AI Under the Hood

See how a language model turns text into tokens with an interactive visualization. Watch byte pair encoding merge characters into subwords, map tokens to ids, and see why token counts differ from word counts.

See how embeddings turn tokens into vectors with an interactive visualization. Watch a lookup table map token ids to numbers, compare vectors with cosine similarity, and see how meaning becomes geometry.

See how self-attention works with an interactive visualization. Watch queries score keys, softmax turn scores into weights, the causal mask hide future tokens, and weighted values mix into a new representation.

See how a language model picks the next token with an interactive visualization. Watch logits become probabilities, compare greedy, temperature, top-k, and top-p sampling, and follow the autoregressive loop token by token.

See how neural networks learn with an interactive visualization. Step through a forward pass, the loss, gradients flowing backward through the chain rule, and a gradient descent weight update on a tiny network.