Embedding

An embedding is the vector of numbers that represents a token inside the model. When the word "dog" enters a model, it is first converted into its embedding, a long list of numbers that encodes its meaning.

The magic of embeddings is that meaning becomes geometry. Words with similar meanings end up with similar embeddings, sitting close together in the space of numbers. "Dog" and "puppy" land near each other; "dog" and "spreadsheet" land far apart. The model learns these positions during training. Embeddings are the bridge that turns language into math.

---