One-Sentence Definition
An embedding is a numerical representation of data -- a word, sentence, image, or any other object -- as a dense vector of numbers in a continuous space, where similar items are positioned close together.
How It Works
Machines cannot natively understand words, images, or audio. They operate on numbers. An embedding converts human-interpretable data into a fixed-length array of floating-point numbers (a vector) that captures the semantic meaning of the original data.
The simplest example is word embeddings. The Word2Vec model, introduced by Google researchers in 2013, learned to represent words as vectors in a space where semantic relationships were encoded as geometric relationships. The famous example: the vector for "king" minus "man" plus "woman" produced a vector close to "queen." Words with similar meanings -- like "car" and "automobile" -- ended up near each other in the vector space.
Modern embedding models go far beyond individual words. OpenAI's text-embedding-3-large and Cohere's Embed v3 produce embeddings for entire passages of text, capturing nuanced meaning in vectors with 1,536 or more dimensions. These sentence and document embeddings power semantic search: instead of matching keywords, a search system compares the embedding of a query to the embeddings of documents and returns the closest matches by meaning.
Embeddings are not limited to text. CLIP produces joint embeddings for images and text in the same vector space, enabling you to search images with text queries. Audio embeddings represent sound characteristics. Graph embeddings represent relationships between entities. In recommendation systems, users and products are both embedded into a shared space, and recommendations are generated by finding products close to a user's vector.
The training process varies by model, but the core idea is consistent: similar inputs should produce similar vectors, and dissimilar inputs should produce distant vectors. This is typically achieved through contrastive learning, where the model is trained on pairs of related and unrelated examples.
Why It Matters
Embeddings are the connective tissue of modern AI systems. Retrieval-augmented generation (RAG) depends on embeddings to find relevant documents. Vector databases store and search embeddings at scale. Recommendation engines at Spotify, Netflix, and Amazon are built on embedding similarity. Semantic search in Google and Bing uses embeddings to understand query intent beyond literal keyword matching.
In 2026, embeddings are also central to AI safety and content moderation. Companies like OpenAI and Anthropic use embedding-based classifiers to detect harmful content. Pinecone, Weaviate, and Chroma have built entire businesses around storing and querying embeddings efficiently. The quality of an AI application's embeddings often determines the quality of its retrieval, recommendations, and understanding.
Key Takeaway
Embeddings translate data into numerical vectors that capture meaning, enabling AI systems to measure similarity, perform semantic search, and connect different types of information in a shared mathematical space.
Part of the AI Weekly Glossary.