Embedding

An embedding is a fixed-length array of floating-point numbers — typically 256 to 3072 dimensions — produced by a neural network to represent the meaning of a piece of text, image, or audio in a way computers can compare mathematically. The core property that makes embeddings useful: semantically similar inputs produce vectors that are close together in that high-dimensional space, while unrelated inputs produce vectors that are far apart. This is the foundation of nearly every "AI understands meaning" feature in modern SaaS — semantic search, RAG retrieval, recommendation engines, clustering, deduplication, and content moderation classifiers. Embeddings are generated by specialized models (OpenAI's text-embedding-3, Cohere's embed-v3, Voyage AI, or open-weight options like BGE and E5) that are distinct from generation-focused LLMs — an embedding model's only job is to output a vector, not text. A concrete worked example: embedding the phrases "cancel my subscription" and "how do I stop my monthly billing" through the same embedding model produces two vectors with high cosine similarity (say, 0.89) even though they share almost no words — because the model has learned they mean nearly the same thing. Compare that to embedding "cancel my subscription" against "what's the weather today," which would produce a low similarity score (say, 0.12). In practice, a builder calls an embeddings API like `POST https://api.openai.com/v1/embeddings {"model": "text-embedding-3-small", "input": "cancel my subscription"}` and receives back a JSON array of 1536 floats, which is then stored in a vector database for later similarity search. Embedding dimensionality and model choice matter: larger dimensions capture more nuance but cost more to store and search; and embeddings from different models are not directly comparable to each other, so a RAG pipeline must use the same embedding model for both indexing documents and embedding queries. One practical gotcha builders hit early: embeddings from different models (or even different versions of the same model) live in incompatible vector spaces — you cannot meaningfully compare an embedding produced by OpenAI's text-embedding-3 against one produced by Cohere's embed-v3, even though both are valid embeddings of text. This means switching embedding models requires re-embedding your entire document corpus from scratch, which is a real migration cost worth planning for before locking in a vendor at scale. Embedding models are also evaluated on public benchmarks like MTEB (Massive Text Embedding Benchmark), which is a reasonable starting point for comparing retrieval quality across providers before committing to one for a production RAG pipeline.