Vector Index

A vector index is the data structure that makes similarity search fast. Comparing a query against every stored embedding is exact and linear; an index (HNSW, IVF, ScaNN and friends) narrows the search to a promising neighbourhood and answers in milliseconds instead of seconds. The trade is approximation. These are approximate nearest-neighbour structures: they return the right answers most of the time, and the tuning parameters decide how often. Higher recall costs memory and latency; lower recall silently drops the one relevant document from a retrieval set nobody inspects. That silence is why recall belongs in your monitoring. A RAG assistant with a badly tuned index does not error — it answers confidently from the four documents it did find, and the missing fifth is the one that contained the exception. Practical notes: filtered search ("only this tenant's documents") is where naive indexes fall over, so check that your database applies filters during traversal rather than after it. And rebuild cost matters — an index that takes six hours to rebuild shapes how often you can re-embed a corpus.