5. Embeddings and Vector Representations
Understand embedding spaces, model selection, similarity metrics, and approximate nearest-neighbor indexes.
How Embeddings Represent Meaning
An embedding maps text or another supported input into a numeric vector. Similar meanings may be close in the learned space, but embeddings are not a perfect representation of truth. Query and document vectors must use compatible models and preprocessing. Store the embedding model identifier and dimensions with each indexed vector.
Embeddings support semantic retrieval, clustering, recommendations, and deduplication. They are not encryption and should not be assumed anonymous.
const vector = await embeddingModel.embed("How do I rotate an API key?");
// vector is a numeric array in a model-specific embedding space- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
What is an embedding?
Choose an Embedding Model
Evaluate embedding models on representative queries: exact names, acronyms, code, multilingual text, domain vocabulary, paraphrases, and long passages. Compare Recall@K, ranking metrics, dimensions, latency, throughput, price, input limits, deployment choices, and data-handling constraints.
Keep a gold set of queries and relevant source IDs. Change one factor at a time. When switching models, use a versioned index or migration because vectors from incompatible spaces generally cannot be mixed.
Compare models on:
- Recall@K / MRR
- Domain and language coverage
- Latency and throughput
- Cost per indexed token/query
- Data handling and operational fit- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Why test embedding models on your own corpus?
Cosine, Dot Product, and Distance
Cosine similarity compares vector direction; dot product is sensitive to magnitude unless vectors are normalized; Euclidean distance measures geometric distance. The right metric depends on the model's intended use and index configuration. Raw scores across models are not universal probabilities of relevance.
Calibrate thresholds on labeled queries. A similarity score of 0.82 does not inherently mean an 82% chance of correctness. Confirm whether the database expects normalized vectors and which metric is configured.
function cosine(a,b) {
if (a.length!==b.length) throw new Error("dimension mismatch");
const dot=a.reduce((sum,x,i)=>sum+x*b[i],0);
const na=Math.sqrt(a.reduce((sum,x)=>sum+x*x,0));
const nb=Math.sqrt(b.reduce((sum,x)=>sum+x*x,0));
return dot/(na*nb);
}- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
What does cosine similarity compare?
Vector Indexes and ANN Search
Exact nearest-neighbor search can be expensive at scale. Approximate nearest-neighbor indexes trade some recall for speed. Graph-based and inverted-file/quantization approaches have different memory and latency trade-offs. Filter behavior can interact with ANN search, so benchmark realistic tenant and metadata filters.
Tune index parameters using recall and latency targets. Keep canonical document records recoverable independently of the index, and monitor retrieval quality after rebuilds and corpus growth.
Query vector + trusted metadata filter
-> ANN search -> top-k IDs
-> fetch authorized text and metadata- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
What is the main ANN trade-off?