7. Embeddings and Semantic Search

Understand embeddings, chunking, similarity metrics, vector indexes, hybrid retrieval, and evaluation.

What Are Embeddings?

An embedding maps text or another supported input into a numeric vector intended to represent semantic features. Similar meanings may be near one another in the embedding space, depending on the model and training. Embeddings enable semantic retrieval, clustering, deduplication, recommendations, and classification features.

Do not assume embeddings are facts, explanations, or guarantees of relevance. Their quality depends on the embedding model, domain, language, chunking, query form, and retrieval method. Keep model identity and dimensions consistent within an index, and plan re-embedding when changing models.

Diagram

Embedding pipeline

Document text -> normalization -> embedding model -> vector + metadata
Query text    -> same embedding space -> nearest-neighbor search
Nearest chunks -> ranking/filtering -> application context

💡 Queries and documents should use compatible embedding models and preprocessing. Store document IDs, chunk IDs, access-control metadata, and version information alongside vectors.

Key Takeaways

Use compatible embedding models for queries and indexed documents.
Track model and document versions.
Evaluate retrieval quality on representative queries.

Knowledge Check

What does an embedding primarily represent?

Chunking, Overlap, and Metadata

Chunking splits long documents into retrievable units. Very large chunks can dilute relevance and exceed context budgets; very small chunks can lose context. Chunk by semantic structure where possible—headings, paragraphs, sections, code blocks—then tune chunk size and overlap using evaluation data.

Metadata should include source, document version, tenant or access scope, section heading, timestamps, and content type. Apply authorization filters before returning chunks to the model. A vector similarity score does not imply permission to disclose a document.

Chunk a document by paragraphs

function chunkParagraphs(text, maxChars = 1200) {
  const paragraphs = text.split(/\n\s*\n/);
  const chunks = [];
  let current = "";

  for (const paragraph of paragraphs) {
    if (current && (current.length + paragraph.length + 2) > maxChars) {
      chunks.push(current);
      current = "";
    }
    current += (current ? "\n\n" : "") + paragraph;
  }
  if (current) chunks.push(current);
  return chunks;
}

💡 This simple baseline groups paragraphs by character count. Production chunking should consider token limits, oversized paragraphs, headings, overlap, code blocks, and document structure.

Key Takeaways

Chunk size and overlap should be evaluated, not guessed permanently.
Preserve source and version metadata.
Enforce access control before model context assembly.

Knowledge Check

Why store access-control metadata with indexed chunks?

Similarity Search, Hybrid Search, and Reranking

Vector search retrieves semantically similar vectors using a distance or similarity metric. Exact search can be expensive at scale, so vector databases often use approximate nearest-neighbor indexes. Hybrid search combines lexical retrieval (such as BM25/full-text search) with vector retrieval; this can help when exact names, identifiers, or rare terms matter alongside semantic meaning. Reranking uses a second model to reorder a smaller candidate set.

Similarity scores are not universally comparable across models or databases. Tune candidate count, filters, distance metric, hybrid weights, and reranking against a labeled evaluation set. Monitor recall and latency as the corpus grows.

Hybrid retrieval concept

Candidates = merge(
  lexicalSearch(query, filters),
  vectorSearch(embed(query), filters)
)
TopResults = rerank(query, Candidates).slice(0, 8)

💡 This is pseudocode showing the stages. Deduplicate candidates, preserve source metadata, enforce filters on both retrieval paths, and benchmark whether reranking improves answer quality enough to justify latency and cost.

Key Takeaways

Tune retrieval against real queries.
Use hybrid search when exact terms matter.
Reranking adds latency and should be justified by measured gains.

Knowledge Check

Why can hybrid search outperform vector-only retrieval for some queries?

What Are Embeddings?

Chunking, Overlap, and Metadata

Similarity Search, Hybrid Search, and Reranking

STAY UPDATED WITH BYTES!