Batch Embeddings and Reuse Work
Batching can reduce network overhead during ingestion. Use bounded concurrency, persist progress, and deduplicate unchanged content before embedding. Cache vectors only when normalized content hash, embedding model, and preprocessing version match. Larger batches can increase memory and failure blast radius, so benchmark realistic workloads.
Implementation example
const key = hash({
contentHash: sha256(normalizedText),
modelId: embeddingModelId,
preprocessingVersion: "normalize-v3"
});💡 The key must include all factors affecting vector compatibility.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
What belongs in an embedding cache key?