Query and Answer Caching
RAG answers depend on permissions, corpus version, prompt version, model, and current data. Cache keys should include relevant scope and versions. Re-check authorization before returning private cached content. Use TTLs and invalidation on source updates, permission changes, or reindexing. Caching retrieval results may be safer than caching final answers for dynamic data.
Implementation example
const key = hash({
tenantId: auth.tenantId,
userScope: auth.cacheScope,
query: normalizeQuery(query),
corpusVersion, promptVersion, modelId
});💡 The key is illustrative; include every state variable that affects correctness.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
Why include corpus version in a cache key?