Threat Model a RAG System
Assets include source documents, ACLs, credentials, conversation history, indexes, and tools. Threats include unauthorized retrieval, prompt injection, poisoned sources, stale permissions, SSRF, unsafe rendering, cost abuse, and sensitive logging. Map each threat to a control and a test.
The model should not decide whether a user may access a document. Trusted application policy must govern access before content enters context.
Diagram
Implementation example
Upload -> validate -> parse safely -> index
Query -> authenticate -> ACL filter -> retrieve
Context -> model -> validate citations -> safe render💡 Each boundary needs explicit controls and negative tests.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
What should determine whether a chunk can be returned?