RAG Architecture and When to Use It
Retrieval-augmented generation retrieves relevant external material and supplies it to the model at answer time. It is useful when answers depend on private, changing, or large corpora that should not be placed entirely in a prompt. RAG does not guarantee correctness: retrieval can miss evidence, chunks can conflict, and a model can still misinterpret retrieved content.
A standard pipeline has offline ingestion (parse, clean, chunk, embed, index) and online answering (authenticate, retrieve, rank, assemble context, generate, validate citations). Keep these paths observable and independently testable. If the task is a simple fixed lookup, a normal database query may be more reliable than RAG.
Diagram
RAG pipeline
OFFLINE: files -> parse -> normalize -> chunk -> embed -> index
ONLINE: question -> auth filter -> retrieve -> rerank -> prompt
-> grounded answer + citations -> validate -> response💡 Ingestion quality determines much of retrieval quality. Online generation should use only authorized and relevant evidence, and should be able to say that the evidence is insufficient.
Key Takeaways
- RAG separates ingestion from online question answering.
- Authorization filters belong in retrieval, before context is assembled.
- Allow an abstention when evidence is insufficient.
Knowledge Check
What is the main purpose of retrieval in RAG?