1. RAG Foundations: Why Retrieval-Augmented Generation Exists
Understand RAG, its use cases, architecture, trade-offs, and common failure modes.
What Is Retrieval-Augmented Generation?
Retrieval-Augmented Generation (RAG) retrieves relevant external information at query time and supplies it to a language model before generation. It is useful when answers depend on private, changing, large, or domain-specific knowledge that may not be reliably present in model weights.
RAG is not a training method: the usual pattern leaves model weights unchanged and changes the context for a request. Quality depends on source quality, parsing, chunking, indexing, retrieval, ranking, context assembly, generation, citation validation, and evaluation. Retrieval can miss the right passage; sources can conflict; a model can misread good evidence. RAG improves grounding opportunities but does not guarantee truth.
Question -> retrieve authorized evidence -> assemble bounded context
-> generate answer -> validate claims and citations- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
What does RAG add to a normal model request?
When to Use RAG—and When Not To
RAG is appropriate when an answer needs evidence selected from a corpus at runtime: internal policies, product documentation, support knowledge, or long technical manuals. It helps when data is too large for every prompt or changes more frequently than model training.
Do not use RAG for every task. Exact account state should come from an authorized database query; arithmetic belongs in deterministic code; stable formatting may need only instructions. Fine-tuning changes model behavior but is usually not the right way to keep a changing document collection current. Hybrid applications often combine SQL, APIs, RAG, and model generation.
Exact account balance -> authorized database query
Policy explanation -> RAG + citations
Arithmetic -> deterministic code
Complex research -> bounded multi-step retrieval- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Which task is usually better served by a direct database query?
Classic RAG vs. Agentic RAG
Classic RAG follows a fixed pipeline: query, retrieve, assemble context, generate. It is easier to control and evaluate. Agentic RAG lets a model choose sources, rewrite queries, decompose questions, or retrieve again. This can help with multi-part questions but increases latency, cost, and failure modes.
Start with classic RAG and add agentic decisions only when evaluation demonstrates a need. Bound tool steps, deadlines, and budgets. The agent must use the same authorization filters and citation validation as a fixed pipeline.
Classic: query -> fixed retrieval -> answer
Agentic: query -> plan/search -> inspect -> search again? -> answer- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Why start with classic RAG?
RAG Quality Is a Pipeline Property
A wrong answer may begin with a missing source, a parser that lost a table, a bad chunk boundary, incompatible embeddings, an overly strict filter, poor ranking, truncated context, or unsupported generation. Debug from the source forward: inspect extracted text, chunks, metadata, raw retrieval, reranked results, final context, answer, and citations.
Do not tune the prompt first if the needed evidence never reached the model. Maintain known questions with expected source IDs and stage-level traces to localize the earliest failure.
Source -> parse -> chunk -> index -> retrieve -> rank
-> context -> generate -> validate- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
If the correct policy never appears in retrieved chunks, what should you inspect first?