RAG Quality Is a Pipeline Property
A wrong answer may begin with a missing source, a parser that lost a table, a bad chunk boundary, incompatible embeddings, an overly strict filter, poor ranking, truncated context, or unsupported generation. Debug from the source forward: inspect extracted text, chunks, metadata, raw retrieval, reranked results, final context, answer, and citations.
Do not tune the prompt first if the needed evidence never reached the model. Maintain known questions with expected source IDs and stage-level traces to localize the earliest failure.
Implementation example
Source -> parse -> chunk -> index -> retrieve -> rank
-> context -> generate -> validate💡 The first incorrect intermediate artifact often explains the final failure.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
If the correct policy never appears in retrieved chunks, what should you inspect first?