8. Retrieval-Augmented Generation (RAG) from Ingestion to Answer

Build an evidence-grounded RAG system with ingestion, retrieval, context assembly, citations, and failure handling.

RAG Architecture and When to Use It

Retrieval-augmented generation retrieves relevant external material and supplies it to the model at answer time. It is useful when answers depend on private, changing, or large corpora that should not be placed entirely in a prompt. RAG does not guarantee correctness: retrieval can miss evidence, chunks can conflict, and a model can still misinterpret retrieved content.

A standard pipeline has offline ingestion (parse, clean, chunk, embed, index) and online answering (authenticate, retrieve, rank, assemble context, generate, validate citations). Keep these paths observable and independently testable. If the task is a simple fixed lookup, a normal database query may be more reliable than RAG.

Diagram

RAG pipeline

OFFLINE: files -> parse -> normalize -> chunk -> embed -> index
ONLINE: question -> auth filter -> retrieve -> rerank -> prompt
        -> grounded answer + citations -> validate -> response

💡 Ingestion quality determines much of retrieval quality. Online generation should use only authorized and relevant evidence, and should be able to say that the evidence is insufficient.

Key Takeaways

RAG separates ingestion from online question answering.
Authorization filters belong in retrieval, before context is assembled.
Allow an abstention when evidence is insufficient.

Knowledge Check

What is the main purpose of retrieval in RAG?

Ingestion: Parsing, Chunking, Deduplication, and Reindexing

Ingestion should be idempotent: running it twice should not create duplicate chunks or stale copies. Store a stable document ID, content hash, source URI, version, parser version, chunking configuration, and embedding model ID. When content changes, delete or supersede prior chunks consistently and ensure failed ingestion jobs can resume.

Different file types require different parsers. Preserve headings and page/section provenance for PDFs and office documents; extract code without destroying indentation; handle OCR uncertainty explicitly. Validate file size and type, scan uploads where appropriate, and treat document contents as untrusted.

Ingestion record example

{
  "documentId": "policy-returns-v4",
  "contentHash": "sha256:...",
  "source": "internal://policies/returns",
  "version": 4,
  "parserVersion": "pdf-parser-2",
  "embeddingModel": "embedding-model-id",
  "status": "indexed"
}

💡 Metadata makes reprocessing, deletion, and debugging traceable. The hash is a placeholder; compute it from normalized content using a cryptographic hash function in your application.

Key Takeaways

Make ingestion idempotent and resumable.
Preserve source provenance and parser version.
Plan for deletion and reindexing when content changes.

Knowledge Check

Why should an ingestion pipeline record document and embedding versions?

Grounded Answers, Citations, and Abstention

A grounded answer should distinguish retrieved evidence from inference. Include stable source identifiers and, where possible, page, heading, or URL references. Validate that cited sources were actually retrieved and that citation spans support the associated claim. Do not ask the model to fabricate page numbers or URLs; supply citation metadata in the context.

Define an abstention policy: if retrieval returns no adequate evidence, the answer should say so or route the user to a human. A fluent answer without supporting evidence is not a successful RAG result. Measure citation correctness and answer support separately from readability.

Note

Citations are not decoration. A citation must support the specific claim it accompanies.

Context with provenance

[
  {
    "sourceId": "returns-policy",
    "section": "Opened products",
    "text": "Opened items may be returned within 14 days...",
    "url": "https://internal.example/policies/returns"
  }
]

💡 Use actual internal source metadata in production. The example URL is illustrative; do not treat it as a real policy reference. The answer renderer should link only to sources the current user may access.

Key Takeaways

Preserve source IDs through retrieval and generation.
Validate that citations refer to retrieved authorized sources.
Evaluate citation support and abstention behavior.

Knowledge Check

What should a RAG assistant do when the retrieved evidence does not support an answer?

RAG Architecture and When to Use It

Ingestion: Parsing, Chunking, Deduplication, and Reindexing

Grounded Answers, Citations, and Abstention

STAY UPDATED WITH BYTES!