PDFs, OCR, and Layout
PDF extraction may fail on scans, columns, headers, footers, and tables. Scans need OCR, which can introduce errors. Preserve page numbers and, where possible, offsets or bounding boxes so evidence can be traced to the original. Inspect representative page images during ingestion QA.
Tables need row/column relationships; concatenating cells may change meaning. Evaluate parsing quality separately from retrieval and generation so extraction defects are not misdiagnosed as model hallucinations.
Implementation example
{
"documentId":"handbook",
"page":12,
"section":"Leave policy",
"text":"Eligible employees may request...",
"extractionMethod":"ocr"
}💡 Page provenance supports citations and helps diagnose OCR errors.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
Why preserve PDF page provenance?