Monitor Ingestion Health and Freshness
Track source counts, last successful sync, ingestion lag, stage failures, dead letters, deletion lag, index version, and age of newest indexed content. Reconcile source counts and sample content: a successful job does not prove all documents were parsed correctly. Alert when freshness SLOs are missed.
Implementation example
Metrics:
ingestion lag | indexed documents | parse failure rate
deletion lag | dead-letter count | last successful sync
index version | stale-source rate💡 Monitor freshness and content quality, not only worker uptime.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
Which metric captures whether updates become searchable promptly?