Incident Response and Rollback
Incidents include leakage, stale policy answers, poisoned sources, provider outages, and index corruption. Prepare runbooks to disable a source or tool, switch to a prior index, limit traffic, or fall back to safe search. Preserve evidence securely, rotate exposed credentials, assess impact, and add regression cases after remediation. Audit emergency controls and practice rollback before incidents.
Implementation example
Detect -> contain source/tool/index -> protect users
-> preserve evidence -> remediate -> validate -> restore gradually💡 A tested runbook reduces recovery time.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
What should follow a RAG incident fix?