Abstention and Clarification
A reliable assistant can say it lacks sufficient evidence. It may ask for clarification when the request is ambiguous or escalate when evidence conflicts or the decision is high impact. Retrieval score thresholds can help, but scores are not universal probabilities. Calibrate thresholds on labeled queries and measure false abstentions as well as unsupported answers.
Measure whether the system correctly chooses to answer, clarify, escalate, or abstain.
Implementation example
Adequate + consistent evidence -> answer with citations
Weak or missing evidence -> abstain / clarify
Conflicting evidence -> disclose / escalate
High-impact action -> separate approval policy💡 Abstention policy should reflect the cost of wrong answers versus unnecessary refusals.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
Why is a similarity score not a probability of correctness?