API Contracts, Streaming, and Error Semantics
Define stable endpoints and response states such as answered, insufficient_evidence, needs_clarification, and temporary_failure. Validate payload size, identity, conversation ownership, and deadlines before external calls. For streaming, specify event types, terminal events, cancellation, and partial-output behavior. Hide provider-specific errors from clients while retaining internal diagnostics.
Implementation example
{
"status":"answered",
"answer":"The policy allows...",
"citations":[{"sourceId":"returns-v4","section":"Opened items"}],
"requestId":"req_123"
}💡 The public contract should be independent of provider response details.
Key Takeaways
- Inspect the relevant pipeline stage instead of guessing.
- Preserve source provenance, versions, and access scope.
- Measure quality on representative queries.
Knowledge Check
Why distinguish abstention from temporary failure?