Build an Evaluation Dataset
An evaluation dataset captures representative inputs, expected properties, and edge cases. It should include ordinary cases, ambiguous cases, adversarial inputs, empty or malformed data, and cases where the correct behavior is to abstain. Keep evaluation examples versioned and separate from training or prompt examples where possible to avoid overfitting.
Evaluate the whole pipeline: retrieval recall, citation support, structured-output validity, tool argument correctness, authorization decisions, latency, and user outcomes. A single “good answer” score hides important failure modes. Use human review for nuanced judgments and calibrate automated graders against human ratings.
Evaluation case shape
{
"id": "returns-ambiguous-03",
"input": "Can I return this if I opened it?",
"expectedBehavior": "Ask for product category if policy differs",
"mustNot": ["Invent a universal return rule"],
"tags": ["ambiguity", "abstention"],
"policyVersion": "returns-v4"
}💡 The expected behavior is expressed as criteria, not necessarily a single exact response. Include the relevant policy version and make evaluation cases stable across runs.
Key Takeaways
- Version representative evaluation cases.
- Measure component-level and end-to-end quality.
- Calibrate automated graders against human review.
Knowledge Check
Why include cases where the correct answer is to abstain?