Project 1: AI Support Ticket Triage
Build a service that classifies support tickets into a known category, estimates priority, extracts a concise summary, and routes uncertain cases to a human. Define a strict output schema and create a labeled evaluation set. Measure per-class precision/recall, invalid-output rate, escalation rate, and average processing cost.
The model should not automatically resolve sensitive account issues or grant access. Routing rules and permissions remain deterministic. Log the model version and output metadata, but minimize storage of raw ticket content and redact sensitive fields when possible.
Diagram
Ticket triage architecture
Ticket -> redact sensitive fields -> classify/extract
-> schema validation -> confidence/quality policy
-> queue assignment or human review
-> evaluation feedback💡 This project exercises structured outputs, validation, rate limiting, evaluation, and human escalation without requiring an autonomous agent.
Key Takeaways
- Use a closed category set.
- Evaluate per-class behavior and escalation.
- Keep routing and permissions deterministic.
Knowledge Check
What should happen when a ticket classification is invalid or uncertain?