AI triage for email and tickets: route to the right person in seconds
TL;DR
For each new email/ticket the LLM returns JSON {category, priority, language, summary, confidence}; code uses it to label, assign and create tickets. Low confidence or VIP customers → human review queue. Measure accuracy weekly against a hand-labeled sample.
Flow
- Trigger: new email (Gmail/Outlook) or ticket (Jira, Zendesk) via webhook.
- Pre-filter with rules: spam, auto-replies, newsletters → skip (no tokens spent).
- LLM classifies and returns JSON against a fixed schema.
- Code routes: labels, assignee, SLA by priority.
- Confidence below threshold → “needs a human” queue.
Suggested schema
{
"category": "bug | billing | how_to | feature_request | other",
"priority": "P1 | P2 | P3",
"language": "vi | en",
"summary": "one sentence",
"confidence": 0.0
}Metrics
| Metric | Starting target |
|---|---|
| Classification accuracy (100 hand-labeled tickets) | > 90% |
| Share escalated to humans | 10–20%, shrinking as prompts improve |
| Time from receipt to correct assignee | Hours → minutes |
| Cost per ticket | Track it; use a small model + batch when not urgent |
References: Tool use — Claude Docs · Building effective agents — Routing