Key takeaways
- Triage decides what a ticket is and how urgent it is; routing decides who should own it. Keep those decisions separate so each can be tested.
- Use deterministic checks for identity, entitlement, duplicates and explicit emergency phrases; use a model only for meaning rules cannot reliably read.
- Store the model label, confidence, evidence spans and policy version before applying queue rules. A tag without its decision record is not auditable.
- Low confidence, conflicting signals, security reports and threats of harm go to a named human queue instead of a generic exception bucket.
- Measure correction rate by category and source channel, not just average triage speed. Fast classification is worthless if one queue absorbs the errors.
Start with a triage contract, not a list of clever labels
A useful contract says when this job starts, what it may read and what a downstream queue can rely on. The trigger is one newly registered ticket or a material customer reply. Inputs are the original message, channel, verified customer and account identifiers, product metadata, entitlement, open tickets and the current category policy. The output is a versioned triage record; no customer reply and no ownership assignment happen here.
The actors are the intake service, a bounded classifier, the support duty lead and the queue rules engine. Frequency is every inbound ticket and every customer update that changes the reported issue. Success means a ticket becomes actionable once, uncertain cases arrive visibly in a review queue, and corrections improve a named category rather than disappearing into a generic “other” bucket.
| Field | Owner | Evidence required |
|---|---|---|
| Customer and account | Code | Verified internal IDs; the sender address is a lookup key, not proof |
| Duplicate or related case | Code | Exact message/thread identifiers plus bounded similarity candidates |
| Intent and product area | Model | Label, confidence and short spans from the customer's words |
| Urgency | Rules + human | Observed impact, affected scope and explicit policy trigger |
| Language | Model + code | Detected language, confidence and channel locale |
| Exception status | Code | Named reason, destination and required next evidence |
The manual baseline reveals where classification actually fails
Before automation, an experienced coordinator opens each message, searches the customer, checks for related cases, decides what the request means, estimates impact, adds fields and drops it in a queue. The work looks like one judgment call because all seven actions happen in one browser tab. Sampling twenty completed tickets usually exposes different failure modes: a missing account, two issues in one message, a reply attached to the wrong case, a category that means different things to two teams, and “urgent” used as a synonym for frustrated.
Record the coordinator's evidence and disagreements for a week. That set becomes the evaluation fixture. If two experienced people cannot agree after reading the same policy, the automation cannot solve the ambiguity by returning a higher confidence number; the category or policy needs repair.
The automated workflow keeps facts, interpretation and consequences separate
- 01Register the eventTriggerCreate one immutable intake event keyed by channel and source-message ID. Store the original before transformations, so retries cannot create a second ticket.
- 02Normalize and link identityCodeStrip signatures only in a derived copy, detect attachments, resolve trusted customer/account IDs and record missing or conflicting matches.
- 03Find duplicates and related casesCodeApply exact keys first. Similarity may suggest candidates, but only deterministic thread rules or a person merges records.
- 04Classify the messageAI judgmentReturn a schema-bound intent, product area, language, sentiment signal, named entities, confidence per field and evidence spans. No free-text route instruction is accepted.
- 05Derive urgencyCodeMap verified impact and scope through the written priority matrix. Explicit security, safety or account-takeover signals override the model and enter a protected review path.
- 06Apply confidence and conflict gatesCodeCommit only permitted combinations above field-specific thresholds. Low confidence, mutually exclusive labels and contradictions with account facts create named exceptions.
- 07Review consequential ambiguityHuman approvalThe duty lead sees the original, proposed fields, evidence spans and failed rule. Their correction is recorded as an event, never used to overwrite history.
- 08Commit the triage recordCodeWrite fields atomically with policy and model versions, emit one triage-completed event and hand the stable record to routing and SLA selection.
Give every exception a destination and a way back
| Exception | Safe action | Return condition |
|---|---|---|
| No trusted account match | Identity-resolution queue; restrict account data | Verified internal account ID attached |
| Several unrelated issues | Propose child tickets; person confirms the split | Each child has source-span provenance |
| Possible duplicate | Link candidates; do not auto-close either case | Coordinator confirms merge or independence |
| Security, safety or threat signal | Protected urgent queue with acknowledgement timer | Authorized responder accepts ownership |
| Low confidence or label conflict | Category-owner review with failed field highlighted | Corrected label and reason recorded |
| Classifier or write failure | Retry from the immutable intake event | One atomic triage record exists |
Worked sample: a frustrated message that is not automatically urgent
A customer writes: “Third time today. Export says complete but the file is empty. I need it for tomorrow's board pack.” Intake links the sender to account A-82 and finds T-1038, closed after a browser refresh. The classifier proposes reporting/export failure at 0.91 and negative sentiment at 0.88, quoting the two relevant clauses.
The urgency rule does not map frustration to severity. One user is affected, there is a workaround, and the deadline is tomorrow, so the verified impact produces P2 rather than P1. The related case is linked but not merged because the symptom differs. The triage record is committed with the evidence spans and passed to routing for a reporting specialist. If the customer had said all exports across the workspace were empty, the scope rule would have raised the priority.
Judge the system by corrections and queue harm
Track coverage, abstention rate, human correction rate per field, corrections by category and channel, high-priority false positives, high-priority misses, duplicate merge reversals and time from arrival to committed triage. Review a random sample of auto-accepted records and every consequential miss. Zendesk documents automated topic, sentiment, language and entity classification; those available outputs are useful inputs, not proof that your local labels or thresholds are correct.
Keep a fixed evaluation set across model and policy changes. A new version ships only when it does not regress protected categories, and the run record identifies which version produced each field. This follows the practical direction of NIST's AI risk framework: risk controls and measurement belong around the system in its context of use, not in a generic accuracy claim.
Support ticket triage SOP
A runnable intake procedure with field contracts, decision bands, exception queues, sampling and audit requirements.
Preview the fileHide preview
# Support ticket triage SOP
Version: 1.0
Owner: Support operations
Review cadence: Monthly and after any category, policy, model, or channel change
## Purpose
Turn each new or materially updated support request into one versioned triage record. This procedure classifies the case; it does not assign an individual, send a customer reply, merge cases automatically, or make a consequential decision.
## Trigger and completion
- Start: a new source-message ID is registered or a customer reply materially changes an open ticket.
- Complete: one atomic triage record exists and either `triage.completed` or a named exception event has been emitted.
- Idempotency key: `{channel}:{source_message_id}:{triage_policy_version}`.
## Required inputs
- Original message reference and approved derived text
- Channel and source-message/thread IDs
- Trusted customer/account lookup result
- Open and recent related-ticket references
- Current intent taxonomy, priority matrix and exception policy
- Classifier version and field-specific confidence thresholds
## Procedure
1. Register the original event before transforming content.
2. Normalize a derived copy; preserve the original and attachment references.
3. Resolve trusted customer and account IDs. If missing or conflicting, restrict account data and open `identity_resolution`.
4. Check exact duplicate keys, then retrieve bounded similarity candidates. Never merge or close from similarity alone.
5. Ask the classifier for schema-bound intent, product area, language, entities, field confidence and evidence spans.
6. Derive urgency with the priority matrix using verified impact and scope. Sentiment alone cannot set severity.
7. Apply hard stops for security, safety, account-takeover, low confidence, conflicting labels and unsupported label combinations.
8. Present exceptions with the original, proposed value, evidence spans, failed rule and permitted corrections.
9. Record the reviewer correction as a new event with reason; do not overwrite the proposal.
10. Commit all accepted fields atomically with taxonomy, policy and classifier versions.
11. Emit one completion event for routing and SLA-policy selection.
## Named exception queues
- `identity_resolution`: trusted account ID absent or conflicting
- `possible_duplicate`: merge requires a coordinator
- `multi_issue_split`: person confirms proposed child cases
- `protected_urgent_review`: security, safety, threat or takeover signal
- `category_owner_review`: low confidence, conflict or unsupported combination
- `triage_system_recovery`: classifier, source-read or atomic-write failure
Each exception needs an owner, age target, reason code, required evidence and return condition.
## Quality controls
- Daily random sample of auto-accepted records during rollout; reduce only after stable evidence.
- Review every high-priority miss, protected-path miss and incorrect merge.
- Compare releases on a fixed, representative evaluation set.
- Track correction rate per field, category and channel; abstention; high-priority false positives/misses; duplicate reversals; and time to committed triage.
## Audit record
Keep intake ID, source reference, trusted identity IDs, candidates, proposed and final fields, confidence, evidence spans, rules fired, exception, reviewer, timestamps, and all model/taxonomy/policy versions. Do not place unnecessary message content or credentials in logs.
## Rollback
Disable classifier acceptance, retain deterministic intake and route every new classification to `category_owner_review`. Reprocess only from immutable intake events and only with a new triage version.
Support ticket triage workflow blueprint
The same process as a build contract: states, inputs, idempotency, branches, events and acceptance tests.
Preview the fileHide preview
# Support ticket triage workflow blueprint ```yaml name: support-ticket-triage version: 1.0.0 trigger: events: [ticket.created, customer.reply_material] idempotency_key: channel + source_message_id + triage_policy_version input: required: [ticket_id, source_message_id, original_reference, channel, policy_version] optional: [authenticated_account_id, locale, attachments, related_ticket_ids] output: required: [triage_id, intent, product_area, language, priority, exception_state, provenance, versions] side_effects: allowed: [write_triage_record, emit_triage_event, open_named_exception] forbidden: [send_customer_reply, merge_ticket, close_ticket, restrict_account, issue_refund] ``` ## States `registered -> normalized -> identity_resolved -> duplicate_checked -> classified -> rules_checked -> accepted|review_required -> committed` Terminal exception states: `identity_resolution`, `possible_duplicate`, `multi_issue_split`, `protected_urgent_review`, `category_owner_review`, `triage_system_recovery`. ## Step contracts | Step | Type | Reads | Writes | Retry rule | |---|---|---|---|---| | register | deterministic | source event | immutable intake | create-if-absent | | identity | deterministic | trusted identity service | account IDs or exception | bounded retry, no broad search | | duplicate | deterministic + retrieval | exact keys, bounded candidates | candidate references | read-only replay | | classify | model | approved derived text, taxonomy | labels, confidence, spans | pin version; no side effect | | priority | deterministic | verified impact/scope, matrix | priority, fired rules | pure function | | gate | deterministic | proposed record, thresholds | accept or named exception | pure function | | review | human | original, proposal, evidence, failed rule | correction event | never auto-repeat approval | | commit | deterministic | accepted fields and versions | atomic triage record/event | compare-and-set triage ID | ## Invariants 1. Exactly one committed record per idempotency key and version. 2. Every model field has confidence and evidence spans. 3. Sentiment cannot independently increase priority. 4. Similarity cannot merge or close tickets. 5. Protected signals never enter ordinary auto-accept. 6. Every correction preserves proposed and final values. ## Events - `triage.registered` - `triage.classification_proposed` - `triage.exception_opened` - `triage.correction_recorded` - `triage.completed` - `triage.failed` Each event includes `triage_id`, `ticket_id`, `occurred_at`, `policy_version`, `taxonomy_version`, `classifier_version`, `actor`, and `correlation_id`. ## Acceptance tests 1. Replayed source message creates no duplicate record. 2. Missing account enters identity resolution without retrieving account data. 3. Exact source/thread duplicate is linked deterministically. 4. Similar text produces candidates but no automatic merge. 5. Two intents in one message enter split review. 6. Low-confidence intent enters category review. 7. Security phrase enters protected review regardless of general confidence. 8. Negative sentiment alone does not set P1. 9. Validation conflict blocks commit. 10. Reviewer correction preserves the proposed label. 11. Classifier timeout can replay without duplicate side effects. 12. Atomic write failure emits no completion event. 13. Taxonomy change creates a new version rather than rewriting history. 14. Logs contain references and decisions but no credentials.
Limitations and when not to use this
- This is a process-design pattern, not a claim that any named support product will achieve a particular classification accuracy on your tickets.
- Do not use this flow to make refunds, access restrictions, fraud findings, safety determinations or other decisions with an individual consequence.
- A small or rapidly changing taxonomy may be better served by manual triage until operators agree on labels and can produce a representative evaluation set.
Sources
- About intelligent triage — Zendesk Documentation Accessed 2 August 2026
- Intelligent triage use cases and workflows — Zendesk Documentation Accessed 2 August 2026
- Artificial Intelligence Risk Management Framework 1.0 — NIST Accessed 2 August 2026
Map one week of triage decisions
Capture the inputs, labels, hard stops and correction destinations before selecting automation software.
Map one week of triage decisionsUli Prantz
Builds and operates all-agents
Uli Prantz builds all-agents, the process-automation platform this site documents. He writes about the operational side of automating recurring business work: where deterministic code beats model judgment, where it does not, and where a human still has to approve.