# Support QA review SOP

Version: 1.0

Owner: Support quality lead

Purpose: Produce a reproducible service-quality signal and route confirmed findings to improvement without automatically deciding consequences for people.

## Scope and prerequisites

Applies to completed support conversations covered by an approved QA rubric. Required: eligible-population query, privacy/access rule, versioned scorecard, trained reviewers, dispute path and defined random, risk and calibration sample sizes.

## Procedure

1. Freeze the cycle dates, eligible channels, queues, languages and exclusion rules.
2. Record the population query and count.
3. Draw the random sample with a stored seed and selection probability.
4. Add risk-triggered cases with a separate selection class and reason.
5. Select a calibration overlap set for independent double scoring.
6. Retrieve permitted conversation events, source records and the effective policy version.
7. Mask or restrict sensitive fields the reviewer does not need.
8. If model assistance is used, request criterion-level pass/fail/not-applicable/uncertain suggestions with exact evidence spans.
9. The human reviewer verifies evidence, records final criterion results, root causes and comments.
10. A second reviewer independently scores overlap cases.
11. Calibrate disagreements by criterion; preserve original scores and apply guidance changes prospectively.
12. Route confirmed findings to procedure, knowledge, tooling or coaching owners.
13. Resolve disputes through an append-only adjudication record.
14. Publish random and risk-sample results separately with denominators and rubric version.

## Exception handling

- Unscorable record: record reason and replace using the original selection rule.
- Rubric change: finish the cycle under its frozen version or restart it.
- Model drift: disable suggestions for the affected slice while human review continues.
- Reviewer conflict: preserve both scores, adjudicator and resolution.
- Sensitive case: restrict to a qualified reviewer or exclude under documented rules.

## Measures

Quality by criterion and slice, risk-defect discovery, reviewer agreement, model-to-final agreement, not-applicable rate, dispute and overturn rate, review time and finding-to-fix closure.

## Prohibited uses

Do not automatically rank workers, allocate compensation, issue discipline, infer protected traits or turn one QA score into an employment decision.

## Revision log

- 2026-08-03: Initial public template.