Agent-to-Human Output Validation

Placement rationale

C20 sits in Layer 3 alongside C13 (semantic observability) and C19 (model behaviour monitoring) because it is an observation-and-response control that operates on the agent's output stream rather than on its tool calls or its memory reads. C13 captures per-run intent telemetry. C19 watches the statistical distribution of behaviour over time. C20 inspects the content of every final response immediately before delivery and routes it through a configured action matrix. Placing C20 in Layer 2 would misrepresent it as a per-call decision gate on the same axis as C05 and C09; placing it in Layer 1 would imply that output classification is an identity property. Output validation is observation of the response stream with a governance response path attached, exactly the shape Layer 3 already carries.

Why

Without this control, an agent that passes every existing GATE check can still deliver a regulated-content response to a user with no record that the response was classified, no opportunity for human review, and no redaction of fields the response should never have contained. The agent's identity is valid (C01). The tool calls were policy-compliant (C05). The invariants held (C09). The model is not drifting (C19). The replay reproduces (C10). And the agent told a customer that a specific medication is safe to take, or that a specific contract term is enforceable, or returned a record that included an unredacted social security number alongside the answer the customer asked for. The failure mode is at the output boundary, and GATE v1.3 had no control there.

This happens through several routes. An agent answers a benign question by retrieving information that happens to fall into a regulated category (medical, legal, financial, HR, immigration) without the calling system being aware that the answer crosses a regulatory line. An agent producing a structured response includes a field containing personal data the consumer was not entitled to see, because the upstream tool returned more than the agent's prompt expected and the agent included it verbatim. An agent with general-purpose capabilities answers a question outside its intended scope with apparent confidence, and the calling user has no signal that the answer is low-confidence relative to the agent's calibrated distribution.

Prompt-based constraints fail here for two reasons. First, "regulated" is jurisdiction-specific and context-specific; a prompt cannot encode the full action matrix. Second, the agent's output is the very thing the prompt is meant to constrain - there is no independent check that the constraint was honoured. Conventional content filters (toxicity scoring, basic PII redaction) catch a subset of failures but operate on a different signal: they look at words, not at the regulatory category of the response or the agent's calibrated confidence. Transparency obligations in regulated jurisdictions raise the bar further: EU AI Act Articles 13 and 14 require that high-risk AI systems be transparent to deployers and subject to effective human oversight, and Article 50 requires that people interacting with AI systems are informed of that fact. None of these obligations is met by an unclassified, unreviewed, unredacted output stream.

What

A pre-delivery classification and obligation service that sits between the agent runtime and any downstream consumer of the agent's final response. The control consists of three mechanisms:

1. Classification engine - runs on every final agent response immediately before delivery. It consumes the response content and a signed output classification bundle defining the organisation's sensitivity tiers (a normative enumeration, not a free list), the regulated content categories it tracks, the confidence threshold below which a response is held for review, and the action matrix mapping sensitivity tier, regulated category, and autonomy tier to obligations.
2. Confidence scoring component - produces a structured confidence score in the [0, 1] range, derived from the model's logprobs where available and from a calibrated proxy where not. The score is a property of the response as a whole; per-claim scoring is out of scope for v1.4. The score is included in the classification event and feeds the hold-for-review obligation.
3. Obligation router - applies the obligations the action matrix defines. Three obligation types: redact_fields (named fields are zeroed or replaced with the standard redaction marker), hitl_review (the response is held until a HITL approval record arrives), and hold_for_review (the response is held until a confidence-threshold review record arrives). Obligations compose: a single response can require both redaction and HITL review.

Invariants the control guarantees:

• Every final agent response produces exactly one gate.output.classification event in the ledger before delivery. There is no delivery path that bypasses classification.
• The output classification bundle is signed (C03) and its hash is recorded in the classification event so a classification decision can be replayed (C10).
• Classification events are immutable evidence (C11 ledger) and correlate to the run ID and trace ID of the originating agent run.
• Where a regulated category and high-privilege tier combination triggers a hitl_review obligation, delivery is gated on the HITL Decision Record arriving and resolving to approve. The fail-closed default is hold, not deliver.

Distinction from C13 and C19

The boundary with C13 and C19 is the architectural property that prevents three controls from collapsing into one. C13 captures intent telemetry on a per-run basis, produced during the run before the final response is composed: the per-run intent signal. C19 observes the distribution of behaviour over time and emits drift decisions on a defined cadence; a single response does not move a C19 distribution by a detectable amount: the aggregate drift signal. C20 inspects the content of each final response at delivery and emits one classification event per response; it does not look at the agent's reasoning, tool choices, or trajectory: the per-response content signal.

These are different signals at different cadences on different artifacts, and they must not be merged. A system that runs only C13 has rich intent telemetry and delivers regulated content without classification. A system that runs only C19 detects a shift toward regulated-category responses weeks after the first one was delivered. A system that runs only C20 classifies every response correctly and learns nothing about intent or drift. All three are required at high_privilege tier. The three controls share an evidence destination (C11 ledger) and a common correlation key (run_id, trace_id) but have independent decision logic, separate bundles, and distinct ledger event types: gate.observability.semantic_event (C13), gate.assurance.drift_decision and gate.assurance.response_action (C19), and gate.output.classification (C20).

How

Control-plane flow

The classification engine runs as the last hop before the agent runtime returns its final response to the calling system. It receives the response content, the run context (run_id, trace_id, tenant_id, environment, autonomy_tier), and a reference to the active output classification bundle. It produces a classification result (sensitivity tier, regulated categories, confidence score) and an obligation list, then emits a gate.output.classification event into the C11 ledger. If the obligations are empty, the response is delivered. If redact_fields is present, the field-level redactions are applied before delivery. If hitl_review is present, the response is held in a review queue and the calling system receives a hold acknowledgement with a review ticket identifier; delivery resumes when the HITL Decision Record arrives with approve. If hold_for_review is present, the response is held pending a confidence-threshold review record from the designated reviewer.

Deployment

The engine runs inside the agent runtime container as a deterministic post-generation hook, or as a sidecar the runtime calls synchronously before delivery. Both shapes are valid; the choice is driven by the runtime's tolerance for additional in-process logic. The output classification bundle is loaded at agent start with signature verification (C03), and the bundle hash is held in memory for inclusion in each classification event. The review queues are persistent, with retention sufficient to cover the review SLA.

Action matrix

The action matrix is configuration-driven, not hardcoded: a YAML structure inside the output classification bundle. Each entry specifies a sensitivity tier, zero or more regulated categories, an autonomy tier, and a confidence band over [0, 1], and lists the obligations to apply. Multiple matching entries merge their obligation lists. The bundle MUST contain a default entry that matches any unmatched combination at high_privilege tier; the default is hold_for_review to enforce fail-closed behaviour. The bundle is versioned, signed, and ships under the same change-control bar as the policy bundle.

Streaming output

Streaming output to the user is disabled at high_privilege tier for any agent whose action matrix can produce a hitl_review or hold_for_review obligation. The response is buffered until the classification event is emitted and the obligations are applied. Streaming to internal logging and trace destinations is unaffected. This constraint applies in v1.4 pending the streaming-aware classification path scoped to v1.5.

Safe rollout

Begin in flag-only mode for thirty days: classification events are emitted and the action matrix is evaluated, but no obligation is applied. This establishes the false-positive rate per regulated category and the baseline confidence score distribution. After flag-only, promote redact_fields first (lowest blast radius), then hold_for_review (delivery delayed, not blocked indefinitely), and hitl_review last, only after the HITL review SLA is demonstrated to be reliable. An unreliable HITL queue with hitl_review enforced produces a denial-of-service against the agent's users.

Testing

Synthetic input injection in CI. For each regulated category, maintain a test corpus of responses that should classify into the category and a counter-corpus that should not; run both on every bundle change. For confidence scoring, maintain a calibration test on a held-out set. For the action matrix, maintain a coverage test confirming every reachable cross-product of inputs produces a non-empty obligation list (no silent passes).

Integration with C13, C19, and C09

C13 events and C20 events share run_id and trace_id: an auditor reads the C13 events for intent and trajectory and the C20 event for the classification of the final response. The C19 baseline can include the distribution of C20 classification outcomes as additional dimensions; a drift in the rate of regulated-category responses is itself a C19 signal, and the integration is recommended at high_privilege tier. Where a response is held and never approved, it is permanently held - a delivery suspension, not an invariant halt. C09 is not invoked by C20 obligations. For hitl_review, C20 uses the existing HITL Decision Record contract, which is single-approver in v1.4; operators that require dual approval today use the break-glass record path as a manual workaround until the multi-approver HITL extension arrives in v1.5.

Evidence

• gate.output.classification event per final response: schema_version, event_type, time, run_id, trace_id, tenant_id, environment, output_hash (sha256 over canonical serialisation of the response), classification (sensitivity_tier, regulated_categories, confidence_score, requires_human_review), obligations, bundle_hash, ledger_event_id.
• HITL Decision Record per response that triggered hitl_review (existing schema; see C09 and the contracts repository).
• Hold-for-review approval record per response that triggered hold_for_review (re-uses the HITL Decision Record schema with a review_type discriminator).
• Output coverage metric: % of final agent responses with a corresponding classification event. Target 100% at bounded and high_privilege tier.
• Regulated category review SLA metric: median and 95th percentile time from obligation emission to HITL Decision Record arrival, per regulated category, computed weekly.
• Obligation distribution: count of responses per obligation type per agent per week, used to track operational load on the review queues.
• Bundle integrity report: signed output classification bundle hash matches the hash recorded in classification events.

Failure modes

• Bypass via streaming output. The runtime streams tokens to the user as generated; the engine runs only on the final response, and tokens reach the user before classification can hold them. Mitigation: at high_privilege tier, streaming to the user is disabled for any agent whose matrix can produce a hold or review obligation; the response is buffered until obligations are applied.
• Action matrix coverage gaps. The matrix has a default entry but the default is pass, not hold. Unenumerated combinations slip through. Mitigation: the bundle's default entry at high_privilege tier MUST be hold_for_review per the schema; a CI check rejects bundles whose default is pass.
• Review queue saturation. A poorly tuned matrix puts a large fraction of responses into hitl_review; the queue grows without bound and users experience the system as broken. Mitigation: per-category, per-tier rollout with volume forecasts; a queue-depth metric alerts before saturation.
• Redaction over-application. redact_fields is applied broadly and useful fields are zeroed in normal responses. Mitigation: redaction applies only to the named fields per regulated category; a negative test confirms unaffected fields are unchanged.
• Classifier false-negative on sensitive content. The classifier misses content that should classify into a regulated category, even when the matrix is right and the score is calibrated. Mitigation: routine evaluation against a signed, versioned held-out test set per regulated category, with results stored as evidence and alerts on accuracy degradation.
• Synchronous classifier latency. The engine runs on every response before delivery; if it is slow, every response carries the latency and p99 becomes a user-facing outage. Mitigation: an explicit latency SLO in the bundle, continuous p50/p99 metrics per regulated category, and a documented degraded mode (recommended: emit the classification event and decline delivery rather than degrade silently).
• Confidence score misuse. Operators treat the score as per-claim accuracy rather than a calibrated whole-response proxy and tune thresholds against a metric the score does not represent. Mitigation: the bundle documents the score's semantics; the conformance check asks whether the operator has documented their interpretation.
• Confidence signal unavailable. The provider stops returning logprobs; the proxy degrades; scores drift toward an uninformative midpoint. Mitigation: a drift detector (per C19) on the score distribution itself raises an alert when the distribution shifts.
• Bundle versioning drift between runtime and ledger. The runtime holds bundle N while the bundle store has moved to N+1. Mitigation: the classification event records bundle_hash, not a version label; the auditor verifies by hash.
• Confusion with C13 or C19. Operators diagnosing an output-related incident start at C13 traces or C19 dashboards and miss the C20 classification event. Mitigation: runbooks for output-related incidents start at the C20 event for the affected run_id; the event type distinction keeps the boundary explicit.
• Single-approver HITL bottleneck. A regulated category that should require dual approval is gated on a single approver because v1.4 HITL is single-approver. Mitigation: documented scope boundary; operators requiring dual approval use the break-glass record path as a manual workaround until v1.5.

NIST AI RMF alignment

C20 maps to GOVERN, MEASURE, and MANAGE. GOVERN: implements GV-3 by routing regulated content to humans whose role is to evaluate the response in context. MEASURE: implements MS-2.10 (output quality is monitored over time) via the structured classification event per response, MS-2.11 (fairness and bias of outputs examined and documented) where the regulated category set includes bias-relevant categories, and MS-3 (mechanisms for tracking identified AI risks over time). MANAGE: implements MG-3 by enforcing redaction and review obligations on responses that touch third-party data. Rationale: per-response content classification with documented obligations and human-in-the-loop review for regulated categories.

ISO/IEC 42001 alignment

C20 maps to A.6.2.6 (AI system intended use), A.7.4 (data quality) to the extent that response quality is bound to the response generation step, A.8.3 (information for interested parties) by ensuring the response delivered is the response that was classified and approved, A.9 (performance monitoring of AI systems), and clause 8.1 (operational planning and control) for the review queues and action matrix maintenance. Typical evidence produced: signed output classification bundle per ABOM version, classification events per response, HITL Decision Records for reviewed responses, action matrix change history.

OWASP AISVS alignment

C20 maps to AISVS chapter C7 (Model Behavior and Output Control) and one requirement in chapter C5 (Access Control and Identity). AISVS chapter numbers collide visually with GATE control numbers; the v1.0-Cx.y.z requirement form disambiguates. Full coverage: v1.0-C7.2.1 (confidence estimation on generated answers - the confidence_score field on every classification event), v1.0-C7.2.3 (an additional verification pass for responses classified high-risk - the hold_for_review and hitl_review obligations), v1.0-C7.3.1 (automated classifiers scan every response and block content matching defined harmful categories), and v1.0-C5.2.4 (post-inference filtering of data the requester is not authorised to receive - the redact_fields obligation). Partial coverage: v1.0-C7.1.1 (output schema validation), v1.0-C7.1.2 (length limits and termination controls), and v1.0-C7.3.2 (detection of system prompt or backend disclosure - completeness depends on the classifier bundle). See the full per-requirement mapping in owasp-aisvs.yaml.

MITRE ATLAS alignment

C20 maps to six ATLAS techniques and sub-techniques. Full coverage: AML.T0048 (External Harms) - C20 gates output content before delivery, complementing C05's tool-call authorisation for the case where a harmful response required no tool call; AML.T0048.003 (User Harm); AML.T0024 (Exfiltration via AI Inference API) - C20 gates the output path while C07 limits exfiltration volume on the tool-call side; AML.T0067 (LLM Trusted Output Components Manipulation) - the Check20 fail-closed guardrail catches the case where the action matrix yields no obligations on a response that should be held; and AML.T0067.000 (Citations) - C18 derives citations from retrieval metadata and C20 gates the output. Partial coverage: AML.T0056 (Extract LLM System Prompt) - C20 output filters can block responses that disclose system prompt content, with C08 defending the injection surface. See the per-technique detail in mitre-atlas.yaml.

NIST SSDF alignment

C20 falls outside the NIST SSDF intersection scope. SSDF (NIST SP 800-218 v1.1) is software-development-lifecycle scope; C20 is an agent-runtime output-validation control, and the two surfaces do not intersect. C20 is listed in nist-ssdf.yaml only by exclusion. This is a deliberate boundary, not a gap: operators pair GATE with an SDLC discipline upstream of agent deployment. The GATE controls that do intersect SSDF are C03 (PS.2.1, PW.4.1), C18 (PW.4.1), and C05 (PW.9.1).

EU AI Act alignment

C20 supports compliance with three articles. The control does not by itself make a system Act-compliant; compliance is a property of the system as a whole and includes obligations GATE does not cover (conformity assessment, registration, Annex IV technical documentation). Where the Act's obligations relate to output behaviour, C20 is the GATE-side mechanism.

• Article 13 (Transparency and provision of information to deployers). The classification event is the per-response transparency artifact: sensitivity tier, regulated categories, confidence score, and the obligations that attached. A deployer reading the ledger has the per-response evidence needed to evaluate whether the system is operating within its documented limits.
• Article 14 (Human oversight). The hitl_review and hold_for_review obligations are the mechanism by which oversight is exercised at the response level. Check20 verifies that the oversight mechanism is configured, the review SLA is documented, and the queue is monitored.
• Article 50 (Transparency obligations for certain AI systems). The classification event records that a response was AI-generated; the response can carry a machine-readable marker derived from the event, and the event records the model identity from the ABOM, satisfying the provenance side of Article 50.

These mappings are operational, not legal. Counsel determines whether a specific system is in scope for any specific article; the C20 conformance evidence supports that determination, it does not make it.

Related controls

• C13 Semantic Tracing - the per-run intent signal; C13 and C20 events share run_id and trace_id for forensic correlation.
• C19 Behaviour Monitoring - the aggregate drift signal; the C19 baseline can consume C20 classification outcomes as additional dimensions.
• C11 Audit Ledger - classification events are immutable ledger evidence; C20 depends on C11.
• C03 Artifact Integrity - signs the output classification bundle whose hash is recorded in every classification event.
• C10 Deterministic Replay - the bundle hash in the classification event makes classification decisions replayable.
• C09 Invariant Enforcement - a held response is a delivery suspension, not an invariant halt; the break-glass record path is the v1.4 workaround where dual approval is required.