Runtime proof · OVERT
AI governance documentation is not operational evidence
Documentation can evidence intent and process. Signed operational records add checkable integrity and key attribution for covered reports; they do not, by themselves, prove event truth, complete coverage, control effectiveness, safety, or compliance.
Governance is strong at saying what ought to be done. Policies, control narratives, and operator-controlled logs can be evidence of intent, process, or reported operation. They may not independently establish integrity, signer attribution, completeness, or what a separate control reported for a specific event.
If you own that documentation, none of this is a failing on your part. You built the right artifacts. The standards asked for policies and you wrote them; the auditors asked for narratives and you produced them. The trouble is structural, not personal: a document usually records an intention or a retrospective account. It does not, by itself, make a claim about a particular operational event independently checkable. As AI systems begin to act, calling tools, making decisions, and taking irreversible steps at runtime, the distance between “we specified a control” and “here is a signed record of the configured control outcome for this event” matters.
Narrowing that distance means preserving scoped operational records alongside the documentation and control-testing evidence a reviewer already needs. The documentation stays; something checkable sits next to it.
What AI governance documentation actually captures
Walk through a typical AI governance binder and you will find three useful kinds of evidence. Each one answers a different question, and none of them establishes the full operational account on its own.
Policies state intent. “Our models will not process protected data without authorization. High-risk outputs require human review.” These are commitments. They are necessary, and a commitment is still a statement about the future rather than a record of the past.
Audit narratives describe process. They explain how a control is meant to operate and how the organization believes it operated over a period. They are written after the fact, by the party being assessed, and they summarize rather than show. An auditor reading a narrative is reading your account of events, then deciding how much of it to trust.
Self-reported logs record activity and can be valuable operational evidence. But when their integrity, retention, and selection remain entirely under the operator’s control, an outside reviewer must also evaluate those controls. The log alone is not independently verifiable evidence of the event it describes.
These artifacts can answer important questions about intent, process, and observed activity. They do not necessarily answer the narrower evidentiary question: can another party check a bounded claim about this covered event, including who signed it and which fields are covered?
Intent versus evidence
The shorthand worth keeping: documentation states intent; operational evidence records bounded claims about particular events.
A policy says the guardrail should exist. A narrative explains how it is meant to work. A log reports activity. A signed artifact can add tamper evidence and a disclosed verification procedure for covered fields. That makes the signed claim checkable; it does not establish that the claim is complete or true, that every event was covered, or that the control was effective.
Why a stronger AI governance framework still leaves the gap open
It is tempting to treat this as a maturity problem: adopt a more rigorous AI governance framework, add more controls, and the gap closes. Most frameworks specify what to govern and what to document, rather than how to preserve independently checkable records of what a configured path reported.
You can map every requirement in a recognized framework and generate strong documentation for each while still lacking a portable artifact for a particular action. A signed record can add integrity and key attribution for its covered report; determining which component was active, whether the report was true, and whether the control worked still requires deployment and testing evidence.
This is the part that catches careful teams off guard. The documentation may be complete and the audit may have gone well. Then something goes wrong at runtime, whether a jailbreak, an unexpected tool call, or an output that should have been blocked, and the question turns evidentiary rather than procedural. “Show me what the configured path reported for this event.” At that moment, policy documents and operator-controlled logs may need additional integrity, provenance, and coverage evidence.
The five things mature operations actually need
When a security or governance team moves past describing controls toward making operational claims checkable, the same set of needs tends to surface:
- Explicit claim scope: which component and configuration the record identifies, and which outcome it reports for a covered event.
- Coverage accounting: what was in scope, what was excluded, and how denominators and bypass paths were derived.
- Tamper-evident telemetry: a change to covered bytes causes supported signature verification to fail, subject to the disclosed key, trust, and verification procedure.
- Independent verification: a disclosed procedure another party can run, with signer and witness provenance stated rather than assumed.
- Configured data minimization: a deployment path that can avoid routinely exporting protected content while retaining bounded verification material.
These are properties of the evidence system around a runtime artifact. The artifact alone cannot prove that collection was complete, that an omitted path did not exist, or that the signer was organisationally independent.
What evidence looks like: the OVERT receipt
This is the gap OVERT was written to address. OVERT is an open, royalty-free format for signed operational records. It gives an outside party a disclosed way to check supported signatures and covered fields; it does not make every statement in a record true or prove that the record set is complete.
The mechanism is deliberately undramatic. For an event that passes through a configured evidence path, the system emits a signed record, a receipt, containing bounded claims and verification metadata. A data-minimizing deployment can keep protected content inside the operator’s configured boundary while exporting hashes, outcomes, signatures, and other selected fields. The actual deployment data flow still has to be inspected.
Four commitments hold the standard together, and each one maps directly onto a weakness in conventional documentation:
- Explicit scope. A receipt states the bounded claim, covered fields, signer, and recorded outcome that a third party can check.
- Configured containment. Data minimization depends on which fields are selected and how the deployment is wired; a receipt is not proof that no other data path exists.
- Disclosed provenance. A reviewer can identify and validate supported signatures. A second signature or service-operated witness does not, by itself, establish organisational independence.
- Measurement, not adjective. Coverage is stated with denominators, exclusions, and bypass paths rather than words such as “robust” or “comprehensive”.
This is where attestation becomes more useful than an opaque dashboard: a reviewer can apply the verification procedure directly. The result establishes integrity and signer correspondence for covered fields, not the truth of every underlying claim or the effectiveness of the control.
Documentation still has a job
This is not an argument against AI governance documentation. Policies still set the intent that controls implement. Frameworks still organize the work. Narratives still explain context that a signed record, by design, does not. The narrower point is that documentation and operational evidence answer different questions. You need both.
The healthiest posture links the two: documentation names the intended rule and control; scoped operational records report what the configured evidence path observed for covered events. Reviewers still need coverage, deployment, and effectiveness evidence before drawing broader conclusions.
Where to start
You do not have to rebuild your governance program to close the evidence gap. The move is additive. Keep the documentation, and add a layer that produces verifiable proof at the points where your AI actually acts: the inference call, the tool call, and the agent boundary.
Start by asking one question of any control you currently document. If this were challenged tomorrow, what bounded claim about a covered event could another party check, and what would remain unproven? If the answer is only a policy and an operator-controlled log, add the integrity, provenance, and coverage evidence the review would require.
The standard is public at overt.is and at /standard; you can Verify a record to see which checks the artifact supports. To discuss a consequential workflow, Talk to us.