AI Governance and Runtime Assurance Glossary

Runtime assurance

The vocabulary for what an AI system does while it is running, and for the record it leaves behind.

Runtime assurance #

Runtime assurance is the practice of keeping awareness, control, evidence, and insight over an AI system while it is operating, rather than only before it ships.

Pre-deployment work describes a system as it was on the day it was examined. A model evaluation, a red-team exercise, a risk assessment, a signed policy: each is a photograph. Runtime assurance covers the much longer period after the photograph was taken, during which the model was updated, the prompt was edited, a new tool was connected, and the behavior moved.

It answers a different question from whether a system was approved. The question is what it did, under which controls, and whether anyone outside the organization can check the answer.

Governed action #

A governed action is any AI step that a runtime control was asked to evaluate: a model inference, a tool call, a retrieval, or an agent decision.

It is the unit a receipt describes, and it is deliberately narrower than everything the system did. If a control evaluated the step, there is a receipt for it. If the step was out of scope, there is no receipt, and an honest evidence pack states the scope it covered rather than implying it covered everything.

Scope a reviewer can read is worth more than coverage a reviewer has to assume. Vendors who blur the two are usually hiding the gap between the actions they govern and the actions their system takes.

Runtime control #

A runtime control is a rule evaluated at the moment an AI system acts, which can allow, block, redact, restrict, escalate, or require human review.

A control that exists only in a policy document is a statement of intent. A runtime control is a decision point in the execution path, and the difference shows up the first time somebody asks for proof. An auditor can be shown a policy saying protected health information must not leave the environment, or a record that a redaction control ran on a specific action at a specific time and returned a specific outcome. Only one of those is evidence.

Action boundary #

The action boundary is the point in an AI system where a decision becomes an effect: output leaves the model, a tool is invoked, a record is written, or a message reaches a person.

It is the last place where a control can still change the outcome, which makes it the only genuinely useful place to put one. Instrumentation downstream of the boundary can describe what happened. A control at the boundary can prevent it.

The boundary is specific to the system. For an ambient clinical scribe it is model egress, the instant a draft note would leave the environment. For an agent it is the tool call. For a screening workflow it is the moment a recommendation reaches the person who will act on it.

Runtime receipt #

A runtime receipt is a signed, hash-chained, content-free record of one governed action, stating which controls ran, what each decided, and what the input and output were committed to.

A receipt written to the OVERT standard carries a receipt identifier, the format version, the named workflow, a signed timestamp, the outcome of each control that ran, the guardrail action together with how many rules were evaluated and how many triggered, a SHA-256 hash of the input, a SHA-256 hash of the output, one or more Ed25519 signatures, and the receipt’s position in a hash chain alongside the hash of the one before it.

It contains no prompt text, no output text, and no records. Only cryptographic commitments to them. A receipt proves faithful recording of what a control did under an enrolled identity; it is never a compliance certificate and never a safety guarantee.

Witnessed receipt #

A witnessed receipt is one that an independent party has countersigned and bound into a transparency log, so that its existence at a point in time does not rest on the operator’s word alone.

Not every receipt is witnessed, and the distinction is worth being pedantic about. Receipts minted through the Glacis portal are operator-signed and then countersigned by an independent Glacis witness, with an inclusion proof binding them into a log. Receipts minted offline through the SDK are self-signed, and a verifier should say exactly that rather than coloring them green.

The difference is between asserting a signature and having somebody else confirm the signature existed when it is claimed to have existed.

Self-signed receipt #

A self-signed receipt carries a single signature made with a key the operator holds, binding its own fields together honestly but without any independent countersignature.

This is not weak evidence. It is precisely scoped evidence. Any later edit to the record breaks the signature, and the hash chain still exposes a deleted or reordered entry. What a self-signed receipt cannot rule out is that the whole chain was produced later than it claims.

A verifier that reports which of the two it is looking at is doing its job. One that treats both as a green tick has quietly destroyed the value of the witnessed badge.

Arbiter #

The arbiter is the component that runs inside the operator’s own infrastructure, evaluates content in place at the action boundary, and signs the resulting receipt.

Its location is the whole argument. Because it sits next to the model or agent it governs, prompts, outputs, documents, and records are examined where they already are. What crosses the network is a hash, a control outcome, and a signature, which is what makes the egress property structural rather than a setting somebody could change.

Evidence pack #

An evidence pack is an artifact assembled from signed runtime receipts and handed to an auditor, a customer’s security reviewer, a regulator, or an underwriter.

A pack typically gathers a coverage summary, a control-execution report, a framework mapping, the underlying audit trail, and a machine-readable export. Framework mapping is informational for a reviewer’s reference and is not a certification claim.

The property that matters is that the reviewer does not have to trust the pack. Each receipt inside it can be checked independently, which turns a document that asks to be believed into a document that can be tested.

Zero sensitive-data egress #

Zero sensitive-data egress means the design has no path by which prompts, outputs, or customer records travel off the operator’s infrastructure; only hashes, control outcomes, signatures, and metadata leave.

The phrase deserves to be read literally, because most products that say something adjacent mean either that they do not retain the data or that they encrypt it in transit. Both still require the data to leave.

The reason to hold the harder line is simple: proving that a control ran should not create a second copy of the thing the control was protecting. An evidence trail that introduces a new egress path has taken away with one hand what it offered with the other.

Attested-blind enclave #

An attested-blind enclave is a deployment mode in which the vendor can demonstrate cryptographically that it is unable to read customer payloads, rather than merely promising not to.

The enclave produces a hardware-rooted attestation describing exactly what code is running and what it can reach, which a customer checks before sending anything. The distinction a reviewer cares about is between a contractual commitment and a structural one. A contract is enforceable after a breach; an enclave attestation is checkable before one.

Verification and the OVERT standard

The cryptography a receipt rests on, and the standard that lets somebody else check it without asking permission.

OVERT #

OVERT, short for Observable Verification Evidence for Runtime Trust, is the open, royalty-free standard that defines the runtime receipt schema, the signing rules, and the verification procedure.

It is published at overt.is with an intellectual property policy, a version history, and machine-readable feeds. Publishing it is what stops a receipt being a proprietary blob: an auditor, a customer, or a competitor can implement a conformant verifier straight from the specification, with no involvement from the vendor whose receipts they are checking.

Glacis authored the standard and does not own it in any sense that constrains an implementer. A proof format controlled by the party being examined is not a proof format.

Attestation #

Attestation is a signed statement by one party that a specific thing was true at a specific time, checkable by a third party who does not trust the signer.

In AI systems the thing attested is usually the execution of a control, not the quality of an output. That is a deliberately modest claim, and it is why attestation is not certification. Attestation records that something happened, under a named identity, in a form that later editing would break. Certification is a judgment about whether what happened was good enough, and it is made by a person or a body, never by a signature.

Ed25519 #

Ed25519 is a public-key digital signature scheme, widely deployed and fast enough to sign every action in a production AI workflow without becoming the bottleneck.

It produces 64-byte signatures from 32-byte keys, has no dependence on a per-signature random value, and is implemented natively by WebCrypto in current browsers. That last property is what allows a receipt to be checked in a reviewer’s own browser with no install, no account, and no call home. OVERT uses it as the signing primitive.

SHA-256 #

SHA-256 is a cryptographic hash function that reduces any input to a fixed 32-byte digest for which finding a second matching input is computationally infeasible.

Inside a receipt it does the work that makes content-free evidence possible. The receipt stores the digest of the prompt and the digest of the output rather than the text of either. Anyone holding the original can recompute the digest and confirm the match; anyone who does not hold it learns nothing from the digest itself.

Hash chain #

A hash chain links each record to the one before it by including the previous record’s hash, so that removing, reordering, or editing any entry invalidates every entry after it.

This is what turns a pile of individually signed records into a sequence with integrity. An unchained log can be silently pruned: the awkward entry is simply not there, and nothing in the remaining records shows a gap. Chaining converts deletion into a detectable event.

That property is what record-keeping obligations are reaching for when they ask for logs that cannot be altered. Nothing prevents a determined operator from destroying a log; a chain ensures the destruction cannot be made to look like it never happened.

Inclusion proof #

An inclusion proof is a short cryptographic path showing that a specific record is part of a larger log, without requiring the verifier to download the whole log.

It is built from a Merkle tree: hash the records in pairs, hash those pairs, and continue to a single root value. Proving that one record belongs under that root needs only the handful of sibling hashes along its path, which is a few hundred bytes no matter how large the log has grown.

Certificate Transparency made the pattern ordinary for the public web. Runtime evidence borrows it unchanged.

Transparency log #

A transparency log is an append-only, publicly checkable record whose operator can be caught if it ever removes or rewrites an entry.

The design assumption is adversarial. The log operator is not trusted; the log operator is audited. Consistency proofs let any observer confirm that today’s log is a strict extension of the one they saw yesterday, so a quiet rewrite is detectable by anyone who kept the earlier root value.

This is the mechanism that makes a witnessed receipt mean more than a countersignature. The witness is itself checkable.

Canonicalization #

Canonicalization is the process of rewriting structured data into one unambiguous byte form, so that two systems signing the same content produce the same signature.

Without it, verification fails for reasons that have nothing to do with tampering: a reordered JSON key, a different Unicode escape, a trailing zero on a number, a stray space. OVERT specifies the JSON Canonicalization Scheme defined in RFC 8785, which fixes key ordering and number formatting.

It is the least interesting part of a receipt format and the part that decides whether two independent implementations can actually check each other’s work.

Verifier #

A verifier is a program that checks a receipt against the standard and reports what it can confirm and what it cannot.

A good verifier is uncooperative. It checks signature validity, the hash binding between fields, structural and temporal consistency, and any inclusion proof present, and it then reports a self-signed receipt as self-signed instead of as passing.

Verification at Glacis runs in the browser, requires no account, and is never metered. A verification step the vendor can switch off, price, or observe is not independent verification.

Content-free evidence #

Content-free evidence proves something about data without containing the data, by committing to it with a hash instead of copying it.

It inverts the usual trade-off in audit. Traditionally, more evidence means more copies of sensitive material sitting in more places, each one a new opportunity for it to leak. Content-free evidence is safe to hand over, which is what lets a health system’s reviewer examine a vendor’s controls without a data-sharing agreement between them.

AI systems that act

Terms for the shift from models that answer to systems that do things, which is where governance stops being a paperwork exercise.

Agentic AI #

Agentic AI describes systems that plan and carry out multi-step work by calling tools, holding credentials, and reacting to results, rather than returning a single answer.

The shift matters because the failure mode changes shape. A chatbot that is wrong produces bad text and a person decides what to do about it. An agent that is wrong takes the action: it sends the message, moves the money, updates the record. Everything that made review tractable for a single prompt and response gets harder, because the decision points multiply and almost none of them are seen by anyone.

AI agent #

An AI agent is a model-driven process that is given a goal, chooses its own sequence of steps toward it, and acts on systems outside itself.

For governance, the interesting property is not the model. It is the credential. An agent that can read a calendar is a convenience. An agent holding an API token for a records system is a user, and should be governed as one, with an identity, a scope of permitted actions, and a record of what it actually did with them.

Tool call #

A tool call is the moment a model stops generating text and invokes external code: a search, a database write, an email send, a payment.

It is the highest-value control point in an agentic system, because it is exactly where language turns into consequence. It is also where the case for a signed record is easiest to make. The parameters of a tool call are a decision the organization may have to explain months later, and they were produced by a component that is not deterministic.

Model Context Protocol #

The Model Context Protocol is an open protocol for connecting models to external tools and data sources through a common interface.

Its adoption turns tool access into infrastructure, which is excellent for interoperability and consequential for security. A protocol layer that every tool call passes through is either a convenient cache or a policy enforcement point, and which one it becomes is a decision somebody has to make deliberately.

Non-human identity #

A non-human identity is a credential held by software rather than a person: a service account, an API key, a workload identity, or an agent’s token.

These outnumber human accounts in most enterprises by a wide margin, and they are governed far less carefully: rarely rotated, frequently over-scoped, and seldom attributable to a specific action after the fact. An agent inherits every one of those weaknesses and adds a new one, because what the credential will be used for is decided at runtime by a model.

Human in the loop #

Human in the loop describes a design in which a person must approve, or can intervene in, a step before it takes effect.

It is the control most often claimed and least often evidenced. Two questions decide whether it is real. Was the person given enough context and enough time to actually decide, or were they presented with a default and a deadline. And is there a record showing that review happened on this specific action, rather than on the process in general.

Without the second, the control cannot be tested, and reviewers have started treating untestable controls as absent.

Failure modes

The ways AI systems go wrong in production, stated without the vendor habit of implying each one has been solved.

Prompt injection #

Prompt injection is an attack in which instructions hidden in the data a model reads are treated by the model as instructions from its operator.

There is no complete fix, and claims to the contrary should be treated as a warning sign. The model has no reliable way to distinguish content from command, because both arrive as text in the same context window. Indirect prompt injection, where the payload sits inside a retrieved document, a web page, or an email the agent processes, is the harder variant and the one that scales.

The defensible posture is containment plus proof: constrain what an injected instruction can reach, and keep a record of what the controls did when one arrived.

Jailbreak #

A jailbreak is a prompt that persuades a model to disregard its own safety training and produce output it was tuned to refuse.

Jailbreaks differ from prompt injection in what they subvert. Injection overrides the operator’s instructions using untrusted data; a jailbreak overrides the model’s own alignment using the user’s own turn. The practical worry for a deployer is transferability. A technique published on Monday works across many deployments of the same base model by Wednesday, which is precisely why a point-in-time evaluation ages so badly.

Excessive agency #

Excessive agency is the condition of an AI system holding more permission, autonomy, or functionality than its task requires.

Named in the OWASP Top 10 for LLM applications, it is the reason an ordinary model error becomes an extraordinary incident. The mistake is unremarkable; the blast radius is not. The remedy is unglamorous and comes straight from conventional access control: narrow the tool surface, scope the credential to the task, and require confirmation for anything irreversible.

What is new is the need to prove the narrowing was in force at the time of the incident rather than added in response to it.

Model drift #

Model drift is the gradual divergence between how a model behaved when it was validated and how it behaves in production.

It has two sources. The world moves, so the distribution of inputs stops matching the one the model was validated against. Or the model moves, because the vendor shipped an update that nobody downstream approved. The second is the newer problem and the harder one, since the deployer often learns about it from behavior rather than from a release note.

Point-in-time approval does not survive either. Where behavior is recorded continuously, a change after a vendor update surfaces in the evidence rather than being discovered months later by a customer.

Data poisoning #

Data poisoning is the deliberate corruption of training, fine-tuning, or retrieval data so that a model learns a behavior the attacker chose.

Retrieval poisoning is the variant most deployers will actually meet, because nobody needs access to a training run if they can get a document into the index the agent searches. It is a supply-chain problem, and it is answered the way supply-chain problems are answered: provenance for what goes in, and a record of what came out.

Shadow AI #

Shadow AI is the use of AI tools inside an organization without the knowledge or approval of the people accountable for the risk.

It is rarely defiance. It is usually a capable tool being faster than the process for approving it. The governance failure it exposes is the same one behind shadow IT a decade ago: the register lists what was approved, not what is running.

An inventory assembled from what systems actually did is the only kind that stays accurate, because it does not depend on anyone remembering to declare something.

Governance practice

Categories of tooling and activity that are routinely confused with one another, and with evidence.

AI governance #

AI governance is the set of structures by which an organization decides what its AI systems may do, who is accountable, and how it demonstrates that the rules were followed.

Most programs are strong on the first two clauses and thin on the third. Policies, risk tiers, an inventory, and an approval workflow describe intent very well. Demonstrating adherence requires artifacts produced by the running system, and the distance between those two things is where most enterprise security reviews now stall.

AI guardrails #

AI guardrails are filters and policies applied to model input or output to block, redact, or reroute behavior that is not permitted.

They are necessary, and they are not evidence. A guardrail that fires and leaves nothing behind is an operational feature: a reviewer has to take on faith that it was enabled, correctly configured, and running on the day in question. A guardrail whose decision is signed and independently checkable is a different artifact, and answers a different question.

AI audit trail #

An AI audit trail is a durable, ordered record of what an AI system did and which controls governed each step.

The word audit sets a higher bar than the word log. An application log is written by the system for its operators, and can be rotated, filtered, or amended without leaving a trace. An audit trail has to survive the interest of the party it describes, which in practice means signing and chaining rather than retention policy alone.

AI observability #

AI observability is instrumentation that lets an operator understand how an AI system is behaving: traces, latencies, token counts, evaluation scores, and error rates.

It is genuinely useful and it is self-reported. The party being examined produced the record and could in principle have amended it, which is unremarkable for operating a service and weak in an adversarial review.

Observability and runtime evidence answer different questions at different layers. A receipt sits underneath observability rather than replacing it, and a team that has one still wants the other.

AI red teaming #

AI red teaming is structured adversarial testing that tries to make a system misbehave before an attacker or an ordinary user does.

It produces findings about one version of a system at one moment. That is valuable, and it is not coverage: the model updates, the prompt changes, a tool is added, and the configuration that was tested no longer exists.

Red teaming and runtime evidence are complements. One finds the weakness; the other shows the fix was still in force six months later.

AI bill of materials #

An AI bill of materials is an inventory of the components in an AI system — models, datasets, weights, prompts, tools, and dependencies — together with their provenance.

It extends the software bill of materials idea to the parts of an AI system a conventional dependency manifest misses entirely. Procurement in medical devices and the public sector increasingly asks for one.

Its limit is the same as its predecessor’s. It describes what a system is made of. It says nothing about what a deployed instance actually did.

Regulation and frameworks

The obligations that create demand for runtime records, with the record-keeping clause of each one pulled to the front.

EU AI Act #

The EU AI Act is the European Union’s horizontal regulation for artificial intelligence, which sorts systems by risk and attaches obligations, including record-keeping, to the higher tiers.

It reaches providers and deployers placing systems on the EU market regardless of where they are established, and its duties phase in over several years rather than arriving at once. For a vendor, most of the practical weight lands on four things: high-risk classification, technical documentation, post-market monitoring, and the automatic logging duty in Article 12.

High-risk AI system #

A high-risk AI system, under the EU AI Act, is one used in a listed sensitive context — medical devices, employment, credit, education, essential services, law enforcement and others — where failure carries consequences for a person’s rights or safety.

Classification is the first question a program has to answer, because everything else follows from it: risk management, data governance, technical documentation, human oversight, accuracy and robustness, and automatically generated logs. Getting the tier wrong is expensive in both directions, and the temptation to argue a system out of scope tends to produce a worse outcome than accepting it.

EU AI Act Article 12 #

Article 12 of the EU AI Act requires high-risk AI systems to technically allow the automatic recording of events over their lifetime, at a level of traceability appropriate to their intended purpose.

Three words in that sentence carry the weight. Automatic rules out a record assembled by hand after an incident. Lifetime rules out logging that begins when somebody remembers to switch it on. Traceability sets the standard for what the record must support, which is reconstructing a specific decision rather than summarizing a month of them.

Hash-chained receipts are one way to answer the obligation. The obligation itself is technology-neutral, and any vendor claiming their product satisfies it on its own is overstating the case.

Conformity assessment #

A conformity assessment is the procedure by which a provider demonstrates that a high-risk AI system meets its regulatory requirements before it is placed on the market.

Depending on the system it is carried out by the provider itself or by a notified body, and it concludes in a declaration of conformity and CE marking. It is a point-in-time judgment about a documented system, which is exactly why post-market monitoring exists alongside it, and why what the system does after the assessment is signed is a separate evidentiary problem.

ISO/IEC 42001 #

ISO/IEC 42001 is the international management-system standard for artificial intelligence, certifiable by an accredited body in the same shape as ISO/IEC 27001.

It certifies a management system, never a model. An auditor confirms that the organization has set objectives, assigned roles, assessed impacts, and operates the controls it claimed to operate. Where it bites at runtime is that last clause: the auditor asks for records showing the controls ran, and a policy document is not a record.

NIST AI RMF #

The NIST AI Risk Management Framework is a voluntary United States framework organized around four functions — Govern, Map, Measure, and Manage — for identifying and managing AI risk.

Being voluntary turns out to be its strength. It is spreading through contracts, vendor questionnaires, and procurement language faster than any mandate could move. Measure and Manage are the functions that demand runtime artifacts, and they are the two that most programs currently answer with intentions.

SR 11-7 #

SR 11-7 is the United States supervisory guidance on model risk management, issued in 2011 and now the default lens applied to AI inside regulated financial institutions.

Its core demands predate modern machine learning: effective challenge, independent validation, ongoing monitoring, and a maintained model inventory. Several translate awkwardly to models a bank did not build and cannot inspect. Ongoing monitoring is the one that translates cleanly, and it is where runtime evidence has the most obvious role.

OSCAL #

OSCAL is a NIST-developed set of machine-readable formats for expressing security controls, system security plans, and assessment results.

It exists because control evidence has historically traveled as prose inside documents, which cannot be diffed, queried, or checked by a machine. Exporting runtime evidence into OSCAL lets a reviewer’s own tooling consume control results directly, instead of a person re-keying them out of a PDF and introducing errors on the way.

Colorado AI Act #

The Colorado AI Act is a United States state law imposing duties of reasonable care on developers and deployers of AI systems that make consequential decisions about people.

It is the most EU-shaped of the US state laws, with a risk management program, impact assessments, notice to affected individuals, and rights of explanation and correction. That makes it the practical bellwether for what a US posture will look like. It has been amended and its start date moved more than once, so effective dates should always be checked against the current text rather than a summary.

Regulated clinical AI

Terms that decide how AI reaches patients, and what a health system asks a vendor for before it does.

Predetermined Change Control Plan #

A Predetermined Change Control Plan is an FDA mechanism that lets a manufacturer specify in advance how an AI-enabled device may change after authorization, and how each change will be validated.

It exists because the alternative, a fresh submission for every model update, is incompatible with how machine-learning products improve. The bargain is specificity: the plan states the scope of permitted modification, the methods for validating each type, and an assessment of the impact.

Showing that the plan was followed is a runtime question rather than a documentation question, which is why change records and control-execution evidence matter long after the submission is accepted.

De Novo #

De Novo is the FDA pathway for a novel device of low to moderate risk that has no legally marketed predicate to claim substantial equivalence to.

Many AI-enabled diagnostics have reached the market this way, for the plain reason that there was nothing before them to be equivalent to. A granted De Novo creates the classification that later devices can then claim equivalence to, which is why the first authorization in a category matters well beyond the company that obtained it.

Ambient clinical documentation #

Ambient clinical documentation is the use of AI to listen to a clinical encounter and draft the note, so that the clinician writes less of it.

It is the most widely deployed category of clinical AI and the one whose risk surface is most often misread. The model hears the entire consultation, so the control point that matters is the egress boundary, at the instant a draft would leave the environment.

The questions health-system reviewers actually ask follow from that: what left, under which controls, and how is any of it shown rather than asserted.

Business associate agreement #

A business associate agreement is the HIPAA contract that must be in place before a vendor may handle protected health information on a covered entity’s behalf.

It is a necessary condition and a weak signal. Signing one commits a vendor to safeguards and to breach notification. It demonstrates nothing about whether any particular safeguard ran on any particular day.

That gap is why health-system reviewers have moved on from asking whether a BAA exists to asking what evidence exists for the controls it promises.

Check a definition against the thing

Definitions are only worth as much as the artifacts behind them. Every term above describes something that can be inspected, and the fastest way to test whether a vocabulary is honest is to look at what it produces.

Read a receipt field by field — what each element commits to, and what it does not
Verify one yourself — in the browser, no account, nothing sent anywhere
The OVERT standard — the schema, the signing rules, and the verification procedure
Questions we get asked — including the ones where the honest answer is no
Research and regulatory guides — longer treatments of most terms on this page