The Proof Gap in Healthcare AI
Why Compliance Claims Are No Longer Enough, and What to Demand Instead
Healthcare organizations need more than vendor compliance claims. Signed operational records can make selected control events independently checkable, while clinical effectiveness, complete workflow coverage, and legal compliance still require separate evidence.
Download White PaperYour vendors can document controls. Can they show checkable evidence about what ran?
Policy, security controls, and monitoring each answer part of the review. The remaining question is whether a covered event carries independently checkable evidence connecting the intended rule to the configured path’s reported control invocation and outcome, with execution and coverage corroborated separately.
Policy and controls
Define what should happen and apply controls at relevant workflow boundaries. Effectiveness must be tested.
Monitoring and investigation
Show selected events and support investigation. A vendor’s own logs remain assertions within its trust boundary.
Independently checkable records
Support integrity and signer attribution for selected fields from configured paths. They do not prove source truth, effectiveness, complete coverage, clinical quality, or compliance.
When “HIPAA Compliant” Isn’t Enough
A patient says “I had one beer at a wedding last month.” The AI writes: “Patient reports daily heroin use.”
The Failure Cascade
| Stage | What Happened | Evidence Available |
|---|---|---|
| Spoken | “I had one beer at a wedding last month.” | None retained |
| ASR Transcript | “I had one beer... heroin last month” | Possibly logged, not linked |
| LLM Processing | Interpreted as substance use disclosure | No trace of reasoning |
| Generated Note | “Patient reports daily heroin use...” | Final output only |
| EHR Write | Hallucinated diagnosis entered | Timestamp only |
What the Vendor Provided
- + 40-page architecture diagram
- + SOC 2 Type II attestation
- + API logs (HTTPS transmission)
- + PHI scanner configuration docs
What the Vendor Couldn’t Provide
- - Per-encounter trace of the processing pipeline
- - Evidence of which guardrails executed
- - Model version digests with timestamps
- - Cryptographically verifiable receipt
The Four Pillars of Inference-Level Evidence
The evidentiary standard healthcare organizations should demand from AI vendors before procurement approval.
Guardrail Execution Trace
Signed traces of the configured control identities, reported sequence and outcomes, and covered event-time fields. Surrounding evidence must establish execution, timing, effectiveness, and coverage.
Selected Decision Context
Purpose-selected references to prompts, redactions, retrieved data, and configuration state for covered outputs. These records do not reconstruct internal model reasoning or provide complete forensic context.
Independent Verifiability
Cryptographically signed, tamper-evident records whose supported fields third parties can check without relying only on a vendor dashboard. Key custody, source truth, effectiveness, and coverage remain separate questions.
Framework Anchoring
Reviewer cross-references to relevant ISO 42001, NIST AI RMF, and EU AI Act concepts. A mapping does not establish that a control objective or legal requirement is satisfied.
Get the Complete White Paper
16 pages of analysis including regulatory timeline, case studies, and the complete evidence framework.
Jennifer Shannon, MD
Chief Medical Officer, GLACIS Technologies
University of Washington-trained psychiatrist with extensive regulatory experience. Previously helped develop the first FDA-authorized AI diagnostic device for autism at Cognoa. She still practices clinically in Seattle and serves as courtesy teaching faculty at UW.