Healthcare AI Compliance Briefing

Executive Summary

Healthcare AI enters 2026 facing a convergence of regulatory deadlines, precedent-setting litigation, and increasingly sophisticated governance committee scrutiny. This briefing covers what matters most for JPM conversations:

• State AI laws — Colorado AI Act has a June 30, 2026 effective date; enforcement is currently stayed in xAI v. Weiser and replacement bill SB 26-189 (effective Jan 1, 2027 if enacted) is in committee. Either statute requires the same underlying evidence; either way, planning windows of 6–12 months still apply.
• Consent litigation — Sharp HealthCare lawsuit creates ambient AI scribe liability template
• HIPAA gaps — Standard BAAs don’t cover AI-specific risks; model training exposure
• Governance shift — Committees now asking for evidence, not just attestations

2025–2026 Regulatory Timeline

Healthcare AI faces a compressed compliance window. Here’s what’s coming and when:

FDA’s comprehensive AI-enabled device lifecycle guidance (issued draft January 2025) expected to finalize. Over 1,250 AI devices now authorized; PCCP framework already in place for modification management.

U.S. Magistrate Judge Cyrus Y. Chung (D. Colo., 1:26cv1515) granted a joint motion in xAI v. Weiser staying SB 24-205 enforcement through the 14th day after the court rules on xAI’s preliminary-injunction motion. DOJ has intervened.

Polis-backed replacement bill: drops mandatory impact assessments and the rebuttable-presumption framework, pivots to disclosure-and-recordkeeping, raises threshold to “materially influence,” pushes effective date to January 1, 2027. Healthcare AI remains in scope under either statute. Colorado adjourns May 13.

SB 24-205 (delayed from February 2026 per SB 25B-004) requires risk assessments, impact statements, and consumer disclosures for “high-risk AI systems” including healthcare decisions — subject to the federal-court stay above.

High-risk AI system requirements fully applicable. Any US company with EU patients or customers must comply.

Connecticut, Texas, Illinois, Washington, Oregon and New York are advancing state AI laws on different vectors (companion-chatbot disclosure, frontier-model safety, automated-decision-making). Patchwork compliance challenge emerging.

Key Insight for JPM

Organizations typically need 6–12 months to implement AI governance programs that satisfy the Colorado AI Act’s reasonable-care framework or its SB 26-189 disclosure successor. The April 2026 stay and the pending replacement reset the deadline, not the work — healthcare AI ends up under one of two regimes by mid-2027 either way, and the underlying NIST AI RMF / ISO 42001-aligned evidence layer is durable across both.

The Consent Litigation Wave

Ambient AI scribes have become one of healthcare’s most widely-adopted AI categories. They also became a significant liability exposure in 2025.

Sharp HealthCare: The Template Case

Filed November 2025, Saucedo v. Sharp HealthCare alleges that since Abridge’s deployment in April 2025, over 100,000 patient encounters may have been recorded without adequate consent. The complaint includes several allegations that create templates for future litigation:

• Auto-inserted consent statements — The AI allegedly inserted false attestations into medical records claiming patients had been "advised of and consented to" recording when they had not
• CIPA violations — California’s Invasion of Privacy Act allows $5,000 per violation (or three times actual damages) for wiretapping
• CMIA violations — Confidentiality of Medical Information Act claims for unauthorized disclosure
• Negligence claims — Failure to implement adequate consent procedures

Potential Liability Exposure

If courts count each recorded patient encounter as a separate CIPA violation, the $5,000 statutory penalty could result in exposure exceeding $500 million based on plaintiff’s estimates. Even at a fraction of this figure, exposure would dwarf typical HIPAA penalties.

The "Capability Test" Expansion

Perhaps more concerning is the emerging "capability test" from Ambriz v. Google LLC (2025). The court held that an AI vendor can be liable as a third-party eavesdropper if it merely possesses the technical capability to use intercepted data for its own purposes—regardless of whether it actually exercises that capability.

Most AI vendors’ terms of service reserve rights to use customer data for model training. Under the capability test, this reservation alone may create CIPA liability even if training never occurs.

HIPAA’s AI Blind Spots

HIPAA was written for fax machines and filing cabinets. While its principles apply to AI, significant gaps exist:

BAA Coverage Gaps

The Audit Trail Problem

HIPAA’s Security Rule requires audit controls for PHI access. For AI systems, this means logging not just who accessed data, but what the AI did with it:

• What PHI was sent to the model
• What the model output was
• What version of the model was used
• What safety controls executed
• Whether output was modified before display

Most AI platforms provide application-level access logs. They don’t provide inference-level audit trails. This creates a compliance gap that governance committees are increasingly identifying.

What Governance Committees Are Asking

Health system AI governance committees have evolved rapidly. In 2023, they asked if you had policies. In 2024, they asked about your certifications. In 2025, they’re asking for evidence.

The New Questions

Based on conversations with health system CISOs, CMIOs, and compliance officers, here are the questions that now regularly appear in procurement reviews:

1. "Can you demonstrate that PHI was not used to train your model?" — Not a policy statement; actual technical evidence
2. "What happens if your AI hallucinates clinical information?" — Looking for incident response, not just disclaimers
3. "How do you prove content filtering ran on a specific patient interaction?" — The execution evidence question
4. "What’s your consent workflow for AI-assisted documentation?" — Sharp lawsuit made this mandatory
5. "How will you comply with state AI disclosure requirements?" — Colorado AI Act preparation

From Attestation to Evidence

The fundamental shift is from trust to verify. Policy documents and attestation letters no longer satisfy sophisticated governance committees. They want:

• Cryptographic proof that controls executed on specific interactions
• Audit trails that can be independently verified
• Real-time dashboards showing compliance status, not annual reports
• Incident evidence that can be produced within hours, not weeks

This is the "evidence gap" that’s stalling healthcare AI procurement. Vendors can describe their controls but can’t prove they ran.

Ambient AI Scribe: The 2025 Flashpoint

Ambient AI clinical documentation is healthcare’s killer app—and its biggest compliance headache. Every major health system is either deploying, piloting, or evaluating ambient scribes. Most are underestimating the liability exposure.

The Consent Challenge

In California and other "all-party consent" states, recording a conversation without all parties’ consent is illegal. Doctor-patient conversations have heightened protection as "confidential communications."

Yet many ambient AI deployments rely on:

• Implicit consent — "The patient didn’t object when I mentioned we use AI"
• General consent forms — Buried in admission paperwork signed weeks earlier
• Verbal mention — Not documented, hard to prove in litigation

The Sharp lawsuit shows where this leads. Explicit, documented, per-encounter consent is becoming the standard.

Best Practice Framework

Ambient AI Consent Checklist

• ☐ Written consent form specific to AI recording (not general treatment consent)
• ☐ Consent obtained before recording begins, not retrospectively
• ☐ Consent captured in EHR with timestamp
• ☐ Patient can review and request deletion of recordings
• ☐ Clear disclosure of what AI does with the recording
• ☐ Opt-out doesn’t affect care quality

Let’s Talk at JPM

We help healthcare AI vendors and health systems generate verifiable compliance evidence. Meet with us in San Francisco, January 12–15.

Recommendations for 2026

For Healthcare AI Vendors

1. Upgrade your BAA — Standard cloud BAAs don’t cover AI-specific risks. Work with counsel to add model training prohibitions, inference logging requirements, and AI incident definitions.
2. Build evidence infrastructure — Governance committees want proof controls ran. Implement inference-level logging with cryptographic verification.
3. Prepare for state AI laws — Map your products against Colorado’s high-risk categories under SB 24-205 and SB 26-189 simultaneously. Either statute requires the same underlying evidence; start the impact-assessment work now even though the deadline is in flux.
4. Document consent workflows — For ambient AI, create auditable consent capture that can withstand litigation discovery.

For Health Systems

1. Audit existing AI deployments — Particularly ambient scribes. Verify consent procedures meet CIPA/CMIA requirements.
2. Strengthen vendor diligence — Add AI-specific questions to security questionnaires. Require evidence, not just attestations.
3. Establish AI governance — If you don’t have a formal AI governance committee, create one. If you do, update its charter for 2026 requirements.
4. Plan for state law compliance — Colorado AI Act affects any AI used for healthcare decisions. Map your exposure.

Related Resources