When AI-Assisted Diagnosis Is High-Risk
“Diagnostic AI” describes a broad set of products and workflows, not a single legal class. The EU AI Act analysis begins with the particular system’s intended purpose, whether it is itself a regulated product or safety component, which product law applies, and whether that product must undergo third-party conformity assessment.
Annex I Product Pathway (Article 6(1))
Article 6(1) classifies an AI system as high-risk when two conditions are met: it is a safety component of, or is itself, a product covered by Union harmonisation legislation listed in Annex I; and that product must undergo third-party conformity assessment under the relevant product law.
The Medical Device Regulation (MDR) 2017/745 and In Vitro Diagnostic Medical Devices Regulation (IVDR) 2017/746 appear in Annex I. Software may be a medical device or in-vitro diagnostic medical device based on its manufacturer-defined intended purpose. Device classification and the need for notified-body involvement then follow the applicable product law and rules; neither can be inferred from the marketing label “diagnostic AI” alone.
Medical-device status alone is not the entire Article 6(1) test. The third-party-conformity-assessment condition must also be satisfied. The result can differ across systems, intended purposes, classifications and pathways.
Annex III Is Not a General Healthcare Catch-All
Annex III identifies specified use cases, but it does not declare every AI system used in healthcare or diagnosis high-risk. For example, Annex III separately addresses certain emergency-call triage and health- or life-insurance risk-assessment uses. Those categories should not be expanded into a general diagnostic-AI rule.
A system outside the Annex I product pathway therefore needs its own Annex III and Article 6 analysis. Clinical importance or potential harm may inform risk management, but does not replace the regulation’s classification tests.
What Physician Oversight Does and Does Not Change
A physician’s final sign-off does not create a blanket exemption. Conversely, calling software “decision support” does not establish that it is high-risk. Intended purpose, product-law status, device classification and the actual high-risk pathway remain determinative.
Where a system is high-risk, Article 14 human-oversight requirements may apply. That is an obligation for a covered system, not a shortcut for determining whether the system is covered in the first place.
Types of AI-Assisted Diagnosis Covered
The following categories illustrate where a classification analysis is often needed. None is definitively high-risk by specialty name alone; intended purpose, product-law classification and the conformity pathway control.
Radiology AI
AI systems that analyze medical imaging including X-rays, CT scans, MRI scans, mammograms, and ultrasounds. These systems detect abnormalities, identify suspected malignancies, measure anatomical structures, and prioritize worklists. Examples include lung nodule detection, breast cancer screening AI, and fracture detection systems.
Pathology AI
AI systems that analyze digitized tissue samples and histopathology slides. Applications include cancer grading, tumor margin assessment, biomarker quantification, and automated cell counting. Digital pathology AI is increasingly used for prostate cancer Gleason scoring and breast cancer HER2 analysis.
Dermatology AI
AI systems that analyze skin lesions from dermoscopic images or clinical photographs. These systems assist in melanoma detection, skin cancer screening, and differential diagnosis of skin conditions. Consumer-facing skin analysis apps may also qualify if they make diagnostic claims.
Ophthalmology AI
AI systems analyzing retinal images for diabetic retinopathy screening, age-related macular degeneration detection, glaucoma risk assessment, and other ocular conditions. Fundus photography and OCT analysis systems are commonly deployed in screening programs.
Cardiology AI
AI systems analyzing electrocardiograms (ECGs/EKGs) for arrhythmia detection, atrial fibrillation screening, and cardiac event prediction. Also includes echocardiogram analysis AI and cardiac MRI interpretation systems.
Other Diagnostic Modalities
Other systems requiring case-specific analysis include endoscopy AI, pulmonary-function analysis, genomic analysis for disease prediction, clinical decision support and symptom checkers. Some may be regulated medical devices or IVDs; others may not be, depending on the claimed intended purpose and actual function.
Dual Regulation: MDR + EU AI Act
A diagnostic AI system covered by MDR or IVDR may also be a high-risk AI system under Article 6(1). Where both regimes apply, the provider or manufacturer must map the duties that attach to its role and system. It is inaccurate to assume every diagnostic tool is subject to every obligation in both regimes.
How the Regulations Interact
For covered Annex I product systems, the applicable product-law conformity procedure coordinates assessment of relevant AI Act requirements. The precise assessment route, responsible actor and notified-body scope depend on the product law, device class, operator role and transition rules. A CE mark or notified-body review should not be described as a universal conclusion about every AI Act duty.
| Requirement Area | MDR Obligation | AI Act Additional Requirement |
|---|---|---|
| Risk Management | ISO 14971 risk management process | AI-specific risk categories per Article 9 |
| Data Governance | Clinical data quality requirements | Training data governance per Article 10 |
| Technical Documentation | MDR Annex II/III documentation | AI-specific documentation per Article 11 |
| Logging | Post-market surveillance data | Purpose-appropriate automatic event logging per Article 12 |
| Human Oversight | Instructions for use | Specific oversight measures per Article 14 |
| Accuracy/Robustness | Clinical performance validation | Accuracy, robustness, cybersecurity per Article 15 |
Conformity Assessment Integration
For an Annex I product system that is high-risk, applicable AI Act requirements are coordinated through the relevant MDR or IVDR conformity procedure. That does not mean every diagnostic system needs a notified body, every existing assessment automatically covers the AI Act, or a notified body gives a standalone guarantee of compliance. Confirm the pathway and division of responsibilities for the specific product.
High-Risk Requirements (Articles 9 to 15)
If the diagnostic AI system is classified as high-risk, Articles 9 to 15 establish provider-facing system requirements. Their application and implementation remain system-, role- and pathway-specific:
Article 9: Risk Management System
Establish and maintain a risk management system throughout the AI system’s lifecycle. For diagnostic AI, this includes: identification of known and foreseeable risks (misdiagnosis, algorithmic bias across patient populations, failure modes), estimation and evaluation of risks, adoption of risk mitigation measures, and continuous monitoring of residual risks.
Article 10: Data and Data Governance
Training, validation, and testing datasets must meet quality criteria. For diagnostic AI: data must be representative of the intended patient population, training data provenance must be documented, statistical properties and potential biases must be examined, and appropriate data preparation steps must be applied.
Article 11: Technical Documentation
Comprehensive documentation enabling assessment of AI Act compliance. For diagnostic AI, this includes: detailed algorithm description, training methodology, validation approach, clinical performance data, integration specifications, and instructions for use by healthcare professionals.
Article 12: Record-Keeping (Logging)
The system must technically allow automatic recording of events over its lifetime to the extent appropriate to its intended purpose. Article 12 does not prescribe one universal diagnostic-AI event schema.
Article 13: Transparency and Information
AI systems must be designed to enable deployers to interpret outputs and use the system appropriately. For diagnostic AI: clear indication of AI involvement in diagnosis, explanation of confidence levels, disclosure of known limitations, and guidance on appropriate clinical contexts for use.
Article 14: Human Oversight
High-risk AI must be designed for effective human oversight. For diagnostic AI: clinicians must be able to understand AI outputs, override AI recommendations, and intervene in system operation. The system must not induce automation bias or over-reliance.
Article 15: Accuracy, Robustness, Cybersecurity
AI systems must achieve appropriate levels of accuracy, robustness against errors and attacks, and cybersecurity protection. For diagnostic AI: validated clinical accuracy across patient subgroups, robustness to imaging artifacts and edge cases, and protection against adversarial manipulation.
Article 12 Logging: Purpose-Appropriate Events, Not a Universal Field List
Article 12 requires covered high-risk AI systems to technically allow automatic recording of events throughout the system lifetime, to the extent appropriate to the intended purpose. The logging capability must support an appropriate level of traceability and facilitate post-market monitoring. The regulation does not generally require every invocation payload or every conceivable workflow field.
What Must Be Logged
At a high level, Article 12 logging should support:
- Traceability: an appropriate level of traceability for the system’s functioning over its lifetime
- Risk and modification review: identification of situations that may result in risk or amount to a substantial modification
- Post-market monitoring: records that facilitate provider monitoring and deployer oversight where the relevant provisions apply
Designing a Diagnostic-AI Event Schema
Inputs, outputs, confidence scores, model versions, user interactions, anomalies and configuration changes can be useful events in a particular diagnostic workflow. But Article 12 does not generally mandate that full list for every system. Providers should select and document purpose-appropriate events through the system’s risk, monitoring, technical-documentation and privacy analysis.
- Purpose: what traceability, risk-monitoring or post-market question the event helps answer
- Scope: which system boundary, workflow stage and actor the event represents
- Data minimization: whether sensitive clinical payloads can remain outside the evidence record
- Coverage: which invocations or pathways are instrumented, and which are not
- Interpretation: what the event establishes during review, and what it cannot
Separate Retention and Integrity Questions
Do not collapse distinct provisions into one “Article 12 retention” rule. Article 18 separately requires providers to retain specified documentation for ten years after the system has been placed on the market or put into service. Article 26 generally requires deployers to keep logs automatically generated by a high-risk system and under their control for at least six months, unless other applicable Union or national law provides otherwise. MDR, IVDR, device-specific and national rules can impose different records or longer periods.
Article 12 does not generally state that all logs must be “tamper-proof.” Integrity controls can be appropriate and may be required by another applicable rule or the system’s risk design, but the legal basis and assurance claim should be identified precisely.
What Signed Operational Records Can and Cannot Show
GLACIS can create signed records for selected, in-scope events at a configured workflow boundary. Those records can support integrity and traceability review for the fields actually captured. They do not prove that a control was effective, that every relevant workflow was covered, that all legal requirements were satisfied, that a system passed conformity assessment, or that a regulator will accept the evidence.
GLACIS relevance: Where a team has defined purpose-appropriate events, GLACIS can preserve signed operational evidence showing which configured supervision step and selected fields were recorded. The organization remains responsible for classification, event coverage, retention, control design, conformity and regulator-facing conclusions.
Notified Body Assessment Requirements
Notified-body involvement depends on the applicable MDR or IVDR classification and conformity-assessment route. Many diagnostic medical-device and IVD systems do require third-party assessment, but that conclusion cannot be made from the presence of AI or a diagnostic function alone.
When Notified Body Assessment is Required
Under MDR, Rule 11 and other applicable classification rules can place software in different classes based on intended purpose and the significance of the information it provides. IVDR uses its own classification rules. Illustrative outcomes under MDR can include:
- Class IIa: software providing information used to make diagnostic or therapeutic decisions, unless a higher class applies
- Class IIb: where a wrong decision could cause serious deterioration or require surgical intervention
- Class III: where a wrong decision could cause death or irreversible deterioration
Higher-risk classifications generally bring notified-body involvement, but assessment modules and sampling or technical-documentation review depend on the specific conformity route. Confirm the result for the actual device or IVD rather than treating this summary as a classification opinion.
Notified Body Competence for AI
A manufacturer should confirm that its selected notified body is designated for the relevant product scope and can address the AI Act requirements that enter the coordinated product-law assessment. Competence, designation and assessment scope are pathway-specific.
Plan the Pathway, Not a Generic Timeline
Cost, evidence scope and review timing vary by device class, notified body, conformity route, clinical or performance evidence and the maturity of the quality system. Obtain current estimates from the relevant notified body; a generic range is not a dependable planning assumption.
US Regulatory Comparison: FDA SaMD Guidance
For organizations operating in both the EU and US markets, understanding regulatory differences is essential for efficient compliance strategies.
FDA Approach to AI Diagnostics
FDA treatment is function- and product-specific. Some diagnostic AI functions are device software functions reviewed through existing device pathways; some clinical decision-support functions may fall outside the device definition, and enforcement policies can also matter. FDA has no separate horizontal AI Act:
- 510(k) Clearance: For AI systems substantially equivalent to predicate devices
- De Novo Classification: For novel AI systems without predicates but with reasonable safety/effectiveness
- Premarket Approval (PMA): For high-risk AI systems requiring clinical trial evidence
Emerging FDA AI/ML Guidance
The FDA has issued evolving guidance on AI/ML-based medical devices, including the Predetermined Change Control Plan (PCCP) framework allowing manufacturers to define anticipated algorithm updates that don’t require new submissions. This addresses the challenge of continuously learning AI systems.
Key Differences from EU Approach
| Aspect | EU (MDR + AI Act) | US (FDA) |
|---|---|---|
| Regulatory Structure | Dual regulation (MDR + AI Act) | Single framework (FDA device regulation) |
| AI-Specific Obligations | Explicit (Articles 9 to 15) | Integrated into general requirements |
| Logging Requirements | Purpose-appropriate automatic events for covered high-risk systems | Case-by-case (post-market surveillance) |
| Third-Party Assessment | Depends on product classification and pathway | FDA review pathway; no EU-style notified body |
| Algorithm Updates | Substantial modification triggers reassessment | PCCP allows predetermined changes |
Evidence Requirements for Regulators
A covered system’s evidence set depends on its intended purpose, classification, operator role, conformity route and applicable MDR, IVDR and AI Act obligations. Operational evidence can support review, but does not by itself demonstrate compliance.
Documentation Categories
- Design and Development: Algorithm architecture, training methodology, feature engineering, model selection rationale
- Data Governance: Training data provenance, quality measures, representativeness analysis, bias assessment
- Validation Evidence: Clinical performance data across patient subgroups, external validation studies, real-world performance monitoring
- Risk Management: Hazard analysis, failure mode assessment, residual risk evaluation, mitigation measures
- Operational Logs: Purpose-appropriate Article 12 events and clearly documented coverage boundaries
- Post-Market Surveillance: Ongoing performance monitoring, incident reports, corrective actions
The Evidence Boundary
Technical documentation, clinical or performance evidence, risk-management records and operational logs answer different questions. A signed runtime record can make selected events harder to alter unnoticed and easier to trace, but it cannot establish clinical effectiveness, control effectiveness, complete coverage, conformity or regulator acceptance without the rest of the evidence and assessment process.
Implementation Checklist
Use this checklist to scope a system-specific legal, product and evidence review. It is not a conformity checklist and does not replace regulatory counsel or the applicable notified body.
Pre-Compliance Assessment
- Document intended purpose and determine whether MDR, IVDR or another product law applies
- Determine device or IVD classification and whether notified-body involvement is required
- Confirm the AI Act classification, operator roles and applicable obligations with counsel
- Establish a compliance timeline working back from the applicable 2 August 2028 Annex I product-embedded date
Risk Management (Article 9)
- Document AI-specific risks beyond traditional medical device hazards
- Assess algorithmic bias across patient populations (age, sex, ethnicity, comorbidities)
- Define risk mitigation measures for identified AI-specific risks
Data Governance (Article 10)
- Document training data sources, provenance, and quality controls
- Conduct representativeness analysis of training data vs. intended population
- Establish data governance policies for ongoing data quality
Logging Infrastructure (Article 12)
- Define purpose-appropriate automatic events and the traceability question each event answers
- Map Article 18 documentation retention, the Article 26 deployer-log floor, and applicable device or national rules separately
- Document event coverage, exclusions, payload boundaries and monitoring ownership
- Select proportionate integrity controls for in-scope operational records
Human Oversight (Article 14)
- Design user interface to support clinician interpretation and override
- Develop training materials addressing automation bias risks
- Document intended clinical workflow and oversight mechanisms
Conformity Assessment
- If required, engage a notified body designated for the relevant product scope
- Map MDR or IVDR documentation to applicable AI Act requirements for the chosen pathway
- Compile clinical performance evidence for target indications
- Confirm applicable quality-management obligations and standards with qualified advisors
Frequently Asked Questions
Our AI only assists. The physician makes the final diagnosis. Is it still high-risk?
Possibly. Physician review does not itself exempt a system, but assistance does not make every system high-risk either. Classify the actual intended purpose, product-law status, device or IVD class, third-party-assessment requirement and AI Act pathway.
We already have MDR certification. Do we need to do anything additional for the AI Act?
A covered system may need to satisfy both MDR/IVDR and applicable AI Act requirements, including Articles 9 to 15. Under the AI Omnibus, relevant Annex I product-embedded high-risk obligations apply from 2 August 2028. Confirm classification, transition rules, and the coordinated conformity pathway with counsel and the notified body.
What if our diagnostic AI doesn’t qualify as a medical device?
Analyze its intended purpose and claims under MDR, IVDR and applicable national rules first. If it is outside the Annex I product pathway, assess whether a specific Annex III use case or another AI Act provision applies. Annex III is not a general healthcare catch-all.
How do we handle algorithm updates under the AI Act?
A substantial modification can trigger provider and conformity-assessment consequences. Define and document the change-control analysis for the specific system, update the applicable technical documentation, and record the purpose-appropriate events needed to trace the change. Do not assume that every model update has the same legal effect.
What are the penalties for non-compliance?
The AI Act uses tiered administrative fines, including a maximum of €15 million or 3% of worldwide annual turnover for certain violations, subject to the provision, actor and statutory calculation rules. Product-law enforcement and corrective measures are separate. Obtain advice for the specific role and alleged breach.
Do US-developed AI diagnostic systems need EU AI Act compliance?
The AI Act can apply to providers and deployers outside the EU in specified circumstances, including placing a system on the EU market or where the system’s output is used in the EU. The obligations still depend on role, scope, classification and any relevant exclusions or transition rules.
Next Steps
AI diagnostic systems require system-specific classification under the EU AI Act and applicable medical-device law. For covered Annex I product-embedded systems, relevant high-risk obligations apply from 2 August 2028. Organizations should begin pathway and evidence planning early.
Key actions: Document the system’s intended purpose, determine its MDR or IVDR and AI Act classification, map responsible roles, confirm whether a notified body is required, and define purpose-appropriate Article 12 events. GLACIS can support selected operational-evidence records after those boundaries are defined; it does not make the classification or conformity decision.