Healthcare AI 2025

FRANCE Law and Practice Contributed by: Liliana Eskenazi, Julie Ernewein and Pauline Lecrenais, Fréget Glaser et Associés

5.3 Bias and Fairness Addressing Algorithmic Bias in Healthcare AI The AI Act requires high-risk AI systems to be subject to risk assessments targeting the bias and discrimi - nation which can result from unrepresentative data, clinical assumptions or algorithmic limitations. Testing, Monitoring, and Mitigation of Bias The AI Act requires developers to test AI systems with large, diverse, and representative datasets, and to continuously monitor performance to detect bias affecting health and safety and implement mitigation strategies throughout the AI lifecycle. Similarly, under the MDR, medical AI must comply with strict safety and performance standards. Protections for Vulnerable Populations and Data Diversity Requirements French and EU regulations emphasise non-discrim - ination and require demographic diversity in training data to avoid biased outcomes. French ethical guide - lines prioritise justice and inclusivity in dataset devel - opment, particularly regarding vulnerable populations.

• The Ethics by Design Guide (2022) outlines five core principles (non-maleficence, justice, auton - omy, transparency and sustainability) and applies them across five AI development stages. • The 2025 Implementation Guide for Ethical AI Systems in Healthcare provides detailed, phase- aligned ethical criteria tailored to developers and providers (guide submitted for public consultation from 12 May to 6 June 2025). Binding vs Voluntary Standards While GDPR, MDR, and French law impose manda - tory rules, other guidelines remain voluntary but are encouraged to foster responsible innovation. They may later be incorporated into formal regulatory standards. 5.2 Transparency and Explainability Transparency and Explainability Requirements for Healthcare AI Systems In France, as in the broader European Union, health - care AI systems are subject to strict transparency and explainability obligations aimed at protecting patients’ rights and ensuring trust in AI-assisted care. Disclosure to Patients Healthcare providers are required to inform patients when AI technologies are used in their diagnosis, treatment or care management (Article L. 4001-3 of the French Public Health Code and GDPR). Information Available to Healthcare Providers and Patients Healthcare professionals must have access to clear, understandable information on how the AI system works – its scope, performance, errors and limitations – so they can maintain clinical judgment, assess the relevance of AI-generated results, and justify medical decisions. While detailed technical disclosures are not gener - ally required due to IP and trade secret protections, the level of explainability must be sufficient for both healthcare professionals and patients to understand function of the AI and its impact on care decisions.

Health Equity in Regulatory Frameworks Health equity is increasingly reflected in:

• transparency and explainability requirements to help clinicians detect biased AI recommendations; • outcome monitoring to prevent worsening of exist - ing disparities; and • public efforts to ensure equitable access to AI tools across populations and regions. French health authorities/institutions also support research and pilot programmes to validate AI in diverse clinical settings, ensuring fair and effective use. 5.4 Human Oversight Human Oversight Requirements for Healthcare AI Systems French and EU regulations emphasise human over - sight as essential, particularly for high-risk healthcare AI. Limits on Autonomous AI Decision-Making Current rules generally prohibit fully autonomous AI in healthcare. The AI Act mandates that high-risk sys -

CHAMBERS.COM