USA Law and Practice Contributed by: Nadia de la Houssaye, Andy Lee, Jason Loring and Graham Ryan, Jones Walker LLP
PCCPs The guidance recommends information to include in a PCCP as part of a marketing submission for a medical device using AI. The PCCP should include a descrip - tion of the device’s planned modifications; methods to develop, validate and implement the modifications; and an assessment of the modification’s impacts. This innovative approach enables AI developers to modify their systems without additional pre-market submis - sions when changes fall within predetermined param - eters. Clinical Evidence Requirements AI system developers must provide clinical evidence demonstrating safety and effectiveness for intended uses. Evidence requirements vary based on risk clas - sification, with higher-risk systems requiring more extensive clinical validation. The FDA emphasises real-world evidence and post-market surveillance. Expedited Pathways The FDA provides several expedited pathways for breakthrough medical devices, including AI systems, that address unmet medical needs or provide sig - nificant advantages over existing treatments. These pathways offer enhanced FDA communication and expedited review timelines while maintaining safety and effectiveness standards. 2.4 Software as a Medical Device (SaMD) Regulatory Framework for AI-Based SaMD On 6 January 2025, the FDA published the Draft Guid - ance: Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations. This comprehensive guidance addresses unique challenges posed by AI- enabled software medical devices. Other issues include the following. • Continuous learning systems: The FDA’s traditional paradigm of medical device regulation was not designed for adaptive AI and ML technologies. • Algorithm transparency and explainability: Health - care AI systems must provide sufficient transpar - ency to enable healthcare providers to understand system recommendations and limitations. The FDA emphasises the importance of explainable AI
that allows clinicians to understand the reasoning behind algorithmic recommendations. • Training data requirements: Clinical study partici - pants and datasets should be representative of the intended patient population. AI-based SaMD developers must ensure training datasets avoid bias and ensure generalisability across diverse clinical settings. • Post-market surveillance requirements. AI-ena - bled medical devices require robust post-market surveillance programmes that monitor real-world performance and detect potential safety issues or performance degradation. 2.5 Data Protection and Privacy Developers and users of healthcare AI must adhere to a number of data privacy and security requirements, including: • HIPAA compliance – a 2025 Department of HHS- proposed regulation states that entities using AI tools must include those tools as part of their risk analysis and risk management compliance activi - ties, including risk assessments, security meas - ures, and breach notification procedures; • minimum necessary standard application – AI tools must be designed to access and use only the PHI strictly necessary for their purpose, even though AI models often seek comprehensive datasets to optimise performance; • de-identification requirements – healthcare AI systems must meet HIPAA’s safe harbour or expert determination standards and prevent re-identifica - tion when datasets are combined; • BAAs – BAAs must include language covering permissible data use and safeguards, as well as AI-specific risks including algorithm updates, data retention policies and security measures for ML processes; and • patient consent requirements – healthcare AI deployment requires careful consideration of patient consent, particularly when such systems influence clinical decisions or when data is used for secondary purposes. 2.6 Interoperability and Standards A number of mandated and voluntary standards regimes apply to healthcare AI, and multiple standards
129 CHAMBERS.COM
Powered by FlippingBook