USA Trends and Developments Contributed by: Nadia de la Houssaye, Andy Lee, Jason Loring and Graham Ryan, Jones Walker LLP
Navigating Regulatory Evolution, Market Dynamics and Emerging Challenges in an Era of Rapid Innovation The use of artificial intelligence (AI) tools in healthcare continues to evolve at an unprecedented pace, fun - damentally reshaping how medical care is delivered, managed and regulated across the United States. As 2025 progresses, the convergence of technological innovation, regulatory adaptation (or lack thereof) and market shifts has created remarkable opportunities and complex challenges for healthcare providers, technology developers, and federal and state legisla - tors and regulatory bodies alike. The rapid proliferation of AI-enabled medical devices represents perhaps the most visible manifestation of this transformation. With nearly 800 AI and machine learning (ML)-enabled medical devices authorised for marketing by the US Food and Drug Administration (FDA) in the five-year period ending September 2024, the regulatory apparatus has been forced to adapt traditional frameworks designed for static devices to accommodate dynamic, continuously learning algo - rithms that evolve after deployment. This fundamen - tal shift has prompted new approaches to oversight, such as the development of predetermined change control plans (PCCPs) that allow manufacturers to modify their systems within predefined parameters and without requiring additional pre-market submis - sions. Regulatory Frameworks Under Pressure The regulatory environment governing healthcare AI reflects the broader challenges facing federal agen - cies as they attempt to balance innovation with patient safety. The FDA’s approach to AI-enabled software as a medical Device (SaMD) has evolved significantly, culminating in the January 2025 publication of com - prehensive draft guidance addressing life cycle man - agement and marketing submission recommenda - tions for AI-enabled device software functions. This guidance represents a critical milestone in establish - ing clear regulatory pathways for AI and ML systems that challenge traditional notions of device stability and predictability. The traditional FDA paradigm of medical device regu - lation was not designed for adaptive AI and ML tech -
nologies. This creates unique challenges for continu - ously learning algorithms that may evolve after initial market authorisation. The FDA’s January 2021 AI/ ML-based SaMD Action Plan outlined five key actions based on the total product life cycle approach, includ - ing tailoring regulatory frameworks with predeter - mined change control plans, harmonising good ML practices, developing patient-centric approaches, supporting bias elimination methods and piloting real- world performance monitoring. However, the regulatory landscape remains fragment - ed and uncertain. The rescission of Executive Order (EO) 14110 on Safe, Secure, and Trustworthy Artifi - cial Intelligence by the Trump administration, and the administration’s issuance of its own EO on AI (Remov - ing Barriers to American Leadership in Artificial Intelli - gence) in January 2025, has created additional uncer - tainty regarding federal AI governance priorities. While the EO has been rescinded, its influence persists through agency actions already underway, including the Section 1557 final rule on non-discrimination of the US Department of Health and Human Services (HHS) and the final rule on algorithm transparency of the Office for Civil Rights (ONC). Consequently, enforcement priorities and future regulatory develop - ment remain uncertain. State-level regulatory activity has attempted to fill some of these gaps, with 45 states introducing AI- related legislation during the 2024 session. Califor - nia’s AB 3030, which specifically regulates generative AI use in healthcare, exemplifies the growing trend towards state-specific requirements that healthcare organisations must navigate alongside federal regu - lations. This patchwork of state and federal require - ments creates particularly acute challenges for healthcare AI developers and users operating across multiple jurisdictions. Data Privacy and Security: The HIPAA Challenge One of the most pressing concerns facing healthcare AI deployment involves the intersection of AI capa - bilities and healthcare data privacy requirements. The Health Insurance Portability and Accountability Act (HIPAA) was enacted long before the emergence of modern AI systems, creating significant compliance challenges as healthcare providers increasingly rely on
143 CHAMBERS.COM
Powered by FlippingBook