USA Law and Practice Contributed by: Nadia de la Houssaye, Andy Lee, Jason Loring and Graham Ryan, Jones Walker LLP
tutional review boards (IRBs) and ethics commit - tees. These bodies provide oversight for AI deploy - ment decisions while addressing ethical dilemmas that arise during implementation and use. • International ethical standards: Healthcare AI development increasingly references international ethical frameworks, including those developed by the World Health Organization (WHO) and other global health organisations. While US participation in the WHO is likely to be limited, at best, during the Trump administration, voluntary adherence to these standards is likely given patient and other stakeholder pressures. 5.2 Transparency and Explainability To minimise risk to patients, providers and health sys - tems, a number of tools can be implemented, includ - ing the following. • Patient disclosures: At the state level, many health - care providers face increasing requirements to dis - close AI system use to patients and obtain appro - priate consent for AI-assisted care. For example, California’s AB 3030 regulates the use of genAI in healthcare provision. • Algorithmic transparency standards: The FDA’s guiding principles regarding transparency for ML- enabled medical devices require that AI systems offer sufficient transparency that balances the information needs of healthcare providers and patients against proprietary algorithm details and trade secrets that companies may wish to protect. • Decision explanations: AI systems must provide understandable explanations for their recommen - dations, which healthcare providers in turn use to communicate with patients. 5.3 Bias and Fairness Given the rapid turnabout in executive-branch pol - icy toward DEI and anti-discrimination initiatives, it remains to be seen how federal healthcare AI regula - tions with respect to bias and fairness will be affect - ed. The following review looks at policies that existed before the Trump administration; it is fair to say that many of these will be revised this year. • Regulatory anti-discrimination requirements: On 26 April 2024, the Department of HHS issued a final
rule under Section 1557 of the ACA advancing pro - tections against discrimination in healthcare which – with respect to AI – underscored the importance of inclusive data practices and continuous evalua - tion of AI tools and algorithms to promote equita - ble health outcomes. • Training data diversity: AI system developers have, until recently, faced increasing regulatory pressure to ensure training datasets adequately represent diverse patient populations. Most healthcare AI developers and practitioners continue to maintain that relevant characteristics – including age, gen - der, sex, race and ethnicity – should be appropri - ately represented and tracked in clinical studies to ensure that results can be reasonably generalised to the intended use population. • Bias testing and mitigation: Healthcare organisa - tions should implement systematic bias testing and mitigation strategies throughout the AI life cycle, with the following goals in mind: (a) promotion of health and health care equity; (b) ensuring that healthcare algorithms and their uses are transparent and explainable; (c) engaging with – and earning the trust of – pa - tients and communities; (d) identification of healthcare algorithmic fairness issues and trade-offs; and (e) ensuring accountability for equity and fairness in outcomes. • Protection of vulnerable populations: Special atten - tion must be paid to protecting vulnerable popula - tions, including paediatric patients, elderly individu - als, racial and ethnic minorities, and individuals with disabilities. 5.4 Human Oversight In most federal and state regulatory schemes, ultimate responsibility for healthcare AI systems is assigned to the people and organisations that implement it — not to the AI itself. Specific best practices include: • providers maintain clinical decision-making author - ity – healthcare providers must maintain ultimate authority for clinical decisions even when using AI-powered decision support tools; • humans in the loop – healthcare AI applications must require meaningful human involvement in
134 CHAMBERS.COM
Powered by FlippingBook