CHINA Law and Practice Contributed by: Gil Zhang, Diana Li, Muran Sun and Yongqi Tao, Fangda Partners
in clinical trials for AI medical devices, pertaining to whether the trial has sufficiently considered ethical principles, is a must-have. In particular, ethics com - mittee approval for an AI medical device to collect data is also required as part of the algorithm research report to be submitted for medical device registration. 5.2 Transparency and Explainability For healthcare AI systems that qualify as medical devices, the instructions of the product shall comply with the requirements of transparency and explainabil - ity, and shall include basic algorithm information. If an AI system’s security level is severe (such as in the case of using a black-box algorithm or auxiliary decision- making), additional algorithm research summaries, use restrictions and necessary precaution informa - tion shall also be provided, as required in the Guiding Principles for AMD Registration Review. HCPs are only explicitly required to disclose to patients when healthcare AI is being used in their care in limited situations. For example, before using AI- assisted diagnostic technology for invasive examina - tions or performing surgery assisted by an AI surgical system, the purpose of the examination/surgery, risks, precautions, potential complications and preventive measures should be communicated by the HCP to the patient and their family members in advance. An informed consent form should also be signed. For other healthcare AI systems that may process HCPs’ and patients’ personal information, general transparency requirements under the PIPL will apply, and the purpose and means of data processing shall also be made available to the HCPs and patients con - cerned. 5.3 Bias and Fairness Maintaining fairness and preventing algorithm bias is one of the key principles in healthcare AI-related regulations and guidelines. For example, the Gen AI Measures (where applicable) require GenAI service providers to take effective measures to prevent bias during algorithm design, the selection of training data, model generation and optimisation, service provision, etc.
For AI medical devices, the Guiding Principles for AMD Registration Review issued by the NMPA provide that, to ensure data quality and control data bias dur - ing the training of AI systems, the collection of sample data must consider the compliance, sufficiency and diversity of data sources (such as disease composi - tion, population distribution, the scientific and rational distribution of data, and the sufficiency, effectiveness and accuracy of data quality control). In the registra - tion materials for AI medical devices, it is also required that the NMPA be provided with algorithm risk man - agement information, specifying control measures for risks such as overfitting and underfitting, false nega - tives and false positives, and data contamination and bias. For healthcare-related GenAI services (such as GenAI tools for diagnostics and treatment planning or patient consultation), the Gen AI Measures require service providers to carry out a security assessment, under which training data and outputs that contain discrimi - native, unreliable or imprecise content that does not meet the security requirements in healthcare informa - tion services must be strictly managed and controlled during sampling tests. Content monitoring and a user complaint mechanism shall also be adopted during service provision. 5.4 Human Oversight Healthcare AI adheres to a human-centred principle, and automatically generating prescriptions, falsely using an HCP’s name or replacing an HCP in pro - viding diagnosis and treatment services is explicitly prohibited. The final diagnosis and treatment must be determined by a qualified HCP. Healthcare AI systems can only serve as a tool for users (HCPs or patients) to collect medical referential information, or to assist users (HCPs or patients) with auxiliary decision-making. In addition, highly autono - mous AI systems that involve safety or health risks are subject to ethical review and expert re-examination.
47
CHAMBERS.COM
Powered by FlippingBook