CHINA Law and Practice Contributed by: Gil Zhang, Diana Li, Muran Sun and Yongqi Tao, Fangda Partners
Medical Institutions Medical institutions deploying AI systems are gener - ally expected to establish internal oversight mecha - nisms, including risk identification, adverse event tracking and algorithm performance monitoring. AI is typically treated as an assistive tool, and liability remains with licensed HCPs, reinforcing the need for robust human-in-the-loop safeguards. Developers AI-based medical devices shall comply with existing medical device regulations. Local guidance, such as that from the Beijing Medical Products Administra - tion, requires risk documentation covering the full life cycle – risk identification, control measures, residual risk evaluation and traceability – particularly for AI- specific risks like false negatives or model drift. Risk Assessment and Insurance There is no mandatory AI-specific insurance, but some policy proposals encourage tailored coverage. In practice, a few insurers and medical institutions have piloted AI-related liability coverage or internal reserve mechanisms to manage emerging risks. 4.4 Defences and Limitations In generic/medical device product liability cases, under general tort law and product liability law the burden of proof lies mainly with the patient rather than the producer/manufacturer (developer), seller or medical institute. There is no reversal of the burden of proof. Consequently, it remains relatively difficult for the patient to hold AI developers or users liable. In current judicial practice, courts will rely on experts to review AI system algorithms and determine wheth - er there is obvious room for improvement that could have prevented the harm (ie, design defects). To date, no cases have involved “black-box” AI systems that are completely opaque and cannot be reviewed, and no relevant precedents exist. In medical institute malpractice cases, there are also no specific liability limitations or “safe harbour” provi - sions available to healthcare users who used AI tools in treatment or diagnosis. The medical institute still needs to independently review and verify the AI sys -
tem’s conclusion according to the medical standards prevailing at the time.
5. Ethical and Governance Considerations for Healthcare AI 5.1 Ethical Frameworks The ethical framework for healthcare AI consists of various mandatory requirements, recommend - ed guidelines and industrial standards. This ethical framework emphasises the ethical review process to ensure compliance and AI’s human-centred nature. One of the key milestones is the promulgation of the Measures for Scientific and Technological Eth - ics Review (Trial) in 2023. Companies engaging in life science, healthcare and AI research that involves sensitive fields of sci-tech ethics have to establish an internal ethics review committee to assess com - pliance with applicable laws, ethical codes and sci- tech ethical principles promoting human well-being, respecting the right to life, adhering to fairness and impartiality, reasonably controlling risks, maintaining openness and transparency, etc. For sci-tech activi - ties that may pose a greater possibility of ethical risks, such as R&D of automated decision-making systems with a high degree of autonomy (AI models) for sce - narios with safety or personal health risks, additional expert ethical review is required. Various recommended guidelines have also been developed as sectoral best practice for developers and health institutions. For example, the Code of Ethics for the New Generation Artificial Intelligence, issued in 2021, which emphasises privacy and data security and echoes the ethical principles listed in the foregoing, provides ethical codes from R&D, sup - ply, use and management perspectives. Similar ethi - cal principles are also seen in the Industrial Expert Consensus of Deployment of DeepSeek by Medical Institutions. In practice, ethical considerations, like human wel - fare, privacy and data security, and accountability are integrated into regulatory processes through product registration, clinical trials and post-market monitor - ing on adverse incidents. Ethics committee approval
46
CHAMBERS.COM
Powered by FlippingBook