BELGIUM Trends and Developments Contributed by: Benjamin Docquir and Margo Cornette, Osborne Clarke
To give a more practical example, a physician should explain to caregivers and patients the specifics of the environments in which manu - facturers or providers specifically state that their AI system will not operate accurately. Suppose that a remote patient monitoring tool is designed to detect skin cancer or follow up on the stages of skin cancer. The provider specifies that the tool requires specific lighting conditions to func - tion correctly. If the physician or caregiver is not aware of the limitations of this AI system and the specific conditions are not met – such as poor lighting when seeking to detect skin lesions – the AI system may fail to provide accurate readings. Instructions for use In accordance with Article 13, Section 2 of the AIA, providers must give instructions for use and make them available to deployers. These instructions should include comprehensive infor - mation on the system’s characteristics, capabili - ties and performance limitations. They should also outline potential risks related to the use of high-risk AI systems, including “any actions by the deployer that could influence system behav - iour, under which the system might pose risks to health, safety, and fundamental rights, on the changes that have been pre-determined and assessed for conformity by the provider and on the relevant human oversight measures, includ - ing the measures to facilitate the interpretation of the outputs of the AI system by the deployers”. Deployers of high-risk AI systems must take the necessary technical and organisational meas - ures to ensure that the systems are used cor - rectly and in accordance with these instructions. Human oversight Providers are responsible for the basic imple - mentation of human oversight tools, and deploy - ers are subsequently obliged to assign human
oversight to natural persons who have the nec - essary competence, training and authorisation. This requirement is intended to prevent or mini - mise risks to health, safety or fundamental rights that may arise from the use of a high-risk AI sys - tem (such as biased output or false negatives). Input data screening Data provided to or directly captured by an AI system, on the basis of which the system pro - cesses an output – defined as input data – must be relevant and sufficiently representative with respect to the intended purpose of the high-risk AI system. This obligation, however, only applies to the extent the deployer exercises control over such input data. In healthcare, for exam - ple, this could mean including diverse patient data to avoid bias. Alternatively, if the AI system is designed for remote monitoring of a specif - ic condition, such as diabetes, the input data should include relevant medical records and diagnostic information. Post-market surveillance and vigilance In accordance with Article 26, Section 5 of the AIA, healthcare professionals using AI systems for remote patient monitoring must monitor the operation of any such AI system on the basis of the accompanying instructions for use. If they identify that the use of the AI system may result in a significant risk, or if they identify a serious incident, they may need to inform the provider (and where legally required also the distributor and/or the relevant market surveillance authori - ties) and suspend the use of the system. Log keeping The AIA mandates the automatic recording of logs on high-risk AI systems. This ensures a level of traceability of the AI systems’ function - ing throughout their life cycle, and facilitates the monitoring of high-risk AI systems to detect situ -
32
CHAMBERS.COM
Powered by FlippingBook