Healthcare AI 2025

POLAND Law and Practice Contributed by: Barbara Kiełtyka, Jakub Gładkowski and Małgorzata Kiełtyka, Kieltyka Gladkowski KG Legal

An additional threat that arises is the possibility of third parties providing false data, to which high-risk AI systems should be resistant pursuant to Article 15 (5) of the AI Act. For testing high-risk AI systems, EU law introduces requirements regarding the real-world conditions under which a supplier or potential supplier should conduct such testing. The testing plan specified in Article 60 of the AI Act requires special attention to potential discrimination and good representativeness of data subjects at particular risk due to age or dis - ability, which is a particularly important provision for high-risk AI systems related to health. 5.4 Human Oversight Recital 73 of the AI Act mandates that high-risk AI systems be created with appropriate oversight by indi - viduals, and that such systems have specific mecha - nisms for providing guidance and information to those responsible for oversight to avoid potential errors. The AI Act also establishes particularly enhanced oversight of AI systems that process such unique biometric data, which is crucial in the healthcare sec - tor. The entity using the system should not be able to act on the basis of identification made by the system until it has been validated by at least two individuals. Human oversight is addressed in more detail in Article 14 (2) of the AI Act, which lists among its purposes preventing or minimising risks to health, safety or fun - damental rights. 6. Data Governance in Healthcare AI 6.1 Training Data Requirements Under the GDPR and the AI Act, data governance frameworks ensure that AI systems are trained on high-quality, unbiased data, leading to more accurate diagnoses and treatment plans. In the health domain, the EHDS will facilitate non- discriminatory access to health data and the training of AI algorithms on these datasets in a secure, timely, transparent, reliable and privacy-friendly manner, with appropriate institutional governance.

The status of AI systems intended for use in the field of healthcare as high-risk AI systems is also uncon - troversial in the context of Article 6 and Annex III of the AI Act. As part of the EDPB’s activities, in December 2024, an Opinion was published on the technical side of prop - er data training including in the health sector, which recommends training AI models by directly collecting data from data subjects. In practical terms, it is important to consider the exist - ence of horizontal LLMs and vertical LLMs, which can train the horizontal LLMs and create multi-agent mod - els themselves. It is at this point in feeding the model with data that transparency and reliability become essential. 6.2 Secondary Use of Health Data From the perspective of the GDPR, the secondary use of medical data for AI training and development pur - poses is considered secondary data processing. The definition of processing indicated in Article 4 (2) of the GDPR is so broad that its understanding of “process - ing” includes activities such as data sharing, includ - ing the anonymisation procedure as defined in Article 4 (5) of the GDPR, carried out in order to process anonymised data by outsourcing such anonymised data. The GDPR permits secondary use of data, but under certain conditions that must be met for it to be lawful. For example, training an AI model for a diagnostic device for reading lung cancer X-rays requires feeding the model data, which should be partially anonymised. It is not essential for the AI system to know that the training data, ie, the tumour image and the patient’s actual condition, comes from a person named x, but it is crucial for the system to properly read and process that the image comes from a man of a specific age, race and location, who has other additional condi - tions and addictions, and a family history of illness. This makes the requirement for pseudonymisation a very complex problem for training diagnostic devices in medicine.

CHAMBERS.COM