POLAND Trends and Developments Contributed by: Barbara Kiełtyka, Jakub Gładkowski and Małgorzata Kiełtyka, Kieltyka Gladkowski KG Legal
security and the rights guaranteed by the Charter of Fundamental Rights of the European Union. However, AI users in the healthcare sector face sev - eral barriers that obstruct the use of new technolo - gies. A key issue in regulatory law is data management and data processing rules.. This facilitates compliance with Regulation (EU) 2016/679 (GDPR – namely per - sonal data, including non-medical data) and Regula - tion (EU) 2023/2854 (Data Act – namely non-personal data, including medical data). Another prerequisite for the proper application of AI in healthcare is under - standing the market and business. AI is not an end in itself, but a tool for implementing strategies. In Poland, the acceptance of AI systems in the healthcare sector by users, especially physicians, also raises legal ques - tions. AI models will be trained at medical universities, in collaboration with start-ups. A key obstacle identified by Polish healthcare compa - nies is legislation, particularly the AI Act. It significantly limits model training for business purposes. Obtaining multiple consents is becoming increasingly difficult, especially when they concern data. Recital 69 of the AI Act emphasises respect for privacy and personal data protection throughout the entire life cycle of an AI system, including the training phase. It requires the application of principles such as data minimisation, privacy by design, anonymisation, encryption, etc. According to Article 10 of the AI Act, training AI mod - els, especially high-risk ones, requires rigorous quality management and data compliance. The AI Act does not seek to hinder pure research and development activities, and therefore excludes AI systems devel - oped and put into use solely for scientific purposes (Recital 25). However, when an AI model is also inno - vated for business purposes, restrictions apply (Article 2 (6)). However, some companies expect deregulation because, in their expert view, this is the only way for the EU to be competitive with global technological powers. Prohibited Practices Regarding the Use of AI in Medicine The AI Act prohibits the use of AI systems that use techniques that influence humans so that they are unable to control their own behaviour and engage in behaviour they would not normally engage in. An
example of this potential threat would be an AI system that identifies a patient’s hesitancy over undertaking expensive treatment and, as a result, presents manip - ulative images or messages intended to decrease their hesitancy and influence them to change their mind. Another category of prohibited practices involves influencing the behaviour of individuals who are par - ticularly vulnerable to such influence due to age, dis - ability or difficult financial circumstances. An example would be manufacturers of special care robots (whose software is based on AI) for the elderly, installing a special function into those robots encouraging the purchase of expensive but as yet unproven medica - tions. Another prohibited AI activity is so-called social sys - tems scoring, which evaluates people based on their characteristics and behaviours. This could lead to unlawful discrimination and result in unjustified restric - tions on citizens’ rights or worse treatment of indi - viduals in terms of access to healthcare. Using AI to assess medical risk is permissible as long as it does not result in denial of access to healthcare services or violation of patient rights. A significant prohibition is on the creation of illegal facial recognition databases if this is done without the consent of the data subjects. A potential situa - tion in which an AI system mass-downloads images of symptoms and faces of sick people, even if this were intended to eliminate a given condition, could violate fundamental rights to privacy and personal data protection. AI systems are also prohibited from analysing emo - tions in specific contexts. However, in the healthcare sector, such systems may be permissible as long as their goal is to improve the patient’s health. For their comfort and safety, the patient should be informed about the system’s operation and consent to it, and the system itself should undergo appropriate assess - ment and verification beforehand. The EU also prohibits the use of biometric systems to categorise individuals based on characteristics such as race, political views or religion. Automatic patient classification based on unverified data and the gen -
113 CHAMBERS.COM
Powered by FlippingBook