GERMANY Trends and Developments Contributed by: Tanja Pfitzner and Claudia Löschhorn, Pfitzner Legal (in collaboration with aginom)
Removing identifying information Where identifying details are irrelevant to a defined evaluative task, the party names, nationalities and other identifiers (whether concerning individuals or legal entities) should be removed before the system sees the material. Where possible, the same proce - dure should be followed for details that could serve as indirect proxies for identity. Facts that may be legally or evidentially relevant should not be removed and an unaltered version of the record should always be retained. Running a comparative assessment Each comparative assessment should be conducted in both orders and multiple times under materially identical conditions, to the extent that the AI system, model version, prompt, context and settings can be controlled. A reversed or unstable answer should be treated as a warning of position bias or unreliability. If the instability is material, the arbitral tribunal should not use the output or should substantially discount it. Deciding the language question consciously Either a working language with human-checked trans - lation should be designated or both languages should be assessed and any divergence noted. If evaluating the entire record in both languages would entail a dis - proportionate amount of effort, the parallel review may be limited to those passages that could be decisive for the case. The choice of language for the analysis should be a deliberate procedural decision. A diver - gence does not establish which output is correct. Requiring a justification that can be tested Only source-linked issue-by-issue justification that can be checked against the record should be accepted. The output should identify the passages and sources on which it relies, state material assump - tions and uncertainties and distinguish between the record, inference and external material. The arbitral tribunal must independently reconstruct the analysis, rather than accept the system’s explanation as its own “reasoning”. Never rely on a bare conclusion, ranking or score. Keeping a record The AI system and, where known, its model and model version, the task performed, the prompt, context and
ited and what disclosure obligations apply. The pro - tocol should be proportionate. Purely administrative or technical uses need not be regulated. The CIArb templates provide a starting point. In particular, the scope of AI use should be discussed with the parties. This allows for the exclusion of areas in which the parties and the arbitral tribunal have little confidence in AI, for example in legal matters unless the AI is specifically trained for the relevant legal sys - tem, or in the evaluation of witness testimony. Over - all, it is recommended that the arbitral tribunal estab - lish the structure of the case resolution without the involvement of AI. In this way, AI is only incorporated with respect to issues where, in the view of the par - ties and the arbitral tribunal, a reasonable and helpful outcome can be expected. Knowing the tool before relying on it Before relying on an AI system for evaluative tasks, reasonable enquiries should be made: What is it and which version was assessed and when? What model does it run on and where has this been disclosed? What is its intended purpose and what are its capa - bilities and known limitations? How is the submitted data processed and how are the generated outputs handled? Is that data retained, reused for training pur - poses or otherwise disclosed? What confidentiality and security safeguards apply? And can the outputs relevant to the decision actually be checked against authoritative sources? Answers to some of these questions regarding the particular model in use can be found in the model cards. These cards are short documents that summarise the intended use, training data, performance, and limitations of the model. They must be published by general-purpose AI (GPAI) pro - viders, while providers of high-risk AI systems must submit equivalent documentation to regulatory regis - tries and downstream deployers. If some of the above questions cannot be answered, that limitation should be noted rather than skipping the question. Confiden - tial information should not come anywhere near the tool unless adequate safeguards are in place. And if the risks cannot reasonably be assessed or the out - puts cannot be independently verified, the tool has no place in an evaluative task.
399 CHAMBERS.COM
Powered by FlippingBook