GERMANY Trends and Developments Contributed by: Tanja Pfitzner and Claudia Löschhorn, Pfitzner Legal (in collaboration with aginom)
and an arbitral award) were non-existent and “hallu - cinated”. German arbitration law, which, like Canadian arbitra - tion law, is based on the UNCITRAL Model Law, con - tains a provision in Section 1059 (2)(1)(d) of the Ger - man Code of Civil Procedure (ZPO) that is comparable to that of Article 646 of the Code de procédure civile of Québec, providing for the setting aside of an arbi - tral award if the arbitral procedure was not in accord - ance with the agreement of the parties. The Canadian decision therefore also serves as a landmark decision for arbitration proceedings in Germany and any other jurisdiction which is based on the UNCITRAL Model Law or is a contracting state to the New York Con - vention. Counsel, experts and arbitrators alike are account - able for the consequences of unsupervised or other - wise irresponsible AI use. Given the rapidly increasing use of generative AI in arbitration proceedings, it is to be expected that cases addressing the distinction between permissible, supportive use of AI and imper - missible delegation to AI will gain prominence in the near future. Bias as the Underestimated Risk While hallucinations, as in the Québec case, can be identified fairly quickly upon review, as the sources can be verified, identifying bias – which the Québec court also briefly addressed – is a different challenge. Output of an AI tool can be skewed and still look per - fectly fair at first glance. Both biased and non-biased outputs can read as fluent and confident analysis. That makes bias a hidden risk, especially now that the legal profession has started to move beyond AI- assisted document review and into genuinely evalu - ative work. Arbitrators may ask AI to help test a case theory, while counsel may use it to simulate the other side’s arguments or to play judge and predict how a dispute based on one strategy or another will land. This article focuses on biases contained in output of non-deterministic AI and the risks inherent in its use in arbitration. It does not cover other concerns that may also need to be considered when using AI, such as data protection, intellectual property, data govern - ance, IT security, confidentiality etc.
When exploring which biases are relevant to arbitra - tion and how they should be addressed in practice, it is important to recognise that bias in AI systems is broader and often less intuitive than the concerns about discrimination that typically come to mind. AI governance professionals distinguish various types of biases and other related issues concerning data governance, and only a few of them can be covered In the context of AI use, most professionals link the term “bias” to the fact that the models underlying the AI systems learn from vast amounts of human writing, which contains human prejudice and thus the output may vary depending, for example, on a party’s name, place of residence, nationality or gender, leading to so-called “implicit bias”. in this article. Types of bias A prominent early case took place at Amazon. The company had used an experimental AI recruiting sys - tem and discovered that it systematically downgraded résumés containing indicators associated with women (eg, graduates of women’s colleges). It had learned historical hiring patterns that reflected existing gen - der imbalances and was discontinued. Such effects have been documented by researchers across many settings. Where a model’s training data or a system’s retrieval sources overrepresent particular regions, languages or legal traditions, the system may treat their concepts, authorities and commercial norms as the default while underweighting those of less well- represented jurisdictions or cultures. Speech recognition tools often tend to understand American speech better than Indian, Nigerian or Scot - tish accents, as the underlying model was trained on far more American speech than the other accents. Models in the healthcare sector trained on hospital data from wealthy countries can run into trouble in poorer settings, much like a map that shows the roads of one city and is then used to navigate another. Dif - ferent disease prevalence, different patient popula - tions and different clinical routines can all shift per - formance, which is why geographically representative data matters.
395 CHAMBERS.COM
Powered by FlippingBook