Abstract
The increasingly widespread use of AI tools in various stages of a contract’s life cycle has brought many challenges, including human rights protection. Discriminatory practices have been detected in many areas of private law where algorithms are used in the selection or decision-making process (e.g. in the context of loan financing, marketing, employment, and insurance). By looking into the EU legal framework, this chapter aims to analyse the selected instances of discriminatory practices caused by AI systems that occur in horizontal relationships (i.e. relationships between private individuals). More precisely, it focuses on two major fields of private law where the EU offers protection against discrimination, that is, employment matters and access to and the supply of goods and services. Although EU Member States may provide a higher level of protection in their national laws, this analysis takes a supranational approach and focuses exclusively on the protection guaranteed by EU law.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The consolidated version of the Treaty on the Functioning of the European Union, OJ C 326 of 26 October 2012.
- 2.
The European Convention for the Protection of Human Rights and Fundamental Freedoms, as amended by Protocols Nos. 11 and 14, of 4 November 1950.
- 3.
The consolidated version of the Treaty on European Union, OJ C 326 of 26 October 2012.
- 4.
The Charter of Fundamental Rights of the European Union, OJ C 326 of 26 October 2012.
- 5.
The Proposal for a Council Directive on implementing the principle of equal treatment between persons irrespective of religion or belief, disability, age, or sexual orientation of 2 July 2008.
- 6.
Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin, OJ L 180 of 19 July 2000.
- 7.
Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation, OJ L 303 of 2 December 2000.
- 8.
Council Directive 2004/113/EC of 13 December 2004 implementing the principle of equal treatment between men and women in the access to and supply of goods and services, OJ L 373 of 21 December 2004.
- 9.
Directive 2006/54/EC of the European Parliament and of the Council of 5 July 2006 on the implementation of the principle of equal opportunities and equal treatment of men and women in matters of employment and occupation (recast), OJ L 204 of 26 July 2006.
- 10.
For more, see: Ebers and Navas (2020, p. 77).
- 11.
However, there are certain rules or general concepts of private law limiting the freedom of contract, such as the principle of protecting the economically weaker party, the principle of good faith, etc. For more, see: Trstenjak et al. (2016, pp. 11, 12).
- 12.
Zakon o varstvu pred diskriminacijo (ZVarD), Official Gazette of the Republic of Slovenia, Nos. 33/16 and 21/18.
- 13.
For details, see: Articles 1 and 2 of the Protection against Discrimination Act. See also: Štajnpihler (2016).
- 14.
BGBl. I S. 1897 of 14 August 2006, last amendment in 2022 (BGBl. I S. 768).
- 15.
See, e.g., Sheard (2022).
- 16.
Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC, OJ L 119 of 4 May 2016.
- 17.
Article 3(1) (a) of Directive 2000/43/EC; Article 3(1) (a) of Directive 2000/78/EC, and similar in Article 1 (a) of Directive 2006/54/EC.
- 18.
For more, see: Sheard (2022).
- 19.
- 20.
Riddick and others v. Facebook, Case 3:18-cv-04529-LB.
- 21.
Ibid.
- 22.
- 23.
European Parliament Resolution of 3 September 2008 on how marketing and advertising affect equality between women and men, OJ C 295E of 4 December 2009.
- 24.
- 25.
Association Belge des Consommateurs Test-Achats ASBL and Others v Conseil des ministres (Case C-236/09), Judgment of the Court (Grand Chamber) of 1 March 2011.
- 26.
E.g. O'Neil Risk Consulting & Algorithmic Auditing (ORCAA), Parity AI.
- 27.
For instance, Siemens has developed such an internal self-auditing system (Hempel 2018).
- 28.
https://www.recruiter.com/recruiting/entelo-diversity-isnt-racist-its-just-the-opposite/. Accessed on 5 August 2022.
References
Angwin, Julia et al. 2017. Dozens of Companies Are Using Facebook to Exclude Older Workers from Job Ads. https://www.propublica.org/article/facebook-ads-age-discrimination-targeting#:~:text=Series%3A%20Machine%20Bias-,Dozens%20of%20Companies%20Are%20Using%20Facebook%20to%20Exclude%20Older%20Workers,said%20one%20employment%20law%20expert. Accessed on 5 August 2022.
Baldwin, Grace. 2018. What’s the Difference Between Dynamic and Personalized Pricing? https://www.omniaretail.com/blog/difference-dynamic-and-personalized-pricing. Accessed on 5 August 2022.
Buchholtz, Gabriele; Scheffel-Kain, Martin. 2022. Algorithmen und Proxy Discrimination in der Verwaltung: Vorschläge zur Wahrung digitaler Gleichheit. Neue Zeitschrift für Verwaltungsrecht (NVwZ), pp. 612–617.
Corrêa, Ana Maria. 2021. Addressing discrimination in data-driven advertising: Regulatory opportunities and failures within the EU. https://www.diplomacy.edu/blog/addressing-discrimination-in-data-driven-advertising-regulatory-opportunities-and-failures-within-the-eu/. Accessed on 5 August 2022.
Dastin, Jeffrey. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G. Accessed on 5 August 2022.
Ducato, Rossana; Kullman, Miriam; Rocca, Marco. 2018. Customer ratings as a vector for discrimination in employment relations? Pathways and pitfalls for legal remedies. https://orbi.uliege.be/bitstream/2268/222031/1/RD%20MK%20MR%20-%20Customer%20ratings%20as%20a%20vector%20for%20discrimination.pdf. Accessed on 5 August 2022.
Ebers, Martin; Navas, Susana (eds.). 2020. Algorithms and Law. Cambridge: Cambridge University Press.
Engler, Alex. 2021. Auditing employment algorithms for discrimination. https://www.brookings.edu/research/auditing-employment-algorithms-for-discrimination/. Accessed on 5 August 2022.
Epping, Volker; Hillgruber, Christian (eds.). 2022. BeckOK Grundgesetz. 51st Edition. Munich: Beck.
Freedland, Mark; Lehmann, Matthias. 2013. Non-discrimination and the “Constitutionalization of Contract Law”. In: Dannemann, Gerhard et al. (eds.). The Common European Sales Law in Context. Oxford: Oxford University Press, pp. 160–189.
Gillum, Jack; Tobin, Ariana. 2019. Facebook Won’t Let Employers, Landlords or Lenders Discriminate in Ads Anymore. https://www.propublica.org/article/facebook-ads-discrimination-settlement-housing-employment-credit#:~:text=Series%3A%20Machine%20Bias-,Facebook%20Won't%20Let%20Employers%2C%20Landlords%20or%20Lenders%20Discriminate%20in,sparked%20lawsuits%20and%20widespread%20outrage. Accessed on 5 August 2022.
Griffin, Rachel. 2022. Tackling Discrimination in Targeted Advertising: US Regulators Take Very Small Steps in the Right Direction—But Where Is the EU? https://verfassungsblog.de/targeted-ad/. Accessed on 5 August 2022.
Grochowski, Mateusz; Jabłonowska, Agnieszka et. al. 2022. Algorithmic Price Discrimination and Consumer Protection. A Digital Arms Race? Technology and Regulation, Special Issue: Should Data Drive Private Law? pp. 36–47.
Hempel, Jessi. 2018. Want to Prove Your Business Is Fair? Audit Your Algorithm. https://www.wired.com/story/want-to-prove-your-business-is-fair-audit-your-algorithm/. Accessed on 5 August 2022.
Kellerbauer, Manuel et al. (eds.). 2019. The EU Treaties and the Charter of Fundamental Rights: A Commentary. Oxford: Oxford University Press.
Möllers, Thomas M.J. 2018. Working with Legal Principles–Demonstrated Using Private Autonomy and Freedom of Contract as Examples. European Review of Contract Law (14), pp. 101–114.
Orwat, Carsten. 2020. Risks of Discrimination through the Use of Algorithms. Berlin: Federal Anti-Discrimination Agency.
Paal, Boris. 2019. Missbrauchstatbestand und Algorithmic Pricing - dynamische und individualisierte Preise im virtuellen Wettbewerb. Gewerblicher Rechtsschutz und Urheberrecht (GRUR), pp. 43–53.
Rosenblat, Alex et al. 2017. Discriminating Tastes: Uber's Customer Ratings as Vehicles for Workplace Discrimination: Customer Ratings and Workplace Discrimination. Policy and Internet, Vol. 9 (4).
Schulze, Reiner (ed.). 2011. Non-discrimination in European Private Law. Tübingen: Mohr Siebeck.
Sheard, Natalie. 2022. Employment Discrimination by Algorithm: Can Anyone Be Held Accountable? University of New South Wales Law Journal, Vol. 45(2).
Štajnpihler, Tilen. 2016. Prepoved diskriminacije v pravu varstva potrošnikov. In: Vlahek, Ana et al. (eds.). Pravo in politika sodobnega varstva potrošnikov. Ljubljana: GV Založba, pp. 67–110.
Thüsing, Gregor. 2021. Allgemeines Gleichbehandlungsgesetz (AGG). In: Jürgen Säcker, Franz et al. (eds.). Münchener Kommentar zum BGB, 9th Edition. Munich: Beck.
Trstenjak, Verica et al. (eds.). 2016. The Influence of Human Rights and Basic Rights in Private Law. Cham, Heidelberg, New York, Dordrecht, London: Springer.
Wachter, Sandra. 2020. Affinity Profiling and Discrimination by Association in online Behavioural Advertising. Berkley Technology Law Journal, Vol. 35 (2).
Wagner, Gerhard; Eidenmüller, Horst. 2019. Down by Algorithms? Siphoning Rents, Exploiting Biases, and Shaping Preferences: Regulating the Dark Side of Personalized Transactions. The University of Chicago Law Review, Vol. 86, pp. 581–609.
Xenidis, Raphaele; Senden, Linda. 2020. EU non-discrimination law in the era of artificial intelligence: Mapping the challenges of algorithmic discrimination. In: Bernitz, Ulf et al. (eds.). General Principles of EU Law and the EU Digital Order. Kluwer Law International, pp. 151–182.
Acknowledgements
The research leading to this chapter has received funding from the Slovenian Research Agency, under the research project “Human Rights and Regulation of Trustworthy Artificial Intelligence” (No. V5-1930).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lutman, K. (2023). Artificial Intelligence and the Prohibition of Discrimination in the EU: A Private Law Perspective. In: Završnik, A., Simončič, K. (eds) Artificial Intelligence, Social Harms and Human Rights. Critical Criminological Perspectives. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-19149-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-19149-7_4
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-031-19148-0
Online ISBN: 978-3-031-19149-7
eBook Packages: Law and CriminologyLaw and Criminology (R0)