Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI

Introduction

• EU contextual equality leaves evidence tests, group definitions, and justifications to case-by-case decisions, producing contradictory measures and fragmented protection across Europe.The relevant considerations include the type and severity of harm, whether a minority group is affected, and the legal and political background.Source: §I · p. 5
• Opaque algorithmic decisions operate at speeds and scale that defy intuition, may affect groups unlike historically protected classes, and can leave disadvantaged people unaware of harm.The authors argue that this weakens intuition as a primary way to identify and assess discrimination and may prevent people from starting a legal claim.Source: §I · p. 6
• The paper argues that human and algorithmic discrimination differ, while group composition and the type and severity of harm require case-specific normative or political choices.These contextual requirements make current EU evidential standards difficult to automate.Source: §I · p. 7
• CDD is proposed as a standard baseline aligned with the ECJ’s “gold standard,” supporting consistent assessment procedures without automating judicial interpretation.The authors argue this can prevent developers from arbitrarily choosing and freezing fairness metrics or thresholds in code.Source: §I · pp. 6–7
• CDD could give judges, regulators, industry, and claimants a baseline for investigating or responding to possible discrimination while preserving contextual judicial interpretation.The authors call for legal and technical communities to combine procedural consistency with recognition of contextual equality.Source: §I · pp. 7, 9

II. THE UNIQUE CHALLENGE OF AUTOMATED DISCRIMINATION

• Automated discrimination can be subtle and intangible, eroding claimants’ ability to notice disadvantage through familiar comparisons such as colleagues’ outcomes or prices.Consumers may not know whether they received the best price or whether advertisements were withheld from them.Source: §II · p. 10
• Because automated discrimination may not be felt and system information can be restricted, claimants may struggle to find evidence even as discriminatory practices persist.Explicit and implicit biases can be recorded in training or operational data without producing an intuitive feeling of inequality.Source: §II · pp. 10–11
• AI can create unintuitive patterns through data, features, classification rules, and system design, while new proxies may obscure effects on protected groups.Disparities may also affect groups that do not map to legally protected characteristics, raising questions about the law’s scope.Source: §II · pp. 11–12
• Complex AI systems may resist decomposition into discrete policies, limiting the common-sense reasoning that has helped judges assess discrimination in other complex settings.Decomposition can also strengthen statistical reasoning and make the choice of test less significant.Source: §II · p. 12
• Without readily available statistical evidence and information about system rules, scope, and outputs, claimants may struggle to identify the rule or define a legitimate comparator group.They may not know which attributes or groups are involved or how others are treated.Source: §II · pp. 12–13

Scope of protection

• Protection varies across the four EU directives by protected grounds and sector, and national transposition creates a fragmented standard across Member States.The directives differ in coverage of employment, welfare, education, and goods and services; the Charter has broader grounds but applies to EU and Member State public bodies, not the private sector.Source: §III · pp. 13–15

Types of discrimination

• Direct discrimination concerns adverse treatment based on a protected attribute, while indirect discrimination concerns a neutral rule or practice that disproportionately disadvantages a protected group.Indirect discrimination can expose structural unfairness through patterns of behavior, but it is relatively new in many Member States and has comparatively little case law.Source: §III · p. 15

Evidential requirements

• Claimants must show harm, significant manifestation within a protected group, and disproportionate impact versus others in a similar situation to establish prima facie discrimination and shift the burden of proof.The alleged offender may then justify the contested rule or practice or refute the claim.Source: §III · pp. 15–16

Contextual equality

• Courts make interconnected, case-specific normative choices about disadvantaged groups, comparators, and particular disadvantage, reflecting local political, social, and legal context; few static rules or thresholds guide them.The paper argues that this contextual flexibility is intentional, not a defect, and should be respected and facilitated in automated systems, although replicating it at scale will be difficult, if not impossible.Source: §III · p. 16

A. COMPOSITION OF THE DISADVANTAGED GROUP

• Direct discrimination is assessed individually through explicit reference to a protected characteristic, whereas indirect discrimination requires group-level comparison.The contested rule must be apparently neutral and significantly disadvantage a legally protected group compared with people in a similar situation.Source: A. COMPOSITION OF THE DISADVANTAGED… · pp. 16–17
• EU jurisprudence supplies no comprehensive rule for group composition: the appropriate broad or narrow traits and level of abstraction depend on the case facts.The analysis accounts for the contested rule’s reach and potential comparators.Source: A. COMPOSITION OF THE DISADVANTAGED… · p. 17
• The ECJ ties the disadvantaged and comparator groups to the contested rule as implemented by the alleged offender, whose reach determines who could be affected.Potential sources of that rule include state legislation, company contracts, regional law, sectoral agreements, hiring, firing, promotion, and payment practices.Source: A. COMPOSITION OF THE DISADVANTAGED… · pp. 17–19
• Case-specific judgments about a contested rule’s reach may work for intuitive discrimination but are difficult to apply at scale to online platforms and multinational companies.For platforms that advertise products or rank search results, the potential reach is not self-evident; the reach remains dependent on contextual factors and case arguments.Source: A. COMPOSITION OF THE DISADVANTAGED… · pp. 19–20

1. Multi-dimensional discrimination

• Additive discrimination concerns disadvantage on multiple protected grounds considered individually, while intersectional discrimination concerns disadvantage based on those grounds considered together.Source: §1 · p. 20
• In Parris [83], the ECJ held that a rule cannot discriminate through the combined effect of age and sexual orientation unless it discriminates on either ground separately [84].The Court also rejected creating a new protected group or category of discrimination from the case's facts [85].Source: §1 · pp. 20–21
• The Parris judgment has been criticized for harming people affected by additive and intersectional discrimination, while multi-dimensional discrimination case law remains scarce and national provisions have yielded no successful case.This lack of jurisprudence reflects the lack of clear guidance for developing and governing automated systems.Source: §1 · p. 21

B. COMPOSITION OF THE COMPARATOR GROUP

• Equal-treatment law requires comparing groups in comparable situations, but EU directives do not clearly specify how to compose the advantaged comparator group.A comparison between groups is implicit in the principle of equal treatment.Source: B. COMPOSITION OF THE COMPARATOR GROUP · p. 22
• Comparator legitimacy turns on whom claimants may compare themselves with, what evidence supports similarity, and whether a concrete or hypothetical comparator is sufficient.These requirements vary with case facts, and protected attributes such as gender, ethnicity, and disability are socially constructed and context-dependent.Source: B. COMPOSITION OF THE COMPARATOR GROUP · pp. 22–23
• The scope of protected characteristics varies across Member States, as national transposition and interpretation leave some grounds—including gender identity—with fragmented or unsettled protection.Other grounds, including ethnicity, disability, religion, and sexual orientation, are also defined and protected through national law with varying scope.Source: B. COMPOSITION OF THE COMPARATOR GROUP · p. 23
• Hypothetical comparators may suffice, but jurisprudence varies; in Maniero and Jyske Finans, the ECJ required a specific concrete comparison rather than an abstract one.In Jyske Finans, country of birth alone could not establish a general presumption of ethnic-group membership.Source: B. COMPOSITION OF THE COMPARATOR GROUP · pp. 24–25
• Comparator legitimacy depends on case circumstances, party arguments, and judicial judgment, with national courts exercising broad interpretive discretion and outcomes varying across Member States.The contextual assessment is often a matter of skilled argumentation, and it is unclear how system controllers could legitimately make it without judicial interpretation.Source: B. COMPOSITION OF THE COMPARATOR GROUP · p. 26

C. REQUIREMENTS FOR A PARTICULAR DISADVANTAGE

• A “particular disadvantage” turns on harm’s nature, severity, and significance; significance can outweigh severity, yet EU law sets no consistent thresholds for these elements.Source: C. REQUIREMENTS FOR A PARTICULAR… · p. 27
• Whether harm must be actual or may be hypothetical remains unsettled, and case-specific judicial assessments of practices make general automated presumptions difficult.Source: C. REQUIREMENTS FOR A PARTICULAR… · pp. 27–28
• Courts may assess disparity at different times, and national courts decide the appropriate timing and whether statistical evidence is valid, representative, and significant.Because inequalities can change over time, a rule’s impact when enacted may differ from its later impact; these flexible judgments are difficult to replicate automatically without case-specific judicial interpretation.Source: C. REQUIREMENTS FOR A PARTICULAR… · p. 31

D. ADMISSIBILITY AND RELEVANCE OF EVIDENCE

• Four evidence types can establish a particular disadvantage and prima facie discrimination: significant-disparity statistics, common-sense assessment, situation testing, and circumstantial inferences.Source: D. ADMISSIBILITY AND RELEVANCE OF… · p. 32

1. Statistical evidence

• Statistics are infrequently used in EU discrimination cases, but are especially relevant to indirect discrimination and must match the geographic and demographic reach of the contested rule.The relevant population may be a Member State or region, company, sector, trade, or other social or professional grouping, depending on the rule.Source: §1 · pp. 32–34
• Statistical evidence can trigger concerns about a “battle of numbers,” faulty assumptions, and parties’ limited resources, while courts retain case-by-case control over its admissibility.Source: §1 · p. 33
• Sensitive-data gaps and unavailable employee records can obstruct statistical proof, and requiring more statistics risks further marginalising groups for whom data are lacking.European privacy and data-protection law also aims to minimise the collection and storage of sensitive personal data.Source: §1 · pp. 34–35
• Legal and policy communities value statistics for detecting and adjudicating discrimination, but collecting sensitive data alone cannot provide adequate remedies or assessment procedures.Some policy bodies have advocated ethical responsibility to collect sensitive data to facilitate legal proceedings, alongside mitigation strategies, effective remedies, and assessment procedures.Source: §1 · p. 35

2. Intuitive and traditional evidence

• Common-sense assessments may treat a rule as intuitively discriminatory, including an apparent workforce ethnic imbalance or bans on turbans or beards.Examples of intuitive evidence include common knowledge, obvious facts, or convictions; an ethnic imbalance may itself indicate indirect discrimination.Source: §2 · p. 36
• The ECJ has allowed indirect discrimination to be established without statistics and treated a military-service preference as self-evidently discriminatory against women.In Meister, the Court said indirect discrimination may be established by any means; in Schnorbus, it considered the gendered military-service requirement sufficiently obvious without statistics.Source: §2 · pp. 36–37
• Traditional evidence can be ineffective for indirect discrimination and no easier to obtain, while automated discrimination may only be observable statistically.The need to demonstrate a correlation between a protected attribute and a non-protected attribute can make documentary evidence, witness statements, or expert opinions ineffective; statistics may therefore become a default option.Source: §2 · p. 37
• The ECJ leaves evidentiary standards largely to national courts, whose contextual and inconsistent assessments make automating fairness particularly challenging.National courts have a high margin of appreciation to assess disadvantage and decide which evidence is reliable and significant.Source: §2 · p. 37

E. A COMPARATIVE ‘GOLD STANDARD’ TO ASSESS DISPARITY

• Single-group examination is especially problematic where protected groups are not evenly distributed or where intersectional or additive disparities are obscured.If a rule disadvantages protected groups at equal rates, looking only at those groups can conceal the greater disparity experienced by people belonging to more than one group.Source: E. A COMPARATIVE ‘GOLD STANDARD’ TO… · pp. 38, 40, 41
• The ECJ’s “best approach” compares the respective proportions of both groups who can and cannot satisfy a contested rule, rather than relying on affected-person counts alone.The Court explained that counts depend on the size of the workforce and the percentages of men and women employed in the Member State.Source: E. A COMPARATIVE ‘GOLD STANDARD’ TO… · p. 38
• Despite the Court’s ‘gold standard,’ courts have not consistently compared disparities: Gerster and Bilka examined only disadvantaged groups, while Seymour-Smith explicitly examined only the advantaged group.Gerster noted that 87% of part-time workers were women; Bilka found 81.3% of pensions went to women although women were 72% of the workforce; Seymour-Smith cited qualification rates of 77.4% for men and 68.9% for women.Source: E. A COMPARATIVE ‘GOLD STANDARD’ TO… · pp. 38–39
• Examining both groups can reveal disparities hidden by the disadvantaged group alone, such as a hiring rule that rejects equally across ethnicity and gender but hires only white men.A disadvantaged group’s composition may not itself reveal that a homogeneous advantaged group is being favoured over a heterogeneous group.Source: E. A COMPARATIVE ‘GOLD STANDARD’ TO… · pp. 40–41
• Any automated assessment must accommodate the courts’ flexible, pragmatic comparison test and ideally facilitate full comparisons between disadvantaged and advantaged groups.The judiciary must determine whether a group has been or is likely to be significantly harmed relative to a comparator in a similar situation.Source: E. A COMPARATIVE ‘GOLD STANDARD’ TO… · pp. 37, 41

F. REFUTING PRIMA FACIE DISCRIMINATION

• Prima facie discrimination can shift the burden to the alleged offender, including when causation is only probable or likely or relevant information is withheld, though national law determines when the shift occurs.Source: F. REFUTING PRIMA FACIE DISCRIMINATION · pp. 41–42
• The alleged offender can refute a causal link or justify indirect discrimination through a legitimate aim pursued necessarily and proportionately, while both parties may submit statistical evidence.Direct discrimination is lawful only when explicitly permitted by law; the causal-link defense also applies to direct discrimination.Source: F. REFUTING PRIMA FACIE DISCRIMINATION · pp. 41–42
• Evidence about the advantaged group can rebut a causal link, as in pension statistics showing women received 81.3% of pensions while making up 72% of employees and inclusive hiring policies in Feryn and Accept.Courts have also recommended examining whether immigrants and gay people are routinely recruited in the direct-discrimination cases Feryn and Accept.Source: F. REFUTING PRIMA FACIE DISCRIMINATION · p. 43
• Whether an interest is legitimate and a measure acceptable depends on the case context, national legislation, and courts’ interpretation, adding complexity to judicial assessment of automated-fairness cases.The ECJ gives national courts a high margin of appreciation, subject to limitations.Source: F. REFUTING PRIMA FACIE DISCRIMINATION · p. 44

IV. CONSISTENT ASSESSMENT PROCEDURES FOR AUTOMATED DISCRIMINATION

• Statistical evidence is rarely and inconsistently used to prove prima facie discrimination, and courts lack standard thresholds or a coherent assessment approach.Fairness has historically been specified contextually through judicial intuition, leaving developers, controllers, regulators, and users without clear requirements for system design and governance.Source: §IV · p. 44
• Direct discrimination cases are likely to be rare in automated systems, while widespread and subtle indirect discrimination poses a growing risk.The passage links this expectation to stronger legal protections for special category data and AI’s greater capacity to identify new proxies for protected attributes.Source: §IV · pp. 44–45
• As automated indirect discrimination grows, statistical evidence is likely to become more important, reflecting growing research on fairness measures and international concern about bias.The passage describes the growth of statistical fairness research as reflecting international consensus that automated bias and discrimination require new detection methods and remedies.Source: §IV · p. 45

A. TOWARDS CONSISTENT ASSESSMENT PROCEDURES

• Contextual interpretation varies across national, regional, and case-specific settings, leaving fragmented protection and no general answer about which statistical evidence or tests are needed.Judicial interpretation shapes key concepts and evidential requirements; national laws rarely specify disparity tests, thresholds, or metrics.Source: A. TOWARDS CONSISTENT ASSESSMENT… · p. 45
• Without regulatory intervention, system controllers may be the only actors with access and expertise to produce statistical evidence, reinforcing the need for high-quality standards aligned with the ECJ’s “gold standard.”This risk does not negate the need for consistent assessment procedures.Source: A. TOWARDS CONSISTENT ASSESSMENT… · p. 45
• Technical measures can inform judicial assessment, but only courts and regulators have democratic legitimacy to decide normative questions such as comparator groups, particular disadvantage, and acceptable disparity.Fairness metrics often imply thresholds, but should not resolve these legal questions generally or independently of context.Source: A. TOWARDS CONSISTENT ASSESSMENT… · p. 46
• Developers and controllers should not set local, subjective discrimination thresholds alone; technical expertise should support, not replace, judicial and regulatory judgment.The proposal brings technical tools and statistical measures together with the judiciary’s interpretive flexibility and democratic legitimacy.Source: A. TOWARDS CONSISTENT ASSESSMENT… · pp. 46–47
• Systems should consistently produce statistical evidence aligned with the ECJ’s comparative “gold standard” as an early warning for potential discrimination, not automatically detect, evaluate, or correct it.This evidence can help judges make informed normative decisions and help controllers detect potential discrimination before it occurs.Source: A. TOWARDS CONSISTENT ASSESSMENT… · p. 47
• Intuition suited to familiar forms of human discrimination is poorly equipped to detect automated discrimination that may be subtle, widespread, and based on complex patterns.Consistent detection and assessment tools can help close this accountability gap and support effective remedies for potential victims.Source: A. TOWARDS CONSISTENT ASSESSMENT… · p. 48

V. STATISTICAL FAIRNESS AND THE LEGAL ‘GOLD STANDARD’

• Across the ECJ and Member States, flexible but inconsistent discrimination tests leave developers, controllers, and regulators without common evidential standards for statistical tests and group comparisons.Judges have often relied on common sense and direct reasoning rather than statistical evidence, amid limited legislative guidance and the need to accommodate contextual equality.Source: §V · pp. 48–49

A. STATISTICAL FAIRNESS TESTS IN EU JURISPRUDENCE

• Demographic parity compares protected-group representation across advantaged and disadvantaged groups; negative dominance requires both a majority of the disadvantaged group to be unprotected and only a minority of the protected group to be advantaged.EU jurisprudence has not set fixed requirements for statistical evidence or thresholds of illegal disparity.Source: A. STATISTICAL FAIRNESS TESTS IN EU… · pp. 49–50
• Negative dominance is harder to satisfy than demographic disparity, so whenever negative dominance exists, demographic disparity is also present.Source: A. STATISTICAL FAIRNESS TESTS IN EU… · p. 51
• When the protected group makes up 50% of the population, the tests are functionally equivalent; the authors argue negative dominance is inappropriate for minority groups.Source: A. STATISTICAL FAIRNESS TESTS IN EU… · pp. 51–52
• 100% of white employees would need to be disadvantaged for demographic disparity in Company B, but negative dominance would treat disparity as non-discriminatory if 10% were disadvantaged.For Company A, with equal numbers of women and men, both tests require all men to be disadvantaged when all women are disadvantaged.Source: A. STATISTICAL FAIRNESS TESTS IN EU… · p. 52
• In the beard-hiring example, demographic disparity is present, but the Sikh protected group’s share falls far below negative dominance’s threshold of more than 50% of the disadvantaged group.The example concerns refusing to hire men with beards, treated as religious discrimination against Sikhs.Source: A. STATISTICAL FAIRNESS TESTS IN EU… · p. 53

B. SHORTCOMINGS OF NEGATIVE DOMINANCE

• Statistical testing cannot fully replace judicial intuition, but measures used for consistent application across Member States should match intuition about which disparities are unfair or discriminatory.Source: B. SHORTCOMINGS OF NEGATIVE DOMINANCE · p. 53
• Negative dominance makes protection depend on group size, leaving minorities less protected precisely because they are minorities, particularly in cases involving ethnicity and sexual orientation.Source: B. SHORTCOMINGS OF NEGATIVE DOMINANCE · p. 53
• Demographic disparity avoids negative dominance’s “divide and conquer” loophole: intersectional splitting can defeat aggregate negative-dominance findings but raises the offender’s burden under disparity testing.Source: B. SHORTCOMINGS OF NEGATIVE DOMINANCE · pp. 53–54
• The judiciary should unambiguously adopt demographic disparity as its statistical-evidence standard, aligning comparisons with the ECJ’s “gold standard”; negative-dominance precedent need not bind future cases.Source: B. SHORTCOMINGS OF NEGATIVE DOMINANCE · p. 54

VI. CONDITIONAL DEMOGRAPHIC DISPARITY: A STATISTICAL ‘GOLD STANDARD’ FOR AUTOMATED FAIRNESS

• Automated systems should provide standard evidence for external adjudication, while courts retain contextual interpretation and developers can use the evidence for pre-emptive bias testing.The authors frame this as a re-alignment of automating fairness, not simply correcting potential discrimination during system development and deployment.Source: §VI · p. 55
• Across the affected population, CDD compares protected groups’ outcome distributions to identify potential disparities and inform the definition of disadvantaged and comparator groups.These comparisons can help assess both groups under the Court’s gold standard, while group composition may depend on normative, political, and social aspects of the case.Source: §VI · pp. 55–56
• CDD adds conditions to demographic disparity; for example, if 70% of white applicants with a 4.0 GPA are admitted, parity requires the same rate for black applicants with a 4.0 GPA.Source: §VI · p. 56
• CDD is a proposed standard for EU automated-discrimination cases that reports disparity magnitude rather than a binary pass/fail or finding of illegality.It is intended to assist contextual analysis of potential discrimination.Source: §VI · pp. 54–56

VI. CONDITIONAL DEMOGRAPHIC DISPARITY: A STATISTICAL ‘GOLD STANDARD’ FOR AUTOMATED FAIRNESS’

• Searching all attribute combinations for violations risks false positives, because an unbiased system may appear biased in some groups by chance.Source: §VI · p. 57

VI. CONDITIONAL DEMOGRAPHIC DISPARITY: A STATISTICAL ‘GOLD STANDARD’ FOR AUTOMATED FAIRNESS

• Splitting people sharing attributes R by protected attributes and selecting the top k_R% within each subgroup satisfies the stated legal requirements; conditional demographic parity can be interpreted as this procedure.Source: §VI · p. 57

A. CONDITIONAL DEMOGRAPHIC PARITY IN PRACTICE

• Women made up 46% of Berkeley’s rejected applicants but 32% of its advantaged group, showing demographic disparity even though the rejection figure fell below the 50% negative-dominance threshold.The example therefore shows demographic disparity where the legal test of negative dominance is not met.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · p. 59, Table 2 · p. 59
• Conditioning on department makes the pattern less clearly discriminatory; fluctuations, largest in department A where few women applied, are expected by chance and do not establish departmental bias.Across the university, the application pool was already biased in favor of men, and women were more likely to apply to more competitive departments.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · pp. 58–59
• A weighted average of departmental statistics finds a slight bias in favor of women after conditioning on department, consistent with prior analyses.The average weights each department’s statistic by its total number of applicants.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · p. 60, Table 3 · p. 60
• In large datasets, very small effects can become statistically significant while remaining meaningless for an individual, and some bias should be expected in real-world systems.The authors note that no system can be perfectly calibrated and future-proof.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · pp. 60–61
• CDD testing may isolate a group when prior evidence indicates likely bias, but should use aggregate testing otherwise and avoid testing every possible group because chance can produce spurious patterns.Isolating a group can prevent noise from other groups’ data from overwhelming evidence of bias.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · p. 61
• Courts must set context-dependent disparity thresholds and decide which factors are reasonable to condition on; the authors favor summary statistics that show magnitude over significance-only tests.Relevant considerations include the system’s purpose, associated harms, and the number of people affected.Source: A. CONDITIONAL DEMOGRAPHIC PARITY IN… · pp. 60–61

B. SUMMARY STATISTICS TO SUPPORT JUDICIAL INTERPRETATION

• CDD is recommended as a baseline evidential standard, using descriptive summary statistics to show disparity’s magnitude rather than statistical tests.The authors note that the measure, called conditional demographic parity in algorithmic fairness, predates the field and has been used for more than 45 years to examine bias amid socially acceptable confounders.Source: B. SUMMARY STATISTICS TO SUPPORT… · p. 62
• In the preceding gender-discrimination example, negative dominance and non-conditional demographic disparity failed to capture the disparity’s full nature, severity, and significance.The authors warn that latitude in the law could allow ethically significant disparities between protected groups to go unflagged.Source: B. SUMMARY STATISTICS TO SUPPORT… · p. 62
• CDD cannot determine whether disparity is illegal or justified; its summaries instead guide contextual investigation of affected groups, comparators, and particular disadvantages across a system’s population.Unlike a binary test, it helps decision-makers weigh the apparent magnitude of bias against competing rights, evidence, or justifications.Source: B. SUMMARY STATISTICS TO SUPPORT… · pp. 62–63
• A shared CDD baseline could give claimants and alleged offenders common statistical evidence, reduce the burden of producing it, and help prevent potential discrimination from being missed in automated decisions.The authors argue that intuitive signals of discrimination are likely to decrease when decisions are automated, making tools to detect bias and disparity important.Source: B. SUMMARY STATISTICS TO SUPPORT… · p. 63
• CDD summaries can expose heterogeneous, minority-based, and intersectional discrimination, while leaving legality thresholds, conditioning factors, and the rule’s reach to case-specific local judgment.The authors seek to avoid AI systems or their developers and controllers setting acceptable-disparity thresholds by default, which could lack democratic legitimacy.Source: B. SUMMARY STATISTICS TO SUPPORT… · pp. 63–64

VII. AUTOMATED FAIRNESS: AN IMPOSSIBLE TASK?

• Legal and technical communities must align fairness measurement and system governance with contextual equality, since statistical metrics alone may not be relevant in court.The authors call for closer dialogue so technical approaches can measure disparity while accounting for flexible legal interpretation.Source: §VII · pp. 64, 66
• Judges, regulators, controllers, developers, and claimants need coherent, consistent evidence procedures to detect algorithmic disparity while preserving contextual, case-specific judicial interpretation.The proposed baseline can support prima facie investigations, regulatory scrutiny, pre-emptive audits, and claims without automating judicial decisions.Source: §VII · pp. 64, 68, 69
• EU non-discrimination assessments turn on context-specific choices about disadvantaged and comparator groups, harm, evidential thresholds, admissibility, and justification.Courts also vary in their reliance on statistics versus intuitive evidence, producing fragmented and sometimes inconsistent standards that are difficult to automate at scale.Source: §VII · pp. 65–66
• Algorithmic discrimination can be subtle, counterintuitive, and unseen by victims, so intuition and individual complaints may miss harms that pre-emptive testing can reveal.Unlike human prejudice and bias, algorithmic systems lack equivalent mechanisms that signal discrimination to potential victims; periodic testing is needed to detect it at scale.Source: §VII · p. 67
• Comparisons should consider both advantaged and disadvantaged groups; examining only the disadvantaged group can obscure disparity and enable subgroup splitting to evade thresholds.The authors describe this two-group comparison as the ECJ’s “gold standard” and link it to CDD.Source: §VII · pp. 68–69
• CDD summary statistics can standardize evidence of disparity across affected protected groups while leaving comparator choices and legal thresholds to judges.The authors propose CDD as a baseline for identifying and assessing potential discrimination, not for making normative, case-specific determinations. Its use depends on protected-attribute data being available or inferred.Source: §VII · pp. 69–70

• The authors include the proof solely for completeness, acknowledging its simplicity.Source: APPENDIX 1 – MATHEMATICAL PROOF FOR… · p. 71
• The algebraic proof starts from the definition of conditional demographic parity and uses inversion, assuming neither B_A nor B_D is zero.Source: APPENDIX 1 – MATHEMATICAL PROOF FOR… · pp. 71–72
• Conditional demographic parity implies that the proportions of advantaged black and white people are equal, denoted k_R%, after conditioning on selected factors.Source: APPENDIX 1 – MATHEMATICAL PROOF FOR… · pp. 71–72