Source-linked AI summary
The Forgotten Margins of AI Ethics
Abeba Birhane, Elayne Ruane, Thomas Laurent, Matthew S. Brown, Johnathan Flowers, Anthony Ventresque, Christopher L. Dancy
TL;DR
AI ethics is examined against concerns about ethical frameworks that can reproduce exclusions and the disproportionate harms experienced by marginalized groups. The paper reviews FAccT and AIES papers using an annotation scheme, and finds an overall inadequacy in how scholarship addresses harms.
Problem
The paper examines how AI ethics addresses fairness and justice amid social and structural power asymmetries, given that those disproportionately harmed are already marginalized.
Source: §1 · p. 1, §2 · pp. 3–4Method
The authors reviewed and annotated papers from FAccT and AIES published from 2018 to 2021.
Source: §3.1 · p. 6Results
The authors report an overall inadequacy in AI ethics scholarship that either mentions harms implicitly or does not address them at all.
Source: §4.2 · p. 11Takeaways & Limitations
The analysis foregrounds the connection between ethical inquiry, historical exclusions, and the disproportionate harms borne by marginalized groups.
Source: §2 · pp. 3–4The annotation scheme did not fit a paper on the Chinese Social Credit System, whose subject and framing fell outside its categories.
Source: §3.2 · p. 7
Abstract
from arXiv · showhide
How has recent AI Ethics literature addressed topics such as fairness and justice in the context of continued social and structural power asymmetries? We trace both the historical roots and current landmark work that have been shaping the field and categorize these works under three broad umbrellas: (i) those grounded in Western canonical philosophy, (ii) mathematical and statistical methods, and (iii) those emerging from critical data/algorithm/information studies. We also survey the field and explore emerging trends by examining the rapidly growing body of literature that falls under the broad umbrella of AI Ethics. To that end, we read and annotated peer-reviewed papers published over the past four years in two premier conferences: FAccT and AIES. We organize the literature based on an annotation scheme we developed according to three main dimensions: whether the paper deals with concrete applications, use-cases, and/or people's lived experience; to what extent it addresses harmed, threatened, or otherwise marginalized groups; and if so, whether it explicitly names such groups. We note that although the goals of the majority of FAccT and AIES papers were often commendable, their consideration of the negative impacts of AI on traditionally marginalized groups remained shallow. Taken together, our conceptual analysis and the data from annotated papers indicate that the field would benefit from an increased focus on ethical analysis grounded in concrete use-cases, people's experiences, and applications as well as from approaches that are sensitive to structural and historical power asymmetries.
1 INTRODUCTION
AI systems can cause real harms to marginalized groups, while abstract ethical analysis risks leaving existing harms to actual people out of view. The paper traces the field’s roots, develops a three-part taxonomy, and surveys FAccT and AIES scholarship to identify patterns and recommend more concrete, power-aware analysis.
1 INTRODUCTION
- Algorithmic systems have harmed marginalized groups through discriminatory policing, medical-care allocations, and targeted ads, while threatening human welfare, freedom, and privacy.The introduction presents these as real impacts of systems deployed in the social world.Source: §1 · p. 2
- Abstract ethical inquiry can let questions of AI ethics drift into speculation at the expense of harms already experienced by actual people.The paper argues for research and practice grounded in concrete particulars.Source: §1 · p. 2
- The paper traces AI ethics scholarship and groups it into overlapping computational, philosophical, and critical categories.It also argues that critical approaches are promising for benefiting groups disproportionately impacted at society’s margins.Source: §1 · p. 2
- The authors examine FAccT and AIES publications to identify emerging trends and themes in AI ethics scholarship.The paper also reviews historical tensions between abstract and concrete ethical approaches and discusses how critical approaches challenge power asymmetries.Source: §1 · p. 3
2 THE IRREMOVABLE TENSION BETWEEN THE ABSTRACT AND CONCRETE
The section examines the tension between abstract, universal approaches to AI ethics and analysis grounded in concrete applications, affected people’s experiences, and social contexts. It argues that fairness assessments must account for historical and structural power asymmetries, which can shape or reproduce harm against marginalized populations.
2 THE IRREMOVABLE TENSION BETWEEN THE ABSTRACT AND CONCRETE
- Canonical ethics’ aspiration to universal principles rests on assumptions that ethics can be separated from context and has centered a white Western ontology.The section argues that AI Ethics reproduces this exclusion by drawing on existing philosophical inquiry that marginalizes other philosophies and systems of ethics.Source: §2 · p. 3
- Black Feminism, Care Ethics, and Asian and African traditions challenge canonical assumptions with practical, power-conscious approaches, yet remain marginalized in philosophy and AI Ethics.Attempts to incorporate non-Western and feminist care ethics remain marginal, while limited teaching expertise risks reifying narrow ideas of how ethics should be applied.Source: §2 · p. 3
- Algorithmic failures produce concrete harms, and disproportionately harmed groups include Black women, welfare recipients, and women of colour.Examples include face-recognition inaccuracies for Black women compared with white men, heavier costs to vulnerable people from automated welfare decisions, and search engines perpetuating stereotypes about women of colour.Source: §2 · p. 4
- Ethical analysis should account for historical and social context because efficiency or inclusion can strengthen conservative institutions and entrench existing power imbalances.The section calls for attention to concrete harms and conditions on the ground, with the aim of material change for those most negatively affected by technology.Source: §2 · p. 5
2.1 The limits of fairness
- Abstract fairness and neutrality can encourage shallow dataset fixes that obscure structural causes and the individuals behind each data point.The section argues that fairness in statistics may be convenient for people insulated from AI harms, while those bearing algorithmic failures experience fairness in daily life.Source: §2.1 · p. 5
- For people who bear the brunt of algorithmic failures, fairness is inseparable from daily life, not merely an abstract statistic.The section links this difference in how fairness is experienced to power asymmetries and the risk of overlooking impacted individuals behind data points.Source: §2.1 · p. 5
- Ignoring historical, societal, and structural power risks equating fair processes and outcomes with equity and justice.Structural systems of power and oppression should be central to AI ethics and fairness considerations.Source: §2.1 · p. 5
- AI systems can scale existing systems of power and oppression, intensifying their effects on marginalized populations.Understanding the power structures underlying AI-related algorithmic systems is necessary to address the dangers of their scale.Source: §2.1 · p. 5
- AI ethics and fairness should center structural power in system design and development, reject context-free fairness assumptions, and contend with applications that may themselves be inequitable or oppressive.The section warns against overly techno-optimistic perspectives that assume context-less fairness will produce equitable outcomes.Source: §2.1 · pp. 5–6
3 SURVEYING THE FIELD
The paper follows how ethics is conceptualized and researched in FAccT and AIES, reading and annotating conference papers against an iteratively developed scheme. The scheme considers concreteness, disparate impacts, and whether impacted groups are explicitly named, while the authors note that some work does not fit its categories.
3 SURVEYING THE FIELD
- The review uses FAccT and AIES publications to trace how ethics is conceptualized, operationalized, researched, and bounded in AI ethics.Following ethics through these publications is intended to offer insight into what counts as legitimate inquiry in the field.Source: §3 · p. 6
3.1 Method
- The authors read and annotated all 535 papers published in FAccT and AIES from 2018 to 2021, then analyzed 526; 91% were annotated by two independent annotators.Cohen’s Kappa values were 0.93 for concreteness, 0.91 for disparity, and 0.92 for explicitness.Source: §3.1 · p. 6
3.2 Annotation Scheme
- The annotation scheme assessed whether papers were abstract or concrete, how they engaged with disparate impacts, and whether they explicitly named impacted groups.Concrete work is grounded in applications, specific demographics, lived people, or context-specific socio-technical phenomena; explicit papers name impacted groups, while implicit papers discuss harm without naming them.Source: §3.2 · pp. 6–7
- The authors acknowledge that binary coding has limitations and that some papers did not fit the annotation scheme.They supplemented the coding with an in-depth qualitative analysis using COMPAS coverage as a case study.Source: §3.2 · pp. 6–7
4 QUANTITATIVE SUMMARY
Across FAccT and AIES papers from 2018–2021, concrete work and stronger disparity-focused work became more common, but most papers remained agnostic or weak and often lacked engagement with structural histories and marginalized perspectives.
4.1 Results
- From 2018 through 2021, analysis of FAccT and AIES papers found that FAccT paper counts grew significantly and consistently, while AIES paper counts showed some variation.Source: §4 · p. 8, §4.1 · p. 8
- Concrete papers rose from 41% in 2018 to 59% in 2021, including 107 concrete versus 73 abstract papers in 2021; FAccT accepted a higher concrete-paper proportion than AIES.The two conferences were more similar in concreteness by 2021 than when they began.Source: Table 1 · p. 8, §4.1 · pp. 8–9, §4.2 · p. 11
- Overall, agnostic, weak, and medium papers represented 40%, 31%, and 17%, respectively; from 2018 to 2021, medium/strong rose from 12% to 37%, while agnostic fell from 61% to 29%.AIES had 49% agnostic papers overall.Source: §4.1 · pp. 9–10
- Explicitness increased with disparity focus: 62% of weak papers were implicit, while 76% of medium and 89% of strong papers explicitly named impacted groups; race and gender were most mentioned.Only 37% of weak papers explicitly named impacted groups in the discussion's comparison.Source: Figure 4 · p. 10, §4.1 · p. 10, §4.2 · p. 11
4.2 Discussion of the results
- FAccT consistently named negatively impacted groups, while AIES's inaugural proceedings were more abstract and largely agnostic; by 2021, the venues appeared more similar in concreteness and non-agnostic positions.Most papers remained agnostic or weak, indicating limited contextualization of socio-technical systems.Source: §4.2 · pp. 10–11
- The authors interpret the prevalence of agnostic or weak papers as evidence that much AI Ethics scholarship lacked historical context and engagement with marginalized groups' perspectives and experiences.They suggest the post-inaugural decline in agnostic papers and rise in medium/strong papers may reflect growing recognition of socio-technical systems and oppression.Source: §4.2 · p. 11
- In a predictive-modeling example, the authors find equity discussion shallow when it gives little attention to the re-arrested person's institutional interactions or racial, class, and gender systems.They warn that modeling within existing racist, capitalist, and unjust criminal-justice institutions risks becoming anti-Black.Source: §4.2 · p. 11
- A separate affirmative-action example is criticized for excluding past inequity, which risks presenting current policy as detached from unjust histories and may bolster racism.The authors characterize this historyless exploration as potentially anti-equity.Source: §4.2 · pp. 11–12
5 QUALITATIVE ANALYSIS: PROPUBLICA’S COMPAS AS A CASE STUDY
The COMPAS case study finds that fairness-focused work often treats algorithms apart from the histories and institutions shaping their use. It argues for centering affected communities and the structural context of harm.
5 QUALITATIVE ANALYSIS: PROPUBLICA’S COMPAS AS A CASE STUDY
- Many papers focus on improving re-offense prediction or fairness within incarceration while leaving re-offense’s social meaning and dataset sourcing and curation largely unaddressed.Source: §5 · p. 12
- Statistical fairness tests can use COMPAS data yet omit the historical processes and institutional context shaping who is deemed part of society.Source: §5 · p. 12
- Institutional deployments carry socio-cultural contexts that shape implementation and adaptation, including anti-Blackness underlying the systems being improved.Source: §5 · pp. 12–13
- Tuning risk algorithms for fairness or accuracy can sustain punitive systems while pushing reintegration questions outside AI ethics.Source: §5 · p. 13
- Krafft et al. designed an auditing toolkit with organizers and people with lived experience of marginalization, centering affected communities.Source: §5 · p. 13
- The authors argue that fairness work should center affected communities’ input and contextualize systems within historical and ongoing oppression.Source: §5 · p. 13
6 CONCLUSION — MISSING THE FOREST FOR THE TREES
The review finds that AI ethics papers often discuss ethics abstractly, with shallow attention to structural context and harms. It calls for work grounded in lived experience, actual harms, and historical power asymmetries.
6 CONCLUSION — MISSING THE FOREST FOR THE TREES
- Ethical analysis should be guided by negatively impacted stakeholders’ lived experiences and perspectives, and situated in historical, social, and cultural contexts.Source: §6 · p. 13
- The FAccT and AIES review finds a strong tendency toward abstract ethics discussion detached from structural and social factors and specific or potential harms.Source: §6 · p. 13
- Papers agnostic to or weakly addressing oppression remained the majority in each conference, despite a welcome decline in agnostic papers.Source: §6 · p. 13
- Many COMPAS-citing papers failed to center structural systems and institutions, resulting in inadequate ethical treatment; some bordered on anti-equity.Source: §6 · pp. 13–14
- The authors argue AI ethics work should attend to structural factors, actual harms, historical power asymmetries, and uneven distributions to benefit the least powerful.Source: §6 · p. 14