PaperlayerView exact revision on arXiv

Source-linked AI summary

No computation without representation: Avoiding data and algorithm biases through diversity

Caitlin Kuhlman, Latifa Jackson, Rumi Chunara

arXiv:2002.11836v1cs.CY

TL;DR

Fairness research addresses structural inequalities reflected in data and algorithms, but the computing community has not fully captured the perspectives of populations most affected by algorithmic systems. The paper connects this gap to examples in fairness research and develops recommendations, including educational collaborations, community-domain partnerships, and mentoring. Its conclusion calls for broadening perspectives in computing as part of pursuing algorithmic fairness.

  • Problem

    Fairness research must address structural inequalities reflected in data and algorithms, yet the community has not fully captured the perspectives of populations most affected by algorithmic systems.

    Source: §1 · p. 1

  • Method

    The paper examines fairness research and cases of bias, and draws on interdisciplinary and community collaboration to develop recommendations for broadening participation.

    Source: §2.2 · p. 4, §3 · p. 5, §4.2 · pp. 6–7

  • Results

    The authors propose three areas for action: educational collaborations with minority-serving institutions, research collaborations with community domain experts, and educational mentoring to develop diverse future leaders.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The authors conclude that growing representation may reduce the need for algorithmic manipulation and avoid placing the burden of representing a group on one underrepresented member.

    Source: §5 · p. 8

Abstract

from arXiv · show

The emergence and growth of research on issues of ethics in AI, and in particular algorithmic fairness, has roots in an essential observation that structural inequalities in society are reflected in the data used to train predictive models and in the design of objective functions. While research aiming to mitigate these issues is inherently interdisciplinary, the design of unbiased algorithms and fair socio-technical systems are key desired outcomes which depend on practitioners from the fields of data science and computing. However, these computing fields broadly also suffer from the same under-representation issues that are found in the datasets we analyze. This disconnect affects the design of both the desired outcomes and metrics by which we measure success. If the ethical AI research community accepts this, we tacitly endorse the status quo and contradict the goals of non-discrimination and equity which work on algorithmic fairness, accountability, and transparency seeks to address. Therefore, we advocate in this work for diversifying computing as a core priority of the field and our efforts to achieve ethical AI practices. We draw connections between the lack of diversity within academic and professional computing fields and the type and breadth of the biases encountered in datasets, machine learning models, problem formulations, and interpretation of results. Examining the current fairness/ethics in AI literature, we highlight cases where this lack of diverse perspectives has been foundational to the inequity in treatment of underrepresented and protected group data. We also look to other professional communities, such as in law and health, where disparities have been reduced both in the educational diversity of trainees and among professional practices. We use these lessons to develop recommendations that provide concrete steps for the computing community to increase diversity.

1 INTRODUCTION

The introduction highlights concerns about using demographic data for consequential decisions, while recognizing that data analysis can also support accurate, reproducible, and transparent work on societal issues.

1 INTRODUCTION

  • Automated decisions using large-scale demographic data raise concerns about fairness and ethical use.The passage names criminal justice, education, housing, and healthcare as legally regulated domains where such decisions affect people’s lives.Source: §1 · p. 1
  • These decisions affect people’s lives in domains including criminal justice, education, housing, and healthcare.Source: §1 · p. 1
  • Data analysis also offers opportunities for accurate, reproducible, and transparent approaches to societal issues.Source: §1 · p. 1

R. Chunara New York University

The paper argues that limited representation in computing leaves the ethics-in-AI community short of perspectives from populations affected by algorithmic systems. It connects this gap to biases in data analysis and models, and advocates diversifying computing through collaborations, examples from the literature, and recommendations informed by other fields.

R. Chunara New York University

  • Vulnerable populations are defined by limited social capital to represent themselves, while underrepresented groups are defined by historically discriminated-against ethnic minority or gender groups and their underrepresentation in technology workforces.Source: R. Chunara New York University · p. 1
  • The ethics-in-AI community has not fully captured affected populations’ perspectives, and an approach centered on recruitment underuses potential diversity resources.Source: R. Chunara New York University · p. 1
  • The authors advocate making computing and the AI research community a priority for diversification, while focusing mainly on North American fairness research.Source: R. Chunara New York University · p. 1
  • The paper connects limited dataset-community diversity to the types and breadth of biases in data analysis and interpretation.Source: R. Chunara New York University · p. 1
  • The authors highlight fairness-and-AI cases where limited perspective diversity may have contributed to methods or models unfairly treating protected groups.Source: R. Chunara New York University · p. 1
  • The paper identifies positive efforts and recommends drawing on practices from fields outside computing to address the diversity deficit.Source: R. Chunara New York University · p. 2

2 WHY IMPROVING DIVERSITY IS ESSENTIAL TO THE ETHICS IN AI COMMUNITY

The section argues that fair algorithms cannot be assessed apart from structural inequality and the computing community’s own lack of diversity. It connects complex, context-dependent sources of bias with narrow fairness analyses and disparities in computing participation.

2 WHY IMPROVING DIVERSITY IS ESSENTIAL TO THE ETHICS IN AI COMMUNITY

  • The section situates algorithmic bias in unequal data representation and objective functions, and notes that computing’s diversity deficit may affect innovation and technology development.The paper points to evidence from other fields, including healthcare, management, science, and economics, when discussing diversity’s potential relevance.Source: §2 · p. 2, §2.2 · p. 4
  • Fairness criteria based on group status, outcomes, and predictions omit context that may produce or perpetuate inequality, so statistical parity may not capture its full extent.Even when protected groups are represented and statistical parity holds, the measure may be insufficient under structural inequality.Source: §2 · p. 2

2.1 Impact of Structural Inequality on Algorithmic Fairness Analysis

  • In healthcare, equal treatment rates and group representation can coexist with unobserved confounding, while multiple social, access, and human-judgment pathways may shape outcomes.The authors use this example to argue that fairness problems may require further domain insight.Source: §2.1 · pp. 2–3
  • Structural inequality can involve many interacting relations, making it difficult to represent with a single confounder or quantify within standard models.The authors argue that domain insight is needed to formulate problems and solutions in the context of these conditions.Source: §2.1 · p. 3

2.2 Impact of Structural Inequality on the Computing Community

  • 44 of 57 experiments at FAT and AIES 2019 targeted race or gender, illustrating that fairness research focused on a narrow set of protected attributes.The survey covered papers experimentally evaluating bias-mitigation algorithms or fairness metrics.Source: §2.2 · p. 4
  • Computing education shows gender and racial disparities: 77.7% of computer science students and 80.7% of computer engineering students were male; Black and Latinx students were 2.0% and 1.7%.The figures come from a 2018 survey of 169 PhD-granting programs; the paper also reports disparities in industry technical roles.Source: §2.2 · p. 4
  • The authors connect limited computing diversity with gaps in domain insight and diverse perspectives, including practitioners’ susceptibility to blind spots about real-world users.They suggest greater workforce diversity could aid understanding of real-world needs in product development.Source: §2.2 · p. 4

3 FAIRNESS IN THE LITERATURE AND POSSIBLE CONFOUNDING

The section relates diverse perspectives in computing to bias risks across datasets, algorithm design, and missing protected-class labels. It illustrates those risks with examples from facial recognition, healthcare risk scoring, and fairness analysis that assumes protected attributes are known.

3 FAIRNESS IN THE LITERATURE AND POSSIBLE CONFOUNDING

  • The authors connect greater diversity among algorithm designers with proactive attention to data and algorithm biases in fairness research.They next examine cases from the literature to illustrate how diverse representation could have mattered.Source: §3 · p. 5

3.1 Biases in Data

  • Underrepresentation of female and dark-skinned faces in training datasets contributed to disparate performance in commercial facial recognition systems, as discussed in Gender Shades.The authors also note that historical stereotypes are reflected in text corpora used to train word embeddings [10] [44].Source: §3.1 · p. 5

3.2 Algorithmic Bias

  • A healthcare risk score used cost as a proxy rather than health-related variables, resulting in Black patients being sicker than white patients at the same score.The authors suggest a diverse team may have identified the issue during design, while noting that the cited study itself did not discuss this possibility.Source: §3.2 · p. 5

3.3 Missing Labels

  • Fairness work often assumes protected attributes are known, while membership labels may be unavailable; the authors argue proactive recording and diverse dataset teams may help identify missing attributes.The paper notes that assuming known protected attributes is reasonable for work using existing data.Source: §3.3 · p. 5

4 RECOMMENDATIONS FOR INCREASING DIVERSITY WITHIN COMPUTING AND THE ETHICS IN AI COMMUNITY

The recommendations move beyond a pipeline-only approach: connect with minority-serving institutions, collaborate with affected communities and interdisciplinary experts, and expand mentorship for trainees. Together, these steps aim to bring broader perspectives into computing and ethics-in-AI research, while recognizing that increasing diversity alone does not guarantee equity.

4 RECOMMENDATIONS FOR INCREASING DIVERSITY WITHIN COMPUTING AND THE ETHICS IN AI COMMUNITY

  • The recommendations target diversity beyond a pipeline problem through institutional partnerships, community engagement, and opportunities to support future leaders.The authors note that more diversity does not automatically fix equity, though exposure to different values and backgrounds is thought to improve it at individual and community levels.Source: §4 · p. 5

4.1 Building Collaborations with Minority Serving Institutions

  • Minority-serving institutions may broaden computing’s talent pool: they educate nearly half of underrepresented computing trainees and have traditions of contextualizing marginalized populations’ lives.The authors point to law-school enrollment patterns as a potential lesson and recommend meaningful, ongoing partnerships to contextualize biases in data.Source: §4.1 · p. 6

4.2 Prioritizing Research Collaboration Between the Ethics in AI Community and Underrepresented/Interdisciplinary Groups

  • Partnerships with underrepresented and vulnerable communities can bring their knowledge into problem design and engagement across data collection, analysis, and interpretation.The paper argues that culturally responsive technologies and pedagogy should account for intersectionality and inclusion proactively rather than rely on after-the-fact accommodations.Source: §4.2 · p. 6
  • Collaboration with social scientists can connect quantitative fairness research to established work on structural inequality across education, political science, sociology, health, and urban studies.The authors describe identifying and quantifying sources of structural inequality as a potential area of interdisciplinary synergy.Source: §4.2 · p. 6
  • Community advisory boards and research open houses can formalize partnerships, while community input may generate research questions around immediate needs and help return findings to communities.The authors also suggest these ties could increase interested trainees and help community activists amplify algorithmic messages identified through research.Source: §4.2 · p. 7

4.3 Providing Enhanced Mentorship to Trainees at Ethics in AI Research Conferences

  • Conference mentors can bridge trainees’ thinner professional networks to senior researchers, potentially expanding newcomers’ connections and likelihood of long-term contributions.The paper identifies a lack of onsite mentorship and shallower professional networks as barriers to recruitment and advancement for many students from diverse communities.Source: §4.3 · p. 7
  • Travel support matters because merit-only awards can reinforce resource imbalances, leaving trainees at institutions educating many underrepresented graduates facing greater financial barriers to conference participation.The authors cite the Society for Molecular Biology and Evolution’s travel support for trainees whose participation would broaden the organization’s diversity capacity.Source: §4.3 · p. 7
  • Mentoring and affinity workshops can build a cadre of research leaders who continue investing in the field and strengthen organizational capacity.The paper describes the Broadening Participation in Data Mining workshop as bringing together underrepresented trainees and early-career scientists with mentors and career exposure.Source: §4.3 · pp. 7–8

5 CONCLUSION

The conclusion calls for bottom-up capacity building that increases meaningful representation and empowers underserved communities alongside efforts to improve algorithmic fairness. It frames education, research collaboration, and mentoring as ways to meet communities where they are and develop more sustainable fairness in socio-technical systems.

5 CONCLUSION

  • The authors propose three routes to address structural inequalities: collaborations with minority-serving institutions, community and domain-expert research partnerships, and mentoring diverse future leaders.They describe meeting communities where they are by identifying institutions producing underrepresented trainees and creating conference participation opportunities through mentoring.Source: §5 · p. 8
  • Meaningful representation may reduce the need for algorithmic manipulation and help retention without placing the burden of representing a group on one person.Source: §5 · p. 8
  • The authors support an organic, bottom-up approach that seeks both fairer algorithmic systems and greater equity for underserved gender and ethnic communities.They warn that without educational initiatives, the field may miss an opportunity to build diverse-perspective capacity and sustainably improve fairness, accountability, and transparency.Source: §5 · p. 8
Loading 2002.11836v1…