Source-linked AI summary
Participation is not a Design Fix for Machine Learning
Mona Sloane, Emanuel Moss, Olaitan Awomolo, Laura Forlano
TL;DR
The paper addresses how participation in ML can reproduce extractive and unjust relations rather than provide meaningful democratic involvement. It critically examines participation as work, consultation, and justice, concluding that participation must be situated, context-specific, and maintained through genuine long-term relationships rather than treated as scalable by default.
Problem
ML participation can obscure exploitative community involvement and existing power dynamics, while industrial scaling separates systems from the contexts in which datasets and designs arise.
Method
The paper critically examines participatory design and ML literature, expands participation beyond conventional design involvement, and organizes it as work, consultation, and justice.
Results
The paper concludes that existing participation should be recognized as potentially exploitative and that participatory design is situated and context-dependent, conflicting with extraction and context-independent scalability.
Takeaways & Limitations
ML practice should recognize participation as work, make consultation context-specific, and pursue genuine, long-term participation as justice.
Takeaways & Limitations
Long-term participation and justice require constant maintenance and do not scale in frictionless ways, while consultation can become performative or reproduce systemic inequalities.
Abstract
from arXiv · showhide
This paper critically examines existing modes of participation in design practice and machine learning. Cautioning against 'participation-washing', it suggests that the ML community must become attuned to possibly exploitative and extractive forms of community involvement and shift away from the prerogatives of context-independent scalability.
1. Introduction
The paper situates participatory ML within mounting evidence of disparate impacts on oppressed and disadvantaged groups, while warning that participation can reproduce existing power structures and legitimize injustice.
- ML systems have disparate impacts on already oppressed and disadvantaged groups, whose experiences reflect complex structural challenges.
- The paper cautions that participation initiatives may become “participationwashing” rather than meaningful community involvement.
- Participation in international development can rely on manufactured consent and postcolonial global power structures.
- Corporate co-creation, philanthropic problem definition, and stakeholder engagement can involve communities while legitimizing unequal outcomes.
2. Participatory Design
Participatory design has evolved from collaborative workplace technology design into a broader family of approaches addressing values, ethics, futures, and public engagement. The paper argues that ML should learn from these traditions while recognizing participation’s wider forms and power dynamics.
- Participatory design originated in 1970s Scandinavian collaborations between workers and designers developing workplace technologies.
- Later approaches connected participatory design with codesign, co-creation, value-sensitive design, and ethics and values in design.
- Design research expanded participation to include algorithms, machines, and multispecies actors as stakeholders.
- Participatory design and futures practices have been combined in design fiction, speculative design, speculative civics, and critical futures.
- Participatory methods have been used to address difficulties understanding complex technologies and anticipated public resistance.
- ML already contains forms of participation, but practices from other domains offer important lessons for understanding its design and societal integration.
- The paper frames participation as work, consultation, and justice to expose extractive collaboration and tensions with context-independent scalability.
3. Different Forms of Participation
The paper distinguishes participation as work, consultation, and justice, showing that ML relies on extensive often-unrecognized labor, episodic stakeholder input, and longer-term reciprocal partnerships.
- 3.1. Participation as Work: ImageNet combines images from hundreds of thousands of people with labeling by mTurk workers, making photographers, web designers, and labelers participants in ML applications.
- 3.1. Participation as Work: Wikipedia has supplied training language corpora for Natural Language Processing applications for over a decade.
- 3.1. Participation as Work: Billions of web users generate behavioral data and improve models through activities such as clicking, navigation, reCAPTCHA, and ranking.
- 3.1. Participation as Work: ML participation also includes hidden human labor such as transcription and content moderation, alongside affective, emotional, and social labor.
- 3.2. Participation as Consultation: Consultation uses episodic projects, workshops, design sprints, hackathons, or online crowdsourcing to identify context-specific stakeholder needs.
- 3.2. Participation as Consultation: Consultation is limited by short-term or unaffordable partnerships, embedded inequalities, weak stakeholder selection, and performative workshops.
- 3.3. Participation as Justice: Participation as justice relies on longer-term relationships built through mutual benefit, reciprocity, equity, justice, and frequent communication.
- 3.3. Participation as Justice: Its related traditions include participatory action research, crip technoscience, data feminism, and technology activism and resistance.
4. Critiques of Participation
ML participation is entangled with extraction and context-independent scaling, so equitable practice requires recognizing participation as labor, designing consultation for specific contexts, and budgeting for sustained justice-oriented partnerships.
- ML datasets are context-bound, but scaling applications across contexts can discard the context and appropriateness that shaped those datasets.
- Meaningful participation must be funded because consultative involvement loses influence as products scale beyond the contexts where it shaped design.
- Equitable participation also requires expanding value beyond monetary and extractive logics, including attention to access, control, and governance of Indigenous data.
- Users participate as laborers whose activities generate training data and whose interactions can improve model performance, often without explicit recognition or compensation.
- Participation as work should include consent, opt-out options or alternatives, and compensation when users contribute labor to ML systems.
- Consultation should be context-specific, revisited to gather information from the right people, and give marginalized stakeholders space to co-design and co-produce ML systems.
3. Participation as justice must be genuine and long
Genuine participation as justice requires long-term, transparent relationships rather than symbolic ethical language. The paper also calls for cross-disciplinary learning from design harms and failures.
- Participation as justice can be co-opted when organizations invoke design-justice or ethical-AI language without adopting corresponding participatory practices.
- Holistic futuring uses lateral thinking across applications and disciplines to identify how shared ways of thinking can produce design harms.
- The proposed searchable database would cross-reference design failures, especially participation failures, with socio-structural dimensions across sectors and domains.
5. Conclusion
The paper cautions against participation-washing in ML and expands participation to include subtle, possibly exploitative community involvement. It frames participation as situated and context-dependent, challenging extraction and context-independent scalability.
- The paper classifies existing participation in design practice and ML as work, consultation, and justice.
- It argues that ML participation should recognize subtle and possibly exploitative forms of community involvement.
- Participatory ML design is necessarily situated and context-dependent, conflicting with industrial prerogatives of extraction and context-independent scalability.
- The paper calls for recognizing participation as work, making consultation context-specific, and ensuring justice is genuine and long term.
- It proposes a cross-sectoral database of design-participation failures cross-referenced with socio-structural dimensions and edge cases.