PaperlayerView exact revision on arXiv

Source-linked AI summary

Power to the People? Opportunities and Challenges for Participatory AI

Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Díaz, Madeleine Clare Elish, Iason Gabriel, Shakir Mohamed

arXiv:2209.07572v1cs.CY

TL;DR

The paper addresses unclear expectations for meaningful participation in AI amid documented harms affecting society. It reviews participatory approaches and presents three case studies, finding that participation offers desirable forms while requiring attention to its scope, limitations, and possible conflation with other activities.

  • Problem

    AI faces documented harms in areas including security, justice, employment, and healthcare, while participation’s scope and expected role require deeper interrogation.

  • Method

    The paper presents three Participatory AI case studies across the machine learning lifecycle and examines participation’s attributes, including reciprocity and reflexivity.

  • Results

    The case studies identify desirable forms of participation that are already available and clarify participation’s understanding and role in AI.

  • Takeaways & Limitations

    Realizing participation’s benefits requires effectively incorporating participatory methods and drawing inspiration from available desirable forms where participation is lacking.

  • Takeaways & Limitations

    Participation can be conflated with consultation, inclusion, or labour and cannot be expected to solve all concerns or problems.

Abstract

from arXiv · show

Participatory approaches to artificial intelligence (AI) and machine learning (ML) are gaining momentum: the increased attention comes partly with the view that participation opens the gateway to an inclusive, equitable, robust, responsible and trustworthy AI.Among other benefits, participatory approaches are essential to understanding and adequately representing the needs, desires and perspectives of historically marginalized communities. However, there currently exists lack of clarity on what meaningful participation entails and what it is expected to do. In this paper we first review participatory approaches as situated in historical contexts as well as participatory methods and practices within the AI and ML pipeline. We then introduce three case studies in participatory AI.Participation holds the potential for beneficial, emancipatory and empowering technology design, development and deployment while also being at risk for concerns such as cooptation and conflation with other activities. We lay out these limitations and concerns and argue that as participatory AI/ML becomes in vogue, a contextual and nuanced understanding of the term as well as consideration of who the primary beneficiaries of participatory activities ought to be constitute crucial factors to realizing the benefits and opportunities that participation brings.

1 INTRODUCTION

The paper argues that participation can help align AI with prosperity and empower marginalized communities, but its meaning, beneficiaries, and role in AI development require clarification. It reviews participation’s history and AI/ML applications, presents three case studies, and characterizes associated limitations and concerns.

  • Participation is presented as a way to align AI with prosperity, particularly for marginalized communities, while requiring scrutiny of its scope, uses, and limitations.
  • Participatory AI aims to move beyond individual opinion toward inclusion, plurality, collective safety, ownership, co-design, and community empowerment.
  • Participation can be weakly executed or co-opted to advance predetermined aims, a concern described as participation-washing.
  • The paper calls for clearer accounts of what participation means, whom it serves, how it operates in AI, and how it relates to existing mechanisms.
  • The paper reviews participatory approaches historically, examines participatory methods in AI and ML, and presents three case studies across participatory AI.
  • The paper introduces a characterization for comparing multiple forms of participation and discusses their potential limitations and concerns.

2 GENEALOGY OF PARTICIPATION

The genealogy of participation shows both emancipatory possibilities and recurring risks of exploitation, co-optation, and unchanged power relations. These histories inform the paper’s analysis of participatory AI and its emphasis on purpose, reciprocity, and beneficiaries.

  • Participation has been used across sectors to cultivate skills, social capital, networks, self-determination, and community welfare.
  • Participation can incorporate knowledge from people directly affected by undertakings, whereas its absence can leave projects based solely on technocratic or elite perspectives.
  • Scandinavian approaches linked participation to workplace democracy, worker control, technological adoption, knowledge-sharing, and collective negotiation.
  • Historical examples show that participation can legitimize colonial power, mask uneven relations, and preserve extractive or unjust arrangements.
  • Arnstein’s critique emphasizes that participation without redistribution of power can let powerholders claim consideration while only some sides benefit.
  • Participatory design in technological innovation sought to improve products and reduce time and costs by involving end-users in design.
  • Participation for technological development may improve products without benefiting or empowering those engaged in co-design.
  • The paper frames participatory AI through three case studies and a characterization involving objectives such as reciprocity, reflexivity, and collective exploration.

3 THREE CASE STUDIES IN PARTICIPATORY AI

The paper presents three participatory AI case studies spanning community-driven machine translation, Māori data sovereignty, and participatory dataset documentation. Together, they illustrate participation as a route to community-centered technology, methodological innovation, reciprocity, refusal, and improved dataset practices.

  • Case studies: Participatory AI projects take diverse forms, including stakeholder involvement in design and implementation, grassroots organizing, algorithm building, sociocultural data collection, and dataset documentation.The paper presents three case studies to illustrate different models of participation.
  • Case 1: Machine translation for African languages: Masakhane involved participants in language-data sourcing, model-output evaluation, benchmark production, and collaboratively defined, iterative research processes.Participants had no fixed prerequisites or roles; agendas were public and democratically voted on.
  • Case 2: Fighting for Māori data rights: Māori data sovereignty practices restricted sharing to prevent commercial actors from shaping the language’s technological future without connection to the language.The community established Māori Data Sovereignty Protocols to take control of its data and technology.
  • Case 2: Fighting for Māori data rights: The Māori case study centered community needs, goals, benefits, and interests while combining methodological innovation with reciprocity and avenues for refusal.The Māori community recorded and annotated 300 hours of Te Reo Māori audio data and developed data sovereignty protocols.
  • Case 3: Participatory dataset documentation: Participatory dataset documentation is presented as a people-centered approach intended to improve dataset quality, validity, reproducibility, reflexivity, critical evaluation, and meaningful feedback.The approach aims to make datasets accessible to a wider set of stakeholders and improve trust.

4 LIMITATIONS AND CONCERNS

The paper identifies limitations and concerns involving democracy, conflation, exclusion, cooptation, measurement, expertise, incentives, and uneven power. These concerns constrain what participatory AI can legitimately do and require context-sensitive practices.

  • Effectiveness and Measurement: Participatory methods are not solutions for every problem and require monitoring, evaluation, learning, and further research to become regular practice.The paper emphasizes that metrics used to evaluate participation can themselves obstruct effective use.
  • Democratic governance: Participation cannot substitute for democratic institutions when decisions involve significant public concern, coercive law enforcement, or requirements for stronger legitimacy.Private or parallel participatory activities cannot stand in for democratic politics.
  • Conflation with other activities: Inclusion is related to but distinct from participation: people may be included in a group without voting, writing, or otherwise participating.Addressing inclusion also requires attention to systemic barriers such as racism, patriarchy, and wealth exclusion.
  • Conflation with other activities: Participatory invitations never reach everyone, often excluding people with low literacy, limited time, or insufficient network access; abstention, refusal, dissent, and protest can also constitute participation.Exclusion may sometimes be needed for safe and open participatory action.
  • Cooptation: Participation can be coopted when corporate actors use grassroots efforts and shared data for products that maximize profits while providing little community benefit.Such dynamics can disempower participants and position corporations as legitimate arbiters of language technology.
  • Effectiveness and Measurement: The benefits of participation, including empowerment, knowledge transfer, social capital, and social reform, are often gradual and intangible, making impact and attribution difficult to measure.Blunt instruments such as cost-benefit analysis may make investments in iterative co-learning appear wasteful.
  • Expertise and Incentives: Participatory processes must account for both technical expertise and communities’ lived experience, while recognizing the epistemic burden and potentially distorted incentives involved.Simplified assumptions about why people participate can obscure uneven power distributions and dynamics hidden by the notion of community.

5 CONCLUSION

Participatory AI can support justice, prosperity, community empowerment, and more responsive systems, but its value depends on clarifying its purposes, beneficiaries, and limitations. The paper frames participation as historically situated, potentially empowering, and requiring reflexive assessment across the AI pipeline.

  • Participation can bring communities’ knowledge, expertise, and interests into AI development aimed at strengthening justice and prosperity.
  • Misunderstanding participation’s role can reduce its effectiveness and increase the risks of harm and exploitation for participants.
  • The paper characterizes participation as historically connected to power, oriented toward vibrant engagement, and composed of methods with specific uses and limitations.
  • Participatory practices span the AI pipeline and can be assessed along dimensions of empowerment and reflexive assessment.
  • The case studies identify desirable forms of participation that can inform future participatory AI practice.
  • The paper seeks to clarify how participatory methods can be incorporated and how critique can address inadequate participation.

A REFLEXIVE ASSESSMENT OF PARTICIPATORY PRACTICES

The paper proposes a reflexive assessment of participatory practices because their goals, incentives, contexts, and power relations vary. Its questions ask researchers to make participation’s objectives, governance, participant experience, ownership, timing, and adaptability explicit.

  • Participatory efforts differ in motivations, objectives, attributes, and sociotechnical contexts, so meaningful participation cannot be assessed uniformly.
  • The assessment is especially important when researchers, technologists, or institutions with power call for participation.
  • The questions are intended as a reflexive guide rather than an exhaustive checklist for clarifying goals, objectives, and limitations.
  • The proposed questions examine whether project goals support community interests and how trust and power imbalances will be addressed.
  • The framework asks whether communication is transparent, participants can question the product itself, and disagreement is possible.
  • It also asks how participants experience the process, what they own and gain, and whether participation exceeds isolated individual experiences.
  • Further questions address engagement duration, pipeline timing, withdrawal without harm, and whether earlier decisions, data, or evaluations can be revised.

B CURRENT MODES OF PARTICIPATION IN AI

Participatory AI has no single unified vision: it encompasses overlapping practices and goals ranging from technical improvement to social reform and redistribution of power. The paper therefore offers a contextual characterization organized around motivations and objectives.

  • There is no single unified understanding of what participation is or what it should achieve.
  • Participatory practices range from improving model performance, accuracy, and data quality to pursuing social reform and redistributing power and resources.
  • A universal definition and vision of participatory AI is considered futile because participation has different implications across forms and contexts.
  • The paper presents three broad, overlapping categories of participation to characterize their implications and consequences in ML projects.
  • The characterization examines participation through its driving motivations and objectives.

B.0.1 Participation for algorithmic performance improvement.

Participation for algorithmic performance improvement uses bounded contributions to improve technical components such as data or models, while potentially improving representation. Its benefits are constrained when participants lack influence over the wider system or downstream outcomes.

  • This mode uses participation to improve the quality, robustness, diversity, accuracy, or efficiency of technical system components.
  • Community data labeling, annotation, entry, and cleaning commonly assign participants predefined roles and discrete tasks.
  • Rigid participant roles can support particular objectives while foreclosing others.
  • Including historically marginalized communities and their subject-matter expertise can improve representation and potentially benefit those communities.
  • Data from marginalized communities may nevertheless be misunderstood or misinterpreted.
  • Because participants’ influence is often limited to specific components, community benefit is not guaranteed.
  • One-way communication and limited access to system goals or downstream impacts can leave participants with little room for negotiation.
  • Piecemeal participation may be extractive when it does not produce net benefits for participants or technology that benefits marginalized groups.

B.0.2 Participation for process improvement.

Participation for process improvement incorporates participants’ beliefs, attitudes, needs, and feedback into project or product design, while designers retain the main agenda and boundaries. It can support mutual benefit but does not generally challenge whether the system should exist or broader power asymmetries.

  • It can generate mutual benefits by helping organizations improve products while addressing the interests of marginalized customers and communities.
  • Although project goals may be discussed and revised, participants generally cannot question whether the design or service should exist at all.
  • Participation remains bounded because designers and organizational actors typically set the agenda, make major decisions, and define the project’s technical objectives.
  • This form includes participants in designing projects, products, or services through activities such as feedback, surveys, focus groups, and UX studies.
  • Compared with participation for algorithmic improvement, this approach is more reciprocal but still leaves major power asymmetries and oppressive structures largely unchallenged.

B.0.3 Participation for collective exploration.

Participation for collective exploration centers community needs, goals, futures, and lived expertise, with decisions made through sustained, active involvement across the project. It can question or discard the project itself, but requires substantial time, resources, and tolerance for unpredictable outcomes.

  • Participants help shape the central agenda, organization, planning, decision making, and construction through ongoing and sometimes unpredictable discussion.
  • This approach treats products or model improvements as possible by-products rather than its originating focus, and it is rarely found within AI.
  • Collective exploration prioritizes local knowledge and lived experience, with participants and stakeholders engaging in active, iterative co-learning.
  • Collective exploration is slow-paced and resource-intensive, requiring sustained time, money, physical interaction, and outcomes that may not be monetarily measurable.
  • Active involvement can extend from defining boundaries and conceptualizing problems through designing tools, collecting data, and annotating it.
  • Because community needs and futures drive the process, participants can question or discard a project, product, or tool that conflicts with community goals.
Loading 2209.07572v1…