Source-linked AI summary
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Fernando Delgado, Stephen Yang, Michael Madaio, Qian Yang
TL;DR
Stakeholder participation in AI design has broad support but varies substantially in goals, methods, and the agency it grants. This paper synthesizes participatory scholarship and analyzes participatory AI research and practitioner interviews, finding that practice is largely consultative and proposing ways to align participation goals with methods and practical constraints.
Problem
Despite growing calls to involve communities affected by AI, participatory approaches vary widely in their methods and goals, making stakeholder agency difficult to assess.
Method
The paper synthesizes literature across participatory scholarship, develops a conceptual framework, analyzes published participatory AI research, and conducts semi-structured interviews with AI researchers and practitioners.
Results
Participatory AI practice largely uses consultative interactions that elicit stakeholder preferences and values, while few projects allow stakeholders to drive or own parts of the design process.
Takeaways & Limitations
Researchers and practitioners should align participatory methods with specific goals, levels of stakeholder agency, and decision-making authority rather than treating participation as all-or-nothing.
Takeaways & Limitations
The corpus may omit relevant projects and the interviews may miss important perspectives, especially those of policymakers, civil society organizations, communities, and people affected by AI systems.
Abstract
from arXiv · showhide
Despite the growing consensus that stakeholders affected by AI systems should participate in their design, enormous variation and implicit disagreements exist among current approaches. For researchers and practitioners who are interested in taking a participatory approach to AI design and development, it remains challenging to assess the extent to which any participatory approach grants substantive agency to stakeholders. This article thus aims to ground what we dub the "participatory turn" in AI design by synthesizing existing theoretical literature on participation and through empirical investigation and critique of its current practices. Specifically, we derive a conceptual framework through synthesis of literature across technology design, political theory, and the social sciences that researchers and practitioners can leverage to evaluate approaches to participation in AI design. Additionally, we articulate empirical findings concerning the current state of participatory practice in AI design based on an analysis of recently published research and semi-structured interviews with 12 AI researchers and practitioners. We use these empirical findings to understand the current state of participatory practice and subsequently provide guidance to better align participatory goals and methods in a way that accounts for practical constraints.
1 INTRODUCTION
As calls for stakeholder participation in AI design grow, approaches vary widely in their goals and methods, making substantive stakeholder agency difficult to assess. This paper responds with a literature-grounded framework and empirical analysis of participatory AI practice.
- Participation is increasingly advocated so AI systems reflect stakeholders’ values, preferences, and needs or empower them to shape system design.
- Participatory AI approaches differ substantially in their methods, theories, and underlying goals, spanning community, collective, and commercial practices.
- The paper synthesizes participation theories across fields to develop a framework distinguishing participatory strategies and tactics.
- The authors analyze 80 research papers and interview 12 AI researchers and practitioners to map current practice and understand its motivations, challenges, and aspirations.
- Most current efforts consult stakeholders about discrete implementation parameters rather than empowering them to make key AI design decisions.
- The paper contributes a framework connecting participation goals to techniques, empirical findings on current practice, and opportunities for future research.
2 STAGE ONE: ESTABLISHING A FRAMEWORK FOR EVALUATING PARTICIPATION 2.1 Related Work
The paper develops a descriptive framework for evaluating participatory AI by connecting participation’s goals, scope, methods, and stakeholder agency. Its modes range from consultation to stakeholder ownership, without treating ownership as universally appropriate.
- Related Work: The framework addresses the difficulty of evaluating participatory approaches amid heterogeneous goals and practices.
- Related Work: It synthesizes participation scholarship from technology design, political theory, and the social sciences to establish a theoretical point of reference.
- Related Work: Prior traditions include user-centered design, service design, participatory design, and co-design, which differ in stakeholder roles and design involvement.
- 2.4 Parameters of Participation: The authors use affinity diagramming across nine traditions to derive Parameters of Participation, mapping modes from consultation to ownership and dimensions that shape goals and methods.
- 2.4 Parameters of Participation: The framework’s dimensions are goals, scope, and methods: why participation is needed, what decisions and stakeholders are involved, and how participation occurs.
- 2.4.2 Modes of Participation: The four participation modes form a spectrum from consulting and including stakeholders to collaborating with them and enabling ownership of the design process.
- 2.4.2 Modes of Participation: The schema is descriptive rather than a universal prescription, evaluating how configurations of participatory dimensions shape stakeholder agency.
3 STAGE TWO: EXAMINING CURRENT PARTICIPATORY APPROACHES TO AI DESIGN 3.1 Methods
The study examines participatory AI through a corpus analysis and interviews with researchers and practitioners. It codes published projects with the framework and analyzes interview data using deductive and inductive approaches.
- 3.1 Methods: The analysis combines 80 research articles with 12 semi-structured interviews involving researchers and practitioners who authored corpus papers.
- 3.1 Methods: The article corpus was gathered from ACM Digital Library proceedings and major AI venues using successive searches for participatory AI terminology.
- 3.1 Methods: Articles were included when they described AI system design using participatory techniques, yielding 80 relevant articles.
- 3.1 Methods: Two authors coded the corpus using the framework’s participation dimensions and modes, resolving disagreements through discussion and consensus.
- 3.1 Methods: Interviews examined motivations, stakeholder selection and roles, participatory processes, challenges, and participants’ aspirational visions.
- 3.1 Methods: Interview data were analyzed with a hybrid codebook-based and inductive thematic analysis approach.
3.2 Findings: Current Participatory AI Landscape - A Largely Consultative Terrain
The reviewed participatory AI projects largely consulted stakeholders about preferences, values, and interface features after core system decisions had already been made. Interviews also revealed that organizational constraints narrowed participation and encouraged pragmatic compromises.
- Participation goals: 80 of 80 projects aimed to improve user experience, while 52 of 80 sought better alignment with stakeholders’ preferences and values.Only 8 of 80 projects aimed to involve stakeholders in shaping the system’s scope and purpose.
- Scope of decisions: All projects involved stakeholders in shaping the user interface, but only 8 of 80 solicited input on models, features, objectives, loss functions, or decision thresholds.None allowed stakeholders to rule out AI as a solution.
- Stakeholder selection: 74 of 80 projects had project leads choose stakeholders, whereas only 6 of 80 used community-engaged designation and 3 of 80 involved community stakeholders throughout the lifecycle.The authors connect lead-controlled selection and sparse lifecycle involvement with reduced community agency.
- Participation methods: Most projects elicited stakeholder preferences or values, with 68 of 80 using methods such as surveys, interviews, role-playing, or collaborative story-boarding.Fifty-seven of those 68 projects engaged stakeholders only once.
- Timing and continuity: Fewer than a quarter of articles—18 of 80—described engaging the same participants more than once, and none involved stakeholders early in reflective deliberations.Participation therefore tended to occur in designated windows after system scope and purpose were determined or around interface components.
- Practical constraints: Interviewees reported that organizational timelines, resources, management metrics, funding demands, and publication pressures constrained stakeholder involvement and long-term relationship development.These pressures often made only how—not whether—AI would be deployed open to discussion.
3.3 Findings: Proxy-Based Participation
Participatory AI projects often used proxies to incorporate stakeholder perspectives when direct engagement was difficult. These proxies included human stand-ins, UX/HCI mediators, and algorithmic representations, each of which introduced limits on agency or representativeness.
- Proxy tactics: The authors identify three proxy tactics: human stand-ins, UX/HCI practitioners mediating between groups, and algorithmic proxies representing stakeholder interests.These tactics were adopted to ostensibly incorporate stakeholders’ voices without directly engaging all affected stakeholders.
- Human stand-ins: Human stand-ins included policymakers, educators, employees, and people with similar demographic or lived backgrounds who were expected to voice affected stakeholders’ preferences.Project teams selected these representatives based on perceived familiarity, work experience, demographics, or lived experience.
- UX/HCI mediators: UX/HCI practitioners were positioned as mediators who scoped, designed, and managed participation while translating between stakeholders and AI researchers.AI experts relied on them to equip non-experts to discuss technical capabilities, limitations, and design impacts.
- UX/HCI mediators: Mediating UX/HCI expertise could temper algorithmic technosolutionism, but it also placed stakeholder interaction at the periphery for some algorithm-focused team members.The corpus and interviews describe this empowerment of intermediaries as potentially coming at stakeholders’ expense.
- Algorithmic proxies: Algorithmic proxies solicited stakeholder preferences through systems in which participants trained models that voted on binary outcomes.This represented stakeholder preferences through model outputs rather than sustained direct participation.
- Algorithmic proxies: Model-training participation captured a snapshot of stakeholders’ values, while changing preferences made representative value modeling difficult even with repeated engagement.In one matching-system case, stakeholder preferences were further simplified to make computation tractable.
4 DISCUSSION
Current participatory AI practice largely elicits stakeholder preferences through consultative methods rather than granting stakeholders authority over core design decisions. Researchers and practitioners also rely increasingly on human and algorithmic proxies to manage practical constraints, raising questions about representation, power, and agency.
- Current practice: Participatory AI projects largely elicit stakeholder preferences through consultative approaches rather than granting stakeholders substantial design agency.Researchers and practitioners report difficulty mapping participatory ambitions to practical constraints.
- Proxy-based participation: Proxy-based tactics include representative stand-ins, UX/HCI mediators, and algorithmic models that elicit or model stakeholder preferences.Algorithmic proxies are especially relevant because machine-learning systems automate decisions and generate simulated user content.
- Implications: The paper recommends aligning participatory methods with explicit goals for stakeholder agency and decision-making authority rather than treating participation as all-or-nothing.It also identifies opacity, scale, and human or algorithmic proxies as challenges requiring further attention.
4.1 Moving beyond idealized aspirations toward specific outcomes
The paper argues that participation should be specified through project-level dimensions and concrete goals for stakeholder agency, while recognizing that meaningful participation must be designed around practical constraints. It cautions that participation can be harmful when it is extractive, tokenistic, or pseudo-participatory.
- Specific outcomes: Participation should be treated as a set of project-level dimensions with explicit goals for stakeholder agency and decision-making authority.This approach rejects both all-or-nothing participation and the assumption that any participation is automatically beneficial.
- Specific outcomes: The framework identifies consulting, involving, collaborating, and owning as intermediate modes between transactional consulting and transformative empowerment.These modes help distinguish levels of stakeholder agency across participation dimensions.
- Practical constraints: Moving beyond consultation requires ambitious but reasonable goals that account for stakeholder access, timelines, and available resources.The authors also call for practitioners to co-design and conduct participation collaboratively with stakeholders.
- Risks: Participation may be worse than no participation when it is extractive, tokenistic, or pseudo-participatory.Such practices can harm participants, reinforce power structures, or foreclose more meaningful participation.
4.2 Unique challenges for participation in AI
AI’s sociotechnical complexity creates distinctive challenges for meaningful participation, including systems that exceed ordinary human understanding and operate across large spatial and temporal scales. The paper therefore emphasizes explaining both specific system mechanisms and AI’s broader potentials before and during participation.
- Unique challenges: AI’s sociotechnical complexity creates participation challenges that existing frameworks from other domains do not fully address.The paper focuses especially on opacity and scale.
- Opacity: High-dimensional mathematical optimization and iterative retraining can make operationally useful AI systems difficult for humans to grasp.This interpretability dilemma distinguishes meaningful participation from mere inclusion.
- Opacity: Practitioners should explain specific AI mechanisms, foster localized understanding during participation, and build broader understanding of AI beforehand.These activities span system explanation, participation contexts, and general AI literacy.
- Scale: AI’s spatial and temporal scale raises questions about whether meaningful participation is feasible across large datasets, geographic deployments, and rapid development cycles.Narrow participation in individual components may not map clearly to system-wide decisions about data or pretrained models.
4.3 Risks in using proxies for participation
Proxy-based participation helps address limited stakeholder time and limited team resources, but it creates risks concerning representativeness, power imbalances, and agency. These risks include speaking for heterogeneous communities, filtering experiential knowledge, and freezing preferences that change over time.
- General risks: Proxy-based participation is used to address limited stakeholder time and AI teams’ limited time and resources, but raises concerns about representativeness, power, and agency.The paper frames proxies as potentially labor-saving or discounted forms of participation.
- Representativeness: Representative stand-ins may not adequately capture heterogeneous community preferences and can exclude marginalized members through selection bias.These concerns become especially serious when proxies inform value-based design or fairness and ethics evaluations.
- Mediation: UX/HCI mediators may undermine participants’ experiential knowledge and inadvertently elevate some voices while excluding others.The practice has been criticized in participatory design and HCI literature.
- Stale modeling: Algorithmic preference models can become stale as people’s preferences change, creating a distinction between human preferences and algorithmic voting.The paper connects this risk to the idea of technologies of de-politicization.
4.4 Limitations
The study’s corpus and interview sample may not represent all participatory AI work or stakeholder perspectives. Relevant projects and viewpoints could have been missed through venue, keyword, recruitment, or participation boundaries.
- The literature corpus may omit relevant projects published outside included venues or described with different participation terminology.Examples include HCI journals and policy institutes’ grey literature.
- The interviews may exclude important perspectives from sampled researchers and practitioners who were not reached or declined participation.
- Future work should include policymakers, civil-society and community organizations, AI participants, and people affected by the systems studied.
5 CONCLUSION
The paper develops an empirically grounded framework for participatory AI and finds that current practice is largely consultative rather than decision-making. It also identifies practical-constraint responses that can reinforce existing power dynamics and aims to guide more deliberate participation.
- Current participatory AI largely consults stakeholders about particular AI-system modules rather than involving them as active decision makers throughout design.
- Researchers and practitioners face tension between participatory ambitions and practical constraints while implementing AI projects.
- Projects negotiate constraints through stakeholder stand-ins, UX expertise, and algorithmic models that elicit stakeholder preferences.
- The framework is intended to help practitioners articulate participatory goals and implement methods more deliberately and strategically.
A CORPUS DESCRIPTION
The corpus spans participatory AI projects across diverse sectors, purposes, stakeholders, and methods. Examples include assistive living, education, healthcare, public services, online moderation, and creative or social applications.
- Methods: Participatory methods include design sessions and workshops, interviews, focus groups, surveys, ethnography, contextual inquiry, role playing, Wizard of Oz, and prototype evaluation.
- Participation mechanisms: Some projects aggregate stakeholder inputs into system specifications or use preference aggregation to support decisions and resource allocation.
- Domains and purposes: The corpus covers domains including assistive living, education, healthcare, public services, online moderation, communication, agriculture, and arts and entertainment.Examples address independence, inclusive education, diagnosis, resource allocation, content filtering, sociality, and creativity.
- Stakeholders: Stakeholders include children, older adults, people with disabilities, patients, caregivers, social workers, parents, teachers, moderators, job seekers, and community members.