Source-linked AI summary
Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices
Bogdana Rakova, Jingying Yang, Henriette Cramer, Rumman Chowdhury
TL;DR
Responsible AI initiatives face organizational barriers that make it difficult to translate concern about AI harms into accountable practice. Through qualitative interviews and a workshop, the paper maps prevalent, emerging, and aspirational organizational practices, finding that structural support and reduced individual labor can enable broader responsible AI work while implementation depends on organizational context.
Problem
Practitioners need to operationalize responsible AI within organizational structures, but how growing attention to AI harms can effectively drive industry change remains an open question.
Method
The paper analyzes 26 semi-structured interviews with industry practitioners and uses a workshop to explore organizational structures, practices, barriers, enablers, and transitions.
Results
The study identifies accountability gaps, ill-informed trade-offs, misaligned incentives, emerging structural support, and aspirational practices integrating responsible AI across organizations.
Takeaways & Limitations
Responsible AI work is better supported when organizations provide structures and processes that reduce individuals’ burden of advocacy, change management, and organizational redesign.
Takeaways & Limitations
The study’s observations are situated in the nascent state of responsible AI functions, with some interviewees reporting stress and others leaving their organizations during the study period.
Abstract
from arXiv · showhide
Large and ever-evolving technology companies continue to invest more time and resources to incorporate responsible Artificial Intelligence (AI) into production-ready systems to increase algorithmic accountability. This paper examines and seeks to offer a framework for analyzing how organizational culture and structure impact the effectiveness of responsible AI initiatives in practice. We present the results of semi-structured qualitative interviews with practitioners working in industry, investigating common challenges, ethical tensions, and effective enablers for responsible AI initiatives. Focusing on major companies developing or utilizing AI, we have mapped what organizational structures currently support or hinder responsible AI initiatives, what aspirational future processes and structures would best enable effective initiatives, and what key elements comprise the transition from current work practices to the aspirational future.
1 INTRODUCTION
Responsible AI has moved from an academic concern toward practical organizational work, but practitioners still face barriers translating principles into accountable industry practices. This study uses interviews and a workshop to map current practices, desired organizational futures, and transitions between them.
- Responsible AI practitioners must translate theoretical concerns about algorithmic inequity into organizational practices within corporate structures.
- The gap between academic priorities and practitioner needs includes organizational tactics, stakeholder management, and operationalization beyond technical methods.
- Practitioners encounter tensions between research-community expectations and the concrete changes achieved inside large corporations, including successes and failures within the same company.
- The study analyzes 26 semi-structured interviews with professionals working on responsible AI or fairness-aware machine learning in practice.
- Interview data are used to compare prevalent, emergent, and aspirational organizational structures and practices, while a workshop gathered reflections and organizational recommendations.
2 LITERATURE REVIEW
Prior work shows that responsible AI guidelines remain difficult to translate into context-specific organizational processes. Organizational change, internal legitimacy, external pressure, and cross-organizational collaboration therefore frame the paper’s focus on how adoption occurs in industry.
- The large number of high-level AI ethics guidelines has not resolved how organizations should translate them into concrete practices in specific contexts.
- Responsible AI requirements must reflect technologies, usage contexts, and local norms, while organizational change and stakeholder alignment support implementation.
- Compliance, diversity, inclusion, and privacy research provides related examples of the difficulty of turning principles and processes into ethical organizational behavior.
- External reputational or legislative pressure interacts with internal processes, audits, managerial diffusion, and legitimization of responsible AI work.
- Organizational change is treated as fluid and socially constructed within open systems characterized by interdependent information, activities, participants, resources, and institutions.
- The paper examines how practitioners experience organizational shifts and which structures or processes drive or hinder adoption of responsible AI practices.
3 STUDY AND METHODS
The study combines interviews across diverse industry roles and organizations with qualitative affinity diagramming and a workshop-based design exercise. These methods examine how responsible AI work is situated, supported, constrained, and changed within organizations.
- Researchers conducted 26 semi-structured interviews with practitioners across four continents and 19 organizations.
- Participants were recruited for roles connected to product, policy, or legal teams whose work directly affected machine-learning products and involved responsible AI.
- Researchers established trust and transparency through ongoing practitioner conversations to support open, nuanced discussion of sensitive and often unvoiced challenges.
- The interview sample covered functions including strategy, engineering, human resources, legal, marketing and sales, research, policy, and product management.
- Interview questions examined current work, organizational placement, accountability, incentives, performance review, and the evolution of responsible AI practices.
- Interview data were analyzed through interpretation sessions and iterative bottom-up affinity diagramming that grouped codes into themes and relationships.
- A conference workshop used group design exercises to connect findings to participant experiences and map organizational practices, structures, interdependencies, and feedback loops.
4 RESULTS: INTERVIEWS
The section introduces a high-level overview of the interview findings before presenting the key themes that emerged from practitioners’ accounts.
- The findings are presented first at a high level.
- The section then turns to key themes from the interviews.
- The interview findings are organized as an overview followed by thematic discussion.
4.1 Overview
The analysis organizes practitioners’ perspectives into prevalent, emerging, and aspirational practices, while highlighting differing role framings, nascent responsibilities, and the need to answer four organizational questions.
- Most participants worked on responsible AI initiatives as individuals, while 11 of 26 volunteered time outside their official job functions.
- Practitioners described responsible AI work using function-specific language, such as product life cycles for project managers and governance guidelines for legal practitioners.
- The study notes stress-related challenges among a few interviewees and identifies these observations as opportunities for further study.
- The analysis distinguishes prevalent practices, emerging practices, and an aspirational future state within responsible AI work in industry.
- Organizations need processes and structures to determine when and how to act, measure success, rely on internal structures, and resolve tensions.
- Scaling responsible AI requires transitioning from prevalent or emerging practices toward aspirational structures and processes, though emerging practices may not all lead there.
4.2 When and how do we act?
Organizations commonly approached responsible AI reactively, while emerging champions and review processes enabled more proactive work; practitioners envisioned anticipatory, organization-wide support.
- Many organizations acted reactively, fewer acted proactively, and respondents aspired to anticipatory approaches for responsible AI decisions.
- Prevalent work practices: Catastrophic media attention and declining media tolerance for the status quo were prevalent incentives for responsible AI action.
- Prevalent work practices: Reactive practices included role uncertainty, uncompensated work, and volunteer-led investigations that sometimes preceded full-time responsible AI teams.
- Emerging work practices: A few organizations introduced proactive evaluation and review processes, distributing responsible AI work and accountability across teams with legal support and educational initiatives.
- Emerging work practices: Proactive champions used grassroots action and internal advocacy to build organizational support while still carrying responsibility for structural change.
- Aspirational future: Respondents envisioned integrated tools and organization-wide processes that identify and address risks before machine-learning systems reach products.
4.3 How do we measure success?
Practitioners struggled to measure and communicate responsible AI impact because existing business metrics often miss long-term and societal outcomes. Emerging and aspirational practices seek broader value frameworks.
- Many responsible AI initiatives target societal impact, creating a mismatch with traditional business metrics such as revenue and profitability.
- Prevalent work practices: The majority of respondents reported inadequate impact metrics and difficulty communicating the importance of their responsible AI work.
- Prevalent work practices: Practitioners identified misleading metrics, differences between academic and industry measures, and product indicators such as click rate and time spent using a product.
- Prevalent work practices: Short development timelines, time pressure, and low prioritization of qualitative work pushed teams toward short-term, easier-to-measure goals.
- Emerging work practices: Some organizations implemented metrics frameworks and processes to evaluate responsible AI risks while accommodating diverse and long-term goals.
- Aspirational future: In the aspirational future, responsible AI would enter product-team key performance indicators, ethical decision-making, and employee performance evaluations.
4.4 What are the internal structures we rely on?
Internal accountability is commonly ambiguous and dependent on individual authority, while emerging structures distribute responsibility and future processes would integrate it across product work.
- Prevalent organizations lacked internal structures ensuring accountability, whereas emerging practices distributed accountability and aspirational practices integrated it into product processes.
- Prevalent work practices: Practitioners reported ambiguity about role definitions and responsibilities, partly because responsible AI work evolves rapidly.
- Prevalent work practices: Existing accountability often depended on individuals’ resources, interests, seniority, and situational power rather than scalable organizational processes.
- Emerging work practices: Emerging enablers included dynamic roles, distributed accountability, escalation processes, review boards, responsible AI research groups, and cross-functional roles.
- Aspirational future: Respondents envisioned responsible AI reviews, reports, and product-specific artifacts being integrated throughout product development.
4.5 How do we resolve tensions?
Practitioners described unresolved ethical tensions arising from misaligned incentives, unclear responsibilities, and organizational processes that remain reactive. They envisioned organization-level values, leadership support, and structures that align responsible AI work with organizational missions.
- Current tensions: Organizations must update prevalent practices to resolve ethical and unintended-consequence tensions through explicit prioritization and trade-offs.Responsible AI introduces tensions that existing organizational processes may not yet be equipped to handle.
- Current tensions: Misalignment between individual, team, and organizational incentives often led practitioners to perform ad hoc responsible AI work based on personal values.Information sharing also frequently depended on individual relationships.
- Current tensions: Practitioners encountered recurring tensions around whether prototypes should scale and how to communicate that models remained incomplete or under validation.These tensions involved geographic scope, expectations, and work-in-progress status.
- Current tensions: Rigid incentives and organizational inertia could demotivate practitioners from addressing identified ethical tensions.Emerging structures shifted labor toward organization-wide processes, but those processes were not always aligned with responsible AI goals.
- Current tensions: Limiting factors included rewards for unnecessary complexity, unclear consequences, diffuse impact, and inadequate support and communication.These conditions made it harder for organizations to resolve tensions in ways that enabled responsible AI work.
- Aspirational future: Respondents wanted leadership to understand, support, and deeply engage with responsible AI concerns as part of high-level organizational values and mission statements.Their aspirational vision placed responsible AI within the organization’s broader context rather than treating it as an isolated effort.
5 RESULTS: INTERDISCIPLINARY WORKSHOP
The interdisciplinary workshop examined how responsible AI enablers should fit specific socio-technical contexts. Participants emphasized veto powers, internal and external pressure, broad perspective-sharing, and coordinated implementation, while noting that combined enablers are difficult to implement.
- Workshop framing: Workshop participants organized discussion around four context-sensitive questions: when to act, how to measure success, which internal structures to rely on, and how to act.The questions were intended to be considered within each team’s and organization’s socio-technical context.
- When and how do we act?: Before addressing fairness or societal implications, groups recommended determining whether an AI system is appropriate at all.If the answer is negative, development should stop, supported by veto powers across employees, oversight committees, investors, and boards.
- Pressure and accountability: Internal evaluation offers greater information access and transparency, while external processes can mobilize stakeholders and build momentum.External pressure may also be easier to apply because employees can fear repercussions for speaking up internally.
- Communication and participation: Participants identified structured opportunities for people with different perspectives to exchange input as a key organizational enabler.One recommendation was a regular semi-public town hall where employees could contribute to organization-wide values.
- Implementation challenges: The proposed enablers often worked best in tandem, creating implementation challenges.For example, whistleblower protections and a supportive culture were both needed for candid participation in town halls.
- Scaling responsible AI: Organization-level structures and processes were identified as mechanisms that can support and amplify individual responsible AI efforts.The workshop themes were offered as starting points for experimentation, with pooled results expected to accelerate collective learning and progress.
6 DISCUSSION AND CONCLUSION
The discussion frames responsible AI as an organizational change problem in which individuals navigate tensions between current practices and an aspirational future. Effective transition requires structures that shift labor away from individuals and support ongoing adaptation.
- Organizational change: Responsible AI requires organizational structures to adapt so they support rather than hinder practitioners’ work.The paper connects this transition with advocacy, early successes, and proactive leadership steering.
- Transition pathways: Practitioners need to map routes from prevalent practices to aspirational goals while building momentum through useful emerging practices.They must also avoid emerging practices that work against desired long-term outcomes.
- Current organizational burden: Prevalent practices often place responsibility for identifying issues and changing outcomes on individual practitioners.This burden appears across the organizational questions examined in the paper.
- Current organizational burden: Individuals may have to perform their regular jobs, unpaid responsible AI work, organizational redesign, and change management simultaneously.These stacked demands can make organizational incentives appear misaligned with responsible AI goals.
- Implications for research and practice: The research community should provide organizational insight, training, mentorship, and sponsorship alongside technical or research skills.These supports would help practitioners communicate impact, build legitimacy, and navigate internal structures and tensions.
- Emerging practices: Emerging structures generally reduced the labor burden on individuals, although rigid incentives in high-inertia contexts could still hinder responsible AI work.Organizations were beginning to implement new structures or adapt existing ones.
- Aspirational future: In the aspirational future, organizational processes would monitor and adapt system-level practices while assigning internal advocacy and change management to dedicated roles.Practitioners could then focus on specific responsible AI issues within their functions.
- Method and conclusion: The study uses qualitative interviews to map barriers, enablers, and transitions that are persistent steps and coalition-building rather than linear movement between fixed states.Its focus is the organizational structure and culture surrounding people who build and deploy machine-learning systems.
A QUESTIONNAIRE
The questionnaire examined practitioners’ roles, how responsible AI efforts begin, accountability, incentives, risk culture, and organizational relationships. Its questions covered both formal structures and how work operates in practice.
- A.1.1 Describe your role.: The role section asked about formal titles, actual responsibilities, role alignment, organizational flexibility, role origins, scope changes, and decision-making autonomy.It also distinguished planned expansion, scope creep, stretch assignments, and other forms of role change.
- A.1.1 Describe your role.: The questionnaire probed how practitioners exercise impactful decision-making autonomy and what organizational conditions shape it.Respondents were asked to explain both the presence and absence of autonomy.
- A.1.2 How did your fairML effort start?: The fairML-start section asked whether efforts were officially sponsored, who launched them, why they began, and how they were communicated.It also examined whether efforts were programs or stand-alone initiatives tied to specific products or launches.
- A.1.2 How did your fairML effort start?: Questions about fairML efforts covered team membership, volunteering, recognition, planned activities, and collaboration with external groups.The questionnaire connected initiative origins with staffing, incentives, and external engagement.
- A.1.3 Responsibility and accountability.: The accountability section asked who identifies risks, develops solutions, fixes mistakes, manages negative impact, and connects sponsorship with risk management.It also examined stakeholders, collaborating departments, adjacent efforts, support, cultural alignment, and scalability.
- A.1.4 Performance, rewards, and incentives.: The performance section examined how algorithmic accountability is defined, how practitioners are evaluated, and what performance-management and compensation systems actually incentivize.It also asked which kinds of people receive consistent rewards.
- A.1.5 Risk culture.: The risk-culture section asked about relationships with communications and legal teams, ownership, visibility, authority, community engagement, ethical tensions, and perceived risk tolerance.The questions distinguished communications responsibilities from organizational risk-management capacity.
A.2 Future dream state - a structured way of geting a mapping of future dream state
The future-state mapping asks practitioners to describe current fairML practices, envision future practices, and identify what must change, retire, or be repurposed. It also elicits what works, the largest challenges, and any additional considerations.
- Practitioners were asked to characterize their company’s current fairML practice across people, process, and technology.
- The mapping asks practitioners to define their vision for the future state of fairML practices.
- Practitioners were asked what needs to change, retire, or be salvaged and repurposed to reach the future state.
- The interview questions also identify what practitioners value in their current setup and summarize its largest challenges.
- Participants could add topics not covered by the preceding questions.