Source-linked AI summary
Designing Fair AI for Managing Employees in Organizations: A Review, Critique, and Design Agenda
Lionel P. Robert, Casey Pierce, Liz Morris, Sangmi Kim, Rasha Alahmad
TL;DR
AI fairness research for managing employees lacks a theoretical and systematic approach to organizing design solutions. The paper reviews 25 design papers and finds that fairness types receive uneven attention, with more focus on distributive than interactional fairness.
Problem
Research lacks a theoretical and systematic approach for organizing design solutions concerning AI fairness in organizations.
Method
The paper develops a theoretical framework, reviews 25 design papers, and proposes a design agenda for AI fairness in organizations.
Results
The reviewed literature often overlooks differences among fairness types, focuses more on algorithm fairness, and gives more attention to distributive than interactional fairness.
Takeaways & Limitations
The paper provides a design agenda intended to guide, organize, and integrate AI-fairness design solutions across the HCI community.
Takeaways & Limitations
The paper identifies privacy, autonomy, organizational context, broader fairness perspectives, equity versus equality, accountability, and auditability as issues for future work and limitations.
Abstract
from arXiv · showhide
Organizations are rapidly deploying artificial intelligence (AI) systems to manage their workers. However, AI has been found at times to be unfair to workers. Unfairness toward workers has been associated with decreased worker effort and increased worker turnover. To avoid such problems, AI systems must be designed to support fairness and redress instances of unfairness. Despite the attention related to AI unfairness, there has not been a theoretical and systematic approach to developing a design agenda. This paper addresses the issue in three ways. First, we introduce the organizational justice theory, three different fairness types (distributive, procedural, interactional), and the frameworks for redressing instances of unfairness (retributive justice, restorative justice). Second, we review the design literature that specifically focuses on issues of AI fairness in organizations. Third, we propose a design agenda for AI fairness in organizations that applies each of the fairness types to organizational scenarios. Then, the paper concludes with implications for future research.
1. INTRODUCTION
Organizations are expanding AI use to manage workers, but AI decisions can be biased, opaque, and unfair. The paper responds with a justice-based review and design agenda for fairer AI management.
- AI in organizations: AI is increasingly used to direct, supervise, monitor, coordinate, and control organizational workers.A survey found that 86% of executives planned to use AI for employee management, while 78% of managers trusted it.
- The fairness problem: AI decisions may be unfair because of encoded biases or learning from biased human behavior, and their independent operation can make impacts unpredictable and invisible.The introduction also cites bias against female applicants in an AI-powered recruitment engine.
- The fairness problem: Worker unfairness associated with AI could decrease effort and increase turnover, motivating systems that support fairness and redress unfairness.The paper frames fairness as necessary for AI to manage workers effectively.
- Research gap: Existing AI-fairness discussions lack a theoretical and systematic design agenda and often fail to distinguish outcome, process, and interactional fairness.The authors argue that this can produce a one-size-fits-all approach and a fragmented design space.
- Contributions: The paper presents organizational justice theory, reviews design literature on organizational AI fairness, and proposes a framework for fairer AI design research.The framework is intended to provide conceptual clarity and organize, integrate, and guide current conversations and design solutions.
- Contributions: The proposed design agenda applies each fairness type to AI models, fairness evaluation, and redress of unfairness in organizational scenarios.The agenda is introduced as a guide for design research on fairer AI in organizations.
2. THEORETICAL BACKGROUND
AI has moved from narrow decision-support applications toward more autonomous organizational management, increasing concern about bias and fairness. Organizational justice theory distinguishes fairness types and links perceived unfairness to worker responses.
- AI in organizations: AI has expanded from narrow decision-support applications to real-time processing and more autonomous organizational activities, including applicant selection and work evaluation.The paper describes AI as supporting planning, organizing, controlling, and leading.
- AI in organizations: AI now automatically searches, filters, selects, and recommends job applicants and evaluates work with minimal human intervention or awareness.These uses extend beyond earlier applications designed primarily to aid human decision-making.
- AI and bias: AI systems are not necessarily less susceptible to human prejudice or bias despite claims that they provide more consistent, reliable, or impartial decisions.The paper contrasts earlier rhetoric about AI impartiality with broader acknowledgment of human bias in AI systems.
- Theoretical framework: Without a theoretical and systematic approach, researchers and designers lack a road map for interventions that promote fairer AI.The paper introduces organizational justice theory to address this need.
- Organizational justice theory: Adams’ equity theory explains fairness through reciprocal relationships between worker contributions and organizational rewards, while organizational justice theory explains reactions to unfairness.Workers may respond to perceived unfairness by contributing less or leaving the organization.
- Organizational justice theory: Organizational justice theory identifies distributive, procedural, and interactional fairness as distinct types with related positive work outcomes.The types concern outcomes, decision processes, and treatment by the organization.
- Fairness types: Distributive fairness concerns outcome allocation, procedural fairness concerns transparent and consistent decision processes, and interactional fairness concerns respectful treatment and adequate information.Procedural fairness includes allowing parties to be heard, while interactional fairness includes interpersonal and informational aspects.
- Fairness types: Fairness types are conceptually distinct even though each has been linked to outcomes such as trust, commitment, performance, and satisfaction.The paper illustrates distributive fairness through matching compensation to work contributions and comparable workers’ pay.
3. Literature Review on Artificial Intelligence (AI) Fairness in Organizations
The review examined design-focused AI fairness research in organizational contexts and used organizational justice theory to identify gaps and organize findings. Across 25 included papers, distributive fairness received the most attention, while procedural and interactional fairness and redress received more limited coverage.
- Review scope: The review identified 61 AI fairness articles and included 25 papers addressing fairness types, organizational contexts, and design implications.The review searched several search engines and screened studies using inclusion and exclusion criteria.
- Shortcomings: The literature review found limited differentiation among fairness types and little to no coverage of how to redress unfairness.The authors use fairness for distributive, procedural, and interactional types, and justice for restorative and retributive approaches to redress.
- Distributive fairness: Distributive fairness appeared in 18 of 25 design papers (72%), making it the most commonly discussed fairness type.The literature focused on fair allocation of outcomes such as organizational resources.
- Distributive fairness: Distributive fairness was overwhelmingly assessed by comparing outcomes across people or groups rather than inputs and outputs within individuals.The reviewed literature primarily used external measures such as mathematically validated algorithms or legal standards.
- Procedural fairness: Users reported higher procedural fairness when AI made objective decisions and lower procedural fairness when it made more human, subjective decisions.The review also reports that procedural justice matters regardless of whether a human or AI agent makes the decision.
4. Design Agenda for Artificial Intelligence (AI) Fairness in Organizations
The paper proposes a context-sensitive design agenda for organizational AI fairness, organized around fairness types, design components, and user affordances. It emphasizes operationalizing fairness, accounting for data and model bias, evaluating fairness, and enabling redress rather than treating fairness as an arbitrary general principle.
- Design principles: The agenda applies specific fairness types to organizational AI scenarios instead of prescribing one universally suitable type.Designers should move beyond vague fairness statements and consider how each fairness type is enacted in practice.
- Primary components: The framework has four primary components: operationalizing fairness, selecting data and computational models, evaluating fairness, and redressing unfairness.These components structure the proposed framework for supporting fairness in organizational AI systems.
- Operationalizing fairness: Operationalizing fairness requires identifying the management practice, defining it through a specific fairness type, and translating fairness into practice.This approach addresses practices such as hiring, promotion, and compensation rather than treating fairness as an abstract label.
- Data and model fairness: Data and model choices can directly shape how fairness types are enacted because measurement, variable selection, sampling, and model weighting introduce bias.Performance scores may reflect managerial relationships and organizational context, while unrepresentative samples can reduce prediction quality for under-represented groups.
- Redressing unfairness: AI systems should detect possible unfairness and provide restorative or retributive paths for redress, an area the reviewed literature has understudied.The agenda recommends considering redress before implementation and including viable options for restoring justice.
- Affordances: The agenda proposes transparency, explainability, visualization, and voice as affordances through which employees can understand, assess, and respond to AI decisions.These affordances treat employees as users who interact with systems rather than as passive recipients of AI actions.
5. Future Work and Limitations
The design agenda identifies several fairness issues it does not cover and presents them as limitations and opportunities for future research. These include privacy, autonomy, organizational context, fairness perspectives, equity versus equality, accountability, and auditing.
- Scope of the agenda: The agenda does not cover privacy, autonomy, organizational context, wider fairness perspectives, equity versus equality, accountability, or AI audits and auditability.The paper presents each issue as a limitation of the agenda and an opportunity for future directions.
- Protecting Worker Privacy: Fairness assessments may require data from multiple workers, while workers reporting unfairness may need anonymity to avoid identification and targeting.The paper calls for future designs that balance data requirements with privacy.
- AI Autonomy: Future research should determine how much autonomy AI receives, including when it identifies unfairness, recommends actions, seeks permission, or acts automatically.The appropriate degree of autonomy may depend on the situation or fairness type.
- Organizational Context: Organizational policies, priorities, and culture may influence which fairness definition an organization promotes and constrain the effectiveness of fairness designs.Future agendas should account for differences among organizations rather than relying only on similarities among them.
- Fair to Whom?: Future agendas could change the entity whose fairness is prioritized, including customers, organizations, workers, or competing platform participants.The paper highlights tensions between fairness for service providers and customers in platform settings.
- Equity versus Equality: The agenda uses equality rather than equity as its fairness basis, although equity may be preferred when individuals have widely different needs.Equality treats people the same, whereas equity provides what individuals need to be successful.
- AI Accountability: Accountability becomes difficult when bias reflects model variables, samples, multiple AIs, or shared responsibility between organizations and third-party firms.The paper defines accountability as assigning legal and financial responsibility, but notes that this assignment is not always direct.
- AI Audits and Auditability: An AI audit inspects underlying logic, decision criteria, and data sources to validate compliance, and auditability measures how much the AI supports such inspection.Audits may be manual, automated, simulation-based, or periodic.
6. Conclusion
The paper advances AI fairness in organizations through a literature review, an organizational-justice framework, and a design agenda. Reviewing 25 design papers revealed that fairness types were often overlooked, with more attention to distributive than interactional fairness.
- The paper combines a literature review, organizational-justice framework, and design agenda to advance fair AI design in organizations.The review included 25 design papers.
- 25 design papers formed the basis for the literature review underlying the paper’s conclusions.
- The reviewed literature often overlooked differences among fairness types and instead focused more on algorithm fairness.
- More attention was paid to distributive fairness than to interactional fairness after classifying the literature with the organizational-justice framework.
- The proposed design agenda is intended to help overcome these limitations and provide a starting point for future work.
Coding Process
The researchers coded the reviewed articles by fairness type and justice type, allowing papers to receive multiple classifications. Three researchers independently reviewed each paper, discussed disagreements, and reached consensus.
- The articles were coded by fairness type—distributive, procedural, or interactional—and justice type—retributive or restorative.
- Three researchers independently reviewed every paper and assigned fairness and justice types using the paper’s summarized definitions.
- Papers could receive multiple fairness and justice classifications.
- The research team discussed coding disagreements and resolved them through consensus.
- Table 3 summarizes how the literature was organized by fairness type.