Fairness Perceptions of Algorithmic Decision-Making: A Systematic Review of the Empirical Literature

1 Introduction

• Unfair ADM can reinforce racial or gender stereotypes, marginalize minorities, or denigrate people (Veale & Binns, 2017; Žliobaitė, 2017).Source: §1 · p. 2
• Across 39 empirical studies and over 25,000 unique observations, the review synthesizes citizens’ fairness perceptions of ADM.Source: §1 · p. 3
• The review combines systematic online and citation searches, screens more than 2,500 entries, and uses two coders in two coding steps.Source: §1 · p. 3
• The review organizes perceived algorithmic fairness around algorithmic predictors, human predictors, comparisons of HDM and ADM, and consequences of ADM.Source: §1 · p. 3

2 Bias in Algorithmic Decision-Making

• Yet AI-based systems have also reduced fairness, including erroneous exclusion from food support, cuts to disability benefits, and false fraud accusations (Barocas & Selbst, 2016; Richardson et al., 2019).Source: §2 · p. 3
• Historical training data can reproduce or exacerbate societal biases that harm minorities, while incomplete or unreliable feature measurements can under- or overrepresent groups.Source: §2 · pp. 3–4
• Careless model selection, design, or specification can yield discrimination because fairness varies by task; random forests and neural networks capture variable synergies better than linear regression.Source: §2 · p. 4
• Bias reduction is not merely technical: implementation can violate privacy or apply ADM to sensitive decisions, and fairness judgments depend on the underlying fairness concepts (Lepri et al., 2018; Wong, 2020; Köbis et al., 2021).Source: §2 · p. 4

3 Concepts of Algorithmic Fairness

• Formal definitions cluster into statistical measures, similarity-based measures, and causal reasoning, represented by statistical parity and equalized odds, fairness through awareness, and counterfactual fairness.Verma and Rubin (2018) reviewed more than 20 definitions and grouped formal concepts into these three categories.Source: §3 · p. 5
• Formal fairness definitions refer to different, often incompatible notions, making social context important when assessing appropriate algorithmic fairness trade-offs (Kleinberg et al., 2017; Lepri et al., 2018; Wong, 2020).Source: §3 · p. 5
• Fairness assessment should consider how inequality is produced, not only its distribution, and may require information about historical structural injustice against minorities (Binns, 2018a).Source: §3 · p. 6
• Organizational-justice approaches assess ADM fairness across distributive allocation, procedural criteria, informational transparency, and interpersonal respect for privacy and protected data (Greenberg, 1987, 1990, 1993; Leventhal, 1980).Source: §3 · p. 6
• Social-science concepts of equality of resources and equality of capability of functioning have not been adequately addressed in machine-learning literature (Gajane & Pechenizkiy, 2017).These concepts concern resource distribution that enables individual choice and equalized opportunities despite unequal social or natural endowments.Source: §3 · pp. 6–7
• Because ADM operates in social contexts and affects people, scholars advocate human-centred fairness research, and this review examines empirical studies of human perceptions (Grgić-Hlača, Redmiles, et al., 2018; Marcinkowski et al., 2020; Srivastava et al., 2019).Source: §3 · p. 7

4 Method

• The review followed Petticrew and Roberts’s (2006) seven-step approach, using predefined and transparent criteria to identify, screen, evaluate, summarize, and disseminate studies.The steps begin with defining a research question and inclusion criteria, followed by a comprehensive search and relevance screening.Source: §4 · pp. 7–8

4.1 Establishing the research question

• Using PICOC, the review asked how individuals perceive ADM fairness, defining the population broadly, the intervention as fairness in and through algorithms, and the outcome as individual perceptions.Comparison was left unspecified, and country and domain were not narrowed, giving the review global scope.Source: §4.1 · p. 8

4.3 Comprehensive Literature Search

• Database searches identified 4,045 contributions; 2,467 remained after deduplication, and manual gray-literature searches found 99 potentially relevant articles.The search covered Web of Science, PsycINFO, IEEE Xplore, and Scopus, with Google Scholar used to find gray literature.Source: §4.3 · p. 10

4.4 Screening & Critical Evaluation

• The initial screening involved two expert raters examining 2,566 publications, and the final sample consisted of 34 publications; Cohen’s κ rose from .55 to .74 after raters discussed reliability.Source: §4.4 · p. 12

5.1 Descriptive Results

• The studies were concentrated in Western democracies: 23 were conducted in the US, two each in the UK and Netherlands, and four collected data in multiple countries.One study each was conducted in Germany, South Korea, and China.Source: §5.1 · p. 13
• Seven studies used qualitative methods, 22 quantitative methods, and ten mixed methods.Source: §5.1 · p. 13
• Criminal justice was the most common domain (11 studies), followed by work-related decisions (seven); other domains included recommendations, donations, admissions, loans, and targeted ads.Source: §5.1 · pp. 13–14

5.2 Concepts of (Algorithmic) Fairness

• Fairness research varied conceptually, focusing mainly on decision outcomes while also addressing procedures and, in some studies, interactional, informational, and opportunity fairness.Computer science studies sometimes treated procedural fairness narrowly as the choice of sensitive input features, unlike broader social-science approaches; the literature lacks a coherent multidimensional concept.Source: §5.2 · pp. 14–15

5.3 Measurements of Algorithmic Fairness

• Measurements ranged from single-item ratings and adapted existing scales to preference-based measures, but no validated multidimensional scale for ADM fairness perceptions exists.All reviewed studies used simple single-item measures or scales originally designed for human decision-making.Source: §5.3 · p. 15

5.4 Algorithmic predictors of perceived algorithmic fairness

• Qualitative accounts split between viewing data-based ADM as inherently unfair and as impartial; judgments also invoke impartiality, accuracy, model features, omissions, errors, and flawed inputs.Source: §5.4 · p. 16
• Perceived fairness was context-dependent: discrimination in housing or job recommendations was seen as more harmful than in music or movie recommendations, with no globally approved definition.Source: §5.4 · p. 16
• Preferences for equality, equity, and efficiency varied across groups, while demographic parity best matched most respondents’ choices in both scenarios.In high-stakes situations, respondents weighed accuracy more heavily.Source: §5.4 · p. 17
• Feature-accuracy fairness was rated fairest, followed by feature-a-priori and feature-disparity fairness, while studies also found disagreement over criteria and preference for equalizing false-positive rates (Koene et al., 2017; Harrison et al., 2020).Koene et al. reported that users could not agree on an algorithm because their fairness criteria could conflict; Harrison et al. found respondents favored equalizing false-positive rates between groups.Source: §5.4 · pp. 18–19
• Relevance, whether a feature causes the outcome, and reliability were most important to fairness judgments; people generally judged relevant features fairer and unrelated ones less fair.Discriminatory advertising based on demographics rather than online behavior was rated more problematic.Source: §5.4 · p. 19
• Respondents initially opposed race and gender as inputs, but support increased when told these features could improve minority outcomes; missing data and subgroup misrepresentation were also seen as reasons for unfairness (Sambasivan et al., 2021).Respondents also judged race and political affiliation more unfair than age and health status.Source: §5.4 · pp. 19–20
• Explanations increased fairness perceptions in a news-recommender study, but outcome explanations reduced fairness when clarifying uneven distributions and increased it when clarifying equal utility distributions (Shin, 2021; Lee, Jain et al., 2019).The direction of outcome-explanation effects depended on context.Source: §5.4 · p. 20
• Explanation-style effects varied: case-based explanations lowered perceived fairness relative to sensitivity-based explanations, while input-influence and demographic explanations raised confidence and fairness; transparency findings were mixed.Transparency increased perceived fairness in one study, whereas varying its degree had no significant effect in another.Source: §5.4 · pp. 20–21

5.5 Human predictors of perceived algorithmic fairness

• Few studies found sociodemographic effects: women opposed gender as an input feature more than men, while education and age also shaped fairness perceptions and reasons for them.Young respondents viewed sensitive-feature use as unfair, and younger people demanded global regulation and ethical guidelines.Source: §5.5 · p. 21
• People tended to perceive algorithms as fairer when ADM produced a positive outcome for them; larger gaps between predicted and actual outcomes were associated with algorithms being perceived as more unfair.Source: §5.5 · pp. 21–22
• Familiarity findings were mixed: understanding a fairness definition increased rejection of it, while programming knowledge was associated with lower fairness perceptions in one study and higher perceptions in another (Saha et al., 2019; Lee and Baykal, 2017; Araujo et al., 2020).Evidence about knowledge of AI and algorithms did not point consistently in one direction.Source: §5.5 · p. 22
• People’s online self-efficacy—the belief they can protect their data—tended to increase perceived algorithmic fairness, whereas general concerns about personal-data collection and use had a negative effect on respondents’ fairness perceptions.Source: §5.5 · p. 22

5.6 Comparative effects (HDM vs. ADM)

• A review identified fourteen studies examining whether human or algorithmic decisions are perceived as fairer; empirical findings were inconsistent and highly context-dependent, with some studies finding no significant difference.Source: §5.6 · pp. 23–24
• Survey respondents more often believed AI makes fairer decisions (54%) than humans (33%), while human decision processes were viewed as fairer than algorithms in criminal justice, hiring, and social-division tasks.Source: §5.6 · p. 23
• Black respondents favored ADM only when their group lacked representation in HDM; mechanical-skill tasks were tied, while human-skill tasks favored HDM (Miller & Keiser, 2020; Lee, 2018).Algorithms were perceived to lack intuition and subjective judgment in human-skill tasks.Source: §5.6 · p. 24
• High-complexity tasks favored humans or hybrid systems, low-complexity tasks favored algorithms, and outcome control increased perceived algorithmic fairness (Nagtegaal, 2021; Lee et al., 2019).ADM was also perceived as fairer than HDM for high-impact health and justice decisions (Araujo et al., 2020); varying human involvement in development and deployment made no significant difference (Wang et al., 2020).Source: §5.6 · pp. 24–25

5.7 Consequences of perceived fairness

• Perceived fairness positively affected satisfaction with algorithms and, in causal analyses, trust in algorithmic recommendations; trust moderated the satisfaction effect (Shin and Park, 2019; Shin et al., 2020, 2021).Kasinidou et al. (2021) found no correlation between perceived fairness and trust; the review describes fairness perceptions as important to algorithm adoption.Source: §5.7 · pp. 25–26
• In hiring, perceived procedural and interactional unfairness was associated with lower organizational attraction or commitment and job-pursuit intention, while low interactional fairness increased litigation likelihood.Marginalized respondents said algorithmic fairness or its absence could substantially affect trust in a company (Woodruff et al., 2018).Source: §5.7 · p. 26
• In university admissions, distributive and procedural fairness perceptions reduced protest intention; procedural fairness reduced exit likelihood, while distributive fairness improved institutional reputation (Marcinkowski et al., 2020).The authors cautioned that perceived unfairness could have detrimental implications for implementing admissions ADM systems.Source: §5.7 · p. 26
• Distribution-norm preferences varied by domain, task, and design; respondents favored straightforward fairness definitions, and explanations showed a tentative positive effect.Source: §5.7 · p. 27
• Sociodemographic predictors yielded inconclusive findings, whereas political ideology and self-interest influenced citizens’ fairness perceptions of ADM.Source: §5.7 · p. 27
• Comparisons of fairness perceptions for human and algorithmic decisions produced ambiguous, context-sensitive results across application areas, precluding generalizations.Source: §5.7 · p. 27
• Although little empirical research examines consequences of perceived algorithmic unfairness, initial findings indicate reputational harm and greater willingness to exit institutions using unfair ADM.Source: §5.7 · pp. 27–28

6.1 Theoretical Groundwork

• Fairness perceptions of ADM depend on algorithm design, application domain, and task, while inconsistent theories and concepts contribute to inconclusive findings.The review calls for theoretical work to systematize contextual factors and unify approaches to perceived algorithmic fairness; it also notes that people may evaluate ADM and human decision-making using different factors.Source: §6.1 · pp. 28–29

6.2 Diversification vs. Harmonization of Research

• The reviewed studies were conducted almost exclusively in Western democracies, and fairness perceptions vary across sociocultural contexts, designs, and application areas.The authors call for research in more countries, domains, and tasks, alongside systematic comparisons across contexts; they particularly identify China and South Korea as settings where ADM is already widely implemented (Sambasivan et al., 2021).Source: §6.2 · pp. 29–30
• The authors also urge harmonized, validated fairness measures that enable comparable findings and identify specific process deficiencies, rather than only judging a process fair or unfair.They argue that diagnostic measurements could help researchers, developers, and policymakers fine-tune ADM to citizens’ concerns.Source: §6.2 · pp. 30–31

6.3 Understanding Consequences of ADM Systems

• Few empirical studies examine the consequences of perceived algorithmic unfairness, despite its potential to affect institutions and people’s attitudes or behavior.The authors encourage research on attitudinal and behavioral implications, especially in high-stakes settings; unfairness perceptions could, for example, deter qualified applicants from applying.Source: §6.3 · pp. 31–32

6.4 Understanding Emerging Trade-offs

• The discussion identifies trade-offs among accuracy, fairness, transparency, accountability, explainability, privacy, and security, and calls for research into how citizens want them resolved.The authors note that maximizing accuracy and fairness can be infeasible and ask how citizens evaluate such design trade-offs in terms of fairness.Source: §6.4 · p. 32
• Fairness evaluations of trade-offs merit study across domains: people might prioritize accuracy in high-stakes medical decisions but fairness in social-benefit distribution.The review recommends conjoint experiments to examine multiple incompatible goals jointly.Source: §6.4 · p. 33

6.5 From Human-in-the-Loop to Society-in-the-Loop

• Evidence comparing human and algorithmic decision-making is inconclusive, with fairness judgments varying across domains and tasks, so each algorithm requires thorough investigation before broad use.The review reports evidence both for and against the view that algorithms are more objective, impartial, and fair than human decision-makers.Source: §6.5 · p. 33
• Human oversight ranges from intervention throughout the decision cycle to design-and-operation monitoring and broader command, and fairness perceptions may differ across these hybrid configurations.Source: §6.5 · p. 34
• The review identifies a blind spot: studies rarely consider institutional implementation questions beyond algorithm design and human involvement in decisions.Such questions include how systems are introduced and how affected people are informed, allowed to appeal, and protected through clear accountability.Source: §6.5 · p. 34
• The proposed society-in-the-loop approach joins human oversight with a social contract, involving interdisciplinary expertise, civil society, institutional leaders, and public opinion.Source: §6.5 · pp. 34–35
• Simple fairness definitions such as demographic parity are often seen as fairest, but explaining minority harms can increase support for complex algorithms.The review cautions that designing systems solely around popular opinion might backfire.Source: §6.5 · p. 35
• The review calls for multiple stakeholders to shape ADM design, establish shared ethical standards, and integrate those standards into institutional and legal frameworks.Source: §6.5 · p. 35

• The review responds to fairness concerns arising as ADM increasingly penetrates society and to calls for a human-centric approach to designing and implementing it.Source: §7 · p. 35
• The review synthesizes 39 empirical studies across algorithmic predictors, human predictors, HDM-versus-ADM comparative effects, and consequences of ADM.Source: §7 · p. 36
• The review calls for research beyond Western contexts and theoretical and methodological groundwork to harmonize concepts and measurements of perceived algorithmic fairness.Source: §7 · p. 36
• The authors advocate interdisciplinary research using a society-in-the-loop framework to contribute to fairer and more responsible ADM.Source: §7 · p. 36

• Measures ranged from single-item ratings to short multi-item Likert scales, with examples using 5-, 7-, and 9-point scales.Source: Perceived fairness of algorithmic · pp. 50, 52
• Fairness concepts included distributive, procedural, interactional, and informational fairness, as well as group, substantive, and procedural fairness.Source: Perceived fairness of algorithmic · p. 50
• Other conceptualizations included economic fairness, numerical and proportional equality, distributive justice, outcome fairness, equality of opportunity, and equity.Source: Perceived fairness of algorithmic · pp. 51, 53
• Some studies assessed adjacent outcomes instead of fairness itself, including process preferences, targeted-advertising problems, demographic-parity comprehension, model approval, or predictors perceived as fair.Source: Perceived fairness of algorithmic · pp. 52–53