Source-linked AI summary
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N'Mah Yilla, Jess Gallegos, Andrew Smart, Emilio Garcia, Gurleen Virk
TL;DR
Computing research lacks a synthesized overview of the diverse harms arising from algorithmic systems and their social contexts. The paper addresses this gap with a scoping review and reflexive thematic analysis, producing an applied taxonomy organized around five major harm types. The taxonomy supports systematic reflection and shared vocabulary while remaining bounded by partial, predominantly Western scholarly perspectives.
Problem
Computing research on algorithmic harms is vast and disparate, limiting access to a synthesized overview of harms for systematic consideration.
Method
The paper conducts a scoping review and reflexive thematic analysis of computing research on sociotechnical harms.
Results
The analysis identifies five major harm types: representational, allocative, quality-of-service, interpersonal, and social system harms.
Takeaways & Limitations
The taxonomy offers an initial shared vocabulary and guide for practitioners and researchers to reflect on adverse impacts and anticipate harms.
Takeaways & Limitations
The taxonomy reflects a partial scholarly literature, primarily English-language and Western-oriented, and does not include vital community-based advocacy.
Abstract
from arXiv · showhide
Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems. Based on a scoping review of computing research $(n=172)$, we present an applied taxonomy of sociotechnical harms to support a more systematic surfacing of potential harms in algorithmic systems. The final taxonomy builds on and refers to existing taxonomies, classifications, and terminologies. Five major themes related to sociotechnical harms - representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms - and sub-themes are presented along with a description of these categories. We conclude with a discussion of challenges and opportunities for future research.
1 INTRODUCTION
Existing research identifies many sociotechnical harms, but computing lacks a synthesized overview that organizes them for systematic consideration. This paper addresses that gap through a scoping review and reflexive thematic analysis intended to support harm identification, interdisciplinary communication, and harm reduction.
- Harms research examines impacts across individuals, communities, and social systems, including dynamics involving social exclusion and power.
- Existing work defines and evaluates many harms, but remains vast and disparate, often treating particular notions of harm in narrow contexts.
- The study uses a scoping review to map prior articulations of computational and contextual harms and a reflexive thematic analysis to organize them.
- The research asks what harms prior studies describe, how they affect micro-, meso-, and macro-levels, where concepts align, and what gaps remain.
- Its applied taxonomy provides terms, definitions, examples, and future-work directions to help practitioners and researchers surface harms more systematically.
2 BACKGROUND
Algorithmic harms arise from entangled technical, cultural, social, and power dynamics rather than from technical components alone. Existing taxonomies organize harms by domains, model functions, or system failures, motivating a broader sociotechnical framework.
- 2.1 Sociotechnical Harms: Algorithmic harms emerge through interactions among design decisions, norms, power, and technical systems, and can encode systemic inequalities.
- 2.1 Sociotechnical Harms: Anticipating harms requires examining technological affordances, differing groups’ uses and impacts, benefit and harm distributions, and existing social hierarchies.
- 2.2 Taxonomies of Harm, Risk, and Failure: Domain-specific taxonomies address settings such as online content, social media, online abuse, misinformation, and malicious uses.
- 2.2 Taxonomies of Harm, Risk, and Failure: Model-focused taxonomies organize harms around functions or models including large language models, image captioning, GPT-3, and BERT.
- 2.2 Taxonomies of Harm, Risk, and Failure: Failure-oriented taxonomies support auditing by focusing on faulty inputs or outputs, limited testing, proxy discrimination, surveillance capitalism, and problematic system behaviors.
- 2.2 Taxonomies of Harm, Risk, and Failure: Because sociotechnical harms retain social and technical elements, technical fixes alone cannot remedy them; social and cultural change is also required.
3 METHODOLOGY
The study maps computing research through a PRISMA-ScR-aligned scoping review and analyzes harm definitions using iterative, reflexive thematic analysis. The process combines broad source identification, data charting, clustering, and taxonomy refinement while acknowledging scholarly and conceptual limits.
- 3.1 Overview of Methodology: The review follows scoping-review practice and the PRISMA-ScR extension to map literature, clarify concepts, and accommodate diverse study designs.
- 3.1.2 Identify and select relevant studies.: Researchers searched scholarly databases, citation networks, targeted organizational and conference sources, and relevant NGO and professional outputs.
- 3.1.2 Identify and select relevant studies.: The initial ACM search found 85 articles after duplicate removal, while citation-based and targeted searches added 125 resources before exclusions.
- 3.1.3 Data charting.: Two researchers independently charted source characteristics and harm descriptions from full texts using a descriptive-analytical process.
- 3.1.3 Data charting.: Reflexive thematic analysis used iterative coding, discussion, condensation, and clustering of overlapping harm terms and contexts.
- 3.1.3 Data charting.: The taxonomy prioritized comprehensive yet manageable major categories for practitioners, while recognizing definitional variability across fields, contexts, technologies, and knowledge states.
- 3.2 Limitations: The review is bounded by academic and English-language scholarship, and therefore omits some community-based advocacy and perspectives.
- 3.2 Limitations: The taxonomy may support systematic analysis while also narrowing practitioners’ imagination and diverting attention from harms outside its structure.
4 TAXONOMY OF SOCIOTECHNICAL HARMS
The taxonomy organizes sociotechnical harms across five major types while recognizing that harms can overlap and occur concurrently within a system or use case.
- Five harm types cover representational, allocative, quality-of-service, interpersonal, and social system harms across micro-, meso-, and macro-level impacts.The categories address unjust social representations, resource distribution, performance disparities, interpersonal relations, and broader inequity or destabilization.
- Representational harms concern unjust social beliefs and hierarchies reflected in model inputs and outputs.
- Allocative harms concern how representations shape model decisions and distribute resources.
- Quality-of-service harms arise when optimization for imagined users produces performance disparities across groups.
- Interpersonal and social system harms concern adverse effects on relationships, communities, inequity, and social stability.
- Categories may contain gray areas, and multiple harms can occur in one system or use case, so the taxonomy is not prescriptive in ordering.
4.1 Representational Harms: Unjust Hierarchies in Technology Inputs and Outputs
Representational harms occur when algorithmic systems reflect or reinforce unjust beliefs and hierarchies about social groups. The taxonomy includes established subtypes and newly surfaced concerns about essentializing social categories.
- Representational harms reproduce unjust societal hierarchies by reinforcing subordination or unequal visibility along identity categories.Examples include disability, gender, race and ethnicity, religion, and sexuality.
- Stereotyping social groups: Stereotyping occurs when system outputs reflect beliefs about groups’ characteristics, attributes, behaviors, and their presumed relationships.Data, coding schemas, and design choices can convey explicit or implicit stereotypes.
- Demeaning social groups: Demeaning social groups casts them as lower-status and less deserving of respect.
- Erasing social groups: Erasure occurs when systems fail to recognize people, attributes, or artifacts associated with specific social groups, making them illegible to algorithmic systems.It represents an extreme of under-representation and can normalize dominant social ideas through design choices and training data.
- Alienating social groups: Alienating systems fail to acknowledge the relevance of someone’s social-group membership, diminishing dignity and recognition of group-specific injustices.
- Denying people the opportunity to self-identify: Automatic classification can deny autonomy when systems assign people to social categories without their knowledge or consent.One example is categorizing a non-binary person into a gendered category they do not belong to.
- Reifying essentialist social categories: Reifying essentialist categories treats socially constructed classifications as inherent, static, and natural.Such classifications can produce existential harm by portraying people reductively, especially through phenotype-based assumptions about gender, race, or sexual orientation.
4.2 Allocative Harms: Inequitable Distribution of Resources
Allocative harms arise when algorithmic decisions distribute information, opportunities, or resources unevenly across groups, particularly where distribution affects material well-being. The taxonomy distinguishes opportunity loss from direct economic loss.
- Allocative harms withhold information, opportunities, or resources from historically marginalized groups in domains affecting material well-being.
- Opportunity loss: Opportunity loss is disparate access to information and resources needed for equitable participation in society.Examples include housing withheld through race-targeted advertising and social services distributed along class lines.
- Economic loss: Economic loss refers to financial harms co-produced through algorithmic systems and linked to poverty and economic inequality.It is often entwined with opportunity loss and can reinforce feedback loops involving existing inequality.
4.3 Quality-of-Service Harms: Performance Disparities Based on Identity
Quality-of-service harms occur when algorithmic systems underperform disproportionately for groups defined by social categories of difference. These disparities can create alienation, increased labor, and degraded or lost benefits.
- Quality-of-service harms are performance disparities in which systems underperform disproportionately for certain identity groups.The corpus identified 10 different articulations of this harm type.
- Alienation: Alienation as a quality-of-service harm is self-estrangement during technology use when systems underperform for marginalized people or reinforce social alienation.Users may experience annoyance, disappointment, frustration, or anger when systems fail to recognize identity characteristics; algorithmic invisibility can also render marginalized topics unseen.
- Increased burden: Automatic speech recognition showed word error rates of 0.35 for Black speakers and 0.19 for white speakers.Related disparities have also been reported by sociolect, gender, age, and region.
- Increased burden: Increased burden requires some social groups to spend more time or effort making systems work as well for them as for others.
- Service or benefit loss: Service or benefit loss is the degraded or total loss of algorithmic-system benefits caused by inequitable performance based on identity.In consequential domains, degraded service can stigmatize users and lead to allocative harms.
4.4 Interpersonal Harms: Algorithmic Affordances Adversely Shape Relations
Interpersonal harms arise when algorithmic systems adversely shape relations among people, communities, and institutions, while also diminishing individual agency, well-being, dignity, and privacy.
- Scope of interpersonal harms: Interpersonal harms capture adverse effects of algorithmic systems on relations between people or communities, including relations mediated through institutions.These harms can emerge without direct person-to-person interaction because systems mediate institutional relationships.
- Agency loss: Agency loss occurs when algorithmic systems reduce autonomy, including through profiling, social sorting, and difficult-to-contest decisions.Insufficient ability to contest or remedy algorithmic decisions can amplify profiling-related agency loss.
- Technology-facilitated violence: Technology-facilitated violence includes harassment, stalking, sexual abuse, sextortion, coercive control, and non-consensual sexual imagery.Such violence can produce distress, fear, and humiliation while infringing dignity, privacy, bodily integrity, autonomy, and expression.
- Health and well-being: Algorithmic systems can diminish health and well-being through behavioral exploitation, emotional manipulation, safety failures, and incorrect health inferences.The resulting harms may be physical or emotional, including distress.
- Privacy violations: Privacy violations include unwanted disclosure or collection of private information, surveillance experiences, and data collection without explicit informed consent.These harms are also framed as data harms affecting the interests of people, entities, or society.
4.5 Societal Harms: System Destabilization and Exacerbating Inequalities
Societal harms are macro-level effects through which algorithmic systems reshape social systems, including knowledge, culture, politics, socioeconomic relations, and the environment.
- Societal harms: Societal harms concern adverse macro-level effects such as systematized bias, inequality, institutional exclusion, and accelerated scales of harm.They reflect the widespread, repetitive, or accumulative character of algorithmic systems and their effects on emergent properties of social systems.
- Knowledge systems: Algorithmic systems can produce information harms involving misinformation, disinformation, and malinformation within knowledge systems.Generative models and recommender systems are identified as sources of these information harms.
- Culture: Cultural harms include loss of communication means, loss of cultural property, harm to social values, and restrictions on alternative understandings or possible futures.These harms affect cultural stability and safety.
- Politics and civic life: Political harms arise when people are disenfranchised or deprived of political power and influence through algorithmic governance and surveillance.Identified effects include destabilized governance systems, eroded human rights, weapons of war, and disproportionate targeting of people of color.
- Socioeconomic systems: Algorithmic systems can increase socioeconomic power imbalances by exacerbating digital divides, systemic inequalities, labor exploitation, and technological unemployment.Examples include unethical data collection, worsening worker conditions, deskilling, and devaluing human labor.
- Environment: Environmental harms span depletion or contamination of resources and damage to built environments across digital technologies’ lifecycle.The burdens and benefits of extractivism are unevenly distributed between economic cores and peripheries.
5 DISCUSSION
The review synthesizes diverse computing research into a taxonomy intended to help practitioners and researchers anticipate sociotechnical harms more systematically. It also identifies methodological differences, shared concerns about inequality, and boundaries on how the taxonomy should be used.
- The review identifies five harm categories: representational, allocative, quality-of-service, interpersonal, and social system harms.
- 5.1 Synthesizing methodological distinctions in studying harms: Individual studies rarely capture the full scope of harms that an algorithmic system may produce because computing disciplines differ in what they study and how they frame harm.
- 5.2 Towards a shared harms vocabulary with flexible structure: Inequality is a recurring normative concern, including the possibility that algorithmic systems exacerbate or scale existing social inequalities.
- 5.2 Towards a shared harms vocabulary with flexible structure: The taxonomy provides a shared language that supports discussion at different levels of specificity, from broad policy guidance to granular evaluation work.
- 5.3 Navigating tensions between known and emergent harms: Known relationships between systems and harms can support systematic analysis of how harms extend across categories and become interdependent in practice.The review gives recommender systems and allocative harms, and identity-related failures requiring additional community labor, as examples.
- 5.4 Towards multidisciplinary proactive and reflexive harm anticipation: The taxonomy is an analytic starting point rather than a final or comprehensive list, and it does not provide normative guidance for identifying, evaluating, or controlling harms.It should be interpreted alongside specific contextual features and supplemented by existing assessment processes.
6 CONCLUSION
The paper offers an initial taxonomy of sociotechnical harms based on a scoping review and reflexive thematic analysis of computing research. It presents the taxonomy as a guide for addressing adverse impacts while recognizing that it should evolve with further research and engagement.
- The taxonomy is an initial guide for practitioners and researchers addressing a range of adverse impacts informed by algorithmic systems.
- The synthesis finds greater consensus and depth for some harms, including representational and allocative harms, while other possible harms remain under-articulated.
- The authors argue that richer understanding of harms can create more generative paths toward reducing their likelihood in algorithmic systems.