PaperlayerView exact revision on arXiv

Source-linked AI summary

Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms

Shalaleh Rismani, Renee Shelby, Leah Davis, Negar Rostamzadeh, AJung Moon

arXiv:2510.10339v1cs.HCcs.AIcs.LG

TL;DR

Existing AI ethics evaluation is fragmented and often lacks systems-level attention to components, interactions, hazards, harms, and meaningful thresholds. A scoping review and reflexive analysis identified measures, mapped them to system components, attributes, hazards, and harms, and compiled the results into a dataset and interactive visualization. Approximately 90% of measures address fairness, transparency, privacy, or trust, while most assess model or output components and provide limited links to harms or thresholds.

  • Problem

    Existing AI ethics evaluation is fragmented and often lacks systems-level attention to components, interactions, hazards, harms, and meaningful thresholds.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    A scoping review and reflexive analysis identified measures, mapped them to system components, attributes, hazards, and harms, and compiled the results into a dataset and interactive visualization.

    Source: §3 · p. 3, Data charting and summarization · p. 4

  • Results

    Approximately 90% of measures address fairness, transparency, privacy, or trust, while most assess model or output components and provide limited links to harms or thresholds.

    Source: §1 · p. 2, §4.1 · p. 5

  • Takeaways & Limitations

    Framing measures through system attributes, hazards, and harms supports more comprehensive, integrated, and systems-aware evaluation practices.

    Source: §1 · p. 2, §6 · p. 10

    The review captures a snapshot of a rapidly evolving field, and its selection, extraction, and categorization reflect the team’s interpretive lens.

    Source: §3.4 · pp. 4–5

Abstract

from arXiv · show

Over the past decade, an ecosystem of measures has emerged to evaluate the social and ethical implications of AI systems, largely shaped by high-level ethics principles. These measures are developed and used in fragmented ways, without adequate attention to how they are situated in AI systems. In this paper, we examine how existing measures used in the computing literature map to AI system components, attributes, hazards, and harms. Our analysis draws on a scoping review resulting in nearly 800 measures corresponding to 11 AI ethics principles. We find that most measures focus on four principles - fairness, transparency, privacy, and trust - and primarily assess model or output system components. Few measures account for interactions across system elements, and only a narrow set of hazards is typically considered for each harm type. Many measures are disconnected from where harm is experienced and lack guidance for setting meaningful thresholds. These patterns reveal how current evaluation practices remain fragmented, measuring in pieces rather than capturing how harms emerge across systems. Framing measures with respect to system attributes, hazards, and harms can strengthen regulatory oversight, support actionable practices in industry, and ground future research in systems-level understanding.

1 Introduction

The paper asks how AI ethics measures relate to system components and algorithmic harms, arguing that fragmented component-level evaluation is insufficient for sociotechnical systems.

1 Introduction

  • Existing measures can lack construct validity and reliability, risking misleading or incomplete assessments of system attributes and performance.The paper identifies this as a measurement-theory concern across current AI ethics evaluation practices.Source: §1 · p. 1
  • Most measures assess isolated models or datasets rather than system-wide properties or interactions, although harms can emerge from otherwise intended component interactions.This motivates a systems-level perspective on safety and harm evaluation.Source: §1 · p. 1
  • The review examines how AI ethics measures map to system components, attributes, hazards, and harms through two research questions.It focuses on measures of adherence to AI ethics principles and their signals about hazards and harms.Source: §1 · p. 1
  • The scoping review extracted 791 measures from 257 computing articles and analyzed their relationships to principles, components, attributes, hazards, and harms.The analysis uses a multi-dimensional mapping of the existing measurement landscape.Source: §1 · pp. 1–2
  • Fairness, transparency, privacy, and trust dominate the measures, while other principles are underrepresented and links to experienced harms and actionable thresholds remain limited.The authors therefore call for more temporally responsive, systems-level evaluation approaches.Source: §1 · p. 2

2 Background

Background work identifies validity, reliability, component isolation, and fragmented practice as limitations of responsible-AI evaluation. A system-safety lens instead connects measures to attributes, hazards, harms, and feedback-driven intervention.

2.1 RAI Evaluation Practices and Challenges

  • Quantitative responsible-AI measures operationalize ethical principles for risk assessment, audits, and trade-offs beyond conventional performance evaluation.Examples include fairness, trust, privacy, and emissions measures alongside qualitative impact and harm assessments.Source: §2.1 · p. 2
  • Existing measures may fail to capture the constructs they target and often focus on isolated outputs rather than harms across components and deployment contexts.This narrow scope can leave a gap between model behavior and downstream human impact.Source: §2.1 · p. 2

2.2 A System Safety Approach to Evaluation

  • System safety defines a relational chain in which component attributes are measured against criteria, deviations create hazards, and hazardous states can produce sociotechnical harm.Harm is framed as adverse lived experience resulting from system deployment and operation.Source: §2.2 · p. 3
  • Measures function as feedback signals about dataset, model, interaction, and perception conditions, enabling stakeholders to intervene as system elements drift toward unacceptable states.Different signals support data refinement, design choices, and improvements to user-facing behavior and outcomes.Source: §2.2 · p. 3
  • The paper applies this lens to assess whether existing measures can anticipate and mitigate harm across the sociotechnical system and its context.This shifts evaluation from what is measured alone toward how measurement supports harm prevention and response.Source: §2.2 · p. 3

3 Methodology

The study combines a scoping review, reflexive mapping, and a public dataset and visualization to analyze AI ethics measures in computing literature.

3 Methodology

  • The methodology comprises a scoping review, reflexive analysis mapping measures to attributes, hazards, and harms, and compilation into a dataset and interactive visualization.These three steps structure the paper’s systems-level analysis.Source: §3 · p. 3, Figure 3 · p. 4

3.1 Scoping Review: Identifying the Measures

  • The review targets measures associated with eleven AI ethics principles using a five-stage scoping-review process managed in Covidence.The principles follow Jobin et al. (2019), selected as a comprehensive synthesis of ethical AI principles.Source: §3.1 · p. 3

Identify relevant studies

  • Searches across database, citation-based, and expert-consultation phases identified studies published mainly from 2017 to August 2023, with the collection concluding in February 2024.The initial search produced over 8,000 results, and 22 refined queries yielded 1,677 papers for screening.Source: Identify relevant studies · p. 3

Study selection

  • After duplicate removal and screening, 257 papers were included for data extraction from the 1,618 papers entering title and abstract screening.Four researchers screened papers using predefined inclusion and exclusion criteria, with conflicts resolved by the lead author.Source: Study selection · pp. 3–4

Data charting and summarization

  • Researchers extracted 791 individual measures and recorded their principles, names, processes, assessment types, system components, application areas, and publication metadata.Measures were mapped to data/input, model, output, user-output interaction, or full-system components.Source: Data charting and summarization · p. 4

3.2 Reflexive Analysis: Identifying the Criterion, Attribute, Hazard, and Harm

The reflexive analysis identifies explicit criteria and maps each measure to attributes, hazards, and sociotechnical harms using inductive coding and an established harm taxonomy.

3.2 Reflexive Analysis: Identifying the Criterion, Attribute, Hazard, and Harm

  • For each measure, the researchers examined its measurement process and source context, revisiting papers to extract additional information when needed.They also recorded whether authors supplied an explicit criterion for the measure.Source: §3.2 · p. 4
  • Measures were linked from criteria to attributes, hazards, and primary and secondary harms, such as output distribution, disparate output, and loan denial.The mapping preserves the paper’s relational account of how measurements connect to harmful outcomes.Source: §3.2 · p. 4
  • Criteria, attributes, and hazards received inductive codes, while harms were categorized using five sociotechnical harm types: allocative, representational, quality of service, interpersonal, and social system harms.Two researchers independently coded the first 100 measures before collaborative coding and iterative team review.Source: §3.2 · p. 4

3.3 Dataset Compilation and Visualization

The study compiled a structured dataset of 791 measures with attributes, hazards, harms, and publication metadata, then created an interactive visualization. Its scoping review is limited by field evolution, incomplete principle framing, interpretive subjectivity, and reliance on one harm taxonomy.

3.3 Dataset Compilation and Visualization

  • The final dataset contains 18 columns covering all 791 measures, their measurement processes, system components, assessment factors, harms, and publication metadata.An interactive Sunburst diagram supports exploration of the categorized dataset.Source: §3.3 · p. 4

3.4 Limitations of Our Study

  • The review captures a snapshot of a rapidly evolving field, so relevant measures may be unpublished or available only as preprints.Source: §3.4 · p. 4
  • The analysis is shaped by the team’s interpretive lens, introducing subjectivity in measure selection, extraction, and categorization.Source: §3.4 · p. 5
  • The findings use one harm taxonomy, although alternative frameworks could provide complementary perspectives.Source: §3.4 · p. 5

4 Findings

The findings section first surveys how 791 measures map onto ethics principles, system components, and sociotechnical harms, then examines attributes and hazards before synthesizing cross-cutting themes.

4 Findings

  • The section proceeds from an overview of measure distributions to mappings between harms, attributes, and hazards, ending with themes from reflexive analysis.Source: §4 · p. 5

4.1 Overview of the Measures

The 791 measures are concentrated in four ethics principles and primarily assess models and outputs, while harm coverage varies substantially across principles. Cross-listing means aggregate percentages can exceed 100%.

4.1 Overview of the Measures

  • Approximately 90% (n = 717) of measures address fairness (46.4%, n = 367), transparency (21.1%, n = 167), privacy (14.2%, n = 112), or trust (10.6%, n = 84).The remaining seven principles account for approximately 10% (n = 79), and 2.9% (n = 23) are coded under multiple principles.Source: §4.1 · p. 5, Figure 4 · p. 5
  • Across all measures, 57.3% (n = 453) target the model and 42.6% (n = 337) target the output, compared with 16.2% (n = 128) for data/input and 25.4% (n = 201) for user-output interaction.Component distributions differ by principle: fairness emphasizes model/output, privacy data/input, and trust user-output interaction.Source: §4.1 · p. 5
  • Low-prevalence principles mostly concern user-output interaction (62.0%, n = 49), while sustainability measures primarily assess the full system (88.2%, n = 15).All solidarity, dignity, and freedom-and-autonomy measures concern user-output interaction.Source: §4.1 · p. 5
  • Measures span all five sociotechnical harm types, led by interpersonal (29.2%, n = 231), social system (25.9%, n = 205), and allocative harms (25.0%, n = 198).Representational harm accounts for 13.1% (n = 104), and quality-of-service harm for 6.7% (n = 53).Source: §4.1 · p. 5, Figure 5 · p. 6
  • High-prevalence principles concentrate on particular harms: 95.6% (n = 351) of fairness measures map to representational, allocative, or quality-of-service harms, while 83.2% (n = 139) of transparency measures map to social-system harms.Privacy measures are linked to interpersonal harms at 95.5% (n = 107), and trust measures to interpersonal or social-system harms at 96.4% (n = 81).Source: §4.1 · pp. 5–6

4.2 Attributes, Hazards, and Harms Signaled

The analysis maps five harm types to system components, measurable attributes, and hazards, revealing recurring concentration on model and output measures and narrower coverage of interaction and full-system effects.

Representational harm (104 measures)

  • Representational-harm measures mainly assess model (79.8%) and output (71.2%) attributes, covering identity representation, social-hierarchy reinforcement, and alignment with user identity.Hazards include imbalanced training representation, stereotypical embeddings, omitted groups, unequal benefits or errors, and identity-misaligned outcomes.Source: Representational harm (104 measures) · p. 6

Allocative harm (198 measures)

  • Allocative-harm measures primarily assess model (83.8%) and output (69.7%) attributes through disparate outcomes, performance, perceived decision fairness, and design-level disparities.These measures signal unequal benefits or errors, unequal access, perceived bias or unintelligibility, and disparities arising from data or modeling choices.Source: Allocative harm (198 measures) · p. 7

Quality of service harm (53 measures)

  • Quality-of-service measures assess model or output components in 90.6% of cases, focusing on group performance disparities and user-experience quality.The associated hazards are uneven reliability or accuracy and outputs that are less useful or satisfactory for certain groups.Source: Quality of service harm (53 measures) · p. 7

Interpersonal harm (231 measures)

  • Interpersonal-harm measures span user-output interaction (49.4%), data/input (31.2%), and model (22.5%) components, covering privacy, influence, autonomy, and well-being.Hazards include privacy attacks, undue influence, disempowerment, harmful content, distorted risk perceptions, and diminished dignity or respect.Source: Interpersonal harm (231 measures) · pp. 7–8

Social system harm (205 measures)

  • Social-system measures cover information loss and environmental harms across components, with model (52.2%), output (31.2%), and user-output interaction (33.2%) most represented.Their hazards include unintelligible or misleading explanations and excessive resource use or emissions.Source: Social system harm (205 measures) · p. 8

4.3 Key Themes and Takeaways

The review identifies fragmented evaluation practices: measures often isolate components, remain disconnected from lived harms, lack actionable thresholds, overlook temporal accumulation, and leave perception-based signals difficult to interpret.

Measures are fragmented across system components

  • Measures mainly assess isolated models or outputs, while few evaluate cross-component dependencies or the system holistically.Examples that span components or integrate attributes across development stages are uncommon, limiting systems-level understanding of unsafe outcomes.Source: Measures are fragmented across system… · pp. 8–9

Measurements are taken far from where harm is experienced

  • No reviewed component-level measure empirically traced a measurement to downstream harm experienced by users, and many measures assess hazards rather than lived harm.Hazard indicators such as offensive language or privacy-leak risk can identify potential problems without measuring users’ experiences of violation.Source: Measurements are taken far from where… · p. 9

Measures lack clear criteria for identifying hazards

  • Only 26.9% (n = 213) of measures had explicit interpretation criteria, and few of those criteria discussed appropriate thresholds.Thresholds vary with application domains, user populations, and risk tolerances, making context-sensitive guidance important for identifying unacceptable behavior.Source: Measures lack clear criteria for… · p. 9
  • Criteria such as parity, sparse explanations, and stable explanations are often under-specified, leaving practitioners to decide what counts as acceptable behavior.The review argues that evaluations should disclose thresholds distinguishing tolerable variation from hazard-signaling conditions.Source: Measures lack clear criteria for… · p. 9

Measures overlook how harms accumulate over time

  • Measures are commonly taken at one point in time even though sociotechnical harms may emerge gradually, and no reviewed paper explained long-term monitoring.Environmental measures tracking energy or carbon across training and deployment were among the few exceptions.Source: Measures overlook how harms accumulate… · p. 9

The role of perception-based measures in signaling harms remains unclear

  • Perception-based measures comprised 17.3% (n = 137) of the corpus, but their links to specific hazards and harms remained difficult to determine.Ignoring user perceptions may overlook experienced harm, while optimizing solely for subjective perceptions is inappropriate for designing safe AI.Source: The role of perception-based measures… · p. 9

5 Discussion

The discussion argues that AI governance, industry practice, and research need more explicit, structured connections between measures, system components, hazards, and harms. It also identifies general-purpose AI and distributed development as areas requiring more adaptive systems-level evaluation.

5.1 Implications for Governance and Policy

  • Governance frameworks need broader principle and hazard coverage, clearer criteria, and explicit links between what is measured, why it matters, and potential harms.The dataset and visualization are offered as resources for making evaluation more traceable and accountable.Source: §5.1 · p. 10

5.2 Implications for Industry and Practice

  • Practitioners can use the mapping and visualization to select measures deliberately, prioritize resource-intensive evaluations, and justify trade-offs across organizational roles.Making explicit what a measure captures and omits can support more transparent and defensible evaluation choices.Source: §5.2 · p. 10

5.3 Implications for Researchers

  • Researchers should report which component and attribute a measure assesses and how it relates to a specific hazard or harm.The discussion also calls for interdisciplinary collaboration to address narrow component and hazard coverage and better integrate user experiences.Source: §5.3 · p. 10

5.4 Future Work

  • General-purpose AI systems require more adaptive, multi-layered evaluation because their use cases are broader and less predictable.Future work should also examine responsibility across organizations, expand the component framework, and test visualization use across organizational contexts.Source: §5.4 · p. 10

6 Conclusion

The paper provides a systems-level mapping of AI ethics measures to principles, components, and harms, finding persistent gaps in measuring cross-component dynamics and connecting measurements to real-world harm. Its dataset and visualization support more grounded, transparent, and context-sensitive evaluation.

6 Conclusion

  • Current practices lack measures that capture cross-component dynamics and remain separated from where real-world harms occur.The analysis, dataset, and visualization provide a foundation for more comprehensive evaluation infrastructures.Source: §6 · p. 10

Positionality Statement

The authors’ disciplinary and geographic backgrounds shaped their interpretation of AI measurement practices. They acknowledge subjectivity in selecting, extracting, and framing the reviewed material despite using quality checks.

Positionality Statement

  • The analysis reflects expertise spanning computer science, software engineering, HCI, robotics, sociology, and STS across industry and academia.The authors acknowledge that this positioning influenced measurement interpretation and analysis.Source: Positionality Statement · p. 11
  • The authors identify subjectivity in data selection, extraction, and analytical framing despite multiple quality checks.They state that interpretations were intended to remain faithful to the original sources while resolving inconsistencies and gaps.Source: Positionality Statement · p. 11
Loading 2510.10339v1…