PaperlayerView exact revision on arXiv

Source-linked AI summary

SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models

Alok Abhishek, Tushar Bandopadhyay, Lisa Erickson

arXiv:2601.21235v1cs.CLcs.AI

TL;DR

Mean-centered evaluation can obscure multidimensional and worst-case social harm in high-stakes LLM deployments. SHARP addresses this gap with decomposed, distribution-aware risk profiling and finds substantial model-dependent differences in tail exposure across eleven frontier models.

  • Problem

    Mean-centered scalar benchmarks obscure distributional structure, cross-dimensional interactions, and worst-case behavior in social harm evaluation.

  • Method

    SHARP models harm as a multivariate risk variable, decomposes it into four dimensions, aggregates compounded failures as additive cumulative log-risk, and summarizes tails with CVaR.

  • Results

    Across eleven frontier LLMs evaluated on 901 socially sensitive prompts, similar average risks corresponded to substantially different tail-risk profiles and heterogeneous dimension-wise tail behavior.

  • Takeaways & Limitations

    Tail-sensitive, multidimensional profiling supports comparative evaluation and governance that scalar benchmarks cannot provide, including risk-sensitive model exclusion or tiering.

  • Takeaways & Limitations

    SHARP reports ordinal, rubric-dependent operational risk measures under a specified protocol, not calibrated real-world harm rates or deployment-ready safety guarantees.

Abstract

from arXiv · show

Large language models (LLMs) are increasingly deployed in high-stakes domains, where rare but severe failures can result in irreversible harm. However, prevailing evaluation benchmarks often reduce complex social risk to mean-centered scalar scores, thereby obscuring distributional structure, cross-dimensional interactions, and worst-case behavior. This paper introduces Social Harm Analysis via Risk Profiles (SHARP), a framework for multidimensional, distribution-aware evaluation of social harm. SHARP models harm as a multivariate random variable and integrates explicit decomposition into bias, fairness, ethics, and epistemic reliability with a union-of-failures aggregation reparameterized as additive cumulative log-risk. The framework further employs risk-sensitive distributional statistics, with Conditional Value at Risk (CVaR95) as a primary metric, to characterize worst-case model behavior. Application of SHARP to eleven frontier LLMs, evaluated on a fixed corpus of n=901 socially sensitive prompts, reveals that models with similar average risk can exhibit more than twofold differences in tail exposure and volatility. Across models, dimension-wise marginal tail behavior varies systematically across harm dimensions, with bias exhibiting the strongest tail severities, epistemic and fairness risks occupying intermediate regimes, and ethical misalignment consistently lower; together, these patterns reveal heterogeneous, model-dependent failure structures that scalar benchmarks conflate. These findings indicate that responsible evaluation and governance of LLMs require moving beyond scalar averages toward multidimensional, tail-sensitive risk profiling.

1 Introduction

SHARP frames social harm in LLMs as multidimensional, distributional, and concentrated in consequential tail failures, addressing limits of mean-centered evaluation with risk-sensitive profiling.

  • High-stakes deployment makes rare LLM failures capable of producing durable or irreversible harm across healthcare, finance, hiring, welfare, and criminal justice.
  • Mean-centered benchmarks collapse heterogeneous harm into point estimates and can miss sharply divergent upper-tail behavior among models with similar averages.
  • Social harm varies across prompts and interacting mechanisms, including bias, fairness violations, ethical misalignment, and epistemic unreliability.
  • SHARP decomposes harm into four dimensions, preserves prompt-level variability, and aggregates compounded failures through additive cumulative log-risk with CVaR95 tail statistics.
  • Across eleven frontier LLMs and 901 socially sensitive prompts, similar average risks coincided with markedly different tail exposure and volatility.
  • SHARP provides comparative tail-sensitive risk profiling under explicit measurement assumptions rather than real-world harm rates or population-level fairness guarantees.

2 Methodology

SHARP evaluates prompt-level social harm as a four-dimensional risk vector, compounds dimensions through an operational union-of-failures structure, and profiles the resulting risk distribution with tail-sensitive statistics.

  • The protocol evaluates fixed-corpus responses using three blinded LLM-based judges whose structured assessments produce normalized harm estimates.
  • The evaluation covers 11 frontier LLMs on 901 socially sensitive prompts derived from BEATS and including demographic, contextual, and intersectional scenarios.
  • SHARP decomposes harm into bias, fairness, ethics, and epistemic reliability, constructing each sub-index from aggregated judge outputs.
  • Judge disagreement is aggregated with log-sum-exp, which interpolates between mean and max pooling; τ = 0.20 emphasizes severe assessments while retaining differentiability.
  • Each prompt response is represented by a four-dimensional harm vector whose coordinates are normalized harm intensities, not calibrated real-world harm probabilities.
  • The monotone compounding operator increases with elevated harm in any dimension and amplifies coordinated activation across dimensions, while remaining an operational risk measure.
  • Negative-log reparameterization converts bounded harm intensities into additive, decomposable cumulative log-risk and uses ε = 10^-6 to avoid saturation near one.
  • SHARP summarizes each model’s empirical prompt-risk distribution with expected log-risk, volatility, and CVaR0.95, its primary statistic for worst-case compounded failures.

3 Empirical Evaluation

SHARP’s empirical evaluation shows that tail-sensitive, multidimensional risk profiles distinguish models that scalar averages conflate. Across statistical tests and robustness checks, worst-case compounded harm is model-dependent, dimensionally heterogeneous, and suitable for risk-constrained rather than total-ordering decisions.

  • Model-Level Social Harm Risk Profiles: 1.69 versus 5.73 and 5.99: Claude Sonnet 4.5, DeepSeek-Chat, and Mistral-Large show more than threefold separation in CVaR95 of cumulative log-risk.LLaMA-3 405B reaches 8.40, exceeding the lowest-risk model by over fourfold, while mean log-risk spans a narrower range.
  • Model-Level Social Harm Risk Profiles: 0.46 versus 0.45 mean log-risk still corresponds to 3.50 versus 3.78 CVaR95, showing that mean risk understates tail exposure.The passage states that this pattern recurs across the model set and that the profiles are conditional on observed outputs.
  • Model-Level Social Harm Risk Profiles: Most models exceed 0.96 CVaR95 in any-harm probability, with several approaching 0.99, limiting probability metrics’ discriminative power under compounding.Cumulative log-risk remains more spread and stable across models in the tail.
  • Model-Level Social Harm Risk Profiles: Similar harm-radius CVaR95 profiles can diverge substantially in cumulative log-risk because coordinated elevation across dimensions amplifies compounded risk.Magnitude metrics capture overall severity but not interaction-driven amplification from joint activation.
  • Risk-Constrained Model Selection: CVaR0.95(L) supports filtering, exclusion, and tiered comparisons under explicit risk tolerance rather than fragile total rankings.Higher-risk tiers can be excluded, while models in the same tier may be treated as interchangeable absent additional domain constraints.
  • Statistical Validation: 80% of model pairs are statistically separable at the 95% level, while ambiguity concentrates among near-neighbor models.The resulting interpretation is a tiered risk structure rather than a fragile total ordering.

4 Limitations

SHARP’s conclusions are constrained to comparative, distributional risk patterns under specified measurement assumptions, rather than calibrated real-world harm or deployment guarantees. Interpretation is further limited by judge dependence, response behavior, single-turn evaluation, and the evaluated prompt distribution.

  • SHARP’s harm indices and cumulative log-risk are operational measures, not calibrated probabilities or normative ground-truth labels.Conclusions concern relative behavior, distributional structure, and tail exposure under a specified measurement configuration.
  • LLM-based judges introduce rubric, ensemble, prompt-sensitivity, and calibration limitations, with no claim of judge neutrality or external human calibration.Inter-judge disagreement is highest for epistemic harm, and scores remain conditional on the chosen judge ensemble.
  • Low measured risk can reflect refusal, deflection, or minimal responses because SHARP does not model utility, coverage, or task completion.Therefore, low estimated risk alone does not establish safe and useful performance.
  • SHARP evaluates single-turn intrinsic behavior and does not model population-level fairness guarantees, downstream decision effects, or sociotechnical feedback loops.Its fairness dimension captures inequitable treatment cues within responses rather than statistical fairness across individuals or outcomes.
  • Risk profiles are conditional on a primarily English-language, Western-centric prompt distribution and single-sample decoding, leaving distribution shift and generation stochasticity unaddressed.Bootstrap uncertainty covers sampling variability, but not these broader sources of variation.
  • SHARP is a reproducible comparative profiling protocol, not a calibrated estimate of real-world harm or a substitute for context-specific deployment validation.Its results should be interpreted within the protocol’s explicit measurement assumptions.

5 Conclusion

SHARP frames social harm as multidimensional and distributional, addressing the limits of mean-centered scalar evaluation with risk-sensitive profiling. Across eleven frontier LLMs, it finds materially different tail-risk structures among models with similar mean risk, supporting evaluation that exposes worst-case behavior.

  • Contributions: SHARP decomposes harm into bias, fairness, ethics, and epistemic reliability, while additive cumulative log-risk and CVaR0.95 expose compounded tail exposure.This design targets failure modes and worst-case behavior that scalar aggregates conflate.
  • Empirical findings: Models with similar mean risk can exhibit substantially different tail-risk profiles, including elevated CVaR alongside low expected risk.These differences reveal susceptibility to rare but severe failures invisible to mean-centered summaries.
  • Empirical findings: Dimension-wise decomposition reveals heterogeneous marginal tail behavior, with no single harm dimension uniformly driving outcomes across models.Statistical validation attributes these differences to systematic model effects while also showing strong prompt dependence in extreme-risk regimes.
  • Implications for evaluation and governance: Risk-sensitive thresholds support excluding or tiering models based on unacceptable worst-case exposure regardless of mean performance.Dimensional decomposition also supports targeted mitigation and comparative governance analysis.
  • Limitations and outlook: SHARP’s conclusions remain bounded by intrinsic single-turn evaluation, a fixed prompt distribution, specified judges, and uncalibrated harm measures.The corpus is primarily English-language and Western in context, and low measured risk may reflect refusal or deflection without utility constraints.
  • Conclusion: Meaningful LLM risk assessment requires multidimensional, distribution-aware profiling of worst-case behavior beyond scalar averages.SHARP provides a reproducible foundation for comparative tail-sensitive evaluation under explicit measurement assumptions.

Broader Impact

SHARP frames social-harm measurement as a multidimensional, distribution-aware task, while exposing explicit normalization, weighting, and aggregation choices. Its broader impact is methodological: supporting more risk-aware comparison and governance without claiming population-level guarantees or calibrated societal-impact estimates.

  • Broader impact: SHARP treats social harm as a multidimensional, distributional object rather than a scalar quantity.The framework represents harm across bias, fairness, ethics, and epistemic reliability, preserving prompt-level variability and using risk-sensitive summaries.
  • Practical boundary: SHARP may support risk-aware model selection, auditing, benchmarking, and governance, but it is not intended to estimate real-world harm rates or provide population-level fairness guarantees.The paper cautions against ranking or deploying models solely from intrinsic harm profiles without considering utility, context, and stakeholders.
  • Measurement design: SHARP normalizes heterogeneous judge outputs to [0, 1], with larger values uniformly indicating greater estimated harm.Boolean, categorical, and ordinal outputs are made commensurable while preserving ordinality.
  • Normative choices: The framework uses fixed, interpretable coefficients as explicit governance parameters rather than statistically optimized or uniquely correct weights.These parameters can reflect institutional priorities, regulatory thresholds, or domain-specific risk tolerances.
  • Fairness scope: Fairness is measured as inequitable treatment cues within a single response, not as demographic parity, equalized odds, or population-level statistical fairness.The fairness sub-index uses response content without requiring counterfactual generation or across-prompt aggregation.

A.9 Methodological contributions (technical view)

SHARP contributes a technical architecture that decomposes harm, emphasizes elevated judge assessments, aggregates compounded failures additively, and summarizes model-level risk distributions. Its agreement diagnostics show dimension-dependent disagreement alongside stable tail-risk rankings under judge ablation.

  • Methodological contributions: SHARP decomposes harm into bias, fairness, ethics, and epistemic dimensions for targeted diagnosis and transparent reporting.The framework also uses risk-sensitive aggregation and cumulative log-risk for compounded multidimensional harm.
  • Evaluation architecture: The evaluation uses three blinded LLM judges, normalizes their structured outputs, and aggregates prompt-level assessments into harm dimensions.Results use 901 prompts across 11 models, with log-sum-exp aggregation at temperature τ = 0.2 and CVaR0.95 for tail-risk summaries.
  • Judge agreement: Epistemic assessments show the largest average inter-judge dispersion, followed by fairness and bias, while ethics shows the lowest dispersion.Mean absolute deviation is aggregated over 9,911 prompt-model pairs per dimension; lower MAD indicates tighter agreement.
  • Judge robustness: Removing a single judge leaves CVaR0.95 model rankings highly stable under the specified aggregation protocol.Removing the OpenAI judge yields identical ordering with Kendall’s τ = 1.0.
  • Prompt-level concordance: Judges exhibit moderate but imperfect prompt-level concordance, with systematic variation across judge pairs and harm dimensions.Several dimensions show higher average concordance for the Claude-Gemini pair than for pairs involving GPT-based judges.

C Judge Ensemble Sensitivity and Overlap Considerations

SHARP’s dependence analysis tests whether its union-of-failures aggregation is interpretable despite correlations among harm dimensions. Dependence is generally lower in the highest-risk prompts, supporting an approximate-independence interpretation of tail aggregation while leaving judge-overlap concerns unresolved.

  • Overlap considerations: Judge-model overlap remains a known concern because shared training distributions or architectural lineage can affect LLM-as-a-judge evaluations.SHARP accommodates alternative judge configurations, but reported scores and rankings remain conditional on the chosen ensemble.
  • Future robustness: The study leaves a comprehensive cross-judge sensitivity analysis to future benchmark instantiations with larger and more diverse judge pools.A fully disjoint ensemble could quantify overlap-induced shifts in absolute harm levels and relative rankings.
  • Empirical dependence: Harm dimensions are generally moderately to highly correlated across all prompts, but correlations consistently decrease in the top 5% risk tail.Several models show near-zero mean tail correlations, indicating heterogeneous and asymmetric activation patterns among extreme-risk prompts.
  • Aggregation interpretation: The reduced tail dependence supports interpreting multiplicative aggregation as an operational risk construction under approximate independence, not strict global independence.The authors argue that worst-case harm is frequently driven by dimension-specific failures rather than fully coupled effects.

E LSE Temperature Sensitivity Analysis

The temperature sensitivity analysis separates rank robustness from changes in absolute tail-risk magnitudes. Model ordering, including the safest and riskiest systems, remains stable across the tested LSE temperatures, while CVaR values vary modestly.

  • Interpretation: The results distinguish invariant comparative conclusions from mild, structured variation in absolute CVaR magnitudes.This behavior reflects LSE interpolation between more pessimistic and more averaging aggregation regimes.
  • Rank stability: τ = 0.15, 0.20, and 0.25 produce identical model rankings under CVaR0.95(L).Both Kendall’s τb and Spearman’s ρ relative to τ = 0.20 equal 1.0 at the two alternative temperatures.
  • Extreme rankings: The safest and riskiest model identities and ordering remain unchanged across the tested temperatures.The analysis treats stability at the head and tail of the ranking as safety-relevant for risk profiling.
  • Cardinal sensitivity: Absolute CVaR0.95(L) values change modestly with temperature, with the largest observed spread equal to 0.0915 for llama3-1-405b.The smallest observed spread is 0.0208 for claude-sonnet-4-5-20250929.

F Aggregation Robustness and Comparison to Simpler Risk Aggregates

SHARP’s tail-risk aggregation is broadly consistent with simpler aggregates, yet it can reveal practically important differences that mean-based summaries miss.

  • Baseline aggregates: Post hoc comparisons evaluate CVaR0.95(L), CVaR0.95 of harm radius, CVaR0.95 of the maximum sub-index, and mean any-harm probability over the same prompt distribution.The comparison uses materialized prompt-level metrics without rerunning model inference.
  • Rank consistency across aggregates: CVaR0.95(L) is highly correlated with other tail-sensitive aggregates, with Spearman ρ ≥0.95 in all reported comparisons.Correlation with mean any-harm probability is also strong.
  • Concrete decision flip: A concrete decision flip occurs when two models have nearly identical mean L but differ by approximately 0.45 in CVaR0.95(L).The different tail-risk values produce a different model ordering.
  • Implications: The decision flip shows that mean-only aggregation can mask rare but severe failures that tail-sensitive evaluation is designed to surface.The findings support additional discriminative power from CVaR beyond mean-based or max-based summaries.

G Supplementary Analysis for Harm Decomposition

Supplementary analyses assess judge reliability, dependence, uncertainty, separability, model differences, and variance sources in SHARP’s risk profiles.

  • Judge reliability and sensitivity: Model-level CVaR rankings remain stable despite moderate, dimension-dependent judge disagreement, which is highest for epistemic assessments.The result indicates that comparative findings are not artifacts of a particular judge configuration.
  • Dependence among harm dimensions: Prompt-level dependence weakens substantially within the top 5% cumulative-log-risk tail, supporting multiplicative aggregation in that regime.Moderate correlations are observed across all prompts.
  • Uncertainty and design: The analyses use a repeated-measures design with all 11 models evaluated on the same 901 prompts and a paired bootstrap over prompts.The bootstrap uses 10,000 resamples with replacement and preserves prompt pairing across models.
  • Primary tail metric: CVaR0.95(L) is defined as the conditional expectation of L over each model’s worst 5% of prompts.Mean log-risk E[L] is additionally reported as a central-tendency diagnostic.
  • Tail-threshold sensitivity: Kendall’s τb = 0.9636 and Spearman’s ρ = 0.9909 for both α = 0.90 versus 0.95 and α = 0.975 versus 0.95.Model rankings therefore remain highly stable across the tested CVaR thresholds.
  • Pairwise separability: 44 of 55 model pairs (80.0%) are separable under paired-bootstrap 95% confidence intervals for CVaR0.95 differences.The remaining ambiguous pairs are statistically near-neighbors under this criterion.
  • Model-level differences: The Friedman test rejects identical prompt-level risk distributions, with χ2(10) = 1629.91 and p ≈0.Kendall’s W = 0.1809 indicates non-trivial but not dominant between-model separation relative to prompt heterogeneity.
  • Post-hoc paired tests: Holm-corrected Wilcoxon tests find 48 of 55 pairs (87.3%) significant, while non-significant pairs concentrate among near-neighbor models.This supports interpreting the results as risk tiers rather than a strict total ordering.

H.4 LSE temperature robustness

SHARP’s tail-risk rankings are robust to tested LSE temperatures, while distributional visualizations support comparisons of prompt-level harm shapes and tail mass.

  • LSE temperature robustness: Across τ ∈ {0.15, 0.20, 0.25}, CVaR0.95(L) rankings are invariant relative to τ = 0.20.Both Kendall τb and Spearman ρ equal 1.0, although absolute CVaR magnitudes vary mildly.
  • Distributional visualizations: Violin plots emphasize distributional shape and tail mass, whereas box plots emphasize median and IQR for cross-model comparison.These visualizations are intended for heavy-tailed behavior.
  • Sub-index panels: The displayed sub-index panels include bias and fairness measures across evaluated models, including Claude, Gemini, DeepSeek, Mistral, and Llama variants.The supplied labels identify model series and the bias and fairness sub-index panels.
Loading 2601.21235v1…