PaperlayerView exact revision on arXiv

Source-linked AI summary

The Anatomy of Socio-Demographics in Pluralistic Offensiveness Modeling: When and Where It Helps, and the Role of Reasoning

Yifan Wang, Leixin Zhang, Jinyi Mu, Yu Wang, Mayank Jobanputra, Isabel Valera, Vera Demberg

arXiv:2610.37896v1cs.AI

TL;DR

Prior findings on demographic information in pluralistic offensiveness modeling are mixed, leaving its effectiveness and mechanisms uncertain. Across three datasets, the paper evaluates aggregation levels and decomposes predictions into baseline and demographic shift. Gains are clearer after aggregation, and explicit reasoning improves component estimates and predictions compared with direct conditioning.

  • Problem

    Prior studies report mixed evidence about whether socio-demographic information benefits pluralistic modeling, motivating systematic analysis of when it helps and how it affects predictions.

    Source: §1 · pp. 1–2

  • Method

    Across three multi-annotator datasets with prompted and trained LLMs, the paper compares evaluation levels and demographic conditioning with explicit reasoning over baseline and demographic-specific shift.

    Source: §1 · p. 2

  • Results

    Demographic gains are limited for individual predictions but clearer at group and population levels, while explicit reasoning yields more accurate baseline and shift estimates than conditioning alone.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Limited individual-level gains should not be interpreted as evidence that socio-demographic information is uninformative; evaluation across aggregation levels can reveal systematic patterns.

    Source: §5 · p. 10

    The observed deviation is a noisy proxy for the demographic-specific deviation, inheriting individual residuals and capturing only effects large enough to change the final discrete prediction.

    Source: §4.3 · p. 8

Abstract

from arXiv · show

Subjective judgments such as offensiveness often vary substantially across individuals, motivating pluralistic modeling approaches that incorporate annotators' socio-demographic information to capture this variation. However, prior findings on the benefits of socio-demographic information for pluralistic modeling are mixed, calling for a systematic analysis of both its effectiveness and the mechanisms through which it improves prediction. In this work, we study $\textbf{when}$ and $\textbf{where}$ socio-demographics benefit pluralistic offensiveness modeling, as well as $\textbf{the role of explicit reasoning}$ in leveraging this information. Across three multi-annotator datasets with prompted and trained LLMs, we first show that demographic gains are small and inconsistent for individual predictions but become clearer at the group and population levels, because aggregation attenuates the idiosyncratic variation that masks small but systematic demographic signals. Second, using a reasoning and prediction format that explicitly decomposes individual judgment into an overall baseline and a demographic-specific deviation, we show that the gains arise from both components: demographic information improves baseline estimation and enables meaningful demographic-specific shifts, and interventions show that better component estimates improve final predictions. Finally, comparing direct demographic conditioning with explicit reasoning about how the text interacts with annotator demographics, we find that reasoning yields more accurate baseline and shift estimates as well as final predictions, whereas conditioning alone rarely produces informative shifts. These results suggest that the mixed evidence in prior work may partly stem from individual-level evaluation, and that the value of socio-demographic information depends on both how it is evaluated and how models are guided to use it.

1 INTRODUCTION

The introduction asks when socio-demographic information helps pluralistic offensiveness modeling and where its benefits arise. It finds clearer gains after aggregation and with explicit reasoning that separates baseline and demographic-specific shifts.

1 INTRODUCTION

  • Prior studies report mixed benefits from demographic information, motivating a systematic examination of when it helps, where gains arise, and how modeling choices affect its use.Pluralistic modeling aims to preserve differences in subjective judgments rather than treat disagreement as noise.Source: §1 · pp. 1–2
  • Demographic gains are limited for individual judgments but more pronounced at group and population levels, where aggregation attenuates residual variation while preserving systematic demographic effects.The paper argues that individual-level metrics can obscure small demographic signals.Source: §1 · p. 2
  • Socio-demographic information improves both baseline and demographic-specific shift estimates, and interventions on these components change final predictions in the expected direction.The prediction format makes the broad-population baseline and annotator-specific deviation separately inspectable.Source: §1 · p. 2
  • Explicit reasoning about how text interacts with annotator demographics yields more accurate baseline and shift estimates than direct conditioning, improving aggregated predictions.The authors compare demographic conditioning with an explicit demographic-reasoning variant across three datasets and prompted and trained LLMs.Source: §1 · p. 2

2 INCORPORATING SOCIO-DEMOGRAPHICS IN OFFENSIVENESS MODELING

This section formulates offensiveness judgments using annotator profiles and decomposes individual predictions into an item baseline, demographic deviation, and residual. It introduces model variants that expose these components and describes the shared experimental setup.

2 INCORPORATING SOCIO-DEMOGRAPHICS IN OFFENSIVENESS MODELING

  • The task represents each item with its text, annotator assignments, binary offensiveness labels, and annotators’ multi-attribute socio-demographic profiles.A demographic group denotes a particular value along a demographic axis.Source: §2 · p. 2
  • An individual judgment is decomposed into an item-level baseline, a demographic-specific deviation, and a zero-mean residual for idiosyncratic differences not explained by the profile.The decomposition allows evaluation of both final predictions and the effects of demographic information on each component.Source: §2 · pp. 2–3
  • NoDemo predicts a crowd baseline without annotator demographics, DemoCond adds a profile and shift without explicit demographic reasoning, and DemoReason adds reasoning about demographic effects.The format emits the baseline, demographic-specific shift, and final individual prediction separately.Source: §2 · p. 3
  • Experiments use DICES, D3CODE, and POPQUORN, with four trained open-source models and nine prompted models.The prompted models include open-source Llama and Qwen models and two commercial models.Source: §2 · p. 4

3 WHEN DO SOCIO-DEMOGRAPHICS HELP? THE ROLE OF AGGREGATION

The section evaluates demographic-aware models at individual, group, and population levels using discrete and continuous metrics. Demographic gains are more visible after aggregation, which reduces residual variation while preserving systematic differences.

3.1 SOCIO-DEMOGRAPHIC GAINS BECOME MORE VISIBLE WITH AGGREGATION

  • At population level, prompted-model Acc gains rise from 0.006/0.009 at group level to 0.007/0.017 for DemoCond/DemoReason, with DemoReason significantly outperforming NoDemo on both metrics.Table 1 compares individual-, group-, and population-level results for prompted and trained models, averaged across dataset–model pairs.Source: Table 1 · p. 4, §3.1 · p. 5
  • Demographic variants offer little consistent individual-level advantage, but show clearer group-level gains, including significant continuous-metric improvements in prompted and trained settings.These patterns also appear with other baselines and transfers to unseen datasets; explicit demographic reasoning generally yields larger gains than conditioning, especially at group and population levels.Source: §3.1 · pp. 4–5

3.2 AGGREGATION REVEALS DEMOGRAPHIC SIGNAL OBSCURED BY INDIVIDUAL NOISE

  • Under the stated residual-independence assumptions, aggregation reduces residual variance at rate 1/n while preserving systematic differences in expected judgments.At the individual level, idiosyncratic preferences can mask improvements in the systematic component; aggregation makes those improvements easier to observe, not demographics more informative.Source: §3.2 · p. 5
  • No single demographic axis explains more than 6.2% of within-item label variation, and the largest same-versus-different-group agreement gap is 3.1 percentage points.Overall agreement rates are 63–71%, consistent with demographic effects being small relative to residual individual variation.Source: §3.2 · p. 5

4 WHERE DO SOCIO-DEMOGRAPHICS HELP IN THE PREDICTION PROCESS?

Socio-demographics improve baseline and demographic-shift estimates, and interventions show that better component estimates improve final predictions. Explicit reasoning strengthens these gains, while the size and consistency of benefits vary across evaluation levels and analyses.

4.1 INCORPORATING SOCIO-DEMOGRAPHICS IMPROVES BASELINE ESTIMATION

  • Demographic-aware variants improve baseline estimation over NoDemo, with clearest gains at the item level; DemoReason matches or improves on DemoCond across reported metrics and settings.Emitted crowd estimates also outperform averaged individual predictions, and DemoReason has particular advantages on prompted annotation-level MAE and trained W1.Source: §4.1 · p. 6, Table 2 · p. 6

4.2 WHY DOES SOCIO-DEMOGRAPHIC INFORMATION IMPROVE BASELINE ESTIMATION

  • Demographic-aware models gain more on high-disagreement items, while conditioning reduces excessive consensus in crowd estimates, with DemoReason reducing it most.Items with empirical baselines inside [0.4, 0.6] make up 16.9% of items; NoDemo, DemoCond, and DemoReason predict crowd values below 0.1 for 37.1%, 26.9%, and 25.6%, respectively, versus 16.8% empirically.Source: §4.2 · p. 7, Table 3 · p. 7
  • DemoReason produces more stable baseline estimates across annotators for prompted models, whereas this advantage is not observed for trained models.Separating demographic-specific perceptions into a distinct reasoning block is associated with lower within-item variation and helps item-level improvements carry over to annotation-level estimates.Source: §4.2 · p. 7, Figure 2 · p. 7

4.3 LLMS CAPTURE MEANINGFUL DEMOGRAPHIC-SPECIFIC DEVIATIONS

  • DemoReason recovers global group tendencies across metrics and settings and beats random on individual–majority deviations, while DemoCond gains are limited or inconsistent.DemoCond predicts zero shift for most instances; individual–majority gains remain modest, consistent with socio-demographics explaining only a small fraction of individual-level variation.Source: §4.3 · p. 8, Figure 3 · p. 8
  • Prompted DemoReason alone performs significantly above chance on item–group interactions, but its gain over chance remains below 0.04.Its performance improves at higher |γ| thresholds, consistent with stronger interactions being less affected by annotation noise.Source: Figure 3 · p. 8, §4.3 · p. 9

4.4 BETTER BASELINE AND DEVIATION ESTIMATES IMPROVE FINAL PREDICTIONS

  • Oracle interventions mostly improve final predictions, while adversarial interventions generally degrade them, showing that component-estimate quality directly influences outcomes.Across trained and prompted LLMs, prediction correctness is also positively correlated with crowd quality (r = 0.24–0.27) and shift correctness (r = 0.40–0.57).Source: §4.4 · p. 9, Table 4 · p. 9

5 CONCLUSION

Socio-demographic information captures systematic group- and population-level patterns more clearly than individual judgments, where idiosyncratic variation can obscure gains. Its benefits involve both baseline estimation and demographic-specific shifts, with explicit reasoning strengthening both components.

  • Socio-demographic information better captures systematic group- and population-level patterns than individual judgments, where idiosyncratic variation can obscure gains until aggregation.Limited individual-level gains should not be interpreted as evidence that socio-demographic information is uninformative.Source: §5 · p. 10
  • Demographic gains involve both improved baseline estimation and meaningful, though modest, demographic-specific deviations.Source: §5 · p. 10
  • Explicit reasoning strengthens baseline and shift estimates beyond direct conditioning, leading to better predictions.Source: §5 · p. 10

AI USE STATEMENT

The authors used generative AI for code development, related-work discovery, and manuscript editing, but not to formulate research ideas or interpret results.

  • Generative AI assisted with code, related-work discovery, and manuscript editing; authors developed and analyzed the research ideas and results.The authors state that they reviewed and verified all AI-assisted code and text and take responsibility for the final content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The paper cautions that demographic attributes can invite stereotypes and that transparent reasoning does not ensure unbiased rationales. Its related-work discussion notes mixed results and limited analysis of evaluation levels, prediction components, and demographic-specific reasoning.

ETHICS STATEMENT

  • Demographic attributes may encode broad group associations that do not accurately reflect individuals; generated rationales may also reproduce stereotypes.The authors recommend careful evaluation and human inspection for deployment or real-world decision support.Source: ETHICS STATEMENT · p. 10

A RELATED WORK

  • Prior studies report gains in some demographic settings (Xu et al., 2026; Zhang & Coltekin, 2026) but limited benefits or negative effects in others (Orlikowski et al., 2023; 2025; Sorensen et al., 2025).Source: A RELATED WORK · p. 14
  • Prior work largely evaluates individual, group, and population modeling separately and typically examines final predictions rather than baseline and demographic-specific shifts.This leaves when demographic benefits emerge across aggregation levels and where they arise in the prediction process less systematically characterized.Source: A RELATED WORK · p. 14
  • Earlier reasoning studies generally assess reasoning as a complete modeling paradigm, whereas this work examines how demographic reasoning affects baseline, shift, and final predictions.Source: A RELATED WORK · p. 14

B DETAILS ON EXPERIMENTAL SETUP

The experimental setup uses three multi-annotator datasets, prompted and trained language models, and three prompt variants that differ in demographic information and explicit audience reasoning.

  • Dataset statistics and socio-demographic coverage are summarized for the test sets; training-data amounts vary with SFT and GRPO optimization steps.Source: B DETAILS ON EXPERIMENTAL SETUP · p. 14
  • The setup includes prompted LLMs and trained open-source models; thinking mode is disabled for most models, except Qwen3.5-122B-A10B in prompting.The authors report that enabling thinking mode substantially reduces output parseability for smaller models.Source: B DETAILS ON EXPERIMENTAL SETUP · p. 14
  • DemoReason requests tagged reasoning about the text, norms, a broad-pool baseline, and the annotator audience, followed by crowd, shift, and offensiveness fields.Source: B DETAILS ON EXPERIMENTAL SETUP · pp. 14–15
  • DemoCond removes the explicit audience-reasoning block but retains demographic inputs and shift prediction; NoDemo also removes annotator information and the shift field.Source: B DETAILS ON EXPERIMENTAL SETUP · p. 15
  • Training uses the same SFT and GRPO hyperparameters across datasets and variants, with GRPO rewarded only for final offensiveness-label correctness.At inference, the models use greedy decoding.Source: B DETAILS ON EXPERIMENTAL SETUP · p. 15
  • The comparison includes prompting baselines with and without profiles and annotator-blind and annotator-aware trained baselines, including DEM-MoE.Source: B DETAILS ON EXPERIMENTAL SETUP · p. 15
  • Table 6 reports prompted-model and baseline performance averaged across three datasets, with overall scores averaged across backbones and significance marked against NoDemo or both variants.Source: Table 6 · p. 16

C FULL PERFORMANCE RESULTS

Across models, socio-demographic benefits are limited for individual predictions but become clearer with aggregation.

D GENERALIZATION RESULTS

Socio-demographic modeling retains its aggregation-level advantages in cross-dataset transfer, while held-out-group results are mixed. A qualitative example illustrates how explicit reasoning can improve both baseline and shift estimates.

D GENERALIZATION RESULTS

  • DemoCond and especially DemoReason improve group- and population-level performance in cross-dataset transfer, with DemoReason best on most population metrics and continuous group-level metrics.NoDemo remains strongest at the individual level, while overall performance stays comparable on unseen datasets.Source: D GENERALIZATION RESULTS · pp. 16–17
  • On held-out demographic groups, DemoReason performs comparably overall, but unseen groups show lower population-level performance and weaker calibration on continuous metrics.It performs better on held-out groups for individual- and group-level discrete metrics.Source: D GENERALIZATION RESULTS · p. 17

E QUALITATIVE COMPARISON

  • In the D3CODE example, DemoReason estimates the crowd baseline at 0.4 versus 0.4306, predicts shift=+1, and correctly identifies the offensive judgment.DemoCond estimates 0.3 but predicts shift=0 and an incorrect non-offensive judgment; NoDemo estimates 0.1 and is also incorrect.Source: E QUALITATIVE COMPARISON · p. 18, Figure 4 · p. 18, Figure 6 · p. 19
  • The example links improved final prediction to both a more accurate baseline and an explicit demographic-specific deviation.The authors also flag a risk that DemoReason may rely on stereotypical assumptions about demographic groups.Source: E QUALITATIVE COMPARISON · p. 18

F FULL BASELINE ESTIMATION RESULTS

Socio-demographics generally improve baseline estimation, and intervention results provide stronger evidence for baseline quality affecting final predictions than for shift quality.

F FULL BASELINE ESTIMATION RESULTS

G BASELINE- AND SHIFT-ONLY INTERVENTION TESTS

  • For separate interventions on crowd and shift components, the baseline-only intervention more consistently followed the expected Oracle > Original > Adversarial ordering, providing stronger evidence that baseline quality directly affects.Source: G BASELINE- AND SHIFT-ONLY… · p. 19
  • In shift-only interventions, the expected ordering is less consistent, and adversarial shifts commonly outperform original shifts.This is consistent with the reported modest effect sizes of model-emitted shifts.Source: G BASELINE- AND SHIFT-ONLY… · p. 20

H LIMITATIONS

The findings are constrained by proxy-based measurement, the evaluated modeling pipeline, and coarse socio-demographic representations.

  • Some demographic-specific shift analyses rely on empirical or proxy targets for quantities that are not directly observable, and these targets can be distorted by annotation noise.Empirical baselines reflect each dataset's available annotator population and may not represent a broader population.Source: H LIMITATIONS · p. 21
  • Findings are specific to the prompting templates, reasoning format, and training pipeline examined, leaving generalization to other approaches and settings unclear.The authors also do not establish that models internally follow the assumed decomposition.Source: H LIMITATIONS · pp. 21–22
  • Available socio-demographic attributes provide only a coarse representation, and intersectional effects are difficult to estimate reliably because of data sparsity.Many individual, contextual, and experiential factors remain unobserved.Source: H LIMITATIONS · p. 22

Related papers

Loading 2610.37896v1…