PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions

Matthias Orlikowski, Jiaxin Pei, Paul Röttger, Philipp Cimiano, David Jurgens, Dirk Hovy

arXiv:2502.20897v1cs.CL

TL;DR

This paper asks whether LLMs can be trained to model annotators’ subjective text perceptions from sociodemographics, given poor zero-shot prompting results. Across five tasks, models appear to improve mainly by learning specific annotators’ behavior rather than generalizable links between demographics and annotation.

  • Problem

    Zero-shot sociodemographic prompting has performed poorly, leaving open whether training can make LLMs accurate models of annotator variation.

    Source: §1 · p. 1

  • Method

    The authors fine-tune LLMs using a curated set of five tasks, formatted annotator information, and evaluation partitions that test known and new annotators.

    Source: §1 · p. 1, §4.2 · p. 4

  • Results

    Across tasks, models do not generalize from sociodemographic patterns to new annotators; apparent attribute gains mainly come from examples of identifiable annotators’ behavior.

    Source: §7 · p. 8, §8 · p. 8

  • Takeaways & Limitations

    The results suggest LLMs learn little meaningful connection between sociodemographics and annotation, and that annotator identity better captures disagreement than a text-only baseline.

    Source: §1 · p. 2, §7 · p. 8

    All study annotators are from the US, so the datasets do not support cross-geocultural comparisons of how results transfer.

    Source: §9 · p. 9

Abstract

from arXiv · show

People naturally vary in their annotations for subjective questions and some of this variation is thought to be due to the person's sociodemographic characteristics. LLMs have also been used to label data, but recent work has shown that models perform poorly when prompted with sociodemographic attributes, suggesting limited inherent sociodemographic knowledge. Here, we ask whether LLMs can be trained to be accurate sociodemographic models of annotator variation. Using a curated dataset of five tasks with standardized sociodemographics, we show that models do improve in sociodemographic prompting when trained but that this performance gain is largely due to models learning annotator-specific behaviour rather than sociodemographic patterns. Across all tasks, our results suggest that models learn little meaningful connection between sociodemographics and annotation, raising doubts about the current use of LLMs for simulating sociodemographic variation and behaviour.

1 Introduction

The paper asks whether training can make LLMs more accurate models of annotator variation than zero-shot sociodemographic prompting. Across its research questions, it finds that apparent gains from attributes largely reflect learning specific annotators rather than generalizable sociodemographic patterns.

1 Introduction

  • Annotation disagreements can reflect errors, adversarial behavior, or individual preferences, and models have been developed to capture some regularities in this variation.The introduction gives group-based differences in offensiveness ratings as an example of regular variation.Source: §1 · p. 1
  • The study tests whether training can address poor zero-shot sociodemographic prompting, which prior work found inaccurate for modeling annotators.The authors note that the causes of low zero-shot performance may be multifaceted.Source: §1 · p. 1
  • The proposed approach combines persona prompting with annotator modelling, using formatted annotator information in fine-tuned decoder-only LLMs with prediction heads.Training uses five subjective classification datasets with annotator IDs and age, gender, race, and education attributes.Source: §1 · pp. 1–2
  • Models improve with sociodemographics for known annotators, but do not generalize to new annotators; gains largely reflect unique attribute combinations acting as identifiers.Annotator IDs improve prediction of disagreement compared with a text-only baseline.Source: §1 · p. 2

2 Related Work

Related work connects subjective annotation, annotator modelling, and sociodemographic prompting. Prior studies examine both annotator-specific prediction and LLM simulation of people based on background descriptions.

2 Related Work

  • Research on annotator characteristics finds that attributes and preferences can relate to judgments, though some studies find no relevant background associations and recent work examines within-group differences.Reported effects can be strongest when annotated content aligns with annotator attributes.Source: §2 · p. 2
  • Annotator modelling predicts individual annotators’ labels for specific inputs, often using IDs or annotation statistics; prior work found such models do not beat content-only baselines for unseen annotators.Related work also reports improvements on high-disagreement instances.Source: §2 · p. 2
  • Studies of sociodemographic prompting report conflicting findings on its value relative to annotator identity, while other work reports demographic gains without an ID comparison (Jaggi et al., 2024).Concurrent work fine-tuned on one dataset with 600 training instances and did not compare against IDs.Source: §2 · pp. 2–3
  • LLM simulation studies report both successful applications and concerns about caricature or misportrayal of social groups, including work prompting models with sociodemographic profiles.Concurrent work found prompted LLMs misaligned with behavioral-experiment outcome distributions while fine-tuning on relevant examples improved alignment (Gao et al., 2024).Source: §2 · p. 3

3 DEMO

DEMO unifies five published datasets of subjective text perceptions and standardizes four shared sociodemographic attributes. The collection spans multiple tasks and annotators, with per-dataset statistics presented in Table 1.

3 DEMO

  • DEMO unifies five datasets of subjective text perceptions and standardizes age, gender, race, and education across them.These four attributes are the largest intersection available across all five datasets.Source: §3 · p. 3
  • The combined datasets contain 21,632 texts, 2,614 annotators, and 147,297 annotations.Table 1 presents statistics for each dataset.Source: §3 · p. 3, Table 1 · p. 4
  • The tasks cover intimacy, offensiveness, politeness, conversational safety, and sentiment, using datasets with differing annotation settings and annotator counts.Intimacy uses English MINT tweets rated by seven annotators each; the safety data apply the source authors’ low-quality annotator filtering.Source: §3 · p. 3

4 Experimental Setup

The experiments fine-tune LLMs with different annotator information and evaluate generalization under instance- and annotator-based partitions. They compare formatted inputs against zero-shot prompting.

4 Experimental Setup

  • The parent setup evaluates LLMs as annotator models through fine-tuning and baseline comparisons.The experimental section introduces the methods and setups used to test these models.Source: §4 · p. 4

4.1 Weights and Architecture

  • The study fine-tunes base Llama 3 8B with a decoder-only transformer and prediction head, following reward-model architecture for text rating tasks.Supplementary experiments with Mistral 7B found minimal differences from Llama 3.Source: §4.1 · p. 4

4.2 Data Partitions

  • The instance split holds out texts while allowing annotators to appear across splits, testing generalization to new instances for annotators with known sociodemographics.The annotator split holds out annotators to assess simulation of new annotators from their sociodemographics.Source: §4.2 · p. 4

4.3 Prompt Formats

  • Inputs vary between text alone, text with age, gender, race, and education, text with annotator ID, and a combined ID-plus-attributes format.Attribute values are formatted as short descriptions, and the setup assumes all possible values are known before testing.Source: §4.3 · pp. 4–5

4.4 Baseline System

  • The zero-shot baseline prompts Llama 3 Instruct 8B with content-only or attribute variants adapted to a chat template.Robustness checks use a larger quantized model and a prompt variant that simply lists attributes.Source: §4.4 · p. 5

5 Experiments

The experiments evaluate whether fine-tuned Llama 3 8B can model individual annotators using sociodemographics or identity. Performance gains on known annotators are much larger with IDs, while neither attributes nor IDs meaningfully improve predictions for unseen annotators.

5.1 Training and Evaluation

  • The experiments evaluate Llama 3 8B on DEMO using instance and annotator splits, treating ratings on three-point Safety or five-point task scales as classes.The main experiments use macro-average F1 and report means over 30 random seeds with bootstrap 95% confidence intervals.Source: §5 · p. 5, §5.1 · p. 5

5.2 Results: Sociodemographic Modelling

  • On the instance split, sociodemographics improve macro-average F1 over text-only predictions, but annotator IDs yield larger gains and adding attributes to IDs adds little.The ID-only gains subsume those from attributes; Offensiveness is a notable exception where zero-shot performance slightly exceeds the fine-tuned content-only model because of strong label imbalance.Source: Figure 2 · p. 5, §5.2 · pp. 5–6

5.3 Results: Annotator Generalization

  • For unseen annotators, attributes, IDs, or both yield negligible gains over text-only predictions, indicating little generalization from sociodemographics.ID gains are not expected when test annotators' IDs were absent from training; the attribute result suggests few meaningful generalizable links among text, attributes, and ratings.Source: §5.3 · p. 6

6 Analyses: What Are LLMs Learning About Sociodemographics?

The analyses test whether sociodemographic attributes act as annotator-identity proxies and whether annotator information helps model disagreement. Gains concentrate on uniquely identifiable profiles, while ID-based models better capture distributions in disagreement cases.

6 Analyses: What Are LLMs Learning About Sociodemographics?

  • The analyses address whether attribute combinations proxy annotator IDs and whether adding attributes or IDs improves modeling of disagreement.The proxy analysis compares attribute-related gains for unique and frequent profiles; the disagreement analysis groups instances by label entropy.Source: §6 · p. 6, §6.1 · p. 6, §6.2 · p. 7

6.1 Sociodemographics as Proxies

  • Attribute gains are largest for annotators with unique profiles and show no consistent or substantial improvement for frequent profiles, supporting the proxy-for-identity account.Unique profiles are combinations not shared by other test annotators; frequent profiles are shared by many.Source: §6.1 · pp. 6–7, Figure 4 · p. 7

6.2 Modelling Disagreement

  • ID-based models have the smallest distances to actual rating distributions on higher-disagreement cases, although disagreement remains harder than agreement and Offensiveness is an exception to near-equal performance across entropy levels.Instances are split at median label entropy; Wasserstein distance measures the difference between predicted and actual rating distributions.Source: §6.2 · p. 7

7 Discussion

The discussion concludes that sociodemographics alone do not support reliable modeling of new annotators; apparent gains mainly reflect identifiable annotators’ learned behavior. Modeling from examples can capture diverging views without specifying their governing factors.

7 Discussion

  • Models can learn a given set of annotators from examples, but do not generalize from seen sociodemographic patterns to new annotators.Unique attribute combinations can identify known annotators, making the apparent benefit of attributes an identity-proxy effect.Source: §7 · pp. 7–8
  • Adding sociodemographics to annotator IDs does not improve results over IDs alone, consistent with some prior findings, though other work reports cases where IDs perform worse than content-only input.The discussion notes that future work could examine how dataset characteristics and architectures influence these differences.Source: §7 · p. 8
  • Learning from examples of identifiable annotators can model within-group variation that demographic personas may oversimplify.The discussion connects this possibility to concerns about uniform responses and flattened group differences.Source: §7 · p. 8

8 Conclusion

Across five datasets and two partitions, LLMs primarily learn specific annotators’ behavior from examples, not perceptions from sociodemographics alone.

8 Conclusion

  • The study asks whether training can overcome poor sociodemographic zero-shot prompting for predicting individual perceptions.Source: §8 · p. 8
  • Across five datasets and two data partitions, LLMs primarily learn specific annotators’ behavior rather than predicting perceptions from sociodemographics alone.The study evaluates partitions based on annotators and instances.Source: §8 · p. 8
  • Examples of individuals’ ratings yield much better performance than both zero-shot and content-only baselines.Source: §8 · p. 9

9 Limitations

The study’s evidence is bounded by its US-only annotators and primarily Llama 3 experiments, limiting cross-geocultural comparison and broad model-family conclusions.

9 Limitations

  • US-only annotators prevent cross-geocultural comparisons, and available datasets from other regions lack the same sociodemographic attributes.The English-language Intimacy subset also excludes non-US annotators; suitable annotator-modeling datasets are rare.Source: §9 · p. 9
  • Because experiments primarily use Llama 3, results with other LLM families might differ.Small-scale Mistral 7B experiments show the same pattern in a smaller setting, but include fewer runs and tasks.Source: §9 · p. 9
  • The study prioritizes 30 runs for more reliable variability estimates, trading off the number of model families and experimental breadth.Source: §9 · p. 9

10 Ethics

The ethics discussion cautions against demographic prompting and notes that the study’s limited demographic coverage may omit important attributes; the appendix details dataset normalization, splits, and profile distributions.

10 Ethics

  • Fine-tuning with demographics does not significantly improve performance, while individual annotations help capture annotators’ ratings; the authors urge caution about potential demographic-prompting biases.Source: §10 · p. 9

A.1 Normalizing annotator attributes

  • The five datasets were normalized into shared categories for gender, race, age, and education.The normalization process identifies common attributes across datasets and groups them into consistent categories.Source: §A.1 · p. 13

A.2 Dataset Splits

A.3 Distribution of Sociodemographic Profiles

  • Attribute combinations can effectively identify annotators, and many unique profiles may be rare in the general population; the reasons and their relationship to annotation behavior warrant further investigation.The profile tables show frequent and sampled unique Intimacy profiles; possible links to personal experiences are presented as an alternative interpretation.Source: §A.3 · pp. 13–14, Table 4 · p. 14

B Fine-Tuning Implementation Details

The supplementary experiments report that Mistral 7B reproduces the larger gains from annotator IDs over sociodemographics. The appendices document fine-tuning settings and zero-shot prompts and experiments.

B Fine-Tuning Implementation Details

  • Training truncates texts at 232 tokens, adds 7 ID or 22 sociodemographic tokens, and adjusts batch sizes to fit GPU memory.Batch sizes vary by task, and Safety accumulates updates to an effective size of 16 while other datasets use 64.Source: B Fine-Tuning Implementation Details · pp. 14–15
  • For Offensiveness, Politeness, Safety, and Sentiment, appendix tables report counts for the ten most-frequent and ten randomly sampled unique sociodemographic profiles.Source: Table 5 · p. 14, Table 6 · p. 14, Table 7 · p. 15
  • Learning rates were selected by validation grid search over 0.0003, 0.00008, 0.00006, and 0.00003; LoRA used r = 8, α = 16, and 0.05 dropout.The experiments also used bf16, Adam, and 10 warmup steps.Source: B Fine-Tuning Implementation Details · pp. 14–15

C Evaluating Additional Model Families

D.1 Zero-Shot Prompt Templates

  • Zero-shot prompts present the task, text, and multiple-choice answers, with sociodemographic attributes optionally described in conversational full sentences.The task prompts were adapted from Hu and Collier (2024).Source: §D.1 · p. 17

D.2 Zero-Shot Experiments

  • Zero-shot experiments compare content-only prompts with prompts that add annotator attributes, using Llama 3 Instruct 8B and additional 70B and prompt-format checks.The 70B and alternative attribute-list robustness experiments were limited to the instance split.Source: §D.2 · p. 17

D.3 Results: Inconsistent Effects of Sociodemographic Prompting

Zero-shot sociodemographic prompting has inconsistent effects across tasks. The 8B model mostly shows no or slightly negative changes, while the 70B model has small gains on some tasks and declines on others.

D.3 Results: Inconsistent Effects of Sociodemographic Prompting

  • Listing attributes in a list-like format produces less accurate predictions than describing profiles conversationally in full sentences.The authors use the conversational format in their other zero-shot experiments.Source: §D.3 · pp. 17–18, Figure 7 · p. 18
  • For the 8B model, adding attributes mostly has no or slightly negative effects, with Politeness showing a robust increase of about 3 macro-average F1 points.Source: §D.3 · p. 18, Figure 7 · p. 18
  • For the larger 70B model, annotator attributes have slightly stronger but directionally inconsistent effects on DEMO performance: Intimacy, Politeness, and Sentiment improve slightly, while Safety and Offensiveness decrease.Figure 7 presents zero-shot results on the instance split, with mean scores over 30 seeds and 95% confidence intervals from bootstrap sampling.Source: §D.3 · p. 18, Figure 7 · p. 18

Related papers

Loading 2502.20897v1…