Source-linked AI summary
Whose Opinions Do Language Models Reflect?
Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, Tatsunori Hashimoto
TL;DR
Language models increasingly express opinions in open-ended settings, but it is unclear whose views they reflect and how to measure that alignment. The paper builds a survey-based framework and OpinionQA to compare model distributions with 60 US demographic groups, finding substantial, persistent misalignment even after steering.
Problem
Subjective LM responses can affect users and shape beliefs, creating a need to identify whose opinions models reflect beyond broad human alignment.
Method
The paper converts public opinion polls into OpinionQA and compares 9 LM opinion distributions with overall and demographic human distributions using ordinal-aware alignment metrics.
Results
Across topics, current LMs substantially misalign with US demographic groups, at a level comparable to the Democrat-Republican climate-change divide, and steering does not resolve this misalignment.
Takeaways & Limitations
OpinionQA exposes representation failures, including poor reflection of groups such as people aged 65+ and widowed individuals, while showing that steerability improvements are limited.
Takeaways & Limitations
The alignment lens may reward reproducing human biases, so the metrics should help diagnose model behavior rather than be blindly optimized.
Abstract
from arXiv · showhide
Language models (LMs) are increasingly being used in open-ended contexts, where the opinions reflected by LMs in response to subjective queries can have a profound impact, both on user satisfaction, as well as shaping the views of society at large. In this work, we put forth a quantitative framework to investigate the opinions reflected by LMs -- by leveraging high-quality public opinion polls and their associated human responses. Using this framework, we create OpinionsQA, a new dataset for evaluating the alignment of LM opinions with those of 60 US demographic groups over topics ranging from abortion to automation. Across topics, we find substantial misalignment between the views reflected by current LMs and those of US demographic groups: on par with the Democrat-Republican divide on climate change. Notably, this misalignment persists even after explicitly steering the LMs towards particular demographic groups. Our analysis not only confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs, but also surfaces groups whose opinions are poorly reflected by current LMs (e.g., 65+ and widowed individuals). Our code and data are available at https://github.com/tatsu-lab/opinions_qa.
1 Introduction
The paper asks whose opinions language models reflect and introduces a quantitative survey-based framework to measure alignment with US populations and demographic groups. Across topics, models show substantial misalignment, limited gains from steering, and topic-dependent group alignment.
- Public opinion reflected by LMs matters because subjective responses can affect user experience and shape subsequent beliefs.
- The framework repurposes public opinion surveys because expert-selected, nuanced, demographic-specific, multiple-choice questions support quantitative LM-human comparisons.
- 1498 questions and 60 demographic groups form OpinionQA, which is used to evaluate 9 LMs across science, politics, personal relationships, and other topics.
- Representativeness: LM opinions substantially misalign with the general US populace, roughly matching the Democrat-Republican divide on climate change.
- Steerability: Steering generally improves alignment with a prompted group, but improvements are modest and do not resolve representativeness problems.
- Consistency: Alignment preferences vary across topics, while some groups—including people aged 65+, Mormons, and widowed individuals—are poorly represented across models.
- The dataset and metrics are intended as probes for understanding representation failures, not benchmarks to optimize indiscriminately.
2 The OpinionQA Dataset
OpinionQA addresses the challenge of evaluating LM opinions by adapting multiple-choice public opinion surveys with human responses and demographic information. The instantiated dataset is US-centric and aggregates survey data into broad and fine-grained topics.
- Studying LM opinions requires relevant topics, extractable responses, and reference distributions from representative human groups.
- Public opinion surveys address these challenges through expert topic selection, carefully designed questions, and responses from demographic groups.
- Each survey question has multiple possible answers and records one selected answer for every participant.
- Human opinion distributions are formed by aggregating respondent answers, optionally using survey weights, for the overall population or demographic groups.
- OpinionQA uses 15 American Trends Panel polls, typically containing about 100 questions each, with respondent demographics and weights.
- The dataset provides overall and group-level distributions for 60 demographic groups and organizes questions into 23 coarse and 40 fine-grained topics.
- OpinionQA is English and US-centric, so its subsequent analysis is limited to US populations and demographic groups.
3 Measuring human-LM alignment
The paper probes LM opinions with multiple-choice survey prompts and compares their answer distributions with aggregated human distributions. It uses ordinal-aware Wasserstein distance and a normalized alignment score, treating refusals separately.
- Representativeness uses a standard QA prompt, whereas steerability prepends context describing the demographic group the LM should emulate.
- Steering: Steering contexts use QA responses, biographical answers, or explicit instructions to portray a demographic group.
- Model opinion distributions are extracted from next-token log probabilities over answer choices, rather than selecting a single correct answer.
- Refusal is excluded from the normalized opinion distribution and measured separately as a model refusal probability.
- The comparison uses 1-Wasserstein distance, mapping ordinal answer choices to positive integers while omitting refusals and specially handling hedging options.
- The normalized alignment metric ranges from 0 to 1, with 1 indicating a perfect match between opinion distributions.
- Alignment is defined relative to particular human groups, so higher alignment is not necessarily desirable or simultaneously achievable for disagreeing groups.
4 Whose views do current LMs express?
Across nine LMs, default opinions are substantially misaligned with the US population and demographic groups, with alignment patterns varying by model type and topic. Steering usually improves subgroup alignment modestly but does not eliminate disparities, while models show inconsistent group alignment across topics.
- Evaluation setup: 9 LMs spanning 350M to 178B parameters are evaluated on OpinionQA using default, unprompted opinion distributions.The models include base and human-feedback-tuned systems from OpenAI and AI21 Labs.
- Overall representativeness: None of the models is perfectly representative of the general US populace, and human-feedback-tuned models are often less representative than base models.Every comparison group used as a human baseline is more representative of the overall populace than the evaluated LMs.
- Overall representativeness: Most models have overall representativeness comparable to the alignment between agnostic and orthodox people on abortion or Democrats and Republicans on climate change.These human comparisons contextualize the magnitude of LM-population misalignment.
- Group representativeness: Base LMs tend to align with lower-income, moderate, and Protestant or Roman Catholic groups, whereas RLHF-trained models shift toward liberal, high-income, and well-educated groups.The demographic groups with the highest representativeness change systematically between base and RLHF-trained models.
- Group representativeness: Groups including people aged 65+, widowed individuals, and people with high religious attendance have low representativeness scores across LMs.The authors note that age-65+ representation was also scarce among crowdsourcing participants in related work, while not identifying the production RLHF crowdworker distribution.
- Modal representativeness: Text-davinci-003 often assigns more than 0.99 probability to one answer, collapsing within-group opinion diversity into a single modal response.This sharp distribution can make the model appear representative of a group while failing to reflect the diversity of that group’s views.
- Steerability: Steering generally improves subgroup alignment, but none of the models’ group disparities disappears after steering.Improvements are often approximately constant-factor gains, so models still align better with some groups than others.
- Consistency: LM alignment is not fully consistent across topics: even generally liberal models can express conservative views on topics such as religion.Overall consistency scores are fairly low, indicating patchwork opinions across demographic groups and topics.
5 Related work
Prior work has studied LM personas, political slants, subjective evaluation, alignment, and bias, but this paper evaluates opinion representativeness across diverse human subpopulations using public-opinion surveys.
- Persona research examines whether LMs reproduce human experiments and simulate human-like personas.
- Prior studies find that human-feedback-trained models often express left-leaning and pro-environmental opinions.
- Unlike modal slant evaluations, this work tests whether LMs reflect varied human subpopulation perspectives consistently across topics.
- Subjective-evaluation research highlights that humans can disagree about correct responses across toxicity, ethics, and inference tasks.
- Human-LM alignment research seeks values to encode in models, whereas this paper asks which humans models are aligned with.
- Bias, toxicity, and truthfulness studies target undesirable outcomes with relatively defined standards; this work quantifies opinion representativeness on inherently subjective survey questions.
6 Conclusion
The paper introduces a public-opinion-poll framework and OpinionQA to measure how LM opinions align with US populations and demographic groups. It identifies misalignment in overall and subgroup representativeness, steerability, and consistency-related behaviors.
- The framework uses public-opinion polls to examine which human populations’ opinions are reflected by LMs.
- OpinionQA reveals poor alignment in overall representativeness, subgroup representativeness, and steerability, including for 65+, Mormon, and widowed groups.
- The analysis examines whether instruct-tuning distorts opinion distributions and whether models exhibit consistent liberal biases.
7 Limitations
The paper cautions that alignment metrics, survey data, and multiple-choice evaluation each impose important limits on interpreting LM opinion alignment.
- Limitations of alignment: Perfectly representing human opinions may replicate human biases, so alignment metrics should not be blindly optimized.
- Limitations of the ATP and surveys: OpinionQA uses the American Trends Panel, whose survey responses may be affected by question specificity and social desirability bias.
- Limitations of the multiple-choice format: The dataset’s multiple-choice format differs from open-ended generation, leaving transfer of measured alignment to dialogue and open-ended QA uncertain.
Acknowlegements
The dataset adapts Pew American Trends Panel surveys into OpinionQA by selecting and minimally processing multiple-choice questions, while preserving weighted demographic response distributions.
- Data collection: Pew’s American Trends Panel draws roughly 10,000 US participants recruited through address-based sampling, including households without internet access.Participants may take surveys repeatedly, and panelists receive payment incentives.
- Questionnaire design: Pew develops questions through topic selection, piloting, cognitive testing, and wording designed to be clear and non-leading.Open-ended pilots can help determine valid answer choices.
- Data quality: ATP data include sample weights that account for sampling bias and non-response, alongside checks for design errors and respondent or answer quality.The survey is first tested with 60 fast panelists.
- Dataset construction: OpinionQA derives questions and human reference distributions from 15 ATP surveys collected between 2017 and 2021 across broad topics and demographic traits.The surveys support subgroup-level analysis using respondent demographics.
- Post-processing: The dataset directly extracts multiple-choice questions, making minimal edits to resolve cross-references, variable dependence, formatting, and repeated-list structures.Questions may also be manually categorized into multiple topics, and steerability analysis uses 500 questions where subgroups frequently disagree.
A.4 Metrics
The metrics account for the ordinal structure of survey answer choices and compare LM distributions with sharpened human distributions using Wasserstein distance and temperature scaling.
- Distributional comparison: Wasserstein distance maps ordinal answer choices into a metric space before comparing human and LM opinion distributions.Most ordered options map to consecutive integers; non-ordinal penultimate choices receive an intermediate value.
- Distributional comparison: Human opinion distributions are sharpened toward their dominant mode with temperature scaling at 1e-3 for the Section 4.1 comparison.The reported results are fairly robust to the temperature choice.
B Additional experimental Results
Additional experiments check whether LM answer probabilities form meaningful distributions and visualize how models allocate probability mass across answer choices.
- Probability mass: At least 30% average cumulative probability mass is assigned to answer choices across models, supporting the meaningfulness of normalized LM opinion distributions.The measure sums exponentiated log probabilities over non-refusal options, and varies across models.
- Probability mass: Figure 7 shows the distribution of probability mass that different models assign to one answer choice.
B.1 Representativeness
Additional analyses examine subgroup representativeness, refusal behavior, response entropy, post-steering alignment, and topic-level demographic alignment.
- Subgroup representativeness: Figure 8 extends subgroup representativeness scores to demographic attributes omitted from the main paper for space.
- Refusal and response distributions: Figure 9 compares the entropy of per-question response distributions for humans and different LMs.
- Refusal and response distributions: Human-feedback-tuned models have refusal rates closest to humans and generally refuse less often than human respondents.Refusal options are excluded from opinion-distribution similarity calculations and analyzed separately.
- Post-steering alignment: Figure 11 compares how successfully different LMs personalize opinions toward specified demographic subgroups.
- Topic-level alignment: Figure 12 encodes fine-grained topic alignment by political ideology, education, and income, with dot size representing the ratio of best to worst subgroup representativeness.A large red dot indicates strong alignment with conservatives on that topic.
B.4 Robustness
The robustness analysis tests whether LM opinion distributions change with answer-option ordering and prompt formatting. Option permutations slightly reduce representativeness, but overall and subgroup trends remain largely consistent, while formatting variants are evaluated on 500 questions.
- Option ordering: Randomly permuting answer choices across LMs produces a small drop in all models’ representativeness scores.The same permutation is used across LMs for each question, replacing the default ordinal ordering.
- Option ordering: The drop may partly reflect a mismatch between randomly ordered LM prompts and human reference responses collected with ordinal options.Because humans are also sensitive to option ordering, the reference human opinion distribution may itself depend on presentation order.
- Option ordering: Despite the ordering perturbation, overall and subgroup-level trends remain largely consistent.The appendix presents these comparisons in Figure 13.
- Prompt formatting: Prompt-format robustness is tested by adding either general instructions or an example-based instruction before each dataset question.The standard prompt is compared with these variants on 500 questions from Section 4.2, using selected demographic attributes because the results are similar across attributes.
- Prompt formatting: The prompt-formatting comparisons report overall and subgroup representativeness, with results shown in Appendix Figure 14.The analysis includes standard, general, and example-based prompt variants.