PaperlayerView exact revision on arXiv

Source-linked AI summary

Parametric Social Identity Injection and Diversification in Public Opinion Simulation

Hexi Wang, Yujia Zhou, Bangde Du, Qingyao Ai, Yiqun Liu

arXiv:2603.16142v2cs.CL

TL;DR

LLM-based public opinion simulation is scalable but struggles to preserve social diversity, with distinct identities becoming less distinguishable in hidden representations. The paper introduces PSII, which injects demographic and value vectors into intermediate states, and reports improved diversity and distributional fidelity across WVS experiments with multiple open-source models.

  • Problem

    Existing LLM-based opinion simulations often produce homogeneous responses and insufficient inter-group and intra-group diversity, limiting their fidelity to real populations.

  • Method

    PSII injects parametric demographic and value vectors into intermediate hidden states, using persistent signals and stochastic perturbations to control identity diversity without fine-tuning.

  • Results

    Across WVS experiments and multiple open-source LLMs, PSII consistently improves prediction accuracy, diversity metrics, and distributional fidelity while counteracting higher-layer Diversity Collapse.

  • Takeaways & Limitations

    Representation-level identity control can produce synthetic populations that are more coherent, heterogeneous, and representative of real-world survey responses.

  • Takeaways & Limitations

    PSII’s diversity effects vary across model scales because hidden-state geometry, depth, noise sensitivity, and injection stages differ between models.

Abstract

from arXiv · show

Large language models (LLMs) have recently been adopted as synthetic agents for public opinion simulation, offering a promising alternative to costly and slow human surveys. Despite their scalability, current LLM-based simulation methods fail to capture social diversity, producing flattened inter-group differences and overly homogeneous responses across demographic groups. We identify this limitation as a Diversity Collapse phenomenon in LLM hidden representations, where distinct social identities become increasingly indistinguishable across layers. Motivated by this observation, we propose Parametric Social Identity Injection (PSII), a general framework that injects explicit, parametric representations of demographic attributes and value orientations directly into intermediate hidden states of LLMs. Unlike prompt-based persona conditioning, PSII enables fine-grained and controllable identity modulation at the representation level. Extensive experiments on the World Values Survey using multiple open-source LLMs show that PSII significantly improves distributional fidelity and diversity, reducing KL divergence to real-world survey data while enhancing overall diversity. This work provides new insights into representation-level control of LLM agents and advances scalable, diversity-aware public opinion simulation.

1 Introduction

LLM-based public opinion simulation offers scalable alternatives to traditional surveys but often collapses social diversity. The paper analyzes this representational collapse and proposes PSII to preserve heterogeneous identities and improve survey fidelity.

  • Motivation: LLM-based simulation reduces the logistical burden of public-opinion research but often produces homogeneous populations that differ from real human behavioral distributions.Traditional surveys face escalating cost and scalability problems, while insufficient simulated diversity can mislead conclusions about societal dynamics.
  • Diversity Collapse: Standard training favors probable continuations, while prompt-injected identities can dominate generation and flatten inter-group or intra-group diversity.These mechanisms underrepresent minority viewpoints and treat demographic attributes as fixed explanatory variables.
  • Diversity Collapse: 500-agent KPCA projections reveal non-monotonic representation diversity: lower layers cluster, intermediate layers spread, and higher layers contract into dense clusters.The authors name this upper-layer contraction Diversity Collapse and link it to unstable conditions for hidden-state evolution.
  • PSII: PSII injects demographic and value vectors into hidden states, using persistent signals for inter-group diversity and stochastic perturbations for intra-group variation.The framework modulates hidden-state trajectories rather than relying on surface prompts, while requiring minimal storage and no fine-tuning.
  • Evaluation: Across WVS experiments and multiple open-source LLMs, PSII improves prediction accuracy, diversity metrics, and the persistence of higher-layer representation diversity.PSII-generated representations sustain or increase diversity where baseline representations undergo Diversity Collapse.
  • Contributions: The paper characterizes Diversity Collapse, introduces PSII for stable heterogeneous identity modeling, and reports improved diversity and distributional fidelity.These contributions target synthetic populations that better reflect real-world heterogeneity.

2 Related Work

Related work spans zero-shot prompting, persona conditioning, fine-tuning, demographic population generation, activation control, sampling, prompt engineering, and hybrid human-LLM approaches. Across these directions, researchers seek better realism, representativeness, diversity, and bias mitigation.

  • LLM-based Personality Simulation Agents: Zero-shot LLM simulation is a baseline, but default outputs can exhibit political bias and underrepresent marginalized groups.The related work identifies direct prompting as an early approach to LLM-based personality simulation.
  • LLM-based Personality Simulation Agents: Persona-based prompting improves prediction accuracy but can trade sensitivity against robustness and may induce stereotyping through simple attribute injection.These methods add demographic attributes such as age, gender, and race to prompts.
  • LLM-based Personality Simulation Agents: Fine-tuning and alignment on social-survey data improve distribution fitting, while backstories and distribution-shift alignment address values and changing contexts.SimVBG models complex values through individual backstories, and DSA adapts models to context changes.
  • LLM-based Personality Simulation Agents: Demographic generation methods use hierarchical trees or importance sampling to reduce population bias, while activation-space control regulates personality traits.These approaches provide alternative mechanisms for constructing topic-adaptive, population-aligned, or precisely controlled agents.
  • Enhancing Diversity and Representativeness: Diversity remains difficult because models favor generic opinions over minority voices.This challenge motivates methods that explicitly seek differentiated perspectives and broader population coverage.
  • Enhancing Diversity and Representativeness: Temperature and top-k sampling increase randomness but can reduce coherence, motivating stratified and negatively correlated sampling strategies.SimpleStrat explores different solution spaces, while NC Sampling suppresses probable responses to elicit differentiated perspectives.
  • Enhancing Diversity and Representativeness: Prompt-based methods such as step-by-step recall, collective critique, and multilingual prompting aim to expand viewpoint and cultural coverage.Multilingual prompting can activate embedded cultural knowledge as an implicit cue.
  • Enhancing Diversity and Representativeness: Hybrid Human-LLM crowds combine human diversity with LLM reasoning to mitigate bias and enhance collective intelligence.This approach responds to concerns that relying solely on LLM populations may exacerbate bias.

3 Parametric Social Identity Injection

PSII combines prompt-level agent profiles with representation-level demographic and language vectors to provide stable, heterogeneous identity signals throughout generation. It constructs demographic vectors from value-conditioned responses, injects them hierarchically with controlled noise, and uses language vectors to add cultural and linguistic variation.

  • Identity Construction: PSII combines semantic agent profiles at the prompt level with identity vectors injected directly into hidden states.The dual-level design provides demographic context before generation and representation-level identity control during generation.
  • Identity Construction: Demographic vectors are built for selected survey-relevant features and value-conditioned through synthetic questions and role instructions.For each demographic value, the method generates responses, computes layer-wise hidden-state averages, and derives a value-specific vector relative to the variable’s marginal mean.
  • Performance Evaluation: Additional analyses report that demographic-vector semantics remain consistent across instruction models, prompt variants, and random seeds.This supports the reported robustness of the vector-construction procedure across those settings.
  • Parametric Injection: PSII injects demographic vectors into selected layers during prompt encoding and autoregressive generation, continuously steering hidden-state trajectories.Vectors are layer-specific, added to prompt-token representations, and reapplied to newly generated tokens at each step.
  • Parametric Injection: Language vectors are injected at the final layer after randomly selecting and translating into a representative language, adding language-specific expression and reasoning tendencies.The base LLM remains frozen while trainable language vectors approximate cultural and worldview-related value orientations.
  • Parametric Injection: Gaussian perturbations model individual differences among agents sharing demographic attributes, while noise magnitude is calibrated to avoid disrupting reasoning or demographic consistency.The perturbation standard deviation controls variability and should be selected separately according to model sensitivity.

4 Experimental Setup

The experiments evaluate PSII on World Values Survey opinion data across four instruction-tuned LLMs, using group-level distributional accuracy and diversity metrics. The setup compares PSII with multiple diversity and persona baselines and calibrates Gaussian noise according to model sensitivity.

  • Dataset: The study uses WVS responses Q1–Q259 as opinion targets and Q260–Q290 for demographic feature modeling and identity alignment.The 259 opinion questions are regrouped into four theory-driven thematic categories.
  • Evaluation Metrics: Performance is evaluated by comparing model-generated and human response distributions using KL divergence, where lower values indicate better alignment.The evaluation is conducted at the group level.
  • Evaluation Metrics: Entropy Deviation measures the absolute difference between normalized model and human response entropy, with lower values indicating closer diversity matching.This metric evaluates whether simulated response diversity resembles the human distribution.
  • Baselines: The comparison includes Direct, High-Temp, Multilingual, DivReq, and persona-engineering baselines designed to vary output diversity or approximate respondents.Implementations of the representative baselines are detailed in the appendix.
  • Models and Sampling: Experiments use Qwen2.5-7B, Qwen2.5-14B, Llama-3.1-8B, and Mistral-24B, sampling 100 individuals from the 97,220-person WVS dataset.Additional resampling experiments are reported as confirming robustness to random sampling variation.
  • Noise Calibration: Noise is calibrated using model sensitivity, defined through ranking changes in answer options under perturbation; higher sensitivity requires a smaller noise standard deviation.When sensitivity exceeds 0.4, the calibration sets σ=0.

5 Experiments and Results

Across WVS experiments, PSII achieves a strong accuracy–diversity trade-off across models and question categories, while response distributions more closely match human survey diversity. Ablations indicate that its modules contribute complementarily, but performance and diversity remain sensitive to question type and model scale.

  • Main Results: PSII consistently achieves the best trade-off between simulation accuracy and diversity across models, parameter settings, and question categories.It substantially narrows the gap between simulated and human survey responses.
  • Main Results: After PSII, all evaluated models show marked gains in accuracy and diversity, with Llama-3.1-8B benefiting most, followed by Qwen2.5-7B.Under direct prompting, Mistral-24B is closest to human data, followed by Llama-3.1-8B and the Qwen models.
  • Main Results: PSII performs best on Economic Progress questions, while Beliefs & Life questions remain relatively weaker.The authors suggest economic evaluations may align more naturally with PSII’s representation shifts, whereas beliefs and values are harder to simulate faithfully.
  • Response Distribution Analysis: PSII produces more evenly distributed responses that more closely match human data, whereas baselines concentrate responses on fewer options.The comparison uses randomly selected questions from four categories and 100 simulated agents per method.
  • Ablation Studies: Removing any single PSII component reduces simulation accuracy, and most ablations also degrade diversity matching.Demographic vectors are most critical for both metrics, while parametric noise is especially important for diversity.
  • Ablation Studies: PSII’s component effects are broadly consistent across LLM backbones, supporting robustness across different models.The authors attribute performance to complementary identity and value representations, injected noise, and layered interventions.
  • Ablation Studies: Injecting all vectors at approximately 70% of network depth achieves the best overall performance in the fixed-layer comparison.For individual demographic attributes, the optimal injection layer varies and is selected by minimizing KL divergence.

6 Ethical Concerns and Societal Implications

PSII raises ethical and societal concerns involving privacy, demographic stereotyping, misuse, and interpretation. The paper recommends safeguards and limits PSII-generated simulations to supplementary use in controlled research settings.

  • Experiments use publicly available value datasets and avoid accessing or inferring private individual data, but extensions require privacy-preserving practices.
  • Demographic modeling may introduce stereotyping or oversimplification, requiring careful use of established survey definitions and group-level analysis.
  • PSII is intended for controlled research settings rather than real-world decision-making.
  • Unregulated synthetic-agent deployment could enable propaganda or social manipulation, motivating access control, transparency reports, and usage auditing.
  • PSII simulations should supplement rather than definitively represent societal opinions, with attention to cultural nuances, minority representation, and model bias.

7 Conclusion

The study identifies Diversity Collapse in LLM-based public opinion simulation and proposes PSII to address it through representation-level identity control. Experiments on WVS show improved distributional fidelity and diversity.

  • Diversity Collapse is characterized by inter-group homogenization and insufficient intra-group representativeness in synthetic populations.
  • Upper-layer representation contraction causes distinct social identities to converge toward a uniform state at the end of the reasoning chain.
  • PSII embeds parametric demographic and value-orientation vectors into intermediate hidden states for stable, fine-grained identity modulation.
  • WVS experiments demonstrate that PSII enhances simulation distributional fidelity and diversity.

A Limitations

PSII's effectiveness is bounded by the available demographic coverage and by the scope of identity attributes represented in its current vectors.

  • Synthetic-population quality is constrained by the granularity and completeness of available demographic data.
  • Rare or underrepresented groups may remain insufficiently modeled, limiting simulation fidelity for those populations.
  • Current PSII vectors primarily encode coarse-grained demographic attributes and value orientations.
  • Fine-grained personality traits, dynamic opinion changes, and context-specific behavioral nuances require additional mechanisms because they are not explicitly captured.

B Baseline Implementation Details

The experiments compare population-simulation baselines spanning direct prompting, sampling variation, multilingual prompting, explicit diversity requests, demographic persona prompts, and activation steering. These implementations establish references for evaluating PSII in fixed-choice WVS settings.

  • Baseline selection: Baseline selection focuses on methods applicable to population-level simulation and structured or semi-structured WVS questions.
  • Direct: Direct prompts generate survey responses without explicit diversity control or identity conditioning.
  • High-Temp: High-Temp uses the Direct strategy with sampling temperature 2 to test whether stochastic decoding induces population-level diversity.
  • Multilingual: Multilingual randomly selects Arabic, English, Spanish, Russian, or Chinese prompts using fixed random seed 42.
  • DivReq: DivReq augments Direct with an instruction encouraging diverse outputs, testing prompt-level diversity control.
  • PE (Prompt Engineering): PE converts each WVS subject's structured demographic information into natural-language profiles used to guide simulated responses.
  • PSII comparison: PSII retains prompt-level demographic injection while adding representation-level identity modeling, unlike prompt conditioning alone.
  • SimVBG: SimVBG generates candidate responses independently across cognitive, affective, and behavioral dimensions before aggregating them.

C.1.1 Dataset Overview.

The WVS provides a cross-national benchmark for modeling values, beliefs, social attitudes, and demographic heterogeneity. The paper reorganizes its questions and demographic information to construct value and identity representations, then tests the robustness of demographic-vector construction.

  • Dataset Overview: The WVS is a large-scale, cross-national survey covering human values, beliefs, social attitudes, and demographic characteristics.
  • Dataset Overview: The original WVS questionnaire groups questions into thematic categories mapped to contiguous question identifiers.
  • Value Modeling: The paper reorganizes the original categories into four higher-level semantic groups guided by conceptual coherence and prior sociological and political-science theory.
  • Identity Modeling: Demographic questions Q260–Q290 are used for profile descriptions, while a subset supplies demographic vectors for identity modeling.
  • Robustness Analysis: The demographic-vector construction remains stable across instruction models, prompt templates, and random seeds, with average KL divergence 0.5017 and average ED 0.0236.

D.2 Sampling Robustness

Sampling robustness tests assess whether PSII’s results depend on the randomly selected WVS respondents. Across five independent samples, performance remains consistent, while supplementary analyses report gains over baselines and degradation when core components are removed.

  • Sampling Robustness: The main experiments sample 100 respondents from 97,220 WVS respondents to balance computational cost with comparability to prior baselines.
  • Sampling Robustness: Five independent 100-person samples produce average KL divergence 0.4732 and average ED 0.0290, with small variances of 6.39 × 10^-4 and 1.5 × 10^-5.
  • Supplementary Results: Across four WVS question categories and overall, PSII consistently outperforms baselines on JS divergence and MAE.
  • Ablation Analysis: Removing any core PSII module degrades JS divergence and MAE, confirming that each component contributes to distributional fidelity.

E.3 Layer-Wise Injection Sensitivity Analysis

The layer-wise sensitivity analysis shows that demographic attributes perform best when assigned to calibrated network layers rather than injected arbitrarily or at one shared depth. Optimal layer configuration achieves the strongest combined accuracy and diversity, while deviations cause measurable losses.

  • Layer-Wise Sensitivity: The analysis compares optimal, random, shifted, and single-layer assignments of demographic vectors across Transformer depths.
  • Sensitivity Results: Optimal Layer Configuration achieves the best overall performance, while nearby global shifts remain competitive.
  • Sensitivity Results: Random assignments substantially degrade accuracy and diversity, and single-layer strategies underperform the optimal configuration.
  • Representation Analysis: Across four open-source LLMs, PSII preserves or increases hidden-state dispersion in higher layers compared with baseline methods.
Loading 2603.16142v2…