Source-linked AI summary
CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations
Myra Cheng, Tiziano Piccardi, Diyi Yang
TL;DR
LLM simulation evaluation has limited means to assess open-ended responses and concerns remain about caricatured portrayals. The paper presents CoMPosT and a method based on individuation and exaggeration; it finds greater caricature susceptibility for certain GPT-4 demographics and general, uncontroversial topics.
Problem
Evaluation of LLM simulations has relied on replication or believability, leaving open-ended generations difficult to assess and believability vulnerable to human judgment biases.
Source: §1 · p. 1Method
Results
GPT-4 simulations of political and marginalized race/ethnicity groups and of general, uncontroversial topics were more susceptible to caricature.
Source: §1 · p. 2, §6.2.1 · p. 7Takeaways & Limitations
Caricature can limit simulations’ ability to capture human nuance and perpetuate misleading descriptions and stereotypes about demographic groups.
Source: §1 · p. 2The metric measures susceptibility to caricature rather than comprehensive simulation quality and may yield false positives for simulations with other problems.
Source: §10 · p. 10
Abstract
from arXiv · showhide
Recent work has aimed to capture nuances of human behavior by using LLMs to simulate responses from particular demographics in settings like social science experiments and public opinion surveys. However, there are currently no established ways to discuss or evaluate the quality of such LLM simulations. Moreover, there is growing concern that these LLM simulations are flattened caricatures of the personas that they aim to simulate, failing to capture the multidimensionality of people and perpetuating stereotypes. To bridge these gaps, we present CoMPosT, a framework to characterize LLM simulations using four dimensions: Context, Model, Persona, and Topic. We use this framework to measure open-ended LLM simulations' susceptibility to caricature, defined via two criteria: individuation and exaggeration. We evaluate the level of caricature in scenarios from existing work on LLM simulations. We find that for GPT-4, simulations of certain demographics (political and marginalized groups) and topics (general, uncontroversial) are highly susceptible to caricature.
1 Introduction
The paper addresses the lack of comprehensive ways to evaluate LLM simulations, especially open-ended responses. It introduces CoMPosT and a caricature metric, finding that some GPT-4 demographic simulations and general, uncontroversial topics are more susceptible.
1 Introduction
- Existing evaluation relies on replication or believability, which do not adequately assess open-ended generations or avoid the biases of human judgment.Replication is limited to reproducing known behavior, while believability judgments can reflect stereotypes and other biases.Source: §1 · p. 1
- The proposed caricature metric tests individuation from a topic-default response and exaggeration of persona characteristics over topic characteristics.The paper describes caricature as an exaggerated persona narrative rather than a meaningful response to the topic.Source: §1 · p. 2
- GPT-4 simulations of political and marginalized race/ethnicity groups, and of general, uncontroversial topics, were more susceptible to caricature.The paper evaluates scenarios from existing LLM simulation work.Source: §1 · p. 2
- For the topic Computers and Electronics, GPT-4’s nonbinary-person responses focus on identity issues, unlike the topical responses for a person.The caption describes the nonbinary-person outputs as constructing a homogeneous narrative centered only on LGBTQ+ activism.Source: Figure 2 · p. 2
- The paper contributes CoMPosT, a four-dimension framework, a caricature measure, experiments across contexts, and recommendations for LLM simulation.The framework’s dimensions are Context, Model, Persona, and Topic.Source: Table 1 · p. 1, §1 · p. 2
2 CoMPosT: Taxonomizing Simulations
CoMPosT provides a shared vocabulary for characterizing LLM simulations through Context, Model, Persona, and Topic. The framework supports describing existing work and examining how these dimensions relate to caricature.
2 CoMPosT: Taxonomizing Simulations
- CoMPosT characterizes simulations along Context, Model, Persona, and Topic.Context, persona, and topic are specified in the prompt; the model is determined externally.Source: §2 · pp. 2–3
- Context covers the situation’s norms, prompt phrasing, and requested response granularity, including multiple-choice or open-ended output.The framework treats formal interviews and online comments as examples of differing contexts.Source: §2 · p. 3
- Existing simulation studies typically select a model and context, then vary persona and topic; the paper uses these dimensions to analyze caricature.The scenario notation includes persona, topic, and context.Source: §2 · p. 3
3 Background: Caricature
The paper defines caricature as exaggeration that makes a depiction meaningfully distinguishable, and discusses its connection to stereotypes and homogeneous narratives.
3.1 Definition of Caricature
- Caricature exaggerates a subject’s defining characteristics in a way that differentiates it from others while retaining some fidelity to those characteristics.In LLM simulations, it means exaggerating persona characteristics beyond a meaningful, topical response; individuation alone does not establish caricature.Source: §3.1 · p. 3
3.2 Implications of Caricature
- Caricatures can depict stereotypes or foster homogeneous narratives that omit within-group diversity, a concern linked to harms including misprediction and medical misdiagnosis.The paper distinguishes caricature from stereotypes, noting that a caricature may depict a stereotype but need not contain one.Source: §3.2 · p. 4
4 Caricature Detection Method
The method measures caricature through a sequential test of individuation and exaggeration, using persona- and topic-specific defaults as comparison points. Exaggeration is assessed by comparing outputs with an axis built from defining characteristics of persona and topic.
4 Caricature Detection Method
- The method first tests individuation and only measures exaggeration if the simulation can be individuated.Individuation is necessary for caricature, so the procedure can stop after that step when outputs are not distinguishable.Source: §4 · p. 4, Figure 3 · p. 6
4.1 Defining Defaults
- The defaults are model- and context-specific simulations without a specified persona or topic, used to isolate the characteristics of those dimensions.They are comparison points, not universal defaults.Source: §4.1 · pp. 4–5
4.2 Measuring Individuation
- Individuation is measured by whether a classifier can distinguish target outputs from default outputs; outputs that are not differentiable cannot be caricatures.The paper reports classifier test accuracy using contextualized embeddings.Source: §4.2 · p. 5
4.3 Measuring Exaggeration
- The persona-topic semantic axis is built from seed words that distinguish persona-only from topic-only defaults, with each pole represented by mean word embeddings.The seed words are selected using statistically significant weighted log-odds ratios.Source: §4.3 · p. 5
- Exaggeration is the average cosine similarity of simulation outputs to the axis, normalized against the default-persona and default-topic simulations.The method uses this score to assess persona exaggeration relative to topic characteristics.Source: §4.3 · p. 5
5 Experiments
The experiments test caricature in online-forum and interview simulations, using GPT-4 and varying personas and topics. The settings adapt prior simulation work while sampling topic specificity and controversy in forums and contentious survey questions in interviews.
5 Experiments
- Experiments use GPT-4 and generate 100 outputs per simulation setting, averaging results across outputs.The authors report that older and open-source models yielded less realistic, lower-quality simulation outputs.Source: §5 · p. 6
5.1 Online Forum
- Forum simulations follow Park et al.’s prompting format and vary 15 personas across 30 topic pairs.Personas include race/ethnicities, genders, political ideologies, ages, and a neutral person; paired topics vary in specificity and controversy.Source: §5.1 · p. 6
- Forum topics sample WikiHow categories and ProCon.org debate topics, with general and more specific wording used for each sampled subject.The WikiHow sample uses 15 popular categories and an associated how-to for each; ProCon.org contributes 15 randomly sampled debate topics.Source: §5.1 · p. 6
5.2 Interview
- Interview simulations reproduce Santurkar et al.’s public-opinion survey setting, converting 30 contentious Pew questions to open-ended prompts and using the same 15 personas.The prompt asks respondents to describe their identity and answer a question; multiple-choice options are removed.Source: §5.2 · pp. 6–7
6 Results and Discussion
The analysis finds that caricature varies with topic and persona, while individuation from the default-persona is generally possible. It uses exaggeration as a proxy for caricature and cautions that low caricature scores do not establish that simulations are stereotype-free.
6.1 Simulations of all personas can be individuated from the default-persona
- Every persona is meaningfully individuated from the default-persona above random chance, but interview scores above 0.95 are not useful for comparing personas.Woman and man personas have lower mean individuation scores in the online forum; lower output variability in interviews makes personas easier to individuate.Source: §6.1 · p. 7, Figure 4 · p. 8
6.2 Exaggeration scores reveal the personas and topics most susceptible to caricature
- Because almost all simulations can be individuated, the analysis uses exaggeration scores as a proxy for caricature.Exaggeration measures similarity between simulations and their corresponding persona-topic axes.Source: §6.2 · p. 7
6.2.1 Caricature ↑: Topic specificity ↓
- In the online forum, more general topics have higher caricature rates than specific topics, and the pattern holds as specificity increases for Health.General uncontroversial topics have the highest exaggeration scores; the inverse relationship with specificity also appears in interview and Twitter contexts.Source: §6.2.1 · p. 7
6.2.2 Caricature ↑: Political ideology, race, and marginalized personas
- Political-leaning, non-white, and nonbinary personas, alongside general and uncontroversial topics, broadly show the highest caricature rates.In the online forum context, nonbinary, Black, Hispanic, Middle-Eastern, and conservative personas have the highest mean exaggeration; Asian and woman personas have relatively low rates, and some persona-topic combinations are especially susceptible.Source: §6.2.2 · pp. 7–8, Figure 5 · p. 8
- Conservative and liberal personas have the widest gap between uncontroversial and controversial topic scores in online forums.In interviews, the highest mean exaggeration scores include nonbinary, Hispanic, 80-year-old, conservative, and Middle-Eastern personas; Twitter results also identify particularly susceptible persona-topic combinations.Source: §6.2.2 · pp. 7–8
6.3 Stereotypes
- Low caricature scores do not rule out stereotypes, because the measure captures persona exaggeration in a setting rather than cataloguing stereotypes.The paper notes gender stereotypes in simulated woman responses despite their low caricature scores.Source: §6.3 · p. 8
7 Recommendations
The recommendations urge researchers to assess simulations in their specific contexts, document positionality, and attend to variation and subgroup differences. They favor more specific topics as a general way to mitigate caricature, while emphasizing care when simulating politicized or marginalized groups.
Recommendations
- Researchers should test simulations in their particular context and critically examine whether they illuminate the intended phenomenon.The relationship between topic, persona, and context in caricature is described as nuanced.Source: §7 · p. 9
- More specific topics are generally encouraged to mitigate caricature, with particular care urged when simulating politicized or marginalized groups.The recommendation is qualified by the nuanced relationship among topic, persona, and context.Source: §7 · p. 9
Documenting Positionality
- Researchers are encouraged to document their positions, motivations, and process because identity and perspectives may influence simulation research.The paper also recommends reporting how researchers’ identities and perspectives may influence their work.Source: §7 · p. 9
Understanding Difference
- Understanding individual variation and using more subtle aggregation can help prevent minority opinions and subgroup disparities from being overlooked.A proposed future direction is injecting variation and using multifaceted personas to capture meaningful differences rather than shallow generalizations.Source: §7 · p. 9
8 Positionality
The authors state that their perspectives shaped the paper and disclose aspects of their identities and institutional setting.
8 Positionality
- The authors explicitly acknowledge that their positionality shaped the perspectives introduced in the paper.The disclosure identifies Myra Cheng as a Chinese-American woman and describes the authors’ academic roles and departmental composition.Source: §8 · p. 9
- Myra Cheng identifies as a Chinese-American woman.Source: §8 · p. 9
- The authors identify their roles as PhD student, postdoctoral scholar, and professor in Stanford Computer Science, a predominantly male and white/Asian department.Source: §8 · p. 9
9 Ethical Considerations
The authors oppose harmful LLM simulation applications and warn that their caricature measure cannot establish that a simulation is acceptable or high quality. They also connect implicit defaults and gaps in empirical data to concerns about future caricature.
9 Ethical Considerations
- The authors oppose harmful applications of LLM simulations and research that could enable bad actors without guardrails.They describe CoMPosT as a shared language for critique and suggest that people might agree to avoid simulating certain topics, personas, and contexts.Source: §9 · p. 9
- A lack of caricature under their measure does not establish that a simulation is acceptable or high quality.The measure identifies a concerning limitation, not a comprehensive assessment.Source: §9 · p. 9
- The least-caricatured personas are also those other work has found to be implicit defaults in LLMs, which may shift with prompts, context, and social factors.The authors raise concern that training future models on generated data could further amplify caricatures; empirical studies may underrepresent many demographics, including non-Western populations and groups beyond American college students.Source: §9 · pp. 9–10
10 Limitations
The authors present caricature detection as a limited evaluation, not a comprehensive measure of simulation quality. Their study focuses on subpopulation personas and one-round responses, while noting that the framework could extend to more complex settings.
10 Limitations
- The measure assesses susceptibility to caricature, not overall simulation quality, and may produce false positives when simulations have other problems.A simulation that appears acceptable and caricature-free under this method may still have other problems.Source: §10 · p. 10
- Avoiding caricature is necessary but insufficient for simulation quality, so the metric should be used alongside other evaluations.The authors describe this work as a pilot and hope it lays groundwork for more comprehensive evaluation, potentially with human evaluation.Source: §10 · p. 10
- The study focuses on personas reflecting subpopulations and one-round response formats, although the framework could extend to more complex simulations.For multi-round settings, the authors suggest using the full text across rounds or splitting it into parts depending on length and structure.Source: §10 · p. 10
- The semantic-axis Pp characterizes words distinguishing a persona simulation in a particular context, not a universal representation of that demographic group.The authors state that their work is not a comprehensive evaluation of stereotypes or demographic representations.Source: §10 · p. 10
A Examples of Caricatures in Simulation
The examples show caricature when simulated nonbinary responses foreground generic identity and inclusion themes instead of the requested topic, unlike the more topical man responses. The accompanying analyses test whether individuation patterns hold across measures and contexts and whether topic specificity relates to exaggeration.
A Examples of Caricatures in Simulation
- Nonbinary responses to guitar and vehicle topics foreground generic identity, authenticity, and inclusion themes, while man responses remain topical and do not exaggerate persona traits.The paper describes this contrast as an unrealistic caricature that narrows the range of possible nonbinary responses.Source: A Examples of Caricatures in Simulation · pp. 14–15
B Robustness of Individuation Measure
- Across alternative individuation measures, all interview personas remain individuable, while marginalized online-forum personas are easiest to distinguish and man and woman personas remain challenging.The alternatives use unsupervised V-measure differentiation or all-distilroberta-v1 embeddings; their broad patterns resemble the main result.Source: B Robustness of Individuation Measure · p. 15, Figure A1 · p. 16
C Internal and External Validation of Semantic Axes
- Persona and topic seed-word sets are distinguishable on each semantic axis, and leave-one-out validation finds higher similarity to the original pole for every constructed axis.The race/ethnicity and nonbinary persona seed words also reflect stereotypes reported by Cheng et al. (2023b).Source: C Internal and External Validation of… · pp. 15–16
D Experimental Details: Topics and Personas
- The study varies age, political ideology, race/ethnicity, and gender personas, pairing persona and topic prompts in online-forum and interview settings.The online-forum prompts also include default-topic and default-persona variants; the supplied interview passage points to its prompt details in Table A5.Source: D Experimental Details: Topics and… · p. 16, §D.1 · p. 16, §D.2 · p. 17
D.1.1 Fine-Grained Specificity Experiment
- Exaggeration is negatively correlated with topic specificity: the broadest health topic has the highest caricature rate, while Level 5 topics come from AskReddit search results.The authors constructed the intermediate specificity levels by interpolating between these endpoints.Source: §D.1.1 · pp. 16–17, Figure A2 · p. 17
E Power Analysis
- With 100 samples per setting, the chosen sample size exceeds the power-analysis requirements of 28 for individuation and 41 for exaggeration.The reported analyses use desired power 0.8 for both metrics.Source: E Power Analysis · p. 17
F Influence of the Context Dimension
- Switching online-forum and interview contexts preserves caricature trends for the same topics, although exaggeration scores are slightly higher in the switched settings.The authors suggest memorization dynamics may contribute to the difference, but identify this as a question for future work.Source: F Influence of the Context Dimension · p. 17, Figure A3 · p. 18
G Twitter Context
- Twitter-context experiments corroborate the paper’s main trends, using Republican and Democrat personas and 30 topics drawn from people and groups.The prompt follows the form “[persona] user posted on Twitter: [topic] is/are.”Source: G Twitter Context · p. 17, §G.1 · p. 18
G.2 Results
In the Twitter setting, Republican and Democrat simulations are highly individuable, while exaggeration varies with both persona and topic. Interview exaggeration is highest for questions about military confidence, social class, and community ties.
G.2 Results
- 0.94 and 0.88 are the average individuation scores for Democrat and Republican personas, respectively, in the Twitter context.Source: §G.2 · p. 18
- Republican personas have higher mean exaggeration in Twitter simulations; broad race and group topics score higher than specific public figures, with race topics especially high for Democrat personas.Republican public-figure topics yield more exaggeration than Democratic public-figure topics; Dr. Anthony Fauci has the lowest rate among the public figures considered.Source: §G.2 · p. 18, Figure A4 · p. 19
H Interview Setting Result Details
- The interview questions with highest exaggeration scores concern confidence in the military, self-identified social class, and the importance of community ties.Source: H Interview Setting Result Details · pp. 18–19, Figure A5 · p. 19