PaperlayerView exact revision on arXiv

Source-linked AI summary

MetaPersona: Task-Grounded Synthetic Populations from Empirical Social Science

Jinyi Ye, Yuangang Li, Chenxiao Yu, Preyashi Poddar, Priyanka Dey, Longtian Ye, Zihan Wang, Xiyang Hu, Emilio Ferrara, Yue Zhao

arXiv:2609.38392v1cs.AI

TL;DR

Persona generation lacks a principled basis for choosing task-relevant attributes and representing their empirical dependencies. MetaPersona-DB and MetaPersona use empirical studies to construct task-specific persona graphs and sample populations; results are strongest for misinformation belief and AI-tool sentiment, while income-redistribution findings are mixed.

  • Problem

    Existing persona methods often use fixed or LLM-nominated schemas, leaving unresolved which attributes a task-specific persona should include and how to represent empirical population distributions and dependencies.

    Source: §1 · p. 1

  • Method

    MetaPersona-DB organizes empirical studies, and MetaPersona retrieves task-relevant evidence to build persona dependency graphs and sample synthetic populations using empirical priors.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    Gains are strongest for misinformation belief and AI-tool sentiment, while income-redistribution results are mixed; on Stack Overflow with GPT-5.2, quadratic-weighted kappa rose from the best baseline value of 0.073 to 0.320.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    Faithful synthetic populations require the right attributes and their empirical dependencies, and grounding them in measured social-science evidence moves persona generation toward a principled approach.

    Source: §7 · p. 8

    MetaPersona simulations reflect aggregate patterns rather than lived experience and should not be treated as evidence about actual individuals or as a substitute for engagement with affected communities on consequential decisions.

    Source: Ethical considerations · p. 9

Abstract

from arXiv · show

Personas used to seed LLM social simulations face a cold-start problem: existing methods lack a principled basis for deciding which attributes to include and how to assign their values. As a result, synthetic populations may misrepresent the demographic composition, latent attributes, and dependency structure that shape downstream behavior. We introduce MetaPersona-DB, a dataset of 11,000+ empirical human-subjects studies annotated with task-relevant variables, reported relationships, and aggregate-level population statistics. Building on this resource, we propose MetaPersona, a framework that retrieves task-relevant evidence, constructs literature-derived persona dependency graphs, and samples synthetic populations from empirical priors linking demographics, latent attributes, and outcomes. Across three downstream case studies, three baselines, and three frontier models, results vary by task and model: MetaPersona performs strongly on misinformation belief and AI-tool sentiment, while results on income redistribution are mixed. It also reduces persona-construction cost to under $0.5 per task using GPT-5.2. Finally, we present MetaPersona-Studio, a prototype interactive interface for empirically grounded persona generation.

1 Introduction

Persona generation lacks a principled way to choose task-relevant attributes and represent their empirical dependencies. MetaPersona-DB and MetaPersona ground attribute selection and population sampling in social-science evidence, with downstream results varying by task and model.

1 Introduction

  • Current persona methods leave unresolved which task-specific attributes to include and how to structure them around real population distributions and empirical dependencies.The paper calls this the persona cold-start problem; existing pipelines often start from a fixed or LLM-nominated schema chosen independently of the downstream task.Source: §1 · p. 1
  • MetaPersona-DB curates 11,000+ Q1-journal human-subjects studies, and MetaPersona retrieves task-relevant evidence to build dependency graphs and sample populations from empirical priors.The graph links demographic and latent factors to outcomes, specifying task-relevant attributes and values that reflect population structure.Source: §1 · p. 2
  • Results vary by task and model: MetaPersona gains are strongest for misinformation belief and AI-tool sentiment, while income-redistribution results are mixed and sometimes favor baselines.The evaluation spans three case studies, three baselines, and GPT-5.2, Claude Haiku 4.5, and DeepSeek-V3.2, measuring response accuracy, distributional alignment, and structural realism.Source: §1 · p. 2
  • Persona-based simulation is costly and technically demanding for interdisciplinary communities, motivating the low-cost interactive MetaPersona-Studio prototype.The manuscript describes the work as in progress and says code and a public demo will be released with a later version.Source: §1 · pp. 1–3
  • MetaPersona generates a full population for under $0.5 per task, 10–50× below other LLM-based baselines, and sampling cost is essentially independent of population size after graph construction.The authors also present MetaPersona-Studio, a prototype interactive interface designed for interdisciplinary users.Source: §1 · p. 3

2 Related Work

Prior persona-generation work has improved assigned values and narrative richness, but often leaves the attribute schema fixed rather than grounding latent attributes in task-relevant empirical evidence.

2 Related Work

  • Scalable persona generation has been associated with systematic biases, flattened identity groups, and regression toward typical profiles, reflecting a “demographics-in, behavior-out” approach.This approach moves from coarse demographics to predicted outcomes.Source: §2 · p. 3
  • Distribution alignment and free-text backstories improve assigned values or richness, but do not determine which latent attributes predict a task outcome (Hu et al., 2025; Park et al., 2024).Hu and Collier (2024) report that persona prompting helps only when selected attributes predict the outcome, while existing persona variables explain under 10% of annotation variance.Source: §2 · p. 3
  • MetaPersona extends task-sensitive persona generation by retrieving empirical studies to derive latent attributes and sample their values conditioned on demographics under literature-derived priors.The schema itself is grounded in measured social-science evidence before values are assigned.Source: §2 · p. 3

3 METAPERSONA-DB: A Variable-level Empirical Social Science Dataset

MetaPersona-DB organizes empirical human-subjects research into variable-level evidence for persona generation, including variable descriptions, distribution shapes, and relationships between variables.

3 METAPERSONA-DB: A Variable-level Empirical Social Science Dataset

  • MetaPersona-DB is motivated by the need to systematically mine social-science evidence about which attributes matter for personas.The dataset is described as variable-level and distilled from thousands of empirical human-subjects studies.Source: §3 · p. 3

3.1 Paper Selection

  • The corpus selects post-2000, Q1 JCR journal studies with at least 20 citations across six social-science disciplines.The selected disciplines are psychology, political science, economics, sociology, communication, and multidisciplinary sciences.Source: §3.1 · p. 3

3.2 Identifying Empirical Human Studies

  • The dataset includes empirical human-involving studies and systematic reviews or meta-analyses, yielding 11,638 studies.GPT-5.2 classified studies with a structured prompt, and two independent human annotators validated that classification.Source: §3.2 · p. 4

3.3 Variable and Path Extraction

  • Annotations identify individual demographic, psychographic, and behavioral variables, assign coarse distribution shapes, and encode directed paths with relationship direction and effect-size bands.When abstracts omit strength, a model-elicited prior is used; human evaluators audit samples of full-text papers, and repeated paths are merged into consensus estimates accounting for between-study heterogeneity.Source: §3.3 · p. 4

4 METAPERSONA: Task-Grounded Synthetic Population Generation

MetaPersona builds synthetic populations through a five-stage pipeline, from retrieving task-relevant studies to generating demographic-conditioned latent attributes.

4.1 Literature Retrieval and Ranking

  • Task-query retrieval combines semantic similarity and keyword overlap, then an LLM reranks the top 100 papers to retain 20.The keyword-overlap weight is λ=0.05.Source: §4.1 · pp. 4–5

4.2 Persona Ontology Graph Construction

  • Extracted variable paths form a task-specific graph whose weighted, signed edges summarize support and retain demographic-to-latent and latent-to-outcome links.Edge weights count supporting paths; edge signs use the majority-vote direction.Source: §4.2 · p. 5

4.3 Latent Attribute Selection

  • Centrality favors latent attributes linked to both demographics and the task outcome, and the top K_Z form each persona’s latent schema.The passage gives K_Z=3 as an example.Source: §4.3 · p. 5

4.4 Demographic Base Population Sampling

  • The base population samples demographic records from the task’s ground-truth survey or a specified census microdata source, preserving the empirical joint distribution.For non-benchmarked tasks, the described sources include U.S. and cross-national public-use microdata.Source: §4.4 · p. 5

4.5 Population-Conditioned Attribute Generation

  • A Gaussian copula samples latent attributes conditioned on demographics while separating their marginal distributions from dependence, avoiding per-agent LLM calls.Generation is mostly statistical after the graph is constructed.Source: §4.5 · pp. 5–6

5 Evaluation of METAPERSONA on Downstream Tasks: Three Case Studies

The evaluation compares METAPERSONA with three persona-construction baselines on three social-science tasks, using individual accuracy, population-distribution fit, and structural realism. Results vary by task and model: METAPERSONA performs strongly on MIST, but ISSP findings are mixed.

5 Evaluation of METAPERSONA on Downstream Tasks: Three Case Studies

  • The evaluation compares METAPERSONA with existing persona-construction practices across three case studies and three frontier LLMs.Each case study is evaluated against individual-level human ground truth.Source: §5 · p. 6

5.1 Task Selection and Data Sources

  • The three tasks were selected for relevant empirical attribute–outcome evidence and individual-level surveys that match synthetic personas to observed respondents.The tasks span disciplines and outcome types; demographics come from each ground-truth dataset.Source: §5.1 · p. 6

5.4 Aggregated Results and Cross-Task Analysis

  • METAPERSONA achieves the best MIST RMSE/MAE under all three models and the best Bias, W-dist, and KS, while ISSP distributional results are mixed.On ISSP, METAPERSONA has the lowest W-dist only with DeepSeek-V3.2; LLM-HEURISTIC leads with GPT-5.2 and Claude Haiku 4.5, and METAPERSONA leads on no reported ISSP metric with GPT-5.2. Table 2 reports results across the methods, tasks, and backbones.Source: Table 2 · p. 7, §5.4 · pp. 7–8

5.2 Baseline Construction

  • The baselines are demographics-only, generated life narratives, and LLM-inferred latent attributes; METAPERSONA instead selects and assigns latents using database evidence.LLM-HEURISTIC and METAPERSONA infer the same number of latent attributes (K=3).Source: §5.2 · p. 7

5.3 Realism Metrics

  • Realism is assessed through individual-response accuracy, distributional alignment, and preservation of literature-supported relationships.Individual metrics vary by outcome type, distributional measures test response-shape fit, and structural realism tests whether predicted outcomes recover supported associations.Source: §5.3 · p. 7

5.4 Aggregated Results and Cross-Task Analysis

  • On SO, QWK rises from the best baseline value of 0.073 to 0.320 with GPT-5.2, and from 0.089 to 0.402 with DeepSeek-V3.2.For ISSP individual accuracy, METAPERSONA has the lowest MAE with Claude Haiku 4.5 and DeepSeek-V3.2 but not GPT-5.2; W-1 Acc and QWK leaders also vary by backbone.Source: §5.4 · p. 7
  • METAPERSONA is closest to ground-truth correlation on two diagnostic paths, with |∆r| of 0.020 for MIST and 0.036 for SO.The paths test different structures: CRT to misinformation discernment for MIST, education to redistribution support for ISSP, and seniority to sentiment for SO.Source: §5.4 · p. 8

6 Application: METAPERSONA-STUDIO

METAPERSONA-STUDIO is a work-in-progress interface for specifying a task and grounding population, then inspecting or exporting generated personas and their graph.

6 Application: METAPERSONA-STUDIO

  • Researchers specify a task query and grounding population; the prototype retrieves relevant literature, builds a persona graph, surfaces latent attributes, and generates a customizable population for inspection and export.Demographic filters include country, age, education, and employment.Source: §6 · p. 8

7 Discussion and Conclusion

MetaPersona’s gains are strongest where outcomes depend on latent attributes that demographics alone cannot capture, while methods converge on income redistribution. The paper presents its evidence-grounded population-generation pipeline as a move from heuristic persona construction toward a more principled approach.

7 Discussion and Conclusion

  • MetaPersona-DB distills 11,000+ human-subjects studies into variable-level structure that MetaPersona uses to retrieve task-relevant attributes and sample populations under literature-derived priors.Source: §7 · p. 8
  • Table 3 compares Pearson r along one diagnostic path per task with ground-truth correlations, using absolute deviation from ground truth as the fidelity measure.Lower |∆r| indicates closer agreement; the caption gives no cell values.Source: Table 3 · p. 8
  • On Stack Overflow with GPT-5.2, quadratic-weighted kappa rises from the best baseline value of 0.073 to 0.320, while methods converge on income redistribution support on ISSP.The authors report no MetaPersona advantage where the education–redistribution relationship is weak.Source: §7 · p. 8
  • The conclusion argues that faithful synthetic populations require the right attributes and their empirical dependencies, not merely richer descriptions.Source: §7 · p. 8

Limitations and Future Work

The evidence base and demographic substrate set boundaries on MetaPersona’s coverage, with broader disciplines, venues, and dependency structures identified as future directions.

Limitations and Future Work

  • The evidence base covers six social science disciplines and journal articles, excluding other JCR fields and archival venues such as HCI conferences.Source: Limitations and Future Work · p. 9
  • Extracting variables from titles and abstracts favors primary, well-powered effects, while access constraints exclude many studies that are not open access.The authors propose broader corpus coverage and full-text extraction where licensing permits.Source: Limitations and Future Work · p. 9
  • The current demographic grounding assumes a sampling frame based on observable attributes, and the dependency graph does not yet capture richer moderation or context-dependent effects.The authors identify validating the substrate on nonbenchmarked tasks and extending the graph as future work.Source: Limitations and Future Work · p. 9

Ethical considerations

MetaPersona is intended to support rather than replace human-subjects research, and its simulations have ethical boundaries in use and interpretation.

Ethical considerations

  • Simulated responses reflect aggregate patterns, not the lived experience of real people, and should not be treated as evidence about actual individuals.Source: Ethical considerations · p. 9
  • The authors caution against substituting simulations for engagement with affected communities on consequential decisions, especially where underlying evidence is limited.Source: Ethical considerations · p. 9
  • Realistic synthetic audiences could be used to optimize persuasive messaging; the authors intend MetaPersona-Studio for research and transparent analysis.Source: Ethical considerations · p. 9

Broader Impacts

MetaPersona lowers persona-construction cost, although simulation itself typically dominates pipeline cost; the authors frame grounded populations as a complement to human data.

Broader Impacts

  • MetaPersona reduces persona construction to under $0.5 per task and makes its cost flat in population size.Source: Broader Impacts · p. 9
  • Persona generation is not the dominant cost in most simulation pipelines; the simulation itself typically dominates.Source: Broader Impacts · p. 9
  • Grounded populations can support messaging pre-screening, hypothesis screening, and opinion forecasting, but should complement rather than displace human data.Source: Broader Impacts · p. 9

Code and Demo Availability

The related work argues that realistic personas require task-relevant attributes and empirical dependencies, not simply richer descriptions. The manuscript’s code and public demo are still being prepared for release.

Code and Demo Availability

  • The research code and public METAPERSONA-STUDIO demo are being prepared for release with the next manuscript version.The manuscript describes this work as in progress.Source: Code and Demo Availability · p. 9

A Extended Related Work

  • Naive persona generation can amplify bias, collapse profiles toward typical cases, exaggerate associations, and flatten within-group diversity.The cited work motivates selecting task-relevant attributes and preserving their empirical dependencies rather than merely adding persona detail.Source: A Extended Related Work · p. 12
  • Existing approaches use demographic sampling, text-grounded personas, or post-hoc alignment, but the task-specific schema-selection problem remains largely unaddressed.Post-hoc alignment such as Hu et al. (2025) aligns a preselected Big Five schema rather than choosing attributes for a task.Source: A Extended Related Work · p. 12
  • Response-tuned simulators and METAPERSONA are complementary: one adapts model weights, while the other structures populations with task-relevant variables and dependencies.METAPERSONA’s literature-grounded dependency graphs operationalize accounts that faithful social simulation needs cognitive and social structure beyond demographics.Source: A Extended Related Work · p. 12
  • Evaluation spans individual accuracy, distributional alignment, and structural realism, including preservation of associations, predictive relationships, and life-course patterns.The related work cautions that individual accuracy alone can reward degenerate mean prediction without representing the population.Source: A Extended Related Work · pp. 12–13

B Cost Analysis

This section details MetaPersona’s persona-construction cost mechanism, annotation and validation procedures, task-specific metrics, extended structural-realism findings, and prompts for study classification, variable and relationship extraction, persona generation, and persona-conditioned survey responses.

B Cost Analysis

  • MetaPersona avoids generating a new profile for every person: it builds a task-specific graph once and samples individuals, so its cost advantage over LLM-Heuristic and Backstory widens with population size.The baselines require one or more LLM calls per persona, whereas MetaPersona uses statistical sampling after graph construction.Source: B Cost Analysis · p. 13, Table 4 · p. 13

C.1 Annotation Taxonomy

  • The annotation schema focuses on measurable person-level attributes, beliefs, behaviors, and outcomes, while excluding study-level descriptors unless treated as attributes or moderators.The schema groups annotations into study-level descriptors, variable-level annotations, and path-level annotations.Source: §C.1 · p. 13

C.2 Dataset Statistics

  • The corpus spans academic disciplines and publication venues, as shown by the distribution of articles across disciplines and journals.The figure illustrates the disciplinary breadth of the corpus used for variable and path extraction.Source: §C.2 · p. 13, Figure 5 · p. 13

C.3 Human Validation

  • Human validation found reliable empirical-paper labeling and 91% usable extracted annotations, with strong annotator agreement across both tasks.For paper labeling, annotator agreement was Cohen’s κ = 0.94 (raw agreement = 97%); LLM classification achieved 97% recall, 86% precision, and 91% F1. For full-text annotation, agreement was κ = 0.86 (raw agreement = 91%); support rates were variables 94%, measurement forms 90%, paths 91%, directions 93%, distribution priors 88%, and mediators/moderators 86%.Source: §C.3 · p. 14, Table 5 · p. 14

D Metric Definitions

E Extended Experiment Results

  • Across additional models, MetaPersona generally yields more empirically plausible structural relationships, especially on Stack Overflow, but results vary by model and task.The prose notes that heuristic baselines can sometimes achieve comparable or lower absolute deviation when expected relationships are weak or ambiguous; structural realism also depends on the underlying LLM’s capabilities and inductive biases.Source: E Extended Experiment Results · pp. 16–17, Table 7 · p. 17

Prompt 1: Empirical Paper Classification

  • The empirical-paper prompt includes original human-data studies and systematic reviews or meta-analyses of individual-level relationships, while excluding theory or work without applied human data.It also excludes natural-system studies without humans and nonsystematic narrative reviews.Source: Prompt 1: Empirical Paper Classification · p. 17

Prompt 2: Variable, Distribution, and Path Extraction

  • The extraction prompt requests study context and findings alongside persona-relevant variables annotated by role, type, measure, and distributional shape.Requested study fields also include academic subfield, agent types, and theoretical constructs.Source: Prompt 2: Variable, Distribution, and… · p. 18
  • The extraction prompt encodes variable roles and distribution shapes, maps reported relationships by type and direction, and requires evidence-based paths that distinguish mediators from moderators.It also records evidence and confidence, and limits extraction to information reported or strongly implied by the title and abstract.Source: Prompt 2: Variable, Distribution, and… · p. 18

Prompt 3: Backstory Persona Generation (Stack Overflow Case Study)

  • The Stack Overflow backstory prompt asks for a first-person professional narrative consistent with a supplied description without directly answering survey questions.It permits details such as education, career path, developer role, work environment, and day-to-day responsibilities.Source: Prompt 3: Backstory Persona Generation… · p. 18

Prompt 4: LLM-Heuristic Persona Attribute Selection

  • The attribute-selection prompt requests the top specified number of person-level attributes relevant to the task outcome, with reasons, explanations, and measurement approaches.It favors attributes that plausibly predict individual differences and excludes system- or environment-level factors.Source: Prompt 4: LLM-Heuristic Persona… · p. 19

Prompt 5: Persona-Conditioned Survey Response (Stack Overflow Case Study)

  • The survey-response prompt directs the model to answer in character as the described software developer, choose a valid scale option, and return a code with a one-sentence reason.The response should reflect the persona’s background, experience, role, industry, and compensation rather than aim for neutrality or balance.Source: Prompt 5: Persona-Conditioned Survey… · p. 19
Loading 2609.38392v1…