PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing LLMs with Cognitive-Affective Personality Inference for Simulating Human Social-Psychological Behavior

Zhibo Deng, Dongyuan Li, Shuwen Ge, Ziqing Zhang, Ying Zhang, Renhe Jiang

arXiv:2610.08328v1cs.SI

TL;DR

LLMs can approximate aggregate human responses, but end-to-end prompting entangles stable dispositions with situation-specific interpretations. SPIN stages personality compilation, state elicitation, and decision readout; across two study families and four models, it achieves the strongest alignment among the evaluated prompting variants.

  • Problem

    End-to-end prompting entangles stable dispositions with situation-specific interpretations, while behavioral simulation should match theory-relevant human summaries rather than merely produce plausible responses.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Method

    SPIN uses a zero-shot staged pipeline to compile a task-blind personality core, elicit condition-specific states, and read out decisions grounded in those states.

    Source: §1 · p. 2

  • Results

    SPIN achieves the strongest benchmark-level alignment across two reconstructed study families and four LLMs, outperforming blank, demographic, narrative, and chain-of-thought variants.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Structured personality–state inference can improve benchmark-level behavioral alignment beyond richer persona descriptions or generic multi-step reasoning.

    Source: §1 · p. 2

    Evaluation covers two reconstructed study families and four base LLMs, so the evidence supports benchmark-level effectiveness rather than broad claims about all social judgment tasks.

    Source: §I.1 · p. 36

Abstract

from arXiv · show

Large language models are increasingly used to simulate human participants in social and behavioral studies, yet static persona prompting typically maps a participant profile and an experimental scenario directly to a response, entangling stable dispositions with situation-specific interpretations. To address this limitation, we introduce \textbf{SPIN}, a cognitive-affective personality system-inspired inference pipeline for simulating human social-psychological behavior. Specifically, SPIN implements this structured inference process through three zero-shot LLM calls that compile a task-blind participant core, elicit condition-specific cognitive-affective states, and read out decisions from those states, thereby reusing stable personality structure while routing each trial-specific response through an explicit state representation. We evaluate SPIN on two reconstructed social-psychological study families spanning uncertainty reasoning and pluralistic ignorance, across four base LLMs. Compared with blank, demographic, narrative, and chain-of-thought prompt variants, SPIN consistently delivers the strongest overall alignment performance across base LLMs and study families. Ablations and state analyses further show that both personality compilation and structured state elicitation contribute to the gains, and that the elicited states shift interpretably across informational and normative conditions. These results suggest that structured personality-state inference can improve benchmark-level behavioral alignment beyond richer persona descriptions or generic multi-step reasoning.

1 Introduction

LLMs can approximate aggregate human responses, but end-to-end prompting entangles stable dispositions with situation-specific interpretations. SPIN separates those representations and achieves the strongest benchmark-level alignment among the evaluated prompt variants across two study families and four LLMs.

1 Introduction

  • End-to-end prompting entangles stable participant dispositions and situation-specific interpretations in one generation step, complicating consistent participant reuse and sensitivity to small manipulations.Prior limitations include compressed human variability, miscalibrated effects, missed cross-condition contrasts, and drift from assigned personas.Source: §1 · p. 2
  • SPIN uses three zero-shot LLM calls to compile a task-blind personality core, elicit condition-specific cognitive-affective states, and read out decisions grounded in those states.Stagewise input restrictions keep trial information out of personality compilation and ground decisions in elicited states; the core is reused across trials.Source: §1 · p. 2
  • The paper defines simulation quality as matching theory-relevant human summaries, including choice rates, rating statistics, and cross-condition contrasts, rather than recovering full individual psychologies.The introduction situates this goal against evidence that LLMs can approximate aggregate patterns and treatment effects, while emphasizing that surface plausibility is insufficient for pilot studies and preliminary theory testing.Source: §1 · pp. 1–2
  • SPIN achieves the strongest benchmark-level alignment across two reconstructed study families and four LLMs, outperforming blank, demographic, narrative, and chain-of-thought variants.Ablations and state analyses indicate that personality compilation and state elicitation both contribute to the gains, with interpretable state shifts across conditions.Source: §1 · p. 2

2 Related Work

Prior work uses LLMs as synthetic respondents and reports both aggregate-pattern replication and persistent alignment limitations. SPIN builds on cognitive and psychological approaches, using CAPS to distinguish stable dispositions from situation-specific activation.

2 Related Work

  • LLMs conditioned on demographic, attitudinal, or biographical profiles can reproduce some aggregate response patterns, and large-scale replications report correlations with actual treatment effects.Validation studies also find compressed variability, miscalibrated effects, missed subgroup or cross-condition structure, and persona drift.Source: §2 · p. 2
  • Existing remedies enrich personas or add generic reasoning, but do not ensure consistent participant reuse across trials or isolate the situation-specific state mediating a decision.Related approaches include narrative backstories, character-consistency methods, chain-of-thought, and context-navigation strategies.Source: §2 · p. 2
  • SPIN draws on cognitive and psychological approaches by using CAPS to separate stable dispositions from situation-specific activation for benchmark-level social judgment alignment.The related work places this approach alongside cognitive benchmarks, value-tradeoff models, belief elicitation, and executable models of others’ routines.Source: §2 · p. 3

3 Task Formulation

The paper defines behavioral alignment at the level of study summaries, not complete individual psychologies. It contrasts this target with monolithic prompting, which combines a participant profile and trial stimulus in one inference call.

3 Task Formulation

  • Benchmark-level behavioral alignment means matching simulated study summaries, such as response proportions, rating statistics, and cross-condition contrasts, to human targets.The goal is not to recover each participant’s full latent psychology or complete response trajectory.Source: §3 · p. 3
  • The task pairs participant profiles containing demographic, behavioral, or personality information with trials specifying a scenario, experimental condition, response space, and measurement protocol.The evaluator maps each generated response to study-level behavioral summaries.Source: §3 · p. 3
  • Monolithic prompting concatenates the participant profile and trial stimulus, asking one generation step to infer both stable tendencies and condition-specific interpretations.The paper notes this can produce inconsistent reuse of a profile across trials and under- or over-weight subtle condition changes.Source: §3 · p. 3

4 Methodology

SPIN stages social-psychological response generation into stable personality compilation, condition-specific state elicitation, and state-grounded decision readout. Its stagewise input restrictions and reuse of the personality core are designed to preserve participant consistency while allowing situational variation.

4 Methodology

  • SPIN decomposes generation into three sequential stages, compiling stable personality characteristics, eliciting a situation-conditioned cognitive-affective state, and producing a decision grounded in that state.The architecture is inspired by CAPS, which describes stable units selectively activated by situational features.Source: §4 · p. 3, §4.1 · p. 3

4.1 Framework Overview: A CAPS-Inspired Architecture

  • The framework instantiates the stable-versus-transient distinction as a reusable personality core and trial-specific states, implemented through separate zero-shot calls with stage-specific inputs.The core is cached per participant, states are elicited per trial or item condition, and the final stage produces item choices and confidence.Source: Figure 1 · p. 4, §4.1 · p. 4

4.2 Stage 1: Personality Compilation

  • Personality compilation converts a participant profile into a task-blind core containing trait scores, cue-triggered disposition rules, and a default decision-style summary.The implementation uses five trait dimensions; the core is created without access to trial information.Source: §4.2 · p. 4

4.3 Stage 2: Condition-Specific State Elicitation

  • State elicitation combines the personality core with the current trial to create an indexed, condition-specific state for each distinct item or condition.Each state includes six continuous variables, promoting and inhibiting considerations, a decision mode, and a rationale; the raw profile is not supplied.Source: §4.3 · p. 5

4.4 Stage 3: Decision Readout

  • Decision readout converts indexed elicited states into benchmark-format per-item choices, a confidence score, and a canonical answer line.Each item decision is grounded in its corresponding state, with neither the raw profile nor personality core available to this stage.Source: §4.4 · p. 5

4.5 Structural Design Principles

  • Stagewise input restrictions create a representational bottleneck: the core is reused across trials, while states are recomputed for each trial or item condition.The authors also describe a causal ordering in which personality precedes situation-activated state, which precedes behavior; finalized stage outputs prevent later stages from retroactively reshaping earlier representations.Source: §4.5 · pp. 5–6

5 Experiments

Across two reconstructed study families and four LLM backbones, SPIN ranks first among the prompting variants on study-level alignment. Ablations and behavioral analyses indicate contributions from both personality compilation and condition-specific state elicitation.

5.1 Experimental Setup

  • The benchmark covers uncertainty reasoning and pluralistic-ignorance measures, with held-out human outcomes and normalized family-level summaries; baselines include blank, demographic, narrative, and CoT prompting.The two families include choice, rating, norm-gap, dispersion, and social-action observables.Source: §5.1 · pp. 6–7

5.2 Main Results

  • SPIN ranks first in Avg. and both study-level alignment scores for all four backbones, with positive paired differences against every baseline in both families (40/40).Table 1 reports Shafir–Tversky and Prentice–Miller alignment scores and their mean, Avg.Source: Table 1 · p. 7, §5.2 · p. 7
  • Gains are strongest on condition-sensitive Shafir–Tversky components, while Prentice–Miller gains are generally smaller and SPIN still leads each backbone.Across models, SPIN improves the two-study macro-average without trading off one study family against the other.Source: §5.2 · p. 8

5.3 Ablations

  • Removing compilation or elicitation lowers Avg. for all four backbones, with larger drops without elicitation (0.036–0.055 vs. 0.021–0.033).The smaller but systematic degradation without compilation suggests raw-profile conditioning does not fully substitute for a task-blind participant core.Source: Table 2 · p. 8, §5.3 · p. 8
  • Replacing SPIN’s fixed trait/state schema with freeform reasoning lowers Avg. for every backbone, especially on Shafir–Tversky (0.089–0.152).SPIN_freeform_intermediate slightly exceeds SPIN for Qwen 3.5-max on Prentice–Miller, an exception the authors describe as not robust.Source: §5.3 · pp. 8–9

5.4 Behavioral Calibration by State Elicitation

  • Removing state elicitation raises focal-action rates especially in known-cooperate Prisoner’s Dilemma, Newcomb, and information seeking, while the full pipeline is generally more moderated and closer to human targets.The authors describe elicitation as calibrating readout to inferred situational state, not uniformly increasing human-like responding.Source: §5.4 · p. 9

5.5 Aggregate State Sensitivity

  • Figure 2 reports task-specific focal-action rates across condition groups: cooperation, choosing Box B only, and paying for opponent information.The figure covers Prisoner’s Dilemma, Newcomb’s problem, and information seeking.Source: Figure 2 · p. 9

6 Conclusion

SPIN improves benchmark-level alignment across two reconstructed study families and four base LLMs, while using a staged personality-and-state inference process. The conclusion frames SPIN as a behavioral-alignment method rather than a validated psychological measurement model.

6 Conclusion

  • SPIN stages task-blind personality compilation, condition-specific state elicitation, and state-grounded readout.Source: §6 · pp. 9–10
  • Across two reconstructed study families and four base LLMs, SPIN improves alignment over blank, demographic, narrative, and chain-of-thought-augmented variants.Source: §6 · p. 10
  • SPIN is a structured inference method for behavioral alignment, not a validated psychological measurement model; broader study coverage and cost-matched baselines remain open directions.Source: §6 · p. 10

A.1 LLM Social Simulation: Capabilities, Limitations, and Evaluation

Prior work shows that LLMs can reproduce some aggregate behavioral patterns, but alignment concerns persist and richer persona prompts or generic reasoning do not resolve the modeling gap. Psychological theory offers a route to structured behavioral representations, including separating stable dispositions from situation-specific activation.

A.1 LLM Social Simulation: Capabilities, Limitations, and Evaluation

  • LLMs reproduce aggregate behavioral patterns, but simulations can compress variability, miscalibrate effects, miss subgroup or cross-condition structure, and drift from personas; benchmarks target behavioral summaries.Source: §A.1 · p. 14

A.2 Persona Conditioning and Multi-Step Prompting

  • Persona conditioning can yield subgroup- or trait-consistent outputs but remains brittle, while generic multi-step reasoning lacks a reusable participant representation and condition-specific state bottleneck.Source: §A.2 · pp. 14–15

A.3 Psychological Theory as Computational Scaffolding for LLM Behavior

  • Psychological frameworks inform both LLM behavior analysis and agent design, while CAPS motivates separating stable dispositions from situation-specific activation in benchmark-level alignment.Source: §A.3 · p. 15

B Trait and State Dimension Rationale

SPIN’s trait and state vocabulary is a compact CAPS-inspired interface for social judgment and strategic interaction, designed to cover behavioral variation in the evaluated task families rather than instantiate a single psychometric scale.

B Trait and State Dimension Rationale

C Experimental Scenario Information

The appendix summarizes two experimental scenario families used to evaluate SPIN. Shafir and Tversky’s uncertainty paradigms vary information across matched decisions, while Prentice and Miller’s pluralistic-ignorance protocol distinguishes private attitudes, attributed norms, and related behavior.

C.1 Shafir and Tversky (1992)

  • Shafir and Tversky’s uncertainty paradigms include matched informational conditions, Newcomb’s Problem, and an information-seeking Prisoner’s Dilemma variant.Source: §C.1 · p. 16

C.2 Prentice and Miller (1993)

  • The Prentice–Miller source links misperceived campus drinking norms to private attitudes, behavioral adjustment, and social alienation.Source: §C.2 · pp. 16–17
  • The benchmark measures self and peer comfort, including whether attributed peer attitudes are shifted and less variable than self-ratings, with counterbalanced question order.Source: §C.2 · p. 17
  • A policy-relevant block measures collective action and related outcomes after participants assess their standing relative to others’ attitudes.Source: §C.2 · p. 17

D Metric Design

The metrics normalize model–human behavioral-summary errors by each observable’s scale, then aggregate valid components and family scores with equal weighting.

D Metric Design

  • Each observable’s model–human error is normalized by its scale bound, and valid components and primary-family scores are averaged equally, including structural ordering checks.Auxiliary collapse diagnostics are reported separately and excluded from the headline score.Source: D Metric Design · p. 17

D.1 Shafir and Tversky (1992)

  • Uncertainty scores cover information-seeking and one-box rates plus Prisoner’s Dilemma cooperation levels, condition gaps, and ordering, preserving structure beyond marginal calibration.Responses are canonicalized into benchmark-facing decision labels before choice rates are aggregated.Source: §D.1 · pp. 17–18

D.2 Prentice and Miller (1993)

  • Pluralistic-ignorance scoring compares rating means, self–other gaps, and target orderings across question orders, treating gap size and ordering as structural features.Source: §D.2 · p. 18
  • The evaluation also scores self- and peer-rating dispersion and group-specific policy outcomes and contrasts, but only for instruments aligned with published marginal statistics.Dispersion captures the variance pattern in which attributed peer attitudes are less variable than private self-ratings.Source: §D.2 · pp. 18–19

E Backbone-Paired Robustness Test

SPIN’s advantage was positive across baselines and backbones in both study families, though the check is a model-level diagnostic. The appended analyses show condition-dependent behavioral changes, aggregate state shifts, and illustrative individual outputs.

E Backbone-Paired Robustness Test

  • SPIN’s advantage was positive for every baseline on every backbone in both study families; with four paired units, authors treat this as a robustness diagnostic.The exact one-sided sign-test p-value is 0.0625 for every row.Source: E Backbone-Paired Robustness Test · p. 19, Table 4 · p. 19

E.1 Behavioral Consequences of Removing State Elicitation

  • Removing state elicitation raises focal-action rates across five groups unevenly; the known-compete condition stays near the floor, while several others become less selective.The analysis cautions against reading this as uniform condition-level improvement.Source: §E.1 · p. 20

E.2 Aggregate State Sensitivity

  • Across Shafir–Tversky, elicited states distinguish informational conditions; Figure 4 shows Prentice–Miller variation emphasizing social concern and conflict over trust and exploitation cues.The aggregate views are diagnostic rather than inferential, and the states are not externally validated psychological measurements.Source: §E.2 · pp. 20–21, Figure 4 · p. 20

F.1 Shafir–Tversky: information-seeking PD variant (participant 196)

  • In the paid-information example, exploitation risk rises and trust collapses despite an initial information purchase; the excerpt pairs textual drivers and inhibitors with pooled heatmap means.The example’s state is comparatively defensive.Source: §F.1 · p. 22

F.2 Shafir–Tversky: Prisoner’s triad across three informational conditions (participant 10)

  • Across three Prisoner’s Dilemma items, trust shifts from moderate to very low under known competition and high under known cooperation, accompanying choices Q1=a, Q2=b, Q3=a.The choices illustrate non-monotone cooperative responses across symmetric payoffs.Source: §F.2 · p. 25

F.3 Shafir–Tversky: computerized Newcomb task (participant 80)

These appendix examples illustrate how SPIN uses stable participant traits and dispositions to interpret task-specific cues and produce benchmark-format choices. They show the separation between eliciting a psychological state from the personality core and stimulus, then using that state for decision readout, with rationales reflecting norm tracking, self-protection, and personal–social conflicts.

F.3 Shafir–Tversky: computerized Newcomb task (participant 80)

  • Participant 80’s Newcomb-task rationale pits self-protective causal-dominance reasoning against the norm of one-boxing, with the final response recorded as Q1=A.The elicited mode is calculative guardedness, and the notes say self-protection and perceived exploitation risk override cooperation drivers favoring alignment with the program’s apparent norm.Source: §F.3 · pp. 26–27

F.4 Prentice–Miller: comfort vs. inferred peer norms (participant 127)

  • Participant 127’s norm-reflective response gives Q1=5 and Q2=6, with the higher peer estimate attributed to an institutional norm reference and group projection.Norm sensitivity is strongly activated, while fairness sensitivity is minimally activated because the task asks for perception rather than an explicit fairness judgment.Source: §F.4 · pp. 27–28

F.5 Prentice–Miller: comparative comfort ratings across self, norms, friends (participant 332)

  • Participant 332’s comparative comfort responses are Q1=8, Q2=4, and Q3=7, separating attributed average-student comfort from personal comfort and friends’ comfort.The passage cautions that narrative notes sometimes misattribute question values and says structured per-question choices should be treated as authoritative when composing tables.Source: §F.5 · pp. 29–30

F.6 Prentice–Miller: keg-policy and alienation block (participant 376)

  • Participant 376’s policy-block responses range from moderate opposition to near-zero behavioral intentions, while the elicited state retains high social concern and floor-level perceived exploitation.The six responses are Q1=5, Q2=2, Q3=0, Q4=0, Q5=50, and Q6=5; the notes attribute them to normative evaluation and conflict between fairness and norm alignment.Source: §F.6 · p. 31

G Prompts

  • The prompts appendix presents rendered templates with runtime payloads replaced by descriptive placeholders and organizes them according to the staged pipeline.The text identifies placeholders such as <PROFILE_JSON> and <TRIAL_PROMPT>.Source: G Prompts · p. 31

G.1 Personality Compilation Prompt

  • The personality-compilation prompt instructs the model to infer a task-blind core from the participant profile, including five trait dimensions and two to five disposition rules.The prompt specifies allowed trigger labels and asks for a default decision style and a short natural-language personality portrait.Source: §G.1 · pp. 31–32

G.2 State Elicitation Prompt

  • State elicitation asks how the personality core responds to the trial, identifying salient cues, activated dispositions, internal states, and relevant strategic incentives without making the final choice.It considers the participant’s subjective perspective, including emotions, trust, exploitation risk, uncertainty, and payoff incentives.Source: §G.2 · p. 32

G.3 Decision Readout Prompt

  • Decision readout maps the elicited state to item-level benchmark responses, using its drivers and inhibitors without reconstructing personality or reanalyzing the scenario.The prompt requires choices for every question in the benchmark’s specified format.Source: §G.3 · p. 33

H Baseline Details

The baseline set contrasts task-only prompting, increasingly rich participant profiles, and generic two-call reasoning against SPIN’s structured inference. These conditions test whether identity cues or extra reasoning steps improve behavioral alignment.

H Baseline Details

  • Blank gives the model only the reconstructed task, establishing a task-structure-only baseline without participant-level information.It includes the scenario, condition assignments, response options, and measurement instructions.Source: H Baseline Details · p. 34
  • Demographic conditioning adds sampled age, gender, and education attributes, testing whether compact identity cues steer population-appropriate behavior [3] [2] [19].Attributes are sampled from the source study’s participant distribution.Source: H Baseline Details · p. 34
  • Narrative conditioning replaces the attribute list with a life-history sketch covering personality, routines, and social context, following memory-rich agent approaches [29] [4].The condition tests whether richer persona descriptions improve alignment beyond demographic cues.Source: H Baseline Details · p. 34
  • Demographic+CoT and Narrative+CoT add a free-form reasoning call before a format-constrained answer, controlling for extra reasoning steps without SPIN’s structured states.The variants retain their corresponding participant specifications; their first-call notes have no fixed trait or state dimensions or stagewise input restrictions.Source: H Baseline Details · p. 35

I.1 Limitations and Future Work

This section bounds SPIN’s claims to benchmark-level alignment, calls for broader robustness and replication work, and cautions against treating it as a substitute for human evidence. It also notes that LLMs are an important, original, or non-standard component of the core methods and directs readers to Appendix J for details.

I.1 Limitations and Future Work

  • SPIN is a structured inference interface, not a validated psychological measure, and it neither guarantees faithful individual simulation nor recovers human cognitive mechanisms.Its base-LLM-generated traits and states use hand-designed semantics without independent psychometric validation; future work could test alternative or learned schemas and anchor variables to external measures.Source: §I.1 · pp. 35–36
  • The evaluation covers only two reconstructed study families and four base LLMs, while added calls and JSON structure leave the source of SPIN’s advantage uncertain.The authors propose cost-matched structured baselines, tests of state stability under prompt and decoding changes and unseen domains, and evaluation on more study families, preregistered replications, and participant-level holdouts.Source: §I.1 · p. 36

I.2 Compute Resources

  • Experiments use inference-only hosted API calls, with each backbone–method evaluation pass completed within one hour.No model training, fine-tuning, or local GPU inference is used; local hardware handles orchestration, parsing, and metrics, so cost is dominated by API latency.Source: §I.2 · p. 36

I.3 Broader Impacts

  • SPIN should support methodological analysis and preliminary simulation, not individual assessment, high-stakes decisions, or replacement of human-subject validation.Risks include treating outputs as human evidence or individual measurements and relying on responses shaped by LLM, prompt, or benchmark-reconstruction biases; the work only partially mitigates these risks through aggregate alignment and held-out outcomes.Source: §I.3 · p. 36

J The Use of Large Language Models (LLMs)

  • LLMs assisted only with manuscript language refinement; the authors state that scientific content and conclusions remained their original intellectual contribution.The stated assistance covered grammar, vocabulary, and sentence structure, not substantive scientific content or interpretation.Source: J The Use of Large Language Models… · p. 36

NeurIPS Paper Checklist

10. Broader impacts

  • The LLM-usage declaration answers “Yes” and directs readers to Appendix J.Source: §10 · p. 43

Related papers

Loading 2610.08328v1…