PaperlayerView exact revision on arXiv

Source-linked AI summary

Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams

Yirui Qi, Xiaoming Zhang, Jaehong Yoon

arXiv:2610.15788v1cs.AIcs.CLcs.MA

TL;DR

LLM teams can lack an explicit, validated account of how directed relationships evolve over time, despite requiring sustained social interaction. Agent Sociodrama models evolving emotional and relational states, and TPR uses social trajectories to recast personas. The authors report that these trajectories support team-composition improvements across collaborative tasks.

  • Problem

    Many agent systems use static role instructions or unstructured memory without an explicit, validated account of how directed relationships evolve over time.

    Source: §1 · p. 1

  • Method

    Agent Sociodrama models evolving emotional and directed relational states under fixed personas, while TPR uses interaction transcripts and social trajectories to diagnose team composition and recast personas.

    Source: §1 · pp. 1–2, §3 · p. 3

  • Results

    The casting controller turns social trajectories into targeted persona interventions and matches or surpasses the strongest manual casting or published reference in collaborative tasks.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Social dynamics can serve as measurable properties of LLM teams and as actionable signals for improving their composition.

    Source: §6 · p. 10

    Fidelity evaluation uses one unified LLM judge and five designed scenarios, while optimization experiments use 12 episodes per casting and do not make per-cell significance claims.

    Source: A LIMITATIONS · p. 14

Abstract

from arXiv · show

Large language model (LLM) agents are increasingly deployed as teams, yet they fail in characteristically social ways: information holders stay silent, dominant speakers anchor groups on wrong answers, and norm violations go unsanctioned. We present Agent Sociodrama, a framework for modeling and validating evolving social dynamics in LLM agent teams. Under fixed personas, agents' emotional states and directed, asymmetric relationships are updated every round by a unified external LLM judge and shape their subsequent interactions. Across six LLM backbones and five social scenarios, we assess social fidelity through classical social-theory patterns and agreement with human judgments. The resulting trajectories exhibit structured social patterns and model-specific differences. We then introduce Trajectory-Guided Persona Recasting (TPR), which diagnoses group-dynamic failures from interaction transcripts and social trajectories and recasts agent personas before rerunning the task. Across four collaborative benchmarks, TPR improves performance from both heterogeneous and randomly initialized agent persona compositions, yielding up to 83.3%p of accuracy improvement over the initial casting. On representative distributed-information cooperation and mixed-motive bargaining tasks, we outperform SOTOPIA and AgentVerse in three of four comparisons, by up to 37.5%p in Pareto rate. These results show that validated social trajectories provide actionable signals for repairing LLM agent teams.

1 INTRODUCTION

Agent Sociodrama treats LLM teams as evolving social systems, modeling relationships alongside fixed personas. The paper validates these dynamics and uses social trajectories to guide persona recasting for collaboration.

1 INTRODUCTION

  • Existing systems often use static role instructions or unstructured memory without an explicit, validated account of how directed relationships evolve over time.The paper frames longitudinal role play as requiring socially faithful relationship changes, not only changing relations.Source: §1 · p. 1
  • Agent Sociodrama jointly models stable personas, evolving emotional states, and directed, asymmetric relational states as a sustained social process.Its overview connects social-fidelity assessment with social-theory patterns and human judgments, then uses trajectories to guide team-composition optimization.Source: §1 · p. 2, Figure 1 · p. 2
  • Across six LLMs and five social scenarios, Agent Sociodrama’s long-horizon relational dynamics are validated against social theory and human evaluation as structured, interpretable, and socially coherent.The study also assesses social fidelity across four established patterns of social interaction.Source: §1 · p. 2
  • Across four collaborative task families, trajectory-guided persona casting substantially improves team performance.The controller diagnoses team composition and recasts personas to seek more effective task-oriented collaboration.Source: §1 · p. 2

2 RELATED WORK

Related work spans LLM social simulation and coordination, as well as persona composition and multi-agent optimization. Agent Sociodrama combines trajectory-based diagnosis with persona assignment search between task runs.

2 RELATED WORK

  • LLM social-simulation work covers memory-based societies, social-intelligence evaluation, coordinated role play, and social reasoning; team studies address information pooling, expertise, and coordination failures.Agent Sociodrama models evolving affect and directed relationships as persona-conditioned states whose trajectories complement transcripts in diagnosing team dynamics and guiding persona recasting.Source: §2 · p. 3
  • Prior optimization targets prompts, topologies, workflows, computational graphs, code-defined agents, or agent activation, while Agent Sociodrama searches persona assignments from a fixed library between task runs.Social trajectories guide proposed changes, while task outcomes determine configuration selection.Source: §2 · p. 3

3 AGENT SOCIODRAMA: SIMULATION WITH MEASURABLE SOCIAL STATE

Agent Sociodrama models repeated interaction as feedback between agents’ actions and evolving emotional and directed relational states. Fixed personas condition this process, while agents act on locally visible events and histories.

3 AGENT SOCIODRAMA: SIMULATION WITH MEASURABLE SOCIAL STATE

  • The framework replaces static or unmodeled relationships with round-by-round updates that shape subsequent communication and can yield shifting alliances, declining trust, and growing disagreement.Each agent has a fixed persona, while actions, emotions, and directed relationships evolve across rounds.Source: §3 · p. 3

3.1 PERSONA-CONDITIONED AGENT STATE

  • Each agent combines a time-invariant persona with a dynamic emotional state and directed relationships toward other agents.Personas specify identity, traits, and behavioral boundaries; emotions encode affective valence, while relationships include favorability, categorical status, and event memory.Source: §3.1 · pp. 3–4

3.2 EVENTS, ACTIONS, AND VISIBILITY

  • Agents respond to ordered scenario events using public and private interactions, so each acts on the events and utterances visible to it.Emotional and relational trajectories can support coordination or amplify conflict, and help the casting controller assess team compositions.Source: §3.2 · p. 4

3.3 PER-ROUND INTERACTION AND STATE UPDATES

  • Each round assembles locally visible context, generates actions in two phases, and revises emotional and relational states from observed events.Agents first choose urgency, communication mode, target, and intent under shared information, then speak in urgency order; update calls are conditioned on persona and visible events, with changes constrained to limit abrupt shifts.Source: §3.3 · pp. 4–5

4 THE CASTING CONTROLLER

The casting controller treats persona composition as an optimization choice, using social trajectories to diagnose team failures and guide selective recasting. In one example, changing only the persona casting raised GPT-5-mini’s Student-council hidden-profile accuracy from 16.7% to 100.0%.

The Casting Controller

  • The framework treats persona composition as consequential and uses social trajectories to guide per-agent recasting.Figure 3 presents the trajectory-guided recasting approach.Source: §4 · p. 5, Figure 3 · p. 6
  • The controller selects a casting to maximize expected task utility, measured by accuracy, negative L1 distance to an expert ranking, or Pareto-efficient settlement rate.The recasting procedure, archetype library, and diagnostic logic remain fixed across tasks; task context and utility definition vary.Source: §4 · p. 6
  • Each iteration evaluates runs, diagnoses suppressed information or unhelpful dominance, then recasts agents by majority vote across runs.The loop ends when the task target is reached, no change is proposed, or after at most three recasts.Source: §4 · p. 6
  • Diagnosis uses emotional states, directed ties, and group-level relationship patterns to localize failures to particular agents and relationships.These signals can distinguish withdrawal after repeated dismissal from neutral silence and reveal disregarded information holders, alliances, polarization, and responses to norm violations.Source: §4 · p. 6

5 EXPERIMENTS

The experiments evaluate social-dynamics fidelity across scenarios and LLMs, then test whether those trajectories improve persona recasting. TPR improves collaboration from both initial cast types and outperforms established baselines in most compared settings.

5.1 EXPERIMENTAL DESIGN

  • The study tests six LLM backbones across five social scenarios and evaluates recasting on four collaborative benchmarks using Hetero and Random initializations.Fidelity runs use five independent nine-round episodes per model-scenario condition; recasting runs use 12 episodes per casting and model.Source: §5.1 · pp. 6–8

5.2 SOCIAL DYNAMICS FIDELITY ON AGENT SOCIODRAMA

  • Across scenarios, terminal favorability falls from 0.75 in Hospital to 0.23 in Demolition as balance and repair also decline, while reciprocity remains high in Demolition at 0.68.Relationships warm under low social pressure and flatten or decline as conflict intensifies; repair falls from 58.5% to 18.9%.Source: Figure 4 · p. 7, Table 1 · p. 7, §5.2 · p. 8
  • Final favorability differs significantly across six models in every scenario (all p < 0.01), and model-specific social dynamics vary across multiple relational dimensions.In Tavern, GPT-5-mini has FavT = 0.75 and E-I = −0.22, while Claude has Recip. = 0.93 and Repair = 80.0%.Source: Table 1 · p. 7, §5.2 · pp. 8–9, Figure 5 · p. 8
  • Among updates where at least seven of thirteen human raters agreed, the judge matched 76.2% of relationship labels and 88.2% of emotional labels.Inter-human agreement was low (Fleiss’ κ = 0.288 for relationships and 0.184 for emotions), and adding the judge as a rater left κ nearly unchanged.Source: §5.2 · p. 9

5.3 TRAJECTORY-GUIDED TEAM COMPOSITION OPTIMIZATION

  • TPR improves performance across all four tasks from both Hetero and Random initializations, with GPT-5-mini rising from 16.7% to 100.0% on Hetero Student Council.Across six backbones, the controller performs up to three recasts; Table 3 reports initial and final averages over 12 episodes per model row.Source: §5.3 · p. 9, Table 3 · p. 10
  • Agent Sociodrama achieves the highest final mean in three of four baseline-comparison settings, with Negotiation Pareto rates of 48.6% under Hetero and 44.4% under Random.AgentVerse narrowly leads on Random NASA (12.89 vs. 12.92).Source: §5.3 · p. 9, Table 4 · p. 10
  • Removing emotional and relational updates weakens model-average performance, while a Lost at Sea case improves mean L1 from 21.83 to 11.33 after three recasts.The first two recasts lower L1 to 18.00 and 14.33; the example assigns distinct personas rather than converging on one preferred persona.Source: §5.3 · pp. 9–10, Figure 7 · p. 10

6 CONCLUSION

The paper presents a framework for modeling and validating evolving social dynamics in LLM agents, then uses those trajectories to guide targeted persona recasting for collaborative tasks.

6 CONCLUSION

  • Agent Sociodrama models directed, asymmetric relationships and emotional states, and validates trajectories against social theory, cross-model comparability, and human judgments.Source: §6 · p. 10
  • A casting controller uses relational and emotional trajectories together with transcripts to identify problematic agents and relationships and recast personas.Source: §6 · p. 10
  • The framework matches or surpasses the strongest manual casting or published reference in collaborative tasks, supporting social dynamics as actionable signals for team composition.Source: §6 · p. 10

A LIMITATIONS

The authors identify limits in the fidelity evaluation and optimization experiments, and describe how gradual threshold updates interact with event-conditioned relationship-status changes.

A LIMITATIONS

  • The fidelity evaluation uses one unified LLM judge and five designed scenarios, so additional judges and structurally different scenarios would strengthen external validity.The authors also report only consistent optimization directions, not per-cell significance claims, because each casting uses 12 episodes against substantial interaction variance.Source: A LIMITATIONS · p. 14
  • Optimization uses 12 episodes per casting, which the authors consider small relative to LLM-interaction variance.Accordingly, they report consistent directions across tasks, backbones, and initializations rather than per-cell significance claims.Source: A LIMITATIONS · p. 14
  • Casting is a coarse episode-level intervention; the authors identify runtime mediation, targeted information routing, and norm enforcement as possible finer-grained next steps.These are proposed future directions using the same measured trajectories.Source: A LIMITATIONS · p. 14

B CATEGORICAL-STATUS UPDATE

  • Relationship status follows favorability thresholds unless the update model judges a consequential event significant enough to override them.The event-conditioned judgment uses agent traits, current favorability and status, the event, and the three most recent relationship-memory entries.Source: B CATEGORICAL-STATUS UPDATE · p. 14

C FIDELITY EVALUATION PROTOCOL

The fidelity protocol samples five scenarios and six LLM drivers, then characterizes social trajectories with relationship, affect, balance, reciprocity, faction, and repair measures.

C FIDELITY EVALUATION PROTOCOL

  • The fidelity evaluation covers 30 scenario–model conditions and 150 trajectories, with five agents interacting for nine rounds across five episodes per condition.This yields 1,350 team-level interaction rounds and 6,750 agent-round observations.Source: C FIDELITY EVALUATION PROTOCOL · p. 14

D METRIC DEFINITIONS

  • Except for Repair, metrics are computed per episode and averaged across five episodes; Repair is pooled across episodes using edge counts..Source: D METRIC DEFINITIONS · p. 14
  • FavT averages favorability over directed edges at the final round, while ST is mean affective valence across agents.The passage defines these terminal measures at the end of an episode.Source: D METRIC DEFINITIONS · p. 14
  • Balance counts triads with an even number of negative dyads as balanced, while reciprocity is the correlation between opposite-direction terminal favorability.High reciprocity correlation can accompany either positive or negative attitudes.Source: D METRIC DEFINITIONS · p. 14
  • The E–I index contrasts mean cross-faction and within-faction favorability; it can fall outside [−1, 1] and become unstable when their sum approaches zero.The authors attribute large dispersion in some Tab. 11 cells to this instability.Source: D METRIC DEFINITIONS · p. 15
  • Repair pools directed edges that deteriorate at least once and recover by the final round, using deteriorated edges—not all edges or episodes—as its denominator.An episode with no deteriorated edges contributes zero to both sums; if none deteriorate across the five episodes, Repair is reported as N/A.Source: D METRIC DEFINITIONS · p. 15

E PERSONA-RECASTING EXPERIMENTAL SETUP

The experimental setup tests persona recasting on three distributed-information tasks and one mixed-motive negotiation task, using matched initial castings and a bounded recasting loop.

E PERSONA-RECASTING EXPERIMENTAL SETUP

  • Student Council, NASA Moon Survival, and Lost at Sea integrate privately distributed knowledge; Solar-farm Negotiation seeks a feasible, Pareto-efficient agreement from private preferences.Task metrics are computed by rules from final outputs rather than by an LLM judge.Source: E PERSONA-RECASTING EXPERIMENTAL SETUP · p. 15
  • Solar-farm negotiation evaluates feasibility and Pareto optimality across 243 possible agreements, which the authors enumerate exactly.An agreement is Pareto-optimal if no other feasible agreement improves one party’s utility without making another worse off.Source: E PERSONA-RECASTING EXPERIMENTAL SETUP · p. 15, Table 5 · p. 16
  • Hetero assigns four distinct archetypes, while Random samples from the same library and may repeat archetypes; each task uses the same casting across methods and backbones.Archetypes change communication only, not task role, private information, or scoring.Source: E PERSONA-RECASTING EXPERIMENTAL SETUP · p. 15, Table 6 · p. 16
  • Each evaluated casting runs 12 episodes, and the controller permits at most three recasts, for up to four evaluated castings.The loop stops early if the target is reached or no persona change is proposed; the final score is from the last evaluated casting.Source: E PERSONA-RECASTING EXPERIMENTAL SETUP · p. 15
  • The social-dynamics ablation uses fixed-order public turn-taking without emotional or relational updates, while retaining the same tasks, castings, backbones, and recasting budget.SOTOPIA and AgentVerse use the same recasting procedure and initial castings.Source: E PERSONA-RECASTING EXPERIMENTAL SETUP · p. 16

F COMPLETE PERSONA-RECASTING RESULTS

The supplied results passages identify complete TPR trajectories and the per-backbone tables underlying the reported ablation and external-framework comparisons.

F COMPLETE PERSONA-RECASTING RESULTS

  • Tables 8 and 9 provide per-backbone results underlying the model-average social-dynamics ablation and external-framework comparison, respectively.The supplied passage specifies that these figures use the same 12-episode runs as the main text.Source: F COMPLETE PERSONA-RECASTING RESULTS · pp. 16–17
  • Table 7 reports TPR’s per-iteration trajectories for every task, backbone, and initialization; its final evaluated iteration corresponds to Tab. 3’s Final score.Iteration 0 is the initial casting, and later columns show mean scores over 12 episodes.Source: F COMPLETE PERSONA-RECASTING RESULTS · p. 16, Table 7 · p. 17
  • Table 8 reports the social-dynamics ablation by backbone, averages 12 episodes per cell, and bolds the better endpoint within each pair.Its model-average row matches the ablation row in Tab. 3.Source: Table 8 · p. 17

G HUMAN ANNOTATION PROTOCOL

Human ratings were collected on stratified dialogue excerpts and compared with the judge. Agreement was limited by ambiguity, while the judge followed human majorities on consensus items and showed a relationship-direction skew.

G HUMAN ANNOTATION PROTOCOL

  • Thirteen raters labeled 25 stratified dialogue excerpts, and strict-majority consensus was used to assess judge agreement only where humans converged.All raters passed an attention check.Source: G HUMAN ANNOTATION PROTOCOL · p. 17
  • Human agreement was low, but the judge followed the majority on most consensus items; relationship increases were labeled about 76% by the judge versus about 40% by humans.The authors attribute low inter-human agreement to ambiguity in interpreting relational and emotional change from short excerpts.Source: G HUMAN ANNOTATION PROTOCOL · p. 17, Table 10 · p. 18
  • Because the same judge scored every condition, its directional tendency affects absolute favorability levels but not cross-scenario ordering or cross-model comparisons.The stated boundary applies to conclusions based on Tables 1 and 11.Source: G HUMAN ANNOTATION PROTOCOL · p. 18

H FULL SOCIAL-DYNAMICS FIDELITY RESULTS

The results show scenario-level shifts in social-dynamics profiles alongside persistent model differences. Dynamic plots complement terminal metrics by showing emotional trajectories and the frequency of low final emotional states.

H FULL SOCIAL-DYNAMICS FIDELITY RESULTS

I DYNAMIC TRAJECTORIES

  • Terminal metrics hide state evolution: Fig. 10 plots per-agent emotional trajectories, while Fig. 11 reports the share of final agent-episode states with ST ≤0.1.These figures use the same 150 episodes as Tab. 11; Fig. 10 averages five episodes and shows one-standard-deviation shading across nine rounds.Source: I DYNAMIC TRAJECTORIES · p. 18, Figure 10 · p. 20, Figure 11 · p. 20

Related papers

Loading 2610.15788v1…