PaperlayerView exact revision on arXiv

Source-linked AI summary

SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation

Han Luo, Guy Laban

arXiv:2604.09212v1cs.CLcs.MA

TL;DR

Long-horizon LLM dialogue generation can suffer from persona drift, role confusion, and echoing, motivating more stable synthetic dialogue infrastructure. SPASM combines modular persona-driven simulation with Egocentric Context Projection, which reduces persona drift and eliminates echoing under human validation across tested configurations.

  • Problem

    Long-horizon multi-turn LLM interactions and synthetic LLM–LLM dialogues require reliable role, persona, and behavioral consistency as context accumulates.

  • Method

    SPASM combines persona sampling, validation, and crafting with Client–Responder simulation, termination detection, and ECP-based projection of shared history into agent-specific views.

  • Results

    ECP substantially reduces persona drift and, under human validation, eliminates echoing across the tested client–responder backbone combinations.

  • Takeaways & Limitations

    SPASM provides a stability-first framework for generating controllable, role-consistent multi-turn dialogue data and analyzing persona and interaction structure.

  • Takeaways & Limitations

    Evaluation is limited to a small set of primarily English-language, instruction-tuned models and two-agent interactions with clearly defined roles.

Abstract

from arXiv · show

Large language models are increasingly deployed in multi-turn settings such as tutoring, support, and counseling, where reliability depends on preserving consistent roles, personas, and goals across long horizons. This requirement becomes critical when LLMs are used to generate synthetic dialogues for training and evaluation, since LLM--LLM conversations can accumulate identity-related failures such as persona drift, role confusion, and "echoing", where one agent gradually mirrors its partner. We introduce SPASM (Stable Persona-driven Agent Simulation for Multi-turn dialogue generation), a modular, stability-first framework that decomposes simulation into (i) persona creation via schema sampling, plausibility validation, and natural-language persona crafting, (ii) Client--Responder dialogue generation, and (iii) termination detection for coherent stopping. To improve long-horizon stability without changing model weights, we propose Egocentric Context Projection (ECP): dialogue history is stored in a perspective-agnostic representation and deterministically projected into each agent's egocentric view before generation. Across three LLM backbones (GPT-4o-mini, DeepSeek-V3.2, Qwen-Plus) and nine Client--Responder pairings, we construct a dataset of 4,500 personas and 45,000 conversations (500 personas X 10 conversations per pairing). Ablations show ECP substantially reduces persona drift and, under human validation, eliminates echoing; embedding analyses recover persona structure and reveal strong responder-driven interaction geometry. Our code is available at https://github.com/lhannnn/SPASM.

1 Introduction

SPASM addresses instability in LLM-generated multi-turn dialogues by combining modular persona-driven simulation with Egocentric Context Projection, targeting role consistency and long-horizon behavioral stability.

  • Multi-turn LLM interactions can accumulate factual inconsistency, goal drift, instruction-adherence failures, and interaction incoherence as context grows.
  • Synthetic LLM–LLM dialogues offer explicit control over roles, personas, and interaction constraints for controllable data construction.
  • SPASM modularizes persona-driven simulation into persona generation, Client–Responder dialogue simulation, and termination detection.
  • SPASM integrates persona validation, natural-language crafting, and termination detection to support high-quality, controllable multi-turn dialogue generation.
  • ECP projects perspective-agnostic dialogue history into agent-specific views, reducing role confusion and mitigating long-horizon persona drift without changing model weights.
  • The dataset spans nine Client–Responder backbone combinations and supports geometric and behavioral analysis of persona stability and interaction dynamics.

2 SPASM

SPASM constructs plausible personas, simulates role-specific Client–Responder interactions, detects coherent stopping, and normalizes dialogue history into each agent’s relative perspective for stability analysis.

  • Figure 1 organizes SPASM into persona generation, dialogue simulation with ECP, and termination detection.
  • Persona generation: Persona profiles sample demographics, interaction context, emotional state, and behavioral patterns from predefined schema fields.
  • Persona generation: The Persona Validator checks sampled profiles for coherence and plausibility before natural-language crafting.
  • Dialogue simulation: The Client enacts the crafted persona while the Responder follows a user-defined role-specific prompt, such as listener, expert, or advisor.
  • Termination: The Termination Detector examines recent turns and predefined closure rules to stop interactions at coherent stopping points.
  • Egocentric Context Projection: ECP stores absolute speaker identities and utterance contents in a perspective-agnostic history, then maps speakers into SELF and PARTNER views for each agent.
  • Egocentric Context Projection: The projection preserves utterance content and temporal order while deterministically changing speaker labels, reducing role ambiguity and role-induced drift or echoing.
  • Drift measurement: Persona drift is measured by embedding distance between baseline and turn-t responses to the same persona probes, with higher values indicating greater deviation.

3 Experiments and Analysis

The experiments evaluate semantic structure, persona retrieval, backbone effects, and the ECP history-construction ablation across nine model pairings. Results show meaningful persona organization, responder-driven geometry, and reduced drift under ECP, with strongest gains on concerns and emotion.

  • Experimental Setup: Nine Client–Responder configurations pair GPT-4o-mini, DeepSeek-V3.2, and Qwen-Plus, using 500 personas and 10 conversations per configuration.Temperatures are 0.7 for generative components and 0.3 for validation and termination judgments.
  • Experimental Setup: Conversations are embedded after concatenating client utterances, reduced to 50 PCA components, and analyzed with cluster, distance, and retrieval measures.The analyses include Silhouette score, Davies–Bouldin index, within- versus between-persona distances, and nearest-neighbor retrieval.
  • Dataset Semantics: Same-backbone interactions produce more compact and better-separated persona clusters, including a GPT-4o-mini/GPT-4o-mini Silhouette score of 0.60 and within-cluster distance of 0.09 ± 0.07.Cross-backbone settings generally degrade clustering quality relative to same-backbone interactions.
  • Dataset Semantics: The Responder Model backbone primarily shapes interaction geometry, while the Client mainly modulates variance.GPT-4o-mini as Responder maintains Silhouette scores above 0.60 across Client backbones, whereas DeepSeek-V3.2 as Responder degrades clustering structure.
  • Dataset Semantics: Cross-backbone interactions increase intra-cluster variance while leaving persona separability largely intact, with within-cluster distance reaching 0.34 ± 0.22 versus approximately 0.09 in same-backbone settings.Between-cluster distances remain near 0.5, and the first two principal components explain approximately 68–77% of variance.
  • Persona Retrieval: Persona retrieval accuracy rises with K and remains above random-label baselines, but Top-1 accuracy varies across pairings, indicating non-deterministic conversation-level consistency.The results support a meaningful persona signal alongside substantial intra-persona variability.
  • History Construction Ablation: ECP renders perspective-specific views from perspective-agnostic history and is evaluated against CONCAT with matched roles, schedules, backbones, and deterministic decoding.The ablation uses 50 personas, three conversations each, 20-utterance caps, and probe-based drift measures for concerns, emotions, and motivations.
  • History Construction Ablation: ECP consistently lowers persona drift across backbones, with the largest reported emotion reduction under GPT-4o-mini at Cohen’s d = −0.75.Turn-level trends show lower ECP drift after the initial turns for concerns, emotion, and motivation; motivation improvements vary by backbone, with no reliable DeepSeek change.

4 Related Work

Prior work uses LLM self-chat and role-playing to synthesize multi-turn dialogue, but standard concatenated histories do not explicitly stabilize long-horizon role and persona fidelity. SPASM addresses this gap with a stability-first framework for controllable LLM–LLM generation.

  • Synthetic Multi-turn Dialogue Generation: Recent synthetic-data methods extend single-turn instruction generation to multi-turn self-chat and agent-based role-playing.Examples include Ultra-Chat, Baize, and CAMEL.
  • Stability Gap: Standard history concatenation leaves long-horizon role and persona fidelity as a central bottleneck because existing setups lack explicit stabilizing mechanisms.Related failures include instruction drift, personality shift, and echoing.
  • Stability Gap: SPASM fills this gap with a stability-first framework for controllable LLM–LLM dialogue generation.The framework targets identity-related failures during scalable multi-turn data generation.

5 Conclusion

SPASM combines modular persona-driven simulation with ECP and termination detection to generate stable multi-turn dialogues. Across backbone configurations, the analyses find persona structure, responder-dominated interaction geometry, reduced drift, and no echoing under human validation.

  • Framework and Findings: SPASM combines persona sampling, validation, crafting, ECP-based history construction, and natural termination detection in a practical data-generation pipeline.The framework is designed for persona-driven multi-turn dialogues with long-horizon behavioral stability.
  • Framework and Findings: Across three LLM backbones and nine Client–Responder configurations, conversations exhibit clear persona structure in embedding space and systematic backbone-pairing effects.The responder model dominates emergent interaction geometry.
  • Framework and Findings: ECP reduces persona drift across probe dimensions and eliminates echoing under full human validation, which is prevalent with standard history concatenation.This conclusion is based on the reported ablations and human-validated echoing comparison.
  • Implications: The released dataset and framework support future work on controllable dialogue synthesis, robust evaluation, and stable agent simulation.The stated support scope follows the paper’s conclusion.

Limitations

The evaluation focuses on controlled two-agent Client–Responder dialogue simulation and leaves several broader settings and representation questions open.

  • The evaluation covers a small set of primarily English-language, instruction-tuned models, leaving other architectures, languages, and smaller models unexplored.
  • SPASM is designed for two-agent interactions with clearly defined roles, excluding multi-agent groups and dynamically changing roles.
  • Structured schemas and natural-language descriptions may not fully capture the richness or variability of real human personas.
  • Human assessments of echoing are inherently subjective and may not scale easily to larger datasets or broader domains.

Ethical Considerations

The paper frames its simulation framework as research infrastructure while recognizing both expressiveness benefits and the need for safeguards and oversight.

  • SPASM enables controllable persona-driven simulation for synthetic dialogue data generation.
  • The framework could potentially be misused to generate deceptive or manipulative interactions if applied irresponsibly.
  • Appropriate safeguards, usage policies, and human oversight are necessary when applying synthetic dialogue data to downstream systems.
  • Per-role generation strictly contains single-agent one-shot generation because role-specific prompts and decoding policies provide additional flexibility.
  • The containment result does not imply better data quality, which requires a separate empirical comparison.
  • The simulation algorithm abstracts persona sampling and validation, Client–Responder interaction, and natural termination using a sliding window of recent turns.

C Theoretical Justification for the Evaluation Metric

The metric treats persona consistency as continued adherence to a system-prompt persona despite accumulating dialogue history and contextual interference.

  • Persona consistency mainly depends on whether the model continues following its system prompt as dialogue history grows.
  • Accumulating history may introduce contextual interference that weakens adherence to the persona specification.
  • The evaluation obtains baseline responses to persona probes before interaction begins using the same system prompt and temperature 0.
  • Before each dialogue turn, a separate probe-only call uses the accumulated history and the same persona specification to query the fixed persona probes.
  • Larger embedding distance from baseline responses serves as a tractable proxy for reduced persona consistency under paraphrase-invariant embeddings.

D Details of Semantic Metrics

The semantic analyses embed client utterances, reduce dimensionality with PCA, and quantify persona cohesion, separation, clustering quality, and drift geometrically.

  • Client-side utterances are concatenated and encoded with text-embedding-3-large, with cosine distance used as the base dissimilarity.
  • PCA retains the top 50 components, and clustering and distance statistics are generally computed in that reduced space using cosine distance.
  • The Silhouette score averages each conversation’s separation from its own cluster relative to the nearest alternative cluster.
  • The Davies–Bouldin index compares within-persona scatter with inter-centroid distance, with lower values indicating better cluster separation.
  • Within-persona distance measures each conversation’s cosine distance to its own persona centroid, while between-persona distance uses the nearest other-persona centroid.
  • The reported statistics summarize the means and standard deviations of within- and between-persona distances for each backbone setting.
  • Drift compares the semantic embedding of a turn-t persona-probe response with its baseline, and larger drift corresponds to greater angular separation between normalized embeddings.

G Persona Drift Probes and Metrics

Persona drift is measured by probing stable values, coping patterns, and motivations across interaction turns, then comparing probe embeddings with pre-interaction baselines. The resulting cosine-distance trajectories and AUC quantify longitudinal semantic change, where lower AUC indicates less cumulative drift.

  • Probe dimensions: Three introspective questions probe persona values, emotional coping patterns, and underlying motivations across interactions.These dimensions target psychologically grounded attributes expected to remain relatively stable over short- to medium-term interactions.
  • Probe protocol: Probe responses are collected at predefined turns and compared with baseline responses recorded before each interaction.The protocol enables consistent comparisons across rounds, models, and experimental conditions.
  • Drift computation: Cosine distance between each probe embedding and its corresponding baseline embedding defines drift for each persona dimension.The reported computation is dcos(a, b) = 1 − a⊤b ∥a∥∥b∥.
  • Aggregation: Turn-wise drift is averaged across personas and conversations separately for Concerns, Emotion, and Motivation dimensions.The analysis plots mean trajectories with uncertainty bands across persona–conversation units.
  • AUC summary: Lower area under the drift curve indicates reduced cumulative persona drift over the interaction horizon.AUC summarizes each unit’s cumulative drift before condition-level averaging.

H UMAP of Dataset

The UMAP and retrieval analyses examine how persona information appears in Client conversation embeddings across nine Client–Responder pairings. Cross-backbone differences primarily increase within-persona dispersion, while original-label retrieval remains above shuffled-label performance.

  • Embedding geometry: Cross-backbone settings primarily increase within-persona dispersion rather than collapsing separation between personas.The resulting two-dimensional projections may appear more overlapped, but the quantitative interpretation attributes differences mainly to intra-cluster variance.
  • UMAP visualization: Across nine Client–Responder pairings, UMAP panels show conversation embeddings as points colored by persona identity.The visualization compares embedding layouts across all backbone combinations.
  • Persona retrieval: Original persona-label retrieval consistently outperforms the shuffled-label baseline across all settings.Top-k nearest-neighbor retrieval is used as a diagnostic for non-trivial persona information rather than as an optimization target.
  • Retrieval figure: Figure 4 plots persona retrieval accuracy against Top-K nearest neighbors for nine Client →Responder pairings.Green curves represent original persona labels, while purple curves represent the shuffled-label baseline.

L Inter-Annotator Agreement and LLM Judge Evaluation

The paper evaluates echoing with human annotation and LLM-judge comparisons, while analyzing role-label ambiguity, symmetric feedback, and post-training alignment as possible sources of drift. A case study illustrates how a Client can adopt the Responder’s advisory and supportive behavior.

  • Human agreement: Two trained annotators independently label a random sample of 200 conversations to measure inter-annotator agreement.Observed agreement and Cohen’s κ are reported for the binary task with class imbalance.
  • LLM-judge evaluation: LLM judges are compared with averaged human references on the CONCAT condition using agreement, precision, recall, and F1.The evaluation focuses on a condition containing positive echoing cases.
  • Echoing annotation: Human annotation labels a conversation as echoing if any turn exhibits partner-role adoption, otherwise assigning no-echoing.The Conversation Dataset Viewer supports blinded inspection and binary annotation across the dataset.
  • Role-label ambiguity: Role-label ambiguity can make partner messages appear to be an agent’s own continuation target under shared absolute-role templates.ECP addresses this mismatch by presenting consistent SELF versus PARTNER interpretations.
  • Alignment boundary: ECP mitigates alignment-related assistant-like drift without changing model weights, but does not eliminate the proposed alignment root cause.Its context-level intervention reduces assistant-continuation cues rather than modifying post-training behavior.
  • Feedback mechanism: A symmetric feedback loop can amplify small role deviations because each agent conditions on the other’s outputs.ECP is described as damping this loop by preventing role leakage from becoming a global conversational norm.
  • Case study: In the CONCAT case study, the Client gives a responder-like budgeting suggestion at turn 3 and supportive language at turn 13.These utterances exemplify adoption of advisory and emotionally supportive roles associated with the Responder.
Loading 2604.09212v1…