Source-linked AI summary
Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles
Kuang Wang, Xianfei Li, Shenghao Yang, Li Zhou, Feng Jiang, Haizhou Li
TL;DR
Existing user simulators can lack authentic utterances, conversation-level control, and realistic user diversity, while user traits in real conversations are often implicit. USP extracts profiles from conversations and conditions multi-stage simulation on them; it reports stronger authenticity and diversity with comparable consistency, and dynamic evaluations align closely with established benchmarks.
Problem
Method
USP extracts implicit user profiles from dialogues and uses profile-conditioned supervised fine-tuning and cycle-consistency reinforcement learning for utterance- and conversation-level simulation.
Source: §1 · p. 2Results
USP significantly outperforms role-playing and direct-simulation baselines in authenticity and diversity while achieving comparable sentence- and conversation-level consistency.
Source: §7 · p. 9Takeaways & Limitations
USP-based dynamic LLM evaluation aligns closely with established benchmarks and enables finer-grained performance assessment across diverse user groups.
Source: §1 · p. 2Experiments use a single dataset with minimal validation on others, and the English-dialogue focus may limit applicability to other languages and cultural contexts.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
User simulators are crucial for replicating human interactions with dialogue systems, supporting both collaborative training and automatic evaluation, especially for large language models (LLMs). However, current role-playing methods face challenges such as a lack of utterance-level authenticity and user-level diversity, often hindered by role confusion and dependence on predefined profiles of well-known figures. In contrast, direct simulation focuses solely on text, neglecting implicit user traits like personality and conversation-level consistency. To address these issues, we introduce the User Simulator with Implicit Profiles (USP), a framework that infers implicit user profiles from human-machine interactions to simulate personalized and realistic dialogues. We first develop an LLM-driven extractor with a comprehensive profile schema, then refine the simulation using conditional supervised fine-tuning and reinforcement learning with cycle consistency, optimizing at both the utterance and conversation levels. Finally, a diverse profile sampler captures the distribution of real-world user profiles. Experimental results show that USP outperforms strong baselines in terms of authenticity and diversity while maintaining comparable consistency. Additionally, using USP to evaluate LLM on dynamic multi-turn aligns well with mainstream benchmarks, demonstrating its effectiveness in real-world applications.
1 Introduction
USP addresses gaps in role-playing and direct user simulation by extracting implicit profiles and conditioning multi-stage dialogue reconstruction on them. It reports stronger authenticity and diversity, comparable consistency, and close alignment with established benchmarks in dynamic LLM evaluation.
1 Introduction
- Predefined-profile role-playing can lack utterance authenticity and confuse user and assistant roles, while text-only simulators lack dialogue control and authentic user-level diversity.These limitations constrain conversation-level consistency and diverse simulation without seed context.Source: §1 · p. 2
- USP treats user simulation as dialogue reconstruction, first extracting implicit user characteristics and then generating dialogue conditioned on the resulting profile.The framework is proposed to address limitations of existing simulation approaches.Source: §1 · p. 2
- Its LLM-driven extractor uses objective facts and subjective characteristics, then turns profile attributes into natural descriptions intended to support generalization.The schema contains a dozen attributes for comprehensive user description.Source: §1 · p. 2
- Conditional supervised fine-tuning targets utterance-level simulation, while cycle-consistency reinforcement learning targets conversation-level consistency; a diverse sampler captures authentic user-profile distributions.USP-based dynamic evaluation aligns closely with established benchmarks and supports finer-grained assessment across user groups.Source: §1 · p. 2
- USP improves semantic and stylistic similarity in reconstructed multi-turn dialogues by approximately 34% and 43%, respectively, versus the leading baseline, and halves reconstruction errors.It also outperforms baselines in authenticity and diversity while maintaining comparable consistency.Source: §1 · p. 2
2 Related Works
Related work moves from early constrained simulators to LLM-based conversational approaches, then considers persona-based methods for improving controllability and diversity. The paper positions its focus on authenticity, consistency, and diversity within this latter line.
2.1 General User Simulator
- LLM advances enabled flexible self-chat and dual-LLM simulation, while trained systems such as PlatoLM and Parrot learn discourse patterns from human-LLM conversations.Earlier agenda- and model-based simulators focused on limited action prediction under natural-language-generation constraints.Source: §2.1 · p. 2
2.2 Persona-based User Simulator
- Persona-based work includes generated goals and retrieved historical personas, while prompt-driven LLM methods simulate users with predefined profiles, personalities, or preferences.USP follows this line while focusing on authenticity, consistency, and diversity.Source: §2.2 · p. 3
3 Task Definition
The task is to reconstruct a target user's multi-turn behavior while preserving fidelity at both utterance and dialogue levels. USP reformulates this as profile extraction followed by profile-conditioned reconstruction.
3 Task Definition
- User simulation is formulated as reconstructing multi-turn user behavior while minimizing both utterance-level and dialogue-level distance.The target dialogue consists of user utterances paired with corresponding answers.Source: §3 · p. 3
- Direct simulation has difficulty capturing personalized traits, and real-world user profiles are often implicit and difficult to derive.The passage contrasts this with role-playing using specific profiles to enable diverse simulations.Source: §3 · p. 3
- USP extracts a profile from the target dialogue and reconstructs the dialogue using ground-truth context, with a hyperparameter balancing utterance- and dialogue-level distances.The profile is defined as the extractor's output for the target dialogue.Source: §3 · p. 3
4 Modeling User Simulator with Implicit Profiles
USP minimizes a dialogue-reconstruction objective through profile extraction, conditional SFT, RLCC, and corpus distribution fitting, targeting utterance authenticity, conversation consistency, and user diversity.
4 Modeling User Simulator with Implicit Profiles
- USP combines profile extraction, conditional SFT, RLCC, and corpus distribution fitting to target utterance authenticity, conversation consistency, and user-level diversity.The framework organizes these components as stages for minimizing its dialogue-reconstruction objective.Source: §4 · p. 3
4.1.1 User Profile Schema
- The profile schema combines objective facts about personal and situational attributes with subjective characteristics covering communication style and Big-Five traits.The attributes are reformulated as coherent narrative descriptions rather than discrete profile fields.Source: §4.1.1 · pp. 3–4
4.1.2 User Profile Extractor
- An LLM-driven extractor identifies user attributes in human–LLM conversations, collects valid attributes, and polishes them into natural-language profile descriptions.The extractor uses a designed prompt and can use an advanced LLM such as GPT-4o.Source: §4.1.2 · p. 4
4.1.3 Profile Quality Verification
- Dialogue Profile Consistency treats profile–dialogue consistency as retrieval, combining precision and recall; a separate 1–5 Validation Score rates subjective-characteristic descriptions.Factual Consistency scores atomic facts against a source using NLI; DPC is the harmonic mean of its precision and recall components.Source: §4.1.3 · pp. 4–5
4.2 Conditional Supervised Fine-Tuning
- Conditional SFT trains utterance generation on both the extracted user profile and preceding dialogue context.Its language-modeling loss optimizes the user utterance.Source: §4.2 · p. 5
4.3 Diverse Profile Sampling
- Diverse Profile Sampling fits a distribution over embedded profiles, then samples realistic profiles and creates plausible variants by combining nearest-neighbor descriptions.The method embeds profiles with SimCSE, reduces dimensionality with UMAP, and fits a Gaussian kernel density estimate.Source: §4.3 · p. 5
4.4 Reinforcement Learning with Cycle Consistency
- RLCC addresses SFT’s incomplete profile reflection by rewarding similarity between target and dialogue-reflected profiles and optimizing profile recall with PPO.Its reward also includes AI detection as an auxiliary component, with λ = 0.8 prioritizing cycle consistency.Source: §4.4 · p. 5
5 Experiments
USP is evaluated for profile quality and for utterance- and conversation-level authenticity, consistency, continuity, and diversity. Across these evaluations, it performs strongly against baselines, including in human judgments and alignment with target dialogue distributions.
5.4 Metrics
- Evaluation covers authenticity, profile-generation consistency, diversity, and multi-turn continuity, using embedding similarity, reverse profile-dialogue metrics, ADV, and early stop rate.The study assesses utterance-level responses given golden context and multi-turn dialogues generated through interactions with GPT-4o.Source: §5 · p. 5, §5.4 · p. 6
5.1 Datasets
- The LMSYS-USP dataset contains 94,874 filtered conversations, and extracted profiles exceed 84% DPC with manual average scores above 4 out of 5.The data are split into 87,882 training, 4,626 validation, and 2,366 test conversations; profile quality was also evaluated on Persona-Chat and ConvAI21.Source: §5.1 · p. 6
5.2 Configurations; 5.3 Baseline Models
- The experiments compare profile-agnostic simulators, profile-conditioned role-playing systems, and USP, which is trained from LLaMA-3-8B-Base using conditional SFT and an RLCC phase.The reported setup uses 4 A100 GPUs for three SFT epochs and two H20 GPUs for one RLCC epoch.Source: §5.3 · p. 6, §5.2 · p. 6
5.5.1 Utterance-Level Evaluation
- USP leads utterance-level baselines in Sem-Sim (53.38) and Style-Sim (46.60), while profile-conditioned role-playing scores higher in r-DP.P through direct keyword copying.The reported balance of authenticity and consistency contrasts with high P.Cover from GPT-4o (User w/ Profile).Source: Table 3 · p. 7, §5.5.1 · p. 7
5.5.2 Conversation-Level Evaluation
- USP outperforms conversation-level baselines in authenticity, consistency, and continuity, with the lowest ESR (10), r-DP.R of 74.38, and r-DPC of 64.05.Its multi-turn authenticity advantage is especially evident, while profile-based role-playing baselines can have high P.Cover and r-DP.P but lower overall profile-dialogue consistency.Source: §5.5.2 · p. 7, Table 4 · p. 8
5.5.3 Human Evaluation
- USP wins against GPT-4o (User w/ Profile) in human-rated authenticity (74 vs. 13) and consistency (61 vs. 35).The USP versus USP (w/o RLCC) consistency gap (43 vs. 30) exceeds the authenticity gap (37 vs. 31).Source: §5.5.3 · pp. 7–8, Table 5 · p. 8
5.5.4 Diversity Sampling Evaluation
- USP series have lower ADV than baselines across percentiles; 60% of samples achieve ADV below 5%, compared with at least 10% for cited baselines.Figure 3’s red cross marks the 60% threshold for USP (w/o RLCC).Source: Figure 3 · p. 8, §5.5.4 · p. 8
6 Analysis
The analysis tests profile polishing and RLCC reward weights, considers dynamic multi-turn evaluation of LLMs, and illustrates how USP expresses profile traits in dialogue. It reports trade-offs in reward weighting and concrete differences from profile-conditioned role-playing.
6.1 Ablation Study
- Polishing profile attributes into natural-language descriptions improves generalization across most continuity, authenticity, and consistency metrics; attribute-only profiles prompt direct replication and P.Cover of 50.35.The comparison is between polished profiles and a baseline using attributes without polishing.Source: §6.1 · p. 8
- Among tested RLCC reward weights, λ = 0.8 best balances model capability and dialogue consistency; λ = 0.9 sacrifices style authenticity, while λ = 0.5 lacks sufficient consistency.The higher setting raises P.Cover without improving r-DPC, whereas the lower setting leaves performance stagnant.Source: §6.1 · p. 8
6.2 Applications: Dynamic Multi-turn Evaluation For LLMs
- USP can interact dynamically with LLMs over multiple rounds, adapting to response quality and user traits to reveal fine-grained weaknesses across user groups.The paper presents this as an application beyond static evaluations using preset questions.Source: §6.2 · pp. 8–9, Table 7 · p. 9
6.3 Case Study
- In examples, USP conveys abstract traits through concrete, coherent details—such as mentioning a son and daughter for “father of two”—rather than repeating profile wording verbatim.The comparison is with GPT-4o (User w/ Profile), which often copies abstract traits directly.Source: §6.3 · p. 9, Table 8 · p. 9
7 Conclusion
USP integrates extracted user profiles through conditional SFT and RLCC. Automatic and human evaluations report stronger authenticity and diversity than role-playing and direct-simulation baselines, comparable consistency, and usefulness for dynamic multi-turn LLM evaluation.
7 Conclusion
- USP combines extracted user profiles with conditional SFT and RLCC, outperforming role-playing and direct-simulation approaches in authenticity and diversity while maintaining comparable consistency.The reported comparison applies at sentence and conversation levels and is supported by automatic metrics and human evaluations.Source: §7 · p. 9
Limitations
The paper notes that its experiments cover one dataset with little cross-dataset validation and focus on English dialogues. Its preprocessing removes non-English, toxic, and duplicate content and preserves context when truncating dialogues.
Limitations
- Experiments use a single dataset with minimal validation elsewhere, and the English-dialogue focus may limit applicability to other languages and cultures.These limitations bound evidence for broader scenario applicability and linguistic or cultural transfer.Source: Limitations · p. 9
A.1 Preprocessing
- Preprocessing removes non-English, toxic, and duplicate dialogues, then segments conversations while ensuring truncated sequences start with an assistant turn.The stated aim is to preserve discourse integrity, context consistency, and coherence.Source: §A.1 · p. 13
A.2 Profile Dataset
The profile dataset combines scene-consistent, scene-related, and deep intrinsic attributes extracted from conversations, then rewrites them as natural descriptions. The appendices also report construction costs, model setups, and human assessments of profile quality.
A.2 Profile Dataset
- Profiles combine scene-consistent, scene-related, and deep intrinsic attributes; extracted attributes are filtered, shuffled, and rewritten as natural descriptions.Profiles average 231 tokens; dataset splits average 1,149 training, 1,295 validation, and 1,438 testing tokens.Source: §A.2 · p. 13
- Big Five traits are retained for polishing only when scores are significantly high or low; moderate scores are excluded as average human behavior.The attribute schema also draws on cited Big Five traits and metric definitions.Source: §A.2 · p. 13
A.3 Resource Consumption in Implementation
- $1,400 is the reported total dataset-construction cost, including $940 for extracting attributes from 94,000 samples.Attribute rewriting adds $0.05 per sample.Source: §A.3 · p. 13
B.1 Trainable Model Setup
- The profile generator is fine-tuned from LLaMA3-8B-Instruct to distill GPT-4o’s two-stage profile generation, using four A100 GPUs for three epochs.The training run lasted two days.Source: §B.1 · p. 14
B.2 Train-Free Model Setup
- Train-free setups distinguish response models, which need a generated opening query, from simulators that directly generate user utterances from a seed prompt.GPT-4o (User with Profile) and PlatoLM are examples, respectively.Source: §B.2 · p. 14
B.3 Human Evaluation
- Human evaluators were required to have IELTS scores of at least 6.5 and a computer-science background with dialogue-systems knowledge.Eight annotators later evaluated conversation-level authenticity and consistency.Source: §B.3 · p. 14
B.3.1 Profile Evaluation
- Profile evaluation used two annotators to rate objective-fact accuracy, completeness, and reasonableness of subjective descriptions on a 1–5 scale.The paper reports moderate-to-high annotator agreement.Source: §B.3.1 · p. 14
- 89.2% objective-fact, 74.3% subjective-character, and 88.4% naturalness 1-point agreement rates were reported for profile evaluation.Two annotators assessed accuracy, completeness, and reasonableness on a 1–5 scale.Source: Table 10 · p. 14, §B.3.1 · p. 14
B.3.2 Dialogue Evaluation
Dialogue evaluation combines human judgments of authenticity and profile consistency with a density-sampler assessment and illustrative generated-dialogue comparisons. The sampler is reported to balance preservation of majority patterns with coverage of minority cases.
B.3.2 Dialogue Evaluation
- Eight annotators assessed authenticity against reference dialogues and consistency against user profiles, with dialogue assignments and presentation order randomized.Four pairs independently reviewed randomly assigned samples.Source: §B.3.2 · p. 14
C.1 Sampling strategy effectiveness
- The sampler’s overall result is described as balancing distribution preservation with targeted sampling of minority cases.High-density sampling captures majority patterns, while low-density weighting seeks minority coverage; LDL and Uniformity Loss assess structure and global coverage.Source: §C.1 · pp. 14–15
C.2 Case Study
- In the case study, USP preserved stylistic and semantic alignment; PlatoLM diverged by turn four, while GPT-4o with a profile repeated praise.USP also integrated objective details and subjective traits in a consistency example involving unseen profiles.Source: §C.2 · p. 15
C.3 Downstream analysis
The downstream analysis reports performance patterns across demographic groups and dialogue turns, while the appendix supplies prompts for profile extraction, rephrasing, and evaluation. These materials address both user-trait capture and consistency or authenticity judgments.
C.3 Downstream analysis
- LLMs performed less effectively with minority demographic groups, while virtual-group performance remained reasonable and instruction-following declined over dialogue turns.The analysis also reports greater turn-to-turn performance volatility for minority groups.Source: §C.3 · p. 15, Figure 6 · p. 16
D Prompt Templates
- Case-study figures compare profile-based simulators over dialogue turns, including USP and GPT-4o using profiles and PlatoLM using first-turn golden context.The examples target reconstruction of reference dialogues.Source: Figure 9 · p. 17, Figure 10 · p. 18
- Profile-extraction prompts seek specific persona attributes from direct and implicit cues, including communication style, scene goals, and Big Five traits.The prompt asks for high, low, or inconclusive trait scores and evidence-based justifications.Source: D Prompt Templates · pp. 19–21
- The appendix includes prompts that rephrase key-value attributes into coherent second-person descriptions and prompts that compare a generated user with a reference dialogue.Additional prompts assess consistency between profiles, dialogue histories, and utterances, as well as subjective-characteristic quality.Source: D Prompt Templates · pp. 16, 22, 23, 24