Source-linked AI summary
SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue
Yuqin Dai, Ning Gao, Wei Zhang, Jie Wang, Zichen Luo, Jinpeng Wang, Yujie Wang, Ruiyuan Wu, Chaozheng Wang
TL;DR
Service-dialogue agents are limited by scarce, noisy conversational data and difficulty simulating authentic, goal-oriented users. SEAD addresses this with decomposed user modeling for adaptive scenario generation and realistic role-play, and reports stronger performance than foundation-model and commercial baselines without large-scale annotation.
Problem
Robust multi-turn service dialogue is limited by scarce, expensive, low-quality conversational data and difficulty modeling dynamic user behavior.
Method
SEAD separates user modeling into a Profile Controller that generates diverse initial states for curriculum learning and a User Role-play Model that simulates realistic responses.
Results
SEAD outperforms open-source foundation and closed-source commercial models with minimal parameters and zero annotation.
Takeaways & Limitations
SEAD provides a resource-efficient self-evolving framework for multi-turn service dialogue in data-scarce settings.
Takeaways & Limitations
Evaluation currently prioritizes task completion, does not yet assess user satisfaction and emotional comfort adequately, and has not extended to multiscenario environments.
Abstract
from arXiv · showhide
Large Language Models have demonstrated remarkable capabilities in open-domain dialogues. However, current methods exhibit suboptimal performance in service dialogues, as they rely on noisy, low-quality human conversation data. This limitation arises from data scarcity and the difficulty of simulating authentic, goal-oriented user behaviors. To address these issues, we propose SEAD (Self-Evolving Agent for Service Dialogue), a framework that enables agents to learn effective strategies without large-scale human annotations. SEAD decouples user modeling into two components: a Profile Controller that generates diverse user states to manage training curriculum, and a User Role-play Model that focuses on realistic role-playing. This design ensures the environment provides adaptive training scenarios rather than acting as an unfair adversary. Experiments demonstrate that SEAD significantly outperforms Open-source Foundation Models and Closed-source Commercial Models, improving task completion rate by 17.6% and dialogue efficiency by 11.1%. Code is available at: https://github.com/Da1yuqin/SEAD.
1 Introduction
SEAD addresses scarce, low-quality service-dialogue data and shortcomings in existing static, interactive, and self-evolving training paradigms. It decomposes user modeling to support diverse, realistic, curriculum-adaptive scenarios and reports stronger performance without large-scale annotations.
- Motivation: Service-dialogue training is constrained by scarce, expensive, and low-quality conversational data, especially failed and nonstandardized human logs.These limitations make robust multi-turn state tracking and adaptive strategy learning difficult.
- Prior limitations: Static synthesis cannot capture reactions to agent behavior, while interactive simulation often lacks curriculum-based difficulty adjustment for coherent long-term interactions.The resulting limitations are particularly relevant to complex multi-turn service dialogues.
- SEAD framework: SEAD decomposes the user side into a profile generator and a role-play model, restricting adversarial training to initial-state selection rather than response generation.The profile generator samples and checks diverse user scenarios, while role-play simulates responses during interaction.
- Evaluation: 120 initial user-state combinations span 5 cooperation levels, 4 emotion levels, and 6 trust levels in an outbound-call enterprise scenario.The combinations are evaluated through multi-level metrics.
- Results: SEAD outperforms foundation-model and large-API baselines while enabling smaller models, lower costs, and effectiveness in data-scarce domains.The introduction reports better goal guidance, conversation efficiency, user-state understanding, and authentic behavior simulation without large-scale annotations.
- SEAD framework: SEAD requires no large-scale annotated dialogue data and uses user profiles and standard operating procedures as inputs for self-evolving service-dialogue training.It is presented as the first self-evolving framework for multi-turn service dialogues.
2 Related Works
Prior task-oriented dialogue systems largely rely on static fine-tuning, while self-evolving agents have expanded from self-rewarding and verifiable domains toward external environments and curated data. SEAD extends this line to realistic multi-turn service dialogue with joint user-role-play and agent evolution.
- Task-oriented Dialogue: Task-oriented dialogue systems support complex inquiries, but traditional neural systems and early user simulations face architectural limitations.Recent LLM-based approaches predominantly use static fine-tuning, sometimes supplemented by retrieval.
- Self-evolving Agents: Self-evolving agents use iterative generation and refinement with minimal supervision, including self-rewarding mechanisms and Coder-Tester frameworks for verifiable domains.Later work incorporates external environments and curated data to enhance evolution.
- Self-evolving Agents: SEAD distinctively evolves a user role-play model and a customer service agent within a realistic environment using a reward that measures task completion and user satisfaction.Its framework samples user states, role-plays users in multi-turn dialogues, and trains the service agent through interaction.
3 Methodology
SEAD separates user-state generation from realistic role-playing, then co-evolves service-agent training through adaptive profile sampling, multi-turn interaction, reward optimization, and mistake analysis.
- Framework Components: SEAD decouples user modeling into a Profile Controller that samples initial states and a User Role-Play Model that simulates responses without controlling outcomes.Only the profile generator participates in adversarial training, while the role-play model focuses on realistic simulation.
- Framework Components: The framework samples diverse user profiles, enacts them in multi-turn dialogues with a service agent, and feeds dialogue data back into the evolving loop.Profiles include initial cooperation, emotion, trust, and behavior configurations.
- User State Space Design: User states evolve autonomously from agent behavior, with effective responses improving cooperation, emotion, and trust while poor interactions degrade them.These evolving states support reward computation and training realism.
- Self-Evolving Training Loop: SEAD prioritizes profiles with completion rates near 0.5 after initial random sampling, while validating consistency and deduplicating profiles.This adaptive curriculum targets moderate difficulty rather than fixed scenarios.
- Self-Evolving Training Loop: Multi-turn dialogues produce trajectories whose task-completion outcomes provide rewards for online service-agent optimization.The task reward is 1 for successful completion and 0 otherwise.
- Self-Evolving Training Loop: Mistake Analysis classifies configurations as too easy, too difficult, or ideal using completion-rate thresholds and feeds the analysis back into profile sampling.Ideal configurations satisfy CR ∈[0.4, 0.6], maintaining learning difficulty near a 50% completion rate.
- Training Optimization: GRPO updates only the service agent using batch-relative advantages, avoiding a separate value network and reducing computational requirements.The policy gradient uses trajectory rewards and the service-agent policy over dialogue histories.
4 Experiments
SEAD is evaluated on a restaurant-promotion outbound-call task requiring agents to handle varied reactions, build trust, address concerns, and sustain engagement across multiple turns without dialogue training data.
- Evaluation Setting: SEAD evaluates outbound-call agents on restaurant-service promotion, where success requires convincing restaurant owners to participate.The task emphasizes diverse reactions, trust building, concern handling, and multi-turn engagement.
- Evaluation Setting: The training setting requires no dialogue data and uses only standard operating procedures, a task objective description, and user profiles.Agents autonomously explore service strategies under these inputs.
- Experimental Configuration: The experiment uses Qwen2.5-14B-Instruct for all components and constructs 120 initial states from cooperation, emotion, and trust dimensions.The state dimensions contain 5 cooperation levels, 4 emotion levels, and 6 trust levels.
4.3 Evaluation Metrics
The evaluation measures service-agent success, efficiency, user-state understanding, state changes, inference cost, and user-role-play quality across dialogue simulations.
- Completion Rate measures the percentage of dialogues in which users agreed, while Average Turns to Target measures successful-dialogue length, with lower values indicating greater efficiency.
- User Portrait Accuracy measures predicted user-state accuracy using errors across cooperation, emotion, and trust dimensions.
- Emotion, Trust, and Cooperation Improvement measure average changes from initial to final user states.
- Total Cost measures cumulative inference cost in CNY for 1,000 multi-turn dialogue samples for API-based models.
- User role-play quality is assessed across humanness, emotion, trust, cooperation, and violation using human-annotated few-shot evaluation criteria.
4.4 Baselines
SEAD is compared with open-source foundation models and closed-source commercial APIs, while supervised fine-tuning is excluded because suitable data and annotation resources are unavailable.
- The study evaluates 14B, 32B, and 72B Qwen2.5-Instruct models using prompts containing task descriptions and standard operating procedures.
- Commercial baselines include GPT-4o, DeepSeek-Chat, Qwen3-235B-A22B, and LongCat-Flash-Chat, all evaluated with dialogue-task prompts.
- Supervised fine-tuning baselines are omitted because annotated dialogue data is unavailable and manual annotation is prohibitively costly.
- 52.0% completion rate is achieved by SEAD with a 14B model, exceeding GPT-4o at 44.2% by 17.6%.
- SEAD records the lowest Average Turns to Target at 9.6, indicating more concise successful dialogues.
- SEAD remains competitive with LongCat-Flash on user-state tracking despite LongCat-Flash having 40× more parameters and extensive service-dialogue pre-training.
4.6 User Role-Play Model Performance
The user role-play model is evaluated against human and failed-model behavior using anonymized enterprise dialogue patterns, with results indicating realistic, diverse, and strategy-responsive simulation.
- Table 2 assesses user-role-play quality using humanness, emotion, trust, cooperation, and violation dimensions.
- All humanness metrics exceed 4.5/5, indicating highly realistic simulated behavior under the table’s scoring convention.
- The simulator’s violation score is 1.15/5, reflecting hesitation and pauses rather than severe communication disruptions.
- The Profile Controller generates diverse users ranging from cooperative to skeptical, capturing behavioral heterogeneity for training.
- Consistent scores across perfect-human, SEAD, and low-quality agent tiers indicate that the simulator responds to different agent strategies rather than following fixed scripts.
4.7 Ablation Study
The ablation study tests decomposed user modeling, Profile Sampling, and Mistake Analysis, finding that the full combination supports realistic simulation and effective adaptive training.
- The ablation compares configurations that remove or retain decomposed user modeling, Profile Sampling, and Mistake Analysis.
- Adversarially training the User Role-play Model causes reward hacking, producing arbitrary acceptance or hang-ups, URM-H 3.3, and CR 35.2%.
- Without Profile Sampling, anomalously high TI 1.87 alongside low UPA reveals a bias toward unrealistically easy scenarios.
- The full SEAD configuration combines a fixed User Role-play Model, intelligent Profile Sampling, and adaptive Mistake Analysis in a closed training loop.
- SEAD achieves approximately 50% success-rate training scenarios and a 47.7% relative CR improvement over adversarial training while maintaining URM-H 4.7.
4.8 Case Study
The case study presents heterogeneous, challenging user personas and shows how the service agent adapts through empathy and identity defense. The ablation description identifies MA, PS, and non-adversarial role-play training as essential components.
- Ablation study: Table 3 evaluates MA, PS, adversarial URM training, and metrics including CR, ATT, UPA, EI, TI, CI, and URM-H.The caption states that all three components are essential for optimal performance.
- Challenging interactions: The User Role-play Model generates heterogeneous personas, including rude, irrational, and AI-skeptical users, to create challenging interactions.These cases span irrational hostility to deep skepticism.
- Agent strategies: The Service Agent uses empathy and identity defense to retain users, prevent premature hang-ups, and pursue task completion.The reward mechanism encourages state improvement while avoiding timeouts associated with purely empathy-driven models.
5 Conclusion
SEAD addresses data scarcity and user role-play fidelity in multi-turn service dialogues by separating curriculum learning from realistic interaction. The paper reports improved performance against open-source and closed-source baselines with minimal parameters and zero annotation.
- Conclusion: SEAD decouples user modeling into a Profile Controller for curriculum learning and a User Role-play Model for authentic interaction.This design addresses user role-play fidelity and avoids traditional adversarial-training unfairness.
- Conclusion: SEAD outperforms Open-source Foundation Models and Closed-source Commercial Models with minimal parameters and zero annotation.The conclusion states this result without reporting a specific metric in the supplied passage.
Limitations
SEAD’s evaluation currently emphasizes task completion and one scenario, leaving user satisfaction, emotional perception, comfort, and multiscenario performance for future study.
- Evaluation metrics: SEAD prioritizes task completion, but its evaluation does not yet fully assess user satisfaction, emotion perception, or user comfort.The limitation concerns evaluation metrics and the gap between intent fulfillment and broader interaction quality.
- Scenario diversity: SEAD has not yet been extended to multiscenario environments.The authors identify scenario diversity as a limitation and plan to validate the framework in broader service dialogues.