Source-linked AI summary
MADS: Multi-Agent Dialog Simulation for Robust Conversational Agents
Arkajyoti Chakraborty, Adam Zernik, Ishika Kulkarni, Mudit Arora, Hugo Lin, Justin Chiu, Alessandro Di Bari, Neha Gupta, Andreas Stolcke
TL;DR
Existing function-calling benchmarks do not model failure-prone conversation trajectories, leaving limited training resources for robustness under realistic, non-cooperative conditions. MADS is a synthetic data generation pipeline based on a multi-agent LLM simulation framework, and MADS-Bench spans six domains, 72 tools, and 55 personas. Training on MADS-Bench improves robustness beyond cooperative datasets alone, while RLVR improves action correctness and trajectory-level task success across small and large models.
Problem
Existing function-calling benchmarks do not model failure-prone conversation trajectories, leaving limited training resources for robustness under realistic, non-cooperative conditions.
Source: §1 · p. 1Method
MADS is a synthetic data generation pipeline based on a multi-agent LLM simulation framework, and MADS-Bench spans six domains, 72 tools, and 55 personas.
Source: §1 · p. 1Results
Training on MADS-Bench improves robustness beyond cooperative datasets alone, while RLVR improves action correctness and trajectory-level task success across small and large models.
Source: §6 · p. 8Takeaways & Limitations
MADS provides a scalable way to expose conversational agents to realistic failure modes in developing more robust task-oriented agents.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Task-oriented conversational agents remain fragile under realistic conditions, especially when users exhibit non-cooperative behavior. Existing function-calling benchmarks underrepresent adversarial conversation trajectories, thereby limiting the training resources available for developing robust agents. We present MADS (Multi-Agent Dialog Simulation), a synthetic data generation pipeline that produces structured, diverse dialogue trajectories. We inject adversarial conversation, producing validated conversation trajectories that capture realistic, unhappy paths. Using MADS, we construct MADS-Bench: a benchmark spanning six domains, 55 personas, and 1095 conversations, with approximately 66% of conversations containing failure-prone conversation scenarios. We evaluate the robustness of the trained models using a two-stage training approach that combines supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR).
1. Introduction
Task-oriented dialogue systems remain brittle when users deviate from scripted paths, while existing function-calling benchmarks underrepresent failure-prone multi-turn conversations. MADS addresses this resource gap with simulated, structured dialogues that include non-cooperative behavior and tool supervision.
1. Introduction
- Rigid, finite-state dialogue managers struggle with changing goals, incomplete information, and deviations from expected user behavior.These brittle interactions can increase user friction, particularly in spoken dialogue systems.Source: §1 · p. 1
- MADS targets the shortage of training resources for realistic multi-turn dialogues involving goal changes, policy violations, or infeasible requests.The paper says existing function-calling benchmarks do not model failure-prone conversation trajectories.Source: §1 · p. 1
- MADS simulates user, assistant, and tool agents to produce structured multi-turn traces with reasoning, planning, and action supervision.The traces include successful and failure-prone scenarios categorized as unhappy-path and impossible scenarios.Source: §1 · pp. 1–2
- MADS-Bench spans six domains, 72 tools, and 55 personas, with unhappy-path scenarios forming the majority of tasks.The benchmark is used to fine-tune open-source models and evaluate them across two levels.Source: §1 · p. 1
2. Related Work
Prior work addresses tool use, dialogue interaction, synthetic data, and reinforcement learning, but the cited work leaves a gap in systematic coverage of non-cooperative dialogue trajectories. MADS combines adversarial behavior injection with structured data and reward-based training.
2.1. Benchmarks and Datasets
- Existing function-calling and simulated-user benchmarks do not systematically expose agents to goal changes, tool failures, or infeasible requests.Human-annotated dialogue corpora provide naturalistic conversations but are limited in scale and lack explicit tool-use supervision.Source: §2.1 · p. 2
2.2. Synthetic Data for Tool-Augmented Dialogue
- MADS complements synthetic dialogue pipelines that prioritize coherence and tool coverage by injecting non-cooperative behavior into validated multi-turn scenarios.The cited ToolFlow and xLAM approaches generate predominantly cooperative scenarios.Source: §2.2 · p. 2
2.3. Reinforcement Learning for Tool-integrated Reasoning
- The training pipeline combines SFT with GRPO-based RLVR, computing rewards against gold reference actions rather than live execution outcomes.The paper presents this choice as applicable to simulated environments without API access.Source: §2.3 · p. 3
3. MADS-Bench
MADS simulates tool-augmented conversations with configurable personas, scenarios, and adversities, then validates and evaluates the resulting trajectories. MADS-Bench includes impossible and unhappy-path cases, with turn-level and trajectory-level evaluation procedures.
3.1. MADS Pipeline
- Three LLM-based agents simulate user, assistant, and tool interactions to generate dialogues with intermediate reasoning, planning, and tool-use traces.A validation pipeline combines rule-based checks and LLM-as-judge evaluations, accepting a conversation only when it passes all four checks.Source: §3.1 · p. 3
3.2. Dataset Construction
- Scenario configurations specify objectives, typed tools, backend seed data, and personas drawn from a catalog of 55 communication styles.Hard constraints must be satisfied for task success, while preferences may be relaxed at the user's discretion.Source: §3.2 · p. 3
3.3. Taxonomy of Unhappy Paths
- The adversity taxonomy covers partial or incrementally revealed information, goal revisions, adversarial tone, backend failures, empty results, timeouts, and policy constraints.Literally unattainable objectives receive an impossible flag and a fallback behavior for graceful refusal with alternatives; approximately 66% of the final dataset contains unhappy-path scenarios.Source: §3.3 · p. 4
3.4. Dataset Statistics
- MADS-Bench contains 189 validated conversations and 1,095 turns across 87 scenarios and six domains, split into 106 training and 83 test conversations.Home Services is held out entirely from training for cross-domain generalization evaluation.Source: §3.4 · p. 4
3.5. Evaluation
- Evaluation uses two levels: turn-level comparison with ground-truth responses and trajectory-level scoring of multi-turn reasoning and planning.The evaluation setup includes tool-use and user-facing response decisions.Source: §3.5 · p. 4, §3.5.1 · p. 4
3.5.1. Turn Level Evaluation (L-1)
- Turn-level evaluation measures action classification, tool-call accuracy, and semantic similarity for direct user-facing responses.Tool-call accuracy checks tool selection and parameter matching, including required values retrieved from conversation context.Source: §3.5.1 · pp. 4–5
3.5.2. Trajectory Level Evaluation (L-2)
- Trajectory-level L-2 scoring evaluates a conversation in temporal order and stops at the first incorrect turn, using the mean conversation score as the success rate.A turn is correct when the action type matches the reference; tool actions must also match the reference tool name under the stated matching rule.Source: §3.5.2 · p. 5
4. Experiments
The experiments train structured dialogue agents with SFT followed by RLVR, then evaluate both turn-level behavior and full-conversation success. They also establish an APIGen-trained baseline for testing performance on MADS-Bench.
4. Experiments
- Models produce structured reasoning, planning, and action outputs, and are trained with supervised fine-tuning followed by reinforcement learning with verifiable rewards.The task is framed as sequential decision-making, with tool calls or user-facing responses as actions.Source: §4 · p. 5
4.1. Training Setup
- SFT trains on 106 MADS-Bench conversation trajectories using next-token prediction over reasoning, planning, and action tokens.The reported learning rate is 2×10−4, with batch size 4 and gradient accumulation over 4 steps.Source: §4.1 · p. 5
4.2. Supervised Fine-Tuning
- SFT improves format-following and tool use but leaves hallucinated completion, fabricated tool arguments, and contradictions of tool outputs.The authors use environment-verifiable signals to motivate RL post-training for action correctness, tool invocation, and response similarity.Source: §4.2 · p. 5
4.3. Reinforcement Learning with Verifiable Rewards (RLVR)
- RLVR refines the SFT model using full rollouts and deterministic rewards for action type, exact tool and parameter matches, and semantic similarity of user-facing answers.The total reward sums the three components and ranges from 0 to 3.Source: §4.3 · pp. 5–6
- The test split holds out Home Services for cross-domain evaluation and contains 71.1% normal scenarios, with unhappy-path and impossible scenarios making up 28.9%.The held-out domain is shown alongside the scenario-complexity distribution.Source: Figure 5 · p. 6
4.4. Cross-Dataset Generalization: APIGen Baseline
- The APIGen baseline converts 198 conversations into 750 training samples, fine-tunes Qwen2.5-7B-Instruct on that data, and evaluates it on the MADS-Bench test set.The conversion pipeline adds synthesized reasoning and planning blocks and applies the MADS-Bench validation pipeline.Source: §4.4 · p. 6
5. Results and Analysis
Turn-level results show that prompt strategy interacts with supervision and that RLVR improves structured grounding. At the trajectory level, RLVR raises task success across training stages and domains, while results vary by scenario complexity and model size.
5.1. L-1 Evaluation
- Pre-Act generally yields higher action recall under MADS supervision, especially for smaller models, while ReAct can produce stronger tool F1 in some settings.The reported prompt effects vary with supervision and model capacity.Source: §5.1 · p. 6, Table 2 · p. 7
- RLVR consistently improves structured grounding metrics across both model sizes, with more reliable gains in tool selection and parameter matching than in semantic answer quality.The comparison is between MADS-supervised SFT and SFT plus RLVR.Source: §5.1 · p. 6, Table 3 · p. 7
5.2. L-2 Evaluation
- RLVR consistently improves trajectory-level task success over SFT across model sizes on the MADS-Bench test set.The authors attribute the gain to better prediction of correct actions and tool invocation throughout the conversation.Source: Table 4 · p. 7, §5.2 · p. 7
- Qwen2.5-32B trajectory success rose from 44.08% with SFT to 53.96% with RLVR.The authors also report reduced failure cascades after refinement.Source: Table 4 · p. 7, §5.2 · p. 7
- APIGen-trained models sometimes outperform MADS SFT, but RLVR improves success across domains, including held-out Home Services, for both model sizes.Home Services was excluded from training for the cross-domain comparison.Source: §5.2 · pp. 7–8, Table 5 · p. 8
- RLVR improves success on normal and unhappy-path scenarios and outperforms the APIGen baseline, but degrades performance on impossible scenarios.The RL fine-tuning set contained only five impossible cases; gains were largest for the small model but substantial for the 32B model as well.Source: §5.2 · p. 8
- The 32B model performs better overall than the 7B model, with a larger advantage on unhappy and impossible conversations.The authors connect this advantage to stronger reasoning ability on more complex tasks.Source: §5.2 · p. 8
6. Conclusions
Training on MADS-Bench improves robustness beyond cooperative datasets alone, and MADS generates structured conversations with adversities across six task-oriented domains. The paper identifies failure-prone conversational dynamics as a training-resource gap and points to broader simulator coverage as future work.
6. Conclusions
- Failure-prone dynamics such as incomplete information, changing goals, and infeasible requests remain underrepresented in training resources for robust dialog managers.Real-world task-oriented systems must handle conditions that diverge from cooperative benchmark settings.Source: §6 · p. 8
- MADS-Bench spans six task-oriented domains, 72 tools, and diverse personas, with many scenarios representing unhappy-path or impossible interactions.MADS injects conversational adversities and execution-level failures into structured multi-turn conversations.Source: §6 · p. 8
- RLVR significantly improves action correctness and trajectory-level task success across small and large models beyond SFT's structural competence.The reported robustness gains from MADS-Bench exceed those achievable with cooperative datasets alone.Source: §6 · p. 8
- Future work could expand tool ecosystems and adversarial behaviors, incorporate real or sandboxed APIs, and extend simulation to multimodal or speech-to-speech systems.These directions target broader scenario coverage, richer reward signals, more faithful evaluation, and robustness in next-generation conversational assistants.Source: §6 · p. 8
- MADS provides a scalable approach to generating structured training data that exposes agents to realistic failure modes.The paper presents this as supporting development of more robust and reliable task-oriented agents.Source: §6 · p. 8