Source-linked AI summary
CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification
Jinpeng Chen, Cheng Gong, Hanbo Li, Ziru Liu, Zichen Tian, Xinyu Fu, Shi Wu, Chenyang Zhang, Wu Zhang, Suiyun Zhang, Dandan Tu, Rui Liu
TL;DR
Multi-turn tool-use agents must translate ambiguous user needs into deterministic actions, but existing training-data synthesis struggles to ensure both complexity and correctness. CoVe uses explicit constraints to guide fuzzified, interactive trajectory generation and deterministic verification, and CoVe-4B achieves strong τ 2-bench performance while supporting SFT and RL. The framework is evaluated only on Airline and Retail, and its sequential SFT+RL pipeline currently underperforms pure SFT because of the open-weight simulator’s capability gap.
Problem
Training multi-turn interactive tool-use agents is difficult because complex, ambiguous user needs must be clarified and converted into precise executable actions, while existing synthesis struggles to ensure trajectory complexity and correctness.
Method
CoVe samples explicit task constraints, fuzzifies them to guide a User Simulator LLM in generating multi-turn trajectories, and uses the original constraints for deterministic verification and rewards.
Results
CoVe-4B achieves 51.2% overall success on τ 2-bench Airline and Retail, outperforming similar-scale baselines and remaining competitive with models up to 17× larger.
Takeaways & Limitations
CoVe provides a data-synthesis pathway that supports both SFT and RL while producing complex interactive trajectories with verified quality.
Takeaways & Limitations
The study evaluates only Airline and Retail, and its sequential SFT+RL pipeline underperforms pure SFT because of the open-weight User Simulator’s capability gap.
Abstract
from arXiv · showhide
Developing multi-turn interactive tool-use agents is challenging because real-world user needs are often complex and ambiguous, yet agents must execute deterministic actions to satisfy them. To address this gap, we introduce \textbf{CoVe} (\textbf{Co}nstraint-\textbf{Ve}rification), a post-training data synthesis framework designed for training interactive tool-use agents while ensuring both data complexity and correctness. CoVe begins by defining explicit task constraints, which serve a dual role: they guide the generation of complex trajectories and act as deterministic verifiers for assessing trajectory quality. This enables the creation of high-quality training trajectories for supervised fine-tuning (SFT) and the derivation of accurate reward signals for reinforcement learning (RL). Our evaluation on the challenging $τ^2$-bench benchmark demonstrates the effectiveness of the framework. Notably, our compact \textbf{CoVe-4B} model achieves success rates of 43.0\% and 59.4\% in the Airline and Retail domains, respectively; its overall performance significantly outperforms strong baselines of similar scale and remains competitive with models up to $17\times$ its size. These results indicate that CoVe provides an effective and efficient pathway for synthesizing training data for state-of-the-art interactive tool-use agents. To support future research, we open-source our code, trained model, and the full set of 12K high-quality trajectories used for training.
1 Introduction
CoVe addresses the difficulty of training agents that must resolve ambiguous user needs through multi-turn dialogue while executing precise tool actions. It synthesizes complex trajectories from explicit constraints and verifies them deterministically, supporting strong performance and both SFT and RL.
- Multi-turn tool-use agents must clarify complex, implicit, and ambiguous user needs before translating them into deterministic tool commands.
- Existing synthesis methods struggle to guarantee query solvability, verification correctness, and sufficiently complex trajectories because they rely on uncontrollable LLM generation and evaluation.
- CoVe samples explicit deterministic task constraints, fuzzifies them into ambiguous expressions, and uses them to guide a User Simulator LLM through multi-turn interactions.
- The original constraints verify tool execution exactly, enabling filtered SFT trajectories and precise RL reward signals from exploration.
- 51.2% overall success rate was achieved by CoVe-4B, outperforming similar-scale baselines and remaining competitive with open-source models up to 70B.
- CoVe generates multi-turn trajectories with deterministic verification, while its synthesized data supports both SFT and RL.
- The authors open-source the code, trained model, and 12K high-quality interactive tool-use trajectories.
2 Related Work
Prior work established tool-using agents and automated trajectory synthesis, but the literature spans increasingly complex applications and multiple approaches to scalable multi-turn data generation.
- Foundational systems such as TALM, Toolformer, and ReAct demonstrated that LLMs can autonomously interact with external APIs.
- Tool-use agents have since been applied to deep research, code generation, and GUI navigation.
- Automated synthesis frameworks address the costly, difficult-to-scale acquisition of high-quality multi-turn interactive tool-use data.
- APIGen-MT uses a blueprint-to-dialogue pipeline with quality verification, while Simia simulates environment and user feedback for SFT and RL training.
3 Problem Formulation
The paper models multi-turn tool-use interaction as a simplified POMDP in which agents act under hidden user intent and changing environment state. CoVe’s framework connects explicit constraints, ambiguous user simulation, and deterministic trajectory verification.
- Problem Formulation: The interaction is formalized as a simplified POMDP with state, action, observation, and transition components.
- Problem Formulation: The latent state contains the user’s hidden intent and the external environment state.
- Problem Formulation: At each step, the agent either executes a structured tool call or generates a natural-language response.
- Problem Formulation: Observations consist of either user instructions or execution results returned by tool calls.
- Problem Formulation: The transition function governs how environment state and user intent evolve according to agent actions.
- CoVe Framework: CoVe samples and fuzzifies explicit constraints to generate ambiguous queries, then reuses the constraints as a checklist for verifying tool invocations.
- Problem Formulation: A trajectory unfolds over multiple turns as the agent selects actions from interaction history to fulfill latent intent under environmental constraints.
4 CoVe: Constraint-Guided Trajectory Generation and Deterministic Verification
CoVe constructs solvable but ambiguous tasks from deterministic constraints, generates multi-turn interactions, and verifies trajectories against those constraints with a rule-based evaluator. The resulting mechanism supports both supervised fine-tuning and reinforcement learning.
- Constraint Sampling: Deterministic constraints are sampled from sandbox database states and available tools, guaranteeing that generated tasks are executable.The constraints define comprehensive task requirements and use precise identifiers for uniquely identifying environment elements.
- Constraint Fuzzification: Constraint fuzzification replaces explicit identifiers with ambiguous but uniquely identifying descriptions that resemble natural user requests.Retail examples include email or name-plus-zip-code user references, item combinations for orders, and product names with attributes for items.
- Trajectory Generation: Fuzzified instructions are given to a User Simulator LLM, which progressively reveals requirements while the Agent clarifies ambiguities and invokes tools across multiple turns.The simulator emits a termination marker when it considers the task complete, after which the interaction history is recorded as trajectory τ.
- Trajectory Verification: A rule-based verifier checks tool invocation records against every constraint and accepts any valid execution path that achieves the required outcome.The verifier evaluates outcome satisfaction rather than enforcing a predefined action sequence, while also identifying redundant operations.
- Training Applications: CoVe supports SFT by retaining only trajectories with maximum score 1 and supports RL by directly feeding verifier scores back as rewards.SFT uses a teacher model to generate candidate trajectories, whereas RL uses the framework as both an interactive environment and reward provider.
5 Experiment
Experiments on Airline and Retail τ^2-bench evaluate CoVe-4B against models across scales and analyze data quality, training paradigms, and simulator diversity. CoVe-4B performs strongly, while pure SFT outperforms sequential SFT+RL under the reported setup.
- Experimental Setup: Experiments evaluate CoVe-4B on Airline and Retail τ^2-bench domains using pass1 through pass4 metrics against proprietary and open-source baselines.The protocol disables the think tool and compares models across parameter scales.
- Main Results: 51.2% average pass1 places CoVe-4B ahead of Simia-Tau-RL-8B at 47.7% and xLAM-2-8b-fc-r at 41.2% within the ≤8B group.CoVe-4B achieves the highest pass1 score in both evaluated domains and improves 18.6 percentage points over its base model, from 32.6% to 51.2%.
- Main Results: 51.2% average pass1 lets CoVe-4B outperform xLAM-2-32b-fc-r at 49.5% and approach xLAM-2-70b-fc-r at 51.5%.The reported gaps to Qwen3-235B-A22B-Instruct-2507 and GPT-4o are 4.9% and 4.6%, respectively.
- Data Quality and Scaling: 44.7% average pass1 from CoVe-5K exceeds APIGen-MT-5K at 41.7% and Simia-5K at 39.7%, while slightly surpassing Simia-90K at 44.3%.Expanding to CoVe-12K raises average pass1 to 51.2% across both domains.
- Training Paradigms: 51.2% for pure SFT exceeds pure RL at 40.7% and sequential SFT+RL at 46.9%, despite all using the CoVe framework.The reported degradation after adding RL is attributed to the narrower, weaker online user simulator used under resource constraints.
- Synthesis Dynamics: 74.0% average trajectory yield for Gemini-3-Pro exceeds weaker simulators because its instruction-following better times dialogue termination signals.The final SFT dataset nevertheless uses a diverse simulator ensemble, and Retail yields consistently exceed Airline yields across simulators.
6 Limitations and Future Work
The study’s current evidence is limited to Airline and Retail τ^2-bench domains, and sequential SFT+RL underperforms pure SFT because of the online simulator’s capability gap. Future work targets stronger or specialized simulators and broader benchmark coverage.
- Simulator Limitations: Sequential SFT+RL currently underperforms pure SFT because online interaction uses an open-weight simulator with a capability gap.The authors propose adopting more capable models, training a dedicated simulator, or refining the simulator through prompt engineering.
- Evaluation Scope: The current study evaluates CoVe strictly on the Airline and Retail domains of τ^2-bench.Future research will test Telecom and other multi-turn interactive tool-use benchmarks such as BFCL.
7 Conclusion
CoVe anchors interactive tool-use data synthesis in explicit constraints, using them to connect ambiguous requests with deterministic tool execution. It supports complex, realistic dialogues and exact verification while achieving competitive performance with much larger models.
- CoVe uses explicit task constraints to bridge ambiguous real-world requests and deterministic tool-execution requirements.
- Constraint fuzzification elicits complex, realistic multi-turn dialogues, while the original constraints provide a deterministic checklist for trajectory verification.
- CoVe-4B achieves performance competitive with open-source models up to 17× its size on τ 2-bench.
- CoVe provides a scalable foundation for advancing reliable and capable interactive AI agents through open-source models and a curated 12K trajectory dataset.