Source-linked AI summary
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors
Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong
TL;DR
Smaller tool-using models struggle to recover from execution errors in multi-turn interactions. Fission-GRPO turns those failures into on-policy corrective supervision, improving Qwen3-8B error recovery and accuracy on BFCL while also gaining across TAU-Bench and TAU2-Bench.
Problem
Smaller models often fail to interpret execution feedback and recover, instead repeating invalid tool calls in multi-turn environments.
Method
Fission-GRPO intercepts failed trajectories, augments them with diagnostic feedback from a fine-tuned Error Simulator, and resamples recovery rollouts on-policy.
Results
Fission-GRPO improves Qwen3-8B error recovery by 5.7% on BFCL v4 Multi-Turn and achieves consistent gains across TAU-Bench and TAU2-Bench.
Takeaways & Limitations
The framework enables smaller models to learn robust self-correction rather than collapsing into repetitive execution-error loops.
Takeaways & Limitations
Fission-GRPO adds computational cost through additional recovery rollouts, which may increase substantially for very large models or expensive real API calls.
Abstract
from arXiv · showhide
Large language models (LLMs) can call tools effectively, yet they remain brittle in multi-turn execution: after a tool-call error, smaller models often fall into repetitive invalid re-invocations instead of interpreting the feedback and recovering. This failure mode persists because current training paradigms do not explicitly teach models how to recover from execution errors. In particular, standard reinforcement learning (RL) collapses rich failure experience into sparse negative rewards, while pre-collected error-correction datasets become mismatched to the policy's evolving failure modes. To bridge this gap, we propose Fission-GRPO, a framework that converts execution errors into on-policy corrective supervision within the RL training loop. Our core mechanism fissions each failed trajectory into a new training instance by augmenting it with diagnostic feedback from a fine-tuned Error Simulator, then resampling multiple recovery rollouts on-policy. This enables the model to learn from the precise errors it makes during exploration, rather than from static, pre-collected error cases. On BFCL v4 Multi-Turn, Fission-GRPO improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% (from 42.75% to 46.75%), outperforming both RL baselines and specialized tool-use agents. The method further generalizes to TAU-Bench and TAU2-Bench, achieving leading results across most settings with gains up to +17.4%.
1 Introduction
Multi-turn tool-use reliability depends on recovering from execution errors, yet smaller models often repeat invalid retries. FISSION-GRPO addresses this gap by converting failures into on-policy corrective training instances.
- Motivation: Smaller tool-using models must recover from execution errors to operate reliably in dynamic multi-turn environments.This matters especially for low-latency and on-device deployment settings.
- Problem: On BFCL v4 Multi-Turn, Claude Sonnet 4 exceeds 50% recovery while Qwen3-8B averages around 20%.Recovery rate is defined as eventual success conditioned on at least one prior execution error.
- Problem: Static error-correction datasets become stale as the policy’s error distribution evolves, while standard GRPO reduces errors to sparse negative rewards.These approaches provide limited alignment with the agent’s current failure modes.
- Approach: FISSION-GRPO uses standard GRPO exploration, diagnostic feedback from a learned Error Simulator, and fission-based resampling of recovery attempts.Each failed rollout is augmented into a corrective context before generating G′ parallel recovery attempts.
- Evaluation: The framework is evaluated on BFCL v4 Multi-Turn, TAU-Bench, and TAU2-Bench, with improvements across benchmarks and model scales.The authors also report a fine-tuned simulator with 96% non-leakage and Cohen’s κ = 0.71 in human evaluation.
2 Related Work
Prior tool-use and RL methods improve capability or synthesize error cases, but offline supervision can drift from the policy’s evolving errors. FISSION-GRPO instead integrates error simulation and recovery learning into the training loop.
- RL for Tool Use: GRPO estimates baselines from group averages without a value network, making it suitable for tool-calling tasks with binary or scalar rewards.Its reliance on intra-group reward variance creates additional failure modes when sampled groups are uniformly incorrect.
- RL for Tool Use: Homogeneously incorrect GRPO groups can produce zero reward variance and null gradients, while indiscriminate negative feedback may suppress valid reasoning steps.These limitations motivate methods such as DAPO, NGRPO, and NTHR.
- RL for Tool Use: Existing mitigations reshape negative signals but leave the scarcity of positive guidance during exploration unaddressed.Fission instead constructs recovery trajectories from zero-reward errors to provide denser learning signals.
- Tool Utilization: Tool-utilization research has progressed from single-turn syntactic correctness toward reliability in multi-turn workflows and recovery from environment errors.Benchmarks such as BFCL and StableToolBench codify recovery as a robustness metric.
- Error-Correction Data: Diagnosis-and-repair methods and model-based synthesis expand error coverage, but predominantly offline construction creates temporal mismatch with on-policy errors.FISSION-GRPO integrates error simulation directly into training to keep supervision aligned with current policy limitations.
3 Method
FISSION-GRPO adds a three-stage corrective loop to GRPO: it identifies failed tool-use trajectories, augments them with diagnostic feedback, and resamples recovery attempts for targeted updates.
- Three-stage framework: FISSION-GRPO alternates standard GRPO exploration with conditional fission updates that target execution-error recovery.Stage 1 maintains base tool-use performance, while Stages 2 and 3 focus on targeted error correction.
- Error identification: The method evaluates format validity first, then flags semantically incorrect trajectories when functional correctness falls below δcorr.Format-invalid trajectories are immediately treated as errors; otherwise, correctness is compared with a tunable threshold.
- Feedback synthesis: Format errors receive deterministic schema feedback, whereas semantic errors receive concise diagnostic messages from a fine-tuned Error Simulator.The simulator is a Qwen3-32B model fine-tuned via SFT to emulate runtime environment responses.
- Corrective sample construction: Failed trajectories become corrective contexts by appending the failed call and diagnostic feedback to the original multi-turn input.Corrective instances may be deduplicated by the pair (x, τerr), then stored in a LIFO buffer so recent policy errors are consumed first.
- Multiplicative resampling: Each corrective context generates a fission group of G′ parallel recovery rollouts, converting one observed error into multiple training attempts.The freshest contexts are sampled first, densifying training signals around current failure modes.
- Corrective batch training: Explicit feedback typically increases outcome diversity within recovery groups, making normalized relative advantages more informative for corrective updates.The corrective objective retains the GRPO-style clipped surrogate form while shifting training toward current failure modes.
4 Experiments
Experiments evaluate FISSION-GRPO across multi-turn tool-use benchmarks, model scales, error-recovery components, simulator quality, trigger frequency, compute matching, and qualitative recovery behavior. Results show consistent accuracy and recovery gains, with improvements linked to dynamic feedback and active diagnosis.
- Experimental Setup: 630 high-quality training instances were retained after schema curation, trajectory synthesis, hierarchical filtering, and dual-model verification.The initial pool contained approximately 2,000 trajectories, and only unanimously verified samples were retained.
- Experimental Setup: FISSION-GRPO was evaluated on BFCL v4 Multi-Turn, TAU-Bench, and TAU2-Bench, which support interactive error feedback and retries.BFCL permits up to 20 retry attempts per error, while TAU-Bench and TAU2-Bench use simulated-user interactions with distinct APIs and error distributions.
- Main Results: 12.58 points: FISSION-GRPO raises Qwen3-1.7B BFCL accuracy from 7.80% to 20.38%, with gains also reaching 46.75% for Qwen3-8B.The Qwen3-8B model also obtains TAU-Bench scores of 51.3% in Retail and 40.0% in Airline.
- Error Recovery Analysis: 5.7%: Qwen3-8B Error Recovery Rate improves on average across categories, including +11.8% in Long Context and +5.5% in Base.One-Shot Success Rate is preserved and improves by an average of 1.75%.
- Impact of Feedback Quality: +3.62 points: Fission-Dynamic outperforms Fission-Static on Qwen3-4B, while Fission-Static already exceeds GRPO.The results associate simulated, precise feedback with additional gains over generic prompts, especially for Miss Param and Long Context subsets.
- Trigger Frequency and Compute Efficiency: FISSION-GRPO retains gains under matched compute and shows stable performance for small-to-moderate correction intervals, while sparse corrections degrade performance.Matched-compute gains concentrate on Miss Param and Miss Func; qualitative traces characterize the method as using verification before correction rather than repetitive retries.
5 Conclusion
FISSION-GRPO turns execution failures into on-policy corrective supervision so smaller tool-using models can learn recovery instead of repeating invalid calls. It improves Qwen3-8B recovery and overall accuracy on BFCL v4 Multi-Turn while also gaining consistently across additional benchmarks.
- FISSION-GRPO intercepts failed calls, adds simulated diagnostic feedback, and resamples recovery attempts as on-policy corrective supervision.The framework transforms each failure into a corrective training context for learning self-correction.
- 5.7% absolute improvement in Qwen3-8B error recovery and 4.0% improvement in overall BFCL v4 Multi-Turn accuracy.Overall accuracy rises from 42.75% to 46.75%.
- FISSION-GRPO shows consistent gains on TAU-Bench and TAU2-Bench in addition to BFCL v4 Multi-Turn.
Limitations
The evaluation is limited to tool-calling benchmarks with interactive error feedback and retry attempts, while the fission mechanism adds computational cost that may grow in expensive settings.
- The evaluation remains within tool-calling agents on BFCL v4 Multi-Turn, TAU-Bench, and TAU2-Bench.These benchmarks cover diverse tool APIs and error dynamics, but not broader agent settings.
- The evaluated benchmarks feature interactive error feedback mechanisms with retry attempts.
- Extending evaluation to interactive code debugging or web navigation with fallback remains future work.
- Resampling G′ rollouts for each intercepted error adds computational cost, potentially amplifying absolute costs for very large models or expensive real API calls.A configurable trigger interval N trades correction frequency against training efficiency, while further scheduling optimization remains open.
A Prompt Template for the Error Simulator
The error simulator is queried with a two-message chat template that specifies its role and constraints, then injects the original context and tool-call attempts.
- The template uses a system message specifying the simulator role and output constraints.
- A user message injects the original context, ground-truth tool calls, and the model’s failed attempt.
B Representative Training Curves
For Qwen3-8B, GRPO and FISSION-GRPO show similar smoothed training dynamics: rapid early improvement followed by stable behavior without reward collapse or divergence.
- Both methods improve rapidly during early training and remain stable thereafter in smoothed reward trajectories.Step-level rewards remain noisy, but the macro-level dynamics are similar.
- The curves provide no evidence of reward collapse or divergence for FISSION-GRPO.The reported dynamics suggest gains without optimization instability.
C Training Algorithm Details
FISSION-GRPO alternates standard exploration with fission-based updates, preserving general capability while learning specific recovery strategies.
- The framework alternates standard exploration with fission-based updates.Exploration maintains general capability and mines errors; fission-based updates learn recovery strategies.
- Standard exploration contributes both general capability and error discovery.
- Fission-based updates target specific recovery strategies.
D Extended Case Study Analysis
The case study contrasts state-tracking failures in Base and GRPO models with FISSION-GRPO’s active diagnosis and successful recovery. The training procedure uses exploration, diagnostic feedback, corrective-context construction, deduplication, and corrective updates.
- Extended Case Study Analysis: Training alternates exploration and fission-based updates, while reward trajectories remain stable despite noisy step-level rewards.The procedure is presented as Algorithm 1, and the representative curves show rapid early improvement without collapse or divergence.
- Extended Case Study Analysis: The task requires tracking partial success because mkdir fails while cd and mv succeed, changing the later file-search requirement.Since the file was moved into archive, a direct grep fails and the agent must locate it first.
- Extended Case Study Analysis: The Base model enters invalid-operation loops after failing to update its state following a successful cd.It redundantly retries cd workspace after the mkdir error.
- Extended Case Study Analysis: GRPO tracks that the file moved but responds to failed searches with an ungrounded path guess and hallucination.It first tries grep("log.txt"), then grep("archive/log.txt"), before inventing a non-existent path.
- Extended Case Study Analysis: FISSION-GRPO resolves a missing-file error by using find to verify the file location before updating state and running grep.It confirms the path, changes into the archive folder, and then executes grep successfully.
- Extended Case Study Analysis: The training flow identifies failed trajectories, generates diagnostic feedback, constructs corrective contexts, deduplicates them, and applies corrective updates.The algorithm defines an error set, invokes the Error Simulator for diagnostics, hashes samples for deduplication, and accumulates corrective gradients.