PaperlayerView exact revision on arXiv

Source-linked AI summary

D-CORE: Incentivizing Task Decomposition in Large Reasoning Models for Complex Tool Use

Bowen Xu, Shaoyu Wu, Hao Jiang, Kai Liu, Xin Chen, Lulu Hu, Bin Yang

arXiv:2602.02160v1cs.CL

TL;DR

Current LRMs struggle to translate extensive reasoning into effective complex tool use because they lack task decomposition and exhibit Lazy Reasoning. D-CORE first bootstraps decomposition through self-distillation, then uses diversity-aware GRPO to restore reflection. It achieves state-of-the-art results on BFCLv3 and τ-Bench, including 77.7% accuracy for D-CORE-8B and 79.3% for D-CORE-14B.

  • Problem

    Current LRMs lack task decomposition in complex tool-use scenarios, producing Lazy Reasoning and limited performance gains despite greater reasoning-token consumption.

  • Method

    D-CORE combines self-distillation for task decomposition with diversity-aware GRPO using entropy-based advantage functions to maintain decomposition and reflection.

  • Results

    D-CORE achieves state-of-the-art results on BFCLv3 and τ-Bench, reaching 77.7% overall accuracy for D-CORE-8B and 79.3% for D-CORE-14B on BFCLv3.

  • Takeaways & Limitations

    D-CORE converts inefficient reasoning cycles into task decomposition and effective sequential tool-use processes across complex scenarios.

  • Takeaways & Limitations

    Self-distillation suppresses reflective reasoning and exploration, producing near-zero reward variance that hinders GRPO optimization.

Abstract

from arXiv · show

Effective tool use and reasoning are essential capabilities for large reasoning models~(LRMs) to address complex real-world problems. Through empirical analysis, we identify that current LRMs lack the capability of sub-task decomposition in complex tool use scenarios, leading to Lazy Reasoning. To address this, we propose a two-stage training framework D-CORE~(\underline{\textbf{D}}ecomposing tasks and \underline{\textbf{Co}}mposing \underline{\textbf{Re}}asoning processes) that first incentivize the LRMs' task decomposition reasoning capability via self-distillation, followed by diversity-aware reinforcement learning~(RL) to restore LRMs' reflective reasoning capability. D-CORE achieves robust tool-use improvements across diverse benchmarks and model scales. Experiments on BFCLv3 demonstrate superiority of our method: D-CORE-8B reaches 77.7\% accuracy, surpassing the best-performing 8B model by 5.7\%. Meanwhile, D-CORE-14B establishes a new state-of-the-art at 79.3\%, outperforming 70B models despite being 5$\times$ smaller. The source code is available at https://github.com/alibaba/EfficientAI.

1. Introduction

Current LRMs can spend substantially more reasoning tokens yet gain little on complex tool use, particularly because they lack task decomposition and exhibit Lazy Reasoning. D-CORE addresses this with self-distillation for decomposition and diversity-aware GRPO to preserve reflection and diversity.

  • D-CORE trained LRMs decompose tasks into executable subtasks instead of repeating inefficient reflection cycles.
  • LRMs consume substantially more reasoning tokens while yielding marginal gains over LLMs in complex tool-use scenarios.
  • Qwen3 LRMs exhibit Lazy Reasoning in complex multi-turn contexts, generating extensive but meaningless reasoning that impedes RL optimization.
  • D-CORE uses self-distillation to bootstrap task decomposition by organizing subtask executions into reasoning trajectories.
  • DA-GRPO adds entropy regularization to the advantage function, balancing structural decomposition with reasoning diversity and restoring reflection.

2. Tool Use Reasoning: Patterns and Limitations

Complex tool use requires reasoning over subtask dependencies, irrelevance, and conversation history. Empirical analyses find that multi-turn failures coincide with insufficient decomposition and excessive reflection, while explicit decomposition improves performance.

  • 2.1. Preliminary: Tool-use tasks include sequential subtasks, parallel subtasks, and queries requiring no tool use.
  • 2.1. Preliminary: Multi-turn tool use additionally requires considering both the current query intent and long-term intent embedded in conversation history.
  • 2.1. Preliminary: A reasoning process is an ordered sequence of intermediate thoughts through which an LRM arrives at its final answer.
  • 2.2. Reasoning Process Enhances Tool Use Awareness: Standard LRMs outperform no-think Qwen3 and xLAM2 on BFCLv3 parallel and irrelevance tasks but lag behind specialized models on complex multi-turn scenarios.
  • 2.3. Lazy Reasoning in Tool Use: Multi-turn reasoning shows minimal task decomposition and excessive reflection, with Lazy Reasoning prevalence strongly correlating with interaction failures.
  • 2.3. Lazy Reasoning in Tool Use: Performance degrades as subtasks increase, while imposing explicit step-by-step decomposition produces significant gains in multi-turn LRM scenarios.

3. D-CORE

D-CORE uses self-distillation to acquire task decomposition and subtask execution, then applies diversity-aware GRPO to preserve reflective reasoning during tool-use training.

  • Framework overview: D-CORE trains LRMs in two stages: self-distillation for task decomposition and subtask execution, followed by diversity-aware GRPO for reflective reasoning.The framework targets complex tool-use scenarios while maintaining reasoning diversity.
  • Task decomposition: Self-distillation prompts an LRM to decompose queries into subtasks using contextual information, reference trajectories, and few-shot examples.Context includes system policy, tools, and conversation history.
  • Reasoning generation: The model generates reasoning processes and tool calls for each subtask, processing sequential subtasks iteratively and parallel subtasks simultaneously.Tool-irrelevant queries receive explanations for why decomposition does not apply.
  • Composition and distillation: Composed subtasks, reasoning, tool calls, and responses form complete trajectories that are used for self-distillation through supervised fine-tuning.The training objective maximizes the probability of each trajectory token conditioned on context and prior tokens.
  • Diversity-aware GRPO: Self-distillation can reduce reward variance and reflective exploration, making GRPO advantages negligible and motivating an entropy-aware update.DA-GRPO reshapes the advantage with an entropy-based term to mitigate gradient collapse and incentivize complex reasoning.
  • Diversity-aware GRPO: DA-GRPO preferentially increases the probability of high-entropy tokens when the standard advantage is zero, thereby reducing their entropy and making them more likely to be generated.The entropy-based term is detached during backpropagation and acts as a fixed offset affecting update magnitude.

4. Experiments

Experiments show that D-CORE improves complex tool-use accuracy across benchmarks, generalizes to unseen tasks, and reduces Lazy Reasoning. Ablations indicate that self-distillation supplies decomposition while DA-GRPO restores reflection and balances exploration with exploitation.

  • Main Results: D-CORE improves τ-bench accuracy by 18.6% for Qwen3-8B and 17.7% for Qwen3-14B.The Qwen3-14B model reaches 46.0% accuracy on τ-airline.
  • Main Results: 77.7% overall accuracy makes D-CORE-8B state-of-the-art among 8B models on BFCLv3, while D-CORE-14B reaches 79.3%.D-CORE-14B uses 5× fewer parameters than the previous 70B state-of-the-art model.
  • Training Dynamics: With α=0.1, DA-GRPO achieves the highest reward while increasing reflection tokens; excessive α causes exploration collapse.At α=1.0, reflection tokens increase progressively but rewards remain lower than GRPO.
  • Generalizability: D-CORE remains competitive on ACEBench, τ 2-Bench, and BFCLv4-agentic out-of-distribution tasks, surpassing the Qwen3 baseline.The evaluation includes dual-control environments, complex system and user prompts, Web-search, and Memory scenarios.
  • Ablation Study: Self-distillation improves with increasing sample size and yields gains of 9.9% and 8.6% on the respective benchmarks, whereas GRPO alone is marginal or negative.The comparison uses identical settings.
  • Ablation Study: Pseudo-labels generated by Qwen3-Max maintain high task-decomposition effectiveness while offering a scalable alternative to inaccessible reference trajectories.This approach bridges ideal supervision and practical applicability.
  • Lazy Reasoning: D-CORE reduces incorrect Multi-Turn BFCLv3 errors attributed to Lazy Reasoning from 45% to 6% for the 8B model.The reduction brings Lazy Reasoning proportions toward levels comparable with Parallel and Irrelevance categories.
  • Ablation Study: DA-GRPO restores some reflection after self-distillation increases decomposition while reducing reflection, but excessive entropy can confuse rewards.Tool accuracy first rises and then declines as α increases; α=0.1 provides the selected balance between exploration and exploitation.

5. Related Work

Prior work improves tool use through curated datasets, long-chain-of-thought training, and reinforcement learning. The paper positions its contribution as a systematic study of reasoning processes for complex tool-use tasks.

  • Tool Use: Tool-use research has built datasets resembling real-world multi-turn scenarios to improve LLM tool-use competencies.Examples include APIGen-MT and Magnet.
  • Tool Use: ToolRL and Nemotron-N1 combine long chain-of-thought with reinforcement learning and evaluate tool use on BFCLv3 single-turn tasks.The cited work addresses generalizability of models trained using supervised fine-tuning.
  • Large Reasoning Models: Large Reasoning Model research includes GRPO with outcome rewards and analyses of how reasoning processes affect outcomes.DeepSeek-R1 is cited as demonstrating enhanced reasoning capabilities with this optimization and reward combination.

6. Conclusion

The paper attributes Lazy Reasoning in complex tool-use scenarios to insufficient task decomposition and proposes D-CORE to address it. D-CORE achieves state-of-the-art results on BFCLv3 and τ-Bench, while future work targets multimodal models and advanced reinforcement learning.

  • Conclusion: D-CORE addresses Lazy Reasoning through self-distillation and DA-GRPO, achieving state-of-the-art results on BFCLv3 and τ-Bench.The paper attributes Lazy Reasoning in complex scenarios to insufficient task decomposition.
  • Conclusion: Future work will extend D-CORE to multimodal models and explore advanced reinforcement learning algorithms for efficient reasoning.

Impact Statement

The paper explores Large Reasoning Models’ tool-use techniques as a contribution to machine learning. It identifies potential benefits for model efficiency, scalability, and broader applicability, without emphasizing specific societal implications.

  • Impact Statement: The paper explores Large Reasoning Models’ tool-use techniques as a contribution to advancing machine learning.
  • Impact Statement: The techniques are described as having potential to enhance model efficiency and scalability and support broader applicability across domains.
  • Impact Statement: The authors identify no societal implications requiring specific emphasis at this time.

A.1. Details of Lazy Reasoning

Qwen3-8B exhibits Lazy Reasoning through insufficient subtask planning and missing reasoning for individual subtask executions. Its reasoning instead follows an inefficient trial-negation-retry pattern in both correct and incorrect responses.

  • Lazy Reasoning lacks detailed planning for the subtasks needed to complete the main task.
  • The models omit specific reasoning for each subtask execution.
  • The resulting reasoning follows an inefficient, random trial-negation-retry pattern.
  • This pattern appears in both correct and incorrect responses.

A.1.1. CASE STUDY ON BFCLV3-MULTI-TURN 196 Qwen3-8B

The BFCLv3 multi-turn cases contrast inefficient, repetitive reasoning with more explicit decomposition into sequential tool-use subtasks. The examples involve travel-budget conversion and flight-reservation management.

  • The budget case requires converting 50,000 RMB to USD before setting the budget limit.
  • The decomposed budget trajectory calls compute exchange rate first and then set budget limit using the converted value.
  • When the reservation ID is unavailable, the reasoning considers checking multiple reservations and obtaining missing flight-date information.
  • The flight-change case requires retrieving user details before identifying the relevant reservation and modifying the flight.
  • The overlapping-flight case requires examining each reservation because the user reports multiple flights on the same days.

A.2. Reward Function in DA-GRPO

DA-GRPO uses the ToolRL-aligned reward components for formatting and structural correctness of tool calls. These components compare predicted and ground-truth tool-call elements using exact matching.

  • The DA-GRPO reward includes formatting and structural components aligned with ToolRL.
  • Rformat rewards exact matching of the required <think>...</think> format.
  • Rstruct evaluates whether predicted and ground-truth tool-call structures match.

A.2.1. ANALYSIS OF LAZY REASONING.

The analysis distinguishes effective reasoning on MATH from Lazy Reasoning in BFCLv3 multi-turn tasks. It motivates filtering and training interventions based on token length and reflection frequency.

  • Figure 8 compares reasoning-length and reflection-count distributions for correct and incorrect Qwen3-8B MATH responses.
  • Figure 9 analyzes the corresponding reasoning-length and reflection-count distributions for BFCLv3 multi-turn responses.
  • On MATH, greater reasoning length and reflection frequency define an effective reasoning region dominated by correct responses.
  • BFCLv3 multi-turn correct and incorrect responses have highly consistent distributions, with no effective reasoning region.
  • Lazy Reasoning is filtered using a common threshold of more than 300 tokens and over 3 reflections.
  • The filtering analysis covers Qwen3-8B and Qwen3-32B on BFCLv3 using reflection- and token-number thresholds.

A.2.3. WHY LRMS HAVE LAZY REASONING?

Lazy Reasoning arises in complex multi-turn tool use when LRMs generate extensive ineffective reflection without adequately decomposing tasks. Decomposition-based prompting, self-distillation, and diversity-aware training are presented as ways to make reasoning more structured and effective.

  • Lazy Reasoning: LRMs exhibit Lazy Reasoning when low-accuracy cases contain extensive ineffective reflection, making the phenomenon detectable through evaluation.The paper characterizes Lazy Reasoning as a posterior concept tied to incorrect outcomes and unproductive reflection.
  • Task decomposition: Decomposed prompting mitigates Lazy Reasoning by guiding models to split queries into subtasks and execute their results sequentially.The workflow manually replaces original queries with decomposed subtasks and collects each subtask’s execution result.
  • Diversity-aware training: DA-GRPO selectively rewards sampled high-entropy tokens when the advantage is zero, encouraging exploration while reducing those tokens’ entropy and consolidating behaviors.The mechanism is described as self-limiting and distinct from standard entropy regularization, which increases overall distribution entropy.
Loading 2602.02160v1…