PaperlayerView exact revision on arXiv

Source-linked AI summary

TodoEvolve: Learning to Architect Agent Planning Systems

Jiaxi Liu, Yanzuo Jiang, Guibin Zhang, Zihan Zhang, Heng Chang, Zhenfei Yin, Qibing Ren, Junchi Yan

arXiv:2602.07839v1cs.CLcs.AIcs.LG

TL;DR

Fixed, manually engineered planners do not adapt readily to structurally diverse tasks. TodoEvolve synthesizes task-specific planning systems through PlanFactory and IGPO, and evaluations report that it outperforms static baselines while balancing success rates and computational efficiency.

  • Problem

    Existing planning systems fix planning structures manually, despite task domains and instances differing in the planning priors they require.

    Source: §1 · p. 2, §3.1 · p. 3

  • Method

    TodoEvolve trains Todo-14B with IGPO to synthesize task-adaptive planners that configure planning topology and strategies within PlanFactory’s unified design space.

    Source: §1 · p. 2

  • Results

    Across evaluations, TodoEvolve outperforms static baselines, with gains of up to 16.37% on GAIA over frameworks such as Smolagents.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    TodoEvolve’s reported Pareto balance between success rates and computational efficiency supports autonomous synthesis of task-specific planning architectures.

    Source: §6 · p. 10

Abstract

from arXiv · show

Planning has become a central capability for contemporary agent systems in navigating complex, long-horizon tasks, yet existing approaches predominantly rely on fixed, hand-crafted planning structures that lack the flexibility to adapt to the structural diversity of open-ended problems. To address this limitation, we introduce TodoEvolve, a meta-planning paradigm that autonomously synthesizes and dynamically revises task-specific planning architectures. Specifically, we first construct PlanFactory, a modular design space that standardizes diverse planning paradigms within a unified codebase encompassing topology, initialization, adaptation, and navigation, thereby providing a common interface for heterogeneous planning patterns. Leveraging PlanFactory, we collect high-quality planning trajectories and train Todo-14B via \textit{Impedance-Guided Preference Optimization} (IGPO), a multi-objective reinforcement learning objective that encourages the generation of planning systems that are performant, stable, and token-efficient across arbitrary tasks and agent backbones. Empirical evaluations on five agentic benchmarks demonstrate that TodoEvolve consistently surpasses carefully engineered planning modules while maintaining economical API costs and runtime overhead.

1 Introduction

The introduction argues that no fixed planning structure can fit diverse agent tasks, motivating TodoEvolve, which synthesizes and revises task-adaptive planners. PlanFactory unifies planning designs, while IGPO trains Todo-14B to generate planning systems that balance performance, stability, and token efficiency.

Introduction

  • Planning structures vary across agent settings, representations, and domains, while differing task priors and revision needs make a universal planner unrealistic.The introduction contrasts simple linear planning with DAG structures for more complex settings and minimal revision with continuous restructuring in high-conflict environments.Source: §1 · pp. 1–2
  • TodoEvolve trains Todo-14B with IGPO to synthesize task-adaptive planners that dynamically revise planning states and tailor topology, revision cadence, and navigation strategy.IGPO jointly promotes performance, stability, and token efficiency; Todo-14B is designed to work across single- and multi-agent frameworks, diverse LLM backbones, and heterogeneous domains.Source: §1 · p. 2
  • PlanFactory organizes planning systems across topology, initialization, adaptation, and navigation, and re-implements ten representative architectures as a unified codebase and trajectory-generation engine.The framework is intended to support future research and generate planning trajectories for training TodoEvolve.Source: §1 · p. 2
  • On four challenging agentic benchmarks, TodoEvolve improves Smolagents by up to 16.37% on GAIA and boosts GPT-5-Mini to 75% on xBench-DS.The introduction reports substantial performance gains and generalization across diverse LLM backbones.Source: §1 · p. 2

2 Related Works

Related work advances agent planning from structured reasoning and execution loops to domain-informed architectures and reinforcement learning, but existing approaches largely optimize within fixed planning structures. This motivates training a meta-planner to synthesize task-specific structures.

Agent Planning Systems

  • Planning methods from CoT (Wei et al., 2022), ToT (Yao et al., 2023a), and GoT (Besta et al., 2023) to ReAct (Yao et al., 2023b) and Reflexion add decomposition and self-correction but retain rigid topologies.Recent frameworks add DAG parallelism, hierarchical coordination, or structured verification, yet still use pre-designed architectures.Source: §2 · p. 2
  • Table 1 organizes PlanFactory’s agentic planning paradigms by agent compatibility, planning scope, and execution structure.Its columns distinguish single- versus multi-agent compatibility, step-wise versus task-wise scope, and linear versus graph structure.Source: Table 1 · p. 3

RL for Agent Planning

3 PlanFactory: Unified Planning Codebase

Section 3 separates agent execution from high-level planning control and introduces PlanFactory to standardize planning architectures. It motivates automatically tailoring planning configurations because existing systems fix them through manual engineering.

3.1 Preliminary

  • The framework distinguishes an Agent System that executes environment interactions from a Planning System that governs high-level control logic.The Agent System is formalized as an execution substrate with agents, state and action spaces, dynamics, context aggregation, and trajectory reward.Source: §3.1 · p. 3
  • Existing planning paradigms are static configurations manually engineered into policy, limiting adaptability and motivating task-tailored planning configurations.The paper frames these configurations as P and the automatically synthesized configuration as P∗.Source: §3.1 · p. 3

3.2 PlanFactory Codebase

  • PlanFactory decouples high-level planning logic from low-level execution through a shared lifecycle interface with topology, initialization, adaptation, and navigation modules.Its polymorphic design lets heterogeneous strategies be swapped within one runtime and benchmarked concurrently without refactoring the agent loop.Source: §3.2 · pp. 3–4
  • PlanFactory evaluates dynamic information-seeking tasks with an LLM-as-a-Judge that measures task success rates and the logical coherence of generated plans.The evaluation suite is designed for reliable assessment in open domains.Source: §3.2 · p. 4

4 TodoEvolve: Training Meta-Planners

TodoEvolve trains Todo-14B to synthesize task-specific planning systems through a pipeline that builds verified planning data and uses SFT followed by Impedance-Guided Preference Optimization (IGPO) to improve execution efficiency.

4.1 Data Construction

  • PlanFactory standardizes seven planning paradigms as tools in a shared plan space, then samples structural references to guide query-specific plan generation.For each query, the context includes three randomly sampled static planning examples; generated plans may integrate or modify those patterns rather than merely copy templates.Source: §4.1 · pp. 4–5
  • Only plans that execute to the ground-truth answer are retained, supplying SFT demonstrations and preference data that prioritizes correctness before efficiency.When both candidates are correct, cognitive impedance breaks the tie; pairs where both fail are discarded.Source: §4.1 · p. 5

4.2.1 Stage 1: Structural Competence via SFT

  • SFT warm-starts Todo-14B on verified plans, teaching code generation and architectural reasoning while grounding outputs in context and API constraints.Training minimizes the negative log-likelihood of expert target plans, establishing syntactically valid, executable architectures before preference alignment.Source: §4.2.1 · p. 6

4.2.2 Stage 2: Impedance-Guided Preference Alignment

  • Because SFT does not ensure efficiency, IGPO samples and evaluates candidate plans, then selects successful preference pairs only when their impedance gap exceeds a threshold.The evaluation metric aggregates token use, latency, and runtime errors; the selected pairs provide objective execution-based efficiency signals.Source: §4.2.2 · p. 6
  • IGPO uses negative expected impedance as an implicit reward and a KL constraint to guide policy probabilities, avoiding an explicit reward model.Its final loss maximizes the margin between efficient and inefficient architectures while maintaining functional correctness.Source: §4.2.2 · pp. 6–7
  • IGPO aligns the Meta-Planner with execution by preferring functionally correct architectures with lower cognitive impedance, rather than optimizing correctness alone.Preference pairs require a successful plan and a significant impedance gap; impedance aggregates token consumption, latency, and runtime errors, while penalizing planning costs that outweigh execution costs.Source: §4.1 · p. 5, §4.2.2 · pp. 6–7

5 Experiments

The experiments evaluate TodoEvolve across agent frameworks, execution backbones, and task types. They report benchmark gains, analyze planning specialization and efficiency, test training components, and illustrate state-adaptive planning in a case study.

5.1 Experiment Setup

  • Training uses teacher-generated planning traces verified for actionable outcomes, and evaluation covers GAIA, XBench-DS, and distinct TaskCraft and WebWalkerQA test splits.The training corpus draws from TaskCraft, MoNaCo, WebWalkerQA, and DeepSearchQA; Qwen3-14B is the backbone, agent execution uses DeepSeek V3.2, Kimi-K2, or GPT-5-mini, and Gemini-3-Flash judges trajectories. Test samples from TaskCraft and WebWalkerQA do not overlap with training splits.Source: §5.1 · p. 7

5.2 Main Results

  • With GPT-5-Mini, TodoEvolve scores 72.12% on GAIA Avg, 16.37% above vanilla Smolagents, and improves across GPT-5-Mini, DeepSeek V3.2, and Kimi K2.With DeepSeek V3.2, it reaches 70.91% on GAIA Avg, over 10 percentage points above Flash-Searcher using the same model, and 53.85% on GAIA Level 3, surpassing Agent KB with GPT-4.1 and matching Agent KB with pass@3 voting. It also outperforms specialized same-backbone frameworks on GAIA Avg and shows versatility on WebWalkerQA and xBench-DS.Source: §5.2 · pp. 7–8

5.3 Structural Specialization

  • Fixed planning priors show task-specific inductive biases: centralized systems favor reasoning depth, while DAGs favor extraction speed, limiting any single static topology across tasks.The TaskCraft multi-category evaluation motivates dynamically synthesizing architectures that combine complementary planning paradigms for cross-domain robustness.Source: §5.3 · p. 8, Figure 2 · p. 9

5.4 Inference Efficiency

  • On WebWalker-QA and DeepSearch-QA, TodoEvolve exceeds the best static baseline by 10.0% and 14.0%, respectively, while maintaining comparable costs and latency.The Kimi-K2 evaluation reports dominant accuracy without a proportional increase in resource consumption; the authors attribute this Pareto advantage to reduced redundant loops and fewer premature failures.Source: §5.4 · pp. 8–9, Table 4 · p. 8

5.5 Ablation Study

  • On GAIA Level 2, Zero-Shot reaches 55.8% accuracy with lower API costs than SFT-Only, while full TodoEvolve reaches 72.1% with more steps and cost.The Base Model fails to synthesize executable plans because it lacks syntactic grounding, which SFT-Only establishes; the full framework combines both training stages with reference-augmented inference.Source: §5.5 · p. 9, Figure 3 · p. 9

5.6 Case Study

  • TodoEvolve adapts planning to task state, using Fork-Join breadth to break information deadlocks, linear constraints to prune noisy searches, and fallback sources when access barriers arise.The case study describes state-aware scaffolding spanning macro-topology initialization, granular adaptation, and navigation to prevent reasoning drift.Source: §5.6 · p. 10, Figure 4 · p. 10

6 Conclusion

TodoEvolve replaces rigid, one-size-fits-all agent planning with autonomous synthesis of task-specific planning architectures. Across diverse benchmarks, it outperforms static baselines while balancing success rates and computational efficiency.

6 Conclusion

  • TodoEvolve outperforms static baselines across diverse benchmarks, achieving Pareto optimality between success rates and computational efficiency.The evaluations support its performance and efficiency claim across diverse benchmarks.Source: §6 · p. 10
  • TodoEvolve uses IGPO to navigate PlanFactory’s unified design space and dynamically configure task-specific planning topologies and strategies.The approach aims to make architectural design adaptive to task demands.Source: §6 · p. 10
  • The paper frames TodoEvolve as a shift from manually engineered, one-size-fits-all workflows toward autonomous synthesis for diverse, open-ended, long-horizon tasks.It presents this approach as a blueprint for self-evolving agents that bridge internal reasoning and external architectural scaffolding.Source: §6 · p. 10

A PlanFactory Details

PlanFactory characterizes established planning systems through their topology, initialization, navigation, and adaptation mechanisms. The systems instantiate these dimensions through distinct coordination, dependency, and revision strategies.

Co-Sight

  • Co-Sight combines a cross-check net with inconsistency-triggered planning, conflict-resolution navigation, and meta-verification adaptation.Planning activates only when internal modules detect conflicting information or divergent perspectives; meta-verification assesses the verification logic used to build consensus.Source: A PlanFactory Details · p. 15

AgentOrchestra

  • AgentOrchestra uses a command hierarchy, role-based task decomposition, centralized routing, and environment-feedback-driven plan recalibration.A planning agent dispatches work to specialized sub-agents and synthesizes execution data and feedback while monitoring progress.Source: A PlanFactory Details · p. 16

OAgents

  • OAgents models objectives as dependent modules, dispatches ready nodes through dynamic programming, and periodically refines task order using critic feedback and prior trajectories.Every N steps, results are checked against global constraints; prior trajectories become soft constraints or behavioral preferences that bias sub-task selection.Source: A PlanFactory Details · p. 16

JoyAgent

  • JoyAgent pairs supervisor-led plan-and-execute oversight with responsive single-agent ReAct loops and consensus voting over pooled outputs.Individual agents adjust from local feedback while preserving the overarching trajectory.Source: A PlanFactory Details · p. 16

Flash-Searcher

  • Flash-Searcher builds a dependency-parsed task DAG and dispatches nodes aggressively in parallel once prerequisites or auxiliary validation permit.At defined intervals, workflow pruning removes resolved nodes, reevaluates pending dependencies, and can inject branches for environmental contingencies.Source: A PlanFactory Details · p. 16

FlowSearch

  • FlowSearch incrementally builds a thought graph, dispatches dependency-ready nodes, and dynamically adds, deletes, or modifies nodes and edges as knowledge changes.A knowledge flow planner decomposes tasks or adds context, while a knowledge refiner transforms the graph using newly acquired insights.Source: A PlanFactory Details · pp. 16–17

OWL

  • OWL separates strategic management from tactical execution, dispatches tasks by worker capability, and replans when decentralized failure signals indicate an impasse.Worker failure signals reach a dedicated task channel, prompting the planner to revise sub-tasks using execution feedback.Source: A PlanFactory Details · p. 17

B Datasets

The evaluation uses five datasets: GAIA, WebWalkerQA, xBench-DeepSearch, TaskCraft, and DeepSearchQA.

B Datasets

  • GAIA contains 165 tasks—53 Level-1, 86 Level-2, and 26 Level-3—while WebWalkerQA evaluates multi-turn web interactions using 170 sampled queries.WebWalkerQA comprises 680 real-world queries across four domains and more than 1,373 webpages.Source: B Datasets · p. 17
  • xBench-DeepSearch contains 100 tasks assessing planning, tool use, and reasoning; TaskCraft contributes 300 queries from a synthetic benchmark.TaskCraft is generated via an autonomous data pipeline.Source: B Datasets · p. 17
  • DeepSearchQA targets agents’ long-horizon research capabilities, with 50 queries collected as a valid subset.Source: B Datasets · p. 17

C Case Study

TodoEvolve synthesizes task-specific planning architectures, adapting control flow from linear pipelines to parallel graphs according to task dependencies and complexity. Three examples illustrate distinct approaches to filtering, structured extraction, and entity resolution.

C Case Study

  • The three case studies show TodoEvolve configuring topology, initialization, navigation, and adaptation triggers to match each task’s cognitive impedance and dependency requirements.The examples range from linear sequential logic to complex parallel graph structures.Source: C Case Study · p. 17
  • For multi-criteria filtering, TodoEvolve uses a linear fetch-and-filter pipeline with periodic validation before final synthesis and verification.The sequential topology manages data dependencies while minimizing branching overhead.Source: Figure 5 · p. 18
  • For structured data extraction, TodoEvolve builds a Knowledge Flow Graph with state-aware routing that selects parallel extraction or sequential reasoning based on node status.The topology uses acquisition, filtering, and finalization nodes to prune the search space while respecting numerical constraints.Source: Figure 6 · p. 19
  • For complex entity resolution, TodoEvolve parallelizes independent subject sub-goals and merges graph updates only when dependency conditions are met.A Parallel Executor minimizes latency while the adaptation layer monitors synchronization across concurrent streams.Source: Figure 7 · p. 20

Related papers

Loading 2602.07839v1…