PaperlayerView exact revision on arXiv

Source-linked AI summary

ProgAgent: Learning Tool Orchestration through Programmatic Training

Hao Li, Keqing He, Longtao Zheng, Junlong Li, Weihao Zeng, Qian Liu, Yuzhen Huang

arXiv:2610.14774v1cs.AI

TL;DR

Long-horizon tool orchestration with DTC can accumulate tool outputs in context, while the training benefits of PTC and suitable verifiable tasks remain unclear. ProgAgent provides a PTC training stack, and its models outperform evaluated comparable-size baselines across four benchmarks, with cross-harness results extending to DTC.

  • Problem

    The benefits of training agents with PTC remain unclear, and publicly available verifiable long-horizon tasks involving diverse tools are scarce.

    Source: §1 · p. 1

  • Method

    ProgAgent combines verifiable long-horizon task synthesis, a lightweight PTC runtime, and cold-start SFT followed by RL.

    Source: §1 · p. 2

  • Results

    On LOCA-bench, ProgAgent-14B achieves 30.7% at a 64k-token environment state, while comparable-size baselines score 0%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Across four benchmarks, ProgAgent outperforms evaluated comparable-size baselines, and PTC-trained models outperform DTC-trained models under both evaluation interfaces.

    Source: §1 · p. 2, §6 · p. 9

    When non-compliant MCP servers omit the isError field, disabling server-level error detection lowers LOCA-bench pass rates for both model sizes.

    Source: §5.4 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are increasingly expected to orchestrate diverse tools over long-horizon tasks. Existing agents typically use Direct Tool Calling (DTC), where the model invokes tools sequentially, and tool responses accumulate in the model context, which can lead to context rot and orchestration breakdown. In contrast, Programmatic Tool Calling (PTC) invokes tools through executable code and stores intermediate results in runtime variables rather than appending them to the context. Prior work has explored PTC mainly as an inference-time optimization, but its benefits for agentic training remain unclear, and suitable verifiable training tasks are scarce. In this work, we introduce ProgAgent, a full PTC training stack, comprising a pipeline for synthesizing verifiable long-horizon tasks, a lightweight PTC runtime, and a two-stage training recipe. Using only 215 tasks instantiated at four environment scales, ProgAgent yields higher pass rates than the evaluated baselines of comparable size across four benchmarks. On LOCA-bench, ProgAgent-14B achieves a 30.7% pass rate at a 64k-token environment state, where these baselines score zero. Cross-harness experiments show that PTC-trained models outperform DTC-trained models under both evaluation interfaces, suggesting that the benefits of PTC training extend beyond the interface.

1 INTRODUCTION

Long-horizon tool orchestration remains difficult, and the training benefits of PTC are unclear despite its potential to use code for orchestration and limit context growth. ProgAgent addresses gaps in verifiable tasks and runtime support, then improves performance across four benchmarks.

1 INTRODUCTION

  • Long-horizon open-world tool environments challenge LLM agents, while DTC accumulates raw tool outputs in context, which can contribute to context rot and orchestration breakdown.Complex orchestration also involves data transformation, branching, and error handling that are difficult to express through sequential calls.Source: §1 · p. 1
  • PTC’s training benefits remain unclear, and systematic study is constrained by scarce verifiable long-horizon tasks and the lack of runtimes suited to controlled training.Existing inference-time evaluations do not establish whether PTC training improves orchestration or transfers to DTC.Source: §1 · pp. 1–2
  • ProgAgent combines verifiable long-horizon tasks, a state-preserving PTC runtime, and cold-start SFT followed by RL.The tasks are difficulty-scaled and use Python verifiers that return binary rewards.Source: §1 · p. 2
  • ProgAgent improves performance across four benchmarks and beats comparable-size evaluated baselines; on LOCA-bench, ProgAgent-14B scores 30.7% at 64k tokens while comparable baselines score 0%.Cross-harness evaluations also find PTC-trained models outperform DTC-trained models under both interfaces.Source: §1 · p. 2

2 REFRAMING TOOL USE AS CODE

The paper models multi-turn tool use as partial observation, then contrasts sequential DTC with code-based PTC. PTC may draw on coding capabilities for orchestration while retaining intermediate results outside the model context.

2.1 MULTI-TURN AGENTIC INTERACTION AS A PARTIALLY OBSERVABLE MARKOV DECISION PROCESS

  • The POMDP framing separates environment state from interaction history, while the agent receives observations rather than the full state and is rewarded by an end-of-episode verifier.The environment contains databases, workspace files, and executable tools; interaction history tracks instructions, observations, and reasoning context.Source: §2.1 · p. 3

2.2 FROM DIRECT TOOL CALLING TO PROGRAMMATIC TOOL CALLING

  • DTC appends each tool action and returned observation to context, so execution logic and raw outputs accumulate and can contribute to context rot and orchestration breakdown.The figure contrasts this with PTC, which uses code and selectively prints outputs.Source: §2.2 · p. 3, Figure 2 · p. 3
  • PTC runs programs that can coordinate multiple tool calls using control flow, data transformation, and exception handling, while keeping intermediate results in runtime variables across turns.Only the program and its printed output are appended to context, allowing selective printing to limit tool-output tokens.Source: §2.2 · p. 3

2.3 PTC AS AN EFFECTIVE TRAINING PARADIGM

  • PTC may transfer pretrained coding capabilities to tool orchestration by applying programming structures to related tool calls, while runtime variables and selective printing provide a mechanism for context management.The paper investigates whether this paradigm improves performance with fewer training tasks and whether training can teach context filtering to mitigate context rot.Source: §2.3 · p. 4

3 PROGAGENT: FROM TASK SYNTHESIS TO AGENTIC TRAINING

ProgAgent combines task synthesis, a lightweight PTC runtime, and cold-start SFT followed by RL. Its task pipeline controls environment-state scale while keeping task logic and tool schemas fixed, and filters tasks through auditing and difficulty checks.

3 PROGAGENT: FROM TASK SYNTHESIS TO AGENTIC TRAINING

  • ProgAgent combines PTC-tailored verifiable tasks with controllable difficulty, a persistent-kernel runtime with sandbox isolation, and cold-start SFT followed by RL.The runtime preserves execution state across turns; the training stack is designed for long-horizon tool orchestration.Source: §3 · p. 4

3.1.1 DESIGN PRINCIPLES

  • Environment-state scaling keeps instructions, task logic, and tool schemas fixed while increasing state size, making agents retrieve and process information from larger environments.Each scale roughly doubles the environment state; DeepSeek V4 Pro’s DTC pass@8 falls from 0.95 to 0.42 from v0 to v3, while its PTC pass@8 stays above 0.90 at all four scales.Source: §3.1.1 · p. 4
  • Tasks distribute related data across relational tables and put constraints in unstructured files, requiring information combination that loops and branching can support.Each task receives a Python verifier that checks the final response and environment state; an oracle solution must pass, with automated repair if it fails.Source: §3.1.1 · p. 4

3.1.2 TASK CONSTRUCTION PIPELINE

  • The construction pipeline generates intents, databases, workspaces, tools, instructions, a verifier, and an oracle solution, then creates four state-scaled variants while preserving task logic and tool schemas.The oracle is used to iteratively repair tasks until it receives full reward.Source: §3.1.2 · pp. 4–5

3.1.3 VERIFICATION-BASED FILTERING

  • After independent consistency audits and repairs, tasks are difficulty-filtered using eight PTC trajectories per scale, retaining tasks with pass rates in (0, 0.8) at every scale.Each retained task is packaged with a data generator, environment module, and instruction.Source: §3.1.3 · p. 5

3.2 A LIGHTWEIGHT PTC RUNTIME

  • The PTC runtime uses a persistent sandboxed IPython kernel to execute programs, route tool calls, retain variables across turns, and return printed output; the verifier returns a binary reward.The runtime isolates episodes during rollouts and prevents access to the verifier implementation and oracle solution.Source: §3.2 · p. 5

3.3 TRAINING WITH PROGAGENT

  • Training uses cold-start SFT on teacher trajectories followed by RL with GSPO, applying binary verifier rewards across DTC, PTC, and Hybrid harnesses.The same task pool and training hyperparameters are used across the three harnesses to isolate the harness effect.Source: §3.3 · p. 5

4 EXPERIMENTS

Across four agentic benchmarks, ProgAgent outperforms comparable baselines, and PTC training achieves the highest aggregate score on each benchmark at both model scales. The experiments train 8B and 14B models on tasks instantiated at four environment scales and compare DTC, PTC, and Hybrid harnesses.

4.1 EXPERIMENTAL SETUP

  • The experiments use 215 tasks instantiated at four scales and train Qwen3 8B and 14B models with DTC, PTC, and Hybrid harnesses.Training uses cold-start fine-tuning followed by GSPO; evaluations cover MCPMark, MCP-Atlas, Toolathlon-Verified, and LOCA-bench.Source: §4.1 · pp. 5–6

4.2 MAIN RESULTS

  • ProgAgent beats comparable baselines across benchmarks, with MCPMark pass rates of 18.3% (8B) and 29.7% (14B), and Toolathlon-Verified pass rates of 9.3% (8B) and 17.1% (14B).On MCP-Atlas, ProgAgent-14B reaches 45.6%, comparable to Claude Sonnet-4.5 at 43.8%.Source: Table 2 · p. 6, §4.2 · p. 6
  • After SFT and RL, the PTC recipe achieves the highest aggregate score on each benchmark at both model scales, reversing the pattern for untrained backbones.For untrained backbones, switching to PTC provides limited benefits and often yields the lowest scores among the three harnesses.Source: §4.2 · p. 6, Table 3 · p. 7
  • At 64k environment-state size, ProgAgent maintains LOCA-bench pass rates of 17.3%–30.7%, while comparable-size baselines score zero at 32k and 64k.Across environment-state sizes, PTC sustains performance as the environment grows, while DTC suffers from context rot; Hybrid performs worse than PTC.Source: §4.2 · pp. 6–7, Figure 3 · p. 7

5 ANALYSIS

The analysis finds that PTC-trained agents use less context and fewer calls while outperforming DTC-trained models across harnesses. It also reports benefits from environment scaling and identifies behavior and runtime limitations.

5.1 WHAT DOES PTC CHANGE DURING TRAINING?

  • PTC reward reaches 0.70 and Hybrid 0.72, versus DTC’s 0.15; PTC cuts context tokens roughly 20%, and both programmatic paradigms use roughly half DTC’s tool calls.DTC reward rises only from 0.08 to 0.15, while PTC and Hybrid start near 0.50.Source: §5.1 · pp. 7–8, Figure 4 · p. 8

5.2 DOES ENVIRONMENT STATE SCALING HELP?

  • 0.95 to 0.42: DeepSeek V4 Pro’s DTC pass@8 declines from v0 to v3; training on all four scales yields higher LOCA-bench averages than v0-only at both model sizes.The reported v0-only versus four-scale averages are 16.0% versus 24.3% at 8B and 23.7% versus 38.3% at 14B.Source: §5.2 · p. 8, Table 4 · p. 8

5.3 HOW DOES PTC FACILITATE TOOL ORCHESTRATION?

  • Loops, conditional execution, and error handling appear in PTC training, alongside programs coordinating dependent calls and selectively printing aggregated results.The Toolathlon-Verified example coordinates dependent calls; the MCP-Atlas example aggregates intermediate results and selectively prints a summary.Source: §5.3 · p. 8, Table 5 · p. 8
  • 11.9 percentage points: under DTC evaluation, the 14B PTC-trained model beats its DTC-trained counterpart on MCP-Atlas; under PTC, PTC-trained models lead on both benchmarks.The cross-harness comparison covers LOCA-bench and MCP-Atlas.Source: §5.3 · p. 9, Table 6 · p. 9

5.4 WHAT CHALLENGES REMAIN FOR PTC AGENTS?

  • Unpredictable or highly unstructured returns can prompt DTC-like behavior; on Playwright, 40.7% of ProgAgent-14B PTC calls contain a single invocation without programmatic control features.The authors say this behavior limits the benefits of programmatic orchestration and context management.Source: §5.4 · p. 9
  • When non-compliant servers omit MCP’s isError field, exception-based recovery can be bypassed and tool errors may be processed as successful returns.Disabling server-level error detection reduces LOCA-bench pass rates from 24.3% to 19.3% at 8B and from 38.3% to 34.5% at 14B.Source: §5.4 · p. 9

6 CONCLUSION

ProgAgent combines verifiable tasks at multiple environment scales, a lightweight PTC runtime, and cold-start SFT followed by RL. It improves performance across four benchmarks, with cross-harness evaluations indicating that PTC-training benefits extend to DTC.

6 CONCLUSION

  • ProgAgent combines verifiable tasks at multiple environment scales, a lightweight PTC runtime, and cold-start SFT followed by RL.The stack is presented as a training approach for long-horizon tool orchestration.Source: §6 · p. 9
  • Using 215 tasks at four scales, ProgAgent improves performance across four benchmarks and outperforms the evaluated baselines of comparable size.Source: §6 · p. 9
  • Cross-harness evaluations show that the benefits of PTC training extend to DTC, supporting its use as a training paradigm as well as an inference interface.The conclusion motivates further study of how programmatic training shapes tool-use behavior across interfaces.Source: §6 · p. 9

AI USE STATEMENT

The paper reports its use of generative AI tools and reviews related work on programmatic tool calling and environment synthesis. Its appendix also describes the task-domain and tool distributions.

AI USE STATEMENT

  • Generative AI tools were used for language editing, synthetic task construction and auditing, and trajectory generation for filtering and training; the authors reviewed the content and take responsibility.Source: AI USE STATEMENT · p. 10

A RELATED WORK

  • Related work spans executable-code tool use, long-context processing, and simulated or executable environment synthesis; ProgAgent combines verifiable tasks at controlled scales with a state-preserving runtime.The related-work discussion contrasts ProgAgent’s long-horizon training focus with prior work’s differing aims and task support.Source: A RELATED WORK · p. 13

B.1 TASK AND TOOL DISTRIBUTIONS

  • The appendix reports 215 tasks across 23 top-level domains and notes that each task’s tool set remains unchanged across all four environment scales.Figure 5 presents the task-domain distribution; Figure 6 presents the number of available tools per task.Source: §B.1 · p. 13, Figure 5 · p. 13

B.2 TASK CONSTRUCTION PIPELINE IN DETAIL

This appendix details how ProgAgent constructs, certifies, scales, and filters long-horizon tool tasks, then describes its runtime, prompts, and training setup. As environment scale increases, DeepSeek V4 Pro’s pass@8 falls under DTC but remains above 0.87 under Hybrid and PTC.

B.2 TASK CONSTRUCTION PIPELINE IN DETAIL

  • Task synthesis starts with LLM-generated intents from curated domains, then builds a relational database and file workspace alongside single-purpose tools and task instructions.Tasks require combining database records with constraints in unstructured workspace files; tools return bounded data, so retrieving more can require multiple calls.Source: §B.2 · p. 14
  • A Python verifier checks the final response and environment state against task criteria, while an oracle solution is repaired against the verifier until it passes.The verifier returns 1 only when all criteria are met and does not use an LLM judge.Source: §B.2 · p. 14
  • Each task is instantiated at four scales by expanding database rows from 1× to 8× and workspace files from 1× to 4×, while keeping task logic and tools fixed.The instruction and verifier also remain fixed across scales.Source: §B.2 · p. 15
  • As environments scale from v0 to v3, pass@8 falls from 0.95 to 0.42 under DTC, while remaining above 0.87 under Hybrid and PTC.Figure 7 shows the Hybrid and PTC results; the DTC counterpart is in Figure 1.Source: Figure 7 · p. 14, §B.2 · p. 15

B.3 VERIFICATION-BASED FILTERING IN DETAIL

  • An independent solver audits candidate tasks at v3, then eight-trajectory-per-scale difficulty filtering retains tasks with pass rates strictly between 0 and 0.8 at every scale.This filtering removes 35 of 250 candidates, leaving 215 tasks.Source: §B.3 · p. 15

B.4 PROMPTS FOR TASK SYNTHESIS AND AUDITING

  • The task-construction prompts require realistic, verifiable workflows and byte-for-byte reuse of the environment and prompt across scales, with only the data scale configuration changing.An independent harness agent checks solvability and consistency across scale variants at v3.Source: Intent generation prompt (Stage I) · p. 15, §2 · p. 16, §4 · p. 16

C.2 PTC RUNTIME ARCHITECTURE

  • Under PTC, programs invoke environment tools through a persistent Python kernel; variables persist across turns, while printed output is returned to the model.Tool returns are native Python values, and the tool instructions recommend inspecting an initial result before processing many items.Source: Code-execution tool description (PTC) · pp. 17–18, §C.2 · p. 18

C.3 TRAINING HYPERPARAMETERS

  • The three harness settings use equal-sized SFT datasets covering all 215 tasks, followed by GSPO reinforcement learning with the same task pool and verifiers.DTC reinforcement learning uses only v0 environments because the other versions are too difficult for the model to obtain rewards.Source: §C.3 · p. 19

C.4 EVALUATION PROTOCOL

The evaluation keeps each benchmark’s official servers, verifier, prompts, and completion logic while testing models under specified harnesses. Reported scores combine the paper’s evaluations with results from prior benchmark papers and Agent-World.

C.4 EVALUATION PROTOCOL

  • Main evaluations use each trained model’s training harness, while DTC is used for baselines and untrained backbones; PTC calls have a 120-second timeout.Hybrid exposes both interfaces, while PTC requires benchmark tool calls to go through programmatic tool call.Source: §C.4 · p. 19
  • The MCPMark GitHub suite is excluded after repeated account suspensions; its reported average covers Filesystem, Notion, Playwright, and PostgreSQL.LOCA-bench is evaluated at four environment-state sizes, and MCP-Atlas uses the final pre-version-2.0 task set.Source: §C.4 · p. 19
  • Score provenance varies by benchmark: several comparison scores come from Agent-World or original benchmark papers, while the remaining scores come from the authors’ evaluations.Toolathlon scores marked † are from the original Toolathlon benchmark; other scores in that column are from Toolathlon-Verified.Source: §C.4 · p. 19

D.1 FULL ABLATION RESULTS

The ablations compare harnesses and training paradigms, test the role of error detection, and characterize interaction costs. PTC-trained models outperform DTC-trained models under the same harness in the v0-only comparison, while disabling server-error detection lowers PTC pass rates.

D.1 FULL ABLATION RESULTS

  • Pass rates remain near zero for untrained 8B and 14B backbones under DTC, Hybrid, and PTC.This establishes performance before training across the three harnesses.Source: §D.1 · p. 20
  • Selected AWM, EnvFactory, and EnvScaler checkpoints also remain near zero under PTC, indicating that access to the PTC interface alone is insufficient for effective orchestration.The passage describes this as consistent with §5.3.Source: §D.1 · p. 20
  • With training restricted to v0, PTC-trained models have higher average pass rates than DTC-trained models under the same harness after both SFT and RL at 8B and 14B.Both paradigms use all 215 tasks and equal numbers of SFT trajectories.Source: §D.1 · p. 20
  • Disabling server-level error detection lowers average pass rates for final PTC checkpoints at both model sizes.The authors relate this decrease to the importance of explicit exceptions for error recovery.Source: §D.1 · p. 20

D.2 INTERACTION COSTS ON MCPMARK

  • Playwright has the highest proportion of DTC-like PTC calls and more tool-output tokens per trajectory and context-limit hits than Filesystem and PostgreSQL.The comparison covers four MCPMark subsets and both model sizes.Source: §D.2 · p. 20

D.3 TRAINING DYNAMICS AT 8B

At 8B, PTC and Hybrid show higher episode rewards than DTC while using fewer tool calls and reducing tool-output tokens entering context. The qualitative examples illustrate PTC aggregation and dependent tool calls, with stated limits on what one example’s success checks verify.

D.3 TRAINING DYNAMICS AT 8B

  • Episode reward rises from approximately 0.05 to 0.08 under DTC, versus 0.38 to 0.56 under both PTC and Hybrid, as the latter use roughly half as many tool calls per episode.Tool-output tokens stay near 30k under DTC, but decrease from 26.2k to 22.5k under PTC and from 29.6k to 22.1k under Hybrid.Source: §D.3 · p. 21

E.1 MCP-ATLAS: FROM CAMPAIGN RECORDS TO A STATISTICAL ANSWER

  • For campaigns spending less than $800, Social Media was the most common ad type; Banner was the least common, with an average audience reach of 19380.5.Source: §E.1 · p. 24, §3 · p. 25

4. Check completeness before the final response

  • The campaign query returns 95 records with no further page, and the example checks that spending ranges from 509 to 798 across four ad types.These checks support completeness of the example’s query results.Source: §4 · p. 25

E.2 TOOLATHLON-VERIFIED: FROM AN EMAILED SCHEDULE TO PUBLISHED CANVAS COURSES

  • The course-schedule example uses returned identifiers to create and enroll instructors in 27 Canvas courses, then generates course announcements.The logged announcement examples include instructor names and class times.Source: §E.2 · p. 26, §2 · p. 27, §3 · p. 27

3. Create announcements using course IDs

  • The example’s success checks cover course publication, teacher count, public-syllabus status, and announcement presence, but not instructor identity, content, or resource sharing.Resource sharing is an explicit requirement when an instructor teaches the same course multiple times.Source: §E.2 · p. 26, §3 · p. 27

4. Check selected course settings and summarize results

The section checks each listed course’s availability, teacher assignment, public syllabus, and announcements, then reports instructor-course totals.

4. Check selected course settings and summarize results

  • Each course is checked for availability, exactly one correct teacher, a public syllabus, and at least one announcement.Sample course records report one teacher, a public syllabus, and one announcement.Source: §4 · pp. 27–28
  • 27 courses were verified successfully against the section’s checks.The final verification reports success for all 27 courses.Source: §4 · p. 28
  • 11 unique instructors are mapped across 27 courses, with the summary listing course assignments by instructor.The mapping includes instructors assigned to three courses each.Source: §4 · p. 28

Related papers

Loading 2610.14774v1…