PaperlayerView exact revision on arXiv

Source-linked AI summary

daVinci-Agency: Unlocking Long-Horizon Agency Data-Efficiently

Mohan Jiang, Dayuan Fu, Junhao Shi, Ji Zeng, Weiye Si, Keyu Li, Xuefeng Li, Yang Xiao, Wenjie Li, Dequan Wang, Pengfei Liu

arXiv:2602.02619v2cs.LGcs.AIcs.SE

TL;DR

Long-horizon agents need training data that captures cross-stage dependencies and evolution, but such supervision is scarce. daVinci-Agency mines interdependent task chains from real-world Pull Request sequences; fine-tuning GLM-4.6 on 239 samples yields broad benchmark improvements, including a 47% relative gain on Toolathlon.

  • Problem

    Long-horizon agent learning lacks scalable training data that captures cross-stage dependencies and evolution, which are needed to supervise task decomposition, consistency, and refinement.

    Source: §1 · p. 2

  • Method

    daVinci-Agency organizes real-world Pull Requests sharing a functional goal into interdependent task chains that model iterative submission, feedback, and refinement.

    Source: §1 · p. 2

  • Results

    47% relative gain on Toolathlon; fine-tuning GLM-4.6 on 239 daVinci-Agency samples also yields broad improvements across general benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings support high-quality structural data from software evolution as a data-efficient basis for agents to master complex engineering workflows.

    Source: §7 · p. 11

    Task chains are currently limited to five Pull Requests because of success-rate constraints.

    Source: §7 · p. 11

Abstract

from arXiv · show

While Large Language Models (LLMs) excel at short-term tasks, scaling them to long-horizon agentic workflows remains challenging. The core bottleneck lies in the scarcity of training data that captures authentic long-dependency structures and cross-stage evolutionary dynamics--existing synthesis methods either confine to single-feature scenarios constrained by model distribution, or incur prohibitive human annotation costs, failing to provide scalable, high-quality supervision. We address this by reconceptualizing data synthesis through the lens of real-world software evolution. Our key insight: Pull Request (PR) sequences naturally embody the supervision signals for long-horizon learning. They decompose complex objectives into verifiable submission units, maintain functional coherence across iterations, and encode authentic refinement patterns through bug-fix histories. Building on this, we propose daVinci-Agency, which systematically mines structured supervision from chain-of-PRs through three interlocking mechanisms: (1) progressive task decomposition via continuous commits, (2) long-term consistency enforcement through unified functional objectives, and (3) verifiable refinement from authentic bug-fix trajectories. Unlike synthetic trajectories that treat each step independently, daVinci-Agency's PR-grounded structure inherently preserves the causal dependencies and iterative refinements essential for teaching persistent goal-directed behavior and enables natural alignment with project-level, full-cycle task modeling. The resulting trajectories are substantial--averaging 85k tokens and 116 tool calls--yet remarkably data-efficient: fine-tuning GLM-4.6 on 239 daVinci-Agency samples yields broad improvements across benchmarks, notably achieving a 47% relative gain on Toolathlon. Beyond benchmark performance, our analysis confirms...

1. Introduction

daVinci-Agency uses chains of real-world Pull Requests to create long-horizon agent training data with cross-stage dependencies and refinement. Fine-tuning on 239 samples yields broad benchmark improvements, while case studies and scaling experiments examine agent skills and longer interaction budgets.

1 Introduction

  • Long-horizon learning requires supervision for task decomposition, consistency, and refinement across stages, where dependencies and error accumulation expose challenges single-feature tasks may miss (Guo et al., 2025; Radhakrishnan et al., 2023; Yao et al., 2022).The stated bottleneck is scalable data with long dependency structures and cross-stage evolution.Source: §1 · p. 2
  • PR sequences link multiple submissions to one functional goal, with later PRs addressing defects or incorporating feedback, thereby providing evolving states and external verification for long-cycle task modeling.The introduction contrasts this modality with synthetic trajectories limited by generation distributions and manual annotation that is costly to scale (Zhang et al., 2025; Wang et al., 2023; Wang and Ammanabrolu, 2025; Xiao et al., 2026).Source: §1 · p. 2
  • Chains of up to five PRs produce trajectories averaging 85k tokens and 116 tool calls, organized around a shared objective as interdependent submission-feedback-refinement tasks.Agents must progress through phases while maintaining overall goal consistency and correcting errors amid evolving code and external feedback.Source: §1 · p. 2
  • 239 daVinci-Agency samples fine-tuned on GLM-4.6 yield broad benchmark improvements, including a 47% relative gain on Toolathlon.The paper also reports that case studies confirm robust planning and error correction.Source: §1 · p. 2
  • Case studies report stronger task decomposition, long-term consistency, and refinement, while scaling experiments indicate that longer closed-loop training data and larger interaction budgets can advance agent performance.The stated contributions also identify expanding both training horizon and inference-time interaction budgets as mechanisms for continued improvement.Source: §1 · p. 2

2. Related Works

Related work traces agentic models from tool invocation and reasoning-action integration toward long-horizon benchmarks, but existing systems and data methods still struggle to teach complex interaction and long-horizon skills explicitly. daVinci-Agency responds with dense multi-turn interaction trajectories and PR-derived supervision for task decomposition, consistency, and iterative refinement.

2 Related Works

  • Toolformer (Schick et al., 2023) introduced self-supervised API invocation, ReAct (Yao et al., 2022) interleaved reasoning and action, and later foundation models targeted agentic performance.Examples include GLM-4.5 (Zeng et al., 2025) and Kimi-K2 (Team et al., 2025), which provide unified agentic capabilities and native tool use, respectively.Source: §2 · p. 3
  • Agentic models still struggle with coherent tool use and efficient feedback processing in complex multi-turn interactions, motivating daVinci-Agency’s high-interaction-density, long-horizon trajectories.The approach trains frequent tool calling and environmental-state interaction throughout extended reasoning dialogues.Source: §2 · p. 3
  • SWE-rebench (Badertdinov et al., 2025) and SWE-Smith (Yang et al., 2025b) extend GitHub-based execution but leave decomposition, consistency, and refinement implicit rather than trainable objectives.daVinci-Agency mines code evolution across Pull Requests to provide explicit supervision for these long-horizon skills; benchmarks include SWE-bench (Jimenez et al., 2023), Toolathlon (Li et al., 2025a), UltraHorizon (Luo et al., 2025), and SWE-bench Pro (Deng et al., 2025).Source: §2 · p. 3

3 Preliminary

The section formalizes agent-environment interaction and models software repositories as evolving through dependent PR chains. It then defines how an agent uses a chain-derived query to produce evaluated trajectories for training.

3.1 Agent-Environment Interaction Paradigm

  • Agent behavior is modeled as an MDP in which reasoning messages and tool executions interleave to transform a repository from an initial state toward a query-satisfying target.A trajectory consists of observations and corresponding reasoning and tool actions over a horizon of length N.Source: §3.1 · p. 3, §3.2 · p. 4

3.2 Software Evolution and Chains of Pull Requests

  • Repositories are modeled as evolving through ordered chains of semantically linked PRs, where each PR’s success can depend on correctly resolving its predecessor.A PR pairs natural-language context, such as issue descriptions and comments, with a ground-truth code patch; later PRs may iterate on, fix, or extend earlier features.Source: §3.2 · p. 4

3.3 Task Formulation

  • For a query derived from a PR chain, the policy conditions reasoning and tool actions on interaction history, then submits a patch evaluated against the ground truth.Rejection sampling retains trajectories with evaluation score s ≥ 0.8 for training, which minimizes negative log-likelihood loss.Source: §3.3 · p. 4

4 daVinci-Agency: A Long-Horizon Data Synthesis Paradigm

daVinci-Agency mines PR chains to create supervision for long-horizon agentic behavior, then collects sequential rollouts that preserve evolving code state. Its pipeline combines repository curation, query construction, stateful execution, and rejection sampling.

4.1 Data Resource

  • Nine repositories with over 7,000 effective PRs each were selected for scale, active code review, and linguistic diversity across technology stacks.The selection used an expert-led, human-in-the-loop assessment.Source: §4.1 · p. 5

4.2 Data Curation Pipeline

  • PR chains reconstruct ordered dependencies from commit and review references, including non-contiguous iterations, to model feature evolution and long-term consistency.The chain topology is built from GitHub API metadata and explicit citations in commit messages and review comments.Source: §4.2 · p. 5
  • LLM-generated sub-queries explain each PR’s problem, reasoning, and relevant symbols while withholding implementation details; the initial query also gives a global chain overview.The query is formalized as q = f(x, ˆp, R), using natural-language context and the PR’s ground-truth patch.Source: §4.2 · p. 5, §4.3 · p. 6

4.3 Rollout Data Collection for Training Dataset

  • Rollouts proceed sequentially, carrying each stage’s cumulative agent patch into the next base codebase so later tasks build on the agent’s evolved state.This recursive state transition is intended to support consistency and refinement across accumulated errors.Source: §4.3 · p. 6
  • GLM-4.6 evaluates generated patches against ground truth and retains only trajectories scoring at least 0.8 after up to three refinement iterations.Trajectories that fail the semantic-alignment threshold are discarded.Source: §4.3 · p. 6
  • GLM-4.6 runs with SII-CLI and miniswe-agent scaffolds, logging rollout observations, reasoning, and tool execution.The paper describes these scaffolds as capturing diverse agentic behaviors.Source: §4.3 · p. 6

5 Experiments

The experiments compare daVinci-Agency with agent-dataset baselines and PR-structure variants, evaluate it on long-horizon benchmarks, and test transfer across model architectures and scales.

5.1 Experiment Setups

  • The setup compared open-source model baselines and external agent datasets, tested two PR-structure variants, and evaluated feature-level, tool-use, interactive, and file-level coding tasks.Variants restricted tasks to isolated PRs or concatenated chronologically adjacent PRs; benchmarks included SWE-bench Verified, Toolathlon, τ 2-Bench, DS-1000, and SciCode.Source: §5.1 · p. 7

5.3 Model Performance Comparison

  • 0.475 overall average score and leads on SWE-bench, Toolathlon, and τ 2-bench show daVinci-Agency’s benchmark strength against the tested models.On AgencyBench Code, it scores 15.9. Training trajectories were generated exclusively by GLM-4.6, supporting the paper’s characterization of the setup as on-policy self-distillation rather than distillation from a stronger teacher.Source: Table 2 · p. 8, §5.3 · p. 8, Table 3 · p. 8

5.2 Main Results

  • Despite its smaller sample size, daVinci-Agency outperformed external agent-dataset baselines across benchmarks; PR evolution variants showed the value of semantically linked cross-stage supervision.SinglePR lacked cross-stage evolution, while TemporalChain concatenated chronologically adjacent PRs without explicit semantic links; daVinci-Agency achieved higher data efficiency.Source: §5.2 · p. 8

5.4 Model Generalization Analysis

  • Qwen3-30B-A3B reached 0.307 average and 0.262 SWE-bench from 0.242; Qwen3-32B reached 0.292 average, while Qwen3-8B also improved overall and on coding benchmarks.The Qwen3 evaluation covered Mixture-of-Experts and dense architectures at different parameter counts.Source: §5.4 · p. 9

6 Long Horizon Analysis

Section 6 shows that daVinci-Agency supports more structured long-horizon behavior, reduces token and tool use, and benefits from longer training trajectories and larger inference budgets.

6.1 Meta-Skills Performance Comparison

  • On Django Issue #11149, the baseline drifts into irrelevant modules and relies on simplistic scripts, while daVinci-Agency decomposes validation and catches a recursion error before execution.The trained model configures the environment before running tests from general to specific, then fixes the recursion error using super().Source: §6.1 · p. 9

6.2 Efficiency Gains in Token and Tool Usage

  • GLM-4.6 uses 113.6K fewer tokens and Qwen3-32B uses 288.8K fewer tokens on SWE-bench; tool use falls by up to 25.8% on SWE-bench and 13.3% on Toolathlon.The paper attributes the efficiency to denser reasoning that prunes redundant actions and preserves context for long-term dependencies.Source: Figure 6 · p. 10, §6.2 · p. 10

6.3 Long Horizon Data Scaling

  • On SWE-bench, longer PR chains unlock previously unrealized gains for Qwen3-32B, supporting trajectory length as a scaling dimension for agentic problem-solving.The authors report that this strategy can improve training data that had not previously yielded performance gains.Source: §6.3 · p. 10
  • On SWE-bench, the performance gap over baselines widens as the tool-call limit increases, indicating greater stability and less error accumulation in long sequences.The reported effect concerns more ample inference budgets on long-horizon tasks.Source: §6.3 · pp. 10–11, Figure 7 · p. 11
  • Extending trajectories from 59.39K to 84.82K tokens yields relative gains of up to 8% on SWE-bench and τ 2-bench.The longer trajectories are constructed by completing additional pull requests in previously unfinished chains.Source: §6.3 · p. 10, Figure 7 · p. 11

7 Conclusion · A. Repositories in daVinci-Agency SII-GAIR

daVinci-Agency mines real-world software evolution into semantically dependent PR chains for long-horizon agent training, with results showing data efficiency from 239 samples and future work targeting longer chains.

7 Conclusion

  • By mining real-world software evolution, daVinci-Agency reconstructs semantically dependent PR chains that turn isolated coding tasks into coherent multi-stage workflows.The approach is presented as a scalable, verifiable training paradigm that avoids prohibitively expensive human annotation and teacher-bounded distillation.Source: §7 · p. 11
  • Fine-tuning on 239 daVinci-Agency samples significantly outperforms baselines trained on much larger datasets, with substantial gains on SWE-bench and Toolathlon.The conclusion attributes this efficiency to agents internalizing task decomposition, long-term consistency, and iterative refinement through structurally rich data.Source: §7 · p. 11
  • The PR-chain structure is intended to teach task decomposition, long-term consistency, and iterative refinement that standard distillation methods fail to capture.The conclusion identifies these behaviors as critical to long-horizon agency.Source: §7 · p. 11
  • Current chains connect at most 5 PRs because of success-rate constraints, while further extending them is presented as a viable path for improvement.Future work aims to overcome those bottlenecks and construct longer evolutionary chains.Source: §7 · p. 11

A Repositories in daVinci-Agency

daVinci-Agency draws on nine source repositories selected to cover diverse technology stacks and development scales. The selection spans scientific computing, distributed systems, and modern systems tooling.

A Repositories in daVinci-Agency

B Efficacy of Rejection Sampling … D. Evaluator Prompt SII-GAIR

Rejection sampling improves fine-tuned performance over both unfiltered training and the base model. The accompanying prompts define how PR changes become intent-based queries and how generated patches are evaluated.

B Efficacy of Rejection Sampling

  • Rejection sampling recovers baseline capabilities and raises the model to a superior average score of 0.421; on SciCode-MP, score more than doubles from 0.062 in the base model to 0.169.Source: B Efficacy of Rejection Sampling · p. 14
  • Unfiltered trajectories drop average score to 0.205, below the base model’s 0.405, indicating substantial degradation from noisy self-generated data.The controlled ablation compares the base model with variants fine-tuned with and without the filter.Source: B Efficacy of Rejection Sampling · p. 14, Table 5 · p. 14

C Query Constructor Prompt

  • The query constructor turns PR data into a benchmark prompt by combining the change’s goal, conceptual location, abstract logic, and a high-level directive.It is intended to test whether a separate coding agent can reproduce the PR’s functional changes.Source: C Query Constructor Prompt · p. 15
  • The query must explain intent, identify file paths and variable purposes, and avoid literal code identifiers, detailed execution instructions, step lists, URLs, and code snippets.The prompt requires a conceptual reverse-instruction rather than a description of specific code changes.Source: C Query Constructor Prompt · p. 15

D. Evaluator Prompt SII-GAIR

  • For the UserProxyAgent example, the generated query directs changing the NEVER-mode description to define a code terminal that cannot handle natural-language requests.The input provides PR data and a specific execution plan; the output must contain only an English query string.Source: D. Evaluator Prompt SII-GAIR · p. 16

D Evaluator Prompt

E. Training Parameters SII-GAIR

The section specifies a patch-review rubric and requires the verdict to be returned as a strict JSON object. The score reflects the presence and severity of critical errors, while the comment summarizes the verdict.

E. Training Parameters SII-GAIR

  • Review patches for syntax, runtime, logic, and resource errors such as memory leaks or infinite loops.The rubric also checks for compilation or parsing issues, crashes or exceptions, and algorithmic correctness.Source: E. Training Parameters SII-GAIR · p. 18
  • Return only a strict JSON object with a score and comment, without Markdown fences or surrounding prose.The specified shape contains the keys “score” and “comment.”Source: E. Training Parameters SII-GAIR · p. 18
  • The score must be a floating-point number between 0.0 and 1.0.Source: E. Training Parameters SII-GAIR · p. 18
  • Use the concise comment to explain the overall verdict by highlighting a key blocking issue or stating that the patch aligns with ground truth.The comment should describe the overall verdict in natural language.Source: E. Training Parameters SII-GAIR · p. 18

E Training Parameters

The training configuration specifies supervised fine-tuning data and batching, loss behavior, and model-parallel and memory-related settings.

E Training Parameters

  • SFT uses two epochs with rollout and global batch sizes of 64, reading prompts from messages and tools from the tools key.The rollout configuration enables shuffling and uses the SFT rollout function.Source: E Training Parameters · p. 18
  • Training uses SFT loss with per-token loss calculation, disables advantage and return computation, and runs in train-only debug mode.Source: E Training Parameters · p. 18
  • Parallelism settings are tensor 8, pipeline 4, context 2, and expert 16, with sequence parallelism enabled and expert tensor parallelism set to 1.Source: E Training Parameters · p. 18
  • The configuration uses full uniform recomputation every layer and dynamic batching capped at 65,536 tokens per GPU.Source: E Training Parameters · p. 18

E. Training Parameters SII-GAIR

SII-GAIR training uses Adam with cosine learning-rate decay, warmup, weight decay, CPU optimizer offload, and precision-aware optimization.

E. Training Parameters SII-GAIR

  • Training uses Adam (lr 2e-6, cosine decay to min-lr 1e-7, warmup fraction 0.1, weight decay 0.1, betas 0.9/0.98), CPU offload with D2H/H2D overlap, and a precision-aware optimizer.Source: E. Training Parameters SII-GAIR · p. 19

Related papers

Loading 2602.02619v2…