PaperlayerView exact revision on arXiv

Source-linked AI summary

Marathoner: Ultra-Long-Horizon Autonomous Intelligence

Zhang Ruiyang, Ou Jinpeng, Xie Yifan, Zhou Jingang, Pan Lirui, Guo Qingpei, Zheng Zhedong

arXiv:2609.34378v1cs.CV

TL;DR

Open models lack the disclosed training methods and long-horizon capability seen in proprietary systems. Marathoner post-trains a base model with synthesized tasks, teacher trajectories, and reinforcement learning; it improves across five benchmarks and sustains execution for 10+ hours with 1000+ tool calls.

  • Problem

    Training methods for proprietary long-horizon models are rarely disclosed, while open-source models lack this capability for challenging, complex scenarios.

    Source: §1 · p. 2

  • Method

    Marathoner post-trains a base model using GitHub-PR-derived tasks, rejection-sampled teacher trajectories, and sandboxed reinforcement learning with Later Stage Bonus Reward.

    Source: §1 · p. 2, §2.3 · p. 6

  • Results

    Across five ultra-long-horizon benchmarks, Marathoner-9B consistently improves on Qwen3.5-9B and surpasses Gemini-3.1-Pro on FrontierSWE and SWE-Marathon.

    Source: §3.2 · p. 8

  • Takeaways & Limitations

    Marathoner can consistently execute for 10+ hours and make 1000+ tool calls on highly challenging tasks.

    Source: §1 · p. 2

Abstract

from arXiv · show

Humans naturally possess the ability to work persistently toward long-term goals. Given a challenging task, humans can continuously work for months or even years to accomplish a specific objective. In this paper, we propose Marathoner, an autonomous agentic model possessing the ability of ultra-long-horizon execution. Specifically, we propose a comprehensive post-training pipeline to instill this critical capability into base model. For Ultra-Long-Horizon Task Synthesis, we leverage major release PRs containing 1000+ lines of new code from diverse GitHub repositories as the primary source for synthesizing challenging task-level data. Additionally, we introduce Multi-Task Chaining, which chains multiple generated tasks into a single more challenging task, enabling the synthesis of tasks with frontier-level difficulty. For rejection sampling finetuning, we combine strong teacher model with diverse harnesses to generate trajectories on our synthesized tasks and conduct supervised finetuning on base model with rejection sampled trajectories. For reinforcement learning, cold-started model performs real-world execution through harnesses in independent sandboxes during rollout process, effectively facilitating the acquisition of genuine ultra-long-horizon execution capability. We further propose a novel reward strategy, Later Stage Bonus Reward, which explicitly encourages model to perform meaningful maneuvers during later stages of execution. Through extensive evaluation on 5 benchmarks containing ultra-long-horizon tasks, Marathoner achieves consistent and substantial performance improvements over base model and even surpasses performance of strong proprietary model. Further analysis shows that Marathoner can consistently work for 10+ hours and conduct 1000+ tool calls on highly challenging tasks.

1 INTRODUCTION

Humans can persist at complex goals for months or years, but methods behind proprietary models’ long-horizon capability are rarely disclosed and open models lack it. Marathoner uses post-training to develop that capability and reports benchmark gains alongside sustained execution exceeding 10 hours and 1,000 tool calls.

  • Marathoner combines task synthesis, rejection-sampling finetuning, and sandboxed reinforcement learning, with Later Stage Bonus Reward for meaningful actions during later execution stages.Source: §1 · pp. 2–3
  • Task synthesis mines 100,000 major release PRs from 10,000 repositories and chains generated tasks to create more complex, longer-horizon challenges.Source: §1 · p. 2
  • Across five ultra-long-horizon benchmarks, Marathoner improves substantially over the base model and surpasses a strong proprietary model, while sustaining 10+ hours and 1000+ tool calls on challenging tasks.Source: §1 · p. 2

2 METHODOLOGY

The methodology builds ultra-long-horizon execution through synthesized repository tasks, rejection-sampled supervised finetuning, and reinforcement learning.

2.1 ULTRA-LONG-HORIZON TASK SYNTHESIS

  • Tasks are mined from 100,000 release PRs across 10,000 repositories, graded by new-code size, represented with unit-test verifiers, and optionally chained in fives.The three difficulty levels use PRs with 100–200, 200–1,000, and 1,000+ lines of new code; chained tasks combine five atomic tasks.Source: §2.1 · p. 4

2.2 REJECTION SAMPLING FINETUNING

  • A teacher generates trajectories for each task through multiple harnesses; only reward-1 trajectories are retained for supervised finetuning of the base model.The teacher uses Claude Code, Codex, and OpenClaw to generate trajectories for the same task.Source: §2.2 · pp. 5–6

2.3 REINFORCEMENT LEARNING

  • RL uses GRPO on real-world harness rollouts, adding a 0.5 bonus when valuable operations occur in the latter half of execution.The correctness reward remains binary and is determined by whether the task's unit tests pass.Source: §2.3 · pp. 6–7

3 EXPERIMENT

Across five ultra-long-horizon benchmarks, Marathoner-9B improves over its base model and surpasses Gemini-3.1-Pro on two benchmarks. Analyses examine environment scaling, training dynamics, execution behavior, and key design ablations.

3.1 SETTING

  • The evaluation covers five challenging benchmarks, including software engineering, repository construction, and terminal-use tasks.The listed benchmarks include FrontierSWE, NL2Repo, SWE-Marathon, Terminal Bench 2.0, and SWE-Bench Verified.Source: §3.1 · pp. 7–8

3.2 MAIN RESULTS

  • Marathoner-9B consistently improves over Qwen3.5-9B across five benchmarks and surpasses Gemini-3.1-Pro on FrontierSWE and SWE-Marathon.The evaluation compares proprietary and open-source models without benchmark-specific optimization.Source: Table 1 · p. 8, §3.2 · p. 8

3.3 ABLATIONS AND ANALYSIS

  • Increasing RL training tasks from 1,000 to 8,000 yields consistent, substantial gains on FrontierSWE and Terminal Bench 2.0.The analysis presents environment scaling as important for general ultra-long-horizon execution capability.Source: §3.3 · p. 8, Figure 3 · p. 9
  • RL reward increases consistently during training while policy entropy remains stable.The authors interpret this pattern as effective optimization without excessive exploration or policy collapse.Source: §3.3 · p. 8, Figure 4 · p. 9
  • Marathoner can operate for several hours and issue hundreds of tool calls on challenging tasks, resembling behaviors of strong proprietary models.The execution-process statistics cover FrontierSWE and Terminal Bench 2.0.Source: §3.3 · pp. 8–9, Table 2 · p. 9
  • Adding any single proposed design improves performance over the vanilla model trained with ordinary RFT and RL.The ablated designs are Multi-Task Chaining, Diverse-Harness Trajectory Generation, and Later Stage Bonus Reward.Source: Table 3 · p. 9, §3.3 · p. 9

4 CONCLUSION

Marathoner combines task synthesis, rejection-sampled finetuning, and reinforcement learning to train ultra-long-horizon execution, with Later Stage Bonus Reward encouraging valuable operations late in execution.

4 CONCLUSION

  • Marathoner combines Ultra-Long-Horizon Task Synthesis, rejection sampling finetuning, and reinforcement learning to train an agent for ultra-long-horizon execution.The pipeline also uses Multi-Task Chaining to construct frontier-level tasks.Source: §4 · p. 9
  • Later Stage Bonus Reward encourages the agent to continue performing valuable operations during later stages of execution.Source: §4 · p. 9
  • Across diverse benchmarks, Marathoner achieves strong performance, and further analysis reports 10+ hours of execution and 1000+ tool calls on challenging tasks.Source: §4 · p. 9

A RELATED WORK

Related work spans proprietary long-horizon models, coding-agent paradigms, and process-level rewards; Marathoner focuses on training open-source coding agents for sustained execution.

Ultra-Long-Horizon Capability

  • Proprietary models are described as exhibiting ultra-long-horizon behavior, while Marathoner proposes post-training to instill this capability in open-source models.Source: A RELATED WORK · p. 13

Autonomous Coding Agent

  • Coding agents include interactive assistants, autonomous task-oriented agents, and planning-centric agents, whereas Marathoner targets ultra-long-horizon execution.Source: A RELATED WORK · p. 13

Process-Level Reward

  • Process-level reward methods include discriminative, generative, and rubric-based approaches; Marathoner introduces Later Stage Bonus Reward for ultra-long-horizon reinforcement learning.Source: A RELATED WORK · p. 13

B.1 ULTRA-LONG-HORIZON TASK SYNTHESIS

Ultra-long-horizon tasks are synthesized from diverse GitHub repositories and major release pull requests, then made more difficult by chaining multiple tasks.

Repository Collection

  • The synthesis pool draws on 10,000 GitHub repositories across domains, including frontend, backend, low-level, mature, trending, and LLM-related codebases.Source: §B.1 · p. 13

Major Release PR Mining

  • The pipeline mines 100,000 major release PRs and assigns task difficulty by new-code volume, with Easy:Medium:Hard tasks distributed at 2:3:5.Easy tasks use PRs adding 100–200 lines, Medium 200–1,000 lines, and Hard more than 1,000 lines.Source: §B.1 · p. 14

Task Construction

  • Tasks use Harbor specifications, clean Ubuntu 24.04 sandboxes, pre-PR repository commits, PR comments or fallback instructions, and test-based reward verifiers.Execution limits are 5, 10, and 20 hours for Easy, Medium, and Hard tasks, respectively.Source: §B.1 · p. 14

Multi-Task Chaining

  • Multi-Task Chaining combines five synthesized tasks into a Frontier task, joining their repositories, instructions, and verifiers under a 40-hour limit.The resulting tasks require coordination across multiple large repositories.Source: §B.1 · p. 14

B.2 REJECTION SAMPLING FINETUNING

Rejection-sampling finetuning uses teacher-generated trajectories from multiple harnesses, retains successful runs, and fine-tunes the base model on them.

Trajectory Generation

  • Kimi K3 generated trajectories for 53,000 selected tasks using Claude Code, Codex, and OpenClaw, with Harbor rollouts in network-enabled sandboxes.The set comprised 50,000 sampled source tasks and 3,000 chained tasks.Source: §B.2 · p. 14

Rejection Sampling

  • A trajectory was retained only if its execution passed every unit test in the task reward verifier.Successful retained trajectories numbered 14,159 for Claude Code, 13,583 for Codex, and 13,078 for OpenClaw.Source: §B.2 · p. 14

Supervised Finetuning

  • The base model Qwen3.5-9B was fine-tuned on 40,820 successful trajectories using full-parameter supervised finetuning.Training used a 256k-token sequence length for five epochs.Source: §B.2 · p. 15

B.3 RL TRAINING DETAILS

RL training uses AReaL and Harbor to run agent rollouts through harnesses in independent sandboxes. The training set combines directly synthesized and chained tasks, while a trajectory-based bonus rewards valuable later-stage operations.

B.3 RL TRAINING DETAILS

  • RL training uses AReaL with Harbor to manage sandbox creation, harness installation, agent rollouts, and reward calculation.AReaL decouples rollout from policy updates; the setup uses 32 H800 GPUs.Source: §B.3 · p. 15
  • The 8,000 RL tasks comprise 5,000 directly sampled tasks and 3,000 tasks formed by chaining tasks from a separately sampled pool.The direct tasks do not overlap with RFT data; the chained tasks come from 15,000 independently sampled tasks.Source: §B.3 · p. 15
  • For Later Stage Bonus Reward, Qwen-3.8-Max summarizes trajectories and assigns binary flags to execution phases.Training uses batch size 32, eight rollouts per task, learning rate 1e-6, temperature 0.6, and top_p 0.95.Source: §B.3 · p. 15

B.4 BASELINES

The baseline comparison includes proprietary agentic models and open-source Qwen models. The proprietary set spans models described as capable of ultra-long-horizon execution and models with broader agentic or multimodal strengths.

B.4 BASELINES

  • Proprietary baselines include GLM-5.2, Qwen-3.7-Max, Kimi K3, GPT-6-Astra, Claude Fable 5.1, and Gemini-3.1-Pro.The descriptions emphasize ultra-long-horizon execution, agentic capability, professional-domain performance, coding, and multimodal capability.Source: §B.4 · p. 15
  • Kimi K3 is described as a 2.8T-parameter agentic model with exceptional ultra-long-horizon autonomous execution capability.The passage identifies it as Moonshot's flagship model.Source: §B.4 · p. 15
  • The open-source comparison includes models from the Qwen3.5 and Qwen3.6 series.Source: §B.4 · p. 15

C MORE ABLATIONS

The ablations examine how task-chain size and Later Stage Bonus Reward design affect performance, and compare models across training stages. The accompanying prompts define how trajectories are divided into phases and how valuable operations are judged.

C MORE ABLATIONS

  • Chaining 5 tasks yields the best performance; 2 or 3 may under-challenge, while 7 or 9 create tasks that are too large and diverge from mainstream benchmark distributions.Table 4 describes the direct-task count, multi-task-chain count, and number of atomic tasks per chain.Source: Table 4 · p. 15, C MORE ABLATIONS · p. 16
  • A 0.5 bonus for valuable operations in the 50%–100% execution stage yields the best performance and encourages meaningful operations throughout execution.A 0.2 bonus offers limited reinforcement, while 0.8 may lead some rollouts to underemphasize final task correctness.Source: Table 5 · p. 16, C MORE ABLATIONS · p. 16
  • Marathoner-9B-RFT improves over the raw base model, and reinforcement learning brings further performance gains.The authors describe RL practice in sandboxes as producing stronger and more robust ultra-long-horizon ability.Source: Table 6 · p. 16, C MORE ABLATIONS · p. 16

D PROMPT

  • The trajectory prompt groups consecutive actions by shared intermediate objective, starts a new phase when the objective or strategy changes, and preserves chronological order.It asks for both phase content and concrete outcomes, including unresolved failures rather than inferred achievements.Source: D PROMPT · pp. 16–17
  • The phase-judging prompt assigns a positive flag only when a non-routine operation makes substantial progress and its importance is supported by the outcome.Ambiguous evidence receives 0, and lateness in the trajectory alone is not a reason to reward a phase.Source: D PROMPT · pp. 17–18

E SYNTHESIZED TASKS

The synthesized task examples specify repository changes as detailed software tasks. They cover inference-framework integration, reusable caching for LLM wrappers, and terminal operations for autonomous agents.

E SYNTHESIZED TASKS

  • The LightLLM task replaces existing VLLM kernel integration with modular SGL and VLLM utilities and adds quantization and MoE operations.The task calls for dependency checks, error handling, and logging.Source: E SYNTHESIZED TASKS · pp. 18–19
  • The GPTCache task introduces BaseCacheLLM so custom LLM wrappers can route calls through caching while inheriting configured cache parameters.It specifies handling for an optional wrapped LLM and an actionable error when no cache object is supplied.Source: E SYNTHESIZED TASKS · p. 19
  • The CAMEL task adds a TerminalToolkit for shell execution and environment management within a configured working directory.It specifies environment setup, output queues, optional terminal display, and blocking or non-blocking command execution.Source: E SYNTHESIZED TASKS · pp. 19–20

F CASE STUDY

The case study documents Marathoner’s sustained coding trajectory on a synthesized hard Qiskit task, implementing native Rust control-flow representations while preserving the existing Python API. It also describes a width-check fix for DAGCircuitErrors and reports that native control-flow implementation failures were not among the remaining failures.

F CASE STUDY

  • 11.8 hours, 872 environment-interaction steps, and 1,273 tool calls characterize Marathoner’s trajectory on the synthesized hard task.The repository task began in a clean Ubuntu 24.04 sandbox without a preinstalled toolchain or project dependencies.Source: F CASE STUDY · p. 20
  • The task aimed to move control-flow operations and their block bodies into native Rust CircuitData structures while keeping the Python API backward-compatible.The existing Python Instruction representation prevented Rust transpiler passes from manipulating control flow and forced Python round-trips; the proposed representation would let Rust passes recurse into blocks.Source: F CASE STUDY · pp. 20–21
  • An initial smoke test failed during if_test construction with a CircuitError stating that the if block was not the most recent instruction in the circuit.The trajectory later investigated control-flow operation caching and pickle round-trips, but the supplied passages do not establish the outcome of that smoke-test failure.Source: F CASE STUDY · p. 26
  • 126 control-flow tests passed after a build with the cache_pygates feature enabled.The build succeeded, and the reported test run completed with four warnings.Source: F CASE STUDY · pp. 25, 27
  • The width-check fix resolved 51 DAGCircuitError failures across 20+ transpiler suites; the sweep then had 689 failures and 2,593 passes, down from 740 failures.Source: F CASE STUDY · p. 29
  • Reported results included 154/154 for test_twirling.py + test_control_flow.py, 30/30 for TestSabreSwapControlFlow, 221/221 for test_dagcircuit.py + test_compose.py, and all test_control_flow_builders tests passing.High-level synthesis had 79/165 passing, with 86 plugin-discovery failures and 0 CF-caused.Source: F CASE STUDY · p. 29
  • The remaining failures were classified as pre-existing or environmental, with 0 failures attributable to the native control-flow implementation.The summary identifies about 644 plugin-discovery errors and about 14 Param::Int strict-equality failures among them.Source: F CASE STUDY · pp. 29–30
  • The two width-check sites now use each node’s wire span from the interners, the authoritative width source for native control-flow operations.Those operations report a width of 0 from num_qubits/num_clbits because their width is a property of their blocks.Source: F CASE STUDY · pp. 28, 30

Related papers

Loading 2609.34378v1…