PaperlayerView exact revision on arXiv

Source-linked AI summary

PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents

Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, Chengyue Jiang

arXiv:2608.26530v1cs.AI

TL;DR

Long-horizon agents often use experience only after execution, while existing architectures do not jointly support live correction and a dedicated self-improvement role. PILOT pairs a supervisor and worker for live steering and persistent harness updates, and ranks first in five of six configurations across two frozen backbones and three benchmarks.

  • Problem

    Most self-improvement methods use experience only after execution, and existing architectures do not jointly support live correction and a dedicated self-improvement role.

    Source: §1 · p. 2

  • Method

    PILOT uses a supervisor–worker harness to redirect active workers during execution and distil observed procedures and failure modes into reusable skills and memory.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Across two frozen backbones and three benchmarks, PILOT ranks first in five of six combinations; its best observed Terminal-Bench 2.0 pass rate rises by 14.6 percentage points with GLM-5.1.

    Source: §1 · p. 2, §6 · p. 9

  • Takeaways & Limitations

    The reported results support treating correction of the current trajectory and accumulation of reusable knowledge as one continuous self-improvement loop.

    Source: §6 · p. 9

    The evaluation is limited to three benchmarks and two open-weight backbones, and does not explore heterogeneous supervisor–worker pairings.

    Source: §6 · p. 9

Abstract

from arXiv · show

Long-horizon agent runs generate experience that can improve both the current run and future work. Most self-improvement methods process this experience only after execution ends, so they cannot redirect the active run or immediately apply and validate lessons learned from it. We argue that self-improvement should instead be live, using emerging experience both to redirect the active run and to update the persistent harness. Existing agent architectures do not fully support this goal. Single-agent self-correction combines task execution and trajectory assessment within one context, while subagent delegation separates execution but typically cannot redirect an active subagent. We present PILOT, a supervisor-worker harness for live self-improvement through two coupled mechanisms: (1) live steering lets a separate supervisor redirect or abort the active worker during execution; and (2) live self-evolution distils procedures and failure modes revealed during execution into reusable skills and memory. Across two frozen backbones and three benchmarks, PILOT ranks first in five of six configurations. On Terminal-Bench 2.0, PILOT outperforms counterpart harnesses by up to 9.8 percentage points. In the self-improvement setting, PILOT gains 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6. Mean output tokens fall by 42.9% and 47.4%, while successful evaluations per million output tokens rise by 110.3% and 134.0%, respectively.

1 Introduction

PILOT proposes live self-improvement: a separate supervisor uses emerging experience to redirect the active run and update the persistent harness. Across two frozen backbones and three benchmarks, it combines live steering with live self-evolution.

1 Introduction

  • Post-hoc reflection, judging, and harness updates can preserve lessons from completed traces but cannot redirect the run that produced them or immediately validate those lessons [23] [36] [17] [15].PILOT instead uses emerging experience both to correct the current run and to improve the persistent harness for later runs.Source: §1 · p. 2
  • Single-agent self-correction makes the task executor diagnose its own mistakes in a context crowded with execution details, while subagent delegation usually cannot redirect a worker before it returns [30] [18] [8] [16].PILOT separates task execution from self-improvement so the supervisor can stay focused on the goal and signs that the run is going off track.Source: §1 · p. 2
  • PILOT pairs live steering, which lets a supervisor redirect or abort an active worker, with live self-evolution, which distils procedures and failure modes into reusable skills and memory.The supervisor remains connected throughout execution, receives questions, notifications, errors, and inactivity alerts, and can record reusable knowledge without taking task execution from the worker.Source: §1 · p. 2, Figure 2 · p. 3
  • PILOT ranks first in five of six backbone–benchmark combinations and beats counterpart harnesses on Terminal-Bench 2.0 by up to 9.8 percentage points.In self-improvement, its best observed pass rate rises by 14.6 percentage points with GLM-5.1 and 12.4 points with Kimi-K2.6; mean output tokens per evaluated task fall by 42.9% and 47.4%, while successful evaluations per million output tokens rise by 110.3% and 134.0%, respectively. Across iterations, the reusable skill set grows by 21 and 31 skills, respectively.Source: §1 · p. 2
  • Two case studies trace the supervisor redirecting an active strategy or correcting an implementation error while the worker remains responsible for the solution.The evaluation also includes trajectory and efficiency analyses of task recovery, skill growth, and token efficiency.Source: §1 · p. 2

2 PILOT: A Live Self-Improvement Loop

PILOT makes self-improvement a live closed loop: a supervisor redirects active workers and records reusable knowledge in a persistent harness, which later workers load. Model parameters remain frozen; improvement changes the harness, not the model.

Live self-improvement loop

  • PILOT couples live steering of active workers with live self-evolution of the persistent harness, and later workers load the updated harness into the same loop.The supervisor records reusable procedures or failure modes in the skill library or memory; workers spawned after an update load the evolved harness within the same or a later episode.Source: §2 · pp. 3–4

Roles and persistent state

  • With frozen model parameters, workers execute long-horizon tasks in isolated contexts while the supervisor selectively inspects trajectories and maintains persistent skills and memory across episodes.Workers may be spawned concurrently or as earlier workers settle; their explorations, dead ends, and verbose tool outputs remain isolated by default.Source: §2 · p. 3

Live steering through a two-way channel

  • A two-way channel supports worker notifications, questions, and final results, plus supervisor guidance queued for the worker’s next turn or interruption of an active worker.Workers continue after sending notifications, pause for supervisory replies to questions, and finish their current turn before queued guidance takes effect.Source: §2 · pp. 3–4

Implementation

  • PILOT extends the Pi coding-agent runtime, with the supervisor and in-process workers running as separate agent sessions that use the same frozen model in experiments.The experiments follow a real-world usage scenario in which that model serves as both supervisor and worker.Source: §2 · p. 4

3 Experiments

Across three long-horizon benchmarks and two frozen backbones, the experiments evaluate PILOT’s live steering in one-shot runs and its live self-evolution across iterations. PILOT leads in five of six backbone–benchmark combinations and improves across self-improvement iterations.

3.1 Setup

  • The evaluation covers Terminal-Bench 2.0, SWE-bench Multilingual, and SWE-bench Pro, using the same frozen GLM-5.1 or Kimi-K2.6 backbone across harnesses.The benchmarks span terminal operation and multilingual or harder repository-level repair; holding the backbone fixed isolates harness effects.Source: §3.1 · p. 4
  • In the self-improvement setting, each iteration starts from shared harness state; runs create or revise skills and memory from live trajectories, and only successful-run updates carry forward.Updates occur before verifier outcomes are available, and those outcomes select which updates are retained rather than creating or modifying them.Source: §3.1 · p. 4

3.2 Live steering keeps long-horizon work on track

  • PILOT ranks first in five of six backbone–benchmark combinations, averaging 71.6% on Terminal-Bench 2.0 and 59.9% on SWE-bench Pro.These averages exceed the strongest average single-agent baseline, Pi, by 5.3 and 4.4 percentage points, respectively.Source: Table 1 · p. 5, §3.2 · p. 6

3.3 Live self-evolution makes supervision reusable across tasks

  • 14.6 points on GLM-5.1 and 12.4 points on Kimi-K2.6: PILOT’s best observed pass rates rise across iterations from 66.3% to 80.9% and 68.5% to 80.9%, respectively.The harness accumulates reusable skills and memory across iterations.Source: §3.3 · p. 6
  • 14.6 points: PILOT’s improvement from iteration 0 to its best observed GLM-5.1 result exceeds OpenCode’s 7.9 points and Pi’s 2.3 points.PILOT, Pi, and OpenCode share the same frozen backbone and initial skill library, with identical user input for every task.Source: §3.3 · p. 6

4 Analysis of Live Self-Improvement

Live self-evolution turns execution experience into broader task coverage and reusable skills while improving token efficiency. Live steering supplies corrections during execution, with successful steering-aided runs concentrated on harder tasks.

4.1 Live self-evolution makes accumulated experience more useful

  • Output tokens per evaluated task fall 42.9% (28.5K to 16.3K) for GLM-5.1 and 47.4% (41.9K to 22.1K) for Kimi-K2.6; best successful evaluations per million output tokens rise 110.3% and 134.0%, respectively.Token counts include all supervisor and worker assistant turns; the mean is averaged across five-iteration windows. Efficiency improvements are relative to iteration 0.Source: §4.1 · pp. 6–7, Figure 3 · p. 7
  • At their first maximum, GLM-5.1 gains 2 Easy, 6 Medium, and 8 Hard passes at iteration 14; Kimi-K2.6 gains 1, 7, and 12 at iteration 13.The largest gains occur on Hard tasks, where reusable procedures, recovery strategies, and tool-use patterns reduce repeated exploration and improve later attempts.Source: §4.1 · p. 6, Figure 3 · p. 7
  • Reusable skills grow from 62 to 83 for GLM-5.1 and from 50 to 81 for Kimi-K2.6 as PILOT accumulates execution experience.The skill counts represent distinct skills available to PILOT at each iteration.Source: §4.1 · p. 6, Figure 3 · p. 7

4.2 Live steering concentrates on harder tasks

  • Among successful runs, steering-aided shares rise from Medium to Hard tasks from 1.1% to 6.1% for GLM-5.1 and from 8.1% to 19.7% for Kimi-K2.6.No successful Easy run was classified as aided; overall shares were 2.3% for GLM-5.1 and 10.6% for Kimi-K2.6. Classification required a concrete supervisor correction followed by successful completion along the corrected path, excluding ignored, stale, or redundant interventions.Source: Table 2 · p. 7, §4.2 · pp. 7–8
  • Representative successful corrections redirect an unproductive strategy or fix an implementation error, with the worker adopting the supervisor’s direction.The worker focuses on tool use and implementation while the supervisor tracks the goal and deviations, enabling timely recovery while leaving the final solution to the worker.Source: Figure 4 · p. 8, §4.2 · p. 8

5 Related Work

Related work spans agent execution and improvement, alongside benchmarks for increasingly long, stateful tasks. PILOT is positioned as a runtime layer linking live recovery with reusable learning.

Agent systems and self-evolving agents

  • Prior agent improvement spans single-agent self-correction, delegated roles, post-hoc memories, and persistent evolution of agent components and harnesses.Examples include ADAS and ACE [11] [35], Group-Evolving Agents [26], EvoSkill and Memento-Skills [1], Mem2Evolve, Continual Harness [13], and Self-Harness [34].Source: §5 · pp. 8–9
  • Unlike methods that improve contexts, agents, or harnesses across completed work, PILOT steers the active worker and records reusable harness knowledge during supervision.Its live steering redirects or aborts the worker during execution; live self-evolution distils procedures and failure modes into reusable skills and memory.Source: §5 · p. 9

Long-horizon agent tasks

  • Long-horizon benchmarks increasingly test interdependent work in stateful environments, from shell-based engineering tasks to workflows lasting hours and ultra-long interactions.LongCLI-Bench reports state-of-the-art agents pass fewer than 20% of tasks, OSWorld 2.0 workflows take people a median of 1.6 hours, and EdgeBench tasks sustain at least 12 hours of interaction [6] [39].Source: §5 · p. 9

6 Conclusion

PILOT couples live steering of an active worker with live evolution of reusable harness knowledge. Its conclusion reports benchmark results while noting limits in evaluation coverage and supervisor–worker pairings.

6 Conclusion

  • Across two frozen backbones and three benchmarks, PILOT ranks first in five of six backbone–benchmark combinations; its best observed Terminal-Bench 2.0 pass rate rises 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6.Mean output tokens per evaluated task fall by 42.9% with GLM-5.1 and 47.4% with Kimi-K2.6, while the reusable skill set grows.Source: §6 · p. 9
  • Live steering lets a separate supervisor redirect an active worker during execution, while live self-evolution turns observed procedures and failure modes into reusable harness knowledge.Together, these mechanisms let supervision recover the current trajectory and update the persistent harness for later agents.Source: §6 · p. 9
  • The evaluation is limited to three benchmarks and two open-weight backbones because iterative self-improvement repeats tasks across many iterations, and heterogeneous supervisor–worker pairings remain unexplored.Broader coverage and proprietary models are left for future work; heterogeneous pairings may trade off oversight quality, task performance, and cost.Source: §6 · p. 9

A Appendix · A.1 Excluded SWE-bench tasks

The evaluation excludes SWE-bench tasks whose pinned environments cannot support the JavaScript runtimes required by the harnesses, applying the exclusions uniformly and planning to release the full lists.

A.1 Excluded SWE-bench tasks

  • 43 SWE-bench Multilingual tasks are excluded because their pinned environments cannot bootstrap the evaluated harnesses’ JavaScript agent runtimes.Source: §A.1 · p. 13
  • 198 SWE-bench Pro tasks are excluded for the same runtime-bootstrapping constraint.Source: §A.1 · p. 13
  • SWE-bench Pro exclusions also include task images with Node.js versions below the runtimes’ minimum requirement.Source: §A.1 · p. 13
  • The same task exclusions are applied to every harness.Source: §A.1 · p. 13
  • The complete excluded-task lists will be released in the project GitHub repository for reproducibility.Source: §A.1 · p. 13

A.2 PILOT pseudocode

Algorithm 1 coordinates multiple worker sessions through coupled supervisor and worker loops. Workers report progress, ask for input, and return results; the supervisor can steer or abort them and update the harness with reusable knowledge.

A.2 PILOT pseudocode

  • One supervisor session coordinates concurrently running or later-spawned workers, with events and actions associated with each worker’s index.Workers emit Notification, Question, and Result events; the supervisor issues Steer and Abort actions.Source: §A.2 · p. 13
  • The supervisor answers worker questions, steers when redirection is warranted, aborts workers when continuing is no longer useful, and updates the harness with reusable knowledge.It inspects the relevant trajectory before steering and settles workers after a result, error, or abort.Source: §A.2 · p. 13
  • Each worker incorporates queued supervisor guidance while acting, can notify the supervisor of progress or risk, and can pause to ask for input before resuming.On completion, the runtime sends the worker’s result to the supervisor.Source: §A.2 · p. 13

A.3 Self-improvement instruction

The self-improvement setting gives every evaluated harness the same instruction, with only the harness-native skill and memory paths substituted. It tells agents to use persistent prior experience and preserve useful task-specific procedures for future runs.

A.3 Self-improvement instruction

  • Every harness in the self-improvement setting receives the same instruction, with only its native skill and memory paths differing.PILOT uses ~/.pilot/skills and ~/.pi/agent/AGENTS.md; Pi uses ~/.pi/agent/skills and that same memory path; OpenCode uses ~/.config/opencode/skills and ~/.config/opencode/AGENTS.md.Source: §A.3 · p. 14
  • For a task that took real work, save exactly one skill named exactly after the task, recording the tools, libraries, commands, setup, and any skills reused.Use the test of whether replaying the approach would save effort on a similar task, and err on the side of saving.Source: §A.3 · p. 14
  • Check for an existing task skill first; update it only when the run finds a genuinely better or more reliable approach, and use AGENTS.md for cross-task lessons.The skill library and long-term notes persist, allowing later similar tasks to replay prior approaches.Source: §A.3 · p. 14

Related papers

Loading 2608.26530v1…