Source-linked AI summary
PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents
Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, Chengyue Jiang
TL;DR
Long-horizon agents produce experience that post-hoc methods cannot use to redirect the active run, while existing architectures separate execution and diagnosis incompletely. PILOT addresses this with a supervisor–worker harness coupling live steering and live self-evolution. Across two frozen backbones and three benchmarks, it ranks first in five of six configurations and reports gains in pass rate and token efficiency, within an evaluation limited to three benchmarks and two open-weight backbones.
Problem
Post-hoc self-improvement cannot redirect active runs, while single-agent correction entangles execution with diagnosis and delegation usually cannot steer active subagents.
Method
PILOT uses a separate supervisor to steer or abort active workers and distil procedures, conventions, and failure modes into persistent skills and memory.
Results
PILOT ranks first in five of six backbone–benchmark combinations across two frozen backbones and three benchmarks, with self-improvement pass-rate gains of 14.6 points for GLM-5.1 and 12.4 points for Kimi-K2.6.
Takeaways & Limitations
PILOT’s results support combining live correction and experience accumulation in one continuous self-improvement loop.
Takeaways & Limitations
The evaluation covers only three benchmarks and two open-weight backbones, and same-backbone supervisor–worker pairings leave heterogeneous trade-offs unexplored.
Abstract
from arXiv · showhide
Long-horizon agent runs generate experience that can improve both the current run and future work. Most self-improvement methods process this experience only after execution ends, so they cannot redirect the active run or immediately apply and validate lessons learned from it. We argue that self-improvement should instead be live, using emerging experience both to redirect the active run and to update the persistent harness. Existing agent architectures do not fully support this goal. Single-agent self-correction combines task execution and trajectory assessment within one context, while subagent delegation separates execution but typically cannot redirect an active subagent. We present PILOT, a supervisor-worker harness for live self-improvement through two coupled mechanisms: (1) live steering lets a separate supervisor redirect or abort the active worker during execution; and (2) live self-evolution distils procedures and failure modes revealed during execution into reusable skills and memory. Across two frozen backbones and three benchmarks, PILOT ranks first in five of six configurations. On Terminal-Bench 2.0, PILOT outperforms counterpart harnesses by up to 9.8 percentage points. In the self-improvement setting, PILOT gains 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6. Mean output tokens fall by 42.9% and 47.4%, while successful evaluations per million output tokens rise by 110.3% and 134.0%, respectively.
1 Introduction
PILOT addresses the gap between post-hoc self-improvement and live correction by coupling a supervisor that can redirect active workers with harness updates derived during execution. Across two frozen backbones and three benchmarks, it performs strongly while improving self-improvement and efficiency metrics.
- Motivation: Post-hoc reflection, evaluation, and harness updates can improve future work but cannot redirect an active run.Single-agent self-correction is timely but shares execution and diagnosis in one context, while delegation usually returns only after completion.
- Approach: PILOT couples live steering, which redirects or aborts active workers, with live self-evolution, which distils procedures and failure modes into reusable skills and memory.A separate supervisor remains connected during execution and updates the persistent harness.
- Evaluation: PILOT ranks first in five of six backbone–benchmark combinations across two frozen backbones and three benchmarks.The evaluation includes one-shot and self-improvement settings.
- Evaluation: 9.8 percentage points is PILOT’s maximum advantage over counterpart harnesses on Terminal-Bench 2.0.This is reported for the one-shot comparison.
- Self-improvement results: 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6 are PILOT’s best observed self-improvement pass-rate gains.Mean output tokens also fall by 42.9% and 47.4%, while successful evaluations per million output tokens rise by 110.3% and 134.0%, respectively.
2 PILOT: A Live Self-Improvement Loop
PILOT implements live self-improvement as two coupled loops in a supervisor–worker harness. The supervisor steers active workers through a live channel and records reusable knowledge so later workers load an evolved harness.
- Roles and state: PILOT separates task execution and self-improvement between workers and a supervisor while treating improvement as a closed loop.The persistent harness contains skills and memory that can evolve during and across episodes.
- Roles and state: Workers run in isolated contexts, while the supervisor reads relevant trajectory portions for diagnosis and preserves context for goals and recurring failure patterns.Model parameters remain frozen; self-improvement changes the persistent harness rather than the model.
- Live steering: A two-way live channel carries worker notifications, questions, and results to the supervisor and Steer or Abort actions back to workers.Steering guidance is queued for the worker’s next turn, while abort interrupts an unproductive active session.
- Live self-evolution: Live self-evolution records successful procedures, conventions, and recurring failure modes in the skill library or memory, producing an evolved harness H′ for later workers.Workers spawned after the update load H′ and re-enter both live steering and live self-evolution loops.
- Implementation: PILOT extends the Pi coding-agent runtime with a supervisor session and in-process worker sessions using the same frozen model for both roles.This implementation follows a real-world usage scenario.
3 Experiments
PILOT is evaluated across three long-horizon benchmarks, two frozen backbones, and one-shot and self-improvement settings. It ranks first in five of six backbone–benchmark combinations and improves both immediate task performance and later-task reuse.
- Setup: The evaluation covers Terminal-Bench 2.0, SWE-bench Multilingual, and SWE-bench Pro using frozen GLM-5.1 and Kimi-K2.6 backbones.The same backbone drives the supervisor and worker, while harness comparisons hold the backbone fixed.
- Evaluation settings: The one-shot setting starts every task from a fresh harness state to isolate whether live steering keeps the worker on track.The self-improvement setting instead reuses a shared harness state across Terminal-Bench 2.0 iterations without benchmark feedback during execution.
- One-shot results: 71.6% average pass rate on Terminal-Bench 2.0 exceeds Pi’s 66.3% by 5.3 percentage points.PILOT reaches 71.9% with GLM-5.1 and 71.3% with Kimi-K2.6.
- One-shot results: PILOT ranks first in five of six backbone–benchmark combinations across the three benchmarks.With GLM-5.1 it ranks first on all three benchmarks; with Kimi-K2.6 it ranks first on Terminal-Bench 2.0 and SWE-bench Pro and second on SWE-bench Multilingual.
- Self-improvement results: 14.6 points on GLM-5.1 and 12.4 points on Kimi-K2.6 are the best-observed self-improvement gains from iteration 0.The corresponding pass rates rise to 80.9% for both backbones.
- Self-improvement results: PILOT improves more from accumulated experience than OpenCode and Pi, gaining 14.6, 7.9, and 2.3 points, respectively.This comparison uses the same frozen GLM-5.1 backbone and the same initial skill library.
4 Analysis of Live Self-Improvement
The analysis links PILOT’s gains to broader task coverage, expanding reusable skills, lower token use, and live corrections during difficult runs. Live steering provides concrete mid-run recovery, while live self-evolution improves later execution efficiency.
- Live self-evolution: GLM-5.1 gains 2 Easy, 6 Medium, and 8 Hard passes, while Kimi-K2.6 gains 1, 7, and 12 additional passes.The largest gains occur on Hard tasks, which more often require specific procedures, recovery strategies, and tool-use patterns.
- Live self-evolution: GLM-5.1 skills grow from 62 to 83, while Kimi-K2.6 skills grow from 50 to 81 across iterations.The expanding skill libraries convert execution experience into reusable procedures rather than treating runs as isolated tasks.
- Token efficiency: Successful evaluations per million output tokens rise 110.3% on GLM-5.1 and 134.0% on Kimi-K2.6 relative to iteration 0.The figure reports both per-iteration values and the best value observed at or before each iteration.
- Live steering: Live steering is classified as aiding a successful run only when a supervisor identifies a concrete problem, the worker follows correction, and the task succeeds.The analysis groups these aided runs by task difficulty and backbone.
- Live steering: Figure 4 shows successful corrections of an unproductive strategy and an implementation error, both followed by worker completion.The worker adopts the supervisor’s correction in each representative case.
- Live steering: Separating execution from oversight lets the worker focus on tools and implementation while the supervisor monitors deviations from the plan.This division allows detection of stalled or incorrect branches before the run is lost.
5 Related Work
Related work spans self-correction, delegation, harness evolution, reusable skills, memory, and long-horizon benchmarks. These systems increasingly adapt agent behavior or persistent context from execution experience and feedback.
- Agent systems and self-evolving agents: Single-agent systems such as ReAct, Self-Refine, and CRITIC improve execution through self-correction, while AutoGen and related systems use delegated roles.Other approaches search over agent programs, evolve context playbooks, synthesize harnesses, or optimize broader harness components.
- Agent systems and self-evolving agents: EvoSkill, Memento-Skills, Mem2Evolve, Continual Harness, and Self-Harness refine skills, memory, prompts, subagents, or harness edits from experience.The related methods differ in whether they evolve reusable assets, agent populations, or broader persistent harness state.
- Long-horizon agent tasks: WebArena, OSWorld, SWE-bench, and Terminal-Bench 2.0 represent a shift toward longer sequences of interdependent actions in stateful environments.Terminal-Bench 2.0 focuses on multi-step system and engineering work through an interactive shell.
6 Conclusion
PILOT is a supervisor–worker harness for live self-improvement, combining live steering with live self-evolution. Across two frozen backbones and three benchmarks, it ranks first in five of six combinations, while iterative use improves performance and token efficiency.
- PILOT implements live self-improvement through live steering and live self-evolution over reusable harness knowledge.Live steering redirects an active worker, while live self-evolution turns observed procedures and failure modes into reusable knowledge.
- PILOT ranks first in five of six backbone–benchmark combinations across two frozen backbones and three benchmarks.
- 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6 are the best observed Terminal-Bench 2.0 pass-rate increases in self-improvement.
- Mean output tokens per evaluated task fall by 42.9% with GLM-5.1 and 47.4% with Kimi-K2.6 in the self-improvement setting.
- The evaluation is limited to three benchmarks and two open-weight backbones because iterative self-improvement is substantially more expensive than a single inference run.Heterogeneous supervisor–worker backbone pairings and their trade-offs remain unexplored.
Full Author List
The paper’s author list includes Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, and Chengyue Jiang.
- Authors: Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Wen Da, Zhaokai Luo, Mu Chuan, Yao Hu, Wenjie Li, and Chengyue Jiang.
A.1 Excluded SWE-bench tasks
The evaluation excludes SWE-bench Multilingual and SWE-bench Pro tasks whose pinned environments cannot bootstrap the JavaScript runtimes required by the evaluated harnesses.
- 43 SWE-bench Multilingual tasks and 198 SWE-bench Pro tasks are excluded because their pinned environments cannot bootstrap the required JavaScript agent runtimes.SWE-bench Pro exclusions also include task images with Node.js versions below the runtimes’ minimum requirement.
- The same exclusions are applied to every harness.
- The complete excluded-task lists will be released in the project GitHub repository for reproducibility.
A.2 PILOT pseudocode
Algorithm 1 describes PILOT’s coupled live-steering and live-self-evolution loops across worker sessions. The supervisor coordinates worker events, can redirect or abort workers, and updates the persistent harness when reusable knowledge is found.
- Worker coordination: Algorithm 1 coordinates multiple worker sessions, which may run concurrently or be spawned later in the same episode.Every event and action is associated with a worker index j.
- Channel operations: Workers emit Notification, Question, and Result events, while the supervisor can issue Steer and Abort actions.
- Supervisor loop: The supervisor receives events from active workers, replies to questions, and queues guidance when redirection is warranted.
- Supervisor loop: The supervisor aborts a worker when continuing it is no longer useful.
- Harness evolution: When reusable knowledge is found in a worker trajectory, PILOT updates the harness before later worker sessions continue.
A.3 Self-improvement instruction
The self-improvement setting gives every task a shared instruction that points to harness-specific skill and memory paths. These paths provide reusable skills and persistent notes that grow through newly saved experience.
- Every evaluated harness receives the same instruction, with substitutions for its native skill and memory paths.
- PILOT uses ~/.pilot/skills for skills and ~/.pi/agent/AGENTS.md for memory.
- Pi uses ~/.pi/agent/skills for skills and ~/.pi/agent/AGENTS.md for memory.
- OpenCode uses ~/.config/opencode/skills for skills and ~/.config/opencode/AGENTS.md for memory.
- The reusable library and long-term notes are available immediately, persist when updated, and let later similar tasks replay prior approaches.