Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite

1 Introduction

• Pooling trajectories from different harnesses can mix controller interventions, prompts, workflow logic, and stopping rules that may not transfer to a general harness.RSR aims to preserve useful planning, verification, recovery, and continuation behaviors while rewriting trajectories for that deployment setting.Source: §1 · p. 2
• RSR uses distinct harnesses to collect diverse successful solutions, then rewrites them with a planner, critic, and executor under the general Terminus 2 harness.The planner produces runbooks, the critic screens for verifier and solution leakage, and the executor follows accepted runbooks in fresh sandboxes; only passing trajectories are retained.Source: §1 · p. 2
• RSR improves over Direct SFT by 20.8, 4.1, 4.6, 7.0, and 3.0 percentage points in pass@3 on TB2, TB3, TB4, TBH, and SWR100, respectively.Mean per-run pass rates also improve on all five benchmarks; process reward on LHTB rises from 0.25 to 0.29.Source: §1 · p. 2
• The three discovery harnesses jointly cover 759 unique tasks among approximately 3K curated tasks, expanding the successful experiences available for training.The introduction frames different harnesses as solving complementary task subsets; Figure 1 depicts rewriting harness-assisted solutions into general-harness demonstrations.Source: §1 · p. 2, Figure 1 · p. 3

2.1 Multi-Harness Discovery

• Different discovery harnesses can be effective on different task types, expanding the range of successful solutions the model can produce.The framework's goal is for the model to learn from these successes under a general harness.Source: §2.1 · p. 3

2.2 Trajectory Rewriting for Experince Learning

• Trajectory rewriting turns each successful solution into a runbook-guided re-solve under the general harness, producing cleaner demonstrations for policy improvement.The planner reconstructs a procedure, the critic filters leakage, and the executor solves the task again.Source: §2.2 · p. 3
• The planner compacts each source trajectory and samples runbooks describing the required end state, milestones, checks, recovery strategies, and pitfalls without giving the finished deliverable.The compacted trajectory retains the instruction, model actions, and environment observations while removing harness-specific control messages.Source: §2.2 · p. 4
• The critic screens candidate runbooks with deterministic checks and a model-based review of whether they provide procedure or leak information.The model-based critic sees only the public task instruction and candidate runbook; only passing runbooks proceed to rewriting.Source: §2.2 · p. 4
• The executor follows private runbook guidance to solve the task in a fresh sandbox, but the runbook is excluded from the public trajectory.Trajectory filtering removes demonstrations containing values that cannot be derived from the task or environment; finetuning retains only public interaction history.Source: §2.2 · p. 4
• At inference, the finetuned model uses the general-purpose harness alone, without source harnesses or private runbooks.Planning, checking, recovery, and continuation must therefore come from the model itself.Source: §2.2 · p. 4

3 Experiments

• The experiments cover experience collection with diverse harnesses, reconstruction and auditing, and training and evaluation of the resulting model.These stages structure the experiments around the self-improvement pipeline.Source: §3 · p. 4

3.1 Terminal Tasks

• The task pool contains approximately 3K terminal tasks: about 2.5K self-constructed SWR tasks and 420 filtered and modified RST tasks.SWR spans 50 domains, including software usage, biology, chemistry, physics, hardware, operations, and security.Source: §3.1 · p. 4
• The task domains require the model to select tools, reason about environment feedback, and carry out task-specific procedures.The described domains include software usage, biology, chemistry, physics, hardware, operations, and security.Source: §3.1 · p. 4

3.2 Discovery Harnesses

• The model is held fixed across Terminus 2, StateM, and RSRT, so collected experience varies with external execution support.The discovery harnesses represent different approaches to terminal problem solving.Source: §3.2 · p. 4
• Terminus 2 provides the general-purpose terminal interaction loop and is both the reconstruction target and the inference harness.The model issues commands, observes outputs, and decides how to proceed or when to finish.Source: §3.2 · p. 5, Figure 2 · p. 5
• StateM structures execution around persistent workflow states, phase-local context, and checked transitions.It supports maintaining state, satisfying intermediate conditions, and verifying progress before moving to the next phase.Source: §3.2 · p. 5, Figure 2 · p. 5
• RSRT uses verifier outcomes to prompt reflection and continued execution after failed completion attempts, stopping on success or timeout.It hides verifier internals and reference solutions, uses the final outcome as weak supervision, and continues in the same conversation and sandbox.Source: §3.2 · p. 5, Figure 2 · p. 5

• The three-harness union solves 759 tasks (25.9%), adding 194 tasks—a 34.3% increase over the strongest individual harness.The union improves over the strongest individual harness by 19.4% on RST and 33.8% on SWR.Source: Table 1 · p. 6, §3.3 · p. 6
• With equal rollout counts, the union solves 352 of 1,247 tasks (28.2%), adding 67 over Recursive Self-Reflect Terminus, or 23.5%.Terminus 2, Recursive Self-Reflect Terminus, and StateM each still contribute exclusive solves: 30, 56, and 21, respectively.Source: Table 2 · p. 6, §3.3 · p. 6
• Unequal rollout counts affect full-pool comparisons because sandbox errors leave fewer rollouts for Recursive Self-Reflect Terminus and StateM.The equal-count analysis checks complementarity on a subset with matched sampling.Source: §3.3 · p. 6
• Of 759 tasks solved by at least one harness, 288 are exclusive to one harness, 163 are solved by exactly two, and 308 by all three.The exclusive counts are 129 for Terminus 2, 91 for Recursive Self-Reflect Terminus, and 68 for StateM.Source: §3.3 · p. 6, Figure 3 · p. 7
• Across 87 fine-grained labels, the harnesses individually cover 49, 46, and 42 domains, while their union covers 51; across 88 task types, the union covers 60.The individual task-type counts are 56 for Terminus 2, 52 for Recursive Self-Reflect Terminus, and 49 for StateM.Source: §3.3 · p. 7
• Passing trajectories also differ in execution style: median turns are 17 for Terminus 2, 18 for Recursive Self-Reflect Terminus, and 24 for StateM.Terminus 2 assigns 55.5% of commands to exploration, while StateM devotes 29.6% to state management; failed Recursive Self-Reflect Terminus trajectories average 67.1 turns.Source: §3.3 · p. 7
• Task examples illustrate distinct approaches: continued execution, structured checks, and direct exploration can address different failure modes.Examples include resuming after a rejected completion, catching an exact dependency-version constraint, and directly exploring and implementing a watershed reconstruction.Source: §3.3 · p. 7
• No harness leads every domain: combining them raises media and signal coverage to 22.3% and earth and geospatial coverage to 19.8%.The best individual rates are 13.1% and 12.2%, respectively.Source: §3.3 · p. 7, Figure 4 · p. 8

3.4 Recursive Self-Rewrite and Case Study

• Recursive Self-Rewrite produced 12,893 rollouts, of which 86% passed, yielding 11,094 rewritten trajectories for training.The process samples four candidate runbooks, screens them with a critic, and uses retained runbooks to guide four fresh executions per successful source rollout.Source: §3.4 · p. 8
• In the Markdown inline-parsing case, 7 of 12 rewrites pass (58.3%); the median passing rewrite is 32 turns versus 64 for the source.Rewrites reuse guidance about a compatible parser and reconstruction steps, but shorter setup does not remove the core reconstruction difficulty.Source: Figure 5 · p. 9, §3.4 · p. 10
• In the OpenFOAM PitzDaily case, 7 of 8 rewrites pass (87.5%), with a median passing length of 29 turns.Fresh executions preserve analysis, implementation, checking, and repair while using the general interface; some successful rewrites are longer than the source.Source: Figure 5 · p. 9, §3.4 · p. 10
• In both cases, the same runbook can guide passing and failing rewrites, so shorter or cleaner executions do not guarantee success.Figure 5 compares source trajectories with passing and failing rewrites guided by the same runbook.Source: Figure 5 · p. 9, §3.4 · p. 10

3.4.1 Command Similarity Across Rewrites

• The similarity measures compare execution structure, not semantic equivalence, using command fingerprints, tool sets, transitions, and heuristic action labels.The analysis compares rewrites sharing a runbook with rewrites using different runbooks for the same task.Source: §3.4.1 · p. 10
• Command overlap is higher within shared runbooks in both tasks: Markdown exact overlap is 19.1% versus 12.1%, and OpenFOAM is 12.6% versus 10.4%.Command-order similarity follows the same pattern: 24.6% versus 15.7% for Markdown and 18.2% versus 14.7% for OpenFOAM.Source: Table 4 · p. 10, §3.4.1 · p. 11
• Broader workflow similarity varies by task: Markdown has higher within-runbook tool-transition and action-sequence similarity, while OpenFOAM shows little transition difference and slightly lower action similarity.Markdown values are 34.0% versus 18.9% for tool transitions and 53.6% versus 42.0% for action sequences; OpenFOAM values are 28.4% versus 28.5% and 61.0% versus 63.1%.Source: §3.4.1 · p. 11

3.5 Model Training and Evaluation

• Direct SFT trains on passing source rollouts after removing harness-specific insertions, while Recursive Self-Rewrite trains on verified rewrites plus 766 unrewritten Terminus 2 successes.The Base condition is the original model; Direct SFT omits runbook reconstruction and fresh re-execution.Source: §3.5 · p. 11
• The study evaluates Base, Direct SFT, and Recursive Self-Rewrite using the same Qwen-3.8-27B model.Direct SFT uses passing source rollouts; Recursive Self-Rewrite uses verified rewrite trajectories alongside 766 Terminus 2 passing trajectories.Source: §3.5 · p. 11
• Evaluation covers Terminal-Bench 2, 3, 4, Terminal-Bench Hard, Long-Horizon Terminal Bench, and Software Terminal 100.Software Terminal is self-curated.Source: §3.5 · p. 11

3.6 Results and Discussion

• RSR achieves the best results across TB2, TB3, TB4, TBH, and SWR100, and has higher LHTB process reward than Direct SFT.Table 5 reports pass@3 and mean per-run pass rate for the five benchmarks, and process reward for LHTB.Source: Table 5 · p. 11, §3.6 · p. 11
• Software Terminal 100 is a self-curated benchmark.The results section reports it alongside the other benchmarks.Source: §3.6 · p. 11
• Direct SFT improves pass@3 over Base on TBH, TB3, and TB4, matches Base on SWR100, but falls 3.6 points on TB2.Its TB2 mean per-run pass rate falls from 51.7% to 43.8%; inspection of three direct-SFT trajectories found looping behavior that caused task failures.Source: §3.6 · p. 12
• RSR improves pass@3 over Direct SFT by 20.8, 4.1, 4.6, 7.0, and 3.0 percentage points on TB2, TB3, TB4, TBH, and SWR100, respectively.On LHTB, process reward is 0.29 for RSR versus 0.25 for Direct SFT; all groups complete zero of 46 tasks.Source: §3.6 · p. 12

3.7 Ablation Studies of Harnesses on Sample Benchmarks

• Passing-task sets vary substantially across harnesses on TBH and TB3, while StateM achieves the highest pass@1 on TB2.The evaluation uses the same base model without additional training; it complements the training comparison by testing harness changes alone.Source: §3.7 · p. 12, Table 6 · p. 12
• A more suitable harness can let the same base model solve more challenging tasks without additional training.TBH and TB3 remain challenging for the base model, and passing tasks differ substantially across harnesses.Source: §3.7 · p. 12

4 Related Work

• Harness-engineering work improves performance by changing execution support while keeping model weights fixed; RSR instead uses diverse harnesses as discovery tools.Prior approaches improve verification, continuation, workflow control, or harness search; reported harness changes can increase solve rates and alter model rankings.Source: §4 · p. 13
• Recursive self-improvement methods revise harnesses or co-evolve harnesses and models, whereas RSR reconstructs multi-harness successes as verified demonstrations for supervised learning.The cited methods include harness-module revision, exploration-policy optimization, and nested harness/model adaptation.Source: §4 · p. 13
• Trajectory-reuse methods provide contextual guidance or training signals; RSR focuses on reconstructing completed successes under a shared general harness after specialized support is removed.Related work includes retrieved reflections, verified software-engineering trajectories, and studies of cross-harness compatibility.Source: §4 · p. 13

• RSR reconstructs successful experiences from diverse harnesses into new verified trajectories under a general harness.Source: §5 · pp. 13–14
• Diverse harnesses help the same model discover complementary successful solutions, though their execution conventions may not transfer directly to a general harness.Source: §5 · p. 14
• Rewriting these experiences helps models relearn planning, checking, recovery, and sustained-execution behaviors under a general harness.Source: §5 · p. 14