PaperlayerView exact revision on arXiv

Source-linked AI summary

PAUSE: Search-Free Steering of Coding Agents Before Their First Source-Code Edit

Viral Patel, Boyin Tan, Jason Xue, Youcheng Sun

arXiv:2610.29870v1cs.SEcs.AI

TL;DR

Coding agents can investigate repositories yet still produce unsuccessful repairs, while existing steering approaches rely on periodic reviews or behavioral alarms. PAUSE inserts one plan at the first source-code edit and resumes the same Executor with its state. It improves resolution over LivePlan on both benchmarks at similar Planner cost and also supports offline recovery.

  • Problem

    Coding agents can struggle to turn repository investigation into a correct repair, while existing steering relies on costly periodic reviews or detected behavioral problems.

    Source: §1 · pp. 1–2

  • Method

    PAUSE triggers one Planner-generated repair plan at the first source-code edit and resumes the same Executor with its existing context and workspace.

    Source: §1 · p. 3

  • Results

    In SWE-agent, PAUSE improves resolution over LivePlan by 3.2% on Verified and 3.7% on Multilingual at similar Planner cost.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Across the evaluated benchmarks and settings, one pre-edit plan can improve online resolution and recover failed attempts offline while retaining prior investigation.

    Source: §7 · p. 18

    PAUSE requires a harness to expose the first source-code file-editor request and a repository-appropriate path rule, so workflows using other editing interfaces may receive no intervention.

    Source: §5.1 · p. 16

Abstract

from arXiv · show

Coding agents can invest substantial effort in repository exploration and testing yet still produce unsuccessful patches. Existing online steering methods review trajectories periodically or invoke an advisor when a monitor detects behavioral problems; frequent reviews are costly, and the absence of anomalous behavior does not mean that an agent is on the right track. We propose PAUSE, a framework for steering coding agents with a single planning intervention. Agents reach their first source-code edit at a median of 60% of trajectory progress, after exploration, bug reproduction, and testing. PAUSE intercepts this edit request before it executes, which requires no learned monitor, no search over checkpoints, and no extra model call (search-free). A separate Planner reads the task and the pre-edit transcript and writes one repair plan; the same agent, the Executor, resumes with this plan, keeping its context and workspace. PAUSE applies online, during a live run, and offline, from a recorded trajectory prefix. We evaluate PAUSE on SWE-bench Verified and SWE-bench Multilingual with two Executors and three agent harnesses. With Qwen3-coder as Executor and GPT-5.6 Sol as Planner, online steering improves resolution by 16.8% on Verified in OpenHands and by 14.7% on Multilingual in SWE-agent. In SWE-agent, PAUSE outperforms LivePlan, a rule-triggered advisor, by 3.2% on Verified and 3.7% on Multilingual at similar Planner cost, and is comparable to SICA's periodic overseer at about 11-15% of its Planner cost. Offline, PAUSE recovers 35.4% of eligible failed Verified tasks, and resuming from the recorded prefix uses 33-41% fewer Executor tokens than restarting.

1 Introduction

PAUSE guides coding agents at the first source-code edit, using their accumulated investigation without periodic reviews or waiting for a behavioral alarm. It supports live steering and offline recovery, and improves resolution across the evaluated benchmarks and settings.

1 Introduction

  • Periodic reviews incur inference cost and may delay feedback, while rule-based monitoring depends on predefined signals that may miss orderly but ineffective repair strategies.The absence of a behavioral alarm does not establish that the repair strategy is correct.Source: §1 · p. 2
  • The first source-code edit occurs at a median of 60.0% of trajectory progress, after substantial investigation.The intervention seeks to use accumulated investigation without repeated reviews or waiting for an alarm.Source: Figure 1 · p. 2, §1 · p. 2
  • PAUSE uses the first source-code edit request to trigger one plan from a separate Planner, then resumes the same Executor with its context and workspace.The trigger requires no learned monitor, checkpoint search, or extra model call to locate the intervention point.Source: §1 · p. 3
  • PAUSE supports both online steering during a live run and offline recovery by restoring a failed trajectory to its pre-edit state and resuming with its remaining budget.The evaluation spans two benchmarks, two Executors, and three agent harnesses.Source: §1 · p. 3
  • 3.2% and 3.7%: in SWE-agent, PAUSE improves resolution over LivePlan on Verified and Multilingual, respectively, at similar Planner cost.Against SICA, PAUSE is comparable on both benchmarks at about 11% and 15% of SICA’s Planner cost on Verified and Multilingual, respectively.Source: §1 · p. 3

2 Related Work

Prior work addresses repository repair through agent tools, planning, monitoring, search, trajectory reuse, and feedback. PAUSE instead places one plan within an executing agent’s existing trajectory, using its first source-code edit as the trigger and retaining the Executor’s state.

2.1 Repository-Level Repair and Planning

  • Repository-repair systems combine navigation, editing, execution feedback, localization, structural search, diagnostic exploration, or training to support investigation and repair.Examples include Agentless, SHERLOC [32], SWE-Fixer [40], and SWE-Gym [28].Source: §2.1 · pp. 3–4
  • Planning and collaboration methods separate roles or compare candidate solutions, whereas PAUSE derives one plan from an executing agent’s investigation and resumes that same Executor with its retained state.SWE-Manager’s P2A framework separates proposal generation, managerial selection and synthesis, and implementation [33].Source: §2.1 · p. 4

2.2 Online Steering

  • Online steering methods use periodic assessment or detected behavioral conditions, while PAUSE triggers a single intervention at the first source-code edit regardless of an alarm.SWE-PRM [11], SICA [29], and LivePlan [22] use periodic reviews or detected signals.Source: §2.2 · p. 4

2.3 Trajectory Reuse and Reflection

  • Search and trajectory-reuse methods explore alternatives or branch from intermediate states, while offline PAUSE inserts one plan at the first-edit checkpoint and continues with the remaining budget.Prior approaches include SWE-Search [2], SWE-Replay [7], DARS [1], and SE-Agent.Source: §2.3 · p. 4
  • Feedback methods guide later attempts or revise outputs using verbal or tool-grounded evidence, while PAUSE delivers planning within an existing trajectory and retains the Executor’s conversation and workspace.Examples include Reflexion [31], Self-Refine [24], CRITIC [12], Self-Debugging [6], LDB [49], and ChatRepair.Source: §2.3 · pp. 4–5

3 Method

PAUSE guides repository repair at the first proposed source-code edit, after investigation has produced evidence but before the change executes. It uses one Planner-written plan and resumes the same Executor with its context and workspace preserved, online or from an eligible recorded failure.

3.1 Problem Formulation

  • Online steering uses the issue and available execution evidence to guide repair, excluding evaluator feedback, hidden tests, reference patches, and future actions.The Executor’s conversation records actions and observations, while its workspace reflects the resulting repository state.Source: §3.1 · p. 5

3.2 Pre-Edit Intervention

  • The first file-editor request targeting a repository source-code path triggers PAUSE before execution, without a learned monitor, checkpoint search, or extra model call.If no source-code edit request occurs, the live run continues without intervention.Source: §3.2 · pp. 5–6

3.3 Plan Generation and Online Steering

  • A separate Planner uses the issue and transcript to propose the likely cause, code change, intended repair, and rationale; the Executor remains responsible for implementation and testing.PAUSE blocks the proposed edit, appends the plan, and resumes the same Executor with its history, workspace, and execution settings preserved.Source: §3.3 · p. 6

3.4 Offline Recovery

  • Offline recovery restores an eligible failed trajectory to the last reasoning checkpoint before its first source-code edit, adds a plan, and resumes the same Executor.It requires an explicit think action and a successful fidelity check; the continuation retains the recorded state and receives only the remaining iteration budget.Source: §3.4 · p. 7

4 Experiment

The experiments assess PAUSE’s effectiveness and efficiency online and offline, its generalization across harnesses and reasoning interfaces, and the pre-edit intervention boundary.

4 Experiment

  • The evaluation measures PAUSE’s effectiveness and efficiency for online steering and offline recovery.These are designated RQ1 and RQ2.Source: §4 · p. 7
  • The evaluation tests generalization across harnesses and reasoning interfaces.This is designated RQ3.Source: §4 · p. 7
  • The evaluation tests the effectiveness of the pre-edit boundary.This is designated RQ4.Source: §4 · p. 7

4.1 Experimental Setup

The experiments use two SWE-bench datasets and define outcome, recovery, regression, and resource measures. Comparisons cover online and offline alternatives, harness and reasoning-interface variants, and the model and execution configurations.

4.1.1 Benchmarks.

  • SWE-bench Verified contains 500 Python repair tasks, while SWE-bench Multilingual contains 300 tasks across nine languages.Both benchmarks support online comparisons; recorded Verified trajectories support offline recovery.Source: §4.1.1 · pp. 7–8

4.1.2 Evaluation Metrics.

  • Resolution is the fraction of evaluated tasks solved; offline resumed outcomes retain uncontinued tasks’ original results and adjust for recovered failures and regressions.Recovery rate divides originally failed tasks solved after intervention by evaluated failures; regression rate is the fraction of baseline successes that fail after intervention.Source: §4.1.2 · p. 8
  • Efficiency reporting separates Planner and Executor tokens and measured model calls; Planner dollar costs exclude Executor inference, and offline continuation costs exclude the recorded prefix run unless stated otherwise.Evaluator errors, empty patches, and missing evaluations count as unresolved; paired outcomes are compared with exact McNemar tests.Source: §4.1.2 · p. 8

4.1.3 Baselines and Control Conditions.

  • Online comparisons test PAUSE against LivePlan’s rule-triggered advisor and SICA’s scheduled overseer; offline comparisons test SHERLOC, plan contributions, and resuming versus restarting.Boundary controls also compare donor guidance and several checkpoint cuts, including the default, earlier, later, immediate pre-edit, and later-edit checkpoints.Source: §4.1.3 · p. 8

4.1.4 Experimental Configuration.

  • Executors are Qwen3-coder and Devstral-small; online runs use OpenHands, SWE-agent, and Pi, while offline recovery uses recorded OpenHands trajectories.The main online comparison uses SWE-agent with Qwen3-coder and GPT-5.6 Sol; online Planner calls occur once when triggered, and resumed runs receive the recorded remaining Executor budget.Source: §4.1.4 · pp. 8–9
  • The main comparisons use one designated attempt per task, with specified reruns for online infrastructure failures and three attempts per donor–recipient pair in the donor-guidance diagnostic.Paired checkpoint arms restore independently, obtain a plan from each prefix, and receive equal fresh Executor iterations.Source: §4.1.4 · p. 9

4.2 RQ1: How Effective and Efficient Is Online Steering?

PAUSE’s single intervention improves online resolution while using fewer Executor tokens, and its observed results compare favorably with rule-based steering at similar Planner cost. Against periodic oversight, resolution is not significantly different at substantially lower Planner cost; outcomes also vary by task difficulty and language.

Overall online steering

  • 14.8 percentage points on Verified and 14.7 on Multilingual: PAUSE raises SWE-agent resolution while reducing Executor token use.The gains are from 50.2% to 65.0% on Verified and from 28.3% to 43.0% on Multilingual.Source: §4.2 · pp. 9, 11

Comparison with rule-based steering

  • 3.2 and 3.7 percentage-point leads over LivePlan on Verified and Multilingual are not statistically significant, with similar Planner cost.PAUSE uses 0.93 Planner calls per task versus LivePlan’s 2.33; aggregate Planner tokens and costs are close on Verified.Source: §4.2 · pp. 9–10

Task difficulty

  • On the 45 hard Verified tasks, PAUSE raises resolution from 11.1% to 26.7%, exceeding SICA’s 17.8% at $3.06 versus $29.53 Planner cost.SICA leads on the 194 easy and 261 medium tasks, which together account for 91% of Verified.Source: Figure 3 · p. 10, §4.2 · p. 11

Programming languages

  • PAUSE improves over the unmodified Executor in eight of nine Multilingual languages; Rust is the exception, where all three guidance methods fall below baseline.Ruby, PHP, and JavaScript account for 31 of PAUSE’s 44 additional solved tasks.Source: Figure 4 · p. 10, §4.2 · p. 11

Comparison with periodic oversight

  • Against SICA, PAUSE trails by a non-significant 2.6 percentage points on Verified and 1.3 on Multilingual, while using about 11% and 15% of SICA’s Planner cost, respectively.PAUSE leads on hard Verified tasks.Source: §4.2 · p. 11

4.3 RQ2: How Effective and Efficient Is Offline Recovery?

Offline PAUSE resumes failed trajectories from their recorded pre-edit state with a new repair plan. GPT-5.6 Sol guidance recovers failures and improves net resolution, while resuming uses fewer Executor tokens than restarting; the cost savings apply to continuation after an original run.

Recovery through external guidance

  • GPT-5.6 Sol recovers 79 of 223 failures, compared with 9 for unaided continuation and 13 for self-reconsideration.The conditions share the restored checkpoint and remaining Executor budget.Source: §4.3 · p. 12, Table 4 · p. 12

The choice of Planner matters

  • Recovery depends on the Planner: GPT-5.6 Sol recovers 79 Qwen3-coder failures versus 31 for Gemini 3.8 Flash and 12 for the Executor’s own model as Planner.GPT-5.6 Sol also recovers 53 of 174 Devstral-small failures.Source: §4.3 · p. 12

Reusing a failed investigation

  • On 223 failures, GPT-5.6 Sol resume recovers 79 tasks versus 81 with restart, while restart uses 22% more Executor tokens.Restart requires 2.0× as many Executor calls, or 6,020 additional calls.Source: §4.3 · p. 12

Full-benchmark comparison

  • At full-benchmark scale, GPT-5.6 Sol resume reaches 58.2% resolution versus restart’s 58.6%, saving 171.7M Executor tokens (33%).With Qwen3-30B-Thinking, resume saves 265.9M Executor tokens (41%) and resolves more tasks than restart.Source: §4.3 · p. 12, Table 5 · p. 13

Cost of the recorded investigation

  • The token savings are incremental to an existing trajectory: including the original run, GPT-5.6 Sol resume uses 1,139.3M Executor tokens versus SHERLOC’s 703.9M.The practical setting is recovery after an agent has already attempted the task.Source: §4.3 · p. 13

Recovery and net repair performance

  • 35.4% of eligible Qwen3-coder failures and 30.5% of Devstral-small failures are recovered with GPT-5.6 Sol; full-benchmark resolution rises from 45.6% to 58.2%.The evaluation also finds no more regressions on originally solved tasks than resuming without a plan.Source: §4.3 · pp. 12–14

4.4 RQ3: Does PAUSE Generalize Across Settings?

PAUSE improves resolution across SWE-agent, Pi, and OpenHands, though both the gain and Executor-token effect vary by harness. In OpenHands, gains appear on recorded failures and successes and persist when explicit think content is removed from Planner input.

Across harnesses

  • PAUSE improves Verified resolution by 14.8 points in SWE-agent, 6.0 in Pi, and 16.8 in OpenHands.The respective gains are 74, 30, and 84 additional solved tasks.Source: §4.4 · pp. 14–15

Cost of transfer

  • Executor tokens fall by 5.3% in SWE-agent but rise by 4.9% in both OpenHands and Pi, while Planner cost stays between $0.037 and $0.052 per task.Planner tokens remain below 2% of Executor tokens in every harness.Source: §4.4 · p. 14

Where the online gain arises

  • In OpenHands, resolution rises in all three trajectory strata: recorded recovery-cohort tasks, excluded failures, and recorded successes.On recorded successes, resolution rises from 75.9% to 87.3%, gaining 39 tasks and losing 13.Source: §4.4 · p. 14

Reasoning interfaces

  • Removing think content from the OpenHands Planner input yields 318 solved tasks versus 311 with the full transcript, a non-significant difference.The variants differ on 71 tasks, split 39 to 32 (p = 0.48).Source: §4.4 · p. 15

4.5 RQ4: Is the Pre-Edit Boundary Effective?

The first-edit checkpoint is a productive intervention point: it retains investigation while allowing a Planner to guide the repair. Offline comparisons find the strongest recovery at this boundary, though later-cut differences are not statistically significant.

4.5 RQ4: Is the Pre-Edit Boundary Effective?

  • The first source-code edit occurs at a median of 60.0% of action progress on Verified, after substantial investigation.The study tests whether guidance at that point can redirect repairs and compares it with earlier and later cuts.Source: §4.5 · p. 15
  • Donor guidance enabled at least one successful continuation in 64 of 97 donor–recipient pairs (66.0%), and all three attempts succeeded in 41 pairs (42.3%).The donor and recipient runs used different Executor prompts; this diagnostic tests whether successful continuation is possible at the boundary.Source: §4.5 · p. 15
  • 56/161 tasks (34.8%) were recovered at the PAUSE checkpoint, versus 43 after the 20% cut and 50 after both the pre-edit and 90% cuts.The four-way comparison includes only tasks graded in every arm.Source: §4.5 · pp. 15–16, Table 7 · p. 16
  • The PAUSE checkpoint recovered 65 of 196 shared tasks, compared with 59 at the cut immediately before the first edit; the 90% cut recovered 32.3% of 167, versus 34.7% for PAUSE.Neither later-cut comparison differed significantly from PAUSE (p=0.39 and p=0.62).Source: §4.5 · pp. 15–16
  • On 207 shared tasks, PAUSE solved 26 tasks missed by the 20% cut, while that cut solved 15 missed by PAUSE, a 5.3-point gap.The difference was not significant at the 0.05 level (p=0.12); the direction was consistent with the Planner having seen less investigation at the earlier cut.Source: §4.5 · p. 15
  • A supplementary comparison found 15 recovered tasks at both the first and later source-code edit checkpoints among 29 paired runs.Of 46 candidate trajectories, 17 failed restoration checks before a model call; waiting for another edit yielded no increase in recovered tasks.Source: §4.5 · p. 16

5 Limitations and Threats to Validity

PAUSE depends on pre-edit repository evidence, a single plan, and an exposed file-editor request; its evaluation also has boundaries in validity, transfer, and statistical power.

5.1 Limitations

  • The Planner cannot inspect the workspace or run commands, so omitted code or reproduction feedback can leave its hypotheses unverified against the repository.Further Executor investigation may be needed to assess whether the suggested strategy is appropriate.Source: §5.1 · p. 16
  • PAUSE supplies one plan without refreshing it as new observations arrive, leaving the Executor to adjust obsolete guidance and implement and validate the patch.Steering can redirect strategy, but does not itself produce a correct patch.Source: §5.1 · p. 16
  • The intervention requires a file-editor request exposed before execution and a repository-appropriate path rule; workflows editing source through shell commands may receive no intervention..Source: §5.1 · p. 16
  • Offline recovery requires a recorded trajectory, usable reasoning checkpoint, and faithfully restorable history and workspace; a transcript alone is insufficient.The reported savings cover the additional Executor computation for another attempt, not the cost of producing the original trajectory.Source: §5.1 · p. 17

5.2.1 Internal validity.

  • Public benchmark issues and fixes may have appeared in the Planner’s training data, and the authors’ execution checks cannot establish prior training exposure.All Executor runs mount only the instance ID and problem statement.Source: §5.2.1 · p. 17

5.2.2 External validity.

  • Evaluation covers the tested benchmarks and configurations; transfer to substantially different repositories or longer, multi-stage workflows remains unexamined.Offline recovery and harness comparisons use Verified only, and recovery findings concern trajectories meeting checkpoint and restoration requirements.Source: §5.2.2 · p. 17

5.2.3 Construct validity.

  • The first-edit trigger uses an action type and benchmark-specific path rule, while benchmark tests measure success; broader paths may capture different investigation stages and behavior beyond test coverage is unevaluated.Alternative cuts based on run length are offline references, not live triggers; incomplete grading records count as unresolved.Source: §5.2.3 · p. 17

5.2.4 Conclusion validity.

  • Main comparisons use one attempt per task, and none of the paired pause-position comparisons reaches significance at the 0.05 level.Similar recovery counts do not establish equivalence; repeated paired runs are needed to assess whether small differences persist.Source: §5.2.4 · p. 17

6 Future Work

Future work proposes testing how to select pre-edit evidence for planning and what parts of failed investigations should be retained when resuming.

6 Future Work

  • Test whether selectively retaining code observations, reproduction outputs, test feedback, and reasoning can preserve repair gains with less computation than longer or randomly selected excerpts.The proposed study holds the checkpoint, Planner, and Executor fixed, compares several fixed input budgets, and measures resolution, regressions, and planning cost across repeated paired continuations.Source: §6 · pp. 17–18
  • Compare full histories with versions masking prior diagnostic conjectures, while retaining repository operations and observations, to test what failed attempts contribute to reuse.A restart condition would reset history and workspace under the same plan and continuation budget; outcomes would include repair, plan adoption, and repeated investigation.Source: §6 · p. 18

7 Conclusion

PAUSE gives a coding agent one search-free plan before its first source-code edit, then lets the same Executor continue with its context and workspace. Evaluation finds online resolution gains and offline recovery, while retaining state lowers additional Executor computation relative to restarting.

7 Conclusion

  • PAUSE triggers one search-free repair plan before the first source-code edit, then resumes the same Executor with its accumulated context and workspace.The framework supports live steering and recovery from recorded failures.Source: §7 · p. 18
  • Evaluation on SWE-bench Verified and Multilingual shows that one pre-edit plan can improve online resolution and recover failed attempts offline.Online improvements across three harnesses and recovery with two Executors support applicability across several tested settings, with gains varying by Planner and execution setting.Source: §7 · p. 18
  • Retaining an existing investigation lowers the additional Executor computation required for another attempt compared with restarting.The conclusion frames external planning and state reuse as complementary roles.Source: §7 · p. 18
  • Comparisons with earlier and later pause positions find no point that recovers more failures than the first-edit boundary.The conclusion presents the first source-code edit as an opportunity to turn accumulated evidence into revised repair decisions.Source: §7 · p. 18

Related papers

Loading 2610.29870v1…