PaperlayerView exact revision on arXiv

Source-linked AI summary

The Handoff Tax: Continuing Non-Native Trajectories in LLM Agents

Roy Ganz, Mor Shpigel Nacson, Adi Kalyanpur, Ron Litman

arXiv:2608.24358v1cs.AI

TL;DR

Long-horizon coding agents must balance model capability and cost when switching models mid-run, but the receiver must continue a trajectory it did not create. The paper varies handoff direction, timing, and interface across Claude and GPT model pairs while preserving repository state. Raw escalation recovers less than half of HC’s quality advantage, whereas downshift offers a favorable cost–quality point and benefits from preserving HC trajectory information.

  • Problem

    The paper asks how continuing another model’s trajectory affects cost–quality trade-offs and how transferred information shapes that effect.

  • Method

    The study evaluates Claude and GPT LC/HC pairs on SWE-bench Verified while varying handoff direction, timing, and trajectory interface with preserved working-tree state.

  • Results

    Raw escalation recovers less than half of HC’s quality advantage in both model families, while Raw downshift provides a favorable cost–quality trade-off below HC-only cost.

  • Takeaways & Limitations

    Reducing LC trajectory information improves escalation, whereas removing HC trajectory information harms downshift, making handoff interface a direction-dependent design problem.

  • Takeaways & Limitations

    The coding-agent study uses one benchmark, SWE-bench Verified, and difficulty-conditioned switched subsets are relatively small in hard strata.

Abstract

from arXiv · show

Coding agents perform long-running tasks spanning dozens of model calls, tool uses, and code edits. As these runs unfold, users face a practical cost-quality trade-off: escalating to a stronger model when a cheaper one struggles, or downshifting once the hard reasoning is complete. Each switch requires the receiver to continue a non-native trajectory produced by another model. We study how this handoff affects quality and cost, and how varying the trajectory information inherited by the receiver changes the outcome. Using pairs of low-cost, low-capability (LC) and high-cost, high-capability (HC) models from the Claude and GPT families, we vary handoff direction, timing, and interface, comparing full-trajectory transfer, compaction, and trajectory removal while preserving the repository state. Across both model families, full-trajectory escalation recovers less than half of the LC-to-HC quality gap while incurring a substantial cost premium. We term this cost-quality penalty the handoff tax. By contrast, downshift offers a favorable cost-quality point. Interestingly, the preferred interface also reverses with direction: reducing LC-model trajectory information improves escalation quality, whereas removing the HC-model trajectory reduces downshift quality.

1 Introduction

Long-horizon coding agents create a cost–quality problem because users must choose models before knowing the task’s capability needs, motivating mid-run handoffs. The paper studies how direction, timing, and transferred trajectory information affect these handoffs, finding that escalation performs poorly while downshift offers a favorable trade-off.

  • Coding agents chain dozens to hundreds of model calls, making model choice an economic decision because higher capability costs more while cheaper models offer lower capability.
  • Users can escalate from LC to HC when the cheaper model struggles or downshift from HC to LC after hard reasoning appears complete.LC and HC denote lower-cost/lower-capability and higher-cost/higher-capability models within each family.
  • The study asks how inheriting a non-native trajectory changes cost–quality trade-offs and how the transferred information shapes that effect.It varies handoff direction, timing, and interface on SWE-bench Verified.
  • Raw escalation recovers less than half of HC’s quality advantage in both model families.For Claude, restarting with HC is cheaper and more accurate than continuing the LC trajectory after paying for the LC prefix.
  • Raw downshift offers a favorable cost–quality trade-off, retaining substantial quality gains over LC while remaining cheaper than HC.Claude retains most of LC’s cost advantage, whereas GPT retains most of HC’s quality advantage.

3. Handoff interfaces reveal what trajectory

Trajectory information has direction-dependent value at a handoff: reducing the prefix trajectory helps escalation, whereas preserving the HC trajectory helps downshift.

  • Reducing LC trajectory information improves escalation, with compaction favoring savings and trajectory removal favoring quality recovery.
  • Removing HC trajectory information while preserving working-tree edits reduces downshift quality across both model families.The results indicate that HC trajectories guide LC receivers, whereas LC trajectories burden HC receivers.

2 Related Work

Prior work improves inference economics through model selection and routing, including multi-turn decisions about which model acts next. This paper instead studies active-run switches and how the inherited trajectory affects end-to-end quality and cost.

  • Cascades invoke higher-capability models when lower-cost ones are insufficient, while routers dispatch requests across a model pool.
  • Recent routing work extends model selection to multi-turn interactions by deciding which model acts and when.
  • This work studies switching during active runs between commercial models with different capability and cost profiles.
  • The study varies handoff direction, timing, and interface to measure how continuing a non-native trajectory affects end-to-end quality and monetary cost.
  • Handoff interfaces preserve the prefix model’s edited working tree while varying whether the full trajectory, a summary, or no trajectory reaches the suffix model.The interfaces include Raw, Compactpre, Compactsuf, and Traj-drop.

3 Experimental Framework

The experiments evaluate mid-trajectory handoffs on SWE-bench Verified with matched model pairs, preserved working-tree state, controlled switch percentiles, and trajectory-transfer interfaces. Results are normalized against matched LC-only and HC-only baselines across a large configuration set.

  • Experimental setup: The study uses SWE-bench Verified, a GitHub issue-resolution benchmark with executable pass/fail evaluation and difficulty annotations.All conditions use the same mini-swe-agent scaffold, tools, and prompts.
  • Directions: Escalation switches from LC to HC to test whether HC can rescue an LC-started trajectory and at what cost, while downshift transfers remaining HC work to LC.
  • Handoff Strategies: All handoff strategies preserve the edited working tree W_K and differ only in the trajectory information passed to the suffix model.
  • Handoff Strategies: Raw transfers the full trajectory, Compactpre passes a prefix-written summary, Compactsuf uses a suffix-written summary, and Traj-drop passes only W_K.
  • Switch Points: Switch points use starting-model step-count percentiles estimated separately within difficulty buckets, sweeping {5, 10, 15, 25, 35, 45, 50}.Later percentiles are omitted because too few instances reach the switch for reliable comparison.
  • Evaluation on the Switched Subset: Comparisons restrict every strategy and baseline to the same intersection of instances that switched under all four strategies.This matched switched subset provides comparable quality and cost references.
  • Metrics: Quality Recovery measures the fraction of HC’s quality advantage over LC recovered, while Cost-Savings Retention measures the fraction of LC’s cost advantage over HC retained.QRec is anchored at 0 for LC quality and 100 for HC quality; CSRet is 100 at LC cost and 0 at HC cost.
  • Experimental scale: The experiment evaluates 58 configurations per model family across 500 instances, totaling 58,000 agent runs, 2 million API calls, and 36 billion processed tokens.

4 The Handoff Tax

Across model families, escalation usually recovers limited quality at substantial cost, while downshift offers a favorable intermediate point. Reducing inherited LC context helps HC receivers, whereas preserving HC context helps LC receivers, through direction-specific computational costs.

  • Escalation: Raw escalation recovers less than half of HC’s quality advantage across both model families, with QRec=47% for Claude and 36% for GPT.
  • Escalation: Raw escalation raises cost over LC-only by approximately 4.0× for Claude and 6.1× for GPT.For Claude, Raw costs $1.61 versus $0.72 for HC-only.
  • Escalation: For Claude, restarting HC from scratch is cheaper and more accurate than continuing Raw after charging for discarded LC work.Abort + HC fresh and LC-full + HC-full cost $0.90 and $1.12, respectively, versus $1.61 for Raw.
  • Escalation: Compactpre improves escalation cost retention, while Traj-drop produces larger quality-recovery gains by removing the trajectory while preserving working-tree edits.For Claude, Compactpre raises QRec from 47% to 60%, while Traj-drop raises it to 64%; for GPT, Traj-drop raises QRec from 36% to 84%.
  • Downshift: Raw downshift creates a favorable cost–quality point below HC-only, with Claude retaining most of LC’s cost advantage and GPT retaining most of HC’s quality advantage.For Claude, pass rate rises from 54.6% to 65.6% while cost rises from $0.41 to $0.51.
  • Cost mechanics: The cost penalties arise through different channels: Raw escalation makes HC steps more expensive, whereas Traj-drop downshift requires additional LC steps.Relative to Compactpre, Raw escalation raises Claude’s post-handoff HC cost per step by 2.2×, while Traj-drop requires 1.6× more post-handoff LC steps.

5 Beyond the Coding-Agent Setting

The extensions show that handoff value depends on how task-relevant information becomes available over time. Late-arriving requirements favor escalation, while progressive search allows escalation to recover quality without savings and makes downshift a cost-quality middle ground.

  • Across all five task families, escalation recovers 86% of HC’s quality advantage and retains 36% of LC’s cost advantage under late-arriving requirements.Downshift recovers 31% of HC’s quality advantage while retaining 53% of LC’s cost advantage.
  • Late-arriving requirements favor escalation because the receiver is the first model able to solve the fully specified task.LiC remains underspecified until the final requirement shard, assigning the decisive stage to HC.
  • For GPT on BrowseComp, escalation nearly closes the HC quality gap but produces no savings.QRec = 95.8%, while CSRet is −30.0%, worse than aborting LC and restarting HC at −8.5%.
  • Under progressive search, downshift recovers 56.7% of HC’s quality advantage while retaining 76.8% of LC’s cost advantage.Inherited search progress therefore provides a useful middle ground rather than savings through escalation.
  • Together, these extensions show that handoff value depends on information dynamics and the task-relevant state available at handoff.LiC favors an HC receiver, whereas BrowseComp shows inherited search progress can nearly recover HC quality without producing savings.

6 Discussion and Conclusion

The paper concludes that handoffs are a distinct inference problem because trajectory transfer interacts with receiver selection, switch timing, and persistent repository state. It therefore recommends optimizing these choices jointly.

  • Under Raw handoff, escalation offers a poor cost-quality trade-off, whereas downshift yields a favorable intermediate point.
  • Reducing LC trajectory information improves escalation, while removing HC trajectory information harms downshift.
  • Handoff design should be optimized jointly with receiver selection and switch timing.The receiver continues both an existing work product and a non-native trajectory within persistent repository state.

Limitations

The study covers two model pairs and uses SWE-bench Verified as its primary coding-agent benchmark, while extending information-dynamics evaluations under Raw transfer.

  • The study evaluates two model pairs: Claude Haiku 4.5/Opus 4.7 and GPT-5.6 Luna/Sol.
  • The primary coding-agent study uses one benchmark, SWE-bench Verified.The broader information-dynamics evaluation extends beyond this setting but uses only Raw transfer.
  • Difficulty-conditioned findings are exploratory because matched switched subsets are relatively small for the hard SWE-bench stratum.The reported cell sizes are approximately 24–27 tasks.
  • Single episodes per task and configuration do not estimate variability across repeated runs.Interface comparisons are restricted to tasks in the shared intersection switched under every strategy.

A.1 Experimental Details

The experiments preserve repository state while varying which model continues and what trajectory information it receives. They compare Raw, compaction, and trajectory-removal interfaces under calibrated switch points and explicit cost accounting.

  • Agent and task environment: A wrapper containing prefix and suffix models preserves one agent loop, Docker environment, and repository state throughout each run.Interfaces differ only in the trajectory information supplied to the suffix model.
  • Models and handoff directions: The study evaluates escalation and downshift using LC→HC and HC→LC model directions.Claude uses Haiku 4.5 and Opus 4.7; GPT uses GPT-5.6 Luna and Sol.
  • Switch rule: Switch points use percentiles {5, 10, 15, 25, 35, 45, 50} of prefix-model termination-step distributions within difficulty buckets.The first suffix call occurs at q = sb, after sb prefix actions and observations.
  • Handoff interfaces: Raw transfers the full trajectory verbatim alongside the persistent working tree.The suffix receives the system prompt, task, reasoning, tool calls, and observations.
  • Handoff interfaces: Compactpre and Compactsuf use trajectory summaries, while Traj-drop transfers no trajectory information.Compactsuf lets the suffix model author the continuation summary; Traj-drop supplies only the task, system prompt, static continuation message, and working tree.
  • Accounting: Compact interfaces add one paid summarization call, whereas Raw and Traj-drop add no model calls.Summarization cost and usage are attributed to the first suffix step.
  • Pricing and audit: The experimental artifact records invocation identifiers, pricing entries, implementation revisions, and evaluator revisions, with infrastructure failures rerun rather than scored.
  • Accounting: GPT exhibits the same handoff-cost accounting pattern as Claude: Raw HC steps cost 1.6× Compactpre, while Traj-drop downshift takes 2.0× as many post-handoff LC steps.

A.3 Additional Results and Analyses

Additional analyses show that interface effects are direction-dependent and persist across timing windows, while receiver-side computation helps explain the escalation cost premium.

  • Difficulty: Escalation economics improve with difficulty, with structured interfaces recovering more quality at a more favorable cost.For downshift, context-preserving interfaces consistently outperform Traj-drop in quality recovery and cost efficiency.
  • Timing: Traj-drop achieves the highest escalation quality recovery and the lowest downshift quality recovery across early and late switch windows.Switch timing changes the magnitude, but not the direction, of this reversal.
  • Receiver-side computation: Raw escalation increases the average cost of each post-handoff HC step relative to Compactpre while requiring a similar number of HC steps.The receiver-side decomposition attributes the Raw escalation premium primarily to higher per-step cost.
  • Disclosure ablation: A disclosure-only prompt provides a modest quality gain at slightly higher cost but remains below HC-only and is dominated by Compactpre and Traj-drop.Disclosure alone does not eliminate the Raw handoff tax in Claude escalation.
  • Uncertainty: The matched-subset bootstrap pools contain 460 Claude escalation tasks, 458 Claude downshift tasks, 411 GPT escalation tasks, and 426 GPT downshift tasks.Replicates draw task IDs with replacement while preserving each task’s interface weights.
  • Interface effects: Traj-drop improves escalation pass rate but reduces downshift pass rate relative to Raw across both model families.All four pointwise bootstrap intervals exclude zero.

B.1 Dataset and Protocol

The study evaluates model handoffs using deterministic, matched protocols across coding task families, switch-timing windows, interfaces, and standardized metrics.

  • Dataset: The corpus spans five task families, including Code, Database, Actions, and other sharded-instruction tasks.Code combines 45 HumanEval and 55 LiveCodeBench tasks; Database and Actions contain 107 and 105 tasks, respectively.
  • Protocol: Each episode reveals one released shard per user turn in official order, uses one assistant call after every shard, and evaluates only the final response.Episodes have exactly N assistant turns and no early termination.
  • Models: The experiments use within-family Claude and GPT model pairs under deterministic schedules and evaluators, with no tools in the chat-only LiC setting.Assistant turns are re-fed as visible text without hidden reasoning blocks.
  • Handoff: At switch point K, the receiver answers the remaining shards while inheriting the sender’s full visible history without a model-change announcement.Early, Middle, and Late switches use K = 1, floor(N/2), and N − 1.
  • Reporting: Timing-window tables compare Claude and GPT escalation and downshift using unweighted means over matched switched subsets.Pass is reported in percent, cost in dollars, and steps as mean trajectory length.
  • Metrics: QRec measures recovered HC quality advantage, while CSRet measures retained LC cost advantage relative to matched single-model anchors.Disaggregated QRec is omitted when the absolute LC–HC anchor gap is below five points.

B.2 Results by Switch Position

Switch position changes the magnitude of handoff savings and quality, with later escalation generally saving more and later downshift generally improving quality while saving less.

  • Claude: Later Claude escalation retains more savings while quality remains mostly stable.This pattern is reported by structural switch position.
  • Downshift: Later downshift generally improves quality recovery while retaining less savings across the Claude and GPT position analyses.The main departure is GPT Actions downshift, where a small LC–HC anchor gap makes QRec volatile.
  • GPT: Later GPT escalation retains more savings, while quality is mostly stable unless prior turns accumulate deliverable state.The GPT table reports the same timing pattern with this qualification.
  • BrowseComp: BrowseComp uses a fixed cohort of 200 difficult information-seeking questions selected from a topic-stratified sample after removing 22 questions solved without browsing.The screening used the Claude pair.
  • Browsing protocol: BrowseComp agents use a single-context ReAct loop with web-search and URL-retrieval tools, allowing up to 100 steps before finalization.Research stops at approximately 80% context utilization.
  • BrowseComp handoff: At BrowseComp switch step K, the receiver inherits the sender’s complete response prefix, including reasoning items, tool calls, and tool outputs.Prefix tools are not re-executed, and both models share the original step budget.
  • Interpretation: BrowseComp results are descriptive because each task–policy cell contains one successfully audited generation and retry attempts are not retained as aggregate counts.Exhausted episodes are rerun from scratch and never scored partially.
Loading 2608.24358v1…