PaperlayerView exact revision on arXiv

Source-linked AI summary

EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation

Zeru Shi, Jindong Jiang, Ryo Hachiuma, Yonggan Fu, Byung-Kwan Lee, Peter Belcak, Saurav Muralidharan, Ayush Agarwal, Ruixiang Tang, Ambrish Dantrey, Pavlo Molchanov, Anjul Patney, Subhashree Radhakrishnan

arXiv:2610.17560v1cs.CL

TL;DR

Long-horizon agents often use costly cloud frontier models throughout a trajectory, although capability needs vary and prior routing work has limited long-horizon evaluation and relies on predefined model pools. EdgeWise keeps an SLM local and delegates critical steps; self-delegation maintains performance comparable to LLM-only execution at lower cost, while evaluation remains focused on coding tasks.

  • Problem

    Long-horizon agents commonly rely on costly cloud frontier models for every step, despite varying capability needs and prior routing work's limited evaluation of real-world long-horizon tasks and reliance on predefined model pools.

    Source: §1 · pp. 1–2

  • Method

    EdgeWise uses a local SLM as the primary solver and delegates critical steps to a cloud LLM through external-judge or step-level RL self-delegation.

    Source: §1 · p. 2

  • Results

    Self-delegation maintains performance comparable to LLM-only while reducing Pass@3 cost by 53%/63% on SWE-ReBench and 35%/33% on Terminal-Bench 2.1 with GPT-5.5/Opus 4.8, respectively.

    Source: §4.2 · p. 7

  • Takeaways & Limitations

    Across coding benchmarks, selective delegation reduces cloud-model usage while maintaining competitive task performance, demonstrating the potential of step-level model collaboration for efficient agent systems.

    Source: §6 · p. 9

    The evaluation focuses on long-horizon coding tasks and does not test deep-research or computer-use agents; broader task settings require further experiments.

    Source: A. Limitations · p. 11

Abstract

from arXiv · show

Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon trajectory incurs substantial inference cost, even though not all steps may require the same level of model capability. In this paper, we find that allocating the LLM to only a small subset of key steps can achieve performance comparable to full LLM execution while significantly reducing inference cost. Motivated by this observation, we propose EdgeWise, an edge-cloud collaboration framework that deploys an SLM on the local device as the primary solver and selectively delegates critical steps to a cloud LLM when stronger model capacity is needed. To enable autonomous delegation without additional models at inference time, we introduce a step-level RL method that jointly optimizes task solving and delegation, with fine-grained credit assignment for individual delegation decisions. We also introduce an external judge-based delegation strategy and use it to systematically investigate key design choices in edge-cloud collaboration, including when to delegate, how delegation decisions should be made, what context should be shared, and how frequently the cloud LLM should be invoked. Experiments show that EdgeWise matches or outperforms pure-LLM execution while reducing inference cost up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. We further deploy EdgeWise on a local workstation, demonstrating comparable cost savings in a practical edge-cloud setting.

1. Introduction

EdgeWise addresses the cost of using cloud frontier models throughout long-horizon tasks by keeping an SLM local and selectively delegating critical steps. It combines external-judge and self-delegation strategies and investigates how to allocate cloud-model capacity.

1. Introduction

  • Prior multi-step routing work is described as under-evaluated on real-world long-horizon tasks and dependent on decision models tied to predefined model pools.Model capability needs can vary across trajectory steps, while simpler steps may be handled locally.Source: §1 · pp. 1–2
  • Self-delegation maintains competitive task performance while reducing inference cost by an average of 46% across models and benchmarks.The paper also reports a practical edge–cloud deployment evaluation.Source: §1 · p. 2
  • EdgeWise uses a local SLM as the primary solver and selectively delegates critical steps to a cloud LLM through external-judge or step-level RL self-delegation.The RL method jointly learns task solving and delegation with fine-grained credit assignment, without an additional decision model at inference time.Source: §1 · p. 2
  • At around 40% cloud usage, delegating the right steps achieves performance comparable to the cloud-only model while reducing cost.Figure 3 compares accuracy and average cost across cloud-model usage ratios.Source: Figure 3 · p. 3

2. Observation: Delegating Only Key Steps Matches Large-Model Performance

The section tests whether every step in a long-horizon coding task needs a large model by randomly assigning some steps to the cloud LLM. Results indicate that performance depends on where cloud capacity is allocated, not only how much is used.

2. Observation: Delegating Only Key Steps Matches Large-Model Performance

  • The experiment independently samples each step for cloud delegation while the SLM handles unselected steps and resumes execution after each cloud step.The tasks involve heterogeneous stages, including problem understanding, root-cause localization, editing, testing, and result inspection.Source: §2 · p. 3
  • At a 40% cloud-model ratio, some random allocations matched full-trajectory cloud performance while reducing inference cost by at least 25%.Performance varied substantially across random allocations at the same ratio.Source: §2 · p. 3
  • Effective collaboration depends more on where cloud-model calls are placed than on the total amount of stronger model capacity used.The observed variation across random allocations highlights the importance of placement along the trajectory.Source: §2 · p. 3

3. EdgeWise: A Framework for SLM–LLM Collaboration in Problem Solving

EdgeWise uses a local SLM as the primary solver and selectively delegates steps to a cloud LLM. The section examines which steps to delegate, who decides, what context to share, and how to control delegation frequency.

3.1. When Should the SLM Delegate to the LLM?

  • 0.200 (95% CI [0.099, 0.297]): Write/Edit cloud coverage has the strongest association with task success among action types.Successful trajectories allocate 9.4 percentage points more Write/Edit coverage on Terminal-Bench 2.1 and 5.6 points more on SWE-ReBench.Source: Figure 4 · p. 4, §3.1 · p. 4

3.2. Who Should Decide Whether to Delegate?

  • An external judge can decide whether to delegate each step, while self-delegation integrates that decision into the SLM; both target selective allocation of stronger model capacity.The judge-based mechanism reevaluates the trajectory after each step, while the self-delegation approach jointly learns task solving and delegation.Source: §3.2 · p. 4

3.2.1. Key-Step-Aware Delegation with an External Judge

  • The training-free key-step strategy uses a judge to identify critical steps dynamically and preserves most pure-cloud performance while reducing cloud-model usage.The judge uses patterns including Write/Edit and complex Read/Search operations to guide binary delegation decisions.Source: §3.2.1 · pp. 4–5, Figure 5 · p. 4

3.2.2. Training the SLM for Self-Delegation

  • Self-delegation uses SFT to teach the delegation format, then RL to jointly reward task correctness and selective cloud use with step-level credit for individual decisions.A lightweight critic evaluates decisions against key-step criteria, encouraging cloud calls on valuable steps rather than simply reducing their overall number.Source: §3.2.2 · pp. 5–6

3.3. What Context Should Be Provided for Collaboration?

  • The framework separately studies context for the delegated LLM and the delegation judge, whose information needs differ.LLM options range from full history to relevant steps; judge options include full history, full history plus current analysis, or current analysis alone, with full history adopted by default.Source: §3.3 · p. 6

3.4. How Frequently Should the LLM Be Invoked?

  • A continuous judge score and adjustable threshold provide control over how frequently the LLM is invoked.Lower thresholds increase delegation; higher thresholds leave more steps to the local SLM, allowing operation under different cloud budgets.Source: §3.4 · p. 6

4. Experiments

Across the evaluated coding benchmarks, self-delegation maintains performance comparable to LLM-only execution at lower cloud inference cost. Context choices, delegation thresholds, and local deployment also affect the performance–cost trade-off.

4.1. Experimental Setup

  • Training uses SWE-Smith with Qwen3.5-9B as SLM and Kimi-K3 as LLM, then evaluates EdgeWise-9B on Terminal-Bench 2.1 and SWE-ReBench-03 using Terminus2, limiting each trajectory to 80 turns under default benchmark settings.Training uses 2K SFT examples, followed by 175 RL steps on 40 NVIDIA A100 80GB GPUs, with batch size 8 and 16 rollouts per example.Source: §4.1 · p. 6

4.2. Main Results of Our Framework

  • GPT-5.5/Opus 4.8 self-delegation lowers Pass@3 cost 53%/63% on SWE-ReBench and 35%/33% on Terminal-Bench 2.1 while maintaining performance comparable to LLM-only.The reported pattern holds across both cloud LLMs and benchmarks.Source: §4.2 · p. 7, Table 1 · p. 7

4.3. Effect of Context Passing on SLM/LLM Collaboration

  • Passing full context from the SLM to the LLM gives the best performance but the highest inference cost; no SLM reasoning is second-best at lower cost.The authors attribute the full-context result to retaining historical information needed at critical delegated steps.Source: §4.3 · p. 7, Figure 6 · p. 8
  • Among judge context configurations, Full History performs best; including the SLM’s current analysis may bias delegation and reduce task performance while lowering LLM use and cost.The authors identify complete trajectory information as the more reliable context for delegation decisions.Source: §4.3 · pp. 7–8, Figure 6 · p. 8

4.4. Practical Edge–Cloud Collaboration

  • Thresholds around 0.2–0.4 maintain strong performance while substantially reducing inference cost versus LLM-only on the validation set and evaluated benchmarks.The validation sweep tested thresholds 0.2, 0.4, 0.6, and 0.8; the reported similar trend extends to Terminal-Bench 2.1 and SWE-ReBench with different LLMs.Source: §4.4 · p. 8, Figure 7 · p. 8
  • On an NVIDIA DGX Spark, EdgeWise achieves Pass@3 comparable to the A100-based evaluation while reducing inference cost by approximately 50% versus the vanilla LLM.The local SLM serves as primary solver and delegates steps to a cloud LLM through an API.Source: §4.4 · pp. 8–9, Figure 7 · p. 8

5. Related Works

Related work spans query-level routing, cascading, fine-grained generation routing, and multi-step agent collaboration. The paper identifies limited long-horizon evaluation and dependence on fixed model pools as remaining limitations.

5.1. Adaptive Model Collaboration

  • Routing methods range from query-level model selection to cascading based on a smaller model’s output, and to fine-grained routing during generation.Examples include conformal reliability estimation and RL-based invocation, as well as token-level or reasoning-position routing.Source: §5.1 · p. 9
  • For agentic multi-step tasks, prior work routes models across steps or agents, but existing approaches are described as limited in long-horizon evaluation and tied to predefined model pools.Fixed candidate pools can make routers difficult to generalize when underlying models are updated or replaced.Source: §5.1 · p. 9

6. Conclusion

EdgeWise uses local SLMs with selective cloud-model delegation for long-horizon tasks. The conclusion emphasizes that stronger intelligence is needed at only a small number of critical steps, and reports reduced cloud usage with competitive task performance across coding benchmarks.

6. Conclusion

  • EdgeWise selectively delegates a small number of critical steps to cloud models while keeping the remaining long-horizon task execution local.The conclusion presents this as an efficient collaboration framework for agentic coding benchmarks.Source: §6 · p. 9

A. Limitations

The evaluation is limited to long-horizon coding tasks, while the comparison of training stages attributes more effective delegation to the RL stage than to SFT alone.

A. Limitations

  • The work evaluates long-horizon coding tasks, not other agent settings such as deep research or computer-use agents.The authors say broader task extensions require further experiments.Source: A. Limitations · p. 11

B.1. Comparison between SFT model and RL model

  • SFT teaches delegation format but does not sufficiently identify when delegation is necessary; the subsequent RL stage improves critical-step allocation through fine-grained credit assignment.The comparison reports substantially lower Terminal-Bench 2.1 Pass@1 for the SFT-only model than for the final self-delegation model.Source: §B.1 · p. 11, Table 2 · p. 11

B.2. The Effectiveness of Token Advantage

The ablations and analyses examine how step-level credit assignment, local-model capability, judge choice, and training affect delegation. They report benefits from fine-grained credit assignment and show delegation can work across local models and judge choices.

B.2. The Effectiveness of Token Advantage

  • 43% of delegated steps are write/edit operations with token-level advantage, versus 29% without it, alongside improved performance and fewer average delegations.The study interprets write/edit operations as more critical and finds that the variant without token-level advantage delegates more read/search operations.Source: §B.2 · p. 11, Table 3 · p. 11

B.3. Effect of SLM Capability on Self-Delegation

  • A stronger GRPO-trained SLM that can delegate still underperforms self-delegation, indicating that increased SLM capability alone is insufficient to learn an effective delegation policy.Random placement of cloud calls also produces substantial performance variation, with some allocations comparable to the vanilla LLM using fewer cloud calls.Source: §B.3 · pp. 11–12, Table 4 · p. 12

C.1. Generalization Across Different SLMs

  • With TMax-9B as the local model, EdgeWise achieves the same Pass@1 as GPT-5.5 while reducing inference cost by 30.6%.This evaluation uses GPT-5.5 xhigh as the cloud model on Terminal-Bench.Source: §C.1 · p. 12, Table 5 · p. 12

C.2. The alteration of Judge Model

  • Qwen3.8-27B and GPT-5.4-mini achieve comparable Pass1 performance as judges, while using the SLM itself as judge performs relatively poorly.The authors note that GPT-5.4-mini costs less than Qwen3.8-27B and that the open-source Qwen model is an effective alternative to the proprietary judge.Source: §C.2 · p. 12

C.3. LLM rate in the final answer trajectory

  • The delegation rate falls after SFT, then rises and stabilizes around 40% as training progresses.The authors attribute the initial decline to the delegation penalty and the later increase to token-level advantage supervision of when delegation is beneficial.Source: §C.3 · p. 12

C.4. Generalization of Self-Delegation

The self-delegation checkpoint is evaluated on Terminal-Bench 2.1 with three harnesses unseen during training. It maintains Pass@3 performance comparable to the vanilla LLM while reducing inference costs by at least 25%.

C.4. Generalization of Self-Delegation

  • The evaluation tests transfer from Terminus2 training to OpenHands, π, and mini-SWE-agent on Terminal-Bench 2.1.The reported comparison is against the vanilla LLM using Pass@3.Source: §C.4 · pp. 12–13
  • Table 7 reports Pass@3 for different harnesses on Terminal-Bench 2.1; across all three unseen harnesses, the method reduces inference costs by at least 25% and generalizes effectively.Source: Table 7 · p. 13, §C.4 · p. 13
  • At least 25% lower inference costs accompany Pass@3 performance comparable to the vanilla LLM across three unseen harnesses.The checkpoint was trained exclusively with Terminus2 and evaluated using OpenHands, π, and mini-SWE-agent.Source: §C.4 · pp. 12–13, Table 7 · p. 13

D. Learning Curve

The learning-curve analyses track delegation-token advantage and critic agreement during training. Both trends indicate that delegation decisions become better aligned as step-level credit assignment guides the model.

D. Learning Curve

  • Delegation-token advantage starts negative, rises toward zero, and is interpreted as fewer delegation decisions needing strong corrective updates.The authors take this trend as evidence that step-level credit assignment guides the model toward more appropriate delegation decisions.Source: D. Learning Curve · p. 13
  • Critic agreement falls early in RL training, then recovers to over 90% at its peak before stabilizing below that peak.The authors connect the early drop to SFT teaching delegation format rather than when to delegate, and the later recovery to step-level credit assignment.Source: D. Learning Curve · p. 13, Figure 10 · p. 13
  • The authors interpret the advantage and agreement trends as evidence that training teaches when to delegate, rather than merely reducing cloud calls.The reported training curves use running averages.Source: D. Learning Curve · p. 13

E. Distribution of LLM-Handle Steps

Across both benchmarks and delegation strategies, the LLM is used mainly for more complex operations, particularly write/edit and complex read/search steps. Simpler steps are left to the local SLM.

E. Distribution of LLM-Handle Steps

  • Cloud calls concentrate on write/edit and complex read/search operations rather than being distributed uniformly through the trajectory.The authors associate these operations with codebase understanding, modification, cross-file exploration, and dependency tracing.Source: E. Distribution of LLM-Handle Steps · p. 14
  • The same pattern of concentrating LLM use on complex operations appears under self-delegation and binary delegation across both benchmarks.The authors connect this pattern to stronger model capabilities being more beneficial for complex reasoning and cross-file understanding.Source: E. Distribution of LLM-Handle Steps · p. 14
  • Simpler steps remain with the local SLM while the LLM is reserved for operations requiring more complex reasoning and broader repository context.The passage describes this selective allocation as identifying where stronger intelligence is most valuable.Source: E. Distribution of LLM-Handle Steps · p. 14

F. The Details of Cloud Budget in the Budget-Aware Cloud Delegation

A higher delegation threshold generally reduces cloud-model use, but it controls usage indirectly rather than setting a fixed budget. The paper also reports the token-level advantage rule and deployment runtime characteristics.

F. The Details of Cloud Budget in the Budget-Aware Cloud Delegation

  • Higher delegation thresholds consistently reduce cloud rate, making the threshold an indirect control of cloud usage rather than a fixed delegation budget.The cloud rate is the proportion of agent steps delegated to the cloud LLM; larger thresholds generally yield fewer invocations.Source: F. The Details of Cloud Budget in the… · p. 14, Figure 12 · p. 14

G.1. Details of Token-level Advantage

  • The token-level advantage is +1 when the SLM and critic both choose delegation, and −1 when their decisions disagree.The SLM's decision is d_i and the critic's decision is d̂_i.Source: §G.1 · p. 14

G.2. Details of Deploying on DGXSpark

  • On a single NVIDIA DGX Spark, Qwen3.5-9B averaged 11.19 tokens/s generation throughput, with 11.72-minute average and 8.30-minute median task completion times on Terminal-Bench 2.1.The deployment used vLLM 0.19.1 with BF16 precision and single-device inference.Source: §G.2 · p. 15

Related papers

Loading 2610.17560v1…