When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems

Motivation

• Cloud-hosted frontier models can incur significant operational costs on iterative agent workloads, while on-device SLMs offer a self-hosted option but retain capability and context-length constraints.On-device models can help avoid API costs and address privacy or data-sovereignty concerns; device DRAM limits KV-cache and context length.Source: §1 · p. 1

Systematic study of hybrid MAS design

• The study adapts representative multi-agent architectures to cloud–edge settings and introduces a framework for evaluating accuracy, monetary cost, and edge energy trade-offs.It explores model assignment, agent roles, and cloud-compute levels.Source: §1 · p. 2

Task-dependent behavior

• No single hybrid design dominates: plan-based and advisory paradigms excel in different domains, and greater cloud usage does not consistently improve performance.The systems are evaluated on tasks involving Deep Search and UI assistance.Source: §1 · p. 2

Mechanistic insights into hybrid collaboration

• Supervision frequency, restart policies, and summarization help explain when hybridization helps or harms long-horizon reasoning.Source: §1 · p. 2

Context Efficiency of hybrid MAS

• Context resets and summarization limit KV-cache growth, helping hybrid MAS handle long contexts and better suit memory-constrained edge deployment.Source: §1 · p. 2

Multi-Agent systems

• MAS designs span centralized and decentralized architectures intertwined with planning and memory, while task-specific systems leave limited actionable guidance on architectural trade-offs (Guo et al., 2024; Chen et al., 2024; Tran et al., 2025; Aratchige & Ilmini, 2025; Zhang et al., 2024b; Wang et al., 2024; Nayak et al., 2024; Yang et al., 2025; Zhu et al., 2025; Zhang et al., 2024a; Bogavelli et al., 2025; Zeng et al., 2025; Han et al., 2024; Cemri et al., 2025; Kim et al., 2025; Wynn et al., 2025).Architectural choices include orchestrated star or tree topologies and decentralized mesh- or graph-like interactions; failure modes also depend on task decomposition, communication, and verification.Source: §2 · p. 2

Hybrid AI

• Hybrid AI work explores allocating agentic computation across model sizes and deployment locations, including SLM orchestration, fine-tuned small models, and request routing (Belcak et al., 2025; Su et al., 2025; Jhandi et al., 2026; Ding et al., 2024; Ong et al., 2024; Jitkrittum et al., 2025).Examples include an 8B orchestrator and a targeted fine-tuned 350M model reported to perform strongly on ToolBench-style evaluation.Source: §2 · p. 2
• Deep-search and GUI agents motivate hybrid designs; MAI-UI uses a cloud model to recover from failures of an on-device model (Tongyi DeepResearch Team et al., 2025; Venus Team et al., 2026; Zhou et al., 2025; Veuns-Team et al., 2026; Wang et al., 2025).Deep search interleaves decomposition, retrieval, browsing, evidence aggregation, and synthesis; GUI agents combine perception, grounding, action selection, and multi-step recovery under latency and privacy constraints.Source: §2 · pp. 2–3

Context summarization and reset

• MASs use context summarization and reset to address context limits and degradation, typically compressing histories into task state and integrating summaries with replanning or executor resets (Sinha et al., 2025; Liu et al., 2024b; Hong et al., 2025; Nadar Ponnusamy et al., 2025; Weng, 2023; Du, 2026; Lu et al., 2025; Li et al., 2025b; Qin et al., 2025).AgentFlow retains tool-use trajectories while pruning verbose reasoning, whereas advisor-style frameworks reset executor context with corrective summaries.Source: §2 · p. 3

Design desiderata

• The design targets heterogeneous model capabilities, lower cloud API cost through on-device multi-turn execution, user-tunable cloud involvement, and bounded edge context for DRAM and KV-cache constraints.The study adapts collaboration, orchestration, role specialization, and tool use to hybrid inference without introducing new MAS abstractions.Source: §3 · p. 3

Orchestration

• Orchestration separates long-horizon, token-heavy execution from lighter planning, verification, and redirection, letting edge models execute while cloud models supervise strategically.The section studies two representative architectures, PEVR and EVA.Source: §3 · p. 4

Plan–Execute–Verify–Replan (PEVR)

• PEVR pairs a cloud Supervisor’s initial plan and periodic plan-alignment checks with an on-device Executor, which receives revised plans when execution fails or drifts.The Supervisor defines both the initial strategy and later course corrections.Source: §3 · p. 4

Execute–Verify–Advise (EVA)

• EVA starts with an on-device Executor acting directly on the query in a ReAct-style loop, while a cloud Supervisor periodically provides action summaries and advisory feedback when progress stalls or seems incorrect.Unlike PEVR, supervision responds to observed progress and advises rather than issuing an explicit revised plan.Source: §3 · p. 4

Shared design

• Both architectures keep token-intensive execution on-device and use intermittent cloud supervision, with a user-set verification interval controlling supervision frequency and cloud cost.When the Supervisor intervenes, the Executor’s history is replaced by a fresh prompt derived from a plan or summary, bounding context and KV-cache use while enabling recovery.Source: §3 · p. 4

4.1. Benchmarks

• HotpotQA, FanOutQA, and AppWorld span short-horizon multi-hop reasoning, long-horizon information aggregation, and stateful interactive decision making.HotpotQA uses ROUGE1 F1 on the first 500 questions of the fullwiki validation split; FanOutQA uses ROUGE1; AppWorld reports Test Pass Ratio and Task Success (all tests passing).Source: §4.1 · p. 4

4.2. Efficiency metrics

• Efficiency is measured through cloud API cost, estimated edge-inference energy, and maximum KV-cache footprint over each task trajectory.Multi-turn trajectory costs sum inference costs at each turn; edge energy is estimated with a model rather than measured through on-device deployment.Source: §4.2 · pp. 4–5

4.3. MAS backbones and hyper-parameters

• GPT-4o is the cloud agent, while Qwen3 4B, 8B, 14B, and 32B variants serve as edge models.The 4B and 8B variants can run on smartphones, and the 14B and 32B variants fit on consumer laptops; 32B uses fp8-quantized KV-cache and weights to run on one A100 GPU.Source: §4.3 · p. 5
• Maximum ReAct turns are 10 for HotpotQA, 20 for FanOutQA, and 40 for AppWorld, with verification intervals swept across benchmark-specific step counts.Intervals are [1,2,3,5] for HotpotQA, [1,2,3,5,10] for FanOutQA, and [1,2,4,8,16] for AppWorld.Source: §4.3 · p. 5

5. Exploring the design space of Hybrid MASs

• Hybrid MAS configurations occupy a Pareto frontier: PEVR and EVA retain or improve monolithic-edge accuracy while reducing edge energy by up to 3×.EVA tends to save more energy than PEVR because it summarizes context; multiple configurations allow accuracy–cost trade-offs.Source: §5 · pp. 5–6
• PEVR outperforms EVA on AppWorld, whereas EVA outperforms PEVR on HotpotQA and FanOutQA; performance can also degrade beyond an optimal verification interval, while larger Executors correlate with better performance.Increasing cloud compute usage does not necessarily improve performance.Source: §5 · p. 6
• On AppWorld, PEVR’s detailed initial plans and actionable replanning outperform EVA’s advisory feedback, including when there are zero Supervisor interventions.The paper attributes this advantage to plans that guide tool calls and control flow, especially in stateful environments where early mistakes can have irreversible effects.Source: §5 · p. 6, Figure 3 · p. 6
• On FanOutQA and HotpotQA, EVA’s summarization and lightweight advice outperform PEVR’s frequent replanning, which can harm long-horizon search.On FanOutQA, performance drops sharply as restarts increase; PEVR’s Supervisor often restarts an Executor multiple times in one trajectory, unlike EVA.Source: §5 · pp. 6–7, Figure 3 · p. 6
• PEVR intervenes more aggressively than EVA, and the best verification policy depends on the task: aggressive supervision can suit plan-driven stateful environments, while conservative verification better protects deep search.The comparison measures false positives and false negatives with Qwen3-32B as Executor and a fixed three-step verification interval.Source: Table 1 · p. 6, §5 · p. 7
• Offloading execution to the cloud while keeping supervision on device is less accurate and more expensive than a monolithic cloud agent; on-device execution with cloud supervision is the more effective hybrid setup.The latter setup offers a way to trade performance against cloud API cost.Source: §5 · pp. 7–8, Table 2 · p. 7

A MAS is different from the sum of its parts

• Across benchmarks, MASs solve some tasks that neither their edge nor cloud components solve alone, while the union of systems strictly dominates any single fixed configuration.This indicates that hybrid MASs do more than route each query to an edge or cloud model based on predicted difficulty, and suggests dynamically selecting an agentic configuration for each task.Source: A MAS is different from the sum of its… · p. 8, Figure 4 · p. 8
• In long-horizon AppWorld tasks, MASs improve task success while limiting KV-cache growth through intermediate verification and replanning that reset and summarize context.Monolithic systems encounter out-of-context errors in about 20% of runs at a maximum of 80 turns; estimated GPTQ weight-and-activation footprints are at most 6 GB for Qwen-3 8B and 16 GB for Qwen-32B.Source: A MAS is different from the sum of its… · p. 8, Table 3 · p. 8

6. Limitations

• Conclusions may not generalize beyond evaluated domains or fixed cloud and edge models, since robotics, coding agents, and broader model families remain unexplored.Broader model families could reveal further interactions with architectural choices.Source: §6 · pp. 8–9
• High evaluation costs led the study to prioritize breadth over multi-seed repetitions, limiting statistical rigor despite consistent trends across tasks and configurations.Source: §6 · p. 9

• Cloud–edge collaboration can be beneficial with device-sized edge models, while excessive supervision and intervention from the cloud model can be detrimental.Source: §7 · p. 9
• Neither tested hybrid multi-agent architecture performs best overall: each variant performs best in one of the two domains, Deep Search and UI assistance.Source: §7 · p. 9
• Dynamically adapting a hybrid multi-agent architecture to task difficulty and requirements is identified as a direction for future work.Source: §7 · p. 9

A.1. Energy consumption model

• The appendix models one on-device LLM inference round with an interpretable estimate of dominant scaling by model size, sequence length, and hardware efficiency, not cycle-accurate prediction.Source: §A.1 · p. 13

A.1.1. INFERENCE DECOMPOSITION

• Inference energy is decomposed into prompt prefill and autoregressive token decode stages.Source: §A.1.1 · p. 13

A.1.2. OPERATION COUNT

• For dense Transformers, operations per token scale linearly with parameter count; the approximation ignores sparsity, recomputation, and fused kernels and counts one forward pass through parameter tensors.Source: §A.1.2 · p. 13

A.1.3. HARDWARE EFFICIENCY AND ENERGY

• Hardware efficiency is measured in Ops/J, and reported efficiencies for quantized mobile-NPU inference are bounded by throughput and sustained power, with thermal and memory effects pushing practice toward the range’s lower end.Source: §A.1.3 · p. 13

A.1.4. NUMERICAL EXAMPLE

• The numerical example assumes N = 4 × 10^9 parameters, np = 1000 prompt tokens, nd = 200 generated tokens, and η = 1.5 × 10^12 Ops/J.Source: §A.1.4 · p. 13
• Thousands of example inference rounds are energetically feasible within laptop or smartphone battery budgets under idealized conditions, but the estimate is an upper bound because system-level energy costs are excluded.The example uses a 4 × 10^9-parameter model, a 1000-token prompt, 200 generated tokens, and η = 1.5 × 10^12 Ops/J.Source: §A.1.4 · p. 13

A.1.5. LIMITATIONS

• The energy model omits DRAM access and refresh, CPU overhead, display and thermal management, background power, and sustained-workload deviations from peak Op/J.Source: §A.1.5 · p. 14
• Equation (4) is a lower bound on end-to-end system energy, intended for order-of-magnitude reasoning and comparative analysis rather than precise measurement.Source: §A.1.5 · p. 14

A.2. Cloud Subscription Costs

• Azure OpenAI GPT-4o standard on-demand costs are 2.5$ / 1M prefill tokens, 1.25$ / 1M cached tokens, and 10$ / 1M generated tokens.Token counts are obtained from API outputs.Source: §A.2 · p. 14

A.3. KV-cache size estimation

• When LLM footprint exceeds device DRAM, storage transfers can substantially degrade inference; footprint also affects energy and user experience (Alizadeh et al., 2024).Source: §A.3 · p. 14
• Footprint depends on parameters, context, and quantization; KV-cache grows linearly with context, and Qwen 3 4B's max-context cache is approximately 4.5 GB versus about 2 GB weights alone.The example uses INT4 weights and BF16 activations.Source: §A.3 · p. 14
• Per-sample peak KV-cache is estimated from maximum trajectory context using MKV = 2 * L * H_KV * d_h * b_act * C, with BF16 cached activations set to 2 bytes.L is transformer layers, H_KV key-value heads, d_h per-head dimension, C context length; the factor of 2 accounts for keys and values, and the maximum estimated cache over inference steps is reported.Source: §A.3 · p. 14

B. Ablating Summarization from EVA

• Removing summarization does not significantly affect EVA’s FanOutQA performance, suggesting its gap from PEVR stems from verification criteria or restarting prompts.EVA summarizes past context after a restart, whereas PEVR retains a log of tool calls and outputs; removing summarization makes EVA’s context representation match PEVR’s.Source: B. Ablating Summarization from EVA · pp. 14–15, Figure 5 · p. 15

C.1. Planning

• In PEVR, the planning-only Supervisor creates a clear, self-contained multi-step tool-use plan for a separate Executor and must not act or answer the query.The plan specifies what to do, in what order, and why, and is consumed verbatim by the Executor.Source: §C.1 · p. 15

C.2.1. DIRECT EXECUTION

• In direct execution, the Executor alone decides which tools to call and when to finish, following a reason–act–observe loop without an explicit Planner.It may call at most one provided tool per turn, must not fabricate tool-obtainable information, and must use the finish tool to terminate.Source: §C.2.1 · p. 16

C.2.2. PLAN-BASED EXECUTION

• In PEVR plan-based execution, the Executor must follow the Supervisor’s plan sequentially, without planning, reinterpretation, or strategy changes.The plan is authoritative, while the user query provides context and cannot override it.Source: §C.2.2 · p. 17

C.2.3. ADVICE-BASED EXECUTION RESUMPTION

• After EVA Supervisor intervention, the restarted Executor continues from the supplied summary and binding advice, without replanning or revisiting completed actions.The reset context relies exclusively on the Supervisor’s handoff, with the summary treated as factual.Source: §C.2.3 · p. 18

C.2.4. REPLAN-BASED EXECUTION RESUMPTION

• After PEVR intervention, the restarted Executor follows a new Supervisor plan and uses the memory log only as a record of completed tool calls and outcomes.The replan is binding, and completed steps are not repeated unless the replan instructs otherwise.Source: §C.2.4 · p. 19

C.3.1. PLAN-BASED VERIFICATION AND CONDITIONAL REPLANNING PROMPT

• PEVR verification checks execution against the authoritative plan and emits CONTINUE or INTERVENE, generating a self-contained replacement plan only when intervening.Any replacement assumes a full Executor context reset and includes only remaining steps, based on the observed memory log.Source: §C.3.1 · p. 20

C.3.2. QUERY-BASED VERIFICATION AND CONDITIONAL ADVICE PROMPT

• EVA verification judges query-driven execution from observable evidence and emits CONTINUE or INTERVENE, providing a reset-safe summary and actionable advice only when intervening.The handoff is self-contained and covers completed work and remaining steps, without relying on prior conversational state.Source: §C.3.2 · p. 21

D.1. Benchmarks

• Across progressively increasing interaction horizons and action spaces, HotpotQA, FanOutQA, and AppWorld span a spectrum from short-horizon information retrieval to long-horizon, multi-step interactive decision making.This progression stress-tests planning, memory, tool usage, and recovery from intermediate errors.Source: §D.1 · p. 22
• HotpotQA and FanOutQA are evaluated using F1, on 500 fullwiki validation tasks and 310 development tasks, respectively.HotpotQA also reports Exact Match; FanOutQA supports Exact Match, ROUGE, and BLEURT, but F1 is the main metric for both.Source: §D.1 · pp. 22–23

D.2. AppWorld Environment

• AppWorld tasks require multi-step programs across everyday applications, with challenge tasks emphasizing deeper dependencies and stateful reasoning across applications.The benchmark has 750 tasks and evaluates execution traces and unit tests for task success and unintended state changes; development tasks support sweeps and ablations, while test-challenge supports final error analysis and qualitative results.Source: §D.2 · p. 23, §D.1 · p. 23

D.2.1. ENVIRONMENT OVERVIEW AND TOOLS

• In AppWorld, agents generate Python code executed in a persistent, sandboxed interpreter, calling application APIs and printing outputs to observe tool results.Variables and objects persist across steps, enabling incremental program construction and memory through code state.Source: §D.2.1 · p. 23

D.2.2. TOOL PROMPT

• The AppWorld prompt requires agents to call the supervisor completion API to finish a task.Some tasks are evaluated by environment state, the answer string, or both; the prompt says this is the only way to complete a task.Source: §D.2.2 · p. 25

D.3. Wikipedia Environment

• The Wikipedia environment adapts Wikienv with improved parsing of tables and sideboxes, including biographical data.It provides a lightweight interactive environment for multi-step reasoning with Wikipedia.Source: §D.3 · p. 27

D.3.1. ENVIRONMENT OVERVIEW

• Wikipedia interaction maintains the currently referenced page as implicit state, so agents can continue lookup without repeating full queries.Episodes end only when the finish action is called.Source: §D.3.1 · p. 27

D.3.2. AVAILABLE TOOLS

• The Wikipedia environment offers three tools: search retrieves entity introductions, lookup finds a matching sentence on the current page, and finish returns the answer.Agents may call only one tool per turn, and must invoke tools explicitly by name.Source: §D.3.2 · p. 27

E.1. PEVR replans vs EVA advices in AppWorld

• PEVR’s AppWorld replan is detailed and actionable, whereas EVA’s advice is general and does not give the executor concrete next steps.The figures characterize EVA’s advice as synthetic: it offers some useful pitfalls to avoid but no concrete path forward.Source: §E.1 · p. 28, Figure 15 · p. 29, Figure 16 · p. 29
• The replan directs the executor to discover Amazon APIs, inspect the cart API documentation, retrieve account credentials, and log in.The listed steps specify expected outputs, including API descriptions and documentation, credentials, and an access token for subsequent calls.Source: §E.1 · p. 28

F. Hardware

• Experiments used NVIDIA A100 GPUs, relying on a single GPU; cloud GPT-4o runs used Azure OpenAI’s standard subscription service.The paper refers to Section 4.3 for these experimental details.Source: F. Hardware · p. 30
• Individual experiments took 1–2 hours at the shortest, while longer AppWorld test-set experiments with Qwen 3 32B took up to 1–2 days.Source: F. Hardware · p. 30

G. Societal Impact

• The authors identify a possible positive societal impact: improving LLM-agent consumers’ experience and monetary costs through trade-offs between cloud subscription costs and edge-device power consumption.They state that their research introduces no novel methods and considers no specific applications with potential positive or negative societal impact.Source: G. Societal Impact · p. 30

H. LLM Usage

• LLMs supported the initial literature review and writing parts of the Python codebase, and multiple expert human engineers peer-reviewed the code before experimentation.Source: H. LLM Usage · p. 30