Source-linked AI summary
When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems
Corrado Rainone, Davide Belli, Bence Major, Arash Behboodi
TL;DR
The paper addresses how hybrid cloud–edge multi-agent systems should balance accuracy, monetary cost, and edge energy in a poorly understood design space. It adapts two representative architectures and finds that cloud assistance can benefit small models, but the best architecture depends on the task and more cloud compute does not consistently improve performance.
Problem
Hybrid cloud–edge MAS design couples task accuracy, monetary cost, and edge energy, but lacks general design principles beyond domain-specific ad hoc choices.
Method
The study adapts representative MAS architectures for hybrid inference and examines model assignment, agent roles, cloud usage, supervision, restart policies, and summarization.
Results
No single hybrid architecture dominates: plan-based and advisory designs excel in different domains, and increasing cloud usage does not consistently improve performance.
Takeaways & Limitations
Hybrid MAS design should be treated as task-dependent, with architecture and collaboration mechanisms determining how effectively SLMs benefit from LLM assistance.
Takeaways & Limitations
The study covers limited settings and fixed cloud and edge model families, while prioritizing breadth over multi-seed repetitions because large-scale hybrid evaluations are costly.
Abstract
from arXiv · showhide
The design space of agentic AI inference spans two extremes: frontier large language models (LLMs), typically hosted in the cloud and offering strong performance across a wide range of tasks at substantially high cost, and more cost-efficient small language models (SLMs), which are amenable to on-device inference. Hybrid multi-agent systems (MASs) combining on-device and cloud models offer a promising middle ground, but they also introduce a complex and poorly understood design space in which task accuracy, monetary cost, and edge energy consumption are tightly coupled; in the absence of general design principles, hybrid components, although not the most prevalent choice, are typically introduced through ad hoc decisions tailored to specific domains. In this work, we examine this design space more systematically. We adapt two representative MAS architectures to support hybrid inference and study how individual design choices shift the operating point along the Pareto frontier of power, cost, and performance. Our findings paint a nuanced picture of hybrid MAS design: while SLMs can effectively benefit from LLM assistance, the optimal architecture is highly task-dependent, and greater frontier-level compute does not consistently translate to better performance.
1. Introduction
The paper studies hybrid multi-agent systems that combine cloud and edge models to balance performance, monetary cost, and edge energy. It systematically examines architectural choices and finds task-dependent benefits from cloud assistance, context management, and planning.
- Hybrid MASs combine models with different proficiency levels and roles across cloud and edge, exposing trade-offs among performance, monetary cost, and edge energy.
- The study adapts representative multi-agent architectures to cloud–edge inference and evaluates model assignment, agent roles, and cloud-compute usage across Deep Search and UI assistance.
- No single hybrid architecture dominates: plan-based and advisory paradigms excel in different domains, while increasing cloud usage does not consistently improve performance.
- Supervision frequency, restart policies, and summarization help explain when hybridization benefits or harms long-horizon reasoning.
- Context resets and summarization limit KV-cache growth, improving hybrid systems’ suitability for memory-constrained edge deployment.
2. Related Work
Related work frames hybrid AI as a broad design space spanning architectures, models, deployment locations, and memory mechanisms. Deep-search and GUI agents are especially relevant because they require long-horizon, multi-stage interaction under practical constraints.
- Multi-agent systems vary in centralized or decentralized architecture, planning paradigms, memory mechanisms, and communication patterns, creating a combinatorial design space.
- Most existing systems are tailored to particular tasks or benchmarks, while relatively few studies systematically evaluate architectural decisions across broad task ranges.
- Hybrid AI studies how agentic pipelines allocate computation across models of different sizes and deployment locations, including routing between small and large models.
- Deep-search and GUI agents motivate hybrid designs because they combine multi-step reasoning, tool interaction, recovery, and constraints such as latency or privacy.
- Context summarization and reset are used in multi-agent systems to address context limits and degradation effects during extended interactions.
3. Hybrid Multi-Agent architectures
The paper studies two hybrid architectures that keep token-intensive execution on the edge and use intermittent cloud supervision. PEVR relies on plans and replans, whereas EVA relies on progress-based advice.
- The design aims to support device-to-frontier model scaling, on-device multi-turn execution, user control over cloud involvement, and bounded edge context.
- The architectures separate long-horizon execution from lighter planning, verification, and redirection so cloud models can supervise strategically.
- Plan–Execute–Verify–Replan (PEVR): PEVR assigns a cloud Supervisor to create an initial plan, periodically verify execution, and issue revised plans after detected misalignment or failure.
- Execute–Verify–Advise (EVA): EVA lets an on-device Executor act through a ReAct loop without an initial plan, while a cloud Supervisor provides summaries and advice when progress stalls or appears incorrect.
- Both architectures confine execution to the edge, use intermittent cloud supervision controlled by a verification interval, and reset executor context after intervention.
4. Experimental setup
The evaluation spans three benchmarks with increasing difficulty and compares task performance against cloud API cost and estimated edge energy. Experiments vary models, supervision intervals, and multi-agent architectures.
- Benchmarks: The benchmarks cover short-horizon multi-hop reasoning, long-horizon information aggregation, and stateful interactive decision making.
- Benchmarks: HotpotQA requires combining evidence from multiple Wikipedia articles and reports ROUGE1 F1 on 500 validation questions.
- Benchmarks: FanOutQA requires identifying dynamic relevant entities and aggregating information across many documents, imposing a longer reasoning horizon than HotpotQA.
- Benchmarks: AppWorld evaluates multi-step API programs in a stateful simulated application ecosystem using Test Pass Ratio and Task Success.
- Efficiency metrics: Efficiency is assessed with cloud API cost in dollars, estimated edge energy consumption in Joules, and maximum KV-cache footprint over each trajectory.
- Comparative evaluation: Figure 2 compares edge-only and cloud-only monolithic systems with PEVR and EVA across performance, API cost, and energy, varying verification intervals for MASs.
5. Exploring the design space of Hybrid MASs
Hybrid MAS configurations occupy different points on accuracy, cost, and energy trade-offs, with the best architecture and supervision policy varying by task. Across experiments, on-device execution with cloud supervision, context resets, and selective interventions provide the main benefits, while more cloud usage is not uniformly better.
- Hybrid MASs can outperform monolithic edge agents while costing less than monolithic cloud agents, placing multiple configurations on the Pareto frontier.The frontier enables task-specific tuning of accuracy against cloud API cost and edge energy.
- PEVR is stronger for AppWorld, whereas EVA is stronger for HotpotQA and FanOutQA, showing that no single architecture dominates across domains.PEVR’s detailed initial plans and replanning suit stateful UI assistance, while query-based advice better fits deep-search tasks.
- Increasing cloud supervision does not consistently improve performance because overly frequent interventions and restarts can degrade long-horizon reasoning.The best verification interval depends on both the task domain and the Executor restart policy.
- Both PEVR and EVA retain or improve monolithic-edge accuracy while reducing edge energy consumption by up to 3× through Executor context resets.EVA generally saves more energy than PEVR because summarization limits context growth more effectively.
- Execution on the device with cloud-based supervision is more effective than cloud execution with an on-device Supervisor for both task accuracy and cloud subscription cost.The comparison uses PEVR for AppWorld and EVA for FanOutQA at their best verification intervals.
- Hybrid MASs solve some tasks that neither monolithic edge nor cloud systems solve independently, indicating that their behavior is not equivalent to simple edge–cloud routing.The result motivates dynamically selecting an agentic configuration for each task.
- Intermediate verification and replanning keep KV-cache growth modest while improving task success on long-horizon AppWorld workloads.Monolithic systems encounter out-of-context errors in approximately 20% of runs at 80 turns, whereas MAS context resets prevent unbounded accumulation.
6. Limitations
The study’s conclusions are bounded by the evaluated domains, fixed model set, and limited multi-seed repetition. Broader settings, model families, and repeated runs could strengthen generality and statistical rigor.
- The evaluation covers multiple domains but not settings such as robotics or coding agents, limiting the generality of its conclusions.The authors identify additional settings as a way to strengthen generality.
- The experiments rely on a fixed set of cloud and edge models, so broader model families could reveal further interactions with architectural choices.
- High computational and monetary evaluation costs led the study to prioritize breadth over multi-seed repetitions, limiting statistical rigor.The authors note that consistent trends across tasks and configurations support robustness despite this limitation.
7. Conclusions
The paper evaluates cloud agents collaborating with device agents across Deep Search and UI assistance. Collaboration can help even with device-sized edge models, but excessive cloud supervision can hurt, and the better architecture depends on the domain.
- The study evaluates two cloud–device Multi-Agent architectures across Deep Search and UI assistance, considering both performance and efficiency.
- Cloud–edge collaboration can be beneficial even when the edge model is device-sized.
- Excessive supervision and intervention from an on-cloud model can be detrimental.
- No single MAS architecture performs best overall: each of the two variants performs best on one domain rather than the other.
- Dynamically adapting the hybrid architecture to task difficulty and requirements is identified as a future direction.
A. Cost and Efficiency Metrics
The appendix estimates on-device inference energy by decomposing each round into prefill and decode, then relating operations to model size, token counts, and hardware efficiency. The model is interpretable rather than cycle-accurate, and its example yields an idealized upper bound on feasible rounds.
- Inference decomposition: The energy model decomposes one on-device inference round into prefill and autoregressive decode stages.Prefill processes the input prompt, while decode generates tokens.
- Operation scaling: Total inference operations depend on model parameters, prompt tokens, and generated tokens.These quantities are denoted N, n_p, and n_d, respectively.
- Operation scaling: For dense Transformers, operations per token scale linearly with parameter count when sparsity, recomputation, and fused kernels are ignored.The approximation accounts for one forward pass through all parameter tensors.
- Hardware efficiency: Inference energy is obtained by relating total operations to hardware efficiency measured in operations per joule.η denotes the hardware efficiency.
- Hardware efficiency: Thermal constraints and memory effects push achieved mobile-NPU efficiency toward the lower end of the reported range.
- Illustrative estimate: Thousands of 4 × 10^9-parameter inference rounds are energetically feasible under idealized conditions, but the estimate is an upper bound because of system-level energy costs.
A.1.5. LIMITATIONS
The energy model omits several system-level contributors, so Equation (4) provides a lower-bound estimate suitable for comparison rather than precise measurement.
- A.1.5. LIMITATIONS: The model omits DRAM access and refresh energy, which can dominate decoding because of KV-cache traffic.
- A.1.5. LIMITATIONS: The model excludes CPU-side overheads such as tokenization, runtime, and the driver stack.
- A.1.5. LIMITATIONS: Display power, thermal management, and background system power are also excluded from the estimate.
- A.1.5. LIMITATIONS: The model does not account for deviations from peak Op/J under sustained workloads.
- A.1.5. LIMITATIONS: Equation (4) is a lower bound on end-to-end system energy, intended for order-of-magnitude reasoning and comparative analysis.It should not be interpreted as a precise measurement.
- A.1.5. LIMITATIONS: KV-cache growth affects both on-device memory footprint and inference feasibility as context length increases.The cache is dynamically provisioned according to task requirements and can significantly affect total memory footprint.
B. Ablating Summarization from EVA
The EVA summarization ablation compares restart behavior across architectures and finds that removing summarization does not significantly change performance, implicating verification criteria or restart prompts instead.
- B. Ablating Summarization from EVA: PEVR preserves a log of tool calls and outputs after restart, whereas EVA condenses the ReAct trajectory through summarization.
- B. Ablating Summarization from EVA: Removing summarization from EVA does not significantly affect performance on FanOutQA.The ablation uses Qwen3-14B as Executor and compares PEVR, EVA, and EVA without summarization.
- B. Ablating Summarization from EVA: The PEVR Supervisor generates a self-contained plan while remaining prohibited from executing actions or calling tools.A separate Executor consumes the plan verbatim.
- B. Ablating Summarization from EVA: The EVA Executor directly handles the query in a reason–act–observe loop without an explicit planning phase.It may call at most one tool per turn and must use the finish tool when complete.
- B. Ablating Summarization from EVA: The PEVR Executor follows the Supervisor’s plan sequentially and cannot modify, reorder, skip, or add plan steps.
- B. Ablating Summarization from EVA: After intervention, EVA resumes from a clean context using a Supervisor-provided summary and advice, while PEVR resumes from a new plan.Both restart prompts restrict the Executor to Supervisor-provided guidance rather than independent replanning.
D.1. Benchmarks
The evaluation spans progressively harder agentic settings, from short-horizon multi-hop retrieval to long-horizon, stateful interaction across simulated applications.
- D.1. Benchmarks: The benchmark suite comprises HotpotQA, FanOutQA, and AppWorld, covering increasing interaction horizons and action spaces.Together they stress-test planning, memory, tool use, and recovery from intermediate errors.
- D.1. Benchmarks: HotpotQA evaluates multi-hop retrieval and synthesis from at least two Wikipedia articles, making it a relatively short-horizon task.The main reported performance metric is F1 on the first 500 validation examples.
- D.1. Benchmarks: FanOutQA requires identifying relevant entities and aggregating information across many dynamically discovered documents.It induces a longer reasoning horizon and uses F1 as the main performance metric on 310 development tasks.
- D.1. Benchmarks: AppWorld evaluates multi-step programs with control flow, memory, and tool use in a realistic simulated application ecosystem.Its tasks involve dozens of turns and are assessed through state-based execution traces and unit tests.
- D.1. Benchmarks: AppWorld agents interact through executable Python code in a persistent sandbox, where variables and objects remain available across steps.The environment exposes application functionality through Python APIs and requires code blocks enclosed in <code>...</code> tags.
- D.1. Benchmarks: Because AppWorld exposes hundreds of APIs, agents use API-description and API-documentation tools to discover available operations and their parameters.
E.1. PEVR replans vs EVA advices in AppWorld
In AppWorld, PEVR produces detailed, actionable replans, whereas EVA provides more general advice that lacks a concrete path for the Executor.
- E.1. PEVR replans vs EVA advices in AppWorld: PEVR’s AppWorld replan is more actionable and detailed than EVA’s generalist advice.The comparison is reported qualitatively for representative AppWorld examples.
- E.1. PEVR replans vs EVA advices in AppWorld: The PEVR replan specifies a sequential path from discovering APIs through retrieving cart details and calculating total cost.It includes expected inputs and outputs for intermediate steps, including credentials, authentication, and cart retrieval.
- E.1. PEVR replans vs EVA advices in AppWorld: EVA advice identifies pitfalls but fails to provide a concrete path with next steps for the Executor.