PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton

arXiv:2605.08626v1eess.SPcs.DCcs.LGcs.MA

TL;DR

Cloud-only and on-device LLMs each face resource or capability constraints, motivating collaboration across networked endpoints. This article organizes task-level collaborative inference into device-cloud and multi-agent approaches, and concludes that routing, cooperative learning, and trust mechanisms are needed to achieve quality-latency-cost tradeoffs beyond individual models.

  • Problem

    No single LLM endpoint can satisfy applications’ diverse computation, memory, connectivity, latency, privacy, and inference-volume constraints.

  • Method

    The article synthesizes task-level collaboration among independent LLMs through vertical device-cloud offloading, horizontal multi-agent cooperation, and hybrid topologies.

  • Results

    Collaborative inference can achieve quality-latency-cost tradeoffs beyond individual models, while learned routing improves offloading and sparse communication can reduce token costs without sacrificing performance.

  • Takeaways & Limitations

    Realizing collaborative intelligence at scale requires adaptive protocols, routing and cooperative training, plus mechanisms for trust, privacy, and safety.

  • Takeaways & Limitations

    Collaborative inference moves prompts and intermediate results across trust boundaries, creating unresolved privacy and cascading prompt-injection risks.

Abstract

from arXiv · show

Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.

I. INTRODUCTION

LLMs offer broad capabilities but depend on resource-rich cloud infrastructure, while compact on-device models face capability gaps on complex tasks. Collaborative intelligence addresses this tension by coordinating complete LLMs across device and cloud endpoints through task-level semantic exchanges.

  • Motivation: LLMs support question answering, code generation, and multimodal reasoning across applications including smartphones, autonomous vehicles, and robotic systems.These capabilities rely on substantial computational and memory resources, with frontier models reaching hundreds of billions to trillions of parameters.
  • Motivation: Compact on-device LLMs improve smartphone feasibility but retain a significant capability gap relative to cloud frontier LLMs on complex tasks.This gap motivates collaboration across network tiers rather than relying on a single endpoint.
  • Collaborative Intelligence: Collaborative intelligence coordinates multiple independent, complete LLM instances across device and cloud endpoints through natural-language or structured semantic messages.Nodes exchange queries, responses, and task descriptions rather than model parameters or intermediate tensors, enabling flexible composition of specialized capabilities.
  • Collaborative Intelligence: The paradigm supports applications such as smartphone offloading, vehicle-to-home coordination, and cloud orchestration of UAV inspections without sharing LLM weights or internal states.These examples use natural language or structured commands and task descriptions across networked endpoints.
  • Article Scope: The article frames collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration.It presents these dimensions as part of a broader resource landscape spanning computation, memory, communication, and cost.

II. THE LANDSCAPE OF NETWORKED LLMS: CHALLENGES AND OPPORTUNITIES

Networked LLM deployments span lightweight on-device models and high-quality cloud frontier models, with computation, memory, communication, and cost creating heterogeneous constraints. Because no endpoint dominates across performance dimensions, task-level collaboration offers an opportunity to balance quality, latency, and cost, but requires continuous multi-objective routing.

  • Resource spectrum: On-device LLMs offer low latency and no per-query fees, whereas cloud frontier LLMs provide state-of-the-art quality but require connectivity and incur usage charges.The ecosystem ranges from models with a few billion parameters on smartphones and edge devices to cloud models with hundreds of billions of parameters.
  • Resource constraints: Deployment constraints fall into computation, memory, communication, and cost, with device-side limits restricting model size, context length, and sustained or offloaded inference.Energy and connectivity also bound device operation, while limited compute and memory constrain local deployment.
  • Heterogeneous endpoints: Binding constraints vary across endpoints: smartphones are chiefly limited by computation and memory, while UAVs face simultaneous scarcity across resources.Smartphone quantization can limit local inference quality and context length; UAV inputs also include aerial imagery and flight telemetry.
  • Heterogeneous endpoints: No single endpoint dominates throughput, energy efficiency, latency-quality, and price-quality tradeoffs across network tiers.The performance-resource tradeoffs show that the Pareto frontier shifts with the binding resource and objective.
  • Collaboration opportunity: Task-level collaboration between device and cloud LLMs could improve the quality-latency-cost balance, but routing must continuously navigate multiple competing objectives.The routing challenge extends beyond optimizing any single objective and must be automated.

III. COLLABORATIVE INFERENCE ARCHITECTURE

Collaborative inference uses two complementary architectures: vertical device-cloud collaboration and horizontal multi-agent collaboration. These topologies can also be composed into hybrid systems, with MCP providing a uniform substrate for exchanges while preserving task-level black-box interaction.

  • Vertical collaboration lets on-device LLMs offload difficult tasks to more capable cloud LLMs.
  • Horizontal collaboration enables peer LLM agents to cooperate on tasks collectively.
  • The two topologies can form hybrid systems, while MCP provides a uniform exchange substrate without changing black-box, task-level interaction.

A. Device-Cloud Collaboration

Device-cloud collaboration combines lightweight on-device LLMs with powerful cloud LLMs because neither endpoint alone suffices across task complexity and resource constraints. Effective collaboration requires routing requests among heterogeneous endpoints while managing accumulated context across endpoint switches.

  • Device-Cloud Collaboration: On-device LLMs handle requests within their capabilities, while more complex requests are forwarded to powerful cloud LLMs.This division addresses limited on-device capability and cloud unavailability or infeasibility under resource constraints.
  • Joint LLM and Modality Selection: Routing must match request difficulty and domain with endpoint capability, latency, cost, network conditions, and server load.Factual lookup may suit a small on-device LLM, whereas multi-step reasoning may require a cloud frontier LLM.
  • Context Management across Turns: Multi-turn conversations and multi-step agentic workflows require transferring accumulated dialogue history or action traces when successive turns use different endpoints.Context accumulates over time and must accompany the new query during endpoint switching.
  • Context Management across Turns: A ten-turn conversation can accumulate thousands of tokens, making repeated context retransmission costly in uplink bandwidth, cloud-side prompt tokens, and monetary expense.Agentic workflows can grow faster because they log intermediate observations and tool outputs.

B. Multi-Agent Collaboration

Multi-agent collaboration enables independent LLM instances to coordinate horizontally on decomposable tasks or tasks benefiting from diverse perspectives. Its patterns and communication topologies trade off flexibility, response quality, latency, information sharing, and overhead.

  • Multi-Agent Collaboration: Independent LLM agents collaborate horizontally through message passing when tasks decompose into parallel subtasks or benefit from diverse perspectives.Agents may operate on devices, in the cloud, or across both tiers.
  • Collaboration Patterns: Collaboration patterns vary by interaction, execution, and coordination dimensions, including debate, division-of-labor, hierarchical, parallel, sequential, centralized, and decentralized structures.These dimensions carry different implications for response quality, latency, and communication.
  • Collaboration Patterns: Debate and division-of-labor suit comparable-capability peers, whereas hierarchical workflows assign a privileged coordinator and can mix cloud supervisors with on-device workers.These patterns are agnostic to agent placement across devices and cloud tiers.
  • Communication Topology and Overhead: Fully connected topologies maximize information sharing but scale quadratically in message volume; star, relay, and tree topologies reduce or balance communication at different coordination costs.A star topology reduces per-agent communication but creates a bottleneck, while relay or tree structures balance information sharing and overhead.

IV. LEARNING TO COLLABORATE

This section examines how collaboration strategies are learned through two objectives: routing requests among device-cloud endpoints and enabling effective cooperation among multi-agent LLMs.

  • Learning objectives: Learning to collaborate has two objectives: routing each request to an endpoint in device-cloud settings and teaching LLMs to work together in multi-agent settings.These objectives mirror the paper’s vertical device-cloud and horizontal multi-agent collaboration dimensions.

A. Routing Policy Learning

Routing policy learning replaces hand-crafted device-cloud decisions with policies that balance response quality, latency, and cost. Approaches include classifier-based candidate selection and self-routing through post-training, with self-routing yielding more accurate offloading.

  • A. Routing Policy Learning: Routing policies learn whether each request should be processed locally or in the cloud while balancing quality, latency, and cost.This replaces hand-crafted routing rules with learned decisions.
  • Router-Based Selection: Classifier-based routers profile candidate LLMs offline, then predict which model offers the best tradeoff for each incoming query.Profiles include quality scores, average latency, and per-token cost on representative benchmarks.
  • Router-Based Selection: The classifier objective maximizes expected response quality subject to an average cost or latency budget.A compact model such as fine-tuned BERT can perform the routing prediction.
  • Self-Routing via Post-Training: Self-routing post-trains the on-device LLM with reinforcement learning to attempt tasks locally, inspect its reasoning, and decide whether to escalate.The reward combines a quality bonus for correct local answers with a cost penalty for escalation.
  • Self-Routing via Post-Training: Self-routing yields substantially more accurate offloading than classifier-based routers because decisions follow task engagement rather than prompt surface features alone.The routing decision occurs after the on-device LLM has already engaged with the problem.

B. Cooperative Capability Learning

Cooperative capability learning moves beyond static role prompts by explicitly training LLMs to collaborate and optimizing how agents communicate under resource constraints. Sparse communication topologies can reduce token costs while matching or surpassing fully connected debate.

  • Cooperative Policy Optimization: Prompt-only collaboration is limited because independently pretrained LLMs lack genuine multi-party interaction and may improvise ineffective strategies.Static prompts can produce repetitive exchanges that fail to converge or degrade performance over additional rounds.
  • Cooperative Policy Optimization: Cooperative policy optimization places multiple LLMs in authentic collaborative scenarios and trains their interaction behavior explicitly.This approach addresses the limitations of assigning roles and instructions without cooperative training.
  • Inter-Agent Network Optimization: Sparse communication topologies replace fully connected graphs so each agent observes only a subset of peers’ responses in each round.This addresses communication efficiency under bandwidth and computational constraints.
  • Inter-Agent Network Optimization: Sparse configurations can significantly reduce token costs while matching or exceeding fully connected debate performance.Limiting each agent’s view filters out some peer responses while preserving or improving empirical performance.

V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS · VI. OPEN CHALLENGES AND RESEARCH DIRECTIONS

The case study examines device-cloud routing for multi-modal kitchen conversations, while the paper identifies large-scale deployment of collaborative LLM systems as an area with unsolved challenges. It highlights routing tradeoffs among response quality, latency, cost, and resource budgets, alongside two broad axes of open research problems.

  • V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS: The case study models a kitchen assistant serving requests across four task categories with varying difficulty, from message editing to scene-grounded visual queries.The setting uses a smartphone-class device and a stream of user requests.
  • V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS: Three complementary but partially redundant image modalities—first-person, overhead, and side views—support the multi-modal conversational setting.The views come from wearable and environmental cameras.
  • V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS: On-device-only inference is fast and free but has the lowest response quality, whereas cloud-only inference has the highest quality but frequently violates the budget.These baselines establish the quality-versus-resource extremes in the routing comparison.
  • V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS: LLM-as-Router baselines produce a quality-latency tradeoff in which higher quality requires proportionally higher latency and cost.The baselines prompt off-the-shelf LLMs to make routing decisions.
  • V. CASE STUDY: DEVICE-CLOUD ROUTING IN MULTI-MODAL CONVERSATIONS: The RL router breaks out of the common baseline tradeoff band by internalizing cumulative resource consumption in its state rather than deciding per query.The comparison concerns quality-latency-cost tradeoffs across device-cloud routing strategies.
  • VI. OPEN CHALLENGES AND RESEARCH DIRECTIONS: Deploying collaborative LLM systems over networks at scale exposes a range of unsolved problems.The paper frames the following discussion around two broad axes of open challenges and promising directions.

A. Scaling Under Resource Heterogeneity

Scaling collaborative LLMs requires frameworks that accommodate endpoints with compute, memory, bandwidth, and cost budgets differing by orders of magnitude. It also requires reliable coordination across large agent populations and long-context workloads, but the necessary scheduling, communication, and fault-tolerance co-design remains largely unexplored.

  • Adapting to constantly shifting ecosystems: Collaboration frameworks must accommodate endpoints whose compute, memory, bandwidth, and cost budgets differ by orders of magnitude.The resource gaps are expected to widen as frontier cloud LLMs grow and on-device deployment moves toward smaller form factors.
  • From pairwise collaboration to large-scale swarms: Scaling from a handful of agents to hundreds or thousands of LLM instances introduces coordination challenges for cloud-managed fleets of on-device agents.Example settings include autonomous vehicles and drone swarms.
  • From pairwise collaboration to large-scale swarms: Concurrent long-context workloads across many agents impose severe memory and communication pressure, while reliable cross-level co-design remains largely unexplored.The unresolved mechanisms span scheduling, communication, and fault tolerance.

B. Trustworthy Collaborative Intelligence · VII. CONCLUSION

Trustworthy collaborative intelligence must balance effective cooperation with containment of sensitive data and resilience against attacks that can spread across agent chains. The paper concludes that heterogeneous LLM collaboration across device-cloud and multi-agent dimensions can improve quality-latency-cost tradeoffs beyond any individual model.

  • B. Trustworthy Collaborative Intelligence: Collaborative inference moves queries and intermediate results across trust boundaries, creating risks for personally identifiable, medical, and financial data.Agentic LLMs with access to local files and system interfaces may leak private content into responses and propagate it through multi-agent chains.
  • B. Trustworthy Collaborative Intelligence: Effective collaboration requires openness, but sensitive-data protection requires containment across the collaboration chain.The passage frames this as a tension between the openness needed for collaboration and the containment needed for privacy.
  • B. Trustworthy Collaborative Intelligence: Prompt injection at one node can propagate through a distributed LLM collaboration hierarchy and contaminate downstream agents across the workflow.This extends the single-interface threat model to distributed deployments.
  • B. Trustworthy Collaborative Intelligence: Injected instructions can persist covertly in accumulated context, interfering with later inference steps after the initial attack.The passage identifies persistent context contamination as a mechanism of cascading attacks.
  • VII. CONCLUSION: No single endpoint can meet the full spectrum of user demands as LLMs become embedded across every network tier.The conclusion motivates collaboration across heterogeneous endpoints.
  • VII. CONCLUSION: Collaborative intelligence lets heterogeneous LLMs cooperate at the task level through natural language and structured messages.The conclusion presents collaboration as an organized response to diverse networked deployment needs.
  • VII. CONCLUSION: Task-level collaboration can achieve quality-latency-cost tradeoffs beyond any individual model.The article organizes this landscape along complementary device-cloud offloading and multi-agent collaboration dimensions.
Loading 2605.08626v1…