AI Flow: Perspectives, Scenarios, and Approaches

Hongjun An; Wenhan Hu; Sida Huang; Siqi Huang; Ruanjun Li; Yuanzhi Liang; Jiawei Shao; Yiliang Song; Zihan Wang; Cheng Yuan; Chi Zhang; Hongyuan Zhang; Wenhao Zhuang; Xuelong Li

Source-linked AI summary

AI Flow: Perspectives, Scenarios, and Approaches

Hongjun An, Wenhan Hu, Sida Huang, Siqi Huang, Ruanjun Li, Yuanzhi Liang, Jiawei Shao, Yiliang Song, Zihan Wang, Cheng Yuan, Chi Zhang, Hongyuan Zhang, Wenhao Zhuang, Xuelong Li

arXiv:2506.12479v3Jun 14, 2025cs.AIcs.CLcs.CVcs.DCeess.SP

TL;DR

Large AI models create hardware and communication demands that constrain ubiquitous intelligence and deployment across heterogeneous platforms. AI Flow addresses these challenges through hierarchical AI–communication integration, familial models, and model collaboration, aiming to support emergent intelligence, timely responsiveness, and ubiquitous accessibility.

Problem
Large models require substantial hardware resources and communication bandwidth, limiting practical deployment across heterogeneous platforms and diverse downstream tasks.
Method
AI Flow integrates hierarchical network architectures, device-edge-cloud collaboration, and feature-aligned familial models to coordinate AI models across heterogeneous nodes.
Results
AI Flow supports intelligence emergence beyond any single model while enabling timely responsiveness and ubiquitous accessibility for intelligent services.
Takeaways & Limitations
The framework offers a multidisciplinary approach for tighter integration of AI techniques and communication systems under heterogeneous resource and bandwidth constraints.
Takeaways & Limitations
The paper identifies hardware resource limitations, including redundant parameters, as a boundary for deploying large models across heterogeneous platforms.

Abstract

from arXiv · show

Pioneered by the foundational information theory by Claude Shannon and the visionary framework of machine intelligence by Alan Turing, the convergent evolution of information and communication technologies (IT/CT) has created an unbroken wave of connectivity and computation. This synergy has sparked a technological revolution, now reaching its peak with large artificial intelligence (AI) models that are reshaping industries and redefining human-machine collaboration. However, the realization of ubiquitous intelligence faces considerable challenges due to substantial resource consumption in large models and high communication bandwidth demands. To address these challenges, AI Flow has been introduced as a multidisciplinary framework that integrates cutting-edge IT and CT advancements, with a particular emphasis on the following three key points. First, device-edge-cloud framework serves as the foundation, which integrates end devices, edge servers, and cloud clusters to optimize scalability and efficiency for low-latency model inference. Second, we introduce the concept of familial models, which refers to a series of different-sized models with aligned hidden features, enabling effective collaboration and the flexibility to adapt to varying resource constraints and dynamic scenarios. Third, connectivity- and interaction-based intelligence emergence is a novel paradigm of AI Flow. By leveraging communication networks to enhance connectivity, the collaboration among AI models across heterogeneous nodes achieves emergent intelligence that surpasses the capability of any single model. The innovations of AI Flow provide enhanced intelligence, timely responsiveness, and ubiquitous accessibility to AI services, paving the way for the tighter fusion of AI techniques and communication systems.

I. INTRODUCTION

AI Flow addresses the resource and communication bottlenecks limiting ubiquitous intelligence by integrating device-edge-cloud collaboration, familial models, and connectivity-based intelligence emergence.

Motivation: Large AI models demand substantial computation, storage, and power, hindering deployment on heterogeneous resource-constrained devices.IoT sensors and mobile phones are identified as especially constrained platforms.
Motivation: Distributing computation across cloud and edge servers can alleviate hardware limitations, but high-dimensional data transmission creates communication overhead.The resulting overhead is described as a critical bottleneck for modern communication infrastructure.
Device-Edge-Cloud Collaboration: AI Flow provides a unified device-edge-cloud architecture integrating end devices, edge servers, and cloud clusters for scalable, low-latency inference.Its collaboration paradigms are tailored to hierarchical network architectures and aim to minimize communication bottlenecks.
Familial Models: Familial models are multi-scale, feature-aligned architectures that support knowledge transfer and collaboration across diverse tasks and resource constraints.Feature alignment enables information sharing without additional middleware, while collaborative deployment improves inference efficiency under constrained bandwidth and computation.
Connectivity- and Interaction-based Intelligence Emergence: Connectivity- and interaction-based intelligence emergence coordinates advanced models across networks to produce intelligence surpassing any single model’s capability.The framework links collaboration and dynamic interaction among models such as LLMs, VLMs, and diffusion models.
The Rise of AI: Large generative models have expanded AI beyond specialized tasks, but adoption remains limited in logical reasoning, collective decision-making, and multi-agent collaboration.The paper presents these limitations as motivation for techniques that enhance model capabilities.

B. Towards Ubiquitous AI Applications

Ubiquitous AI aims to provide responsive, adaptable services across everyday environments, but deployment is constrained by hardware resources and communication networks. Large-model growth, inference memory and computation, bandwidth demands, latency, and network instability jointly impede scalable real-time operation.

Application vision: Ubiquitous AI targets responsive and adaptable services across everyday environments and sectors including smart cities, industrial automation, and autonomous systems.AI-powered glasses are presented as one example of applications that augment perception and spatial awareness.
Collaborative intelligence: Collaborative agents must share information, negotiate roles, fuse multimodal data, and adapt to changing environments, but dynamic networks and limited onboard computation impede latency-sensitive decisions.The paper identifies these constraints as obstacles to realizing ubiquitous AI applications in resource-constrained environments.
Hardware resource limitations: 4–32 GBs is the approximate capacity of typical end devices, whereas advanced large models require gigabytes of memory and increasingly undermine low-latency deployment on such hardware.The passage links model-scale growth and rising computational complexity to hardware requirements and latency-sensitive applications such as autonomous driving.
Hardware resource limitations: Quadratic self-attention complexity makes intermediate reasoning tokens increase LLM inference cost, while efficiency techniques may sacrifice model capabilities and reliability.Quantization reduces precision and model pruning removes redundant parameters, but the text frames these approaches as limited compromises.
Communication network constraints: Tens to hundreds of megabytes may be transmitted per split-inference step, and repeated high-dimensional exchanges can increase latency, reduce throughput, and hinder scalability.The paper also identifies packet loss, queuing delays, jitter, congestion, and synchronization of reasoning traces as communication challenges.
AI Flow motivation: AI Flow addresses the dual bottleneck by combining hierarchical network architectures with AI–communication co-designs for scalable, efficient deployment and real-time response.The framework is described as streamlining inference, optimizing heterogeneous resources, and coordinating familial-model interactions.

D. Addressing Challenges with AI Flow

AI Flow addresses ubiquitous-AI deployment challenges through hierarchical resource collaboration, familial models, and communication-efficient inference mechanisms. Its approaches distribute workloads across heterogeneous tiers, align model variants, and support flexible quality-efficiency tradeoffs.

Hierarchical collaboration: AI Flow streamlines model inference through device-edge-cloud collaboration, mitigating hardware constraints and reducing high inference latency.The architecture integrates distributed resources and is intended to support scalability and low-latency inference.
Hierarchical collaboration: The device-edge-cloud architecture dynamically adapts to heterogeneous hardware and application requirements by combining local, edge, and cloud resources.Latency-critical inference is prioritized at device and edge tiers, while resource-intensive operations are offloaded to cloud resources.
Communication-efficient inference: TOFC reduces communication constraints in device-edge cooperative VLM inference by combining bandwidth-aware coding with adaptive token pruning.The mechanism dynamically optimizes task-relevant data payloads under real-time conditions.
Communication-efficient inference: Speculative decoding uses lightweight device models to predict LLM tokens while larger edge and cloud models validate and refine them.This shifts larger models from full sequence generation to error correction and supports adaptive quality-efficiency tradeoffs under changing resource constraints.
Familial models: Familial models use early exit and weight decomposition to scale flexibly across heterogeneous resources while preserving aligned intermediate features.Aligned features allow direct reuse of intermediate results from smaller variants, and weight decomposition provides arbitrary parameter counts with reduced redundancy.
Familial models: Familial models support efficient collaboration across model sizes and hardware settings without severe performance degradation or computational overhead.Early exit enables faster inference with minimal accuracy loss, while the framework is designed to meet diverse task demands under stringent conditions.

3) Connectivity- and Interaction-based Intelligence Emergence:

AI Flow treats connectivity and interaction among heterogeneous models as a basis for collaborative intelligence. Its device-server and diffusion-model collaboration paradigms combine specialized outputs across networked nodes and tiers.

Connectivity-based intelligence: AI Flow reinforces connectivity and interaction among LLMs, VLMs, and diffusion models through a hierarchical network architecture.The framework presents model collaboration as a route to intelligence emergence beyond isolated model operation.
Device-server collaboration: A four-step device-server pipeline selects suitable devices, gathers specialized device responses, aggregates them centrally, and returns the answer for possible revision.The aggregated answer incorporates both global context and specialized device opinions.
Diffusion-model collaboration: Serial diffusion-model collaboration composes temporally controlled motion segments and then refines interactions between people with another model.The paradigm is introduced for complex, coordinated multi-person motion sequences.
Diffusion-model collaboration: Parallel collaboration uses a unified image encoder, separate near- and far-region decoders, and anchor-depth fusion to produce a complete depth map.This paradigm is applied to monocular depth estimation.
Diffusion-model collaboration: Hierarchical networked collaboration aggregates RGB, segmentation, depth, and audio features through distributed encoders and a unified diffusion transformer.Specialized decoding heads produce modality-specific outputs while the transformer enforces spatial and temporal alignment.
Hierarchical network architecture: The device-edge-cloud architecture provides distributed inference pipelines that balance computational scale, transmission latency, and model collaboration across network tiers.End devices, edge servers, and cloud clusters support different resource and latency profiles within the collaborative framework.

B. Efficient Collaboration Paradigms

Task-oriented feature compression reduces visual transmission and inference latency for multimodal models under constrained communication. The proposed TOFC method merges and entropy-codes task-relevant visual features while preserving downstream performance.

Task-Oriented Feature Compression: TOFC dynamically compresses task-relevant visual features through clustering, merging, and adaptive entropy-model selection.DPC-KNN groups visual features for average-pooling, while entropy models encode the merged features according to their characteristics.
Task-Oriented Feature Compression: 25% to 45% and 35% to 60% reductions in data transmission overhead are achieved on RealWorldQA and MME, respectively, at matched performance.The comparisons are against conventional image compression methods.
Task-Oriented Feature Compression: 5.9× and 6.7× larger data sizes are required by WebP and JPEG to reach performance comparable to TOFC under similar system latency.With feature merging, TOFC also saves approximately 25% and 42% of data size for identical MME perception scores compared with WebP and JPEG.
Task-Oriented Feature Compression: 69% and 72% data-size reductions relative to WebP and JPEG are obtained when feature merging is used for inference acceleration.The results are attributed to discarding redundant visual features while maintaining high task performance.
Task-Oriented Feature Compression: TOFC halves RealWorldQA system latency under low communication rates and reaches 32% of server-only latency on MME with minimal performance degradation.Latency reductions become more substantial as communication rates increase.

2) Hierarchical Collaborative Decoding:

Hierarchical collaborative decoding combines lightweight device models with larger edge or cloud models to balance generation speed, resource usage, and output quality. Parallel execution reduces blocking while retaining the larger model’s verification and refinement role.

Hierarchical Collaborative Decoding: A lightweight device model drafts tokens locally, while a larger edge model verifies and corrects them through speculative decoding.This division preserves local generation efficiency while using the larger model for ambiguity resolution and complex reasoning.
Hierarchical Collaborative Decoding: The device-edge framework transmits compact token updates, reducing bandwidth use while retaining the larger model’s accuracy and reliability.The edge model accepts or rejects candidate tokens using the relative probabilities assigned by the two models.
Hierarchical Collaborative Decoding: Parallel collaborative decoding lets the device continue generating while the edge refines received tokens at intervals, minimizing blocking.The pipeline aligns device decoding time with an edge forward pass over the received token sequence.
Hierarchical Collaborative Decoding: The architecture combines fast device generation with edge intelligence to improve final-output quality and generation efficiency.It can extend from two tiers to a three-tiered device-edge-cloud setup.
Hierarchical Collaborative Decoding: 40.09 tokens per second on MATH-500 provides about a 1.25× speedup over standalone edge decoding at the same accuracy.The evaluation uses pass@1 across mathematical reasoning and code-generation benchmarks.
Familial Models: Familial models use different-sized models with aligned hidden features to support overhead-free information sharing and collaboration.They can reuse intermediate results from smaller models and adapt across heterogeneous hardware and bandwidth constraints.

A. Related Works

The related-work discussion motivates familial models through weight decomposition and early exit, then describes extensions for scalable, aligned multi-branch inference. These techniques reduce parameters and computation while supporting collaboration across heterogeneous devices.

Weight Decomposition: The proposed decomposition uses data whitening and truncated SVD to enable layer-wise rank control while limiting approximation distortion.Discarded-component distortion is determined by the squared singular values, supporting adaptive compression ratios.
Familial Models: The familial architecture extends low-rank decomposition to scalable branch networks with shared language-model heads for multi-branch inference and fine-grained tuning.The design generalizes parameter-efficient adaptation beyond conventional low-rank fine-tuning.
Early Exit: Familial models preserve alignment across branches so heterogeneous devices can continue processing intermediate features without redundant computation.This supports cooperative inference and improves efficiency and accuracy in distributed systems.
Weight Decomposition: Weight decomposition replaces a linear layer with two lower-dimensional layers, reducing parameter count and potentially lowering memory use and inference cost.The hidden dimension can be tuned to control model size while maintaining structural consistency.
Weight Decomposition: Adaptive compression ratios across linear-layer categories improve overall model accuracy under a fixed total parameter budget.The selection is guided by the distribution of singular values.
Early Exit: Early exit creates branches with different parameter counts and computational complexities, allowing adaptation to hardware resources and task delay requirements.Branches connect intermediate exit points to a shared language-model head.

C. Implementation Strategies

The paper presents hierarchical principal component decomposition (HPCD) as a familial-model strategy that progressively decomposes weight matrices into low-rank components. By retaining different numbers of components, HPCD provides dynamic control over model accuracy, computation, and memory.

HPCD: HPCD decomposes transformer weight matrices into hierarchical low-rank components that progressively fit residual errors.Each subsequent component is trained against the residual left by earlier components while preserving their learned information.
Results: HPCD captures primary and supplementary features across decomposition stages, enabling dynamic compression and accuracy–efficiency trade-offs.The authors describe progressive component training as preserving critical features while controlling computational demands.
HPCD: During inference, retaining the first k components enables granular control over FLOPs and memory usage under hardware constraints.The retained component count k determines the compact weight matrix used at inference.
Implementation: The implementation trains four variants spanning 2.38B to 6.30B parameters, with k ranging from 1 to 4.All variants use a uniform low-rank decomposition structure while varying the number of components.
Training: HPCD combines autoregressive language-modeling loss with knowledge distillation from TeleChat2-7.68B to preserve generation and teacher alignment.The distillation term minimizes KL divergence between student and teacher logits, with λ fixed at 0.4.
Results: Despite limited training tokens, HPCD models achieve performance comparable to LLaMA2-7B, Baichuan2-7B, and ChatGLM2-6B, with later components improving over predecessors.The reported progression indicates that later components capture knowledge not acquired by earlier ones.

2) Early Exiting with Scalable Branches:

Early exiting with scalable branches (EESB) constructs familial models by combining exit points with decomposed transformer branches. This design tunes parameter counts and inference cost while retaining strong VQA performance across diverse tasks.

EESB design: EESB constructs familial models with multiple exit points and decomposed transformer branches to support nearly arbitrary parameter counts.The branch parameters and shared model components are configured to meet a target parameter budget.
Inference: During inference, loading only blocks before the selected exit and its branch substantially reduces GPU memory and storage requirements.The selected branch can be dynamically tuned for the task’s computational and performance requirements.
Training: Training can freeze, LoRA-tune, or end-to-end fine-tune the main model, while branch losses are combined through a weighted cross-entropy objective.When simultaneous branch training is too memory-intensive, only a subset of branches is trained per step.
Evaluation: EESB evaluates nine VQA benchmarks spanning perception, OCR, reasoning, mathematics, scientific knowledge, and information gathering.The evaluation compares EESB with a baseline that directly uses intermediate features to generate responses.
Results: EESB retains 98.2% of backbone capability on six general-purpose VQA benchmarks using 3.17B parameters.The baseline requires at least 4.63B parameters to reach 90% of backbone capability on the same benchmark group.
Results: For diagram and table information gathering, EESB achieves 82.8% of task performance with 4.18B parameters, versus 28.6% for the baseline with 5.44B.The reported comparison covers ChartQA, DocVQA, and InfoVQA.
Results: EESB lowers the average parameter count needed to maintain 60% of backbone capability from 4.43B to 2.76B.At 4.38B parameters, EESB exceeds 95% of backbone capability, while the baseline reaches 75% with 5.44B.

D. Discussions

The discussion frames familial models as adaptable components for heterogeneous hardware, task requirements, and collaborative inference. It also presents AI Flow as an interaction-driven framework in which connected models can achieve capabilities beyond isolated models.

Familial models: Familial models adapt model size and parameter configuration to heterogeneous computing resources and downstream latency requirements.Smaller models can serve delay-sensitive applications, while larger models support more demanding tasks.
Collaborative inference: Shared intermediate features enable overhead-free split inference between user devices and edge servers.Device-side processing can continue with a preliminary response while edge servers perform further processing when available.
Collaborative inference: Device–edge collaboration can smooth response generation during intermittent wireless disconnections and reduce edge-server computation costs.Offloading part of the computation to user devices also increases the number of concurrent requests an edge server can handle.
Speculative decoding: Aligned intermediate features support speculative decoding by increasing draft-token acceptance and preserving computation savings across devices.The smaller familial model serves as the draft model, while shared computation with the larger model can be omitted.
Future work: Future work includes extending familial models to more downstream tasks and deploying trained models on layered infrastructure.This is identified as a principal direction for further development.
AI Flow: AI Flow coordinates heterogeneous intelligent components through hierarchical interactions to produce outputs that surpass the sum of individual contributions.The framework integrates diverse modalities and domain expertise through enhanced connectivity and inter-model synergy.
LLM collaboration: Across LLM collaboration tasks, weaker models benefit most, with gains reported in dialogue coherence, human preference consistency, and challenging reasoning.The reported improvements span MT-Bench, AlpacaEval 2.0, and Arena-Hard.
Multi-agent collaboration: Device-server multi-agent performance improves as the number of agents increases, with Arena-Hard showing an approximately linear performance curve.The authors interpret this trend as supporting connectivity- and interaction-driven intelligence emergence.

B. Collaboration among Diffusion Models

The paper formulates serial, parallel, and networked collaboration paradigms for diffusion models. In the serial setting, modular components compose and refine motion sequentially, producing stronger semantic alignment and interaction quality.

Collaboration paradigms: AI Flow uses serial, parallel, and networked collaboration paradigms to adapt diffusion-model cooperation to diverse network topologies and hardware.These paradigms organize hierarchical interactions among distributed models.
Serial collaboration: INS fuses solo and interactive motion segments into a temporally controlled sequence, and REC refines coordination for coherent interactions.The conditional diffusion process is guided by text descriptions and temporal cues.
Serial collaboration: The serial design produces richer interaction capabilities by combining modular expertise across composition and coordination stages.The resulting trajectories capture transitions between self-expression and coordinated social behavior.
Serial collaboration: The serial architecture separates high-level motion composition from low-level inter-agent refinement through complementary modules.The INS module composes motion, while REC models spatial relationships and mutual influences between agents.
Results: On InterHuman, serial collaboration reaches Top-1/2/3 R-Precision scores of 0.335, 0.479, and 0.584, with FID of 6.332.The evaluation measures socially synchronized, text-aligned motion.
Results: Compared with InterGen, serial collaboration improves Top-1 R-Precision by 25.3% and reduces FID by 50.6%.The comparison reports gains in semantic accuracy and motion realism.
Results: Serial collaboration reduces MM Distance by 37.0% versus single-person MDM baselines while achieving near-human motion diversity of 7.763 versus 7.748.The reported results also include a multimodality score of 1.601.

2) Parallel Collaboration:

Parallel collaboration distributes computation across peer models or branches, addressing serial latency and supporting depth estimation and multimodal video generation. The reviewed systems improve cross-dataset depth generalization and unify multiple video modalities and tasks.

Parallel collaboration: Parallel collaboration enables simultaneous computation across multiple devices, accelerating collaborative inference compared with serial execution.The framework leverages model diversity among peer models at the same level.
Monocular metric depth estimation: The depth model achieves top performance across all metrics on NYU-V2 and KITTI, outperforming models fine-tuned individually on each dataset.Evaluation uses 10 meters for NYU-V2 and 80 meters for KITTI.
Monocular metric depth estimation: The dual-branch design demonstrates strong cross-dataset generalization by resolving local structure and extending predictions to distant regions.This capability is attributed in the passage to the complementary near-depth and far-depth branches.
Monocular metric depth estimation: A dual-branch depth model combines scaled near-field prediction with tapered far-field prediction using sliding anchor embeddings.The branches predict depth at different ranges, validity masks, and complete fused depth maps.
Networked collaboration: OmniVDiff integrates RGB, depth, segmentation, and Canny edge modalities within one diffusion framework with adaptive modality control.Conditioning and generation roles change by task, guided by learnable modality embeddings.
Networked collaboration: OmniVDiff outperforms existing baselines on depth-conditioned video generation, improving geometric consistency and fidelity to the input depth.The results support a unified model for multiple generative and understanding tasks.

C. Discussions

AI Flow treats connectivity and interaction as primitives for distributing inference and coordinating heterogeneous models. Its hierarchical and multi-agent collaborations support low-latency responses, collective capabilities beyond a single model, and robust operation across real-world environments.

Connectivity and interaction: AI Flow establishes connectivity and interaction as first-class primitives for intelligence emergence.The framework generalizes collaboration across hierarchical and heterogeneous intelligent agents.
Hierarchical collaboration: Hierarchical collaborative decoding distributes partial inference across device, edge, and cloud layers to trade off latency, accuracy, and computational cost.GUI-agent subtasks can be assigned according to complexity and latency sensitivity.
Hierarchical collaboration: Coordinated scheduling enables low response latency while maintaining high-quality outputs for assistants, analytics, and multimodal interaction platforms.Simple tasks remain on devices while complex reasoning and planning can be offloaded.
Multi-agent intelligence: Multi-agent systems can exhibit emergent intelligence by coordinating specialized agents to solve tasks beyond the reach of a single model.AI Flow supports sharing intermediate representations, contextual information, and feedback signals.
Familial models: Familial models share an architectural backbone while adapting to each agent’s hardware constraints, reducing redundant computation and inference overhead.A drone can relay intermediate features to a ground robot, which resumes inference without repeating preprocessing.
Connectivity and interaction: Adaptive interaction mechanisms allow heterogeneous agents to accomplish tasks impossible for individual agents, such as coordinated object delivery by a robotic arm and drone.The arm handles manipulation while the drone handles aerial navigation and delivery.

B. Wearable Devices

AI Flow applies device–edge–cloud coordination to wearable and smart-city settings constrained by computation, memory, energy, bandwidth, and latency. Workload partitioning and familial models distribute intelligence across tiers for responsive, coordinated operation.

Wearable devices: Wearable devices face limited processing power, memory storage, and battery life that constrain on-device AI deployment.These resource constraints create a bottleneck for operating AI models.
Wearable devices: AI Flow partitions wearable workloads across devices, edge servers, and cloud clusters according to latency and resource requirements.Local processing extracts immediate spatial cues, while resource-intensive recognition and segmentation can be offloaded.
Wearable devices: The orchestration minimizes wearable energy consumption while maintaining high inference accuracy and responsiveness.Latency-sensitive tasks receive priority when edge resources are available, whereas latency-tolerant analytics can be deferred to the cloud.
Smart cities: Smart cities require ultra-low-latency coordination under stringent energy and bandwidth constraints while scaling across many connected devices.Examples include emergency vehicle rerouting, medical drone dispatch, and shared low-altitude airspace.
Smart cities: AI Flow disseminates intelligence across end devices, edge servers, and cloud clusters using familial models optimized for each tier.Devices handle localized lightweight tasks, edge models aggregate multimodal regional data, and cloud models train global models and disseminate updates.
Smart cities: Hierarchical task distribution enables smart cities to coordinate heterogeneous systems, minimize bottlenecks, and enhance responsiveness in dynamic scenarios.The framework supports prompt decision-making across IoT sensors, autonomous vehicles, and aerial drones.

B. Distributed Edge Inference

Distributed edge inference partitions and coordinates AI computation across heterogeneous devices and hierarchical networks, but faces resource, communication, and synchronization challenges. AI Flow addresses these constraints through device-edge-cloud collaboration, familial models, and connectivity- and interaction-based intelligence emergence.

Distributed Edge Inference: Distributed edge inference partitions large AI models into smaller modules for collaborative execution across interconnected edge devices.This uses decentralized computational resources for scalable, low-latency deployment while mitigating individual devices’ memory and processing limitations.
Distributed Edge Inference: Heterogeneous device capabilities, memory and energy constraints, and unstable participation complicate model partitioning and workload balancing.Devices may frequently join or disconnect, requiring coordination across differing computational conditions.
Distributed Edge Inference: High-dimensional multimodal data and intermediate features can dominate end-to-end latency and bottleneck system scalability.These inter-device transfer costs are a central challenge for collaborative inference.
Distributed Edge Inference: Existing data and model parallelism often assumes homogeneous infrastructure and stable networks, whereas edge environments produce synchronization and load-balancing bottlenecks.Traditional parallelism therefore struggles with device heterogeneity and intermittent disconnections.
Distributed Edge Inference: Network heterogeneity creates unpredictable latency, bandwidth fluctuations, topology complexity, and unreliable model synchronization across hierarchical systems.Cloud links may experience jitter and variable delays, motivating adaptive communication protocols and resilient designs.
AI Flow: AI Flow integrates AI and communication-network technologies through device-edge-cloud collaboration, familial models, and connectivity- and interaction-based intelligence emergence.The framework targets hardware and communication constraints while enabling model collaboration across heterogeneous nodes.
AI Flow: AI Flow is presented as supporting emergent intelligence beyond a single model, timely responsiveness, and ubiquitous accessibility at the network edge.The paper also analyzes application scenarios and outlines future directions for expanding the framework.