PaperlayerView exact revision on arXiv

Source-linked AI summary

Communication-Efficient Edge AI: Algorithms and Systems

Yuanming Shi, Kai Yang, Tao Jiang, Jun Zhang, Khaled B. Letaief

arXiv:2002.09668v1cs.ITcs.LGeess.SP

TL;DR

Edge AI addresses the difficulty of sending massive, delay-sensitive, and privacy-sensitive data from edge devices to the cloud, while cooperation among edge nodes creates heavy communication overheads. This paper surveys communication challenges and communication-efficient algorithmic and system techniques for edge training and inference. It organizes approaches spanning distributed-learning algorithms, compression, decentralized systems, edge inference, and coding-based edge computing.

  • Problem

    Massive edge-generated data cannot all be sent to the cloud because of varying channel quality, traffic congestion, latency requirements, and privacy concerns, while edge cooperation introduces heavy communication overheads.

  • Method

    The paper provides a comprehensive survey of communication-efficient distributed-learning algorithms and edge AI system architectures for training and inference.

  • Results

    The survey categorizes communication-efficient approaches across optimization algorithms, data- and model-partitioned architectures, computation offloading, edge inference, and general edge computing systems.

  • Takeaways & Limitations

    Communication-efficient edge AI requires coordinated advances in distributed learning algorithms, system architectures, and communication protocols.

Abstract

from arXiv · show

Artificial intelligence (AI) has achieved remarkable breakthroughs in a wide range of fields, ranging from speech processing, image classification to drug discovery. This is driven by the explosive growth of data, advances in machine learning (especially deep learning), and easy access to vastly powerful computing resources. Particularly, the wide scale deployment of edge devices (e.g., IoT devices) generates an unprecedented scale of data, which provides the opportunity to derive accurate models and develop various intelligent applications at the network edge. However, such enormous data cannot all be sent from end devices to the cloud for processing, due to the varying channel quality, traffic congestion and/or privacy concerns. By pushing inference and training processes of AI models to edge nodes, edge AI has emerged as a promising alternative. AI at the edge requires close cooperation among edge devices, such as smart phones and smart vehicles, and edge servers at the wireless access points and base stations, which however result in heavy communication overheads. In this paper, we present a comprehensive survey of the recent developments in various techniques for overcoming these communication challenges. Specifically, we first identify key communication challenges in edge AI systems. We then introduce communication-efficient techniques, from both algorithmic and system perspectives for training and inference tasks at the network edge. Potential future research directions are also highlighted.

I. INTRODUCTION

Edge AI brings AI training and inference closer to data sources to address the bandwidth, latency, traffic, and privacy pressures of cloud-based processing. Because edge devices have limited resources, effective edge AI depends on cooperation among edge nodes and communication-efficient algorithms and architectures.

  • Explosive growth in data and AI applications increases pressure on cloud-based processing, especially for delay-sensitive services and variable network conditions.The passage also identifies privacy as a major concern for cloud-based solutions.
  • Edge AI moves inference and training toward edge nodes near data sources, reducing data transferred to the cloud and alleviating traffic, latency, and privacy concerns.
  • Deploying full AI models on edge devices is often infeasible because model sizes and computational requirements exceed their hardware resources.
  • Cooperation among edge nodes can support intensive edge AI tasks by exploiting different data-storage and processing capabilities.Federated learning is given as an example in which multiple devices collaboratively train a model using local data.
  • The survey covers communication-efficient distributed learning algorithms and edge AI system architectures for communication-efficient training and inference.It also frames the paper around communication challenges and the interplay of computation mechanisms, learning algorithms, and system architectures.

A. Motivations

Edge AI shifts computation toward increasingly capable edge nodes, but training and inference remain constrained by communication, computation, storage, power, heterogeneity, and privacy requirements. These constraints motivate communication-aware algorithms and joint designs across edge devices and servers.

  • A. Motivations: Moving computation from cloud centers to edge nodes can support low-latency AI services and joint designs that reduce execution latency and enhance privacy.
  • A. Motivations: Edge training must coordinate massive edge nodes that each typically hold only a small subset of training data, creating high communication costs over bandwidth-limited wireless links.The paper connects distributed data aggregation in edge training with wireless in-network computation and over-the-air model aggregation.
  • A. Motivations: Edge inference benefits from low latency and enhanced privacy, but DNNs can require substantial storage and computation; AlexNet has over 60 million parameters.
  • A. Motivations: Compared with cloud AI, edge AI faces limited computation, storage, power, and link bandwidth across many edge devices and servers.
  • A. Motivations: Hardware, network, and power variability creates heterogeneous communication, computation, storage, and power capabilities across edge nodes.For example, device participation may depend on battery availability, while metered cellular connectivity can discourage information exchange.
  • A. Motivations: Privacy and security constraints are increasingly important for high-stakes intelligent IoT applications and motivate collaborative approaches such as federated learning.

C. Communication Challenges of Edge AI

Edge AI communication latency depends on exchanged information, communication rate, communication rounds, and computation time. The paper surveys algorithmic and system-level designs that reduce communication overheads while accounting for computation, learning, and architecture.

  • Communication model: Total edge AI latency combines communication cost, determined by exchanged information, rate, and rounds, with computation cost across rounds.The communication term uses L, r, and N, while computation contributes T · N.
  • Communication model: Reducing communication overhead requires fewer training rounds, lower per-round traffic, and improved communication rates for training and inference.Training commonly uses multiple rounds, whereas inference often uses one collaborative round.
  • Algorithm-level challenges: Learning algorithms determine what information is transmitted, influencing both per-round overhead and the number of rounds needed for convergence.This motivates convergence acceleration and communication-efficient algorithm design.
  • System-level challenges: Edge AI system design shapes communication through centralized aggregation, in-network computation, distributed modes, and decentralized modes.System architectures vary in data distribution, model placement, computation, and communication mechanisms.
  • Paper scope: The survey distinguishes algorithm-level designs from system-level designs to address communication challenges in edge training and inference.Algorithm-level techniques include convergence acceleration and compression; system-level techniques include data and model partitioning and communication architectures.

III. COMMUNICATION-EFFICIENT ALGORITHMS FOR EDGE AI

Communication-efficient edge AI algorithms address distributed learning under isolated data, limited resources, and heterogeneous links. The surveyed approaches span zeroth-, first-, second-order, and federated optimization, including probabilistic communication and quantization.

  • Motivation: Distributed edge learning extends beyond cloud settings because data is isolated across mobile devices and communication links are heterogeneous and constrained.These conditions make communication-efficient methods important for using distributed data and available computing resources.
  • Zeroth-order methods: Zeroth-order methods support settings where derivatives are unavailable or difficult to obtain, transmitting function values to a central coordinator.They are also used for model-free reinforcement-learning policy optimization.
  • Zeroth-order methods: Zeroth-order gradient estimators approximate derivatives from function values, including finite-difference quantities such as (l(w+µv)−l(w))v/µ.The estimator approximates the gradient of l(w) at w.
  • Communication efficiency: Communication costs can be reduced by probabilistic neighbor transmissions and by measuring convergence in per-node transmissions rather than iteration count.Quantization further reduces per-round traffic, including schemes that quantize each estimated gradient to 1 bit.
  • Survey scope: The paper organizes training algorithms by optimization order and federated optimization, alongside system and communication approaches covered in its topic overview.Figure 3 illustrates zeroth-, first-, second-order, and federated optimization methods for a generalized linear model.

B. Communication-Efficient First-Order Methods

First-order edge training methods rely on gradient exchange but face substantial communication demands in large-scale, distributed settings. The surveyed responses accelerate convergence and reduce per-round traffic through reuse, quantization, sparsification, sketching, and variance reduction.

  • First-order methods: First-order methods iteratively update model variables opposite the loss gradient using a step size, but each iteration can scale with data volume and model dimension.This scaling makes single-device training of large models with massive datasets generally infeasible.
  • First-order methods: Distributed training mitigates computation costs for large models and datasets but introduces additional communication costs for exchanging information across devices.Stochastic gradient descent uses one training sample per update as datasets grow.
  • Communication bottlenecks: Gradient exchange is a main bottleneck in distributed edge AI, motivating both convergence acceleration and reduced communication overhead per round.Per-round reduction methods include gradient reuse, quantization, sparsification, and sketching-based compression.
  • Convergence acceleration: One communication-round distributed optimization can have each device independently run SGD before a server averages the resulting local model parameters.The cited example reports significantly reduced overall runtime as the number of devices increases for some learning tasks.
  • Convergence acceleration: Mini-batch acceleration can reduce communication cost linearly with mini-batch size when the largest admissible batch does not hurt sample complexity.Other approaches include amortized variance reduction and distributed stochastic variance-reduced gradients for heterogeneous data.

2) Minimizing Communication Bandwidth:

Communication bandwidth can be reduced by avoiding unnecessary updates, compressing gradients, and transmitting only important coordinates while preserving convergence or accuracy under stated conditions.

  • Gradient reuse: Gradient reuse omits uploads from devices whose gradients change slowly, reducing communication overhead per round.LAG uses outdated gradients at the fusion center for such devices.
  • Gradient quantization: Gradient quantization represents gradients with fewer bits, with analyses covering mean-squared error, convergence, and decentralized-network limitations.Scalar quantization can fail without a central aggregation node, motivating vector and Grassmannian quantization methods.
  • Gradient quantization: One-bit gradient quantization achieved a 10 times speed-up on distributed speech-DNN training with a small accuracy loss.Sign-based distributed SGD can match full-precision variance reduction and converge to a stationary point for general non-convex objectives.
  • Gradient sparsification: Gradient sparsification communicates only important gradients, often selected because many gradients are small during training.Methods use thresholds, random coordinate dropping, and error correction to support accuracy or convergence guarantees.

C. Communication-Efficient Second-order Methods

Communication-efficient second-order methods use approximate curvature information or extra local computation to reduce communication rounds and model-transfer costs in edge training.

  • C. Communication-Efficient Second-order Methods: First-order methods reduce local computation but may require many communication rounds because of slow convergence.This motivates incorporating second-order curvature information, although exact methods require substantial computation, storage, and communication.
  • C. Communication-Efficient Second-order Methods: Stochastic quasi-Newton methods estimate inverse Hessians and can update a central approximation using subsampled Hessian-vector products.Updating every L iterations is used to provide more stable approximations.
  • C. Communication-Efficient Second-order Methods: DANE solves approximate local Newton systems with global aggregation while requiring the same communication bandwidth as first-order distributed learning.DiSCO instead approximately solves a global Newton system for a more accurate second-order approximation per round.
  • C. Communication-Efficient Second-order Methods: Federated optimization exchanges additional local computation for fewer communication rounds by performing multiple local update steps before aggregation.CoCoA uses multiple local dual-optimization steps, while FedAvg uses local SGD iterations and model averaging.
  • C. Communication-Efficient Second-order Methods: FedProx adds a proximal term to keep local models close to the global model when device data are non-IID.The method addresses degradation that can occur when FedAvg performs too many local SGD steps.
  • C. Communication-Efficient Second-order Methods: Model compression reduces communication overhead by shrinking transmitted model parameters through quantization, sketching, pruning, sparse regularization, and structural matrix design.These techniques target weight representation, connectivity, sparsity, or parameterized matrix structure.
  • C. Communication-Efficient Second-order Methods: Hashing-based sketching groups weights into shared hash buckets, with theoretical local strong convexity and smoothness around the global minimizer.The CNN extension first converts filter weights to the frequency domain before hashing frequency parameters.
  • C. Communication-Efficient Second-order Methods: Low-rank factorization reduces matrix parameters from mn to mr + nr for rank r, lowering computational complexity and storage.A speech-recognition example reduced DNN parameters by 30%−50% in the final weight layer.

IV. COMMUNICATION-EFFICIENT EDGE AI SYSTEMS

Edge AI systems address limited computation, storage, and communication resources alongside privacy, low-latency, security, and reliability requirements through diverse training and inference architectures.

  • IV. COMMUNICATION-EFFICIENT EDGE AI SYSTEMS: Edge AI system architectures are surveyed because edge nodes face limited computation, storage, and communication resources plus privacy, security, latency, and reliability requirements.The section organizes architectures beginning with a general discussion and then treats them individually.
  • IV. COMMUNICATION-EFFICIENT EDGE AI SYSTEMS: The paper summarizes edge AI architectures into four categories based on data and model-parameter availability.These include data-partition and model-partition training, computation-offloading inference, and general edge-computing systems.
  • IV. COMMUNICATION-EFFICIENT EDGE AI SYSTEMS: The survey covers different edge AI systems and topics for efficient training and inference under these constraints.Its stated scope includes both architectural categories and communication-related techniques.

A. Architectures of Edge AI Systems

Edge AI systems partition data or models across devices and servers, offload inference, and use wireless aggregation or coding to manage communication bottlenecks, stragglers, and bandwidth limits.

  • A. Architectures of Edge AI Systems: Data-partition training distributes dataset subsets across devices, each holding a complete model replica and periodically exchanging local updates.Centralized variants use an edge server for scheduling and aggregation, while decentralized variants omit a central node.
  • A. Architectures of Edge AI Systems: Model-partition training distributes model parameters across nodes and supports very deep models or vertical federated learning with partial data attributes.Device-server cooperation can keep part of the model on the device and the remainder on the server to avoid exposing user data.
  • A. Architectures of Edge AI Systems: Computation-offloading inference moves entire or partial inference tasks to nearby edge servers when device storage, computation, or battery resources are insufficient.The goal is low-latency edge AI service deployment near end users.
  • A. Architectures of Edge AI Systems: General edge-computing systems jointly consider distributed data input and model deployment, making data-shuffling communication a critical concern.MapReduce-like frameworks are an example of this category.
  • A. Architectures of Edge AI Systems: In distributed data-partition training, communication bottlenecks arise from aggregating local updates and handling straggler devices.The system periodically aggregates device updates at a central node.
  • A. Architectures of Edge AI Systems: Over-the-air computation exploits wireless signal superposition to aggregate distributed updates while jointly performing communication and computation.It applies when the aggregation function is a nomographic function, including linear combinations of model updates.
  • A. Architectures of Edge AI Systems: Over-the-air federated learning exposes trade-offs between receive SNR, model-parameter truncation, and the fraction of participating devices.Truncated channel inversion is used for deep-fading channels.
  • A. Architectures of Edge AI Systems: Gradient sparsification with random linear projection over the air reduces gradient dimension under limited channel bandwidth and was reported to converge faster than separated computation and communication.The approach was extended to wireless fading channels.

2) Decentralized System Mode:

Decentralized edge learning exchanges model information directly among devices, making network topology a key determinant of communication and runtime efficiency.

  • Decentralized learning lets edge devices exchange information directly without a central node, including through gossip communication protocols.Gossip protocols randomly select nodes to collect or broadcast local updates among neighbors.
  • In fully connected networks, every device sends local gradients to all others, averages received gradients, and maintains identical model parameters after each iteration.
  • Alternative topologies can improve learning performance compared with standard fully connected communication in deep reinforcement learning tasks.
  • An Erdos-Renyi graph with 1000 devices can compete with a fully connected topology using 3000 devices.
  • Topology design also affects distributed optimization runtime: sparser networks can sometimes significantly reduce convergence time.
  • Ring topologies restrict communication to logical neighbors while allowing all devices to obtain the full gradients through aggregation and forwarding.

C. Model Partition Based Edge Training Systems

Model-partitioned edge training distributes portions of a machine-learning model across heterogeneous devices and servers to balance computation, communication, and privacy requirements.

  • Model partitioning distributes model parameters across mobile devices and edge servers so nodes collaboratively perform training or inference.
  • Partitioning across many heterogeneous edge nodes aims to balance computation and communication demands.
  • A reinforcement-learning approach can deploy computation graphs across edge devices, but the approach is time- and resource-intensive.
  • PipeDream automatically determines DNN model partitions, and multiple mini-batches make the system converge faster than a single machine.
  • Splitting a DNN between device and server keeps early layers local and places later layers on the server to avoid exposing users’ data.
  • Vertically partitioned data motivates privacy-preserving machine learning because isolated attributes create competition, privacy, and administrative barriers to collaborative modeling.
  • Secure multi-party computation has been applied to privacy-preserving k-means, association-rule mining, and linear-regression training with vertically partitioned data.

D. Computation Offloading Based Edge Inference Systems

Computation-offloading edge inference places all or part of DNN processing on edge servers, while communication-efficient methods reduce transmitted raw or intermediate data.

  • D. Computation Offloading Based Edge Inference Systems: Server-based edge inference uploads device input data to an edge server, whereas device-edge joint inference splits processing between the device and server.
  • 1) Server-Based Edge Inference: Server-based inference is suited to resource-limited IoT devices, but limited communication bandwidth makes data transmission the main bottleneck.
  • 1) Server-Based Edge Inference: Partial transmission reduces network volume by sending compressed data or selected key frames instead of complete inputs.
  • 1) Server-Based Edge Inference: DNN-based encoding can convert high-dimensional raw data into sparse latent representations, with reinforcement learning controlling encoding actions to preserve prediction accuracy.
  • 1) Server-Based Edge Inference: Multiple edge servers can cooperatively transmit inference outputs to mobile users to improve downlink communication efficiency.
  • 2) Device-Edge Joint Inference: Device-edge joint inference keeps early DNN layers on the device and sends local outputs to the server without revealing sensitive raw data.
  • 2) Device-Edge Joint Inference: Partition placement can account for network latency, memory, communication costs, local accuracy, and offloading latency.
  • 2) Device-Edge Joint Inference: Early exit reduces communication when early-layer features support high-confidence predictions, while encoded transmission and pruning compress intermediate values.

E. General Edge Computing System

MapReduce-like edge systems jointly handle distributed data and model deployment through map, shuffle, and reduce phases, with coding addressing communication and straggler bottlenecks.

  • E. General Edge Computing System: MapReduce provides a general distributed computing framework that can achieve parallel speedups for machine-learning training and inference.
  • E. General Edge Computing System: The framework jointly considers distributed data inputs and distributed model deployment.
  • E. General Edge Computing System: MapReduce computation proceeds through map, shuffle, and reduce phases.
  • E. General Edge Computing System: The map phase computes map functions in parallel, the shuffle phase exchanges intermediate values, and the reduce phase computes outputs from available intermediates.
  • E. General Edge Computing System: The main bottlenecks are heavy shuffle-phase communication and straggler delays caused by variable computation times across nodes.
  • E. General Edge Computing System: Coding techniques reduce shuffle communication through wireline, wireless, and interference-channel schemes, including node cooperation and task assignment.
  • E. General Edge Computing System: Coded computing mitigates stragglers by assigning redundant subtasks, allowing recovery from results produced by a sufficient subset of nodes.

V. CONCLUSIONS AND FUTURE DIRECTIONS

The survey synthesizes communication challenges and solutions for edge AI, then identifies future directions spanning hardware, software platforms, and domain-specific services.

  • Survey scope: The survey categorizes communication-efficient distributed training algorithms and edge AI system architectures, including data- and model-partitioned training.It also covers computation offloading for edge inference and general edge-computing-based AI systems.
  • Future directions: The rapid growth of edge AI activities and applications motivates continued investigation of these hardware, platform, and service directions.The paper presents these as challenges and future directions for enabling intelligent applications at network edges.
  • Edge AI hardware design: Edge AI hardware remains a future direction, with customized designs needed for different system architectures and applications.Existing examples include Google edge TPUs for high-speed inference and Nvidia Jetson TX2 for power-efficient embedded AI computing.
  • Edge AI software platforms: Edge AI software platforms are evolving from cloud-centric systems that currently treat edge nodes mainly as simple extensions of cloud computing.Examples include Google Cloud IoT, Microsoft Azure IoT, NVIDIA EGX, and AWS IoT.
  • Edge AI as a service: Edge AI as a service must integrate computation, communication, storage, and power resources while accommodating domain-specific targets and constraints.FATE is cited as an industrial-grade federated learning framework for the credit industry.
Loading 2002.09668v1…