PaperlayerView exact revision on arXiv

Source-linked AI summary

Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing

Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, Junshan Zhang

arXiv:1905.10083v1cs.DCcs.AIcs.NI

TL;DR

The proliferation of mobile and IoT devices is producing substantial data at the network edge, while edge intelligence research remains at an early stage. This paper surveys the motivation, architectures, frameworks, and technologies for edge-based AI training and inference, and discusses open challenges and future directions. Its supported outcome is a comprehensive synthesis intended to stimulate discussion and further research.

  • Problem

    Mobile and IoT devices generate substantial data at the network edge, but research on moving AI there remains at an early stage and lacks a dedicated comprehensive venue.

  • Method

    The paper conducts a comprehensive survey of edge intelligence, covering motivation, architectures, frameworks, enabling technologies, model training and inference, and future challenges.

  • Results

    The survey provides an overview of research efforts and technologies for deep learning model training and inference at the network edge.

  • Takeaways & Limitations

    Edge intelligence is presented as a research area connecting edge computing with AI to process data closer to devices and data sources.

Abstract

from arXiv · show

With the breakthroughs in deep learning, the recent years have witnessed a booming of artificial intelligence (AI) applications and services, spanning from personal assistant to recommendation systems to video/audio surveillance. More recently, with the proliferation of mobile computing and Internet-of-Things (IoT), billions of mobile and IoT devices are connected to the Internet, generating zillions Bytes of data at the network edge. Driving by this trend, there is an urgent need to push the AI frontiers to the network edge so as to fully unleash the potential of the edge big data. To meet this demand, edge computing, an emerging paradigm that pushes computing tasks and services from the network core to the network edge, has been widely recognized as a promising solution. The resulted new inter-discipline, edge AI or edge intelligence, is beginning to receive a tremendous amount of interest. However, research on edge intelligence is still in its infancy stage, and a dedicated venue for exchanging the recent advances of edge intelligence is highly desired by both the computer system and artificial intelligence communities. To this end, we conduct a comprehensive survey of the recent research efforts on edge intelligence. Specifically, we first review the background and motivation for artificial intelligence running at the network edge. We then provide an overview of the overarching architectures, frameworks and emerging key technologies for deep learning model towards training/inference at the network edge. Finally, we discuss future research opportunities on edge intelligence. We believe that this survey will elicit escalating attentions, stimulate fruitful discussions and inspire further research ideas on edge intelligence.

I. INTRODUCTION

AI applications and edge-generated data are expanding rapidly, creating pressure to move AI processing closer to devices. This survey introduces edge intelligence, reviews its architectures and technologies for training and inference, and identifies future challenges.

  • Deep learning has driven AI applications across vision, speech, language, recommendation, surveillance, and robotics.
  • Mobile and IoT proliferation is shifting data generation toward the network edge, with nearly 850 ZB projected by 2021 versus 20.6 ZB of global datacenter traffic.
  • Cloud-based analytics can impose high monetary cost and transmission delay while raising privacy concerns, whereas edge computing places services near devices and data sources.
  • Edge intelligence combines edge computing and AI to use distributed edge resources rather than relying entirely on the cloud.
  • Because edge intelligence research remains at an early stage, the survey reviews its motivation, definition, architectures, enabling technologies, training, inference, and open challenges.

B. Deep Learning and Deep Neural Networks

Deep learning uses layered neural networks to learn representations for high-precision inference, with architectures specialized for different data and tasks. The section introduces DNN structure, training and inference, and popular models including MLPs, CNNs, RNNs, LSTMs, GANs, and DRL.

  • Deep learning uses artificial neural networks organized as layers of neurons that learn deep data representations for tasks such as image classification and face recognition.Each neuron generates nonlinear outputs from its inputs, while deeper and more abstract layers support high-precision inference.
  • A DNN propagates inputs through hidden layers, where weighted sums and activation functions transform them before producing results at the output layer.The section identifies MLPs, CNNs, and RNNs as three popular DNN structures.
  • MLPs use fully connected layers, CNNs use convolutional filters to extract representations for vision tasks, and RNNs process sequential inputs using prior samples.The section also describes LSTM memory cells and gates that control information access and limit perturbation from irrelevant inputs.
  • Deep learning is a resource-demanding AI workload that naturally aligns with edge computing, motivating techniques for training and inference under edge constraints.The survey focuses on deep learning at the edge while noting that some optimization techniques may also apply to other AI and machine-learning models.
  • Training optimizes initially assigned weights by evaluating output error against true labels and applying algorithms such as stochastic gradient descent.Inference follows training and uses only feed-forward computation to generate predictions from real-world inputs.
  • GANs combine a generator that learns to produce new data with a discriminator that distinguishes real data from generated data.The model is used for image generation, transformation, synthesis, and super-resolution.
  • Deep reinforcement learning combines reinforcement learning over environment states and actions with DNNs that represent states and approximate action values.A reward measures the distance between predefined requirements and an action's performance, enabling continuous learning for tasks such as gaming.

III. EDGE INTELLIGENCE

Edge intelligence combines AI with edge computing to exploit data and resources across end devices, edge nodes, and cloud datacenters. The paper broadens its scope beyond purely local execution and rates configurations by data offloading.

  • Motivation: Edge intelligence combines AI’s data-driven learning with edge computing’s proximity to generated data and collaborative edge resources.The combination supports applications that analyze multimodal edge data and benefits from low latency and reduced bandwidth consumption.
  • Motivation: Edge intelligence is mutually beneficial with edge computing: it enables ubiquitous AI, while AI supplies demanding application scenarios for edge infrastructures.The paper describes this relationship as technically complementary and mutually beneficial in application popularization.
  • Scope and Definition: The paper defines edge intelligence broadly as exploiting data and resources across end devices, edge nodes, and cloud datacenters for DNN training and inference.This scope includes cloud-edge-device coordination rather than requiring models to run entirely at the edge.
  • Scope and Rating: The paper rates edge intelligence into six levels according to the amount and path length of data offloading.The levels range from Cloud Intelligence through cloud-edge and in-edge cooperation to all-in-edge and all-on-device configurations.
  • Scope and Rating: Higher edge-intelligence levels reduce offloading latency, improve privacy, and reduce WAN bandwidth cost, but increase computational latency and energy consumption.Therefore, no universally best level exists; selection depends jointly on latency, energy efficiency, privacy, and WAN bandwidth cost.

IV. EDGE INTELLIGENCE MODEL TRAINING

The survey focuses on distributed DNN training at the edge, organizing it by architecture and reviewing centralized training in the cloud. These architectures use cloud datacenters, edge servers, and end devices as distinct computational and data-source roles.

  • Overview: Distributed edge DNN training is reviewed through architectures, performance indicators, enabling techniques, and existing systems and frameworks.The section emphasizes that edge-generated data increasingly supplies AI model training.
  • Architectures: The architectures are divided into centralized, decentralized, and hybrid Cloud-Edge-Device training modes.Fig. 8 illustrates the three modes as subfigures (a), (b), and (c), respectively.
  • Architectures: The architecture’s roles distinguish the cloud as the central datacenter, end devices as data sources, and base stations as representative edge servers.Examples of end devices include mobile phones, cars, and surveillance cameras.
  • Centralized: In the centralized mode, end devices generate and gather training data, which is sent to the cloud datacenter for DNN training.The resulting system corresponds to Cloud Intelligence or Levels 1–3 depending on its inference mode.

2) Decentralized:

Decentralized edge training lets computing nodes train locally and exchange model updates without cloud-datacenter intervention. Its evaluation spans optimization, convergence, privacy, communication, latency, and energy.

  • Decentralized: In decentralized training, each computing node trains a local DNN model with local data and exchanges local model updates with other nodes.This preserves private information locally and produces a global model without intervention from the cloud datacenter.
  • Hybrid: The hybrid architecture combines decentralized edge-server updates with centralized cloud training and therefore covers Levels 4 and 5.It is also called Cloud-Edge-Device training because it involves cloud, edge, and device roles.
  • Performance Indicators: Six indicators assess distributed training: training loss, convergence, privacy, communication overhead, latency, and energy efficiency.Training loss and convergence are common objectives, while convergence is specialized for decentralized methods.
  • Performance Indicators: Decentralized convergence measures whether and how fast distributed training reaches consensus, depending on gradient synchronization and update procedures.The consensus represents the training result of the decentralized method.
  • Performance Indicators: Communication overhead increases training latency, energy use, and bandwidth consumption, while energy efficiency is constrained by model size and device resources.Latency also combines computation and communication components, and end devices are energy-constrained.

4) Communication Cost:

The survey reviews technologies that improve communication efficiency and related training objectives in edge DNN training. Federated learning keeps raw data on clients but faces optimization and communication challenges, motivating update and aggregation controls.

  • Overview: The surveyed enabling technologies target one or more key performance indicators for edge DNN training, including communication overhead and privacy.Table I summarizes the highlights of each technology.
  • Federated Learning: Federated learning leaves raw data on clients and trains a shared model by aggregating locally computed updates at a server.Its main challenges are optimization and communication.
  • Federated Learning: SSGD lets clients train independently and selectively share small subsets of model parameters, targeting privacy and training loss.The cited approach reduces training loss compared with training solely on local inputs but does not address unbalanced non-IID data.
  • Communication Cost: Federated learning communication can bottleneck training because clients typically send a full model or full model update over unreliable networks.Structured and sketched updates reduce communication cost, while increasing local computation can reduce the number of training rounds.
  • Communication Cost: Communication-efficient systems control update aggregation content and frequency, including Gaia’s separation of intra- and inter-datacenter communication.Other methods optimize local-update versus global-aggregation trade-offs under resource budgets or use fully decentralized neighbor exchanges.

3) Gradient Compression:

Gradient compression reduces communication overhead in decentralized edge training through quantization and sparsification, while related techniques address privacy, computation, and resource limits.

  • Gradient quantization uses finite-bit values, while gradient sparsification transmits only part of gradient vectors.
  • 99.9% of distributed SGD gradient exchange is reported as redundant, enabling DGC to compress gradients by 270-600× across CNNs and RNNs.DGC preserves accuracy using momentum correction, local gradient clipping, momentum factor masking, and warm-up training.
  • eSGD selects important gradient coordinates and accumulates momentum residuals to track outdated residual gradients.
  • Extrapolation compression and difference compression achieve O(1/sqrt(nT)) convergence, matching full-precision centralized training.Here, n is the number of clients and T is the number of iterations.
  • DNN splitting protects privacy by transmitting partially processed data and can partition a model between end devices and edge servers without losing accuracy.The split occurs between successive layers, with different partitions deployed at different locations.
  • Transfer learning reuses teacher-network features for a student network, but performance varies with student architectures and transfer techniques.Improvement is reported when transferring intermediate- and last-layer knowledge to a shallower student, while some alternatives reduce performance.

6) Gossip Training:

Gossip training replaces centralized coordination with peer-to-peer model exchange for asynchronous, decentralized learning, but scaling introduces communication and convergence challenges.

  • Randomized gossip algorithms exchange information peer-to-peer and can converge toward consensus without centralized nodes or variables.
  • GoSGD trains DNNs asynchronously and decentrally by iterating gradient updates and mixing updates across independent nodes.
  • Gossiping SGD replaces synchronous all-reduce with gossip aggregation to retain features of synchronous and asynchronous SGD.
  • At large scale, trivial gossip algorithms cause communication imbalance, poor convergence, and heavy communication overhead.GossipGraD is introduced as a gossip communication protocol based SGD algorithm intended for practical scaling.
  • Distributed edge training must address data privacy because sources may belong to individuals or organizations with restrictions on sharing private information.Systems listed as considering privacy include FedAvg, BlockFL, and GossipGraD.

2) Device-based:

Device-based inference runs the DNN locally after obtaining the model from the edge server, avoiding inference-time communication but requiring substantial device resources.

  • In device-based inference, the mobile device obtains the DNN model from the edge server and performs inference locally without communicating during inference.
  • Local inference is reliable, but performance depends on the mobile device and requires substantial CPU, GPU, and RAM resources.
  • Edge-device: Edge-device inference partitions the DNN according to network bandwidth, device resources, and edge-server workload.The device executes initial layers, sends intermediate data, and the edge server executes the remaining layers.
  • Edge-device: Edge-device inference is described as more reliable and flexible than edge-based or device-based inference, but may require substantial mobile-device resources.
  • The four edge-centric inference modes can be combined to pool heterogeneous resources across end devices, edge nodes, and clouds.
  • Inference quality is evaluated using metrics including latency and accuracy, with latency covering preprocessing, inference, transmission, and postprocessing.Real-time applications may require deadlines such as 100ms latency.

3) Energy:

Energy efficiency is a central edge-inference concern because end devices are battery-limited; compression and deployment choices must account for computation, communication, memory, and privacy constraints.

  • DNN inference on end devices consumes substantial energy because computation and communication overheads are imposed on battery-limited hardware.Energy efficiency depends on model size and available device resources.
  • Communication overhead strongly affects inference performance in all modes except device-based inference and depends on the inference mode and available bandwidth.
  • DNN compression reduces model complexity and resource requirements, enabling local inference while reducing latency and privacy concerns.Compression targets latency, energy, privacy, and memory footprint.
  • Magnitude-based pruning reduced AlexNet and VGG16 weights by 9x and 13x, respectively, without ImageNet accuracy loss; Deep Compression reached 35-49x.
  • Reducing weight count does not necessarily produce significant energy savings because convolutional layers can dominate energy cost.The passage argues that pruning should become directly energy-aware for end devices.
  • Combining compression techniques can support fast, low-power, accurate inference, while single techniques may not meet diverse device constraints.AdaDeep formulates accuracy, latency, storage, and energy requirements as a unified optimization problem.

2) Model Partition:

Model partition reduces end-device execution pressure by distributing DNN computation across devices and edge servers. The surveyed approaches target latency, energy, privacy, memory, and accuracy through partitioning, compression, early exits, and caching.

  • Model Partition: Model partition offloads computationally intensive DNN components to edge servers or nearby devices to improve inference performance.Its main concerns are latency, energy, and privacy.
  • Server-Device Partition: Neurosurgeon and related methods select a device-server partition point while considering latency, energy efficiency, compression, and accuracy constraints.JALAD formulates partitioning as an integer linear programming problem under a guaranteed accuracy constraint.
  • Device-Device Partition: MoDNN accelerates inference by 2.17-4.28x with 2 to 4 worker nodes, while MeDNN improves inference by 1.86-2.44x and reduces additional computing and communication time.MoDNN and MeDNN partition layers horizontally across mobile devices; DeepThings instead partitions layers vertically to reduce memory footprint.
  • Model Early Exit: Early-exit methods finish inference using partial DNN execution by adding exit branches or policies that classify inputs before the final layer.BranchyNet adds exit branches, while distributed DDNNs maps exits across device, edge-server, and cloud layers.
  • Edge Caching: Edge caching accelerates inference by storing and reusing results or features at the edge, with Glimpse achieving 1.6-5.5x acceleration.Cachier addresses local-cache scale limits by supporting recognition of thousands of objects, while FoggyCache targets redundant computation for similar inputs.

5) Input Filtering:

Input filtering accelerates edge inference by removing uninformative video frames before expensive DNN processing. The surveyed systems combine lightweight detectors, staged filtering, semantic features, and cross-camera models, while broader inference design also considers model selection and resource contention.

  • Input Filtering: Input filtering removes non-target-object frames from video input to reduce unnecessary DNN inference.The technique is especially aimed at video analytics.
  • Single-Camera Filtering: NoScope skips frames with little temporal change using lightweight binary classifiers that detect relevant differences.Frames containing target objects, such as cars, are retained for DNN processing.
  • Multi-Stage Filtering: FFS-VA uses a three-stage pipeline combining background detection, target-object recognition, and Tiny-YOLO filtering.Another system accumulates intermediate features and uses Euclidean similarity to select top-k interesting frames.
  • Cross-Camera Filtering: ReXCam reduces computation workload by 4.6x and improves DNN model inference accuracy by 27% for cross-camera analytics.It uses a learned spatiotemporal model to filter video frames.
  • Related Inference Optimization: Adaptive model selection and concurrent-task scheduling extend inference optimization across latency, accuracy, energy, and resource constraints.Bayesian optimization improves minimum energy per image by up to 6x under accuracy constraints, while NestDNN schedules compact multi-capacity models.

D. Summary of Existing Systems and Frameworks

Existing edge-intelligence systems combine application-specific architectures and enabling techniques for training and inference. Their diversity creates a challenging real-time configuration problem spanning models, networking, resources, and deployment platforms.

  • System Overview: The survey organizes existing inference systems by target application, architecture, edge-intelligence level, optimization objectives, techniques, and effectiveness.These systems are summarized in Table IV.
  • System Diversity: Existing frameworks adopt different subsets of techniques tailored to specific applications and requirements.A generic system would need cooperative techniques and flexible design choices.
  • Platforms: Current edge platforms mainly relay connections to powerful cloud datacenters rather than providing comprehensive edge-AI functionality.Examples include Amazon Greengrass, Microsoft Azure IoT Edge, and Google Cloud IoT Edge.
  • Platforms and Models: Future edge-intelligence services require portability across heterogeneous nodes, while no single edge-AI programming framework currently dominates all metrics.The survey also highlights resource-aware model design using compression, AutoML, and neural architecture search.
  • Computation-Aware Networking: Distributed edge intelligence requires computation-aware networking for efficient sharing of data and computation results across edge nodes.Candidate directions include 5G URLLC, software-defined networking, network-function virtualization, gradient coding, and over-the-air computation.

D. Trade-off Design with Various DNN Performance Metrics

Edge-intelligence deployment requires balancing accuracy, latency, energy, privacy, trust, and resource demands across heterogeneous devices and services. The survey identifies high-dimensional configuration, orchestration, security, and privacy as central design challenges.

  • Performance Metrics: Standard metrics such as top-k accuracy or mean average precision do not fully reflect DNN inference runtime on edge devices.Choosing among candidate models therefore requires considering deployment-specific performance trade-offs.
  • Performance Metrics: Object-recognition performance depends on proposal count, input image size, and feature-extractor selection, and a new combination can outperform the state-of-the-art method.The finding motivates evaluating trade-offs between inference speed and accuracy.
  • Resource Orchestration: Dynamic edge environments require online orchestration and joint optimization of computation, communication, cache allocation, and model configurations.The resulting parameter space is high-dimensional and difficult to search in real time.
  • Security and Privacy: Open edge ecosystems require lightweight distributed security for authentication, access control, data and model integrity, and platform verification.Privacy-sensitive edge data also makes directly sharing original datasets across nodes risky under requirements such as GDPR.
  • Ecosystem Design: Edge intelligence involves collaboration among platform, software, device, network, data, and service providers, with blockchain proposed for decentralized coordination and revenue distribution.The survey calls for resource-friendly blockchain consensus protocols.
Loading 1905.10083v1…