Source-linked AI summary
Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey
Yiwen Xu, Tariq M. Khan, Yang Song, Erik Meijering
TL;DR
Edge deep learning addresses the difficulty of running computationally intensive deep-learning models close to data sources and in resource-constrained settings. This survey synthesizes foundations, hardware platforms, efficient model-design strategies, and computer-vision and medical-diagnostic applications, concluding that continued hardware and software advances could expand Edge DL’s impact while privacy, computational, and deployment constraints remain.
Problem
Computationally complex deep-learning models are difficult to deploy on resource-constrained edge devices, despite the need for processing close to data sources.
Method
The paper comprehensively surveys Edge DL, categorising hardware platforms and reviewing lightweight design, model compression, and applications in computer vision and medical diagnostics.
Results
The survey synthesises Edge DL’s technical merits and practical applications, including reduced latency, conserved bandwidth, and strengthened data privacy.
Takeaways & Limitations
Continued advances in Edge DL hardware and software could expand its transformative impact across applications and support broader healthcare accessibility.
Takeaways & Limitations
Edge DL remains constrained by privacy and computational limitations, while reduced precision can impair accuracy and bit-width selection depends on the network and target platform.
Abstract
from arXiv · showhide
Edge deep learning, a paradigm change reconciling edge computing and deep learning, facilitates real-time decision making attuned to environmental factors through the close integration of computational resources and data sources. Here we provide a comprehensive review of the current state of the art in edge deep learning, focusing on computer vision applications, in particular medical diagnostics. An overview of the foundational principles and technical advantages of edge deep learning is presented, emphasising the capacity of this technology to revolutionise a wide range of domains. Furthermore, we present a novel categorisation of edge hardware platforms based on performance and usage scenarios, facilitating platform selection and operational effectiveness. Following this, we dive into approaches to effectively implement deep neural networks on edge devices, encompassing methods such as lightweight design and model compression. Reviewing practical applications in the fields of computer vision in general and medical diagnostics in particular, we demonstrate the profound impact edge-deployed deep learning models can have in real-life situations. Finally, we provide an analysis of potential future directions and obstacles to the adoption of edge deep learning, with the intention to stimulate further investigations and advancements of intelligent edge deep learning solutions. This survey provides researchers and practitioners with a comprehensive reference shedding light on the critical role deep learning plays in the advancement of edge computing applications.
1 Introduction
Edge deep learning combines local edge computation with deep learning to support increasingly intelligent, responsive applications, including computer vision and medical diagnostics. This survey reviews the field’s foundations, hardware, deployment strategies, applications, and future challenges.
- Edge computing processes data near its source, improving response speed while supporting security, privacy, scalability, and distributed processing.
- Deep learning models provide powerful computer-vision capabilities, but their computational complexity challenges deployment on resource-constrained edge devices.The survey identifies model compression and lightweight design as responses to these deployment constraints.
- Edge DL deploys deep learning algorithms and models on edge devices, combining local computation with artificial intelligence across applications such as autonomous driving, industrial automation, agriculture, retail, and wildlife conservation.
- Edge computing in healthcare was valued at USD 5.28 billion in 2023 and is projected to reach USD 12.9 billion by 2028, with a 26.1% CAGR from 2022 to 2028.
- Deep-learning-based medical devices are seeing steady adoption, highlighting potential improvements in the efficiency and accessibility of medical diagnostics.
- The survey focuses on Edge DL applications in computer vision generally and medical diagnostics particularly, complementing earlier surveys with deeper domain coverage.
2 Edge Computing Overview
Edge computing moves processing, storage, and analysis closer to data sources through several distributed paradigms and supports low-latency, bandwidth-efficient applications. Edge model inference and deployment require compression, lightweight models, and adaptation to constrained and heterogeneous devices.
- Edge computing processes distributed data near its origin rather than in distant cloud servers, reducing latency and improving processing efficiency for latency-sensitive applications.
- Related paradigms and terminologies: Cloudlets provide small-scale edge cloud hubs, fog computing distributes processing across network nodes, and MEC brings resources closer to mobile devices for real-time processing.
- Related paradigms and terminologies: Edge DL performs deep learning inference and training near data sources to address computational and latency challenges in centralised cloud-based frameworks.
- Related paradigms and terminologies: Edge analytics processes source-generated data locally, reducing latency and bandwidth usage while enabling real-time insight.
- Related paradigms and terminologies: Cloud-edge collaboration integrates both paradigms to support data processing, analytics, storage, resource utilisation, and application performance.
- Edge model lifecycle: Edge model training includes communication-efficient schemes and privacy-preserving federated learning, although federated learning requires computational and communication resources.
- Edge model lifecycle: Edge model deployment supports real-time processing through model-management and deployment platforms, while adaptive architectures can adjust after deployment to diverse environments.
- Edge model inference: Model inference on edge devices depends on executing complex computations within limited resources, motivating model compression and lightweight model adoption.
3 Edge Computing Devices
The survey classifies edge devices by computational capability into low-end, medium-range, and high-performance categories, linking each to distinct deployment scenarios. This framework helps match hardware specifications and device diversity to application requirements.
- Device classification: Edge devices are classified into low-end, medium-range, and high-performance categories according to computational capabilities.The classification supports comparison of device specifications, functions, and use cases.
- Low-end devices: Low-end devices prioritize energy efficiency and compact form factors, supporting preprocessing, basic inference, and low-complexity IoT tasks.They commonly have constrained memory and lack advanced accelerators.
- Medium-range devices: Medium-range devices combine multicore processors with some GPU acceleration for image recognition, object detection, and compact-model applications.They are used in robotics, smart cameras, and other scenarios requiring moderate computational capability.
- High-performance devices: High-performance devices provide powerful CPUs, GPUs, and substantial memory for complex deep-learning training and inference.The YOLOv3 comparison highlights Jetson Xavier NX for FPS and inference time, while Jetson Nano balances performance and energy efficiency.
- Deployment considerations: Medium-range and low-end devices are constrained for training but remain suitable for space-constrained or mobile deployments because of their compactness and portability.Local processing also reduces data transmission, supporting privacy and security.
- Device selection: The survey presents device examples across the capability spectrum and emphasizes selecting hardware according to computational needs and application requirements.Examples include microcontrollers, specialized platforms, smartphones, laptops, automobiles, and smart-home devices.
4 Transferring Large DNNs to Lightweight DNNs
Lightweight neural-network architectures address the challenge of deploying deep neural networks on resource-constrained edge devices. They balance computational efficiency and model complexity through smaller memory requirements, faster inference, and lower energy consumption.
- Lightweight design: Lightweight neural networks are designed to deploy deep neural networks on devices with limited computational and memory resources.Their purpose is to balance computational efficiency with model complexity.
- Benefits: Compared with larger models, lightweight architectures provide a smaller memory footprint, faster inference, and reduced energy consumption.
4.1 Network Pruning
Network pruning compresses deep neural networks by removing redundant weights, neurons, channels, or filters while seeking to preserve accuracy. Structured approaches such as channel and filter pruning improve hardware deployability, whereas unstructured weight pruning can achieve substantial complexity reductions but may require specialized hardware.
- Pruning overview: Pruning identifies and removes redundant neural-network elements to reduce computational demands and model size while maintaining accuracy.The survey discusses weight, neuron, channel, and filter pruning.
- Weight pruning: Unstructured weight pruning achieved 9× and 13× complexity reductions in AlexNet and VGG16, respectively.The process trains the network, removes weights below a threshold, and uses the original weights to initialize retraining.
- Neuron pruning: Neuron pruning removes entire neurons or filters with their associated weights and typically requires iterative fine-tuning to recover performance.Pruning criteria identify neurons considered less important or redundant.
- Channel pruning: Channel pruning reduced ResNet-50 parameters by 50% while maintaining 99% of original ImageNet accuracy.Its structured sparsity supports deployment on CPUs and GPUs without requiring specialized inference engines.
- Channel pruning: Channel pruning reduced MobileNetV1 FLOPs by 2× while maintaining 70.9% top-1 ImageNet accuracy, demonstrating hardware flexibility.
- Hybrid compression: Channel pruning can combine with quantisation and knowledge distillation; one pruned and quantised ResNet-34 reached 69.86% top-1 ImageNet accuracy with 5× smaller model size.
- Filter pruning: Filter pruning accelerated ResNet-50 by 2× while maintaining 98.6% of its original ImageNet accuracy.Filter pruning removes redundant filters to create more compact models for deployment across hardware platforms.
- Filter pruning: Learning-based filter pruning reduced ResNet-56 FLOPs by 60.3% while maintaining 99% of its original CIFAR-10 accuracy.The survey identifies this as suitable for real-time edge applications.
4.2 Sparse Representation
Sparse representation reduces neural-network redundancy by promoting zero weights or activations, supporting compact models for resource-constrained deployment. The section also reviews pruning methods and their accuracy–efficiency trade-offs.
- Sparse representation uses relatively few nonzero elements to reduce redundancy and produce more compact, interpretable models.It can also improve generalisation by removing less essential connections and activations.
- L1 regularisation and ReLU activations are common ways to induce sparsity during training and inference.L1 regularisation penalises nonessential parameters, while ReLU can naturally create sparse activations.
- Sparse networks require less computation and memory, improving suitability for resource-constrained environments while supporting interpretability and robustness.Nonzero weights and activations can be associated with influential features and connections.
- Sparse-network optimisation still requires selecting an appropriate sparsity level and managing potential performance trade-offs.Research is pursuing improved pruning algorithms, adaptive sparsity, and methods to mitigate sparse-model drawbacks.
- 3× fewer parameters and 2× fewer FLOPs were achieved by structured pruning of ResNet-50 while retaining 97% of original ImageNet accuracy.The approach pruned entire filters or layers, preserving structural integrity for real-time edge image classification.
- 73.1% top-1 ImageNet accuracy was achieved by combining quantisation and sparsity on MobileNetV2, alongside reductions in model size and inference latency.The resulting model was presented as suitable for mobile real-time image recognition and augmented-reality applications.
4.3 Quantised Neural Network
Quantised neural networks reduce parameter and activation precision to improve efficiency on resource-constrained devices. The section covers quantisation methods, representative edge-deployment results, and the accuracy and bit-width choices that constrain their use.
- Quantised neural networks map high-precision parameters to discrete lower-bit representations, such as 8-bit integers, trading accuracy against efficiency.The approach is designed for resource-constrained environments.
- Weights and activations can be quantised, with binarisation representing the most extreme reduction in precision.These techniques reduce the numerical representation used throughout the network.
- Quantisation reduces memory requirements, accelerates inference, and lowers computational resource and energy demands for edge and mobile applications.These benefits support real-time operation on devices with limited memory, compute, or power.
- Reduced precision can lower accuracy, especially for fine-grained tasks, while selecting an appropriate bit-width depends on the network and target platform.Quantisation-aware training is identified as a mitigation for accuracy degradation.
- Mixed-precision, learning-aware, hardware-aware, adaptive, and combined quantisation methods aim to balance efficiency with acceptable accuracy.Adaptive quantisation dynamically changes bit-widths during training, while other frameworks combine quantisation with pruning or knowledge distillation.
- 75.3% top-1 ImageNet accuracy was achieved by mixed-precision ResNet-50 quantisation while reducing model size by 10× and inference latency versus the full-precision model.The result was reported as suitable for real-time image recognition on mobile devices.
- 72.2% top-1 ImageNet accuracy was achieved by hardware-aware MobileNetV2 quantisation with 220 million FLOPs and significant energy savings.The approach targets edge devices with limited power resources.
4.4 Efficient Lightweight Neural Networks
Efficient lightweight neural networks reduce computation, parameters, or latency while maintaining competitive vision accuracy for mobile, edge, and resource-constrained applications. The surveyed designs span compact CNNs, pruning, dynamic channels, attention mechanisms, and lightweight transformers.
- MobileNet: MobileNet architectures use efficient designs to balance model size, accuracy, and on-device performance across successive versions.MobileNetV1 introduced depthwise separable convolutions; MobileNetV2 added inverted residual blocks and linear bottlenecks, while MobileNetV3 introduced h-swish activation.
- MobileNet: 70.6% top-1 accuracy with 569 million FLOPs was reported for MobileNetV1, while MobileNetV2 reached 72.0% with 300 million MAdds and MobileNetV3-Large reached 75.2% with 219 million MAdds.MobileNetV3-Large was described as suitable for real-time image recognition and augmented reality on mobile and edge devices.
- EfficientNet: 77.1% top-1 accuracy with 5.3 million parameters and 390 million FLOPs was reported for EfficientNet-B0 on ImageNet.EfficientNet was designed to reduce model complexity while maintaining performance in mobile and edge applications.
- Group Convolutional Networks: ShuffleNetV2 achieved 69.4% top-1 accuracy on ImageNet with 146 million FLOPs by using group convolutions, channel shuffling, and residual connections.CondenseNet used pruning and condensation, while MixNet used mixed depthwise convolutions to address resource constraints.
- Squeeze & Excitation: SENet improved MobileNetV2 top-1 accuracy from 70.6% to 74.7% on ImageNet through squeeze-and-excitation channel recalibration.SqueezeNet reduced parameters with small 1×1 convolutions and fire modules, while DiCENet dynamically adjusted active channels at runtime.
- Mobile Transformers: MobileViT achieved 78.4% top-1 accuracy on ImageNet-1k with approximately 6 million parameters, surpassing MobileNetV3 by 3.2% and DeiT by 6.2%.EdgeViT-S combined self-attention and convolutions, reporting 81.0% on ImageNet with 85.3±3.9 ms latency on Samsung Galaxy S21.
4.5 Neural Architecture Search
Neural architecture search automates network design and supports efficient architectures for edge deployment by balancing accuracy, computational cost, and latency. The survey reviews progressive, multi-objective, factorised, and evolutionary NAS approaches, alongside practical edge applications.
- NAS foundations: NAS uses search algorithms to automate neural network architecture design for specific tasks, reducing reliance on manual trial and error.The approach targets architectures that are efficient, effective, and adaptable to application and hardware constraints.
- Progressive search: PNASNet progressively increases network complexity, achieving a 3.41% top-1 error rate on CIFAR-10 with 400 GPU hours for architecture search.This compares with 2,000 GPU hours for NASNet.
- Latency-aware search: MNASNet incorporates latency into multi-objective optimisation and achieved 75.2% top-1 accuracy on ImageNet with 312 billion MAdds.Its design targets mobile and edge devices with limited computational resources.
- Latency-aware search: FBNet uses factorised architecture search that decouples operation selection from connectivity, making its search space more manageable.The surveyed approach targets efficient neural network architectures through AI-assisted search.
- Evolutionary search: AmoebaNet applies evolutionary search and achieved 96.7% top-1 accuracy on CIFAR-10 and 83.9% on ImageNet.The method explores a wide search space and optimises performance alongside computational cost.
- Practical edge deployment: Edge applications use NAS to balance accuracy, latency, memory, and power, including real-time drone navigation and wearable physiological monitoring.The wearable system uses quantisation and network pruning to reduce model size and power consumption.
5 Computer Vision Edge Applications
Computer vision edge applications span transportation, manufacturing, agriculture, retail, environmental monitoring, and other domains requiring real-time processing. The surveyed systems use lightweight models, multimodal sensing, specialised feature processing, and edge deployment for detection, segmentation, monitoring, and conservation.
- Application domains: Edge computer vision is surveyed across autonomous driving, manufacturing, agriculture, retail, wildlife conservation, smart cities, public safety, and sports analytics.The applications emphasise real-time processing across diverse edge devices.
- Autonomous driving and transportation: Autonomous transportation systems use edge vision for real-time object detection, tracking, classification, and traffic-video segmentation.Approaches include YOLOv4 adaptations, RGB-camera and LiDAR fusion, spatiotemporal interest points, and multimodal feature combinations.
- Application examples: Figure 9 illustrates edge vision examples including RGB-camera and LiDAR driving, MobileNet-based container-defect detection, drone pest classification, and mobile-robot retail monitoring.The figure presents representative applications across transportation, manufacturing, agriculture, and retail.
- Autonomous driving and transportation: TrafficSensor uses a calibrated camera, neural networks, and tracking algorithms to classify highway vehicles under poor lighting and adverse weather.The system supports infrastructure planning and traffic management with real-time surveillance.
- Smart manufacturing and industrial automation: Industrial edge vision targets real-time defect detection, using lightweight segmentation and detection networks for tyre X-rays, sanitary ceramics, polarisers, and insert moulding.Reported techniques include MobileNet backbones, feature pyramids, attention modules, Ghost modules, and transformer-based feature extraction.
- Agriculture and crop monitoring: UAV-acquired thermal imagery showed strong correlations between vegetation indices and canopy temperature with fruit yield and leaf stomatal conductance.The study describes thermal imagery as a rapid tool for real-time crop-stress assessment.
- Environmental monitoring and wildlife conservation: Wildlife conservation systems deploy object detection and classification models on camera-equipped edge devices, including wireless updates and active learning in remote locations.Another camera-trap method addresses cluttered images using candidate animal proposals generated with multilevel graph cuts in the spatiotemporal domain.
- Environmental monitoring and wildlife conservation: Urban environmental analysis uses the Normalised Difference Bareness Index to examine how Shenzhen’s expansion affects temperature distribution and the urban heat island effect.The study provides data relevant to urban planning and mitigation of urban heat island effects.
6 Medical Edge Applications
Edge deep learning supports real-time medical diagnosis on resource-constrained devices, with applications spanning gastrointestinal, pulmonary, dermatological, pathological, and telemedicine settings. Surveyed systems combine lightweight models and edge hardware to deliver efficient diagnostic analysis.
- Edge devices such as NVIDIA Jetson and Raspberry Pi support real-time medical diagnostic tasks under power and resource constraints.These platforms have been used for skin and cervical cancer detection and for chest CT and dermatological analysis.
- Gastrointestinal Diagnosis: Traditional colonoscopy has a polyp miss rate as high as 25%, motivating computer-aided systems for earlier adenoma detection.Variation in practitioner skill and fatigue contributes to missed polyps and variability in adenoma detection rates.
- Gastrointestinal Diagnosis: Lightweight models are essential because weaker or absent AI accelerators make direct deployment of conventional deep learning models impractical on diagnostic devices.Low inference rates per second can reduce computer-aided diagnosis efficiency, motivating lightweight architectures.
- Gastrointestinal Diagnosis: On-site edge devices can provide immediate texture and hardness insights for resected colorectal cancer polyps, linking data collection directly with interpretation.This illustrates edge computing’s practical role in histopathological gastrointestinal assessment.
- Pathological Images Analysis: ReducedFireNet occupies 0.391 MB and requires 0.201 GFLOPS while achieving 96.88% average accuracy and a 0.968 F1 score on malignant lymphoma images.The model was designed for histopathological classification on edge devices with limited processing capabilities.
- Pathological Images Analysis: Embedded cervical-cell detection achieved 94.1% accuracy on an NVIDIA Jetson TK1 using optimized convolutions, parameter compression, and deeper feature expression.The model used fewer parameters than ResNet18 and MobileNet.
- Medical Edge Applications: Fast R-CNN-based diabetic foot ulcer detection achieved 91.8% average precision on an NVIDIA Jetson TX2 with 48 ms inference per image.The system used a 1,775-image expert-annotated dataset and transfer learning.
- Telemedicine: SR-Telemedicine enhances very low-resolution video to 720p or 1080p while dynamically adapting resolution through a scalable neural network and DDQN.The approach targets high-quality physician video without extensive network bandwidth.
7 Future Directions
The survey identifies privacy, security, energy, performance, and architectural adaptability as major future directions for edge deep learning. It emphasizes robust, efficient, and privacy-preserving systems for computer vision and medical applications.
- Future Edge DL research should address privacy and security, energy efficiency, adaptive architectures, multimodal fusion, efficient inference, explainability, quantum-inspired methods, and human-centric applications.These areas define the survey’s proposed development agenda for computer vision and medical applications.
- Privacy and Security: Adversarial perturbations can mislead edge models and cause dangerous medical misdiagnoses, motivating adversarial training for improved robustness.Adversarial samples are incorporated during training to strengthen model resilience.
- Privacy and Security: Homomorphic encryption and secure multiparty computation are privacy-preserving approaches for protecting training data and model weights in healthcare.Patient confidentiality is especially important because breaches can affect trust and safety.
- Energy Efficiency and Performance Optimisation: Hardware and software energy optimisation is critical for continuous monitoring, portable devices, and accurate real-time diagnosis in resource-constrained medical settings.Wearable ECG monitors and portable ultrasound scanners require energy-efficient edge processors.
- Energy Efficiency and Performance Optimisation: Lightweight networks, data compression, energy-aware scheduling, adaptive voltage scaling, and power gating reduce computational and storage demands while sustaining performance.Energy-aware scheduling dynamically allocates tasks according to device energy state and computational demand.
7.3 Adaptive Edge Computing Architectures
Adaptive edge architectures tailor neural-network computation to changing environments, application requirements, and device resources. The survey highlights hardware-aware search, dynamic network scaling, and input-dependent pruning as key mechanisms.
- Hardware-aware neural architecture search optimizes network design using device-specific metrics while respecting resource constraints.This approach is positioned as a key tool for adapting architectures to target hardware.
- Slimmable Neural Networks dynamically adjust network width during inference, allowing one model to operate from full capacity to lightweight configurations.The selected scale depends on real-time computational resource availability.
- Resource-aware dynamic pruning removes unimportant channels during inference, reducing computational complexity for less demanding inputs.The strategy jointly targets power consumption and performance on edge devices.
7.4 Multimodal Edge Computing Fusion Techniques
Multimodal edge fusion combines heterogeneous healthcare data to support comprehensive assessment and complex decisions. Transformer-based fusion can integrate visual and textual modalities with limited preprocessing, while efficient inference also depends on scheduling across hardware and cloud resources.
- Healthcare multimodal edge computing integrates medical images, electronic health records, and wearable-sensor data for comprehensive patient assessment.The central challenge is fusing diverse modalities such as vision, audio, and text for complex decision-making.
- ViLT represents images and text in a joint embedding space, supporting medical report generation and cross-modal retrieval with minimal preprocessing.Its simplified cross-modal interactions illustrate an efficient transformer-based fusion strategy.
- Efficient multimodal edge inference requires energy management and task scheduling across CPUs, GPUs, FPGAs, and collaborative edge-cloud resources.Hardware allocation and collaboration complement model-level multimodal fusion.
- SparseNN skips zero-value computations, while ShiftAddNet replaces convolutional multiplications with shift-and-add operations to reduce inference load.Both methods target faster and more efficient execution on constrained edge hardware.
7.6 Explainable AI in Edge Models
The section highlights explainability as important for transparent, trusted, and compliant AI, especially in healthcare, while also discussing quantum-inspired approaches for edge medical diagnostics.
- Explainable AI supports transparency, trust, and regulatory compliance in sensitive applications such as healthcare and autonomous driving.
- LIME and SHAP provide visual explanations showing how features influence model decisions.
- In healthcare, understandable model reasoning assists clinical decision-making and fosters trust among patients and providers.
- Quantum-inspired edge computing applies quantum-computing principles to improve computational methods and resource allocation for medical diagnostics.
7.8 Human-Centric Edge Applications
Human-centric edge applications center technology on human needs, enabling instantaneous health-data analysis and personalized services while raising privacy and security challenges. Learning-based medical-image diagnostics provide complementary advanced methods, including transformers and generative models.
- Human-Centric Edge Applications: Human-centric edge applications use portable devices for instantaneous health-data analysis, personalized feedback, and health advice.
- Human-Centric Edge Applications: These applications can support early health-issue identification and proactive health management.
- Human-Centric Edge Applications: Personalized services in human-centric edge systems require robust security protocols and careful handling of sensitive user data.
- Learning-Based Diagnostics: Medical-image diagnostics increasingly use transformers, contrastive learning, few-shot learning, transfer learning, domain adaptation, and generative models.
8 Conclusion
The conclusion presents Edge DL as a framework for real-time data processing and interpretation near the source in resource-constrained settings, particularly for computer vision and medical diagnostics. It anticipates growing impact from continued hardware and software advances, including potential contributions to universal healthcare accessibility.
- Edge DL enables real-time processing and interpretation of data close to its source in resource-constrained settings.
- The survey focuses on Edge DL’s foundational concepts and technical merits in computer vision and medical diagnostics.
- Continuing hardware and software advances are expected to expand Edge DL’s transformative impact across applications.
- The paper links Edge DL’s prospective societal contribution to improved daily life and universal healthcare accessibility.