Source-linked AI summary
Communication-Computation Trade-Off in Resource-Constrained Edge Inference
Jiawei Shao, Jun Zhang
TL;DR
Resource-constrained edge inference must balance device computation against communication of intermediate features. This paper proposes a three-step device-edge co-inference framework and reports a better trade-off with lower latency than baselines, while noting manual pruning setup and encoder overhead as limitations.
Problem
Edge inference is constrained by limited device computation and excessive communication overhead, motivating a better trade-off for co-inference.
Method
The framework selects a split point, applies communication-aware on-device model compression, and uses task-oriented feature encoding for intermediate features.
Results
Classification simulations verify a better communication-computation trade-off and much lower latency than other baselines.
Takeaways & Limitations
The framework supports effective co-inference across varying device computation capabilities and available bandwidth.
Takeaways & Limitations
Incremental pruning requires manual sparsity-ratio setup and many training iterations, while feature encoding introduces extra DNNs.
Abstract
from arXiv · showhide
The recent breakthrough in artificial intelligence (AI), especially deep neural networks (DNNs), has affected every branch of science and technology. Particularly, edge AI has been envisioned as a major application scenario to provide DNN-based services at edge devices. This article presents effective methods for edge inference at resource-constrained devices. It focuses on device-edge co-inference, assisted by an edge computing server, and investigates a critical trade-off among the computation cost of the on-device model and the communication cost of forwarding the intermediate feature to the edge server. A three-step framework is proposed for the effective inference: (1) model split point selection to determine the on-device model, (2) communication-aware model compression to reduce the on-device computation and the resulting communication overhead simultaneously, and (3) task-oriented encoding of the intermediate feature to further reduce the communication overhead. Experiments demonstrate that our proposed framework achieves a better trade-off and significantly reduces the inference latency than baseline methods.
I. INTRODUCTION
Edge inference faces a trade-off between limited device computation and costly transmission of intermediate features. The paper proposes a three-step co-inference framework combining split-point selection, communication-aware compression, and task-oriented feature encoding.
- Edge inference remains less investigated than edge training despite growing demand for DNN services on resource-constrained devices.
- On-device inference is limited by computation and accuracy, while server-based inference incurs communication overhead and privacy concerns.
- Device-edge co-inference splits a DNN between the device and edge server, making split-point selection critical to the resulting computation and communication costs.
- The framework combines model splitting, communication-aware model compression, and task-oriented feature encoding to reduce co-inference latency.
- Incremental pruning reduces redundant weights and computation, while two-step feature encoding learns a compact intermediate representation to reduce communication latency.
- Classification experiments show that the framework improves the communication-computation trade-off and reduces edge inference latency relative to baselines.
II. COMMUNICATION-COMPUTATION TRADE-OFF IN EDGE INFERENCE
The section frames device-edge co-inference as a communication-computation trade-off and presents split-point selection as the baseline approach. It then motivates model compression and feature encoding as complementary improvements.
- Split-point selection provides a rudimentary trade-off between local computation and communication overhead in device-edge co-inference.
- The framework incorporates three techniques to improve this trade-off and support more effective edge inference.
A. Communication-Computation Trade-Off and Model Splitting
Model splitting controls the division of computation and communication, but data amplification limits split-only methods. Communication-aware pruning therefore targets both redundant on-device computation and the size of transmitted features.
- Device-edge co-inference splits a DNN across device and server, using intermediate features to control local model size and transmission cost.
- Split-point selection alone achieves limited latency gains because intermediate features can be larger than input data through data amplification.
- Communication-aware compression prunes on-device parameters while accounting for the output feature size that affects communication latency.
- Incremental network pruning is used as a magnitude-based compression method focused on the on-device network.
C. Task-Oriented Feature Encoding for Better Trade-Off
Task-oriented feature encoding compresses intermediate features according to inference needs rather than reconstructing all transmitted data. The framework couples encoding with split selection and pruning, with parameters manually tuned for each edge environment.
- Intermediate features contain sparsity and task-irrelevant information, motivating a lightweight DNN encoder trained with the other network layers.
- Unlike traditional coding, task-oriented encoding can discard information unrelated to the inference task instead of reconstructing the transmitted data.
- The framework combines splitting, incremental pruning, and feature compression while tuning split position, sparsity, and compression ratio under an accuracy threshold.
A. Model Compression for DNNs
DNN compression for device-edge co-inference must reduce both on-device computation and the communication overhead caused by transmitting intermediate features. The framework uses structured, communication-aware pruning with iterative channel-wise sparsity and activation pruning.
- Parameter pruning reduces DNN memory footprint and computational cost, but device-edge co-inference requires accounting for communication overhead as well.
- Structured pruning creates regular sparsity patterns, such as channel-wise or kernel-wise pruning, that can accelerate DNN computation.
- The framework uses incremental, magnitude-based pruning during training to achieve high compression with limited performance loss.
- Targeted channel-wise sparsity ratios are set at each pruning iteration, with higher output-layer sparsity reducing communication overhead.
- Each pruning iteration sorts weights by output-channel l2-norm, masks the smallest Si% of weights, updates only unmasked weights, and then restores masked values.
IV. TASK-ORIENTED FEATURE ENCODING
Task-oriented feature encoding reduces communication overhead by optimizing intermediate-feature transmission for inference accuracy rather than faithful data recovery. The proposed approach combines dimension reduction with learning-driven coding in an end-to-end framework.
- Communication-aware compression alone has limited ability to reduce overhead because intermediate features exhibit high dimension-wise correlation.
- Traditional and communication-oriented coding methods target lossless recovery or lossy reconstruction with tolerable distortion, whereas task-oriented encoding targets the inference task.
- The proposed two-step encoder first reduces intermediate-feature dimension, then applies learning-driven coding to map symbols to compact codewords.
1) Dimension Reduction:
The feature-encoding design uses lightweight neural networks for dimension reduction and offers two learning-driven coding alternatives. These components are trained to preserve inference accuracy while reducing transmitted feature data.
- Dimension Reduction: A lightweight encoder-decoder pair reduces intermediate-feature dimension with less than 1% additional computation compared with the original network.
- Learning-Driven Coding: The second encoding step learns a mapping from source symbols to codewords using either source coding or joint source-channel coding.
- Learning-Driven Source Coding: Learning-driven source coding quantizes floating-point intermediate features into fewer bits using learned discrete codewords instead of predefined rounding or truncation codewords.
- Learning-Driven JSCC: Learning-driven joint source-channel coding models channel noise during training and can reduce redundancy relative to separate channel coding.
A. Experimental Setup
The evaluation uses ResNet18 for CIFAR-10 image classification and measures the accuracy-latency trade-off in edge inference. Latency is related to on-device computation and communication overhead, which are quantified using FLOPs and transmitted data or time.
- The experiment evaluates the framework on CIFAR-10 image classification with ResNet18, using 50,000 training images and 10,000 testing images.
- The original ResNet18 reaches around 95% accuracy, while the edge-inference accuracy threshold is set to 93%.
- End-to-end latency is mainly influenced by on-device computation and communication overhead.
- The study compares the original split network without compression against pruning-based baselines to examine the communication-computation trade-off.
B. Communication-Computation Trade-Off
The proposed framework achieves a better communication-computation trade-off than baselines, reducing both on-device computation and communication overhead. Its latency remains robust under poor communication, although gains are marginal when computation is extremely limited.
- For every point on the proposed curve, no other curve simultaneously achieves lower computational cost and communication overhead.The framework is Pareto-superior to the three baselines in the plotted trade-off.
- The method reduces data amplification and requires less on-device computation than the compared approaches.2-Step Pruning leaves few useful split points, while BottleNet++ reduces feature size but retains excessive on-device computation.
- At very low on-device computation, the proposed method provides only marginal performance gains because few layers cannot form a low-entropy representation.A lightweight auto-encoder also struggles with the large intermediate feature produced by data amplification.
- The study identifies neural network architecture design as an important future direction for edge AI.This follows the limited gains observed in the low-computation region.
D. Performance of Learning-Driven JSCC over Noisy Channels
The paper evaluates learning-driven JSCC for transmitting intermediate features over noisy channels. It reports communication overhead comparable to source coding with optimal channel coding and substantially better than two other baselines.
- The noisy-channel experiment compares learning-driven coding and JSCC over AWGN and Binary Symmetric Channels.Noise is characterized by signal-to-noise ratio for AWGN and bit-flipping rate p for BSC.
- JSCC achieves communication overhead slightly lower than or comparable with learning-driven source coding using optimal channel coding.The comparison uses Shannon capacity as the communication rate for baseline methods.
- JSCC outperforms the input-data and original-network intermediate-feature baselines by a large margin in communication overhead.The study compares against both the input data and the original network’s intermediate feature.
- JSCC has much lower encoding and decoding complexity because it is constructed by a DNN.The reported comparison indicates potential reductions in communication overhead, processing time, and energy.
VI. CONCLUSIONS AND FUTURE WORKS
The framework combines model splitting, model compression, and feature encoding to improve the communication-computation trade-off and reduce co-inference latency. Its remaining limitations motivate automated architecture, encoding, and hyperparameter search.
- The three-step framework integrates model splitting, model compression, and feature encoding for supervised edge inference.
- Classification simulations achieved a better communication-computation trade-off and much lower latency than other baselines.
- Incremental pruning requires manually setting sparsity ratios and many training iterations.
- Feature encoding adds computational cost, while discretization may cause performance loss.
- Manual hyperparameter tuning and brute-force search make model selection time-consuming across edge environments.
- Future work includes neural architecture search, information-theoretic feature encoders, and reinforcement-learning-based hyperparameter search.