PaperlayerView exact revision on arXiv

Source-linked AI summary

BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems

Jiawei Shao, Jun Zhang

arXiv:1910.14315v5cs.LGeess.SPstat.ML

TL;DR

Resource-constrained devices need to balance DNN computation against communication when inference is split with an edge server, but prior designs separated splitting, compression, and communication. BottleNet++ jointly compresses and transmits intermediate features with channel-aware CNN encoder and decoder components, achieving up to 64× AWGN bandwidth reduction and up to 256× BEC bit compression with less than 2% accuracy degradation. Its higher compression ratio enables earlier splitting and up to 3× lower on-device computation than other compression methods.

  • Problem

    Device-edge co-inference must balance on-device computation and communication, while existing designs decoupled model splitting, feature compression, and communication.

  • Method

    BottleNet++ uses an end-to-end encoder, non-trainable channel layer, and decoder that implement channel-aware joint source-channel coding with CNNs.

  • Results

    BottleNet++ achieves up to 64× bandwidth reduction on AWGN and up to 256× bit compression ratio on BEC, with less than 2% accuracy degradation versus direct feature transmission.

  • Takeaways & Limitations

    Higher compression enables earlier DNN splitting and up to 3× reduction in on-device computation compared with other compression methods.

Abstract

from arXiv · show

The emergence of various intelligent mobile applications demands the deployment of powerful deep learning models at resource-constrained mobile devices. The device-edge co-inference framework provides a promising solution by splitting a neural network at a mobile device and an edge computing server. In order to balance the on-device computation and the communication overhead, the splitting point needs to be carefully picked, while the intermediate feature needs to be compressed before transmission. Existing studies decoupled the design of model splitting, feature compression, and communication, which may lead to excessive resource consumption of the mobile device. In this paper, we introduce an end-to-end architecture, named BottleNet++, that consists of an encoder, a non-trainable channel layer, and a decoder for more efficient feature compression and transmission. The encoder and decoder essentially implement joint source-channel coding via convolutional neural networks (CNNs), while explicitly considering the effect of channel noise. By exploiting the strong sparsity and the fault-tolerant property of the intermediate feature in a deep neural network (DNN), BottleNet++ achieves a much higher compression ratio than existing methods. Furthermore, by providing the channel condition to the encoder as an input, our method enjoys a strong generalization ability in different channel conditions. Compared with merely transmitting intermediate data without feature compression, BottleNet++ achieves up to 64x bandwidth reduction over the additive white Gaussian noise channel and up to 256x bit compression ratio in the binary erasure channel, with less than 2% reduction in accuracy. With a higher compression ratio, BottleNet++ enables splitting a DNN at earlier layers, which leads to up to 3x reduction in on-device computation compared with other compression methods.

I. INTRODUCTION

Device-edge co-inference balances on-device computation and communication by splitting a DNN, but effective intermediate-feature compression remains essential. BottleNet++ jointly compresses and transmits these features while adapting to channel conditions, achieving substantial bandwidth and computation reductions with limited accuracy loss.

  • I. INTRODUCTION: Device-edge co-inference splits a DNN between a mobile device and edge server to balance computation and communication overhead.The intermediate feature produced on the device is transmitted for continued processing at the edge.
  • I. INTRODUCTION: Feature compression affects both communication overhead and on-device computation because early-layer features can exceed the input data size.Without effective compression, splitting must occur deeper in the network, increasing device-side computation.
  • I. INTRODUCTION: BottleNet++ combines an encoder, non-trainable channel layer, and decoder in an end-to-end architecture for intermediate-feature compression and transmission.Its lightweight CNNs implement joint source-channel coding and explicitly model channel effects.
  • I. INTRODUCTION: Adaptive encoder coding supports different channel conditions and graceful accuracy degradation over noisy channels.The encoder receives channel-condition parameters as input.
  • I. INTRODUCTION: Up to 64× bandwidth reduction on AWGN and up to 256× bit compression ratio on BEC are achieved with less than 2% accuracy degradation versus direct feature transmission.BottleNet++ also enables earlier DNN splitting, producing up to 3× lower on-device computation than other compression methods.

II. PRELIMINARY

Device-edge co-inference uses both mobile devices and edge servers to execute DNNs, targeting a better tradeoff between communication and on-device computation. This setting motivates increasing compression of intermediate features.

  • II. PRELIMINARY: Device-edge co-inference divides DNN execution across a mobile device and edge server to improve the tradeoff between communication and on-device computation.
  • II. PRELIMINARY: The paper focuses on increasing the compression ratio of the intermediate feature transmitted between the device and edge.

A. Network Splitting and Feature Compression

Prior device-edge co-inference methods addressed splitting and feature compression largely through separate source-coding designs that assumed reliable communication. BottleNet++ instead targets inference performance directly through joint source-channel coding and neural-network fault tolerance.

  • A. Network Splitting and Feature Compression: Network splitting and feature compression are both critical because the splitting point must reduce on-device communication and feature-transmission latency.
  • A. Network Splitting and Feature Compression: Earlier methods primarily used source coding while assuming reliable wireless communication and seeking accurate intermediate-feature recovery.Their objectives included lossless recovery or tolerable distortion of the transmitted feature.
  • A. Network Splitting and Feature Compression: Device-edge co-inference can tolerate unreliable intermediate-feature transmission when inference performance is not seriously affected.
  • B. Joint Source-Channel Coding: BottleNet++ uses an end-to-end joint source-channel coding design that exploits DNN fault tolerance for feature compression.The paper identifies this as an opportunity distinct from restoring the transmitted message at the receiver.
  • B. Joint Source-Channel Coding: Learning-based joint source-channel coding studies focused on message restoration, whereas BottleNet++ aligns transmission design with device-edge inference performance.

C. Fault-Tolerance Property of Neural Networks

BottleNet++ exploits neural-network fault tolerance by incorporating channel effects into the DNN pipeline. Its encoder, channel model, and decoder compress, transmit, and restore intermediate features at a selected splitting point.

  • C. Fault-Tolerance Property of Neural Networks: Neural-network fault tolerance permits relaxing transmission reliability to reduce communication overhead in device-edge co-inference.The paper attributes this property to neural networks containing more processing elements than necessary for a task.
  • C. Fault-Tolerance Property of Neural Networks: BottleNet++ inserts a non-trainable wireless-channel layer between a lightweight CNN encoder and decoder at the DNN splitting point.The architecture compresses and transmits cubelike intermediate feature tensors.

A. Encoder

The encoder compresses intermediate DNN features and performs joint source-channel coding through a lightweight CNN, while channel-specific noise models support transmission over AWGN and BEC channels.

  • A. Encoder: The encoder applies lossy compression to reduce the dimension of the DNN’s intermediate feature tensor.Its convolutional layer uses filter count, stride, and kernel size to control output channels and spatial dimensions.
  • A. Encoder: The encoder and decoder together implement joint source-channel coding using lightweight convolutional neural networks.The encoder includes convolution, batch normalization, and activation layers.
  • A. Encoder: The BEC models channel quality with bit erasure rate p and has binary input with ternary output.The erasure bit is assigned the average of the bit values 0 and 1.

C. Decoder

The decoder reconstructs the compressed intermediate feature after channel corruption, while the training strategy uses staged optimization to improve convergence and supports modular retraining for changed channel conditions.

  • C. Decoder: The decoder maps the corrupted feature to a restored feature tensor with the same dimension as before compression.It uses deconvolution, batch normalization, and ReLU activation.
  • C. Decoder: The decoder’s deconvolutional filters and strides restore channel, width, and height dimensions from the compressed representation.The restoration settings mirror the encoder’s spatial transformation.
  • C. Decoder: BottleNet++ uses convolutional and deconvolutional networks instead of fully connected layers because fully connected compression has unacceptable memory and computation costs.The implementation uses 2×2 kernels and stride (2, 2), yielding 2× width- and height-compression in the encoder.
  • D. Training Strategy: The architecture is trained in three steps because directly training the complete noisy model would converge slowly.Adjusting compression ratios for different channel conditions requires retraining only the compression module, not the whole model.

A. Experimental Setup

The experiments evaluate BottleNet++ on CIFAR-100 using VGG16 and ResNet50, testing compression at multiple DNN splitting points under AWGN and BEC conditions.

  • A. Experimental Setup: The evaluation uses CIFAR-100, containing 60,000 32×32 color images across 100 classes.The dataset has 600 images per class and is considered suitable for low-resolution on-device applications.
  • A. Experimental Setup: The study evaluates classical VGG16 and ResNet50 architectures to test feature compression capability.VGG16 reaches 74.04% accuracy, while ResNet50 achieves 77.81% accuracy in the experiment.
  • A. Experimental Setup: PSNR indicates AWGN channel condition, whereas bit erasure rate indicates the quality of a BEC.The encoder’s maximum output is 1, motivating the paper’s PSNR definition.
  • A. Experimental Setup: Compression is evaluated at different splitting points of VGG16 and ResNet50.Sequential networks can split at layer ends, while ResNet splitting points correspond to res-units because of branchy structures.

B. Compression Capability Comparison

BottleNet++ is compared with conventional and neural compression baselines across BEC and AWGN channels, targeting early splitting with low communication overhead and limited accuracy loss. It achieves the lowest communication overhead across nearly all tested splitting points and reduces on-device computation relative to competing compression methods.

  • B. Compression Capability Comparison: The evaluation sets a 2% accuracy-degradation threshold and seeks the earliest splitting point with low communication overhead.On-device computation is approximated by the number of floating-point operations.
  • B. Compression Capability Comparison: BottleNet++ is compared with JPEG, Quantization + Huffman, BottleNet, and raw PNG-image transmission.Baseline methods assume reliable communication with channel coding, whereas BottleNet++ integrates the channel model and avoids extra channel-coding cost.
  • B. Compression Capability Comparison: 256× bit compression ratio is achieved on the BEC at p = 0.01 in the last convolutional layer of ResNet.The feature is reduced from 8192 32bit-floating numbers, or 32 KB, to 128 8bit integers, or 128 Bytes.
  • B. Compression Capability Comparison: BottleNet++ achieves the lowest communication overhead at almost any splitting point in the BEC experiments.The BEC comparisons use ResNet50 and VGG16 and represent compression capability through communication overhead.
  • B. Compression Capability Comparison: BottleNet++ achieves the minimum on-device computation when communication overhead must remain below raw PNG-image transmission.Table I reports the minimum computation at the earliest feasible splitting point for each method and scene.
  • B. Compression Capability Comparison: BottleNet++ achieves up to 64× bandwidth reduction in AWGN by compressing a 2048-symbol ResNet50 feature to 32 symbols.It also reduces on-device computation by approximately 2× for VGG16 and 3× for ResNet50 compared with other methods.

C. Generalization Ability and Robustness Analysis

BottleNet++ remains robust as channel conditions vary across AWGN and BEC, with channel-aware encoding providing the strongest accuracy and generalization. Across the tested ranges, accuracy drops remain below 1%.

  • Robustness to Channel Variations: BottleNet++ accuracy drops less than 1% as PSNR changes from 25 dB to 10 dB in AWGN or bit erasure rate changes from 0.01 to 0.15 in BEC.The experiment evaluates ResNet50 after the last convolutional layer, using 64× compression for AWGN and 256× bit compression for BEC.
  • Channel-Aware Encoding: Case 1 achieves the highest accuracy under every tested channel condition when the encoder knows channel state during training and testing.Channel state is represented by bit erasure rate or PSNR.
  • Channel-Aware Encoding: Case 2 remains very close to Case 1 when channel state is known only during training and fixed assumptions are used during testing.The testing assumptions are 15 dB for AWGN and 0.125 for BEC.
  • Channel-Aware Encoding: Case 3 shows a noticeable accuracy drop relative to Case 1 when channel state is unavailable during both training and testing.The comparison indicates that incorporating channel conditions during encoding improves generalization ability.

V. CONCLUSIONS

BottleNet++ uses end-to-end deep learning to compress intermediate features for device-edge co-inference. Its high compression ratios reduce communication overhead and enable earlier DNN splitting, lowering on-device computation.

  • Conclusions: BottleNet++ exploits intermediate-feature sparsity and fault tolerance to achieve higher compression ratios than existing methods.The approach targets device-edge co-inference with resource-constrained mobile devices.
  • Conclusions: The resulting compression reduces communication overhead and makes earlier DNN splitting feasible, reducing on-device computation.The conclusion frames BottleNet++ as addressing both what to transmit and how to transmit in edge-assisted inference.
Loading 1910.14315v5…