Source-linked AI summary
Deep feature compression for collaborative object detection
Hyomin Choi, Ivan V. Bajic
TL;DR
Collaborative intelligence can reduce mobile-cloud computation costs, but the impact of compressing intermediate features on object-detection accuracy was previously unexamined. This paper studies near-lossless and lossy feature compression and introduces compression-augmented training, achieving up to 70% communication savings without sacrificing accuracy.
Problem
The impact of compressing intermediate feature data on collaborative object-detection accuracy had not been studied, although feature compression could reduce transfer latency and energy.
Method
The paper quantizes split-layer features, compresses them using lossless or lossy coding, and retrains the detector with compression augmentation for lossy transmission.
Results
Up to 70% bit savings at the same mAP are achieved by combining feature compression with compression-augmented training compared with cloud-only processing.
Takeaways & Limitations
Lossless compression of 8-bit or higher quantized features has little accuracy impact, while compression-augmented training extends useful lossy-compression levels.
Takeaways & Limitations
The transmission analysis assumes data is not corrupted during transfer.
Abstract
from arXiv · showhide
Recent studies have shown that the efficiency of deep neural networks in mobile applications can be significantly improved by distributing the computational workload between the mobile device and the cloud. This paradigm, termed collaborative intelligence, involves communicating feature data between the mobile and the cloud. The efficiency of such approach can be further improved by lossy compression of feature data, which has not been examined to date. In this work we focus on collaborative object detection and study the impact of both near-lossless and lossy compression of feature data on its accuracy. We also propose a strategy for improving the accuracy under lossy feature compression. Experiments indicate that using this strategy, the communication overhead can be reduced by up to 70% without sacrificing accuracy.
1. INTRODUCTION
Collaborative intelligence splits deep networks between mobile devices and the cloud, but efficient feature transfer remains insufficiently studied. This work examines feature compression for collaborative object detection and proposes a strategy to preserve accuracy under lossy compression.
- Collaborative intelligence runs a network’s front end on the mobile and uploads intermediate features to the cloud for remaining computation.The approach distributes computation between mobile and cloud rather than using only one location.
- Appropriate network splitting can reduce energy consumption and latency compared with cloud-only or mobile-only execution.Prior work examined split points and proposed Neurosurgeon to optimize the split for energy or latency.
- Feature-transfer efficiency is largely unexplored: prior work omitted compression, while another study used 8-bit quantization followed by lossless compression without examining application impact.The paper identifies feature compression as a way to reduce transfer latency and radio-transmission energy.
- This work studies how near-lossless and lossy feature compression affect accuracy in collaborative object detection.The study focuses on a deep object-detection model and evaluates compression effects on detection accuracy.
2. PRELIMINARIES
The paper uses YOLO-based object detection and compares three feature-transfer strategies: lossless transfer, quantization with lossless coding, and lossy compression. These strategies trade communication efficiency against possible accuracy changes.
- YOLO object detection jointly addresses object localization, classification, and bounding-box prediction through a loss function containing bounding-box and class-error terms.The preliminaries define ground-truth and predicted box coordinates, confidence scores, and class probabilities.
- YOLO9000 provides the experimental model, with feature volume and cumulative computational cost varying across network layers.The paper uses these layer-wise quantities to motivate possible mobile-cloud split points.
- Lossless transfer preserves accuracy but is inefficient because 32-bit floating-point features may contain redundancy.This strategy corresponds to sending split-layer features directly to the cloud.
- Quantizing features to 8 bits and then applying lossless compression is near-lossless, whereas adding lossy compression can save more bits but may further reduce accuracy.The paper illustrates these alternatives as distinct mobile-cloud transfer paths.
3. PROPOSED METHODS
The proposed pipeline quantizes split-layer features, arranges channels for codec compression, and uses compression-augmented training to improve robustness to lossy transmission. It accounts for quantization metadata and selects simple channel tiling for compression.
- 3.1. Quantization: The Q-layer quantizes split-layer feature tensors at 8-, 10-, or 12-bit precision before inverse quantization in the cloud.Quantization operates on an N × M × C tensor and uses its minimum and maximum values.
- 3.1. Quantization: min(V) and max(V) must be transmitted as 32-bit floats, except that min(V) can be omitted when the activation’s minimum is known to be zero.Leaky ReLU in YOLO9000 requires transmitting this parameter, unlike the assumed sigmoid or ReLU cases.
- 3.2. Compression: Quantized feature channels can be encoded as video frames, grouped into larger frames, or combined into one image using tiling or quilting.The paper evaluates channel arrangements for compatibility with existing codecs.
- 3.2. Compression: Simple tiling by channel index gives the best tested results and is therefore used to compress tiled feature channels as a still image.The selected arrangement preserves the channel-index tile structure for subsequent coding.
- 3.3. Compression-augmented training: Compression-augmented training retrains the model with lossy compression at the split point to extend useful compression levels for a desired accuracy.The method acts like regularization by exposing upstream and downstream weights to compression noise.
4. EXPERIMENTS
Experiments compare lossless and lossy feature compression for collaborative object detection using mAP and communication volume. Lossless compression preserves accuracy but offers limited savings, whereas lossy compression substantially reduces bits, especially with compression-augmented training.
- 16,551 VOC2007/VOC2012 images were used for training and 4,952 VOC2007 images for testing across 20 object classes.
- Lossless compression: Lossless compression after 8-, 10-, or 12-bit quantization achieves mAP scores around 0.7465-0.7475 across split points.The reported mAP changes little across the evaluated quantization settings and split points.
- Lossless compression: Lossless feature compression provides limited, if any, bit savings, and splits near the input can be less efficient than uploading input images.Early split points retain too much feature data for lossless transfer to be advantageous.
- Lossy compression: Lossy compression provides significant bit savings over cloud-only processing, while compression-augmented training extends useful compression levels for a given mAP.Figure 5 compares cloud-only input compression with splits after max 11 and max 17 using default or compression-augmented weights.
- Lossy compression: 45% (70%) total bit savings are achieved at the same mAP for max 11 (max 17) splits with compression-augmented training.Default weights provide 6% (60%) savings, while compression-augmented training adds 39% (10%), respectively.
5. CONCLUSIONS
The study finds that quantized lossless compression has little effect on detection accuracy, while lossy compression offers greater bit savings with an accuracy trade-off. Compression-augmented training extends useful lossy compression levels for a desired accuracy.
- Lossless compression of 8-bit or higher quantized data has little impact on detection accuracy.
- Lossy compression provides higher bit savings but affects detection accuracy.
- Compression-augmented training extends the range of useful compression levels for a desired accuracy.