Source-linked AI summary
SqueezeNext: Hardware-Aware Neural Network Design
Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, Kurt Keutzer
TL;DR
Embedded deployment is constrained by neural networks’ memory and power demands. SqueezeNext designs compact architectures using prior architectures and accelerator simulation, achieving competitive accuracy and hardware trade-offs across model variants. The resulting family offers broad speed, energy, model-size, and accuracy choices for target applications.
Problem
Neural networks for embedded systems require lower memory and power consumption while retaining useful accuracy.
Method
SqueezeNext combines two-stage channel bottlenecks, low-rank separable convolutions, skip connections, and hardware-guided architecture variations.
Results
SqueezeNext matches AlexNet’s top-5 accuracy with 112× fewer parameters, exceeds MobileNet’s top-5 accuracy by 1.6% with 1.3× fewer parameters, and achieves hardware-guided speed and energy gains without accuracy degradation.
Takeaways & Limitations
The SqueezeNext family provides speed, energy, model-size, and accuracy trade-offs for selecting models suited to particular embedded applications.
Takeaways & Limitations
Hardware simulations conservatively assume 40% weight sparsity.
Abstract
from arXiv · showhide
One of the main barriers for deploying neural networks on embedded systems has been large memory and power consumption of existing neural networks. In this work, we introduce SqueezeNext, a new family of neural network architectures whose design was guided by considering previous architectures such as SqueezeNet, as well as by simulation results on a neural network accelerator. This new network is able to match AlexNet's accuracy on the ImageNet benchmark with $112\times$ fewer parameters, and one of its deeper variants is able to achieve VGG-19 accuracy with only 4.4 Million parameters, ($31\times$ smaller than VGG-19). SqueezeNext also achieves better top-5 classification accuracy with $1.3\times$ fewer parameters as compared to MobileNet, but avoids using depthwise-separable convolutions that are inefficient on some mobile processor platforms. This wide range of accuracy gives the user the ability to make speed-accuracy tradeoffs, depending on the available resources on the target hardware. Using hardware simulation results for power and inference speed on an embedded system has guided us to design variations of the baseline model that are $2.59\times$/$8.26\times$ faster and $2.25\times$/$7.5\times$ more energy efficient as compared to SqueezeNet/AlexNet without any accuracy degradation.
1. Introduction
Embedded deployment is challenged by neural networks whose accuracy-oriented growth increases memory and power demands. SqueezeNext addresses this by designing smaller architectures that retain benchmark accuracy across multiple model sizes.
- SqueezeNext targets embedded applications with limited power and memory budgets.
- 4.4 Million parameters reaches VGG-19’s baseline accuracy in a deeper SqueezeNext variation.
- SqueezeNext provides a family of width and depth variations spanning a wide range of accuracy levels.
- 112× fewer parameters matches AlexNet’s top-5 performance using an aggressively reduced SqueezeNext model.The reduction uses group convolutions in the 23-module architecture.
2. SqueezeNext Design
SqueezeNext redesigns convolutional blocks to reduce parameters while preserving trainability and avoiding inefficient operations on some embedded systems. Its design combines staged channel reduction, separable spatial convolutions, skip connections, and a final bottleneck.
- Low Rank Filters: K × K convolutions are decomposed into 1 × K and K × 1 convolutions, reducing parameters from K^2 to 2K.The separable convolutions also increase network depth and include ReLU and batch normalization.
- Bottleneck Module: Two bottleneck modules each halve channel size before separable convolutions, reducing the parameter cost of spatial convolutions.A final 1 × 1 expansion module reduces the separable convolutions’ output channels.
- SqueezeNext avoids depthwise-separable convolutions because their low arithmetic intensity can perform poorly on some embedded systems.
- Fully Connected Layers: A final bottleneck reduces the input channel size to the last fully connected layer, considerably reducing total model parameters.Fully connected layers account for 96% of AlexNet’s total model size.
3. Hardware Performance Simulation
The paper evaluates neural-network execution on a simulated embedded accelerator using two dataflows and hardware-aware scheduling choices. It then uses these simulations to guide architecture modifications for improved inference performance and energy consumption.
- Hardware-aware design: Hardware simulation is used to modify the baseline network architecture because lightweight 1×1 convolutions have limited parallelism and data reusability.The paper’s stated goal is to obtain better inference and power performance without generalization loss.
- Dataflow options: The simulator evaluates convolution using Weight Stationary and Output Stationary operating modes.Eyeriss’s loop-based taxonomy motivates these two dataflow options.
- Dataflow options: Weight Stationary minimizes convolution-parameter memory accesses by reusing filter weights across activations, whereas Output Stationary minimizes output-activation accesses by accumulating partial sums over time.The corresponding innermost loops differ: spatial x and y loops for WS, versus channel and filter-position loops for OS.
- Hardware simulation setup: The reference accelerator contains a 16 × 16 or 8 × 8 PE array, a 128KB or 32KB global buffer, DMA, and 16-bit MAC units.It transfers data between DRAM and the global buffer and uses local PE register files.
- Execution and memory management: The accelerator selects the faster operating mode separately for each layer and applies tiling when a layer exceeds global-buffer capacity.Tiling covers the x, y, c, and k convolution loops, with loop choices selected through a cost function.
- Performance estimation: The performance estimator sums modeled layer cycles while approximating DRAM behavior with 100-cycle latency and 16GB/s effective bandwidth.Energy is estimated using a similar methodology to prior accelerator work.
4. Results
SqueezeNext experiments span accuracy, model-size, inference-time, and energy trade-offs, including comparisons with AlexNet, VGG-19, MobileNet, and simulated accelerator configurations.
- Classification Performance Results: 1.3× fewer parameters delivers comparable performance to MobileNet-1.0-224 in 2.0-SqNxt23v5.SqueezeNext achieves slightly better top-5 performance under the reported comparable-training comparison.
- Hardware Performance Results: 12% faster and 17% more energy efficient than the baseline model is achieved by 1.0-SqNxt-23v5 on the 16×16 configuration.The variation redistributes depth across stages while maintaining accuracy.
- Classification Performance Results: 112× fewer parameters matches AlexNet’s top-5 performance using group convolution with group size two.
- Classification Performance Results: 31× fewer parameters matches VGG-19’s performance in the 2.0-SqNxt-44 model.
- Hardware Performance Results: 2.59×/8.26× faster and 2.25×/7.5× more energy efficient than SqueezeNet/AlexNet is achieved without accuracy degradation.
- Hardware Performance Results: Depthwise-separable convolutions can be inefficient on the simulated hardware because of poor arithmetic intensity, especially with more processors.
5. Conclusions
SqueezeNext is a family of embedded-oriented architectures designed around accuracy, model size, speed, and energy trade-offs. Its reported results match or exceed reference-network performance with substantially fewer parameters and use accelerator simulation to guide efficient variants.
- 112× fewer parameters achieves AlexNet’s top-5 performance, while a deeper variation exceeds VGG-19’s accuracy with 31× fewer parameters.
- SqueezeNext exceeds MobileNet’s top-5 accuracy by 1.6% with 1.3× fewer parameters without using depthwise-separable convolutions.
- Hardware simulation and per-layer analysis guide variants that are 2.59×/8.26× faster and 2.25×/7.5× more energy efficient than SqueezeNet/AlexNet without accuracy degradation.
- The SqueezeNext family provides speed, energy, model-size, and accuracy trade-offs for selecting models suited to particular applications.