Source-linked AI summary
SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing
Ao Ren, Ji Li, Zhe Li, Caiwen Ding, Xuehai Qian, Qinru Qiu, Bo Yuan, Yanzhi Wang
TL;DR
Software-based DCNNs require costly, power-hungry servers, limiting deployment in embedded and mobile IoT devices. SC-DCNN provides a comprehensive SC-based design and optimization framework that reduces hardware footprint and power and energy consumption while maintaining high network accuracy.
Problem
Software-based DCNNs require high-performance servers with high power consumption and hardware cost, limiting their suitability for embedded and mobile IoT devices.
Method
SC-DCNN uses a bottom-up framework that jointly optimizes basic function blocks, feature extraction blocks, weight storage, and the overall architecture.
Results
SC-DCNN achieves remarkably low hardware footprint, low power and energy consumption, while maintaining high network accuracy.
Takeaways & Limitations
Stochastic Computing offers a scalable basis for DCNN hardware because multiplications and additions can use simple AND gates and multiplexers.
Takeaways & Limitations
Stanh cannot be applied directly because its inaccuracy is notable across inputs, finite bit-stream lengths require adjustment, and scaling-back is needed after input down-scaling.
Abstract
from arXiv · showhide
With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.
1. Introduction
The paper introduces SC-DCNN to make DCNN hardware more suitable for embedded and mobile IoT devices by exploiting stochastic computation. It develops and jointly optimizes computation blocks, feature extraction blocks, weight storage, and the overall architecture for low cost and energy use while preserving accuracy.
- Software DCNNs typically require high-performance servers, whose power and hardware costs limit deployment in embedded and mobile IoT devices.
- Stochastic Computing represents numbers with bit-streams and implements multiplication and addition using simple logic, creating a large design space for scalable DCNN hardware.SC uses AND gates for multiplication and multiplexers for addition.
- SC-DCNN is presented as the first comprehensive, bottom-up design and optimization framework for SC-based DCNNs.
- The framework designs basic function blocks and four jointly optimized feature extraction blocks, including a hardware-oriented max-pooling design.
- Weight storage is optimized through filter-aware SRAM sharing, effective storage methods, and layer-wise optimizations to reduce area and power or energy consumption.
- Overall SC-DCNN optimization selects feature extraction blocks and layer-wise configurations according to how hardware inaccuracies affect network accuracy across layers.
2. Related Works
Prior DCNN accelerators use GPUs, FPGAs, and ASICs to exploit parallelism, but conventional hardware still leaves performance and power improvements available. The paper positions SC as a candidate paradigm and identifies the lack of comprehensive SC-DCNN design work spanning computation and weight storage.
- GPU- and FPGA-based DCNN implementations exploit parallel resources but retain limitations from general-purpose devices and restricted signal routing.
- ASIC-based designs improve DCNN efficiency through parallel nodes or specialized fully connected layers, but address narrower architectural scopes.
- Earlier stochastic-logic neural-network studies include neurochips, radial-basis-function networks, and deep-belief-network neurons, rather than comprehensive SC-based hardware DCNNs.
3. Overview of DCNN Architecture and Stochastic Computing
DCNNs combine convolution, pooling, and fully connected layers, while SC represents values as bit-stream probabilities and simplifies arithmetic into logic operations. The paper applies these representations to inner products, pooling, and activation, with tanh implemented through an FSM.
- DCNN Architecture Overview: A DCNN commonly stacks convolutional, pooling, and fully connected layers; convolution computes receptive-field dot products with learnable filters.
- DCNN Architecture Overview: The paper treats inner product, pooling, and activation as hardware function blocks whose composition forms a feature extraction block.
- Stochastic Computing: SC represents a probabilistic number by the proportion of ones in a bit-stream and supports bipolar encoding for values in [-1, 1].
- Stochastic Computing: SC offers lower hardware cost for many arithmetic operations, leaving design space to trade area against power, latency, and parallelism in large-scale DCNNs.
- Stochastic Computing: SC multiplication uses AND gates for unipolar values and XNOR gates for bipolar values, while addition can use OR gates, multiplexers, APCs, or a two-line representation-based adder.
- Stochastic Computing: The design uses a K-state FSM for stochastic tanh activation, whose output approximates tanh(K^2 x) according to the stated stochastic formulation.
- Stochastic Computing: Overall network accuracy is a key optimization goal because stochastic hardware blocks introduce inaccuracies that correlate with, but differ from, network accuracy.
4. Design and Optimization for Function Blocks and Feature Extraction Blocks in SC-DCNN
The paper optimizes SC-DCNN function blocks and their feature-extraction combinations to balance hardware cost, energy, latency, and accuracy. It combines block-level analysis with joint and holistic optimization across bit-stream lengths, scaling, and layer configurations.
- Function block design: SC-based function blocks are designed for inner product/convolution, pooling, and activation while targeting lower power, energy, and hardware resources with high accuracy.The framework evaluates the advantages and limitations of alternative block implementations before constructing feature-extraction blocks.
- Inner product/convolution blocks: OR-gate inner products produce unacceptable bipolar accuracy, worsening with input size even after suitable pre-scaling.The reported evaluation fixes the bit-stream length at 1024 and uses the most suitable pre-scaling.
- Inner product/convolution blocks: MUX-based inner products lose accuracy as input size increases because only one input is selected per time step, but longer bit-streams can restore sufficiently good accuracy.The accuracy loss arises because other inputs are omitted at each selection step.
- Inner product/convolution blocks: APC-based inner products incur less than 1% accuracy degradation versus conventional accumulative parallel counters while reducing gate count by about 40%.The APC counts ones in each product-stream column and emits a binary representation.
- Inner product/convolution blocks: Two-line representation-based inner products suffer overflow with multiple inputs and excessive area overhead compared with other implementations.Their non-scaling behavior can cause significant accuracy loss in hardware DCNNs.
- Pooling block designs: The hardware-oriented max-pooling scheme selects locally largest stream segments to reduce the latency and energy cost of full-stream counting while retaining sufficient accuracy.With segment length 16, the design remains sufficiently accurate even for large input sizes.
- Activation and feature-extraction blocks: Stanh requires joint optimization because distributed inputs, finite bit-streams, and scaling effects can prevent direct application and require scaling back.The optimized design selects state number K using scaling factor, bit-stream length, and accuracy requirements, while feature-extraction blocks jointly tune connected components.
- Feature-extraction block optimization: Feature-extraction blocks combine selected MUX- or APC-based inner products, pooling blocks, and Stanh or Btanh, with compatibility and bit-stream effects optimized jointly.For APC-Max-Btanh, accumulators replace counters in hardware-oriented max pooling, allowing the original Btanh design to be used without adjustment.
5. Weight Storage Scheme and Optimization
The paper reduces SC-DCNN weight-storage cost through filter-aware SRAM sharing, reduced precision, and layer-wise precision choices that preserve accuracy.
- Filter-aware SRAM sharing divides SRAM into filter-sized blocks, reducing routing overhead and wire delay.
- Reducing weight precision removes low-impact least significant bits, shrinking SRAM and its read/write circuits with little accuracy loss.
- Layer-wise precision matters: Layer0 is least sensitive, whereas Layer2 is most sensitive because it contains the most weights.
- 10.3× area savings are estimated for SRAM through the proposed precision-reduction method.
- With 7-7-6-bit weights across three LeNet5 layers, error rate is 1.65%, while area and power improve 12× and 11.9× over unreduced storage.
6. Overall SC-DCNN Optimizations and Results
Overall SC-DCNN optimization selects feature-extraction blocks and layer-wise configurations to balance accuracy against area, delay, power, and energy. Evaluations on LeNet5 show substantial efficiency gains, while larger-network validation remains future work.
- 6. Overall SC-DCNN Optimizations and Results: Feature-extraction blocks are compared across input sizes and bit-stream lengths for accuracy, area, delay, power, and energy.
- 6.1 Optimization Results on Feature Extraction Blocks: MUX-Avg-Stanh has the lowest area and energy and shortest delay, but its accuracy is worst and suits small receptive fields.
- 6.1 Optimization Results on Feature Extraction Blocks: MUX-Max-Stanh is more accurate than MUX-Avg-Stanh and can handle small and large receptive fields when bit-stream length increases.
- 6.1 Optimization Results on Feature Extraction Blocks: APC-based blocks provide higher accuracy by preserving inner-product information, but require more hardware, longer delays, and higher energy.
- 6.1 Optimization Results on Feature Extraction Blocks: APC-Max-Btanh achieves the best accuracy and is appropriate when accuracy requirements are very tight, despite highest area and energy.
- 6.2 Layer-wise Feature Extraction Block Configurations: Layer-wise block selection uses differing layer error sensitivities to reduce area and power or energy while maintaining high network accuracy.
- 6.3 Overall Optimizations and Results on SC-DCNNs: On MNIST LeNet5, the optimization evaluates max- and average-pooling designs under a 1.5% error-rate-difference threshold and 1024 maximum bit-stream length.
- 6.3 Overall Optimizations and Results on SC-DCNNs: Compared with Nvidia Tesla C2075, SC-DCNN No.11 achieves 15625× higher throughput and 159604× higher energy efficiency.
7. Conclusion
The paper presents SC-DCNN as a comprehensive framework combining stochastic-computing function blocks, feature-extraction designs, and weight-storage optimizations. The resulting implementation reports high throughput and efficiency with low hardware and energy consumption.
- SC-DCNN is presented as the first comprehensive design and optimization framework for SC-based DCNNs.
- The framework explores function blocks, four feature-extraction designs, and three SRAM weight-storage optimization schemes.
- The implementation achieves 781250 images/s throughput, 45946 images/s/mm2 area efficiency, and 510734 images/J energy efficiency.