Source-linked AI summary
CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices
Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, Xiaolong Ma, Yipeng Zhang, Jian Tang, Qinru Qiu, Xue Lin, Bo Yuan
TL;DR
Large DNNs deliver strong accuracy but require substantial computation, memory, and energy, while existing compression methods have structural, training, and guarantee limitations. CirCNN uses block-circulant weight matrices and FFT-based multiplication to reduce computation and storage while preserving effectiveness. Across FPGA, ASIC, and embedded-processor evaluations, it reports substantial energy-efficiency gains with a small hardware footprint.
Problem
Large DNNs are compute- and memory-intensive, while existing compression methods can introduce irregular structure, training complexity, and weak guarantees for compression and accuracy.
Method
CirCNN represents weights with block-circulant matrices and applies FFT-based multiplication, directly training networks under that structure.
Results
6-102X energy efficiency improvements are reported over the best state-of-the-art results from FPGA implementation and ASIC synthesis.
Takeaways & Limitations
CirCNN provides a configurable DNN inference engine evaluated across FPGA, ASIC, and embedded-processor platforms with a small hardware footprint.
Takeaways & Limitations
The described hardware architecture mainly focuses on inference because online training is difficult under embedded systems’ limited computing power and available datasets.
Abstract
from arXiv · showhide
Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning; 2) the increased training complexity; and 3) the lack of rigorous guarantee of compression ratio and inference accuracy. To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from O(n2) to O(nlogn) and the storage complexity from O(n2) to O(n), with negligible accuracy loss. Compared to other approaches, CirCNN is distinct due to its mathematical rigor: it can converge to the same effectiveness as DNNs without compression. The CirCNN architecture, a universal DNN inference engine that can be implemented on various hardware/software platforms with configurable network architecture. To demonstrate the performance and energy efficiency, we test CirCNN in FPGA, ASIC and embedded processors. Our results show that CirCNN architecture achieves very high energy efficiency and performance with a small hardware footprint. Based on the FPGA implementation and ASIC synthesis results, CirCNN achieves 6-102X energy efficiency improvements compared with the best state-of-the-art results.
1 INTRODUCTION
Large DNNs improve accuracy but impose substantial computation, memory, and energy demands, motivating compression and hardware acceleration. CirCNN addresses these pressures with block-circulant weights, FFT-based computation, mathematical guarantees, and implementations across multiple platforms.
- Large models can improve overall accuracy but increase computational capability and memory requirements, creating scalability, performance, and energy-efficiency challenges.
- Off-chip DRAM accesses can dominate system power because their per-bit access energy is 200× that of on-chip SRAM.
- Existing compression methods offer parameter reduction but can produce irregular structures, complicate training, and lack rigorous guarantees for compression ratio and accuracy.
- CirCNN represents weights with block-circulant matrices, using vectors for circulant blocks to reduce storage; a 6×3 matrix example falls from 18 to 6 stored parameters, a 3x reduction.
- FFT-based multiplication with block-circulant weights reduces fully connected-layer computational complexity from O(n^2) to O(n logn).
- CirCNN directly trains networks with block-circulant structure, and its theoretical foundation shows convergence to the same effectiveness as uncompressed DNNs.
- CirCNN is a configurable DNN inference engine for varied hardware/software platforms, evaluated on FPGA, ASIC, and embedded processors.
2 BACKGROUND AND MOTIVATION
DNNs use cascaded convolutional, fully connected, and pooling layers, but large models increase computation, memory demands, and energy consumption. CirCNN addresses limitations of pruning and prior circulant approaches with block-circulant representations that support compression, acceleration, and theoretical accuracy guarantees.
- 2.1 Deep Neural Networks: DNNs cascade fully connected, convolutional, and pooling layers to extract features at multiple abstraction levels.Fully connected and convolutional layers dominate computation and storage, while pooling has lower complexity.
- 2.2 DNN Weight Storage Reduction and Acceleration: Heuristic pruning reduces parameters but produces irregular structures requiring indexing, which undermines compression and performance improvements.Prior compression methods also focus mainly on weight reduction, use heuristic complexity reductions, and add training complexity through post-training transformations.
- 2.4 Novelty of CirCNN: CirCNN combines asymptotic computational speedup with weight compression and theoretically approaches the effectiveness of uncompressed networks.The approach targets simultaneous reductions in computational and storage complexity during both inference and training.
- 2.4 Novelty of CirCNN: CirCNN represents weights with block-circulant matrices, supporting both fully connected and convolutional layers for whole-network application.For convolutional layers, it exploits inter-filter sparsity by representing a matrix of filters with a vector of filters.
- 2.4 Novelty of CirCNN: Larger block sizes provide higher compression but may degrade accuracy, whereas smaller blocks provide better accuracy with less compression.Block-circulant matrices also avoid storage and computation waste from zero padding when input and output dimensions differ.
3 CIRCNN: ALGORITHMS AND FOUNDATION
CirCNN partitions fully connected and convolutional weights into circulant blocks, enabling FFT-based computation for compressed inference and training. Its theoretical foundation establishes approximation guarantees while experiments report compression, accuracy, and training results across DNNs and datasets.
- 3.1 FC Layer Algorithm: Block-circulant FC layers partition W into square circulant submatrices, representing each block by a vector.For a p×q block partition with block size k, each Wij is defined by its first-row vector wij.
- 3.1 FC Layer Algorithm: FFT→element-wise multiplication→IFFT computes each circulant block product, reducing FC inference to O(n logn) computation and O(n) storage for small p and q.The corresponding layer complexities are O(pqk logk) and O(pqk), respectively.
- 3.1 FC Layer Algorithm: CirCNN directly trains vectors corresponding to circulant submatrices, so the trained network naturally retains the block-circulant structure during inference and training.This avoids converting unstructured weights after training.
- 3.2 CONV Layer Algorithm: The same block-circulant idea extends to CONV tensors by making every spatial-input/output slice circulant and reformulating convolution as matrix multiplication.The reformulated operation applies block-circulant multiplication to reduce complexity from O(W H r^2 C P) to O(W H Q logQ), where Q = max(r^2C, P).
- 3.3 Theoretical Foundation: The theory proves universal approximation for block-circulant networks and derives an approximation error bound of O(1/n) when the layer width n is limited.The bound decreases as the number of neurons or inputs increases.
- 3.4 Experimental Evaluation: Experiments report highly compressed regular models, more parameter reduction than state-of-the-art results for LeNet-5 and AlexNet, and 5× to 9× DBN training acceleration.The DBN acceleration is lower than the model reduction ratio because GPUs are less optimized for FFT than matrix-vector multiplication.
4 CIRCNN ARCHITECTURE
CirCNN is a configurable DNN inference engine built around reusable FFT computing blocks, with pipelining, parallelism, and platform-specific memory optimizations. Its design trades performance and throughput against power, footprint, control complexity, and memory constraints.
- 4 CirCNN Architecture: CirCNN provides a universal inference engine with configurable layer types, sizes, scales, and implementations across hardware and software platforms.The architecture is based on block-circulant matrix algorithms.
- 4.3 Pipelining and Parallelism: Inter-level and intra-level pipelining exploit CirCNN’s compressed regular structure to improve accelerator performance and energy efficiency.Inter-level stages correspond to FFT levels, while intra-level stages divide butterfly computations.
- 4.1 Recursive Property of FFT: FFT serves as CirCNN’s basic computing block because its recursive structure supports universal, reconfigurable, and small-footprint implementations.The same structure can be reused for large FFTs and adapted for IFFT.
- 4.1 Recursive Property of FFT: The universal FFT→component-wise multiplication→IFFT procedure supports FC and CONV layers during both inference and training.Component-wise multiplication has O(n) complexity and a relatively small hardware footprint.
- 4.2 Overall Architecture: The architecture separates major FFT/IFFT work from lower-complexity peripheral operations such as component-wise multiplication, ReLU activation, and pooling.Its subsystems include basic and peripheral computing blocks, control, memory, and I/O.
- 4.3 Pipelining and Parallelism: Larger parallelization degree p and depth d increase performance and throughput but also increase hardware cost, while larger d reduces memory accesses at higher control complexity.Design optimization is constrained by hardware resources, memory bandwidth, and I/O bandwidth.
- 4.3 Pipelining and Parallelism: On Cyclone V FPGA with block size 128, increasing p from 16 to 32 raises performance by 53.8% with less than 10% higher power, while increasing d from 1 to 2 raises performance by 62.2% with 7.8% higher power.These results illustrate the power-performance tradeoff used for design optimization.
5 EVALUATION
CirCNN is evaluated across FPGA, ASIC, embedded-processor, and neurosynaptic-platform comparisons. Across these settings, it delivers strong energy efficiency and performance with a small hardware footprint, while accuracy and platform-specific trade-offs remain relevant.
- Experimental Setup: Order(s) of magnitude in energy efficiency and performance improvements are observed across FPGA, ASIC, and embedded-processor evaluations.The experiments cover MNIST, CIFAR-10, SVHN, and ImageNet, with comparisons against state-of-the-art hardware systems.
- FPGA-Based Testing: 11×-16× energy-efficiency improvement is achieved over prior FPGA works using heuristic model-size reduction techniques.These references use heuristic weight pruning or binary-weighted neural networks.
- FPGA-Based Testing: 60-70× energy-efficiency improvement is achieved over prior FPGA systems using uncompressed or partially compressed deep-learning models.The reported gains are attributed to algorithmic complexity reduction, efficient hardware design, weight reduction, and eliminating off-chip weight access.
- FPGA-Based Testing: CirCNN improves throughput for MNIST and SVHN and achieves energy efficiency at the same magnitude as IBM TrueNorth across the compared benchmarks.For CIFAR-10, CirCNN throughput is lower because TrueNorth uses specific preprocessing and the selected DNN uses small-scale FFTs.
- ASIC Designs Synthesis Results: More than 6 times higher energy efficiency than the best state-of-the-art ASIC is achieved alongside the highest synthesized throughput and energy efficiency.Using 4-bit representations and near-threshold computing adds another 17× improvement, reaching 102× versus the best state-of-the-art.
- Embedded Processors: 570× energy-efficiency improvement over the NVIDIA Jetson TX1 is achieved, increasing to 9,690× with near-threshold computing and 4-bit representations.On ARM Cortex A9, LeNet-5 reaches 0.9ms/image with 96% accuracy; an AlexNet fully connected layer reaches 667 Layers/s versus 573 Layers/s on an NVIDIA Tesla GPU.
- Summary and Discussions: The main reported gains come from algorithmic improvements of around 10×-20% and other factors contributing 2×-5%, while memory power is no longer absolutely dominant.The architecture also uses FFT/IFFT recursion to support medium- to large-scale models in a small footprint and provides configurable reconfigurability.
- Summary and Discussions: The evaluation focuses mainly on inference, although the underlying algorithmic framework applies to both inference and training.The paper states that online hardware training is difficult because embedded systems have limited computing power and may encounter limited datasets.
6 CONCLUSION
The conclusion presents CirCNN as a principled block-circulant approach using FFT-based multiplication to compress and accelerate DNNs. Across FPGA, ASIC, and embedded processors, it reports high energy efficiency and performance with negligible accuracy loss and a small hardware footprint.
- 6 CONCLUSION: CirCNN uses block-circulant matrices and FFT-based multiplication to reduce computation from O(n^2) to O(n logn) and storage from O(n^2) to O(n).The reported complexity reductions apply to both inference and training, with negligible accuracy loss.
- 6 CONCLUSION: 6 - 102× energy-efficiency improvements are reported against the best state-of-the-art results from FPGA implementation and ASIC synthesis.CirCNN is described as a configurable universal DNN inference engine implementable across hardware and software platforms.