Source-linked AI summary
Flattened Convolutional Neural Networks for Feedforward Acceleration
Jonghoon Jin, Aysegul Dundar, Eugenio Culurciello
TL;DR
Large CNNs are computationally expensive, motivating compact models for real-time and mobile execution. The paper trains flattened networks by replacing 3D convolution filters with consecutive 1D filters across channels and spatial directions. The resulting models achieve similar or better accuracy with far fewer parameters and about twofold faster evaluation, without manual tuning or post-processing.
Problem
CNNs offer strong visual and audio performance but are computationally expensive, creating deployment demands for real-time and mobile applications.
Method
The method trains flattened CNNs whose conventional 3D convolution filters are replaced by consecutive 1D filters across channels, vertical, and horizontal directions.
Results
Flattened networks achieve similar or better accuracies with ten times fewer parameters and about two times faster evaluation than baseline models.
Takeaways & Limitations
The flattened pipeline can substitute for 3D filters while reducing feedforward computation without requiring manual tuning or post-processing after training.
Takeaways & Limitations
Direct filter separability can lose information because practical filters may have intrinsic rank higher than one, and flattened training stores N −1 intermediate states for backpropagation.
Abstract
from arXiv · showhide
We present flattened convolutional neural networks that are designed for fast feedforward execution. The redundancy of the parameters, especially weights of the convolutional filters in convolutional neural networks has been extensively studied and different heuristics have been proposed to construct a low rank basis of the filters after training. In this work, we train flattened networks that consist of consecutive sequence of one-dimensional filters across all directions in 3D space to obtain comparable performance as conventional convolutional networks. We tested flattened model on different datasets and found that the flattened layer can effectively substitute for the 3D filters without loss of accuracy. The flattened convolution pipelines provide around two times speed-up during feedforward pass compared to the baseline model due to the significant reduction of learning parameters. Furthermore, the proposed method does not require efforts in manual tuning or post processing once the model is trained.
1 INTRODUCTION
CNNs enable demanding visual and audio applications but remain computationally expensive, especially for mobile devices with limited power and connectivity. This work reduces filter redundancy during training by replacing 3D convolutions with consecutive 1D filters while retaining comparable accuracy with fewer parameters.
- CNNs support visual and audio applications requiring high accuracy and real-time execution, but their computational cost often demands servers or GPUs.
- Mobile devices still struggle to run large CNNs because of limited computing power and battery life, making off-site execution dependent on reliable connectivity.
- CNNs contain substantial parameter redundancy, so compact representations can improve evaluation time and memory while preserving accuracy within 1% of original models.
- The proposed method decreases filter redundancy during training by separating conventional 3D convolution filters into three consecutive 1D filters.
- The flattened networks achieve similar or better accuracies than a baseline network with about ten times more parameters.
2 RELATED WORK
Prior acceleration methods exploit sparsity, separability, low-rank approximations, or inter-layer connectivity, but each has practical or optimization challenges. The proposed approach imposes structural constraints during training to learn fully one-dimensional separated filters without manual tuning or post-processing.
- Sparse filters can reduce computation, but irregular nonzero values and positions make sparsity difficult to exploit in practice.
- Separable filters replace a rank-one 2D filter operation with two consecutive 1D convolutions, significantly shortening CNN evaluation time.
- Low-rank methods approximate pretrained 3D filters and use clustering and post-training tuning to obtain twofold layer speedup while keeping accuracy within 1% of the original.
- Alternative work explores sparse connectivity between convolutional layers, but learning such connectivity for output prediction remains difficult.
- The proposed method learns 1D separated filters through structural constraints and avoids manual tuning or structural changes after training.
3 FLATTENING CONVOLUTION FILTERS
The method represents convolutional filters as consecutive one-dimensional filters across channels and the two spatial directions. Because rank-one separability is restrictive, the network learns these filters through constrained connections, reducing computation and parameters.
- Filter representation: CNN filters are represented as four-dimensional tensors with input channels, two spatial dimensions, and output filters.C and F denote input and output channels, while X and Y denote spatial dimensions.
- Filter separation: Under a rank-one assumption, a 3D filter can be separated into consecutive lateral, vertical, and horizontal one-dimensional filters.The lateral filter operates across channels; the vertical and horizontal filters operate across the Y and X spatial dimensions.
- Filter separation: Direct filter separation is limited because practical filters generally have rank higher than one and distributed eigenvalues, causing information loss.The text links increasing classification difficulty with the need for more leading components.
- Training strategy: Restricting receptive-field connections lets the model learn one-dimensional separated filters during training instead of approximating trained 3D filters afterward.The method applies structural constraints while preserving the CNN training procedure through backpropagation along constrained paths.
- Computational reduction: The constrained layer reduces per-feature-map parameters from XY C to X + Y + C and operations from MNCXY to MN(C + X + Y).These reductions follow from replacing a conventional 3D filter with three one-dimensional filters.
4 EXPERIMENTAL RESULTS
Across classification experiments, flattened convolutional models used separated 1D filters to reduce parameters while retaining baseline-level accuracy and accelerating feedforward execution. Training and memory behavior depended on the implementation and longer serialized gradient paths.
- Experimental setup: Flattened models were evaluated against CNN baselines on different classification tasks using the Torch7 environment.The experiments used customized gradient updates to demonstrate model performance.
- Flattened architecture: The flattened model applies sequential lateral, vertical, and horizontal 1D convolutions, extending channel-wise 1D convolution into spatial directions.Lateral filters operate across channels, followed by vertical and horizontal spatial filters.
- Training flattened model: 2–3% accuracy drops occurred when filters were reduced to a single set of C, X, and Y dimensions, while removing one LVH direction caused significant accuracy loss.These results motivated cascading LVH stages to preserve discriminative capacity.
- Training flattened model: With proper weight initialization, the flattened model achieved comparable accuracy to the baseline on CIFAR-10, with more consistent learning curves and less variation.The baseline saturated earlier and at a lower accuracy under the reported learning-rate schedule.
- Parameter reduction: The flattened representation reduces each feature map’s parameters from XY C to X + Y + C and operations from MNCXY to MN(C + X + Y).Parameter reduction is most favorable when the output-to-input channel ratio supports flattening; the authors selectively flattened later layers.
- Runtime and memory: The flattened layer runs about two times faster than a conventional convolutional layer during feedforward on both platforms.Acceleration tends to increase for larger images, while backpropagation gains can be negligible on GPUs because parameter updates require serial accumulation and frequent global memory access.
5 CONCLUSION
The paper proposes flattening 3D convolutional layers into consecutive 1D convolutions to reduce parameters and accelerate feedforward execution. The flattened networks achieve comparable or better accuracy while requiring substantially fewer parameters.
- The method converts each 3D convolutional layer into consecutive 1D convolutions across channels, vertical, and horizontal directions during training.
- Flattened convolutional networks achieve similar or better accuracies with ten times fewer parameters than the baseline model.
- Around two times speed-up is achieved during evaluation compared with the baseline model.
- The proposed method requires no manual tuning or post-processing after training.