Source-linked AI summary
Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, Ian Reid
TL;DR
Deep CNNs are computationally expensive for resource-constrained mobile platforms, while value-based quantization can introduce error and preserve only local approximations. The paper proposes Group-Net, which redesigns binary network structure through grouped, parallel binary bases and learned connections, extending the approach to task-specific semantic segmentation structures. Experiments report effective performance and computational trade-offs across image classification and semantic segmentation, including CPU inference acceleration.
Problem
Deep CNNs require substantial computation for mobile deployment, while value approximation can incur quantization error and lacks adaptive ability across tasks.
Method
Group-Net decomposes full-precision networks into groups approximated by parallel binary bases, learns their connections, and embeds task-specific atrous structure for segmentation.
Results
Experiments report effective ImageNet classification and PASCAL VOC semantic segmentation performance, with promising CPU test-time acceleration and favorable accuracy–complexity trade-offs.
Takeaways & Limitations
Structure approximation enables binary architectures to preserve higher-level structure and adapt to different tasks while maintaining the efficiency goals of binary computation.
Takeaways & Limitations
Binary quantization functions are often non-differentiable, so their gradients can only be roughly approximated; value approximation also inevitably introduces quantization error for pretrained task-specific models.
Abstract
from arXiv · showhide
In this paper, we propose to train convolutional neural networks (CNNs) with both binarized weights and activations, leading to quantized models specifically} for mobile devices with limited power capacity and computation resources. Previous works on quantizing CNNs seek to approximate the floating-point information using a set of discrete values, which we call value approximation, but typically assume the same architecture as the full-precision networks. In this paper, however, we take a novel 'structure approximation' view for quantization---it is very likely that a different architecture may be better for best performance. In particular, we propose a `network decomposition' strategy, named \textbf{Group-Net}, in which we divide the network into groups. In this way, each full-precision group can be effectively reconstructed by aggregating a set of homogeneous binary branches. In addition, we learn effective connections among groups to improve the representational capability. Moreover, the proposed Group-Net shows strong generalization to other tasks. For instance, we extend Group-Net for highly accurate semantic segmentation by embedding rich context into the binary structure. Experiments on both classification and semantic segmentation tasks demonstrate the superior performance of the proposed methods over various popular architectures. In particular, we outperform the previous best binary neural networks in terms of accuracy and major computation savings.
1 Introduction
Binary neural networks target mobile deployment by replacing floating-point computation with compact binary operations, but value approximation can lose task-relevant structure. Group-Net addresses this gap through structure approximation, decomposing networks into learnable binary groups and adapting them to classification and segmentation.
- Motivation: Accurate deep models can require billions of FLOPs, motivating binary neural networks for resource-constrained mobile platforms.Binary weights and activations enable dot products through XNOR-popcount operations instead of floating-point multiplication.
- Limitations of prior quantization: Value approximation quantizes weights and activations but is limited by local approximation, quantization error, and weak adaptability across tasks.Binary quantization is especially difficult because its functions are often non-differentiable and gradients are only roughly approximated.
- Proposed approach: Group-Net uses structure approximation by partitioning a full-precision network into groups reconstructed from parallel binary bases.The binary bases are optimized end-to-end, while the decomposition can preserve higher-level structural information.
- Task extension: Group-Net extends to semantic segmentation by applying different atrous rates across parallel binary bases, embedding multiscale context without increasing binary-convolution complexity.This task-specific structure is motivated by absorbing ASPP-like context into feature extraction.
- Proposed approach: Learnable connections among groups provide adaptive decomposition instead of enumerating all possible binary network structures.The paper introduces a fusion gate to learn soft connections between blocks.
- Evaluation: Experiments on ImageNet and PASCAL VOC report a favorable accuracy–complexity trade-off, with Group-Net also generalized to semantic segmentation and CPU inference.The contribution summary additionally reports significant improvement from BPAC while maintaining complexity.
2 Related Work
Prior network-quantization work reduces precision using fixed-point values or binary operations, balancing accuracy against computational complexity.
- Network quantization: BNNs constrain weights and activations to +1 and -1, replacing multiply-accumulations with XNOR and popcount operations.Other methods recursively approximate residual error with multiple binary maps to trade accuracy against complexity.
3 Method
Group-Net treats binarization as structure approximation: it decomposes full-precision networks into aggregations of binary branches, learns their connections, and adapts the structure for semantic segmentation.
- Structure approximation: Binary neural networks are designed from a structure-approximation perspective rather than by directly quantizing the original architecture.The approach decomposes a network into binary structures while preserving its representability.
- Layer-wise binary decomposition: Layer-wise decomposition approximates each floating-point convolution with K homogeneous binary branches whose outputs are combined using learned coefficients.Each branch preserves the original layer topology; K = 1 corresponds to direct binarization, while larger K increases approximation capacity and transformation complexity.
- Layer-wise binary decomposition: Parallel binary branches are hardware-friendly because XNOR-popcount operations can run in parallel, while inference requires K binary convolutions and K full-precision additions.For one ResNet layer with cin = 256, a 3 × 3 kernel, 28 × 28 spatial dimensions, and K = 5, the estimated speedup is 12.45×.
- Group-wise binary decomposition: Group-wise decomposition reduces accumulated layer-wise errors by approximating blocks or multi-block groups with binary groups and cascaded homogeneous bases.The additional block dimension expands the structure space and makes the decomposition more flexible.
- Learning for dynamic decomposition: Soft fusion gates dynamically learn connections between neighboring binary blocks by weighting a straight branch path and the preceding block’s aggregated output.This increases information flow and gradient paths during training; the possible connections otherwise grow as 2^N for a network with N blocks.
- Extension to semantic segmentation: For semantic segmentation, BPAC assigns different dilation rates to binary branches and aggregates them to embed multiscale information without additional parameters or computational complexity over naive branching.This extends the structured binary design to atrous convolutional layers used in dense pixel-wise classification.
4 Discussions
Group-Net replaces the K^2 binary convolutions of K-bit fixed-point quantization with K structural bases, reducing computation while preserving comparable accuracy. Its binary weights also reduce bandwidth, although the structural design increases parameter count by K times.
- Complexity and storage: K^2 binary convolutions are required by ABC-Net, whereas Group-Net uses K structural bases and saves approximately K times computational complexity at comparable Top-1 accuracy.Group-Net still incurs element-wise operations between groups, so the actual saving is slightly below K times.
- Complexity and storage: Group-Net requires K times as many parameters as its full-precision counterpart because it uses K structural bases, but binary weights still save memory bandwidth.The method directly designs and end-to-end optimizes the binary network structure rather than obtaining bases from full-precision tensors.
- Complexity and storage: K-bit fixed-point quantization computes K^2 xnor(·) and popcount(·) operations with complexity about O(K^2).Its output range is [−(2K −1)2M, (2K −1)2M].
- Complexity and storage: Group-Net with K bases has computational complexity K and saves (2K −1)2/K accumulator bandwidth compared with K-bit fixed-point methods.Binary weights also reduce weight bandwidth by 32/K times.
- Structural approximation: Unlike sequential multiple-binarization methods, Group-Net uses structure approximation to mimic the full-precision network and can generalize beyond local tensor approximation.The paper reports better performance than tensor-level approximation and highlights applicability to other tasks.
5 Experiment
Experiments evaluate Group-Net against binary and fixed-point models on ImageNet, hardware inference, and PASCAL VOC segmentation. The results indicate favorable accuracy–complexity trade-offs and benefits from group-wise decomposition and task-specific structure.
- ImageNet classification: Group-Net achieves the best trade-off between computational complexity and prediction accuracy against prior binary neural networks.Compared with directly binarized networks, it improves performance but requires K times more storage and complexity; its homogeneous bases can be parallelized on-chip.
- Fixed-point comparison: Group-Net requires less memory bandwidth while achieving comparable accuracy to LQ-Net on fixed-point ImageNet models.The comparison uses K=4 binary bases against 2-bit and 8-bit fixed-point configurations.
- Hardware implementation: More than 100× convolution speedup, approximately 7.5× overall speedup, and approximately 5.8× memory saving are reported for Group-Net** on an 8-core CPU.Floating-point element-wise operations and thread communication limit acceleration in the reported implementation.
- Ablation: 7.2% higher Top-1 accuracy is obtained by Group-Net over the layer-wise LBD strategy on ImageNet.The comparison supports the value of group-wise rather than layer-wise decomposition.
- Semantic segmentation: Using identical dilation rates across binary bases leaves a large gap to the full-precision segmentation counterpart.The paper relates this drop to the quality of extracted features and motivates task-specific structure.
- Semantic segmentation: Task-specific BPAC significantly improves semantic segmentation without adding computational complexity.Group-Net with BPAC achieves comparable performance to fixed-point LQ-Nets while saving considerable complexity, and the ResNet-34 variant can outperform the ResNet-50 counterpart.
6 Conclusion
The paper concludes that Group-Net decomposes full-precision networks into optimizable binary groups and extends this approach from ImageNet classification to semantic segmentation.
- Conclusion: Group-Net decomposes a full-precision network into multiple groups approximated by binary bases optimized end-to-end.The decomposition is learned automatically.
- Conclusion: Experiments demonstrate Group-Net’s effectiveness on ImageNet classification and its generalization to semantic segmentation.
S7 More ablation study on ImageNet classification
The ablation study defines several group-wise binary decomposition variants that differ in the number of blocks contained in each group base.
- S7 More ablation study on ImageNet classification: GBD v1 uses one block per group base, whereas GBD v2 uses two blocks per group base.GBD v1 corresponds to Eq. (5), while GBD v2 is described by Eq. (6).
S7.1 Group space exploration
The group-space ablation finds that learning soft connections between blocks gives the strongest ResNet-18 performance, while hard connections perform relatively worse.
- S7.1 Group space exploration: Learning soft connections between blocks produces the best ResNet-18 performance among the tested group-wise decomposition strategies.Methods using hard connections perform relatively worse.
- S7.1 Group space exploration: The ablation concludes that designing a compact binary structure is essential for highly accurate classification.The authors suggest further gains may come from integrating additional design choices.
S7.2 Effect of the number of bases
The number of binary bases controls how closely Group-Net fits the full-precision structure, trading accuracy against complexity. Increasing the bases improves performance while retaining efficiency advantages.
- With one base, directly binarizing the original network causes an apparent accuracy drop versus its full-precision counterpart.
- Increasing the number of bases steadily improves performance by enabling better fitting of the full-precision structure.
- The decomposition exposes an accuracy–complexity trade-off: enough bases can in principle approximate the full-precision network precisely.
- Group-wise architectures can decompose the network into groups containing one or multiple blocks, with each floating-point block approximated by binarized branches.
- Each binary base uses few resources, while the parallel structure reduces inference time and power consumption and suits FPGA/ASIC implementation.
S8 More discussions
Group-Net resembles ResNeXt through homogeneous branches and a cardinality dimension, but differs by decomposing network structure into binary branches for efficient reconstruction. Its group-wise approximation also supports flexible integration with architecture designs.
- Both Group-Net and ResNeXt use homogeneous multi-branch designs and introduce a cardinality dimension.
- ResNeXt divides input channels into groups for efficient group convolutions, whereas Group-Net divides the network into groups and replicates floating-point structure across binary branches.
- Group-Net aggregates low-precision transformations to reconstruct full-precision structure for complexity reduction in energy-efficient hardware.
- Unlike ResNeXt’s restriction to one block, Group-Net allows structured transformations across multiple blocks and can integrate with neural architecture search.
S9 More ablation study on semantic segmentation
The ablations show that task-specific structure, multiscale dilation, and activation precision affect Group-Net performance. Group-Net with BPAC preserves computational efficiency while incorporating task-specific context, and higher activation precision improves the accuracy–complexity trade-off.
- For FCN-32s and FCN-16s, dilation rates of 4 and 8 in the last two blocks, respectively, produce the best performance.
- Group-Net with BPAC accurately fits the full-precision model while saving considerable computational complexity.
- Using five floating-point branches with different dilation rates increases the FLOPs of full-precision baselines, whereas Group-Net+BPAC does not increase complexity over Group-Net alone.
- The unchanged complexity demonstrates Group-Net’s flexibility in borrowing task-specific properties to approximate the original floating-point structure.
- Activation bit-width can be increased alongside the number of bases to trade higher accuracy for greater computational complexity.
- Clipping ReLU outputs to [0, β] before uniform quantization limits dynamic range and reduces quantization error when bit precision is low.
- With five bases, binary weights and fixed-point activations achieve highly accurate results, including Top-1 accuracy drops of 1.5% for ternary activations and 6.5% for binary activations on ResNet-50.