Source-linked AI summary
Diverse Branch Block: Building a Convolution as an Inception-like Unit
Xiaohan Ding, Xiangyu Zhang, Jungong Han, Guiguang Ding
TL;DR
ConvNets need higher performance without increasing inference-time costs, while multi-branch structures typically complicate deployment. The paper introduces DBB, which enriches training with diverse branches and then converts the block into a single convolution; it reports gains on classification, detection, and segmentation, including up to 1.9% higher ImageNet top-1 accuracy.
Problem
Multi-branch structures can enrich feature representations but usually slow inference, while deployed ConvNets face latency, memory, parameter, and hardware constraints.
Method
DBB replaces selected convolutions with diverse training-time branches and uses structural re-parameterization to convert each block into a single convolution after training.
Results
DBB improves ConvNets on ImageNet classification, COCO detection, and Cityscapes segmentation, with up to 1.9% higher top-1 accuracy on ImageNet.
Takeaways & Limitations
DBB improves off-the-shelf ConvNet architectures without extra inference-time costs by making training-time microstructures more diverse while retaining the deployment structure.
Takeaways & Limitations
Convolution additivity requires the combined convolutions to have matching configurations, including channels, kernel size, stride, and padding.
Abstract
from arXiv · showhide
We propose a universal building block of Convolutional Neural Network (ConvNet) to improve the performance without any inference-time costs. The block is named Diverse Branch Block (DBB), which enhances the representational capacity of a single convolution by combining diverse branches of different scales and complexities to enrich the feature space, including sequences of convolutions, multi-scale convolutions, and average pooling. After training, a DBB can be equivalently converted into a single conv layer for deployment. Unlike the advancements of novel ConvNet architectures, DBB complicates the training-time microstructure while maintaining the macro architecture, so that it can be used as a drop-in replacement for regular conv layers of any architecture. In this way, the model can be trained to reach a higher level of performance and then transformed into the original inference-time structure for inference. DBB improves ConvNets on image classification (up to 1.9% higher top-1 accuracy on ImageNet), object detection and semantic segmentation. The PyTorch code and models are released at https://github.com/DingXiaoH/DiverseBranchBlock.
1. Introduction
DBB inserts diverse, nonlinear training-time microstructures into existing ConvNet architectures, then converts them into regular convolutions for inference without changing the macro architecture or inference-time structure. It improves performance across classification, detection, and segmentation while preserving deployment costs.
- Motivation: DBB separates training-time complexity from deployment costs by allowing additional training resources while keeping deployed models the same size.This addresses the trade-off between higher performance and efficiency-sensitive deployment constraints.
- DBB design: DBB upgrades a regular K × K convolution into a universal building block with diverse branches, including multi-scale convolutions, sequential convolutions, average pooling, and branch addition.These branches provide different receptive fields and path complexities, enriching the feature space.
- Structural re-parameterization: Training-time DBB structures can be equivalently transformed into a single convolution, preserving the original macro architecture and inference-time structure.The conversion is performed once after training, and only the resultant model is deployed.
- Contribution: The approach preserves the original macro architecture while replacing selected regular convolutions with more complicated training-time microstructures.DBB is intended as a universal building block complementary to broader architecture, search, augmentation, and training methods.
- Results: DBB improves ConvNets on ImageNet, COCO detection, and Cityscapes segmentation, including up to 1.9% higher top-1 accuracy on ImageNet.The reported gains span image classification, object detection, and semantic segmentation.
2. Related Work
DBB builds on multi-branch architectures and structural re-parameterization while targeting broader architectural compatibility and faster deployment than a real Inception unit. It also differs from data-dependent and kernel-level re-parameterization approaches.
- Multi-branch architectures: Inception architectures use multi-branch structures, diverse connections, varied receptive fields, and branch combinations to enrich the feature space.DBB borrows this multi-branch idea but packages it as a reusable building block.
- Multi-branch architectures: Unlike a real Inception unit, DBB branches can each be converted into convolutions and merged into a single convolution for faster inference.This enables DBB to retain a simple deployment structure while using diverse training-time paths.
- ConvNet components: DBB is designed for use across numerous ConvNet architectures, whereas related components such as SE, ECA, Octave, Deformable, dilated, and BlurPool convolutions target other improvements.The paper characterizes DBB as complementary because it upgrades a fundamental building block.
- Multi-branch architectures: DBB reports that combining branches with different representational capacities can outperform duplicate strong-capacity branches.The paper presents this as a potential insight for ConvNet architecture design.
- Structural re-parameterization: Structural re-parameterization converts a training-time block into a convolution, encompassing DBB and related methods such as RepVGG, ExpandNet, DO-Conv, and ACNet.The paper distinguishes this from methods that re-parameterize kernels or generate weights conditionally without converting a block structure.
3. Diverse Branch Block
DBB enriches a convolution’s training-time feature space through diverse branches while preserving an equivalent single-convolution form for inference. Six transformations fuse normalization, branch structures, pooling, multi-scale kernels, and convolutional sequences into that deployment form.
- A convolution maps a C-channel feature map to D output channels using kernel F and optional bias b, with padding and stride determining sliding-window correspondence.
- Batch normalization can be fused into a preceding convolution, producing a single convolution with transformed kernel F′ and bias b′ for inference.The transformed parameters are assigned to a single convolution after training rather than recomputed before each forward pass.
- Convolution additivity merges branches with identical configurations by summing their kernels and biases, while differing branches are first transformed into compatible K × K convolutions.Additivity requires matching channels, kernel size, stride, padding, and other configurations so the branches share sliding-window correspondence.
- Average pooling is equivalent to a convolution with the same kernel size and stride, while smaller kernels can be converted to K × K kernels through zero-padding.Average pooling downsamples when stride s > 1 and smooths when s = 1; 1 × 1, 1 × K, and K × 1 convolutions require input padding to align sliding windows.
- DBB uses diverse branches, including multi-scale convolutions, sequential 1 × 1 - K × K convolutions, average pooling, and branch addition, to enrich the feature space.The representative instance includes 1 × 1, 1 × 1 - K × K, and 1 × 1 - AVG branches alongside the original K × K layer.
4. Experiments
Experiments evaluate DBB across classification, detection, and segmentation settings, then analyze branch contributions, training-time nonlinearity, and computational costs. DBB consistently improves several classification models, while retaining the original inference structure after conversion.
- Experimental setup: Replacing every eligible convolution and following BN with DBB preserves the original inference structure after post-training conversion.The experiments exclude larger kernels and test converted DBB-Nets using the original model structure.
- Classification results: DBB outperforms ACNet, suggesting that combining paths with different complexities can benefit performance beyond aggregating multi-scale convolutions.The comparison uses the same training settings, while the authors note that original-model hyperparameters may be less suitable for DBB-Nets.
- Branch analysis: Branch scaling factors differ by stride: the 1 × 1 - AVG branch is more important for stride-2 blocks, while 1 × 1 convolution is more useful with stride 1.For stride-2 DBBs, the three convolutional branches have comparable scaling magnitudes and average pooling is especially important; stride-1 distributions differ across channels.
- Ablation studies: Removing any branch degrades performance, while diverse branches outperform duplicate branches and can provide better accuracy with fewer training-time parameters.The (K × K + 1 × 1 + (1 × 1 - AVG)) DBB reaches 70.40% versus 70.29% for triple K × K while using 2.3× fewer training-time parameters.
- Ablation studies: Training-time BN nonlinearity and diverse connections both contribute to performance, whereas duplicate branches without this nonlinearity provide little improvement.The Double Duplicate block changes from 69.54% to 69.59% when made purely linear, while the comparable DBB reaches 69.83% without BN versus 70.15% with BN.
5. Conclusions
DBB combines diverse branches in a training-time ConvNet block, then retains a single-convolution inference structure without extra inference-time costs.
- DBB combines diverse branches to improve ConvNet performance without adding inference-time costs.