Source-linked AI summary
Dynamic Region-Aware Convolution
Jin Chen, Xijun Wang, Zichao Guo, Xiangyu Zhang, Jian Sun
TL;DR
Standard convolution's shared filters limit semantic variation, while increasing filters is computationally inefficient. DRConv assigns input-dependent filters to spatial regions, and reports strong results across vision tasks, including gains over standard convolution baselines.
Problem
Standard convolution's spatially shared filters limit modeling semantic variations, while increasing filters across channels is computationally inefficient.
Source: §1 · p. 1Method
DRConv uses a learnable guided mask to divide spatial features into regions and generates filters for each input and region.
Source: §1 · p. 2Results
DRConv reports strong performance in classification, face recognition, detection, and segmentation, including a 6.3% gain over ShuffleNetV2 at 0.5× scale.
Source: §1 · p. 2, §4.1 · p. 5Takeaways & Limitations
DRConv is presented as a substitute for standard convolution in existing networks while preserving translation invariance.
Source: §6 · p. 8
Abstract
from arXiv · showhide
We propose a new convolution called Dynamic Region-Aware Convolution (DRConv), which can automatically assign multiple filters to corresponding spatial regions where features have similar representation. In this way, DRConv outperforms standard convolution in modeling semantic variations. Standard convolutional layer can increase the number of filers to extract more visual elements but results in high computational cost. More gracefully, our DRConv transfers the increasing channel-wise filters to spatial dimension with learnable instructor, which not only improve representation ability of convolution, but also maintains computational cost and the translation-invariance as standard convolution dose. DRConv is an effective and elegant method for handling complex and variable spatial information distribution. It can substitute standard convolution in any existing networks for its plug-and-play property, especially to power convolution layers in efficient networks. We evaluate DRConv on a wide range of models (MobileNet series, ShuffleNetV2, etc.) and tasks (Classification, Face Recognition, Detection and Segmentation). On ImageNet classification, DRConv-based ShuffleNetV2-0.5x achieves state-of-the-art performance of 67.1% at 46M multiply-adds level with 6.3% relative improvement.
1. Introduction
DRConv addresses limits of spatially shared standard filters by assigning input-dependent filters to spatial regions while retaining translation invariance. The introduction presents it as a plug-in replacement evaluated across several vision tasks.
1. Introduction
- Repeated standard convolution raises computational cost and creates optimization difficulties, while local convolution sacrifices translation invariance.Source: §1 · pp. 1–2
- DRConv uses a learnable guided mask to group spatial positions into regions and generates filters for each input and region.Source: §1 · p. 2
- The method is presented as improving semantic representation while retaining standard convolution's translation invariance.Source: §1 · p. 2
- The guided-mask sharing pattern is trained through task-loss backpropagation, making the method end-to-end optimizable.Source: §1 · p. 2
- DRConv achieves strong results in classification, face recognition, detection, and segmentation by replacing standard convolution without much added computation.Source: §1 · p. 2
2. Related Work
Prior work varies spatial processing through local or region-aware operations, or makes network behavior data-dependent. DRConv is distinguished by adaptively grouping spatial positions and sharing filters within each region while preserving translation invariance.
2. Related Work
- Local convolution assigns unshared filters per pixel; DeepFace and DeepID demonstrate its advantages for face recognition.Source: §2 · p. 2
- R-FCN [5] forms 3 × 3 blocks from selected channel subtensors, while Sun et al. [27] use part-based convolution for person re-identification.Source: §2 · p. 2
- Spatial Transformer Networks warp feature maps but are difficult to train; ACU offsets become static after training, whereas deformable convolution offsets remain dynamic.Source: §2 · p. 2
- DRConv adaptively divides spatial positions into regions, shares one filter within each, and maintains translation invariance.Source: §2 · p. 2
- SKNet [19] selects receptive fields channel-wise, while CondConv [33] dynamically combines kernels; DRConv instead uses a dynamic mask to assign region filters.Source: §2 · p. 3
3. Our Approach
DRConv assigns multiple filters to spatial regions while retaining translation-invariance among positions with similar features. Its approach combines a learnable guided mask with sample-specific filters generated for those regions.
3. Our Approach
- DRConv uses multiple filters across spatial regions to increase feature diversity while maintaining translation-invariance for positions with similar features.This is presented as a response to standard convolution’s limited semantic variation and local convolution’s loss of translation-invariance.Source: §3 · p. 3
3.1. Dynamic Region-Aware Convolution
- DRConv partitions spatial positions into regions that share one filter, grouping semantically similar features; kernels larger than 1×1 can draw from adjacent regions at borders.The method defines a region mask and assigns one filter to each region.Source: §3.1 · pp. 3–4
3.2. Learnable guided mask
- A learnable guided mask assigns positions to filters using the maximum guided-feature channel; a softmax-based approximate backward pass lets loss gradients update guided features despite argmax.The guided feature has m channels, and the selected channel index identifies the filter assigned to each position.Source: §3.2 · p. 4
3.3. Dynamic Filter: Filter generator module
- The filter generator pools each input sample to k×k, then applies two 1×1 convolutions—sigmoid first, groups=m second—to predict its m region filters.The predicted filters are adjusted to the characteristics of each sample.Source: §3.3 · pp. 4–5
4. Experiments
Experiments evaluate DRConv with 8 learnable regions across classification, face recognition, detection, and segmentation, generally replacing standard convolutions in existing networks. The reported results show gains over baselines on all three task groups represented in these passages.
4.1. Classification
- 6.3% and 3.6% gains over ShuffleNetV2 at 0.5× and 1× scale, respectively, accompany gains over MobileNetV2 and MobileNetV1 at comparable computational cost.DRConv-based models replace all 1×1 standard convolutions in the backbone; Table 1 compares Top-1 ImageNet accuracy.Source: §4.1 · p. 5, Table 1 · p. 6
4.2. Face Recognition
- 4.9% over baseline MobileFaceNet, 1.4% over CondConv, and 1.3% over Local-MobileFaceNet are reported for DRConv-MobileFaceNet on MegaFace.The local-convolution comparison applies local convolution only in MobileFaceNet’s last three layers because of device-memory limits; Table 2 reports rank-1 accuracy with 1M distractors.Source: §4.2 · p. 6, Table 2 · p. 6
4.3. COCO Object Detection and Segmentation
- On COCO, 8-region DRConv improves DetNAS-300M detection by 1.8%, while 16-region DRConv in two Mask R-CNN FPN layers improves detection by 1.2% and segmentation by 1.1%.The comparison replaces 1×1 backbone convolutions in DetNAS-300M and only two FPN layers in Mask R-CNN; the latter gains use COCO’s standard AP metric.Source: §4.3 · p. 7, Table 3 · p. 7
5. Ablation Study
The ablations find that smaller models gain more from DRConv, and that its guided masks assign filters to regions with similar semantics. Region connectivity also tends to differ between shallow and deep layers.
Visualization of dynamic guided mask
- With m = 4, visualized guided masks assign filters to regions with the same semantics in classification and face-recognition images.The authors also report that clearer semantic representation may lead masks to form fewer regions in deeper layers.Source: §5 · p. 8
- Guided-mask regions tend to be discrete in shallow layers and connected in deep layers, corresponding respectively to detailed context and larger receptive fields tied to semantic information.The mask is determined by spatial-information distribution, so a region can contain connected or discrete points.Source: §5 · p. 8
Different model size
- Smaller ShuffleNetV2 and MobileNetV2 models gain more from DRConv than larger models, with the authors attributing this to limited feature-extraction capacity.The compared model scales are [0.5×, 1×, 1.5×, 2×] for ShuffleNetV2 and [0.25×, 0.5×, 0.75×, 1×] for MobileNetV2.Source: §5 · p. 8
6. Conclusion
DRConv uses partial filter sharing across spatial regions while maintaining translation-invariance, and the authors present it as a substitute for standard convolution. Its guided-mask and filter-generator modules support per-sample filter assignment, with experiments reporting effectiveness across several tasks.
6. Conclusion
- DRConv is motivated by partial filter sharing in spatial dimensions and maintains translation-invariance, so the authors present it as a substitute for standard convolution in existing networks.Source: §6 · p. 8
- A learnable guided-mask module assigns the same filter to similar features within a region, while a filter generator produces customized filters for each input.Source: §6 · p. 8
- Experiments across classification, face recognition, object detection, and segmentation report that DRConv outperforms state-of-the-art and other manually designed methods; ablations identify the guided mask as key to filter distribution and performance.Source: §6 · p. 8