Source-linked AI summary
RFAConv: Receptive-Field Attention Convolution for Improving Convolutional Neural Networks
Xin Zhang, Chen Liu, Degang Yang, Tingting Song, Yichen Ye, Ke Li, Yingze Song
TL;DR
The paper addresses the limitation that spatial attention does not fully resolve parameter sharing for large convolutional kernels. It introduces Receptive-Field Attention and RFAConv, which focus attention within receptive fields and substantially improve network performance with almost negligible overhead. Experiments across multiple vision tasks and datasets support the proposed approach, while memory overhead limits use on storage-constrained devices.
Problem
Existing spatial attention mechanisms address convolutional parameter sharing but do not fully resolve it for larger kernels because attention weights remain shared across sliding windows.
Method
The paper expands spatial features into non-overlapping receptive-field features and learns receptive-field attention weights to construct RFAConv and related RFCBAMConv and RFCAConv operations.
Results
RFAConv significantly improves network performance with almost negligible computational overhead and parameter increase across classification, object detection, and semantic segmentation experiments.
Takeaways & Limitations
Prioritizing receptive-field spatial features provides a basis for non-shared-parameter convolutional operations and improves performance over conventional convolution in the reported experiments.
Takeaways & Limitations
Learning attention weights for every receptive-field position increases memory overhead, limiting use of 3 × 3 non-shared convolutions on devices with restricted storage.
Abstract
from arXiv · showhide
In the realm of deep learning, spatial attention mechanisms have emerged as a vital method for enhancing the performance of convolutional neural networks. However, these mechanisms possess inherent limitations that cannot be overlooked. This work delves into the mechanism of spatial attention and reveals a new insight. It is that the mechanism essentially addresses the issue of convolutional parameter sharing. By addressing this issue, the convolutional kernel can efficiently extract features by employing varying weights at distinct locations. However, current spatial attention mechanisms focus on shallow attention to spatial features, which is insufficient to address the fundamental challenge of parameter sharing in convolutions involving larger kernels. In response to this challenge, we introduce a novel attention mechanism known as Receptive-Field Attention (RFA). Compared to existing spatial attention methods, RFA not only concentrates on the receptive-field spatial features but also offers effective attention weights for large convolutional kernels. Building upon the RFA concept, a Receptive-Field Attention Convolution (RFAConv) is proposed to supplant the conventional standard convolution. Notably, it offers nearly negligible increment of computational overhead and parameters, while significantly improving network performance. Furthermore, this work reveals that current spatial attention mechanisms require enhanced prioritization of receptive-field spatial features to optimize network performance. To validate the advantages of the proposed methods, we conduct many experiments across several authoritative datasets, including ImageNet, COCO, VOC, and Roboflow...
1. Introduction
Standard convolution reduces computational cost through parameter sharing, but shared weights cannot fully reflect location-specific information. The paper proposes receptive-field attention to address this limitation and introduces RFAConv as a replacement for standard convolution.
- Shared parameters reduce convolutional neural network computational overhead and model complexity.
- Standard convolution applies identical parameters across receptive fields, limiting its ability to model differences between image locations.The paper motivates assigning different convolutional parameters at different locations.
- Existing spatial attention mechanisms address convolutional parameter sharing but remain restricted to spatial features and larger kernels.
- Receptive-Field Attention considers feature significance within each receptive field and provides the basis for RFAConv.RFAConv is introduced as a replacement for traditional standard convolution operations.
- RFAConv substantially improves network performance with minimal additional parameters and computational overhead across classification, detection, and segmentation experiments.Experiments include ImageNet, COCO, VOC, and Roboflow.
2. Related Works
Related work spans deformable convolution and channel, spatial, coordinate, and triplet attention mechanisms. The paper combines attention with convolution to target receptive-field spatial features and reduce parameter sharing.
- Deformable convolution addresses fixed sampling positions by learning offsets for convolutional kernels.
- Prior attention methods include squeeze-and-excitation, CBAM, coordinate attention, multi-scale channel attention, and TripletAttention.
- The paper combines attention mechanisms with convolution to create non-shared-parameter convolutional operations.Its stated focus is receptive-field spatial features, which existing attention mechanisms do not specifically target.
3. Methods
The method reframes spatial attention as a way to address convolutional parameter sharing, then introduces receptive-field features and attention to avoid shared weights within larger kernels.
- Spatial Attention and Standard Convolutional Operations: Existing spatial attention addresses parameter sharing for 1 × 1 convolutions but remains limited for larger kernels such as 3 × 3.For larger kernels, overlapping receptive-field features cause attention weights to be shared across sliding windows.
- Receptive-Field Spatial Feature: Receptive-field spatial features expand the original feature map into non-overlapping sliding windows whose size is determined by the convolution kernel.For a 3 × 3 kernel, the expanded feature map becomes three times wider and three times higher.
- Receptive-Field Attention Convolution: Receptive-Field Attention assigns different weights to features within each receptive-field slider, transforming standard convolution into a non-shared-parameter operator.The attention map is learned over receptive-field features rather than only over the original spatial feature map.
- Receptive-Field Attention Convolution: RFAConv combines receptive-field attention with standard convolution through Grouping Optimization and a Receptive-Field Attention Process.The approach uses receptive-field feature extraction, attention weighting, and shape adjustment before the subsequent convolution operation.
- Receptive-Field Attention Convolution: GroupConv extracts 3 × 3 receptive-field features with the same 9C × H×W output dimension as Unfold while requiring the same number of parameters.On YOLOv5n and VisDrone, the GroupConv-based method achieved good performance with less training time than Unfold over 300 epochs.
- Receptive-Field Attention Convolution: RFCBAM and RFCA adapt CBAM and CA to prioritize receptive-field spatial features in non-shared-parameter convolutional operations.Their final k×k convolution uses stride = k to extract feature information from the receptive-field representation.
4. Experiments and Discussions
Experiments across classification, detection, segmentation, and diverse datasets show that receptive-field spatial attention improves convolutional networks with small computational and parameter increases. RFAConv and its RFCBAMConv/RFCAConv variants generally outperform standard and existing attention-based convolutions, although gains vary by task and dataset.
- Classification experiments on ImageNet-1k: Replacing 3 × 3 convolutions with RFAConv improves ImageNet recognition while adding only 0.16 M parameters and 0.09 G computational overhead in ResNet18.ResNet18 accuracy increases by 1.64% on TOP1 and 1.24% on TOP5 relative to the original model.
- Classification experiments on ImageNet-1k: RFCBAMConv and RFCAConv improve recognition over CBAMConv and CAConv with only small increases in parameters and computational overhead.These results support placing attention on receptive-field spatial features.
- Detection experiments: RFAConv improves performance on Roboflow-100 across 100 datasets from different domains, supporting evaluation across varied detection settings.The experiments use YOLOv5n and average results over the 100 datasets.
- Semantic segmentation experiments: RFAConv improves semantic-segmentation performance over conventional convolution but remains below leading attention-based counterparts because RFA does not capture long-distance information.The paper reports that RFCBAMConv and related variants further demonstrate the value of receptive-field spatial features.
- Discussions: RFAConv assigns distinct attention weights to each receptive-field slider, enabling location-specific feature emphasis and improved classification over other attention mechanisms.Grad-CAM visualizations show that RFCBAMConv and RFCAConv emphasize key objects more effectively than CBAMConv and CAConv.
- Detection experiments: RFAConv improves detection across benchmark experiments with small parameter and computational increases, while gains are substantially larger on COCO2017 than on VOC.The reported analysis attributes this difference to greater spatial variation and target complexity in COCO2017.
5. Conclusion
The work reframes spatial attention as a way to address convolutional parameter sharing and proposes receptive-field-focused methods to improve network performance. It validates these methods across classification, detection, and segmentation tasks while acknowledging increased memory overhead.
- Existing spatial attention analyses overlook its deeper relationship with convolutional kernels and do not fully resolve parameter sharing within receptive fields.
- RFA emphasizes receptive-field spatial features and feature significance within the sliding window, while RFAConv substitutes traditional standard convolution.
- Improved RFCBAM and RFCA combine with standard convolution to create non-shared parameter convolutional operations.
- The methods are evaluated through classification, object detection, and semantic segmentation experiments on ImageNet, COCO, VOC, and Roboflow.
- Learning distinct attentional weights for each receptive-field position increases network memory overhead, motivating future arbitrarily shaped convolutional kernels.
CRediT authorship contribution statement
The authors’ contributions include conceptualization, methodology, validation, software, analysis, investigation, writing, visualization, resources, data curation, supervision, project administration, and funding acquisition.
- The contribution statement assigns authors roles spanning conceptualization, methodology, validation, software, analysis, investigation, writing, and visualization.
- It also records contributions in resources, data curation, project administration, supervision, and funding acquisition.