PaperlayerView exact revision on arXiv

Source-linked AI summary

CBAM: Convolutional Block Attention Module

Sanghyun Woo, Jongchan Park, Joon-Young Lee, In So Kweon

arXiv:1807.06521v2cs.CV

TL;DR

The paper asks how attention can increase CNN representation power by emphasizing meaningful features and suppressing unnecessary ones. It introduces CBAM, which sequentially refines features along channel and spatial dimensions, and reports consistent improvements across classification and detection benchmarks with negligible overhead.

  • Problem

    The paper addresses how attention can increase CNN representation power by emphasizing meaningful features and suppressing unnecessary ones.

  • Method

    CBAM sequentially applies channel and spatial attention maps to intermediate CNN features for adaptive feature refinement.

  • Results

    CBAM consistently improves classification and object detection across ImageNet-1K, MS COCO, and VOC 2007, with negligible parameter and computation overhead.

  • Takeaways & Limitations

    CBAM is broadly applicable across CNN architectures and tasks while adding little computational or parameter overhead.

Abstract

from arXiv · show

We propose Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks. Given an intermediate feature map, our module sequentially infers attention maps along two separate dimensions, channel and spatial, then the attention maps are multiplied to the input feature map for adaptive feature refinement. Because CBAM is a lightweight and general module, it can be integrated into any CNN architectures seamlessly with negligible overheads and is end-to-end trainable along with base CNNs. We validate our CBAM through extensive experiments on ImageNet-1K, MS~COCO detection, and VOC~2007 detection datasets. Our experiments show consistent improvements in classification and detection performances with various models, demonstrating the wide applicability of CBAM. The code and models will be publicly available.

1 Introduction

The paper introduces CBAM as a lightweight attention module that refines CNN features by focusing on important information and suppressing irrelevant features. It reports improved performance across classification and detection benchmarks and supports the module through ablation studies.

  • Module design: CBAM sequentially applies channel and spatial attention to adaptively refine intermediate feature maps within convolutional blocks.The overview describes channel and spatial sub-modules operating sequentially at every convolutional block of deep networks.
  • Empirical findings: CBAM-enhanced networks improve accuracy over various baseline networks on ImageNet-1K and focus more properly on target objects than baseline networks.Grad-CAM visualizations associate the performance boost with accurate attention and noise reduction of irrelevant clutter.
  • Motivation and proposal: CBAM is a simple, effective module designed to boost CNN representation power by focusing on important features and suppressing unnecessary ones.The module investigates attention as an architectural factor beyond depth, width, and cardinality.
  • Contributions: The authors validate CBAM through extensive ablation studies and report improvements across ImageNet-1K, MS COCO, and VOC 2007 benchmarks.The contribution statement says the lightweight module improves various networks on classification and detection benchmarks.

2 Related Work

Prior CNN research explored architectural scaling, multi-branch designs, and attention mechanisms to improve visual recognition. CBAM builds on these efforts by combining channel and spatial attention in a lightweight plug-and-play module.

  • Network Engineering: CNN architecture research has pursued deeper networks and multi-branch designs to improve performance across vision applications.The passage identifies increased depth and Szegedy et al.’s customized multi-branch Inception architecture as examples of network engineering.
  • Attention Mechanisms: Human vision motivates attention mechanisms that selectively focus on salient scene regions rather than processing the whole scene simultaneously.The passage describes sequential partial glimpses as a way to capture visual structure more effectively.
  • Attention Mechanisms: CNN attention methods include encoder-decoder attention modules that refine feature maps and improve robustness to noisy inputs.Residual Attention Network is presented as an approach for large-scale classification.
  • CBAM: CBAM is designed as a plug-and-play module for pre-existing base CNN architectures.The supplied passage explicitly characterizes the module as plug-and-play.
  • CBAM: Unlike Squeeze-and-Excitation, CBAM uses max-pooled features alongside global average-pooled features for channel attention and adds spatial attention.The passage states that average-pooled features alone are suboptimal for fine channel attention and that spatial attention determines where to focus.

3 Convolutional Block Attention Module

CBAM sequentially refines an intermediate feature map with channel and spatial attention maps, using complementary mechanisms to emphasize what and where is informative. Channel attention uses pooled spatial descriptors and a shared MLP, while spatial attention uses channel-pooled maps and convolution; experiments favor sequential, channel-first placement.

  • 3 Convolutional Block Attention Module: CBAM sequentially infers a 1D channel attention map and a 2D spatial attention map, then multiplies them with the input feature map for refinement.Attention values are broadcast along the complementary spatial or channel dimensions to produce the final refined output.
  • Channel Attention Module: Channel attention models inter-channel relationships to emphasize what is meaningful, aggregating spatial information with average-pooling and max-pooling descriptors.The two descriptors are processed by a shared network to produce the channel attention map.
  • Channel Attention Module: The channel-attention shared network is a one-hidden-layer MLP with hidden activation size R^C/r×1×1, shared across descriptors and followed by ReLU after W0.The outputs from both descriptors are merged by element-wise summation before sigmoid activation.
  • Spatial Attention Module: Spatial attention models inter-spatial relationships to emphasize where informative content occurs, using channel-wise average-pooling and max-pooling followed by concatenation and convolution.The resulting 2D spatial attention map encodes where to emphasize or suppress, with the convolution filter size set to 7 × 7.
  • Arrangement of Attention Modules: Sequential placement performs better than parallel placement, and channel-first ordering is slightly better than spatial-first ordering.The two modules compute complementary attention focused on what and where, respectively.

4 Experiments · 4.1 Ablation studies · Channel attention.

The experiments evaluate CBAM across standard classification and detection benchmarks, using reproduced CNN baselines and ablations to assess channel attention design choices. Channel attention performs best when inferred from both average- and max-pooled features through a shared MLP.

  • 4 Experiments: CBAM is evaluated on ImageNet-1K classification and MS COCO and VOC 2007 object detection using reproduced networks for consistent comparisons.The evaluated networks were reproduced in PyTorch, with reproduced results reported throughout the experiments.
  • 4 Experiments: CBAM outperforms the evaluated baselines across architectures and tasks without additional bells and whistles.The module can be integrated into CNN architectures and jointly trained with the resulting CBAM-enhanced networks.
  • 4.1 Ablation studies: The ablation study uses ResNet-50 on ImageNet-1K to empirically evaluate CBAM’s design choices.The study uses standard ImageNet-1K training augmentation and single-crop 224×224 evaluation.
  • 4.1 Ablation studies: The module design process separately searches channel attention, spatial attention, and their combination.Each component is evaluated in sequence before considering how channel and spatial attention should be combined.
  • Channel attention.: The channel-attention ablation compares average pooling, max pooling, and their joint use, with shared MLP inference for the two pooled features.Average-pooled channel attention is equivalent to the SE module; shared MLP inference saves parameters because both features occupy the same semantic embedding space.
  • Channel attention.: Max-pooled features improve accuracy comparably to average-pooled features, while SE uses only average-pooled features.The experiments therefore identify max pooling as meaningful for channel-attention inference rather than redundant with average pooling.
  • Channel attention.: The reported ablations establish channel-attention design as one stage of a broader CBAM evaluation spanning spatial attention and attention-combination strategies.The accompanying tables separately compare spatial-attention methods and channel–spatial combinations.
  • Channel attention.: CBAM combines average- and max-pooled channel features because salient-part responses complement the softer global statistics captured by average pooling.A shared network processes both features, and its outputs are merged by element-wise summation to improve performance beyond SE without additional learning.

Spatial attention.

The spatial attention module forms a 2D descriptor from channel information, applies a convolution, and sigmoid-normalizes the resulting attention map. Experiments compare descriptor construction, convolution kernel size, and arrangements with channel attention, finding advantages for channel pooling and larger kernels.

  • Spatial attention: Spatial attention computes a 2D channel-information descriptor, applies one convolution layer, and normalizes the raw attention map with sigmoid.The module operates on channel-wise refined features and produces a 2D spatial attention map.
  • Spatial attention: The study compares channel pooling using average- and max-pooling with standard 1 × 1 convolution for reducing the channel dimension.It also evaluates convolution kernel sizes of 3 and 7.
  • Spatial attention: Channel pooling produces better accuracy than learnable weighted channel pooling implemented with 1 × 1 convolution.The authors attribute this to explicitly modeled pooling producing finer attention inference.
  • Spatial attention: Larger convolution kernels generate better accuracy in both descriptor-generation cases, indicating that spatial decisions need a broad view or large receptive field.The compared kernel sizes are 3 and 7.
  • Spatial attention: The experiment compares sequential channel-spatial, sequential spatial-channel, and parallel arrangements of the channel and spatial attention submodules.The comparison tests whether module order affects overall performance because channel attention is globally applied whereas spatial attention works locally.

Arrangement of the channel and spatial attention.

The ablation study finds that sequentially arranging channel and spatial attention produces finer attention maps than parallel arrangement, with channel-first slightly outperforming spatial-first. The final CBAM uses sequential submodules and achieves a 22.66% top-1 error with ResNet50.

  • Arrangement strategy: Sequentially inferring attention maps produces finer attention maps than arranging channel and spatial attention in parallel.The study compares different attention-arrangement methods.
  • Arrangement strategy: Channel-first ordering performs slightly better than spatial-first ordering.Both sequential ordering choices outperform using only channel attention independently.
  • Final module: The final module uses average- and max-pooling for both attention modules, a 7×7 convolution for spatial attention, and sequential channel-spatial arrangement.These design choices define the final CBAM configuration.
  • Final module: 22.66% top-1 error is achieved by ResNet50 + CBAM.This is the reported performance of the final module.

Final module design. · 4.2 Image Classification on ImageNet-1K

CBAM is evaluated on ImageNet-1K across multiple CNN architectures and shows improved classification performance over baselines and SE. Its low parameter and computational overhead also supports application to lightweight MobileNet models.

  • 4.2 Image Classification on ImageNet-1K: ImageNet-1K experiments evaluate CBAM in ResNet, WideResNet, and ResNext architectures.The experiments follow the protocol specified in Sec. 4.1.
  • 4.2 Image Classification on ImageNet-1K: All reported ImageNet-1K results are reproduced in the PyTorch framework.This implementation detail accompanies the classification experiments.
  • 4.2 Image Classification on ImageNet-1K: CBAM networks significantly outperform all baselines and improve accuracy over SE on ImageNet-1K.SE is identified as the winning approach of the ILSVRC 2017 classification task.
  • 4.2 Image Classification on ImageNet-1K: CBAM achieves the lowest training and validation errors across the ImageNet-1K error curves.The curves indicate stronger improvement in baseline-model generalization than SE.
  • 4.2 Image Classification on ImageNet-1K: CBAM has small parameter and computational overhead, motivating its application to the lightweight MobileNet architecture.The evaluation uses both basic MobileNet and a capacity-reduced model with width multiplier α set to 0.7.
  • 4.2 Image Classification on ImageNet-1K: On MobileNet, CBAM significantly boosts baseline accuracy and favorably improves SE performance, indicating potential for low-end-device applications.The experiments cover both the basic and capacity-reduced MobileNet models.

4.3 Network Visualization with Grad-CAM [18]

Grad-CAM visualizations are used to qualitatively analyze which spatial regions different networks attend to for class prediction. The CBAM-integrated network covers target object regions better than the baseline and SE-integrated networks, suggesting improved exploitation and aggregation of target-object information.

  • Visualization method: Grad-CAM uses class-specific gradients to calculate the importance of spatial locations in convolutional layers and reveal attended regions.The analysis applies Grad-CAM to different networks using ImageNet validation images.
  • Visualization results: CBAM-integrated networks’ Grad-CAM masks cover target object regions better than other methods.The comparison includes ResNet50, ResNet50 + CBAM, and ResNet50 + SE, using last-convolutional outputs.
  • Visualization results: CBAM helps networks exploit information in target object regions and aggregate features from them.The visualizations indicate that feature refinement improves the use of target-object information for prediction.
  • Interpretation: Feature refinement by CBAM is conjectured to lead networks to utilize given features well, with scores increasing accordingly.The observation connects improved target-region attention with the networks’ feature utilization.

4.4 MS COCO Object Detection

The study evaluates object detection on Microsoft COCO using its standard train/validation images and average mAP across IoU thresholds from 0.5 to 0.95. Training follows prior setups, using most available images while holding out 5,000 validation examples.

  • Evaluation: Performance is measured by average mAP over IoU thresholds from 0.5 to 0.95.
  • Training setup: Following prior work, training uses all training images plus part of the validation set, while holding out 5,000 examples for validation.The training code is based on [40].

4.5 VOC 2007 Object Detection

On PASCAL VOC 2007, CBAM was applied to SSD- and StairNet-based detectors and improved strong baselines with negligible parameter overhead. Results with a lightweight backbone further suggest CBAM’s applicability to low-end devices.

  • Experimental setup: CBAM was evaluated on the PASCAL VOC 2007 test set by applying the module to detectors rather than base networks.The study adopted the StairNet multi-scale framework based on SSD and reproduced SSD and StairNet in PyTorch.
  • Experimental setup: The reproduced SSD and StairNet detectors achieved 77.8% and 78.9% mAP@.5, respectively.These reproduced baselines were used to estimate CBAM’s performance improvement accurately.
  • Results: CBAM improved the accuracy of all strong baselines across two backbone networks.The passage reports consistent gains in the summarized Table 7 results without specifying individual improvement values.
  • Results: The accuracy gains required only negligible parameter overhead, indicating that they arose from feature refinement rather than a naive capacity increase.The reported interpretation attributes the enhancement to CBAM’s effective feature refinement.
  • Results: Results with a lightweight backbone indicate that CBAM may be useful for low-end devices.The passage identifies the lightweight-backbone result as additional evidence for this application.

5 Conclusion

CBAM improves CNN representation power through sequential channel and spatial attention-based feature refinement, achieving considerable performance gains with small overhead. Its channel attention combines max-pooled and average-pooled features to produce finer attention than SE.

  • 5 Conclusion: CBAM improves CNN representation power using channel and spatial attention modules for attention-based feature refinement.The module applies two distinctive attention-based refinement mechanisms along channel and spatial dimensions.
  • 5 Conclusion: CBAM achieves considerable performance improvement while keeping computational overhead small.
  • 5 Conclusion: Channel attention combines max-pooled and average-pooled features to produce finer attention than SE.
Loading 1807.06521v2…