Source-linked AI summary
Coordinate Attention for Efficient Mobile Network Design
Qibin Hou, Daquan Zhou, Jiashi Feng
TL;DR
Mobile attention mechanisms often neglect positional information, while many alternatives impose computational costs unsuitable for mobile networks. Coordinate attention addresses both issues by embedding positional information into channel attention, improving ImageNet classification, object detection, and semantic segmentation.
Problem
Existing mobile attention methods often neglect positional information, while many attention mechanisms impose computational overhead unsuitable for mobile networks.
Method
Coordinate attention factorizes channel attention into parallel vertical and horizontal 1D feature-encoding processes that preserve positional information and generate direction-aware attention maps.
Results
0.8% top-1 ImageNet accuracy gain was achieved with comparable parameters and computation, alongside significant improvements in object detection and semantic segmentation.
Takeaways & Limitations
Coordinate attention is a lightweight, flexible mechanism that improves mobile-network performance across classification, detection, and semantic segmentation.
Abstract
from arXiv · showhide
Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps. In this paper, we propose a novel attention mechanism for mobile networks by embedding positional information into channel attention, which we call "coordinate attention". Unlike channel attention that transforms a feature tensor to a single feature vector via 2D global pooling, the coordinate attention factorizes channel attention into two 1D feature encoding processes that aggregate features along the two spatial directions, respectively. In this way, long-range dependencies can be captured along one spatial direction and meanwhile precise positional information can be preserved along the other spatial direction. The resulting feature maps are then encoded separately into a pair of direction-aware and position-sensitive attention maps that can be complementarily applied to the input feature map to augment the representations of the objects of interest. Our coordinate attention is simple and can be flexibly plugged into classic mobile networks, such as MobileNetV2, MobileNeXt, and EfficientNet with nearly no computational overhead. Extensive experiments demonstrate that our coordinate attention is not only beneficial to ImageNet classification but more interestingly, behaves better in down-stream tasks, such as object detection and semantic segmentation. Code is available at https://github.com/Andrew-Qibin/CoordAttention.
1. Introduction
The introduction identifies computational cost and lost positional information as key limitations of attention in mobile networks. It proposes coordinate attention, which factorizes channel attention into two 1D directional encodings to preserve positional information with low overhead, and reports gains across classification and downstream vision tasks.
- Motivation: Attention mechanisms improve deep networks, but their use in mobile networks lags because most impose unaffordable computational overhead.Mobile networks have limited model size and computation capacity.
- Motivation: SE attention remains popular for mobile networks because 2D global pooling provides channel attention at low cost, but it neglects positional information needed to capture objects.SE encodes inter-channel information without preserving spatial position.
- Coordinate attention: Coordinate attention embeds positional information into channel attention by replacing 2D pooling with parallel 1D feature encoding along vertical and horizontal directions.The resulting direction-aware maps encode long-range dependencies along their respective spatial directions.
- Coordinate attention: Coordinate attention captures cross-channel, direction-aware, and position-sensitive information while remaining lightweight and pluggable into MobileNetV2 and MobileNeXt building blocks.The introduction describes the method as flexible and suitable for classic mobile-network blocks.
- Experimental results: 0.8% performance gain in top-1 classification accuracy on ImageNet is achieved with comparable learnable parameters and computation, alongside significant improvements in object detection and semantic segmentation.The experiments cover ImageNet classification, object detection, and semantic segmentation.
2. Related Work
Related work spans efficient mobile architectures, channel and spatial attention, and non-local/self-attention models. The proposed approach targets mobile networks by capturing positional information and channel-wise relationships with lightweight one-dimensional encoding.
- Efficient mobile architectures: Recent mobile networks mainly use depthwise separable convolutions and inverted residual blocks, with later designs adding spatial modeling, channel operations, or neural architecture search.HBONet introduces down-sampling inside inverted residual blocks; ShuffleNetV2 uses channel split and shuffle modules; MobileNetV3 searches activation functions and expansion ratios.
- Attention mechanisms: Attention mechanisms improve computer vision tasks, while SENet models channel dependencies by squeezing each 2D feature map and CBAM adds spatial encoding with large kernels.The passage also identifies GENet, GALA, AA, and TA as later attention works.
- Non-local and self-attention models: Non-local and self-attention networks capture spatial or channel-wise attention through non-local mechanisms but are often unsuitable for mobile networks because their modules are computationally expensive.Examples include NLNet, GCNet, A2Net, SCNet, GSoP-Net, and CCNet.
- Proposed approach: The proposed approach efficiently captures positional information and channel-wise relationships for mobile networks by factorizing 2D global pooling into two one-dimensional encoding processes.It is described as lightweight and as performing much better than SENet, CBAM, and TA.
3. Coordinate Attention
Coordinate attention extends channel attention by preserving positional information through separate horizontal and vertical feature encoding. It generates direction-aware attention maps that model channel relationships, capture long-range dependencies, and selectively enhance mobile-network features.
- SE attention: SE attention uses squeeze for global information embedding and excitation for adaptive recalibration of channel relationships.The squeeze operation collects global information from local convolutional descriptors, while excitation captures channel-wise dependencies.
- Motivation and overview: Unlike SE attention, coordinate attention preserves positional information while modeling channel relationships for spatially selective attention maps.SE reweights channels but neglects positional information, which is important for generating spatially selective attention maps.
- Coordinate information embedding: Coordinate attention factorizes global pooling into horizontal and vertical 1D feature encodings, producing direction-aware feature maps instead of one feature vector.Pooling kernels with spatial extents (H, 1) and (1, W) aggregate each channel along the two spatial directions.
- Coordinate attention generation: The attention-generation stage concatenates the directional maps, applies a shared 1 × 1 convolution, splits the result, and transforms each direction into attention weights.The reduction ratio r controls the intermediate block size, with an example value of 32 used to reduce model-complexity overhead.
- Discussion: Applying horizontal and vertical attention simultaneously makes each attention-map element indicate whether an object of interest exists in its corresponding row and column.The resulting maps encode spatial information while attention is applied along both directions to the input tensor.
- Ablation comparison: The ablation comparison reports that either horizontal or vertical attention matches SE, whereas combining both directions produces the best result.The table uses MobileNetV2 as the baseline and compares different reduction ratios and attention settings.
4. Experiments
Experiments show that coordinate attention benefits image classification and remains effective across mobile architectures, attention settings, object detection, and semantic segmentation. Its strongest gains appear in dense prediction tasks, where positional information and long-range dependencies are especially valuable.
- Ablation studies: Combining horizontal and vertical attention produces the best classification result, showing that coordinate information embedding helps beyond either direction alone.The comparison is made at comparable learnable parameters and computational cost, against single-direction attention and SE attention.
- Ablation studies: Coordinate attention remains stronger than SE attention and CBAM when the reduction ratio changes, demonstrating robustness to attention-block compression.Halving the reduction ratio increases model size and can improve performance, while coordinate attention still yields the best results.
- Attention for Mobile Networks: Adding SE attention raises classification performance by more than 1%, whereas CBAM’s spatial module does not improve mobile networks relative to SE attention.The paper attributes coordinate attention’s advantage to reduced information loss and complementary 1D global pooling that captures global dependencies.
- Attention for Mobile Networks: Coordinate attention achieves the best EfficientNet-b0 result with comparable parameters and computations, extending its effectiveness to powerful mobile networks.This result supports the method’s flexibility beyond MobileNetV2 and MobileNeXt.
- Object detection: 24.5 v.s. 22.3: adding coordinate attention to MobileNetV2 substantially improves COCO detection with only 0.5M extra parameters and nearly the same computation.SSDLite320 with coordinate attention also achieves the best results across all reported metrics versus SE and CBAM at nearly matched resources.
- Downstream tasks: 71.7 to 73.1: coordinate attention raises Pascal VOC mean AP, while SE and CBAM do not improve the baseline; it also outperforms alternatives on semantic segmentation.On Cityscapes, coordinate attention improves segmentation by a large margin with comparable parameters, with larger gains than classification or detection.
5. Conclusions
The paper presents coordinate attention, a lightweight mobile-network mechanism that combines channel-interaction modeling with long-range dependencies and precise positional information. Experiments demonstrate its effectiveness in ImageNet classification, object detection, and semantic segmentation.
- Conclusions: Coordinate attention is a novel lightweight attention mechanism for mobile networks.It is presented as the paper’s main proposed mechanism.
- Conclusions: Coordinate attention models inter-channel relationships while capturing long-range dependencies with precise positional information.It inherits the channel-attention advantage of modeling inter-channel relationships and adds positional information.
- Conclusions: Experiments demonstrate coordinate attention’s effectiveness in ImageNet classification, object detection, and semantic segmentation.The conclusion reports effectiveness across all three evaluated task types.