Source-linked AI summary
ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition
Yu-Sheng Liu, Yu-Chen Tung
TL;DR
FGVR channel attention often loses spatial concentration by representing channels only with GAP, making differently distributed responses indistinguishable. ConCA pairs the mean with shift-invariant NegEnt and applies lightweight per-channel gating. It improves over established and richer baselines across six FGVR benchmarks and generalizes across eight backbones on iNat2021-mini.
Problem
GAP-based channel descriptors capture activation magnitude but ignore spatial concentration, so channels with identical means can remain indistinguishable in FGVR.
Method
ConCA combines channel mean with shift-invariant negative-input entropy and maps the dual descriptor to per-channel weights using a lightweight depthwise 1-D convolutional MLP.
Results
ConCA consistently outperforms attention-free, SE-Net, ECA-Net, and four richer descriptor-based modules across six FGVR benchmarks and generalizes across eight backbones on iNat2021-mini.
Takeaways & Limitations
The channel descriptor and the per-channel gate together form an underexplored design axis for lightweight channel attention in FGVR.
Takeaways & Limitations
ConCA assumes channels carry meaningful spatial structure and targets dense-convolution backbones rather than strongly grouped architectures.
Abstract
from arXiv · showhide
Lightweight channel attention mechanisms are widely used in image classification, yet their effectiveness in fine-grained visual recognition (FGVR) remains limited. Most modules summarize each channel by global average pooling (GAP), which captures activation magnitude but ignores spatial concentration, so channels with different spatial distributions but identical means receive the same descriptor. We propose Concentration-Aware Channel Attention (ConCA), which pairs the mean with a shift-invariant negative-input entropy (NegEnt), computed via a softmax over the negated activations, forming a dual descriptor that jointly encodes magnitude and concentration. A depthwise 1-D convolutional multi-layer perceptron (MLP), whose parameter count is linear in the number of channels, maps the pair to a per-channel weight. On six fine-grained benchmarks, ConCA improves over attention-free, SE-Net, and ECA-Net baselines as well as four richer descriptor-based modules under a controlled from-scratch protocol, and it generalizes across eight backbones on iNat2021-mini. These results indicate that the channel descriptor, together with the per-channel gating that maps it to attention weights, is an important but underexplored aspect of lightweight channel attention in FGVR.
1. Introduction
Fine-grained recognition depends on localized visual cues, but GAP-based channel attention discards spatial concentration. ConCA addresses this by pairing channel mean with NegEnt and achieves broad benchmark gains with lightweight per-channel gating.
- FGVR must preserve the spatial characteristics and semantic information of localized discriminative regions.
- GAP captures activation magnitude but assigns identical descriptors to channels with different spatial distributions and equal means.
- The method uses a depthwise 1-D convolutional MLP for independent per-channel gating with parameter cost linear in channel count.
- ConCA combines channel mean with shift-invariant NegEnt, capturing activation magnitude and spatial concentration with minimal parameter overhead.
- ConCA consistently improves over attention-free, SE-Net, ECA-Net, and four richer descriptor modules across six FGVR benchmarks under controlled from-scratch evaluation.
- Cross-architecture generalization is confirmed across eight backbones on iNat2021-mini.
2. Related Work
Prior channel-attention methods enrich either descriptors or gating, while FGVR methods address subtle local cues through localization, high-order interactions, or learned attention. ConCA’s related-work position is descriptor-level and avoids extra annotations or feature-encoding modules.
- SE-Net uses GAP followed by fully connected gating, whereas ECA-Net replaces the bottleneck with shared-kernel 1-D convolution.
- CBAM and FcaNet enrich channel descriptors or attention mechanisms, while CBAM additionally combines channel and spatial attention.
- ConCA differs from CAT through negative-input rather than positive-input entropy and independent per-channel rather than shared channel-mixing gating.
- SRM is the closest counterpart because it combines the mean with a shift-invariant statistic under a linear-cost per-channel gate, but measures dispersion rather than concentration.
- FGVR approaches localize parts, model bilinear interactions, or learn attention end-to-end because global features alone are weakly discriminative for subtle local cues.
- ConCA operates at the descriptor level without extra annotations or additional feature-encoding modules.
3. Method
ConCA augments the mean channel descriptor with shift-invariant NegEnt to distinguish activation magnitude from spatial concentration, then maps both statistics to per-channel gates with a lightweight depthwise MLP. The design preserves this complementary information while keeping parameter growth linear in the channel count.
- Motivation: GAP is many-to-one: localized and spatially uniform channels can share the same mean despite different spatial patterns.This ambiguity motivates adding a descriptor for spatial concentration in FGVR.
- NegEnt descriptor: NegEnt applies entropy to a softmax over negated activations, measuring spatial concentration independently of absolute activation level.Its shift invariance follows because a constant offset cancels in the softmax.
- NegEnt descriptor: The pair (μ_c, E_c) separates channels that mean-only descriptors conflate by retaining the mean while adding a concentration axis.NegEnt decreases as the activated region expands and the low-activation background shrinks.
- Descriptor choice: ConCA uses NegEnt rather than PosEnt because, for localized FGVR responses, PosEnt concentrates on dominant activations and overlaps with information already captured by the mean.Both entropy variants are shift-invariant, but their complementarity with the mean differs.
- Module design: A depthwise 1-D MLP independently maps each channel’s mean and NegEnt to a gate that rescales the channel while preserving spatial resolution.The MLP expands the two descriptors to width 4m and compresses them to one gate; surrounding backbone layers provide cross-channel interaction.
- Complexity: ConCA requires 33C learnable parameters, growing linearly with channel count, while computing both descriptors in O(CN).At C = 2048, ConCA uses about 13% of SE-Net’s parameters; it adds +0.50M parameters and essentially no FLOPs over the attention-free baseline, although eager latency is higher.
4. Experiments
Experiments evaluate ConCA under pretrained and from-scratch protocols, across six fine-grained benchmarks and eight architectures. Results show strongest and most consistent gains from the complementary mean-plus-NegEnt descriptor with per-channel gating.
- Experimental setup: Experiments cover six fine-grained benchmarks, pretrained and from-scratch training protocols, and cross-architecture evaluation on iNat2021-mini.The from-scratch protocol uses ResNet-18 on all six benchmarks, while the transfer-learning study fine-tunes pretrained ResNet-50 models.
- Pretrained setting: Under pretrained ResNet-50 fine-tuning, all methods perform similarly, with average accuracies differing by only 0.65 points.This limited separation motivates focusing on from-scratch experiments, where the module contribution is more directly observable.
- From-scratch benchmark comparison: ConCA exceeds the attention-free baseline by 1.59 percentage points on average across six from-scratch benchmarks and achieves the highest mean accuracy on every benchmark.SE and ECA drop below the baseline by 5.34 and 2.34 percentage points on average, respectively.
- From-scratch benchmark comparison: ConCA is the only evaluated method improving over the baseline on all six benchmarks; SRM remains 3.79 points lower and CAT-c trails ConCA by 6.88 points.These comparisons contrast ConCA with richer descriptor-based modules, not only GAP-based attention.
- Cross-architecture generalization: ConCA attains the highest accuracy on all eight iNat2021-mini architectures, with gains of 1.85 points over the runner-up on ResNet-50 and 1.07 on Inception-v4.The remaining architectures show smaller but consistent improvements.
- Cross-architecture generalization: Significance tests show ConCA outperforming the baseline on seven of eight architectures, SE on six, and ECA on seven.The remaining comparisons do not reach statistical significance with three training seeds.
- Descriptor ablation: Pairing mean with NegEnt raises the best single-descriptor average from 51.96% to 52.76%, whereas adding more descriptors lowers performance.The all-7 combination falls to 48.49%, indicating that descriptor complementarity matters more than descriptor count.
- Descriptor redundancy analysis: NegEnt complements the mean more effectively than PosEnt because NegEnt is less redundant with mean and retains spatial-structure information.PosEnt varies more systematically with the mean, while NegEnt remains comparatively independent.
5. Conclusion
ConCA identifies the channel descriptor and per-channel gating as underexplored design axes, pairing mean with shift-invariant NegEnt for lightweight attention in fine-grained recognition. Its scope targets dense-convolution backbones and motivates future extensions to other descriptors, gates, and architectures.
- ConCA pairs the shift-sensitive mean with shift-invariant NegEnt under a depthwise per-channel gate, with gains attributed jointly to gating and a non-redundant descriptor.
- ConCA consistently outperforms existing channel-attention modules across six benchmarks and eight CNN architectures at minimal parameter cost.
- ConCA assumes channels carry meaningful spatial structure and targets dense-convolution backbones rather than strongly grouped architectures.
- Future work will extend descriptor design to alternative concentration measures, gating strategies, and transformer architectures.
CRediT authorship contribution statement
The authors contributed across writing, methodology, software, investigation, conceptualization, supervision, and funding acquisition.
- Yu-Sheng Liu handled the original draft, methodology, software, and investigation; Yu-Chen Tung handled review and editing, conceptualization, supervision, and funding acquisition.