Source-linked AI summary
Attention-based Dropout Layer for Weakly Supervised Object Localization
Junsuk Choe, Hyunjung Shim
TL;DR
WSOL commonly localizes only the most discriminative object part rather than its full extent. ADL uses self-attention to hide that part while highlighting informative regions, achieving new state-of-the-art localization on CUB-200-2011 with low overhead.
Problem
WSOL methods use image-level labels without location annotations, but often fail to cover the object's entire extent.
Method
ADL uses self-attention to create a drop mask that hides discriminative regions and an importance map that highlights informative regions.
Results
ADL achieves new state-of-the-art accuracy on CUB-200-2011 and comparable accuracy with current state-of-the-art methods on ImageNet-1k.
Takeaways & Limitations
ADL improves WSOL localization while remaining lightweight, requiring no additional trainable parameters and low computation overhead.
Takeaways & Limitations
The authors leave analysis of the drop mask's regularization effect and learning less discriminative regions outside the object for future work.
Abstract
from arXiv · showhide
Weakly Supervised Object Localization (WSOL) techniques learn the object location only using image-level labels, without location annotations. A common limitation for these techniques is that they cover only the most discriminative part of the object, not the entire object. To address this problem, we propose an Attention-based Dropout Layer (ADL), which utilizes the self-attention mechanism to process the feature maps of the model. The proposed method is composed of two key components: 1) hiding the most discriminative part from the model for capturing the integral extent of object, and 2) highlighting the informative region for improving the recognition power of the model. Based on extensive experiments, we demonstrate that the proposed method is effective to improve the accuracy of WSOL, achieving a new state-of-the-art localization accuracy in CUB-200-2011 dataset. We also show that the proposed method is much more efficient in terms of both parameter and computation overheads than existing techniques.
1. Introduction
WSOL methods often localize only the most discriminative object part, so ADL hides that region while highlighting informative features to improve complete localization efficiently.
- Problem: WSOL uses image-level labels to localize objects, but often covers only the most discriminative part rather than the object's full extent.CAM-based classifiers focus on features useful for classification, which narrows the spatial response.
- Existing approaches: Existing methods erase discriminative regions to encourage learning less discriminative object parts, but effective removal can require substantial computation.Approaches may use retraining, auxiliary classifiers, or two forward-backward propagations per iteration.
- Proposed method: ADL uses self-attention to remove the target's most discriminative part with a lightweight method.It generates attention from feature maps and uses it to guide region removal.
- Proposed method: ADL combines a drop mask that hides discriminative regions with an importance map that highlights informative regions for classification.The two components are selected stochastically during training.
- Efficiency: ADL requires one forward-backward propagation in one model and no additional trainable parameters for self-attention extraction.It can be applied to convolutional feature maps regardless of model architecture.
- Results: ADL improves accuracy by more than 15 percentage points over existing state-of-the-art techniques on CUB-200-2011 and is comparable on ImageNet-1k.The reported results concern WSOL accuracy on the two datasets.
2. Related Work
Related work applies dropout and attention to convolutional features, while ADL differs by dropping strongly activated spatial regions without adding comparable computational or parameter overhead.
- Dropout: Conventional dropout randomly zeros hidden nodes, but pixel-based dropout is ineffective on convolutional maps because neighboring pixels share contextual information.Spatial correlation leaves redundant information after individual pixels are dropped.
- Dropout: SpatialDropout addresses this issue by randomly dropping feature-map channels instead of individual pixels.It is a channel-based alternative to pixel-level dropout.
- Dropout: ADL instead drops strongly activated spatial regions, combining region-based dropout with attention-guided selection.This distinguishes it from methods that remove entire channels.
- Attention mechanism: Self-attention methods improve CNN classification using channel, spatial, or combined attention maps, but some require large parameter or auxiliary-convolution overheads.RAN uses uncompressed feature maps, while BAM and CBAM use auxiliary convolutional layers for spatial attention.
- Attention mechanism: ADL uses an importance map to reward informative regions while avoiding additional trainable parameters for extracting its self-attention map.Its attention mechanism is designed for lightweight WSOL training.
3. ADL: Attention-based Dropout Layer
ADL converts convolutional feature maps into self-attention-guided masks that alternately hide discriminative regions and reward informative ones during training, improving localization without trainable overhead.
- ADL architecture: ADL is applied to each classification feature map and generates a self-attention map, drop mask, and importance map.The two masks are computed from the same attention representation.
- Dual components: The drop mask hides discriminative parts, whereas the importance map highlights them; stochastic selection lets ADL use both roles during training.The drop mask encourages learning less discriminative object parts, while the importance map supports classification.
- Self-attention map: Channelwise average pooling compresses F into M_att, whose pixel intensity approximates the spatial distribution of discriminative power.The input feature map has height H, width W, and C channels.
- Drop mask: Thresholding M_att produces M_drop, with 0 in the most discriminative region and 1 elsewhere.The dropped-region size increases as γ decreases.
- Efficiency: ADL has no trainable parameters and can be plugged into multiple feature maps without parameter overhead.It avoids auxiliary classifiers, retraining, and additional forward-backward propagation.
- Deployment: ADL is active only during training, so testing uses the vanilla model and standard heatmap extraction methods.The method does not compensate for training-testing distribution differences.
- Attention extraction: The attention map is not refined by auxiliary layers because existing CNN layers are considered sufficient to produce meaningful attention maps.Training gradients update those existing layers through classification loss.
- Comparison: ADL is more efficient than existing methods, although its classification improvement may be smaller than CBAM's while remaining sufficient for the application.The paper reports that its attention map still improves classification and identifies discriminative regions.
4. Experimental Results
Experiments across CUB-200-2011 and ImageNet-1k show that ADL improves WSOL by encouraging coverage beyond the most discriminative object part, while adding minimal overhead. Its effectiveness depends on drop rate, feature-map level, and dataset background, with a trade-off between localization and classification accuracy.
- Datasets: CUB-200-2011 contains 200 bird species and is challenging for WSOL because the most discriminative region can be much smaller than the entire bird.The dataset has 5,994 training images and 5,794 testing images, with smaller intraclass variation than ImageNet-1k.
- Ablation Study: Higher-level feature maps contain class-specific features, and their drop masks erase the most discriminative part more accurately than lower-level masks.Lower-level maps contain class-agnostic general features, whereas pool4 and conv5-3 capture class-specific features.
- Ablation Study: 75% drop rate gives the best localization accuracy, whereas applying the drop mask at every iteration greatly reduces Top-1 Clas and Top-1 Loc accuracy.At 100% drop rate, the model never observes the most discriminative part, reducing classification power and adversely affecting localization.
- Ablation Study: 68.99% classification accuracy at a 25% drop rate decreases to 67.78% with no dropping, which the authors associate with overfitting.The authors state that rigorously analyzing the drop mask’s regularization effect is beyond the paper’s scope.
- Ablation Study: Applying the importance map alone raises classification accuracy but lowers localization accuracy, while combining it with the drop mask produces better localization than either component alone.The drop mask guides the model toward less discriminative parts, whereas the importance map guides it toward the most discriminative part, creating a localization-classification trade-off.
- Comparison with State-of-the-art Methods: ADL has no parameter overhead and nearly zero computation overhead, including 0.003% with ResNet50-SE, while achieving over 15 percentage points of localization improvement on CUB-200-2011.The method also achieves better localization accuracy than existing state-of-the-art techniques with several other backbone networks.
5. Conclusion
ADL induces CNN classifiers to learn the entire object extent while remaining efficient and lightweight, achieving strong localization across datasets and architectures.
- ADL induces CNN classifiers to learn the entire extent of the object.
- ADL is more efficient and lightweight than existing state-of-the-art methods.
- ADL achieves new state-of-the-art accuracy on CUB-200-2011 and comparable accuracy with current state-of-the-art methods on ImageNet-1k.
- ADL can be applied to various CNN classifiers to improve localization accuracy.
- Future work will analyze the drop mask’s regularization effect and address learning less discriminative regions outside the object.