Source-linked AI summary
Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection
Zeyi Huang, Yang Zou, Vijayakumar Bhagavatula, Dong Huang
TL;DR
WSOD relies on image-level category labels, which do not ensure spatially consistent localization across transformed images. CASD aggregates attention across transformations and feature layers and self-distills it within the network; it achieves state-of-the-art results on standard WSOD benchmarks.
Problem
WSOD trains detectors from image-level category labels, but those labels do not enforce consistent object localization across image transformations.
Source: §1 · pp. 1–2Method
CASD aggregates attention from multiple transformations and feature layers, then self-distills that comprehensive attention within the WSOD network.
Source: §1 · p. 2Results
CASD achieves state-of-the-art WSOD results on PASCAL VOC 2007/2012 and MS-COCO, outperforming other methods by clear margins.
Source: §1 · p. 3Takeaways & Limitations
CASD provides instance-balanced and spatially consistent supervision for robust WSOD bounding-box localization.
Source: §1 · p. 3
Abstract
from arXiv · showhide
Weakly Supervised Object Detection (WSOD) has emerged as an effective tool to train object detectors using only the image-level category labels. However, without object-level labels, WSOD detectors are prone to detect bounding boxes on salient objects, clustered objects and discriminative object parts. Moreover, the image-level category labels do not enforce consistent object detection across different transformations of the same images. To address the above issues, we propose a Comprehensive Attention Self-Distillation (CASD) training approach for WSOD. To balance feature learning among all object instances, CASD computes the comprehensive attention aggregated from multiple transformations and feature layers of the same images. To enforce consistent spatial supervision on objects, CASD conducts self-distillation on the WSOD networks, such that the comprehensive attention is approximated simultaneously by multiple transformations and feature layers of the same images. CASD produces new state-of-the-art WSOD results on standard benchmarks such as PASCAL VOC 2007/2012 and MS-COCO.
1 Introduction
WSOD trains object detectors from image-level labels, but localization can favor salient objects, clusters, or object parts and vary across transformations. CASD aggregates attention across image transformations and feature layers, then self-distills it to encourage balanced, consistent localization; it achieves state-of-the-art results on standard benchmarks.
1 Introduction
- WSOD uses image-level category labels rather than precise box annotations, which are labor-intensive to obtain at scale; its detectors remain behind fully supervised detectors.Many prior WSOD methods use multiple instance learning (MIL).Source: §1 · p. 1
- WSOD can miss inconspicuous instances, merge adjacent objects, focus on object parts, or detect different objects across transformations of the same image.The paper relates these localization issues to biased features and to image-level labels that do not enforce spatial consistency across transformed views.Source: §1 · p. 2
- CASD aggregates attention across transformations and feature layers, then self-distills it within the WSOD network to encourage instance-balanced, spatially consistent supervision.The approach is intended to support robust bounding-box localization.Source: §1 · pp. 2–3
- CASD achieves state-of-the-art WSOD results on PASCAL VOC 2007/2012 and MS-COCO, outperforming other methods by clear margins.The authors also report systematic ablations of transformations and feature layers.Source: §1 · p. 3
2 Related Work
The related work spans MIL-based WSOD, attention regularization, and knowledge distillation. CASD differs by regularizing WSOD feature learning with comprehensive attention that targets both spatial consistency and completeness.
2.1 Weakly Supervised Object Detection
- MIL-based CNN methods include WSDDN [5], OICR [12], and PCL [7]; unlike approaches that regularize WSOD outputs, CASD directly regularizes feature learning.Other cited WSOD approaches use smoothed MIL losses, self-training, segmentation-derived proposals, or detection-segmentation collaboration.Source: §2.1 · p. 3
2.2 Attention Mechanism in Computer Vision DNNs
- Prior attention methods reweight features or regularize attention-map consistency [20] [21] [22] [24] [25]; CASD is presented as the first attention-regularization approach for WSOD.CASD targets both consistent and spatially complete features, including localization of less-salient objects.Source: §2.2 · p. 3
2.3 Knowledge Distillation
- Unlike distillation from a separate teacher, CASD transfers comprehensive attention knowledge within the WSOD model across multiple views of the same data.The passage describes distillation targets including predictive probabilities, intermediate features, and attention maps.Source: §2.3 · p. 3
3 Method
CASD extends OICR with comprehensive attention self-distillation to encourage balanced and consistent WSOD representations. It aggregates attention across transformed inputs and network layers, then trains the network to approximate that attention.
3.1 Background
- OICR extracts proposal features, scores them through an image-level MIL head, and refines localization through sequential instance-classifier branches.At inference, scores from all refinement classifiers are summed to predict boxes with NMS.Source: §3.1 · pp. 3–4, Figure 2 · p. 4
- OICR's proposal scores can favor the most discriminative instances or object parts, while CASD also aggregates proposal scores across transformed views during training.The overall CASD-based training objective combines losses from the OICR framework and CASD.Source: §3.1 · p. 4, §3.2 · p. 6
3.2 Comprehensive Attention Distillation
- CASD combines attention across image transformations and feature layers, then self-distills the comprehensive maps to encourage balanced and consistent representations.Input-wise CASD uses transformed versions of the same image; layer-wise CASD aggregates across network layers.Source: §3.2 · pp. 5–6
- Input-wise CASD forms proposal attention maps from original, flipped, and scaled images and trains each variant to approximate comprehensive attention.The maps can focus on different parts of a proposal, while their union covers more comprehensive object regions.Source: §3.2 · p. 5
- Inverted Attention independently masks highly attended features in each transformation to reduce domination by one attention map and encourage other features to activate.The paper reports that this regularization produces better results in its ablation study.Source: §3.2 · p. 6
- Layer-wise CASD aggregates attention from proposal feature maps at multiple network blocks, combining early local features with deeper global semantics.RoI pooling makes proposal feature maps from different blocks share the same spatial size.Source: §3.2 · p. 6
4 Experimental Results
On VOC 2007/2012 and MS-COCO, CASD outperforms prior state-of-the-art WSOD methods. Ablations assess its input- and layer-wise attention, other components, and experimental settings.
4 Experimental Results
- Experiments use image-level labels on VOC 2007/2012 and MS-COCO, reporting VOC mAP0.5 and COCO mAP0.5 plus mAP averaged over IoU thresholds.VOC evaluation follows training on train+validation and testing on the test set; COCO uses train for training and validation for testing.Source: §4 · p. 6
4.1 Ablation Studies
- Combining input-wise and layer-wise CASD reaches 55.3% mAP0.5, 6.4 points above the 48.9% baseline.Input-wise CASD alone reaches 54.1% and layer-wise CASD 52.3%; the table also reports gains from proposal score aggregation and inverted attention.Source: §4.1 · p. 7, Table 1 · p. 8
- Adding the regression branch raises +IW+LW to 56.1% mAP0.5, while stronger flip+scale+color augmentation yields 56.8%.The regression branch improves the combined configuration by 0.8 points; the stronger augmentation setting is the best CASD result reported in this ablation.Source: §4.1 · p. 7
- Layer-wise CASD performs best with VGG16 blocks B4+B3+B2, reaching 52.3% mAP0.5.The authors report that B2 and B3 provide more spatially distributed discriminative clues than B1, whose low-level attention can add noise; other experiments use B4+B3+B2.Source: §4.1 · p. 7, Table 2 · p. 8
- IW-CASD outperforms prediction consistency and attention consistency by 2.4 and 1.6 mAP0.5 points, respectively.On VOC 2007, CASD reports mAP values of 54.1%, 54.7%, 55.3%, 55.0%, and 55.0% for γ values 0.05, 0.075, 0.1, 0.15, and 0.2, respectively; the authors describe it as robust to γ.Source: §4.1 · p. 8, Table 3 · p. 8
4.2 Comparison with the State-of-the-arts
- CASD beats MIST(+Reg.) [8] by 1.9/1.5 mAP0.5 points on VOC 2007/2012 and 1.4/2.1 mAP/mAP0.5 points on COCO with VGG16.On COCO with ResNet50, CASD reaches 13.9% mAP and 27.8% mAP0.5, exceeding MIST(+Reg.) by 1.3% mAP and 1.7% mAP0.5.Source: §4.2 · p. 8, Table 4 · p. 8, Table 5 · p. 8
5 Conclusion
CASD regularizes WSOD by aggregating attention across image transformations and feature layers, then using self-distillation to enforce instance-balanced and spatially consistent supervision. The paper reports state-of-the-art WSOD results and suggests the training module may extend to other weakly and semi-supervised tasks.
5 Conclusion
- CASD aggregates attention from multiple image transformations and low-to-high feature layers as spatial supervision within the WSOD network.The paper describes these sources as “free” resources of spatial supervision.Source: §5 · p. 8
- Self-distillation uses the comprehensive attention to enforce instance-balanced and spatially consistent supervision over objects.Source: §5 · p. 8
- The authors report that CASD achieves new state-of-the-art WSOD results.Source: §5 · p. 8
- The authors suggest CASD may generalize to weakly and semi-supervised tasks such as instance segmentation and pose estimation.Source: §5 · p. 8
Broader Impact
The paper frames its broader impact as reducing the burden of human annotation and potentially lowering data-training costs. It also presents CASD as a general regularization approach that may benefit several learning settings.
Broader Impact
- The authors present CASD as a general regularization approach that may benefit semi-supervised, weakly supervised, and self-supervised representation learning.Source: Broader Impact · p. 9
- The paper's research vision is to potentially relieve the burden of human annotations on training data.Source: Broader Impact · p. 9
- The authors suggest this effort may reduce training-data costs, balance human bias, and accelerate machine-perception development.Source: Broader Impact · p. 9
A.1 Visualization
The visualizations show CASD successes and failures, and compare its detections with MIST. The reported pattern is stronger localization for CASD, with failures concentrated on small or heavily occluded objects.
A.1 Visualization
- Figure 4 overlays attention maps for high-confidence WSOD proposals on input images; green boxes denote ground truth.The figure presents success cases above and failure cases below.Source: §A.1 · p. 12, Figure 4 · p. 12
- CASD tends to succeed on relatively large, fully visible objects, while failures mainly involve small or heavily occluded objects.The authors suggest poor Selective Search proposals and incomplete appearance as possible contributors to failures on occluded objects.Source: §A.1 · p. 12
- CASD has better success than MIST in detecting high-quality bounding boxes, with comprehensive attention shown alongside detections.Source: §A.1 · p. 12, Figure 5 · p. 13
A.2 CorLoc on Trainval Sets
The appendix reports CorLoc as an additional localization metric on the VOC 2007 and VOC 2012 trainval sets. CASD has the best overall localization accuracy among the compared methods.
A.2 CorLoc on Trainval Sets
- CorLoc evaluates detector localization accuracy, and prior work generally reported it after training and testing on the trainval sets.Source: §A.2 · p. 13
- The paper describes CorLoc as mostly an ablation metric rather than a main algorithm-comparison metric.Source: §A.2 · p. 13
- CASD has the best overall localization accuracy among compared methods on the VOC 2007 and VOC 2012 trainval sets.Source: §A.2 · p. 13, Table 6 · p. 13
A.3 More Ablation Studies and Clarification
The ablations support using a single shared loss weight as a good trade-off between performance and tuning effort. They also report similar performance on flipped and unflipped tests, compare attention-map choices, and note a GPU constraint on branch count.
A.3 More Ablation Studies and Clarification
- A single γ is a good trade-off: separate loss weights reached 55.5% mAP0.5, only 0.2% above the 55.3% mAP0.5 best result with one γ.The separate-weight sweeps vary γIW and γLW independently.Source: §A.3 · p. 14
- 56.5% mAP0.5 on the horizontally flipped VOC 2007 test set was similar to 56.8% on the unflipped set, indicating consistency with flipping.The paper also reports qualitative comparisons in Figs. 1 and 5 as evidence of more consistent and complete object features.Source: §A.3 · p. 14
- 52.0% mAP0.5 with Grad-CAM was slightly below 52.6% with channel-wise average pooling plus sigmoid, which the authors adopted for computational efficiency.The comparison is an ablation of layer-wise CASD on VOC 2007.Source: §A.3 · p. 14
- All experiments use k = 2 OICR branches because of GPU limitations; the authors suggest k = 3 may improve results with sufficient GPU memory.Vanilla OICR uses three branches and suggests larger k can improve results.Source: §A.3 · p. 14