Source-linked AI summary
Self-produced Guidance for Weakly-supervised Object Localization
Xiaolin Zhang, Yunchao Wei, Guoliang Kang, Yi Yang, Thomas Huang
TL;DR
Weakly supervised localization maps often highlight only small, discriminative object regions. SPG generates foreground-background guidance masks stagewise and uses them as auxiliary pixel-level supervision, achieving a 43.83% Top-1 localization error rate on ILSVRC with image-level supervision.
Problem
CAM-based localization maps highlight only the most discriminative regions, so they locate only a small part of the target objects.
Method
SPG stagewise generates foreground and background guidance masks from high-confidence attention-map regions and uses them as auxiliary supervision for classification networks.
Results
43.83% Top-1 localization error rate on ILSVRC with only image-level supervision, reported as a new state-of-the-art result.
Takeaways & Limitations
SPG produces higher-quality guidance and attention maps that help networks detect more object regions for localization.
Abstract
from arXiv · showhide
Weakly supervised methods usually generate localization results based on attention maps produced by classification networks. However, the attention maps exhibit the most discriminative parts of the object which are small and sparse. We propose to generate Self-produced Guidance (SPG) masks which separate the foreground, the object of interest, from the background to provide the classification networks with spatial correlation information of pixels. A stagewise approach is proposed to incorporate high confident object regions to learn the SPG masks. The high confident regions within attention maps are utilized to progressively learn the SPG masks. The masks are then used as an auxiliary pixel-level supervision to facilitate the training of classification networks. Extensive experiments on ILSVRC demonstrate that SPG is effective in producing high-quality object localizations maps. Particularly, the proposed SPG achieves the Top-1 localization error rate of 43.83% on the ILSVRC validation set, which is a new state-of-the-art error rate.
1 Introduction
The paper addresses weakly supervised object localization, where classification-based attention maps highlight only small discriminative object parts. It proposes stagewise Self-produced Guidance masks that use confident foreground and background regions to provide pixel-level spatial supervision.
- Weakly supervised learning reduces reliance on expensive pixel-level annotations, and this paper focuses on weakly supervised object localization.
- CAM-based localization maps typically highlight only the most discriminative regions, leaving much of the target object unlocalized.
- Attention maps provide reliable foreground and background seeds that can guide learning of broader object patterns, while medium-confidence regions remain undefined.
- SPG generates stagewise foreground/background guidance masks from attention maps, treating highly confident regions as foreground, low-confidence regions as background, and medium-confidence regions as undefined.
- The method uses self-produced guidance as auxiliary supervision so classification networks learn pixel correlations and produce better localization maps.
2 Related Work
Related work contrasts fully supervised detection and localization systems with weaker approaches that use cheaper image-level annotations. Existing weakly supervised methods include proposal-based, CAM-based, and top-down response-map techniques, but some require extra computation or remain limited by image-level supervision.
- Fully supervised object detectors can be effective but require large numbers of annotations that may be unaffordable under limited budgets.
- Weakly supervised methods reduce annotation requirements by using image-level labels, including self-taught frameworks that progressively refine selected high-response regions.
- Proposal-based weakly supervised approaches may introduce extra computation through preprocessed region proposals and multiple networks.
- CAM aggregates top-level feature maps with class-specific layers to produce class localization maps, while related work also uses alternative backpropagation schemes for contrastive response maps.
3 Self-produced Guidance
SPG combines classification-based attention maps with progressively learned foreground/background guidance masks. These masks provide auxiliary pixel-level supervision so the network can learn pixel correlations and improve object localization maps.
- Network Overview: SPG uses Stem, SPG-A, SPG-B, and SPG-C to extract features, classify images, learn guidance masks, and reinforce attention maps.Stem produces position-aware feature maps; SPG-A performs classification and generates class-specific attention maps, while SPG-C applies auxiliary SPG supervision.
- Self-produced Guidance Learning: High-confidence attention-map regions become foreground or background seeds, while medium-confidence regions remain undefined.The guidance mask assigns 1 to object regions, 0 to background regions, and 255 to ignored regions; thresholds δl and δh identify these regions.
- Self-produced Guidance Learning: SPG guidance serves as auxiliary pixel-level supervision, encouraging the classification network to learn correlations among pixels.The entire network is trained end-to-end using cross-entropy losses for classification and self-produced guidance learning.
- Inference and Evaluation: During testing, the highest-scoring class determines the attention map, which is resized and thresholded to produce object bounding boxes.The bounding-box thresholds are adjusted using grid search for the evaluation procedure.
4 Experiments
Experiments evaluate SPG on ILSVRC and CUB-200-2011, comparing localization errors, visual outputs, and component effects. SPG improves localization by learning guidance maps from confident regions and using them as auxiliary supervision.
- Localization results: SPG achieves Top-1 51.40 and Top-5 40.00 localization error on ILSVRC, improving over SPG-plain’s Top-1 53.71 and Top-5 41.81.On CUB-200-2011, SPG achieves Top-1 53.36% localization error and outperforms ACoL.
- Localization results: 43.83% Top-1 and 29.36% Top-5 localization error are obtained on ILSVRC when SPG uses ensemble DPN classification results.The ensemble DPN classifier has Top-1 classification error 15.47% and Top-5 classification error 2.70%.
- Qualitative results: SPG generates attention maps that highlight nearly entire objects and produce precise bounding boxes on ILSVRC and CUB-200-2011.Figure 3 uses green predicted boxes and red ground-truth boxes.
- Guidance-map generation: Confident localization regions become foreground and background seeds, while medium-confidence regions remain undefined before guidance maps are learned.The learned maps then provide auxiliary supervision to improve localization maps.
- Ablation studies: With ground-truth labels, SPG-plain has 37.32% Top-1 error, reduced to 35.31% with auxiliary supervision.Removing SPG-C worsens the Top-1 error to 36.06%, while removing shared layers increases it from 35.31% to 36.31%.
- Ablation studies: The two-stage SPG-B structure produces better self-produced guidance maps and more effective attention maps.The second and third layers are shared between B1 and B2 in the reported configuration.
5 Conclusions
The conclusion presents SPG as a weakly supervised approach that uses image-level labels to generate guidance maps and improve object localization. Experiments report higher-quality localization and performance beyond state-of-the-art methods.
- 5 Conclusions: SPG locates target object regions using only image-level labels and generates guidance maps that encourage classification networks to learn pixel-level correlations.The approach thereby enables networks to detect more object regions for localization.
- 5 Conclusions: Extensive experiments show that SPG detects more object regions and outperforms state-of-the-art localization methods.