Source-linked AI summary
Zigzag Learning for Weakly Supervised Object Detection
Xiaopeng Zhang, Jiashi Feng, Hongkai Xiong, Qi Tian
TL;DR
Weakly supervised object detection must identify objects from image-level labels despite uncertain locations and complex backgrounds. This paper combines easy-to-difficult training based on mEAS with feature masking, achieving 47.6% mAP on PASCAL VOC 2007 and surpassing the state of the art.
Problem
With image-level labels alone, weakly supervised detection must disentangle object instances from complex backgrounds despite missing locations and a risk of local-minimum solutions.
Source: §1 · p. 1Method
The method progressively recruits easy-to-hard samples using mEAS and masks discriminative regions in convolutional feature maps during training.
Source: §1 · p. 2Results
47.6% mAP on PASCAL VOC 2007 surpasses the state of the art under the weakly supervised paradigm.
Source: §1 · p. 2Takeaways & Limitations
The paper reports that combining progressive learning and masking benefits object-instance mining while reducing model-overfitting risks.
Source: §1 · p. 2Performance remains limited on categories such as chair and person because the classification model is limited in correctly localizing them, especially in cluttered backgrounds.
Source: §4.3 · p. 8
Abstract
from arXiv · showhide
This paper addresses weakly supervised object detection with only image-level supervision at training stage. Previous approaches train detection models with entire images all at once, making the models prone to being trapped in sub-optimums due to the introduced false positive examples. Unlike them, we propose a zigzag learning strategy to simultaneously discover reliable object instances and prevent the model from overfitting initial seeds. Towards this goal, we first develop a criterion named mean Energy Accumulation Scores (mEAS) to automatically measure and rank localization difficulty of an image containing the target object, and accordingly learn the detector progressively by feeding examples with increasing difficulty. In this way, the model can be well prepared by training on easy examples for learning from more difficult ones and thus gain a stronger detection ability more efficiently. Furthermore, we introduce a novel masking regularization strategy over the high level convolutional feature maps to avoid overfitting initial samples. These two modules formulate a zigzag learning process, where progressive learning endeavors to discover reliable object instances, and masking regularization increases the difficulty of finding object instances properly. We achieve 47.6% mAP on PASCAL VOC 2007, surpassing the state-of-the-arts by a large margin.
1. Introduction
Weakly supervised object detection uses image-level labels, but must disentangle objects from complex backgrounds despite uncertain object locations. The paper proposes combining easy-to-difficult learning with feature masking to mine object instances while reducing overfitting.
- The method targets detection training with only image-level labels, which are easier to acquire than object bounding boxes.Fully supervised detection requires large quantities of high-quality object-level annotations.Source: §1 · p. 1
- The central challenge is disentangling object instances from complex backgrounds when missing locations are latent variables and optimization can get stuck in local minima.Without accurate detections, instances cannot be discovered; without appropriate examples, an accurate detector cannot be learned.Source: §1 · p. 1
- mEAS ranks images by localization difficulty so training can progressively add samples from easy to difficult.The paper argues that learning from easier images first improves models and increases the probability of localizing objects in harder images.Source: §1 · pp. 1–2
- Feature masking randomly erases discriminative regions to direct the network toward less discriminative parts and reduce overfitting to initial samples.The resulting random occluded samples also serve as data augmentation and enhance generalization.Source: §1 · p. 2
- 47.6% mAP on PASCAL VOC 2007 surpasses the state of the art under the weakly supervised paradigm.The paper attributes the result to integrating progressive learning and masking regularization.Source: §1 · p. 2
2. Related Works
Related work covers image-difficulty evaluation and weakly supervised detection. The paper contrasts its network-based difficulty criterion and unified mining-and-training approach with prior cues and iterative methods.
- Prior image-difficulty work used low-level cues or object size, with the latter requiring extra size annotations.The paper presents mEAS as network-based and free of human interpretation.Source: §2 · p. 2
- Related weakly supervised detectors iteratively updated object instances, but some separated feature extraction from training,, while another did not consider overfitting.The paper instead integrates model training and object mining in a unified framework.Source: §2 · p. 2
- Oquab et al. returned object center points rather than tight boxes, while Bilen et al. used region-score classification that could focus on object parts.Source: §2 · p. 2
3. Method
The method estimates localization difficulty from region scores, then progressively trains and relocalizes on easy-to-difficult images while masking discriminative features. Together, these steps aim to mine more reliable object instances and reduce overfitting to initial seeds.
3. Method
- The method combines automatic difficulty estimation, progressive sample addition, and masking of high-responsive convolutional features to regularize training.For masking, a region within each mined object is selected according to ratio τ and its corresponding feature-map values are set to zero.Source: §3 · pp. 2–3, Figure 2 · p. 3, §3.3 · p. 5
3.1. Estimating Image Difficulty
- mEAS averages energy-accumulation scores at eleven energy levels to rank images by localization difficulty; its class-level scores correlate 0.703 with CorLoc.Higher mEAS indicates that fewer regions are needed to reach a target energy; high-scoring images tend to have concentrated heat maps, while low-scoring images tend to have divergent ones.Source: §3.1 · p. 4, Table 1 · p. 4
3.2. Progressive Detection Network
- Training proceeds through easy-to-difficult folds, using each trained model to relocalize instances in the next fold as more images are added.The section also describes weighting the next-step loss by the reliability of mined instances.Source: §3.2 · p. 5
3.3. Convolutional Feature Masking Regularization
- Figure 4 presents detection performance on the PASCAL VOC 2007 test split, comparing different learning folds K in the left panel and masking ratios τ in the right panel.Source: Figure 4 · p. 6
4. Experiments
The experiments evaluate zigzag learning on PASCAL VOC 2007 and 2012, using component and parameter ablations before comparison with state-of-the-art methods. The final model reports higher detection performance than the cited best results on both test splits, while error analysis identifies persistent localization challenges.
4.1. Experimental Setup
- Experiments used VOC 2007 and 2012, measuring CorLoc on trainval and average precision on test; model S was CaffeNet and model L was VGG-VD.Training and test used single-scale images, with approximately 2,000 Edge Boxes proposals per image.Source: §4.1 · p. 6
4.2. Ablation Studies
- 40.7% test accuracy with masking was 1.6% above baseline; mEAS improved on region scores by up to 3.2%, and weighted loss added 1.4%.The authors attribute mEAS's gain to avoiding initial selections focused on object parts, and weighted loss to accounting for mined-instance confidence.Source: §4.2 · p. 6
- With K = 3, CorLoc fell from 72.3% on the easiest fold to 44.3% on the most difficult, supporting mEAS as a localization-difficulty criterion.Progressive learning reached 39.1% detection performance, versus 35.9% for training on all images at once.Source: §4.2 · p. 6
4.3. Comparisons with state-of-the-arts
- 47.6% mAP on VOC 2007 beat (41.7%); on VOC 2012, 42.9% exceeded (38.3%).These comparisons use the final model; the VOC 2012 localization result was comparable to (61.5% vs. 62.1%).Source: Table 5 · p. 7, Table 7 · p. 7, §4.3 · p. 8
4.2. Ablation Studies
- At masking ratio τ = 0.1, test performance was 40.7%, 1.6% above the unmasked baseline; performance dropped as more regions were masked.The authors attribute the decline to masking too many regions and reducing exposure to true positive samples.Source: §4.2 · p. 8
4.3. Comparisons with state-of-the-arts
- Animals and vehicles achieved average precision above 60%, but furniture detection was weaker; about 30% of errors across categories came from inaccurate localization.The authors link furniture difficulty to cluttered chair and table scenes; failed detections often localized object parts or grouped same-class objects.Source: Figure 5 · p. 7, §4.3 · p. 8, Figure 6 · p. 8
5. Conclusion
The paper concludes that zigzag learning combines mEAS-based progressive sample recruitment with masking of high-response feature maps. Experiments on PASCAL VOC benchmarks demonstrate the approach’s effectiveness.
5. Conclusion
- Zigzag learning progressively recruits training samples using mEAS to measure image difficulty.The method automatically measures image difficulty and adds samples during training.Source: §5 · p. 8
- Randomly erasing high responses on the last convolutional feature maps highlights less discriminative parts and improves network generalization.Source: §5 · p. 8
- Experiments on PASCAL VOC benchmarks demonstrate the effectiveness of the proposed approach.Source: §5 · p. 8