Source-linked AI summary
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
TL;DR
Weakly-supervised object detection seeks complete-object localization from image-level labels, but CNN confidence can favor discriminative parts. WSOD2 distills bottom-up boundary evidence into CNNs through adaptive objectness targets and bounding-box regression. The paper reports effective quantitative and qualitative results, while identifying dense scenes and incomplete person detections as remaining challenges.
Problem
CNN-based WSOD can distinguish discriminative local features but struggles to determine whether a proposal contains a complete object without bounding-box supervision.
Method
WSOD2 combines bottom-up object evidence with top-down CNN confidence, bounding-box regression, and adaptive training to distill boundary information into CNN objectness representations.
Results
WSOD2 demonstrates effectiveness quantitatively and qualitatively across standard weakly-supervised object-detection datasets and settings.
Takeaways & Limitations
Bottom-up object evidence can guide WSOD toward object boundaries while complementing discriminative CNN features.
Takeaways & Limitations
WSOD2 still faces challenges in dense scenarios and often detects human faces rather than complete persons.
Abstract
from arXiv · showhide
We study on weakly-supervised object detection (WSOD) which plays a vital role in relieving human involvement from object-level annotations. Predominant works integrate region proposal mechanisms with convolutional neural networks (CNN). Although CNN is proficient in extracting discriminative local features, grand challenges still exist to measure the likelihood of a bounding box containing a complete object (i.e., "objectness"). In this paper, we propose a novel WSOD framework with Objectness Distillation (i.e., WSOD^2) by designing a tailored training mechanism for weakly-supervised object detection. Multiple regression targets are specifically determined by jointly considering bottom-up (BU) and top-down (TD) objectness from low-level measurement and CNN confidences with an adaptive linear combination. As bounding box regression can facilitate a region proposal learning to approach its regression target with high objectness during training, deep objectness representation learned from bottom-up evidences can be gradually distilled into CNN by optimization. We explore different adaptive training curves for BU/TD objectness, and show that the proposed WSOD^2 can achieve state-of-the-art results.
1. Introduction
Weakly-supervised object detection reduces annotation requirements but struggles to distinguish complete objects from discriminative parts. WSOD2 combines bottom-up object evidence with top-down CNN confidence and adaptive training to improve objectness learning.
- Weakly-supervised object detection uses image-level annotations to reduce the costly need for manually annotated bounding boxes.
- Multiple-instance learning, OICR, and PCL transform image-level supervision into instance-level supervision using classifiers and pseudo ground truths.
- CNNs learn discriminative local features but perform poorly at judging whether a proposal contains a complete object without bounding-box supervision.
- WSOD2 combines bottom-up evidences such as color contrast and superpixel straddling with CNN representation to capture object boundaries and objectness.
- The framework jointly uses bottom-up evidence and top-down confidence to assign pseudo labels and regression targets, then gradually distills boundary knowledge through adaptive training.
- The authors report effectiveness on PASCAL VOC 2007/2012 and MS COCO datasets.
2. Related Work
Prior WSOD methods rely heavily on multiple-instance learning, pseudo ground-truth mining, and classification confidence, while bounding-box regression remains less common because bounding-box supervision is unavailable. WSOD2 integrates regression and bottom-up evidence to improve proposal supervision without requiring additional counting annotations.
- Weakly-supervised Object Detection: Most WSOD methods formulate detection as multi-label classification and mine pseudo ground truths from classification confidence or hand-crafted rules.
- Weakly-supervised Object Detection: OICR and PCL refine instance classifiers online using outputs from earlier detectors as pseudo ground truths.
- Bounding Box Regression: Bounding-box regression reduces localization errors in fully supervised detection but is less common in WSOD because bounding-box annotations are unavailable.
- Bounding Box Regression: WSOD2 integrates a bounding-box regressor and assigns regression targets using bottom-up object evidence.
3. Approach
WSOD² combines bottom-up boundary evidence with top-down CNN confidence to learn objectness while refining proposals through classification and bounding box regression. An adaptive distillation strategy shifts guidance from bottom-up evidence toward learned top-down representations.
- 3.1. Based Multiple Instance Detector: The base detector uses two softmax streams to combine class and proposal predictions into region scores and image-level class scores.Region features are processed by separate classification and detection streams, whose element-wise product is aggregated over proposals.
- 3.2. Bottom-up and Top-Down Objectness: WSOD² ranks proposals using objectness that jointly reflects bottom-up boundary evidence and top-down class confidence.Bottom-up evidence captures generic object boundaries, while top-down confidence comes from the preceding classifier branch.
- 3.2. Bottom-up and Top-Down Objectness: Each refinement classifier receives dynamically selected pseudo labels and objectness-based loss weights, with later classifiers supervised by earlier branches.The framework builds K instance classifiers and uses their outputs to provide supervision for subsequent classifiers without instance-level ground truth.
- 3.3. Bounding Box Regression: Bounding box regression updates proposal locations and sizes so positive regions approach nearby boxes with higher objectness.The regressor is trained with pseudo regression references, and updated boxes can become more complete and precise regression targets.
- 3.4. Objectness Distillation: Objectness distillation initially emphasizes bottom-up evidence, then adaptively transfers boundary knowledge into CNN top-down classifiers.The adaptive factor α balances the two sources, reflecting the lower reliability of top-down classifiers at the beginning of training.
4. Experiments
Experiments evaluate WSOD2 through component ablations, weight-decay studies, and comparisons across PASCAL VOC and MS COCO. Results show benefits from bottom-up evidences, adaptive decay, bounding-box regression, and the complete framework, while dense scenes and human bodies remain challenging.
- Experimental Setup: WSOD2 is evaluated on PASCAL VOC 2007/2012 and MS COCO using image-level labels for training and detection metrics for assessment.PASCAL VOC uses mAP on test splits and CorLoc on trainval splits.
- Ablation Study: Bottom-up evidences improve WSOD2 over the baseline, although individual evidences favor different object categories.The study tests multi-scale saliency, color contrast, edge density, superpixels straddling, and their average combination.
- Ablation Study: Linear decay, α = −(n/N) + 1, performs best among the tested weight-decay designs, and is used in later experiments.The study compares constant, polynomial, and cosine decay functions; n and N denote the current and total training steps.
- Ablation Study: The bounding-box regressor contributes at least 2.6 mAP, NMS contributes 0.8 mAP, and bottom-up evidences plus α decay each contribute 2.2 mAP.These gains are reported in the component ablation study on PASCAL VOC 2007.
- Comparisons with State-of-the-Arts: WSOD2 achieves 53.6 mAP on PASCAL VOC 2007, 47.2 mAP on PASCAL VOC 2012, and outperforms compared MS COCO works by at least 2 AP.Training on the combined VOC 2007+2012 trainval splits gives WSOD2* a 56.1 mAP score.
- Visualization and Case Study: WSOD2 can handle multiple discrete instances, but dense scenarios and complete-person detection remain challenging.For person images, detectors often focus on faces while missing other body parts.
5. Conclusion
WSOD2 combines bottom-up object evidence with top-down CNN classification to improve CNN’s deep objectness representation in weakly-supervised object detection. It also integrates bounding box regression and progressive instance classification end-to-end, with experiments analyzing effectiveness and remaining challenges.
- 5. Conclusion: WSOD2 uses bottom-up evidence to distill boundary features into CNN objectness representations through adaptive training.The method targets CNN’s difficulty in determining whether a proposal contains a complete object without bounding-box supervision.
- 5. Conclusion: The framework integrates bounding box regression and progressive instance classification in an end-to-end training strategy.
- 5. Conclusion: Experiments on standard WSOD datasets and settings demonstrate WSOD2’s effectiveness quantitatively and qualitatively.
- 5. Conclusion: Dense scenarios and incomplete localization of person instances remain challenging for WSOD2.The paper identifies missed body parts when detectors focus on human faces and suggests human structure priors as a possible future direction.