Source-linked AI summary
Weakly Supervised Cascaded Convolutional Networks
Ali Diba, Vivek Sharma, Ali Pazandeh, Hamed Pirsiavash, Luc Van Gool
TL;DR
Weakly supervised object detection seeks to replace bounding-box annotations with image-level labels, enabling broader dataset annotation. This paper proposes two end-to-end cascaded CNNs that combine class-specific proposals, optional weak segmentation, and multiple instance learning, reporting state-of-the-art weakly supervised detection and classification with improved localization.
Problem
Weakly supervised object detection aims to detect objects using only image-level labels, because conventional methods rely on bounding boxes and labels for each object.
Method
The paper trains two- and three-stage cascaded CNNs end-to-end, combining class-specific proposal extraction, weakly supervised segmentation in the three-stage model, and multiple instance learning.
Results
The experiments achieve state-of-the-art weakly supervised object detection and classification, while the best PASCAL VOC 2007 CorLoc performance is 56.7% using the three-stage VGG-16 cascade.
Takeaways & Limitations
Jointly learning localization, multiple instance learning, and weakly supervised segmentation provides a single weak-supervision pipeline for improving object detection and localization.
Takeaways & Limitations
The experiments fix the hyperparameter balancing the two loss functions at 1, although cross-validation may improve results.
Abstract
from arXiv · showhide
Object detection is a challenging task in visual understanding domain, and even more so if the supervision is to be weak. Recently, few efforts to handle the task without expensive human annotations is established by promising deep neural network. A new architecture of cascaded networks is proposed to learn a convolutional neural network (CNN) under such conditions. We introduce two such architectures, with either two cascade stages or three which are trained in an end-to-end pipeline. The first stage of both architectures extracts best candidate of class specific region proposals by training a fully convolutional network. In the case of the three stage architecture, the middle stage provides object segmentation, using the output of the activation maps of first stage. The final stage of both architectures is a part of a convolutional neural network that performs multiple instance learning on proposals extracted in the previous stage(s). Our experiments on the PASCAL VOC 2007, 2010, 2012 and large scale object datasets, ILSVRC 2013, 2014 datasets show improvements in the areas of weakly-supervised object detection, classification and localization.
1. Introduction
The paper targets weakly supervised object detection using only image-level object labels, replacing costly bounding-box annotations with cascaded CNNs. Its two- and three-stage architectures jointly improve proposal generation and detection through end-to-end training.
- Image-level labels can train object detection without specifying object counts or bounding boxes, enabling annotation or collection of much larger datasets.
- A common weakly supervised pipeline generates proposals, extracts proposal features, and applies multiple instance learning to infer box labels from image labels.
- The two-stage cascade uses a fully convolutional network with global pooling for class-specific proposals, followed by ROI pooling and multiple instance learning.
- End-to-end cascaded training links proposal generation, feature extraction, and multiple instance learning, allowing less noisy proposals to support improved detection.
- The three-stage cascade adds class-specific segmentation between proposal generation and detection to produce more reliable object proposals.
2. Related works
Prior weakly supervised detection methods commonly treat images as bags of regions and use multiple instance learning to select object instances from image-level labels. Related work spans latent-region regularization, proposal-generation strategies, and CNN-based localization and classification.
- Weakly supervised detection: Multiple instance learning methods model each image as a bag of regions, assuming positive images contain at least one instance of the target category.
- Weakly supervised detection: Earlier approaches regularize latent object locations or enforce similarity between object windows to improve instance selection.
- Weakly supervised detection: Many previous methods train detectors from large collections of noisy object proposals, whereas the proposed method focuses on a smaller set of proposals.
- Object proposal generation: Object proposal research includes dense regions, saliency-based proposals, generic objectness measures, and selective search.
- CNN based weakly supervised object detection: CNN-based weakly supervised methods have combined feature discrimination with localization and classification in learned CNN architectures.
3. Proposed Method
WCCN jointly learns classification, localization, and multiple-instance learning under image-level supervision through two- and three-stage cascaded CNN architectures. The three-stage variant adds weakly supervised segmentation to refine object regions before MIL selects representative boxes.
- WCCN jointly learns classification, localization, and multiple instance learning using only image-level labels.
- Two-stage Cascade: The two-stage cascade uses a fully convolutional location network to generate class-specific candidate boxes, followed by ROI-pooled feature extraction and MIL.The stages are trained together in an end-to-end pipeline.
- Two-stage Cascade: The location stage produces class activation maps through global average or maximum pooling and uses independent binary logistic losses for each class.
- Three-stage Cascade: The three-stage cascade inserts a fully convolutional segmentation stage between localization and MIL to provide more accurate object regions.Its segmentation supervision uses pseudo ground truth derived from the first stage rather than human annotations.
- Three-stage Cascade: All three losses in the three-stage cascade are added, and the network parameters are learned jointly through end-to-end training.
- Deployment: The trained cascade can either directly detect and localize objects or supply pseudo ground-truth boxes to supervised detectors such as R-CNN or Fast R-CNN.At test time, EdgeBoxes proposals are used, with non-maximum suppression also applied.
4. Experiments
The experiments evaluate the cascaded CNNs across standard weakly supervised detection, classification, and localization benchmarks. Results report strong performance across PASCAL VOC and ILSVRC, with segmentation improving the cascade and generated proposals supporting supervised detection.
- Metrics: The evaluation uses average precision for detection and classification, and CorLoc for correct localization.For detection, AP counts a box as correct when IoU exceeds 50%; CorLoc measures positive images with at least one correctly localized instance.
- Implementation: The two- and three-stage cascades are evaluated with AlexNet and VGG-16 backbones pretrained on ImageNet.The first stage uses fully convolutional proposal extraction, while later stages include segmentation and multiple instance learning depending on the architecture.
- Detection performance: The method achieves the best reported weakly supervised detection performance among the compared recent methods on PASCAL VOC 2007, 2010, and 2012.The reported comparison uses average precision on the corresponding test sets and emphasizes end-to-end training without sophisticated clustering or optimized initialization.
- Localization performance: 56.7% CorLoc is achieved by the three-stage cascade with VGG-16 on PASCAL VOC 2007.The AlexNet network also outperformed methods using similar architectures and non-deep-learning methods in the reported comparison.
- Detection performance: Fast R-CNN trained with the cascaded networks’ pseudo-ground-truth boxes performs 0.3% better in average precision than the full detection pipeline on PASCAL VOC 2007.This comparison uses pseudo-ground-truth bounding boxes generated by the cascaded networks.
- Object proposals: EdgeBox yields approximately 1.5% improvement over SelectiveSearch for the two-stage AlexNet cascade on the VOC 2007 detection test set.The passage notes that EdgeBox performs better with CNN-based object detectors in this comparison.
- Cascade architecture study: Adding segmentation improves the AlexNet cascade by approximately 2.5% on detection and 2% on classification in PASCAL VOC 2007.The segmentation stage is reported to exploit better regions and produce more accurate initial object locations.
5. Conclusion
WCCN uses cascaded CNNs for weakly supervised visual learning, combining localization, multiple instance learning, and weakly supervised segmentation across multiple stages. Experiments report state-of-the-art performance in weakly supervised object detection and classification across several PASCAL VOC and ILSVRC datasets.
- WCCN applies cascaded CNNs to weakly supervised object detection, localization, and classification.
- The proposed two multi-stage networks use different loss functions across stages to form an end-to-end weakly supervised learning pipeline.
- The approach combines localization, multiple instance learning, and weakly supervised object-region segmentation as a multi-task learning strategy.
- Experiments on PASCAL VOC 2007, 2010, 2012 and ILSVRC 2013, 2014 report state-of-the-art weakly supervised object detection and classification performance.