Source-linked AI summary
Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals
Minsu Cho, Suha Kwak, Cordelia Schmid, Jean Ponce
TL;DR
The paper asks how dominant objects can be discovered and localized without annotations when image collections contain multiple classes and outliers. It matches proposed object and part regions across images with probabilistic Hough matching and selects salient regions for localization. Experiments show robust discovery in mixed-class settings and strong performance against colocalization methods, while the method remains subject to practical and probabilistic limitations described by the authors.
Problem
The central gap is object discovery and localization without image-level annotations, a single dominant class assumption, or a known number of classes in noisy multi-class collections.
Method
The method matches many bottom-up region proposals for objects and parts across images using probabilistic Hough scores that combine appearance and spatial consistency, then uses standout scores for foreground localization.
Results
The approach markedly outperforms state-of-the-art colocalization methods on standard benchmarks and demonstrates unsupervised discovery in challenging mixed-class data.
Takeaways & Limitations
Part-based matching is a crucial factor for object discovery in challenging mixed-class unsupervised localization.
Takeaways & Limitations
Using more neighboring images does not always improve performance while increasing computation time.
Abstract
from arXiv · showhide
This paper addresses unsupervised discovery and localization of dominant objects from a noisy image collection with multiple object classes. The setting of this problem is fully unsupervised, without even image-level annotations or any assumption of a single dominant class. This is far more general than typical colocalization, cosegmentation, or weakly-supervised localization tasks. We tackle the discovery and localization problem using a part-based region matching approach: We use off-the-shelf region proposals to form a set of candidate bounding boxes for objects and object parts. These regions are efficiently matched across images using a probabilistic Hough transform that evaluates the confidence for each candidate correspondence considering both appearance and spatial consistency. Dominant objects are discovered and localized by comparing the scores of candidate regions and selecting those that stand out over other regions containing them. Extensive experimental evaluations on standard benchmarks demonstrate that the proposed approach significantly outperforms the current state of the art in colocalization, and achieves robust object discovery in challenging mixed-class datasets.
1. Introduction
The paper targets fully unsupervised object localization in collections containing multiple dominant classes, outliers, and no image-level annotations. It proposes matching object and part regions across images, then selecting salient regions for localization.
- Problem: Unlike weakly supervised localization, the setting assumes neither target-class labels nor a single dominant class in the image collection.
- Problem: Fully unsupervised localization must handle multiple dominant object classes and outlier images without image-level or box annotations.
- Contributions and results: The approach markedly outperforms state-of-the-art colocalization methods on standard benchmarks despite requiring no annotations or known number of classes.
- Approach: The method uses bottom-up multi-scale region proposals as candidate regions for both whole objects and their parts.
- Approach: Probabilistic Hough matching scores candidate correspondences using appearance and spatial consistency across images.
- Approach: A standout score selects salient regions containing potential objects, addressing the mismatch between highly confident parts and complete-object localization.
2. Related work
Prior unsupervised discovery methods generally worked in restricted settings, while related localization methods introduced image-level labels or assumed a single dominant class. This paper operates in a more general setting and uses many region proposals without objectness priors.
- Unsupervised discovery: Earlier unsupervised discovery methods were demonstrated mainly with a few distinctive object classes and struggled with in-the-wild localization.
- Related localization tasks: Cosegmentation segments common foreground regions, whereas colocalization localizes common objects with bounding boxes across image collections.
- Weak supervision: Weakly supervised localization uses image-level labels indicating whether a target object class appears in each image.
- Region proposals: Most related region-proposal methods use fewer than 100 top proposals per image to form whole-object hypotheses, often with generic objectness measures.
- Region proposals: This paper instead uses typically 1000 to 4000 region proposals per image as primitive matching elements without objectness priors.
3. Proposed approach
The approach combines probabilistic part-based region matching with unsupervised foreground localization to discover dominant objects in noisy, mixed-class image collections. Candidate regions are matched using appearance and spatial consistency, aggregated across neighbor images, and selected using standout scores.
- Part-based region matching: Off-the-shelf multi-scale region proposals provide candidate regions for both whole objects and object parts.The method uses many proposals as primitive matching elements rather than relying on objectness priors.
- Part-based region matching: Probabilistic Hough Matching combines appearance similarity with spatial consistency to assign confidence scores to candidate region correspondences.A Hough-space score estimates the geometry prior by aggregating votes from possible region matches.
- Part-based region matching: Region confidence is max-pooled over matches, so high-confidence regions indicate correspondences supported by at least one good match.The resulting confidence measure functions as a signal for common regions across images.
- Foreground localization: Multiple source images contribute aggregated region confidences, with more images potentially improving the confidence estimates.Neighbor images are defined as sources in which an object from the target image may appear.
- Foreground localization: The standout score compares a region's confidence with the maximum confidence of larger regions containing it, favoring potential whole-object boundaries over salient parts.Containment uses an area-ratio criterion and an overlap criterion; top standout regions are selected for localization.
- Overall algorithm: A coordinate descent-style discovery algorithm alternates neighbor retrieval, part-based matching, and re-localization of potential object regions.It initializes each image with the entire image region, retrieves 10 neighbors, and updates neighbors and regions over iterations.
4. Experimental evaluation
The experiments evaluate fully unsupervised discovery in separate-class and mixed-class settings on Object Discovery and PASCAL VOC 2007. Results show strong localization, robustness to noisy mixed-class collections, and substantial gains over prior colocalization methods.
- Experimental setup: The evaluation covers separate-class and fully unsupervised mixed-class experiments on the Object Discovery and PASCAL VOC 2007 benchmarks.The mixed-class setting combines images from multiple classes and includes noisy images without target objects.
- Evaluation metrics: CorRet measures the percentage of retrieved nearest neighbors belonging to the same ground-truth class, while CorLoc measures localization under the PASCAL overlap criterion.CorRet uses k = 10 neighbors, and CorLoc requires area(bp∩bgt) / area(bp∪bgt) > 0.5.
- Object Discovery results: The Object Discovery algorithm rapidly improves CorLoc and CorRet, reaches a steady state after roughly 4–5 iterations, and outperforms prior cosegmentation and colocalization methods.In the mixed-class setup, localization is almost the same as in the separate-class setup despite 36 outlier images.
- PASCAL07-6x2 results: Removing multiple object regions, probabilistic Hough matching, or the standout score substantially damages PASCAL07-6x2 performance.The ablations identify part-based matching and standout-score foreground localization as crucial components for robust discovery.
- PASCAL07-6x2 results: On PASCAL07-6x2, the method significantly outperforms previous results without objectness priors or negative training images.The comparison uses CorLoc and includes the mixed-class setting alongside separate-class evaluation.
- PASCAL07-all results: On PASCAL07-all, the method performs very close to the best weakly-supervised localization method that does not use additional supervised data.The benchmark contains realistic images with clutter, occlusion, and diverse viewpoints, making it more challenging than Object Discovery.
- Mixed-class analysis: In mixed-class PASCAL experiments, similar classes can assist discovery, but lower-frequency classes are harder to localize under class imbalance.Average CorLoc drops 1.2% for the highest-frequency classes versus 9.4% for the lowest-frequency classes; CorRet is 30.7% versus 11.0%, respectively.
5. Discussion and conclusion
The paper demonstrates unsupervised object localization in a challenging mixed-class setting. It identifies part-based matching as crucial for object discovery while avoiding additional supervision.
- The method demonstrates unsupervised object localization in a challenging mixed-class setup not previously fully attempted on a challenging dataset.
- Part-based matching is identified as a crucial factor for object discovery.
- The evaluation abstains from additional supervision, including saliency or objectness measures, negative data, and pretrained features.
- Future work includes handling multiple object instances per image and building visual models for classification and detection.