PaperlayerView exact revision on arXiv

Source-linked AI summary

Few-Example Object Detection with Model Communication

Xuanyi Dong, Liang Zheng, Fan Ma, Yi Yang, Deyu Meng

arXiv:1706.08249v8cs.CV

TL;DR

Few-example object detection asks how to train detectors from only a few bounding-box labels per category while exploiting many unlabeled images. The paper alternates detector training with high-confidence sample selection, using multiple detection models to improve generated-sample precision and recall. With very limited supervision, MSPLD achieves competitive performance against state-of-the-art weakly supervised approaches that use more supervised knowledge.

  • Problem

    Few-example object detection must generate many trustworthy training samples from a large unlabeled image pool despite having only a few bounding-box examples per category.

  • Method

    MSPLD alternates detector improvement with reliable sample selection, progressing from easy to harder samples and jointly optimizing multiple detection models.

  • Results

    MSPLD achieves competitive performance against state-of-the-art weakly supervised object-detection approaches using more supervised knowledge.

  • Takeaways & Limitations

    Using only a few bounding-box labels per category, the framework can obtain competitive object-detection performance under extremely limited supervision.

  • Takeaways & Limitations

    The method still requires about 1% of the dataset's images to be annotated, with future work targeting one image per class.

Abstract

from arXiv · show

In this paper, we study object detection using a large pool of unlabeled images and only a few labeled images per category, named "few-example object detection". The key challenge consists in generating trustworthy training samples as many as possible from the pool. Using few training examples as seeds, our method iterates between model training and high-confidence sample selection. In training, easy samples are generated first and, then the poorly initialized model undergoes improvement. As the model becomes more discriminative, challenging but reliable samples are selected. After that, another round of model improvement takes place. To further improve the precision and recall of the generated training samples, we embed multiple detection models in our framework, which has proven to outperform the single model baseline and the model ensemble method. Experiments on PASCAL VOC'07, MS COCO'14, and ILSVRC'13 indicate that by using as few as three or four samples selected for each category, our method produces very competitive results when compared to the state-of-the-art weakly-supervised approaches using a large number of image-level labels.

1 INTRODUCTION

Few-example object detection uses only several annotated bounding boxes per class, making reliable label propagation and detector training difficult. The paper addresses this challenge by alternating detector improvement with reliable sample generation and using multiple detectors to balance precision and recall.

  • Problem: Few-example object detection uses only 3-4 annotated bounding-box examples per class, unlike approaches assuming substantially more annotations.The task is challenging because scarce labels make label propagation and model training difficult.
  • Motivation: FEOD reduces labeling effort, supports rare and stationary objects, and can provide accurate annotations for crowded objects.The paper reports 40.1% mAP for person versus 24.7% for the best weakly supervised algorithm.
  • Approach: Self-paced learning selects high-confidence easy samples first, while multi-modal learning incorporates multiple detection models during training.The integrated scheme is designed to avoid noisy instances and improve generated-sample quality.
  • Approach: The method alternates detector improvement with reliable sample generation to gradually obtain a stable, robust detector.Pseudo boxes are generated from easy to harder samples as the detector improves.
  • Approach: Multiple detection models are embedded in a unified learning scheme to ameliorate the precision-recall trade-off in training-sample generation.The models exchange complementary knowledge during joint optimization.
  • Results: The algorithm produces competitive accuracy against state-of-the-art weakly supervised object-detection methods requiring much more labeling effort.The paper frames this as competitive performance under substantially reduced annotation requirements.

2 RELATED WORK

CNN-based object detectors are commonly organized into proposal-based and proposal-free methods. Proposal-based systems generate candidate object regions before detection, whereas proposal-free systems directly predict bounding boxes.

  • Detection paradigms: CNN object-detection methods are divided into proposal-based and proposal-free categories.The distinction concerns whether methods generate region proposals before predicting objects.
  • Proposal-based methods: Proposal-based methods evolved from R-CNN through SPP-Net and Fast R-CNN, with Faster R-CNN using a region proposal network.The cited progression emphasizes improvements in accuracy, speed, and proposal recall.
  • Proposal-free methods: Proposal-free methods directly predict bounding boxes without generating region proposals.YOLO is given as an example of this approach.

2.2 Semi-supervised Object Detection

Semi-supervised object detection commonly combines image-level labels with some bounding-box labels or auxiliary video and tracking information. The paper distinguishes FEOD by requiring far fewer annotations and no extra supervised auxiliary knowledge.

  • Supervision: Current semi-supervised object-detection methods usually use both image-level labels and some bounding-box labels.Some approaches also exploit video-specific features or large auxiliary datasets for model initialization.
  • Auxiliary supervision: Related methods can use around 350k images with bounding-box annotations to initialize detection models on PASCAL VOC.They may additionally use negative datasets and further labeled image resources.
  • Comparison: Other methods mine instances by fusing detection and tracking information, whereas FEOD does not require extra supervised auxiliary knowledge.The comparison highlights FEOD's extremely low annotation requirement.

2.3 Weakly Supervised Object Detection

Weakly supervised object detection trains detectors from image-level labels rather than bounding-box annotations. Existing approaches either use pretrained CNNs or design architectures that derive object information from classification signals.

  • Setting: Weakly supervised object detection uses each image's image-level label to train object detectors.This supervision differs from direct bounding-box annotation.
  • Approaches: Some weakly supervised methods employ off-the-shelf CNN models.These methods use existing CNN representations within the detection pipeline.
  • Approaches: Other methods design CNN architectures that extract object information from classification loss and use the resulting classifier to derive detectors.Bilen et al. is cited as an example using selective search proposals and image-level classification.

2.4 Object Detection from Few Examples

Few-example learning uses a small number of labeled examples together with unlabeled data, progressively improving an initial model. It differs from few-shot learning because few-shot learning does not use unlabeled data.

  • Few-example learning progressively improves an initial model learned from few labeled data by using unlabeled data.
  • Few-shot learning learns a model from a few training examples without unlabeled data.
  • Compared with earlier methods using 10-100 training samples per class, this approach requires 2-4 examples per class without extra training datasets.

2.5 Webly Supervised Learning for Object Detection

Webly supervised approaches use web data to reduce annotation cost, but may require extensive collection and training resources and still perform poorly in some settings.

  • Web data can reduce annotation cost for object detection.
  • Chen et al. initialize CNN models from easy samples before adapting them to more realistic images.
  • Divvala et al. learn models across many variations through webly supervised learning, but require substantial collection and training time.

2.6 Zero-shot Object Detection

Zero-shot object detection locates instances from novel categories without training examples. This paper instead extracts detectors from extremely few training samples for each object class.

  • Zero-shot object detection locates object instances belonging to novel categories without any training examples.
  • Prior ZSD work models interactions between visual and semantic domain information jointly.
  • This paper focuses on extracting detectors from extremely few training samples rather than detecting new object categories without examples.

2.7 Model Ensemble

Model ensemble methods combine detectors or knowledge from multiple models or tasks. Existing examples include part-detector composition, low-rank knowledge fusion, and verification-classification fusion.

  • Ensemble methods combine multiple detectors or model outputs to construct object detectors.
  • Dai et al. ensemble part detectors into sub-structure detectors that constitute the final object detector.
  • Other approaches ensemble knowledge from different tasks or fuse verification and classification models.

2.8 Progressive Paradigm

The method uses a progressive strategy that organizes examples from easy to difficult while iteratively improving multiple detection models. This progressive paradigm supports repeated model updates as increasingly challenging samples become reliable.

  • The method adapts a progressive strategy to iteratively optimize multiple detection models.
  • Examples are organized in a meaningful order, with training samples selected according to their difficulty.
  • Iterative label generation and detector updating yield reliable pseudo boxes from easy to hard samples while improving detector robustness.

3 THE PROPOSED METHOD

MSPLD integrates self-paced learning with multiple complementary detectors to generate and refine pseudo-labeled training data from unlabeled images. It alternates pseudo-label generation, sample selection, detector retraining, and regularization updates until convergence.

  • 3.2 The MSPLD Model: The method uses multiple detection models whose fused outputs provide pseudo boxes, while selection variables indicate which images and classes train each detector.Fast R-CNN and R-FCN serve as basic detectors with complementary architectures; pseudo boxes and binary selection variables are updated during training.
  • 3.2 The MSPLD Model: The objective uses λ for self-paced selection and γ for multi-modal regularization, with pairwise selection-weight regularization encouraging detectors to share pseudo-annotation confidence.The shared-confidence assumption raises an image’s selection threshold for a detector when another detector has selected it.
  • 3.3 Optimization: Alternative optimization cyclically updates pseudo boxes, selection variables, and detector parameters using initial annotations plus selected pseudo-labeled images.The process repeats across detectors and iterations, updating λ and γ to select more images in later rounds.
  • 3.4 Algorithm Description: The alternating procedure converges after unlabeled samples are traversed and the objective can no longer be minimized, while multi-modal learning improves the precision–recall trade-off over iterations.The reported training behavior shows recall increasing as precision decreases, while detection mAP increases and stabilizes.

4 EXPERIMENTAL EVALUATION

Experiments evaluate MSPLD across multiple detection benchmarks, model configurations, initialization sizes, supervision settings, and noisy-image conditions. The method achieves competitive or improved performance with few annotations, while pseudo-box coverage remains limited in complex images.

  • Datasets: MSPLD is evaluated on PASCAL VOC 2007, PASCAL VOC 2012, MS COCO 2014, and ILSVRC 2013 detection benchmarks.The experiments use standard training, validation, and test splits where available.
  • Benchmark comparisons: 41.7% mAP is achieved on PASCAL VOC 2007 using 60 annotated images, with less than 1% of training images annotated.This performance is reported as competitive with state-of-the-art weakly supervised object detection methods using more image-level labels.
  • Model comparisons: MSPLD outperforms a model ensemble by about 4% in mAP and 3% in CorLoc.The reported comparison attributes the improvement to the proposed multi-model learning strategy, while different architectures also complement one another under SPL.
  • Benchmark comparisons: 65.5% average CorLoc is achieved on the PASCAL VOC 2007 trainval set, while mAP reaches 41.7% versus 34.5% for.The method is also reported to maintain similar performance across different initializations of fully annotated images.
  • Ablation studies: Combining selective search and edge boxes slightly improves performance, whereas edge boxes alone is inferior because of poorer first-iteration initialization.The proposal-generation comparison tests SS, EB, and their combination.
  • Ablation studies: R-R50 benefits from data selection and prior knowledge, while R-R50 and F-VGG16 are complementary under multi-modal learning.OHEM slightly improves R-R50 but harms F-VGG16 and R-R101, potentially because the latter models are large for the relatively small training set.
  • Ablation studies: Using k = 2 initialization is unstable, while adding one example per class can significantly improve MSPLD performance.Using 100 randomly selected initialization images approaches fully supervised accuracy; the paper selects 3–4 images per class for low manual cost.
  • Ablation studies: Filtering pseudo boxes with image-level labels greatly boosts the framework under the simplest form of image-level supervision.The method uses image labels only to remove incorrect pseudo boxes.

5 CONCLUSION AND FUTURE WORK

MSPLD alternates detector improvement with reliable sample selection and uses multiple detection models under extremely limited supervision. The paper reports competitive performance against state-of-the-art weakly supervised approaches, while future work targets lower annotation requirements and novel-class adaptation.

  • MSPLD alternates detector amelioration and reliable sample selection using only a few bounding box labels per category.
  • Multiple detection models are embedded to enhance detector learning under scarce annotation.
  • MSPLD achieves competitive performance against state-of-the-art weakly supervised approaches despite using extremely limited supervision information.
  • The method still requires about 1% of the dataset images to be annotated, motivating future work toward one image per class.
  • Future work includes accommodating novel classes without destroying the accuracy of models trained on previously learned classes.
Loading 1706.08249v8…