PaperlayerView exact revision on arXiv

Source-linked AI summary

A Simple Semi-Supervised Learning Framework for Object Detection

Kihyuk Sohn, Zizhao Zhang, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister

arXiv:2005.04757v2cs.CV

TL;DR

Semi-supervised object detection needs effective use of unlabeled data because detection annotations are expensive and SSL evidence has focused mainly on image classification. STAC combines high-confidence pseudo labels with strong augmentation-based consistency training, achieving higher performance and approximately 2× label efficiency in the reported low-label settings. Its scope includes MS-COCO and PASCAL VOC, while noisy pseudo labels may be problematic with stronger hard-negative mining.

  • Problem

    Object detection has high labeling costs, while most SSL progress has focused on image classification and existing detection methods use limited additional context.

  • Method

    STAC uses a two-stage framework that generates high-confidence pseudo labels from a detector trained on labeled data and trains with strong augmentation-based consistency.

  • Results

    STAC improves MS-COCO mAP from 18.47 to 24.38 with 5% labeled data and from 23.86 to 28.64 with 10%, while reaching 46.01 mAP and 79.08 AP0.5 on PASCAL VOC.

  • Takeaways & Limitations

    STAC provides a simple, effective object-detection SSL framework with approximately 2× label efficiency in low-label regimes and only two new hyperparameters.

  • Takeaways & Limitations

    Noisy pseudo labels could be overly used with detection frameworks employing stronger hard-negative mining.

Abstract

from arXiv · show

Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data. Although there has been remarkable recent progress, the scope of demonstration in SSL has mainly been on image classification tasks. In this paper, we propose STAC, a simple yet effective SSL framework for visual object detection along with a data augmentation strategy. STAC deploys highly confident pseudo labels of localized objects from an unlabeled image and updates the model by enforcing consistency via strong augmentations. We propose experimental protocols to evaluate the performance of semi-supervised object detection using MS-COCO and show the efficacy of STAC on both MS-COCO and VOC07. On VOC07, STAC improves the AP$^{0.5}$ from $76.30$ to $79.08$; on MS-COCO, STAC demonstrates $2{\times}$ higher data efficiency by achieving 24.38 mAP using only 5\% labeled data than supervised baseline that marks 23.86\% using 10\% labeled data. The code is available at https://github.com/google-research/ssl_detection/.

1. Introduction

STAC extends consistency-based self-training to semi-supervised object detection, addressing the high annotation cost of detection with pseudo labels and strong augmentation. Experiments on MS-COCO and VOC evaluate its efficacy under low-label protocols.

  • Object detection requires stronger label-efficient SSL because its annotations are more expensive than image-classification labels.
  • STAC combines pseudo-label self-training with consistency regularization driven by strong data augmentations in a two-stage training framework.A detector trained on labeled data generates high-confidence localized-object pseudo labels for unlabeled images, which are used during augmented training.
  • STAC generates bounding boxes and class labels with teacher-model inference, filters predictions by confidence after NMS, and enforces consistency under strong augmentations.Target boxes are also transformed when global geometric augmentations are applied.
  • STAC improves MS-COCO mAP from 18.47 to 24.38 under the 5% labeled-data protocol and from 23.86 to 28.64 under the 10% protocol.
  • The paper introduces experimental protocols for semi-supervised object detection using 1%, 2%, 5%, and 10% labeled MS-COCO data, plus additional unlabeled-data settings.It also evaluates VOC07 trainval with VOC12 and optional MS-COCO unlabeled data.
  • STAC is designed to remain simple, adding only the confidence threshold τ and unsupervised loss weight λu as new hyperparameters.The framework is evaluated on MS-COCO and PASCAL VOC with Faster RCNN.

2. Related Work

Prior SSL work emphasizes consistency regularization and pseudo labeling, while object-detection methods use limited consistency transformations or additional contextual information. The paper identifies intensive augmentation as insufficiently studied for semi-supervised detection.

  • Existing semi-supervised object-detection methods include flipped-image consistency, context-aware proposal learning, and prediction ensembling across transformations.
  • The flipped-image consistency method requires a more sophisticated Jensen-Shannon Divergence for consistency-loss computation.
  • Consistency regularization trains models to produce consistent predictions across label-preserving augmentations and has become prominent in image-classification SSL.Examples include Mean-Teacher, UDA, and MixMatch.
  • Data augmentation is important for generalization and SSL, but intensive augmentation has not been thoroughly studied for semi-supervised object detection.

3. Methodology

STAC combines pseudo-label self-training with consistency regularization under strong augmentations for semi-supervised object detection. Its two-stage pipeline trains a teacher on labeled data, generates filtered pseudo labels, augments images and boxes, and jointly optimizes supervised and unsupervised losses.

  • STAC framework: STAC combines self-training through pseudo labels with augmentation-driven consistency regularization for object detection.The framework adapts consistency-based SSL ideas from classification to detection.
  • Two-stage training: The first stage trains a teacher detector on available labeled images and uses it to generate bounding-box and class-label pseudo labels for unlabeled images.Pseudo labels are produced through detector inference, including post-processing such as NMS.
  • Data augmentation: Strong augmentations are applied to unlabeled images, with corresponding box-coordinate transformations for geometric operations.The pipeline applies transformations to both the image and pseudo boxes before computing the unsupervised loss.
  • Optimization: The detector is trained jointly with supervised and unsupervised losses on labeled and augmented unlabeled data.For Faster RCNN, the method uses classifier and region proposal network heads with classification and box-regression components.
  • Pseudo-label generation: Confidence-based filtering after NMS retains high-confidence pseudo boxes to reduce potentially incorrect detections.STAC uses a high threshold inspired by FixMatch to improve pseudo-label precision.
  • Optimization: STAC introduces two hyperparameters, τ and λu, with experiments finding τ = 0.9 and λu ∈[1, 2] effective under a simple constant schedule.The confidence threshold controls pseudo-label selection, while λu weights the unsupervised loss.
  • Data augmentation: Its object-detection augmentation strategy sequences global color transformation, global or box-level geometric transformation, and Cutout.The strategy extends RandAugment-style operations with box-level transformations and applies Cutout at multiple random locations.

4. Experiments

STAC is evaluated for semi-supervised object detection on MS-COCO and PASCAL VOC under low-label protocols, comparing against supervised detectors and augmentation strategies. Results show stronger performance and data efficiency, especially with limited labeled data and additional unlabeled data.

  • Supervised Baselines: 2.71 mAP at 5%, 2.32 mAP at 10%, and 1.85 mAP at 100% are the supervised gains from RandAugment over default augmentation.These comparisons use supervised baselines with resizing and horizontal flipping.
  • MS-COCO Results: 5.91 mAP at 5% and 4.78 mAP at 10% are STAC’s improvements over stronger supervised models.STAC reaches 24.36 mAP with 5% labeled data versus 23.86 for a supervised model using 10%, and 28.56 versus 28.63 at the corresponding 10% versus 20% comparison.
  • MS-COCO Results: 39.21 mAP is achieved by STAC on the 100% MS-COCO protocol, below the 39.48 mAP of supervised training with strong augmentation.STAC nevertheless improves over the default supervised baseline of 37.63 mAP.

5. Ablation Study

The ablation studies examine augmentation, learning schedules, unlabeled-data scale, hyperparameters, and pseudo-label quality. They show that strong augmentation, sufficient unlabeled data, confidence filtering, and balanced loss weighting are important to STAC’s performance.

  • Data Augmentation and Learning Schedule: Stronger augmentation increasingly benefits detectors with small labeled datasets, while quick training schedules help in the low-label regime.With all labeled data, strong augmentation can underfit without longer training; extended training eventually makes it outperform default augmentation.
  • Data Augmentation and Learning Schedule: 39.21 mAP is obtained by STAC with strong augmentation, compared with 37.57 mAP for self-training alone and 37.63 mAP for the supervised baseline on the 100% protocol.On the 10% protocol, the supervised baseline rises from 24.05 to 26.27 with default augmentation and to 29.00 with STAC.
  • Size of Unlabeled Data: Larger unlabeled datasets improve STAC, whereas limited unlabeled data yields smaller gains than strong supervised augmentation.On VOC07, combining VOC12 and MS-COCO reaches 79.08 AP0.5, compared with 77.45 using VOC12 alone and 78.21 for the strong supervised model.
  • Hyperparameters λu and τ: λu = 2 and τ = 0.9 produce the best STAC performance on the single-fold 10% protocol.Performance deteriorates when λu is above 2 or below 0.5; larger τ improves pseudo-label precision but can reduce recall.
  • Hyperparameters λu and τ: Confidence filtering is necessary for substantial gains because unfiltered predicted boxes include many inaccurate pseudo labels.The study finds higher pseudo-label precision preferable to higher recall on the 10% protocol.
  • Quality of Pseudo Labels: Improving pseudo-label quality yields an mAP increase on the 5% protocol but no clear correlation with STAC performance on the 10% protocol.The results suggest that incremental pseudo-label improvements may provide limited additional benefit.

6. Discussion and Conclusion

STAC is presented as a simple, effective SSL framework for object detection, with 2× label efficiency in the low-label regime. Its broader applicability is constrained by confirmation-bias risks with stronger hard-negative mining.

  • 6. Discussion and Conclusion: 2× label efficiency is reported in the low-label regime for STAC, which introduces only two hyperparameters that are easy to tune.The hyperparameters are the confidence threshold τ and unsupervised loss weight λu.
  • 6. Discussion and Conclusion: Confirmation bias could be problematic with detection frameworks using stronger hard-negative mining because noisy pseudo labels may be overly used.The authors identify noisy-label learning, confidence calibration, and uncertainty estimation as directions for improvement.
  • 6. Discussion and Conclusion: The appendix specifies short-edge resizing between 500 and 800 when the long edge remains below 1024.This is an implementation setting for one listed training configuration.
  • 6. Discussion and Conclusion: The Faster RCNN head uses a batch size of 64 per image in the corresponding listed configuration.The passage reports the batch setting without linking it to a performance outcome.
  • 6. Discussion and Conclusion: Another listed configuration fixes the short edge at 800 when the resized long edge is below 1333 and uses 512 samples per image for the Faster RCNN head.These are appendix implementation settings.

A.3. VOC

The VOC configuration uses fixed image resizing and a specified Faster RCNN head batch size; the supplied anchor-size entry is not interpretable beyond its label.

  • A.3. VOC: Images use a fixed short edge of 600, with the longer edge capped at 1000 after resizing.This is the listed data-processing configuration.
  • A.3. VOC: The Faster RCNN head uses a batch size of 256 per image.The passage reports this as a training configuration.
  • A.3. VOC: The configuration lists RPN anchor sizes, but the supplied entry contains no interpretable size values.No concrete anchor-size comparison can be made from the passage.

B. Data Augmentation in STAC

STAC’s strong augmentation is treated as important to its performance, and its loss formulation follows the Noisy Student combination of self-training and strong augmentation.

  • B. Data Augmentation in STAC: Strong data augmentation produces significant mAP gains over Self-Training across all reported cases.Self-Training denotes STAC without strong augmentation on unlabeled data.
  • B. Data Augmentation in STAC: The comparison in Table A1 reports mAP means and standard deviations over five data folds for the 5% and 10% protocols.The table compares STAC with and without strong augmentation.
  • B. Data Augmentation in STAC: The extended formulation uses p(x) instead of p(x; θ) for generality and denotes p(x; θ) as the model prediction during training.This notation clarification concerns the prediction function used in the loss formulation.
  • B. Data Augmentation in STAC: STAC’s unsupervised loss follows Noisy Student and combines Self-Training with strong data augmentation.The formulation can be replaced by more complicated SSL formulations such as Mean Teacher or MixMatch/ReMixMatch.

C.1. Bootstrapping (a.k.a. Self-Training) [61, 33]

The supplied passages frame bootstrapping as a model-based training process, while noting that gradient flow may involve both prediction paths and that ramp schedules depend on the optimizer.

  • C.1. Bootstrapping (a.k.a. Self-Training) [61, 33]: The existing model parameter ˜θ usually refers to a model trained only on labeled data until convergence.This defines the model source used in the bootstrapping formulation.
  • C.1. Bootstrapping (a.k.a. Self-Training) [61, 33]: Gradient flows to both q and p in the entropy-minimization formulation.The passage identifies Entropy Minimization as the only method known there to backpropagate through q.
  • C.1. Bootstrapping (a.k.a. Self-Training) [61, 33]: The formulation omits ramp-up and ramp-down behavior for w(·) because it depends on the optimization framework.The passages point to external references for those schedule details.

C.7. Unsupervised Data Augmentation (UDA) [59]

UDA and FixMatch generate pseudo labels with weak augmentation and train with strong augmentation. Related methods extend consistency training with MixUp, while Noisy Student adds class balancing.

  • UDA uses weak augmentation to generate pseudo labels and strong augmentation for model training.Weak augmentation includes translation and horizontal flip; strong augmentation uses RandAugment followed by Cutout.
  • FixMatch follows the same weak-to-strong augmentation pattern, using RandAugment or CTAugment followed by Cutout.
  • Noisy Student adds data balancing across classes, a design choice not represented in the displayed formulation.
  • MixMatch combines weak augmentation with MixUp for the unsupervised loss.MixUp may combine unlabeled data, or labeled and unlabeled data with ground-truth labels for the labeled examples.
  • ReMixMatch uses MixUp for unsupervised loss together with weak and strong augmentation.Its strong augmentation example is CTAugment.
Loading 2005.04757v2…