PaperlayerView exact revision on arXiv

Source-linked AI summary

Humble Teachers Teach Better Students for Semi-Supervised Object Detection

Yihe Tang, Weifeng Chen, Yijun Luo, Yuting Zhang

arXiv:2106.10456v1cs.CV

TL;DR

Semi-supervised object detection is constrained by scarce and costly bounding-box annotations. Humble Teacher addresses this with an EMA-updated teacher, soft pseudo-labels over region proposals, and detection-specific ensembling, outperforming prior methods across benchmarks and improving over a supervised baseline with similarly sized unlabeled data.

  • Problem

    Object-detection annotations remain scarce and costly, and classification-oriented semi-supervised methods do not directly resolve unknown object locations without bounding boxes.

  • Method

    Humble Teacher uses an EMA-updated teacher, asymmetric augmentation, soft pseudo-labels on region proposals, and a detection-specific data ensemble.

  • Results

    Humble Teacher achieves state-of-the-art performance across multiple benchmarks, surpasses prior work in low- and high-data regimes, and improves over a supervised detector with similarly sized unlabeled data.

  • Takeaways & Limitations

    Soft labels with a balanced number of teacher region proposals are key to the reported performance, while EMA updates and teacher ensembling further improve the detector.

  • Takeaways & Limitations

    The EMA teacher relies on the assumption that its predictions are more accurate than the student's, an assumption evaluated on a 10% labeled MS-COCO setup.

Abstract

from arXiv · show

We propose a semi-supervised approach for contemporary object detectors following the teacher-student dual model framework. Our method is featured with 1) the exponential moving averaging strategy to update the teacher from the student online, 2) using plenty of region proposals and soft pseudo-labels as the student's training targets, and 3) a light-weighted detection-specific data ensemble for the teacher to generate more reliable pseudo-labels. Compared to the recent state-of-the-art -- STAC, which uses hard labels on sparsely selected hard pseudo samples, the teacher in our model exposes richer information to the student with soft-labels on many proposals. Our model achieves COCO-style AP of 53.04% on VOC07 val set, 8.4% better than STAC, when using VOC12 as unlabeled data. On MS-COCO, it outperforms prior work when only a small percentage of data is taken as labeled. It also reaches 53.8% AP on MS-COCO test-dev with 3.1% gain over the fully supervised ResNet-152 Cascaded R-CNN, by tapping into unlabeled data of a similar size to the labeled data.

1. Introduction

Humble Teacher addresses scarce and costly object-detection annotations with a teacher-student framework that uses EMA-updated teachers, soft pseudo-labels, and detection-specific ensembling. It outperforms STAC and narrows the gap with fully supervised detection across VOC and MS-COCO settings.

  • Motivation: Object-detection datasets are comparatively scarce, and acquiring bounding-box annotations is costly.MS-COCO has 118,287 annotated images, compared with over 14 million labeled images in ILSVRC.
  • Motivation: Classification-based semi-supervised methods transfer imperfectly because detection requires learning unknown object existence and locations without bounding-box annotations.The paper reports that direct classification-based pretraining was not effective in its experiments.
  • Approach: Humble Teacher dynamically updates the teacher with EMA, uses asymmetric augmentation, and trains the student on soft pseudo-labels from region proposals.The teacher processes weaker views while the student receives stronger augmentations.
  • Approach: Soft labels on many proposals provide richer teacher information and reduce overfitting to missing or incorrect predictions than hard thresholded labels.The method balances broad image coverage with emphasis on useful foreground instances.
  • Results: Humble Teacher outperforms STAC on MS-COCO and VOC and improves over the supervised ResNet-152 Cascade R-CNN when similarly sized unlabeled data are added.The paper also reports that it closes the semi-supervised versus fully supervised gap on VOC.

2. Related Work

Prior semi-supervised methods use pseudo-labeling or consistency regularization, but Humble Teacher adapts these ideas to object detection through soft labels, teacher-student asymmetry, EMA updates, and detection-specific ensembling. It differs from related approaches in how predictions and region proposals are used.

  • Semi-supervised classification: Pseudo-labeling repeatedly generates labels for unlabeled data and trains on mixtures of pseudo-labeled and human-annotated examples.FixMatch retains only highly confident hard pseudo-labels and uses different augmentations for label creation and training.
  • Semi-supervised classification: Consistency regularization penalizes disagreement between predictions from differently perturbed inputs or between prediction and temporal ensembles.Humble Teacher extends soft labels beyond class probabilities to bounding-box regression offsets for all classes.
  • Semi-supervised classification: Mean Teacher updates a teacher from the student by EMA, whereas Humble Teacher uses the teacher for accurate pseudo-label creation and the student for learning from stronger augmentations.The cited comparison distinguishes pseudo-labeling from applying consistency constraints to two strongly augmented copies.
  • Object detection: STAC combines self-learning and consistency regularization, while Humble Teacher uses a separate teacher-student design and reports better performance.The related-work passage introduces these differences without specifying a quantitative result.
  • Object detection: Humble Teacher averages predictions from original and flipped images to create pseudo-labels, unlike CSD, which uses flips for a consistency loss.CSD and ISD focus on grid-sampled boxes in single-stage detectors, while Humble Teacher applies to proposals in two-stage detectors.

3. Approach

Humble Teacher trains a two-stage detector with supervised and unsupervised branches, where an EMA-updated teacher supplies soft targets from many proposals to a strongly augmented student. A horizontal-flip ensemble further improves the teacher’s ROI pseudo-labels while preserving the detector’s geometry assumptions.

  • Overview: The framework jointly trains on labeled and unlabeled images using supervised and unsupervised branches in a two-stage detector.Mixed batches contain equal numbers of labeled and unlabeled images, and the final loss combines supervised and unsupervised losses.
  • Overview: The teacher predicts pseudo-labels from weakly augmented images, while the student learns from strongly augmented views of the same images.The teacher does not back-propagate gradients; the unsupervised loss compares student predictions with teacher pseudo-labels.
  • Soft Labels and Unsupervised Loss: Soft teacher labels supervise both classification and bounding-box regression, retaining class probabilities and offsets for all possible classes.The method applies unsupervised loss in both the RPN and ROI heads rather than selecting only hard categories and offsets.
  • Soft Labels and Unsupervised Loss: Using the top-N region proposals provides abundant pseudo-label instances spanning objects, overlapping boxes, and background regions.The default number of retained proposals is N = 640, avoiding reliance on only NMS- and threshold-selected hard pseudo-ground truths.
  • Exponential Moving Average for the Teacher Model Update: EMA updates make the teacher less sensitive to occasional wrong pseudo-labels by limiting each student update’s influence on teacher weights.The default decay is α = 0.999, so the teacher changes only slightly at each iteration.
  • Teacher Ensemble with Horizontal Flipping: The teacher ensemble averages predictions from an image and its horizontally flipped version for ROI pseudo-label generation, but not for RPN pseudo-labels.The flipped image reuses horizontally transformed proposals from the original image because paired RPN anchors may not be symmetric.
  • Experiments: On Pascal VOC, the model consistently outperforms competing methods across the reported experiment setups.Table 1 evaluates results on the VOC07 test set and includes a ResNet-50-based CSD re-implementation.

4. Experiments

The experiments evaluate Humble Teacher across Pascal VOC and MS-COCO under varying labeled-data regimes, including settings with similarly sized unlabeled datasets. The method consistently improves over prior approaches and fully supervised baselines, with stronger gains in localization-sensitive AP metrics.

  • Experimental setup: Experiments cover Pascal VOC and MS-COCO, including 1%, 2%, 5%, and 10% labeled-data settings and a roughly 1:1 MS-COCO labeled-to-unlabeled setup.VOC experiments use VOC07 trainval as labeled data and VOC12 trainval, optionally with MS-COCO20, as unlabeled data.
  • Pascal VOC results: Humble Teacher outperforms the best existing results across all Pascal VOC setups, exceeding STAC by 8.4% AP in both reported setups.The VOC comparisons use AP50 and MS-COCO-style AP metrics.
  • Pascal VOC results: The VOC COCO-style mAP gap narrows from 9.65% to 1.25% relative to fully supervised training on VOC07+VOC12.The comparison uses labeled VOC07 with unlabeled VOC12 versus a model fully supervised on VOC07+VOC12.
  • Pascal VOC results: Humble Teacher’s gains over CSD and STAC are larger on 0.5:0.95 AP than on AP50, indicating more accurate object localization.The paper attributes this pattern to soft labels over the full set of region proposals, which provide guidance on regions without definite labels.
  • MS-COCO labeled percentages: On MS-COCO, the method consistently outperforms the best existing approach at every tested labeled-data percentage, without diminishing improvement as labeled data increases.The evaluation reports mAP (50:95) on MS-COCO val 2017 using ResNet-50 models.
  • MS-COCO train plus unlabeled data: With full MS-COCO train supervision plus unlabeled data, the method improves AP by 4.74% for Faster R-CNN, 3.15% for Cascade R-CNN, and 3.1% on MS-COCO test-dev.The fully supervised Faster R-CNN baseline achieves 37.63% AP; the experiments use 123,403 additional unlabeled images.

5. Ablation Study

The ablation study finds that EMA-updated teachers, balanced proposal counts, soft labels, and task-specific teacher ensembles each improve semi-supervised detection performance.

  • 5.1. Number of Proposals for Unsupervised Loss: A balanced number of region proposals is crucial because too few miss objects and context, while too many add distracting background samples.Experiments tested proposal counts up to 6000 under a GPU-memory limit.
  • 5.2. Update Rules: EMA updates at every iteration outperform both copying weights every 10K iterations and keeping the teacher fixed.The authors attribute this partly to averaging weight updates over time and sample batches.
  • 5.3. Soft Labels versus Hard Labels: Soft-label training produces higher and continually increasing teacher mAP, whereas hard-label training shows diminishing mAP during training.Hard labels are generated by thresholding prediction confidence in the 10%-labeled MS-COCO setup.
  • 5.4. Teacher Ensemble: A task-specific ensemble of flipped and original images improves performance by 0.64% AP, while a random augmentation ensemble hurts performance.The comparison uses Faster R-CNN with ResNet-50 trained on 10% labeled MS-COCO train 2017.

6. Conclusions

Humble Teacher achieves state-of-the-art semi-supervised object detection performance across multiple benchmarks. The conclusions identify iteration-wise EMA updates, soft labels with balanced proposals, and carefully constructed teacher ensembles as important design elements.

  • 6. Conclusions: Humble Teacher obtains state-of-the-art performance on multiple benchmarks for semi-supervised object detection.The method is reported to outperform prior work in both low-data and high-data regimes.
  • 6. Conclusions: Iteration-wise EMA teacher updates, soft labels with balanced proposals, and carefully constructed teacher ensembles are reported as effective components.The authors specifically describe soft labels coupled with balanced region proposals as key to superior performance.

Supplementary Materials

The supplementary materials identify the paper, its authors, and its arXiv publication record.

  • Supplementary Materials: The paper is authored by Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang of Amazon Web Services.
  • Supplementary Materials: The paper was posted as arXiv:2106.10456v1 on 19 Jun 2021.

1. Weight of Unsupervised Loss

The unsupervised-loss weight must balance supervised and unsupervised learning. Ablations show that performance deteriorates when the weight is too small or too large, leading the authors to use β = 0.5.

  • 1. Weight of Unsupervised Loss: The final loss combines supervised and unsupervised losses, scaling the latter by β nU/nS.nU and nS are the numbers of unlabeled and labeled images, and β weights the unsupervised loss.
  • 1. Weight of Unsupervised Loss: Performance deteriorates when β is too small or too large, showing that supervised and unsupervised learning require balance.The authors set β = 0.5 across the main-paper experiments without optimizing it for individual experiments.

2. Ablation on Unsupervised Localization

The experiment tests whether applying unsupervised loss to bounding-box regression heads improves the final detector. It compares enabling versus disabling both regression heads while holding other parameters constant.

  • The ablation compares models with and without unsupervised localization loss on both bounding-box regression heads.Other parameters are kept unchanged.
  • The setup uses 640 second-stage region proposals, following the main paper.
  • The experiment evaluates the final model's performance under the two localization-loss settings.

3. Hard Label Experiments

The hard-label experiments examine the pseudo-label loss weight and confidence threshold, identifying settings that optimize hard-label performance. Even after tuning these parameters, the soft-label model remains stronger.

  • Hyper-parameters: The hard-label experiments vary β, the pseudo-label loss weight, and θ, the confidence threshold for accepting hard pseudo-labels.
  • β: Pseudo-Label Loss Weight: β between 0.09 and 0.10 gives the best performance on 10% labeled MS-COCO training data.The main paper uses β = 0.1 for hard-label experiments across data splits.
  • Comparison: The soft-label model outperforms the best hard-label model despite extensive hard-label parameter tuning.
  • θ: Confidence Threshold: The best confidence threshold is between 0.7 and 0.8 for filtering low-quality hard pseudo-labels.The main paper selects θ = 0.7 for the best overall performance among five splits.

4. Augmentation Details

The strong augmentation combines a randomly selected color transformation with Cutout. Color changes include blur, sharpening, noise, inversion, brightness shifts, pixel scaling, and contrast changes.

  • Strong augmentation applies one randomly selected color transformation followed by Cutout.
  • Color transformations include identity, Gaussian blur, average blur, sharpening with alpha blending, Gaussian noise, inversion, and pixel-value shifts.
  • Pixel values may be multiplied by a sample from (0.5, 1.5), applied to 50% of pixels per channel.
  • Contrast is multiplied by a random per-channel factor from (0.5, 2).
  • Cutout fills a random α from (1, 5) of square patches, each sized 0 or 0.2 of the input image height.A zero-size patch is canceled.
Loading 2106.10456v1…