PaperlayerView exact revision on arXiv

Source-linked AI summary

Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, Hao Li

arXiv:2103.11402v1cs.CVcs.AI

TL;DR

Object detection depends on laborious precise annotations, whereas SSOD seeks to use unlabeled data effectively. The paper introduces Instant-Teaching with instant pseudo labeling and extended weak-strong augmentations, then adds co-rectification for confirmation bias. The framework achieves state-of-the-art performance across MS-COCO and PASCAL VOC, including 4.2 mAP improvement on MS-COCO with 2% labeled data.

  • Problem

    Object detection requires abundant accurate manual annotations, while semi-supervised object detection has comparatively limited prior literature and needs effective use of unlabeled data.

  • Method

    Instant-Teaching performs instant pseudo labeling with extended weak-strong augmentations each iteration, while Instant-Teaching* adds co-rectification using two models to address confirmation bias.

  • Results

    4.2 mAP improvement over STAC, from 18.25 to 22.45, is achieved on MS-COCO under the 2% protocol, while the method outperforms state-of-the-art methods across reported protocols and datasets.

  • Takeaways & Limitations

    Instant-Teaching* is presented as a general SSOD framework that can leverage unlabeled data and is not restricted to object detection models.

  • Takeaways & Limitations

    The method uses a constant λu; dynamically adjusting λu is left for future work because early pseudo-annotation quality and quantity are low.

Abstract

from arXiv · show

Supervised learning based object detection frameworks demand plenty of laborious manual annotations, which may not be practical in real applications. Semi-supervised object detection (SSOD) can effectively leverage unlabeled data to improve the model performance, which is of great significance for the application of object detection models. In this paper, we revisit SSOD and propose Instant-Teaching, a completely end-to-end and effective SSOD framework, which uses instant pseudo labeling with extended weak-strong data augmentations for teaching during each training iteration. To alleviate the confirmation bias problem and improve the quality of pseudo annotations, we further propose a co-rectify scheme based on Instant-Teaching, denoted as Instant-Teaching$^*$. Extensive experiments on both MS-COCO and PASCAL VOC datasets substantiate the superiority of our framework. Specifically, our method surpasses state-of-the-art methods by 4.2 mAP on MS-COCO when using $2\%$ labeled data. Even with full supervised information of MS-COCO, the proposed method still outperforms state-of-the-art methods by about 1.0 mAP. On PASCAL VOC, we can achieve more than 5 mAP improvement by applying VOC07 as labeled data and VOC12 as unlabeled data.

1. Introduction

Object detection often requires costly precise annotations, motivating SSOD methods that exploit unlabeled data. Instant-Teaching provides an end-to-end framework with instant pseudo labeling, extended augmentations, and co-rectification, achieving state-of-the-art results.

  • Accurate class labels and precise bounding boxes make the large annotated datasets needed for object detection costly and impractical in some applications.
  • Existing SSOD methods such as STAC require a separate teacher-training stage and fixed pre-generated pseudo annotations that are not updated during training.
  • Instant-Teaching generates pseudo annotations from weakly augmented unlabeled images and immediately uses them as targets for strongly augmented versions during each iteration.
  • Instant-Teaching* adds co-rectification, in which two independently initialized models help rectify each other’s false predictions to alleviate confirmation bias.
  • Instant-Teaching* outperforms state-of-the-art methods across the reported PASCAL VOC and MS-COCO experimental protocols.

2. Related Work

Object detection research includes two-stage and single-stage detectors, but both traditionally rely on expensive accurate annotations. This work combines pseudo annotations, augmentations, and co-rectification to improve SSL for detection more simply and efficiently.

  • Two-stage detectors generate regions of interest with a Region Proposal Network before classification and bounding-box regression.
  • Single-stage detectors offer a separate fast-detector line, while both detector families traditionally train on large amounts of expensive, time-consuming annotations.
  • The proposed framework trains object detectors on both labeled and unlabeled data rather than only labeled data.
  • The approach combines pseudo annotations, data augmentations, and co-rectification to improve SSL for object detection more efficiently and simply.

3. Method

Instant-Teaching* is an end-to-end SSOD framework built around instant pseudo labeling, weak-strong augmentation, and co-rectification. It jointly learns from labeled and unlabeled data while addressing pseudo-label quality and confirmation bias.

  • Problem definition: SSOD trains object detectors on both labeled and unlabeled data, reducing reliance on precise manual annotations.The task uses labeled images with class labels and bounding boxes alongside unlabeled images.
  • Co-rectify: Co-rectify trains two same-structure models simultaneously so they can rectify false predictions and alleviate confirmation bias.During inference, only Model-a is used, so training two models does not increase inference time.
  • Instant pseudo labeling: Instant-Teaching generates unlabeled-image pseudo annotations with the current model and uses them immediately for strongly augmented training in the same iteration.This procedure makes pseudo-label generation and model training end-to-end, allowing pseudo annotations to improve as the model converges.
  • Training objective: The framework jointly minimizes supervised and unsupervised losses, with λu balancing the two objectives.The supervised loss combines classification cross-entropy and L1 bounding-box regression, while the unsupervised loss uses pseudo labels from weakly augmented data on strongly augmented images.
  • Weak-strong data augmentations: Weak-strong augmentation enforces consistent predictions between weakly and strongly augmented unlabeled images, encouraging learning from self-generated pseudo annotations.The framework extends strong augmentations with Mixup and Mosaic while keeping labeled data weakly augmented for fair comparison with STAC.
  • Weak-strong data augmentations: Mixup combines an unlabeled image and pseudo annotations with a labeled image using a Beta-distributed mixing coefficient, while Mosaic mixes corresponding images and annotations horizontally or vertically.These augmentations are applied to unlabeled data to improve robustness to pseudo-annotation noise and alleviate overfitting.

4. Experiments

Experiments evaluate Instant-Teaching* on MS-COCO and PASCAL VOC using established semi-supervised object detection protocols and mAP-based comparisons. The framework consistently improves over STAC, with gains particularly evident under limited labels and stricter box-overlap evaluation.

  • MS-COCO evaluation: MS-COCO experiments report mAP over 80 object categories under protocols using 1%, 2%, 5%, 10%, and 100% labeled data.The implementation follows STAC’s experimental settings for fair comparison.
  • PASCAL VOC evaluation: PASCAL VOC experiments report mAP over 20 categories, using VOC07 trainval as labeled data and VOC12 plus selected MS-COCO images as unlabeled data.Performance is evaluated on VOC07 test at IoU=0.5, IoU=0.75, and IoU=0.5:0.95.
  • MS-COCO results: 4.2 mAP improvement is achieved over STAC on MS-COCO with 2% labeled data, increasing mAP from 18.25 to 22.45.At 1% labeled data, mAP rises from 13.97 to 18.05, while improvements also appear under 5%, 10%, and 100% protocols.
  • MS-COCO results: About 1.0 mAP improvement remains under the 100% MS-COCO protocol, relative to the 39.21 mAP benchmark.The result indicates gains persist even when all labeled data are available.
  • PASCAL VOC results: 5.36 absolute mAP improvement is obtained on VOC07 with VOC07 labeled data and VOC12 unlabeled data, increasing mAP from 44.64 to 50.00.Adding the specified MS-COCO subset further increases mAP from 46.01 to 50.80 compared with STAC.
  • PASCAL VOC results: The AP0.75 improvement is more prominent than the AP0.5 improvement, so mAP improvement mainly reflects higher-quality predicted bounding boxes.Table 2 reports AP0.5:0.95, AP0.5, and AP0.75 as the evaluation metrics.

5. Ablation Study

The ablations show that instant pseudo labeling, extended weak-strong augmentation, efficient use of unlabeled data, and co-rectification each improve semi-supervised object detection, while training dynamics motivate future loss-weight scheduling.

  • 5.1. Instant pseudo labeling: As the model converges, the quantity of high-quality pseudo annotations instantly improves during training.
  • 5.1. Instant pseudo labeling: 24.70 mAP versus STAC’s 23.14 is achieved with instant pseudo labeling using only Color+Cutout as strong augmentations.The protocol uses 5% MS-COCO with 8× unlabeled data.
  • 5.2. Strong data augmentation: 25.60 mAP is obtained when Mixup and Mosaic are combined, improving over STAC’s 23.14 mAP.Applying strong augmentations during pseudo labeling causes a 1.54 mAP drop compared with STAC, while Mixup or Mosaic improves Instant-Teaching.
  • 5.3. Size of unlabeled data: 23.60 and 28.80 mAP with 1× unlabeled data exceed STAC’s 23.14 and 27.95 mAP with 8× unlabeled data for 5% and 10% labeled data.Instant-Teaching outperforms STAC across the tested unlabeled-data scales, although both methods exhibit a ceiling effect as performance increases.
  • 5.4. Analysis of τ and λu: τ = 0.9 selects higher-quality pseudo annotations, λu = 1.0 gives the best performance, and smaller λu values perform better early in training.The method is relatively robust to λu, but the paper uses a constant λu and leaves dynamic adjustment for future work.
  • 5.5. Analysis of co-rectify: Co-rectification improves pseudo-annotation performance faster and consistently improves Instant-Teaching across training iterations.At the same training iteration, it filters some false predictions and generates more high-quality pseudo annotations.

6. Conclusion

The paper concludes that Instant-Teaching is an effective end-to-end SSOD framework, while co-rectification addresses confirmation bias and improves performance. Although evaluated with Faster-RCNN, the framework is presented as applicable to other detectors.

  • Instant-Teaching uses instant pseudo labeling with extended weak-strong augmentations in an end-to-end SSOD framework.
  • The co-rectify scheme alleviates confirmation bias and further improves the performance of Instant-Teaching.
  • Experiments on MS-COCO and PASCAL VOC demonstrate significant superiority of the proposed method.
  • The framework is not restricted to Faster-RCNN and can be directly applied to detectors such as SSD and FCOS.Application to other detectors is left for future work.

A.1. Learning Schedules

This subsection provides additional details about the learning schedules used in the experiments.

  • The paper provides more details on the different learning schedules used in its experiments.

A.1.1 MS-COCO: Quick Training

The quick-training schedule specifies batch size and image-processing settings, including distinct resizing policies across configurations.

  • The quick-training configuration uses a batch size of 16.
  • One data-processing configuration samples the short edge between 500 and 800 when the resized long edge is below 1024.
  • The appendix specifies a batch-per-image setting for the Faster-RCNN head and a testing score threshold.The supplied passages do not include the corresponding values.
  • Another data-processing configuration fixes the short edge at 800 when the resized long edge is below 1024.

A.1.2 MS-COCO: Standard, [n] ×

The MS-COCO standard training configuration specifies batch size, learning-rate schedules, and image preprocessing settings.

  • Batch size is set to 16.
  • The 1×, 2×, and 3× learning-rate schedules decay from 0.01 to 0.0001 at progressively later iteration milestones.The schedules end at 180k, 360k, and 540k iterations, respectively.
  • Image preprocessing fixes the short edge at 800 when the resized long edge is below 1333.

A.1.3 PASCAL VOC Training

The experiments use shared hyperparameters across MS-COCO and PASCAL VOC, and evaluate Instant-Teaching* for convergence and backbone effects. Instant-Teaching* reaches steady convergence and performs better with a more powerful backbone.

  • PASCAL VOC Training: The experiments use the same hyperparameters for MS-COCO and PASCAL VOC unless otherwise specified.
  • Convergence: Instant-Teaching* models reach steady convergence during training.The convergence behavior is shown in Fig. 8.
  • Backbone Effects: A more powerful backbone improves Instant-Teaching* performance on the 2% MS-COCO protocol.The comparison replaces ResNet-50 with ResNet-101 and evaluates the supervised baseline, Instant-Teaching, and Instant-Teaching* methods.
Loading 2103.11402v1…