PaperlayerView exact revision on arXiv

Source-linked AI summary

Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detection

Hongyu Zhou, Zheng Ge, Songtao Liu, Weixin Mao, Zeming Li, Haiyan Yu, Jian Sun

arXiv:2207.02541v2cs.CV

TL;DR

Semi-supervised object detection methods based on pseudo-boxes require post-processing and fine-tuned hyperparameters. Dense Teacher uses dense teacher predictions as pseudo-labels and adds region selection to manage noise. It reports state-of-the-art performance on MS-COCO and Pascal VOC.

  • Problem

    Pseudo-box-based semi-supervised object detection requires post-processing and fine-tuned hyperparameters, while inaccurate pseudo-boxes can make label assignment inconsistent with potential ground truth.

    Source: Inconsistent Label Assignment · p. 7

  • Method

    Dense Teacher uses the teacher network's dense predictions as pseudo-labels without ordinary post-processing and applies region division to suppress noise and highlight key regions.

    Source: §1 · pp. 2–3

  • Results

    Dense Teacher reports state-of-the-art performance on MS-COCO and Pascal VOC benchmarks.

    Source: §1 · p. 3, §5 · p. 14

  • Takeaways & Limitations

    The authors conclude that Dense Teacher is a simpler but stronger alternative to pseudo-box-based SS-OD algorithms on the evaluated benchmarks.

    Source: §5 · p. 14

Abstract

from arXiv · show

To date, the most powerful semi-supervised object detectors (SS-OD) are based on pseudo-boxes, which need a sequence of post-processing with fine-tuned hyper-parameters. In this work, we propose replacing the sparse pseudo-boxes with the dense prediction as a united and straightforward form of pseudo-label. Compared to the pseudo-boxes, our Dense Pseudo-Label (DPL) does not involve any post-processing method, thus retaining richer information. We also introduce a region selection technique to highlight the key information while suppressing the noise carried by dense labels. We name our proposed SS-OD algorithm that leverages the DPL as Dense Teacher. On COCO and VOC, Dense Teacher shows superior performance under various settings compared with the pseudo-box-based methods.

1 Introduction

Dense Teacher replaces pseudo-box supervision with dense teacher predictions to avoid post-processing while preserving richer information. A region division strategy addresses noise in those predictions, and the framework reports state-of-the-art performance on MS-COCO and Pascal VOC.

1 Introduction

  • Semi-supervised learning uses labeled and unlabeled data during training, addressing detection networks' reliance on large labeled datasets and the human effort required to label them.The introduction notes that pseudo-labeling and consistency-based regularization are established approaches in image classification.Source: §1 · p. 1
  • Pseudo-box supervision requires NMS, thresholding, and label assignment, adding hyperparameters that substantially affect semi-supervised detection performance.The passage names NMS and score thresholds as examples.Source: §1 · p. 2
  • Dense Pseudo-Label (DPL) uses the teacher network's original output without ordinary post-processing, retaining richer information than pseudo-box labels.An exponential moving average of the student generates DPL for unlabeled data, and the student learns from DPL alongside labeled ground truth.Source: §1 · pp. 2–3
  • Dense Teacher with region division reports state-of-the-art performance on MS-COCO and Pascal VOC.The introduction also reports gain analysis and ablations to assess the framework's components.Source: §1 · p. 3
  • Because dense predictions also contain noise such as low-scoring predictions, region division suppresses noise and directs student attention to key regions.The authors report that this strategy can use information in hard-negative regions to enhance training.Source: §1 · p. 3

2 Related Works

The related work reviews pseudo-labeling and consistency regularization in semi-supervised learning, contrasts anchor-based and anchor-free detectors, and surveys semi-supervised object detection methods. It identifies detector label assignment as a characteristic shared by prior pseudo-box-based strategies.

2.1 Semi-Supervised Learning

  • Pseudo-labeling trains a teacher on labeled data to label unlabeled examples for a student, while consistency regularization encourages similar predictions under perturbations.The review describes Exponential Moving Average teachers as an online pseudo-labeling variant and notes that consistency methods can be combined with other methods.Source: §2.1 · p. 3

2.2 Object Detection

  • Anchor-based detectors predict box offsets and scales from predefined anchors, whereas anchor-free detectors predict box size and location directly on the feature map.FCOS predicts classification scores, four boundary distances, and a quality score at each FPN pixel; the paper uses FCOS for its experiments.Source: §2.2 · p. 4

2.3 Semi-Supervised Object Detection

  • Prior SS-OD methods include consistency losses, static-teacher pseudo-labels, thresholding, class rebalancing, and score-weighted losses for classification or regression.The review says these pseudo-box-based strategies treat boxes or selected boxes as ordinary targets and follow detector label-assignment strategies.Source: §2.3 · p. 4

3 Dense Teacher

Dense Teacher replaces post-processed pseudo-boxes with dense teacher predictions, while selecting learning regions to preserve useful information and suppress noise. The section motivates this design through threshold sensitivity, NMS trade-offs, and inconsistent label assignment.

3.1 Pseudo-Labeling Framework

  • Dense Teacher uses an EMA teacher to label unlabeled images and trains the student on those labels alongside ground truth from labeled images.The student updates the teacher through the same EMA process; the overall loss combines supervised and unsupervised losses.Source: §3.1 · p. 5

Dilemma in Thresholding

  • High confidence thresholds can eliminate true predictions and even prevent convergence, while low thresholds increase false positives, leaving no reliable universal cutoff.The reported threshold sensitivity appears on both COCO and CrowdHuman.Source: Dilemma in Thresholding · p. 5, Figure 2 · p. 6

Dilemma in Non-Maximum Suppression (NMS)

Inconsistent Label Assignment

  • Inaccurate pseudo-box localization can make assigned foreground labels disagree with ground truth, even when a predicted box meets an IoU threshold of 0.5.The authors identify this inconsistency as likely to degrade performance.Source: Inconsistent Label Assignment · p. 7, Figure 3 · p. 7

3.3 Dense Pseudo-Label

  • Dense Pseudo-Label (DPL) uses the model’s post-sigmoid predictions directly, retaining more detailed information than pseudo-box labels without ordinary post-processing.The authors frame DPL as a dense alternative to using pseudo-boxes as an intermediate label.Source: §3.3 · pp. 7–8, Inconsistent Label Assignment · p. 7
  • Because DPL retains low-scoring predictions that can distract the student, the method selects the top k% of pixels by Feature Richness Score for learning and suppresses the rest.The selected predictions are retained as DPL targets; other regions are set to zero.Source: §3.3 · p. 8
  • The learning-region design can include hard negatives and supports unsupervised regression using IoU Loss.The paper presents both as advantages of the region-selection design.Source: §3.3 · p. 8

4 Experiments

The experiments cover MS-COCO and Pascal VOC using labeled and unlabeled data under three settings. COCO-Standard varies the labeled-data fraction, while COCO-Full and VOC Mixture use distinct labeled/unlabeled dataset combinations.

4.1 Datasets and Experiment Settings

  • The evaluation uses MS-COCO and Pascal VOC, with COCO validation on val2017 and VOC validation on VOC07 test.COCO training uses train2017 and unlabeled2017; VOC training uses VOC07 train and VOC12 train.Source: Datasets · p. 9

Datasets.

  • COCO-Standard samples 1%, 2%, 5%, or 10% of train2017 as labeled data and averages results over five dataset folds.The remaining images are treated as unlabeled; the folds follow the division in [18].Source: Datasets · p. 9
  • COCO-Full uses train2017 as labeled data and unlabeled2017 as unlabeled data; VOC Mixture uses VOC07 train as labeled data and VOC12 train plus COCO20cls4 as unlabeled data.Source: Datasets · p. 9

Implementation Details.

On COCO-Standard 10%, Dense Teacher’s final configuration reaches 35.11% mAP and is reported as state of the art for that setting. The method is evaluated with FCOS and a ResNet-50 backbone, and its reported gains build through dense labels, region division, and regression learning.

Implementation Details.

  • The experiments use FCOS with an ImageNet-pretrained ResNet-50 backbone, batch size 16, and a burn-in strategy to initialize the teacher.The unlabeled-data loss weight is 4 on COCO-Standard and 2 on the other settings.Source: Implementation Details · p. 9

4.2 Main Results

  • 35.11% mAP is the final Dense Teacher result on COCO-Standard 10%, reported as a new state of the art for that setting.The progression begins at 31.52% mAP for Unbiased Teacher, reaches 32.0% with DPL alone, gains a further 1.34% with region division, and then adds regression learning.Source: §4.2 · p. 9, Table 1 · p. 10
  • The authors attribute Dense Teacher’s advantages to a simpler, post-processing-free label pipeline and region division that uses hard negatives to enhance training.They report that DPL alone yields only a marginal improvement over the baseline, while the division strategy provides a further gain.Source: §4.2 · pp. 10–11

4.3 Comparison with State-of-the-arts

Table 2 presents experimental results on COCO-Standard.

4.3 Comparison with State-of-the-arts

  • Table 2 reports experimental results on COCO-Standard.An asterisk marks re-implemented FCOS results, and ⊚ marks training with large-scale jittering.Source: Table 2 · p. 11

COCO-Standard.

Dense Teacher reports gains across COCO-Standard, Pascal VOC, and COCO-Full, with results varying by labeled-data setting and training configuration.

COCO-Standard.

  • Dense Teacher performs consistently better than compared methods with 2%–10% labeled data, while at 1% it trails Faster R-CNN Unbiased Teacher but leads its FCOS version by 1.2% mAP.Large-scale jittering produces more significant improvements and a new state-of-the-art, according to the authors.Source: COCO-Standard · p. 11

VOC & COCO-Full.

  • 4.9% mAP improvement, reaching 46.12% mAP, is reported for Dense Teacher with large-scale jittering and a longer scheduler on COCO-Full.The authors report a 3.1% boost from unlabeled2017 compared with other methods’ results.Source: VOC & COCO-Full · p. 11
  • 8.2% AP50 and 10.0% mAP improvements over the supervised baseline are reported on Pascal VOC.Table 3 compares existing methods on VOC07 test.Source: VOC & COCO-Full · p. 11, Table 3 · p. 12

4.4 Ablation and Key Parameters

  • Selecting hard negatives for training outperforms ignoring them, while suppressing them causes a significant drop on both classification and regression branches.The comparison examines three strategies: suppressing, ignoring, and selecting hard negatives.Source: Effect of Hard Negative Samples · p. 12

Effect of Hard Negative Samples

The reported results support learning on selected regions that include reliable positives and hard negatives, including for regression.

Effect of Hard Negative Samples

  • The region-division and pseudo-box approaches yield 1.8% and 1.6% mAP gains, respectively, while using ground-truth positives as the learning region yields about 2% mAP.The authors conclude that their region-division strategy produces sufficiently reliable regions for regression learning.Source: Regression Branch · p. 12

Regression Branch

  • Hard negatives are the difference between the method’s division at k = 1 and the assigned foreground.The COCO-Standard unlabeled images used for this analysis have annotations available for label assignment.Source: Regression Branch · p. 12

Effectiveness on other detectors and datasets

Dense Teacher is evaluated with other detectors and datasets, and the paper studies how region size and unsupervised-loss weight affect performance.

Effectiveness on other detectors and datasets

Size of Learning Region

  • Selecting 1% of samples for unsupervised learning gives the best reported performance; COCO under FCOS label assignment has about 0.4% positive samples.The authors infer that the optimal region includes both ground-truth positives and valuable hard negatives.Source: Size of Learning Region · p. 13, Table 7 · p. 14
  • Dataset statistical characteristics can help determine the learning-region hyperparameter and mitigate model-migration and deployment challenges.This follows the paper’s discussion of the performance sensitivity to region size.Source: Size of Learning Region · p. 14
  • For COCO-Standard 10%, an unsupervised weight of 4 is favorable, whereas a weight of 2 is sufficient for COCO-Full.The authors attribute this difference to different degrees of overfitting across the settings.Source: Unsupervised Loss Weight · p. 14, Table 7 · p. 14

5 Conclusion

The conclusion argues that pseudo-boxes are sub-optimal for unlabeled data and presents Dense Teacher, which uses the teacher’s dense predictions as pseudo-labels. The authors report state-of-the-art performance on MS-COCO and Pascal VOC.

5 Conclusion

  • Dense Teacher achieves state-of-the-art performance on MS-COCO and Pascal VOC benchmarks.The conclusion compares it with other pseudo-box-based SS-OD algorithms.Source: §5 · p. 14
  • Dense Teacher adopts dense predictions from the teacher model as pseudo-labels for unlabeled data.Source: §5 · p. 14
  • The authors describe Dense Teacher as simpler but stronger, arguing that pseudo-boxes are a sub-optimal choice because of flaws from their lengthy generation pipeline.Source: §5 · p. 14

Related papers

Loading 2207.02541v2…