PaperlayerView exact revision on arXiv

Source-linked AI summary

Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detection

Xinjiang Wang, Xingyi Yang, Shilong Zhang, Yijiang Li, Litong Feng, Shijie Fang, Chengqi Lyu, Kai Chen, Wayne Zhang

arXiv:2209.01589v3cs.CV

TL;DR

Inconsistent pseudo-targets can inject noise into student training and lead to overfitting in semi-supervised object detection. Consistent-Teacher combines adaptive assignment, feature alignment, and dynamic thresholding to address this problem. It reports 40.0 mAP using 10% labeled MS-COCO data, about 3 mAP above the state-of-the-art.

  • Problem

    In SSOD, oscillating pseudo-boxes and mismatches between classification confidence and box quality can produce noisy targets and overfitting.

    Source: §1 · pp. 1–2

  • Method

    Consistent-Teacher uses adaptive anchor assignment, 3-D feature alignment, and class-specific GMM thresholds to reduce pseudo-target inconsistency.

    Source: §1 · p. 3, §6 · p. 8

  • Results

    40.0 mAP on MS-COCO with 10% labeled data, about 3 mAP ahead of the state-of-the-art [43]; with 100% labels plus unlabeled data, it reaches 47.7 mAP.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The method reports significant gains across SSOD benchmarks while demonstrating robust anchor assignment and more consistent pseudo-labels.

    Source: §6 · p. 8

    The method is mainly developed for traditional single-stage detectors; application to two-stage and DETR-based detectors remains to be verified.

    Source: §5 · p. 8

Abstract

from arXiv · show

In this study, we dive deep into the inconsistency of pseudo targets in semi-supervised object detection (SSOD). Our core observation is that the oscillating pseudo-targets undermine the training of an accurate detector. It injects noise into the student's training, leading to severe overfitting problems. Therefore, we propose a systematic solution, termed ConsistentTeacher, to reduce the inconsistency. First, adaptive anchor assignment~(ASA) substitutes the static IoU-based strategy, which enables the student network to be resistant to noisy pseudo-bounding boxes. Then we calibrate the subtask predictions by designing a 3D feature alignment module~(FAM-3D). It allows each classification feature to adaptively query the optimal feature vector for the regression task at arbitrary scales and locations. Lastly, a Gaussian Mixture Model (GMM) dynamically revises the score threshold of pseudo-bboxes, which stabilizes the number of ground truths at an early stage and remedies the unreliable supervision signal during training. ConsistentTeacher provides strong results on a large range of SSOD evaluations. It achieves 40.0 mAP with ResNet-50 backbone given only 10% of annotated MS-COCO data, which surpasses previous baselines using pseudo labels by around 3 mAP. When trained on fully annotated MS-COCO with additional unlabeled data, the performance further increases to 47.7 mAP. Our code is available at \url{https://github.com/Adamdad/ConsistentTeacher}.

1. Introduction

Consistent-Teacher addresses oscillating pseudo-targets that destabilize semi-supervised object detection (SSOD) by adapting assignment, aligning classification and regression features, and dynamically adjusting thresholds. It reports strong results on COCO and other evaluations.

1. Introduction

  • Inconsistent pseudo-boxes vary across training and can bias SSOD predictions; static anchor assignment is sensitive to box noise, causing overfitting on unlabeled images.Small teacher-box changes can cause erroneous targets to be assigned to nearby objects.Source: §1 · p. 1
  • A mismatch between classification confidence and regression quality can produce different pseudo-boxes for similarly scored anchors, contributing to noisy labels and label drift.The introduction identifies this mismatch as an important factor in drifting pseudo-labels.Source: §1 · p. 2
  • Static confidence thresholds can yield too few or too many pseudo-boxes as the model changes, creating inefficient and biased student supervision.The proposed GMM estimates class-specific thresholds during training to stabilize positive-sample counts.Source: §1 · pp. 2–3
  • 40.0 mAP on MS-COCO with 10% labeled data, about 3 mAP ahead of the state-of-the-art [43]; with 100% labels plus unlabeled data, it reaches 47.7 mAP.The paper also reports improvements across other labeled-data ratios and datasets.Source: §1 · p. 3
  • The paper frames inconsistent pseudo-targets as an SSOD problem that causes severe overfitting and presents Consistent-Teacher as a new baseline for a broad range of evaluations.The introduction describes its investigation as the first in-depth study of this problem.Source: §1 · p. 3
  • Consistent-Teacher combines adaptive sample assignment, 3-D feature alignment, and GMM-based thresholds to address assignment noise, classification–regression mismatch, and threshold inconsistency.FAM-3D lets classification features select regression features across locations and scales; GMM adapts thresholds by class during training.Source: §1 · pp. 2–3

2. Related Work

Related work situates Consistent-Teacher within semi-supervised object detection and adaptive label assignment. The paper focuses on adapting assignment to SSOD, where static criteria can compound inconsistency and errors.

2. Related Work

  • SSOD commonly uses teacher-generated pseudo boxes to train students expected to predict consistently on augmented inputs.The paper adopts a single-stage SSOD framework and focuses on inconsistency.Source: §2 · p. 3
  • Detection assignment commonly uses hard IoU thresholds or centerness, while recent methods have shifted toward adaptive assignment strategies.PAA fits anchor-score distributions; OTA minimizes assignment cost as an optimal-transport problem.Source: §2 · p. 3
  • The paper argues that existing assignment methods are limited to fully supervised settings, while static assignment in SSOD induces inconsistency and accumulated errors.It presents cost-aware assignment as a way to stabilize label noise and improve SSOD performance.Source: §2 · p. 3

3. Consistent-Teacher

Consistent-Teacher combines adaptive sample assignment, 3-D feature alignment, and GMM-based thresholding to address pseudo-target inconsistency in SSOD. Its Mean-Teacher RetinaNet baseline is paired with modules targeting anchor assignment, classification–regression alignment, and threshold variation.

3. Consistent-Teacher

  • The method comprises Adaptive Sample Assignment, 3D Feature Alignment, and Gaussian Mixture-based thresholding for addressing SSOD inconsistencies.Source: §3 · p. 3

3.1. Baseline Semi-Supervised Detector

  • The Mean-Teacher RetinaNet baseline uses weakly augmented unlabeled images to generate teacher pseudo-boxes and strongly jittered images for student training.The teacher is an exponential moving average of the student; labeled images also train the student for classification and regression.Source: §3.1 · p. 3

3.2. Consistent Adaptive Sample Assignment

  • ASA assigns pseudo-boxes to anchors using matching costs, selecting low-cost anchors so assigned labels align with their predictions.The matching cost incorporates classification, regression, and a small center-distance prior; the authors distinguish its focus on pseudo-label shifting from supervised assignment.Source: §3.2 · p. 4

3.3. BBox Consistency via 3-D Feature Alignment

  • FAM-3D lets classification features select regression features across spatial locations and pyramid scales, calibrating localization with classification confidence.Its extra convolution layers increase computation by approximately 1%.Source: §3.3 · pp. 4–5

3.4. Thresholding with Gaussian Mixture Model

  • GMM-based thresholding models each category’s scores as positive and negative distributions to adapt its pseudo-box threshold during training.The adaptive threshold is intended to stabilize the number of positive samples as training progresses.Source: §3.4 · p. 5

4. Experiments

Experiments evaluate Consistent-Teacher on COCO and VOC semi-supervised benchmarks, analyze training inconsistencies, and ablate its assignment, feature-alignment, and thresholding designs.

4.1. Troubleshooting the Inconsistencies in SSOD

  • Consistent-Teacher reduces pseudo-label discrepancy across training checkpoints while its unlabeled-set label quality improves over time.Figure 3 plots mAP on the unlabeled set and pseudo-label inconsistency; the comparison reports reduced discrepancy relative to MeanTeacher.Source: Figure 3 · p. 5, §4.1 · p. 6
  • MeanTeacher predictions include low-confidence, high-IoU boxes, whereas Consistent-Teacher predictions concentrate in high-confidence, high-IoU regions.Figure 4 visualizes confidence against maximum IoU with ground-truth boxes; FAM-3D is described as reducing high-score, low-quality noise predictions.Source: Figure 4 · p. 6, §4.1 · p. 6
  • With static thresholds of 0.4, 0.5, or 0.6, pseudo-label counts rise during training, while GMM keeps counts nearly constant and raises thresholds over time.The GMM sets higher thresholds when fewer labeled samples are available; Figures 5 and 6 show counts and average thresholds, respectively.Source: Figure 5 · p. 6, §4.1 · p. 6, Figure 6 · p. 6

4.2. Semi-supervised Object Detection

  • 36.1 mAP on COCO 5% and 40.0 mAP on COCO 10% surpass Dense Teacher by ~3.1 and ~3 mAP, respectively.These COCO-PARTIAL results are reported in Table 1.Source: §4.2 · p. 6, Table 1 · p. 7
  • 47.7 mAP on COCO-ADDITION surpasses all previous state-of-the-art works.COCO-ADDITION uses the full labeled train2017 set plus unlabel2017; results are reported in Table 2.Source: Table 2 · p. 7, §4.2 · p. 7
  • Consistent-Teacher improves VOC-PARTIAL performance by 2.2 absolute mAP over the latest state-of-the-art methods.The comparison is reported in Table 3.Source: Table 3 · p. 7, §4.2 · p. 7

4.3. Ablation Study

  • ASA reaches 38.50 mAP on COCO 10%, improving 3 mAP over IoU matching versus 1.7 mAP in fully supervised training; FAM-3D gains 1.0 mAP over no alignment and 0.4 over FAM-2D.Table 4 compares assignment strategies; Table 5 reports detection-head ablations.Source: Table 4 · p. 7, §4.3 · p. 8, Table 5 · p. 8
  • GMM achieves 40.00 mAP on COCO 10%, a 0.5 mAP improvement, and outperforms a hard threshold of 0.4 across labeled-data ratios.The ablation compares GMM-based filtering with several hard thresholds; Figure 8 compares GMM with the 0.4 baseline across data ratios.Source: §4.3 · p. 8, Figure 7 · p. 8, Figure 8 · p. 8

5. Limitations and Future Work

Consistent-Teacher is developed mainly for traditional single-stage detectors, leaving its use with two-stage and DETR-based detectors to be verified.

5. Limitations and Future Work

  • The method is developed mainly for traditional single-stage detectors; application to two-stage and DETR-based detectors remains to be verified.These detector families are identified as future verification targets.Source: §5 · p. 8
  • Pseudo-label self-recurrence can accumulate errors from inaccurate priors and human heuristics.The paper notes that adaptive sample assignment replaces some heuristics, including anchor-based assignments.Source: §5 · p. 8
  • Exploring more end-to-end approaches to semi-supervised learning is left for future research.Source: §5 · p. 8

6. Conclusion

Consistent-Teacher combines adaptive anchor assignment, feature alignment, and dynamic thresholding to address pseudo-target inconsistency. The paper reports improved performance over state-of-the-art methods on various SSOD benchmarks.

6. Conclusion

  • Consistent-Teacher identifies positive anchors by lowest matching costs and aligns classification and regression through 3-D feature-pyramid offsets.Adaptive anchor assignment and FAM address anchor matching and subtask alignment, respectively.Source: §6 · p. 8
  • A Gaussian Mixture Model dynamically adjusts pseudo-bbox thresholds during self-training to address threshold inconsistency.The method integrates this threshold adjustment with adaptive anchor assignment and FAM.Source: §6 · p. 8
  • Consistent-Teacher achieves significant performance improvement over state-of-the-art methods on various SSOD benchmarks.The conclusion attributes this result to the integrated approach and reports robust anchor assignment and consistent pseudo-bboxes.Source: §6 · p. 8

1. More details in Consistent-Teacher

The paper measures pseudo-target inconsistency by tracking prediction changes across training checkpoints. It uses accumulated 1 − mAP against the preceding checkpoint's predictions as its measure.

1. More details in Consistent-Teacher

  • Inconsistency is defined as pseudo-boxes being inaccurate and varying substantially across training stages.The paper quantifies variation in pseudo-bboxes across training steps.Source: §1.1 · p. 11
  • Every 4,000 training steps, the authors run inference on 5,000 unlabeled images and treat the previous checkpoint's predictions as ground truth.The current checkpoint is evaluated against those previous predictions.Source: §1.1 · p. 11
  • Accumulated 1 − mAP across checkpoints measures inconsistency, with higher mAP indicating more consistent pseudo-targets.The mAP compares current predictions to the preceding checkpoint's predictions.Source: §1.1 · p. 11

2. Verification of the Inconsistency in SSOD

The experiments assess how pseudo-label noise affects assignment consistency and how classification confidence relates to box IoU. They report that ASA is more robust to noise than IoU-based assignment and that Consistent-Teacher reduces classification–regression mismatch.

Assignment Inconsistency under Noisy Pseudo Labels

  • The A-IoU compares anchor sets assigned to clean and perturbed boxes; higher A-IoU indicates greater robustness to label noise.Box coordinates are perturbed with Gaussian noise at ratio ρ; the experiments vary ρ and also track A-IoU over training at ρ = 0.1.Source: §2 · p. 11
  • The assignment comparison measures clean-versus-noisy label A-IoU across noise ratios and training stages.Figure 9 depicts mean ± standard deviation A-IoU at different training times and noise ratios.Source: Figure 9 · p. 11, §2 · p. 11
  • ASA maintains assignment consistency under severe label noise, while IoU-based assignment fails to preserve initial assignments at high noise.Across training, ASA and ATSS have higher A-IoU than IoU-based assignment, and ASA improves as the detector becomes more accurate.Source: §2 · p. 11

Classification and Regression Inconsistency

  • Table 6 reports linear-regression (LR) error of 0.080 for Consistent-Teacher versus 0.109 for Mean Teacher RetinaNet, supporting that Consistent-Teacher largely eliminates classification–regression mismatch in semi-supervised detection.Source: Table 6 · p. 11, §2 · p. 12

3. Additional Ablation Study

Consistent-Teacher improves results across anchor-based and anchor-free detectors. The ablation and timing results also show that λdist supports stable training, while some added modules increase iteration time.

3.1. Anchor-based VS Anchor-Free

  • Semi-supervised FCOS reaches 35.8 mAP with MeanTeacher and gains 4.1 mAP with Consistent-Teacher.The paper reports that Consistent-Teacher substantially enhances both anchor-based and anchor-free baselines; Table 9 compares their SSOD performance.Source: §3.1 · p. 12, Table 7 · p. 12

3.2. Ablation on λdist

  • Setting λdist = 0 can destabilize assignment and cause memory overflow early in training, while a significant λdist can cancel ASA's performance advantage.The authors use λdist to ensure stable training and report that early matching is quite inaccurate.Source: §3.2 · p. 12, Table 8 · p. 12

3.3. Training Time

  • ASA reduces assignment time through efficient implementation and fewer required anchors; the RetinaNet baseline takes 1.25 s/iter.The table is described as comparing training time per iteration across methods.Source: §3.3 · p. 12, Table 9 · p. 12
  • FAM3D adds marginal training time, while updating the GMM threshold every iteration increases training time by approximately 10%.The authors describe the GMM overhead as a cost of threshold updating.Source: §3.3 · p. 12

4. Detection results visualization

The qualitative comparison highlights Consistent-Teacher’s advantages in crowded localization, task prediction alignment, and detection of small or crowded instances. Its failure examples also show limitations on some out-of-distribution and small-object cases, including part-whole errors.

4.1. Qualitative comparison with the baseline.

  • Consistent-Teacher better localizes crowded objects than Mean Teacher, which can mistake overlaps for new instances.The authors attribute the improvement to adaptive anchor selection.Source: §4.1 · p. 13
  • FAM3D addresses classification-regression inconsistency, such as birds classified as airplanes despite accurate localization.Mean Teacher may also regress inaccurate boxes despite high classification confidence.Source: §4.1 · p. 13
  • Consistent-Teacher detects small or crowded instances, such as hot dogs on a grill, that Mean Teacher may miss.Source: §4.1 · p. 13

4.2. Good and Failure Cases.

  • Failure cases include some out-of-distribution images and poor detection of small classes such as toothbrushes and hair dryers.Source: §4.2 · p. 13
  • The detector can identify an object part as a whole object, such as a panda’s head as an animal or a clock dial as a clock.Source: §4.2 · p. 13

5. Experiment and Hyper-parameter settings

The experiments use COCO2017 and VOC data, with labeled and unlabeled sets specified for each dataset. Training follows the cited augmentation procedure and uses a multi-GPU setup; the figures also identify ground truths, predictions, and false positives by color.

5.1.1 MS-COCO 2017

  • COCO2017 provides 118K training images, 5K validation images, and bounding-box annotations for 80 categories.Source: §5.1.1 · p. 13

5.1.2 PASCAL VOC 2007-2012

  • VOC experiments use VOC 2007 trainval’s 5,011 images as labeled data and VOC 2012 trainval’s 11,540 images as unlabeled data, evaluated on VOC 2007 test.Source: §5.1.2 · p. 13

5.1.3 Data Augmentations.

  • The data augmentations follow Soft Teacher, with separate procedures for labeled images and weakly or strongly augmented unlabeled images.Source: §5.1.3 · p. 13

5.2. Implementation Details

  • Training uses 8 NVIDIA Tesla V100 GPUs for about 3 days and 180K iterations, with 1 labeled and 4 unlabeled images per GPU.Figures 10–12 use orange for ground truths and violet for predictions; Figure 10 also marks false positives in red.Source: §5.2 · p. 13, Figure 10 · p. 14, Figure 11 · p. 15

Related papers

Loading 2209.01589v3…