PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Domain Adaptive Object Detection with Probabilistic Teacher

Meilin Chen, Weijie Chen, Shicai Yang, Jie Song, Xinchao Wang, Lei Zhang, Yunfeng Yan, Donglian Qi, Yueting Zhuang, Di Xie, Shiliang Pu

arXiv:2206.06293v1cs.CVcs.AI

TL;DR

Existing Faster R-CNN cannot predict localization uncertainty for target pseudo boxes. Probabilistic Teacher adds probabilistic teacher–student consistency, parallel anchor adaptation, and EFL, and reports new state-of-the-art results on multiple source-based/free UDA-OD benchmarks.

  • Problem

    Existing Faster R-CNN cannot predict localization uncertainty, leaving uncertainty in target pseudo boxes unrepresented during self-training.

    Source: §1 · p. 2

  • Method

    Probabilistic Teacher models category and localization labels probabilistically, trains threshold-free teacher–student consistency, adapts anchors alongside localization, and uses EFL to emphasize lower-entropy pseudo boxes.

    Source: §1 · p. 2

  • Results

    Probabilistic Teacher with EFL achieves new state-of-the-art results on multiple source-based/free UDA-OD benchmarks, surpassing previous approaches by a large margin.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Ablations identify strong augmentation as implicit intra-domain alignment and especially important in self-training, while localization-only adaptation improves over source-only.

    Source: §1 · p. 2

Abstract

from arXiv · show

Self-training for unsupervised domain adaptive object detection is a challenging task, of which the performance depends heavily on the quality of pseudo boxes. Despite the promising results, prior works have largely overlooked the uncertainty of pseudo boxes during self-training. In this paper, we present a simple yet effective framework, termed as Probabilistic Teacher (PT), which aims to capture the uncertainty of unlabeled target data from a gradually evolving teacher and guides the learning of a student in a mutually beneficial manner. Specifically, we propose to leverage the uncertainty-guided consistency training to promote classification adaptation and localization adaptation, rather than filtering pseudo boxes via an elaborate confidence threshold. In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter. Together with this framework, we also present a novel Entropy Focal Loss (EFL) to further facilitate the uncertainty-guided self-training. Equipped with EFL, PT outperforms all previous baselines by a large margin and achieve new state-of-the-arts.

1. Introduction

Self-training for domain-adaptive object detection depends on pseudo-box quality, but confidence-threshold filtering can miss localization uncertainty and requires target annotations for tuning. Probabilistic Teacher (PT) instead uses uncertainty-guided consistency, adapts anchors alongside localization, and reports state-of-the-art results across multiple benchmarks.

1. Introduction

  • PT replaces threshold-based pseudo-box filtering with uncertainty-guided teacher–student consistency for classification and localization adaptation.The detector is restructured probabilistically so category and localization labels can be represented as probability distributions.Source: §1 · p. 2
  • PT adapts anchors in parallel with localization because source and target domains can have different bounding-box size distributions.This unifies classification, localization, and anchor adaptation within one framework.Source: §1 · p. 2
  • Entropy Focal Loss (EFL) promotes uncertainty-guided consistency training in both classification and localization branches, emphasizing lower-entropy pseudo boxes.The paper introduces EFL to further facilitate uncertainty-guided self-training.Source: §1 · p. 2
  • The threshold-free approach avoids tuning a confidence threshold on annotated target data and extends to source-free adaptation using only unlabeled target data.The paper reports improvements over previous approaches in the source-free setting.Source: §1 · p. 2
  • The paper reports that strong augmentation bridges the intra-domain gap, matters more in self-training than domain-alignment approaches, and localization adaptation alone improves over source-only training.These are findings from the paper’s ablation studies.Source: §1 · p. 2
  • PT achieves new state-of-the-art results on multiple source-based and source-free UDA-OD benchmarks, surpassing previous approaches by a large margin.The paper also reports new state-of-the-art results for both tested Foggy Cityscapes splits.Source: §1 · p. 2

2. Related Works

Prior self-training methods seek better pseudo boxes, but those boxes remain noisy and confidence-threshold filtering is difficult to tune without annotated target data. The paper instead motivates handling pseudo-box uncertainty during cross-domain self-training.

Unsupervised Domain Adaptive Object Detection

  • Prior pseudo-box generation methods improve accuracy through target-to-source translation or a larger-capacity teacher, yet pseudo boxes remain noisy.The paper proposes dynamically handling this noise with uncertainty-guided cross-domain self-training.Source: §2 · p. 2
  • On Foggy Cityscapes, box variance is reported as better than foreground score for measuring bounding-box localization accuracy.The comparison uses IoU with ground-truth boxes as the localization-accuracy measure.Source: Figure 3 · p. 3

Self Training for Object Detection

  • UDA-OD threshold filtering depends on selecting a category-confidence threshold without annotated target data for tuning, and ignores localization confidence.Prior semi-supervised methods use thresholds; Soft Teacher uses box jitter for localization filtering, while LabelMatch updates the threshold using label-distribution consistency.Source: §2 · pp. 2–3

3. Preliminary

The preliminary section recasts Faster R-CNN as a probabilistic detector so it can represent uncertainty in both classification and localization. It models each box coordinate with a Gaussian and trains its predicted distribution against a ground-truth point distribution using cross-entropy.

Probabilistic Faster R-CNN

  • Probabilistic Faster R-CNN represents both category and localization labels as probability distributions, addressing the standard detector’s lack of localization uncertainty.The standard detector’s classification branch predicts a label-space distribution, while its L1 bbox regression models coordinates as Dirac deltas.Source: §3 · p. 3
  • Each bounding-box coordinate is modeled as a Gaussian with mean and variance, and the variance is constrained between zero and one by a sigmoid.The regression loss is cross-entropy between the ground-truth Dirac-delta distribution and the predicted Gaussian distribution.Source: §3 · p. 3
  • The regression loss sums cross-entropy over matched predicted boxes, using each ground-truth coordinate’s probability under the predicted Gaussian.The passage defines the predicted coordinate mean and variance and the anchor-or-proposal count used in the loss.Source: §3 · p. 3
  • Box variance is reported as better than foreground score for measuring localization accuracy, using IoU with ground-truth boxes.The comparison is shown on Foggy Cityscapes.Source: §3 · p. 3
  • The reformulated training objective uses four cross-entropy losses with equal weights, following the original Faster R-CNN.The text presents this objective as the setup for the subsequent Probabilistic Teacher method.Source: §3 · p. 4

4. Probabilistic Teacher

Probabilistic Teacher uses uncertainty in teacher-generated target pseudo boxes to guide student learning across classification and localization. It combines consistency training, gradual teacher and anchor updates, sharpening functions, and Entropy Focal Loss.

4.1. Overview

  • The teacher generates probabilistic target pseudo boxes from weakly augmented images, and the student learns from them while updating the teacher through EMA.The teacher represents category labels as distributions and box locations as four Gaussian distributions; student knowledge feeds back to the teacher via exponential moving average.Source: §4.1 · p. 4

4.2.1. UNCERTAINTY-GUIDED CONSISTENCY TRAINING

  • The student combines supervised source loss with target self-supervised consistency between teacher and student predictions for classification and localization.Target pseudo-box classification and coordinate distributions are sharpened and used across RPN and ROI-head classification and regression losses.Source: §4.2.1 · p. 4

4.2.2. SHARPENING FUNCTIONS

  • Sharpening functions lower prediction entropy: classification uses temperature SoftMax, while bbox regression sharpens Gaussian predictions.At τ = 1, sharpening is equivalent to the original SoftMax or Gaussian function; the work sets τ < 1.Source: §4.2.2 · p. 4

4.2.3. TEACHER UPDATING

  • EMA updates the teacher from student weights, providing a slowly updated teacher regarded as an ensemble of students across training timestamps.The teacher update is θT = αθT + (1 − α)θS, where α is the EMA rate.Source: §4.2.3 · p. 5

4.2.4. ANCHOR ADAPTATION

  • Anchor shapes adapt slowly through teacher-student mutual learning using EMA to match target-domain bbox distributions.The objective includes the anchor shapes across A anchors.Source: §4.2.4 · p. 5

4.3. Entropy Focal Loss

  • Entropy Focal Loss uses category and location entropy to emphasize less noisy category predictions and more accurate coordinates during consistency training.Its entropy-based weighting further facilitates uncertainty-guided consistency in classification and localization branches.Source: §4.3 · p. 5

5. Strong Augmentation for UDA-OD

The paper identifies an intra-domain adaptation gap in which smaller, more blurred, and occluded target objects adapt less well. It uses strong augmentation to transform large, confident objects to mimic these harder cases.

5.1. Intra-Domain Gap

  • Smaller, more blurred, and occluded target objects tend to have poorer adaptation, a neglected intra-domain gap that the paper identifies as a UDA-OD bottleneck.The contrast is with inter-domain gap, which the passage says previous work has addressed.Source: §5.1 · p. 5
  • Table 1 covers normal-to-foggy adaptation at foggy-level splits 0.01, 0.02, and ALL, and identifies Source only and Oracle reference models.The caption defines Source only as trained on labeled source data and Oracle as trained on labeled target data.Source: Table 1 · p. 6

5.2. Intra-Domain Alignment via Strong Augmentation

  • Strong augmentation transforms large, high-confidence objects to mimic small, blurred, and occluded ones, whose low-entropy pseudo labels guide model attention to those cases.The passage names random resizing, Gaussian blur, and color jitter as examples; it describes this augmentation as implicit intra-domain alignment.Source: §5.2 · p. 6

6. Experiments

PT is evaluated across four kinds of domain shift, with reported improvements over prior methods in each. Ablations examine augmentation, anchor adaptation, EFL, uncertainty, source-free use, and training behavior.

6.1. Experimental Settings

  • Experiments cover four domain shifts across five public datasets; evaluation uses mAP at IoU 0.5, with Faster-RCNN as the base detector and strong augmentation in baseline comparisons.Training runs for 30k iterations: 4k for pretraining and 26k for mutual learning.Source: §6.1 · pp. 6–7

6.2. C2F: Adaptation from Normal to Foggy Weather

  • 42.7 mAP for Foggy Cityscapes level 0.02 and 47.1 mAP across all three levels; PT exceeds MeGA by +0.9/+5.3, respectively.Strong augmentation improves the “source only” result by +4.8 and +6.2 mAP on the two splits.Source: §6.2 · p. 7

6.3. C2B: Adaptation from Small to Large-Scale Dataset

  • 34.9 mAP on C2B, a +5.4 margin over ICR-CCR, with PT outperforming all reported baselines.The comparison reports results on seven common categories across Cityscapes and BDD100k.Source: §6.3 · p. 7, Table 2 · p. 7

6.4. K2C: Adaptation across Cameras

  • PT improves on the best K2C method, GPA, by +12.7 mAP and also outperforms SimROD.SimROD uses YOLOv5 as its base detector and relies on a large-scale teacher model.Source: Table 3 · p. 7, §6.4 · p. 8

6.5. S2C: Adaptation from Synthetic to Real Images

  • PT improves over the current best S2C method by +3.0 mAP on the Cityscapes test set’s car category.The experiment uses Sim10k as source data and the Cityscapes training set as target data.Source: Table 4 · p. 7, §6.5 · p. 8

6.6. Ablation Studies

  • Stronger augmentation improves performance, and its gains are larger for self-training than for domain-alignment counterparts; anchor adaptation and EFL contribute across all four shifts.The authors describe strong augmentation as implicit intra-domain alignment and interpret anchor adaptation as addressing object-scale shifts; EFL uses pseudo-box entropy to further boost performance.Source: Table 5 · p. 7, §6.6 · p. 8
  • PT extends to source-free UDA-OD using only target-domain self-supervised loss; source-free and source-based results are almost comparable for K2C and S2C, but remain far apart for C2F and C2B.The authors leave the remaining C2F and C2B gap for future research.Source: §6.6 · p. 8, Table 6 · p. 8
  • PT is more superior in mean performance and more robust in standard deviation than threshold-based training; PT’s performance modeling gain alone is described as trivial and limited.Performance varies widely across confidence thresholds, while PT is threshold-free; category and box entropy rise briefly, then decrease stably during mutual learning.Source: §6.6 · pp. 8–9

7. Conclusions

The conclusion presents Probabilistic Teacher with Entropy Focal Loss as a framework for uncertainty in cross-domain self-training, reporting state-of-the-art results across source-based and source-free UDA-OD benchmarks.

7. Conclusions

  • Probabilistic Teacher studies the use of uncertainty during cross-domain self-training.Source: §7 · p. 9
  • The framework is equipped with the novel Entropy Focal Loss.Source: §7 · p. 9
  • The authors report new state-of-the-art results on multiple source-based and source-free UDA-OD benchmarks.They suggest the method may inspire other weakly supervised object-detection tasks, such as noisy-label-supervised detection.Source: §7 · p. 9

A. More Experimental Results

The appendix presents additional qualitative target-domain detection results, including C2F visualizations and results from two adaptation tasks.

A. More Experimental Results

A.1. More Qualitative Visualization

  • In Figure 9, green, red, and gold boxes denote true positives, false positives, and false negatives, respectively.Source: Figure 9 · p. 12
  • Figures 9 and 10 show qualitative detection results from two adaptation tasks to demonstrate PT’s improvement.Source: §A.1 · p. 12

B. More Implementation Details

Anchor adaptation retains Faster-RCNN’s initial anchor shapes and adds a small set of learnable parameters.

B.1. Anchor adaptation

  • Initial anchor shapes follow the original Faster-RCNN configuration.Source: §B.1 · p. 12
  • The configuration uses three anchor sizes: 128, 256, and 512.Source: §B.1 · p. 12
  • It uses three aspect ratios—0.5, 1, and 2—at each output sliding-window position.Source: §B.1 · p. 12
  • Anchor adaptation requires only 9 × 2 additional learnable parameters.Source: §B.1 · p. 12

B.2. Data Augmentation Details

The appendix specifies strong target-image augmentations and outlines Probabilistic Teacher’s two-stage training loop, where student learning and teacher updates proceed jointly.

B.2. Data Augmentation Details

  • Strong augmentation applies horizontal flipping, color jitter, grayscale conversion, Gaussian blur, and solarization, following SimCLR except for RandomResizedCrop.Images are also randomly resized and padded to their original size.Source: §B.2 · p. 12
  • Random resizing zooms images out by a factor sampled within (0.5, 1), preserving aspect ratio.The implementation applies this resizing operation to each image.Source: §B.2 · p. 12
  • Figure 10 marks true positives, false positives, and false negatives with green, red, and gold boxes in additional K2C target-domain results.Source: Figure 10 · p. 13

B.3. Pseudo-Code of PT

  • PT pretrains on labeled source data, then uses target pseudo boxes and source labels for uncertainty-guided student training before updating the teacher with EMA.The algorithm also trains anchor shapes by minimizing the target loss.Source: §B.3 · p. 13

C. Mathematical Proofs

The appendix states proofs for entropy and cross-entropy properties used in the paper, alongside the PT training pseudocode. It also gives a discrete-entropy maximum condition.

C.1. The cross-entropy between a Dirac delta distribution and a univariate Gaussian distribution

  • The appendix formulates the cross-entropy between a Dirac delta distribution and a univariate Gaussian, then states that it can be simplified.Source: §C.1 · pp. 13–14
  • PT’s pseudocode initializes teacher and student from a source-trained detector, then trains the student with source data and target pseudo boxes before updating the teacher by EMA.The loop includes uncertainty-guided classification and localization losses and anchor-shape training.Source: §C.1 · p. 14

C.2. The entropy of a univariate Gaussian distribution

  • For a univariate Gaussian, the appendix states that entropy depends only on variance; with σ processed between zero and one, its maximum is 1/2 log (2π) + 1/2.Source: §C.2 · p. 14

C.3. The cross-entropy between two univariate Gaussian distributions

  • The appendix derives cross-entropy between two univariate Gaussians by expanding the second term and simplifying using PDF normalization, mean, and variance properties.The resulting expression includes a constant C.Source: §C.3 · p. 15

C.4. Maximal value of a discrete variable’s entropy

  • A discrete variable’s entropy reaches its maximum when all possible events are equiprobable.Source: §C.4 · p. 16

Related papers

Loading 2206.06293v1…