PaperlayerView exact revision on arXiv

Source-linked AI summary

Contrastive Mean Teacher for Domain Adaptive Object Detectors

Shengcao Cao, Dhiraj Joshi, Liang-Yan Gui, Yu-Xiong Wang

arXiv:2305.03034v1cs.CV

TL;DR

Object detectors lose performance across source and target domains, and mean-teacher adaptation is hindered by low-quality pseudo-labels. CMT unifies mean-teacher self-training with object-level contrastive learning using pseudo-label-derived features, achieving state-of-the-art adaptation results and greater robustness as pseudo-label noise increases.

  • Problem

    Object detectors suffer from source–target domain gaps, while mean-teacher self-training struggles with low-quality pseudo-labels in unsupervised adaptation.

  • Method

    CMT integrates mean-teacher self-training with object-level contrastive learning that uses noisy target pseudo-labels to optimize localized, multi-scale features without target labels.

  • Results

    51.9% mAP improves Cityscapes-to-Foggy-Cityscapes adaptation from 49.8% mAP, while CMT reduces performance variance and provides larger gains as pseudo-label noise increases.

  • Takeaways & Limitations

    CMT is a general-purpose, drop-in framework that can combine with existing mean-teacher methods and consistently improve domain-adaptive object detection.

Abstract

from arXiv · show

Object detectors often suffer from the domain gap between training (source domain) and real-world applications (target domain). Mean-teacher self-training is a powerful paradigm in unsupervised domain adaptation for object detection, but it struggles with low-quality pseudo-labels. In this work, we identify the intriguing alignment and synergy between mean-teacher self-training and contrastive learning. Motivated by this, we propose Contrastive Mean Teacher (CMT) -- a unified, general-purpose framework with the two paradigms naturally integrated to maximize beneficial learning signals. Instead of using pseudo-labels solely for final predictions, our strategy extracts object-level features using pseudo-labels and optimizes them via contrastive learning, without requiring labels in the target domain. When combined with recent mean-teacher self-training methods, CMT leads to new state-of-the-art target-domain performance: 51.9% mAP on Foggy Cityscapes, outperforming the previously best by 2.1% mAP. Notably, CMT can stabilize performance and provide more significant gains as pseudo-label noise increases.

1. Introduction

Object detectors face a costly source–target domain gap, while mean-teacher adaptation is limited by noisy pseudo-labels. CMT unifies mean-teacher self-training with object-level contrastive learning to extract useful target-domain signals and improve adaptation.

  • Motivation: Object detectors often degrade when moving from curated source datasets to real-world target applications, where accurate human labels may be unavailable.Unsupervised domain adaptation aims to improve target-domain performance while minimizing human supervision.
  • Mean-teacher self-training: Mean-teacher adaptation uses teacher-generated target pseudo-labels to supervise the student, while updating the teacher with the student’s exponential moving average.This teacher–student process gradually adapts the detector across domains.
  • Contrastive Mean Teacher: CMT integrates mean-teacher self-training with contrastive learning because their momentum-teacher and student-encoder structures provide stable targets and EMA updates.The framework combines the two paradigms rather than treating contrastive learning as a separate pretraining stage.
  • Object-level contrastive learning: Object-level contrastive learning adapts localized detector features, uses predicted classes from noisy pseudo-labels, and incorporates multi-scale features for additional learning signals.This addresses the fact that natural images contain multiple heterogeneous objects rather than one homogeneous instance.
  • Framework scope: CMT is a drop-in enhancement compatible with existing mean-teacher methods, preserving their training pipelines while improving feature learning.The authors report combinations with methods such as Adaptive Teacher and Probabilistic Teacher.
  • Results: 51.9% mAP improves Cityscapes-to-Foggy-Cityscapes adaptation from 49.8% mAP, while CMT’s gains become more pronounced as pseudo-label noise increases.The contribution passage reports a 2.1% mAP improvement over the previously best result.

2. Related Work

Prior domain-adaptive detectors use adversarial training, image translation, or mean-teacher self-training, while contrastive learning develops unlabeled representations through instance or cluster discrimination. CMT applies object-level contrastive learning directly to domain-adaptive detectors using noisy pseudo-labels and multi-scale features.

  • Unsupervised domain adaptation: Adversarial feature learning aligns domains with a discriminator, whereas image-to-image translation synthesizes source-like or target-like images to mitigate domain gaps.These approaches explicitly model or transform the domain shift.
  • Mean-teacher methods: Mean Teacher has been extended to unsupervised domain adaptation for detection through methods including UMT, Adaptive Teacher, and Probabilistic Teacher.These methods incorporate image translation, weak–strong augmentation, adversarial training, or uncertainty-guided self-training.
  • Contrastive learning: Contrastive learning supports self-supervised representation learning through instance or cluster discrimination, often using large batches, memory banks, asymmetric architectures, or clustering.It has also been extended from image-level pretraining toward masks, objects, and regions for downstream vision tasks.
  • CMT distinction: CMT differs by using noisy pseudo-labels to construct contrastive pairs and optimizing multi-scale object features within domain-adaptive object detectors.The paper positions this as distinct from typical object-level contrastive pretraining.

3. Approach

CMT unifies mean-teacher self-training with Momentum Contrast by adding object-level contrastive feature adaptation to unsupervised domain adaptation for object detection. It uses teacher-generated pseudo-labels to align teacher and student object features while retaining the mean-teacher training structure.

  • Mean-teacher self-training: Mean-teacher self-training uses teacher-generated target-domain pseudo-labels to supervise the student, while the teacher is updated as an EMA of student weights.The teacher provides more stable learning targets and is also used for evaluation.
  • Aligning the paradigms: Mean-teacher self-training and Momentum Contrast share two jointly learned networks, EMA teacher updates, and unlabeled augmented inputs.The student is optimized with detection or contrastive loss, while the teacher supplies stable targets.
  • Contrastive Mean Teacher: CMT integrates contrastive learning as a drop-in feature-adaptation enhancement within the mean-teacher framework.The main training structure remains mean-teacher self-training, with object-level contrastive learning added to it.
  • Object-level contrastive learning: Object-level features are extracted from teacher and student feature maps using pseudo-label bounding boxes and aligned across transformed views.Features are extracted at multiple backbone scales, and the resulting contrastive losses are optimized together.
  • Object-level contrastive learning: The contrastive objective compares student and teacher features, using same-class object pairs as positives and other objects in the batch as negatives.The approach finds that within-one-image-batch features are adequate instead of requiring a large cross-iteration memory bank.

4. Experiments

CMT is evaluated across weather, camera, and image-style domain shifts by adding object-level contrastive objectives to mean-teacher adaptation pipelines. It consistently improves existing methods, reaches state-of-the-art results, and becomes more robust to pseudo-label noise.

  • Experimental setup: CMT is evaluated on Cityscapes, Foggy Cityscapes, KITTI, Pascal VOC, and Clipart1k across weather, camera, and image-style domain adaptation tasks.Experiments use target-domain validation data after unsupervised adaptation with source- and target-domain training splits.
  • Experimental setup: CMT adds object-level contrastive learning objectives to the original Adaptive Teacher and Probabilistic Teacher adaptation pipelines.The experiments use AT and PT as state-of-the-art mean-teacher base methods.
  • Benchmark results: 51.9% mAP is achieved on Foggy Cityscapes, while CMT improves AT by +1.0% mAP on the “0.02” split and +2.1% mAP on the “All” split.CMT consistently improves both PT and AT, with a larger gain when more unlabeled data are available.
  • Benchmark results: 4.1% AP is the additional target-domain improvement from CMT on KITTI →Cityscapes, bringing performance close to an Oracle trained with target-domain labels.This benchmark studies adaptation across different camera configurations and cities.
  • Benchmark results: 1.3% mAP is the improvement of AT + CMT over AT on Pascal VOC →Clipart1k, while CMT exceeds the previous best TIA by 0.7% mAP.The resulting task concerns adaptation from realistic images to artistic images without human supervision.
  • Analysis and ablation: At pseudo-label noise level 1.0, CMT reduces performance standard deviation from 1.4% to 0.4% and increases mean gain from +2.0% to +2.8% as noise rises from 0.0 to 1.0.The analysis attributes these effects to object-level contrastive learning combating noisy pseudo-labels.

5. Conclusion

The paper presents CMT as an integrated framework aligning contrastive learning with mean-teacher self-training. Experiments report consistent gains from object-level contrastive learning and state-of-the-art performance across multiple benchmarks.

  • CMT integrates contrastive learning and mean-teacher self-training into an unsupervised domain adaptation framework.
  • On Foggy Cityscapes, CMT helps correct mis-classification, false-negative, false-positive, and localization errors relative to AT.
  • CMT’s object-level contrastive learning consistently improves several existing methods and achieves state-of-the-art results on multiple benchmarks.

A. Additional Visualization Results

Additional visualizations compare AT with AT + CMT on artistic and adverse-weather target domains. The CMT-enhanced detector corrects incorrect predictions in the displayed examples.

  • Pascal VOC →Clipart1k: Figure 5 compares AT and AT + CMT on Pascal VOC →Clipart1k, with CMT correcting incorrect predictions in the displayed image pairs.
  • Cityscapes →Foggy Cityscapes: Figure 6 compares AT and AT + CMT on Cityscapes →Foggy Cityscapes, with CMT correcting incorrect predictions in the displayed image pairs.

B. Pseudo-code for Contrastive Mean Teacher

Algorithm 1 integrates teacher-generated pseudo-labels, object-level contrastive learning, supervised and unsupervised detection losses, and EMA teacher updates into the CMT training loop.

  • Optimization: The combined loss is optimized through a student SGD step, after which the teacher is updated using the student’s weights in the mean-teacher loop.The pseudocode identifies optimization as the final stage after the total loss is computed.
  • Algorithm inputs and variables: CMT uses student and teacher object detectors, denoted D(·; θQ) and D(·; θK), as its main inputs and outputs.Student-related variables use Q, while teacher-related variables use K.
  • Data loading and augmentation: Each iteration loads labeled source-domain images with boxes and classes, alongside unlabeled target-domain images.The student applies strong augmentation, while the teacher applies weak augmentation.
  • Pseudo-labels and contrastive branch: The teacher generates and optionally post-processes target-domain pseudo-labels, which are then used to locate teacher and student object-level features for contrastive learning.The algorithm obtains multi-scale feature maps, counts pseudo-labeled objects, and computes the contrastive loss from corresponding object-level features.
  • Loss computation: CMT computes unsupervised target-domain and supervised source-domain detection losses before combining them with the contrastive loss.The total loss is weighted as L = λcontrastLcontrast + λunsup detLunsup det + λsup detLsup det.
Loading 2305.03034v1…