PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection

Dayan Guan, Jiaxing Huang, Aoran Xiao, Shijian Lu, Yanpeng Cao

arXiv:2103.00236v2cs.CV

TL;DR

Unsupervised domain-adaptive detection must transfer detectors from labeled source data to unlabeled target data despite domain bias and harmful global alignment. UaDAN uses uncertainty-aware adversarial learning with a curriculum from image-level to instance-level alignment, and reports superior performance across four challenging cross-domain tasks, including 41.1% mAP on Cityscapes → Foggy Cityscapes.

  • Problem

    Domain-adaptive detectors using global adversarial alignment may misalign already well-aligned samples, while collecting target annotations is costly.

  • Method

    UaDAN uses uncertainty to weight adversarial learning by sample alignment and progressively adapts image-level features before instance-level features.

  • Results

    UaDAN achieves superior performance across four challenging cross-domain detection tasks, including 41.1% mAP across all classes on Cityscapes → Foggy Cityscapes.

  • Takeaways & Limitations

    The method protects well-aligned features while focusing adaptation on uncertain samples and harder detection stages within the evaluated domain-adaptation setting.

Abstract

from arXiv · show

Unsupervised domain adaptive object detection aims to adapt detectors from a labelled source domain to an unlabelled target domain. Most existing works take a two-stage strategy that first generates region proposals and then detects objects of interest, where adversarial learning is widely adopted to mitigate the inter-domain discrepancy in both stages. However, adversarial learning may impair the alignment of well-aligned samples as it merely aligns the global distributions across domains. To address this issue, we design an uncertainty-aware domain adaptation network (UaDAN) that introduces conditional adversarial learning to align well-aligned and poorly-aligned samples separately in different manners. Specifically, we design an uncertainty metric that assesses the alignment of each sample and adjusts the strength of adversarial learning for well-aligned and poorly-aligned samples adaptively. In addition, we exploit the uncertainty metric to achieve curriculum learning that first performs easier image-level alignment and then more difficult instance-level alignment progressively. Extensive experiments over four challenging domain adaptive object detection datasets show that UaDAN achieves superior performance as compared with state-of-the-art methods.

I. INTRODUCTION

UaDAN addresses domain bias in object detection by adapting detectors from labeled source data to unlabeled target data while avoiding harmful global alignment. It uses uncertainty-aware sample alignment and progressive image-to-instance adaptation.

  • I. INTRODUCTION: Object detectors can suffer drastic performance drops on new datasets, while annotating target data is time-consuming and unscalable.
  • I. INTRODUCTION: Traditional adversarial adaptation aligns global distributions with equal sample weighting, which may pull well-aligned samples toward incorrect classes.
  • I. INTRODUCTION: UaDAN estimates sample uncertainty and adaptively assigns smaller adversarial-loss weights to well-aligned samples and larger weights to poorly aligned samples.
  • I. INTRODUCTION: UaDAN progressively aligns easier image-level features before harder instance-level features, reducing image-level misalignment and later error accumulation.
  • I. INTRODUCTION: The method is built on Faster R-CNN and targets proposal generation and instance detection without requiring target-domain annotations.

C. Curriculum learning

The method creates a curriculum over the two subtasks of domain-adaptive detection, combining uncertainty-aware alignment with a progression from image-level to instance-level adaptation.

  • C. Curriculum learning: Curriculum learning in UaDAN orders domain-adaptation subtasks from easier image-level alignment to harder instance-level alignment.
  • C. Curriculum learning: The framework trains on labeled source images and unlabeled target images, producing image-level and instance-level features for domain adaptation.
  • C. Curriculum learning: The detector uses Faster R-CNN components—DCNN, RPN, and RCNN—to extract features, generate proposals, and classify candidate boxes.
  • C. Curriculum learning: Image-level adaptation weights adversarial loss by proposal entropy, emphasizing high-entropy under-aligned features while less affecting low-entropy well-aligned features.
  • C. Curriculum learning: Instance-level curriculum loss filters detection entropy through a gate controlled by proposal entropy and activates harder alignment only below a truncation threshold.

C. Training Objective

UaDAN combines supervised source detection with image-level uncertainty-aware adversarial learning and instance-level uncertainty-guided adversarial learning.

  • C. Training Objective: The training objective combines a supervised detection loss with image-level and instance-level adaptation losses.
  • C. Training Objective: During forward processing, source and target image pairs produce proposals, detections, image-level features, and ROI-pooled instance-level features.
  • C. Training Objective: The supervised loss distills detection knowledge from labeled source images using region proposals and object detections generated by the network.

2) Image-level uncertainty-aware adversarial learning:

UaDAN performs image-level uncertainty-aware adversarial alignment using entropy-based sample weighting. Its broader curriculum framework delays instance-level alignment until image-level features are sufficiently well aligned.

  • 2) Image-level uncertainty-aware adversarial learning:: Image-level alignment uses source and target features, region proposals, and a domain classifier in an uncertainty-aware adversarial loss.The loss is formulated from image-level features, proposals, and domain predictions.
  • 2) Image-level uncertainty-aware adversarial learning:: Proposal entropy estimates prediction uncertainty, using the lowest proposal entropy at each location to assess confidence.The entropy function follows information-theoretic uncertainty estimation.
  • 2) Image-level uncertainty-aware adversarial learning:: UaDAN progressively activates instance-level alignment only after image-level features become well aligned, indicated by low proposal entropy.A gate function uses threshold ξ to determine whether image-level representations are sufficiently aligned.
  • 2) Image-level uncertainty-aware adversarial learning:: The training objective jointly optimizes the feature generator and image- and instance-level domain classifiers until the UaDAN loss converges.The overall objective combines the domain-adaptation components and is optimized simultaneously.

D. Analysis

The analysis explains how uncertainty weighting protects confident, well-aligned samples while emphasizing under-aligned ones. It also motivates curriculum learning because early instance alignment can damage semantic structure and cause negative transfer.

  • D. Analysis: Entropy-based weighting protects low-entropy, well-aligned features from excessive re-alignment while focusing adversarial learning on high-entropy features.Well-aligned features produce confident predictions, whereas under-aligned features tend to have higher entropy.
  • D. Analysis: Traditional adversarial alignment can misalign a correctly classified target sample to an incorrect class under negative transfer.The analysis describes this as a consequence of assuming matching source and target joint distributions despite their natural discrepancy.
  • D. Analysis: Uncertainty-aware adversarial learning mitigates this negative transfer by reducing the alignment pressure on well-aligned target samples.The method adjusts the loss weight of the well-aligned sample rather than applying uniform alignment.
  • D. Analysis: The curriculum gate activates instance-level alignment only when corresponding image-level representations have low entropy.This avoids aligning nonsensical instances during early training, when predictions are inaccurate and source knowledge is still being transferred.

A. Experimental Setup

The experiments follow the standard unsupervised domain-adaptive detection protocol, pairing labelled source data with unlabelled target data. They use six public datasets and restrict training and evaluation to shared object categories.

  • A. Experimental Setup: Each adaptation task uses labelled source training data and unlabelled target training data, with the target validation set used for evaluation.Only categories shared by source and target datasets are included in training and evaluation.
  • A. Experimental Setup: The experimental suite contains six public datasets: Mapillary Vistas, Cityscapes, Foggy Cityscapes, PASCAL VOC, Clipart, and SIM10k.The datasets cover autonomous-driving, real-world, stylized, and synthetic imagery.
  • A. Experimental Setup: Mapillary Vistas contains 18,000 training and 2,000 validation images across 37 object categories, while Cityscapes contains 2,975 training and 500 validation images across 8 categories.Mapillary image resolution varies from 768 × 1024 to 4000×6000.
  • A. Experimental Setup: PASCAL VOC provides bounding-box annotations for 20 classes, Clipart1k contains 1,000 images across the same 20 categories, and SIM10k contains 10,000 synthetic car images.Clipart1k has 800 training and 200 validation images; SIM10k uses automatically generated labels.

2) Implementation details:

UaDAN is evaluated across four domain-shift scenarios using standard quantitative and qualitative comparisons with source-only and state-of-the-art adaptation methods.

  • Implementation details:: The evaluation covers cross-camera, weather, dissimilar real-to-artistic, and synthetic-to-realistic adaptation scenarios, comparing UaDAN with established domain-adaptive methods.The datasets include Cityscapes→Mapillary Vistas, Cityscapes→Foggy Cityscapes, PASCAL VOC→Clipart1k, and Sim10k→Cityscapes.
  • Implementation details:: Quantitative comparisons report per-category AP and all-class mAP on the corresponding target-domain validation sets.Tables II–V cover the four adaptation tasks and specify the evaluated target validation sets.
  • Cross camera adaptation:: Qualitative results across four tasks show UaDAN detecting more true positives and fewer false positives than GPA at a 0.5 visualization threshold.Yellow arrows indicate true positives and red arrows indicate false positives in the visualizations.
  • Cross camera adaptation:: On Cityscapes→Mapillary Vistas, UaDAN achieves 32.7% mAP and exceeds other methods by over 2.7% AP for difficult classes such as train.The train class has only 7.1% AP for Source only, while UaDAN focuses on under-aligned features and limits effects on well-aligned features.
  • Cross camera adaptation:: Feature-distribution visualization reports within-class and between-class variances that are highly consistent with the qualitative detection results.The analysis uses t-SNE visualizations of target-domain features.

2) Weather adaptation:

The weather-adaptation experiment transfers detectors from normal-weather Cityscapes images to foggy Foggy Cityscapes images and evaluates them on the target validation set.

  • Weather adaptation:: The experiment addresses performance degradation when detectors trained on normal-weather images are applied to foggy, adverse-weather images.The study specifically examines normal-weather to foggy-weather adaptation.
  • Weather adaptation:: UaDAN achieves the best detection accuracy on Cityscapes→Foggy Cityscapes, reaching 41.1% mAP across all classes.The evaluation uses Cityscapes as source, Foggy Cityscapes as target, and its validation set for evaluation.
  • Weather adaptation:: For difficult classes such as train, UaDAN exceeds other methods by over 1.6% AP, while Source only achieves 9.6% AP.The result is consistent with the cross-camera experiment and supports focusing alignment on under-aligned samples.
  • Weather adaptation:: The results further verify that UaDAN can focus on aligning under-aligned samples.This conclusion is stated alongside the weather-adaptation comparison.

4) Synthetic-to-realistic adaptation:

The synthetic-to-realistic experiment adapts from Sim10k synthetic images to Cityscapes real images for the shared car category, while also examining qualitative errors, feature distributions, and efficiency.

  • Synthetic-to-realistic adaptation:: UaDAN achieves the best synthetic-to-realistic detection accuracy, reaching 48.6% mAP on Sim10k→Cityscapes.The source and target domains share only the car category, and evaluation uses the Cityscapes validation set.
  • Synthetic-to-realistic adaptation:: Across four target datasets, UaDAN identifies more correct objects and fewer false positives than GPA in qualitative comparisons.The visualizations also show that brute-force alignment can map well-aligned features to incorrect categories.
  • Synthetic-to-realistic adaptation:: Feature-distribution variances are highly consistent with the detection results, linking target-domain feature organization to observed object-detection behavior.The analysis uses t-SNE and computes within-class and between-class variances.
  • Synthetic-to-realistic adaptation:: UaDAN reduces misalignment errors by producing fewer false negatives for source-only-correct samples and more true positives for source-only errors.The authors attribute these patterns to protecting well-aligned features while focusing alignment on under-aligned features.
  • Synthetic-to-realistic adaptation:: Entropy computation adds less than 0.0003 second per training iteration, under 0.1% overhead, with no inference-time overhead.The method nevertheless outperforms traditional adversarial methods by over 4.3% AP across the four domain-adaptive tasks.
  • Synthetic-to-realistic adaptation:: Training and testing losses decrease rapidly during the first 10k iterations and fluctuate within a small range after 50k iterations.Losses are evaluated periodically during training.

D. Ablation Studies

Ablation studies show that uncertainty-aware alignment improves both image-level and instance-level adaptation, while curriculum learning further strengthens the complete UaDAN model. Sensitivity experiments across three tasks identify ξ = 0.5 as the consistently best setting.

  • UaAL consistently outperforms traditional AL at both image and instance levels by keeping well-aligned features less affected.
  • Image-level and instance-level adversarial alignment both outperform the source-only baseline, with image-level alignment providing the larger gain.
  • Image-level and instance-level UaAL are complementary, as removing UgCL still outperforms either UaAL component alone.
  • UaDAN outperforms its version without UgCL by a large margin, verifying the effectiveness of uncertainty-guided curriculum learning.
  • Across three domain adaptive detection tasks, the best performance is obtained consistently when ξ = 0.5.
Loading 2103.00236v2…