PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross Domain Object Detection by Target-Perceived Dual Branch Distillation

Mengzhe He, Yali Wang, Jiaxi Wu, Yiru Wang, Hanqing Li, Bo Li, Weihao Gan, Wei Wu, Yu Qiao

arXiv:2205.01291v1cs.CV

TL;DR

Cross-domain object detection must contend with domain shift and a lack of target-domain instance annotations, while existing approaches often focus on only one. TDD combines target-perceived dual detection branches with self-distillation, and the paper reports that it outperforms state-of-the-art methods across multiple benchmarks.

  • Problem

    Cross-domain object detection faces domain shift and scarce target-domain instance annotations, while many existing approaches focus on only one of these coupled challenges.

    Source: §1 · p. 1

  • Method

    TDD combines source-adaptive and target-like detection branches, using a Target Proposal Perceiver and dual-branch self-distillation to integrate object knowledge across domains.

    Source: §1 · p. 2

  • Results

    TDD outperforms state-of-the-art methods on multiple benchmarks, according to the paper’s reported experiments.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The paper concludes that its target proposal perceiver and dual-branch self-distillation tackle domain shift and label deficiency together.

    Source: §5 · p. 8

Abstract

from arXiv · show

Cross domain object detection is a realistic and challenging task in the wild. It suffers from performance degradation due to large shift of data distributions and lack of instance-level annotations in the target domain. Existing approaches mainly focus on either of these two difficulties, even though they are closely coupled in cross domain object detection. To solve this problem, we propose a novel Target-perceived Dual-branch Distillation (TDD) framework. By integrating detection branches of both source and target domains in a unified teacher-student learning scheme, it can reduce domain shift and generate reliable supervision effectively. In particular, we first introduce a distinct Target Proposal Perceiver between two domains. It can adaptively enhance source detector to perceive objects in a target image, by leveraging target proposal contexts from iterative cross-attention. Afterwards, we design a concise Dual Branch Self Distillation strategy for model training, which can progressively integrate complementary object knowledge from different domains via self-distillation in two branches. Finally, we conduct extensive experiments on a number of widely-used scenarios in cross domain object detection. The results show that our TDD significantly outperforms the state-of-the-art methods on all the benchmarks. Our code and model will be available at https://github.com/Feobi1999/TDD.

1. Introduction

Cross-domain object detection is challenged by domain shift and scarce target-domain annotations, while prior approaches often address only one difficulty. TDD combines target-aware dual branches and distillation, and the paper reports state-of-the-art results on widely used benchmarks.

1. Introduction

  • In adverse-weather examples, UBT misses fogged objects and GPA makes some incorrect predictions, while TDD predicts boxes and categories more accurately.The caption identifies a motorcycle and a person as examples of GPA’s incorrect predictions.Source: Figure 1 · p. 1
  • Domain shift and scarce target-domain object annotations both deteriorate target-domain detection performance.Instance-level predictions are sensitive to variation in image styles and contents, while obtaining target-domain annotations is costly and labor-intensive.Source: §1 · p. 1
  • Earlier approaches often addressed either domain shift or label deficiency, leaving their combined challenge insufficiently handled.Adversarial adaptation can have unstable optimization and limited discrimination, while self-training can struggle with complex domain shifts.Source: §1 · pp. 1–2
  • TDD uses source-adaptive and target-like detection branches with a Target Proposal Perceiver and a three-step self-distillation strategy.The perceiver uses iterative cross-attention to extract target-domain proposal contexts; joint-domain pretraining, cross-domain distillation, and dual-teacher refinement integrate knowledge across domains.Source: §1 · p. 2
  • TDD outperforms state-of-the-art methods on widely used benchmarks, according to the paper’s summary of its experiments.The introduction reports a large margin but gives no benchmark-specific values here.Source: §1 · p. 2

2. Related Work

Related work spans detector architectures, adversarial domain adaptation, and semi-supervised approaches to target-label scarcity. The paper distinguishes TDD by combining cross-domain perception with a dual-branch framework.

2. Related Work

  • Object detectors include two-stage and one-stage methods, with anchor-free and transformer-based approaches also emerging.The cited passage groups prior object-detection work into these broad categories.Source: §2 · p. 2
  • Domain-adaptation work commonly aligns features adversarially, with variants targeting different levels, regions, or foreground-background distinctions.The passage describes image- and instance-level classifiers, local and global alignment, multi-level alignment, and other feature constraints.Source: §2 · p. 2
  • Other cross-domain approaches address inaccurate target labels through refined pseudo-labels, consistency learning, or cross-domain distillation.The paper contrasts its dual-branch framework and cross-domain perceiver with the approach in [9].Source: §2 · p. 2
  • Semi-supervised detection methods use consistency or pseudo-labeling, but the cited methods do not account for domain differences.The paper notes that this omission limits their detection performance in cross-domain settings.Source: §2 · p. 2

3. Proposed Methods

TDD addresses domain shift and scarce target annotations with two detection branches, target-guided proposal features, and dual-branch self-distillation. Its training uses labeled source and target-like images, then target-image pseudo-labels refined through teacher–student updates.

3.1. Overview

  • TDD combines source-adaptive and target-like detection branches with a shared proposal extractor; Fourier style transfer creates target-like source images with inherited labels.Source and target-like proposals train their corresponding branches, while real target-image proposals enter both branches.Source: §3.1 · p. 3, Figure 2 · p. 3

3.2. Target Proposal Perceiver

  • Multi-Head Proposal Cross Attention uses SA proposals as queries and TL proposals as keys and values to aggregate target-like contexts for SA.Iterative application progressively enhances source proposal features with target-like contexts; geometry weights encode positional similarity between proposals.Source: §3.2 · p. 4

3.3. Dual-Branch Self Distillation

  • Dual-Branch Self Distillation first jointly pretrains on labeled source and target-like images, then trains the student on target-image pseudo-labels from both teacher branches.The teacher is gradually refined from the student via EMA, and the process iterates; only the SA teacher branch supplies detections at inference.Source: §3.3 · pp. 4–6

4. Experiments

Experiments evaluate TDD across adverse-weather, synthetic-to-real, and cross-camera domain shifts. TDD outperforms prior methods in the reported benchmarks, and ablations assess its dual-branch structure, proposal attention, and self-distillation.

4.1. Implementation details

  • Training uses Faster R-CNN with VGG16 or Res50, and runs 9,000 joint-domain pretraining iterations followed by 16,000 distillation and refinement iterations.Each mini-batch includes one source-domain and one target-domain image per GPU.Source: §4.1 · p. 6

4.2. Adverse Weather Conditions Adaptation

  • For C→F, Cityscapes is the source and unlabeled Foggy Cityscapes at fog level β = 0.02 is the target; Foggy Cityscapes validation is used for evaluation.The authors report TDD as competitive with two oracle models trained using labeled target data or labeled source-and-target data.Source: §4.2 · p. 6
  • 41.8% mAP for VGG-based MeGA, with TDD reporting a +1.3% improvement; TDD also outperforms prior methods in synthetic-to-real and cross-camera transfers.The reported cross-camera comparisons include KITTI-to-Cityscapes and Cityscapes-to-BDD100K.Source: Table 1 · p. 6, §4.2 · p. 6, §4.3 · p. 7

4.3. Synthetic to Real Adaptation

  • +4.4% over VGG-based EPM and +10.7% over Res50-based SFA on car AP for Sim10k-to-Cityscapes adaptation.The evaluation uses the common class car.Source: §4.3 · pp. 6–7, Table 2 · p. 7

4.4. Cross Camera Adaption

  • For KITTI-to-Cityscapes, TDD outperforms the VGG-based approach by 3.8% and the R50-based approach by 1.9%; on BDD100K it surpasses prior works by a large margin.The source-only results on BDD100K increase by 9.8% with the R50 backbone.Source: §4.4 · p. 7, Table 3 · p. 7

4.5. Ablation Studies and Analysis

  • Adding target and target-like images improves performance step by step, and the dual-branch experiment outperforms all single-branch methods.The authors say the dual-branch framework retains useful source knowledge while exploring target information simultaneously.Source: §4.5 · p. 7, Table 4 · p. 7
  • MHPCA improves results, and the asymmetric design—cross-attention in the source-adaptive branch only—performs best among the tested attention variants.The authors attribute the need for source-adaptive-branch cross-attention to that branch's lack of target-domain knowledge.Source: §4.5 · pp. 7–8, Table 5 · p. 7
  • Joint-Domain Pretraining, Cross-Domain Distillation, and Dual Teacher Refinement each improve on the preceding step; a small EMA rate α = 0.96 reduces performance.Qualitative C→F results show TDD localizing and classifying fog-obscured objects more accurately, with target-like proposals providing contextual clues such as a motorcycle and two person proposals for a rider.Source: Table 6 · p. 7, §4.5 · p. 8, Figure 5 · p. 8

5. Conclusion

TDD combines a target proposal perceiver with dual-branch self-distillation to address domain shift and label deficiency together in cross-domain object detection. The authors report that it surpasses existing state-of-the-art models across multiple benchmarks.

  • TDD surpasses existing state-of-the-art models on multiple cross-domain object detection benchmarks.Source: §5 · p. 8
  • TDD uses a target proposal perceiver and dual-branch self-distillation to address domain shift and label deficiency together.Source: §5 · p. 8
  • The work evaluates TDD on multiple benchmarks for cross-domain object detection.Source: §5 · p. 8

Related papers

Loading 2205.01291v1…