Source-linked AI summary
Domain Adaptive Object Detection via Asymmetric Tri-way Faster-RCNN
Zhenwei He, Lei Zhang
TL;DR
Unsupervised domain adaptive object detection must transfer from labeled source data to unlabeled target data despite risks from parameter-shared alignment. ATF uses an asymmetric tri-way Faster-RCNN with an independent, source-supervised ancillary net, and reports stronger target-domain performance while preserving source-domain discrimination.
Problem
Parameter-shared adversarial alignment can collapse source-domain discrimination because reliable labeled-source features are aligned toward vulnerable unlabeled-target features.
Method
ATF adds an independent ancillary net supervised by source labels alongside two parameter-shared chief streams, using shared proposal and pooled-feature processing for detection.
Results
ATF reports higher target performance than DAF and MAF on Cityscapes-to-Foggy-Cityscapes transfer, with gains of 11.1% and 4.7%, respectively.
Takeaways & Limitations
The asymmetric structure is reported to prevent source-domain collapse, preserve source-feature discrimination, and improve target-domain detection across tested settings.
Abstract
from arXiv · showhide
Conventional object detection models inevitably encounter a performance drop as the domain disparity exists. Unsupervised domain adaptive object detection is proposed recently to reduce the disparity between domains, where the source domain is label-rich while the target domain is label-agnostic. The existing models follow a parameter shared siamese structure for adversarial domain alignment, which, however, easily leads to the collapse and out-of-control risk of the source domain and brings negative impact to feature adaption. The main reason is that the labeling unfairness (asymmetry) between source and target makes the parameter sharing mechanism unable to adapt. Therefore, in order to avoid the source domain collapse risk caused by parameter sharing, we propose an asymmetric tri-way Faster-RCNN (ATF) for domain adaptive object detection. Our ATF model has two distinct merits: 1) A ancillary net supervised by source label is deployed to learn ancillary target features and simultaneously preserve the discrimination of source domain, which enhances the structural discrimination (object classification vs. bounding box regression) of domain alignment. 2) The asymmetric structure consisting of a chief net and an independent ancillary net essentially overcomes the parameter sharing aroused source risk collapse. The adaption safety of the proposed ATF detector is guaranteed. Extensive experiments on a number of datasets, including Cityscapes, Foggy-cityscapes, KITTI, Sim10k, Pascal VOC, Clipart and Watercolor, demonstrate the SOTA performance of our method.
1 Introduction
Unsupervised domain adaptive object detection transfers knowledge from labeled source data to unlabeled target data, but parameter-shared adversarial alignment can damage source discrimination. ATF addresses this with an asymmetric tri-way architecture and an ancillary net that supports target adaptation while preserving source structure.
- Motivation: Domain shift from changing environments causes conventional object detectors to degrade, while labeling every target domain is impractical.The task therefore transfers knowledge from a label-rich source domain to a label-agnostic target domain without target training labels.
- Approach: ATF learns ancillary target features that help make target representations both domain invariant and class separable.These features contribute to a new classifier decision boundary for source and target data.
- Motivation: Adversarial alignment can distort domain distributions and destroy structural discrimination because target data are unlabeled.In detection, reliable source features may be aligned toward vulnerable target features, increasing source-collapse risk and harming classification and bounding-box regression.
- Approach: ATF uses two parameter-shared chief streams and an independent ancillary stream, with all three streams using a shared RPN and ROI-Align for final detection.The chief net receives domain-adversarial and source-label detection supervision, whereas the ancillary net receives source-label detection supervision.
- Results: 11.1% and 4.7% higher target performance are reported for ATF than DAF and MAF, respectively, on transfer from Cityscapes to Foggy Cityscapes.The comparison also evaluates an upper target-performance bound using network features with ground-truth target labels.
2 Related Work
Related work frames domain adaptation as reducing cross-domain disparity and extends that idea to object detection under unconstrained scene changes. Prior detection methods align image-level and instance-level features and motivate domain-invariant representations for target detection.
- Object Detection: The paper selects Faster-RCNN as its base detector, a two-stage architecture that uses an RPN to generate region proposals.Related detectors include one-stage models optimized for computational efficiency and two-stage models that first generate proposals.
- Domain Adaptation: Domain adaptation bridges gaps between domains across computer-vision tasks such as image classification and object segmentation.Earlier approaches include minimizing disparity estimates such as maximum mean discrepancy.
- Domain Adaptive Object Detection: Domain adaptive object detection addresses domain shift in unconstrained scenes by transferring knowledge between related source and target domains.The task was first proposed with alignment at both image and instance levels.
3 The Proposed ATF Approach
ATF uses an asymmetric tri-way Faster-RCNN to align target and source-related features while preserving source-domain detection reliability. Its chief and independent ancillary nets jointly perform global and local domain alignment with source-supervised detection losses.
- 3.1 Network Architecture of ATF: ATF extends Faster-RCNN with two shared chief streams for source and target data plus an independent ancillary stream fed only source data during training.All three streams share Faster-RCNN's region proposal network and use ROI-Align-pooled features for detection.
- 3.2 Principle of the Chief Net: Domain adversarial confusion aligns ancillary-net features with target-stream features, connecting target knowledge in the chief net with source knowledge in the ancillary net.The mechanism is applied to reduce domain discrepancy between the two streams.
- 3.2 Principle of the Chief Net: ATF performs both global image-level alignment across convolutional blocks and local object-level alignment on ROI-Align-pooled features.The global module targets low-level convolutional representations, while the local module aligns pooled foreground and background object features.
- 3.2 Principle of the Chief Net: The chief net bounds domain discrepancy, while the ancillary net uses source supervision to bound empirical source risk and prevent the source stream from becoming uncontrolled.The ancillary net is used instead of the chief net's source stream because parameter sharing can make source empirical risk out of control under unlabeled target training.
- 3.4 Training Loss of Our ATF: The total training objective combines domain alignment with source detection losses based on classification and bounding-box regression, weighted by hyper-parameter α.The model is trained end-to-end using stochastic gradient descent.
4 Experiments
Experiments evaluate ATF across visibility, camera, artistic, synthetic-to-real, and diagnostic settings. Across these settings, ATF achieves strong target-domain performance while preserving source-domain discrimination and improving proposal coverage.
- Experimental setup: ATF is evaluated on Cityscapes, Foggy Cityscapes, KITTI, SIM10k, Pascal VOC, Clipart, and Watercolor using established domain-adaptive detection comparisons.The experiments use VGG-16 or ResNet-101 backbones, with fully labeled source data and completely unlabeled target data.
- Different visibility and cameras: ATF achieves the best performance on both KITTI-to-Cityscapes and Cityscapes-to-KITTI tasks, supporting adaptation across camera-induced domain shifts.The evaluation uses target-domain car AP as the reported metric.
- Large domain shift: 42.1% mAP is achieved by ATF on Pascal VOC-to-Clipart, outperforming all compared models.This setting transfers from real images to comic images with the same 20 categories.
- Large domain shift: ATF achieves the best performance among compared models on Pascal VOC-to-Watercolor and outperforms all compared models when transferring from SIM10k to Cityscapes.The latter evaluates car AP for synthetic-to-real adaptation.
- Analysis and discussion: 57.1% of ground-truth boxes are covered by ATF’s RPN, while generated-ROI classification exceeds 90% for all tested models and ground-truth classification accuracy sharply drops.These results identify proposal recall as important in cross-domain detection and associate ATF’s stronger recall with its asymmetric structure and ancillary target features.
- Analysis and discussion: ATF maintains higher source-domain mAP than parameter-shared MAF during training and shows the best qualitative target-domain detection results on Foggy Cityscapes.The analysis attributes source-risk prevention and preserved structural discrimination to the asymmetric structure.
5 Conclusions
The paper proposes ATF, an asymmetric tri-way network for unsupervised domain adaptive object detection, to address risks from parameter-shared Siamese transfer networks. Its independent, source-supervised ancillary net preserves source-feature discrimination and improves target-feature reliability, while experiments report state-of-the-art results across benchmark datasets.
- 5 Conclusions: ATF uses an asymmetric structure to alleviate labeling unfairness between labeled source and unlabeled target domains.The chief and independent ancillary nets avoid relying on a fully parameter-shared Siamese transfer network.
- 5 Conclusions: The source-supervised ancillary net preserves structural discrimination in source features and promotes target-domain feature reliability.It supports joint training of the chief and ancillary nets while retaining discrimination relevant to domain adaptation.
- 5 Conclusions: ATF is implemented end-to-end and achieves state-of-the-art results on multiple benchmark datasets.The reported benchmarks include Cityscapes, Foggy-cityscapes, KITTI, Sim10k, Pascal VOC, Clipart, and Watercolor.