Source-linked AI summary
Collaborative Training between Region Proposal Localization and Classification for Domain Adaptive Object Detection
Ganlong Zhao, Guanbin Li, Ruijia Xu, Liang Lin
TL;DR
Object detectors face costly annotation demands and domain shift, while backbone-focused adaptation does not address the transferability of both RPN and RPC. The paper trains the two modules collaboratively using high-confidence outputs and uses low-confidence ROIs for discrepancy optimization. It reports improved transferability and performance over existing methods across multiple adaptation scenarios.
Problem
Object detection adaptation must address domain shift and the costly object-level annotations needed to train detectors, while accounting for both RPN and RPC rather than only backbone features.
Source: §1 · p. 1, §3.1 · p. 5Method
Collaborative self-training uses high-confidence outputs from RPN and RPC to train each other, while low-confidence ROIs support discrepancy optimization.
Source: §1 · p. 3, §3.3 · p. 8Results
The method improves transferability and outperforms existing methods across various domain-adaptation scenarios.
Source: §6 · p. 14Takeaways & Limitations
Treating RPN and RPC as complementary modules provides a way to adapt proposal generation as well as detection in two-stage detectors.
Source: §1 · p. 3, §6 · p. 14
Abstract
from arXiv · showhide
Object detectors are usually trained with large amount of labeled data, which is expensive and labor-intensive. Pre-trained detectors applied to unlabeled dataset always suffer from the difference of dataset distribution, also called domain shift. Domain adaptation for object detection tries to adapt the detector from labeled datasets to unlabeled ones for better performance. In this paper, we are the first to reveal that the region proposal network (RPN) and region proposal classifier~(RPC) in the endemic two-stage detectors (e.g., Faster RCNN) demonstrate significantly different transferability when facing large domain gap. The region classifier shows preferable performance but is limited without RPN's high-quality proposals while simple alignment in the backbone network is not effective enough for RPN adaptation. We delve into the consistency and the difference of RPN and RPC, treat them individually and leverage high-confidence output of one as mutual guidance to train the other. Moreover, the samples with low-confidence are used for discrepancy calculation between RPN and RPC and minimax optimization. Extensive experimental results on various scenarios have demonstrated the effectiveness of our proposed method in both domain-adaptive region proposal generation and object detection. Code is available at https://github.com/GanlongZhao/CST_DA_detection.
1 Introduction
Labeled data scarcity and domain shift make object detection adaptation difficult, and backbone-only alignment does not address the transferability of both RPN and RPC. The paper proposes collaborative training that uses each module to guide the other and applies discrepancy optimization to low-confidence ROIs.
1 Introduction
- Object detection adaptation is motivated by costly object-level annotations and performance degradation when pretrained detectors encounter domain gaps.The paper frames transfer from a labeled source domain to another domain as a common response to this data constraint.Source: §1 · p. 1
- Prior domain-adaptive detection work largely aligns backbone features, leaving RPN transferability insufficiently addressed.The discussion notes that Zhu et al. use RPN-generated regions but do not adapt the RPN itself, while Saito et al. align backbone layers.Source: §1 · p. 2
- RPC usually transfers better than RPN under large domain gaps, but its performance is limited when RPN fails to provide high-quality proposals.The comparison evaluates RPN and RPC recall on Sim10k-to-Cityscapes adaptation; RPC also shows less degradation from the Oracle model than the source-only model.Source: Figure 1 · p. 2, §1 · p. 3
- Collaborative self-training uses high-confidence outputs from RPN and RPC to train each other, with focal loss weighting confident ROIs.The paper also uses low-confidence ROIs to calculate foreground/background discrepancy, minimizing it in the backbone while RPN and RPC maximize it.Source: §1 · p. 3
- The paper identifies RPN transferability as important because backbone alignment alone cannot guarantee high-quality proposals for RPC.It also presents adapting Maximum Classifier Discrepancy to two-stage detection by focusing on ambiguous ROIs.Source: §1 · p. 3
2 Related Works
Related work spans object detection architectures and domain adaptation methods, while prior domain-adaptive detection approaches primarily align backbone features. The paper argues that detection adaptation must also account for proposal generation and region-level classification.
Object Detection
- Two-stage detectors include Faster RCNN and Mask RCNN, and are described as retaining higher performance than one-stage alternatives such as YOLO and SSD.Object detectors can be divided into one-stage and two-stage categories.Source: §2 · p. 4
Domain Adaptation
- Domain adaptation methods reduce domain gaps through feature or image alignment, including MMD, H∆H, adversarial training, and generative adversarial networks.The review also mentions subspace alignment, asymmetric metric learning, covariance matrix alignment, and entropy minimization.Source: §2 · p. 4
Domain Adaptation for Object Detection
- Prior domain-adaptive object detection methods include subspace alignment [21], global and instance-level alignment, pseudo-labeling, and backbone alignment strategies.Zhu et al. use RPN proposals to mine discriminative regions and align them.Source: §2 · pp. 4–5
- The review characterizes earlier detection adaptation methods as focusing on backbone-stream alignment regardless of RPN transferability.It distinguishes object detection adaptation from classification and segmentation by its inclusion of object proposal detection and region-level classification.Source: §2 · pp. 4–5
3 Method
The method adapts the detector’s region proposal network (RPN) and region proposal classifier (RPC) alongside backbone features. It combines confidence-weighted mutual training, discrepancy optimization on ambiguous ROIs, and RPN-weighted domain alignment.
3.1 Framework overview
- The framework combines collaborative self-training and RPN–RPC discrepancy optimization with a domain discriminator that aligns backbone features using RPN foreground scores.RPN generates ROIs for RPC, while the discriminator predicts domain labels from backbone features and weights its loss using foreground information.Source: §3.1 · p. 5
3.2 Collaborative self-training
- High-confidence RPC outputs train RPN, while high-confidence RPN outputs weight entropy minimization for RPC; ambiguous ROIs are reserved for discrepancy optimization.The method uses pseudo-labels from target images; the RPC-derived pseudo-labels need not include every object or specify each object’s class.Source: §3.1 · p. 5, §3.2 · pp. 6–7, Figure 2 · p. 6
3.3 Maximize discrepancy classifier on detectors
- MCD weights low-confidence ROIs more heavily for RPN–RPC discrepancy optimization, while the feature extractor minimizes discrepancy and the two modules maximize it alternately.ROIs between the decision boundaries are more likely to be misclassified; those farther from both boundaries are treated as reliable pseudo-labels.Source: §3.3 · p. 8, Figure 4 · p. 9
3.4 RPN weighted alignment
- RPN-weighted alignment uses foreground probabilities to focus the domain discriminator’s loss on locations more likely to contain foreground objects.The RPN foreground map may be rough initially and is refined during collaborative self-training.Source: §3.4 · p. 9
3.5 Overall objective
- The overall objective combines source-domain detection losses with target-domain adaptation losses and discriminator loss, with α, β, and γ controlling the trade-off.A gradient reversal layer flips gradient signs during back-propagation to implement the adversarial loss.Source: §3.5 · p. 10
4 Experiment
Experiments evaluate unsupervised domain adaptation across three shifts, using source images and annotations but only target images during training. The method improves on the existing best across all three reported shifts.
4 Experiment
- The unsupervised protocol uses source images and annotations but only target images during training, across three kinds of domain shifts.Source: §4 · p. 10
4.1 Implement details
- Experiments use Faster RCNN with VGG16 and compare against source-only Faster RCNN, DA-Faster, SWDA, and SCDA.Training proceeds in three stages: source pretraining, 10,000 iterations with detection and adversarial losses, then 6,000 iterations with all loss terms.Source: §4.1 · pp. 10–11
4.2 Domain Adaptation for Detection
- The method improves on the existing best by 1.6% mAP for Cityscapes→FoggyCityscapes, 1.5% mAP for Sim10k→Cityscapes, and 1.1% mAP for KITTI→Cityscapes.Source: Table 1 · p. 11, §4.2 · p. 11
5 Ablation Study
The ablations assess component contributions and proposal quality, while visualizations show weighted alignment focusing on objects across domain shifts.
Effectiveness of RPN adaptation
- Proposal-coverage distributions move closer to Oracle with CST, reducing zero-coverage boxes and improving proposals at IOU≥0.5 versus naive alignment.Proposal coverage is the largest IOU between a target-domain ground-truth box and any detected proposal.Source: §5 · pp. 11–12, Figure 5 · p. 12
Effectiveness of Different Components
- 9.94% w.r.t AP: all three components together improve the source-only baseline; CST alone gains 7.76% AP, from 34.57% to 42.33%.The full model gains an additional 2.18% AP over the CST-only version.Source: §5 · p. 13, Table 4 · p. 14
Visualization of detection results
- Across Sim10k→Cityscapes, KITTI→Cityscapes, and Cityscapes→FoggyCityscapes, the model localizes and classifies objects while weighted alignment assigns less loss weight to sky and roads.Brighter regions in the weight map indicate higher attention.Source: §5 · pp. 13–14, Figure 6 · p. 13
6 Conclusion
The paper reports that RPN and RPC transferability differs under large domain gaps, and proposes collaborative self-training to improve adaptation. Experiments report improved transferability and performance over existing methods across various domain-adaptation scenarios.
6 Conclusion
- RPN and RPC show significantly different transferability in two-stage detectors facing large domain gaps.The paper presents this as an empirical finding.Source: §6 · p. 14
- Collaborative self-training uses high-confidence ROIs for mutual RPN–RPC guidance, while low-confidence ROIs support discrepancy-based minimax optimization.Source: §6 · p. 14
- Experiments across various domain-adaptation scenarios report improved transferability and performance over existing methods.Source: §6 · p. 14