Source-linked AI summary
SCL: Towards Accurate Domain Adaptive Object Detection via Gradient Detach Based Stacked Complementary Losses
Zhiqiang Shen, Harsh Maheshwari, Weichen Yao, Marios Savvides
TL;DR
Domain shift makes object detection challenging amid changes in viewpoint, background, appearance, illumination, occlusion, and scenes. SCL combines primary detection and complementary objectives with gradient-detach updates. The authors report that SCL outperforms state-of-the-art approaches by a large margin across varied benchmarks.
Problem
Domain shift makes object detection challenging amid variations in viewpoint, background, appearance, illumination, occlusion, and scenes.
Source: §1 · p. 1Method
SCL combines primary detection and complementary objectives with gradient-detach updates in an end-to-end framework.
Source: §1 · pp. 2–3Results
Across a variety of benchmarks, SCL outperforms state-of-the-art approaches by a large margin.
Source: §5 · p. 8Takeaways & Limitations
The paper argues that diverse deep supervisions should be controlled by their insertion locations, loss types, ordering, and updating strategy.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Unsupervised domain adaptive object detection aims to learn a robust detector in the domain shift circumstance, where the training (source) domain is label-rich with bounding box annotations, while the testing (target) domain is label-agnostic and the feature distributions between training and testing domains are dissimilar or even totally different. In this paper, we propose a gradient detach based stacked complementary losses (SCL) method that uses detection losses as the primary objective, and cuts in several auxiliary losses in different network stages accompanying with gradient detach training to learn more discriminative representations. We argue that the prior methods mainly leverage more loss functions for training but ignore the interaction of different losses and also the compatible training strategy (gradient detach updating in our work). Thus, our proposed method is a more syncretic adaptation learning process. We conduct comprehensive experiments on seven datasets, the results demonstrate that our method performs favorably better than the state-of-the-art methods by a significant margin. For instance, from Cityscapes to FoggyCityscapes, we achieve 37.9% mAP, outperforming the previous art Strong-Weak by 3.6%.
1. Introduction
Domain shift makes object detection challenging, while unsupervised adaptation must align source and target representations and define how they are matched. SCL combines complementary objectives with gradient-detach training in an end-to-end framework.
1. Introduction
- Unsupervised cross-domain detection seeks a common representation space and a defined source–target alignment mechanism without additional target instance annotations.The introduction contrasts this route with target-domain fine-tuning, which requires costly target annotations.Source: §1 · p. 1
- The authors report better feature embeddings than prior approaches on both PASCAL-to-Clipart and Cityscapes-to-FoggyCityscapes visualizations.Figure 1 uses t-SNE; red denotes source examples, blue target examples, and light blue overlapping positions.Source: Figure 1 · p. 2
- SCL extends multi-feature alignment by studying how diverse complementary objectives can be assembled and applied at controlled network locations and with a compatible training strategy.The paper reports that a hierarchical context subnetwork trained with gradient detach obtains its best results on several benchmarks.Source: §1 · p. 2
- SCL adds complementary outputs at intermediate layers, so error backpropagates from them as well as from the final layer.The authors describe this as a deeply supervised formulation and identify gradient detach as critical to learning with the complementary losses.Source: §1 · pp. 2–3
- The paper presents step-by-step ablations as practical guidance for constructing high-performance multi-objective domain-adaptive detectors.It identifies investigation of diverse losses and gradient detach as a contribution.Source: §1 · p. 3
- SCL achieves the highest accuracy on several domain-adaptive or cross-domain object-detection benchmarks.The paper identifies multi-loss interaction and gradient detach as central components of its framework.Source: §1 · p. 3
2. Methodology
The method trains a detector across labeled source and unlabeled target domains using stacked complementary objectives and gradient-detach updating. Its components combine domain alignment with detection training while keeping context representations distinct from the detection pathway.
2. Methodology
- The training setup uses annotated source-domain boxes and unlabeled target-domain images to learn features for detection across both domains.Source: §2 · p. 3
2.1. Multi-Complement Objective Learning
- At layers k = 1, ..., K, SCL cuts in complementary losses; domain classifiers use GRL, while instance-context alignment joins RoI object features with a shared context vector.GRL reverses the gradient passed from the domain classifier to the detection network.Source: §2.1 · p. 3
2.2. Gradients Detach Updating
- Gradient detach blocks context-subnetwork gradients from flowing through the detection-backbone path, aiming for more discriminative context representations distinct from detection features.The authors describe context and instance representations as focusing on perceptually different image parts.Source: §2.2 · p. 4
- The context network is updated with detection loss excluding the RPN and instance-context loss, while the detection network uses detection and complementary objectives.These are the two update steps specified in the backward-pass algorithm.Source: §2.2 · p. 4
2.3. Framework Overall
- The Faster R-CNN framework combines detection and complementary losses, with λ trading off between them; the detection component includes the RPN and other modules.Source: §2.3 · p. 4
3. Empirical Results
The experiments evaluate SCL across seven domain-shift datasets, using labeled source and unlabeled target images. Results report improvements over baselines across adaptation settings, while the ablation study shows that complementary losses must be placed in a controlled way.
3.1. How to Choose Complementary Losses in Our Framework
- The complementary-loss study considers cross-entropy, weighted least-squares, and focal losses, with least-squares used to align low-level features and focal loss to emphasize hard examples.Experiments report mAP at an IoU threshold of 0.5 and use one labeled source image and one unlabeled target image per training mini-batch.Source: §3.1 · p. 5, §3 · p. 5
3.2. Ablation Studies from Cityscapes to FoggyCityscapes
- 37.9% vs. 34.3%: SCL outperforms Strong-Weak on Cityscapes-to-FoggyCityscapes adaptation.The reported comparison is for the densest FoggyCityscapes setting, at 150 m visibility.Source: §3.2 · p. 5, Table 1 · p. 6
3.3. Similar Domains
- SCL exceeds reimplemented Strong-Weak by 4% on K→C and 1.7% on C→K car AP.The comparison covers both KITTI–Cityscapes adaptation directions.Source: Table 2 · p. 6, §3.3 · p. 6
- SCL consistently outperforms the baseline on the reported sunny→night and sunny→rainy INIT adaptations.The sunny training data for sunny→rainy was downsampled to match the smaller rainy training set; sunny→cloudy results are omitted because the domains are nearly identical.Source: Table 3 · p. 6, §3.3 · pp. 6–7
3.4. Discrepant Domains
- SCL outperforms baselines on PASCAL-to-Clipart and PASCAL-to-WaterColor adaptation.For Clipart, replacing focal loss with cross-entropy on the instance-context classifier improves performance from 40.6% to 41.5%.Source: Table 4 · p. 7, Table 5 · p. 7, §3.4 · p. 7
3.5. From Synthetic to Real Images
- SCL consistently outperforms baselines for Sim10K-to-Cityscapes adaptation on the car class.The evaluation uses Cityscapes as the target domain.Source: Table 6 · p. 7, §3.5 · p. 7
4. Analysis
The analysis tests sensitivity to SCL configuration and loss weighting, compares performance across IoU thresholds, and visualizes attention and detections.
Hyper-parameter K
- K = 3 gives the best performance on all three datasets, with K controlling the number of SCL losses and context branches.Table 7 reports the hyper-parameter K analysis.Source: §4 · p. 7, Table 7 · p. 7
Parameter Sensitivity on λ and γ
- 37.9% is the best accuracy on Cityscapes → FoggyCityscapes at λ = 1.0 and γ = 5.0; on Sim10K → Cityscapes, the best settings are λ = 0.1 and γ = 2.0.λ trades off SCL and detection objectives, while γ controls hard-sample strength in Focal Loss.Source: §4 · pp. 7–8
Analysis of IoU Threshold
- Across three datasets, SCL is better than the baselines at different IoU thresholds in most cases.Figure 4 plots AP (%) against IoU threshold; the paper describes higher thresholds as indicating better ground-truth coverage.Source: Figure 4 · p. 8, §4 · p. 8
Why Gradient Detach Can Help Our Model?
- With gradient-detach training, attention maps show stronger contrast between object areas and context, with less attention on background.Figure 5 visualizes feature maps after Conv B3; without detach, the model attends more to background.Source: Figure 5 · p. 8, §4 · p. 8
Detection Visualization
- SCL detects more small, blurry objects in dense FoggyCityscapes scenes and suppresses more false positives on Clipart and Watercolor than the two baselines.The comparisons are against DA and Strong-Weak.Source: §4 · p. 8
5. Conclusion
The conclusion presents SCL with gradient-detach training and multiple complementary losses for unsupervised domain-adaptive detection. It reports broad benchmark gains and identifies detection from scratch without pretrained models as future work.
5. Conclusion
- SCL combines multiple complementary losses with gradient-detach training that suppresses gradients flowing back to the detection backbone.The paper reports extensive experiments and ablations for its components and states that results outperform state-of-the-art approaches across a variety of benchmarks.Source: §5 · p. 8
- SCL outperforms state-of-the-art approaches by a large margin on a variety of benchmarks.Figure 6 compares detection examples from DA, Strong-Weak, and SCL on three datasets; displayed scores exceed 0.3 for FoggyCityscapes and 0.5 for the other two.Source: §5 · p. 8, Figure 6 · p. 9
- Future work will explore domain-shift detection from scratch, without pretrained models such as DSOD, to avoid bias from the pretraining dataset.Source: §5 · p. 8
A. More Ablation Studies
The ablation section reports target-domain results for adaptation from PASCAL VOC to WaterColor and Clipart, using different SCL combinations and ablations.
A. More Ablation Studies
- The reported target domains are WaterColor and Clipart, with adaptation starting from PASCAL VOC.Source: A. More Ablation Studies · p. 12
- The section presents detailed results for both adaptation tasks in Tables 8 and 12.Source: A. More Ablation Studies · p. 12
- The experiments compare different SCL combinations and include diverse ablations.Source: A. More Ablation Studies · p. 12
A.1. From Pascal VOC to Watercolor Dataset
The table caption identifies an AP (%) evaluation for adaptation from PASCAL VOC to WaterColor and defines the L2-loss ablation.
A.1. From Pascal VOC to Watercolor Dataset
- Table 8 reports AP (%) for adaptation from PASCAL VOC to WaterColor.Source: Table 8 · p. 12
- “W/O CLoss (L2)” denotes removing the L2 term from the complementary losses.Source: Table 8 · p. 12
- The caption identifies the task and ablation scope but supplies no AP values or result comparison.Source: Table 8 · p. 12
A.2. From Pascal VOC to Clipart Dataset
The section reports the PASCAL VOC-to-Clipart adaptation results in Table 12.
A.2. From Pascal VOC to Clipart Dataset
- Table 12 presents the PASCAL VOC-to-Clipart adaptation results.The passage does not provide the table’s values or identify a leading method.Source: §A.2 · p. 12
B. Results on Source Domains
The section compares source-domain adaptation results and notes that the target-best model need not also perform best on source data. It raises whether source-domain evaluation is a useful metric for domain-adaptive object detection.
B. Results on Source Domains
- The target-best models do not necessarily perform best on source data, although the reported gaps between DA, Strong-Weak, and SCL are marginal for PASCAL VOC-to-WaterColor.DA [3] obtains the highest source-domain results in that example.Source: B. Results on Source Domains · p. 12
- Source-domain adaptation results appear in Tables 9, 10, 13, and 11; target-trained best models did not perform best on source data, while DA [3] led PASCAL VOC-to-WaterColor results, with marginal gaps.Source: B. Results on Source Domains · p. 12
- The authors conjecture that target-domain adaptation affects source-data learning and performance despite source bounding-box supervision, and propose investigating this further.They question whether source-domain evaluation should serve as a metric for domain-adaptive object detection.Source: B. Results on Source Domains · p. 12
C. Detailed Results of Parameter Sensitivity on λ and γ
The section reports parameter-sensitivity results for two adaptation settings, describes the context network, and visualizes intermediate feature embeddings. The visualizations indicate that gradient-detach models better align source and target image distributions than models without detach.
C. Detailed Results of Parameter Sensitivity on λ and γ
- Parameter-sensitivity results are provided for λ and γ on Cityscapes-to-FoggyCityscapes and Sim10K-to-Cityscapes adaptation.The results are in Tables 15 and 16; the passages do not report the values.Source: C. Detailed Results of Parameter… · p. 12
D. Context Network
- The context network concatenates three 128-dimension branch features into a 384-dimension context feature.Each branch derives its feature from a corresponding SCL backbone layer.Source: D. Context Network · p. 12
E. Visualization of Intermediate Feature Embedding
- Across three adaptation datasets, Figure 7 visualizations show that gradient-detach-based models adapt source and target images to a more similar distribution than models without gradient detach.Source: E. Visualization of Intermediate… · p. 12
G. More Detection Visualization in Fig. 9
The section provides additional SCL detection examples on Clipart and Watercolor, showing detections with scores above 0.5.
G. More Detection Visualization in Fig. 9
- The additional SCL examples on Clipart and Watercolor show detections with scores higher than 0.5.The passage does not report a comparison or quantitative detection outcome.Source: Figure 9 · p. 16