Source-linked AI summary
Harmonizing Transferability and Discriminability for Adapting Object Detectors
Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, Qi Dou
TL;DR
Cross-domain object detectors struggle because domain shift and unequal feature transferability create a tension between transferable and discriminative representations. HTCN hierarchically calibrates transferability through local feature masks, IWAT-I, and CILA, and reports state-of-the-art benchmark performance, including 39.8% mAP on Cityscapes→Foggy-Cityscapes. Its context-aware alignment addresses a limitation of treating instance features independently from global context, while its theoretical analysis assumes unconstrained adversarial training can form a non-conservative adaptation problem.
Problem
Source-trained object detectors generalize poorly to unlabeled target domains, while adversarial alignment can impair discriminability because features are not equally transferable.
Method
HTCN hierarchically calibrates transferability using local feature masks, IWAT-I for global discriminability, and CILA for context-aware instance-level alignment.
Results
39.8% mAP on Cityscapes→Foggy-Cityscapes exceeded recent adversarial adaptation methods by 5.6% on average and approached the 40.3% upper bound.
Takeaways & Limitations
HTCN harmonizes transferability and discriminability and achieves state-of-the-art performance on several cross-domain object-detection benchmarks.
Takeaways & Limitations
The theoretical analysis assumes unconstrained adversarial training represents a non-conservative domain-adaptation problem, and prior concatenation-based fusion treats context and instance features independently.
Abstract
from arXiv · showhide
Recent advances in adaptive object detection have achieved compelling results in virtue of adversarial feature adaptation to mitigate the distributional shifts along the detection pipeline. Whilst adversarial adaptation significantly enhances the transferability of feature representations, the feature discriminability of object detectors remains less investigated. Moreover, transferability and discriminability may come at a contradiction in adversarial adaptation given the complex combinations of objects and the differentiated scene layouts between domains. In this paper, we propose a Hierarchical Transferability Calibration Network (HTCN) that hierarchically (local-region/image/instance) calibrates the transferability of feature representations for harmonizing transferability and discriminability. The proposed model consists of three components: (1) Importance Weighted Adversarial Training with input Interpolation (IWAT-I), which strengthens the global discriminability by re-weighting the interpolated image-level features; (2) Context-aware Instance-Level Alignment (CILA) module, which enhances the local discriminability by capturing the underlying complementary effect between the instance-level feature and the global context information for the instance-level feature alignment; (3) local feature masks that calibrate the local transferability to provide semantic guidance for the following discriminative pattern alignment. Experimental results show that HTCN significantly outperforms the state-of-the-art methods on benchmark datasets.
1. Introduction
Object detectors trained on a source domain generalize poorly to unlabeled target domains because of domain shift, motivating unsupervised domain adaptation. HTCN addresses a tension in adversarial adaptation by hierarchically calibrating transferability to preserve feature discriminability across local-region, instance, and image levels.
- Problem: Source-trained object detectors generalize poorly to new target domains because domain shift changes data distributions.This limits deployment in real-world settings and motivates transferring knowledge from labeled source data to fully unlabeled target data.
- Prior Work: Existing cross-domain detectors apply adversarial alignment at image, instance, local-region, or multiple feature levels within detection frameworks.These approaches include image- and instance-level alignment, strong-local and weak-global alignment, proposal-based local alignment, and prediction-guided multi-level alignment.
- Problem: Adversarial adaptation improves transferability, but strict alignment can impair discriminability because features are not equally transferable.The paper defines transferability as cross-domain representation invariance and discriminability as the ability to localize and distinguish instances.
- Method: HTCN hierarchically calibrates representation transferability to harmonize transferability and discriminability in cross-domain object detection.Its components target global discriminability through IWAT-I, local discriminability through CILA, and local transferability through feature masks.
- Results: 39.8% mAP on Cityscapes→Foggy-Cityscapes exceeded recent adversarial adaptation methods by 5.6% on average and approached the 40.3% upper bound.The reported result is from a representative benchmark adaptation setting.
2. Related Work
Unsupervised domain adaptation has primarily advanced through feature-distribution matching and adversarial learning, while object-detection adaptation remains comparatively limited. Prior detection methods embed adversarial alignment into established detection pipelines to reduce image- and instance-level domain discrepancies.
- Unsupervised Domain Adaptation: UDA transfers knowledge between domains by mitigating distributional variations, with strong prior success in image classification and semantic segmentation.Common approaches match high-order statistics of source and target feature distributions in latent space.
- Unsupervised Domain Adaptation: UDA methods mainly use statistics matching or adversarial learning to reduce domain differences in learned representations.Statistics matching uses distribution divergence, whereas adversarial learning trains representations to reduce domain distinguishability.
- UDA for Object Detection: Domain adaptation for object detection has received relatively limited study compared with single-domain detection.Early work embedded adversarial feature adaptation into Faster R-CNN to reduce divergence at image and instance levels.
3. Hierarchical Transferability Calibration Network (HTCN)
HTCN addresses the tension between transferability and discriminability in cross-domain object detection through hierarchical calibration of local-region, image-level, and instance-level representations. Its modules use interpolated, importance-weighted features, context-aware instance alignment, and local feature masks to guide adaptation and semantic consistency.
- Overall architecture: HTCN hierarchically calibrates transferable local-region, image-level, and ROI-based instance-level features to improve discriminability across domains.The approach targets the potential contradiction between transferability and discriminability in adversarial adaptation.
- Image-level calibration: IWAT-I generates interpolation samples between domains and re-weights image-level features using domain-discriminator uncertainty to calibrate global transferability.Synthetic samples fill the distributional gap, while images that are harder for the discriminator to distinguish receive greater weight.
- Instance-level calibration: CILA aligns instance-level representations after fusing ROI features with aggregated context vectors, capturing complementary interactions between local instances and global scene information.The tensor-product fusion provides informative interactions that concatenation would miss, although it introduces a dimension-explosion concern addressed in the method.
- Local-region calibration: Local feature masks assign higher weights to informative regions and suppress less informative ones, semantically guiding subsequent high-level feature adaptation.The masks are computed from shallow-layer features in both domains to support semantic consistency.
- Theoretical perspective: HTCN frames its adaptation objective as reducing domain divergence through transferable region, image, and instance features while reducing the ideal joint-hypothesis error through hierarchical alignments.This theoretical interpretation links transferability to dH∆H(S, T) and discriminability to C.
4. Experiments
HTCN is evaluated across synthetic-to-real, real-to-foggy, and real-to-artistic adaptation tasks, with comparisons, ablations, threshold analysis, masks, and qualitative detections. Across these experiments, HTCN improves target-domain detection and supports its hierarchical transferability-calibration design.
- State-of-the-arts: 39.8% mAP on Cityscapes → Foggy-Cityscapes, improving over state-of-the-art results by +4.7% on average and approaching the upper bound by 0.5%.Table 1 reports target-domain average precision, with the comparison spanning cross-domain detection methods.
- State-of-the-arts: HTCN achieves state-of-the-art mAP for PASCAL → Clipart, demonstrating robustness between real-image and artistic-image domains.The Clipart target contains the same 20 categories as PASCAL, while the domains remain visually dissimilar.
- State-of-the-arts: HTCN outperforms all comparison methods on Sim10K → Cityscapes, supporting the effectiveness of hierarchical transferability calibration for synthetic-to-real adaptation.Sim10K provides synthetic driving scenes, whereas Cityscapes is a real street-scene dataset.
- Ablation Study: Removing any proposed module lowers performance in the ablation study, including variants that alter context use or tensor-product feature fusion.HTCN-w/o Context Information removes context from CILA while preserving instance-level alignment; HTCN-w/o Tensor Product replaces tensor-product fusion with vanilla concatenation.
- Influence of IoU thresholds: Across IoU thresholds 0.5–0.9, HTCN significantly outperforms Source Only and SWDA, while mAP decreases as the threshold increases.The authors interpret this pattern as indicating more accurate and robust bounding-box regression.
- Visualization of Local Feature Masks: Local feature masks emphasize instance edges and descriptive regions, up-weighting semantically informative areas to improve discriminability during adaptation.The visualized masks show edge-aware patterns around cars, people, buildings, and traffic signs across source and target domains.
- Example of Detection Results: Qualitative results show HTCN consistently outperforming Source Only and SWDA across two transfer tasks, including accurate boxes for obscured Foggy-Cityscapes instances.Figure 5 covers Cityscapes → Foggy-Cityscapes and Sim10K → Cityscapes.
5. Conclusion
HTCN harmonizes transferability and discriminability by exploring hierarchical adaptation across local regions, images, and instances. Experiments demonstrate state-of-the-art performance for adapting object detectors on benchmark datasets.
- The framework explores transferability across local regions, images, and instances.
- HTCN harmonizes transferability and discriminability for adaptive object detection.
- HTCN achieves state-of-the-art performance for adapting object detectors on several benchmark datasets.