PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-domain Detection via Graph-induced Prototype Alignment

Minghao Xu, Hang Wang, Bingbing Ni, Qi Tian, Wenjun Zhang

arXiv:2003.12849v1cs.CV

TL;DR

Cross-domain detection is challenged by domain shift, inaccurate region proposals, multi-modal instance features, and class imbalance. GPA uses graph propagation and confidence-guided prototypes for category-level alignment, with class-reweighted contrastive learning and two-stage Faster R-CNN alignment. The framework achieves state-of-the-art performance across cross-domain detection tasks.

  • Problem

    Domain shift, inaccurate target proposals, multi-modal instances, and class imbalance make reliable cross-domain detection alignment difficult.

  • Method

    GPA propagates information among proposals, merges instances into per-category prototypes, applies class-reweighted contrastive learning, and aligns features in two Faster R-CNN stages.

  • Results

    GPA achieves state-of-the-art performance across cross-domain detection tasks under various scenarios.

  • Takeaways & Limitations

    Prototype alignment integrates critical instance information and category-level multi-modal variation for cross-domain detection adaptation.

Abstract

from arXiv · show

Applying the knowledge of an object detector trained on a specific domain directly onto a new domain is risky, as the gap between two domains can severely degrade model's performance. Furthermore, since different instances commonly embody distinct modal information in object detection scenario, the feature alignment of source and target domain is hard to be realized. To mitigate these problems, we propose a Graph-induced Prototype Alignment (GPA) framework to seek for category-level domain alignment via elaborate prototype representations. In the nutshell, more precise instance-level features are obtained through graph-based information propagation among region proposals, and, on such basis, the prototype representation of each class is derived for category-level domain alignment. In addition, in order to alleviate the negative effect of class-imbalance on domain adaptation, we design a Class-reweighted Contrastive Loss to harmonize the adaptation training process. Combining with Faster R-CNN, the proposed framework conducts feature alignment in a two-stage manner. Comprehensive results on various cross-domain detection tasks demonstrate that our approach outperforms existing methods with a remarkable margin. Our code is available at https://github.com/ChrisAllenMing/GPA-detection.

1. Introduction

Cross-domain detection suffers when domain shift, inaccurate proposals, multi-modal instances, and class imbalance undermine feature alignment. GPA addresses these issues with graph-based instance aggregation, prototype-based category alignment, class-reweighted contrastive learning, and two-stage Faster R-CNN alignment.

  • Domain shift causes performance decay when detectors trained on annotated source domains are applied to distinct, weakly labeled target domains.
  • Target-domain proposals can deviate from objects, while single instances capture limited scale or orientation information despite category-level multi-modality.
  • GPA aggregates proposal features with graph-based region aggregation and derives per-class prototypes for category-level domain alignment.
  • Faster R-CNN uses two-stage alignment, separating foreground and background distributions before fine-grained per-category alignment.
  • Class-reweighted Contrastive Loss assigns greater weight to sample-scarce classes to harmonize multi-class adaptation.
  • The proposed approach achieves state-of-the-art performance across cross-domain detection tasks under various scenarios.

2. Related Work

Related work spans conventional object detectors, unsupervised domain adaptation, and cross-domain detection. GPA differs from prior proposal-based alignment by refining instance representations and integrating multi-modal instances into category prototypes.

  • Object Detection: Object detectors are broadly divided into one-stage methods that predict directly from anchors and two-stage methods that classify generated region proposals.
  • Unsupervised Domain Adaptation: UDA transfers knowledge from labeled source domains to unlabeled target domains using discrepancy minimization or adversarial domain classifiers.
  • Cross-domain Detection: Cross-domain detection methods align domains at image or instance levels, including domain-adaptive Faster R-CNN, strong-weak alignment, and mean-teacher adaptation.
  • Improvements over existing methods: GPA addresses prior proposal-based methods’ inaccurate instance representations and their omission of instances’ multi-modal information.
  • Graph Convolutional Network: Graph models have been applied to learning relations in tasks including action recognition, person re-identification, video understanding, and point-cloud learning.

3. Method

GPA aligns source and target domains through category-level prototypes built from graph-aggregated region proposals, while class-reweighted contrastive training addresses class imbalance. Integrated with Faster R-CNN, it performs domain alignment progressively from class-agnostic foreground/background features to per-category distributions.

  • Two-stage Domain Alignment: Two-stage domain alignment first separates foreground and background for class-agnostic alignment, then aligns each foreground category using finer-grained features.The procedure follows Faster R-CNN’s RPN and R-CNN stages.
  • Graph-induced Prototype Alignment: GPA constructs a relation graph over Faster R-CNN region proposals to propagate information and obtain more precise instance-level representations.The graph uses proposal relationships to aggregate feature embeddings and classification confidence.
  • Graph-induced Prototype Alignment: The adjacency matrix uses proposal overlap through IoU, so proposals with larger overlap receive stronger connections than equally distant proposals.IoU accounts for both proposal location and size, unlike center distance alone.
  • Graph-induced Prototype Alignment: Confidence-guided merging integrates multimodal instance information into class prototypes by using each proposal’s class confidence as a weighting factor.The resulting prototype is a weighted mean embedding and serves as a class proxy for domain alignment.
  • Graph-induced Prototype Alignment: Category-level alignment minimizes distances between same-class prototypes and constrains different-class prototypes to remain farther apart than a margin.These intra-class and inter-class objectives form the prototype-based contrastive alignment.
  • Class-imbalance-aware Adaptation Training: Class-reweighted contrastive training assigns higher weights to sample-scarce categories, whose feature distributions are otherwise harder to align.Weights are based on the maximum class confidence, with a threshold of 1/Nc filtering classes absent from the proposal set.

4. Experiments

The experiments evaluate GPA across three cross-domain detection tasks with distinct domain shifts, using Faster R-CNN and comparisons against existing methods. Two-stage alignment consistently performs strongly, including gains of 3.6% on mAP for Normal to Foggy, 47.6% AP on Synthetic to Real, and 4.3% performance gain for Cross Camera Adaptation.

  • Experimental Setup: GPA is evaluated on Normal to Foggy, Synthetic to Real, and Cross Camera Adaptation tasks with distinct domain shifts.The tasks are Cityscapes → Foggy Cityscapes, SIM 10k → Cityscapes, and KITTI → Cityscapes.
  • Experimental Setup: The evaluation compares RPN Alignment, RCNN Alignment, and Two-stage Alignment configurations against several existing cross-domain detection methods.The configurations use different λ1 and λ2 settings, with ResNet-50 used as the common backbone for fair comparison.
  • Normal to Foggy: 3.6% mAP improvement is achieved by Two-stage Alignment on the Normal to Foggy task, while all three GPA configurations surpass existing methods.The comparison uses Cityscapes as source and Foggy Cityscapes as target, evaluated across eight categories.
  • Synthetic to Real: 47.6% AP is obtained by Two-stage Alignment on the Synthetic to Real task, the highest AP among compared methods for the common category car.SIM 10k is the source domain and Cityscapes is the target domain; the domain shift is mainly attributed to distinct image styles.
  • Cross Camera Adaptation: 4.3% performance gain is achieved by Two-stage Alignment on the Cross Camera Adaptation task, with all GPA configurations exceeding existing works.The task adapts from KITTI to Cityscapes, where different camera setups produce diverse instance patterns.

5. Analysis

Ablation and qualitative analyses examine relation-graph construction, two-stage alignment, parameter sensitivity, feature embeddings, and detection examples. The results support IoU-based graphs, progressive alignment, robust trade-off parameters, clearer cross-domain feature structure, and improved localization with fewer false positives.

  • Effect of relation graph: IoU-based relation graphs outperform Euclidean-distance graphs, showing that proposal size information is essential for relation-graph construction.Adding a learnable parameter matrix causes performance decay relative to parameter-free graph propagation, which the authors associate with limited supervisory signal.
  • Effect of two-stage alignment: Two-stage alignment surpasses either single-stage configuration with a clear margin across the evaluated cross-domain detection tasks.The authors describe this as progressive alignment from coarse-grained foreground alignment to fine-grained per-category alignment.
  • Sensitivity Analysis: Target-domain performance is not sensitive to λ1 and λ2 when either parameter varies from 0.25 to 2.0 while the other remains fixed.These parameters trade off detection loss and domain adaptation loss on SIM 10k → Cityscapes.
  • Sensitivity Analysis: The highest target-domain mAP occurs when the class-balancing parameter γ is around 2.0 on Cityscapes → Foggy Cityscapes.This setting gives the most beneficial weight assignment among different classes according to the analysis.
  • Visualization: After RPN and RCNN alignment, same-category source and target features are better aligned and different categories are more clearly separated than with Source-only.The comparison uses t-SNE visualizations on SIM 10k → Cityscapes.
  • Qualitative Detection Results: GPA localizes objects more precisely under severe occlusion and suppresses false positives compared with Source-only and DA [5].The qualitative examples also show DA [5] incorrectly classifying a garbage can as a car.

6. Conclusion

The paper concludes that GPA performs cross-domain detection by aggregating instance information through graph-based propagation, aligning category prototypes, and reweighting adaptation across classes.

  • Conclusion: GPA aggregates critical instance information through graph-based message propagation and derives prototype representations for category-level domain alignment.The framework also uses Class-reweighted Contrastive Loss to harmonize adaptation training, and experiments support its reported performance.

8. Appendix I: Network Architecture

The GPA implementation uses a Faster R-CNN architecture with RPN and RCNN stages, producing prototypes and domain-alignment losses at both stages.

  • Network Architecture: The implementation uses Faster R-CNN with a ResNet-50 backbone, separating image-wide feature extraction from the detection and adaptation head.The head performs bounding-box recognition, regression, and domain adaptation learning.
  • RPN: The RPN stage applies RoI pooling and a fully connected layer to produce 128-dimensional features for foreground and background prototypes.An RPN domain-alignment loss is calculated from these prototypes alongside classification and localization losses.
  • RCNN: The RCNN stage uses average pooling to form 2048-dimensional features and a fully connected layer to produce 64-dimensional category-prototype features.An RCNN domain-alignment loss is calculated from the category prototypes alongside classification and localization losses.

9. Appendix II: Qualitative Detection Results

Qualitative results across SIM 10k →Cityscapes and Cityscapes →Foggy Cityscapes show GPA producing more accurate detections than Source-only and DA [5]. The comparisons cover vehicle detection and eight shared categories.

  • SIM 10k →Cityscapes: GPA alleviates false positives and accurately localizes small or severely occluded vehicles on SIM 10k →Cityscapes.Source-only produces boxes biased from objects, while DA [5] still generates false positives.
  • Cityscapes →Foggy Cityscapes: On Cityscapes →Foggy Cityscapes, GPA correctly detects most objects and predicts bounding boxes more accurately than Source-only and DA [5].The evaluation uses eight common categories: person, rider, car, truck, bus, train, bicycle, and motorcycle.
  • Cityscapes →Foggy Cityscapes: The qualitative comparisons evaluate Source-only, DA [5], and GPA on Cityscapes →Foggy Cityscapes.Figure 9 presents several groups of detection results for this task.
Loading 2003.12849v1…