Source-linked AI summary
MeGA-CDA: Memory Guided Attention for Category-Aware Unsupervised Domain Adaptive Object Detection
Vibashan VS, Vikram Gupta, Poojan Oza, Vishwanath A. Sindagi, Vishal M. Patel
TL;DR
Object detectors can degrade across domains, while common unsupervised adaptation methods align features without category information, risking negative transfer. MeGA-CDA adds category-wise discriminators and memory-guided attention to route source and target features, and it outperforms existing approaches on several benchmarks. The method assumes fully labeled source images and unlabeled target images.
Problem
Object detection performance can degrade across domains, and category-agnostic feature alignment may align features from different classes, producing negative transfer.
Method
MeGA-CDA uses category-specific discriminators with memory-guided category-specific attention maps to route features despite unavailable target labels.
Results
MeGA-CDA outperforms existing domain-adaptive detection approaches by a considerable margin across several benchmark datasets.
Takeaways & Limitations
Category-aware alignment mitigates negative transfer and results in better overall feature alignment for domain-adaptive object detection.
Takeaways & Limitations
The method assumes fully labeled source-domain images with bounding-box annotations and unlabeled target-domain images without annotations.
Abstract
from arXiv · showhide
Existing approaches for unsupervised domain adaptive object detection perform feature alignment via adversarial training. While these methods achieve reasonable improvements in performance, they typically perform category-agnostic domain alignment, thereby resulting in negative transfer of features. To overcome this issue, in this work, we attempt to incorporate category information into the domain adaptation process by proposing Memory Guided Attention for Category-Aware Domain Adaptation (MeGA-CDA). The proposed method consists of employing category-wise discriminators to ensure category-aware feature alignment for learning domain-invariant discriminative features. However, since the category information is not available for the target samples, we propose to generate memory-guided category-specific attention maps which are then used to route the features appropriately to the corresponding category discriminator. The proposed method is evaluated on several benchmark datasets and is shown to outperform existing approaches.
1. Introduction
Object detectors can degrade when deployed across domains, motivating unsupervised adaptation using labeled source and unlabeled target data. MeGA-CDA adds category-aware alignment with memory-guided attention to address negative transfer from category-agnostic alignment.
- Object detectors can suffer severe performance degradation when evaluated on images from distributions different from their training data.
- Cross-city or cross-environment deployment, such as autonomous driving, requires detectors that generalize beyond their source domain.
- Unsupervised domain adaptation uses labeled source data and unlabeled target data, typically learning domain-invariant features through source-target feature alignment.
- Global alignment can mix features from different categories, causing sub-optimal adaptation, especially when detection images contain multiple object categories.
- MeGA-CDA uses category-specific discriminators and memory-guided attention maps to route features for category-wise distribution matching without target category labels.
2. Related work
Prior work includes established object-detection approaches and unsupervised adaptation strategies, while the proposed overview combines global and category-aware feature alignment. The category-aware component uses K category-specific discriminators and memory-guided attention for unlabeled target features.
- Object-detection methods include two-stage and single-stage approaches, with recent methods benefiting from convolutional neural networks and anchor-based strategies.
- The proposed overview combines category-agnostic global alignment with K category-specific discriminators, routing target features through memory-guided category-specific attention maps.
- Some adaptation methods use large amounts of target-domain video to generate pseudo-labels, while others train with noisy target bounding boxes from source-only detectors.
3. Proposed method
MeGA-CDA combines global and category-wise adversarial feature alignment with memory-guided attention, routing category-specific features to corresponding discriminators despite unlabeled target data. Category memories store source-derived prototypes, support attention-map generation for both domains, and are regularized for compact, unique representations.
- Problem setup: The method assumes fully labeled source images and unlabeled target images, using source labels to train a detector that generalizes to target data.Faster R-CNN is used as the base detector, with feature alignment performed through domain-adversarial training.
- Global discriminator for adaptation: Global alignment uses a discriminator over the entire backbone feature map, but category-agnostic matching can transfer features negatively across object categories.A gradient reversal layer trains the discriminator and feature encoder adversarially, while least-squares loss is used for stability.
- Category-wise discriminators for adaptation: Category-wise discriminators align features for each category, using binary attention maps to select locations containing the corresponding category.The kth discriminator receives source and target features routed through attention maps σ(Fs)k and σ(Ft)k.
- Memory-guided attention: MeGA learns K category-specific memory modules that store object prototypes and retrieves similar elements to compute attention maps for source and target feature queries.The resulting attention maps focus on category-specific information and route it to the appropriate category discriminator.
- Memory module: Memory elements are updated from labeled source features, while compactness and uniqueness constraints encourage each module to represent distinct prototypes.The compactness loss uses an L2 penalty, and a triplet loss reduces redundancy among memory elements.
4. Experiments and results
MeGA-CDA is evaluated across adverse-weather, synthetic-to-real, and cross-camera adaptation settings, with results and ablations examining category-wise alignment and memory-guided attention.
- Experiments cover adverse-weather, synthetic-to-real, and cross-camera domain adaptation categories.
- Adverse weather conditions: 2.5% average absolute mAP improvement over the recent best method is reported for Cityscapes→Foggy-Cityscapes adaptation.
- Synthetic data adaptation: 1.8% absolute mAP improvement over the recent best method is reported for Sim10K→Cityscapes adaptation.
- Cross-camera adaptation: MeGA-CDA achieves considerable improvements over recent best methods in both KITTI→Cityscapes and Cityscapes→KITTI adaptation.
- Ablation studies: Adding cosine-similarity memory-guided category-wise discriminators improves mAP by 0.6%, while applying them at conv4 and conv5 improves it by approximately 3%.
- Ablation studies: Learned similarity improves conv5 MeGA-CDA by 2.1% over the cosine-similarity baseline, and applying it at conv4 and conv5 adds 2%.
- Qualitative analysis: Qualitative results associate global alignment with missed detections and false positives, whereas category-wise alignment reduces false positives and produces high-quality detections.
- Qualitative analysis: Learned-similarity attention maps obtain more accurate category focus than cosine-similarity maps, with attention covering the majority of the car region.
5. Conclusions
The conclusion presents MeGA-CDA as a category-aware feature-alignment approach that uses memory-guided attention to route features to category-specific discriminators and reports considerable performance gains.
- MeGA-CDA incorporates category information into domain alignment through category-aware discriminators.
- Memory-guided attention generates category-specific maps that route features to the appropriate category-specific discriminator without target category labels.
- The method is evaluated on several benchmark datasets and outperforms existing approaches by a considerable margin.
Supplementary Material for “MeGA-CDA: Memory Guided Attention for
The supplied passage identifies supplementary material for the paper titled “Category-Aware Unsupervised Domain Adaptive Object Detection.”
- The supplementary material is associated with the paper title “Category-Aware Unsupervised Domain Adaptive Object Detection.”
- The paper title identifies category-aware unsupervised domain adaptive object detection as its subject.
- The supplied title names unsupervised domain adaptation and category-aware object detection together.
1. Network architecture
The supplied passages identify the paper version and refer to architecture tables for its global and category-wise discriminators and related networks.
- The paper version is arXiv:2103.04224v2 dated 3 Apr 2021.
- Table 2 presents the architecture of the global and category-wise discriminators.
- The architectures of networks Θt and Θk are described as similar in Table 3.
2. Alternative baselines
The ablation compares category-wise and multi-discriminator domain alignment at instance and image levels. Image-level category-wise alignment performs best, while adding discriminators without category-specific routing performs worse.
- Alternative baselines: Adding k multi-discriminators at image level performs 2% lower than category-wise image-level alignment.The result indicates that simply increasing discriminator count does not ensure effective information routing.
- Alternative baselines: The ablation evaluates multi-discriminator configurations at both instance and image levels on Cityscapes → Foggy-Cityscapes.The table defines k as the number of classes.
- Alternative baselines: The proposed category-wise k discriminators route category-specific information to the corresponding discriminator.This routing distinguishes the proposed design from blindly adding multiple image-level discriminators.
3. Qualitative results
The qualitative results visualize detections and memory-guided attention maps on target domains. The proposed method produces high-confidence detection boxes in the Foggy→Cityscapes visualization.
- Qualitative results: The Foggy→Cityscapes visualization compares DAFaster RCNN with the proposed method using confidence-colored bounding boxes.Only detections with scores higher than 0.5 are shown.
- Qualitative results: The proposed method is described as tackling classes’ negative transfer and constructing high-confidence prediction boxes.The visualization uses detector confidence to color the bounding boxes.
- Qualitative results: Additional visualizations show memory-guided attention maps on target domains.