Source-linked AI summary
Strong-Weak Distribution Alignment for Adaptive Object Detection
Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, Kate Saenko
TL;DR
Object detectors need costly labels, and full global alignment can fail when source and target scenes differ in layout or object combinations. The paper proposes weak global alignment with strong local alignment, emphasizing globally similar images while matching local receptive-field features. Across datasets with similar and dissimilar shifts, the authors report large-margin improvements over existing methods, while noting errors under substantial appearance and crowded-scene differences.
Problem
Object detection requires costly bounding-box labels, while full global source–target alignment may be unsuitable when domains differ in scene layouts, object counts, or object combinations.
Method
The Strong-Weak Domain Alignment model weakly aligns globally similar images and strongly aligns local receptive-field features through a local domain classifier.
Results
The method outperformed existing methods with a large margin across several datasets and was effective for both similar and dissimilar domain shifts.
Takeaways & Limitations
Selective global alignment and strong local alignment provide an effective detector-adaptation strategy across varied domain shifts without fully matching global feature distributions.
Takeaways & Limitations
The method can produce unsuccessful detections when object appearance differs substantially and can generate duplicate or overly broad boxes in crowded scenes.
Abstract
from arXiv · showhide
We propose an approach for unsupervised adaptation of object detectors from label-rich to label-poor domains which can significantly reduce annotation costs associated with detection. Recently, approaches that align distributions of source and target images using an adversarial loss have been proven effective for adapting object classifiers. However, for object detection, fully matching the entire distributions of source and target images to each other at the global image level may fail, as domains could have distinct scene layouts and different combinations of objects. On the other hand, strong matching of local features such as texture and color makes sense, as it does not change category level semantics. This motivates us to propose a novel method for detector adaptation based on strong local alignment and weak global alignment. Our key contribution is the weak alignment model, which focuses the adversarial alignment loss on images that are globally similar and puts less emphasis on aligning images that are globally dissimilar. Additionally, we design the strong domain alignment model to only look at local receptive fields of the feature map. We empirically verify the effectiveness of our method on four datasets comprising both large and small domain shifts. Our code is available at \url{https://github.com/VisionLearningGroup/DA_Detection}
1. Introduction
Object detection adaptation is difficult because global source–target matching can confuse scene-layout and object-combination differences with domain variation. The paper therefore combines weak global alignment with strong local alignment to preserve semantics while reducing appearance gaps.
- Motivation: Object detectors require extensive bounding-box annotation, while dataset bias can reduce pretrained models’ generalization to novel domains.Each object instance must receive a precise bounding box, making detection annotation especially burdensome.
- Proposed approach: The approach is designed to address both similar and dissimilar domain shifts without requiring full global distribution matching.The motivation contrasts strong alignment for closed problems with the more selective alignment needed when domains differ in categories, layouts, or object combinations.
- Motivation: Fully aligning global image features is problematic when domains differ in scene layouts, object counts, or object co-occurrences.Forcing invariance between single-object source images and multi-object target images can hurt performance.
- Motivation: Strong local alignment is intended to match texture and color while preserving category information.Here, local features refer to texture or color features with small receptive fields, not instance-level object features.
- Proposed approach: The Strong-Weak Domain Alignment model combines partial global alignment with strict local alignment for unsupervised object-detector adaptation.Its weak global model emphasizes globally similar images and reduces emphasis on globally dissimilar images, while the local classifier focuses on local receptive fields.
2. Related Work
Prior object-detection systems use strong feature alignment, while this paper retains strong alignment only for local appearance features and weakens alignment of global features. Detection adaptation is built on Faster R-CNN but is intended to extend to other detector architectures.
- Object Detection: Faster R-CNN uses region proposals followed by classification and serves as the paper’s base detector.The authors state that the method should also apply to one-stage detectors such as YOLO or SSD.
- Domain Alignment: The paper proposes weak alignment for global features and strong alignment only at the local level to align image style across domains.This contrasts with approaches that align source and target distributions more broadly.
- Domain Alignment: The method also regularizes the domain classifier with source-domain detection loss to stabilize adversarial training.The related-work passage connects this design to task-specific regularization used in segmentation and GAN-based adaptation.
3. Method
The method combines weak global alignment with strong local alignment for object detection, using focal-loss weighting to emphasize difficult domain examples and local receptive-field matching. It also stabilizes adversarial training with context-vector regularization and detection loss.
- Weak Global Feature Alignment: The global domain classifier aligns high-level features, while the feature extractor remains discriminative for object detection but uninformative about domain.The formulation uses labeled source images and unlabeled target images, with Faster R-CNN detection modules included in the objective.
- Weak Global Feature Alignment: Weak global alignment uses focal loss to downweight easy-to-classify domain examples and focus training on hard examples.Cross-entropy assigns non-negligible loss to easy examples, whereas the modulating factor decreases as the estimated class probability increases.
- Strong Local Feature Alignment: Strong local alignment applies a fully convolutional, 1×1-kernel domain classifier to produce location-wise predictions over lower-layer feature maps.Each receptive field is aligned with the other domain using a least-squares loss over the feature-map width and height.
- Context Vector based Regularization: Context-vector regularization concatenates domain-classifier context vectors with region-wise features and trains the classifiers with source detection loss and domain-classification loss.The context vectors are extracted from middle layers and are intended to contain information about the whole input image.
- Overall Objective: The overall objective combines detection and adversarial losses, with a trade-off parameter and gradient reversal, using one labeled source and one unlabeled target example per mini-batch.The detection objective includes region proposal, classification, and localization losses.
4. Experiments
Experiments across dissimilar and similar domain shifts evaluate weak global and strong local alignment against source-only and domain-alignment baselines. The method consistently improves adaptation, including 34.3 mAP on Cityscape-to-FoggyCityscape and better performance than strict alignment in several settings.
- Experimental setup: The evaluation covers PASCAL-to-Clipart, PASCAL-to-Watercolor, Cityscapes-to-FoggyCityscapes, and GTA-to-Cityscapes domain shifts.These experiments test adaptation across both dissimilar and similar domains.
- Adaptation between dissimilar domains: In PASCAL-to-Clipart, weak alignment separates most target features from source features while aligning only some globally similar examples, outperforming strict matching.The source typically contains one or two objects, whereas target images often contain multiple objects.
- Adaptation between dissimilar domains: The method outperformed baseline methods on PASCAL-to-Watercolor, with local alignment contributing about 3% improvement for the target domain’s painting style.The reported improvement is attributed to reducing the domain gap through local-level features.
- Adaptation between similar domains: For Cityscape-to-FoggyCityscape, strong local alignment achieved 27.9 mAP and combining it with weak global alignment increased mAP to 34.3.Because fog noise produces a local-level shift, strong local alignment contributed substantially; the method’s performance was very near the labeled-target oracle.
- Experimental setup: The proposed method combines weak global alignment with strong local alignment, while baselines include source-only Faster R-CNN, BDC-Faster, and DA-Faster.BDC-Faster shares the proposed architecture but uses cross-entropy for its domain classifier and omits local-level alignment.
5. Conclusion
The paper proposes unsupervised detector adaptation through strong local and weak global alignment. Its weak alignment emphasizes globally similar images, while local alignment examines receptive fields.
- The method combines strong local alignment with weak global alignment for unsupervised object-detector adaptation.
- Weak alignment focuses adversarial loss on globally similar images and deemphasizes globally dissimilar images.
- The strong alignment model restricts its analysis to local receptive fields of the feature map.
- Experiments across several datasets found large-margin improvements over existing methods and verified both alignment strategies' effectiveness.
1. Network Architecture
The architecture uses separate global and local domain classifiers. The global classifier aggregates spatial context, whereas the local classifier operates through pointwise convolutions over local features.
- Local features come from conv3_3 in VGG16 and the last res2c layer in ResNet101.
- The architecture includes distinct global and local domain classifiers.
- The global classifier uses three 3 × 3 convolutional layers, global average pooling, and a linear layer.
- Its softmax output supports global domain prediction, while the extracted context vector has 128 dimensions.
- The local classifier uses three 1 × 1 convolutional layers and sigmoid activation for local domain prediction.
2. Pixel-level Adaptation
The pixel-level adaptation experiments use CycleGAN to translate source images toward the target domain, then train Faster R-CNN with source, translated-source, and target images.
- CycleGAN is trained on all images from both domains for 10 epochs.
- Translated source images are used to train the Faster R-CNN model.
- Because some translations corrupt objects with large noise, training combines source, translated-source, and target images.
- Translated images train detection modules but are excluded from domain classification.
3. Additional Results
Additional experiments report source-domain preservation, target-domain and pixel-level adaptation results, parameter sensitivity, and domain-evidence visualizations across multiple adaptation settings.
- Additional Results: The method does not significantly degrade source-domain detection performance across three adaptation scenarios.
- Additional Results: For Clipart, CycleGAN-generated training images did not improve performance, possibly because target styles were diverse and substantially different.
- Additional Results: On Watercolor, pixel-level adaptation greatly improved performance to almost oracle-level performance.
- Parameter Sensitivity: The method outperformed the baseline domain classifier for every λ value from 0 to 1.
- Parameter Sensitivity: For Sim10k to Cityscape, peak performance occurred around γ = 3.0, with AP of 42.3.
- Domain Evidence: Grad-Cam visualizations show dataset-dependent domain-classifier behavior while the feature extractor partially fools the classifier.
- Additional Results: On PASCAL VOC to Clipart, the method preserves source performance while BDC-Faster and DC-Faster degrade it.