Source-linked AI summary
Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector
Cheng-Chun Hsu, Yi-Hsuan Tsai, Yen-Yu Lin, Ming-Hsuan Yang
TL;DR
Domain adaptive detectors must handle domain shifts in object appearance, viewpoints, and backgrounds, while existing image- and proposal-level alignment can mix foreground with background. This paper predicts pixel-wise objectness and centerness to guide center-aware adversarial alignment toward discriminative object regions, and reports favorable or state-of-the-art performance across benchmark settings. Its results also show that center-aware and global alignment together improve adaptation, although underrepresented categories can remain weaker.
Problem
Domain adaptive object detection must handle shifts in object appearance, viewpoints, and backgrounds, but image- and proposal-level alignment can mix foreground and background signals.
Method
The method predicts pixel-wise objectness and centerness, then uses these maps to guide adversarial alignment toward high-confidence, centered object regions.
Results
The method achieves favorable or state-of-the-art performance across numerous domain-adaptation benchmark datasets and settings.
Takeaways & Limitations
Center-aware alignment focuses adaptation on discriminative foreground regions, and combining global with center-aware alignment improves domain adaptation for object detection.
Takeaways & Limitations
Underrepresented categories can perform worse because fewer source-domain foreground pixels receive attention; stronger backbones or category priors are suggested.
Abstract
from arXiv · showhide
A domain adaptive object detector aims to adapt itself to unseen domains that may contain variations of object appearance, viewpoints or backgrounds. Most existing methods adopt feature alignment either on the image level or instance level. However, image-level alignment on global features may tangle foreground/background pixels at the same time, while instance-level alignment using proposals may suffer from the background noise. Different from existing solutions, we propose a domain adaptation framework that accounts for each pixel via predicting pixel-wise objectness and centerness. Specifically, the proposed method carries out center-aware alignment by paying more attention to foreground pixels, hence achieving better adaptation across domains. We demonstrate our method on numerous adaptation settings with extensive experimental results and show favorable performance against existing state-of-the-art algorithms.
1 Introduction
Domain adaptation is needed for detectors facing changes in object appearance, viewpoints, backgrounds, and weather, but existing image- and instance-level alignment can mix foreground and background signals. The proposed center-aware framework predicts pixel-wise objectness and centerness to align discriminative foreground regions, with favorable results across benchmark settings.
- Motivation: Object detectors must adapt to unseen domains containing variations in object appearance, viewpoints, backgrounds, and weather.This need is illustrated by autonomous-driving detectors trained under particular weather conditions.
- Limitations of Existing Alignment: Image-level alignment adapts global features but may be less effective under large domain gaps and can emphasize noisy background pixels.Global alignment considers foreground and background pixels simultaneously.
- Limitations of Existing Alignment: Instance-level alignment pools proposal pixels, potentially mixing foreground and background signals because target-domain proposals may contain more background.This can confuse adaptation toward a source distribution containing mostly foreground pixels.
- Proposed Approach: The proposed method predicts pixel-wise objectness and centerness to focus alignment on centered, discriminative foreground regions.These regions are intended to be less sensitive to irrelevant target-domain background pixels.
- Evaluation: Experiments cover Cityscapes Foggy, Cityscapes, and KITTI adaptation settings, with favorable performance against existing state-of-the-art algorithms.The paper also reports ablations evaluating the usefulness of individual components.
2 Related Work
Related work spans anchor-based and anchor-free object detectors and multiple domain-adaptation alignment strategies. The reviewed approaches include global, instance-level, low-level, pixel-level, and pseudo-label-based methods, while proposal-based alignment remains vulnerable to background effects and inaccurate target proposals.
- Object Detection: Object detectors are categorized into anchor-based methods that generate proposals and anchor-free methods that directly localize objects with fully convolutional networks.Faster R-CNN exemplifies anchor-based detection, while FCOS uses per-pixel prediction without proposal generation.
- Object Detection: Anchor-free methods can use center or corner localization, while FCOS directly predicts class and offset information per pixel.Some keypoint-based approaches require post-processing to group detected points.
- Domain Adaptive Object Detection: Prior domain-adaptation methods use global, instance-level, low-level, pixel-level, or pseudo-label-based alignment schemes.The proposed center-aware alignment is characterized by pixel-wise objectness and centerness.
- Domain Adaptive Object Detection: Global feature alignment can suffer from cross-domain foreground variation and background clutter, motivating lower-level or more localized alignment strategies.Instance-level methods may also introduce clustering hyperparameters or lack end-to-end training.
- Domain Adaptive Object Detection: Proposal-based instance alignment is sensitive to inaccurate target proposals and foreground/background mixtures, while some one-stage methods only partially alleviate background effects.These limitations motivate considering every pixel during feature alignment.
3 Proposed Method
The framework combines global feature alignment with center-aware alignment, using pixel-wise objectness and centerness to emphasize likely foreground regions while training detection only on labeled source images.
- Center-aware Alignment: The fully-convolutional module P predicts classification and centerness maps from feature maps to support center-aware feature construction.Each spatial location provides objectness and centerness information used to identify discriminative object regions.
- Overall Framework: The method jointly performs global alignment and center-aware alignment with separate discriminators and domain prediction losses.Global alignment uses DGA and LGA, while center-aware alignment uses DCA and LCA.
- Global Feature Alignment: Global alignment uses a spatial domain-classification map and adversarial training through a gradient reversal layer to reduce source-target feature differences.DGA predicts a domain label at each feature-map location; its loss is minimized for the discriminator and maximized by the feature extractor.
- Center-aware Alignment: The center-aware map combines class-agnostic objectness with centerness to identify potential object locations and guide feature weighting.Objectness is obtained by sigmoid activation followed by a maximum over categories, while centerness is sigmoid-activated.
- Center-aware Alignment: The weighted feature map is fed to DCA, so alignment emphasizes pixel-wise features likely to belong to objects rather than mixing foreground and background.The center-aware map is duplicated across feature channels before element-wise multiplication with F.
- Detection Objective: The source-domain detector objective combines classification, regression, and centerness losses, while the total objective weights detection, global alignment, and center-aware alignment.The detection loss is applied using source images and ground-truth boxes; α and β balance the alignment terms.
4 Experimental Results
Experiments across weather, synthetic-to-real, and cross-camera adaptation show that center-aware alignment complements global alignment and generally improves detection, while multi-scale alignment helps handle varied object sizes. The method performs favorably against existing approaches, with gains strongest in several challenging metrics and settings.
- Experimental Settings: Experiments cover weather, synthetic-to-real, and cross-camera adaptation on road-scene datasets.The settings are Cityscapes→Foggy Cityscapes, Sim10k→Cityscapes, and KITTI→Cityscapes.
- Overall Performance: 17.6% improvement in mAPr 0.5 over the non-adapted baseline is reported for weather adaptation.Using VGG-16, GA + CA performs best compared with other methods.
- Overall Performance: 8.8% and 9.7% gains over the non-adapted baseline are reported for cross-camera adaptation with VGG-16 and ResNet-101, respectively.Adding center-aware alignment consistently improves the results.
- Alignment Analysis: Global and center-aware alignment are complementary because they emphasize image-level distributions and likely foreground pixels, respectively.Combining GA and CA achieves the best performance, whereas global alignment may attend excessively to background pixels.
- Metric Analysis: 2.2%, 1.4%, and 3.9% improvements over GA-only are obtained by adding CA under the more challenging mAPr M metric.Similar observations are reported for the KITTI adaptation case.
- Multi-scale Alignment: Multi-scale alignment is effective for varying object sizes because different scales specialize in different size ranges.F 3∼F 5 performs better for mAPr M, while F 5∼F 7 is better for mAPr L because it handles larger objects.
5 Conclusions
The paper presents center-aware feature alignment for domain adaptive object detection, using pixel-wise object regions as guidance and combining global and center-aware alignment across object scales.
- 5 Conclusions: The method generates pixel-wise maps to localize object regions and guides adversarial alignment toward features from those regions.It uses center-aware regions so the discriminator focuses on object-region features.
- 5 Conclusions: A multi-scale feature alignment scheme is designed to handle objects of different sizes.
- 5 Conclusions: Combining global and center-aware alignment improves domain adaptation for object detection and achieves state-of-the-art performance across numerous benchmark datasets and settings.