PaperlayerView exact revision on arXiv

Source-linked AI summary

Strong-Weak Distribution Alignment for Adaptive Object Detection

Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, Kate Saenko

arXiv:1812.04798v3cs.CV

TL;DR

Object detectors rely on costly bounding-box annotations, and dataset bias can limit their transfer to new domains. The Strong-Weak Domain Alignment model combines weak global alignment with strong local alignment. Across several datasets, it outperformed existing methods by a large margin.

  • Problem

    Object detection domain adaptation must address annotation burden and dataset bias, while global source-target matching may be unsuitable when domains differ in scene layout or object combinations.

    Source: §1 · pp. 1–2

  • Method

    The Strong-Weak Domain Alignment model combines weak global alignment that emphasizes globally similar images with strong local alignment over local receptive fields.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Across several datasets, the method outperformed existing methods by a large margin, and experiments verified the effectiveness of weak global and strong local alignment.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Across three adaptation scenarios, the method did not significantly degrade detection performance on the source domain.

    Source: §3 · p. 11

    Detection can remain unsuccessful when object appearance differs substantially, and the detector may produce multiple predictions for one object or merge neighboring objects into one box.

    Source: §4.4 · p. 8

Abstract

from arXiv · show

We propose an approach for unsupervised adaptation of object detectors from label-rich to label-poor domains which can significantly reduce annotation costs associated with detection. Recently, approaches that align distributions of source and target images using an adversarial loss have been proven effective for adapting object classifiers. However, for object detection, fully matching the entire distributions of source and target images to each other at the global image level may fail, as domains could have distinct scene layouts and different combinations of objects. On the other hand, strong matching of local features such as texture and color makes sense, as it does not change category level semantics. This motivates us to propose a novel method for detector adaptation based on strong local alignment and weak global alignment. Our key contribution is the weak alignment model, which focuses the adversarial alignment loss on images that are globally similar and puts less emphasis on aligning images that are globally dissimilar. Additionally, we design the strong domain alignment model to only look at local receptive fields of the feature map. We empirically verify the effectiveness of our method on four datasets comprising both large and small domain shifts. Our code is available at \url{https://github.com/VisionLearningGroup/DA_Detection}

1. Introduction

Object detection adaptation is needed because precise box annotation is costly and models trained in label-rich domains may generalize poorly to new data. The paper proposes combining weak global alignment with strong local alignment to reduce domain gaps without forcing dissimilar scenes to match.

Introduction

  • Precise bounding-box annotation makes object detection costly, while dataset bias can limit transferred models’ generalization to novel data.The paper frames transfer from label-rich domains as an attractive way to reduce annotation costs.Source: §1 · p. 1
  • Prior UDA methods align feature [32] or image [21] [14] distributions, often adversarially to reduce discrepancy, but strong alignment can hurt open-set or partial adaptation [34] [42].The authors argue full alignment is reasonable mainly in closed problems where source and target share categories and prior label distributions.Source: §1 · pp. 1–2
  • Global image-level alignment in detection can force backgrounds and scene layouts to match; Adaptive Faster RCNN [5] aligns features at both global-image and instance-object scales.Its domain classifier distinguishes source from target examples while the feature extractor learns to deceive it.Source: §1 · p. 2
  • Global matching may suit small appearance shifts but hurt under larger scene-layout, object-count, or co-occurrence shifts; strong local texture and color alignment can reduce the domain gap without changing category information.Here, “local” means texture or color features with small receptive fields, not instance-level object features.Source: §1 · p. 2
  • The Strong-Weak Domain Alignment model partially aligns globally similar images while directing adversarial loss away from dissimilar ones, and strictly aligns local features using a classifier restricted to local receptive fields.The method is evaluated on adaptation between both similar and dissimilar domains.Source: §1 · p. 2

2. Related Work

Prior domain-adaptation methods often match source and target feature distributions, while this paper proposes weak global and strong local alignment for object detection. It also regularizes the domain classifier with source detection loss.

Object Detection

  • Prior unsupervised detector adaptation aligned proposal-cropped instance features [5], but the paper omits this because target proposals lack ground truth and imprecise localization may hurt performance.Source: §2 · p. 2
  • The method uses Faster-RCNN [30] as its base detector and is intended to apply to other two-stage and one-stage detectors, including YOLO [29] and SSD [22].Source: §2 · p. 2

Domain Adaptation

  • Domain-adaptation methods often use adversarial domain classifiers to strictly align feature distributions [9] [41] [40]; Long et al. designed full feature matching for image classification [24].Source: §2 · p. 2
  • The paper proposes weak alignment for global features and strict local alignment to match image style across domains.The method also regularizes the domain classifier using detection loss on source examples.Source: §2 · pp. 2–3
  • Task-specific classification-loss regularization can stabilize adversarial training [36], motivating the paper's detection-loss regularization of the domain classifier on source examples.Source: §2 · p. 3

3. Method

Strong-Weak DA combines weak global alignment in high-level features with strong local alignment in low-level features, and uses the detection loss to stabilize domain classifiers.

3. Method

  • The architecture extracts global features before the RPN and local features from lower layers, applying weak global and strong local alignment in their respective feature spaces.The model also stabilizes domain classifiers with detection loss.Source: §3 · p. 3

3.1. Weak Global Feature Alignment

  • The weak global classifier uses a modulating factor, exemplified by focal loss [19], to emphasize hard-to-classify examples and reduce the influence of easy ones.Well-classified examples receive very small gradients, limiting attempts to align them across domains.Source: §3.1 · p. 3

3.2. Strong Local Feature Alignment

  • A fully convolutional local classifier with 1×1 kernels predicts domains at each feature-map location and uses least-squares loss to align receptive fields across domains.It operates on F1’s low-level feature map, preserving its width and height in the domain prediction map.Source: §3.2 · p. 4

3.3. Context Vector based Regularization

  • Context regularization concatenates whole-image context vectors from the domain classifiers with region-wise features and trains the classifiers with source detection loss as well as domain loss.The vectors are extracted from the classifiers’ middle layers and forwarded during testing.Source: §3.3 · p. 4

3.4. Overall Objective

  • The overall objective combines source detection loss with adversarial alignment loss, with λ controlling their trade-off; gradient reversal flips gradients, and each mini-batch contains one source and one target example.The detection loss covers region proposals and final classification and localization.Source: §3.4 · pp. 4–5

4. Experiments

Experiments across four domain shifts show that weak global alignment can outperform strict alignment, while local alignment and context-vector regularization further improve adaptation in some settings. Results also indicate that the value of strong alignment depends on how source and target domains differ.

4. Experiments

  • The approach is evaluated on four domain shifts—PASCAL to Clipart, PASCAL to Watercolor, Cityscapes to FoggyCityscapes, and GTA to Cityscapes—with all models trained for 70K iterations.Implementation uses a 600-pixel shorter image side, with default λ = 1.0 and γ = 5.0.Source: §4 · p. 5

4.1. Adaptation between dissimilar domains

  • On PASCAL-to-Clipart, focal loss improved mAP from 25.6 to 36.4; on PASCAL-to-Watercolor, local alignment added about 3% and the method outperformed baselines.Strict alignment reduced source-domain performance, whereas the proposed method did not; feature visualizations showed that weak alignment left most target features separated when domains were dissimilar.Source: §4.1 · pp. 5–6, Table 3 · p. 6

4.2. Adaptation between similar domains

  • For Cityscapes-to-FoggyCityscapes, strong local alignment alone reached 27.9 mAP and adding weak global alignment raised it to 34.3, near the labeled-target oracle.Because fog is a local-level shift and source and target images share layouts and objects, strict alignment also helped relative to source-only training.Source: Table 4 · p. 6, §4.2 · pp. 6–7

4.3. Adaptation from synthetic to real images

  • On Sim10k-to-Cityscapes, weak global alignment reached 36.4 AP versus 31.8 for BDC-Faster, and setting focal-loss γ = 3.0 further improved performance.Exponential focal loss also improved over baseline models; context-vector regularization and local alignment further improved results, while instance-level adaptation showed no positive effect.Source: Table 5 · p. 7, §4.3 · p. 7

4.4. Analysis

  • Detection examples show that the model can localize objects across styles, but struggles with large appearance differences, duplicate predictions, and boxes spanning neighboring bicycles.Grad-CAM analysis for Sim10k-to-Cityscapes suggests that the feature extractor partially aligns global features around cars to deceive the domain classifier.Source: §4.4 · p. 8

5. Conclusion

The paper proposes unsupervised detector adaptation through strong local alignment and weak global alignment. Experiments show the method outperformed existing methods by a large margin across several datasets.

5. Conclusion

  • The method outperformed existing methods by a large margin across several datasets.Extensive experiments also verified the effectiveness of weak global and strong local alignment.Source: §5 · p. 8
  • Weak alignment focuses adversarial loss on globally similar images and reduces emphasis on globally dissimilar ones.Source: §5 · p. 8
  • Strong domain alignment examines only local receptive fields of the feature map.Source: §5 · p. 8

Supplemental Material · 1. Network Architecture

The supplemental material specifies the global and local domain-classifier architectures, including their convolutional layouts, output activations, and 128-dimensional context vectors. It also describes how classifier inputs are selected for VGG16 and ResNet101.

1. Network Architecture

  • The domain-classifier architecture is shared across Faster R-CNN with ResNet101 and VGG16, with channel sizes adjusted; local-classifier inputs come from VGG16 conv3 3 or ResNet101’s last res2c layer.Source: §1 · p. 11
  • The global domain classifier begins with a 3 × 3 × 512 convolution at stride 2 and padding 1, followed by batch normalization, ReLU, and dropout.Source: Table 6 · p. 11
  • The global classifier uses three convolutional layers, global average pooling, and a linear layer, with batch normalization, ReLU, and dropout after each convolution.Source: §1 · p. 11
  • The global classifier applies softmax, and its context vector is extracted after average pooling, yielding 128 dimensions.Source: §1 · p. 11
  • The local classifier uses three convolutional layers with 1 × 1 kernels; its specified convolution is 1 × 1 × 256 at stride 1 and padding 0.Source: §1 · p. 11
  • The local classifier applies sigmoid and extracts a 128-dimensional context vector before its last convolutional layer.Source: §1 · p. 11

2. Pixel-level Adaptation

Pixel-level adaptation uses CycleGAN-translated source images for detector training, while retaining original source and target images to mitigate corrupted translations.

  • CycleGAN was trained for 10 epochs on all training images from both domains, and translated source images were used to train Faster R-CNN.Source: §2 · p. 11
  • Because some translated images were corrupted by large noise that hid cars, each mini-batch included source, translated source, and target images to suppress their effect.Source: §2 · p. 11
  • Translated source images contributed to detection loss only and were not used for domain classification.Source: §2 · p. 11

3. Additional Results

Additional experiments report source-domain performance, target-domain adaptation results, parameter sensitivity, and visualizations of domain evidence.

  • CycleGAN-generated training images did not improve Clipart adaptation; the large style gap and diverse target styles may make generated images unsuitable.Source: §3 · pp. 11–12
  • Across three scenarios, our method did not significantly degrade source-domain detection; on PASCAL VOC, BDC-Faster and DC-Faster did degrade it.Source: §3 · p. 11, Table 7 · p. 12, Table 10 · p. 13
  • With VGG adaptation from PASCAL VOC to Clipart, performance largely improved with our method.Source: Table 9 · p. 12, §3 · p. 12
  • Pixel-level adaptation greatly improved Watercolor performance to almost oracle-level.Source: §3 · p. 12, Table 10 · p. 13

Parameter Sensitivity

Domain Evidence

  • Domain-classifier behavior varied by dataset, while the feature extractor partially fooled the classifier.Source: §3 · p. 12, Figure 7 · p. 14

Related papers

Loading 1812.04798v3…