PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Adaptation for Object Detection via Style Consistency

Adrian Lopez Rodriguez, Krystian Mikolajczyk

arXiv:1911.10033v1cs.CV

TL;DR

Object detectors can perform poorly across domains, and varied target styles make pixel-level adaptation alone insufficient. The paper combines style transfer and feature consistency with robust pseudo-labelling to adapt low- and high-level features. The combined approach reports the highest mAP on the cross-domain benchmark, exceeding state-of-the-art methods across its three datasets.

  • Problem

    Object detectors can perform poorly when training and evaluation use different domains, while varied target styles and high-level appearance changes pose adaptation challenges.

    Source: §1 · pp. 1–2

  • Method

    The method transfers source images into target styles, enforces detector feature consistency, then adapts classifiers using robust pseudo-labels with positive and negative sampling.

    Source: §1 · p. 2, §4.2 · p. 5, §4.3 · p. 6

  • Results

    The combined approach reports higher mAP than state-of-the-art methods by 6.7% in Clipart1k, 4.0% in Watercolor2k, and 9.6% in Comic2k.

    Source: §5.1 · p. 8

  • Takeaways & Limitations

    The approach achieves its highest cross-domain benchmark performance when style transfer, feature consistency, and pseudo-labelled target instances are combined.

    Source: §5.1 · p. 8, §6 · p. 10

    Improvements on the driving datasets are limited where target instances have little style variation and low-level feature mismatch is smaller than in the cross-domain dataset.

    Source: §5.1 · p. 10

Abstract

from arXiv · show

We propose a domain adaptation approach for object detection. We introduce a two-step method: the first step makes the detector robust to low-level differences and the second step adapts the classifiers to changes in the high-level features. For the first step, we use a style transfer method for pixel-adaptation of source images to the target domain. We find that enforcing low distance in the high-level features of the object detector between the style transferred images and the source images improves the performance in the target domain. For the second step, we propose a robust pseudo labelling approach to reduce the noise in both positive and negative sampling. Experimental evaluation is performed using the detector SSD300 on PASCAL VOC extended with the dataset proposed in arxiv:1803.11365 where the target domain images are of different styles. Our approach significantly improves the state-of-the-art performance in this benchmark.

1 Introduction

Object detectors trained and tested across domains can perform poorly, while unsupervised adaptation uses labelled source data and unlabelled target data. This paper combines style transfer and feature consistency with robust pseudo-labelling to address low- and high-level differences.

1 Introduction

  • Object detectors can perform poorly when training and evaluation use different domains, despite strong results on datasets with similar train and test images.The paper situates this challenge against detectors such as Faster R-CNN and SSD.Source: §1 · p. 1
  • The domain-adaptation setting uses a fully annotated source set and a non-annotated target set to train a model for target-domain performance.The source and target share object classes.Source: §1 · p. 1, Figure 1 · p. 2
  • Style transfer can adapt source images instance by instance, which suits target datasets containing varied styles such as texture, shape, pattern, illumination, or color changes.The cited generative models typically produce one translated image per source image and cannot target a specific style.Source: §1 · pp. 1–2
  • High-level feature adaptation is also needed because object appearance can change in ways such as shape, beyond low-level visual differences.The paper identifies pseudo-labelling target instances with the source-trained detector as a way to adapt those features.Source: §1 · p. 2
  • The approach first transfers source images into target styles and enforces feature consistency, then uses robust pseudo-labels with low-noise positive and negative sampling.The target pseudo-labels are used with style transfer to train on style-transferred target images.Source: §1 · p. 2

2 Related work

Related work spans object detection, style transfer, and domain adaptation. Prior domain-adaptation methods include feature alignment, image translation, and approaches specifically targeting object detection.

2 Related work

  • Object detectors include two-stage methods such as R-CNN, Fast R-CNN, and Faster R-CNN [12] [31], and one-stage methods such as YOLO [30] and SSD [25].Faster R-CNN introduced a Region Proposal Network, while one-stage detectors omit an ROI proposal network for more efficient inference.Source: §2 · p. 2
  • Style-transfer work ranges from content-and-style-loss methods to feed-forward extensions [6] [19] [45], unseen-style generalization [17], and translation between real-image styles [22] [47].Related work also includes joint pixel and feature adaptation using a CycleGAN-like approach [16].Source: §2 · pp. 2–3
  • Image-classification domain adaptation has used domain confusion [8] [41], adversarial alignment [42] [35], teacher-based refinement [38], co-training [34], and transformation consistency [33].These approaches align features or refine predictions using source and target data.Source: §2 · p. 3
  • Object-detection adaptation includes synthetic-to-real training [28], merging annotated datasets [29], and instance- or image-level adaptation for Faster R-CNN [2].CycleGAN-based pixel adaptation of PASCAL VOC and a cross-domain dataset with comic, watercolor, and clipart styles were introduced in [18].Source: §2 · p. 3
  • Other detection approaches adapt GANs for pixel translation [43] or use focal loss for high-level feature adaptation [36].A related style-transfer method for semantic segmentation [46] does not enforce feature-level style consistency.Source: §2 · p. 3

3 Preliminaries

The experiments use adaptive instance normalization for style transfer and SSD as the detector. Style transfer aligns feature statistics, while SSD combines confidence and localization losses over matched boxes.

3 Preliminaries

  • Adaptive instance normalization transfers a target image’s style to a source image using a feed-forward network and a decoder.It matches the source VGG19 feature map’s mean and variance to those of the target image; the transfer loss combines content and style losses.Source: §3 · p. 3
  • The style-transfer loss is the sum of content loss LCont and style loss LStyle; the method uses ImageNet-pretrained VGG19 features up to relu4_1, then decodes an affine-transformed source feature map.Source: §3 · p. 3
  • SSD uses anchor boxes from multiple feature maps and combines confidence and localization losses over matched boxes.The confidence loss is averaged over positive matched boxes; experiments use α = 1 and a 3:1 ratio of negative to positive object boxes.Source: §3 · p. 4
  • For a single matched box, SSD’s confidence loss is the cross-entropy loss of the predicted score for the ground-truth class.The score is normalized with softmax.Source: §3 · p. 4

4 Method

The method first builds robustness to low-level style changes, then adapts detection to target-domain high-level differences with pseudo-labelled examples.

4 Method

  • The two-step method trains with style-transferred images and feature consistency, then fine-tunes on pseudo-labelled target examples.The first step targets low-level variation; the second adapts to high-level changes.Source: §4 · p. 4

4.1 Style Transfer with Feature Consistency

  • Feature consistency reduces distance between source and style-transferred images’ high-level features while training on both with their shared ground-truth labels.The consistency loss acts on SSD300’s last six feature maps, where detections are generated.Source: §4.1 · p. 5

4.2 Robust Pseudo Labelling

  • Pseudo-labelled target boxes are accepted above a positive-confidence threshold, while a separate negative threshold avoids treating uncertain detections as negatives.At confidence 0.8, precision is at least 0.8 in both source and DTI domains, while recall drops from 0.6 to 0.16; negative sampling maintains a 3:1 negative-to-positive ratio.Source: §4.2 · pp. 5–6

4.3 Combined Domain Adaptation Loss

  • The combined loss weights style-transfer, feature-consistency, source-detector, and robust-pseudo-labelling losses; the first two also apply to target-style pairs during the second training step.The pseudo-labelling loss uses target images and their style-transferred counterparts.Source: §4.3 · p. 6

5 Experiments

The experiments evaluate the two-step adaptation approach on cross-domain object-detection benchmarks and driving datasets. The full method outperforms state-of-the-art methods on three style-diverse datasets, while driving-dataset gains vary by setting.

5 Experiments

  • Experiments use SSD300 with VOC source data and Clipart1k, Watercolor2k, and Comic2k targets, comparing against prior methods and varying adaptation components.Driving experiments instead use SSD512, with batch size reduced to 2 for memory constraints.Source: §5 · pp. 6–8, Table 1 · p. 7, Table 2 · p. 8

5.1 Domain Adaptation for Object Detection Results

  • Style transfer improves performance over DTI in all three datasets, consistency adds gains, and using transferred target instances with pseudo labels beats using target instances alone.Consistency raises Clipart1k performance by 2.0%; using one randomly chosen target style with consistency yields 36.3%, 6.3% below using all styles.Source: §5.1 · p. 8
  • The tested pseudo-labelling method from [18] performs worse than using no pseudo labels, with discarded predictions, false negatives, and false positives identified as reasons.The method takes only the top-confidence prediction per class, lacks a negative-sampling constraint, and assumes every class is present without image-level labels.Source: §5.1 · p. 8
  • The full method exceeds state-of-the-art mAP by 6.7% in Clipart1k, 4.0% in Watercolor2k, and 9.6% in Comic2k.The reported comparisons cover Clipart1k and the Watercolor2k and Comic2k results.Source: Table 1 · p. 7, §5.1 · p. 8, Table 2 · p. 8
  • In Clipart1k, performance is stable for pseudo-label thresholds sĉ ≥ 0.2; under standard hard-negative mining with s−̂ = 0, performance is 44.0%.Low s−̂ values give slightly lower performance.Source: §5.1 · p. 9
  • In Foggy Cityscapes, adaptation improves performance by 4.1% over the source-trained model; gains are limited in Sim10k→Cityscapes.The authors attribute limited driving-dataset improvements to less style variation, lower low-level mismatch, and distortions in style-transferred images.Source: §5.1 · pp. 9–10

6 Conclusion

The approach combines low-level style-transfer adaptation with high-level adaptation through robust pseudo-labelling. The combined method achieves the highest mAP on the cross-domain dataset and improves over the source-trained model on driving datasets.

6 Conclusion

  • The approach combines low-level adaptation via style transfer with high-level adaptation via robust pseudo-labelling, and achieves its highest performance when both are used.It obtains the highest mAP on the cross-domain dataset and improves over the source-trained model in driving datasets.Source: §6 · p. 10

Related papers

Loading 1911.10033v1…