PaperlayerView exact revision on arXiv

Source-linked AI summary

Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation

Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, Kiyoharu Aizawa

arXiv:1803.11365v1cs.CV

TL;DR

The paper asks whether common objects can be detected across image domains when the source has instance annotations but the target has only image-level annotations. It proposes progressive adaptation using domain transfer and pseudo-labeling, and reports approximately 5 to 20 percentage points of mAP improvement over best-performing baselines on three newly collected domains.

  • Problem

    Cross-domain weakly supervised object detection seeks accurate target-domain detection using source-domain instance annotations and only a small number of target-domain image-level annotations.

  • Method

    A source-trained fully supervised detector is sequentially fine-tuned on domain-transferred images and pseudo-labeled target-domain images.

  • Results

    Approximately 5 to 20 percentage points of mAP improvement are achieved over the best-performing baselines across Clipart1k, Watercolor2k, and Comic2k.

  • Takeaways & Limitations

    The framework provides a simple baseline for cross-domain weakly supervised object detection and is evaluated across three visual domains.

  • Takeaways & Limitations

    Pseudo-labeling uses only the top-1 bounding box per class, so other instances may be treated as negative samples.

Abstract

from arXiv · show

Can we detect common objects in a variety of image domains without instance-level annotations? In this paper, we present a framework for a novel task, cross-domain weakly supervised object detection, which addresses this question. For this paper, we have access to images with instance-level annotations in a source domain (e.g., natural image) and images with image-level annotations in a target domain (e.g., watercolor). In addition, the classes to be detected in the target domain are all or a subset of those in the source domain. Starting from a fully supervised object detector, which is pre-trained on the source domain, we propose a two-step progressive domain adaptation technique by fine-tuning the detector on two types of artificially and automatically generated samples. We test our methods on our newly collected datasets containing three image domains, and achieve an improvement of approximately 5 to 20 percentage points in terms of mean average precision (mAP) compared to the best-performing baselines.

1. Introduction

The paper introduces cross-domain weakly supervised object detection, where source-domain instance annotations and target-domain image-level annotations support detection across visual domains. Its two-step adaptation generates target-like, instance-annotated samples through domain transfer and pseudo-labeling, improving mAP over existing baselines.

  • Motivation: Fully supervised object detectors require many images with instance-level labels containing object classes and bounding boxes.These annotation requirements make detection data-hungry.
  • Task definition: Cross-domain weakly supervised detection uses source-domain instance annotations, target-domain image-level annotations, and shared or overlapping object classes.The objective is accurate target-domain detection under these annotation conditions.
  • Method: The framework fine-tunes a source-trained detector on artificially and automatically generated target-domain samples with instance-level annotations.It uses domain transfer to make source images resemble the target domain, then pseudo-labeling to annotate target images.
  • Results: Approximately 5 to 20 percentage points of mAP improvement are achieved over the best-performing baselines across the collected datasets.The datasets cover clipart, watercolor, and comic image domains.
  • Method: The proposed framework includes domain transfer and pseudo-labeling as sequential stages of progressive domain adaptation.These stages generate complementary training samples for detector fine-tuning.
  • Resources: The paper constructs fully instance-level annotated datasets with multiple object classes across three domains far from natural images.These datasets provide evaluation resources for the proposed task.

2. Related Work

Prior work includes fully supervised, weakly supervised, and unsupervised domain adaptation approaches, but cross-domain detection remains constrained by annotation and benchmark limitations. The paper positions its task as distinct from related adaptations and introduces multi-class, multi-instance datasets across visual domains.

  • Fully supervised detection: Fully supervised detectors use two-stage or single-stage architectures but require large datasets with instance-level annotations.Examples include R-CNN-family detectors, SSD, YOLOv2, and RetinaNet trained on datasets such as PASCAL VOC, MSCOCO, and OpenImages.
  • Weak supervision: Weakly supervised object detection trains from image-level labels and commonly uses region extraction, feature extraction, region selection, and multiple instance learning.These methods address annotation scarcity without providing bounding boxes for training images.
  • Cross-domain adaptation: Adapting a detector to the target domain is essential because using a detector without target-domain training or fine-tuning causes a significant performance drop.The paper distinguishes its cross-domain setting from classifier-to-detector adaptation within one domain and from adaptation using only source image-level annotations.
  • Datasets: Existing cross-domain detection datasets have limitations, including single-class evaluation in People-Art and a one-instance-per-image assumption in Photo-Art.The paper addresses this gap with datasets containing multiple common classes across visual domains.
  • Unsupervised adaptation: Unsupervised domain adaptation learns domain-invariant models from annotated source data and unannotated target images, often through feature-distribution matching.The cited approaches use techniques such as maximum mean discrepancy or domain-classifier networks.

3. Dataset

The paper constructs three target-domain datasets—Clipart1k, Watercolor2k, and Comic2k—to evaluate cross-domain object detection, with target classes either matching or subsetted from the source classes.

  • Dataset overview: 5,000 images and 12,869 instance-level annotations were collected across the three target-domain datasets.
  • Benchmark scope: The datasets are intended as benchmarks for domain adaptation, fully supervised detection, weakly supervised detection, and semi-supervised detection.
  • Clipart1k: Clipart1k uses the same target classes as the source domain and combines images from CMPlaces, Openclipart, and Pixabay.
  • Comic2k and Watercolor2k: Comic2k and Watercolor2k use subsets of the source classes, focusing on bicycle, bird, cat, car, dog, and person.
  • Comic2k and Watercolor2k: For watercolor and comic domains, images with at least one target class were filtered using BAM! labels, then up to 2,000 images per domain received instance-level annotations.

4. Proposed Method

The proposed method progressively adapts a source-domain fully supervised detector using two kinds of automatically generated target-domain samples. Domain transfer preserves bounding-box correctness, while pseudo-labeling provides target-domain image quality but less accurate boxes.

  • Proposed Method: The framework fine-tunes a source-pretrained fully supervised detector on artificially generated target-domain samples with instance-level annotations.
  • Domain Transfer: Domain transfer converts source images into target-domain-like images while retaining correctly annotated bounding boxes.
  • Progressive Adaptation: Progressive adaptation pre-trains on source annotations, fine-tunes on domain-transfer samples, and then fine-tunes on pseudo-labeled samples.
  • Domain Transfer: CycleGAN learns unpaired mappings between source and target image domains to generate target-domain-like training images.
  • Effect of Adaptation: The source-only detector mainly fails through confusion with classes and backgrounds, while pseudo-label fine-tuning dramatically reduces this confusion.
  • Pseudo-Labeling: Pseudo-labeling generates pseudo instance-level annotations from target-domain images using detector outputs and image-level class annotations.

5. Experiments

Experiments evaluate the framework on Clipart1k, Watercolor2k, and Comic2k using multiple detectors and baselines. The two-step adaptation improves mAP, with DT and PL contributing complementary gains and remaining effective across domains.

  • Evaluation setup: Experiments use AP and mAP to evaluate methods across Clipart1k, Watercolor2k, and Comic2k.The study compares multiple methods and examines generality across domains.
  • Compared methods: The comparisons include a baseline SSD300 detector, an ideal target-domain supervision case, weakly supervised detectors, UDA, and ensembles.SSD300 is the baseline FSD, while the ideal case uses target-domain instance annotations.
  • Clipart1k results: PL improves baseline SSD300 by 9.6 percentage points in mAP, while DT+PL reaches 46.0% mAP.The combined approach is complementary: its mAP is 19.2 percentage points above baseline and 9.4 points below the ideal case.
  • Clipart1k results: DT improves baseline SSD300 by 11.2 percentage points in mAP without modification.DT transfers source-domain images with instance annotations into the target style for fine-tuning.
  • Clipart1k results: Target-domain image-level annotations are essential for PL because pseudo-labeling without them introduces substantial inaccuracy and harms performance.PL combines target image-level annotations with predictions from an FSD already fine-tuned using DT.
  • Generality across detectors: The framework generalizes across SSD300, Faster R-CNN, and YOLOv2, although the performance gain is larger for SSD300.Ensembling SSD300 and Faster R-CNN yields 30.2% mAP, while all three detectors yield 31.0% mAP, described as less remarkable than DT+PL.
  • Error analysis: PL reduces confusion with other classes, especially among more confident detections, while DT improves performance particularly for less-confident detections.This error pattern supports complementary roles for the two adaptation steps.
  • Generality across domains: Both DT and PL work in Watercolor2k and Comic2k, while extra noisy image-level labels can further improve detection without manual target-domain instance annotation.On BAM! extra images, methods using noisy labels sometimes outperform the ideal case trained on 1,000 clean instance-level annotations.

6. Discussion

Pseudo-labeling uses only the top-1 bounding box for each class, treating other instances as negative samples and leaving improved localization accuracy for future work.

  • Pseudo-labeling employs only the top-1 bounding box for each class.Other instances may be treated as negative samples.
  • Using all detections could address the current treatment of additional instances as negative samples.The authors identify this issue as future work.
  • Standard multiple instance learning features could improve pseudo-labeling localization accuracy.The proposed future direction extracts same-sized features corresponding to each detection.

7. Conclusion

The paper introduces cross-domain weakly supervised object detection and a two-step progressive adaptation framework, evaluated on original datasets spanning three visual domains.

  • The paper proposes cross-domain weakly supervised object detection as a novel task.
  • The evaluation uses original datasets containing images with instance-level annotations in three visual domains.
  • The results suggest the methods outperform comparable existing methods and provide a simple but solid baseline.

A. Statistics of Our Datasets

The datasets contain substantial object content, with Clipart1k having class and instance densities close to PASCAL VOC, but instance distributions are unbalanced and person is dominant.

  • Clipart1k contains 1.7 classes and 3.2 instances per image.These averages are reported as nearly matching PASCAL VOC.
  • Clipart1k has almost the same number of classes and instances per image as PASCAL VOC.
  • Instance distributions across classes in Clipart1k are unbalanced, as in PASCAL VOC.
  • The person class is dominant in Clipart1k, Watercolor2k, and Comic2k.

B. Visualization of Detections

The paper illustrates typical detection errors and presents detection results across the collected visual domains.

  • The methods often ignore small objects during detection.
  • They may merge highly overlapped objects belonging to the same class.
  • They may localize only the most discriminative part of an object.
  • They may fail to recognize highly deformed objects.
  • Detection results are shown in Figures 9, 10, and 11.

C. Implementation Details

The implementation resizes images before fine-tuning and restores their original dimensions during testing. Domain-transferred images were generated from all 16,551 VOC2007-trainval and VOC2012-trainval images.

  • Images were resized to 286 × 286, randomly cropped to 256 × 256 during fine-tuning, and restored to original size for testing.
  • Domain-transferred images were obtained using all 16,551 images from VOC2007-trainval and VOC2012-trainval.

C.2. Configurations for training FSDs

The training configurations specify detector-specific preprocessing, thresholds, learning rates, and fine-tuning durations. Qualitative figures show DT+PA outputs across Clipart1k, Watercolor2k, and Comic2k, alongside typical errors from Clipart1k and Comic2k.

  • YOLOv2 used 416 × 416 inputs, a 1.0×10−5 learning rate, IoU threshold 0.45, and confidence threshold 0.001.It was fine-tuned for five epochs in DT experiments and one hundred epochs in other experiments.
  • Typical DT+PA detection errors using SSD300 as the baseline FSD were illustrated with images from Clipart1k and Comic2k.
  • Faster R-CNN used a 1.0 × 10−5 learning rate, shorter-edge scaling to 600, and longer-edge scaling to at most 1,000.Its IoU threshold was 0.3 and confidence threshold was 0.05; fine-tuning lasted one epoch for DT and one hundred epochs for other experiments.
  • DT+PA example outputs using SSD300 as the baseline FSD were shown for Clipart1k, Watercolor2k, and Comic2k.Only detection windows with scores above 0.25 were displayed for visibility.
Loading 1803.11365v1…