PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-adversarial Faster-RCNN for Unrestricted Object Detection

Zhenwei He, Lei Zhang

arXiv:1907.10343v2cs.CV

TL;DR

Object detectors can degrade when training and testing domains differ, making unrestricted detection difficult. MAF addresses this with hierarchical and proposal feature alignment, SRM, and WGRL; on Cityscapes-to-Foggy Cityscapes, it outperforms DAF by 6.4%.

  • Problem

    Object detection in the wild remains difficult because domain shifts create discrepancies between training and testing data, causing performance degradation and missing objects.

  • Method

    MAF uses multi-adversarial alignment of hierarchical convolutional and aggregated proposal features, with SRM improving training efficiency and WGRL emphasizing hard confused samples.

  • Results

    MAF achieves the best results among compared methods, outperforming DAF by 6.4% when both domain and proposal feature alignment modules are used.

  • Takeaways & Limitations

    The results support combining hierarchical domain-feature alignment with proposal-feature alignment for unrestricted object detection.

Abstract

from arXiv · show

Conventional object detection methods essentially suppose that the training and testing data are collected from a restricted target domain with expensive labeling cost. For alleviating the problem of domain dependency and cumbersome labeling, this paper proposes to detect objects in an unrestricted environment by leveraging domain knowledge trained from an auxiliary source domain with sufficient labels. Specifically, we propose a multi-adversarial Faster-RCNN (MAF) framework for unrestricted object detection, which inherently addresses domain disparity minimization for domain adaptation in feature representation. The paper merits are in three-fold: 1) With the idea that object detectors often becomes domain incompatible when image distribution resulted domain disparity appears, we propose a hierarchical domain feature alignment module, in which multiple adversarial domain classifier submodules for layer-wise domain feature confusion are designed; 2) An information invariant scale reduction module (SRM) for hierarchical feature map resizing is proposed for promoting the training efficiency of adversarial domain adaptation; 3) In order to improve the domain adaptability, the aggregated proposal features with detection results are feed into a proposed weighted gradient reversal layer (WGRL) for characterizing hard confused domain samples. We evaluate our MAF on unrestricted tasks, including Cityscapes, KITTI, Sim10k, etc. and the experiments show the state-of-the-art performance over the existing detectors.

1. Introduction

The paper targets unrestricted object detection under domain shifts, using labeled source data and unlabeled target data to learn domain-adaptive features. Its MAF framework aligns hierarchical domain and proposal features, improves adversarial training efficiency, and reports superior performance on several benchmarks.

  • Motivation: Domain shifts between restricted training and testing datasets can cause substantial detection degradation, including many missing objects in Foggy Cityscapes.The paper motivates unrestricted detection by showing that conventional detectors are difficult to adapt across domains.
  • Approach: MAF uses labeled source data and completely unlabeled target data to learn domain-invariant features for unrestricted object detection.The source and target domains are semantically related but distributionally different.
  • Approach: MAF aligns hierarchical convolutional features and aggregated proposal features through multiple adversarial domain-classifier submodules.The hierarchical module operates across convolutional blocks, while proposal alignment incorporates detection-related information.
  • Approach: SRM reduces feature-map scale without domain-feature information loss, improving the training efficiency of adversarial domain classifiers.The module reduces feature-map size by increasing channel capacity and merging adjacent features.
  • Approach: WGRL assigns larger gradient weights to hard-to-confuse proposal samples and smaller weights to easily confused samples during domain alignment.Proposal features are aggregated with classification scores and regression coordinates before domain classification.
  • Results: Experiments on Cityscapes, KITTI, SIM10K, and other unrestricted tasks report superior performance over state-of-the-art detectors.The supplied introduction describes exhaustive experiments but does not provide complete benchmark values.

2. Related Work

The related work situates MAF within CNN-based two-stage object detection and domain adaptation, especially methods that reduce distribution discrepancy through feature alignment and adversarial learning.

  • Object Detection: Faster-RCNN integrates Fast-RCNN with a Region Proposal Network, improving detection speed and accuracy within a two-stage architecture.MAF adopts Faster-RCNN as its backbone for cross-domain unrestricted detection.
  • Domain Adaptation: Domain adaptation methods bridge domains by reducing distribution discrepancy, including approaches based on maximum mean discrepancy between domain-specific CNN branches.The cited related work also includes adversarial feature-alignment approaches.
  • MAF Positioning: MAF extends this context by applying feature alignment to both hierarchical domain features and proposal features within a Faster-RCNN framework.Its architecture includes multiple adversarial classifiers at convolutional blocks and a proposal-level alignment module.
  • Domain Adaptation: Adversarial domain adaptation uses a two-player game between feature representations and domain classifiers to promote feature alignment.Prior work also explored multiple domain classifiers and two-step target-encoder training.

3. The Proposed MAF Detector

MAF combines hierarchical and proposal-level adversarial alignment within Faster-RCNN to adapt detection across domains. SRM improves alignment efficiency, while WGRL reweights proposal gradients according to domain confusion.

  • Feature Alignment: MAF aligns hierarchical convolutional features and aggregated proposal features using multiple adversarial domain-classifier submodules.Hierarchical features capture image and object-position information, while proposal features capture proposal semantics.
  • Hierarchical Domain Feature Alignment: The hierarchical alignment module deploys adversarial classifiers across convolutional blocks to reduce source–target distribution differences.The backbone is trained to confuse domain classifiers through gradient reversal, producing domain-invariant representations.
  • Scale Reduction Module: SRM reduces feature-map channels with a 1 × 1 convolution and rearranges spatial features into lower-scale maps without information loss.Its parameter reduction improves the training efficiency of hierarchical adversarial alignment.
  • Aggregated Proposal Feature Alignment: The proposal alignment module feeds proposal features, classification scores, and bounding-box regression results to a domain discriminator.Concatenating detection outputs with proposal features supports semantic alignment while preserving classification and regression information.
  • Weighted Gradient Reversal Layer: WGRL reweights proposal gradients so high-confidence, distinguishable domain samples receive stronger penalties while less-confident samples are down-weighted.The weighting is based on the domain classifier’s predicted scores and is applied through gradient reversal.
  • Overall Training Objective: MAF jointly optimizes Faster-RCNN detection loss with hierarchical and proposal-level adversarial alignment losses in an end-to-end framework.The detection loss uses source-domain labeled data, while the alignment losses support adaptation between domains.

4. Experiments

The experiments evaluate MAF for unrestricted object detection on several datasets and compare it with domain-adaptive and standard Faster-RCNN baselines.

  • Experimental Setup: MAF is evaluated on Cityscapes, Foggy Cityscapes, KITTI, and SIM10K against domain adaptive Faster-RCNN and standard Faster-RCNN.The evaluation follows the unrestricted setting used in prior work.
  • Experimental Setup: The comparisons target object detection performance when labeled source data are available but the target domain is completely unlabeled.Target-domain performance is evaluated using mean average precision.
  • Evaluation Metric: Mean average precision on the target domain is the reported evaluation measure for unrestricted object detection.The experiments compare MAF with both a domain-adaptive Faster-RCNN baseline and the standard detector.

4.1. Implementation Details

Implementation uses a labeled source domain, an unlabeled target domain, and a fixed tradeoff between Faster-RCNN detection and adversarial adaptation losses.

  • Data Setting: The source domain provides annotated bounding boxes and categories, whereas the target domain is completely unlabeled.This setup evaluates adaptation without target-domain annotation.
  • Evaluation: Target-domain performance is compared using mean average precision under the unrestricted object-detection setting.The evaluation follows the experimental setting of prior domain-adaptive Faster-RCNN work.
  • Training Configuration: The loss tradeoff parameter α is set to 0.1 during training, with detection hyperparameters following Faster-RCNN.The detector uses ImageNet-pretrained VGG16 components as specified in the implementation setup.

4.2. Datasets

Four datasets provide varied unrestricted detection settings, including real street scenes, simulated fog, autonomous-driving imagery, and synthetic vehicle images.

  • Dataset Overview: Four datasets—Cityscapes, Foggy Cityscapes, KITTI, and SIM10K—are used to evaluate MAF.The datasets cover both real and simulated domain-shift conditions.
  • Cityscapes: Cityscapes contains 5000 densely labeled outdoor street-scene images collected across 27 cities and multiple seasons.Its annotations were originally created for semantic segmentation and converted into detection bounding boxes.
  • Foggy Cityscapes: Foggy Cityscapes derives foggy images from Cityscapes’ clear-weather imagery and reuses inherited pixel labels to generate bounding boxes.The dataset simulates fog-weather conditions for evaluation.
  • KITTI: KITTI contains 14,999 images and 80,256 bounding boxes captured from an autonomous-driving platform.Experiments use 7,481 training images for adaptation and evaluation.
  • SIM10K: SIM10K is a GTA V-generated simulated dataset with 10,000 images and 58,071 car bounding boxes.All SIM10K images serve as the source domain for training.

4.3. Experimental Results

MAF is evaluated across weather, synthetic-to-real, and cross-scene domain shifts, consistently outperforming the compared detectors. Results indicate that combining hierarchical domain-feature and proposal-feature alignment improves unrestricted detection.

  • Experimental Setup: The experiments cover weather, synthetic-to-real, and cross-scene domain shifts using Cityscapes, Foggy Cityscapes, SIM10k, and KITTI.
  • Cityscapes to Foggy Cityscapes: MAF achieves the best results among compared methods for Cityscapes-to-Foggy-Cityscapes detection.Using both alignment modules, MAF outperforms DAF by 6.4% in mAP.
  • Cityscapes to Foggy Cityscapes: 28.5% mAP is achieved by MAF with only proposal feature alignment, exceeding DAF in the Foggy Cityscapes experiment.
  • Synthetic Data to Real Data: MAF achieves the best performance for synthetic SIM10k-to-Cityscapes detection and outperforms DAF by 2.1% in car AP.The experiment evaluates only the car category on the Cityscapes validation set.
  • One Scene to Another: MAF achieves the best performance in both Cityscapes-to-KITTI and KITTI-to-Cityscapes settings, with the best result 8.1% higher than DAF.The comparison reports car AP under cross-scene domain shifts.

4.4. Analysis of Proposal Feature Alignment

The proposal feature alignment analysis finds that aggregating proposal features with detection results and using WGRL improve domain adaptation. Their combination achieves the best mAP.

  • Proposal Feature Aggregation: Aggregating proposal features with classification scores and regression results provides more semantic information for domain confusion.
  • Weighted Gradient Reversal: WGRL weights easily-confused and hard-confused samples differently, emphasizing samples that are difficult for the domain classifier to confuse.
  • Combined Design: The combination of aggregated proposal features and WGRL achieves the best mAP in the proposal feature alignment analysis.

4.5. Influence of IOU Threshold

The study varies the testing IoU threshold and compares MAF with DAF, Faster-RCNN, and single-module variants. MAF remains strongest across different IoU values, while mAP declines as the threshold increases.

  • Threshold Effect: mAP drops as the IoU threshold increases for Faster-RCNN, DAF, MAF, and single-feature-alignment variants.Larger thresholds exclude more predicted boxes, causing recall and accuracy to fall.
  • Model Comparison: MAF achieves the best results under different IoU values, followed by MAF with only hierarchical feature alignment.The comparison uses Cityscapes as source and Foggy Cityscapes as target.

5. Conclusion

MAF addresses unrestricted object detection by aligning hierarchical image and proposal features across domains while improving adversarial training efficiency. The detector is trained end-to-end and reports state-of-the-art results across several datasets.

  • MAF aligns hierarchical convolutional feature maps and aggregated proposal features for domain-adaptive object detection.The two alignment modules target image-distribution and proposal-semantic alignment, respectively.
  • Multiple adversarial domain classifiers operate across convolutional blocks to confuse domain-specific feature-map representations.The classifiers are trained with gradient reversal while the backbone learns domain-invariant features.
  • The scale reduction module (SRM) reduces feature-map scale to improve adversarial domain-classifier training efficiency.
  • Aggregated proposal features pass through a weighted gradient reversal layer (WGRL) that automatically focuses on hard confused samples.
  • The MAF detector is optimized end-to-end using domain-alignment and Faster-RCNN detection losses, achieving state-of-the-art results on several datasets.
Loading 1907.10343v2…