Source-linked AI summary
A Robust Learning Approach to Domain Adaptive Object Detection
Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, William G. Macready
TL;DR
Object-detection domain adaptation must handle deployment domains that differ from labeled source data. The paper formulates adaptation as robust learning with noisy target-domain bounding boxes and reports state-of-the-art performance across several cross-domain tasks.
Problem
Domain shift limits object-detection deployment when target-domain labels are expensive or unavailable.
Method
The framework retrains a source-trained detector on source data and noisy target-domain detections, robustly refining class labels and bounding-box geometry.
Results
The robust framework achieves state-of-the-art performance on several cross-domain detection tasks.
Takeaways & Limitations
The approach enables unsupervised target-domain training without collecting labeled target data and can integrate additional information through detection re-scoring.
Takeaways & Limitations
The probabilistic localization assumption is exact for an L2 localization error, whereas practical combinations of L2 and L1 norms lack a simple probabilistic output.
Abstract
from arXiv · showhide
Domain shift is unavoidable in real-world applications of object detection. For example, in self-driving cars, the target domain consists of unconstrained road environments which cannot all possibly be observed in training data. Similarly, in surveillance applications sufficiently representative training data may be lacking due to privacy regulations. In this paper, we address the domain adaptation problem from the perspective of robust learning and show that the problem may be formulated as training with noisy labels. We propose a robust object detection framework that is resilient to noise in bounding box class labels, locations and size annotations. To adapt to the domain shift, the model is trained on the target domain using a set of noisy object bounding boxes that are obtained by a detection model trained only in the source domain. We evaluate the accuracy of our approach in various source/target domain pairs and demonstrate that the model significantly improves the state-of-the-art on multiple domain adaptation scenarios on the SIM10K, Cityscapes and KITTI datasets.
1. Introduction
Object-detection domain adaptation is difficult because deployment conditions differ from training data. The paper formulates adaptation as robust learning, retraining on noisy target-domain detections while refining class labels and bounding boxes.
- Domain shifts arise from changes in viewpoint, background, appearance, scene type, illumination, and deployment environment.
- A source-trained detector supplies target-domain detections for retraining, despite inaccuracies in those pseudo-labels.
- The approach trains on the target domain without labels and may implicitly align domains through instances from both source and target.
- The robust objective can revise target-domain class labels and bounding-box geometry according to the model’s current beliefs.
- An auxiliary classifier can use motion, optical flow, alternative architectures, or ensembles during retraining, while test-time detector complexity remains unchanged.
- The paper reframes object-detection domain adaptation as robust learning and proposes a framework that handles noise in labels, locations, and box sizes.
2. Previous Work
Earlier domain-adaptation research largely focused on image classification, while object-detection adaptation remained less developed. This paper differs by treating target-domain pseudo-ground-truth as noisy labels and designing robust training for detection.
- Faster R-CNN is adopted as the base detector, enabling direct comparison with prior state-of-the-art work.
- Object-detection domain adaptation has received less attention than image-classification domain adaptation.
- Prior adaptation methods include adaptive SVMs, feature subspace alignment, and adversarial learning of domain-invariant features.
- The paper takes a fundamentally different approach by reformulating detection adaptation as noisy labeling.
- Robust-learning research previously focused on image classification with few and disjoint classes, whereas this work targets object detection.
3. Method
The method adapts object detection without target-domain labels by generating, refining, and robustly using noisy target bounding boxes alongside labeled source data. Its probabilistic refinement corrects both class-label and localization errors while balancing detector and auxiliary-classifier predictions.
- Three-phase framework: Target images lack bounding-box annotations, so Faster R-CNN proposals augment the target dataset before robust retraining.The framework assumes labeled images in the source domain and unlabeled images in the target domain.
- Detector background: Faster R-CNN combines a region proposal network with an ROI classifier to predict object labels and refine bounding-box locations and sizes.Its losses include classification and localization terms for the proposal and ROI components.
- Three-phase framework: The three-phase pipeline trains a source detector, mines noisy target boxes, refines them with an image classifier, and retrains the detector on source and refined target annotations.Retraining accounts for possible mislabeling in the machine-generated target annotations.
- Robust refinement: The robust objective models noise in both object classes and box geometry, allowing refined annotations to alter labels and locations before phase-three training.This correction mechanism is designed for inaccurate target-domain detections.
- Robust refinement: Classification refinement finds a distribution close to both Faster R-CNN and the auxiliary image classifier, with α controlling their relative influence.The objective minimizes KL divergence to both predictive distributions; large α favors the image classifier, while smaller α favors Faster R-CNN.
- Robust refinement: The closed-form classification solution is a weighted geometric mean, equivalently a softmax distribution whose logits are weighted between the detector and image classifier.Training decreases α so the refinement shifts from greater reliance on the image classifier toward the evolving detector prediction.
4. Experiments
The experiments evaluate robust domain adaptation across synthetic-to-real, weather, and real-dataset shifts using multiple baselines and benchmark datasets. Results show consistent gains, including stronger improvements than prior state of the art and qualitative corrections of detection errors.
- Baselines: Baselines range from a source-only Faster R-CNN detector to pseudo-labeling and adversarial feature learning.The source-only baseline is blind to the target domain, while pseudo-labeling selects target proposals using a fixed threshold.
- Experimental setup: The evaluation covers multi- and single-label detection across SIM 10K, Cityscapes, Foggy Cityscapes, and KITTI.Experiments use different source/target combinations, including synthetic-to-real, weather, and cross-dataset adaptation.
- Weather adaptation: The weather experiment tests adaptation from Cityscapes to Foggy Cityscapes under the highest fog intensity.Average precision is recorded on the Cityscapes validation set, with optional classification-error, box-refinement, and false-negative-correction components.
- Synthetic-to-real adaptation: +3.51% improvement over Pseudo-labeling is achieved by the best method on SIM 10K →Cityscapes, versus +1.05% for Feature Learning.The paper describes this as more than triple the improvement of the incumbent state-of-the-art over the same baseline.
- Weather adaptation: +4.53% mAP improvement is obtained by the method in multi-label adaptation, compared with +2.78% for the state of the art.The experiment reports average precision for eight object categories and their mean average precision.
- New-dataset adaptation: The method significantly outperforms the state of the art on car average precision for both Cityscapes ⇄KITTI directions.Qualitative KITTI results show fewer false positives, corrected box size and location, and detections of cars missed by Faster R-CNN.
5. Conclusion
The paper formulates domain adaptation as robust learning to address deployment under domain shift without labeled target data. Its framework handles noisy classes and bounding boxes while achieving state-of-the-art performance through robust target-domain training.
- Conclusion: Domain shift can limit real-world deployment when collecting labeled target data is expensive or infeasible.The conclusion frames this as the motivating deployment problem.
- Conclusion: The proposed unsupervised framework uses robust learning to handle noise in object classes and bounding boxes.It trains in the target domain using a model trained only in the source domain.
- Conclusion: State-of-the-art performance is achieved without collecting target-domain data, while detection re-scoring integrates other information sources.The conclusion presents these as properties of the proposed approach.