Source-linked AI summary
Automatic adaptation of object detectors to new domains using self-training
Aruni RoyChowdhury, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, Erik Learned-Miller
TL;DR
The paper addresses adapting an existing object detector to a new domain without costly target-domain annotations. It combines high-confidence detector predictions with tracker-derived hard examples and soft-label distillation for retraining. Across face and pedestrian adaptation tasks, soft-label self-training improves over hard labels and is comparable to or better than adversarial methods with minimal hyper-parameter dependence.
Problem
Existing detectors do not reliably generalize to every domain, while supervised retraining for a new scenario requires substantial labeling effort.
Method
The method combines high-confidence detector pseudo-labels and tracker-derived hard examples, then retrains with a modified distillation loss using soft target labels.
Results
Soft-label self-training improves over hard-label training across the evaluated face and pedestrian adaptation tasks and is comparable to or better than adversarial methods.
Takeaways & Limitations
The approach provides a readily applicable unsupervised adaptation method for object detectors, with no extra adversarial parameters and minimal dependence on hyper-parameters.
Abstract
from arXiv · showhide
This work addresses the unsupervised adaptation of an existing object detector to a new target domain. We assume that a large number of unlabeled videos from this domain are readily available. We automatically obtain labels on the target data by using high-confidence detections from the existing detector, augmented with hard (misclassified) examples acquired by exploiting temporal cues using a tracker. These automatically-obtained labels are then used for re-training the original model. A modified knowledge distillation loss is proposed, and we investigate several ways of assigning soft-labels to the training examples from the target domain. Our approach is empirically evaluated on challenging face and pedestrian detection tasks: a face detector trained on WIDER-Face, which consists of high-quality images crawled from the web, is adapted to a large-scale surveillance data set; a pedestrian detector trained on clear, daytime images from the BDD-100K driving data set is adapted to all other scenarios such as rainy, foggy, night-time. Our results demonstrate the usefulness of incorporating hard examples obtained from tracking, the advantage of using soft-labels via distillation loss versus hard-labels, and show promising performance as a simple method for unsupervised domain adaptation of object detectors, with minimal dependence on hyper-parameters.
1. Introduction
The paper studies adapting existing object detectors to new domains using abundant unlabeled video, avoiding costly bounding-box annotation. It combines temporal tracking with self-training and soft-label distillation, and evaluates the approach on face and pedestrian detection.
- Motivation: A single detector is unlikely to generalize across domains, while fully supervised retraining for each scenario requires substantial labeling effort.The motivation concerns domain shifts such as web images versus surveillance footage or clear daytime versus adverse driving conditions.
- Problem: The proposed problem is whether an off-the-shelf detector can improve itself by observing unlabeled target-domain video.The intended benefit is reducing costly bounding-box annotation for large-scale domain adaptation.
- Challenge: Naive self-training treats model predictions as pseudo-labels, but noisy labels are especially problematic for object detectors because wrong instances may become hard training examples.The paper contrasts this challenge with adversarial and generative approaches to domain adaptation.
- Approach: The method combines detection and tracking to recover missed cases, then uses distillation-based soft labels to represent varying target-domain example difficulty.It evaluates face adaptation from WIDER-Face to surveillance video and pedestrian adaptation from clear daytime BDD-100K to other conditions.
- Findings: Across the evaluated tasks, soft-label self-training improves over hard labels and performs comparably to or better than adversarial methods without extra parameters.The authors report minimal dependence on hyper-parameters.
2. Related Work
The related work positions the paper within self-training and video-based detector improvement. Its distinction is adapting to a new target domain while combining tracked hard examples with soft-label distillation.
- Self-training: Self-training begins with a fully labeled source-domain model and uses its predictions as labels for a novel weakly labeled or unlabeled dataset.The approach is presented as a label-efficient learning strategy.
- Self-training: Prior object-detection work used pseudo-labels from an unlabeled dataset for incremental retraining, while data distillation augmented same-domain training with predictions from test-time augmentation.These methods provide precedents for pseudo-labeling but differ in target-domain setting or label-generation procedure.
- Video cues: Tracking has been used to mine hard false-negative examples, but this paper uses temporal cues for adaptation to a new target domain rather than improving performance back on the source domain.The distinction is the adaptation objective, not merely the use of tracking.
- Video cues: Other weakly supervised video approaches use tracking and curriculum-based sample selection, whereas this work combines hard and easy target examples and applies knowledge distillation to address label noise.The paper identifies both differences as central to its approach.
3. Proposed Approach
The approach creates target-domain pseudo-labels from detector confidence and tracker refinements, retrains Faster R-CNN, and replaces uniformly hard target labels with confidence-aware soft labels. It also offers score remapping and constrained hard-example schemes to reduce hyper-parameter dependence.
- Pseudo-label generation: The baseline detector supplies target pseudo-labels when normalized confidence d_i exceeds threshold θ, with θ set to 0.5 for faces and 0.8 for persons.A tracker then refines the labels using temporal information.
- Hard-label training: In the naive retraining scheme, positive target regions from either detection or tracking receive label 1, while negative regions receive label 0 under binary cross-entropy.The training sample X_i denotes a region rather than an entire image.
- Pseudo-label generation: Tracking adds hard positives that the detector missed, so the training set combines high-confidence detections with tracker-only examples.This combination is intended to retain detector positives while recovering missed objects.
- Soft labels: Detector-originated soft scores use d_i, whereas tracker-only samples receive θ because their detector confidence is below the threshold.The example assigns 0.78 and 0.83 to detector samples and 0.5 to a tracker-only sample with d_3=0.32.
- Soft labels: The interpolated target label combines the hard label y_i and soft score s_i through λ, while source data retains the ordinary loss and target data uses the soft-label loss.Larger λ trusts the baseline score more, reducing target training influence while limiting damage from incorrect pseudo-labels.
- Hyper-parameter reduction: Histogram specification remaps target detector scores to match the source score distribution, providing a parameter-free soft-label scheme; constrained hard examples instead emphasize tracker-only cases while preserving easy-example posteriors.The remapping uses G^-1(F(x)) for detector scores and θ for tracker-only samples.
4. Experiments
Experiments evaluate unsupervised adaptation for face and pedestrian detection across substantial source–target domain shifts. Combining detector and tracker pseudo-labels with soft-label training improves performance across both tasks, while requiring limited target-domain tuning.
- Datasets: Experiments cover face adaptation from WIDER to CS6 surveillance videos and pedestrian adaptation from clear daytime BDD images to varied BDD conditions.The pedestrian target includes snowy, rainy, cloudy, night-time, and dusk scenarios.
- Baselines and Ablations: The baselines compare source-only detection, detector-derived pseudo-labels, tracker-derived pseudo-labels, their combination, soft-label distillation, and domain-adversarial Faster R-CNN.Pseudo-labels are generated from high-confidence detections and tracking, then used in joint source–target retraining.
- Face detection results: 17.31 AP improves on 15.66 AP for WIDER→CS6 when detector and tracker pseudo-labels are combined, while tracker-only labels reach 11.73 AP.Using only high-confidence detections gives 17.29 AP; soft target labels provide a further consistent gain, with APλ=0.3 reaching 19.89.
- Pedestrian detection results: 28.59 AP improves on the 15.21 AP baseline for BDD(clear,daytime)→BDD(rest) using soft labels, while combined detection-and-tracking pseudo-labels reach 27.11 AP.Detector-only and tracker-only pseudo-labels reach 26.16 and 26.28 AP, respectively; score-remap and HP-cons achieve 28.02 and 28.43 AP.
- Pedestrian detection results: Soft-label models outperform the domain-adversarial comparison by approximately 5 AP on BDD, and HP-cons consistently outperforms DA across evaluated sub-domains.Night-time images degrade performance for all models, while threshold selection can be automated by mapping a source precision threshold to the target domain.
5. Conclusion
Self-training with soft labels performs at least as well as recent domain-adversarial adaptation on two challenging tasks while avoiding adversarial methods’ extra layers and hyper-parameters.
- Self-training with soft-labels is at par with or better than a recent domain-adversarial approach on two challenging tasks.
- The method significantly boosts pre-trained models on the target domain and consistently improves over hard labels for pseudo-labeled target samples.
- Its minimal dependence on hyper-parameters supports applying it to large-scale unsupervised domain adaptation of object detectors.