Source-linked AI summary
Towards Universal Object Detection by Domain Attention
Xudong Wang, Zhaowei Cai, Dashan Gao, Nuno Vasconcelos
TL;DR
Object detection has rarely been addressed as a universal task despite substantial progress in universal visual representations, while detectors usually specialize to individual domains. The paper introduces a shared universal detector with squeeze-and-excitation adapters and domain attention, and reports that it outperforms individual, multi-domain, and baseline universal detectors on an 11-dataset benchmark.
Problem
Object detectors are usually trained for single datasets, but practical systems need to detect across diverse domains without knowing the domain beforehand.
Method
The paper combines universal squeeze-and-excitation adapters with feature-based domain attention in a single network that shares parameters and computations across domains.
Results
The proposed detector outperforms individual, multi-domain, and baseline universal detectors on UODB, an 11-dataset benchmark.
Takeaways & Limitations
Universal object detection can use shared computation while retaining domain sensitivity through data-driven adaptation.
Takeaways & Limitations
A fully shared universal detector has little flexibility for statistical domain variation and usually underperforms multi-domain detectors; experiments also use mostly common hyperparameters across datasets.
Abstract
from arXiv · showhide
Despite increasing efforts on universal representations for visual recognition, few have addressed object detection. In this paper, we develop an effective and efficient universal object detection system that is capable of working on various image domains, from human faces and traffic signs to medical CT images. Unlike multi-domain models, this universal model does not require prior knowledge of the domain of interest. This is achieved by the introduction of a new family of adaptation layers, based on the principles of squeeze and excitation, and a new domain-attention mechanism. In the proposed universal detector, all parameters and computations are shared across domains, and a single network processes all domains all the time. Experiments, on a newly established universal object detection benchmark of 11 diverse datasets, show that the proposed detector outperforms a bank of individual detectors, a multi-domain detector, and a baseline universal detector, with a 1.3x parameter increase over a single-domain baseline detector. The code and benchmark will be released at http://www.svcl.ucsd.edu/projects/universal-detection/.
1. Introduction
Object detection remains largely domain-specific, while practical applications need detectors that operate across changing and unknown domains. The paper establishes an 11-dataset benchmark and proposes shared universal architectures with domain-sensitive adaptation.
- Motivation: Existing detectors usually specialize in one dataset because detection domains differ in categories, viewpoints, and image styles.High performance generally requires specialization on the target dataset.
- Motivation: Practical applications need domain-agnostic detection because the target domain may change frequently and may be unknown beforehand.Maintaining D specialized detectors also increases total model size linearly with the number of domains D.
- Motivation: Universal object detection is more challenging than universal image classification because modern detectors combine backbones, proposal generators, regressors, and classifiers.The paper identifies object detection as an underexplored universal-learning problem.
- Contributions: The proposed UODB benchmark combines 11 diverse object detection datasets to support evaluation of universal and multi-domain detectors.The authors present it as a new benchmark for this previously underexplored problem.
- Architectures: SE adapters add lightweight domain-specific adaptation layers, improving multi-domain detectors by compensating for domain shift while sharing other parameters.The adapters are based on the squeeze-and-excitation mechanism.
- Architectures: The domain-attentive universal detector shares parameters and computations across domains, using universal SE adapters and feature-based attention to specialize responses without prior domain knowledge.A single network remains active and performs inference once, while the learned number of domains need not equal the number of datasets.
2. Related Work
Prior object detectors achieve high performance on individual datasets but generally require fine-tuning to handle multiple domains. Related work includes multi-scale, position-sensitive, cascaded, single-stage, and earlier dataset-adapted detection approaches.
- Object Detection: Modern two-stage detectors include R-CNN variants, MS-CNN, FPN, R-FCN, and Cascade R-CNN, which improve detection quality, scale handling, or speed.These methods extend the two-stage detection framework in different ways.
- Object Detection: Single-stage detectors such as YOLO and SSD became popular for their speed, complementing the two-stage detection literature.The passage contrasts their speed-oriented appeal with the broader detection framework.
- Object Detection: None of these deep-learning detectors reached high detection performance across more than one dataset or domain without fine-tuning.This motivates methods designed explicitly for multi-domain or universal detection.
- Object Detection: An earlier universal DPM detector added dataset-specific biases, but its relevance is limited because DPM is not comparable to modern deep-learning detectors.The related-work passage distinguishes this pre-deep-learning approach from current detector architectures.
3. Multi-domain Object Detection
This section establishes UODB and compares detector designs for object detection across diverse domains. It motivates lightweight adaptation layers by showing domain-dependent activation statistics and introduces SE-based multi-domain adaptation.
- 3.1. Universal Object Detection Benchmark: UODB comprises 11 diverse object detection datasets spanning everyday objects, visual styles, kitchen scenes, and medical images.The benchmark includes Pascal VOC, WiderFace, KITTI, LISA, DOTA, COCO, Watercolor, Clipart, Comic, Kitchen, and DeepLesions.
- 3.1. Universal Object Detection Benchmark: The Faster R-CNN baseline uses an RPN for class-agnostic proposals followed by a region-of-interest network that outputs final detections.All detectors in this work use Faster R-CNN as their baseline architecture.
- 3.2. Single-domain Detector Bank: A detector bank uses an independent detector per dataset, replicating all detector parameters and forming the most expensive multi-domain baseline.The baseline is illustrated as Figure 2(a).
- 3.2. Single-domain Detector Bank: Activation means and variances differ across datasets and layers, with stronger differences in early layers and similar statistics in many intermediate layers.The observed similarity in intermediate layers suggests that at least some domains can share those layers.
- 3.3. Adaptive Multi-domain Detector: The adaptive multi-domain detector shares convolutional layers while retaining domain-specific output and RPN layers plus lightweight adaptation layers.The adaptation layers are intended to compensate for domain shift while minimizing parameters and computation.
- 3.3. Adaptive Multi-domain Detector: SE adapters provide lightweight feature-based channel modulation, and the SE adapter bank adds one selectable branch per domain.Each SE adapter uses global pooling followed by FC, ReLU, and a second FC layer; the adapter bank assumes the domain is known a priori.
4. Universal Object detection
Universal detection shares a detector across unknown image domains while adding domain sensitivity through adapter banks and attention-based feature adaptation.
- Universal detector: Universal detectors process unknown input-image domains without requiring prior domain knowledge.Unlike multi-domain detectors, they infer domain sensitivity automatically during inference.
- Universal detector: A fully shared detector uses one network across tasks, with only task-specific output layers and no domain-specific parameters.This maximizes parameter sharing and efficiency, but limits flexibility across statistically different domains.
- Domain-attentive detector: The proposed domain-attentive detector combines universal squeeze-and-excitation adapters with feature-based domain attention.The adapter bank concatenates individual adapter outputs into a universal representation space, while attention combines their projections.
- Domain attention: Domain attention globally pools feature maps, applies a softmax weighting layer, and uses the resulting weights to combine adapter responses.The combined responses then channel-wise rescale the activations being adapted.
- Domain attention: The domain-attentive universal detector soft-routes adapter projections without prior domain knowledge and enables information sharing across domains.In experiments, it outperforms the other detector architectures considered.
5. Experiments
Experiments evaluate universal and multi-domain detectors on five-dataset and 11-dataset benchmarks, showing that domain attention improves performance while retaining efficient inference and parameter sharing. Results also examine adapter counts, dataset relationships, and official per-dataset comparisons.
- Baseline: The detector bank is a strong multi-domain baseline, achieving 59.4 average mAP across the evaluated datasets.
- Multi-domain detection: The adaptive multi-domain detector adds approximately 11M parameters and outperforms the single-domain detector bank by 0.7 points on five datasets.
- Multi-domain detection: The universal detector adds only 0.5M parameters but performs 1.7 points below the adaptive multi-domain detector.
- Multi-domain detection: The domain-attentive universal detector outperforms the single-domain bank by 1.6 points, runs about 4× faster than multi-domain detectors, and is only 1.33× slower than the universal detector.
- Multi-domain detection: Fixed domain assignments reduce domain-attentive universal-detector performance by 0.5 point, supporting the value of data-driven attention.
- Adapter analysis: Performance generally improves with more SE adapters until seven, while parameter count grows linearly; around five adapters offers the best reported trade-off.
- Full benchmark: On the full benchmark, the universal detector performs comparably to the single-domain bank with 10 times fewer parameters, while domain-attentive training improves baseline performance.
- Full benchmark: Joint training produces dataset-dependent gains, including a large DeepLesion improvement, mild 1–5-point gains on several datasets, and no gains on COCO, LISA, or DOTA.
6. Conclusion
The paper addresses universal and multi-domain object detection with a single network that operates across tasks without prior domain knowledge. Its domain-attentive detector uses data-driven adaptation and outperforms multiple universal, multi-domain, and single-task detectors on a new benchmark.
- The proposed universal detector uses one network active across all tasks and requires no prior domain knowledge.
- Its domain sensitivity comes from a novel data-driven domain adaptation module.
- The detector outperforms multiple universal and multi-domain detectors, as well as individual detectors optimized for a single task, on the newly established benchmark.