DQ-DETR: DETR with Dynamic Query for Tiny Object Detection

1 Introduction

• Fixed query counts can miss objects in dense images, while large counts add false positives and quadratic decoder self-attention cost in sparse images.AI-TOD-V2 images range from fewer than 10 to more than 1,500 objects.Source: §1 · p. 2
• Learned query positions are image-independent and lack explicit physical meaning, making them poorly matched to aerial images with varying object distributions.Some images have dense objects concentrated in specific areas; others have only a few scattered objects.Source: §1 · pp. 2–3
• DQ-DETR uses a categorical counting module, counting-guided feature enhancement, and dynamic query selection to adjust query number and improve query positions.The density map reinforces encoder features, while the predicted count guides the number of decoder queries.Source: §1 · p. 3
• 16.6% in AP and 20.5% in APvt: DQ-DETR surpasses the state-of-the-art method on AI-TOD-V2.The introduction reports these gains for the two metrics, respectively.Source: §1 · p. 3

2.1 Tiny Object Detection

• Tiny-object methods use augmentation, specialized losses, and Gaussian label assignment, but some rely on predefined thresholds that may be unstable across datasets.DotD uses absolute and relative object size in its loss; [6] copy-pastes small objects.Source: §2.1 · p. 3

2.2 DETR-like Methods

• DETR-like methods address slow convergence through encoder-only designs, dynamic attention, deformable attention, denoising, and revised query formulations.Deformable-DETR attends to selected sampling points, while DAB-DETR represents query positions as 4-D anchor boxes.Source: §2.2 · p. 4
• Despite query-design improvements, existing DETR-like methods retain fixed query counts and are constrained on tiny objects and datasets with widely varying object counts.DQ-DETR is presented as the first DETR-like model specifically designed for tiny-object detection with dynamic query counts.Source: §2.2 · p. 4

3.1 Overview

• The end-to-end architecture adds categorical counting, counting-guided feature enhancement, and dynamic query selection to a CNN backbone and deformable transformer encoder-decoder.The counting module sets the decoder query count; feature enhancement strengthens spatial information, and selection refines query location and size.Source: §3.1 · pp. 4–5, Figure 1 · p. 5

3.2 Unflattening of Encoder’s Feature Map

• Encoder features are flattened and concatenated for transformer input, then reshaped into 2-D multi-scale EMSV features for dilated convolutions.The paper notes that higher-resolution features contain more spatial detail useful for counting and tiny-object detection.Source: §3.2 · p. 5

3.3 Categorical Counting Module

• The counting module extracts a density map from the largest EMSV feature and classifies image counts into four ranges: N ≤10, 10 < N ≤100, 100 < N ≤500, and N > 500.The range boundaries are selected based on the AI-TOD-V2 instance-count mean and standard deviation.Source: §3.3 · p. 6, Density Extractor · p. 6, Counting Number Classification · p. 6

3.4 Counting-Guided Feature Enhancement Module (CGFE)

• CGFE uses density-map spatial attention to emphasize foreground regions and multiply that attention element-wise with encoder features.The spatial attention map is generated from multi-scale counting features using average and max pooling.Source: §3.4 · p. 6, Spatial cross-attention map · pp. 6–7

Channel attention map.

• Channel attention pools spatially intensified features with average and max pooling, combines the results through a shared MLP, and weights the features.The resulting channel-attention map is multiplied with the spatially intensified features.Source: Channel attention map · p. 7

Number of queries.

• The four predicted count classes determine decoder query counts of K = 300, 500, 900, or 1500.For images classified as N ≤10, the decoder uses K = 300 queries.Source: Number of queries · p. 7

Enhancement of queries.

• Dynamic selection scores flattened intensified features and uses the top-K as priors to generate query content and refine 4-D anchor-box positions.Selected features provide the position prior; an FFN predicts biases that refine the original anchor boxes.Source: Enhancement of queries · pp. 7–8

Hungarian Loss

• Training combines Hungarian matching losses for boxes and classification with auxiliary decoder-stage losses and categorical-counting cross-entropy.The Hungarian loss uses L1, GIoU, and focal losses.Source: Hungarian Loss · p. 8

4.1 Datasets

• AI-TOD-V2 has 28,036 images, a mean object size of 12.7 pixels, and 1–2667 objects per image; VisDrone averages 40.7 objects per image across sparse and crowded scenes.AI-TOD-V2 averages 24.64 objects per image, with a standard deviation of 63.94.Source: §4.1 · p. 9

4.2 Implementation Details

• The experiments use a six-layer encoder and decoder with a ResNet50 backbone, Adam, batch size 1, and two-stage training to stabilize counting before feature enhancement.The batch size is set to 1 because of memory constraints.Source: §4.2 · p. 9

4.3 Main Results

• DQ-DETR achieves 30.2 AP on AI-TOD-V2 and 37.0 AP on VisDrone, but records 50.2 AP versus 51.3 AP on COCO.The AI-TOD-V2 and VisDrone results are reported as best among compared methods; the COCO result is slightly lower than the comparison.Source: §4.3 · pp. 10–11, Table 2 · p. 10, Table 3 · p. 11

Main ablation experiment.

• +2.2 AP over baseline accompanies categorical counting plus dynamic query selection, while feature enhancement adds +4.3 AP, +2.6 APvt, and +5.2 APt over baseline.The ablation reports an additional improvement from counting-guided feature enhancement even when query numbers remain static.Source: Main ablation experiment · p. 12, Table 5 · p. 12

Ablation of DQ-DETR with different number of instances in images.

• For N ≤10 and 10 < N ≤100, DQ-DETR uses fewer queries and beats DINO-DETR by 16% and 16.4% AP; for dense images, its APvt gain reaches 42.1%.DINO-DETR uses 900 queries in all instance-count ranges, whereas DQ-DETR adjusts its query count dynamically.Source: Ablation of DQ-DETR with different… · p. 12, Table 6 · p. 13

Ablation of Categorical Counting Module.

• Counting classification reaches 94.6% overall but 56.6% for N > 500; across 1–2267 instances per image, the authors prefer classification to regression.For N > 500, misclassification can yield an inappropriate query count and harm detection performance.Source: Ablation of DQ-DETR with different… · p. 13, Ablation of Categorical Counting Module · pp. 13–14, Table 7 · p. 14

5 Conclusion

• DQ-DETR dynamically adjusts the number of detection queries to address instance-count imbalance across aerial images.The model comprises a categorical counting module, counting-guided feature enhancement, and dynamic query selection.Source: §5 · p. 14
• DQ-DETR improves query positional information to help the decoder localize tiny objects.Source: §5 · p. 14
• 30.2% AP on AI-TOD-V2 is state-of-the-art, outperforming previous CNN-based detectors and DETR-like methods with ResNet50 as backbone.The paper identifies DQ-DETR as the first DETR-like model focused on tiny-object detection.Source: §5 · pp. 14–15

A.1 FP/FN Under Different Density Situation

• Sparse images contain fewer than 100 objects, while dense images contain more than 900; LRP FP and LRP FN are the evaluation metrics.Lower LRP values indicate better performance.Source: §A.1 · p. 17
• DQ-DETR dynamically adjusts query counts, yielding fewer false positives in sparse images and fewer missed instances in dense images.DINO-DETR uses 900 queries regardless of image density; its query count can exceed sparse-image object counts and fall short of dense-image counts.Source: §A.1 · p. 17
• Table A1 reports LRP FP and LRP FN scores across density situations, with DINO-DETR as the baseline.Source: Table A1 · p. 17

A.2 Ablation of Categorical Counting Module

• The module bins image counts as N ≤10, 10 < N ≤100, 100 < N ≤500, and N > 500; these thresholds reflect AI-TOD-V2 count statistics.The long-tailed training distribution limits finer categorization.Source: §A.2 · pp. 17–18
• Only 46 training images have N > 500, where the four-level module's classification accuracy is 56.6%.AI-TOD-V2 has images with up to 2,267 instances, but the long-tailed training distribution prevents more detailed count categories.Source: §A.2 · pp. 17–18
• In the five-class scenario, AP drops 1.4 compared with the four-class scenario.The authors attribute the drop to poor counting classification affecting the query count used for detection.Source: §A.2 · p. 18
• With five classes, total classification accuracy is 93.8%, but accuracy is 37.1% for 500 < N ≤900 and 57.4% for N > 900.The authors report few training images in these two situations.Source: §A.2 · p. 18

A.3 Categorical Counting Module (CCM) for Different Datasets.

• The categorical counting module should be tailored to the properties of each dataset.Source: §A.3 · p. 18
• Dataset properties inform how many queries should be used for object detection.Source: §A.3 · p. 18
• The authors characterize dataset-specific module design and query-number selection as vital.Source: §A.3 · p. 18

• For AI-TOD-V2, the categorical counting module assigns images to four instance-count levels: N ≤10, 10 < N ≤100, 100 < N ≤500, and N > 500.The thresholds 10, 100, and 500 are tailored to AI-TOD-V2.Source: Appendix 19 · p. 19
• For other datasets, the authors recommend adjusting CCM hyperparameters using the mean and variance of objects per image.This is intended to reduce manual CCM design for different datasets.Source: Appendix 19 · p. 19
• The pseudo-code defines four count classes using the dataset mean and variance as boundaries around predicted counts.Its cutoffs are mean − variance, mean, and mean + variance.Source: Appendix 19 · p. 19
• In Fig. B1, green, red, and blue boxes denote true positives, false positives, and false negatives, respectively.Source: Figure B1 · p. 19

• In dense scenes, DQ-DETR detects most tiny objects by selecting an appropriate number of object queries.The passage describes this as demonstrating stronger capture of fine details.Source: B Visualization · p. 20
• In dense scenes, Deformable-DETR misses more objects because it has an insufficient number of object queries.Source: B Visualization · p. 20
• In sparse scenes, Deformable-DETR produces more false positives amid excessive queries and unrefined positional information.Source: B Visualization · p. 20