Source-linked AI summary
Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection
Yangchen Zeng, Zhenyu Yu, Dongming Jiang, Wenbo Zhang, Yifan Hong, Zhanhua Hu, Jiao Luo, Kangning Cui
TL;DR
Small-object detectors struggle with background-induced query noise and costly deep decoders. HELP selectively embeds positional information, retrieves cleaner queries, and reduces decoder depth and parameters while retaining accuracy gains.
Problem
Small-object detection suffers from cluttered backgrounds, sparse informative features, and background-polluted queries that require costly decoder refinement.
Method
HELP uses heatmap-guided positional embeddings, noise-suppressed encoder fusion, high-quality query retrieval, and Linear-Snake Convolution for sparse target features.
Results
59.4% parameter reduction (66.3M vs. 163M) accompanies decoder reduction from eight to three layers and consistent accuracy gains across benchmarks.
Takeaways & Limitations
Selective positional embedding produces cleaner decoder inputs and enables a substantially lighter decoder without sacrificing performance.
Takeaways & Limitations
Gradient-based heatmap generation is restricted to training, so deployment relies on the learned embedding behavior rather than runtime gradient computation.
Abstract
from arXiv · showhide
Transformer-based detectors have advanced small-object detection, but they often remain inefficient and vulnerable to background-induced query noise, which motivates deep decoders to refine low-quality queries. We present HELP (Heatmap-guided Embedding Learning Paradigm), a noise-aware positional-semantic fusion framework that studies where to embed positional information by selectively preserving positional encodings in foreground-salient regions while suppressing background clutter. Within HELP, we introduce Heatmap-guided Positional Embedding (HPE) as the core embedding mechanism and visualize it with a heatbar for interpretable diagnosis and fine-tuning. HPE is integrated into both the encoder and decoder: it guides noise-suppressed feature encoding by injecting heatmap-aware positional encoding, and it enables high-quality query retrieval by filtering background-dominant embeddings via a gradient-based mask filter before decoding. To address feature sparsity in complex small targets, we integrate Linear-Snake Convolution to enrich retrieval-relevant representations. The gradient-based heatmap supervision is used during training only, incurring no additional gradient computation at inference. As a result, our design reduces decoder layers from eight to three and achieves a 59.4% parameter reduction (66.3M vs. 163M) while maintaining consistent accuracy gains under a reduced compute budget across benchmarks. Code Repository: https://github.com/yidimopozhibai/Noise-Suppressed-Query-Retrieval
1 Introduction
Small-object detection in cluttered aerial scenes suffers from sparse, fine-grained features and background-induced query noise. HELP addresses these issues by selectively embedding positional information, suppressing noisy queries, and reducing decoder depth while preserving accuracy.
- Cluttered aerial scenes combine tiny targets, scale variation, dense layouts, and extensive background, degrading localization and recognition.
- Background responses pollute queries and force decoders to repeatedly refine low-quality embeddings, increasing computational cost.
- HELP selectively preserves positional encodings in foreground-salient regions while suppressing them in background-dominant regions.
- HPE provides heatbar visualization for interpretable analysis and fine-tuning of selective positional embedding.
- MOHFE and HQ-Retrieval provide noise-suppressed encoding and gradient-based query filtering, reducing decoder depth from eight to three.
- 59.4% parameter reduction (66.3M vs. 163M) is achieved by integrating LSConv while accelerating training and inference.
- The approach is validated on five benchmarks with consistent gains and good scalability.
2 Related Work
Prior work includes CNN-based detectors and transformer-based detection frameworks, alongside efficiency strategies such as dynamic token pruning and sparse attention. These approaches address efficiency and small-object representation, but the supplied passages provide only a high-level overview.
- CNN-based detectors use hierarchical feature extraction and multi-scale representations, with one-stage methods emphasizing efficiency and two-stage methods using region proposals.
- Transformer detectors use set prediction and query-driven decoding with global context modeling, often combining convolutional backbones, transformer encoders, and learnable queries.
- HELP differs in the supplied overview by combining LSConv, heatmap-guided encoder fusion, and high-quality query retrieval within one pipeline.
- Efficiency-oriented transformer methods include dynamic token pruning and allocation, while other small-object strategies use focal loss, class activation maps, and sparse attention.
3 Heatmap-guided Embedding Learning
HELP learns where to preserve positional information by using heatmap-derived saliency to suppress background clutter and retain foreground cues. HPE supports noise-suppressed encoder features and discriminative decoder query retrieval, while LSConv enriches sparse small-object representations.
- Heatmap-guided Positional Embedding: HELP uses heatmap-guided positional embedding to preserve positional cues in foreground-salient regions and suppress them in background-dominant regions.The heatmap-guided embedding is designed to align positional information with detection semantics.
- Heatmap-guided Positional Embedding: Second- and third-order gradients of classification confidence produce class-discriminative heatmaps, while regression-based backpropagation produces geometry-aware heatmaps.The semantic and geometric heatmaps are combined into a mixed heatmap for subsequent masking.
- Heatmap-guided Positional Embedding: HPE combines class and bounding-box heatmaps into a mixed heatmap, then thresholds it into a binary mask for positional encoding.Locations with Hmap ≤ τ suppress positional encodings; locations with Hmap > τ preserve them.
- Heatmap-guided Positional Embedding: Heatmap supervision is used only during training, so inference uses a standard forward pass without additional gradient computation.The learned heatmap-driven embedding behavior remains in the model parameters after training.
- Encoder–Decoder Integration: MOHFE fuses class-driven and box-driven embeddings into encoder features, while HQ-Retrieval derives compact decoder queries from mixed-heatmap embeddings.These components provide cleaner encoder memory and prioritize foreground-salient positions before deformable-attention decoding.
- Linear-Snake Convolution: LSConv enriches retrieval-relevant features with geometry-aware sampling before heatmap-guided query retrieval.Its complementary sampling paths support sparse, complex small-object representations.
4.1 Datasets and Experimental Protocol
The evaluation spans five public benchmarks and uses standardized splits, fixed hyperparameters, detection-accuracy metrics, and computational-cost measures to compare the method with representative detectors.
- Datasets: Five public benchmarks cover generic and aerial detection, including NWPU VHR-10, PASCAL VOC, DOTA, DIOR, and VisDrone.The datasets assess scalability across data types, object densities, and scene complexity.
- Experimental protocol: Official training/testing splits and identical hyperparameters are used across datasets, with λ = 0.5 and τ = 0.5 selected on validation data.The heatmap threshold τ binarizes the heatmap for query retrieval.
- Comparisons and ablations: Table 1 compares the method with RT-DETR on PASCAL VOC and NWPU VHR-10, while Table 2 evaluates component contributions on both datasets.
- Metrics: Accuracy is reported with mAP@50 and mAP, while GFLOPs and Params (M) measure computational cost and model size.
4.2 Result Analysis
The method improves accuracy and efficiency over RT-DETR on NWPU VHR-10, while HPE visualizations show object-focused heatbar patterns under cluttered and clean scenes.
- Quantitative results: 94.51% mAP@0.5 and 67.20% mAP are achieved on NWPU VHR-10.
- Quantitative results: +1.91 mAP@0.5 and +6.94 mAP are obtained over RT-DETR, while computation falls from 136 to 57 GFLOPs.
- Quantitative results: 59.4% fewer parameters are used than RT-DETR, reducing model size from 163M to 66.3M.
- HPE visualization: HPE heatbars exhibit a hot-center, cold-ends profile that emphasizes object-relevant regions and suppresses background responses during query initialization.
4.3 Ablation Studies and Additional Evaluation
Ablations show complementary gains from the proposed components, reduced decoder depth under suitable training regimes, and consistent improvements across large-scale aerial benchmarks.
- Component-wise ablation: The full configuration achieves the best performance on both NWPU VHR-10 and PASCAL VOC, indicating complementary component gains.
- Component-wise ablation: 94.51% mAP@0.5 and 67.20% mAP are reached on NWPU VHR-10 after adding HQ-Retrieval.
- Component-wise ablation: The full model improves PASCAL VOC mAP@0.5 from 69.4% to 70.6% and mAP from 50.5% to 51.6%.
- Decoder depth: At 100 epochs, Det2 reaches peak performance at 70.5% AP, whereas deeper decoders help under limited training budgets.
- Decoder depth: Three decoder layers provide a favorable trade-off in data-scarce regimes, while GFLOPs increase modestly from 53.7 to 57.0 across depths.
- Additional benchmarks: Relative to DETR, improvements of +6.4/+3.9 on DOTA, +7.6/+5.4 on DIOR, and +7.5/+4.4 on VisDrone are reported for AP50/AP75.
- Additional benchmarks: The method consistently outperforms DETR across DOTA, DIOR, and VisDrone at both AP50 and AP75.
5 Conclusion
The paper argues that selective positional embedding improves query quality and high-IoU localization while allowing substantially lighter decoder stacks without sacrificing performance.
- Conclusion: HELP preserves positional embeddings in foreground-salient regions and suppresses them in background-dominant regions.
- Conclusion: Selective positional embedding produces cleaner decoder inputs, improves high-IoU localization, and reduces reliance on deep decoder stacks.
- Conclusion: The paper identifies embedding placement as a noise-aware allocation problem for improving detector accuracy and efficiency.