Source-linked AI summary
Dynamic Refinement Network for Oriented and Densely Packed Object Detection
Xingjia Pan, Yuqiang Ren, Kekai Sheng, Weiming Dong, Haolei Yuan, Xiaowei Guo, Chongyang Ma, Changsheng Xu
TL;DR
Oriented and densely packed detection remains difficult because receptive fields and inference are insufficiently adaptable, while related benchmark data are limited. The paper introduces DRN, combining adaptive receptive-field selection, object-aware classification and regression refinement, and the SKU110K-R dataset. It reports consistent gains across DOTA, HRSC2016, SKU110K, and SKU110K-R, with representative gains of 3.3% mAP on DOTA and 6.4% mAP on HRSC2016 over the baseline.
Problem
Oriented and densely packed detection is challenged by misaligned axis-aligned receptive fields, static inference for specific test samples, and limited related benchmarks.
Method
DRN uses an FSM to adapt receptive fields to object shapes and orientations, DRH-C and DRH-R for object-aware refinement, and SKU110K-R with oriented bounding-box annotations.
Results
The method shows consistent gains across DOTA, HRSC2016, SKU110K, and SKU110K-R, including 3.3% mAP on DOTA and 6.4% mAP on HRSC2016 over the baseline.
Takeaways & Limitations
Adaptive receptive-field selection and dynamic prediction refinement support oriented and densely packed object detection across multiple datasets.
Abstract
from arXiv · showhide
Object detection has achieved remarkable progress in the past decade. However, the detection of oriented and densely packed objects remains challenging because of following inherent reasons: (1) receptive fields of neurons are all axis-aligned and of the same shape, whereas objects are usually of diverse shapes and align along various directions; (2) detection models are typically trained with generic knowledge and may not generalize well to handle specific objects at test time; (3) the limited dataset hinders the development on this task. To resolve the first two issues, we present a dynamic refinement network that consists of two novel components, i.e., a feature selection module (FSM) and a dynamic refinement head (DRH). Our FSM enables neurons to adjust receptive fields in accordance with the shapes and orientations of target objects, whereas the DRH empowers our model to refine the prediction dynamically in an object-aware manner. To address the limited availability of related benchmarks, we collect an extensive and fully annotated dataset, namely, SKU110K-R, which is relabeled with oriented bounding boxes based on SKU110K. We perform quantitative evaluations on several publicly available benchmarks including DOTA, HRSC2016, SKU110K, and our own SKU110K-R dataset. Experimental results show that our method achieves consistent and substantial gains compared with baseline approaches. The code and dataset are available at https://github.com/Anymake/DRN_CVPR2020.
1. Introduction
The paper targets oriented and densely packed object detection, where axis-aligned receptive fields, static inference, and limited oriented benchmarks create challenges. It proposes DRN with adaptive feature selection, dynamic refinement heads, and SKU110K-R, reporting gains across multiple datasets.
- Challenges: Oriented and densely packed objects challenge detectors because horizontal regions can misalign with objects and static models lack flexibility for test samples.Dense arbitrary orientations can cause multiple instances to be enclosed by one horizontal RoI, hindering accurate feature extraction.
- Method: DRN combines an FSM that adapts receptive fields to object shapes and orientations with DRHs that refine predictions in an object-aware manner.DRH-C and DRH-R target classification and regression, respectively.
- Results: The method shows consistent and substantial gains across DOTA, HRSC2016, SKU110K, and SKU110K-R.The introduction reports this as a contribution across oriented and densely packed object detection.
2. Related Work
Prior work addresses oriented detection through region transformations, feature selection, deformable sampling, and dynamic filters, but these approaches differ in computational cost, rigidity, or prediction scope. DRN combines dense learnable rotation in FSM with content-aware refinement heads.
- Oriented detection: R-CNN-based methods use horizontal RoIs that can misalign with rotated objects, while rotated-anchor methods increase computation through many angle, scale, and aspect-ratio combinations.RoI Trans reduces misalignment by transforming horizontal RoIs into oriented ones and extracting rotation-invariant features.
- Feature selection: Feature-selection methods dynamically choose levels, pixels, or receptive fields, while FSM selects features through dense learnable rotation transformations.Earlier approaches include FPN, FSAF, dynamic feature selection, and SKN.
- Spatial transformation: Active convolution shares offsets spatially, whereas deformable convolution predicts dynamic offsets; FSM learns rotation transformations densely at each position.ORN uses a shared hyperparameter rotation angle, while the paper's transformation predicts angles at each position.
- Dynamic refinement: Dynamic filters and CARAFE generate example-wise or content-aware features, but DRHs specifically refine detection results in a content-aware manner.The related-work passage contrasts DRHs with methods that change feature reassembly or offsets without changing kernel weights.
3. Our Method and Dataset
The method combines feature selection and dynamic refinement to address receptive-field misalignment and sample-specific prediction in oriented, densely packed detection. It also introduces SKU110K-R, an oriented-box extension of SKU110K.
- 3.1. Network Architecture: The network uses CenterNet with an added angle-prediction branch to construct oriented bounding boxes from center, size, offset, and orientation outputs.The corner coordinates are obtained using a rotation matrix and the predicted center offset.
- 3.2. Feature Selection Module: FSM adaptively aggregates features from kernels with different sizes, aspect ratios, and orientations to reduce mismatches with object geometry.Its rotation convolution layers provide different receptive fields, whose outputs are combined through attention.
- 3.3. Dynamic Refinement Head: DRH models each input object’s particularity through separate classification and regression heads that dynamically refine predictions.DRH-C generates example-wise filters and feature refinements, while DRH-R produces object-aware regression refinements.
- 3.4. SKU110K-R Dataset: SKU110K-R extends SKU110K by rotating images at six angles and crowdsourcing oriented bounding-box annotations for every instance.The source dataset contains 11,762 images and 1,733,678 instances collected under varied supermarket imaging conditions.
4. Experiments
Experiments evaluate DRN on multiple oriented and densely packed detection datasets, showing gains over baselines and improvements from FSM and DRH components.
- Experimental Setup: The experiments cover DOTA, HRSC2016, and SKU110K/SKU110K-R using oriented or COCO-style average precision evaluation.DOTA and HRSC2016 use PASCAL VOC mAP, while SKU110K and SKU110K-R use COCO-style mAP, AP75, and AR300.
- Experimental Results: 3.3% mAP gain over the baseline is reported on DOTA for the oriented bounding box task.The method follows an anchor-free paradigm and achieves comparable results with SCRDet.
- Experimental Results: 6.4% mAP gain is reported on HRSC2016, while COCO-style evaluation reports a 1.9% mAP gain.The HRSC2016 improvement is attributed to adaptively adjusting receptive fields; performance also improves as IoU increases.
- Ablation Study: Adding the receptive-field adjustment component improves FSM performance, with rotation and multiple kernel shapes providing further gains.The three-kernel FSM enables adjustment in shape and rotation, while more complex kernel shapes remain future work.
- Ablation Study: DRH-C raises Acc from 0.21 to 0.32 and Rec from 0.81 to 0.89, while providing 0.7% AP50 and 0.6% AP75 gains.DRH-R reduces scale-regression L1 error by 1.24 and improves AP50 and AP75 by 0.7% and 0.6%, respectively.
- Ablation Study: The method improves accuracy over the baseline with a very limited increase in model parameters when DRH-R is applied to the scale head.The comparison includes processing time, parameter count, AP50, and AP75; post-processing time is excluded from the timing measurement.
5. Conclusion
The paper presents DRN as a unified framework for oriented and densely packed object detection, combining adaptive receptive fields, dynamic prediction refinement, and the SKU110K-R dataset.
- 5. Conclusion: FSM aggregates information using receptive fields adapted to object shapes and orientations, addressing misalignment between neurons and objects.The framework uses FSM to adjust receptive fields and DRH-C and DRH-R to refine classification and regression dynamically.
- 5. Conclusion: DRH-C and DRH-R refine predictions dynamically to address the mismatch between generic model knowledge and specific test objects.
- 5. Conclusion: SKU110K-R relabels SKU110K with oriented bounding boxes to support research on oriented and densely packed object detection.
- 5. Conclusion: Experiments report consistent gains across multiple datasets compared with baseline approaches.The paper identifies more effective dynamic-model mechanisms and few-shot oriented detection as future directions.