SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing

Xue Yang; Junchi Yan; Wenlong Liao; Xiaokang Yang; Jin Tang; Tao He

Source-linked AI summary

SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing

Xue Yang, Junchi Yan, Wenlong Liao, Xiaokang Yang, Jin Tang, Tao He

arXiv:2004.13316v2Apr 28, 2020cs.CVcs.LGeess.IV

TL;DR

Small, cluttered, and rotated objects challenge detectors because background contamination and rotation-regression boundary problems degrade localization. SCRDet++ introduces instance-level feature-map denoising and an IoU-adjusted smooth L1 loss, achieving state-of-the-art performance across complex-scene datasets while releasing S2TLD.

Problem
Existing detectors often overlook small size, cluttered arrangements, arbitrary orientations, and rotation-regression boundary problems that complicate robust detection.
Method
SCRDet++ uses end-to-end instance-level feature-map denoising and adds an IoU constant factor to smooth L1 loss for rotated bounding-box regression.
Results
The method achieves state-of-the-art performance on complex-scene rotation-detection datasets, while InLD improves detection by 1.44%, 1.55%, 1.4%, and 0.86% across four datasets.
Takeaways & Limitations
The plug-in InLD module improves detection across tasks, and S2TLD provides 5,786 images with 14,130 traffic-light instances across five categories.
Takeaways & Limitations
ImLD is optional and is omitted from subsequent experiments unless otherwise stated, depending on the dataset and computing environment.

Abstract

from arXiv · show

Small and cluttered objects are common in real-world which are challenging for detection. The difficulty is further pronounced when the objects are rotated, as traditional detectors often routinely locate the objects in horizontal bounding box such that the region of interest is contaminated with background or nearby interleaved objects. In this paper, we first innovatively introduce the idea of denoising to object detection. Instance-level denoising on the feature map is performed to enhance the detection to small and cluttered objects. To handle the rotation variation, we also add a novel IoU constant factor to the smooth L1 loss to address the long standing boundary problem, which to our analysis, is mainly caused by the periodicity of angular (PoA) and exchangeability of edges (EoE). By combing these two features, our proposed detector is termed as SCRDet++. Extensive experiments are performed on large aerial images public datasets DOTA, DIOR, UCAS-AOD as well as natural image dataset COCO, scene text dataset ICDAR2015, small traffic light dataset BSTLD and our released S$^2$TLD by this paper. The results show the effectiveness of our approach. The released dataset S2TLD is made public available, which contains 5,786 images with 14,130 traffic light instances across five categories.

1 INTRODUCTION

SCRDet++ targets small, cluttered, and arbitrarily oriented objects, especially where horizontal boxes include background or nearby objects. It introduces instance-level feature denoising and an IoU-enhanced smooth L1 loss, then evaluates them across diverse datasets.

Motivation: Small, cluttered, and arbitrarily oriented objects remain challenging, particularly in aerial imagery and other complex scenes.Small objects can be overwhelmed by surrounding content, while dense arrangements blur feature boundaries and arbitrary orientations complicate localization.
Results: On DOTA-v1.0, the journal version improves overall accuracy from 72.61% to 76.81% for OBB and from 75.35% to 79.35% for HBB relative to the conference detector.These comparisons are reported as extensions over the earlier SCRDet conference version.
Contributions: Instance-Level Denoising (InLD) suppresses interference from other objects and background in feature maps, using supervised segmentation derived approximately from detection boxes.The module aims to decouple object-category features while reducing background interference, with little additional computation and parameter increase.
Contributions: An IoU-enhanced smooth L1 loss addresses rotated-box regression boundaries associated with angular periodicity and exchangeable edges.These issues can make an ideal rotation path incur a large loss and force more complex regression behavior.
Contributions: 5,786 images and 14,130 traffic-light instances across five categories comprise the released S2TLD dataset.The dataset is used to further verify InLD’s effectiveness and generalization.
Results: SCRDet++ reports state-of-the-art rotation-detection performance on public datasets and notable InLD gains across multiple detection tasks.The experiments cover DOTA, DIOR, UCAS-AOD, COCO, ICDAR2015, BSTLD, and S2TLD.

2 RELATED WORK

Related work spans general object detectors, rotated-object and scene-text detectors, image denoising, and small-object detection. SCRDet++ positions instance-level feature-map denoising and IoU-Smooth L1 loss as additional tools for these challenges.

General Object Detection: CNN-based detection includes region-based, single-stage, anchor-based, and anchor-free detector families.Examples include Faster R-CNN, RetinaNet, YOLO, FCOS, CornerNet, CenterNet, and ExtremeNet.
Rotation Detection: Rotated bounding boxes and quadrangles are widely used to represent multi-oriented objects in aerial and scene-text detection.Prior methods include rotated proposals, vertex regression, single-stage rotated detection, and quadrilateral regression.
Image Denoising: Image denoising research covers additive white noise, real noise, blind denoising, and hybrid noisy or blurred low-resolution images.Denoising has also been applied to improve classification, detection, and semantic segmentation.
Small Object Detection: Small-object detection remains unsolved, with common approaches including augmentation, multi-scale fusion, tailored sampling, generative models, and multi-scale training.SCRDet++ presents denoising as another perspective because small-object features are often overwhelmed by background and similar objects.
Positioning SCRDet++: SCRDet++ combines a general-purpose instance-level feature-denoising module with IoU-Smooth L1 loss for horizontal and rotation detection.The stated scope includes aerial imagery, natural images, and scene texts.

3.1 Approach Overview

SCRDet++ uses a modular pipeline that extracts features, optionally removes image-level noise, suppresses instance-level interference, and predicts class scores and box positions.

Pipeline: The pipeline contains feature extraction, optional image-level denoising, instance-level denoising, and a class-and-box prediction branch.The implementation uses RetinaNet as an embodiment, while the feature extractor can take forms from existing detectors.

3.2 Instance-level Feature Map Denoising

The paper introduces instance-level feature-map denoising to suppress interference from background and other objects in small, cluttered scenes. The method decouples category features across channels while enhancing objects and weakening background spatially.

Motivation and formulation: Instance-level feature-map denoising is presented as an end-to-end object-detection approach, optimized jointly with the detector.The authors position it as the first use of instance-level feature-map denoising for object detection.
Motivation and formulation: The method targets non-object activations, inter-class feature coupling, intra-class boundary blurring, and weak object responses.These effects are illustrated as instance-level noise in feature maps.
Motivation and formulation: Existing attention reweighting separates object from background but does not explicitly weaken mutual interference among objects.The paper motivates category-specific feature decoupling as the missing capability.
Mathematical modeling: InLD approximately assigns different object categories to separate channels and concatenates category, unseen-category, and background components.Responses for background and categories absent from the image are intended to be small.
Implementation and effects: The denoising map is combined with the original feature map so object regions are enhanced, background is weakened, and intra-class boundaries become sharper.This spatial and channel-wise decoupling is intended to alleviate the three identified feature-map problems.
Implementation and effects: Weighting regression loss with the probability map gives ambiguous positive samples smaller weights and higher-quality positives more attention.The authors report faster convergence and improved detection results after introducing this probability map.

3.3 Loss Function Design and Learning

The detector uses standard horizontal box regression before redefining boxes for rotation detection. Rotation handling compares direct and indirect angle regression, then uses an IoU-weighted smooth L1 loss to reduce boundary discontinuities caused by angular periodicity and edge exchangeability.

Horizontal Object Detection: Horizontal detection regresses center, width, and height offsets relative to anchor boxes using normalized coordinate and logarithmic size transformations.The targets use tx and ty for center offsets, and tw and th for width and height ratios.
Horizontal Object Detection: The horizontal multi-task objective combines classification, regression, and instance-level denoising losses with default trade-off weights of 1.Classification uses focal loss, regression uses smooth L1 loss, and InLD uses pixel-wise softmax cross-entropy.
Rotation Object Detection: Rotation angle regression faces large losses near representation boundaries because angular periodicity and exchangeable box edges make equivalent rotations appear geometrically distant.This can force the model into more complex regression paths involving simultaneous rotation and width-height scaling.
Rotation Object Detection: The paper compares direct angle-offset regression with indirect regression that predicts sin θ and cos θ targets, with indirect regression offering a simpler way to avoid boundary problems.The indirect targets are tsin θ = sin(θ · π/180) and tcos θ = cos(θ · π/180).
Rotation Object Detection: The IoU-smooth L1 loss multiplies traditional smooth L1 loss by |−log(IoU)|, reducing the boundary loss increase while scaling gradient magnitude by overlap.The loss separates gradient direction, determined by the regression term, from magnitude, determined by the IoU factor.

4 EXPERIMENTS

Experiments evaluate SCRDet++ across aerial, natural-image, scene-text, and traffic-light datasets, with ablations isolating InLD, ImLD, and IoU-Smooth L1 Loss. The method improves detection across datasets and achieves leading results on several rotation-detection benchmarks.

Ablation Study: 1.95% higher performance is achieved by non-local denoising with Gaussian than the compared denoising operations.The passage identifies non-local Gaussian filtering as the most effective option, except for mean filtering.
Ablation Study: 1.44%, 1.55%, 1.4%, and 0.86% improvements are obtained on DIOR, ICDAR2015, COCO, and S2TLD, respectively, using InLD.The results support general applicability beyond DOTA-v1.0.
Ablation Study: IoU-Smooth L1 Loss improves rotation-detection accuracy and eliminates angle-boundary effects that complicate coordinate regression.The reported accuracies for three methods are 69.83%, 68.65%, and 76.20%; further results are reported across DOTA versions.
Comparison with the State-of-the-Art Methods: 76.56% and 76.81% are achieved on DOTA-v1.0 OBB by R3Det++ with InLD and FPN-InLD, respectively, both reported as best performance.The method also reports 77.80% and 75.11% mAP on DIOR and 96.95% OBB accuracy on UCAS-AOD.

5 CONCLUSION

The paper concludes that feature-map instance-level denoising improves detection of small, densely arranged objects, while IoU-Smooth L1 Loss addresses rotation-regression boundary problems. Experiments across multiple aerial, traffic-light, natural-image, and scene-text datasets demonstrate state-of-the-art detection accuracy.

Conclusion: InLD decouples category features across channels and enhances object features while weakening non-object features spatially.The technique is intended especially for small and densely arranged objects in aerial images.
Conclusion: IoU-Smooth L1 Loss adds an IoU constant factor to smooth L1 loss to address rotation-detection boundary problems and improve rotation estimation.The paper evaluates the approach on DOTA, DIOR, UCAS-AOD, BSTLD, S2TLD, COCO, and ICDAR2015.