PaperlayerView exact revision on arXiv

Source-linked AI summary

HRDNet: High-resolution Detection Network for Small Objects

Ziming Liu, Guangyu Gao, Lin Sun, Zhiyuan Fang

arXiv:2006.07607v1cs.CV

TL;DR

Small-object detection requires detail that deep networks may lose, while simply increasing image resolution worsens scale variation and computational cost. HRDNet addresses this with multi-depth processing of an image pyramid and MS-FPN fusion of the resulting feature groups. It achieves state-of-the-art results across the reported benchmarks, with especially strong small-object performance.

  • Problem

    Small objects lack detailed information and may disappear in deep networks, while higher resolution can worsen object-scale variation and computational cost.

  • Method

    HRDNet uses shallow backbones for high-resolution images, deep backbones for low-resolution images, and MS-FPN to fuse their multi-scale feature groups.

  • Results

    HRDNet achieves state-of-the-art performance across MS COCO, Pascal VOC, and VisDrone2019, particularly for small-object detection.

  • Takeaways & Limitations

    Combining image-pyramid and feature-pyramid processing improves small-object detection while retaining performance across other object sizes.

Abstract

from arXiv · show

Small object detection is challenging because small objects do not contain detailed information and may even disappear in the deep network. Usually, feeding high-resolution images into a network can alleviate this issue. However, simply enlarging the resolution will cause more problems, such as that, it aggravates the large variant of object scale and introduces unbearable computation cost. To keep the benefits of high-resolution images without bringing up new problems, we proposed the High-Resolution Detection Network (HRDNet). HRDNet takes multiple resolution inputs using multi-depth backbones. To fully take advantage of multiple features, we proposed Multi-Depth Image Pyramid Network (MD-IPN) and Multi-Scale Feature Pyramid Network (MS-FPN) in HRDNet. MD-IPN maintains multiple position information using multiple depth backbones. Specifically, high-resolution input will be fed into a shallow network to reserve more positional information and reducing the computational cost while low-resolution input will be fed into a deep network to extract more semantics. By extracting various features from high to low resolutions, the MD-IPN is able to improve the performance of small object detection as well as maintaining the performance of middle and large objects. MS-FPN is proposed to align and fuse multi-scale feature groups generated by MD-IPN to reduce the information imbalance between these multi-scale multi-level features. Extensive experiments and ablation studies are conducted on the standard benchmark dataset MS COCO2017, Pascal VOC2007/2012 and a typical small object dataset, VisDrone 2019. Notably, our proposed HRDNet achieves the state-of-the-art on these datasets and it performs better on small objects.

1 Introduction

HRDNet addresses the trade-off between small-object detail, large-object performance, and computational cost by combining multi-depth image processing with multi-scale feature fusion.

  • Motivation: High-resolution inputs preserve details and position information for small objects but increase scale variation, computation, and memory demands.Deeper backbones improve semantics but also increase computation and memory usage.
  • Results: The approach combines high-resolution detail with multi-depth and multi-scale processing to improve small-object detection while maintaining efficiency and broader object-detection performance.The paper reports state-of-the-art performance, particularly for small objects, across several datasets.
  • MD-IPN: MD-IPN applies shallow backbones to high-resolution images and deep backbones to low-resolution images, producing multi-scale feature groups with different semantic strengths.Its inputs form an image pyramid with decreasing ratio α, and each output group contains multi-level feature maps.
  • MS-FPN: MS-FPN fuses multi-scale feature groups because standard FPN cannot directly handle the additional scale streams.Information propagates both within feature pyramids and between streams.
  • HRDNet: HRDNet combines an image pyramid with a feature pyramid to balance detection performance across small, middle, and large objects.The architecture contains MD-IPN and MS-FPN.

2 Related Work

Prior work improves small-object detection through data, feature, resolution, and architecture changes, while HRDNet combines image- and feature-pyramid processing to address small and other object scales together.

  • Small object detection: Small-object detection methods include oversampling and copy-pasting, super-resolved features, preserved spatial resolution, and focused training regions.These approaches target limited small-object representations or background influence.
  • HRDNet: Unlike these lines of work, HRDNet combines image and feature pyramids to improve small-target detection while retaining performance for other objects.The paper presents this combination as its distinguishing approach.
  • High-resolution detection: High-resolution detection studies explored fast detection on remote-sensing images and 4K or 8K videos but did not fully examine high-resolution inputs for small-object detection.HRDNet concentrates on this high-resolution small-object setting.
  • Feature-level imbalance: Feature-level imbalance arises because multi-level features convey different semantic information, motivating pathways such as FPN and PANet.FPN uses a top-down pathway to transmit semantic information and alleviate imbalance.

3 High-Resolution Detection Network

HRDNet balances the benefits of high-resolution inputs for small-object detection against scale variance and computational cost by combining multi-depth processing with multi-scale feature fusion. Its MD-IPN uses different-depth backbones across image resolutions, while MS-FPN fuses the resulting multi-scale, multi-level features.

  • HRDNet overview: HRDNet uses deep backbones for low-resolution images and shallow backbones for high-resolution images to balance detection quality and computation.The framework is designed to preserve high-resolution detail while obtaining stronger semantics from deeper processing of lower-resolution inputs.
  • MD-IPN: MD-IPN processes an image pyramid through multiple streams, with each stream corresponding to a backbone of different depth.The highest-resolution input is assigned to a shallow stream, whereas lower-resolution inputs are assigned to deeper streams.
  • MD-IPN: MD-IPN produces feature groups that vary across both image scale and feature hierarchy for subsequent detection processing.Each feature group contains multi-level features indexed by stream scale and hierarchy level.
  • MS-FPN: MS-FPN fuses multi-scale feature groups because conventional FPN cannot directly handle their additional stream dimension.Information propagates both across feature levels and between streams of different resolutions.
  • MS-FPN: The highest-resolution feature preserves resolution for small-object detection while incorporating semantically strong features from multiple streams.The final feature group is formed from the fused outputs of the multi-scale pyramid.

4 Experiments

Experiments evaluate HRDNet on VisDrone2019, MS COCO2017, and Pascal VOC, including ablations of resolution, fusion design, efficiency, and comparisons with state-of-the-art methods. Results show improved small-object detection while preserving performance on larger objects across benchmarks.

  • Experiment Details: Experiments use VisDrone2019, MS COCO2017, and Pascal VOC2007/2012, with dataset-specific training and evaluation protocols.VisDrone2019 is the typical small-object benchmark; COCO and Pascal VOC provide broader detection evaluations.
  • Ablation Studies: Higher input resolution improves detection performance on VisDrone2019, with particularly significant gains for small objects under the same settings.The ablation also reports that HRDNet outperforms Cascade R-CNN at the same input resolution.
  • Ablation Studies: 1.1-7.6% performance decreases occur for relatively large objects in single-backbone Cascade R-CNN when input resolution increases.This result illustrates the trade-off between higher resolution and large-object detection.
  • State-of-the-Art Comparisons: HRDNet achieves the best performance on the VisDrone2019 DET validation set, with more than 3.0% AP improvement over HFEA using ResNet152.The comparison uses the state-of-the-art methods listed for the VisDrone2019 benchmark.
  • State-of-the-Art Comparisons: HRDNet achieves state-of-the-art results on COCO2017, including > 4.9% APsmall improvement compared with most recent models.The COCO evaluation uses the MS COCO test-dev benchmark.
  • State-of-the-Art Comparisons: On Pascal VOC2007/2012, HRDNet improves small-object detection while maintaining performance for large objects.The dataset contains relatively few small objects, so this experiment emphasizes preservation of large-object performance.

5 Conclusion

HRDNet addresses the damage to large-object detection and scale variation caused by simply increasing image resolution. With MD-IPN and MS-FPN, it achieves state-of-the-art small-object detection on VisDrone2019 and outperforms on MS COCO and Pascal VOC.

  • Conclusion: HRDNet addresses the limitations of simply increasing image resolution, which can damage large-object detection under severe object-scale variation.The architecture combines MD-IPN and MS-FPN to handle these issues.
  • Conclusion: HRDNet achieves state-of-the-art performance on VisDrone2019 and outperforms on MS COCO and Pascal VOC.The conclusion identifies VisDrone2019 as the small-object detection benchmark.
Loading 2006.07607v1…