PaperlayerView exact revision on arXiv

Source-linked AI summary

Parallel Residual Bi-Fusion Feature Pyramid Network for Accurate Single-Shot Object Detection

Ping-Yang Chen, Ming-Ching Chang, Jun-Wei Hsieh, Yong-Sheng Chen

arXiv:2012.01724v5cs.CV

TL;DR

Feature pyramids can lose accurate localization through pooling, and detecting small and large objects together remains challenging. PRB-FPN uses parallel bi-directional fusion with CORE, a residual design, and bottom-up fusion; the authors report better accuracy and speed than other bi-directional and state-of-the-art one-stage methods.

  • Problem

    Feature pyramids can lose localization detail through pooling, and existing approaches have difficulty detecting small and large objects accurately at the same time.

    Source: §1 · pp. 1–2

  • Method

    PRB-FPN uses parallel bi-directional feature fusion with CORE and a residual design, alongside a bottom-up fusion module.

    Source: §1 · p. 2, §3.1 · p. 5, §3.4 · p. 6

  • Results

    PRB-FPN outperforms other bi-directional methods and state-of-the-art one-stage methods in accuracy and speed.

    Source: §5 · p. 13

  • Takeaways & Limitations

    The authors conclude that PRB-FPN can fuse deep and shallow features for fast and accurate object detection and integrate with popular backbones.

    Source: §5 · p. 13

Abstract

from arXiv · show

This paper proposes the Parallel Residual Bi-Fusion Feature Pyramid Network (PRB-FPN) for fast and accurate single-shot object detection. Feature Pyramid (FP) is widely used in recent visual detection, however the top-down pathway of FP cannot preserve accurate localization due to pooling shifting. The advantage of FP is weakened as deeper backbones with more layers are used. In addition, it cannot keep up accurate detection of both small and large objects at the same time. To address these issues, we propose a new parallel FP structure with bi-directional (top-down and bottom-up) fusion and associated improvements to retain high-quality features for accurate localization. We provide the following design improvements: (1) A parallel bifusion FP structure with a bottom-up fusion module (BFM) to detect both small and large objects at once with high accuracy. (2) A concatenation and re-organization (CORE) module provides a bottom-up pathway for feature fusion, which leads to the bi-directional fusion FP that can recover lost information from lower-layer feature maps. (3) The CORE feature is further purified to retain richer contextual information. Such CORE purification in both top-down and bottom-up pathways can be finished in only a few iterations. (4) The adding of a residual design to CORE leads to a new Re-CORE module that enables easy training and integration with a wide range of deeper or lighter backbones. The proposed network achieves state-of-the-art performance on the UAVDT17 and MS COCO datasets. Code is available at https://github.com/pingyang1117/PRBNet_PyTorch.

1 Introduction

The paper targets the difficulty of accurately detecting small and large objects together: pooling reduces spatial resolution, while deep and shallow backbones present different accuracy-efficiency trade-offs. It proposes PRB-FPN, a parallel bi-fusion design with CORE and residual improvements, and reports state-of-the-art results on Pascal VOC and MS COCO.

1 Introduction

  • Tiny objects can shrink to about a single-pixel feature vector in the final feature-pyramid layer, limiting spatial detail for accurate discrimination.The paper also notes that shallow backbones improve computational efficiency but reduce retained contextual and semantic features.Source: §1 · p. 1
  • Accurately detecting small and large objects together remains difficult, even when methods improve detection of one size range.The cited LPIRC 2019 method improves general-sized object detection but not small-object detection on COCO.Source: §1 · p. 2
  • PRB-FPN fuses three feature-map layers in parallel to produce three prediction maps, aiming to retain semantic and localization information for tiny and large objects.The design introduces CORE for feature fusion and purification, and adds a residual design to support training and integration with popular backbones.Source: §1 · p. 2, Figure 2 · p. 3
  • Parallelization is presented as a way to represent small and large objects without losing efficiency, unlike approaches that concatenate large feature maps and increase their size.The paper describes PRB-FPN as suitable for multiple object classes and sizes, real-time operation, and edge deployment.Source: §1 · p. 2
  • PRB-FPN achieves state-of-the-art results for accurate and efficient object detection on Pascal VOC and MS COCO, with reported generalization across object sizes and types.The authors also describe the network as easily trained and integrated with different backbones.Source: §1 · p. 3

2 Related Works

Related work covers one-stage detectors, feature pyramids, and alternatives for multi-scale detection. The review emphasizes the challenge of retaining localization for small objects while balancing detection accuracy and computational efficiency.

2 Related Works

  • One-stage detectors combine a backbone and predictor; SSD uses multiple feature maps for varying object sizes, while FPN-based designs aim to improve small-object detection.The review also discusses YOLO, RetinaNet, RefineDet, Pelee, and PFPN as developments in one-stage detection.Source: §2 · p. 3
  • Top-down feature pyramids aggregate semantic features for scale invariance, but pooling and quantization can impair accurate localization, especially for small objects.The review surveys FPN, SPP, and PFPN among commonly used feature-pyramid approaches.Source: §2 · pp. 3–4
  • Bi-directional pyramids can recover shallow-layer information for small-object detection, while reviewed alternatives such as M2Det raise efficiency concerns and BiFPN still struggles to detect small and large objects together.Other approaches discussed include gated feature flow, path aggregation, and concatenation-based fusion.Source: §2 · p. 4
  • The review also covers multi-scale context methods, anchor-free detection, and transfer learning, noting trade-offs such as lower efficiency for some regression-based methods and limited transfer gains when training data are sufficient.It describes transfer learning as useful in scenarios with few training samples.Source: §2 · p. 4

3 Method

PRB-FPN combines parallel bidirectional fusion with recursively refined CORE features and bottom-up location fusion to retain information for detecting objects across scales.

3.1 Parallel Concatenation and Re-organization Feature Bi-Fusion Architecture

  • Parallel bi-fusion paths generate concurrent fused maps for prediction, keeping scale-specific features for objects ranging from tiny to large.For N prediction maps, the design runs N concurrent fusion paths and feeds each fused output to its corresponding prediction map.Source: §3.1 · p. 5

3.2 Concatenation and Re-organization for Feature Bi-Fusion

  • CORE recursively fuses top-down semantic features and bottom-up spatial localization features, using learned 1 × 1 convolution to keep fused maps fixed-size.It also re-organizes features from the shallower layer before fusion.Source: §3.2 · pp. 5–6

3.3 Residual Bi-Fusion Feature Pyramid

  • Re-CORE recursively passes features between modules and fuses adjacent scales with residual connections, supporting easier backbone integration and small-object detection.The module combines features from shallow, current, deep, and deeper layers; the paper says the residual design supports integration with a wide range of backbones.Source: §3.3 · p. 6

3.4 Bottom-up Feature Fusion

  • BFM’s Re-Org splits C feature-map channels into 4C, preserving spatial information and enabling 1 × 1 convolution to generate robust semantic features that improve small-object detection; its bottom-up path provides local information for accurate localization, particularly of large and mid-sized objects.Source: §3.4 · p. 6

4 Experimental Results

The evaluations test PRB-FPN across datasets, backbones, module ablations, and comparisons with other detectors. They report accuracy and speed results, including performance on small and large objects and real-time detection benchmarks.

4 Experimental Results

  • The experiments evaluate AP and FPS on MS COCO and UAVDT, using multiple feature-extraction backbones and module ablations to assess accuracy and efficiency.Backbones include PeLee, MobileNet-V2, DarkNet-53, VGG16, ResNet-50, DenseNet, and CSPnet.Source: §4 · p. 8, §4.1 · p. 8

4.2 BFM Accuracy Improvements

  • BFM maintains high AP across four backbones; its AP50 improvements are 6.5% for DarkNet, 3.5% for Pelee, and 0.2% for DenseNet.The authors report that BFM's computational load is very light and can be ignored for all backbones.Source: §4.2 · p. 8
  • With ResNet-50 and EfficientNet backbones, BFM outperforms EfficientDet-D0 and NAS-FPN; FPN-BPN's downsampling lowers small-object accuracy and its deconvolution lowers efficiency.The comparison is described in terms of accuracy and efficiency.Source: §4.2 · p. 9

4.3 BFM with Re-CORE Accuracy Improvements

  • BFM improves small-object detection as input size increases, while its gains for large objects decrease; PRB-FPN outperforms YOLOv3 across all categories.Frame rates with and without BFM are reported as very similar.Source: §4.3 · p. 9
  • On the UAVDT example, PRB-FPN detects tiny distant vehicles that LRFNet misses, though PRB-FPN also produces a false pedestrian detection in a dark-background example.The comparisons are shown in Figure 5 and Figure 8.Source: Figure 5 · p. 9, §4.3 · p. 10, Figure 8 · p. 11

4.4 PRB-FPN vs the Original FPN

  • On UAVDT, the parallel design raises accuracy from 34.3% to 65.47% with MobileNet-V2 and from 64.52% to 76.55% with CSPDarknet-53.The MobileNet-V2 results also report double the accuracy and triple the efficiency over RetinaNet.Source: §4.4 · p. 11, Table 4 · p. 12

4.5 Comparisons with State-of-The-Art Models

  • On MS COCO, PRB-FPN achieves stronger AP-small versus inference-time performance than the compared one-stage detectors, including YOLOv4, YOLOv3, EfficientDet, ATSS, SM_NAS, and NAS-FPN.Figure 9 plots inference time against APs.Source: Figure 9 · p. 11, §4.5 · p. 12

4.6 Comparisons with State-of-The-Art real-time Models

  • PRB-FPN with a scaled P6 backbone surpasses contemporary methods on the real-time COCO benchmark, while its P5 variant exceeds YOLOv7-X at comparable computational cost.The comparison is reported in Table 7.Source: §4.6 · p. 13, Table 7 · p. 13

5 Conclusions

The authors conclude that PRB-FPN fuses deep and shallow pyramid features for fast, accurate detection, and report that it outperforms other bi-directional and one-stage methods in accuracy and speed. They identify anchor-free methods and architecture search as possible future directions.

5 Conclusions

  • PRB-FPN combines deep and shallow pyramid features, and its evaluations report better accuracy and speed than other bi-directional and state-of-the-art one-stage methods.The authors also describe its residual bi-fusion design as easy to train and integrate with popular backbones.Source: §5 · p. 13
  • Future work may explore anchor-free methods to avoid handcrafted anchors and network architecture search across backbones and feature pyramids.The authors state these directions might further improve detection accuracy or find a better architecture.Source: §5 · p. 13
  • In a foggy-scene example, PRB-FPN6 recognizes objects that YOLOv7-E6E fails to detect accurately.The comparison is presented in Figure 10.Source: Figure 10 · p. 15

Related papers

Loading 2012.01724v5…