PaperlayerView exact revision on arXiv

Source-linked AI summary

Centralized Feature Pyramid for Object Detection

Yu Quan, Dong Zhang, Liyan Zhang, Jinhui Tang

arXiv:2210.02093v1cs.CV

TL;DR

Existing feature pyramids emphasize inter-layer interactions but overlook intra-layer regulation and corner regions important for dense prediction. CFP combines a lightweight MLP, learnable visual centers, and top-down centralized regulation to address this gap. On MS-COCO, it reports consistent gains over YOLOv5 and YOLOX baselines, with supporting ablations and efficiency comparisons.

  • Problem

    Existing feature pyramids overlook intra-layer feature regulation and corner regions, while transformer-based approaches addressing broader context can be computationally complex.

  • Method

    CFP combines a lightweight MLP for global long-range dependencies, a parallel learnable visual center for local regions, and top-down regulation from deepest to shallow features.

  • Results

    CFP achieves consistent performance gains on YOLOv5 and YOLOX baselines, while ablations report 1.4% mAP improvement for both models when using EVC.

  • Takeaways & Limitations

    The proposed pyramid efficiently combines global dependencies with an all-round yet discriminative feature representation for object detection.

Abstract

from arXiv · show

Visual feature pyramid has shown its superiority in both effectiveness and efficiency in a wide range of applications. However, the existing methods exorbitantly concentrate on the inter-layer feature interactions but ignore the intra-layer feature regulations, which are empirically proved beneficial. Although some methods try to learn a compact intra-layer feature representation with the help of the attention mechanism or the vision transformer, they ignore the neglected corner regions that are important for dense prediction tasks. To address this problem, in this paper, we propose a Centralized Feature Pyramid (CFP) for object detection, which is based on a globally explicit centralized feature regulation. Specifically, we first propose a spatial explicit visual center scheme, where a lightweight MLP is used to capture the globally long-range dependencies and a parallel learnable visual center mechanism is used to capture the local corner regions of the input images. Based on this, we then propose a globally centralized regulation for the commonly-used feature pyramid in a top-down fashion, where the explicit visual center information obtained from the deepest intra-layer feature is used to regulate frontal shallow features. Compared to the existing feature pyramids, CFP not only has the ability to capture the global long-range dependencies, but also efficiently obtain an all-round yet discriminative feature representation. Experimental results on the challenging MS-COCO validate that our proposed CFP can achieve the consistent performance gains on the state-of-the-art YOLOv5 and YOLOX object detection baselines.

I. INTRODUCTION

Existing detection systems use feature pyramids for varied object scales, but primarily emphasize inter-layer interactions while overlooking intra-layer regulation and corner cues. CFP addresses this gap with explicit visual centers and top-down centralized regulation.

  • Feature pyramids assign objects of different sizes to suitable feature layers with appropriate contextual information.
  • Existing feature-pyramid methods emphasize inter-layer interactions but overlook intra-layer feature regulation, despite its reported benefits for visual recognition.
  • CNN backbones mainly locate discriminative regions because of limited receptive fields, while attention and transformer methods capture wider context but can ignore corner cues.
  • CFP combines a lightweight MLP for global long-range dependencies with a parallel learnable visual center for local key or corner regions.
  • CFP uses visual-center information from deepest features to regulate frontal shallow features through globally centralized top-down regulation.
  • CFP achieves consistent gains on YOLOv5 and YOLOX object-detection baselines in MS-COCO experiments.

II. RELATED WORK

Related work develops multi-scale feature pyramids and mechanisms for global context, but transformer-based recognition remains computationally intensive. CFP is presented within this progression toward efficient, richer visual representations.

  • A. Feature Pyramid in Computer Vision: Feature pyramids represent multi-scale information and support detection of objects with different sizes.
  • A. Feature Pyramid in Computer Vision: FPN builds a top-down path with lateral connections, while PANet adds a bottom-up pathway to share information between feature layers.
  • Attention and non-local operations were introduced to capture broader image context and long-range dependencies beyond CNN local representations.
  • The CFP architecture combines a backbone, centralized feature pyramid, and detection head with classification and regression losses.
  • Transformer-based recognition models can provide global feature interactions but are computationally intensive and complex.

C. MLP in Computer Vision

MLP-based vision methods are motivated as simpler alternatives to attention-based transformer modules while retaining global information processing.

  • Recent vision methods replace attention-based transformer modules with MLPs to alleviate transformer complexity while maintaining strong performance.

D. Object Detection

Object detection seeks comprehensive scene descriptions through object categories and locations, using architectures that address scale variation and global representation. CFP applies explicit intra-layer regulation within a feature pyramid.

  • D. Object Detection: Object detection recognizes image objects and predicts their categories and locations as a comprehensive scene description.
  • D. Object Detection: Two-stage detectors generate region proposals before classification and regression, whereas single-stage detectors directly generate bounding boxes and classify regions.
  • The implementation details are organized around CFP architecture, explicit visual centers, and global centralized regulation on the feature pyramid.
  • CFP targets the intra-layer feature regulation gap left by methods focused mainly on inter-layer feature interactions.
  • CFP extracts a five-level feature pyramid and applies a lightweight MLP to capture global long-range dependencies in its deepest feature level.

B. Explicit Visual Center (EVC)

The Explicit Visual Center combines a lightweight MLP for global long-range dependencies with a learnable visual center for local corner-region information. Its parallel outputs are integrated into a feature representation for downstream recognition.

  • EVC architecture: EVC uses parallel lightweight MLP and learnable visual center mechanisms to capture global dependencies and local corner regions, respectively.The MLP processes top-level features X4, while the visual center mechanism aggregates intra-layer local regional features.
  • Stem block: The Stem block smooths X4 before EVC using a 7 × 7 convolution, batch normalization, and an activation function.The convolution output channel size is 256 in the implementation.
  • EVC architecture: The two branch outputs are concatenated along the channel dimension to form the EVC output.X is formed from the outputs of MLP(Xin) and LVC(Xin).
  • MLP: The lightweight MLP contains depthwise-convolution and channel-MLP residual modules with normalization, channel scaling, and DropPath operations.Depthwise convolution reduces computational costs compared with traditional spatial convolution, while channel MLP reduces complexity compared with space MLP.
  • LVC: LVC encodes Stem features with convolution layers and a CBR block before mapping encoded pixels to learnable codewords using scaling factors.The codebook stores visual centers and the scaling factors map pixel and codeword positions to corresponding information.
  • LVC: LVC predicts channel scaling coefficients from codebook outputs, multiplies them channel-wise with Xin, and adds the resulting local features back to Xin.The scaling function is sigmoid, and channel-wise addition combines the local corner-region features with Stem features.

C. Global Centralized Regulation (GCR)

Global Centralized Regulation applies EVC only to the deepest feature level, then uses its spatial explicit visual center information to regulate shallower feature levels in a top-down manner. This design reduces the computational overhead of applying intra-layer regulation at every pyramid level.

  • Global Centralized Regulation: GCR implements spatial EVC on the top feature X4 and uses its output to regulate frontal shallow features X3 to X2.The deepest features are used because they contain abstract representations scarce in shallow features.
  • Global Centralized Regulation: For each shallow level, the deep regulated features are upsampled to the shallow spatial scale and concatenated along the channel dimension.

A. Dataset and Evaluation Metrics

The experiments use MS-COCO to evaluate CFP with YOLOv5 and YOLOX baselines, while comparing lightweight MLP and attention-based feature-regulation variants under consistent training settings.

  • Dataset: MS-COCO provides 80 object classes across 118k training, 5k validation, and 20k test images.Training images are resized to 640 × 640, with Mosaic and MixUp augmentation.
  • Baselines: YOLOv5 and YOLOX serve as the state-of-the-art baseline models for evaluating CFP generality.Experiments use their default training and inference settings unless otherwise stated.
  • Baselines: YOLOv5 experiments include Small, Media, and Large scales, with FPN necks and coupled detection heads.DarkNet53 is used for state-of-the-art comparisons, while Modified CSPNet v5 supports ablations.
  • Baselines: YOLOX retains the overall network structure while using a decoupled detection head and two experimental backbones.The backbones include DarkNet53 and Modified CSPNet v5.
  • Implementation: The implementation compares MLP and attention-based variants, using CPSLayer and depthwise convolution as lightweight token mixers.The training schedule lasts 150 epochs with five warmup epochs, 640 × 640 inputs, and two GeForce RTX 3090 GPUs.

C. Ablation Study

Ablations on MS-COCO validation measure how local visual centers, lightweight MLPs, and their combination affect accuracy and efficiency. The combined EVC scheme yields the strongest baseline gains, while the selected CFP repetition is R = 1.

  • Study design: The ablation evaluates LVC, MLP, EVC, and CFP using average precision, parameters, computation volume, and latency.The study uses YOLOv5-L and YOLOX-L baselines, with variant comparisons based on YOLOX-L.
  • Component ablations: 1.4% mAP gains are obtained on both YOLOv5-L and YOLOX-L when LVC and MLP are combined as EVC.Using LVC alone improves mAP by 1.0% and 1.3%, while MLP alone improves it by 0.6% and 1.3%, respectively.
  • Component ablations: 49.2% mAP is reached by EVC on YOLOX-L, with lower parameter count and computation volume than the YOLOv5-L counterpart.YOLOX-L is therefore used as the baseline for subsequent ablation experiments.
  • MLP and attention variants: 49.10% mAP is achieved by the proposed lightweight MLP, exceeding YOLOX-L by 1.3%.CSPM is 0.1% below YOLOX-L and adds 0.74ms latency, whereas DWA reaches 49.20% but adds 2.84ms over the lightweight MLP.
  • Repetition ablation: The CFP repetition study selects R = 1 after finding that repeated feature extraction adds computational cost without useful information.The supplied table caption defines R as the number of repetitions.

D. Efficiency Analysis

The efficiency analysis compares MLP and attention-based variants using multiple metrics, then places the lightweight MLP against transformer and MLP detectors. The proposed MLP combines stronger precision with lower resource requirements in the reported comparisons.

  • Variant comparison: The multi-metric comparison evaluates MLP and attention-based variants against a common YOLOX-L baseline on MS-COCO validation.All models use the same data augmentation settings, and Figure 5 reports the comparison across multiple efficiency and accuracy measures.
  • Efficiency comparison: The lightweight MLP provides better precision with fewer parameters, lower computation volume, and lower inference time than the other tested structures.This comparison motivates using MLP instead of attention-based modules for efficient long-range dependency modeling.
  • Comparison with prior methods: 1.3% higher average precision than Mask R-CNN is reported for MLPYOLOX-L, with lower memory use.Compared with DETR, MLPYOLOX-L improves mAP by up to 7.1% without extra computational cost.
  • Comparison with prior methods: 62.43M fewer parameters are reported for MLP at the same mAP as REGO-Deformable DETR.The comparison positions the proposed MLP as both accurate and memory-efficient relative to the cited transformer methods.

E. Comparisons with State-of-the-art Methods.

CFP is evaluated against YOLOv5 and YOLOX baselines and other detectors on MS-COCO, combining accuracy comparisons with qualitative failure analysis. The reported results show gains across baseline sizes and improvements over several prior detectors.

  • Quantitative comparisons: 0.5%, 0.5%, and 1.4% mAP gains are reported over YOLOv5 Small, Media, and Large baselines, respectively.
  • Quantitative comparisons: 7.0%, 0.8%, and 1.6% mAP gains are reported over YOLOX Small, Media, and Large baselines, respectively.
  • Quantitative comparisons: CFPYOLOX-L improves mAP by up to 27.80% compared with YOLOv3-ultralytics and its previous detectors.
  • Speed and accuracy: CFPYOLOv5-M is reported as 1.5 times faster than EfficientDet-D2 at nearly the same average precision, while CFPYOLOX-L is 1.8 times faster than EfficientDet-D3 with 1.9% higher accuracy.
  • Quantitative comparisons: 49.40% maximum average accuracy is reported for CFP, 1.6% higher than YOLOX-L.
  • Qualitative results: Qualitative comparisons visualize YOLOX-L, EVCYOLOX-L, and CFPYOLOX-L, with boxes identifying misses, contextual detection errors, and classification errors.White boxes mark failures linked to occlusion, illumination, or small object size; red boxes mark insufficient contextual semantic relationships.

V. CONCLUSIONS AND FUTURE WORK

The conclusion presents CFP as a globally explicit centralized feature-regulation approach for object detection. It combines visual-center modeling with top-down regulation and identifies future extensions to improve feature representation and broader task applicability.

  • CFP uses globally explicit centralized feature regulation for object detection.
  • The spatial explicit visual center uses a lightweight MLP for global long-range dependencies and a learnable visual center for local corner regions.
  • GCR uses deepest intra-layer visual-center information to regulate all frontal shallow features in a top-down feature pyramid.
  • CFP is reported to capture global dependencies, preserve local corner information, and achieve consistent gains on state-of-the-art object-detection baselines.
  • Future work: Future work targets advanced intra-layer feature-regulation methods and applications to semantic segmentation, object localization, instance segmentation, and person re-identification.
Loading 2210.02093v1…