PaperlayerView exact revision on arXiv

Source-linked AI summary

GiraffeDet: A Heavy-Neck Paradigm for Object Detection

Yiqi Jiang, Zhiyu Tan, Junyan Wang, Xiuyu Sun, Ming Lin, Hao Li

arXiv:2202.04256v2cs.CV

TL;DR

Object detectors traditionally rely on computationally heavy, classification-oriented backbones, while large-scale variation demands efficient exchange between semantic and spatial features. GiraffeDet reverses this emphasis with a lightweight backbone and a deep Generalized-FPN neck, achieving strong accuracy-efficiency trade-offs across resource levels and 54.1% mAP for GiraffeDet-D29 on COCO.

  • Problem

    Large-scale variation challenges object detection, while conventional CNN backbones are computationally expensive and may retain a classification-to-detection domain shift.

  • Method

    GiraffeDet combines a lightweight S2D-chain backbone with a heavy Generalized-FPN neck using Queen-Fusion and skip-layer connections for multi-scale information exchange.

  • Results

    GiraffeDet models achieve higher accuracy and better efficiency across FLOPs levels; GiraffeDet-D29 reaches 54.1% mAP on COCO with multi-scale testing and outperforms other SOTA methods.

  • Takeaways & Limitations

    The heavy-neck paradigm prioritizes efficient feature extraction in the backbone and extensive semantic-spatial fusion in the neck, especially for small and large objects.

  • Takeaways & Limitations

    The paper assumes that FPN is more crucial than conventional backbones for object detection and identifies classification-pretrained backbones as having a domain-shift issue.

Abstract

from arXiv · show

In conventional object detection frameworks, a backbone body inherited from image recognition models extracts deep latent features and then a neck module fuses these latent features to capture information at different scales. As the resolution in object detection is much larger than in image recognition, the computational cost of the backbone often dominates the total inference cost. This heavy-backbone design paradigm is mostly due to the historical legacy when transferring image recognition models to object detection rather than an end-to-end optimized design for object detection. In this work, we show that such paradigm indeed leads to sub-optimal object detection models. To this end, we propose a novel heavy-neck paradigm, GiraffeDet, a giraffe-like network for efficient object detection. The GiraffeDet uses an extremely lightweight backbone and a very deep and large neck module which encourages dense information exchange among different spatial scales as well as different levels of latent semantics simultaneously. This design paradigm allows detectors to process the high-level semantic information and low-level spatial information at the same priority even in the early stage of the network, making it more effective in detection tasks. Numerical evaluations on multiple popular object detection benchmarks show that GiraffeDet consistently outperforms previous SOTA models across a wide spectrum of resource constraints. The source code is available at https://github.com/jyqi/GiraffeDet.

1 INTRODUCTION

Object detection must handle extreme scale variation, but image pyramids are costly and conventional feature pyramids may insufficiently exchange semantic and spatial information. GiraffeDet addresses these issues with a lightweight backbone, richer multi-scale fusion, and a family of models spanning resource constraints.

  • COCO’s smallest and largest 10% of object instances differ in scale by nearly 20 times, making scale variation a central detection challenge.The reported scales are 0.024 and 0.472, respectively.
  • Image pyramids improve detection but process every image scale, making them computationally expensive.
  • Feature pyramid methods reduce image-pyramid cost but may insufficiently exchange high-level semantic and low-level spatial information.
  • GiraffeDet combines a lightweight backbone with sufficient cross-scale fusion through Queen-Fusion and targets both efficient down-sampling and multi-scale representation.
  • The GiraffeDet family is reported to achieve higher accuracy and better efficiency across FLOPs levels, while its components include Lightweight S2D-chain and Generalized-FPN.
  • 54.1% mAP is reported for GiraffeDet-D29 on COCO with multi-scale testing, outperforming other SOTA methods.

2 RELATED WORK

Related work addresses large-scale variation through CNN detector architectures, image pyramids, feature pyramids, and alternative network designs that fuse multi-scale features. GiraffeDet is presented within this broader shift beyond conventional backbone-neck-head designs.

  • CNN-based object detectors are commonly organized as two-stage or one-stage systems for recognizing and localizing objects across scales.
  • Image pyramid methods detect instances by scaling images, whereas feature pyramid methods fuse representations across different scales.
  • Alternative architectures such as FishNet and SpineNet use encoder-decoder or cross-scale designs to fuse multi-scale features instead of relying solely on conventional detector structure.
  • GiraffeDet’s overview uses a body, heavy neck, and head: the neck refines and fuses semantic and spatial features before prediction.
  • The conventional backbone-neck-head architecture remains widely used and has been shown effective in detection tasks.

3 THE GIRAFFEDET

GiraffeDet replaces conventional heavy backbones with a lightweight S2D-chain and concentrates computation in a generalized FPN. Its skip-layer and cross-scale connections support dense information exchange across feature levels while enabling scalable detector variants.

  • Framework: GiraffeDet combines a lightweight space-to-depth chain, generalized FPN, and prediction networks in a one-stage detection framework.The S2D-chain performs lightweight feature down-sampling, while the generalized FPN provides multi-scale fusion.
  • Lightweight Space-to-Depth Chain: The S2D-chain uses initial 3x3 convolutions, stacked S2D blocks, and 1x1 convolutions to down-sample features without additional parameters in the S2D layer.S2D moves spatial information into the depth dimension by uniformly sampling and reorganizing features.
  • Design Rationale: Controlled experiments show that the neck is more crucial than conventional backbones for object detection at the same FLOPs.This finding motivates allocating computation to a deep, wide neck rather than relying on increasingly heavy classification backbones.
  • Generalized-FPN: GFPN extends feature-pyramid designs with queen-fusion cross-scale connections and skip-layer connections alongside conventional top-down and bottom-up pathways.FPN uses top-down fusion, PANet adds bottom-up aggregation, and GFPN combines same- and neighboring-level exchanges with skip connections.
  • Skip-layer Connection: Dense-link concatenates all preceding layers, whereas log2n-link connects at most log2l + 1 preceding layers to reduce complexity and support deeper networks.The log2n-link has O(l · log2l) time complexity instead of O(l2), while increasing inter-layer distance only from 1 to 1+log2l.
  • GiraffeDet Family: GiraffeDet scales primarily the GFPN depth and width to cover different resource constraints while exchanging high-level semantic and low-level spatial information.The heavy neck and lightweight backbone are intended to balance accuracy and efficiency under large-scale variation.

4 EXPERIMENTS

Experiments evaluate GiraffeDet on COCO against comparable detectors and analyze its connections, depth, width, backbones, and deformable convolutions. Results emphasize scale-sensitive detection and the efficiency of a lightweight backbone paired with a generalized FPN.

  • Quantitative evaluation: GiraffeDet achieves the best performance across each pixel-scale range in Figure 6, including a 5.7% mAP advantage over RetinaNet for 0–32 pixels.The largest advantage over other methods occurs in the 192–256 pixel range.
  • Quantitative evaluation: GiraffeDet achieves better performance than previous detectors at each comparable FLOPs level, with stronger results on small and large objects.The comparison uses COCO results grouped by similar computational cost.
  • Connection analysis: Log2n connections outperform dense and no-skip alternatives, indicating more effective transmission from early to later nodes with less redundant information.The connection analysis compares GFPN-dense, GFPN-log2n, and GFPN without skip-layer connections.
  • Depth and width: GFPN outperforms stacked FPN, PANet, and BiFPN across depth and width settings, while a smaller “11” depth and “221” width design supports efficient multi-scale detection.The comparison is conducted at the same FLOPs level, with S2D-chain used as the backbone.
  • Backbone effects: The S2D-chain combined with GFPN outperforms alternative backbones at the same FLOPs level, while performance can decrease as conventional backbone size grows.The authors associate larger backbones with a remaining domain-shift problem.
  • Deformable convolution: Deformable convolution significantly improves GiraffeDet performance, allowing GiraffeDet-D11 with DCN to outperform GiraffeDet-D16.The experiments also examine shallow GFPN configurations with stronger DCN backbones under acceptable inference time.

5 CONCLUSION

The paper proposes GiraffeDet as a heavy-neck framework for large-scale variation, combining a lightweight spatial-to-depth backbone with a generalized FPN. The framework targets information exchange between semantic and spatial features and reports higher accuracy and efficiency, especially for small and large objects.

  • Conclusion: GiraffeDet addresses large-scale variation with a lightweight spatial-to-depth backbone and a heavy generalized-FPN neck.The backbone extracts multi-scale image features, while the neck supports high-level and low-level information exchange.
  • Conclusion: The GiraffeDet family achieves higher accuracy and better efficiency, particularly when detecting small and large object instances.

A ARCHITECTURE DETAILS

The architecture details identify the space-to-depth chain and compare the stacked BiFPN with GFPN-D11. These materials define the principal architectural components used by GiraffeDet.

  • Space-to-depth chain: The space-to-depth chain architecture uses convolutional layers, SiLU activations, S2D layers, and repeated S2D blocks.In the figure notation, “Bx” denotes the number of S2D blocks.
  • Neck architecture: GFPN-D11 is presented in an architecture comparison with stacked BiFPN.

B MORE IMPLEMENTATION DETAILS

The implementation-details appendix lists the hyperparameters used in the experiments.

  • Hyperparameters: Table 8 provides the list of hyperparameters used in the experiments.

C.1 FEATURE FUSION METHODS

The ablation compares concatenation and summation feature fusion, finding that concatenation achieves better performance at matched FLOPs despite higher computational cost.

  • Figure 10 compares concatenation, summation, and summation-at-matched-FLOPs fusion models.The matched-FLOPs summation model is designed to separate fusion-style effects from computational budget.
  • Concatenation-based fusion achieves better performance than summation-based fusion at the same FLOPs level.The comparison indicates that summation's lower computational cost comes with substantially lower performance.
  • The authors judge the performance loss from summation fusion not worth its reduction in FLOPs.

C.2 INFERENCE TIME

Inference-time experiments compare ResNet + FPN with S2D-chain + GFPN under matched FLOPs. GiraffeDet provides improvements with acceptable inference time, and its performance declines more slowly as FPS increases.

  • The inference-time comparison evaluates ResNet + FPN and S2D-chain + GFPN at the same FLOPs level.Figure 11 uses orange for S2D-chain + GFPN and purple for ResNet + FPN; Table 9 reports the corresponding comparison.
  • GiraffeDet achieves significant improvements with acceptable inference time against the ResNet-FPN-GFocalV2 baseline.The authors attribute possible speed effects to GPU friendliness of ResNet inference and memory I/O sensitivity in concatenation-based GFPN fusion.
  • GiraffeDet's performance decreases more slowly than the standard model as FPS increases.
  • GFPN outperforms BiFPN under the same FLOPs level when using a standard backbone.The experiment also reports that increasing GFPN depth improves performance with a ResNet-18 backbone.

D ADDITIONAL QUALITATIVE RESULTS

Qualitative COCO comparisons show that all evaluated methods detect objects, while GiraffeDet detects more instances, particularly small objects.

  • Figure 12 provides qualitative object-detection comparisons on the COCO dataset.
  • All evaluated methods detect object instances in the qualitative examples.
  • GiraffeDet detects more instances than other SOTA methods, especially small object instances.The authors connect this result to the effectiveness of their FPN on datasets with large object-scale variation.
Loading 2202.04256v2…