PaperlayerView exact revision on arXiv

Source-linked AI summary

Hyper-YOLO: When Visual Object Detection Meets Hypergraph Computation

Yifan Feng, Jiangang Huang, Shaoyi Du, Shihui Ying, Jun-Hai Yong, Yipeng Li, Guiguang Ding, Rongrong Ji, Yue Gao

arXiv:2408.04804v2cs.CV

TL;DR

YOLO necks have limited cross-level, cross-position, and high-order feature integration. Hyper-YOLO addresses this with HGC-SCS, MANet, and HyperC2Net, and reports stronger COCO performance, including gains for smaller models.

  • Problem

    Existing YOLO necks do not sufficiently integrate cross-level and cross-position features or exploit high-order correlations among visual features.

  • Method

    Hyper-YOLO transposes backbone features into semantic space, constructs a hypergraph for high-order message propagation, and uses MANet and HyperC2Net in its backbone and neck.

  • Results

    Hyper-YOLO outperforms other models across four scales, achieving 41.8%, 48.0%, 52.0%, and 53.8% APval at the N, S, M, and L scales.

  • Takeaways & Limitations

    Hyper-YOLO combines semantic and structural high-order information with cross-level and cross-position message propagation for object detection.

Abstract

from arXiv · show

We introduce Hyper-YOLO, a new object detection method that integrates hypergraph computations to capture the complex high-order correlations among visual features. Traditional YOLO models, while powerful, have limitations in their neck designs that restrict the integration of cross-level features and the exploitation of high-order feature interrelationships. To address these challenges, we propose the Hypergraph Computation Empowered Semantic Collecting and Scattering (HGC-SCS) framework, which transposes visual feature maps into a semantic space and constructs a hypergraph for high-order message propagation. This enables the model to acquire both semantic and structural information, advancing beyond conventional feature-focused learning. Hyper-YOLO incorporates the proposed Mixed Aggregation Network (MANet) in its backbone for enhanced feature extraction and introduces the Hypergraph-Based Cross-Level and Cross-Position Representation Network (HyperC2Net) in its neck. HyperC2Net operates across five scales and breaks free from traditional grid structures, allowing for sophisticated high-order interactions across levels and positions. This synergy of components positions Hyper-YOLO as a state-of-the-art architecture in various scale models, as evidenced by its superior performance on the COCO dataset. Specifically, Hyper-YOLO-N significantly outperforms the advanced YOLOv8-N and YOLOv9-T with 12\% $\text{AP}^{val}$ and 9\% $\text{AP}^{val}$ improvements. The source codes are at ttps://github.com/iMoonLab/Hyper-YOLO.

I. INTRODUCTION

The paper identifies limitations in YOLO necks for cross-level, cross-position, and high-order feature integration, then proposes hypergraph-based components to address them. Hyper-YOLO combines HGC-SCS, HyperC2Net, and MANet, with reported COCO gains across model scales.

  • Motivation: PANet mainly fuses adjacent layers, while gather-distribute designs do not fully support cross-position or high-order feature interactions.These limitations motivate a neck that integrates information across levels, positions, and semantic relationships.
  • Motivation: High-order correlations among features at different scales, positions, and semantic levels are important but commonly underexplored in visual tasks.The paper links these relationships to combining low-level visual features with high-level semantic information for object identification and localization.
  • Proposed approach: HGC-SCS transposes visual feature maps into semantic space and constructs a hypergraph for high-order message propagation.The framework is intended to capture both semantic and complex structural information beyond conventional feature-focused learning.
  • Proposed approach: Hyper-YOLO uses MANet in its backbone and HyperC2Net in its neck to model high-order associations across five feature scales, levels, and positions.HyperC2Net departs from conventional grid-based feature interaction while MANet enriches information flow and feature extraction.
  • Results: 12% APval improvement over YOLOv8-N and 9% APval improvement over YOLOv9-T are reported for Hyper-YOLO-N on COCO.The reported comparison appears in the paper’s contribution summary.

III. HYPERGRAPH COMPUTATION EMPOWERED SEMANTIC COLLECTING AND SCATTERING FRAMEWORK

HGC-SCS collects multi-stage visual features in semantic space, estimates their high-order relationships with a hypergraph, propagates messages, and scatters enhanced information back to feature maps. Hyper-YOLO instantiates this framework through HyperC2Net while retaining a conventional backbone–neck detection organization.

  • Framework: HGC-SCS processes feature maps and high-order structures together rather than learning only from visual features.Its general computer-vision paradigm combines hypergraph construction, hypergraph convolution, and fusion of original and high-order features.
  • Semantic collecting: The framework first collects and fuses backbone outputs into a mixed semantic-space feature bag called Xmixed.This collection step precedes estimation of potential high-order correlations.
  • Hypergraph computation: A hypergraph is constructed in semantic space, and hypergraph computation generates Xhyper containing high-order structural and semantic information.The hypergraph construction estimates correlations among feature points, after which convolution propagates high-order messages.
  • Semantic scattering: The high-order information is scattered back to each input feature map through a feature fusion function, producing enhanced visual maps.The enhanced maps are denoted {X′1, X′2, · · · } in the framework description.
  • HyperC2Net: HyperC2Net integrates five backbone feature sets across levels and positions to produce final neck features at three detection scales.The neck outputs are denoted {N3, N4, N5}, corresponding to small-, medium-, and large-scale detection.

C. Mixed Aggregation Network

MANet augments Hyper-YOLO’s backbone by combining three convolutional variants to enrich feature extraction, gradient flow, and semantic information across five stages.

  • C. Mixed Aggregation Network: MANet synergistically blends three convolutional variants to strengthen feature extraction in Hyper-YOLO’s backbone.The passage identifies a 1 × 1 bypass convolution for channel-wise feature recalibration and describes the combined design as producing richer gradient flow.
  • C. Mixed Aggregation Network: The architecture produces more variegated gradient flow during training, amplifying semantic depth in base features across five pivotal stages.
  • C. Mixed Aggregation Network: MANet fuses and compresses three feature types through concatenation followed by a 1 × 1 convolution, generating Xout with 2c channels.Each X1, X2, …, X4+n has c channels, while Xmid has 2c channels.

D. Hypergraph-Based Cross-Level and Cross-Position Representation Network

HyperC2Net implements HGC-SCS by collecting multi-stage backbone features, constructing a semantic-space hypergraph, and propagating high-order messages across levels and positions.

  • D. Hypergraph-Based Cross-Level and Cross-Position Representation Network: HyperC2Net is proposed as a hypergraph-based neck for comprehensively fusing cross-level and cross-position information from backbone features.
  • D. Hypergraph-Based Cross-Level and Cross-Position Representation Network: The five backbone stages provide feature maps B1, B2, B3, B4, and B5, which are channel-wise concatenated to synthesize cross-level visual features.
  • D. Hypergraph-Based Cross-Level and Cross-Position Representation Network: A spatial-domain hypergraph convolution with a residual connection performs high-order learning on vertex features.The formulation uses neighbor relationships between vertices and hyperedges, with trainable parameter Θ.
  • D. Hypergraph-Based Cross-Level and Cross-Position Representation Network: The matrix formulation uses diagonal degree matrices Dv and De for vertices and hyperedges, respectively.

3) An Instance of HGC-SCS Framework.:

Hyper-YOLO instantiates HGC-SCS through HyperC2Net, combining five-level features and semantic-space hypergraph processing to capture cross-level and cross-position relationships.

  • 3) An Instance of HGC-SCS Framework.: HyperC2Net is a streamlined instantiation of HGC-SCS that combines hypergraph construction and convolution strategies.
  • 3) An Instance of HGC-SCS Framework.: Xmixed concatenates B1 through B5, and Xhyper applies HyperConv using the constructed hypergraph H.
  • 3) An Instance of HGC-SCS Framework.: The fused backbone representation intrinsically contains cross-level information because it combines features from multiple levels.
  • 3) An Instance of HGC-SCS Framework.: Deconstructing grid features into semantic-space points and constructing distance-based hyperedges enables high-order message passing across varying positions.This captures cross-position information and enriches the model’s understanding of the semantic space.
  • 3) An Instance of HGC-SCS Framework.: HyperC2Net directly fuses features across five backbone levels, overcoming PANet’s restriction to immediately contiguous layers.
  • 3) An Instance of HGC-SCS Framework.: HyperC2Net’s representations combine original backbone semantics with potential high-order structural features for object detection.
  • 3) An Instance of HGC-SCS Framework.: The resulting high-order feature processing is presented as advantageous over PANet and gather-distribute necks.

V. EXPERIMENTS

Hyper-YOLO is evaluated on COCO Val2017 against advanced YOLO baselines under standardized training and evaluation conditions. It outperforms competing models across four scales, while reducing parameters relative to Gold-YOLO and showing especially larger gains at smaller scales.

  • Evaluation Setup: COCO Val2017 evaluates Hyper-YOLO against YOLOv5, YOLOv6-3.0, YOLOv7, YOLOv8, Gold-YOLO, and YOLOv9.Train2017 is used for training and Val2017 for validation.
  • Evaluation Setup: All methods are compared without pre-training or self-distillation, using 640 × 640 inputs and standard COCO AP.Speed measurements use FP16 precision and a Tesla T4 GPU with TensorRT 8.6.1.
  • Main Results: 41.8% APval at -N, 48.0% at -S, 52.0% at -M, and 53.8% at -L make Hyper-YOLO the top performer across all four scales.The reported APval improvements over Gold-YOLO are 2.2, 2.6, 2.2, and 2.0, respectively.
  • Main Results: 28%, 31%, 19%, and 25% parameter reductions relative to Gold-YOLO accompany the gains at the -N, -S, -M, and -L scales.The paper attributes this efficiency to HGC-SCS using cross-level and cross-position information more efficiently.
  • Controlled Comparisons: 38.5 APval versus 37.3 and 3.1M parameters versus 3.2M show Hyper-YOLO-T outperforming YOLOv8-N with fewer parameters.A same-backbone comparison also reports Hyper-YOLOv1.1-T and -S gains of 2.0 and 1.2 APval over YOLOv9-T and -S.
  • Scale Effects: 0.9 to 4.5 APval improvement over YOLOv8 as scale decreases from -L to -N indicates larger reported gains for smaller models.The paper links this pattern to high-order correlations and direct information flow across positions and levels.

C. Ablation Studies on Backbone

Ablation studies examine MANet, kernel sizes, and HGC-SCS learning choices using Hyper-YOLO-S. The results support mixed aggregation and high-order correlation learning as effective components under controlled comparisons.

  • Ablation Setup: Hyper-YOLO-S is selected for ablations of the backbone and neck, with experiments focused on MANet and HGC-SCS components.The backbone and neck enhancements are assessed separately and together across the study.
  • On Basic Block of Backbone: MANet outperforms the C2f module across all metrics when both use the same PANet neck.This isolates the basic-block difference in the backbone comparison.
  • On Kernel Size of Different Stages: Kernel-size experiments vary [k2, k3, k4, k5] across MANet stages to study receptive fields and spatial hierarchies.Mixed aggregation begins at the second stage, so the reported configuration starts with k2.
  • High-Order vs. Low-Order Learning in HGC-SCS Framework: High-order learning outperforms graph-based low-order learning and the None configuration in HGC-SCS ablations.The graph baseline connects a central node with neighbors inside an ϵ-ball, while None removes correlation learning.

2) On the Semantic Collecting Phase:

Ablations examine semantic-space input levels, hypergraph distance thresholds, and backbone/neck enhancements across model scales and segmentation. Results favor richer semantic inputs, threshold 8, and combined enhancements.

  • On the Semantic Collecting Phase:: Using more feature-map levels supplies more abundant information to the semantic space for hypergraph computation.The study compares configurations using 3, 4, or 5 input levels.
  • On Hypergraph Construction:: Hypergraph computation improves overall performance over the “None” configuration without hypergraph computation.Performance remains relatively stable for thresholds 7–9, declines at 6 and 10, and the chosen threshold is 8.
  • More Ablation Studies: Backbone and neck enhancements significantly improve the -N, -S, and -M models, with scale-dependent gains.Backbone gains decrease from 2.6 to 1.5, 0.8, and 0.1 across -N, -S, -M, and -L, while neck gains are 1.9, 1.6, 1.0, and 0.8.
  • More Ablation Studies: For small-object detection, HyperYOLO-L with both enhancements increases APs by 1.6.The combined model outperforms the backbone-only enhancement on this measure.
  • Instance Segmentation: On COCO instance segmentation, Hyper-YOLO improves both APbox and APmask over YOLOv8 across all four model variants.APbox gains are 4.7, 3.3, 2.2, and 1.4 AP, while APmask gains are 3.3, 2.3, 1.3, and 0.7 AP for -N through -L.

G. Visualization of High-Order Learning in Object Detection

The paper uses mathematical rationale and visualizations to support high-order message propagation in Hyper-YOLO. It presents the architecture as advancing object detection and extending the framework toward future visual architectures.

  • Visualization of High-Order Learning in Object Detection: The hypergraph-based neck is designed to overcome grid-like neighborhood limitations in traditional neck message propagation.It enables high-order message propagation across feature semantic spaces.
  • Conclusion: Hyper-YOLO integrates hypergraph computations with YOLO to harness high-order correlations in visual data.The conclusion frames this integration as the central architectural contribution.
  • Conclusion: The paper reports that Hyper-YOLO advances state-of-the-art object detection by addressing cross-level integration and high-order relationship limitations in traditional necks.The conclusion presents this as a benchmark for future object-detection research.

APPENDIX A IMPLEMENTAL DETAILS OF HYPER-YOLO

Hyper-YOLO retains YOLOv8’s overall architecture while replacing C2f with MANet in the backbone and adding hypergraph-specific neck configuration and scaling details.

  • IMPLEMENTAL DETAILS OF HYPER-YOLO: Hyper-YOLO-N, -S, -M, and -L use YOLOv8-like architectures and loss functions with MANet and HyperC2Net added.An efficient decoupled head is used for object detection.
  • Backbone: The backbone replaces YOLOv8’s C2f modules with MANet while preserving the same layer structure [6] and channel counts.The module swap is the stated backbone change.
  • Backbone: MANet uses depthwise separable convolutions and expands a 2c input to a 4c output.The passage equates 2c with c_out.
  • Backbone: Depth multipliers are 1/3, 1/3, 2/3, and 1 for Hyper-YOLO-N, -S, -M, and -L, respectively.The multipliers scale the stage depths based on model size.
  • Neck: The neck configuration defines c_in and c_out as HyperConv input and output channels and ε as the hypergraph distance threshold.These parameters specify the neck’s channel interfaces and graph-construction threshold.

B. Neck

HyperC2Net implements HGC-SCS in the neck to fuse cross-level and cross-position information through hypergraph-based high-order correlations. Visual results report improved recognition, class inference, categorization, and boundary delineation.

  • Neck: HyperC2Net replaces the YOLOv8 neck and encapsulates high-order correlations within feature semantic space.It is presented as the neck realization of the HGC-SCS framework.
  • Neck: HyperC2Net fuses cross-level and cross-position information by modeling interdependencies among feature points across layers and positions.The resulting representation is intended to capture subtle image nuances.
  • Visualizations: Visualizations cover both object detection and instance segmentation tasks.The object-detection and segmentation examples are shown in figures S2 and S3, respectively.
  • Object Detection: Object-detection visualizations report superior recognition and a degree of class inference ability from the hypergraph-based neck.One example infers that additional detected birds belong to the same class as another bird.
  • Instance Segmentation: Segmentation visualizations report improved categorization and boundary delineation compared with YOLOv8, including complex scenes and inaccurate ground-truth annotation.The reported examples contain no missed cookies.

APPENDIX C TRAINING DETAILS OF HYPER-YOLO

Hyper-YOLO training used a standardized multi-GPU protocol across model scales, with scale-specific augmentation and hypergraph-threshold settings. Speed tests used ONNX-to-TensorRT conversion at two batch sizes, with a custom distance function to address torch.cdist conversion errors.

  • Training protocol: All Hyper-YOLO-N, -S, -M, and -L variants were trained from scratch for 500 epochs on 8 NVIDIA GeForce RTX 4090 GPUs.Each GPU used a uniform batch size of 20, and training did not use ImageNet pre-training.
  • Training protocol: Learning rate, augmentation, and hypergraph-threshold settings were adjusted for different model scales, with progressively stronger augmentation from N and S to M and L.M and L used moderate and high augmentation, respectively, and shared a close-mosaic setting of 20 epochs.
  • Training protocol: The hypergraph threshold depended on batch size and model scale, decreasing relatively with larger single-GPU batches and increasing for larger models.The reported settings assume a batch size of 20 per GPU.
  • Speed benchmarking: Hyper-YOLO required replacing torch.cdist with a custom feature-distance function during testing because ONNX conversion produced large tensors and batch-32 errors.A backbone-enhanced variant was also tested for speed.
  • Speed benchmarking: Speed evaluation converted models to ONNX and then TensorRT, testing batch sizes 1 and 32 with a fixed 640 × 640 input.The environment used fixed versions of Python, PyTorch, CUDA, cuDNN, TensorRT, and ONNX.
Loading 2408.04804v2…