Source-linked AI summary
FBRT-YOLO: Faster and Better for Real-Time Aerial Image Detection
Yao Xiao, Tingfa Xu, Yu Xin, Jianan Li
TL;DR
High-resolution aerial detection must identify small or obscured objects while meeting real-time constraints on resource-limited flight devices. FBRT-YOLO addresses this with FCM for spatial-semantic integration and MKP for multi-scale perception, and reports a balanced accuracy-efficiency trade-off across three aerial datasets.
Problem
Aerial detectors must handle small or background-obscured objects while balancing accuracy and real-time efficiency on resource-constrained flight equipment.
Method
FBRT-YOLO combines FCM, which integrates shallow spatial location information with deeper semantic features, and MKP, which uses diverse kernels for multi-scale target perception.
Results
FBRT-YOLO achieves a balanced accuracy-efficiency trade-off across VisDrone, UAVDT, and AI-TOD and outperforms prior real-time YOLO-series detectors across model scales.
Takeaways & Limitations
The proposed detector targets real-time aerial detection by improving small-object representation while reducing computational and structural redundancy.
Abstract
from arXiv · showhide
Embedded flight devices with visual capabilities have become essential for a wide range of applications. In aerial image detection, while many existing methods have partially addressed the issue of small target detection, challenges remain in optimizing small target detection and balancing detection accuracy with efficiency. These issues are key obstacles to the advancement of real-time aerial image detection. In this paper, we propose a new family of real-time detectors for aerial image detection, named FBRT-YOLO, to address the imbalance between detection accuracy and efficiency. Our method comprises two lightweight modules: Feature Complementary Mapping Module (FCM) and Multi-Kernel Perception Unit(MKP), designed to enhance object perception for small targets in aerial images. FCM focuses on alleviating the problem of information imbalance caused by the loss of small target information in deep networks. It aims to integrate spatial positional information of targets more deeply into the network,better aligning with semantic information in the deeper layers to improve the localization of small targets. We introduce MKP, which leverages convolutions with kernels of different sizes to enhance the relationships between targets of various scales and improve the perception of targets at different scales. Extensive experimental results on three major aerial image datasets, including Visdrone, UAVDT, and AI-TOD,demonstrate that FBRT-YOLO outperforms various real-time detectors in terms of performance and speed.
Introduction
FBRT-YOLO addresses small-object detection and real-time efficiency in high-resolution aerial images through FCM and MKP, aiming to balance accuracy with computational cost.
- Introduction: High-resolution aerial detection remains difficult because targets may be small or background-obscured, while flight devices impose real-time computational constraints.Increasing image resolution can improve small-object detection but adds computational burden.
- Introduction: Deep semantic features and shallow spatial features remain mismatched because backbone networks struggle to preserve and integrate shallow information.This mismatch can produce feature alignment problems for small-target localization.
- Introduction: FCM transfers shallow spatial location information into deeper layers and guides complementary learning between spatial and channel information.The module is embedded across backbone stages to improve spatial-semantic consistency and small-object localization.
- Introduction: MKP uses convolutional kernels of different sizes and spatial point convolutions to capture target features across multiple scales.It is designed for objects represented by only a few pixels and also simplifies the network structure.
- Introduction: FBRT-YOLO uses fewer training parameters and reduced computational load than YOLOv8 while targeting a balanced accuracy-efficiency trade-off across aerial benchmarks.The comparison is reported on VisDrone, UAVDT, and AI-TOD across different model scales.
Related Work
Prior real-time detectors and small-object methods often incur efficiency costs or retain feature-imbalance problems in aerial imagery. FBRT-YOLO instead combines spatial-semantic integration, multi-scale perception, and redundancy reduction.
- Related Work: Real-time detectors developed for low-resolution natural images perform unsatisfactorily on high-resolution aerial images and may contain structural redundancy.Aerial platforms emphasize model size, memory, and computational efficiency.
- Related Work: Small-object solutions such as dataset augmentation, high-resolution inputs, clustering, and density-map cropping generally add model complexity or slow detection.These approaches target limited small-object representation through additional data, resolution, or spatial-context processing.
- Related Work: Feature Pyramid Network and related multi-scale methods combine deep semantic features with shallow spatial information, but backbone integration and preservation of shallow information remain challenging.FBRT-YOLO focuses on performing this integration within the backbone rather than relying on an image pyramid that consumes substantial resources.
Method
FBRT-YOLO combines lightweight feature fusion and multi-scale perception modules to improve small-object representation while reducing redundant computation for real-time aerial detection.
- Feature Complementary Mapping Module: FBRT-YOLO uses FCM to integrate spatial positional information into deeper semantic features, improving small-object representation.FCM uses split, transformation, complementary mapping, and feature aggregation to address spatial-semantic mismatch.
- Feature Complementary Mapping Module: FCM splits input channels into αC and (1 − α)C branches, with the split ratio controlling low-level spatial-information acquisition.The paper states that appropriately enhancing low-level information can improve performance as the network deepens.
- Feature Complementary Mapping Module: The FCM branches separately extract richer channel information and preserve more shallow spatial-location information before complementary fusion.Standard 3×3 convolution produces XC, while point-wise convolution produces XS with more original spatial information.
- Feature Complementary Mapping Module: Channel and spatial interactions generate weights that are mapped across branches and concatenated into features containing dual spatial-semantic mappings.Depthwise convolution, global average pooling, sigmoid weighting, spatial convolution, BN, and sigmoid are used in these interactions.
- Multi-Kernel Perception Unit: MKP uses convolutional kernels of different sizes to capture target information across scales and strengthen contextual and small-target features.The unit is designed for small objects with limited effective information and establishes spatial relationships across scales.
- Targeted Reduction of Redundancy-Driven Network Design: The network decouples spatial downsampling and channel expansion by applying group convolutions first and point convolutions second.This design addresses channel interference associated with channel expansion before depthwise-convolution sampling and reduces structural redundancy.
Implementation Details
Experiments use three aerial-image benchmarks with standardized GPU and training settings, while inference speed is measured separately on an RTX 3080.
- Experiments cover VisDrone, UAVDT, and AI-TOD aerial-image detection benchmarks.
- Training uses 300 epochs of SGD with momentum 0.937, weight decay 0.0005, batch size 4, and initial learning rate 0.01.
- Experiments run on an NVIDIA GeForce RTX 4090 GPU, except inference speed testing on a single RTX 3080 GPU.
Results on Visdrone Dataset
On VisDrone, FBRT-YOLO is reported to improve the accuracy–efficiency trade-off over real-time detectors, while heatmaps show stronger focus on small, densely packed targets.
- 72% and 74% fewer parameters for FBRT-YOLO-N/S than YOLOv8-N/S accompany AP improvements of 0.6% and 2.3%, respectively.
- FBRT-YOLO-M reduces GFLOPs by 26% versus YOLOv8-M and 23% versus YOLOv9-M, with AP gains of 1.3% and 1.2%, respectively.
- FBRT-YOLO-X uses 66% fewer parameters than YOLOv8-X and 23% fewer than YOLOv10-X, while improving AP by 1.2% and 1.4%, respectively.
- Table 2 presents comparison results between FBRT-YOLO and other state-of-the-art methods on VisDrone.
- Heatmaps indicate that FBRT-YOLO focuses more on small and densely packed targets than the baseline model.
Results on UAVDT Dataset
On UAVDT, FBRT-YOLO is reported to surpass existing methods and achieve an AP of 18.4%, while qualitative results address detection in complex backgrounds.
- 18.4% AP is reported for FBRT-YOLO on UAVDT, surpassing methods including GLSAN and CEASC.
- Qualitative visualizations report improved detection performance in complex backgrounds through propagation of target spatial information across network layers.
Results on AI-TOD Dataset
On AI-TOD, which contains many small objects, FBRT-YOLO improves detection metrics while reducing parameters and computation relative to the baseline.
- 74% fewer parameters and 20% fewer GFLOPs accompany AP50 and AP increases of 2.2% and 1.1%, respectively, versus the baseline.
- AI-TOD provides an evaluation setting with a significant proportion of small objects.
Ablation Study
Ablation experiments use YOLOv8-S as the baseline and show that model simplification, FCM, and MKP each contribute to FBRT-YOLO’s design. The experiments also examine mapping configurations, FCM partition ratios, and MKP kernel sizes.
- Experimental Setup: YOLOv8-S serves as the baseline for ablation experiments on VisDrone.The experiments are designed to validate the effectiveness of FBRT-YOLO’s core modules.
- Overall Ablation: Reducing baseline redundancy cuts parameters by 18% and computational load by 11%, with a slight accuracy decrease.The optimized network also converges faster during training than the baseline.
- FCM Ablation: Adding FCM increases AP50 by 1.4% while further reducing computational resources.FCM incorporates spatial positional information into deeper backbone layers.
- MKP Ablation: Replacing final-layer downsampling with MKP increases AP by 1.6% by supporting detection across multiple target scales.MKP uses varied kernel sizes to capture multi-scale target features.
- Mapping Configuration: Combining channel and spatial mapping performs better than using either mapping alone or no mapping relationship.The optimal configuration improves AP50 by 2 compared with the model without mapping relationships.
- FCM Partition Ratio: Increasing the spatial-feature proportion during downsampling improves experimental effects, consistent with retaining deeper spatial location information.The reported interpretation connects this retained spatial information with target localization and feature matching.
- Kernel-Size Study: Smaller kernels limit receptive fields, whereas larger kernels introduce background noise; varied kernels capture multi-scale target features.The MKP experiments motivate combining kernel sizes rather than relying on a single receptive-field scale.
Conclusion
FBRT-YOLO combines lightweight FCM and MKP modules with reduced detector redundancy for real-time aerial image detection. Results on VisDrone, UAVDT, and AI-DOT support a balanced trade-off between accuracy and efficiency.
- Contribution: FBRT-YOLO introduces FCM to fuse semantic information with precise spatial location details.The module targets improved representation alignment for aerial image detection.
- Contribution: MKP enhances multi-scale target perception and feature capture across varying target scales.It complements FCM by addressing scale variation in aerial targets.
- Efficiency: Reducing redundancy in conventional detectors further accelerates the network for aerial image detection.This design supports the paper’s emphasis on real-time deployment.
- Conclusion: Experiments on VisDrone, UAVDT, and AI-DOT demonstrate a balanced trade-off between aerial detection accuracy and efficiency.The conclusion presents this balance as the overall outcome of FBRT-YOLO.