Source-linked AI summary
Large Selective Kernel Network for Remote Sensing Object Detection
Yuxuan Li, Qibin Hou, Zhaohui Zheng, Ming-Ming Cheng, Jian Yang, Xiang Li
TL;DR
Remote sensing detectors have largely emphasized oriented bounding boxes while overlooking aerial-image priors: objects may require broad context, and the needed range varies by category. LSKNet uses input-dependent spatial selection over decomposed large kernels to adapt its receptive field. The model achieves state-of-the-art performance across HRSC2016, DOTA-v1.0, and FAIR1M-v1.0.
Problem
Remote sensing detection lacks sufficient use of aerial-image priors, although accurate recognition can require broad and category-dependent contextual information.
Method
LSKNet dynamically adapts the feature-extraction backbone’s receptive field by spatially selecting and merging features from sequential large depth-wise kernels.
Results
State-of-the-art performance is reported across three benchmarks: 98.46% mAP on HRSC2016, 81.64% mAP on DOTA-v1.0, and 47.87% mAP on FAIR1M-v1.0.
Takeaways & Limitations
LSKNet’s behavior aligns with the two identified contextual priors, supporting the effectiveness of large and selective kernels for remote sensing detection.
Abstract
from arXiv · showhide
Recent research on remote sensing object detection has largely focused on improving the representation of oriented bounding boxes but has overlooked the unique prior knowledge presented in remote sensing scenarios. Such prior knowledge can be useful because tiny remote sensing objects may be mistakenly detected without referencing a sufficiently long-range context, and the long-range context required by different types of objects can vary. In this paper, we take these priors into account and propose the Large Selective Kernel Network (LSKNet). LSKNet can dynamically adjust its large spatial receptive field to better model the ranging context of various objects in remote sensing scenarios. To the best of our knowledge, this is the first time that large and selective kernel mechanisms have been explored in the field of remote sensing object detection. Without bells and whistles, LSKNet sets new state-of-the-art scores on standard benchmarks, i.e., HRSC2016 (98.46\% mAP), DOTA-v1.0 (81.85\% mAP) and FAIR1M-v1.0 (47.87\% mAP). Based on a similar technique, we rank 2nd place in 2022 the Greater Bay Area International Algorithm Competition. Code is available at https://github.com/zcablii/Large-Selective-Kernel-Network.
1. Introduction
Remote sensing detection has emphasized oriented bounding-box representation while underusing aerial-image priors: small objects often require broad context, and different categories require different context ranges. LSKNet addresses this by dynamically selecting large receptive fields and reports state-of-the-art benchmark performance.
- Remote sensing object detection identifies objects in aerial images, where oriented bounding boxes fit object orientations more accurately than horizontal boxes.
- Aerial-image detection often requires contextual information because small objects can be difficult to identify from appearance alone.
- Limited context can cause incorrect classifications, including confusing junctions with intersections or misclassifying intersections obscured by trees.
- Required context varies by object type: soccer-ball fields need less additional context, whereas roundabouts and tree-covered intersections need larger receptive fields.
- LSKNet dynamically adjusts the backbone receptive field through spatially selective weighting of features processed by sequential large depth-wise kernels.
- 98.46% mAP is reported for LSKNet on HRSC2016, alongside 81.64% on DOTA-v1.0 and 47.87% on FAIR1M-v1.0.
2. Related Work
Related work includes oriented detection frameworks, large-kernel convolution, and attention or kernel-selection mechanisms. LSKNet combines decomposed large kernels with spatial rather than channel-wise adaptive aggregation for remote sensing.
- Remote Sensing Object Detection Framework: High-performance remote sensing detectors include two-stage RCNN variants and one-stage frameworks that directly classify and regress oriented bounding boxes.
- Remote Sensing Object Detection Framework: Existing rotation-variance methods generally do not account for strong prior information in aerial images, which LSKNet targets without modifying the detection framework.
- Large Kernel Networks: Large-kernel networks use increasingly broad convolutional receptive fields, with examples ranging from ConvNeXt’s 7×7 kernels to SLaK’s 51×51 kernels.
- Large Kernel Networks: Large-kernel convolutions had been studied in general object recognition but lacked dedicated investigation for remote sensing detection.
- Attention/Selective Mechanism: Attention mechanisms reweight feature channels, spatial locations, or both to enhance neural representations and contextual modeling.
- Attention/Selective Mechanism: Unlike SKNet’s channel-wise selection, LSKNet uses decomposed large kernels and adaptively aggregates them across spatial dimensions.
3. Methods
LSKNet uses repeated blocks containing large-kernel selection and feed-forward sub-blocks. Its LSK module decomposes large convolutions into multiple receptive fields, spatially selects among them, and fuses the result with the input.
- LSKNet Architecture: LSKNet is built from repeated blocks containing Large Kernel Selection and Feed-forward Network residual sub-blocks.The LSK module is embedded within the Large Kernel Selection sub-block.
- Large Kernel Convolutions: A large kernel is decomposed into sequential depth-wise convolutions with growing kernel sizes and increasing dilation rates.This produces multiple long-range contexts while expanding the receptive field quickly.
- Large Kernel Convolutions: The decomposition yields multiple large receptive fields for later selection and is more efficient than directly applying a single larger kernel.Two or three decomposed convolutions provide theoretical receptive fields of 23 and 29, respectively.
- Large Kernel Convolutions: Each decomposed feature is processed by a 1×1 convolution before a selection mechanism dynamically chooses kernels for different objects.The 1×1 convolution enables channel mixing for each spatial feature vector.
- Spatial Kernel Selection: Spatial selection concatenates multi-range features, derives pooled spatial descriptors, generates attention maps, and weights and fuses the kernel outputs.Average and maximum channel-based pooling provide the spatial descriptors used to produce the selection maps.
- Spatial Kernel Selection: The LSK module returns an attention feature multiplied element-wise with the input feature, expressed as Y = X ⋅S.This output adaptively collects the large receptive field corresponding to each object.
4. Experiments
Experiments show that LSKNet benefits from decomposed large kernels and spatially selective receptive fields, achieving strong results across three remote sensing benchmarks. Analysis further indicates that receptive-field selection varies by object category and network depth.
- Ablation Study: Two depth-wise large kernels provide the best speed–accuracy trade-off at a fixed theoretical receptive field of 29.The ablation evaluates inference FPS and mAP for different decomposition counts.
- Ablation Study: A receptive field of approximately 23 is most effective, while excessively small or large fields hinder LSKNet performance.Spatial selection also outperforms channel attention for remote sensing object detection.
- Backbone Comparisons: Under similar model size and complexity budgets, LSKNet outperforms six other large-kernel or selective-attention backbones on DOTA-v1.0.This comparison evaluates alternative high-performance backbone designs.
- Main Results: 98.46% mAP is achieved by LSKNet-S on HRSC2016 under the VOC 2012 metric, outperforming the 12 compared methods.The corresponding VOC 2007 mAP is 90.65%.
- Main Results: 81.64% mAP on DOTA-v1.0 and 47.87% mAP on FAIR1M-v1.0 are achieved by LSKNet-S, with 18.1 FPS reported on 1024x1024 images.LSKNet-T achieves 81.37% mAP on DOTA-v1.0 and 46.93% mAP on FAIR1M-v1.0.
- Analysis: Bridge requires more additional context than other categories, whereas Soccer-ball-field requires minimal context because of distinctive court boundary lines.The normalized receptive-field analysis links these differences to category-specific visual context.
- Analysis: Larger kernels are selected more often for Bridge than Roundabout and Soccer-ball-field, while shallow layers generally use larger kernels than higher layers.This pattern is consistent with category-dependent context needs and progressive semantic processing across depth.
5. Conclusion
LSKNet is designed for remote sensing object detection by adapting its large spatial receptive field to varying contextual needs, achieving state-of-the-art benchmark performance.
- LSKNet adapts its large spatial receptive field to model the varying contextual nuances of different remote sensing object types.
- Extensive experiments show that the lightweight LSKNet achieves state-of-the-art performance on competitive remote sensing benchmarks.
A.1. LSKNet Block
The LSKNet block is a repeated ConvNeXt-inspired backbone unit composed of large-kernel selection and feed-forward sub-blocks, with dynamic receptive-field adjustment.
- A.1. LSKNet Block: Each LSKNet block contains two residual sub-blocks: Large Kernel Selection and Feed-forward Network.
- A.1. LSKNet Block: The Large Kernel Selection sub-block dynamically adjusts the network’s receptive field as needed.
- A.1. LSKNet Block: Figure 8 illustrates an LSKNet block as a repeated unit in the backbone network.
A.2. 2022 the Greater Bay Area International Algorithm Competition
The competition used FAIR1M-v2.0 data for rotated object detection, and the authors further enhanced their detector with model ensemble strategies.
- A.2. 2022 the Greater Bay Area International Algorithm Competition: The competition required Jittor-based remote sensing object detection models producing rotated bounding boxes and object types.
- A.2. 2022 the Greater Bay Area International Algorithm Competition: The competition dataset contained 5,000 training images, 576 preliminary test images, and 577 final test images.
- A.2. 2022 the Greater Bay Area International Algorithm Competition: The full competition scoreboard is available through the cited CVMART race page.
- A.2. 2022 the Greater Bay Area International Algorithm Competition: The authors used model ensemble strategies to improve performance beyond their single detection model.
- A.2. 2022 the Greater Bay Area International Algorithm Competition: Model output ensemble merges detector outputs using non-maximal suppression, whereas model weight ensemble combines model weights through weighted averaging.
A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version)
LSKNet differs from SKNet through decomposed large-kernel selection and spatial rather than channel-wise adaptive aggregation; the section also presents LSKNet results and module illustrations.
- A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version): LSKNet’s selective mechanism uses a sequence of decomposed large kernels, unlike most existing attention-based approaches.
- A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version): LSKNet adaptively aggregates information across large kernels in the spatial dimension rather than the channel dimension.
- A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version): Figure 9 shows examples of FAIR1M-v2.0 test results produced with LSKNet.
- A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version): Figure 16 conceptually illustrates the LSK module with the proposed spatial selection in LSKNet.
- A.3. SKNet v.s. LSKNet v.s. LSKNet-CS (channel selection version): Figure 10 provides a conceptual comparison of SKNet, LSKNet, and LSKNet-CS module architectures.
A.4. Experiment Implementation Details
The experiments follow mainstream dataset-processing procedures, using multi-scale patch-based processing for DOTA-v1.0 and FAIR1M-v1.0 and fixed long-side rescaling for HRSC2016.
- Multi-scale processing: DOTA-v1.0 and FAIR1M-v1.0 use multi-scale training and testing at scales 0.5, 1.0, and 1.5.Images at each scale are cropped into 1024×1024 sub-images.
- Multi-scale processing: Each scaled DOTA-v1.0 and FAIR1M-v1.0 image is cropped with 500-pixel patch overlap.The crop size is 1024×1024 pixels.
- HRSC2016 processing: HRSC2016 images are rescaled by setting the longer side to 800 pixels while preserving aspect ratio.
A.5. Spatial Activation Visualisations
Spatial activation visualisations show that different remote-sensing categories require different amounts of contextual information, with Bridge needing more additional context than other categories and Plane and Court needing little.
- Category-dependent context: Bridge requires more additional contextual information than other DOTA-v1.0 categories.Its feature similarity to roads makes surrounding water context useful for determining whether the bridge is enveloped by water.
- Visualisation setup: Figure 11 presents receptive-field activation maps from LSKNet’s Eq. (8) spatial activation for additional DOTA-v1.0 object categories.Categories are ordered by the ratio of expected selective receptive-field area to ground-truth bounding-box area.
- Category-dependent context: Ships and vehicles require a large receptive field to reference whether their surrounding context is sea or land.
- Category-dependent context: Plane and Court categories require minimal contextual information because of distinctive shapes, textures, and court boundary lines.Soccer-ball-field is given as an example of a Court category.
- FAIR1M-v1.0 comparison: Table 13 compares fine-grained category results between LSKNet and state-of-the-art methods on FAIR1M-v1.0.The compared LSKNet backbones and R3Det, S2ANet, and Oriented RCNN backbones are pretrained on ImageNet for 300 epochs.