Source-linked AI summary
Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network
Xue Yang, Hao Sun, Xian Sun, Menglong Yan, Zhi Guo, Kun Fu
TL;DR
Remote-sensing ship detection must handle complex scenes, redundant regions, dense arrangements, and direction prediction. The paper proposes a multitask rotational-region CNN combining DFPN, adaptive ROI Align, rotational regression, prow-direction prediction, and R-NMS. Experiments on DOTA and SRSS report competitive rotation-detection performance.
Problem
Remote-sensing ship detection is challenged by complex scenarios, redundant detection regions, dense ship arrangements, and the need to determine berthing or sailing direction.
Method
The paper builds an end-to-end rotational-region detector combining DFPN, adaptive ROI Align, rotational bounding-box regression, prow-direction prediction, and R-NMS.
Results
Experiments on DOTA and SRSS report competitive performance for rotation-region detection.
Takeaways & Limitations
The framework is presented as able to handle complex scenarios, detect dense objects, reduce redundant regions, and recover ship berthing or sailing direction.
Abstract
from arXiv · showhide
Ship detection is of great importance and full of challenges in the field of remote sensing. The complexity of application scenarios, the redundancy of detection region, and the difficulty of dense ship detection are all the main obstacles that limit the successful operation of traditional methods in ship detection. In this paper, we propose a brand new detection model based on multitask rotational region convolutional neural network to solve the problems above. This model is mainly consist of five consecutive parts: Dense Feature Pyramid Network (DFPN), adaptive region of interest (ROI) Align, rotational bounding box regression, prow direction prediction and rotational nonmaximum suppression (R-NMS). First of all, the low-level location information and high-level semantic information are fully utilized through multiscale feature networks. Then, we design Adaptive ROI Align to obtain high quality proposals which remain complete spatial and semantic information. Unlike most previous approaches, the prediction obtained by our method is the minimum bounding rectangle of the object with less redundant regions. Therefore, rotational region detection framework is more suitable to detect the dense object than traditional detection model. Additionally, we can find the berthing and sailing direction of ship through prediction. A detailed evaluation based on SRSS for rotation detection shows that our detection method has a competitive performance.
I. INTRODUCTION
Remote-sensing ship detection is challenged by complex scenes, redundant horizontal regions, dense arrangements, and the need to predict berthing or sailing direction. The paper proposes an end-to-end rotational-region framework addressing these issues with multiscale features, adaptive proposals, direction prediction, and rotational suppression.
- Remote-sensing ship detection supports defense, port management, cargo transportation, and maritime rescue, while ship aspect ratios make detection difficult.
- Horizontal detection produces redundant and overlapping regions for inclined, densely arranged ships, allowing noise to interfere with features and NMS to discard objects.
- The proposed framework combines DFPN, adaptive ROI Align, rotational bounding-box regression, prow-direction prediction, and R-NMS.
- Adaptive ROI Align reduces redundant proposal noise while retaining semantic and spatial information, and direction prediction identifies berthing and sailing direction.
- The rotational-region CNN is presented as more suitable for dense ship detection than predominantly horizontal-region methods.
III. PROPOSED METHOD
The proposed detector uses a densely connected multiscale feature pyramid and a two-stage rotational detection framework. It combines adaptive proposal features with horizontal and rotational prediction branches, including ship-direction prediction and stringent rotational suppression.
- Proposed Method: The framework obtains proposals with an RPN, applies adaptive ROI Align, and predicts horizontal and rotational bounding boxes in its second stage.
- Proposed Method: The rotational branch predicts the ship’s berthing and sailing direction in addition to its rotational bounding box.
- Proposed Method: R-NMS applies more stringent constraints to produce the final predictions.
- Dense Feature Pyramid Network: DFPN uses dense connections and concatenation to merge feature maps, while the implementation assigns multiple scales and aspect-ratio anchors.
- Dense Feature Pyramid Network: DFPN densely fuses multiscale feature maps to propagate and reuse features for detecting objects at different scales.
B. Adaptive ROI Align
Adaptive ROI Align is designed for inclined ships whose elongated proposals contain substantial redundant noise. It filters noise with a trainable mask while retaining spatial information needed for later prediction.
- Adaptive ROI Align: Inclined ships create large redundant proposal regions that can reduce or overwhelm feature quality during extraction.
- Adaptive ROI Align: ROI Align retains substantial noise, whereas RROI Align removes noise through affine transformation but loses object spatial information.
- Adaptive ROI Align: Adaptive ROI Align uses a trainable mask obtained by convolving proposals to automatically filter noise while retaining spatial information.
- Adaptive ROI Align: Visualization indicates that Adaptive ROI Align produces higher-quality feature maps than ROI Align and RROI Align for angle regression and prow-direction prediction.
C. Prow Direction Prediction
The method represents ships with rotated bounding boxes and separately predicts prow direction, using a multitask objective to jointly train classification, regression, and direction-related tasks.
- Rotated bounding box representation: Rotated bounding boxes use center coordinates, width, height, and angle to represent arbitrary-oriented ships.The angle is constrained to [-90,0), with width and height swapped when needed to preserve box position.
- Prow direction prediction: Because the rotation angle alone cannot identify a ship’s prow, the model predicts berthing and sailing direction from the rotation branch.The prow direction is defined relative to the four sides of the minimum bounding rectangle.
- Prow direction prediction: The network reportedly learns the long-edge direction first and then selects which side corresponds to the prow, although some predictions confuse prow and stern.Figure 6 covers side-by-side, docked, large-object, and small-object scenes.
- Rotated bounding box regression: Rotational-box regression incorporates angle information and uses parameterized offsets for the predicted, anchor, and ground-truth boxes.The regression formulation keeps the angle within the specified range and handles width-height interchange for odd rotation indices.
E. Rotational Nonmaximum Suppression
The proposed R-NMS addresses inaccurate overlap calculations and suppression errors that arise when densely packed ships are represented by skewed bounding boxes.
- Motivation: Axis-aligned IoU can inaccurately measure overlap between skewed ship boxes, potentially damaging bounding-box prediction in dense scenes.The method therefore uses a triangulation-based Skew IoU implementation.
- Rotation sensitivity: For a ship with aspect ratio 1:7, a 15-degree angle difference yields an IoU of only 0.38, demonstrating rotation-sensitive overlap.
- R-NMS design: R-NMS preserves predictions with IoU below 0.7 and applies an angle-difference rule when IoU lies between 0.3 and 0.7.The supplied passage states that predictions with angle differences greater than 15 degrees are discarded in this overlap range.
IV. EXPERIMENTS
The experiments compare the proposed framework’s detection performance using an NVIDIA GeForce GTX 1080 GPU with 8 GB of memory.
- Experimental setup: All experiments were conducted on an NVIDIA GeForce GTX 1080 GPU with 8 GB of memory.
A. Dataset and Settings
The evaluation uses the collected SRSS ship dataset and the public DOTA aerial-image dataset, with dataset-specific training schedules and preprocessing settings.
- Datasets: The framework is evaluated on SRSS, a collected satellite remote-sensing ship dataset, and DOTA, a public aerial-image dataset.
- SRSS dataset: SRSS contains 50 large 10,000 × 10,000-pixel scenes covering 25 square kilometers at 0.5-meter resolution.The imagery includes red, green, and blue bands and scenes such as civilian ports, naval bases, and offshore areas.
- SRSS dataset: SRSS preprocessing produces about 8,000 1,000 × 1,000-pixel overlapping subimages, using a 1:4 training-to-test ratio.The overlap between subimages is 0.4, and images without ships are filtered out.
- DOTA dataset: DOTA contains 2,806 aerial images from varied sensors and platforms, spanning roughly 800 × 800 to 4,000 × 4,000 pixels and 15 object categories.Its annotations cover varied object scales, orientations, and shapes.
- DOTA dataset: DOTA images are divided into subimages following the SRSS procedure, with one-third assigned to testing.
- Training settings: The model is initialized with pretrained ResNet-101 and trained for 40k iterations on SRSS with staged learning rates.The learning rate is 0.001 for the first 20k iterations, 0.0001 for the next 10k, and 0.00001 for the final 10k.
- Evaluation strategy: SRSS is used to verify individual model components, especially prow prediction, while DOTA evaluates overall performance and scalability.
B. Evaluation Indicators
The paper evaluates detection using precision–recall curves, F1, and mAP, each capturing different aspects of performance.
- Precision measures the proportion of predicted positives that are correct, using true positives and false positives.
- Recall measures the proportion of actual positives detected, using true positives and false negatives.
- F1 combines precision and recall through their harmonic mean, with higher values indicating better performance.
- mAP integrates precision over recall to reflect global detection performance rather than a single operating point.
C. Evaluation of DFPN
Experiments examine multiscale feature fusion, proposal feature extraction, and comparisons with existing detectors. The proposed method achieves strong detection performance, especially for recall and precision–recall trade-offs, but is slower and produces more false alarms.
- Evaluation of DFPN: Multiscale detection networks outperform single-scale networks, particularly for small-object detection.The paper attributes the improvement to effective fusion of information from multiple feature layers.
- Evaluation of Adaptive ROI Align: Adaptive ROI Align achieves the best results among ROI Align, RROI Align, and Adaptive ROI Align, especially in recall.It reduces redundant-region noise while preserving spatial information, whereas RROI Align loses spatial information.
- Comparisons with Other Ship Detection Methods: FPN-Soft-NMS improves performance by about 1.2% without additional training or computational burden.FPN also obtains the highest precision among the five compared methods, at 89.3%.
- Comparisons with Other Ship Detection Methods: The proposed detection model achieves the best overall performance and highest recall among the compared frameworks.Its precision is not the highest, indicating a higher false-alarm rate than the traditional method with the highest precision.
- Comparisons with Other Ship Detection Methods: The proposed method is the slowest detection algorithm, although the paper describes it as still very efficient.LSTM-Based has the fastest detection speed in the comparison.
F. Evaluation of Prow Direction Prediction
The paper evaluates prow-direction prediction alongside rotational ship detection. Direction accuracy is close to recall across directions, while predictions commonly identify either the prow or stern.
- Prow-direction prediction accuracy depends on recall and is close to recall for each direction.The authors regard this correspondence as evidence that the prediction method is efficient and feasible.
- Final predictions often appear only at the prow and stern.The authors speculate that the network first learns the ship’s long-edge direction and then selects which end is the prow.
G. Evaluation of overall performance and scalability
On DOTA, the framework scales beyond ships to multiple object categories and improves rotation-region detection across varied orientations and sizes, while retaining a precision limitation from false alarms.
- Dataset and scalability: The DOTA evaluation covers 15 object categories, although direction prediction is blocked for categories whose annotations lack directional concepts.The dataset tests overall performance and scalability beyond ship detection.
- Overall comparison: The proposed method achieves higher mAP than horizontal-region detectors, especially for dense or large-aspect-ratio objects such as ships, vehicles, planes, harbors, and bridges.The comparison includes SSD-inception-v2, YOLOv2, R-FCN, and FR-H.
- Component contributions: Adaptive ROI Align improves results over R-DFPN, while DFPN is identified as a primary contributor to the gains over other rotation-region methods.The paper attributes the improvement over R-DFPN specifically to adaptive ROI Align.
- Qualitative results: Despite varied object orientations and sizes, the proposed approach successfully detects and locates most objects in the DOTA examples.The qualitative results are shown in Fig. 10.
- Limitations: More false alarms produce lower Precision than Faster-RCNN and FPN, leaving false-alarm reduction as future work.This limitation remains despite the method's reported best performance.