Source-linked AI summary
Are We Ready for Unmanned Surface Vehicles in Inland Waterways? The USVInland Multisensor Dataset and Benchmark
Yuwei Cheng, Mengxin Jiang, Jiannan Zhu, Yimin Liu
TL;DR
USVInland addresses the lack of a public multisensor dataset and benchmark for USVs in inland waterways. It provides synchronized sensor data and benchmarks for three tasks, where common algorithms perform worse in inland-waterway scenes.
Problem
There was no public standard dataset for inland-waterway USVs with multisensor data supporting multiple tasks and sensor fusion.
Source: §I · p. 1Method
USVInland collects multisensor inland-waterway data and provides benchmarks for SLAM, stereo matching, and water segmentation.
Source: §I · p. 2Results
Common algorithms perform worse on SLAM, stereo matching, and water segmentation benchmarks because of inland-waterway scene characteristics.
Source: §I · p. 2Takeaways & Limitations
The authors conclude that common autonomous-driving approaches may not meet inland USVs’ safe-navigation requirements.
Source: §V · p. 7
Abstract
from arXiv · showhide
Unmanned surface vehicles (USVs) have great value with their ability to execute hazardous and time-consuming missions over water surfaces. Recently, USVs for inland waterways have attracted increasing attention for their potential application in autonomous monitoring, transportation, and cleaning. However, unlike sailing in open water, the challenges posed by scenes of inland waterways, such as the complex distribution of obstacles, the global positioning system (GPS) signal denial environment, the reflection of bank-side structures, and the fog over the water surface, all impede USV application in inland waterways. To address these problems and stimulate relevant research, we introduce USVInland, a multisensor dataset for USVs in inland waterways. The collection of USVInland spans a trajectory of more than 26 km in diverse real-world scenes of inland waterways using various modalities, including lidar, stereo cameras, millimeter-wave radar, GPS, and inertial measurement units (IMUs). Based on the requirements and challenges in the perception and navigation of USVs for inland waterways, we build benchmarks for simultaneous localization and mapping (SLAM), stereo matching, and water segmentation. We evaluate common algorithms for the above tasks to determine the influence of unique inland waterway scenes on algorithm performance. Our dataset and the development tools are available online at https://www.orca-tech.cn/datasets.html.
I. INTRODUCTION
Inland-waterway conditions challenge USV perception and navigation, while no public standard dataset supports inland USVs across multiple sensors and tasks. USVInland provides multisensor data and benchmarks for evaluating these challenges.
I. INTRODUCTION
- GPS attenuation, fog, reflections, and frequent pose changes complicate localization, perception, and navigation for inland-waterway USVs.GPS signals can be attenuated by vegetation, bridges, and urban settlements; water waves and low surface friction can also hinder landmark matching.Source: §I · p. 1
- Before USVInland, no public standard dataset targeted inland-waterway USVs or provided multisensor data for multiple tasks and sensor fusion.Source: §I · p. 1
- Commonly used algorithms perform worse in inland-waterway scenes, according to benchmarks for SLAM, stereo matching, and water segmentation.The paper reports that these scene characteristics reduce algorithm performance.Source: §I · p. 2
- The dataset and calibration data are accompanied by tools for loading and visualizing synchronized multisensor data.Source: §I · p. 2
- USVInland introduces multisensor inland-waterway data and benchmarks for SLAM, stereo matching, and water segmentation.The dataset includes lidar, stereo cameras, millimeter-wave radar, GPS, and IMUs, with synchronized data and calibration results.Source: §I · p. 2
A. USVs for the Inland Waterway
Prior work applies USVs to inland-waterway monitoring, cleaning, and transportation, while related methods address SLAM, stereo matching, and obstacle detection. The reviewed approaches span lidar, visual, and radar sensing.
A. USVs for the Inland Waterway
- Inland-waterway USVs have been developed for structural monitoring, water-quality monitoring, floating-waste collection, and autonomous transportation.The Roboat project includes localization, perception, planning, and control for autonomous transportation tasks.Source: A. USVs for the Inland Waterway · p. 2
B. SLAM
- Prior USV SLAM work includes lidar localization under bridges, LIO-SAM on Roboat II, visual odometry for river monitoring, and radar-based coastal navigation.Source: B. SLAM · pp. 2–3
C. Stereo Matching
- Stereo matching supports obstacle detection through reconstructed 3D points and through image-pair alignment for segmentation consistency.Classical and newer methods provide depth information for self-driving vehicles.Source: C. Stereo Matching · p. 3
D. Water Segmentation
Water detection methods range from shoreline-based approaches to deep-learning segmentation, while the acquisition platform carries sensors for diverse benchmark tasks.
D. Water Segmentation
- Deep-learning water segmentation can handle boundaries beyond straight shorelines, unlike methods that rely on fitted waterlines or extracted line segments.Prior approaches may fail with extremely irregular shorelines or interference over the water surface.Source: D. Water Segmentation · p. 3
A. Platform
- The platform places a lidar above the boat, a forward-facing stereo camera, and three radars at the front and sides to broaden radar coverage.Source: A. Platform · p. 3
- The sensor suite includes a 16-beam lidar, stereo camera with IMU, three millimeter-wave radars, INS, and two GPS antennas.One GPS antenna uses RTK to provide up to centimeter-level accuracy.Source: A. Platform · p. 3
B. Data Collection
USVInland was collected from May to August 2020 along inland waterways in typical cities. The collection covers varied weather and waterway settings.
B. Data Collection
- From May to August 2020, the inflatable boat was driven manually and automatically along inland waterways in cities with rich hydrographic networks.Source: B. Data Collection · p. 3
- Data were collected under varied weather conditions to cover real-world inland-waterway scenes.The figure caption describes the different weather conditions as revealing perception and navigation challenges.Source: B. Data Collection · p. 3, Figure 3 · p. 4
- The selected waterways vary in width and length and include diverse bankside landscapes.Source: B. Data Collection · p. 4
C. Synchronization and Calibration
The section describes timestamp-based synchronization and the calibration procedures used to register the platform’s sensors.
C. Synchronization and Calibration
- Sensor frames are synchronized by combining each sensor’s recorded absolute starting timestamp with its per-frame relative timestamps.The authors report a maximum 25 ms synchronization drift between radar and camera, which they consider suitable for the boat’s speed.Source: C. Synchronization and Calibration · p. 4
- Camera–lidar extrinsics are calibrated by matching image checkerboard corners to lidar points, while radar–lidar registration is optimized using physical targets.The relative radar positions are measured first; raw calibration data are also provided for researchers who want to obtain their own results.Source: C. Synchronization and Calibration · p. 4
- Stereo-camera intrinsics and camera–IMU extrinsics within the same device come from factory calibration.The lidar–camera and lidar–radar registrations are handled separately.Source: C. Synchronization and Calibration · p. 4
D. Benchmark Selection
The benchmarks draw on 27 raw sequences covering more than 26 km of inland waterways, with separate selection procedures for SLAM, stereo matching, and water segmentation.
D. Benchmark Selection
- The raw dataset covers more than 26 km across 27 continuous sequences collected under different weather conditions, with waterways 10–40 m wide and covered lengths ranging from 100 to 3400 m.Figure 4 shows part of the collection trajectories from 5 of the 27 raw sequences; sequence names encode collection date and location.Source: D. Benchmark Selection · p. 4, Figure 4 · p. 4
- The SLAM benchmark contains 33 sequences cut from the raw data, each 100–2600 seconds long, and retains only sequences with RTK Fixed GPS data.The GPS localization precision factor distinguishes RTK Fixed and RTK Float accuracy levels.Source: D. Benchmark Selection · p. 4
- The stereo-matching benchmark contains 324 image pairs selected by downsampling and clustering for sample diversity, with some obscured or overexposed images removed manually.The selection procedure is described as similar to the one in.Source: D. Benchmark Selection · p. 4
- The water-segmentation benchmark contains 700 manually selected images spanning waterline shapes and varied weather and lighting conditions.Unlike marine water-segmentation datasets dominated by straight water–sky boundaries, it includes varied inland waterline structures, such as bridge areas and vegetated banks.Source: D. Benchmark Selection · p. 4, Figure 5 · p. 4
- The SLAM evaluation reports relative pose error over trajectory segments from 5 to 400 m, alongside absolute trajectory errors plotted against distance traveled.The trajectory figure compares LOAM and LeGO-LOAM with ground truth on six sequences.Source: D. Benchmark Selection · p. 5, Figure 6 · p. 5
E. Ground-Truth Generation
Ground truth is generated separately for SLAM, stereo matching, and water segmentation; experiments then use the three benchmarks to evaluate relevant algorithms in inland-waterway scenes.
E. Ground-Truth Generation
- SLAM ground truth uses GPS location and IMU pose information.Source: E. Ground-Truth Generation · p. 5
- Stereo depth ground truth is produced by aggregating synchronized lidar point clouds from adjacent frames with ICP, then projecting the denser points onto the image plane.No water-surface depth ground truth is provided because reflected-light intensity is reduced and lidar detects nearly no points there.Source: E. Ground-Truth Generation · p. 5
- Water-segmentation ground truth is manually polygon-annotated with LabelMe, and annotations are repeatedly checked and corrected.Source: E. Ground-Truth Generation · p. 5
IV. EXPERIMENT
- Experiments on the three benchmarks assess dataset validity and evaluate relevant algorithms on inland-waterway scenes.Source: §IV · p. 5
A. SLAM
The SLAM evaluations show substantial visual-SLAM error, lower lidar-SLAM performance in rain and fog, and radar-SLAM results that vary with the availability of nearby targets.
A. SLAM
- LOAM and LeGO-LOAM performance decreases on sequences collected in harsh environments such as rain and fog.The lidar-SLAM evaluation uses six sequences collected under different weather conditions and assesses trajectories in 2-D.Source: A. SLAM · p. 5
- More than 20% average relative translation error was reported for ORB-SLAM2 across the 33 public sequences, with estimates differing substantially from ground truth.The figure shows ground-truth trajectories as dashed lines and ORB-SLAM2 estimates as green lines on four sequences.Source: A. SLAM · p. 5, Figure 7 · p. 5
- Fog makes feature detection harder, while faraway water reflections and unstable reflection keypoints can increase visual-SLAM pose errors.When the boat is mid-channel, extracted features are mostly distant reflections; water waves can distort those reflections between frames.Source: A. SLAM · p. 6, Figure 8 · p. 6
- 3.95% relative translational error was obtained by radar SLAM, averaged over trajectories of 5–200 m on selected sequences.The evaluation uses eight sequences where the boat sails close to the bankside or in narrow rivers.Source: A. SLAM · p. 6
- Radar SLAM performs better with distinct waterway-side targets, while moving farther from the banks reduces valid targets and increases drift.The cited example includes stone balustrades along the bank as distinct targets.Source: A. SLAM · p. 6, Figure 9 · p. 6
B. Stereo Matching
The section reports weak stereo-matching performance on inland-waterway scenes and describes water-segmentation evaluation and its accuracy challenges. The evaluations highlight limitations tied to water-surface ground truth and variable scenes.
B. Stereo Matching
- SGM performs poorly on inland-waterway stereo scenes; Fig. 10 contrasts relatively unacceptable and good disparity-map results.The SGM result is reported in Table II.Source: B. Stereo Matching · p. 6, Figure 10 · p. 7, Table II · p. 7
- Stereo matching uses 180 low-resolution 640×320 image pairs, with evaluation limited to pixels having lidar-derived ground-truth disparity.Water areas are excluded because they lack ground-truth data; the authors prioritize bankside depth for collision avoidance.Source: B. Stereo Matching · p. 6
C. Water Segmentation
- Water segmentation is evaluated with Deeplab v3+ on high- and low-resolution images, using train and test sets from different scenes and weather conditions.The model is also evaluated independently on images of both resolutions.Source: C. Water Segmentation · p. 6
- Low-accuracy segmentation examples reflect variable inland environments and the visual similarity between bankside objects and their reflections.The authors identify both as difficulties for accurate water-area segmentation.Source: C. Water Segmentation · p. 6
V. CONCLUSION
The conclusion presents USVInland as the first multisensor USV dataset for inland waterways, with benchmarks for three tasks. Evaluations indicate that common autonomous-driving approaches may not meet safe-navigation requirements in these scenes.
V. CONCLUSION
- USVInland is presented as the first multisensor USV dataset for inland waterways, with benchmarks for SLAM, stereo matching, and water segmentation.The dataset was collected under varied weather conditions to represent real-world inland-waterway scenes.Source: §V · p. 7
- Evaluations find that common autonomous-driving approaches may perform poorly in inland-waterway scenes and may not meet USV safe-navigation requirements.Source: §V · p. 7
- The authors plan to extend the dataset with historical data for research on long-term SLAM in real-world inland waterways.Source: §V · p. 7