Source-linked AI summary
NTU VIRAL: A Visual-Inertial-Ranging-Lidar Dataset, From an Aerial Vehicle Viewpoint
Thien-Minh Nguyen, Shenghai Yuan, Muqing Cao, Yang Lyu, Thien Hoang Nguyen, Lihua Xie
TL;DR
NTU VIRAL addresses the relative lack of public datasets for autonomous aerial systems by collecting multimodal data from a drone-mounted sensor suite. It combines lidar, synchronized stereo vision, IMUs, UWB ranging, and laser-tracker ground truth across challenging environments, while documenting timing and practical limitations.
Problem
Public datasets for autonomous aerial navigation are relatively scarce compared with datasets for autonomous driving and ground robots.
Method
The paper constructs a drone-based benchmark suite with multiple lidars, synchronized cameras, IMUs, UWB ranging, calibration results, and laser-tracker ground truth.
Results
The datasets cover challenging indoor and outdoor conditions, and onboard-sensor ground-truth temporal alignment is within 0.1s in the current datasets.
Takeaways & Limitations
NTU VIRAL provides ROS-compatible datasets, analysis code, and example localization implementations for investigating autonomous UAV navigation.
Takeaways & Limitations
Fixed exposure settings can produce darker images that challenge direct visual SLAM, and practical data-collection issues limit achievable accuracy.
Abstract
from arXiv · showhide
In recent years, autonomous robots have become ubiquitous in research and daily life. Among many factors, public datasets play an important role in the progress of this field, as they waive the tall order of initial investment in hardware and manpower. However, for research on autonomous aerial systems, there appears to be a relative lack of public datasets on par with those used for autonomous driving and ground robots. Thus, to fill in this gap, we conduct a data collection exercise on an aerial platform equipped with an extensive and unique set of sensors: two 3D lidars, two hardware-synchronized global-shutter cameras, multiple Inertial Measurement Units (IMUs), and especially, multiple Ultra-wideband (UWB) ranging units. The comprehensive sensor suite resembles that of an autonomous driving car, but features distinct and challenging characteristics of aerial operations. We record multiple datasets in several challenging indoor and outdoor conditions. Calibration results and ground truth from a high-accuracy laser tracker are also included in each package. All resources can be accessed via our webpage https://ntu-aris.github.io/ntu_viral_dataset.
1 Introduction
NTU VIRAL addresses the shortage of public datasets for autonomous aerial navigation in GPS-denied environments by combining a comprehensive drone-mounted sensor suite with challenging flight conditions and high-accuracy ground truth.
- Public datasets reduce the hardware, calibration, and field-operation burden while enabling navigation methods to be tested and compared across scenarios.
- Ground-air dichotomy: Aerial datasets typically emphasize high-frame-rate cameras for VIO and visual SLAM because payload constraints often exclude 3D lidar.
- Ground-air dichotomy: Ground and aerial datasets differ in motion, duration, environment size, and access to GPS, motivating a dataset designed for complex UAV motion and stringent evaluation.
- A new dataset for autonomous drone: The datasets cover indoor and outdoor NTU environments and include ROS bags, analysis code, and plug-and-play examples for several state-of-the-art localization methods.
2 Sensor setup
The sensor setup combines inertial, lidar, stereo-vision, UWB-ranging, and laser-tracking hardware on a DJI M600 Pro hexacopter. The arrangement supports complementary environmental observation, range sensing, and position ground truth.
- A DJI M600 Pro hexacopter carries the sensor setup, whose messages are timestamped by their ROS publish times.
- The main VectorNav VN100 IMU is the body-frame center and publishes inertial data at an effective rate of roughly 385Hz.
- Two 16-channel OS1 lidars provide complementary coverage of the UAV’s sides, ground, and front and back directions.
- Two externally triggered monochrome global-shutter cameras face forward, with synchronized image timestamps typically differing by less than 3 ms.
- UWB ranging sensors: Four UAV UWB nodes and three anchors create 12 UAV-to-anchor ranging pairs, with UAV-to-anchor measurements published at 68.571 Hz.
- Ground truth: A Leica Nova MS60 MultiStation tracks a crystal prism on the UAV to provide position ground truth in a frame aligned with gravity.
3 Dataset characteristics
NTU VIRAL divides its recordings into EEE, SBS, and NYA sequences spanning indoor and outdoor NTU campus environments with distinct sensing challenges.
- Sequence overview: The datasets comprise EEE, SBS, and NYA sequences collected across indoor and outdoor locations on the NTU campus.Table 3 summarizes the sequences and their statistics.
- EEE sequences: EEE sequences were collected in a carpark surrounded by tall buildings, trees, road markings, and buildings that support lidar and visual localization.
- SBS sequences: SBS sequences were collected in an open square with low-rise buildings and large glass surfaces, where distant visual features can produce noisy depth.
- Trajectory recording: The SBS trajectories were recorded by a Leica laser tracker and are shown for the collected sequences.
- NYA sequences: The auditorium environment challenges lidar SLAM with semi-transparent surfaces and visual SLAM with flight dynamics and low lighting.UWB measurements also exhibit multipath effects and signal loss.
4 Dataset format
NTU VIRAL packages flight recordings as synchronized rosbags with calibration and sensor-specific metadata, while documenting units, timing, message types, and coordinate conventions.
- 4.1 Files organization: Each flight produces a single rosbag by temporally synchronizing onboard recordings with separately recorded ground truth.The bags are stored on the drone’s onboard DJI Manifold 2C computer before merging.
- 4.1 Files organization: Calibration files accompany each rosbag and contain camera coefficients, sensor-to-body transforms, IMU noise, and camera-IMU time delay.They can be parsed with OpenCV and ROS APIs; accessories for pointclouds and UWB data are also provided.
- 4.2 Units and timestamps: Measurements use SI units, while message timestamps use ros/Time format assigned when each sensor driver publishes over ROS.
- 4.2 Units and timestamps: Lidar message timestamps mark scan end-time, and each point includes its time relative to scan start for deskewing.
- 4.3 Message types: Custom uwb_driver/UwbRange messages include distance, signal-to-noise ratio, line-of-sight diagnosis, antenna coordinates, and anchor coordinates.Anchor coordinates are fixed using the first messages in /nodes_pos_sc.
- 4.3 Message types: UWB measurements include a 5 cm fluctuation during static flight in SBS 03, while NYA 03 shows signal loss and multipath effects.
- 4.4 Coordinate transformations: Extrinsics use ROS and EuRoC parent-child naming, with T_Body_Cam represented as an SE(3) transform containing rotation and translation.The convention supports transforming observed objects between camera, body, and other coordinate frames.
- 4.4 Coordinate transformations: For an object observed in the camera frame, its position and orientation are converted to the body frame using the documented extrinsics.This coupling is required when tracked features or objects are used in SLAM estimation.
5 Sensor calibration
The calibration pipeline separates rigid camera-IMU calibration from flexible sensor compensation, then evaluates temporal alignment and localization performance against laser-tracker ground truth.
- Calibration strategy: Cameras and IMU are mounted on rigid titanium-alloy parts, whereas other sensors are flexibly mounted through carbon-fiber tubes or dampers.The flexible mounting introduces greater variance in spatial relations.
- Calibration strategy: Rigid calibration covers stereo intrinsics and extrinsics plus the IMU-camera transform, while flexible-sensor transforms rely mainly on VICON and laboratory priors.Calibration results are subsequently verified.
- Camera calibration: Stereo calibration uses synchronized chessboard images, pinhole and radial-tangential models, reprojection-error filtering, and lidar depth verification.Large chessboard roll increases reprojection error for the 120-degree FOV lens.
- Camera-IMU calibration: Camera-IMU calibration uses Kalibr with stereo-derived intrinsics, manufacturer-calibrated IMU biases, and full-6-DOF excitation.
- Flexible parts and ground truth: The payload bay’s flexible relation to GPS, compass, and prism is compensated using VICON, tilt testing, flight-motion assumptions, and hand-eye alignment.The measured prism displacement between static and maximum tilt is around 2 cm.
- Localization evaluation: Table 4 reports ATE in meters for state-of-the-art localization methods across NTU VIRAL datasets, highlighting the best odometry result and underlining the second best.
- Temporal alignment: Temporal alignment error is bounded within 0.1 s because VINS, LIO-SAM, and VIRAL SLAM all minimize ATE within the [-0.1, 0.1] interval.
6 Evaluation recommendations
The evaluation procedure converts estimated body poses to prism trajectories, resamples ground truth, aligns coordinate frames, and computes ATE for consistent comparison.
- 6.1 Are we tracking the same point?: Evaluation compares an estimated body trajectory with ground-truth crystal-prism samples, using supplied MATLAB scripts for accuracy metrics.
- 6.1 Are we tracking the same point?: A body-centered comparison can incur up to 0.4 m bias because the crystal prism is displaced from the body origin.The recommendation is to transform poses into an estimated prism trajectory before comparison.
- Evaluation visualization: Figure 8 evaluates ATE sensitivity to ground-truth time shifts, while Figure 9 visualizes VIRAL SLAM’s aligned trajectory, map, keyframes, and ground truth.
- 6.2 Resampling: The resampling step matches each ground-truth time to an estimated trajectory interval with adjacent samples separated by less than 0.1 seconds.Ground truth within the interval is linearly interpolated.
- 6.3 Coordinates transform: The coordinate-transform stage maps estimated and ground-truth trajectories into a common reference frame before direct comparison.The transform has a closed-form solution from Umeyama’s method.
7 Known issues and limitations
The dataset has practical limitations that constrain achievable accuracy, including camera exposure trade-offs, temporary ground-truth tracking loss, and UWB measurement loss.
- 7.1 Camera exposure setting: Fixed exposure settings improve image sharpness and reduce motion blur but produce darker images that can challenge direct visual SLAM.The settings trade brightness consistency and reduced glare for lower image brightness.
- 7.2 Partial loss of information: The Leica station temporarily lost track of the prism in SBS experiments because its vantage point was lower than in other environments.The provided evaluation method accounts for these tracking interruptions.
- 7.2 Partial loss of information: Significant UWB loss occurred in some experiments when low received signal strength arose at certain node-anchor relative positions.The loss is linked to the radiation pattern and represents a real-scenario challenge for algorithm development.
8 Conclusion and future works
The paper presents a drone-mounted dataset suite for advancing autonomous UAV navigation and acknowledges remaining hardware and data-collection issues alongside planned updates.
- 8 Conclusion and future works: The datasets combine an autonomous-driving-style sensor suite with a drone platform to support autonomous UAV navigation research.The paper positions this collection as a resource for investigation using up-to-date technologies.
- 8 Conclusion and future works: Remaining issues include UWB loss, limited ground-truth tracking vantage, temporal alignment, and camera hardware updates.The authors identify these as development issues rather than presenting the dataset as final.
- 8 Conclusion and future works: Future work will address hardware and processing improvements, while adding more datasets to the suite.Planned changes include updated UWB hardware, improved tracking vantage, batch temporal optimization, and newer RGBD-type sensors.
Dataset access
The dataset and supporting resources are available through the project webpage.
- Dataset access: The dataset is accessible from the NTU VIRAL project webpage.The site also provides calibration, evaluation, and update details to facilitate research use.
- Dataset access: Calibration details are provided online to support use of the dataset.
- Dataset access: Evaluation and update information is provided online for individual research use.