Source-linked AI summary
The Newer College Dataset: Handheld LiDAR, Inertial and Vision with Ground Truth
Milad Ramezani, Yiduo Wang, Marco Camurri, David Wisth, Matias Mattamala, Maurice Fallon
TL;DR
The paper addresses limited systematic evaluation of mobile localization and mapping when precise ground truth is unavailable across varied environments. It constructs a handheld vision-and-LiDAR dataset and derives 6 DoF poses by registering scans to a detailed prior map. The resulting dataset provides centimeter-scale ground truth and supports visual, LiDAR, reconstruction, and navigation research.
Problem
Existing datasets lack precise ground truth across large or GPS-deprived environments, limiting evaluation of localization and mapping systems.
Method
The authors collect handheld stereo-inertial camera and LiDAR data, build a detailed prior map, and register individual scans to it using ICP.
Results
The dataset provides accurate 6 DoF ground truth for the entire dataset, with stationary ICP standard deviations of 2 cm, 1.6 cm, and 2 cm along x, y, and z.
Takeaways & Limitations
The dataset supports systematic evaluation of visual and LiDAR localization, mapping, reconstruction, and navigation across built, open, and vegetated environments.
Abstract
from arXiv · showhide
In this paper we present a large dataset with a variety of mobile mapping sensors collected using a handheld device carried at typical walking speeds for nearly 2.2 km through New College, Oxford. The dataset includes data from two commercially available devices - a stereoscopic-inertial camera and a multi-beam 3D LiDAR, which also provides inertial measurements. Additionally, we used a tripod-mounted survey grade LiDAR scanner to capture a detailed millimeter-accurate 3D map of the test location (containing $\sim$290 million points). Using the map we inferred centimeter-accurate 6 Degree of Freedom (DoF) ground truth for the position of the device for each LiDAR scan to enable better evaluation of LiDAR and vision localisation, mapping and reconstruction systems. This ground truth is the particular novel contribution of this dataset and we believe that it will enable systematic evaluation which many similar datasets have lacked. The dataset combines both built environments, open spaces and vegetated areas so as to test localization and mapping systems such as vision-based navigation, visual and LiDAR SLAM, 3D LIDAR reconstruction and appearance-based place recognition. The dataset is available at: ori.ox.ac.uk/datasets/newer-college-dataset
I. INTRODUCTION
The paper introduces a handheld vision-and-LiDAR dataset designed for systematic localization and mapping evaluation with centimeter-accurate ground truth. It combines diverse environments, walking-speed sensing, and a prior 3D map to support multiple navigation tasks.
- Motivation: Existing benchmarks use GPS/INS, laser trackers, or motion capture, each limiting precise local accuracy, line of sight, experiment scale, or environment.These limitations motivate a dataset with accurate ground truth across larger outdoor environments.
- Contribution: The dataset provides centimeter accuracy for evaluating both short length-scale odometry and large-scale drift.The raw measurements are accompanied by a precise 3D model, and device locations are inferred with LiDAR ICP at 10 Hz.
- Data collection: A handheld device carried at walking speed combines a stereo-inertial camera and a 3D LiDAR with self-contained IMUs.The sensors are commercially available Intel Realsense D435i and 64-beam Ouster OS-1 devices.
- Research scope: The New College site includes structured buildings, vegetation, and open spaces lacking texture, with original and faster aggressive sequences for navigation and mapping research.The sequences target visual navigation, LiDAR SLAM, reconstruction, and place recognition.
- Paper organization: The paper details the device, dataset, ICP-based ground truth, and example mobile-robotics applications before summarizing the work.
II. RELATED WORK
Related benchmarks span vehicle platforms and large traverses, but their ground truth often depends on GPS and is unreliable in GPS-deprived areas. The review positions the dataset against these scale and localization constraints.
- Vehicle-based datasets: Vehicle-based datasets cover substantial distances and support autonomous-navigation research using platforms such as UGVs and cars.Examples include the 90 km MIT traverse, the 37 km Malaga trajectory, and the 1000 km Oxford RobotCar dataset.
- Vehicle-based datasets: The reviewed datasets collectively target varied challenges, including dust, smoke, rain, GPS loss, multilane highways, and dynamic urban entities.
- Vehicle-based datasets: KITTI provides 6 DoF trajectories with below-10-cm RTK-GPS/INS accuracy, but GPS accuracy is not guaranteed in urban canyons and its images and IMU readings are unsynchronized.
- Vehicle-based datasets: Ground truth in driverless-car datasets is highly dependent on GPS, so its use is not recommended in GPS-deprived areas for localization and mapping evaluation.
B. Mobile Robot or Human-carried Datasets
Mobile-robot and human-carried datasets provide campus, indoor, outdoor, and visual-inertial scenarios, but their ground-truth methods face scale, line-of-sight, GPS, or accuracy constraints. The paper proposes prior-map LiDAR registration as an alternative.
- Existing mobile and handheld datasets: The original New College dataset offers synchronized laser, stereo, and omnidirectional imagery with 5 DoF odometry over a 2.2 km Segway traverse, but no ground truth.
- Existing mobile and handheld datasets: NCLT spans 147.4 km over 15 months with LiDAR scan matching and RTK-GPS ground truth, but GPS multipath can cause drift indoors or near buildings.
- Visual-inertial datasets: EuRoC, Zurich Urban MAV, and PennCOSYVIO target visual-inertial odometry or visual SLAM across aerial and handheld trajectories.Their trajectories range from 0.6 km to 2 km in the cited examples.
- Ground-truth limitations: Laser trackers, motion capture, photogrammetry, and external sensor networks provide ground truth but are constrained by orientation coverage, small indoor areas, image accuracy, or continuous line of sight.
- Proposed ground truth: The proposed approach registers individual LiDAR scans against an accurate prior map using ICP to determine ground truth.
III. THE HANDHELD DEVICE
The handheld platform rigidly mounts a stereo-inertial camera and dense 64-beam LiDAR, with documented coordinate frames, calibration, and software synchronization. Its design supports reproducible multimodal sensing for mobile robotics.
- Mechanical design: The custom device rigidly attaches its sensors to a precisely 3D-printed base and provides an open-source URDF model.The Ouster LiDAR is mounted on top with a 45-degree clockwise rotation for cable routing.
- Sensors: The platform combines a commodity-grade Intel Realsense stereo-inertial camera with a 64-beam Ouster LiDAR that provides denser data than many other LiDAR datasets.
- Coordinate frames: The paper defines coordinate frames for the LiDAR IMU, LiDAR point clouds, stereo cameras, and stereo IMU.
- Calibration and synchronization: Kalibr computes camera intrinsic and extrinsic calibration and supports spatio-temporal calibration across the cameras and both IMUs.The device is not hardware synchronized, making software and network synchronization important.
- Mapping and recording: The device uses a survey-grade Leica BLK360 scanner to build the ground-truth map, while the onboard computer records the handheld sensors.
IV. DATA COLLECTION
The dataset contains varied handheld walking sequences through the Quad, Mid-Section, and Parkland, supplemented by faster and more aggressive motion experiments.
- Additional experiments: Additional datasets vary walking speed and turning, and are organized by motion aggressiveness.These include a faster Quad sequence, dynamic spinning at 2.5 rad/sec, and Parkland mound loops.
- Main collection: The main route follows the original New College dataset through the Quad, Mid-Section, and Parkland.The environments include repeating architecture, illumination changes, open areas, vegetation, gravel, and muddy paths.
- Main collection: The handheld device was carried at roughly 1 m/s, above the shoulder, with unavoidable vibration from human walking and hand motion.The collection was not intended to be highly dynamic, but its walking gait introduces motion relevant to mobile sensing.
- Main collection: The full traversal lasted 2300 seconds and included repeated loops, back-and-forth sections, and return paths between the Quad and Parkland.The collection reached second 2180 before the final return to the starting point.
- Additional experiments: Further experiments used faster walking and aggressive device motions to extend the dataset’s testing scope.A shorter 1500-second experiment was also collected at the same walking speed.
V. GROUND TRUTH
Ground truth is obtained by registering handheld LiDAR scans against a survey-grade prior map using ICP. Stationary accuracy is approximately 3 cm overall, while motion and difficult environments make evaluation harder.
- Pose estimation: ICP computes ground-truth poses by iteratively minimizing Euclidean distances between corresponding points in a reading cloud and reference cloud.The method requires a prior pose and estimates relative displacement during registration.
- Prior map: A survey-grade Leica BLK360 scanner captured 47 static point clouds to build the prior map of the approximately 135×225 m2 environment.The merged map contains about 290 million points.
- Registration: The map is downsampled to 1 cm resolution and dynamically cropped to a 100 m by 100 m reference region around the sensor pose.Filtered scans are registered against this local reference cloud, with ground-truth poses expressed in the base frame.
- Limitations: Ground-truth evaluation is challenging in confined Mid-Section areas and the natural Parkland environment, especially while the device is moving.Dynamic effects such as LiDAR motion distortion may reduce accuracy during motion.
- Accuracy: 3 cm overall: stationary ground-truth position has standard deviations of 2 cm, 1.6 cm, and 2 cm along x, y, and z.These values come from the first 10 seconds, when the device was stationary.
VI. EXAMPLE DATASET USAGE
The dataset supports example evaluations across localisation and mapping research, covering LiDAR-SLAM, loop closure, 3D reconstruction, and visual-inertial odometry.
- The paper presents example uses of the dataset in localisation and mapping research.
- The demonstrated research areas include LiDAR-SLAM and appearance-based loop closure.
- The dataset is also used for 3D reconstruction and visual (inertial) odometry.
A. LiDAR SLAM
A LiDAR-SLAM system estimates ego-motion and detects loop closures geometrically at 2 Hz across the entire dataset, demonstrating a localisation use case.
- The LiDAR-SLAM system estimates ego-motion at 2 Hz across the dataset.
- The system identifies loop closures geometrically.
- The resulting trajectory demonstrates the dataset’s usefulness for LiDAR-based localisation systems.
B. Visual Appearance-Based Loop-Closure
The dataset supports visual place recognition by comparing ORB-feature views across previously visited poses and identifying loop candidates from similarity scores.
- DBoW2 compares ORB-feature views against poses traveled previously to identify loop candidates using a similarity threshold.The poses are sampled at 2 m intervals.
C. LiDAR 3D Reconstruction
The dataset supports 3D reconstruction by comparing meshes generated from the Leica ground-truth map and registered Ouster scans.
- The top row shows a Poisson-reconstructed surface mesh generated from the Leica ground-truth map.
- The bottom row shows a mesh created from Ouster scans registered against the prior map using ground-truth localization.
D. Visual (Inertial) Odometry
The dataset provides synchronized visual-inertial measurements and calibration files for evaluating visual odometry, while also demonstrating applications across mobile robotics and navigation.
- Software time-synchronized visual and inertial measurements, calibration files, and image data support evaluation of visual odometry methods.
- ORB-SLAM2 was tested for stereo odometry with loop closures disabled on the first 1483 seconds of data.
- The dataset’s accurate 6 DoF ground truth distinguishes it from many other datasets.
- The paper also demonstrates the dataset for different subproblems in mobile robotics and navigation.