Source-linked AI summary
M2DGR: A Multi-sensor and Multi-scenario SLAM Dataset for Ground Robots
Jie Yin, Ang Li, Tao Li, Wenxian Yu, Danping Zou
TL;DR
Ground-robot SLAM lacks datasets that combine rich sensing with diverse indoor and outdoor scenarios. M2DGR introduces and benchmarks such a dataset, finding that existing SLAM systems perform poorly in some situations and releasing the dataset and tools publicly.
Problem
Existing SLAM datasets and solutions inadequately cover the practical, diverse indoor and outdoor conditions encountered by ground robots.
Method
M2DGR constructs a large-scale, multi-sensor ground-robot dataset with calibrated and synchronized sensing, trajectory ground truth, diverse scenarios, and a benchmark of state-of-the-art SLAM systems.
Results
Existing visual and LiDAR SLAM systems perform poorly in at least one scenario, while LiDAR methods generally outperform vision-based methods, especially outdoors.
Takeaways & Limitations
M2DGR provides a public test field for analyzing SLAM defects and advancing ground-robot localization and mapping.
Abstract
from arXiv · showhide
We introduce M2DGR: a novel large-scale dataset collected by a ground robot with a full sensor-suite including six fish-eye and one sky-pointing RGB cameras, an infrared camera, an event camera, a Visual-Inertial Sensor (VI-sensor), an inertial measurement unit (IMU), a LiDAR, a consumer-grade Global Navigation Satellite System (GNSS) receiver and a GNSS-IMU navigation system with real-time kinematic (RTK) signals. All those sensors were well-calibrated and synchronized, and their data were recorded simultaneously. The ground truth trajectories were obtained by the motion capture device, a laser 3D tracker, and an RTK receiver. The dataset comprises 36 sequences (about 1TB) captured in diverse scenarios including both indoor and outdoor environments. We evaluate state-of-the-art SLAM algorithms on M2DGR. Results show that existing solutions perform poorly in some scenarios. For the benefit of the research community, we make the dataset and tools public. The webpage of our project is https://github.com/SJTU-ViSYS/M2DGR.
I. INTRODUCTION
Existing SLAM datasets and methods do not adequately represent the diverse, challenging conditions faced by ground robots. M2DGR addresses this gap with a rich multi-sensor dataset and benchmark covering practical scenarios.
- Ground robots operate across applications requiring reliable navigation in indoor and mixed indoor–outdoor environments.
- Existing visual and LiDAR SLAM methods can fail in darkness, textureless scenes, long corridors, open areas, and unusual motions such as lift travel.
- Results show that existing SLAM systems perform poorly in at least one situation, indicating that further improvement is required.
- M2DGR provides large-scale ground-robot sequences with six surround-view fisheye cameras, a sky-pointing camera, VI-sensor, event camera, infrared camera, LiDAR, IMU, and two GNSS receivers.
- The dataset includes challenging practical situations such as entering lifts and complete darkness that previous datasets do not contain.
- The authors benchmark state-of-the-art SLAM algorithms of varied designs and analyze their characteristics and defects.
II. RELATED WORK
Related work spans vision-based, LiDAR-based, and multi-sensor SLAM, along with specialized sensor configurations. Existing datasets and sensing approaches each leave gaps for robust ground-robot operation across difficult environments.
- SLAM with different sensors: Visual SLAM includes monocular, binocular, and multi-camera settings, but pure vision struggles in textureless or low-illumination scenes.
- Dataset context: The dataset is visualized as sensory data captured by a rich sensor suite across various scenarios.
- SLAM with different sensors: LiDAR SLAM is generally more reliable in difficult visual conditions but can struggle with long corridors, highly dynamic movements, and fog.
- Multi-sensor fusion: Multi-sensor fusion combines vision, LiDAR, IMU, and GNSS in different SLAM systems to improve practical accuracy and robustness.
- Sensor configurations: Multiple cameras broaden field of view and can improve robustness in dynamic scenes while resolving scale ambiguity through known baselines.
- Sensor configurations: Thermal-infrared cameras improve visibility beyond the visible spectrum in fog, smoke, and darkness.
- Sensor configurations: Event cameras provide low-delay, low-power, high-dynamic-range measurements with advantages for quick motions.
- Sensor configurations: GNSS provides high-precision outdoor positioning, while a sky-pointing camera can monitor satellite availability.
B. Existing benchmark datasets
Existing SLAM benchmarks are often tailored to autonomous cars, aerial robots, or limited ground-robot settings. M2DGR addresses these gaps with long-term, diverse scenarios and richer sensor information for ground-robot evaluation.
- Most existing SLAM datasets target autonomous driving or UAVs rather than ground robots.
- Existing ground-robot datasets contain only a few specific sensors or particular scenes.
- Some related datasets have short sequences, low image rates, or simulated rather than real-world scenes, limiting evaluation conditions.
- Challenging conditions such as lifts, complete darkness, and indoor–outdoor transitions are seldom included in existing SLAM benchmarks.
- M2DGR provides long-term trajectories in diverse real-world scenarios with a rich pool of sensory information for comparing algorithm designs.
A. Acquisition platform
The authors built a three-layer ground robot for synchronized, high-throughput multi-sensor data collection. Computing, power, and display components occupy the bottom layer, while sensors occupy the upper layers.
- The data-collection robot has three layers containing computing and power components below, with different sensors mounted on the middle and top layers.Data are recorded on a high-end laptop with a high-speed NVMe SSD.
B. Sensor setup
The platform combines panoramic vision, depth, thermal, event, inertial, and positioning sensors. Ground-truth tracking uses GNSS-IMU outdoors and motion capture or laser tracking indoors.
- Six fisheye cameras provide a 360° view through forward, rear, and side-facing configurations.
- The sensor suite includes a 32-beam Velodyne LiDAR, infrared camera, VI-sensor, event camera, IMU, and GNSS receivers.
- A GNSS-IMU navigation system tracks outdoor sequences, while motion capture and laser tracking provide indoor trajectory references.
C. Calibration and synchronization
M2DGR calibrates sensor intrinsics and inter-sensor poses with established camera, IMU, LiDAR, and calibration-toolbox procedures. Sensor timestamps use software synchronization, achieving less than 10 ms camera-to-camera timing error.
- Calibration: Camera, fisheye, infrared, and IMU intrinsics are calibrated using specialized models, a heated checkerboard, and calibration toolboxes.
- Calibration: LiDAR is the reference frame for extrinsic calibration between LiDAR, IMU, and cameras.
- Synchronization: Less than 10 ms synchronization is achieved between cameras by software triggering and shared system timestamps.
- Evaluation: Evaluation uses ATE after estimated trajectories are aligned with ground truth using EVO.
D. Data collection
M2DGR was recorded across outdoor, indoor, mixed indoor–outdoor, and lift-transition scenarios, using environment-specific procedures and ground-truth systems. The sequences were designed to test loop closing, indoor–outdoor transitions, and floor changes.
- Outdoor environments: Outdoor sequences were collected on the Shanghai Jiao Tong University campus, including repeated circular routes for testing visual-SLAM loop closing.GNSS-RTK provided high-accuracy trajectory ground truth under good satellite visibility.
- Indoor environments: Indoor sequences used motion capture in a room and laser tracking elsewhere, with complete-darkness recordings included to test SLAM robustness.The motion-capture system used twelve high-speed tracking cameras operating at 50 Hz.
- Indoor–outdoor switching: Mixed sequences moved from GNSS-covered outdoor areas through a door into a hall and back outdoors to evaluate indoor–outdoor localization transitions.These sequences evaluated both SLAM and GNSS positioning at the boundary between indoor and outdoor environments.
- Lift transitions: Lift sequences carried the robot from a first-floor hall to a second-floor hall to test localization while entering and leaving a lift.The robot first traveled around the hall, entered the lift, and was carried to another floor.
E. Data usage and tools
The dataset was recorded as ROS rosbags with synchronized topics for its cameras, VI-sensor, event camera, infrared camera, IMU, and GNSS data. Export scripts, ground-truth trajectories, calibration results, and evaluation instructions accompany the release.
- Recorded topics: ROS rosbags record topics for the RGB cameras, VI-sensor, event camera, infrared camera, IMU, and GNSS measurements.The listed topics include seven RGB image streams, VI-sensor imagery, event data, thermal imagery, IMU data, and raw GNSS outputs.
- Recorded topics: The RGB camera topics cover left, right, third, fourth, fifth, sixth, and head image streams.Each stream is listed with raw/compressed recording variants where provided.
- Recorded topics: Additional topics include the VI-sensor color image, event-camera data, infrared images, IMU measurements, and raw GNSS messages.The GNSS entries include aidalm, rxmraw, fix, and navstatus topics.
- Tools: The release provides scripts for exporting data to other formats, ground-truth trajectories and calibration results for each sequence, and detailed SLAM-evaluation instructions.These tools support both data conversion and reproducible algorithm evaluation.
IV. EVALUATION
The evaluation tests visual and LiDAR SLAM systems on seven representative sequences using aligned Absolute Trajectory Error. Results show complementary strengths but substantial failures in low illumination, lift transitions, indoor–outdoor switching, dynamic motion, and multi-camera matching.
- Evaluation setup: Seven representative sequences were evaluated with Absolute Trajectory Error after estimated trajectories were aligned to ground truth using EVO.The tested systems included ORB-SLAM3 variants, CubemapSLAM, Multicol-SLAM, VINS-Mono, A-LOAM, LeGO-LOAM, LINS, and LIO-SAM.
- Overall results: LiDAR-based methods generally outperform vision-based methods, especially outdoors at large scale, but both categories fail in particular cases.Estimated trajectories are reported quantitatively in Table V and visualized in the ENU frame.
- Low illumination: Low illumination causes pinhole and fisheye ORB-SLAM3 to fail, whereas thermal-infrared ORB-SLAM3 is more robust but can lose texture on some objects.Thermal cameras distinguish objects under low visibility, but colorful flat curtains may appear textureless.
- Entering and leaving the lift: None of the tested systems tracked the complete lift trajectory or reconstructed a complete map, with LIO-SAM drifting severely after lift entry.The reported LIO-SAM failure is attributed to a mismatch between IMU pre-integration and LiDAR odometry.
- Outdoor–indoor switching: During outdoor–indoor switching, all SLAM systems completed the sequence, but pure-visual methods produced large errors as GNSS signals degraded near the door.RTKLIB’s indoor GNSS localization failed despite an apparently normal APE, reflecting the loss of usable satellite visibility indoors.
- Dynamic motion and multiple cameras: Abrupt zigzag motion caused most visual methods to fail and produced large ATE errors for LiDAR SLAM, while Multicol-SLAM lost track in almost every sequence.The multi-camera failure may result from false feature matches in highly distorted fisheye images.
V. CONCLUSION
M2DGR is released as a large-scale multi-sensor ground-robot dataset for localization and mapping, intended to support multi-sensor-fusion research. The authors also evaluate existing systems, analyze their limitations across scenarios, and plan continued expansion toward a comprehensive benchmark.
- Dataset release: M2DGR provides a large pool of sensory information for ground-robot localization and mapping and encourages research on multi-sensor fusion.The dataset is explicitly released for SLAM research on ground robots.
- Benchmarking: The paper evaluates state-of-the-art SLAM systems and analyzes their defects and limitations in different scenarios.The analysis is intended to point out potential directions for further SLAM development.
- Future work: The project is planned for ongoing updates and extensions toward a comprehensive ground-robot SLAM benchmark similar to KITTI.The stated future work is to extend the project over time.