Source-linked AI summary
EU Long-term Dataset with Multiple Sensors for Autonomous Driving
Zhi Yan, Li Sun, Tomas Krajnik, Yassine Ruichek
TL;DR
Autonomous-vehicle perception and localization depend on sensors and their data processing, while long-term autonomy involves localization, mapping, and prediction in changing environments. This paper presents an eleven-sensor platform and a publicly available ROS-based dataset, recorded across varied driving conditions and intended especially for long-term autonomy research.
Problem
Perception and localization are essential to autonomous driving, and both are affected by the sensors used and how their data are processed.
Source: §I · p. 1Method
Results
Takeaways & Limitations
The dataset is available to the community for research such as persistent mapping, long-term prediction, and online or lifelong learning.
Source: §V · p. 8
Abstract
from arXiv · showhide
The field of autonomous driving has grown tremendously over the past few years, along with the rapid progress in sensor technology. One of the major purposes of using sensors is to provide environment perception for vehicle understanding, learning and reasoning, and ultimately interacting with the environment. In this paper, we first introduce a multisensor platform allowing vehicle to perceive its surroundings and locate itself in a more efficient and accurate way. The platform integrates eleven heterogeneous sensors including various cameras and lidars, a radar, an IMU (Inertial Measurement Unit), and a GPS-RTK (Global Positioning System / Real-Time Kinematic), while exploits a ROS (Robot Operating System) based software to process the sensory data. Then, we present a new dataset (https://epan-utbm.github.io/utbm_robocar_dataset/) for autonomous driving captured many new research challenges (e.g. highly dynamic environment), and especially for long-term autonomy (e.g. creating and maintaining maps), collected with our instrumented vehicle, publicly available to the community.
I. INTRODUCTION
Autonomous-vehicle perception and localization depend on the sensors used and how their data are processed. This paper presents a multisensor platform and ROS-based dataset intended as a practical reference for autonomous-driving research, including long-term autonomy.
I. INTRODUCTION
- A typical autonomous-navigation framework combines sensors, perception and localization, path planning and decision making, and motion control.The vehicle gathers environmental data and self-measurements, then uses them to understand its surroundings and localize in a pre-built map.Source: §I · p. 1
- Perception and localization are essential to safe, reliable autonomous driving, and both are affected by sensor choice and data processing.Perception covers internal and external environmental information; localization includes visual odometry/SLAM, map-based localization, and place recognition or relocalization.Source: §I · p. 1
- The paper presents an eleven-sensor platform and a ROS-based dataset with calibrated sensors, approximate synchronization, GPS-RTK ground-truth trajectories, and lidar-odometry baselines.The dataset includes urban and suburban driving challenges and daily and seasonal changes, making it especially suitable for long-term autonomy research.Source: §I · p. 2
- The paper aims to provide a practical reference for readers facing the design, budgeting, and implementation challenges of autonomous vehicles.Its account draws on the authors’ experience building an autonomous car, particularly the multisensor platform.Source: §I · p. 2
II. THE PLATFORM
No single sensor type is sufficient for efficient autonomous-driving perception and localization. The section motivates combining sensors to draw on their different strengths while accounting for their limitations.
II. THE PLATFORM
- GNSS can navigate in all weather, but its update frequency and accuracy may fall short of autonomous-driving needs, and urban structures can obstruct its signals.The cited examples include urban canyons, tunnels, and underground parking lots.Source: §II · p. 2
- Lidars, cameras, and radar have different limitations: lidar can struggle with semantic perception and adverse weather, cameras with illumination changes, and radar with range accuracy.The passage also notes lidar scan distortion at high vehicle speed and the limited field of view of 2D lidar.Source: §II · p. 2
- No single sensor type is sufficient for efficient autonomous-driving perception and localization, motivating multisensor systems that complement individual sensors.Table I summarizes typical advantages and disadvantages of commonly used sensors.Source: §II · p. 2, Table I · p. 2
A. Hardware
The platform’s hardware is configured to broaden sensor coverage and maximize overlap between sensors. Its choices balance perception needs, safety, and hardware cost.
A. Hardware
- The hardware design seeks to strengthen the vehicle’s visual scope and maximize the area perceived by multiple sensors.The sensor configuration and visual scope are shown in Figs. 2 and 3.Source: A. Hardware · p. 2, Figure 2 · p. 3, Figure 3 · p. 3
- Front- and rear-facing stereo cameras use global shutters, which expose every pixel at the same instant and are advantageous at high vehicle speeds.The front camera is a Bumblebee XB3 and the rear camera a Bumblebee2.Source: A. Hardware · pp. 2–3
- The platform uses phase-locking to reduce interference between its nearby Velodyne lidars and post-processes data to remove shadows behind them.The authors note that one lidar may pick up a reflection intended for the other.Source: A. Hardware · p. 3
- Two lateral-facing fisheye cameras provide a 185° view, extending perception on both sides and adding semantic information to the Velodyne lidars.They are mounted midway along the roof.Source: A. Hardware · p. 3
- A front-mounted ibeo lidar and radar are used together to support safety around the vehicle’s front, with the radar and lidar cross-checking object detections.The radar is described as reliable for detecting moving-object velocity, though less angularly accurate than lidar.Source: A. Hardware · p. 3
- A downward-tilted 2D lidar scans the road for markings and boundaries, complementing the ibeo lidar’s role in detecting dynamic objects.The SICK LMS100-10000 provides a 270° field of view.Source: A. Hardware · p. 3
- The system includes a roof-mounted GNSS receiver with two antennas and an in-vehicle IMU that outputs acceleration, angular velocity, and orientation.RTK enhancement is used to reduce GPS positioning error from meters to centimeters.Source: A. Hardware · p. 3
- Two side-by-side Velodyne 32-layer lidars were chosen as a trade-off between sensing efficiency and hardware cost.The authors contrast this arrangement with a single roof-mounted lidar and more expensive 64-layer models.Source: A. Hardware · pp. 3–4
B. Software
The software system uses ROS for data collection, with sensors connected to a workstation and ROS nodes running locally. Raw data are recorded for offline post-processing.
B. Software
- Local ROS nodes timestamp the sensor data to maximize software-level synchronization, and CPU-only recording discovered no data delay.The collection records raw data and leaves post-processing to offline playback.Source: B. Software · p. 4
- The system provides raw data from different devices so users can perform advanced processing, including motion compensation.The paper focuses on vehicle-perception experience using ROS-1, which the authors say can serve as a reference for ROS-2.Source: B. Software · p. 4
1) Sensor Calibration:
The cameras and lidars were calibrated intrinsically, with the stereo cameras also calibrated relative to the Velodyne lidars. The paper describes the calibration procedures and notes that hardware-level synchronization is outside its scope.
1) Sensor Calibration:
- Camera intrinsics were calibrated with a chessboard and ROS camera calibration package, while lidar intrinsics used factory parameters.The stereo cameras were also calibrated relative to the Velodyne lidars.Source: 1) Sensor Calibration · p. 4
- Lidar extrinsics were estimated by minimizing voxel-wise L2 distance between sensor point clouds while driving in a structured environment with landmarks.The stereo-camera-to-Velodyne transform was manually aligned using point clouds of two building walls and the ground.Source: 1) Sensor Calibration · p. 4
- Hardware-level synchronization was beyond the scope of the paper.Source: 1) Sensor Calibration · p. 4
- The calibrated lidar and stereo-camera point clouds are shown aligned in the ROS Rviz visualization.The image also displays data from the Velodyne, SICK, ibeo LUX, and stereo cameras.Source: Figure 5 · p. 4
2) Configuration of two Velodyne lidars:
The two side-by-side Velodyne lidars were phase-locked with different offsets, and an equation was used to remove spurious data from inter-sensor blockage or reflections. Their packets were directed to specific workstation IP ports to avoid network congestion.
2) Configuration of two Velodyne lidars:
- The left and right Velodyne lidars were assigned phase-lock offsets of 90° and 270°, respectively, using the GPS receiver's PPS signal.The offsets synchronize their relative rotational positions.Source: 2) Configuration of two Velodyne lidars · p. 4
- The subtended-angle equation was used to remove spurious data caused by blockage or reflections from the opposing lidar.The relevant quantities are the far sensor's diameter and the distance between sensor centers.Source: 2) Configuration of two Velodyne lidars · pp. 4–5
- Each lidar transmitted packets to a specific, non-broadcast workstation IP address through a unique port to avoid network congestion.The configuration also applied to the SICK and ibeo LUX lidars.Source: 2) Configuration of two Velodyne lidars · p. 5
III. DATASET
The dataset was recorded in downtown and suburban Montbéliard, France, with routes selected to capture varied driving conditions. Its long-term data follows one route across repeated rounds and times, while separate recordings cover ten roundabouts.
III. DATASET
- Collection used a human-driven vehicle with ADAS functions disabled, at speeds limited to 50 km/h, and stored all data in rosbag files.The recordings came from downtown and suburban areas; the cited itinerary figure distinguishes the long-term and roundabout routes.Source: §III · p. 5, Figure 8 · p. 5
- The long-term dataset records the same approximately 5 km route for eleven rounds at different times, spanning city, park, residential, commercial, and river-bridge areas.The route includes small and large loops for loop closure and was followed during day, night, weekdays, summer, and winter, including snow.Source: §III · p. 5
- The long-term route was selected to capture periodic daily, weekly, and seasonal environmental changes.It also includes uphill and downhill roads, a shared zone, a road diversion, and highly dynamic or dense traffic.Source: §III · p. 5
- The roundabout recordings cover ten roundabouts of varying sizes across an area of approximately 0.75 km² near the UTBM Montbéliard campus.Source: §III · p. 5
A. Lidar Odometry Benchmarking
The dataset includes lidar-odometry baselines using LOAM Velodyne and LeGO-LOAM. The paper invites comparisons across devices with different computational capabilities but accepts only real-time results.
A. Lidar Odometry Benchmarking
- The dataset provides lidar-odometry baselines using LOAM Velodyne and LeGO-LOAM implementations.LOAM Velodyne is described as a real-time 3D-lidar SLAM method with scan restoration for fast driving; LeGO-LOAM is lightweight and ground-optimized.Source: A. Lidar Odometry Benchmarking · p. 5
- LeGO-LOAM addresses LOAM performance deterioration under limited resources and noisy operating conditions.Its point-cloud segmentation discards potentially unreliable features after ground separation.Source: A. Lidar Odometry Benchmarking · p. 5
- Users are encouraged to compare methods on devices with different computational capabilities, but only real-time results are accepted.Results can be submitted to the baseline GitHub repository.Source: A. Lidar Odometry Benchmarking · p. 5
B. Long-term Autonomy
The paper frames long-term autonomy around robust localization, map maintenance, and prediction of dynamic objects. Its dataset also provides continuous roundabout recordings for studying surrounding-vehicle behavior.
B. Long-term Autonomy
- Long-term autonomy calls for vehicle localization and mapping across changing environments, alongside prediction of dynamic-object occupancy and presence.The paper proposes self-aware localization and liability-aware long-term mapping, including updating landmark variance and assessing scan-map registration.Source: B. Long-term Autonomy · p. 6
- The dataset's multiple driving sessions vary lightness and landmarks, and the authors offer long-term localization, mapping, and dynamic-object prediction as open problems.The paper notes frequency modelling or recurrent neural networks for tracking periodic layouts and human activity patterns.Source: B. Long-term Autonomy · p. 6
C. Roundabout Challenge
- Full-circle roundabout recordings provide continuous data for learning and predicting surrounding vehicles' trajectories.Images can reveal turn signals and wheel steering; the authors aim for research on vehicle behavior prediction and reducing crashes in roundabouts.Source: C. Roundabout Challenge · p. 6
IV. RELATED WORK
The related-work discussion compares autonomous-driving platforms and datasets by sensor configuration, collection scope, and research focus. It highlights limitations in KITTI and Oxford that motivate the paper's multisensor design.
IV. RELATED WORK
- KITTI combines lidar and cameras, producing 10 lidar frames per second with 100k points per frame for localization and 3D object detection.Its visual data also support visual odometry, optical flow, object detection, tracking, and semantic-understanding benchmarks.Source: §IV · p. 6
- Oxford RobotCar provides a 360° fisheye-camera view and was collected over one year and about 1000 km.Its 2D/3D lidars and stereo cameras yield data streams at 11 fps and 16 fps, respectively.Source: §IV · p. 6
- KAIST covers complex urban settings with 13 sensors, while Waymo offers synchronized, calibrated lidar-camera data and the Canadian dataset targets snowfall conditions.The cited Waymo data are exhaustively annotated.Source: §IV · pp. 6, 8
- The paper identifies KITTI's single-session capture and incomplete camera field of view, and Oxford's lack of modern 3D lidar, as limitations.The authors describe their platform as leveraging strengths of both designs while addressing these limitations.Source: §IV · p. 6
- The proposed platform combines four lidars, including two Velodynes, with four cameras to provide stronger range and visual sensing.The paper presents this as its multisensor design response to the earlier platform comparison.Source: §IV · pp. 6–7
- Apollo, Autoware, and Udacity are cited as open-source repositories for sharing and contributing autonomous-driving software.The paper also notes that several other datasets focus on visual perception and release only images and videos.Source: §IV · pp. 6, 8, Table II · p. 7
V. CONCLUSION
The conclusion presents a multisensor platform and a publicly available ROS-based dataset for autonomous-driving research. It emphasizes the dataset's long-term challenges and describes privacy processing for released images.
V. CONCLUSION
- The platform integrates eleven heterogeneous sensors, including lidars, cameras, radar, and GPS/IMU, to enhance visual scope and perception capability.The authors expect sensor heterogeneity to improve localization and situation awareness and ultimately improve autonomous-driving safety.Source: §V · p. 8
- The publicly available ROS-based dataset contains periodic changes and is especially suited to long-term autonomy research, including persistent mapping and long-term prediction.The authors also identify online and lifelong learning as research applications.Source: §V · p. 8
- To protect privacy, the authors post-process camera images to blur faces and license plates under GDPR-related data handling.Images were released successively starting in the first quarter of 2020.Source: §V · p. 8