Source-linked AI summary
Boreas: A Multi-Season Autonomous Driving Dataset
Keenan Burnett, David J. Yoon, Yuchen Wu, Andrew Zou Li, Haowei Zhang, Shichen Lu, Jingxing Qian, Wei-Kang Tseng, Andrew Lambert, Keith Y. K. Leung, Angela P. Schoellig, Timothy D. Barfoot
TL;DR
Autonomous-driving datasets often provide limited seasonal and adverse-weather variation, while many systems depend on maps that can degrade seasonally. Boreas addresses this gap with a year-long, repeated-route multimodal dataset, accurate post-processed poses, and benchmarks spanning odometry, metric localization, and 3D object detection. Its main scope boundaries are sensor asynchrony and pose-error increases in urban canyons or poor satellite visibility.
Problem
Autonomous-driving research needs datasets that evaluate reliability across adverse weather and seasonal variation rather than primarily ideal conditions and stable maps.
Method
Boreas collects over 350km on a repeated route over one year using lidar, camera, radar, and post-processed GNSS/IMU/wheel-encoder ground truth poses.
Results
The dataset supports benchmarks for odometry, metric localization, and 3D object detection, including 3D object labels for a sunny-weather subset.
Takeaways & Limitations
Boreas enables research on long-term localization across seasons and adverse weather and comparisons among vision-, lidar-, and radar-based pipelines.
Takeaways & Limitations
Measurements are asynchronous and can exhibit motion distortion, while residual RMS position error is typically below 5cm nominally but can reach 20–40cm in urban canyons.
Abstract
from arXiv · showhide
The Boreas dataset was collected by driving a repeated route over the course of one year, resulting in stark seasonal variations and adverse weather conditions such as rain and falling snow. In total, the Boreas dataset includes over 350km of driving data featuring a 128-channel Velodyne Alpha Prime lidar, a 360$^\circ$ Navtech CIR304-H scanning radar, a 5MP FLIR Blackfly S camera, and centimetre-accurate post-processed ground truth poses. Our dataset will support live leaderboards for odometry, metric localization, and 3D object detection. The dataset and development kit are available at https://www.boreas.utias.utoronto.ca
1 Introduction
Boreas addresses the limited reliability of autonomous driving in adverse weather and dependence on costly, seasonally degradable maps. It provides a year-long, multimodal dataset with accurate poses and benchmarks for odometry, metric localization, and 3D object detection.
- Motivation: Boreas targets autonomous driving research under rain, snow, and seasonal variation, where many systems remain outside their operational envelope.The paper also identifies reliance on highly accurate maps that are costly to maintain and may degrade seasonally.
- Dataset: Over 350km of data were collected on a repeated route over one year with stark seasonal variations and multiple adverse-weather sequences.The collection includes rain and falling snow.
- Dataset: The platform combines a 128-beam lidar, 5MP camera, 360° scanning radar, and post-processed GNSS/IMU-based centimetre-accurate ground truth poses.The sensor configuration is identified as a high-quality suite, and the poses are globally consistent.
- Benchmarks: Boreas supports benchmarks for odometry, metric localization, and 3D object detection through a live, open leaderboard.The contribution list also specifies 3D object labels collected in sunny weather.
2 Related Work
Existing autonomous-driving datasets often lack seasonal and weather variation or radar data, limiting evaluation of sensing robustness. Boreas supports comparisons of lidar- and radar-based localization across varied weather conditions.
- Dataset gaps: Many published datasets focus on perception but lack substantial weather and seasonal variation, and many do not provide radar data.The paper notes that automotive radar is robust to precipitation, dust, and fog because of its longer wavelength.
- Boreas: Boreas combines a high-density 128-beam lidar with a 360° scanning radar and significant weather variation for localization comparisons.The paper states that previous datasets could not support these comparisons because they lacked radar or sufficient weather variation.
- Comparison: Compared with RADIATE, Boreas focuses on localization, is larger, includes repeated route traversals, and provides higher-quality localization ground truth.The comparison passage also reports higher-resolution radar, lidar, and camera data.
3 Data Collection
Boreas was collected primarily by repeatedly driving a route near the University of Toronto over one year. The collection captures seasonal changes across sequences ordered chronologically, though the sequence spacing is uneven.
- Routes: The primary Boreas collection repeatedly traversed the Glen Shields route near the University of Toronto over one year.Additional routes were collected as standalone sequences or with a small number of repeated traversals.
- Seasonal coverage: Figure 2 orders camera images chronologically from November 2020 through November 2021 to depict seasonal changes.Each image represents a camera image taken on a different day.
- Seasonal coverage: The sequences shown in the seasonal overview are not evenly spaced in time.
4 Sensors
The dataset documentation provides detailed sensor specifications and figures showing how the sensors are positioned on the Boreas platform.
- Sensor specifications: Table 2 provides detailed specifications for the sensors used in Boreas.
- Sensor placement: Figures 7 and 8 illustrate the placement of the different sensors on Boreas.
5 Dataset Format
Boreas organizes each drive as a timestamped sequence containing synchronized sensor streams, poses, calibration, and metadata, while documenting sensor-specific formats and motion-related limitations.
- 5.1 Data Organization: Each Boreas sequence contains all sensor data and ground truth poses from one drive and is named using its collection date and time.The sequence format is boreas-YYYY-MM-DD-HH-MM.
- 5.2 Timestamps: Sensor timestamps use UNIX epoch microseconds and are synchronized to UTC through the Applanix POS LV, PPS/NMEA links, camera pulses, and radar NTP.The lidar uses a hardwired PPS/NMEA connection, while the radar derives UTC through the synchronized recording computer.
- 5.2 Timestamps: Figure 3 contrasts weather effects: falling snow adds detections to lidar pointclouds, whereas radar remains relatively unperturbed.The comparison spans falling snow, rain, and sun conditions.
- 5.2 Timestamps: Lidar and radar scan timestamps represent scan middles, while individual lidar points and radar azimuths retain their own relative or epoch timestamps.Radar azimuth timestamps use the same UNIX epoch format as filenames.
- 5.3 File Formats: Camera images are rectified and anonymized PNGs, lidar clouds use compact binary records, and radar scans use 2D polar images with embedded timestamp and encoder metadata.Lidar points contain position, intensity, laser ID, and point time; radar data is organized by azimuths and range bins.
- 5.3 File Formats: The dataset includes figures documenting the Glen Shields route, metadata-tag frequencies, sensor configuration, approximate sensor placement, and per-sequence organization.Sensor-placement distances are approximate; precise measurements are available through calibrated extrinsics.
- 5.3 File Formats: Measurements sharing an index are not necessarily synchronous, but timestamps and relative poses permit fusion across sensors.Lidar clouds are not motion-corrected, and radar scans contain motion and Doppler distortion; the devkit provides or references compensation methods.
- 5.3 File Formats: The format materials also include pose-file entries for camera, IMU, post-processed GPS, lidar, radar, and calibration data.These entries appear alongside the dataset’s sensor and sequence organization.
6 Ground Truth Poses
Boreas provides post-processed sensor poses from GNSS, IMU, wheel encoders, and RTX-corrected Applanix processing, expressed in a fixed ENU reference frame. Nominal residual position error is below 5 cm, but global estimates can degrade in urban canyons and relative short-horizon poses are more accurate.
- Ground truth poses combine GNSS, IMU, wheel encoder, and RTX-corrected Applanix POSPac measurements.
- Positions and velocities are expressed in a fixed East-North-Up reference frame aligned to the first sequence pose.The frame uses x East, y North, and z up under the WGS-84 geoid convention.
- Each sensor pose record contains timestamp, position, velocity, roll, pitch, yaw, and angular velocity.Post-processed IMU measurements are also provided at 200 Hz in the Applanix frame.
- Residual RMS position error is typically less than 5 cm in nominal conditions but can reach 20–40 cm in urban canyons.The estimate varies with satellite visibility.
- Relative pose estimates are more accurate over short time horizons than the reported global estimates.
7 Calibration
Boreas calibrates camera–lidar and lidar–radar relationships and provides the resulting transforms for sensor fusion. Lidar points are transformed into the camera frame before projection into rectified images.
- Camera–lidar calibration: Camera intrinsics are calibrated with MATLAB’s camera calibrator, and rectified images use the stored camera matrix P.Calibration files include camera0 intrinsics.yaml and P camera.txt.
- Camera–lidar calibration: Lidar points are transformed from the lidar frame into the camera frame using time-specific camera and lidar poses.The transform is Tcl = T−1ec Tel, followed by xc = Tclxl.
- Camera–lidar calibration: Projected image coordinates are computed from the transformed camera-frame points using the rectified camera model.
- Lidar–radar calibration: Lidar–radar rotation is estimated by Fourier Mellin correlative scan matching across stationary lidar–radar pairs, then averaged.
- Calibration outputs: Extrinsic calibrations are supplied as 4x4 homogeneous transformation matrices under the calib/ folder.
8 3D Annotations
Boreas-Objects-V1 supplies 3D bounding-box annotations for sunny-weather data, organized into continuous training and held-out test scenes. Labels include tracked object identities, semantic types, box dimensions, positions, and yaw in the lidar frame.
- Dataset scale: 7111 lidar frames annotated at 5 Hz produce 326,180 unique 3D box annotations.Because lidar was collected at 10 Hz, annotations can be interpolated between frames at slightly lower fidelity.
- Dataset split: The annotations cover 53 continuous scenes, divided into 37 training scenes and 16 test scenes with withheld ground-truth labels.Each scene lasts 20–70 seconds.
- Label format: Each label row stores [uuid, type, dx, dy, dz, x, y, z, yaw].The uuid remains consistent across frames within a scene, while box location and orientation use the lidar frame.
- Label taxonomy: Semantic types are Car, Cyclist, Pedestrian, and Misc.Car includes several vehicle categories, Cyclist includes motorcycle riders, and Misc includes buses, industrial trucks, streetcars, and trains.
- Annotation overview: Figures report annotation statistics and show examples of the 3D boxes.
9 Benchmark Metrics
Boreas benchmarks odometry, metric localization, and 3D object detection using defined evaluation protocols. The localization benchmark evaluates relative pose accuracy and uncertainty consistency without GNSS or future sensor information.
- The benchmark supports odometry, metric localization, and 3D object detection.
- Metric localization: Metric localization builds a map from training traversals, then localizes live test data against that pre-built map.
- Metric localization: GNSS is unavailable during testing, and methods cannot use future sensor information acausally.
- Metric localization: Users choose lidar or camera as the 3D reference sensor, while radar frames are the sole reference for 2D localization.
- Metric localization: Localization errors include lateral, longitudinal, vertical, and rotational components computed from estimated and ground-truth poses.
- Metric localization: A consistency score near 1 is ideal; values below 1 indicate over-confidence, while values above 1 indicate conservatism.
10 Development Kit
The Boreas development kit wraps the dataset for Python-based access and provides tutorials and evaluation support. It helps users query sensor frames, ground truth, and benchmark outputs.
- The devkit provides a Python wrapper for querying frames and associated ground truth across odometry, localization, and 3D object detection.
- Introductory Jupyter tutorials demonstrate projecting lidar onto camera images and visualizing 3D bounding boxes.
- Evaluation scripts let users validate algorithms before submitting them to the benchmark.
11 Conclusion
Boreas is a multi-season autonomous driving dataset spanning over 350km with a high-quality multimodal sensor suite and accurate ground truth. It targets long-term localization across seasons and adverse weather while supporting online leaderboards for three tasks.
- Boreas contains over 350km of driving data collected over one year.
- Its sensor suite combines a 128-beam lidar, 5MP camera, and 360° radar with accurate Applanix-derived ground-truth poses.
- The dataset includes 3D object labels for a subset of data collected in sunny weather.
- Its primary purpose is research on long-term localization across seasons and adverse weather conditions.
- The website provides online leaderboards for odometry, metric localization, and 3D object detection.