PaperlayerView exact revision on arXiv

Source-linked AI summary

Canadian Adverse Driving Conditions Dataset

Matthew Pitropov, Danson Garcia, Jason Rebello, Michael Smart, Carlos Wang, Krzysztof Czarnecki, Steven Waslander

arXiv:2001.10117v3cs.CV

TL;DR

Autonomous driving perception remains difficult in snowfall because adverse weather degrades sensor returns and obscures the environment. CADC responds with an annotated multimodal winter-driving dataset collected by the Autonomoose in Waterloo, including synchronized cameras, lidar, and positioning data. Its varied snowfall and road-coverage conditions support evaluation of object detection and localization and mapping techniques.

  • Problem

    Snowfall degrades sensor returns and obscures surroundings, challenging autonomous-driving perception tasks across changing winter environments.

  • Method

    The paper constructs CADC by collecting annotated lidar and image data with synchronized sensors on the Autonomoose during winter driving in Waterloo.

  • Results

    CADC provides 75 driving sequences spanning over 20 km, with varying traffic, snowfall intensity, and road snow coverage.

  • Takeaways & Limitations

    The dataset enables researchers to test object-detection and localization-and-mapping techniques under challenging winter weather.

Abstract

from arXiv · show

The Canadian Adverse Driving Conditions (CADC) dataset was collected with the Autonomoose autonomous vehicle platform, based on a modified Lincoln MKZ. The dataset, collected during winter within the Region of Waterloo, Canada, is the first autonomous vehicle dataset that focuses on adverse driving conditions specifically. It contains 7,000 frames collected through a variety of winter weather conditions of annotated data from 8 cameras (Ximea MQ013CG-E2), Lidar (VLP-32C) and a GNSS+INS system (Novatel OEM638). The sensors are time synchronized and calibrated with the intrinsic and extrinsic calibrations included in the dataset. Lidar frame annotations that represent ground truth for 3D object detection and tracking have been provided by Scale AI.

I. INTRODUCTION

CADC addresses the challenge of autonomous driving in adverse winter weather by providing multimodal, annotated driving data from the Waterloo region. It offers varied snowfall and traffic conditions alongside a sensor suite intended for perception research.

  • Snowfall degrades sensor returns and obscures surroundings, making localization, mapping, detection, segmentation, tracking, and prediction more difficult.
  • CADC contains 75 driving sequences covering over 20 km across varying traffic and snowfall levels, collected over three days in Waterloo during March 2018 and February 2019.
  • Earlier multimodal datasets generally emphasized clear or diverse non-snow weather, forward-facing views, localization, or mapping rather than snowy object annotations.

A. Time Synchronization

The Autonomoose synchronizes sensor data through GPS-derived timing signals, computer clocks, and hardware triggering. Lidar and cameras are aligned around a shared phase-lock timing scheme, while other vehicle devices use computer timestamps.

  • GPS Timestamps: GPS provides NMEA, PPS, and 10 Hz VARF signals used to synchronize the computer, lidar, Xsens IMUs, and eight cameras.VARF hardware-triggers camera acquisition, while PPS and NMEA signals are distributed to timing-capable devices.
  • Computer Timestamps: The computer derives a GPS reference clock from NMEA and PPS inputs, then uses Chrony to synchronize its system clock and correct clock drift.
  • Lidar Timestamps: Each lidar point cloud is timestamped at a phase-lock angle, and motion correction transforms left-hemisphere points forward and right-hemisphere points backward in time.
  • Cameras Timestamps: All eight cameras are hardware-triggered at 10 Hz when lidar reaches the phase-lock angle, but their timestamps are assigned in ROS and truncated to the VARF period.The camera timestamp is truncated within 0.1 s to better represent image acquisition time.
  • Vehicle-control and Xsens messages use computer-based timestamps because their devices cannot directly receive the relevant GPS timing signals.

IV. SENSOR FRAMES

The Autonomoose sensor-frame convention defines vehicle, GPS/IMU, lidar, camera, and IMU coordinate placements and orientations. Calibration materials document the sensor-frame relationships needed to use the collected data.

  • The base-link frame originates at the rear axle center, with x forward, y left, and z upward.
  • The GPS/IMU frame is inside the trunk and rotated 90° clockwise about z relative to the base-link frame.
  • The lidar frame matches the base-link orientation at the lidar optical center, while each camera frame is located at its optical center.The camera axes point with z toward the image plane, x right, and y down; three forward cameras form a trinocular configuration.
  • The calibration archive includes files documenting camera intrinsics and extrinsics between sensor frames.

A. Camera intrinsics

CADC supplies camera intrinsic and sensor-extrinsic calibration information for using its multimodal data. The camera files store imaging parameters, while extrinsics.yaml stores transformations between specified sensor frames.

  • Eight camera YAML files store each camera’s name, image dimensions, camera matrix, distortion model, and distortion coefficients.
  • The camera matrix K contains focal lengths fx and fy, skew s, and optical-center coordinates cx and cy.
  • CADC provides Brown-Conrady distortion coefficients, with k1, k2, and k3 as radial terms and p1 and p2 as tangential terms.
  • The extrinsics archive contains 4x4 homogeneous transforms linking base link, lidar, GPS, and camera frames.It includes direct camera-to-lidar transforms for convenience and a measured lidar-to-base-link transform.

VI. DATASET FORMAT

CADC organizes each drive into raw sensor outputs, sampled labeled data, 3D annotations, and daily calibration files. The format exposes synchronized camera, lidar, GPS/IMU, vehicle-control, and Xsens data through distinct downloadable archives and sensor folders.

  • Each drive provides full-rate raw outputs, approximately 3 Hz labeled data, corresponding 3D annotations, and calibration data for its recording day.
  • The downloadable structure includes labeled.zip, raw.zip, 3d ann.json, and calib.zip.
  • Figure 6 depicts the folder structure used to download and organize CADC data.
  • Camera images are PNG files, lidar point clouds are binary files, and other sensor outputs are space-separated text files documented by dataformat.txt.
  • The labeled archive contains data from eight cameras, one lidar, and one GPS/IMU source.
  • Raw data archives contain outputs from all available Autonomoose sensors.
  • Raw sensor folders cover camera images, lidar, Novatel GPS/IMU, vehicle control, and Xsens IMUs.

A. Image data

The image-data section specifies the storage and timing conventions for camera and navigation streams, including missing-image handling and motion-corrected lidar products. Sensor messages retain detailed position, orientation, uncertainty, and inertial measurements.

  • There are eight camera image folders, image 00 through image 07, with 15 missing camera-00 images excluded from labeled data in sequence 0066.
  • Original camera images are stored as 1280x1024 PNG files of approximately 2 MB each.
  • Each lidar point-cloud file is timestamped when the lidar beams pass the phase-lock angle.
  • Motion-corrected lidar points compensate for vehicle-motion distortion using post-processed GPS/IMU data at the phase-lock timestamp.
  • Novatel outputs provide current position, velocity, orientation, best position, RTK estimates, and pose uncertainty.
  • Novatel IMU messages contain gravity-, earth-rotation-, and sensor-error-corrected orientation and acceleration changes, sampled at 100 Hz.

D. Vehicle control data

CADC includes vehicle-control and auxiliary inertial data alongside synchronized labeled sensor streams. These records describe vehicle actuation, diverse platform telemetry, and corrected or undistorted labeled modalities.

  • Raw sensor data include braking, gear, steering, surround, throttle, wheel speed, turn-signal, and miscellaneous vehicle reports.
  • Vehicle-control messages are decoded from the CAN bus by the Dataspeed ADAS kit and transmitted over USB with computer timestamps.
  • Xsens 300 and Xsens 30 IMU streams contain orientation, angular velocity, linear acceleration, and covariance matrices.
  • Xsens velocity streams contain linear and angular velocity.
  • Labeled data are downsampled to one frame every 300 ms, yielding approximately 50–100 frames per 15–30-second drive.
  • Labeled images are undistorted, lidar points are motion-corrected, and Novatel data use post-processed RTK outputs.

A. cuboid

CADC cuboid annotations identify tracked objects across frames and encode their camera source, lidar-frame geometry, motion state, point count, labels, and attributes. The annotation taxonomy includes multiple object categories but uses one broad Car class without unique attributes.

  • A. cuboid: Each cuboid receives a UUID that identifies it across all frames within a drive.
  • A. cuboid: Cuboid annotations record the labeling camera, lidar-frame position, dimensions, yaw, stationary status, and lidar-point count.
  • A. cuboid: Each 3D annotation JSON file stores frame lists containing detected cuboids and an example cuboid structure.
  • A. cuboid: The taxonomy includes Car, Truck, Bus, Bicycle, Horse and Buggy, Pedestrian, Pedestrian with Object, Animal, Garbage Container on Wheels, and Traffic Guidance Object.
  • A. cuboid: Figures summarize the geographic collection map, cuboid counts per frame, unique labeled objects, and total object instances.
  • A. cuboid: Vehicle classes carry Parked, Stopped, and Moving attributes in every frame, while nonvehicle categories use category-specific attributes.
  • A. cuboid: The Car label combines cars, SUVs, and vans without unique attributes, leaving vehicle body sizes undivided.

X. DATASET STATISTICS

The dataset statistics characterize geographic coverage, data volumes, annotation distributions, snowfall intensity, and road-surface conditions. They also report storage characteristics and object-label summaries.

  • GPS points show the full dataset and labeled subset geographically, with red marking all data and green marking labeled data.
  • Raw and labeled data have similar compressed and uncompressed sizes because images dominate sensor storage and are already losslessly compressed.
  • Annotation statistics include frame-level annotation counts, unique objects by label, object persistence across frames, and vehicle and pedestrian attributes.
  • Snowfall levels were assigned using DROR-filtered lidar point counts, with bins spanning light, medium, heavy, and extreme snowfall.The reported bins are Light (25-249), Medium (250-499), Heavy (500-749), and Extreme (750-1500).
  • 18 driving sequences had bare roads, while 57 had snow covering the road.

XI. DEVELOPMENT KIT

The development kit provides Python tools for inspecting vehicle paths and dataset data. Its path tool converts drive GPS messages into an ENU frame and plots them as axis frames.

  • The Python development kit can view vehicle paths, project lidar onto images, and project 3D annotations onto images and lidar.
  • The vehicle-path script loads all GPS messages from a drive and converts them to an ENU frame.
  • The script uses the first GPS message as the ENU origin and plots each subsequent message as an axis frame.

B. run demo lidar.py

The lidar demonstration loads synchronized lidar and camera data with calibration, then projects lidar points or 3D annotation cuboids into visual views. Additional examples show snowfall conditions and vehicle-path outputs.

  • B. run demo lidar.py: The lidar script loads a camera image, its corresponding lidar file, and calibration data before projecting lidar points onto the image.Projected point color is scaled by depth.
  • B. run demo lidar.py: Snowfall examples pair top-down lidar views with front-camera images for light, medium, heavy, and extreme snowfall.
  • B. run demo lidar.py: The demo vehicle-path output shows Autonomoose following a path while making a left-hand turn on drive 0027.
  • B. run demo lidar.py: The lidar demonstration includes a front-camera projection of lidar points and birds-eye views with 3D cuboid boxes overlaid.
Loading 2001.10117v3…