PaperlayerView exact revision on arXiv

Source-linked AI summary

The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes

Abhishek Patil, Srikanth Malla, Haiming Gang, Yi-Ting Chen

arXiv:1903.01568v1cs.CVcs.RO

TL;DR

Existing datasets lack standardized, full-surround 3D detection and tracking benchmarks for crowded scenes with interacting and occluded traffic participants. The paper introduces H3D, an efficiently labeled LiDAR dataset with a standardized benchmark, and reports detection and tracking results while discussing their error sources.

  • Problem

    Existing datasets provide insufficient data and labels for full-surround 3D multi-object detection and tracking in crowded scenes with highly interactive and occluded traffic participants.

  • Method

    The paper constructs H3D from naturalistic driving data, develops an efficient large-scale labeling methodology, and establishes a standardized benchmark for full-surround 3D detection and tracking.

  • Results

    The evaluated detectors show poor yaw estimation with few object points and pedestrian-detection failures under crowded-scene occlusion, while tracking is also affected by occlusions.

  • Takeaways & Limitations

    H3D provides a large-scale benchmark intended to stimulate research on full-surround 3D multi-object detection and tracking in crowded urban scenes.

Abstract

from arXiv · show

3D multi-object detection and tracking are crucial for traffic scene understanding. However, the community pays less attention to these areas due to the lack of a standardized benchmark dataset to advance the field. Moreover, existing datasets (e.g., KITTI) do not provide sufficient data and labels to tackle challenging scenes where highly interactive and occluded traffic participants are present. To address the issues, we present the Honda Research Institute 3D Dataset (H3D), a large-scale full-surround 3D multi-object detection and tracking dataset collected using a 3D LiDAR scanner. H3D comprises of 160 crowded and highly interactive traffic scenes with a total of 1 million labeled instances in 27,721 frames. With unique dataset size, rich annotations, and complex scenes, H3D is gathered to stimulate research on full-surround 3D multi-object detection and tracking. To effectively and efficiently annotate a large-scale 3D point cloud dataset, we propose a labeling methodology to speed up the overall annotation cycle. A standardized benchmark is created to evaluate full-surround 3D multi-object detection and tracking algorithms. 3D object detection and tracking algorithms are trained and tested on H3D. Finally, sources of errors are discussed for the development of future algorithms.

I. INTRODUCTION

H3D addresses limitations in existing 3D traffic datasets by providing full-surround annotations for crowded, interactive urban scenes. It also introduces an efficient labeling process and standardized benchmark for detection and tracking.

  • Motivation: KITTI provides more than 200k manually labeled 3D objects but only annotates the frontal view and lacks sufficient complexity and label richness.These limitations hinder full-surround reasoning and evaluation in crowded urban scenes with interacting and occluding traffic participants.
  • Dataset goal: H3D targets full-surround 3D multi-object detection and tracking in diverse, rich, and complex traffic scenes selected from four major urban areas.The dataset is gathered from the HDD naturalistic driving dataset in the San Francisco Bay Area.
  • Contributions: H3D introduces a large-scale 3D bounding-box labeling methodology and establishes a standardized full-surround detection and tracking benchmark.The labeling process is designed to speed up the overall annotation cycle.
  • Contributions: 1,071,302 3D bounding box labels cover 8 common traffic participants in H3D.The dataset is presented as a large-scale resource for detection and tracking research.

II. TRAFFIC SCENE DATASETS

Existing traffic datasets provide useful 2D or localization resources, but the reviewed datasets do not supply the 3D boxes and tracks needed for 3D traffic detection and tracking.

  • 2D datasets: Many 2D traffic datasets emphasize semantic segmentation through high-quality labels and scalable dataset-generation methodologies.Cityscapes provides 5,000 finely annotated images and 20,000 coarse images, while Mapillary expands pixel-level annotation scale and diversity.
  • 3D datasets: The reviewed 3D urban dataset contains point clouds from multiple LiDAR devices but lacks 3D bounding boxes and object tracks for traffic participants.The passage also notes that manually labeling large-scale datasets is non-trivial.

III. H3D DATASET

H3D’s dataset-generation pipeline calibrates and synchronizes sensors, corrects and registers point clouds, annotates objects in World coordinates, and converts labels back to sensor coordinates.

  • Preprocessing: Sensor calibration aligns GPS/IMU and LiDAR coordinate systems, while all five sensors are time-synchronized with GPS timestamps.The calibration uses a hand-eye calibration method.
  • Preprocessing: Point-cloud undistortion removes motion artifacts before annotation.This preprocessing step prepares the spinning-LiDAR data for labeling.
  • Coordinate processing: Point-cloud registration estimates ego-vehicle odometry and transforms each scenario into fixed World coordinates.The transformation supports annotation in a common coordinate frame.
  • Annotation: Annotators label objects in World coordinates, after which bounding boxes and point clouds are converted back to Velodyne coordinates and raw point-cloud format.The pipeline separates annotation from the original sensor-coordinate representation.

A. Sensor Setup

The instrumented vehicle combines three cameras, a 3D LiDAR, and an automotive motion analyzer to capture synchronized multimodal data for H3D.

  • Cameras: The vehicle uses three color cameras operating at 30 Hz with 1,920×1,200 resolution and 80°–90° fields of view.The left and right cameras have 90° fields of view, while the center camera has 80°.
  • LiDAR: The Velodyne HDL-64E S2 LiDAR operates at 10 Hz with 64 laser beams, 100 m range, and 26.9° vertical field of view.These specifications define the primary 3D sensing configuration.
  • Motion sensing: An ADMA sensor supplies DGPS-output gyros, accelerometers, and GPS at 100 Hz.Sensor data is recorded on a multi-core Ubuntu workstation with 16 GB of memory and four 2 TB SSDs.

B. Data Collection

Data was collected across four San Francisco Bay Area urban areas using an instrumented vehicle, with synchronized and corrected sensor data registered for annotation.

  • B. Data Collection: Data were collected in four San Francisco Bay Area urban areas from April to September 2017 using an instrumented vehicle.Sensor data were synchronized through ROS via customized hardware.
  • B. Data Collection: Raw spinning-LiDAR point clouds were undistorted using fused GPS/IMU data and linear interpolation.The correction addresses motion distortion before annotation.
  • B. Data Collection: NDT registered each independent point-cloud sequence to its initial World frame for odometry estimation.This registration compensates for unreliable GPS in enclosed urban spaces.

2) Data Annotation:

H3D annotation uses registered point clouds, camera assistance, temporal propagation, and frame-by-frame verification to reduce labeling effort and maintain consistency.

  • 2) Data Annotation:: Registered point clouds let annotators label static objects in one shot and associate objects across frames more easily.Three cameras assist in determining object categories, while bounding boxes and track IDs are annotated on registered clouds.
  • 2) Data Annotation:: Final annotations are verified frame by frame by projecting boxes onto images and visually inspecting them in bird’s-eye view.This combines image-based and BEV quality checks.

D. Statistics

H3D statistics characterize a large, class-balanced benchmark whose traffic-participant density substantially exceeds KITTI under the stated comparison assumption.

  • D. Statistics: 15 times higher traffic-participant density was observed in H3D than in KITTI.The comparison assumes H3D’s 360° annotations contain four times KITTI’s frontal-view annotation count.
  • D. Statistics: H3D reports total bounding-box annotations across multiple traffic-participant classes, with car and pedestrian proportions consistent across dataset splits.The split statistics are summarized in Table I and class distributions in Fig. 5.

IV. 3D DETECTION

The paper evaluates VoxelNet on H3D using a KITTI-like protocol and identifies sparse-point yaw errors and crowded-scene pedestrian occlusion as key challenges.

  • IV. 3D DETECTION: VoxelNet is evaluated on H3D to provide baseline values and assess the dataset’s complexity.H3D enables full 360° object detection in point clouds.
  • IV. 3D DETECTION: Detection evaluation adapts the KITTI protocol, using IoU thresholds of 0.5 for cars and 0.25 for pedestrians.Car and truck classes are combined for car-detection evaluation.
  • IV. 3D DETECTION: Yaw estimation degrades when objects contain few points, while pedestrian detection fails under crowded-scene occlusion.These challenges are highlighted as sources of 3D detection error.

V. 3D MULTI-OBJECT TRACKING

The tracking evaluation uses an Unscented Kalman Filter with Euclidean-centroid data association and reports separate MOT results for cars and pedestrians, including occlusion-related failures.

  • 3D objects are tracked with an Unscented Kalman Filter through prediction, data association, update, and track management.
  • Euclidean distance between object centroids is used for data association.
  • Cars achieve 1.00 MOTA, 1.00 MOTP, 1.00 MT, and 0.00 ML at 0.5 3D IoU.
  • Pedestrians achieve 0.83 MOTA, 1.00 MOTP, 0.77 MT, and 0.11 ML at 0.5 3D IoU.
  • Pedestrian tracking is affected by occlusions, including missing tracks and changes in data association over time.

VI. CONCLUSION

The conclusion presents H3D as a dataset and benchmark for full-surround 3D multi-object detection and tracking in crowded urban scenes, supported by an efficient labeling methodology.

  • H3D targets full-surround 3D multi-object detection and tracking in crowded urban scenes.
  • The authors discuss tracking failure cases involving pedestrian occlusion and data-association changes.
  • Its labeling methodology enables efficient large-scale annotation of 3D objects and their track IDs.
  • The paper establishes a standard benchmark for future 3D point cloud detection and tracking algorithm development.
Loading 1903.01568v1…