PaperlayerView exact revision on arXiv

Source-linked AI summary

Lost and Found: Detecting Small Road Hazards for Self-Driving Vehicles

Peter Pinggera, Sebastian Ramos, Stefan Gehrig, Uwe Franke, Carsten Rother, Rudolf Mester

arXiv:1609.04653v1cs.CVcs.RO

TL;DR

Small road hazards are difficult to detect reliably with vision, yet they are critical for autonomous driving. The paper uses local stereo-image hypothesis tests for free space and obstacles, and reports stronger performance than the evaluated baselines while supporting efficient processing.

  • Problem

    Small hazardous road objects are difficult to detect because they cover very small image areas and vary in shape and appearance, while stereo accuracy decreases at long range.

  • Method

    The method performs statistical hypothesis tests between free-space and obstacle models directly on stereo image data using independent local patches.

  • Results

    The proposed PHT/FPHT approaches significantly outperform both baselines, achieving iInt values of approximately 0.4 at an average of 3 false positives per frame.

  • Takeaways & Limitations

    The Lost and Found dataset and proposed methods provide an evaluated basis for detecting small road hazards across varied locations, road conditions, object types, and sizes.

Abstract

from arXiv · show

Detecting small obstacles on the road ahead is a critical part of the driving task which has to be mastered by fully autonomous cars. In this paper, we present a method based on stereo vision to reliably detect such obstacles from a moving vehicle. The proposed algorithm performs statistical hypothesis tests in disparity space directly on stereo image data, assessing freespace and obstacle hypotheses on independent local patches. This detection approach does not depend on a global road model and handles both static and moving obstacles. For evaluation, we employ a novel lost-cargo image sequence dataset comprising more than two thousand frames with pixelwise annotations of obstacle and free-space and provide a thorough comparison to several stereo-based baseline methods. The dataset will be made available to the community to foster further research on this important topic. The proposed approach outperforms all considered baselines in our evaluations on both pixel and object level and runs at frame rates of up to 20 Hz on 2 mega-pixel stereo imagery. Small obstacles down to the height of 5 cm can successfully be detected at 20 m distance at low false positive rates.

I. INTRODUCTION

The paper targets difficult small-road-hazard detection for autonomous driving, using stereo vision despite its range-accuracy limits. It extends prior geometric detection, introduces the Lost and Found dataset, and situates the work among stereo-based obstacle methods.

  • Motivation: Small road obstacles and debris are safety-critical hazards for autonomous driving.Reported figures include 25,000 debris-related crashes in the US in 2004 and approximately 150 deaths involving lost hazardous cargo in 2011.
  • Motivation: Small hazardous objects are difficult to detect because they occupy few pixels and vary widely in shape and appearance.Cameras offer high spatial resolution at relatively low cost, whereas active range sensors typically have lower resolution and higher cost.
  • Motivation: Stereo systems provide dense range maps but have comparatively low distance accuracy, especially at long range, where timely detection depends on accuracy.
  • Contributions: A tenfold speedup extends prior work while maintaining the reported result quality.The paper also introduces a mid-level obstacle representation based on the original point-based output.
  • Contributions: The proposed mid-level representation improves robustness and compactness, aiding subsequent processing.
  • Contributions: Lost and Found is introduced as a dataset dedicated to visual lost-cargo detection, alongside application-focused evaluation metrics.
  • Related Work: Prior stereo obstacle methods include Stixels, digital elevation maps, and geometric point clusters, while small-road-obstacle literature remains limited.These methods rely on precomputed stereo disparity maps and generally perform best for medium-sized objects at close to medium range.
  • Related Work: The paper builds on local plane models that test obstacle and free-space hypotheses directly on underlying image data.

III. METHODS

The method formulates small-obstacle detection as local statistical testing between free-space and obstacle plane hypotheses. It derives likelihoods from stereo image data, optimizes each patch independently, and reports only sufficiently reliable decisions.

  • Direct Planar Hypothesis Testing (PHT): Direct Planar Hypothesis Testing formulates geometric obstacle detection as statistical hypothesis testing on image data.
  • Direct Planar Hypothesis Testing (PHT): Free-space is the null hypothesis and obstacles are the alternative, with each local plane parameterized by its normal vector and distance from the origin.
  • Direct Planar Hypothesis Testing (PHT): Each local image patch receives an independent Generalized Likelihood Ratio Test using maximum-likelihood estimates and threshold γ.
  • Direct Planar Hypothesis Testing (PHT): The likelihood model treats left and right stereo intensities as noisy samples of an observed image signal.The model includes local bias and zero-mean noise terms.
  • Direct Planar Hypothesis Testing (PHT): A plane- and camera-dependent warp transforms left-image coordinates into the right image using the plane-induced mapping.
  • Direct Planar Hypothesis Testing (PHT): Each hypothesis is evaluated by summing pixel-wise residual losses over the local patch, with parameters estimated through constrained nonlinear optimization.
  • Direct Planar Hypothesis Testing (PHT): Projected Levenberg–Marquardt solves the constrained optimization iteratively.
  • Direct Planar Hypothesis Testing (PHT): The system reports decisions only in sufficiently textured image areas where the approximate Hessian has a sufficiently large minimum eigenvalue.

B. Fast Direct Planar Hypothesis Testing (FPHT)

FPHT simplifies PHT for calibrated stereo by reparametrizing plane hypotheses in disparity space, reducing optimization complexity and achieving an order-of-magnitude speed-up without sacrificing detection performance.

  • FPHT uses a simplified parametrization for calibrated stereo cameras, reducing the number of free parameters and optimization complexity.
  • An order-of-magnitude computational speed-up is achieved without sacrificing detection performance in practice.
  • The reparametrization assumes rectified stereo pairs and plane models without yaw or roll, represented by lines in stereo disparity space.
  • The reduced parameter vector contains only disparity slope a and offset b, which directly relate to the 3D plane parameters.
  • Global independent bounds on disparity slope and offset are unavailable because their values jointly vary with plane orientation and camera distance.
  • FPHT uses the left-image sample as the reference estimate and applies a Gaussian noise model with squared-error loss ρ.

C. Point Compatibility (PC)

Point Compatibility detects obstacles by testing relative geometric compatibility between 3D point pairs, producing pixelwise labels and meaningful point clusters without a global road model.

  • Point Compatibility labels a point P2 as an obstacle when it lies within a truncated cone defined around point P1.
  • The cone uses maximum slope angle, minimum relevant obstacle height, and maximum connection height as geometric criteria.
  • The method tests every point in a precomputed stereo disparity map and projects the cones into the image plane for pixel labeling.
  • Point Compatibility simultaneously produces pixelwise obstacle labels and clusters of geometrically compatible obstacle points.
  • Although it avoids a global surface or road model, Point Compatibility depends directly on the quality of the underlying point cloud.

D. Stixels

Stixels compactly represent 3D scenes using a global ground model and vertical obstacle segments, but their results depend on disparity quality and road-model estimation.

  • Stixels provide a compact and robust 3D-scene description, particularly in man-made environments dominated by horizontal and vertical structures.
  • The algorithm separates a global ground surface model from vertical obstacle segments with variable heights.
  • Column-wise segmentation is optimized by dynamic programming from a precomputed stereo disparity map.
  • Stixel results depend on both the quality of the disparity map and the estimated road model.

E. Mid-level Representation: Cluster-Stixels (CStix)

Cluster-Stixels extend point-wise obstacle detectors with a compact Stixel-like representation through adaptive clustering and disparity-guided splitting, reducing output complexity while improving robustness.

  • Cluster-Stixels extend PHT, FPHT, and PC point-wise detectors to reduce output data and increase robustness.
  • The representation uses clustering and splitting rather than the column-wise optimization of the original Stixel algorithm.
  • Adaptive DBSCAN forms obstacle clusters using distance-adaptive neighborhoods and minimum point counts tied to stereo disparity uncertainty.
  • For Point Compatibility, clustering is omitted because the detector already produces meaningful obstacle-point clusters.
  • Each cluster is split horizontally into fixed-width vertical boxes and optionally split vertically until disparity variance falls below a threshold.
  • The pipeline accepts obstacle points and a dense or sparse disparity map, then outputs Cluster-Stixels after clustering, splitting, and bounding-box fitting.

IV. EVALUATION

The evaluation uses the Lost and Found dataset, designed around challenging small-road-obstacle scenarios and diverse objects, with separate training/validation and unseen test environments.

  • A. Lost and Found Dataset: 13 street scenarios and 37 obstacle types cover irregular roads, far distances, varied surfaces, and strong illumination changes.Objects vary in size and material; objects below 5 cm are treated as non-hazardous and excluded from results.
  • A. Lost and Found Dataset: 2,104 annotated frames come from 112 stereo video sequences with coarse free-space and fine-grained obstacle annotations.Each object receives a unique ID for later refinement into categories such as obstacle size.
  • A. Lost and Found Dataset: The test subset uses completely different surroundings, includes nine unseen objects, and contains more difficult road-profile geometries than training scenarios.The subsets contain similar numbers of sequences, frames, and objects.
  • A. Lost and Found Dataset: The stereo setup uses a 21 cm baseline, 2300-pixel focal length, and 2048×1024 spatial resolution, while evaluated methods use grayscale data.The imagery also has 12-bit radiometric resolution.
  • A. Lost and Found Dataset: The dataset is presented as the first publicly available dataset focused on small road hazards and lost cargo detection.Its stated purpose is to support further research on this safety-critical topic.

B. Metrics

The evaluation combines pixel-level ROC analysis with an instance-level metric to measure detection performance while addressing the area bias of conventional pixel aggregation.

  • B. Metrics: Pixel-level evaluation uses a ROC curve plotting pixel-wise True Positive Rate against False Positive Rate.The curve is generated through a parameter sweep and convex hull computation across evaluated configurations.
  • B. Metrics: TP and FP count true and false pixel-wise predictions relative to annotated image areas.Scaling factors compensate for subsampling and downsampling, while GTObstacles and GTF reeSpace count ground-truth obstacle and free-space pixels.
  • B. Metrics: The ROC curve is biased toward object instances covering large image areas.This motivates a second metric that weights instances rather than only accumulated pixels.
  • B. Metrics: Instance Intersection over Union measures per-instance intersection between Cluster-Stixels and pixel-wise ground truth against false-positive Stixels per frame.A Stixel is false positive when its overlap with labeled free space exceeds the 50% evaluation threshold.

2) Instance-level Metric:

The instance-level evaluation compares parameterized detection methods and efficiency settings, using Cluster-Stixels extensions and test-oriented parameter selection.

  • 2) Instance-level Metric:: PHT, FPHT, PC, and Stixels are evaluated with parameter sweeps over their principal parameters.The sweep includes patch size, optimization eigenvalue, likelihood threshold, or Stixel dynamic-programming cut costs, depending on the method.
  • 2) Instance-level Metric:: Point-based methods use stride-two subsampling by default, with optional additional twofold image downscaling for faster execution.The authors report that default subsampling provides higher efficiency without significant detection-performance loss based on their experience.
  • 2) Instance-level Metric:: PHT and FPHT use fixed plane-normal bounds, while dense SGM disparity maps initialize parameters and provide input to PC and Stixel approaches.Cluster-Stixels use a fixed manually optimized parameter set with lower impact on final results than swept parameters.
  • 2) Instance-level Metric:: Training/Validation convex hulls determine the best method-specific parameter configurations before test-set ROC evaluation.Direct comparisons between ROC curves are reserved for the test evaluation after parameter selection.

2) Quantitative Results:

On the test subset, Cluster-Stixels improve the proposed and baseline approaches, while PHT/FPHT outperform the baselines at instance level and FPHT delivers a large speed advantage.

  • 2) Quantitative Results:: All methods except Stixels produce consistent results across Training and Test subsets.Stixels perform notably worse on challenging test road profiles because road-estimation failures strongly increase the False Positive Rate.
  • 2) Quantitative Results:: The proposed mid-level Cluster-Stixels representation provides a considerable gain in the test ROC curves.The comparison includes the primary approaches and their corresponding Cluster-Stixels extensions.
  • 2) Quantitative Results:: iInt values of approximately 0.4 are achieved at an average of 3 false positives per frame by PHT/FPHT, significantly outperforming both baselines.Downsampling mainly harms the smallest, distant object instances, which the instance metric weights equally.
  • 2) Quantitative Results:: FPHT performs on par with or better than PHT while reducing full-image processing time from approximately 500 ms to 50 ms on a state-of-the-art GPU.The timing comparison refers to the algorithmic core with downsampling.

3) Qualitative Results:

The qualitative evaluation shows that FPHT methods handle challenging distances and road geometries better than the comparison methods, while Cluster-Stixels improves detection performance and robustness.

  • All methods detect a medium-sized hazard on a flat road profile in the residential-area example.
  • At large distances on a bumpy surface, FPHT remains reasonably effective while Stixel and PC fail to detect the relevant objects.The disparity signal-to-noise ratio drops substantially at these distances, degrading the constructed 3D point cloud.
  • On a road with a double kink, only FPHT detects the tire that PC and Stixel miss.The case requires highly accurate road-model estimation for Stixel, whereas PC and FPHT are invariant to the road-profile condition.
  • Across the qualitative scenarios, FPHT methods show the best performance, while PC has increased false positives and Stixel depends on correct road-profile estimation.
  • Cluster-Stixels provides an additional gain in detection performance and robustness while significantly reducing output complexity.
Loading 1609.04653v1…