PaperlayerView exact revision on arXiv

Source-linked AI summary

In Defense of Classical Image Processing: Fast Depth Completion on the CPU

Jason Ku, Ali Harakeh, Steven L. Waslander

arXiv:1802.00036v1cs.CV

TL;DR

Sparse LIDAR depth maps limit machine-vision and robotics systems, motivating depth completion without relying on dense measurements or extensive learned models. The paper proposes a training-free, non-guided classical image-processing algorithm and reports state-of-the-art KITTI performance at 90 Hz on the CPU. Its results support using carefully designed classical processing as a competitive embedded preprocessing option.

  • Problem

    Sparse LIDAR depth maps limit the performance and operational range of perception algorithms that rely on depth.

  • Method

    The paper uses a non-guided sequence of classical image-processing operations on sparse LIDAR data, requiring no training data.

  • Results

    At submission, the algorithm ranked first among published KITTI methods and outperformed NN+CNN by 131.29 mm RMSE and 113.54 mm MAE.

  • Takeaways & Limitations

    The CPU-based algorithm runs at 90 Hz and is presented as a competitive embedded preprocessing step for SLAM and 3D object detection.

Abstract

from arXiv · show

With the rise of data driven deep neural networks as a realization of universal function approximators, most research on computer vision problems has moved away from hand crafted classical image processing algorithms. This paper shows that with a well designed algorithm, we are capable of outperforming neural network based methods on the task of depth completion. The proposed algorithm is simple and fast, runs on the CPU, and relies only on basic image processing operations to perform depth completion of sparse LIDAR depth data. We evaluate our algorithm on the challenging KITTI depth completion benchmark, and at the time of submission, our method ranks first on the KITTI test server among all published methods. Furthermore, our algorithm is data independent, requiring no training data to perform the task at hand. The code written in Python will be made publicly available at https://github.com/kujason/ip_basic.

I. INTRODUCTION

The paper argues that carefully designed classical image processing can outperform deep-learning approaches for depth completion while remaining fast, CPU-only, and training-free.

  • Motivation: Depth completion matters because sparse projected LIDAR limits perception algorithms that depend on sufficiently dense depth measurements.The paper specifically connects this limitation to 3D object detection, whose bounding-box regression requires enough points on an object.
  • Motivation: Deep-learning depth completion is attractive for reducing human design decisions, but GPU power requirements constrain deployment on embedded systems.The paper also warns that insufficient problem understanding can produce sub-optimal high-capacity network designs.
  • Contribution: The proposed non-guided algorithm uses only LIDAR data and classical image processing, requiring no training data and remaining independent of image-quality changes.Its non-guided design avoids reliance on an RGB image as guidance.
  • Contribution: 90 Hz on the CPU and first among all published methods on KITTI depth completion benchmark are the reported speed and ranking claims.The benchmark ranking and runtime are presented as headline contributions.
  • Contribution: The algorithm is reported to outperform CNN approaches designed for sparse input representations by a wide margin.This comparison targets networks specifically developed for sparse depth inputs.

II. RELATED WORK

Prior depth-completion methods include guided approaches using color images and non-guided approaches using sparse depth alone, spanning classical filters, patch methods, and neural networks.

  • Scope: Depth completion supports applications including stereo vision, optical flow, and 3D reconstruction from sparse LIDAR data.
  • Guided Depth Completion: Guided methods use color images to complete depth maps, including joint bilateral filtering, color-guided median filters, and deep learning.These methods have been demonstrated to produce higher-quality depth maps but require substantial data.
  • Non-Guided Depth Completion: Non-guided methods complete dense depth maps from sparse depth alone, including repetitive-patch methods and Nadaraya-Watson kernel regression.The Nadaraya-Watson baseline handles sparsity by excluding missing values from Gaussian filtering.
  • Non-Guided Depth Completion: A sparsity-invariant CNN architecture processes sparse inputs by considering only valid values in output computations.

III. PROBLEM FORMULATION

Depth completion estimates a dense depth map from an image and sparse depth map, while the non-guided formulation removes dependence on the image.

  • Problem Formulation: Given image I and sparse depth map Dsparse, the task is to estimate a function f̂ whose output approximates the true dense-depth function.The target function maps the image and sparse depth inputs to a dense depth map.
  • Problem Formulation: Ddense has the same dimensions as I and Dsparse, with empty values replaced by estimated depths.For non-guided completion, the formulation becomes independent of image I.

IV. PROPOSED ALGORITHM

The proposed implementation uses standard OpenCV and NumPy image-processing operations to produce a dense depth map for downstream 3D perception tasks.

  • Implementation: The Python method addresses sparsity through a sequence of OpenCV and NumPy operations that exploit larger pixel values overwriting lower ones.This property is used to fill empty pixels through an eight-step depth-completion algorithm.
  • Output: The algorithm outputs a dense depth map suitable for 3D object detection, occupancy-grid generation, and SLAM.

1) Depth Inversion:

The algorithm uses morphological transformations for sparsity handling, but first inverts valid depth values because empty and close pixels both encode as zero. It then begins filling holes nearest to valid depths with a custom dilation kernel.

  • OpenCV morphological transformations overwrite smaller pixel values with larger ones, forming the algorithm’s main sparsity-handling mechanism.
  • Depth inversion prevents dilation from overwriting closer objects with farther depths and losing edge information.Valid depths are transformed as Dinverted = 100.0−Dinput.
  • A custom kernel initially dilates valid depth pixels into nearby empty pixels that are likely to share similar depths.The kernel reflects the sparsity of projected points and the structure of LIDAR scan lines.

3) Small Hole Closure:

The algorithm closes small holes by connecting nearby dilated depth patches while preserving object edges, then fills remaining small-to-medium holes selectively. It also extrapolates tall structures above the highest LIDAR points.

  • A 5 × 5 full-kernel morphological close connects nearby depth values and preserves object edges.
  • A 7 × 7 full-kernel dilation fills only empty pixels, leaving previously computed valid depths unchanged.
  • Column-wise extrapolation extends the top value to the image top for tall objects such as trees, poles, and buildings.

6) Large Hole Fill:

The large-hole fill step uses nearby-value extrapolation where no depth or image data exists, then removes dilation outliers and smooths local geometry before restoring the original depth encoding.

  • A 31x31 full-kernel dilation fills remaining empty pixels while leaving valid pixels unchanged.Because these regions contain no points and no image data is used, their depths are extrapolated from nearby values.
  • Qualitative KITTI results compare dense depth maps and pixel-wise errors against Sparse CNN and Nearest Neighbour Interpolation with CNN.The error visualization ranges from blue for low error to red for high error.
  • A 5 × 5 median blur removes dilation outliers while maintaining local edges, followed by a 5×5 Gaussian blur that smooths local planes and sharp object edges.
  • The output depth encoding is restored using Doutput = 100.0 −Dinverted.

V. EXPERIMENTS AND RESULTS

The algorithm is evaluated on KITTI depth completion data formed by projecting LIDAR scans into image coordinates, with RGB images provided but unused by the unguided method. Evaluation uses a 1000-image validation set, a 1000-image test submission, and four error metrics.

  • KITTI depth completion provides sparse depth maps by projecting LIDAR points into image coordinates using front-camera calibration matrices.The resulting depth maps have the same size as the corresponding RGB images.
  • The unguided algorithm does not use the corresponding RGB images.
  • A validation set of 1000 images evaluates all experiments, while final results on a 1000-image test set are submitted to KITTI’s test server.
  • Performance is measured with iRMSE, iMAE, RMSE, and MAE.

A. Performance on the Depth Completion Task

On the KITTI depth completion test set, the proposed method ranks first among published methods in RMSE and MAE while running at 90 Hz on the CPU.

  • Benchmark performance: The proposed algorithm ranks first among all published methods in both RMSE and MAE on KITTI’s depth completion test set.The comparison includes Nadaraya-Watson, Sparse CNN, and Nearest Neighbour Interpolation with CNN baselines.
  • Benchmark performance: 131.29 mm lower RMSE and 113.54 mm lower MAE distinguish the method from NN+CNN, the runner-up on KITTI.The paper describes this as an 11 cm mean-error difference in the final point cloud.
  • Runtime: 90 Hz on an Intel Core i7-7700K lets the method run on the CPU, whereas the second- and third-ranking methods require GPUs at 50 and 100 Hz.The comparison concerns runtime and hardware requirements reported for the KITTI methods.

B. Experimental Design

The experiments use a greedy sequence of depth-completion design choices, evaluating dilation kernels and blur methods for accuracy, runtime, and qualitative point-cloud behavior.

  • Design procedure: The algorithm fills smaller holes before larger ones, then extrapolates remaining regions and applies final blurring to reduce noise and smooth local planes.This ordering gradually increases each valid pixel’s area of effect while preserving local structure.
  • Design procedure: Kernel size, kernel shape, and blur type are optimized sequentially, carrying the best result from each experiment into the next design step.The greedy procedure first selects kernel size, then shape, and finally blur settings.
  • Dilation kernels: A 5 × 5 dilation kernel provides the lowest RMSE, while 3 × 3 under-expands pixels and 7 × 7 dilates beyond their actual areas.The size experiment compares 3 × 3, 5 × 5, and 7 × 7 full kernels.
  • Dilation kernels: A diamond shape provides the lowest RMSE among 5 × 5 kernels and preserves rounded-edge outlines while connecting edges during hole closing.The comparison uses full, circular, cross, and diamond binary kernel shapes.
  • Runtime: Dilation-kernel size and shape do not significantly affect runtime, separating their accuracy effects from computational cost.This runtime observation accompanies the kernel-size and shape experiments.
  • Blurring: Median blur removes outlier depth pixels while preserving local edges, bilateral blur preserves local structure with minimal metric effect, and Gaussian blur adds only 1 ms.Median blur adds 2 ms and bilateral blur adds 4 ms; the final algorithm combines median and Gaussian blur for lowest RMSE.
  • Qualitative behavior: For applications not requiring a fully dense map, the paper recommends limiting upward extension and dilation size because Gaussian blur adds many 3D points despite the lowest RMSE.Figure 5 compares bilateral and Gaussian blur on a projected KITTI point cloud, with points colorized using RGB.

VI. CONCLUSION

The paper presents a training-free, traditional image-processing depth-completion method that achieves state-of-the-art KITTI results at real-time CPU speed without additional GPU hardware.

  • Conclusion: The method converts sparse depth maps into dense maps using traditional image-processing techniques and no training.The authors describe the approach as robust to overfitting.
  • Conclusion: The algorithm achieves state-of-the-art KITTI results, runs at 90 Hz in real time, and requires no additional GPU hardware.The paper identifies embedded preprocessing for SLAM or 3D object detection as a competitive deployment context.
  • Conclusion: The work argues that carefully designed classical methods can serve as powerful baselines alongside deep learning systems.The authors state that the paper is not intended to undermine deep learning.
Loading 1802.00036v1…