Source-linked AI summary
Embedded real-time stereo estimation via Semi-Global Matching on the GPU
Daniel Hernandez-Juarez, Alejandro Chacón, Antonio Espinosa, David Vázquez, Juan Carlos Moure, Antonio Manuel López
TL;DR
Real-time stereo depth estimation is computationally demanding for robotics, ADAS, and autonomous vehicles. This work implements an SGM-based disparity pipeline on an energy-efficient embedded GPU, achieving 42 fps on a Tegra X1 under a specified configuration.
Problem
Real-time, robust, dense stereo depth computation is computationally demanding in robotics, ADAS, and autonomous vehicles.
Method
The paper develops and optimizes GPU-parallel disparity-estimation schemes and data layouts, including vectorization and CTA-to-Warp conversion, for embedded Tegra X1 hardware.
Results
42 fps is achieved on a Tegra X1 for 640×480 images, 128 disparity levels, and 4 SGM path directions.
Takeaways & Limitations
The results indicate that low-consumption embedded GPUs can attain real-time stereo processing demands and support real-time video-processing applications.
Takeaways & Limitations
Future work targets larger images, more disparity levels, and more complex algorithms, with postfiltering methods intended to improve accuracy.
Abstract
from arXiv · showhide
Dense, robust and real-time computation of depth information from stereo-camera systems is a computationally demanding requirement for robotics, advanced driver assistance systems (ADAS) and autonomous vehicles. Semi-Global Matching (SGM) is a widely used algorithm that propagates consistency constraints along several paths across the image. This work presents a real-time system producing reliable disparity estimation results on the new embedded energy-efficient GPU devices. Our design runs on a Tegra X1 at 42 frames per second (fps) for an image size of 640x480, 128 disparity levels, and using 4 path directions for the SGM method.
1 Introduction
Stereo depth estimation is important but computationally demanding for robotics, ADAS, and autonomous vehicles. This work targets a complete real-time disparity pipeline on an energy-efficient embedded GPU, achieving high throughput with optimized parallel designs.
- Motivation: Dense, robust, real-time stereo depth computation supports ADAS, robotics navigation, and autonomous vehicles.Stereo information can also improve pedestrian-detection accuracy and performance.
- Stereo depth estimation: Epipolar geometry restricts correspondence search to a one-dimensional image line, while disparity is used to infer depth through triangulation.The camera baseline and focal length determine the relationship between disparity and distance.
- Stereo depth estimation: SGM addresses ambiguous correspondences by aggregating consistency constraints along multiple image paths.Combined with local similarity metrics, it handles noise, lighting variation, untextured regions, and preserves edges.
- System challenge: SGM’s computational load and memory-bandwidth demands challenge fast, low-energy implementations; desktop GPUs consume too much power for many embedded settings.Dedicated hardware is efficient but inflexible to algorithm changes.
- Contribution: The work implements and evaluates a complete disparity-estimation pipeline on an embedded GPU-accelerated device.The objective is motivated by the emergence of low-cost, low-consumption platforms such as Tegra X1 systems.
- Results: 42 fps is achieved on a single Tegra X1 for 640×480 images, 128 disparity levels, and 4 SGM path directions.Optimizing baseline parallel schemes with vectorization and CTA-to-Warp conversion improves performance around 3 times.
2 Disparity Estimation Pipeline
The pipeline extracts local features, computes matching costs, aggregates SGM smoothing costs, derives disparity, filters outliers, and transfers images between host and GPU memory. Its design combines stereo-matching methods with GPU-aware parallelization and memory layouts.
- Disparity estimation pipeline: The pipeline copies captured images to the GPU, computes local matching costs, aggregates SGM costs, filters disparity, and returns the result to host memory.A 3×3 median filter removes outliers after disparity computation.
- Local matching cost: Census-based matching was selected because Center-Symmetric Census Transform with a 9×7 window provides a compact representation with similar accuracy.Similarity is measured using the Hamming distance between CSCT bit-vector features.
- Local matching cost: Census features compare pixel values around a central pixel, while local intensity invariance and outlier tolerance support outdoor robustness.These properties arise from neighboring-pixel comparisons and the bit-vector representation.
- Semi-Global Matching: SGM approximates a two-dimensional global energy minimum by aggregating costs along several one-dimensional paths.The number of paths trades off disparity quality and performance.
- GPU architecture and performance: GPU performance depends on parallelization and data layout because they determine thread-level parallelism and memory-access patterns.Coalesced accesses and reuse in shared memory and registers help address device-memory bandwidth bottlenecks.
- Related work: Earlier CPU and GPU implementations were substantially slower or used lower-resolution settings, while some modifications reduced accuracy.Reported examples include 5.43 fps on CPU, 12 fps for reduced SGM, and 8 fps for early GPU implementations.
- Related work: The reported embedded evaluation was the first, to the authors’ knowledge, for disparity estimation on an NVIDIA GPU-accelerated embedded system and the Maxwell architecture.The work therefore emphasizes parallelization schemes adapted to current embedded hardware features.
3 Algorithm Description and Massive Parallelization
The paper maps each disparity-estimation stage onto cooperative GPU schemes, using tiled data reuse for CSCT and matching cost, path-wise recurrence for SGM aggregation, and targeted optimizations to reduce memory and synchronization overhead.
- Matching Cost Computation: The disparity pipeline uses tiled, read-cooperative GPU schemes for CSCT and matching-cost computation, with shared memory enabling input reuse.The CSCT scheme follows a 2D stencil, while matching cost uses a 1D tile and disparity-fast data layout.
- Matching Cost Computation: 12 times fewer global memory accesses are achieved for tiled CSCT than for a straightforward embarrassingly parallel design.The reduction is attributed to shared-memory tiling and data reuse for the 2D-stencil computation.
- Matching Cost Computation: 4 times lower memory bandwidth requirements result from representing matching cost with one byte instead of 32-bit integers.The read-cooperative matching-cost scheme trades some thread-level parallelism for approximately 8 times fewer global memory accesses than the naïve design.
- Smoothing Cost Aggregation (SGM) and Disparity Computation: SGM aggregates matching costs along multiple path directions using recurrent dynamic programming, then sums path costs and selects the minimum-cost disparity.The recurrence uses neighboring disparity costs and penalties P1 and P2, while the proposed scheme parallelizes across perpendicular slices, disparity levels, and path directions.
- Additional Optimizations: Almost 3× combined performance improvement comes from vectorization, CTA-to-warp conversion, and reduced synchronization and communication overhead.CTA-to-warp conversion enables register shuffles and reduces instruction count, although both strategies reduce thread-level parallelism.
- Additional Optimizations: 1.35x Tegra X1 speedup results from fusing the final path aggregation with cost summation and disparity computation.The fusion reduces memory traffic by avoiding separate intermediate data accesses.
4 Results
The evaluation measures throughput, performance per watt, and disparity accuracy across Tegra X1 and Titan X systems and multiple SGM path configurations. The Tegra X1 achieves 42 fps with high accuracy using four paths, while the Titan X exceeds its performance but is less energy efficient.
- Figure 6 reports performance throughput, performance per watt, and accuracy for 640×480 images, 128 disparity levels, and 2, 4, and 8 path directions.These measurements compare different SGM configurations on both GPU systems.
- 42 fps is achieved on the Tegra X1 for 640×480 images, 128 disparity levels, and 4 SGM path directions.The evaluation covers 2, 4, and 8 path directions.
- More than 10× higher performance is provided by the Titan X, while the Tegra X1 delivers around 2× more performance per Watt.The comparison uses the high-end Titan X and embedded Tegra X1 GPU systems.
- Figure 7 provides an example of the disparity computed by the proposed algorithm.
5 Conclusions
The work develops and optimizes GPU-parallel disparity-estimation schemes for embedded systems. Its Tegra X1 implementation reaches real-time performance, while future work targets larger inputs, more disparity levels, and more complex accuracy-improving algorithms.
- The implementation reaches real-time depth computation performance on the low-power Tegra X1.The authors identify embedded GPUs as suitable platforms for real-time video processing.
- Around 3× performance improvement comes from optimizations including vectorization and CTA-to-Warp conversion.The baseline schemes use GPU performance-model rules and are intended to scale to forthcoming architectures.
- Future work targets larger images, more disparity levels, and more complex algorithms on embedded NVIDIA Pascal GPUs.Planned postfilters include left-right consistency checking, subpixel calculation, and adaptive P2 to improve accuracy.