Source-linked AI summary
StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matching
Bohan Li
TL;DR
Stereo matching methods still struggle to preserve fine-grained geometry, especially around boundaries, thin structures, and sharp edges. StereoDiffuer uses SAP and confidence-guided iterative diffusion conditioned on initial disparity to refine these details; Scene Flow and KITTI experiments show competitive overall accuracy with improvements in evaluated boundary and foreground regions.
Problem
Existing stereo methods often produce blurred edges and over-smoothed predictions because fine-grained geometric details are not fully preserved.
Method
StereoDiffuer combines SAP-derived saliency cues and cost-volume confidence with the initial disparity map to condition iterative denoising diffusion refinement.
Results
Scene Flow and KITTI experiments show competitive overall accuracy and improvements in the evaluated boundary and foreground regions.
Takeaways & Limitations
The results support saliency-conditioned progressive refinement for explicit geometric modeling in stereo matching.
Takeaways & Limitations
Four-step StereoDiffuer requires 0.28 s per stereo pair on an NVIDIA RTX 4090, and the method is evaluated primarily on Scene Flow and KITTI.
Abstract
from arXiv · showhide
With the advance of deep neural networks, the quality of disparity maps obtained through stereo matching has steadily improved. However, existing stereo matching methods still struggle to preserve fine-grained geometric details, resulting in blurred edges and over-smoothed predictions in challenging regions. To address these limitations, we propose StereoDiffuer, an iterative diffusion-based stereo matching framework that explicitly models geometric details and progressively refines disparity estimates. The framework incorporates a Saliency Attention Perception (SAP) module to extract salient geometric cues, including object boundaries, thin structures, and sharp edges. Confidence-guided SAP features are combined with the initial disparity estimate to condition an iterative denoising diffusion process, which corrects residual disparity errors and restores geometric details suppressed during cost-volume regularization and upsampling. Experimental results on the Scene Flow and KITTI benchmarks demonstrate the effectiveness of the proposed framework and its competitive performance relative to the compared stereo matching methods.
1. Introduction
StereoDiffuer addresses persistent blurred edges and over-smoothed disparity boundaries by explicitly modeling fine-grained geometric details. It combines saliency-guided cues and initial disparity estimates to condition iterative diffusion refinement, with evaluations showing effective, geometrically consistent predictions.
- Stereo matching remains difficult in textureless, repetitive, slender, occluded, and reflective regions, producing inaccurate disparity predictions.
- Regularization and coarse-to-fine methods improve matching precision or efficiency, but blurred edges and over-smoothed boundaries persist because fine-grained geometry is insufficiently preserved.
- The diffusion process progressively corrects residual disparity errors and reduces boundary over-smoothing in geometrically challenging regions.
- StereoDiffuer introduces SAP to extract high-frequency geometric cues, including object boundaries, thin structures, and sharp edges.
- The method combines confidence-guided saliency features with the initial disparity map to condition iterative denoising diffusion for continuous disparity refinement.
- Experiments on Scene Flow and KITTI report effective, accurate, and geometrically consistent disparity maps.
2. Related Work
Related stereo-matching work develops stronger cost-volume construction, regularization, and coarse-to-fine computation, while diffusion models provide an iterative denoising paradigm that can be guided for predictive tasks.
- Early stereo methods use Siamese or 2D convolutional architectures for feature similarity, correspondence prediction, and end-to-end disparity estimation.
- GCNet, PSMNet, and related approaches improve cost-volume construction and contextual aggregation through feature concatenation, 3D regularization, hourglass networks, and global guidance.
- Coarse-to-fine frameworks reduce the computational burden of 3D aggregation while preserving multi-scale or global contextual information.
- Denoising diffusion models approximate empirical data distributions through iterative denoising procedures across computer vision, language, and scientific applications.
- Conditional diffusion adapts generative models to predictive tasks by guiding outputs with task-specific inputs, as demonstrated in super-resolution and radiance-field synthesis.
3. Method
StereoDiffuer builds an initial disparity estimate from multi-scale stereo features and cost volumes, then refines it with saliency-conditioned iterative diffusion. Its SAP module emphasizes geometric details, while confidence-guided saliency features supplement the initial disparity during denoising.
- Overall framework: The framework extracts feature pyramids, constructs multi-scale cost volumes, regresses an initial disparity, and performs saliency-conditioned diffusion refinement.The cost-volume pyramid supplies a multi-scale geometric prior before the refinement stage.
- Conditional diffusion refinement: The conditional diffusion module progressively refines the initial disparity estimate using the collaborative condition rather than replacing the preceding stereo-matching blocks.The refinement strategy addresses inaccuracies introduced around thin objects, edges, and sharp object boundaries.
- Cost-volume construction and disparity regression: Coarse-to-fine cost-volume regularization exchanges context across scales, after which softmax probability volumes support soft-argmin disparity regression and bilinear upsampling.The resulting maps match the input resolution but may contain inaccurate disparities around thin objects, edges, and sharp boundaries.
- Saliency Attention Perception: SAP combines spatial-wise and channel-wise saliency blocks to emphasize spatially distinctive structures and geometry-related feature channels.The fused saliency representation highlights boundaries, thin structures, and other high-frequency geometric details.
- Saliency Attention Perception: SAP extracts saliency maps from image features to complement the coarse geometric consistency supplied by cost-volume aggregation and disparity regression.This design targets high-frequency details that regularization and upsampling may smooth around boundaries and thin structures.
- Conditional diffusion refinement: A confidence map selects reliable saliency information through cross-attention, and the selected features are concatenated with the initial disparity to form the collaborative condition.The condition guides the diffusion process throughout iterative denoising.
4. Experiments
Experiments evaluate StereoDiffuer on Scene Flow and KITTI using quantitative benchmarks, qualitative visualizations, ablations, edge-region analysis, runtime measurements, and cross-domain discussion. The results show accurate disparity estimation with improved geometric detail preservation, while accuracy–runtime trade-offs and domain shifts constrain deployment.
- Benchmark Results: StereoDiffuer obtains an EPE of 0.47 on the Scene Flow test set with 12 reverse steps, matching IGEV-Stereo and Selective-RAFT.The result is lower than the other methods listed in Table 2.
- Benchmark Results: The KITTI evaluations compare erroneous-pixel and disparity-outlier metrics across KITTI 2012 and KITTI 2015 test sets.KITTI 2012 reports 2-pixel, 3-pixel, Noc, and All metrics; KITTI 2015 reports D1 for background, foreground, and all pixels.
- Qualitative Results: Qualitative results show refined disparities with sharper object boundaries on Scene Flow and preserved thin structures with sharper boundaries on KITTI 2015.Comparisons on KITTI test sets also show finer thin structures, sharper edges, and more coherent estimates in low-texture regions.
- Edge-Region Evaluation: Edge-region evaluation on KITTI 2015 reports the lowest compared errors, with an Edge EPE of 0.76 and an Edge 3-pixel error of 2.58%.The evaluation uses common masks, full-resolution predictions, and no boundary-specific post-processing.
- Ablation Study: Adding the complete SAP module reduces KITTI 2015 3-pixel error from 1.74% to 1.52% and Scene Flow EPE from 0.90 to 0.51.Removing either the spatial or channel-wise block produces higher errors, indicating complementary information from both components.
- Ablation Study: Four-step diffusion lowers KITTI 2015 3-pixel error by 0.21 percentage points and Scene Flow EPE by 0.30 pixels versus four-update ConvGRU.Increasing reverse steps from four to eight improves the metrics modestly but raises runtime from 0.28 s to 0.45 s; four steps are used as the default compromise.
5. Conclusion
StereoDiffuer uses saliency-conditioned iterative diffusion to model geometric details and progressively refine disparity estimates, achieving competitive accuracy with improvements in evaluated boundary and foreground regions.
- StereoDiffuer explicitly models geometric details through iterative diffusion-based refinement of disparity estimates.The framework progressively refines disparity rather than relying solely on initial estimates.
- Its SAP module extracts spatial- and channel-wise saliency cues for geometric modeling.These cues provide salient geometric information for the refinement process.
- Cost-volume confidence selects reliable geometric information to condition reverse diffusion alongside the initial disparity map.Confidence guidance controls which SAP features enter the diffusion conditioning.
- Experiments on Scene Flow and KITTI show competitive overall accuracy and improvements in evaluated boundary and foreground regions.The conclusion presents these results as support for saliency-conditioned progressive refinement, without claiming universal superiority.