Continuous Markov Random Fields for Robust Stereo Estimation

• Approximately one hour per low-resolution Middebury stereo pair illustrates why slanted-plane MRF computation can be impractical for robotics or autonomous driving.Each plane has three continuous parameters, making non-convex continuous-MRF inference computationally challenging.Source: §1 · p. 1
• Junction potentials constrain physically realizable combinations of boundary labels, adapting line-labeling ideas [14, 15] to segment boundaries.Source: §1 · p. 1
• Occlusion labels express ownership of the pixels composing a segment boundary, which is owned by the foreground object.Source: §1 · p. 2
• Particle inference based on PCBP avoids premature quantization of continuous variables, while primal-dual optimization learns each potential’s contribution.Source: §1 · p. 2
• The approach outperforms the state of the art on high-resolution Middlebury imagery and the more challenging KITTI dataset.Source: §1 · p. 2

• Local methods can suffer border bleeding or correspondence ambiguity; adaptive support windows improve border-discontinuity performance at greater computational cost.Source: §2 · p. 2
• Semi-global matching propagates information along 16 orientations and supplies the disparity map used to build this paper’s potentials.Seed-grown disparities can be accurate and fast but do not provide dense matching and struggle in textureless or distorted areas.Source: §2 · p. 2
• The paper combines boundary labels, junction potentials, edge ownership, convex PCBP inference, and primal-dual approximate-inference training.Source: §2 · p. 2
• Global MRF methods enforce smoothness explicitly, while pixel-level formulations define it very locally.Source: §2 · p. 2

3 Continuous MRFs for stereo

• The model jointly infers segment depth as continuous slanted 3D planes and neighboring-boundary labels as discrete co-planar, hinge, or directional occlusion states.Source: §3 · pp. 2–3

3.1 Occlusion Boundary and Segmentation Potentials

• Given an input disparity image, potentials score plane agreement with matching results, enforce boundary consistency, and penalize occlusion labels unsupported by disparity observations.Source: §3.1 · pp. 3–5
• Higher-order potentials reject physically impossible 3-way junctions and enforce feasible 4-way junctions, whose discrete variables have only four states.Source: Figure 1 · p. 3, Figure 2 · p. 3, §3.1 · pp. 5–6
• A color-histogram potential favors similar statistics for coplanar neighboring segments, though those statistics are less meaningful for controlled-environment Middlebury imagery.Source: Figure 3 · p. 4, §3.1 · p. 6

3.2 Inference in Continuous MRFs

• Inference is NP-hard for this hybrid model; PCBP iteratively samples particles, solves a discretized MRF, and updates the current MAP solution.Source: §3.2 · pp. 6–7

3.3 Learning in Continuous MRFs

• Learning estimates potential weights by minimizing surrogate partition loss, approximating integration over continuous variables by summing over 30 particles.Source: §3.3 · p. 7

Number of superpixels

• Inference scales linearly with superpixel count, averaging 5.5 minutes per image at 1,200 superpixels and 2.5 minutes at 300.Performance degrades gracefully as the number of superpixels is reduced.Source: Figure 5 · p. 8, §4 · p. 9

Number of re-sampling iterations

• Two resampling iterations reach a good local optimum and reduce inference cost from 5.5 to 2.2 minutes per high-resolution image.The comparison is with five iterations.Source: Figure 6 · p. 8, §4 · p. 9

Datasets and robustness to noise

• Evaluation spans high-resolution Middlebury and KITTI, plus a synthetic Gaussian-noise test with 10 training and 90 test images.The synthetic images are 320 × 240 pixels; Table 8 reports disparity RMS and boundary-variable error as noise varies.Source: §4 · pp. 8, 10, Table 8 · p. 10

Comparison with the state-of-the-art

• The approach significantly outperforms all baselines across KITTI and high-resolution Middlebury settings and all reported thresholds.The reported thresholds are greater than 2, 3, 4, and 5 pixels.Source: §4 · p. 8

Segmentation strategy

• Intersecting SLIC and UCM superpixels outperforms either strategy on KITTI and yields similar results on Middlebury.UCM alone performs poorly because its segments are few and sometimes very large; the intersection retains UCM's better boundary alignment.Source: §4 · p. 9

Training set size

• Training with a single image still outperforms all baselines.Table 5 examines estimation errors as training-set size varies and notes that few images suffice to learn good parameters.Source: §4 · p. 9, Table 5 · p. 9

Oracle performance

• The oracle achieves strong performance, indicating that errors from 3D slanted-plane discretization are negligible.This is an upper-bound evaluation that assumes an energy whose MAP is ground-truth disparity and solvable NP-hard inference.Source: §4 · pp. 9–10, Table 6 · p. 9, Table 7 · p. 10

Learning settings

• Performance remains state-of-the-art from CRFs (ϵ = 1) to structural SVMs (ϵ = 0), while the choice of task loss has little effect.The learning settings are evaluated on KITTI.Source: §4 · p. 10

Conclusion

• The model jointly reasons about occlusion boundaries and depth.It formulates inference in a hybrid MRF with continuous slanted 3D planes and discrete occlusion-boundary variables.Source: §5 · p. 10
• Particle convex belief propagation is used to tackle inference in the hybrid MRF.Source: §5 · p. 10
• The approach is demonstrated on high-resolution Middlebury imagery and the more challenging KITTI dataset; future work includes alternative inference algorithms and segmentation potentials.Source: §5 · p. 10