Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras

1.1. Real-time Visual Odometry

• Camera-based odometry is increasingly sought for versatile, compact, low-cost sensing despite requiring sufficient illumination and texture; desired systems combine accuracy, robustness, and reconstruction density.Camera-based sensors also offer higher resolution, range, and sensing rate, as well as lower weight and size, than active laser sensing.Source: §1.1 · pp. 1–2

1.2. Related Work

• Prior direct methods span dense, semi-dense, and sparse reconstructions, while monocular DSO [7] lacks metric scale and suffers scale drift, calibration dependence, and fast-motion/rolling-shutter sensitivity.Earlier keypoint-based systems track selected points across frames to jointly infer 3D locations and camera motion.Source: §1.2 · p. 2

1.3. Contribution

• Stereo DSO combines temporal multi-view and static stereo within bundle adjustment using Schur-complement marginalization while retaining real-time capability.Unlike prior monocular-to-stereo extensions that filter geometry without bundle adjustment, it extends the energy and bundle-adjustment pipeline. Stereo initialization enables faster-converging scale estimation and reduces sensitivity to missing photometric calibration and rolling shutter.Source: §1.3 · p. 2
• KITTI/Cityscapes evaluations found Stereo DSO superior to Stereo ORB-SLAM and Stereo LSD-SLAM, especially on the KITTI testing set; it also beat globally optimized Stereo ORB-SLAM over longer ranges without loop closure.The authors report that the KITTI testing-set result indicates better generalization to unknown settings.Source: §1.3 · pp. 2–3

2. Direct Sparse VO with Stereo Cameras

• Static stereo provides absolute scale from the known baseline, while temporal multi-view stereo resolves static stereo’s limited accurate triangulation depth range.The two sources also complement each other in cases where edges are parallel to epipolar lines.Source: §2 · p. 3

2.5. Windowed Optimization

• Joint active-window optimization combines temporal and static stereo constraints to estimate keyframe poses, affine brightness parameters, point depths, and camera intrinsics.Old keyframes and points are marginalized using the Schur complement to keep the window bounded.Source: §2 · p. 3, §2.4 · p. 4, §2.5 · p. 5

2.1. Notation

• Points are represented by image coordinates and inverse depth relative to their host keyframe, a parameterization reported as advantageous when image errors are modeled as Gaussian.Source: §2.1 · p. 3

2.2. Direct Image Alignment Formulation

• Stereo DSO follows direct image alignment with a fixed number of points sampled across gradient-rich regions, using stereo images to verify points and assist depth initialization.Affine brightness parameters model changes between images when exposure and camera response information is unavailable.Source: §2.2 · pp. 3–4

2.3. Tracking

• Frames are tracked against the newest keyframe by coarse-to-fine Gauss–Newton optimization with depths fixed; the initial frame’s semi-dense depths come from stereo matching along horizontal epipolar lines.The matching uses normalized cross-correlation (NCC) over 3 × 5 neighborhoods.Source: §2.3 · p. 4

2.4. Frame Management

• Keyframe creation depends on scene change measured by mean squared optical flow and illumination change measured by relative brightness; candidate points are selected per image block using adaptive gradient thresholds.Static stereo matching with NCC initializes candidate-point depths, which are refined by subsequent non-keyframes before activation.Source: §2.4 · p. 4

2.5. Windowed Optimization

• Each active point’s observations create photometric factors, while static stereo constraints introduce scale information and geometric priors for temporal multi-view stereo.The static stereo camera transform is fixed rather than optimized, and a coupling factor weights static stereo constraints relative to temporal ones.Source: §2.4 · p. 4, §2.5 · p. 5

3.1. KITTI Visual Odometry Benchmark

• On example sequence 06, stereo coupling weights λ = 1 or 2 significantly reduce translational and rotational errors; λ > 3 worsens performance by increasing sensitivity to incorrect static-stereo matches.Source: §3.1 · p. 6
• On KITTI training data, Stereo DSO is almost always better than Stereo LSD-SLAM; versus ORB-SLAM2, its rotational errors are all lower while translational errors are mixed.The reported translational and rotational RMSEs are averaged over 100m to 800m intervals; ORB-SLAM2 loop closure and global bundle adjustment were disabled for the comparison.Source: §3.1 · p. 6, Table 1 · p. 6
• Qualitative KITTI comparisons show that state-of-the-art feature-based and direct monocular VO methods cannot handle scale drift properly, illustrating the benefit of using a stereo camera.Figure 6’s caption identifies KITTI sequences 00 and 05 and notes that monocular VO results are similarity-aligned to ground truth.Source: §3.1 · p. 6, Figure 6 · p. 7
• On KITTI testing sequences, Stereo DSO outperforms LSD-SLAM and ORB-SLAM2 SLAM results across all plotted distance-interval and driving-speed settings, despite running as pure VO.The comparison includes SLAM results with loop closure, and ORB-SLAM2 also uses global bundle adjustment.Source: §3.1 · p. 6, Figure 5 · p. 7

3.2. Cityscapes Dataset

• For rolling-shutter Cityscapes imagery, fast pixel clocks and synchronized corresponding stereo rows let static stereo compensate rolling-shutter errors across views, without a dedicated calibration treatment.Cityscapes provides high-dynamic-range images at a fairly high 17Hz frame rate.Source: §3.2 · pp. 6–7
• On Cityscapes, severe brightness changes combined with strong vehicle rotation can still make Stereo DSO fail, as its affine brightness model is insufficient for extreme cases.Inaccurate GPS/vehicle odometry and moving objects also make accurate tracking evaluation difficult.Source: §3.2 · p. 7
• On Cityscapes, Stereo DSO produces precise, denser 3D reconstructions than feature-based methods and more accurate ones than prior dense or semi-dense approaches, though its reconstructions are sparser.The authors attribute the accuracy to bundle adjustment in windowed optimization.Source: §3.2 · p. 8

4. Conclusion

• On the KITTI testing set, Stereo DSO provides more accurate results than Stereo ORB-SLAM2 with loop closing and global bundle adjustment, even without closing large loops.The paper also reports that evaluations on KITTI and Cityscapes demonstrate accurate and robust tracking in challenging real-world scenarios.Source: §4 · p. 8
• Stereo DSO integrates temporal multi-view stereo and static stereo within a marginalization framework using the Schur complement.The method is presented as capable of real-time tracking and mapping from a stereo camera.Source: §4 · p. 8
• Future work plans to add loop closing and a map-maintenance database to extend Stereo DSO into a full SLAM system, and to handle dynamic objects explicitly.The authors aim for explicit dynamic-object handling to further improve visual-odometry accuracy and robustness.Source: §4 · p. 8

• It examines how different weighting of static-stereo constraints influences tracking accuracy.Source: Abstract · p. 10
• It presents full KITTI training-set trajectory estimates compared with state-of-the-art monocular visual odometry methods.Source: Abstract · p. 10
• It presents additional Cityscapes Frankfurt results that qualitatively demonstrate the method’s tracking accuracy.Source: Abstract · p. 10
• A supplementary video demonstrates method performance on selected datasets and the quality of its 3D reconstructions.Source: Abstract · p. 10

1. Effect of Stereo Coupling Factor

• Estimated trajectories on KITTI Seq. 06 are shown for coupling factors (λ) 0–3.The trajectories are presented in Fig. 1.Source: §1 · p. 10

• Across KITTI training sequences, monocular ORB-SLAM and DSO show scale drift, while Stereo DSO is aligned with a rigid 6DoF transform rather than their 7DoF similarity transform.Source: §2 · p. 10
• Monocular DSO appears to drift more in scale than ORB-SLAM, which the authors attribute to direct methods’ sensitivity to low frame rates, large optical flow, and unmodeled image effects.The cited effects include non-Lambertian reflectance and illumination changes that were not sufficiently corrected.Source: §2 · pp. 10–11
• On KITTI sequence 06, increasing the static-stereo constraint weight significantly reduces translational drift.Source: Figure 1 · p. 10

3. More Results on Cityscapes

• The full Frankfurt sequence (~107,000 frames) is split into 5,000–6,000-frame sections because the method cannot yet handle moving objects and sudden strong brightness changes across the entire sequence.The sections provide coverage comparable to KITTI sequences; example estimated trajectories and GPS-derived ground truth are shown in Figure 4.Source: §3 · p. 11, Figure 4 · p. 15
• GPS-derived ground-truth poses are visibly inaccurate in some Frankfurt sections, including the example in Figure 4g.The poses are calculated from GPS coordinates using the Mercator projection.Source: §3 · p. 11, Figure 4 · p. 15