Source-linked AI summary
BaseBoostDepth: Exploiting Larger Baselines For Self-supervised Monocular Depth Estimation
Kieran Saunders, Luis J. Manso, George Vogiatzis
TL;DR
Self-supervised monocular depth methods use narrow baselines, while wider ones bring brightness and occlusion challenges. BaseBoostDepth uses staged baseline expansion and guided pose and reconstruction strategies; it reports state-of-the-art performance on KITTI and SYNS across depth metrics without extra test-time computation.
Problem
Self-supervised monocular depth methods commonly use narrow baselines, while wider temporal windows introduce brightness inconsistencies and increased occlusions.
Source: §1 · p. 2Method
BaseBoostDepth uses wider monocular baselines with curriculum-inspired training and guided pose estimation, including error-induced reconstructions.
Source: §1 · p. 2, §3.5 · p. 6, §5 · p. 10Results
BaseBoostDepth reports state-of-the-art performance on KITTI and SYNS and improvements in image-, edge-, and point-cloud-based metrics without increased test-time computational cost.
Source: §1 · p. 3, §5 · p. 10Takeaways & Limitations
Brightness-contrast cues enhance depth-boundary definition without edge-based supervision.
Source: §1 · p. 2Brightness-contrast effects may make road signs and lane markings appear as road irregularities, and inverse augmentations did not satisfactorily mitigate this issue.
Source: Brightness-Contrast Cues & Limitations · pp. 19–20
Abstract
from arXiv · showhide
In the domain of multi-baseline stereo, the conventional understanding is that, in general, increasing baseline separation substantially enhances the accuracy of depth estimation. However, prevailing self-supervised depth estimation architectures primarily use minimal frame separation and a constrained stereo baseline. Larger frame separations can be employed; however, we show this to result in diminished depth quality due to various factors, including significant changes in brightness, and increased areas of occlusion. In response to these challenges, our proposed method, BaseBoostDepth, incorporates a curriculum learning-inspired optimization strategy to effectively leverage larger frame separations. However, we show that our curriculum learning-inspired strategy alone does not suffice, as larger baselines still cause pose estimation drifts. Therefore, we introduce incremental pose estimation to enhance the accuracy of pose estimations, resulting in significant improvements across all depth metrics. Additionally, to improve the robustness of the model, we introduce error-induced reconstructions, which optimize reconstructions with added error to the pose estimations. Ultimately, our final depth network achieves state-of-the-art performance on KITTI and SYNS-patches datasets across image-based, edge-based, and point cloud-based metrics without increasing computational complexity at test time. The project website can be found at https://kieran514.github.io/BaseBoostDepth-Project.
1 Introduction
Wider baselines are underexplored in self-supervised monocular depth and bring brightness and occlusion challenges. BaseBoostDepth combines staged baseline expansion with multiple reconstruction and pose strategies to use them.
1 Introduction
- BaseBoostDepth reports state-of-the-art performance on KITTI and SYNS and outperforms Monodepth2 on image- and edge-based metrics.The paper also examines point-cloud metrics and reports that brightness-contrast cues sharpen depth boundaries without edge-based supervision.Source: Figure 1 · p. 2, §1 · pp. 2–3
- The role of baseline width in self-supervised monocular depth has received less attention than in multi-baseline stereo, where narrow baselines yield poorer depth estimates.Source: §1 · p. 2
- Wider temporal baselines can introduce brightness inconsistencies and increased occlusions, complicating their use in self-supervised monocular depth.Monodepth2 instead uses consecutive frames and narrow stereo frames for reconstruction.Source: §1 · p. 2
- Simply combining large and small baselines can bias minimum aggregation toward smaller baselines because their inaccurate depths may still produce lower photometric errors.Source: §1 · p. 2
- BaseBoostDepth transitions from smaller to wider baselines through warmup and boosting, and reconstructs the center frame from past-and-future frame triplets.Source: §1 · p. 3
- Incremental pose estimation breaks large-interval pose estimates into smaller increments, while error-induced reconstructions add controlled error to pose estimates.The latter addresses the observation that incremental pose estimation does not benefit every reconstruction.Source: §1 · p. 3
2 Related Work
The section situates monocular view-synthesis depth estimation among prior methods, then describes BaseBoostDepth’s staged use of wider baselines and its pose and reconstruction strategies.
2 Related Work
- Prior monocular methods use photometric view synthesis, while wider-baseline work identified occlusion and brightness inconsistencies; Madhu et al. addressed brightness for two-frame separation.Source: §2 · p. 3
Method:
- BaseBoostDepth is depth-backbone-agnostic and uses warmup and boosting stages, adding incremental pose estimation and error-induced reconstructions only during boosting.Source: Method · p. 4, Figure 2 · p. 4
3.1 Preliminaries
- Depth and pose networks synthesize a target image from successive source frames through inverse warping, with photometric loss and per-pixel minimum aggregation used for training.Source: §3.1 · pp. 4–5
3.2 Curriculum-learning-inspired Optimization Strategy
- The curriculum treats smaller-baseline reconstructions as easier and larger-baseline ones as harder, progressing through warmup and boosting.Source: Method · p. 4, §3.2 · p. 5
3.3 Tri-minimization:
- Tri-minimization reconstructs the center frame from temporally separated sources to address occlusion, brightness inconsistency, and dynamic objects while encouraging wider baselines.Source: §3.3 · pp. 5–6
3.4 Incremental Pose Estimation
- Incremental pose estimation addresses drift at wider separations, but helps only the nearest source-frame reconstruction; its rotation estimates benefit every reconstruction.Source: §3.4 · p. 6
3.5 Error-induced Reconstructions
- Fixed translation errors create additional reconstructions that broaden depth-network alternatives, alongside standard incremental-pose reconstructions in the final photometric-plus-smoothness loss.Source: §3.5 · pp. 6–7
4 Results:
BaseBoostDepth’s boosting phase improves on the baseline across the reported benchmarks, with pretrained variants achieving the strongest KITTI comparison and gains across SYNS image, edge, and point-cloud metrics. The ablations show that wider baselines alone are challenging, while tri-minimization and pose-handling choices improve the balance of image- and edge-based results.
4.1 Datasets
- KITTI validation uses the official Zhou split, training uses 39,810 images, and testing uses 697 Eigen images; SYNS-Patches supplies 1,438 outdoor images with measured depth.KITTI assessment focuses on image-based metrics, while SYNS evaluation includes edge- and point-cloud-based metrics; the training setup uses pretrained ImageNet weights and separate warm-up and boosting stages.Source: §4.1 · p. 7, §4.2 · p. 7, §4 · p. 7
4.2 Ablation Study:
- Using up to four-frame separation with standard minimum aggregation yields no significant quantitative improvement; larger baselines face occlusion, brightness changes, and dynamic objects.The four-frame variant improves edge-based metrics despite declining image-based metrics.Source: §4.2 · p. 8
- Curriculum optimization retains edge gains but has worse image metrics than the one-frame baseline; tri-minimization improves image metrics and surpasses the four-frame variant on edge metrics.Incremental pose alone further declines image metrics while increasing edge metrics; partial incremental pose reduces both, and error-induced reconstruction with α = 5.5 further decreases both metric types.Source: §4.2 · p. 8
4.3 Comparison with SotA
- Pretrained boosting variants outperform the original approach, and BaseBoostDepth†pre sets a new state-of-the-art benchmark at the evaluated resolution with a MonoViT backbone.Table 2 reports KITTI Eigen results without test-time refinement, using single-frame inference at 640 × 192 and no edge supervision.Source: §4.3 · p. 9, Table 2 · p. 9
4.4 Evaluation of Edge and Point Cloud Performance
- On SYNS, boosting improves image-, edge-, and point-cloud metrics across backbones; pretrained BaseBoostDepth maintains similar image metrics to Monodepth2 while improving edge and point-cloud results.BaseBoostDepth†pre leads on the SYNS dataset but is slightly behind BaseBoostDepth in edge composition; backbone choice depends on user requirements.Source: §4.4 · pp. 9–10, Table 3 · p. 9
5 Conclusion
The study concludes that wider baselines can improve self-supervised monocular depth metrics when paired with curriculum-inspired optimization and guided pose estimation. The boosting strategy is backbone-agnostic and does not increase test-time computational cost.
5 Conclusion
- Wider baselines improve image-, edge-, and point-cloud-based metrics in the study’s self-supervised monocular depth setting.The conclusion attributes the edge improvements to curriculum-inspired optimization and carefully guided pose estimation.Source: §5 · p. 10
- The boosting strategy can start from a warm-up phase or pretrained weights and is depth-backbone-agnostic.Source: §5 · p. 10
- The reported improvements do not increase computational cost at test time.Source: §5 · p. 10
7 Hyperparamter Tuning
The error-induced reconstruction tuning balances KITTI image-based metrics against SYNS edge-based metrics by varying the pose-error parameter α. The selected α depends on which metric emphasis is preferred.
7 Hyperparamter Tuning
- Table 4 evaluates depth on KITTI and SYNS using the main-paper parameters and extends the error-induced reconstruction results from Table 1, row 7.Source: Table 4 · p. 14
- Error-induced reconstruction introduces controlled pose-estimation error into image reconstruction, and α tunes that error to balance image- and edge-based metrics.Source: §7 · p. 14
- α = 5.5 yields the most favorable KITTI image-based metrics, while α = 5 gives the best SYNS edge detail.The authors select α = 5.5 to exceed the baseline in image metrics and improve edge results relative to row 7 of Table 1, while noting it can be retuned for edge emphasis.Source: §7 · p. 14
8 Incremental Pose Estimation
Incremental pose estimation reduces the error growth associated with larger frame separations, although error continues to rise. The ablation visualization also shows that boundary quality varies across the pose and reconstruction choices.
8 Incremental Pose Estimation
- Standard pose error grows exponentially after three-frame separation, whereas incremental pose error grows linearly with smaller variance.The comparison uses ground-truth pose on KITTI odometry sequence 9 and averages over one frame; error still increases at larger separations.Source: §8 · p. 14, Figure 4 · p. 15
9 Ablation Visualisation
- Larger baselines improve depth boundaries, but high brightness-contrast areas can make nearby objects appear farther away.Source: §9 · p. 15
- The visualization shows fewer boundaries with incremental pose, recovery with partial incremental pose, and further edge accentuation with error-induced reconstructions.Source: §9 · p. 15
10 Evaluation Metrics
The section defines the depth metrics used for KITTI and SYNS-Patches. SYNS-Patches evaluation includes both edge-based and 3D reconstruction metrics.
10.1 KITTI Eigen
- KITTI evaluation uses AbsRel, Sq Rel, RMSE, RMSE Log, and threshold accuracy for δ < 1.25^k, where k ∈ 1,2,3.Ground-truth depth is d and predicted depth is ˆd.Source: §10 · p. 15, §10.1 · pp. 16–17
10.2 SYNS-Patches
- SYNS-Patches 3D evaluation uses Chamfer Distance, precision, recall, F-Score, and IoU to assess point-cloud agreement and reconstruction quality.Precision and recall use a distance threshold δ = 0.1; the metrics follow [17] and its application in [23].Source: §10.2 · p. 17
- The authors note that SYNS ground truth is not easy to use and say they will release its creation steps and evaluation code for reproducibility.The metrics are informed by the SYNS analysis in [23].Source: §10.2 · pp. 17–18
- SYNS-Patches edge evaluation considers accuracy and completeness of predicted depth boundaries using an Euclidean Distance Transform threshold of δ = 10.The metrics are from [13], following [23].Source: §10.2 · pp. 17–18
11 Eigen Benchmark
On the improved-ground-truth KITTI Eigen test dataset, BaseBoostDepth remains competitive against prior methods with equivalent backbones. The paper also reports that its variants generally improve image-, edge-, and point-cloud-based results on SYNS.
11 Eigen Benchmark
- BaseBoostDepth remains competitive across metrics on improved-ground-truth KITTI with equivalent backbones; its variants generally improve image-, edge-, and point-cloud-based results on SYNS.The KITTI results use no test-time refinement and require one frame for depth inference.Source: Table 5 · p. 18, §11 · p. 18
- KITTI benchmark results are reported for the improved-ground-truth Eigen test dataset.The table caption specifies no test-time refinement and one-frame inference.Source: Table 5 · p. 18
12 Ethical Considerations
The paper identifies privacy concerns in KITTI and SYNS-Patches and cautions that their Western data sources constrain geographic and cultural generalization. It also reports that brightness-contrast cues can sharpen boundaries but can mislead depth estimates, with lane-marking errors unresolved by the tested inverse augmentations.
12 Ethical Considerations
- The Western sourcing of KITTI and SYNS-Patches makes extrapolation to other geographic regions and cultural contexts challenging.The authors also note privacy concerns because pedestrians, cyclists, drivers, and registration plates may appear without explicit consent.Source: §12 · p. 18
Brightness-Contrast Cues & Limitations
- BaseBoostDepth shows a moderate brightness-disparity correlation—described as four times Monodepth2’s cue impact—mainly in high-frequency regions, with little effect in low-frequency regions.The effect is observed particularly in high-contrast regions near the camera.Source: Brightness-Contrast Cues & Limitations · p. 19
- Brightness-contrast cues can overemphasize high-contrast regions, producing closer estimates for bright pixels and misranking foreground branches behind a white background.Figure 7 illustrates this contrast-related error.Source: Brightness-Contrast Cues & Limitations · p. 19, Figure 7 · p. 20
- Lane markings may be misread as road irregularities, and inverse augmentations did not provide a satisfactory fix, leaving the issue for future work.The authors nevertheless report improvements across all evaluation metrics and overall performance over baseline methods.Source: Brightness-Contrast Cues & Limitations · pp. 19–20
14 Final Results
The qualitative results compare BaseBoostDepth variants with Monodepth2 and MonoViT on the KITTI test split. The authors report sharper edge definitions and finer detail, including on fences and shrubbery.
14 Final Results
- BaseBoostDepth produces sharper edge definitions and finer details that are often blurred in current state-of-the-art methods.The comparisons include Monodepth2 and MonoViT, as well as pretrained BaseBoostDepth variants.Source: §14 · p. 20, Figure 8 · p. 21
- The KITTI test-split visual comparison includes BaseBoostDepth, its pretrained Monodepth2 and MonoViT variants, and the Monodepth2 baseline.Figure 8 shows these qualitative results.Source: Figure 8 · p. 21
- Additional qualitative examples show sharper fence and shrubbery structures.These examples are presented in Figure 9.Source: Figure 9 · p. 22