PaperlayerView exact revision on arXiv

Source-linked AI summary

On the uncertainty of self-supervised monocular depth estimation

Matteo Poggi, Filippo Aleotti, Fabio Tosi, Stefano Mattoccia

arXiv:2005.06209v1cs.CV

TL;DR

Self-supervised monocular depth estimation avoids ground-truth labels, but uncertainty estimation for its predictions remains insufficiently explored despite the task’s practical importance. The paper evaluates uncertainty methods across self-supervised paradigms and introduces Self-Teaching, which is designed for this setting. The experiments report improved depth accuracy and strong uncertainty-estimation performance, with outcomes depending on the supervision strategy.

  • Problem

    Uncertainty estimation for self-supervised monocular depth networks is insufficiently explored, although failure detection matters for practical applications and the task is ill-posed.

  • Method

    The paper reviews and evaluates uncertainty-estimation techniques and introduces a Self-Teaching paradigm tailored to self-supervised monocular depth estimation.

  • Results

    The experiments report that Self-Teaching always improves depth accuracy, while uncertainty-estimation outcomes vary across supervision strategies and settings.

  • Takeaways & Limitations

    Uncertainty modelling can improve depth accuracy and provides a basis for identifying wrong predictions in self-supervised monocular depth estimation.

  • Takeaways & Limitations

    Predictive uncertainty modelling is sub-optimal when monocular training leaves pose unknown because the formulation jointly explains uncertainty for depth and pose.

Abstract

from arXiv · show

Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all. Despite the astonishing results yielded by such methodologies, learning to reason about the uncertainty of the estimated depth maps is of paramount importance for practical applications, yet uncharted in the literature. Purposely, we explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy, proposing a novel peculiar technique specifically designed for self-supervised approaches. On the standard KITTI dataset, we exhaustively assess the performance of each method with different self-supervised paradigms. Such evaluation highlights that our proposal i) always improves depth accuracy significantly and ii) yields state-of-the-art results concerning uncertainty estimation when training on sequences and competitive results uniquely deploying stereo pairs.

1. Introduction

Self-supervised monocular depth estimation avoids costly ground-truth depth annotations but remains ill-posed and safety-critical when it fails. This section motivates uncertainty estimation and introduces the paper’s evaluation and Self-Teaching contribution.

  • Depth estimation supports autonomous driving and augmented reality, while standard cameras offer lower cost, higher resolution, and higher frame rates than active sensors.
  • Single-image depth estimation is attractive because it avoids multi-view limitations, but its ill-posed nature makes the task extremely challenging.
  • Self-supervised learning removes the need for expensive depth labels by leveraging image reprojection, with monocular sequences additionally requiring unknown scale and camera poses.
  • Uncertainty estimation matters because incorrect distance estimates for pedestrians or vehicles can have dramatic consequences in autonomous driving.
  • The paper investigates how self-supervised training paradigms affect uncertainty and depth estimation, introducing Self-Teaching to model uncertainty and improve depth accuracy when pose is unknown.
  • Experiments on KITTI comprehensively evaluate uncertainty-estimation approaches tailored to self-supervised monocular depth estimation.

3. Depth-from-mono and uncertainty

Self-supervised monocular depth replaces ground-truth depth with image reconstruction, but uncertainty modelling must account for the indirect supervision and unknown pose. The section presents empirical and predictive approaches, including Self-Teaching to decouple depth and pose.

  • Self-supervised formulation: Self-supervised depth uses reconstructed images instead of ground-truth maps, with camera intrinsics and relative pose mediating the training loss.Images may come from a moving monocular camera or a calibrated stereo rig; monocular pose is usually learned jointly with depth.
  • Self-supervised formulation: Because supervision targets image reconstruction rather than depth directly, predictive uncertainty methods can jointly model uncertainty in depth and pose when monocular pose is unknown.The resulting formulation is described as sub-optimal for unknown pose, whereas stereo supervision has a single unknown depth parameter.
  • Empirical estimation: Image flipping estimates uncertainty from disagreement between depth predictions for an image and its horizontally flipped counterpart.The method can be applied to a pretrained framework without architectural modification, but requires twice as many test-time forward passes.
  • Empirical estimation: Dropout, bootstrapped, and snapshot ensembles estimate empirical uncertainty through prediction variance across sampled or separately trained network instances.Dropout samples networks at test time, bootstrapping trains specialized models on random data subsets, and snapshots reuse one training run with cyclic learning rates.
  • Predictive estimation: Learned Reprojection trains an uncertainty output to mimic the self-supervised per-pixel loss, capturing ambiguities such as occlusions and low texture without ground-truth labels.The approach relies on the loss being high when estimated depth is wrong.
  • Predictive estimation: Self-Teaching trains a teacher network self-supervised and a same-architecture student to mimic the teacher’s noisy depth distribution, decoupling depth from pose.The authors report that the student is more accurate than the teacher and that uncertainty estimation becomes more effective under monocular supervision.

4. Experimental results

The experiments evaluate joint depth and uncertainty estimation across monocular, stereo, and combined self-supervision on KITTI. Self generally improves depth accuracy and uncertainty modelling, with performance depending on whether pose and scale are known.

  • Evaluation protocol: The study evaluates self-supervised strategies for joint depth and uncertainty estimation on KITTI using monocular, stereo, and combined supervision.Models use the Eigen split with improved ground truth and standard depth and uncertainty metrics.
  • Monocular supervision: Self improves all reported depth metrics over the baseline under monocular supervision, including after post-processing.Other predictive methods produce worse depth estimates, while empirical methods generally fail to improve most metrics.
  • Monocular supervision: Boot+Self achieves the best overall uncertainty results under monocular supervision.Predictive methods perform better than empirical methods, and combining empirical and predictive methods often improves over single choices.
  • Stereo supervision: Under stereo supervision, Self improves the baseline on all depth metrics but does not outperform Post, whereas Log performs best for uncertainty estimation.Known pose allows Log to model only depth uncertainty, while Self ranks second for uncertainty despite more accurate depth.
  • Monocular+stereo supervision: With combined monocular and stereo supervision, Self consistently outperforms the baseline and is competitive with Post for depth accuracy.Self also outperforms Log for uncertainty estimation, while Self-based Bayesian combinations are typically better than their Log counterparts.
  • Sparsification curves: Sparsification curves show that Self-based methods perform best with monocular and combined supervision, while Log performs better with stereo supervision.The curves report average RMSE for M, S, and MS supervision settings.

5. Conclusion

The paper investigates uncertainty modelling in self-supervised monocular depth estimation and introduces a Self-Teaching paradigm. Results show that the best uncertainty strategy depends on supervision, while uncertainty estimation consistently improves depth accuracy.

  • The study evaluates up to 11 strategies for estimating uncertainty in self-supervised monocular depth predictions.
  • The proposed Self technique performs best when pose is unknown, a setting considered especially appealing for practical applications.
  • Among empirical methods, Dropout sampling performs well with unknown scale but fails when scale is known.
  • Uncertainty estimation consistently improves depth accuracy across all training paradigms.

– Supplementary material

The supplementary document provides expanded experimental results for the paper and simplifies the terminology by treating self-supervision as supervision.

  • The listed authors are Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia.
  • The document reports more detailed results for experiments from “On the uncertainty of self-supervised monocular depth estimation”.
  • The supplementary text often uses “supervision” as a simplified term for self-supervision.

1. Insights about sparsification over different metrics

Sparsification evaluates whether uncertainty modelling identifies erroneous pixels so that removing them reduces depth-map errors or outlier rates.

  • Abs Rel and RMSE sparsification assess how well uncertainty identifies pixels with the largest error magnitudes.
  • Removing pixels ranked as highly erroneous can reduce the overall average depth error.
  • δ ≥1.25 sparsification treats pixels whose estimates differ from ground truth by more than a 1.25 ratio as outliers.
  • Under δ ≥1.25 sparsification, the objective is to reduce the percentage of outliers in each depth map.

2. Detailed depth evaluation

The supplementary evaluation reports standard depth metrics for Monodepth2 variants on the reduced Eigen test split under monocular, stereo, and combined supervision.

  • The evaluation covers seven metrics traditionally used for depth estimation.
  • Results use the Eigen test split with improved ground truth and are reduced from 697 to 652 images because ground truth is unavailable for all images.
  • Tables 1, 2, and 3 report results for monocular, stereo, and monocular-plus-stereo supervision, respectively.

3. Sparsification curves

Sparsification Error curves compare uncertainty quality across monocular (M), stereo (S), and monocular+stereo (MS) supervision for Abs Rel, RMSE, and δ ≥1.25. Self-Teaching outperforms log-likelihood maximization for M and MS, whereas Log performs better for S.

  • 3. Sparsification curves: Figures 1–3 report Sparsification Error curves for Abs Rel, RMSE, and δ ≥1.25 under M, S, and MS supervision.The curves evaluate all three metrics across the three supervision paradigms.
  • 3. Sparsification curves: Self-Teaching strategies outperform traditional log-likelihood maximization under M and MS supervision.This pattern is consistent across the evaluated metrics.
  • 3. Sparsification curves: Log-likelihood maximization yields better results than Self-Teaching under S supervision.The supervision-dependent reversal is visible in the sparsification curves.

4. Depth evaluation – 50 meters cap

The evaluation caps the maximum depth at 50 meters and compares this setting with the standard 80-meter evaluation. The reported margin between 80 and 50 meters is much lower when using improved ground truth than when evaluating raw LiDAR traditionally.

  • 4. Depth evaluation – 50 meters cap: The margin between evaluating at 80 or 50 meters is much lower than in traditional raw-LiDAR evaluation.This comparison follows the evaluation protocol used in previous works.
  • 4. Depth evaluation – 50 meters cap: The 50-meter evaluation is reported consistently with previous works that cap the depth range.The passage contrasts this setting with evaluation on raw LiDAR traditionally performed by existing works.
  • 4. Depth evaluation – 50 meters cap: 50-meter depth caps are evaluated on the Eigen test split with improved ground truth for M, S, and MS supervision.Tables 4–6 provide the corresponding depth evaluations.

5. Uncertainty evaluation – 50 meters cap

Uncertainty modelling is evaluated with a 50-meter maximum depth under M, S, and MS supervision. The results and sparsification curves retain the same supervision-dependent behavior observed at 80 meters.

  • 5. Uncertainty evaluation – 50 meters cap: The same uncertainty-modelling behavior occurs with maximum depths of 50 and 80 meters.This comparison is made against Tables 1–3 from the main paper.
  • 5. Uncertainty evaluation – 50 meters cap: The 50-meter uncertainty evaluation uses improved ground truth rather than the raw-LiDAR evaluation reported separately.The supplied tables identify the improved-ground-truth protocol for M, S, and MS, while the raw-LiDAR tables are listed separately.
  • 5. Uncertainty evaluation – 50 meters cap: Sparsification curves confirm that the behavior observed at 80 meters is preserved at 50 meters for M, S, and MS.Figures 4–6 report the corresponding curves.

8. Uncertainty evaluation – raw LiDAR (80 meters)

Raw-LiDAR uncertainty evaluation at 80 meters compares Self and Log variants across M, S, and MS supervision. Self is better for M and MS, while Log is better for S, including RMSE sparsification.

  • 8. Uncertainty evaluation – raw LiDAR (80 meters): Raw LiDAR is used as ground truth for uncertainty-modelling evaluation under M, S, and MS supervision.Tables 13–15 summarize the results for the three supervision settings.
  • 8. Uncertainty evaluation – raw LiDAR (80 meters): Self solutions outperform Log solutions for M and MS supervision.The passage reports this pattern directly for the raw-LiDAR evaluation.
  • 8. Uncertainty evaluation – raw LiDAR (80 meters): Log outperforms Self under S supervision, particularly for RMSE sparsification.Figures 7–9 provide sparsification curves for the corresponding supervision settings.

10. Qualitative results

The paper presents qualitative depth and uncertainty results across eleven strategies, three KITTI Eigen-split images, and monocular, stereo, or combined supervision. Depth and uncertainty are visualized with standardized colormaps, while supplementary video highlights supervision-dependent behavior.

  • Qualitative comparisons: The qualitative figures report depth and uncertainty maps for variants trained with monocular, stereo, or combined supervision across the selected KITTI images.The reported examples are intended to illustrate outcomes from the different evaluated methods.
  • Visual encodings: Depth maps use the magma colormap, while uncertainty maps use hot, ranging from far to close depth and low to high uncertainty.These encodings are introduced to make the qualitative visualizations easier to read.
  • Qualitative-results organization: Eleven strategies are compared by reporting both depth and corresponding uncertainty maps in a common qualitative-results layout.Figure 11 provides the legend explaining the contents of each qualitative-results cell.
  • Observed behavior: Supplementary video observations indicate that Drop provides reasonable uncertainty estimation with monocular supervision but fails with stereo and combined supervision.The video uses the KITTI 2011 09 26 drive 0101 sync sequence and presents results in M, S, and MS order.
Loading 2005.06209v1…