PaperlayerView exact revision on arXiv

Source-linked AI summary

Zero-Shot Monocular Depth Estimation for Visual Perception in Lower-Limb Prostheses

Ishi Vijayvergiya, Marc Hölle, Youssef Dawoud, Amir El-Ghoussani, Michele De Vita, Rohan Asthana, Philipp Beckerle, Vasileios Belagiannis

arXiv:2610.35018v1cs.CV

TL;DR

Monocular depth foundation models have shown broad zero-shot generalization, but their performance at the dynamic knee-level viewpoint relevant to lower-limb prostheses has been little explored. The paper introduces the LP dataset and evaluates four models zero-shot; results show effective generalization, with different models leading on stair and corridor scenes.

  • Problem

    How well do monocular depth foundation models generalize zero-shot to the dynamic knee-level viewpoint of a prosthesis, a setting not represented by conventional benchmarks?

    Source: §I · p. 1

  • Method

    The authors introduce synchronized RGB-depth data from an emulated knee-level prosthetic viewpoint and use it to evaluate four monocular depth foundation models zero-shot.

    Source: §I · p. 2

  • Results

    VGGT achieved staircase Abs Rel of 0.1532, while Marigold led corridor performance with Abs Rel of 0.1963; all models showed effective zero-shot generalization.

    Source: A. Quantitative Results · pp. 3–4

  • Takeaways & Limitations

    The results demonstrate the feasibility of zero-shot foundation models as support for downstream prosthetic perception.

    Source: §VI · p. 5

Abstract

from arXiv · show

Monocular depth estimation has advanced rapidly with the development of foundation models that demonstrate strong zero-shot generalisation across diverse visual domains. However, the applicability of these models to visual perception for lower-limb prosthetics remains largely unexplored. Reliable depth perception is particularly important for prosthesis users, who often experience significant challenges when navigating stairs, uneven terrain, and other complex environments. To address this gap, we introduce the Lower-Limb Prosthesis (LP) dataset, a benchmark designed to evaluate depth perception from an emulated knee-level prosthetic viewpoint. Using this dataset, we evaluate four state-of-the-art monocular depth estimation foundation models. Our results provide the first systematic benchmark of monocular depth estimation performance from a prosthetic viewpoint, while highlighting both the challenges and the potential of deploying foundation models in prosthetic perception systems. The code and dataset are available at https://github.com/Ishivijay/lp_dataset.

I. INTRODUCTION

The paper addresses the lack of depth evidence for the unusual, dynamic knee-level viewpoint of lower-limb prostheses, where safe navigation is important. It introduces the LP dataset and evaluates four foundation models zero-shot to benchmark their performance in this setting.

I. INTRODUCTION

  • Reliable environmental perception matters for prosthesis safety, particularly during navigation of stairs, uneven terrain, and cluttered environments.The introduction identifies falls as a significant risk for users in these settings.Source: §I · p. 1
  • Existing depth benchmarks use conventional viewpoints, while prior locomotion datasets lack paired ground-truth depth from a lower-limb-mounted camera.The paper identifies the knee-level perspective, motion patterns, and close-range stair navigation as unrepresented in existing benchmarks.Source: §I · pp. 1–2
  • The LP dataset provides synchronized RGB-depth pairs from an emulated knee-level prosthetic viewpoint during stair ascent, descent, and indoor locomotion.It was collected across six recording sessions and includes staircases, corridors, landings, and doorways.Source: §I · p. 2
  • The study evaluates four state-of-the-art monocular depth foundation models zero-shot and analyzes performance across scene types.The paper names Marigold, Depth Anything V2, Depth Pro, and VGGT among its contributions.Source: §I · p. 2
  • The benchmark is intended to provide empirical evidence about model robustness and limitations for prosthetic-viewpoint perception.The authors describe this as enabling further research in visual perception for medical prostheses.Source: §I · p. 2

A. Monocular Depth Estimation

Monocular depth estimation has progressed from supervised and self-supervised methods to foundation models trained for broader generalization. Wearable-camera datasets and conventional depth benchmarks still offer limited evidence about lower-limb prosthetic viewpoints.

A. Monocular Depth Estimation

  • Earlier monocular depth methods used paired RGB-depth supervision, while self-supervised approaches used photometric consistency across video frames or stereo pairs.The passage describes a subsequent shift toward foundation models designed to generalize across diverse domains.Source: A. Monocular Depth Estimation · p. 2
  • Foundation models use varied approaches: Depth Anything V2 scales labeled and unlabeled training, Marigold uses diffusion denoising, Depth Pro estimates focal length, and VGGT uses vision transformers.Their robustness at unconventional lower-limb prosthetic viewpoints remains largely unexplored.Source: A. Monocular Depth Estimation · p. 2

B. Egocentric and Wearable Vision Datasets

  • Wearable and egocentric datasets capture general perception but not the low viewpoint, reduced camera height, and distinctive motion of prosthetic locomotion.The passage says such datasets therefore provide limited insight into depth-estimation performance in prosthetic navigation.Source: B. Egocentric and Wearable Vision… · p. 2

C. Visual Perception for Assistive Devices

Prior prosthetic perception work has emphasized mechanical design, control, and semantic environmental sensing, while explicit geometric depth perception remains largely unexplored. The LP dataset was collected indoors with knee-mounted RGB and depth cameras to emulate a prosthetic viewpoint.

C. Visual Perception for Assistive Devices

  • Existing prosthetic vision work provides semantic information but does not explicitly capture environmental geometry, which depth-aware perception could use for obstacle detection, step-height estimation, terrain assessment, and path planning.The application of monocular depth estimation to lower-limb prosthetic navigation remains largely unexplored.Source: C. Visual Perception for Assistive… · p. 2

A. Dataset Acquisition and Setup

  • The LP dataset contains 3,144 synchronized RGB-depth pairs across six indoor sequences of stair ascent, stair descent, and corridor walking.Three healthy male participants walked at constant speed in a relatively uncluttered setting; door thresholds, densely furnished spaces, other people, and moving animals were excluded. Four mounting configurations placed the camera above or below either knee.Source: A. Dataset Acquisition and Setup · pp. 2–3, Table I · p. 3

B. Dataset Structure and Formats

The dataset provides multiple processed or visualized depth representations alongside RGB images.

B. Dataset Structure and Formats

IV. EXPERIMENTS

The experiments compare four monocular depth foundation models using zero-shot inference and a shared evaluation pipeline. Per-image scale-and-shift alignment and evaluation metrics are computed over valid ground-truth pixels.

1) Model Configurations:

  • Four models were evaluated: Marigold, Depth Anything V2, Depth Pro, and VGGT, using specified pretrained checkpoints and configurations.Depth Pro produced metric-depth predictions; VGGT used the VGGT-1B checkpoint and official crop-mode preprocessing.Source: §IV · p. 3, 1) Model Configurations · p. 3

2) Evaluation Protocol:

  • All models were evaluated on the complete LP sequences under identical conditions, resizing predictions with bilinear interpolation when dimensions differed from ground truth.The shared pipeline comprised zero-shot inference, depth-map post-processing and alignment, then quantitative evaluation.Source: 2) Evaluation Protocol · p. 3, §IV · p. 3
  • Per-image least-squares alignment estimates a global scale and shift, and metrics are computed only on valid ground-truth pixels (GT > 0).The reported metrics include Abs Rel, Sq Rel, RMSE, RMSElog, and threshold accuracies δ1, δ2, and δ3.Source: 2) Evaluation Protocol · p. 3

A. Quantitative Results

Across the staircase and corridor scenes, all four models generalized to the emulated knee-level viewpoint, but the strongest model differed by scene. Qualitative comparisons show broad agreement in scene structure alongside differences in smoothness and local depth variation.

B. Qualitative Results

  • All models captured global scene geometry and depth gradients, while Depth Anything V2 and VGGT generally produced smoother, more spatially coherent maps than Marigold.Marigold showed finer-grained depth variation around local structures and discontinuities.Source: B. Qualitative Results · p. 4, Figure 2 · p. 4

A. Quantitative Results

  • VGGT led on staircases, while Marigold led across corridor metrics; Marigold and VGGT showed the strongest overall performance, and Depth Anything V2 had the largest errors and lowest threshold accuracies.On staircases, VGGT achieved Abs Rel 0.1532, Sq Rel 0.0818, RMSE 0.3449, and δ1 0.8099; in corridors, Marigold achieved Abs Rel 0.1963, Sq Rel 0.0813, RMSE 0.3357, RMSElog 0.2205, δ1 = 0.7592, δ2 = 0.9371, and δ3 = 0.9669.Source: A. Quantitative Results · pp. 3–4, Table II · p. 5, Table III · p. 5

B. Qualitative Results

  • Depth Pro separated foreground from background, but its outputs were occasionally more over-smoothed than those of the other models.Across the challenging knee-level viewpoint and motion setting, all methods’ predictions remained broadly consistent with scene structure.Source: B. Qualitative Results · p. 5

C. Future Work

The evaluated foundation models generalize well but are too large and computationally demanding for direct deployment on active prosthetic hardware. Future work should explore model optimization, lightweight architectures, and uncertainty and out-of-distribution detection.

C. Future Work

  • Large parameter sizes and high computational demands currently prevent direct deployment of the evaluated foundation models on active prosthetic hardware.Future directions include compression, pruning, and quantization, as well as lightweight monocular architectures for real-time edge processing; gait adaptation requires visual feedback within the swing phase, typically under 400 ms.Source: C. Future Work · p. 5
  • Uncertainty estimation and pixel-wise out-of-distribution detection are identified as essential for flagging unreliable depth predictions and unexpected terrain hazards.The LP dataset is proposed as a benchmark for evaluating optimization impacts under prosthesis-specific operating conditions.Source: C. Future Work · p. 5

VI. CONCLUSION

Four foundation models trained on conventional benchmarks generalise effectively to the emulated prosthetic viewpoint on the LP dataset. The dataset and baseline results provide a foundation for research into prosthetic visual perception and its potential downstream applications.

VI. CONCLUSION

  • Four foundation models trained on conventional benchmarks generalise effectively to the previously unexplored prosthetic viewpoint.The authors describe these baseline results as demonstrating the feasibility of zero-shot foundation models for downstream prosthetic applications.Source: §VI · p. 5
  • The LP dataset pairs synchronised RGB and ground-truth depth data from a knee-level camera mounted on healthy participants.The camera emulates a lower-limb prosthesis perspective during stair navigation and indoor locomotion.Source: §VI · p. 5
  • The benchmark establishes baseline performance and a foundation for future research into visual perception in lower-limb prostheses.Potential downstream applications include obstacle detection, terrain assessment, and adaptive gait planning.Source: §VI · p. 5

Related papers

Loading 2610.35018v1…