PaperlayerView exact revision on arXiv

Source-linked AI summary

StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes

Zhengri Wu, Yiran Wang, Yu Wen, Zeyu Zhang, Biao Wu, Hao Tang

arXiv:2509.16415v1cs.CVcs.RO

TL;DR

Underwater stereo estimation must adapt foundation encoders to a domain with limited labels and combine monocular priors with fragile stereo correspondences. StereoAdapter joins LoRA adaptation with recurrent stereo refinement and synthetic pretraining; it reports RMSE improvements on TartanAir and SQUID over state-of-the-art methods.

  • Problem

    Underwater stereo depth estimation must adapt large foundation encoders with limited labels and fuse scale-ambiguous monocular priors with photometrically fragile stereo correspondences.

    Source: §I · p. 1

  • Method

    StereoAdapter combines a LoRA-adapted monocular foundation encoder with recurrent stereo refinement, dynamic LoRA rank selection, and UW-StereoDepth-40K pretraining.

    Source: §I · pp. 1–2

  • Results

    6.11% on TartanAir and 5.12% on SQUID are reported as improvements in RMSE over state-of-the-art methods.

    Source: §VII · p. 10

  • Takeaways & Limitations

    The authors present the approach as a practical solution for AUV navigation, infrastructure inspection, and marine ecology monitoring.

    Source: §VII · p. 10

    Synthetic supervision has narrower domain breadth and scene co-occurrence statistics than real-world underwater imagery.

    Source: §VI · p. 10

Abstract

from arXiv · show

Underwater stereo depth estimation provides accurate 3D geometry for robotics tasks such as navigation, inspection, and mapping, offering metric depth from low-cost passive cameras while avoiding the scale ambiguity of monocular methods. However, existing approaches face two critical challenges: (i) parameter-efficiently adapting large vision foundation encoders to the underwater domain without extensive labeled data, and (ii) tightly fusing globally coherent but scale-ambiguous monocular priors with locally metric yet photometrically fragile stereo correspondences. To address these challenges, we propose StereoAdapter, a parameter-efficient self-supervised framework that integrates a LoRA-adapted monocular foundation encoder with a recurrent stereo refinement module. We further introduce dynamic LoRA adaptation for efficient rank selection and pre-training on the synthetic UW-StereoDepth-40K dataset to enhance robustness under diverse underwater conditions. Comprehensive evaluations on both simulated and real-world benchmarks show improvements of 6.11% on TartanAir and 5.12% on SQUID compared to state-of-the-art methods, while real-world deployment with the BlueROV2 robot further demonstrates the consistent robustness of our approach. Code: https://github.com/AIGeeksGroup/StereoAdapter. Website: https://aigeeksgroup.github.io/StereoAdapter.

I. INTRODUCTION

Underwater imaging creates a domain shift that complicates adaptation of foundation encoders and fusion of monocular priors with stereo geometry. StereoAdapter addresses these challenges through parameter-efficient adaptation and recurrent stereo refinement, against a backdrop of work on stereo matching, foundation models, and underwater depth estimation.

I. INTRODUCTION

  • Underwater attenuation, scattering, and refraction create domain shift, while adaptation must be parameter-efficient and fuse scale-ambiguous monocular priors with photometrically fragile stereo matches.The paper motivates adapting rather than replacing a foundation encoder and using stereo constraints to correct global-scale or texture-induced errors.Source: §I · p. 1
  • StereoAdapter combines a LoRA-adapted monocular foundation encoder with recurrent stereo refinement, using dynamic rank selection and UW-StereoDepth-40K pretraining.The method uses monocular-prior guidance, photometric reconstruction, occlusion-aware masking, and edge-aware smoothness without dense underwater ground truth.Source: §I · pp. 1–2
  • 6.11% and 5.12% improvements in RMSE on TartanAir and SQUID, respectively, are reported against state-of-the-art methods; BlueROV2 evaluations show consistent robustness.The framework combines a LoRA-adapted monocular foundation encoder with recurrent stereo refinement, dynamic LoRA rank selection, and UW-StereoDepth-40K pretraining.Source: §I · p. 2

II. RELATED WORK

  • Earlier work spans cost-volume and iterative stereo methods, monocular foundation models, and frameworks combining monocular priors with stereo geometry, including Stereo Anywhere.The related-work discussion also names GC-Net, PSMNet, GA-Net, RAFT-Stereo, Depth Anything, and Depth Anything V2 [27].Source: §II · p. 2
  • Self-supervised monocular and stereo methods address scarce annotations, while underwater benchmarks and methods remain constrained by limited stereo labels and complex adaptation needs.TiO-Depth unifies monocular and binocular tasks; existing underwater datasets include UW-Stereo, and prior methods include UWStereo and UWNet and Fast-UWNet.Source: §II · p. 2
  • Terrestrial stereo datasets do not capture underwater challenges, and LoRA offers parameter-efficient adaptation when target-domain data are scarce.The paper motivates combining foundation-model robustness, self-supervised learning, and LoRA for underwater depth estimation.Source: §II · pp. 2–3

III. THE PROPOSED METHOD

StereoAdapter uses a two-stage self-supervised pipeline: a LoRA-adapted monocular foundation model supplies an initial disparity, then recurrent stereo refinement uses stereo evidence to produce metric depth. Its architecture combines multi-scale features, sparse metric anchors, and stereo-consistency losses.

A. Overview

  • The two-stage pipeline first adapts Depth Anything V2 for monocular underwater disparity, then initializes recurrent stereo refinement to produce fine-grained disparities and depth.Only LoRA parameters and decoder components are optimized in the monocular stage while pretrained encoder weights remain frozen.Source: A. Overview · p. 3, B. Architecture · p. 3

B. Architecture

  • LoRA adapts Depth Anything V2’s multi-scale feature pyramids to underwater scenes while preserving pretrained geometric priors.Features are taken from DPT reassemble modules at H/4, H/8, H/16, and H/32 resolutions.Source: B. Architecture · p. 3
  • A multi-scale stereo correlation pyramid supplies coarse-to-fine matching cues, while combined context features initialize and guide recurrent refinement.The context encoder reuses adapted foundation features and adds lightweight CNN features for spatial precision.Source: B. Architecture · pp. 3–4
  • Sparse stereo correspondences verify and refine the monocular stage’s scale, combining globally consistent depths with local metric accuracy.The mono-stage depths are checked against sparse metric measurements, with corrective scale and shift applied when the scale is unreliable.Source: B. Architecture · pp. 3–4
  • A ConvGRU iteratively updates disparity using correlation features, current disparity, and multi-scale context, then upsamples the final disparity and converts it to depth.The stereo objective combines photometric reconstruction, monocular disparity guidance, occlusion handling, and edge-aware smoothness.Source: B. Architecture · pp. 4–5

C. Dynamic LoRA

Dynamic LoRA addresses the differing adaptation needs of network layers by learning importance weights for rank components, encouraging sparsity, and merging retained components into the base weights after adaptation.

C. Dynamic LoRA

  • Fixed-rank LoRA can provide too little adaptation capacity or impose unnecessary parameter overhead when layer requirements vary.Dynamic LoRA adjusts rank according to the importance of individual components.Source: C. Dynamic LoRA · p. 5
  • Learnable importance weights modulate low-rank components, and ℓ1 sparsity regularization drives less important weights toward zero for automatic rank selection.The importance weights are optimized jointly with the low-rank matrices.Source: C. Dynamic LoRA · p. 5
  • Proximal-gradient soft-thresholding handles the sparsity constraint, with the threshold increasing during training to progressively strengthen pruning.Training begins with a dense phase before soft-thresholding activates during the sparse phase.Source: C. Dynamic LoRA · p. 5
  • After adaptation, components with non-zero importance are merged into the base network weights, removing auxiliary-module inference overhead while retaining domain-specific knowledge.Only rank components with significant importance weights contribute to the final update.Source: C. Dynamic LoRA · pp. 5–6

D. Data Synthesis

UW-StereoDepth-40K is synthesized in UE5 to provide diverse underwater stereo data with geometrically consistent views. The pipeline varies camera and environmental conditions and filters and checks the resulting pairs.

Rendering Pipeline

  • UW-StereoDepth-40K uses UE5 ray tracing and global illumination across coral reefs, industrial structures, shipwrecks, and seabed scenes populated with diverse 3D assets.The assets include scanned corals, marine flora, underwater robots, submerged vehicles, and marine structures.Source: D. Data Synthesis · p. 6
  • UE5 renders calibrated left-right camera views with pixel-aligned geometry, avoiding the stereo inconsistency attributed to stochastic diffusion-based generation.The passage contrasts this geometric consistency with diffusion-based methods while retaining photorealism for domain transfer.Source: D. Data Synthesis · p. 6

Environmental Variation

  • Camera baselines vary across 4, 10, 20, and 40 cm to cover stereo configurations used by different underwater ROV platforms.The range spans narrow baselines on compact inspection ROVs and wider stereo separation on survey vehicles.Source: D. Data Synthesis · p. 6
  • The synthesis adds simulated caustics, physics-based floating particles, and depth-dependent color attenuation to represent underwater conditions.Source: D. Data Synthesis · p. 6

Dataset Generation and Quality Control

  • Automatic filtering removes frames with insufficient texture or depth distributions beyond typical underwater robotic ranges (>50m).Source: D. Data Synthesis · p. 6
  • The final dataset contains 40,000 stereo pairs at 1280×960 with dense depth ground truth and semantic masks, checked using stereo similarity metrics and expert inspection.Source: D. Data Synthesis · p. 6

A. Datasets and Evaluation Metrics

UW-StereoDepth-40K supplies synthetic training data, while TartanAir underwater and SQUID provide real-world evaluation sets. Evaluation uses depth-error and threshold-accuracy metrics.

Datasets

Evaluation Metrics

  • Depth accuracy is measured with REL, SQ REL, RMSE, and LOG RMSE, alongside A1, A2, and A3 threshold-accuracy metrics.The A metrics report the percentage of pixels within the specified δ < 1.25^i error thresholds for i = 1, 2, and 3.Source: A. Datasets and Evaluation Metrics · p. 6

B. Implementation Details

The implementation enhances underwater images with MobileIE and initializes the encoder from DepthAnything v2-B. Training proceeds in two stages on UW-StereoDepth-40K.

Visual Augmentation

  • MobileIE enhances underwater images at the input stage to reduce water-induced visual degradation before downstream processing.The text describes MobileIE as lightweight, fast, and suitable for resource-constrained underwater ROVs.Source: B. Implementation Details · pp. 6–7

Evaluation Context

  • Table I reports evaluation on the TartanAir underwater subset.Source: Table I · p. 7

Model Details

  • Training initializes the encoder with DepthAnything v2-B and uses 20 epochs followed by 40 epochs for stereo depth estimation.The stated setup uses AdamW, a constant learning rate of 1 × 10^-4, and batch size 8 on UW-StereoDepth-40K.Source: B. Implementation Details · p. 7

C. Main Results

StereoAdapter reports state-of-the-art zero-shot performance on TartanAir underwater and SQUID, with further gains after TartanAir fine-tuning. Its depth maps are also reported as more visually coherent than baseline outputs.

Evaluation Setup

  • The evaluation includes a combined set of five stereo-matching datasets: Scene Flow, Sintel, ETH3D, InStereo2K, and CREStereo.Source: C. Main Results · p. 7

Overall Results

  • StereoAdapter achieves state-of-the-art zero-shot performance and further gains when fine-tuned with TartanAir and UW-StereoDepth-40K.Source: C. Main Results · p. 7

Quantitative Results

  • RMSE is 2.8947 zero-shot and 2.7834 after fine-tuning on TartanAir underwater; on SQUID, it is 1.8843 and 1.8621, respectively.The reported TartanAir zero-shot results also include REL 0.0527 and A1 94.67%; after fine-tuning, A1, A2, and A3 are 95.12%, 98.36%, and 99.04%. SQUID accuracies are 94.13% (A1), 97.48% (A2), and 98.52% (A3).Source: C. Main Results · p. 7

Overall Results

  • The experiments describe UW-StereoDepth-40K pretraining as valuable for underwater applications and robust zero-shot generalization.Source: C. Main Results · p. 7

Qualitative Results

  • StereoAdapter produces more accurate, visually coherent depth maps than baselines, including better far-range scale estimation.Source: C. Main Results · p. 7

D. Real World Evaluation

The real-world evaluation uses a BlueROV2 stereo platform and compares methods in a single indoor water-tank scene across three obstacle layouts and nine trajectories.

D. Real World Evaluation

  • BlueROV2 carries a Zed 2i stereo camera, with onboard perception and logging on a Jetson Orin NX and low-level actuation handled by an STM32 controller.The experiments use the camera’s built-in synchronization circuitry.Source: D. Real World Evaluation · pp. 7–8
  • Nine synchronized stereo sequences cover dispersed, side-by-side, and clustered obstacle layouts, with three robot trajectories per layout.All methods receive the same rectified stereo pairs, resolution, and preprocessing; disparity outputs are converted to metric depth using calibrated focal length and baseline.Source: D. Real World Evaluation · pp. 8–9
  • Predictions are evaluated against mesh-rendered reference depth in the left-camera frame, excluding pixels without valid reference depth and reporting standard depth metrics over valid pixels.Metrics are averaged across the nine sequences; A1 uses δ < 1.25.Source: D. Real World Evaluation · p. 8

E. Ablation Study

The ablations examine recurrent refinement, Dynamic LoRA, and training settings; the paper also reports onboard latency, discusses limitations, and concludes with benchmark and BlueROV2 results.

E. Ablation Study

  • Four GRU layers achieve the best ablation results, while the final three-layer, 128-hidden-dimension, 32-iteration configuration balances performance and computational efficiency.The ablation varies GRU layer count, hidden dimensions, and refinement iterations.Source: E. Ablation Study · p. 9
  • The ablations cover Dynamic LoRA and training settings; batch size 8 is reported as optimal after learning monocular features before stereo-domain training, with REL=0.051 and RMSE=2.783.The final training configuration uses batch size 8, learning rate 1 × 10−4, 20 first-stage epochs, and 40 second-stage epochs.Source: E. Ablation Study · p. 9, Table VI · p. 9

V. TEST-TIME EFFICIENCY

  • On the Jetson Orin NX, with batch size 1, StereoAdapter is fastest at 1113 ms—327 ms faster than Stereo Anywhere and 702 ms faster than FoundationStereo.It uses a LoRA-adapted DepthAnything-B encoder only for feature extraction.Source: §V · pp. 9–10

VI. LIMITATION AND FUTURE WORK

  • The current local-matching decoder has limited global reasoning in severe turbidity, highlights, and large textureless regions; synthetic training data also leaves a measurable sim-to-real gap.The authors identify limited domain breadth and partially captured underwater phenomena, and propose broader domain randomization, physics-based rendering, and real-sequence self-training.Source: §VI · p. 10

VII. CONCLUSION

  • 6.11% on TartanAir and 5.12% on SQUID: StereoAdapter improves over state-of-the-art methods, while BlueROV2 deployment confirms robust performance with favorable latency.These are the conclusion’s reported benchmark improvements.Source: §VII · p. 10

VIII. APPENDIX: VISUALIZATION

The appendix presents qualitative comparisons on BlueROV2, TartanAir, and SQUID, alongside example image-depth pairs from UW-StereoDepth-40K.

a) Real world evaluation

  • BlueROV2 visualizations show sharper, more geometrically faithful depth maps than competing baselines, especially in textureless water and reflective regions.The comparison includes captured ground truth and competing methods across multiple key frames.Source: §VIII · p. 10, Figure 7 · p. 11

b) Benchmark evaluation

  • TartanAir and SQUID visualizations show fine-grained scene structure with fewer artifacts across diverse synthetic-to-real domain shifts.These are qualitative visualization results.Source: §VIII · p. 10, Figure 8 · p. 12

c) UW-StereoDepth-40K visualization

  • UW-StereoDepth-40K examples vary in viewpoint, illumination, and scene composition.The paper presents representative image-depth pairs from the dataset.Source: §VIII · p. 10, Figure 9 · p. 13
Loading 2509.16415v1…