PaperlayerView exact revision on arXiv

Source-linked AI summary

Image-Space Refraction Correction for Underwater 3D Reconstruction: Warping Flat-Port Views into Pinhole Perspective

Chelim Lim, Tobias Fischer, Emilio Olivastri, Beverley Gorry, Michael Milford, Alejandro Fontan

arXiv:2610.07788v1cs.CV

TL;DR

Flat-port refraction distorts underwater geometry and can undermine the accuracy needed for mapping and navigation. The paper ray-traces this distortion and corrects it in image space before reconstruction. The correction improves geometric fidelity against conventional and refractive SfM and transfers across vision pipelines.

  • Problem

    Flat-port refraction violates the pinhole assumption and can create bent, bowl-shaped reconstructions that are unsuitable for localization, navigation, and ecological monitoring.

    Source: §I · p. 1

  • Method

    A physics-based image-space correction ray-traces refractive displacement and warps images to their equivalent perspective appearance before downstream vision tasks.

    Source: §I · pp. 1–2

  • Results

    Geometric fidelity improves by up to 10× over conventional and refractive SfM, and the approach achieves the highest registered image ratio and geometric fidelity across evaluated sequences.

    Source: §I · p. 2, B. Reconstruction Completeness and… · p. 7

  • Takeaways & Limitations

    Corrected images can be used with standard 3D reconstruction or VSLAM pipelines without modifying them.

    Source: §VII · p. 7

    The method tunes one operational scene distance using prior scene-structure knowledge, though that distance should instead be determined per pixel; accurate underwater reconstruction ground-truth datasets are also lacking.

    Source: §VII · p. 8

Abstract

from arXiv · show

Consumer-grade cameras in flat-port housings are widely used for underwater exploration and mapping of coral reefs and seafloor habitats due to their low cost and accessibility. However, refraction at flat-port interfaces causes bowl-shaped deformation in reconstructed scenes and camera trajectories, compromising the metric accuracy required for mapping and navigation. To remove the dominant refractive distortion before reconstruction, we introduce a physics-based refraction correction in image space. Our method is downstream-agnostic: the refraction-corrected images can be directly used as input to existing reconstruction and SLAM algorithms. We characterize the refractive distortion through ray-tracing simulations and validate our correction on two real underwater datasets with differing scene structures. Compared with conventional and refractive Structure-from-Motion (SfM), our approach removes reconstruction deformation while registering more frames and maintaining low reprojection error. The correction further generalizes across diverse reconstruction and VSLAM backends, demonstrating its broad applicability to downstream vision pipelines.

I. Introduction

Refraction through flat-port housings violates the pinhole assumption and can deform underwater reconstructions, while existing correction strategies either approximate refraction or couple its parameters to reconstruction. The paper instead ray-traces and corrects distortion in image space before downstream vision tasks.

I. Introduction

  • Flat-port refraction can produce bent, bowl-shaped scenes and camera trajectories, making reconstructions unsuitable for localization, navigation, and habitat monitoring.Light bends at the camera–housing interface, invalidating the single-viewpoint assumption.Source: §I · p. 1
  • Conventional distortion parameters only approximate refraction, which is not perfectly radial and varies with scene depth; jointly refining them can hinder convergence and introduce systematic errors.The approximation is especially problematic when parameters are estimated with camera pose and structure.Source: §I · p. 1
  • Dedicated refractive camera models can improve accuracy over perspective approximations, but jointly handling refraction, pose, and structure increases coupled variables and hinders convergence.The paper contrasts this with image-space correction, which separates refraction correction from reconstruction.Source: §I · p. 1
  • Corrected images can be used with existing reconstruction, localization, and mapping methods without modifying their camera models or optimization procedures.The stated design goal is to simplify optimization and improve geometric fidelity and reconstruction robustness.Source: §I · pp. 1–2
  • The proposed physics-based preprocessing ray-traces light through the housing interfaces and inverts the displacement into a backward warp that gives images an equivalent perspective appearance.The correction is reported to improve geometric fidelity by up to 10× over conventional and refractive SfM and to transfer across diverse vision pipelines.Source: §I · p. 2

A. Underwater Camera Modeling and Refraction

  • A flat-port camera has no single viewpoint, and exact compensation varies with scene depth; radial components can nevertheless be partially approximated by conventional intrinsics and distortion parameters.Self-calibration degrees of freedom can trade off, with doming most pronounced on near-planar scenes and amplified by near-parallel viewing.Source: A. Underwater Camera Modeling and… · p. 2

B. Refractive Structure-from-Motion

Prior refractive SfM methods incorporate refraction into reconstruction geometry, while other underwater approaches adjust conventional camera models or correction terms. This paper instead corrects refraction as image-domain preprocessing so standard pipelines can be used without modification.

B. Refractive Structure-from-Motion

  • Refractive SfM methods incorporate refraction into reconstruction geometry, including a refractive fundamental matrix [2], a dedicated framework, and an extension of COLMAP.These methods reformulate estimation around the refractive model.Source: B. Refractive Structure-from-Motion · p. 2

C. Camera Calibration Underwater

  • Other approaches represent refraction through conventional perspective models, physical housing calibration, or virtual-pinhole and radial-distortion components,,,.The paper distinguishes its approach from proposing a new distortion model.Source: C. Camera Calibration Underwater · p. 2
  • The proposed image-domain preprocessing investigates whether correcting refraction improves reconstruction robustness and geometric fidelity while remaining compatible with standard pipelines.State-of-the-art reconstruction methods can be used without modification.Source: C. Camera Calibration Underwater · p. 2, B. Refractive Structure-from-Motion · p. 2

III. Methodology

The method models flat-port camera geometry and traces rays through the housing interfaces to derive refractive displacement. The model accounts for scene depth and housing parameters.

A. Refractive Camera Geometry

  • A flat port is modeled with parallel air–glass and glass–water interfaces, with projection depending on scene depth and housing geometry.Source: A. Refractive Camera Geometry · p. 3

B. Forward Refractive Displacement Model

  • Rays are traced from image pixels through the air–glass and glass–water interfaces, applying Snell’s law at each boundary to determine their water-side direction.Source: B. Forward Refractive Displacement Model · p. 3
  • The parameter sensitivity panels vary depth, housing-to-camera distance, their ratio, water refractive index, glass thickness, or focal length while holding other parameters nominal.Source: Figure 3 · p. 4

C. Backward Refraction Warping

The refractive displacement is inverted into a backward resampling map that warps the raw image toward its perspective appearance. A scale factor adjusts the corrected projection to limit image shrinkage.

  • The backward resampling map relocates refracted pixels toward their perspective positions to correct the input image.Source: C. Backward Refraction Warping · p. 4
  • Because the housing narrows the captured field of view, a scale factor adjusts corrected focal length and output dimensions to keep corrected-image scale closer to the original.Source: C. Backward Refraction Warping · p. 4
  • For each corrected-image pixel, the method back-projects using corrected intrinsics, finds the raw pixel whose ray-traced intersection matches the target, and samples its intensity by bilinear interpolation.Source: C. Backward Refraction Warping · p. 4

D. Depth Assumption in Practice

Experiments use one constant depth per sequence rather than a separate depth for every pixel. Scenes with substantial depth variation are only approximately corrected under this assumption.

E. Simulation and Sensitivity Analysis

The sensitivity analysis finds that refractive displacement depends most strongly on the ratio of scene depth to housing-to-camera distance. Example images show raw and corrected captures from both datasets.

IV. Experimental Setup

The experiments compare image-space correction followed by conventional COLMAP against conventional and refractive COLMAP. They use real underwater datasets spanning different capture platforms and scene geometries.

A. Baselines

  • OURS + COLMAP applies physics-based image correction before conventional COLMAP, unlike Refractive COLMAP, which jointly optimizes refraction, poses, and structure.COLMAP (ref) is the perspective baseline with refined radial-tangential distortion parameters.Source: A. Baselines · p. 5
  • The baseline comparison tests five VSLAM systems and two learning-based multi-view models on raw and corrected images.The systems span handcrafted and learned features, RNNs, and transformers.Source: A. Baselines · p. 5

B. Datasets

  • The Malaysia and Lizard Island datasets contrast diver-handheld with robot-mounted capture and near-planar with topographically varied scenes.Malaysia camera-to-seafloor distance varies across passes; the Lizard Island vehicle maintains a fixed height above the water surface.Source: B. Datasets · p. 5

C. Evaluation Metrics

Because absolute ground truth is impractical at survey scale, the evaluation combines structural consistency with standard Structure-from-Motion statistics. Structural deformation is measured against a sequence-specific planar reference, while SfM statistics describe registration and model fit.

C. Evaluation Metrics

  • Absolute ground truth is impractical for these surveys, so reconstructions are evaluated using structural consistency and standard SfM statistics.Underwater GPS is unavailable, and more precise positioning or scanning systems are expensive and impractical to deploy in the field.Source: C. Evaluation Metrics · p. 5
  • Structural deformation is measured against a fitted plane: Malaysia uses reconstructed scene points, while Lizard Island uses camera centers.The reference reflects each dataset’s near-planar seafloor or constant-altitude camera trajectory.Source: C. Evaluation Metrics · p. 5
  • Signed point-to-plane residuals are normalized by projected scene extent to account for arbitrary monocular scale.The plane is fitted by SVD, with its normal oriented toward the mean camera center.Source: C. Evaluation Metrics · p. 5
  • A radially symmetric paraboloid models systematic bowl or dome curvature; κ = 0 is flat, and |κ| measures deformation strength.Negative κ indicates a bowl and positive κ a dome.Source: C. Evaluation Metrics · pp. 5–6
  • SfM statistics include registered image ratio and point count, mean track length, and reprojection error; lower reprojection error does not guarantee more accurate geometry.Track length is the average number of images observing a reconstructed point.Source: C. Evaluation Metrics · p. 6

V. Results

Across the Malaysia and Lizard Island datasets, the comparison shows more complete trajectories and less bending with corrected images. Reconstructions use the operational depth selected in the ablation study.

V. Results

  • Across both datasets, OURS + COLMAP produces more complete trajectories with fewer camera gaps and more geometrically consistent reconstructions without bending.The comparison is against COLMAP (ref).Source: Figure 5 · p. 6, §V · p. 6
  • The figure compares 3D point clouds and camera trajectories reconstructed with COLMAP (ref) and OURS + COLMAP.The comparison spans the Malaysia and Lizard Island datasets.Source: Figure 5 · p. 6
  • The reported reconstructions use operational depth Z0 selected in the ablation study.The study also uses calibrated in-air intrinsics and standard flat-port parameters: 2 mm glass thickness and 2 cm housing-to-camera distance.Source: §V · p. 6

A. Geometric Fidelity via Structural Deformation

Image-space correction yields the smallest structural curvature across both datasets, while improving registration coverage and point-track statistics. The section also examines depth selection, backend generalization, and runtime.

A. Geometric Fidelity via Structural Deformation

  • COLMAP (ref) has κ from −0.06 to +0.19 in Malaysia and −0.37 to +0.31 in topographically varied Lizard Island scenes.The Malaysia scene’s generally planar structure makes refraction’s radial effect more amenable to radial distortion parameters.Source: A. Geometric Fidelity via Structural… · pp. 6–7
  • The depth ablation reconstructs scenes after undistorting images with candidate Z0 values and compares normalized planarity-residual distributions.The figure caption identifies this as the procedure for selecting operational depth.Source: Figure 7 · p. 7
  • Refractive COLMAP does not consistently outperform COLMAP (ref), with κ ranges of +0.10 to +0.69 in Malaysia and −0.45 to +0.07 in Lizard Island.The authors suggest that explicit refraction modeling may hinder optimization convergence and lead to local minima.Source: A. Geometric Fidelity via Structural… · p. 7
  • The correction produces the smallest structural curvature across sequences: κ ranges from −0.07 to +0.08 in Malaysia and −0.02 to +0.05 in Lizard Island.The planarity maps show the strong dome pattern for Refractive COLMAP and its elimination with Ours + COLMAP in Malaysia p1_s02.Source: A. Geometric Fidelity via Structural… · p. 7, Figure 6 · p. 7

B. Reconstruction Completeness and Model Fit

  • The correction achieves the highest registered image ratio across sequences, highest point count in 9/10 sequences, and highest mean track length across all sequences.The authors interpret the track-length gain as more consistent observation of recovered points across views, strengthening reconstruction constraints.Source: B. Reconstruction Completeness and… · p. 7
  • Table III reports trajectory planarity and registered-image counts for corrected inputs across 3D reconstruction and VSLAM backends.The caption identifies the table as a generalization evaluation.Source: Table III · p. 7
  • The method has the fastest runtime across all stages and the fastest total runtime despite adding refractive undistortion.Source: Table IV · p. 7

C. Generalization Across 3D Reconstruction and VSLAM

The correction supports standard reconstruction and VSLAM backends, while the ablations examine operational scene depth and computational performance. The conclusion identifies scene-depth selection and the lack of accurate underwater geometry ground truth as limitations.

C. Generalization Across 3D Reconstruction and VSLAM

VI. Ablation Studies

  • Geometric fidelity is highest at specific operational Z0 values, but a single scene-level value averages over locally varying optimal depths.Source: §VI · p. 7
  • The approach speeds up feature matching and mapping compared with COLMAP and Refractive COLMAP, respectively.The passage attributes the gains to a better-fitting model than COLMAP and a simpler model than Refractive COLMAP.Source: §VI · p. 7

VII. Conclusion and Future Work

  • The method enables standard 3D reconstruction or VSLAM pipelines to run unmodified.Source: §VII · p. 7
  • Selecting one operational scene distance currently relies on prior scene-structure knowledge, while accurate underwater ground-truth geometry remains unavailable.The authors say Z0 should instead be obtained per pixel, for example through stereo triangulation or monocular depth prediction.Source: §VII · p. 8
Loading 2610.07788v1…