PaperlayerView exact revision on arXiv

Source-linked AI summary

SURGE: Sonar-fUsed Reconstruction and localization via image-gated Graph Estimation

Mohammed Ibrahim M, Vallabh Deogaonkar, Trung Dong, Jane Shin, Abhilash Somayajula, Xiaomin Lin

arXiv:2610.07472v1cs.RO

TL;DR

Underwater ROVs need reliable localization and metric scene geometry, but monocular vision and 2D sonar each provide incomplete support. SURGE fuses camera and sonar observations in a factor graph to estimate metric poses and target location, then uses those poses for sonar Gaussian splatting. On real underwater data, it reports consistent pose estimates and a compact, natively metric reconstruction compared with RGB Gaussian-splatting baselines.

  • Problem

    Underwater localization and metric reconstruction are difficult when external positioning is unavailable: monocular vision can drift in scale and trajectory, while 2D sonar leaves elevation ambiguous.

    Source: §I · p. 1

  • Method

    SURGE fuses visual and acoustic observations in a factor graph to jointly estimate the ROV trajectory and target location, then uses recovered metric poses for target-scoped sonar Gaussian splatting.

    Source: §I · p. 2, §III · p. 3

  • Results

    874/880 frames were retained at 10.6 cm median radial and 3.33° median azimuth residuals; the sonar cloud had 0.65 m median distance, versus 1.4–2.2 m for RGB baseline means.

    Source: C. Localization · p. 6, D. Quantitative reconstruction · p. 6

  • Takeaways & Limitations

    Under the same views, the natively metric sonar cloud near the localized landmark is stronger support for reconstruction than RGB densification on COLMAP poses.

    Source: §VI · p. 8

    Reconstruction remains bounded by 2D FLIS elevation ambiguity and limited viewpoint diversity.

    Source: §VI · p. 8

Abstract

from arXiv · show

Remotely operated vehicles (ROVs) are widely used to explore and inspect underwater environments such as caves, shipwrecks, and submerged infrastructure. These missions require accurate 3D understanding of the surrounding environment, which depends on both reliable vehicle localization and metric scene reconstruction. However, external positioning is often unavailable underwater, requiring small ROVs to rely primarily on onboard perception. Optic vision provides rich visual and geometric information but suffers from scale ambi- guity and trajectory drift, whereas 2D imaging sonar provides metric range but incomplete 3D geometry. Existing underwater reconstruction approaches typically address these limitations separately or assume known sensor poses, leaving localization and reconstruction disconnected. We present SURGE, a camera sonar framework that jointly estimates the ROV trajectory and target location by integrating visual and acoustic observations within a factor graph, then uses the recovered metric poses for sonar Gaussian splatting. Experiments on real underwater RGB sonar observations show that SURGE substantially improves localization consistency over conventional vision based pose estimation and produces a more compact, natively metric reconstruction than RGB Gaussian splatting baselines.

I. INTRODUCTION

Underwater missions need reliable localization and metric scene geometry, but monocular vision and 2D sonar each leave important gaps. SURGE combines camera and sonar observations to estimate the trajectory and target together, then uses the metric poses for reconstruction.

I. INTRODUCTION

  • Underwater missions need accurate vehicle localization and metric scene geometry, while external positioning is often unavailable to small ROVs.Monocular vision can suffer scale ambiguity and trajectory drift, whereas 2D imaging sonar provides metric range but leaves elevation ambiguous.Source: §I · p. 1
  • Accurate reconstruction depends on the accuracy and metric consistency of the estimated sensor trajectory.The paper identifies this dependence as a practical limitation of treating localization and reconstruction separately.Source: §I · p. 2
  • SURGE jointly estimates the ROV trajectory and target location from monocular RGB and 2D forward-looking sonar, then uses recovered metric poses for target-scoped sonar Gaussian splatting.Its contributions include multi-view sonar aggregation and a target-extent-aware acoustic residual.Source: §I · p. 2

A. Multi-modal Reconstruction with Known Poses

  • AONeuS, Z-Splat, and Collado-Gonzalez et al. combine optical and acoustic measurements for reconstruction, but primarily assume externally provided sensor poses.These approaches address reconstruction once the trajectory is known rather than recovering localization and reconstruction from the same measurements.Source: A. Multi-modal Reconstruction with… · p. 2

B. Single-Modality Reconstruction with Estimated Poses

Prior work has explored reconstruction with estimated poses within individual sensing modalities and trajectory estimation from underwater sensor constraints. SURGE combines monocular RGB and 2D sonar to estimate metric motion and target location, then uses those poses for reconstruction.

C. Underwater Factor-Graph SLAM

  • For compact extended targets, SURGE models sonar observations relative to target extent because dominant returns can move across the visible surface as viewpoint changes.This avoids forcing each acoustic return to coincide with a single geometric center.Source: C. Underwater Factor-Graph SLAM · p. 2

B. Single-Modality Reconstruction with Estimated Poses

D. Position of the Present Work

  • SURGE couples visual and acoustic observations in a factor graph to estimate metric vehicle motion and target location, then uses the recovered trajectory for reconstruction without external positioning.The pipeline overview shows fusion of a multi-view-consistent peak, segmentation centroid, prior visual pose, and object-position anchor before sonar Gaussian splatting.Source: D. Position of the Present Work · p. 2, Figure 2 · p. 3

III. METHOD

SURGE pairs RGB and 2D sonar observations, fuses visual and acoustic constraints to recover metric ROV poses and a target center, then reconstructs the target with sonar Gaussian splatting.

III. METHOD

  • RGB and 2D sonar frames feed a factor graph that estimates metric camera poses against a multi-view-consistent sonar observation and a target box, then sonar Gaussian splatting uses the recovered poses.Vision supplies the object's image location, while sonar supplies range; the polar disk residual accommodates returns on a near face rather than at the geometric center.Source: §III · p. 3

A. Sensors, frames, and inputs

  • The data platform is a BlueROV2 with a rigid, forward-facing RGB camera and 2D forward-looking sonar; the assumed target is a static upright box with unknown center.The target half-extents are h=[0.10, 0.10, 0.20] m; camera–sonar extrinsics are fixed, and only yaw is refined over COLMAP poses.Source: A. Sensors, frames, and inputs · pp. 3–4, Figure 3 · p. 4

III. METHOD

  • SURGE forms one range–azimuth sonar observation per frame by using RGB to gate candidate peaks and multi-view voting to select a view-consistent return.The consensus observation addresses clutter, tether, and sidelobes that can outshine the target.Source: Figure 4 · p. 4

B. Sonar measurement layer

SURGE builds one range–azimuth sonar observation per frame by combining local peak detection, RGB masking, and multi-view consensus. This observation provides a world-consistent acoustic measurement for later estimation.

B. Sonar measurement layer

  • Fig. 5 shows the RGB silhouette, box, and centroid alongside the matching sonar fan, cones, projected brick, locked return, and projected centre.The figure distinguishes the RGB-mask cone from the factor-graph cone.Source: Figure 5 · p. 4
  • The RGB mask is projected through the lever into polar range–azimuth coordinates to discard local sonar peaks outside its cone.Local maxima are detected with nonmaximum suppression inside a range gate.Source: B. Sonar measurement layer · p. 4
  • Multi-view consensus intersects sonar elevation arcs with RGB-centroid rays, bins hits in 18 cm voxels, and selects the modal world point as the locked observation.Of 881 cone-gated frames, 880 retain a consensus flash.Source: B. Sonar measurement layer · p. 4

C. Factor graph

The factor graph jointly estimates camera centres, small yaw corrections, and a static target centre from RGB and sonar observations. Its residuals accommodate the target’s spatial extent while combining image constraints with pose priors and geometric gates.

C. Factor graph

  • The graph jointly estimates camera centres, small yaw corrections, and one static target centre so RGB and sonar observations agree on the same metric object.The first valid frame is gauge-fixed to its COLMAP centre, with its yaw correction set to zero.Source: C. Factor graph · p. 4
  • The sonar residual allows the projected target centre to lie within a 12 cm disk around the locked return, avoiding the inconsistent assumption that a face return equals the geometric centre.Outside the disk, a Huber pull acts on the cameras; the factor remains active.Source: C. Factor graph · p. 5
  • RGB factors compare the projected target-box AABB with the SAM box and the projected centre with the mask centroid, constraining bearing without overpowering the metric sonar disk.Source: C. Factor graph · p. 5
  • A weak COLMAP anchor keeps the trajectory near its SfM shape, while nearby weights are reduced when reported inter-frame motion exceeds 0.9 m.The anchor uses σc=0.8 m; nearby weights are scaled by 0.05 for those large motions.Source: C. Factor graph · p. 5
  • Smoothness factors prefer small consecutive camera-centre changes and sometimes match COLMAP motion, while yaw factors penalize heading changes that could later shift the orange return laterally.COLMAP motion is matched when it is below 0.4 m.Source: C. Factor graph · p. 5
  • An elevation gate keeps the target inside the FLIS aperture, and a weak depth prior keeps it in front of the camera; the target centre is frozen after a short box-only warm start.Optimization uses GPU Adam for 400 iterations at learning rate 10^-3, with all factors active and no staged schedule.Source: C. Factor graph · p. 5

D. Reconstruction

SURGE reconstructs a localized target neighbourhood by sonar Gaussian splatting with recovered metric poses. Because each sonar bin leaves elevation ambiguous, the method uses multi-view evidence and restricts seed placement around the target.

D. Reconstruction

  • SURGE uses recovered metric poses and target centre b for target-neighbourhood sonar Gaussian splatting, with seeds restricted to short elevation arches near the target.Cone-arch seeding keeps the cloud on the object rather than filling the seafloor.Source: D. Reconstruction · p. 5
  • An FLIS return gives range and azimuth but is consistent with points along an elevation arc, so reconstruction hypothesizes mass along elevation and resolves it through multi-view consistency.The M750d elevation interval is −25° to 25°.Source: D. Reconstruction · p. 5
  • The scene is represented by anisotropic Gaussians, whose means, covariances, opacities, and acoustic reflectivities are rendered into polar sonar fans.The renderer follows, transforms means into sonar coordinates, and alpha-composites along range; the representation has no RGB appearance.Source: D. Reconstruction · p. 5
  • SURGE reuses elevation-arc initialization and the polar renderer from, but conditions splatting on estimated poses held fixed during reconstruction rather than known poses.Seeds are restricted to cone-gated peaks near the localized box; full-fan seeding produces a seafloor pancake.Source: D. Reconstruction · p. 5
  • The output is a compact, metric target cloud from real freshwater observations, conditioned on the joint pose–landmark solution; estimated camera poses are held fixed during splatting rather than jointly refined.Training uses unmasked 128×128 polar fans for 15 epochs.Source: D. Reconstruction · p. 5

A. Dataset and protocol

The evaluation uses an offline freshwater spring-cave sequence of a single cinder block, then assesses localization and target-cloud compactness against RGB Gaussian-splatting baselines. RGB clouds are similarity-aligned for distance scoring, whereas the sonar cloud remains in the graph’s metric frame.

A. Dataset and protocol

  • The dataset contains 1,199 RGB–FLIS pairs of one cinder block, with 880 frames retained after synchronization and removal of a COLMAP teleport window; 874 enter the graph as inliers.The ROV orbits at 2–5 m range at approximately similar depth; no surveyed brick size is used as ground truth.Source: A. Dataset and protocol · p. 5

B. Baselines and metrics

  • The evaluation compares localization and reconstruction compactness, explicitly disclaiming face-accurate mesh or full-scene mapping claims.The reconstruction measures include opaque-mean distance to b, the share within 0.5 m, projected mass inside the SAM mask, and native metric status.Source: B. Baselines and metrics · p. 6
  • RGB baselines include Vanilla 3DGS, SeaSplat, WaterSplatting, and AquaSplat, while their clouds are Sim(3)-aligned before distance scoring.The recovered scale is approximately 0.38; the cone-arch sonar cloud is not re-aligned.Source: B. Baselines and metrics · p. 6

C. Localization

Pose adjustment yields localization consistent with sonar and RGB observations, while the target-scoped sonar cloud is closer to the landmark than aligned RGB splats.

C. Localization

  • 874/880 frames were retained, with 10.6 cm median radial and 3.33° median azimuth residuals and 93 px mean projected-box error; no surveyed trajectory supports absolute-error evaluation.The reported localization measure is consistency with multi-view sonar observations and RGB segmentation.Source: C. Localization · p. 6
  • 1.89 m median range error remains when COLMAP poses are frozen and only the landmark is optimized; pose adjustment is needed to recover scale consistency.The disk residual further reduces face bias and bounding-box error.Source: C. Localization · p. 6
  • A 6.26 m COLMAP teleport becomes a 1.70 m step in the recovered trajectory rather than being copied as a jump.Early free frames can still move to place the sonar observation inside the disk under the gauge pin.Source: Figure 6 · p. 6, C. Localization · p. 6

D. Quantitative reconstruction

  • The cloud is not face-accurate: its vertical structure is limited by sensor elevation ambiguity and the orbit.The representative-view overlays show the compact cloud near the target; vertical spread follows the unmeasured ±25° FLIS aperture under a mostly horizontal orbit.Source: D. Quantitative reconstruction · p. 6, Figure 7 · p. 7
  • 0.65 m median distance to landmark b, 30% within 0.5 m, and 36% mean in-mask projection for the natively metric sonar cloud, versus RGB baselines’ 1.4–2.2 m, at most 7.1%, and 8–20%.RGB clouds were Sim(3)-aligned to factor-graph cameras before scoring; the reconstruction is target-scoped, not a full-scene map.Source: D. Quantitative reconstruction · p. 6, Table IV · p. 7

V. DISCUSSION

SURGE recovers metric poses, a static target centre, and a compact sonar cloud, but the available views and measurements constrain how much geometry the cloud can claim.

A. What the pipeline recovers

  • SURGE recovers metric-scale ROV poses, a static target centre, and a compact sonar Gaussian cloud from monocular camera and 2D FLIS observations.The cloud is target-scoped, not a full-scene map, and its vertical structure is limited by FLIS angular resolution, the unmeasured ±25° elevation aperture, and a mostly horizontal orbit.Source: A. What the pipeline recovers · p. 7

B. Why the disk residual matters

  • A 12 cm polar disk tolerance keeps ambiguous near-face sonar observations active instead of allowing robust weighting to down-weight informative frames and collapse range by about a metre.The dominant return usually lies on the target’s near face rather than at its geometric centre.Source: B. Why the disk residual matters · p. 7

C. Limits of open-water reconstruction

  • The reconstruction claim is a compact, correctly located, natively metric cloud near the landmark, not denser, sharper, face-accurate, or full-scene geometry.The orbit has little elevation diversity, FLIS provides no elevation measurement, and the target occupies only a small fraction of each fan.Source: C. Limits of open-water reconstruction · p. 7

D. Discarded design alternatives

SURGE reports failed design alternatives, limits its evaluated scope to assumed-size compact targets, and identifies viewpoint diversity as its main remaining limitation. Its conclusion reports centimetre-scale acoustic residuals and a natively metric target reconstruction on a real freshwater orbit.

D. Discarded design alternatives

  • Peak-as-centre, frozen COLMAP, strong absolute priors, full-fan seeding, and sub-bin covariance failed through glint chasing, scale lock, copied teleports, seafloor spread, or opacity collapse.Source: D. Discarded design alternatives · p. 7

E. Scope and future work

  • Assumed target half-extents set both the box prior and disk scale; unknown-size targets require jointly estimated extents, which SURGE does not evaluate.Source: E. Scope and future work · p. 7
  • Viewpoint diversity, rather than the renderer, is the main remaining limit; active next-best-view control is proposed to reduce joint pose–landmark and Gaussian uncertainty.Source: E. Scope and future work · p. 7

VI. CONCLUSION

  • SURGE recovers metric ROV poses and a compact target reconstruction without external positioning using a camera–sonar factor-graph pipeline.Source: §VI · p. 7
  • On a real freshwater orbit, SURGE reports centimetre-scale orange residuals and a natively metric cloud with stronger support than RGB densification on COLMAP poses under the same views.Multi-view orange consensus stabilizes acoustic measurements; a polar disk residual tolerates face-versus-centre geometry under an assumed box prior, while a jump-gated SfM prior absorbs COLMAP teleports without copying them.Source: §VI · p. 8

Related papers

Loading 2610.07472v1…