Source-linked AI summary
3D-USE: From Image-Level to Scene-Level Underwater Enhancement
Jieyu Yuan, Yuanlin Zhang, Jihong Li, Chunle Guo, Huimin Lu, Chongyi Li
TL;DR
Underwater 3D reconstruction preserves visibility loss, while physical inversion can leave errors in recovered scene appearance and independent view enhancement lacks a stable shared target. 3D-USE reconstructs a medium-aware Gaussian scene, transfers paired 2D enhancement through ATC, and realizes it with U-BAF for persistent enhanced novel-view rendering. Experiments on real underwater scenes report improved visibility and cross-view consistency while maintaining reconstruction quality.
Problem
Underwater reconstruction can preserve degraded visibility, and physical inversion may leave scene-appearance errors without paired enhanced 3D supervision.
Method
3D-USE combines MediumRBF medium-aware reconstruction with ATC transition targets and U-BAF transformations of Gaussian radiance and medium appearance.
Results
Experiments on real underwater scenes show improved visibility and cross-view consistency while maintaining reconstruction quality.
Takeaways & Limitations
The learned enhancement is stored in a persistent 3D scene that directly renders enhanced novel views without a 2D UIE model at inference.
Abstract
from arXiv · showhide
Underwater 3D reconstruction faithfully reproduces the color shifts and visibility loss of captured views, while physical inversion may leave estimation errors in the recovered scene appearance. We formulate Underwater Scene-level Enhancement (USE) as learning a persistent, visibility-enhanced 3D scene representation from degraded multi-view underwater observations, enabling consistent enhanced rendering. Realizing USE requires both a reliable scene representation for enhancement and a consistent enhancement target without paired enhanced 3D data. Therefore, we present 3D-USE, a two-stage framework. First, the Medium Radial Basis Anchor Representation (MediumRBF) establishes a medium-aware Gaussian scene by representing water effects with shared radial-basis anchors and explicitly decomposing object and medium contributions. Based on this fixed scene representation, Appearance Transition Consensus (ATC) transfers paired 2D underwater image enhancement (UIE) knowledge into scene-global and Gaussian-local targets, avoiding direct supervision from inconsistent enhanced views. An Underwater Bilateral Appearance Field (U-BAF) then realizes these targets in Gaussian radiance and medium appearance. The scene directly renders enhanced novel views without a 2D UIE model at inference. Experiments on real underwater scenes show improved visibility and cross-view consistency while preserving reconstruction quality.
Introduction
3D-USE reframes underwater enhancement as learning a persistent, visibility-enhanced 3D scene rather than reproducing degraded observations or independently enhancing views. Its two-stage design combines medium-aware reconstruction with consensus-based transfer of 2D enhancement knowledge into a consistent renderable representation.
- Underwater imaging causes wavelength-dependent attenuation and veiling light, so faithful 3D reconstruction can preserve color shifts, contrast loss, and limited visibility.
- Physical inversion can absorb medium-estimation errors into scene radiance, producing medium-dependent color and spatial artifacts despite reconstructing captured views.
- USE therefore learns a persistent visibility-enhanced 3D scene representation that directly renders consistent enhanced novel views from degraded multi-view images.
- MediumRBF establishes a medium-aware Gaussian scene with stable geometry and explicit separation between object appearance and observer-dependent water effects.
- ATC consolidates calibrated view-wise 2D UIE transitions into scene-global and Gaussian-local guidance instead of using independently enhanced images as RGB targets.
- U-BAF realizes the guidance through coordinated changes to Gaussian radiance and medium appearance, producing persistent enhanced rendering without a 2D UIE model at inference.
Methodology
The methodology reconstructs a medium-aware Gaussian scene, derives fixed global and local transition targets from calibrated 2D enhancement proposals, and learns a renderable appearance field against those targets. The resulting system preserves shared medium structure and realizes enhancement through complete underwater compositing.
- Medium Radial Basis Anchor Representation.: Stage 1 jointly reconstructs Gaussian geometry and radiance with a MediumRBF water-medium representation, then freezes the scene for enhancement.
- Medium Radial Basis Anchor Representation.: MediumRBF interpolates medium properties from top-K isotropic radial-basis anchors in observer space and represents directional variation with spherical harmonics.
- Medium Radial Basis Anchor Representation.: The learned centers and widths share medium estimates locally across nearby observations while adapting that sharing to the camera distribution, supporting unseen observer positions.
- Medium Radial Basis Anchor Representation.: The medium coefficients encode asymptotic color, wavelength-dependent backscatter, and attenuation, which are constrained to valid non-negative values before underwater compositing.
- Appearance Transition Consensus: ATC converts calibrated proposal transitions into one scene-global operator and fixed Gaussian-local targets, separating common color and exposure changes from spatial corrections.
- Underwater Bilateral Appearance Field: U-BAF is a trainable low-rank 4D appearance field that transforms Gaussian and medium appearance using position, appearance, viewing direction, and camera position.
- Optimization Objectives: Stage 2 supervises transitions re-extracted after complete underwater compositing, matching global and centered local realized parameters to ATC targets while keeping Stage-1 and ATC targets fixed.
- Optimization Objectives: The complete pipeline qualitatively improves background consistency and reduces medium-related artifacts while maintaining coherent geometry and sharp object boundaries.
Experiment Results
Across real underwater scenes, 3D-USE improves reconstruction fidelity, visibility, and cross-view appearance consistency while reducing medium-related and spatial artifacts.
- Reconstruction Quality: 3D-USE achieves the best PSNR and SSIM on all four reconstruction benchmarks while maintaining competitive LPIPS and the highest rendering speed.
- Reconstruction Quality: 0.249 and 1.163 dB PSNR improvements over the second-best methods are reported on D3 and D5, respectively.
- Enhancement Quality: 3D-USE leads multiple no-reference enhancement metrics across SeaThru-4, DRUVA-20, D3, and D5, while achieving the lowest wLPIPS macro-average over 26 scenes.The results indicate improved visibility and stable enhanced appearance across views.
- Qualitative Results: Explicit medium modeling improves background consistency and reduces medium-related artifacts while maintaining coherent geometry and sharp object boundaries.
- Qualitative Results: Qualitative comparisons show balanced color correction, improved visibility, clearer foreground structures, and more spatially coherent backgrounds with fewer artifacts than existing methods.
- Ablation Study: Global transition supervision sets the scene-wide enhancement direction, local supervision constrains spatial realization, and their combination suppresses localized color artifacts.Global-only corrections can be spatially uneven, whereas local-only residuals may provide insufficient overall enhancement.
Conclusion
3D-USE addresses missing paired enhanced 3D data and inconsistent view-wise 2D guidance by learning a persistent, scene-consistent underwater enhancement representation.
- Conclusion: 3D-USE reconstructs a medium-aware Gaussian scene with MediumRBF, then uses ATC to consolidate paired 2D UIE transitions into a scene-consistent enhancement objective.
- Conclusion: U-BAF realizes the objective through spatially coherent transformations of Gaussian radiance and medium appearance before underwater compositing.
- Conclusion: The resulting scene directly renders enhanced novel views without applying a 2D UIE model at inference, improving visibility and cross-view consistency while preserving reconstruction quality.
Overview
The paper motivates underwater scene-level enhancement and reviews Gaussian splatting and underwater rendering foundations needed for medium-aware reconstruction.
- Overview: 3D Gaussian Splatting represents scenes with explicit Gaussian primitives and differentiable rasterization, supporting efficient rendering and direct appearance manipulation.
- Overview: Each Gaussian stores a center, covariance, opacity, and view-dependent color coefficients, with covariance controlled by rotation and scaling matrices.
- Overview: Projected Gaussians are rasterized and combined per pixel through depth-ordered alpha blending of view-dependent colors and projected opacities.
Underwater Gaussian Compositor
The underwater Gaussian compositor combines attenuated object appearance with interval-wise and far-end medium backscatter while separating foreground occlusion, attenuation, and scattering.
- Underwater Gaussian Compositor: The rendered underwater color is the sum of an attenuated object term, an interval-wise medium term, and a far-end medium term.
- Underwater Gaussian Compositor: Foreground Gaussian opacities determine object transmittance along each camera ray before attenuation is applied to the object contribution.
- Underwater Gaussian Compositor: The object attenuation coefficient is RGB-valued, and element-wise multiplication applies attenuation to the object contribution.
- Underwater Gaussian Compositor: Medium backscatter is accumulated between neighboring Gaussian distances using the medium color and RGB backscatter coefficient.
- Underwater Gaussian Compositor: The exponential is evaluated channel-wise, separating foreground occlusion, object attenuation, and medium backscatter in the rendering formulation.
Medium-Aware Reconstruction
MediumRBF reconstruction models underwater medium effects alongside Gaussian scene appearance. The objective combines regularized photometric, structural, and depth-prior supervision during staged optimization.
- Representation: MediumRBF uses eight learnable anchors and retains the K = 4 highest-affinity anchors for each camera position.Degree-3 spherical harmonics model directional variation, while decoded medium color, attenuation, and backscatter remain non-negative.
- Reconstruction Objective: The rendered underwater image is used to define a pixel-wise inverse-intensity weighting scheme.Stop-gradient prevents the weights from receiving back-propagated gradients while increasing dark-region contributions.
- Reconstruction Objective: The reconstruction loss combines regularized L1 and MS-SSIM terms with λ = 0.2.The formulation is Lrec = (1 −λ)LReg-L1 + λLReg-MS-SSIM.
- Geometry Supervision: Rendered depth is converted to disparity and aligned with a Depth Anything V2 prior for coarse geometry supervision.The disparity is bd = 1/(10 bD + 1).
- Optimization: Stage 1 runs for 15k iterations, with Gaussian densification ending at 10k.
Appearance Transition Consensus
ATC converts paired image-enhancement transitions into shared scene-global and Gaussian-local targets. Consensus and visibility-aware lifting produce fixed guidance for subsequent scene optimization.
- Transition Extraction: The transition extractor decomposes paired-image changes into a scene-scale affine operator and a local diagonal correction.The global operator is identity-centered, while local cells store RGB log-gains and biases.
- Transition Parameterization: The global operator contains nine color-mixing coefficients and three bias values, whereas each local cell contains three log-gains and three biases.
- Calibration: An offline calibration set uses 800 UIEB underwater inputs paired with fixed enhanced targets.A residual calibrator maps frozen Nu2Net proposals and source-image descriptors to paired targets using normalized Smooth-L1 losses.
- Scene-Global Consensus: ATC evaluates global candidates on ten RGB probes and solves consensus using Weiszfeld iterations followed by linear least squares.Consensus on observable responses avoids directly averaging different affine parameterizations.
- Local Guidance: The local calibrator predicts per-view gain-and-bias maps conditioned on the shared operator, with spatial means removed to prevent duplication of the global grade.
- Gaussian-Local Targets: Visibility-weighted observations are accumulated through the frozen splatting operator into one fixed Gaussian-local target.After two supporting views, influence is reweighted by a Cauchy factor measured in paired-data IQR units.
Underwater Bilateral Appearance Field
U-BAF realizes ATC’s fixed global and local targets through a bilateral field conditioned on spatial position and appearance. It jointly transforms Gaussian and medium appearance while preserving the Stage-1 scene structure.
- Field Parameterization: U-BAF conditions local transformations on space and a learned signed-log appearance guide derived from Stage-1 Gaussian appearance.
- Field Parameterization: Four one-dimensional factor grids form a rank-R code for normalized spatial and appearance coordinates.
- Global and Local Factors: A trainable identity-centered scene affine represents the global enhancement grade, while spatially constant local outputs are removed.This assigns the global grade only to the global affine component.
- Medium Appearance: The same field transforms Gaussian and medium appearance using camera position and MediumRBF color as inputs.Bounded scalar log-ratios adjust attenuation and backscatter, while cross-channel mixing remains in the appearance field.
- Supervision: U-BAF training compares realized transitions from Stage-1 renderings and enhanced renderings with fixed ATC targets.
Evaluation Details
Evaluation measures reconstruction fidelity, enhancement quality, qualitative behavior, and cross-view consistency on held-out underwater views. Across these tests, 3D-USE combines strong structural fidelity with scene-consistent enhancement, while wLPIPS results are complementary rather than uniformly dominant.
- Evaluation Protocol: Reconstruction scores average views within each scene and then report unweighted scene means for SeaThru-4 and DRUVA-20.SSIM uses RGB images and LPIPS uses the AlexNet backbone.
- Evaluation Protocol: Enhancement quality uses UCIQE, URanker, and MUSIQ on held-out views with unweighted scene-macro means.Inputs with long sides above 1024 pixels are resized while preserving aspect ratio.
- Quantitative Results: 3D-USE achieves the best SSIM in every SeaThru-4 scene and the best or second-best LPIPS in Curasao, Japanese Gardens, and Panama.
- Quantitative Results: 3D-USE leads all three enhancement metrics on IUI3 and Japanese Gardens, while achieving the best MUSIQ on Panama and ranking second in MUSIQ on Curasao.
- Qualitative Reconstruction: Qualitative reconstruction comparisons show fewer spatially varying artifacts while preserving ropes, calibration targets, coral boundaries, and surrounding backgrounds.The comparison covers representative scenes and water conditions.
- Qualitative Enhancement: Enhancement comparisons reveal a recurring trade-off between enhancement strength and spatial stability across methods.WaterSplatting strongly increases contrast but clips bright seabed regions in IUI3.
- Cross-View Consistency: 3D-USE obtains the lowest wLPIPS on both multi-scene benchmarks and the lowest 26-scene macro average.Less favorable D3 and D5 results make wLPIPS complementary evidence rather than a claim of uniform trajectory-level dominance.