PaperlayerView exact revision on arXiv

Source-linked AI summary

BenthicFlow: Generating Extensible Underwater Environments via Flow Matching

Joaquín Figueira, Camile Lendering, Manfred Gonzalez-Hernandez, Giacomo D'Amicantonio, Erkut Akdag, Egor Bondarev

arXiv:2608.23173v1cs.CV

TL;DR

Underwater 3D scene understanding is limited by scarce structured data and weak generalization from surface imagery, while prior large-scene generators reconcile independently sampled tiles after generation. BenthicFlow uses one conditional flow-matching model to generate aligned RGBD content, reconcile overlapping windows during sampling, and lift mosaics with surface-aligned Gaussian surfels; experiments show coherent large-scale scenes across distinct sites that preserve site-specific appearance and match target distributions.

  • Problem

    Underwater 3D scene understanding lacks high-quality structured data, and surface-trained models generalize poorly to underwater scenes.

  • Method

    BenthicFlow jointly generates aligned RGB-depth pairs with one conditional flow-matching model, extends them through trajectory-level window reconciliation, and lifts mosaics using surface-aligned Gaussian surfels.

  • Results

    BenthicFlow generates coherent, large-scale 3D scenes across geographically distinct survey sites while preserving site-specific appearance and closely matching target distributions.

  • Takeaways & Limitations

    The framework provides a scalable basis for generating synthetic underwater environments for downstream perception and robotics applications.

  • Takeaways & Limitations

    The pipeline remains fundamentally 2.5D and cannot recover geometry absent from the generated view.

Abstract

from arXiv · show

Computer vision applications for 3D scene understanding in underwater environments remain challenging due to the lack of high-quality 3D data and the inability of surface-trained models to generalize to underwater scenes. To address this challenge, an emerging trend is to employ generative models to close the data domain gap. However, existing methods assemble large scenes by stitching independently generated tiles post hoc with separately trained models, while demonstrating heterogeneous landscapes only within individual survey sites. We introduce BenthicFlow, a unified framework based on a single conditional flow-matching model that jointly generates aligned textures and depth maps. A MultiDiffusion-inspired sampling procedure reconciles overlapping windows throughout the generative trajectory, enabling spatially extensible RGBD mosaics without a separate stitching model. The generated mosaics are subsequently lifted into explicit 3D benthic environments using surface-aligned Gaussian surfels. Experiments across geographically distinct survey sites demonstrate that BenthicFlow preserves site-specific appearance while generating coherent, large-scale 3D scenes that closely match the target distributions. Code and trained models are available at https://github.com/jacomof/BenthicFlow.

1 Introduction

BenthicFlow addresses the scarcity of usable underwater 3D data with a unified generative pipeline that jointly produces aligned RGBD mosaics and lifts them into extensible 3D benthic environments. Its windowed flow-matching sampler reconciles overlaps during generation, avoiding separately trained stitching or in-painting models while spanning geographically distinct survey sites.

  • Usable underwater 3D data remains scarce despite abundant image collections because missing poses, dense supervision, and underwater degradation hinder SfM, SLAM, and LiDAR.
  • Synthetic seafloor generation can support robotics simulation, ecological anomaly detection, and improved underwater representation in foundation-model training data.
  • BenthicFlow jointly synthesizes aligned RGB-depth pairs and extends them into unbounded RGBD mosaics using conditional flow matching and MultiDiffusion-inspired windowed sampling.
  • Overlapping windows are reconciled by averaging predicted velocities throughout sampling, so one model handles generation and in-painting without a secondary stitching network.
  • A single conditional velocity field spans multiple geographically distinct survey sites and interpolates smoothly between them.
  • RGBD mosaics are lifted into spatially continuous 3D environments using surface-aligned Gaussian surfels.

2 Related Works

Related work spans underwater image restoration, synthetic-data generation, continuous-time flow matching, latent representations, and spatial blending. BenthicFlow combines these directions while differing from prior extensible terrain generation in reconciling windows within the generative trajectory rather than afterward.

  • Underwater computer vision supports marine biology, infrastructure inspection, AUV navigation, mapping, and archaeology, with much prior work focused on optical image restoration.
  • Synthetic underwater-data methods include surface-to-underwater image transfer and models that generate data for depth estimation, while the closest prior work jointly generates images and depth with differentiable rendering.
  • Prior extensible terrain generation samples tiles independently and reconciles them afterward with a separately trained inpainting model, whereas BenthicFlow resolves overlaps during sampling.
  • Conditional flow matching regresses velocity fields along tractable conditional paths and can provide competitive sample quality with fewer function evaluations than denoising diffusion models.
  • Representation Autoencoders use high-dimensional semantic spaces derived from frozen vision foundation models to ease velocity learning relative to strict VQ-GAN or VAE bottlenecks.
  • MultiDiffusion unifies localized generation paths by averaging overlapping crops during sampling, and BenthicFlow extends this fusion mechanism to continuous flow trajectories.
  • The dataset contains over 600k color-normalized 518 × 518 benthic nadir frames collected across Australian reefs and Hawaii.
  • Deployment-level partitioning assigns approximately 90k images each to validation and testing, with approximately 500k images for training, preventing temporal leakage.

4 Methodology

BenthicFlow encodes RGB and depth into a shared latent space, interpolates reference-image conditioning across a grid, and generates large canvases through overlapping conditional-flow windows. The resulting latent canvas is decoded into RGBD and lifted for 3D rendering.

  • Latent initialization and conditioning interpolation: Reference images are encoded into a conditioning grid, while a Gaussian noise canvas is denoised through overlapping latent windows whose trajectories are averaged before one-pass RGBD decoding.
  • Latent initialization and conditioning interpolation: The d-RAE jointly encodes RGB images and depth maps into fused 1024-dimensional patch-token latents using DINOv2 RGB features and a ViT depth encoder.
  • Latent initialization and conditioning interpolation: Four reference-image descriptors are bilinearly interpolated into a full conditioning grid, allowing appearance to vary smoothly while preserving shared latent coherence.
  • Extensible generation with conditional flow matching: BenthicFlow applies rectified flow matching between isotropic Gaussian noise and d-RAE latents, generating 16 × 16 latent windows conditioned on interpolated appearance vectors.
  • Extensible generation with conditional flow matching: The model is trained at a fixed low resolution rather than explicitly on windowed denoising of large canvases.
  • Extensible generation with conditional flow matching: At each Euler step, concurrent window velocities are averaged per token with a sine window that downweights borders, integrating extensibility directly into sampling.
  • Extensible generation with conditional flow matching: A single sampler fuses neighboring regions during generation, eliminating secondary networks, post-hoc blending, and dedicated treatment of window boundaries.
  • 3D lifting and rendering: The fully convolutional decoder produces arbitrary-size RGBD mosaics, which are unprojected with a pinhole model and lifted into explicit 3D representations for novel-view rendering.

5 Experiments

BenthicFlow is evaluated for image generation, geographic retention, extensible terrain synthesis, and depth consistency against real data and FLUX.2-dev. The experiments show strong distributional fidelity, site-specific visual retention, and overlap-dependent seam quality, while depth consistency is assessed indirectly because direct comparison is not applicable to localized crops.

  • Evaluation Setup: BenthicFlow is evaluated across single-image generation, reference adherence, extensible landscape generation, and 3D benthic environment rendering.The evaluation suite covers four complementary aspects of the proposed system.
  • Single Image Generation: BenthicFlow is compared with 4-bit quantized FLUX.2-dev using FID, KID, and DINOv2 cosine similarity under different conditioning protocols.BenthicFlow uses random test crops, while FLUX.2-dev uses full CLAHE-normalized images with a domain-specific text prompt.
  • Single Image Generation: FID ≤20 across all reefs and preprocessing variants, with higher fidelity and conditional similarity than FLUX.2-dev.Qualitative comparisons also report structural and optical artifacts in FLUX.2-dev outputs; generated depth maps are slightly diffused but remain consistent with generated images.
  • Geographic Retention: BenthicFlow retains over 95% of real-data balanced accuracy and macro F1 for geographic reef classification, while substantially outperforming FLUX.2-dev.BenthicFlow-DPF retains 95% and 94% of real DPF-Net balanced accuracy and macro F1, respectively.
  • Extensible Generation: 62.5% window overlap optimizes the Seam Gradient Ratio, whereas both zero overlap and very high overlap such as 87.5% perform worse.The analysis generates 500 images for each of seven overlap configurations at 16 times the CFM output scale.
  • Depth Consistency: Direct generated-image versus generated-depth comparison is not applicable because localized crops remove peripheral context to which Depth Anything V2 is sensitive.Depth-image consistency is instead approximated through d-RAE encoding and decoding of unseen image-depth pairs, producing near-perfect alignment metrics.

3D Rendering

BenthicFlow generates coherent, diverse underwater scenes at large scale and lifts RGBD mosaics into 3D using surface-aligned surfels with anchored refinement.

  • Extensible scene generation: BenthicFlow interpolates reference images across geographically different reefs into coherent landscapes with diverse substrate transitions and biological detail.Sand transitions into rocky structures, with plating corals and algae distributed across the generated scene.
  • Extensible scene generation: BenthicFlow-DPF produces a larger color range and greater texture quality by mitigating underwater optical attenuation.
  • Extensible scene generation: 32× larger than the CFM training resolution, BenthicFlow and BenthicFlow-DPF synthesize large underwater scenes from reference seeds.The reference seeds appear in corner frames of the scenes.
  • 3D lifting ablation: Anchored refinement reduces LPIPS from 0.138 to 0.046 and increases SSIM from 0.884 to 0.925 on paired single-view reprojection.The ablation uses 300 generated RGBD samples and compares PSNR, SSIM, and LPIPS across surfel, refinement, and perceptual-polish configurations.
  • 3D lifting ablation: Surface-aligned initialization outperforms isotropic initialization before refinement, while perceptual color polishing provides no further reprojection benefit.Surfels preserve surface alignment needed for multi-view coherence despite isotropic Gaussians achieving a 0.39 dB higher converged PSNR.

6 Conclusions and Future Work

BenthicFlow provides a unified framework for extensible 2.5D underwater environments, combining flow-matching synthesis, RGBD generation, and surfel-based 3D lifting. The authors identify representation, geometry, memory, and resolution constraints as directions for future work.

  • Conclusions: BenthicFlow unifies diverse marine-biome modeling with extensible 2.5D underwater-environment generation.The framework is evaluated across image generation, depth consistency, extensible synthesis, and 3D reprojection fidelity.
  • Conclusions: MultiDiffusion-inspired sampling integrates directly into flow-matching trajectories, avoiding a separately trained inpainting model for extensible synthesis.Surface-aligned Gaussian surfels improve surface coverage while preserving estimated geometry during lifting.
  • Future work: The current pipeline remains fundamentally 2.5D and cannot recover geometry absent from the generated view.The authors suggest direct 3D generation using pseudo-3D supervision as future work.
  • Future work: High-resolution generation is constrained by the memory cost of decoding the full latent canvas.The authors propose consistent windowed decoding and higher-resolution full-image training as possible improvements.

Supplementary Material for BenthicFlow: Generating Extensible Underwater Environments

The supplementary material identifies the authors and their Eindhoven University of Technology affiliation and contact information.

  • Authors: Joaquín Figueira, Camile Lendering, Manfred Gonzalez-Hernandez, Giacomo D’Amicantonio, Erkut Akdag, and Egor Bondarev are listed as authors.
  • Affiliation: The authors are affiliated with the AIMS Group in the Department of Electrical Engineering at Eindhoven University of Technology, Eindhoven, The Netherlands.
  • Contact: Contact information is provided for j.figueira and c.r.lendering at tue.nl.

S1 Additional Single Image Generation Results

Supplementary evaluations report per-campaign reconstruction and conditional-generation metrics, with results generally consistent across geographic locations and campaigns. Batemans campaigns perform relatively worse, which the authors hypothesize reflects moderate training-data imbalance.

  • Additional results: Table S1 reports per-campaign reconstruction fidelity and depth consistency for the two flow-based pipelines.Metrics include FID, KID, DINOv2 cosine similarity, RMSE, and Pearson correlation r.
  • Additional results: The supplementary results remain consistent across geographic locations and campaigns, while Batemans campaigns show relatively inferior performance.The authors state that the tables cover every campaign available in the dataset.
  • Additional results: The authors hypothesize that Batemans underperformance is caused by moderate training-data imbalance.Scott Reef and Hawaii have more images per campaign, providing flow-matching models with a stronger learning signal for those locations.
  • Additional results: Table S2 reports per-campaign conditional-generation distributional fidelity and semantic alignment for the two models.FID and KID are lower-is-better metrics, while DINOv2 cosine similarity is higher-is-better.

S2 Extended Analysis of the Extensible Generation

The extended analysis shows that overlapping windows reduce seam artifacts in large mosaics, while sine-weighted merging achieves the optimal seam gradient ratio across overlap settings.

  • Overlap effects: 16× mosaics without overlap show lighting and color discontinuities between windows.The mosaics are 4× larger vertically and horizontally than BenthicFlow’s standard output.
  • Overlap effects: Optimal overlap removes visible discontinuities and abrupt color changes between windows.Gradient plots also show few peaks near the expected seam regions.
  • Overlap effects: Few gradient peaks near expected seams indicate a more uniform gradient profile under optimal overlap.
  • Window weighting: The sine window is the only tested weighting strategy that achieves an optimal seam gradient ratio.Uniform and sine-weighted windows are compared as functions of overlap percentage.
  • Seam analysis: Seam artifacts are expected near overlap centers, where the pull of multiple trajectories equalizes.

S3 Benthic Conditioning of FLUX.2-dev

This section describes FLUX.2-dev conditioning with a reference benthic image and a prompt that preserves survey properties while varying substrate distribution, alongside an inference-speed comparison with BenthicFlow.

  • Conditioning: FLUX.2-dev is conditioned on a reference 512 × 512 image and a prompt describing benthic survey-image properties.
  • Conditioning: The prompt varies benthic substrate distribution while preserving perspective, lighting, and style.It specifies hard corals, rubble, sand, artificial strobe lighting, and slight underwater attenuation and backscatter.
  • Inference: 28 diffusion steps provide the selected trade-off between image quality and inference speed.
  • Inference: At 28 steps, quantized FLUX.2-dev generates one image every ∼6 seconds, while BenthicFlow generates more than 120 images in the same timeframe on the same hardware.

S4 Additional Generated Scenes

Additional renderings demonstrate BenthicFlow generation at 36× the model window resolution, with smooth transitions and varied benthic biomes; the section also documents deployment-level data splits for reproducibility.

  • Additional Generated Scenes: Additional 3D renderings are generated at 36× the model window resolution.The scenes are 6× larger horizontally and 6× larger vertically.
  • Additional Generated Scenes: The additional scenes show smooth transitions between seeds and natural benthic biome variations.
  • Additional Generated Scenes: Reference seeds for the supplementary scenes are sampled from different deployments and campaigns.
  • Data Reproducibility: The reproducibility subset is selected from Squidle+ at the deployment level, with split and campaign identified for each deployment.
  • Data Reproducibility: Validation and test sets each hold out one complete deployment per campaign, preventing spatially overlapping imagery from leaking across splits.The totals are 510,307 train, 76,480 validation, and 87,574 test images across 52, 7, and 7 deployments, respectively.
Loading 2608.23173v1…