PaperlayerView exact revision on arXiv

Source-linked AI summary

Coastal Environment Generation with HoloOcean

Abigail Austin, Brady Moon, Joshua G. Mangelson

arXiv:2609.10484v1cs.RO

TL;DR

Marine robotics simulators need high-fidelity environments, but manual environment creation requires substantial time, data, and expertise. This paper presents a UE5 pipeline that generates coastal HoloOcean levels from one overhead image, combining terrain generation with automated asset selection and placement. Asset placement averaged less than 0.5 meters xy location RMSE and less than 2 degrees orientation RMSE, while the terrain results were non-ideal.

  • Problem

    Creating high-fidelity simulation environments often requires substantial time, data, and specialized environment-system knowledge.

    Source: §I · p. 1

  • Method

    The pipeline uses one overhead coastal image to generate terrain and semantic information, then detects and matches assets for placement in a HoloOcean environment.

    Source: §I · p. 1, §III · p. 3, §V · p. 6

  • Results

    Asset placement averaged less than 0.5 meters xy location RMSE and less than 2 degrees orientation RMSE.

    Source: A. Synthetic Environment Regeneration… · p. 5

  • Takeaways & Limitations

    The pipeline produces customized, HoloOcean-compatible coastal levels while reducing reliance on expert Unreal Engine environment-development knowledge.

    Source: §I · p. 2

    The Seabed-Net-generated height maps were non-ideal, particularly because they lacked gradual terrain changes between land and water.

    Source: A. Synthetic Environment Regeneration… · p. 5

Abstract

from arXiv · show

Marine robotic simulation provides a safe and inexpensive method of developing and testing algorithms for unmanned underwater vehicle (UUV) and unmanned surface vessel (USV) autonomy and perception before full field deployment. However, these simulations are often limited by the availability of simulated environments. Current marine robotics simulation suites offer manual ways to edit or create environments, but they require existing data or specialized knowledge of the environment system. To address these issues, we introduce a novel Unreal Engine 5 level generation pipeline that enables automatic creation of coastal environments for HoloOcean. Our pipeline relies on a user-provided overhead image of a coastal scene. The pipeline then uses the image to generate height map data, as well as automatically select assets and place them in the environment.

I. INTRODUCTION

Marine simulation environments support safe and inexpensive UUV and USV autonomy and perception testing, but creating high-fidelity environments often requires substantial time, data, and expertise. This work introduces an automated UE5 pipeline that builds coastal HoloOcean environments from overhead imagery.

  • Environment fidelity can limit large-scale, high-fidelity marine robotics testing because environment creation is often time-, data-, and expertise-intensive.Source: §I · p. 1
  • The proposed UE5 pipeline automatically creates coastal simulation environments for HoloOcean from a user-provided overhead image.Source: §I · p. 1
  • The pipeline combines automatic terrain generation using semantic imagery and height-map estimation with asset selection and placement from an asset library.Source: §I · p. 1
  • The resulting levels support customized HoloOcean environments without requiring expert Unreal Engine environment-development knowledge and remain compatible with HoloOcean sonar implementations and robotic agents.Source: §I · p. 2

II. RELATED WORK

HoloOcean builds on Unreal Engine tools, assets, and rendering, while related simulators offer different mechanisms for creating or customizing environments. These alternatives vary in realism, required input data, and flexibility.

C. Environment Generation

Procedural and language-guided systems can generate diverse simulated scenes, but replicating specific real-world coastal locations remains difficult. The proposed context centers on image-based coastal environment generation and sensor-compatible outputs.

  • Infinigen generates diverse Blender scenes through procedural templates for assets, materials, and animals, with automatic object annotations for computer vision.Source: C. Environment Generation · p. 2
  • SceneX and UnrealLLM use text prompts and language models to control procedural scene-generation parameters in Blender or Unreal Engine.Source: C. Environment Generation · p. 2
  • Text-prompted procedural approaches are not adequate for replicating real-world locations, while other systems retrieve landscape data from GPS coordinates.Source: C. Environment Generation · p. 2
  • The generated environment can be evaluated with Raycast Semantic LiDAR material identification and Imaging Sonar results.Source: Figure 3 · p. 3

D. DEM and Bathymetry Generation

DEM and bathymetry prediction remains challenging, with existing models often specialized to particular terrain or coastal regions. The proposed pipeline uses a base Unreal level in which generated landscapes and assets are later spawned.

  • DEM and bathymetry prediction remains an ongoing challenge, and existing methods target specific terrain types such as mountains or lunar regions.Source: D. DEM and Bathymetry Generation · p. 3
  • Seabed-Net provides bathymetry and pixel-classification prediction models trained on coastal data, but its models struggle to generalize beyond their corresponding locations.Source: D. DEM and Bathymetry Generation · p. 3
  • The pipeline requires only a true or generated overhead image and produces a functional HoloOcean level through landscape generation and asset selection and placement systems.Source: §III · p. 3
  • A mostly empty Unreal base level supplies HoloOcean weather, water, lighting, and underwater post-processing controls before landscapes and assets are spawned.Source: A. Base Level Setup · p. 3

B. Image Input and Semantic Processing

Users provide a scaled aerial coastline image, which is cleaned and semantically segmented into terrain classes for landscape texturing and vegetation placement.

D. Asset Library Generation

The system builds an asset library from rendered user assets, matches detected manmade objects to that library, and uses biome rules to place natural objects procedurally.

Asset Library Generation

  • User-provided assets are rendered under three lighting conditions and two background colors before DINOv3 creates their embedding database.Source: D. Asset Library Generation · p. 3

Asset Selection and Placement

Procedural Natural Placement

A. Synthetic Environment Regeneration Experiments

Synthetic tests compare regenerated HoloOcean environments with manually created ground truth across resolutions, measuring placement accuracy, matching, segmentation, terrain quality, and runtime.

Experimental Setup

Asset Accuracy

Resolution Effects

Limitations

Runtime

B. Real World Imagery Test

A real-world Hale‘iwa Boat Harbor image was hand-labeled for evaluation; the generated level matched the scene visually but struggled with occluded objects and fine object identity.

Evaluation Setup

Results

Limitations

  • The pipeline struggled with partially obscured or closely spaced objects and sometimes selected the wrong object type despite identifying the correct class.Source: B. Real World Imagery Test · p. 6

V. CONCLUSION

The pipeline generates coastal HoloOcean environments by combining height-map modeling, semantic segmentation, and automated asset matching. Generation takes seconds rather than the days required for manually creating complicated coastal levels, while future work targets improved terrain modeling and customization.

  • The pipeline generates coastal HoloOcean environments using height maps, semantic segmentation, and automated asset matching.Semantic segmentation supports texturing and vegetation placement, while YOLO detects objects and DINOv3 selects similar assets.Source: §V · p. 6
  • Future work will replace Seabed-Net with a custom DEM and bathymetry model trained on paired RGB imagery and DEM data.Source: §V · p. 6
  • Generation time scales linearly with detected assets but remains on the order of seconds rather than days of manual coastal-level creation.Source: §V · p. 6
  • Future work will also improve user customization of generated levels.Source: §V · p. 6

Related papers

Loading 2609.10484v1…