Source-linked AI summary
LOTUSim-Energy: A Maritime Simulator for Human-Drone Interaction in Autonomous Offshore Operation \& Maintenance
Juliette Grosset, Marie Dubromel, Hélène Lechêne, Quentin Arzel, Cédric Buche
TL;DR
Offshore O&M needs coordinated multi-domain robots, realistic environmental physics, and human supervision, while existing simulators often lack integrated HITL interaction and environmental physics. LOTUSim-Energy unifies heterogeneous vehicle dynamics, wind–wave–current forcing, immersive supervision, repeatable O&M tasks, energy monitoring, and modular autonomy pipelines. A multi-domain monopile and transition-piece inspection scenario evaluates waypoint following, AIS-referenced trajectory tracking, and real-time energy monitoring within the simulator.
Problem
Offshore O&M needs coordinated multi-domain robots, realistic environmental physics, and human supervision, while existing simulators often lack integrated HITL interaction and environmental physics.
Method
LOTUSim-Energy unifies heterogeneous vehicle dynamics, wind–wave–current forcing, immersive supervision, repeatable O&M tasks, energy monitoring, and modular autonomy pipelines.
Results
A multi-domain monopile and transition-piece inspection scenario evaluates waypoint following, AIS-referenced trajectory tracking, and real-time energy monitoring within the simulator.
Takeaways & Limitations
LOTUSim-Energy is a robotics-centric simulator unifying vehicle dynamics, environmental forcing, multi-user supervision, and benchmarking metrics for guidance, planning, and perception.
Abstract
from arXiv · showhide
Offshore maintenance requires operations in the air, the surface, and the subsea domain and include human supervision. This paper presents LOTUSim-Energy, a real-time maritime simulator designed for multi-domain human--drone interaction for offshore operation and maintenance. The plat- form unifies heterogeneous unmanned vehicles (Unmanned Aerial Vehicles: UAVs, Unmanned Surface Vehicles: USVs, Autonomous Underwater Vehicles: AUVs, Remotely Operated Vehicles: ROVs) within a distributed architecture coupling environment forcing (wind, waves, currents) and provides immersive user interfaces for supervision (desktop and virtual reality). A structured offshore task library enables repeatable evaluation of autonomy stacks under realistic metocean disturbances. The simulator supports realistic physics, energy-aware battery modeling, and fault-detection pipelines as modular validation tools. System-level performance is demonstrated on a multi-domain inspection scenario for monopile and transition piece structure, where we evaluate the reliability of integrated waypoint-follower plugin and Automatic Identification System (AIS)-referenced trajectory tracking under real-time energy monitoring. By combining unified environmental physics, heterogeneous vehicle simulation, and immersive supervision, LOTUSim-Energy provides an integration testbed for prototyping and rehearsing offshore human--robot collaboration workflows, as a step toward de-risking sea deployment.
I. INTRODUCTION
Offshore multi-robot maintenance needs realistic environmental physics and human supervision, but existing simulators often do not integrate both. LOTUSim-Energy addresses this gap with a unified simulator, repeatable task library, and modular autonomy stack.
- Motivation: High-fidelity simulation is presented as necessary for validating human–robot coordination before offshore sea trials because weather windows and safety constraints are narrow.Perception failures or incorrect energy estimates can cause severe asset-loss consequences.
- Research gap: Existing simulation platforms commonly excel in rendering, sensors, or AI training but lack integrated human-in-the-loop interaction and environmental physics.These omissions limit validation of coordinated offshore autonomy under realistic conditions.
- Core contribution: LOTUSim-Energy integrates cross-domain vehicle dynamics, physics-consistent wind, waves, and currents, and immersive human supervision.The architecture supports manual takeover, multi-user interaction, and mission procedure rehearsal.
- Benchmarking task library: A configurable offshore task library enables repeatable evaluation of heterogeneous autonomy systems and human intervention under realistic maritime disturbances.The scenarios target multi-domain offshore operation and maintenance inspection.
- Autonomy stack: The integrated autonomy stack includes AIS-referenced trajectory tracking, waypoint-following and geometric controllers, battery monitoring, and vision-based crack and corrosion detection.Battery simulation monitors charge and discharge in relation to propulsion effort.
A. Offshore Wind Turbine Technology & O&M
Offshore wind O&M is costly and difficult because harsh marine conditions, limited weather windows, and coordination demands constrain operations. Robotics and high-fidelity simulation are positioned as tools for safer, more proactive inspection and validation across air, surface, and subsea domains.
- O&M challenges: Offshore wind O&M remains technically challenging and costly because of harsh marine conditions, limited weather windows, and complex operational coordination.O&M is identified as a substantial part of the turbine lifecycle and contributes 15–35% of lifetime costs.
- Role of robotics: Robotics and digitalisation can shift O&M toward remote, data-driven, and proactive operations while enabling safer inspections, faster surveys, and improved predictive monitoring.Blade inspection times are projected to decrease by up to ∼40%.
- Multi-domain robotics: Aerial, surface, and subsea robots support coordinated offshore O&M, while reliable deployment requires realistic cross-domain physics and human interaction for supervision and mission validation.The relevant platforms include UAVs, USVs, AUVs, and ROVs.
- Simulation gap: Existing simulators often lack native human-in-the-loop support, realistic underwater current disturbances, or unified multi-domain operation.Common current representations include fixed CFD fields, constant unidirectional currents, or simplified profiles.
- LOTUSim-Energy response: LOTUSim-Energy combines unified air, surface, and subsea physics, depth-dependent current drift and shear, and immersive human-in-the-loop interfaces for pre-deployment validation.These capabilities target realistic validation of autonomous O&M strategies before deployment.
III. LOTUSIM-ENERGY ARCHITECTURE
LOTUSim-Energy uses a distributed server–client architecture to coordinate physics, rendering, and agent interaction across maritime domains. Gazebo manages timing and asset orchestration while external modules provide extensible dynamics, communication, and immersive rendering.
- Architecture: LOTUSim-Energy is a distributed server–client framework in which Gazebo orchestrates assets and simulation timing through a deterministic step scheduler.Separate client modules execute specialized simulation tasks.
- Client modules: The architecture separates Physics, Agent Interaction, and Rendering clients for modular execution locally or remotely.ROS 2 supports inter-agent messaging and hardware-in-the-loop bridges, while Unity provides optional high-fidelity HRI rendering.
- Scheduling: Each timestep requires client updates to return before the next synchronization barrier, supporting consistent world state and both real-time and accelerated-time execution.Requests are randomly permuted with a reproducible seed to reduce scheduling bias; supported modes include RTF≈1 and RTF>1.
- Positioning: The simulator’s architecture is presented alongside a state-of-the-art comparison focused on autonomous O&M and immersive operator decision-making.The comparison table marks unreported capabilities as N/A.
- Physics interface: The unified physics interface exchanges asset pose, twist, actuation, and environment parameters with external engines and re-ingests wrenches or updated poses.Marine vehicles use six-degree-of-freedom Xdyn dynamics with added mass, damping, wave, and current forces, connected through gRPC/WebSocket.
B. Agent Interaction
LOTUSim-Energy separates physics, visualization, and agent logic while supporting ROS 2 autonomy integration and desktop/VR human supervision. Its interfaces support distributed multi-vehicle interaction, operator control, and scalable rendering or sensor generation.
- Agent interfaces: ROS 2 exposes topics, actions, and services for guidance, planning, perception, and logging, enabling autonomy-stack or hardware-bridge integration.
- Human supervision: Desktop and virtual-reality Unity interfaces support authority handover, operator-action logging, and real-time human–robot interaction.
- Rendering: Optional Unity rendering provides photorealistic views and synthetic RGB/IR, depth, segmentation, lidar, and sonar outputs with pixel-accurate ground truth.
- Operator roles: The simulator includes physics-constrained embodied control and free-flight supervisor modes for mission rehearsal, oversight, and dynamic waypoint placement.
B. Aerial
The simulator models aerial wind disturbances and underwater current fields across a defined local ENU space–time domain. These environmental models provide configurable atmospheric forcing and depth-dependent current representations for heterogeneous agents.
- Aerial environment: Gazebo’s wind plugin provides dynamic wind fields varying in time and space, so aerial drones respond consistently to configured atmospheric conditions.
- Aerial interface: The platform includes X500 drone agents and a user interface for setting different wind forces.
- Current environment: LOTUSim-Energy implements baseline constant flow and an Ekman-inspired current model whose velocity components depend on x, y, z, and t.
- Domain conventions: The current field is defined over a local ENU wind-farm domain bounded horizontally by Ωh and vertically by bathymetric depth H(x, y).
- Depth conventions: Surface and seabed depths are represented by zs and zb, with both constrained to the interval [0, H(x, y)].
2) Parameters:
The current-model parameters specify Coriolis effects, wind drag, wind stress, and a surface-current formulation combined with Airy-wave orbital velocities. These choices determine spiral orientation, wind forcing, and surface velocity profiles.
- Parameters: The Coriolis parameter f = 2Ωsin(φ) uses latitude and Earth’s rotation rate, with its sign setting spiral rotation by hemisphere.
- Parameters: Wind stress is modeled as τs = CDρairU^2_10 using the 10 m wind speed U10.
- Parameters: Surface currents combine Airy-wave orbital velocities with a northern-hemisphere Ekman spiral.
- Parameters: The formulation identifies zs as depth, Ds as surface Ekman-layer depth, V0 as surface current velocity, and ϕ as surface wind orientation.
4) Middle Layer:
The middle current layer is modeled as unaffected by waves or the seabed. This separates the interior flow from the boundary-layer formulations used near the surface and seabed.
- 4) Middle Layer:: The middle layer is unaffected by waves or the seabed.
- 4) Middle Layer:: Bottom-layer corrections for irregular seabeds maintain continuity and satisfy boundary conditions, while seabed-proximal tasks encounter bottom-layer shear.
V. AUTONOMOUS OFFSHORE O&M
LOTUSim-Energy integrates a multi-turbine offshore wind-farm scene, heterogeneous air/surface/subsea vehicles, calibrated metocean fields, and energy-aware mission monitoring for repeatable O&M rehearsal.
- Environmental Modeling: MetOcean measurements are assimilated to calibrate Ekman-layer coefficients and generate next-day subsurface-current forecasts for safety analysis, risk management, and scheduling.The model ingests time-stamped wind, wave, and current fields from the Copernicus Marine Service.
- Simulation Environment and Assets: The simulator combines turbine digital twins and wind, wave, and current fields with UAV, USV, ROV, and AUV agents for offshore O&M rehearsal.Users can rehearse launch/recovery, waypoint transit, station-keeping, blade inspection, and subsea transects while logging time, energy, and safety-hold states.
- Subsea Operations: The platform supports BlueROV close-range inspection and LRAUV wide-area surveys under teleoperation, shared-control, or fully autonomous modes.BlueROV targets visual inspection, cathodic-protection checks, and light intervention, whereas LRAUV supports cable and structure transects.
- Energy-Aware Mission Planning: Its battery plugin estimates state of charge from each vehicle’s instantaneous propulsive effort instead of assuming constant power draw.Real-time voltage and state-of-charge outputs support mission planning and operator evaluation under realistic power constraints.
VI. EXPERIMENTAL INSPECTION SCENARIO
The experimental scenario evaluates a cross-domain monopile and transition-piece inspection path using coordinated surface, underwater, and aerial tasks, including AIS-referenced vessel guidance.
- Cross-Domain Inspection Path: The inspection path spans surface, underwater, and aerial waypoint sequences for monopile and transition-piece inspection.The task library covers representative offshore wind O&M activities and identifies empirically demonstrated tasks.
- Surface Monitoring: Surface monitoring uses a support vessel or USV following a wide-area AIS-based waypoint trajectory.The trajectory is derived from a real AIS track rather than a ground-truth controller-tracking reference.
- Underwater and Aerial Inspection: BlueROV2 performs close-range underwater TP surveys to detect marine growth, corrosion, and cracks, while X500 drones inspect above-water structures for corrosion or damage.The two vehicle classes cover complementary subsea and aerial inspection domains.
- Waypoint Following: The waypoint follower uses closed-loop PID heading control, bang–bang linear-velocity regulation, waypoint tolerance, acceleration limits, and velocity saturation.Dynamic replanning is exposed through a ROS2 service interface.
- AIS-Referenced Trajectory Following: Figure 6 compares the real AIS plot with the simulated plot for 10 points.The comparison illustrates AIS-referenced guidance under the disturbances and GNSS noise associated with an independently controlled vessel.
D. Visual Inspection Using Onboard Detection
The visual-inspection pipeline combines YOLO-based anomaly detection across underwater and aerial vehicles with current-disturbance experiments, battery monitoring, and immersive operator control.
- Visual Inspection: YOLO-based real-time detection identifies structural anomalies such as cracks and corrosion in BlueROV2 underwater and X500 aerial inspections.Figure 7 illustrates detections in both domains during structural and blade inspection.
- Environmental Disturbance: An actively propelled LRAUV still requires continuous thruster compensation because depth-dependent currents cause lateral drift and rotational flow.The vehicle is commanded to hold a straight trajectory while exposed to the vertically resolved Ekman-based current field.
- Operator Supervision and Energy Monitoring: Operators monitor AIS traffic and control inspection cameras while the battery plugin tracks vehicle capacity and discharge in real time.Battery information lets operators adapt inspection intensity to preserve reserve for safe recovery.
- Immersive Interfaces: The Desktop/VR interface supports hand-gesture camera control and an education mode with real-time wind and weather controls.Leap Motion enables camera zooming and reorientation without requiring a full VR headset.
- Scope and Future Work: Future work includes physics-based underwater image formation, biodiversity-planning constraints, and sim-to-real benchmarking for energy-aware planners and fault detection.These extensions define the current boundary of the demonstrated simulator capabilities.