PaperlayerView exact revision on arXiv

Source-linked AI summary

Reinforcement Learning-Based 3D Beam Adaptation for Underwater Wireless Optical Communication with AUVs

Viviana Centritto~Arrojo, Ama Bandara, Sergi Abadal, Evgenii Vinogradov

arXiv:2610.06407v1eess.SPcs.ETcs.SYeess.SY

TL;DR

Misalignment and current-driven drift threaten connectivity for mobile AUVs using underwater optical links. The paper combines 3D DRL beam control with a channel model that includes depth-dependent impairments. Across 1,500 trajectories, mean excess outage is 8.5% without positional uncertainty and 16.2% under current-induced drift.

  • Problem

    Transmitter–receiver misalignment can cause complete link outages, and ocean currents introduce drift and jitter that compromise pointing accuracy.

    Source: §I · p. 1

  • Method

    The paper develops a 3D UWOC channel model and a DRL control framework that adjusts beam steering and divergence while accounting for current-driven positional uncertainty.

    Source: §I · p. 1, §II · p. 2

  • Results

    Mean excess outage was 8.5% under positional certainty and 16.2% when the receiver trajectory was perturbed by ocean currents.

    Source: §VI · p. 5

  • Takeaways & Limitations

    The agent tracked the receiver under both positional certainty and current-induced drift, with excess outage quantifying losses attributable exclusively to pointing errors.

    Source: §VI · p. 5, §V · p. 5

Abstract

from arXiv · show

Underwater Wireless Optical Communications (UWOC) provide essential high data rates for Autonomous Underwater Vehicles (AUVs), but reliable connectivity is critically affected by transmitter-receiver misalignment. This work addresses the beam pointing problem for a moving AUV subject to unknown ocean currents through a Deep Reinforcement Learning (DRL) framework. We develop a comprehensive 3D UWOC channel model incorporating depth-dependent attenuation, turbulence, and a discrete-ray method to accurately quantify geometric and misalignment losses. The resulting agent jointly optimizes beam steering and divergence angles, learning a policy that prioritizes continuous link maintenance. Evaluated using real oceanographic data, the framework's performance is assessed via the excess outage metric, which isolates outages occurring exclusively due to pointing errors. Relative to perfect alignment between nodes, the proposed method bounds this metric to 8.5% under ocean current-free tracking conditions and limits it to 16.2% when subjected to current-induced drift. The findings obtained in this work confirms that DRL-enabled beam control is capable of adaptive tracking and mitigating link outages in dynamic underwater environments.

I. INTRODUCTION

UWOC can provide high data rates, but underwater impairments and transmitter–receiver misalignment threaten link reliability, especially for mobile AUVs affected by currents. The paper addresses the gap in 3D tracking for submerged AUVs with a DRL framework and a 3D channel model.

I. INTRODUCTION

  • Misalignment can cause complete link outage, while ocean currents introduce drift and jitter that increase positional uncertainty.UWOC is presented as a high-data-rate, low-latency option for underwater links facing RF attenuation and acoustic bandwidth limits.Source: §I · p. 1
  • Prior work largely uses 2D models or surface-bound receivers, leaving 3D tracking for submerged, mobile AUVs unresolved.The proposed framework incorporates vertical link characteristics and current-induced positional uncertainty, and evaluates individual optical rays that fail to reach the receiver aperture.Source: §I · p. 1
  • The work characterizes the 3D UWOC environment and designs and evaluates a DQN control framework to maintain beam alignment and link quality.The approach quantifies geometric and pointing losses by evaluating the individual optical rays that fail to reach the receiver aperture.Source: §I · p. 1

II. SYSTEM MODEL

  • The system models a fixed transmitter and mobile receiver AUV, with the transmitter adjusting beam steering and divergence to track the receiver’s trajectory.Ocean-current disturbances are represented as depth-dependent northward and eastward velocity components.Source: §II · pp. 1–2
  • The evaluation considers a current-free scenario with zero positional uncertainty and a second scenario with dynamic current-driven drift.The AUVs are assumed to know their starting positions and planned waypoints, estimate position and velocity using IMUs, and exchange state information over a bidirectional UWOC link.Source: §II · p. 2

III. DRL IMPLEMENTATION

The DRL model keeps its action space, reward function, and UWOC environment consistent across scenarios, while changing the state formulation to represent current-driven drift. The current-induced state adds environmental data alongside tracking information.

III. DRL IMPLEMENTATION

  • Across both scenarios, the DRL model uses the same action space, reward function, and UWOC environment; only the state-space formulation differs.Source: §III · p. 2

A. State space vector (St)

  • The current-free tracking state, Strack, contains 11 variables.Source: A. State space vector (St) · p. 2
  • For current-driven drift, the augmented state has 24 continuous variables in two branches, adding 13 components to separate beam dynamics from environmental uncertainties.The environmental data are generated using HYCOM; the two branches comprise Strack and Senv.Source: A. State space vector (St) · p. 2

B. Action space

The agent selects discrete steering and divergence updates and receives a reward that favors alignment while penalizing loss of optical capture. The channel model represents attenuation, turbulence, geometric spreading, and pointing losses.

B. Action space

  • The agent chooses among 27 actions combining increases, decreases, or holds for two steering angles and the divergence angle.The steering-angle step sizes are 10 mrad, and the divergence-angle step size is 1 mrad.Source: B. Action space · pp. 2–3

C. Reward function

  • The reward uses the ratio of instantaneous to ideal SNR for alignment efficiency and assigns −1 when misalignment exceeds the beam radius plus receiver aperture radius.The beam radius w(z) is agent-controlled, while Ar is a fixed hardware constraint.Source: C. Reward function · p. 3

IV. UWOC ENVIRONMENT: CHANNEL MODELING

  • The received electrical signal is modeled from transmitted power, photodetector responsivity, overall channel loss, and additive noise.The overall loss comprises channel attenuation, turbulence, geometric spreading, and pointing losses.Source: §IV · p. 3

A. Channel attenuation

The attenuation model follows Beer–Lambert decay and represents attenuation as depth- and water-type-dependent absorption and scattering. It models chlorophyll with a vertical profile to capture water-column heterogeneity.

A. Channel attenuation

  • Attenuation follows Beer–Lambert power decay with propagation distance and a coefficient formed from absorption and scattering.Both coefficients are based on the Haltrin model.Source: A. Channel attenuation · p. 3
  • Absorption sums aquatic-constituent contributions scaled by concentration; this work considers pure water and chlorophyll as its main components.Scattering sums contributions from pure sea water, small particles, and large organic particles.Source: A. Channel attenuation · p. 3
  • Chlorophyll concentration is modeled as a Gaussian vertical profile to represent heterogeneity associated with the Deep Chlorophyll Maximum.The profile uses surface background concentration and gradient, peak depth, integrated peak biomass, and layer thickness.Source: A. Channel attenuation · p. 3
  • Adjusting the surface background chlorophyll concentration simulates pure sea water, clear ocean, coastal, and harbor water types.The corresponding B0 values are 0.005, 0.31, 0.83, and 5.9, respectively.Source: A. Channel attenuation · p. 3

B. Turbulence

The turbulence model divides the water column into discrete layers and combines their fading under a weak-turbulence assumption. Its environmental characterization derives layer scintillation from temperature and salinity profiles and numerical integration.

B. Turbulence

  • Under weak turbulence, total fading is the product of independent lognormal fading coefficients for the water-column layers.The water column is divided into K layers, each characterized by thickness and scintillation index.Source: B. Turbulence · p. 3

1) Characterization of the turbulence environment:

  • Temperature and salinity profiles from HYCOM are used to compute the density ratio, with vertical gradients and seawater coefficients characterizing the turbulence environment.The thermal expansion and saline contraction coefficients are calculated using the Gibbs-SeaWater Oceanographic Toolbox.Source: 1) Characterization of the turbulence… · pp. 3–4
  • Each layer’s scintillation index is derived from its Rytov variance, which requires numerical integration over normalized path distance and spatial frequency.The auxiliary parameters include the optical wavenumber, modified Nikishov spatial power spectrum, Fresnel ratio, and estimated curvature parameter.Source: 1) Characterization of the turbulence… · p. 4

3) Estimation of the total turbulence fading coefficient:

The total turbulence fading coefficient is built from layer-level fading under an independence assumption, then stochastically generated with its mean logarithmic amplitude normalized for unity power expectation.

3) Estimation of the total turbulence fading coefficient:

C. Geometric spreading and pointing losses

The model represents a Gaussian-profile laser beam with discrete rays and quantifies geometric and pointing losses by how many rays miss the receiver aperture. Rays are projected onto the receiver plane at its depth.

C. Geometric spreading and pointing losses

D. Noise model

The noise model treats total noise variance as a sum of three independent components and expresses ambient solar noise in terms of receiver and bandwidth parameters.

D. Noise model

  • Total noise variance is defined as the sum of three independent variances.Source: D. Noise model · p. 4
  • Solar background often constitutes the most significant interference source [14], and Psol depends on Rx field of view, aperture radius, and depth-dependent upwelling solar radiance.Source: D. Noise model · p. 4
  • Solar-background noise variance is 2qSPsolB, with q, S, B, and Psol denoting electronic charge, Rx sensitivity, bandwidth, and background solar power.Source: D. Noise model · p. 4

V. RESULTS

Across unseen trajectories, the agent tracks the receiver and adapts beam divergence under drift. Mean excess outage is higher with current-induced trajectories than in current-free tracking.

V. RESULTS

  • In a current-free test, the Tx beam closely followed the Rx-AUV through planned waypoints, with minor oscillations from discrete-step decisions.Source: §V · p. 4, Figure 5 · p. 5
  • At 2.5 s, alignment below Ar led the agent to narrow the beam and achieve -21.82 dBm; beyond 3.5 s, it increased divergence as tracking stability deteriorated.Source: §V · p. 5
  • Mean excess outage was 8.5% under current-free conditions and 16.2% in current-induced trajectories across 1,500 unseen episodes.Source: §V · p. 5
  • Under induced currents, the Rx-AUV deviated from its planned path, and the agent increased beam divergence to maintain connectivity.Source: Figure 6 · p. 5, §V · p. 5

VI. CONCLUSION

The work presents a 3D DRL framework for mobile-AUV misalignment, combining a depth-dependent UWOC channel model with discrete-ray pointing-loss simulation. In evaluation, mean excess outage was 8.5% without current-induced drift and 16.2% with it.

VI. CONCLUSION

  • The framework models 3D UWOC channel impairments and uses a discrete-ray mechanism to simulate pointing loss for mobile AUVs.Source: §VI · p. 5
  • Mean excess outage was 8.5% under positional certainty and 16.2% when ocean currents perturbed the Rx trajectory.Source: §VI · p. 5
  • The learned baseline policy estimates oceanic drift and preserves link reliability by controlling beam direction and divergence.Source: §VI · p. 5
  • When alignment is precise, the agent narrows the beam.Source: Figure 7 · p. 6
Loading 2610.06407v1…