PaperlayerView exact revision on arXiv

Source-linked AI summary

Joint Geometric and QoS-Aware Routing in Optical LEO Satellite Networks via DRL

Abdulrahman Al-Hababi, Meysam Ghanbari, Mohammad Taghi Dabiri, Rula Ammuri, Mazen Hasna, Khalid A. Qaraqe

arXiv:2610.01631v1eess.SP

TL;DR

Optical LEO routing must account for changing geometry, jitter-induced outage, and heterogeneous link reliability, constraints not jointly captured by prior routing approaches described here. The paper combines optical feasibility modeling with geometry-aware masked DRL, reporting near-optimal routes and robustness to changing pointing jitter on a Starlink-like constellation.

  • Problem

    Routing must jointly handle dynamic topology, optical-link reliability, and link constraints, while existing approaches described here do not scale well across changing snapshots.

    Source: §I · p. 2, A. Scientific Motivation and Design… · p. 7

  • Method

    The framework combines optical feasibility and a geometry-aware corridor with masked DRL, while reward design allows routing priorities to shift between latency and relay count.

    Source: §VIII · p. 13, D. Results and Discussions · p. 13, Appendix B · p. 14

  • Results

    For Doha→Shanghai, the optimal delay is approximately 32.34 ms and the DRL policy attains 34.66 ms; for Tokyo→Amsterdam, they are about 41.2 ms and 42.91 ms, respectively.

    Source: D. Results and Discussions · p. 12

  • Takeaways & Limitations

    On a Starlink-like constellation, AC-MDQN is reported to provide reliable, near-optimal performance, robustness under varying tracking jitter, and a reward-tunable latency–relay trade-off.

    Source: §VIII · pp. 13–14

Abstract

from arXiv · show

Optical inter satellite links ISLs are becoming the backbone of modern LEO constellations offering high capacity and low latency but introducing stringent geometric and physical layer constraints Routing in such networks must therefore account for time varying topology jitter induced outage and the heterogeneous reliability of intra and inter plane optical links aspects that classical shortest path or existing learning based schemes do not fully capture This paper develops a joint geometric and QoS aware routing framework for optical LEO networks We derive a closed form outage expression under Gaussian beam propagation with pointing errors and obtain analytical maximum feasible link ranges for different ISL classes These relations remove beam divergence from the optimization variables and embed optical feasibility directly into the routing layer leading to a latency reliability capacity constrained routing formulation that is proved to be NP hard To enable scalable decision making we cast snapshot routing as a Markov decision process and introduce an angle constrained masked deep Q network AC MDQN that integrates optical feasibility masks potential based latency shaping and a geometry aware corridor filter around the source destination great circle path This design significantly reduces the effective action space complexity while preserving near optimal routing choices Simulations on a Starlink like constellation demonstrate that AC MDQN achieves end to end latency within 1 to 2 percent of constrained shortest path solutions remains robust under varying pointing jitter and supports controllable hop latency trade offs through reward design The results confirm that the proposed framework provides an efficient and physically consistent routing solution for large scale optical LEO networks

I. Introduction L

Optical LEO routing must handle changing visibility and link distances alongside jitter-dependent outage and heterogeneous link reliability. This paper combines optical feasibility modeling with geometric, QoS-aware routing and a scalable DRL approach.

I. Introduction L

  • Dynamic topology, pointing losses, and differing intra- and inter-plane reliability make routing depend jointly on geometry, physical-layer QoS, and network-layer metrics.The paper identifies link distance and great-circle structure, outage probability and SNR, and end-to-end latency, hop count, and capacity as relevant factors.Source: §I · p. 2

A. Related Work and Research Gaps

  • Prior routing, DRL, and optical-ISL studies leave a gap: they do not jointly enforce jitter-aware optical feasibility and use geometric corridor structure to reduce decision complexity.The proposed framework uses outage-derived, class-dependent link ranges and action masking alongside an angular corridor.Source: A. Related Work and Research Gaps · p. 2

A. Network Entities and Graph Representation

B. Optical Inter-Satellite Link Model

C. Propagation Delay Model

The routing model separates geometric propagation latency from other delays and restricts routing to visible, sufficiently reliable links. A selected set of feasible links forms the source-to-destination path.

C. Propagation Delay Model

  • Link propagation delay is determined by geometric path length, and path propagation latency aggregates this component across the route.Queuing and processing delays are introduced separately in the routing formulation.Source: C. Propagation Delay Model · p. 4

A. Feasible Link Set

  • The feasible-link set restricts routing to links that meet visibility and outage-based reliability conditions, with optical effects reflected in link feasibility.Binary link-selection variables enforce a single end-to-end path through selected links.Source: A. Feasible Link Set · p. 4

B. End-to-End Latency Model

The routing formulation minimizes end-to-end latency subject to reliability and capacity requirements, with link propagation and per-hop overhead represented in the latency model. Its combinatorial, reliability-constrained structure is NP-hard, motivating learned decisions under dynamic geometry and link quality.

B. End-to-End Latency Model

  • Total latency combines propagation delay with per-hop overheads, and the length–hop surrogate penalizes excessive relays when reliable longer links are available.The surrogate’s physical propagation term depends on link geometry; the hop penalty may favor fewer relays when reliability permits.Source: B. End-to-End Latency Model · p. 4, 2) Length–Hop Surrogate Formulation · p. 5
  • The routing objective minimizes end-to-end latency while enforcing reliability and capacity constraints, with an optional global latency budget.The formulation includes an end-to-end reliability requirement and limits aggregated flow demand to available link capacities.Source: C. Objectives · p. 4, B. End-to-End Latency Model · p. 4, 1) Exact Latency Minimization · p. 4
  • Binary routing choices, reliability constraints coupled to optical parameters, and evolving link geometry motivate learning-based route selection for dynamic conditions.The learning-based method is intended to approximate near-optimal decisions under dynamic geometry and probabilistic link quality.Source: 2) Length–Hop Surrogate Formulation · p. 5
  • The reliability- and capacity-constrained routing problem is NP-hard, including its surrogate form.The proof reduces a simplified case to a constrained shortest-path problem with an additive reliability budget; the full formulation adds coupling to optical and geometric parameters.Source: A. Computational Complexity · p. 5

B. Temporal Dynamics and Snapshot Optimization

Because link geometry and outage probabilities change over time, routing is optimized on snapshots. The paper derives outage-based feasible link ranges and shows that maximum reliable range decreases with pointing jitter.

B. Temporal Dynamics and Snapshot Optimization

  • Satellite motion changes link distances and outage probabilities, so the paper optimizes a route and optical settings at each snapshot for the following interval.The sequence of snapshot decisions is modeled as an MDP for learning policies that adapt to geometry and link statistics.Source: B. Temporal Dynamics and Snapshot… · p. 5

C. Outage Constraint as a Function of Distance

  • The closed-form outage-based range bound removes beam divergence from routing optimization because terminal optics adjust divergence to meet the outage target.The resulting range depends on pointing jitter and divergence tuning limits.Source: C. Outage Constraint as a Function of… · pp. 5–6
  • Maximum feasible link range scales inversely with pointing jitter, even under optimal beam tuning.The closed-form range prediction agrees with exhaustive two-dimensional optimization within 0.5–1.5% under the adopted far-field model and link budget.Source: C. Outage Constraint as a Function of… · p. 6

D. Reformulated Optimization with Link-Class-Specific Constraints

The reformulation embeds optical feasibility through link-class-specific geometric range bounds and removes beam divergence as a continuous optimization variable. It retains the routing problem’s combinatorial structure, motivating a DRL approach to dynamic snapshots.

D. Reformulated Optimization with Link-Class-Specific Constraints

A. Scientific Motivation and Design Principles

  • Snapshot-by-snapshot deterministic routing is difficult to scale because dynamic topology and changing feasibility make precomputation intractable and complexity grows exponentially.The paper therefore formulates routing as a sequential decision process and uses DRL to learn latency-minimizing actions under dynamic feasibility constraints.Source: A. Scientific Motivation and Design… · p. 7

B. MDP Formulation

Routing is formulated as a snapshot MDP with local state features and feasible-neighbor actions. Reward shaping aligns learning with latency or reliability objectives, with a stated caveat for the high-reliability approximation.

B. MDP Formulation

  • The agent observes normalized local features for destination distance, progress, feasible-neighbor availability, revisits, and average link cost.The features are scaled to [0, 1] for stability and invariance across constellation scales.Source: 1) State Representation · p. 7
  • Actions are restricted to feasible outgoing links, and an empty feasible set terminates the episode with failure under the static-snapshot assumption.The available action set is defined at the current node.Source: 2) Feasible Actions · p. 7

C. Reward Design: Latency-Driven vs. Reliability-Aware Schemes

  • The per-step hop cost is derived from the routing latency surrogate, aligning learning signals with the routing objective.The paper defines the trajectory as a sequence of links from source to destination.Source: C. Reward Design: Latency-Driven vs.… · p. 7
  • Latency-aligned potential-based shaping accelerates credit assignment while preserving the latency objective up to trajectory-independent constants.The potential uses geometric distance to the destination, and its shaping terms telescope over an episode.Source: C. Reward Design: Latency-Driven vs.… · p. 7, 1) Latency-aligned shaping · p. 7
  • A reliability-aware reward uses a Lagrangian penalty on outage, with a first-order approximation of log(1−p) in the high-reliability regime.The shaped reward aligns with the constrained objective up to constants; λ is the dual variable.Source: 2) Incorporating reliability via… · p. 8
  • An online projected subgradient update increases the dual weight when the realized outage budget is exceeded and decreases it otherwise.The update steers the policy toward the target reliability.Source: 3) Choosing and calibrating the reward · p. 8
  • When the first-order approximation is loose, the paper suggests replacing outage probability with −log(1−Pout,e) to better reflect multiplicative end-to-end success.This alternative requires evaluating a logarithm at each step.Source: 3) Choosing and calibrating the reward · p. 8

D. Learning Algorithm: Masked DQN for Dynamic LEO Graphs

Masked DQN restricts decisions to feasible local links and provides a stable approximation under changing geometry. The paper also identifies sparse-reward exploration limits on long routes and uses corridor pruning to reduce the effective branching factor.

D. Learning Algorithm: Masked DQN for Dynamic LEO Graphs

A. Complexity and Practical Limits

  • Destination-only rewards make sample complexity grow essentially exponentially with route hop count H.Sparse terminal rewards are rarely collected early in training on long routes, while latency shaping partially mitigates the issue; dependence on H persists when neighborhoods are large and near-optimal continuations are few.Source: A. Complexity and Practical Limits · p. 9
  • Geometric corridor refinement reduces effective branching, while wider corridors retain more optimal routes at increased learning cost.The retained satellite count grows approximately linearly with corridor width for small θ.Source: A. Complexity and Practical Limits · p. 9, Table 3 · p. 9

B. Geometry-Aware Evidence on Angular Corridors

The section evaluates whether optimal feasible routes remain near the source–destination great-circle corridor and weighs that coverage against the cost of a wider search region. Moderate corridor widths capture most optimal routes, while wider corridors increase the retained network and learning complexity.

B. Geometry-Aware Evidence on Angular Corridors

  • Across 105 static constellation snapshots, the study evaluates optimal-route containment for corridor half-widths from 10° to 40° using globally optimal deterministic routing.The baseline is solved over each snapshot’s instantaneous network graph.Source: B. Geometry-Aware Evidence on Angular… · p. 9
  • Moderate corridor half-widths already contain most optimal routes, and increasing the width expands the retained region and raises the probability of full route containment.Figure 1 illustrates the retained regions for θ = 20° and θ = 30°.Source: B. Geometry-Aware Evidence on Angular… · pp. 9–10, Figure 1 · p. 10
  • For small θ, the retained satellite count grows approximately linearly with corridor width, increasing the learning agent’s local action space.The proportionality depends on constellation density and source–destination arc length.Source: C. Corridor Width vs. Complexity: A… · p. 10
  • Wider corridors increase both neighborhood size and the effective horizon explored by the agent, raising learning cost.The routing graph admits directed links only when geometric and reliability constraints are satisfied; its deterministic cost captures propagation delay and average per-hop processing penalty.Source: B. Geometry-Aware Evidence on Angular… · p. 9, C. Corridor Width vs. Complexity: A… · p. 10
  • The selected balanced corridor contains 99% of optimal routes while remaining narrow enough to limit exploration cost.Larger corridors capture more optimal routes but increase the retained-node count and branching factor.Source: C. Corridor Width vs. Complexity: A… · p. 10

D. Angle-Constrained Masked DQN: Method Update

The method refines masked DQN routing with a cross-track corridor filter around the source–destination geodesic. The resulting AC-MDQN reduces the masked branching factor while retaining near-optimality on evaluated snapshots.

D. Angle-Constrained Masked DQN: Method Update

  • The corridor-based bound reduces per-episode cost to O(Ā(θ)T_ep) and preserves near-optimality with probability at least 0.98 on evaluated snapshots.The conservative exploration bound substitutes the corridor-limited branching factor for the unfiltered maximum.Source: D. Angle-Constrained Masked DQN:… · p. 10
  • AC-MDQN replaces feasible-action enumeration with the angularly filtered set A_θ⋆(v), while leaving subsequent reward evaluation, replay updates, and target computation unchanged.The filter retains neighbors within the selected cross-track half-width of the source–destination geodesic plane.Source: D. Angle-Constrained Masked DQN:… · pp. 10–11
  • Geometric pruning reduces exploration complexity from O(A_maxT_ep) to O(Ā(θ⋆)T_ep), with average training time cut by more than 40%.The framework retains the baseline DQN’s convergence guarantees.Source: D. Angle-Constrained Masked DQN:… · p. 11

VII. Simulation Results

The evaluation uses randomized Starlink-like constellation snapshots and globally distributed gateways to represent routes of varied geodesic lengths. It then applies a corridor filter and link-class-specific geometric limits to construct a reduced, physically meaningful routing subgraph.

A. Constellation Generation and Ground Sites

  • Twelve globally distributed city gateways and randomized orbital snapshots provide short, medium, and long intercontinental source–destination routes.Snapshot times span 0–5400 seconds, with uniformly randomized initial phase offsets across orbital planes.Source: A. Constellation Generation and Ground… · p. 11

B. Corridor Filtering and Feasible Link Construction

  • The default corridor retains satellites within θ⋆ = 25° of the source–destination path and between the source and destination projections along the track.This produces a reduced but physically meaningful subgraph.Source: B. Corridor Filtering and Feasible… · p. 11
  • Feasible ISLs are further restricted by link-class-dependent geometric range limits, with implementation thresholds tied to typical divergence and jitter values.Figure 2 depicts example routes in red, the constellation in gray, and gateway endpoints in blue.Source: B. Corridor Filtering and Feasible… · p. 11, Figure 2 · p. 12

C. Reinforcement Learning Configuration

The simulation configures masked DQN with a fixed set of candidate next hops and shaped rewards, then trains the agent on the corridor-filtered environment to minimize surrogate latency.

C. Reinforcement Learning Configuration

  • The state includes geometric progress, local neighborhood degree, and visitation indicators, while the action mask enforces feasibility and corridor constraints.The agent uses the masked DQN architecture described for the routing framework.Source: C. Reinforcement Learning Configuration · p. 12
  • At each node, up to five feasible neighbors are sorted by geometric improvement and offered as candidate next-hop slots.Larger action sets improve routing flexibility but increase learning complexity; smaller sets simplify learning at possible cost to optimality.Source: C. Reinforcement Learning Configuration · p. 12
  • Delay penalties, relay penalties, and positive progress rewards shape policies toward minimum-latency links or minimum-hop paths.The trade-off is explicitly evaluated in the performance results.Source: C. Reinforcement Learning Configuration · p. 12
  • Training runs for 30,000 episodes with replay sampling and target updates in the corridor-filtered environment, optimizing the surrogate latency objective.The learned policy is evaluated on unseen constellation snapshots.Source: C. Reinforcement Learning Configuration · p. 12

D. Results and Discussions

The geometry-aware policy stays close to exhaustive shortest-path delays, while unconstrained RL performs worse and tracking jitter sharply limits QoS-feasible routing. Reward design shifts the learned routes between latency and relay-count priorities, and the angular corridor narrows the geometric search space.

D. Results and Discussions

  • 34.66 ms versus 32.34 ms for Doha→Shanghai, and 42.91 ms versus 41.2 ms for Tokyo→Amsterdam, show learned-route delays near the exhaustive shortest-path optimum.The comparisons use the same feasible subgraph; the learned Doha→Shanghai route has a nearly identical hop count.Source: D. Results and Discussions · p. 12
  • The unconstrained RL baseline typically reached around 11 relays and delays near 42 ms for Doha→Shanghai, worse than both geometry-aware RL and the exhaustive solution.The baseline omits angular corridoring and link-class filtering.Source: D. Results and Discussions · pp. 12–13
  • Policies were trained for 30,000 episodes and evaluated on 100 random snapshots across three intercontinental routes, recording average hop count and propagation delay.The routes were Doha→Shanghai, London→New York, and Tokyo→Amsterdam.Source: 1) Evaluation Methodology · p. 13
  • Latency-oriented RL favors lower end-to-end delay, while hop-oriented RH favors fewer relays through longer in-plane hops, sometimes adding propagation delay.The paper notes that reducing relays may help alleviate ISL congestion during peak hours; reward parameters tune this trade-off while retaining feasible routing.Source: 1) Evaluation Methodology · p. 13, D. Results and Discussions · p. 13
  • At 100→400 µrad, delay rises from 26.2 to 41.2 ms; QoS success falls from 100% at 100–200 µrad to 87% at 300, 32% at 400, and 0% at 500 µrad.At 500 µrad, the outage constraint cannot be met for the considered constellation and QoS threshold.Source: 1) Evaluation Methodology · p. 13, Table 6 · p. 13
  • The angular corridor is a spherical band of width 2θ⋆ around the source–destination great-circle segment; widening θ retains more satellites and raises the chance the optimum lies inside.Learned routes sometimes favor longer in-plane hops to reduce relay count while remaining within reliability limits.Source: D. Results and Discussions · p. 12, Appendix B · p. 14, Appendix C · p. 15

Related papers

Loading 2610.01631v1…