PaperlayerView exact revision on arXiv

Source-linked AI summary

RL-based Network Slice Embedding over Space Division Multiplexed Elastic Optical Networks

Divya Khanure, Riti Gour†, Congzhou Li, Jason P. Jue

arXiv:2608.29444v1cs.NI

TL;DR

Jointly embedding slices in SDM-EONs requires coordinating compute and spectrum resources in an NP-hard, multidimensional optimization problem. The paper proposes PPO-Full, a PPO-based joint optimizer, and reports the highest acceptance under hotspot traffic, with its advantage widening as load increases.

  • Problem

    SDM-EON slice embedding jointly involves compute, spectrum, spatial-core, and routing constraints, creating multidimensional bin-packing complexity that makes exact optimization intractable.

  • Method

    PPO-Full jointly optimizes compute-node selection and RMCSA over K candidate nodes and paths using a multidimensional action space, with slot-efficiency reward shaping.

  • Results

    PPO-Full maintains the highest acceptance rate under 30% hotspot traffic, reaching ≈80% at λ = 75 versus ≈76% for WMSM-F and lower values for the other baselines.

  • Takeaways & Limitations

    PPO-Full combines high acceptance with low provisioning cost and low fragmentation, while decoupled or distance-based approaches exhibit resource-efficiency or blocking trade-offs.

Abstract

from arXiv · show

Network slicing over space-division-multiplexed elastic optical networks (SDM-EONs) requires jointly managing spectrum, spatial cores, and compute resources, a coupling that many existing studies ignore by treating compute placement independently from routing and spectrum decisions. This disconnect can cause the spectrum to be allocated along a path, only for the request to fail due to insufficient compute resources along the path, or may result in compute resources being allocated without consideration for spectrum resource availability on the path between compute nodes. We propose a path-constrained reinforcement learning framework that addresses compute node selection and RMCSA, being aware of both resources, restricting the RL agent's action space to nodes along $k$-shortest paths between request endpoints. Training incorporates reward shaping to improve robustness under high load. We propose PPO-Full (Proximal Policy Optimization-Full), which jointly selects compute nodes and routing paths via a multi-dimensional action space, against distance-based heuristics, a greedy baseline, and a decoupled VONE-DRL baseline on a 24-node USNET topology under hotspot traffic conditions. Results demonstrate consistent improvements in acceptance rate over all baselines at high load, with gains becoming more pronounced as traffic intensity increases.

I. BACKGROUND LITERATURE

Prior SDM-EON and reinforcement-learning studies address optical allocation or compute placement separately, while this paper formulates slice embedding with coupled routing, spectrum, core, and compute decisions.

  • Earlier VNE formulations improved spectral efficiency but used single-core architectures without compute-aware placement.
  • Crosstalk-aware SDM-EON studies and congestion-aware RMCSA heuristics generally treated compute placement independently from optical allocation.
  • Existing optical-resource RL methods addressed RMSA or RMCSA without incorporating compute placement, limiting their applicability to joint slice embedding.
  • Prior VNE work studied co-location and coordinated node-link mapping, while masked DRL for VONE used decoupled agents on single-core EONs.
  • The system model represents an SDM-EON as a graph with spatial cores, spectrum slots, compute-capable nodes, and finite compute capacity.
  • Each request is accepted only when VNFs are placed at feasible compute nodes and spectrum is provisioned on all resulting segments.

B. Cost Model

The cost model combines compute and spectrum consumption for accepted VNRs, while feasibility requires reach, spectrum, core-group, and compute-capacity constraints.

  • B. Cost Model: Average provisioning cost sums allocated compute units and spectrum slots across successfully provisioned VNRs, with equal compute and spectrum weights.Lower cost indicates more resource-efficient allocation, while the objective also maximizes acceptance rate.
  • C. Constraints: Requests are blocked when slice-mapping constraints are violated.
  • C. Constraints: Segment paths must remain within the maximum transmission reach of the selected modulation format.Supported formats include 16-QAM up to 500 km, QPSK up to 1000 km, and BPSK up to 2000 km.
  • C. Constraints: Each request occupies a contiguous spectrum-slot set whose size is determined by bitrate and modulation efficiency.
  • C. Constraints: Spectrum continuity is required across links within a segment but not across segments separated by OEO conversion.
  • C. Constraints: Compute allocation must respect node capacity limits.
  • C. Constraints: Core-group selection assigns requests to non-adjacent core groups to mitigate inter-core crosstalk.

D. Performance Metrics

Performance is assessed using acceptance, spectrum utilization, provisioning cost, and fragmentation, reflecting both provisioning success and resource efficiency.

  • D. Performance Metrics: Acceptance rate measures the fraction of successfully provisioned requests.
  • D. Performance Metrics: Spectrum utilization measures consumed slots relative to total network slots across links, cores, and slots per core.
  • D. Performance Metrics: External fragmentation is computed from the largest contiguous free block and total free slots, then averaged across cores.
  • D. Performance Metrics: The problem is NP-hard because it combines routing and spectrum assignment, modulation reach, inter-core crosstalk, and compute placement.PPO-Full requires approximately 135 µs per request for policy inference on CPU, with training performed offline.

III. PROPOSED FRAMEWORK

PPO-Full restricts decisions to small candidate sets and jointly selects compute nodes and routing paths through a multidimensional PPO action space.

  • III. PROPOSED FRAMEWORK: PPO-Full ranks candidate compute nodes by path distance and uses a learned PPO policy to jointly select a compute node and routing path.
  • III. PROPOSED FRAMEWORK: The framework uses C = 3 candidate nodes and K = 3 candidate paths to balance decision quality with action-space tractability.Larger candidate sets increase joint combinations, slowing convergence and producing sparser rewards.
  • III. PROPOSED FRAMEWORK: K = 3 was selected after sweeping K ∈{2, 3, 4, 5} as the best trade-off between candidate diversity and action-space tractability.
  • III. PROPOSED FRAMEWORK: MultiDiscrete([C,P]) yields C×K = 9 joint node-path combinations per decision step.

A. Action Space

PPO-Full reduces the decision space by selecting among filtered compute-node and routing-path candidates, while observing request, network, and per-candidate resource features.

  • A. Action Space: PPO-Full ranks and filters compute nodes by path distance and feasibility, reducing node selection from O(|Vc|) to O(C).It uses C = 9 candidate compute nodes derived from K = 3 shortest paths.
  • A. Action Space: The MultiDiscrete() action jointly selects one of three compute-node candidates and one of three path candidates.This yields nine joint node-path combinations per decision step.
  • A. Action Space: The observation combines request features, network utilization and fragmentation, and nine per-candidate features describing compute, distance, utilization, free spectrum, and fragmentation.The per-candidate features describe both resulting path segments.

C. Reward Shaping

The reward shapes PPO training toward accepted requests with more spectrum-efficient allocations, while the implementation uses a discrete-action PPO policy and a defined training checkpoint procedure.

  • C. Reward Shaping: The reward function incentivizes request acceptance while penalizing spectrum-inefficient allocations.The supplied passage introduces the reward's two objectives but does not provide the complete displayed formula.
  • C. Reward Shaping: The slot-efficiency bonus decreases linearly with spectrum usage and reaches zero at allocations consuming 200 or more slots.Here S denotes the total spectrum slots consumed by the accepted request.
  • C. Reward Shaping: PPO-Full uses an MLP policy with two hidden layers of 256 neurons for its discrete action space.The implementation uses net_arch=[256,256].
  • C. Reward Shaping: The best checkpoint is selected by mean acceptance rate evaluated every 50,000 training steps at λ ∈ {15, 30, 50}.The cited implementation passage identifies the evaluation loads but not a reward-shaping outcome for them.
  • C. Reward Shaping: The allocation procedure decodes a policy action into an anchor node and path index, partitions VNFs, and blocks requests when placement or spectrum allocation fails.Accepted allocations then reserve compute units and update network state.

E. Baseline Approaches

The baselines use first-fit spectrum allocation and differ in how they select compute nodes, with distance-based ranking followed by feasibility checks.

  • E. Baseline Approaches: All baselines use first-fit spectrum allocation across core groups and differ only in compute-node selection.
  • E. Baseline Approaches: Endpoint-Biased ranking favors proximity to either request endpoint, whereas Equidistant ranking minimizes endpoint-distance imbalance.Both strategies rank compute nodes relative to the source and destination.
  • E. Baseline Approaches: The baselines attempt the top five ranked candidates in order, assigning VNFs to the first feasible node pair before first-fit spectrum allocation.

2) Greedy Heuristic Baseline (WMSM-F):

WMSM-F selects compute waypoints by jointly minimizing path length and segment imbalance, then applies first-fit spectrum allocation; the related VONE-DRL baseline separates node and link decisions.

  • 2) Greedy Heuristic Baseline (WMSM-F):: WMSM-F jointly minimizes total waypoint path length and segment imbalance δW when selecting compute waypoints.The imbalance is the maximum segment distance minus the minimum segment distance.
  • 2) Greedy Heuristic Baseline (WMSM-F):: WMSM-F uses first-fit spectrum allocation and blocks a request when no feasible configuration exists.
  • 2) Greedy Heuristic Baseline (WMSM-F):: The VONE-DRL baseline adapts separate node and link mapping agents to the four-VNF chain and uses PPO-Full's hyperparameters and training budget.The shared budget is intended to support comparison under identical computational constraints.

A. Simulation Setup

The evaluation compares PPO-Full with distance-based, greedy waypoint, and decoupled RL baselines on a 24-node USNET topology under hotspot traffic. Results cover acceptance, blocking, provisioning cost, fragmentation, and utilization.

  • Evaluation strategies: The study evaluates five strategies, including PPO-Full, Endpoint-Biased, Equidistant, WMSM-F, and VONE-DRL.Heuristic baselines use first-fit spectrum allocation on resulting path segments.
  • Acceptance rate: PPO-Full reaches ≈80% acceptance at λ = 75, exceeding WMSM-F (≈76%), Endpoint-Biased and VONE-DRL (≈74%), and Equidistant (≈70%).Its advantage widens progressively with load under 30% hotspot traffic.
  • VONE blocking: PPO-Full achieves the lowest non-trivial VONE blocking (≈12%), while WMSM-F has the highest (≈19%).VONE-DRL has near-zero VONE blocking but low acceptance, reflecting poor spectrum efficiency from decoupled routing.
  • RMCSA blocking: WMSM-F records the lowest RMCSA blocking (≈5%), while PPO-Full (≈8%) outperforms Endpoint-Biased (≈13%) and Equidistant (≈18%).VONE-DRL performs worst at approximately 26% RMCSA blocking when λ = 75.
  • Cost and resource use: PPO-Full maintains the lowest provisioning cost (≈59–67), while VONE-DRL ranges from ≈85 at low load to ≈71 at λ = 75.PPO-Full has ≈0.06 fragmentation and ≈29% utilization; WMSM-F has lower fragmentation but higher blocking.

V. CONCLUSION

The paper concludes that PPO-Full jointly selects compute nodes and routing paths for SDM-EON slice embedding, achieving strong performance across hotspot traffic conditions.

  • PPO-Full jointly optimizes compute node selection and RMCSA over candidate nodes and paths using a multi-dimensional action space.
  • Spectrum utilization is evaluated against arrival rates alongside the reported acceptance, blocking, cost, and fragmentation measures.
Loading 2608.29444v1…