Source-linked AI summary
An Overview on Application of Machine Learning Techniques in Optical Networks
Francesco Musumeci, Cristina Rottondi, Avishek Nag, Irene Macaluso, Darko Zibar, Marco Ruffini, Massimo Tornatore
TL;DR
Optical networks face difficult decisions because QoT depends on many tunable parameters and failures require rapid detection, localization, and recovery. The paper provides an ML tutorial and literature survey, reporting applications across optical communication and networking while identifying the field as fast-growing and still open to further research.
Problem
QoT optimization is difficult because readability-related parameters depend on many tunable design choices, while failure management must support rapid detection, localization, and recovery.
Method
The paper combines an introductory tutorial on ML methods with a survey and classification of optical communication and networking use cases.
Results
The survey finds ML applications spanning traffic prediction, resource allocation, failure diagnosis, and path computation, including a Bayesian classifier with 0.8% misclassified instances.
Takeaways & Limitations
ML for optical networking is a fast-growing area, with unified SDN-coordinated control and ML-based failure recovery identified as promising directions.
Takeaways & Limitations
Many supervised approaches assume static conditions, but optical-network scenarios evolve with traffic and component aging, and training may not transfer across topologies.
Abstract
from arXiv · showhide
Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from these data and take decisions pertaining to the proper functioning of the networks from the network-generated data. Among these mathematical tools, Machine Learning (ML) is regarded as one of the most promising methodological approaches to perform network-data analysis and enable automated network self-configuration and fault management. The adoption of ML techniques in the field of optical communication networks is motivated by the unprecedented growth of network complexity faced by optical networks in the last few years. Such complexity increase is due to the introduction of a huge number of adjustable and interdependent system parameters (e.g., routing configurations, modulation format, symbol rate, coding schemes, etc.) that are enabled by the usage of coherent transmission/reception technologies, advanced digital signal processing and compensation of nonlinear effects in optical fiber propagation. In this paper we provide an overview of the application of ML to optical communications and networking. We classify and survey relevant literature dealing with the topic, and we also provide an introductory tutorial on ML for researchers and practitioners interested in this field. Although a good number of research papers have recently appeared, the application of ML to optical networks is still in its infancy: to stimulate further work in this area, we conclude the paper proposing new possible research directions.
I. INTRODUCTION
Optical networks are becoming more complex and data-rich, creating demand for machine-learning methods that support analysis, configuration, and management. The paper introduces ML for optical communications and networking, surveys applications, and identifies future research directions.
- Contribution: The paper provides an introductory ML tutorial and surveys and classifies applications across optical communication and networking.
- Scope: The survey covers physical-layer and network-layer applications and highlights cross-layer interactions such as BER monitoring triggering routing or spectrum decisions.
- Motivation: Optical networks have gained adjustable, interdependent parameters through coherent technologies and flexible networking, increasing design and operation complexity.Examples include modulation formats, symbol rates, coding rates, channel spacing, and resource allocation.
- Motivation: Network monitors provide heterogeneous data, including traffic traces, signal-quality indicators, alarms, and user behavior, for ML-based analysis.
- ML overview: The paper organizes ML methods into supervised, unsupervised, reinforcement, and semi-supervised learning categories.
1) Parametric models:
Parametric models use a fixed number of learned parameters, with neural networks serving as a principal example. Neural networks transform inputs through layered nonlinear units and are trained by minimizing task-specific error functions.
- Parametric models: Parametric models use a fixed number of parameters estimated from training data, which can then be discarded during prediction.
- Neural networks: Neural networks apply successive nonlinear transformations, with weights and bias parameters learned during training.Biases are represented through dummy inputs with unit value.
- Neural networks: Neural networks are organized in layers, and one hidden layer can approximate continuous functions on compact domains when sufficiently many hidden units are used.
- Neural networks: Training minimizes an error function: sum of squared error is typical for regression, whereas cross-entropy is used for classification.
- Pre-processing: Input variables are typically rescaled to zero mean and unit standard deviation before training, while categorical data use 1-of-K coding.
- Nonparametric models: Nonparametric methods retain training data for prediction, whereas SVMs select a smaller subset of basis functions and support vectors.
B. Unsupervised learning
Unsupervised learning operates without output labels, most commonly through clustering. The section also introduces semi-supervised learning, self-training, and reinforcement learning as approaches that use unlabeled data or interaction feedback.
- Unsupervised learning: Unsupervised learning uses input vectors without labels, with clustering as its most common task.
- Clustering: Clustering groups data to maximize within-cluster similarity and minimize between-cluster similarity.
- Clustering: K-means assigns each point to one cluster, whereas Gaussian mixture models provide soft assignments and associated uncertainty.
- Semi-supervised learning: Semi-supervised learning combines a small labeled set with many unlabeled samples, which can improve performance when labels are scarce or expensive.
- Semi-supervised learning: Self-training iteratively labels selected unlabeled points using a supervised model and retrains with the expanded labeled set.
- Reinforcement learning: Reinforcement learning learns policies through state-action interaction and evaluative rewards, optimizing long-term performance rather than only immediate reward.
E. Overfitting, underfitting and model selection
Model selection must balance underfitting and overfitting while assessing generalization beyond training data. The paper presents held-out evaluation, cross-validation, and regularization as tools for selecting suitable models.
- Model selection: Overfitting occurs when a model is too complex for the dataset and fits noise, producing poor predictions on new data.
- Model selection: Training error alone poorly indicates generalization, so models are evaluated on separate training and test sets.Cross-validation is another option for limited datasets.
- Model selection: In overfitting, test error is high while training error is small; underfitting usually produces high errors on both sets.
- Regularization: Regularization adds a penalty to the training error, including weight decay or lasso, with coefficient λ controlling penalty strength.
- Optical-network context: Optical-network ML applications are motivated by increased system complexity, abundant monitoring data, and evolving traffic and management requirements.
A. Physical layer domain
Physical-layer ML applications address monitoring, transmission-quality estimation, amplifier control, modulation recognition, and nonlinear impairment mitigation. These applications use observed network data to support prediction, configuration, and protection decisions.
- Physical-layer applications: QoT estimation predicts whether candidate lightpaths will satisfy quality thresholds despite many interacting parameters and nonlinear propagation effects.Existing approaches include accurate but computationally heavy analytical models and approximate marginated formulas; ML classifiers learn direct relationships from configurations to observed QoT.
- Physical-layer applications: ML regression uses historical monitoring data to predict EDFA power excursions caused by adding or dropping wavelengths during dynamic lightpath provisioning.Dynamic reconfiguration can produce wavelength-dependent power excursions and signal distortion between lightpaths.
- Physical-layer applications: Modulation-format recognition addresses receiver uncertainty when flexible coherent transmitters select arbitrary modulation formats.Modern transmitters and receivers can flexibly vary bandwidth, carrier frequency, and modulation format to meet bit-rate and reach requirements.
- Physical-layer applications: Supervised ML models can learn input-output relationships that capture optical-fiber nonlinearities and their effects on monitored performance parameters.The targeted nonlinearities include Kerr effect, self-phase modulation, and cross-phase modulation, which can contribute to receiver signal distortion.
- Physical-layer applications: Optical performance monitoring estimates BER, Q-factor, chromatic dispersion, and polarization-mode dispersion during lightpath operation.These estimates can support polarization compensation, launch-power adjustment, modulation-format changes, and lightpath rerouting.
- Network-layer applications: Network-layer applications use ML for fast provisioning, traffic prediction, virtual-topology reconfiguration, failure management, and traffic-flow classification.These tasks support resource planning, rerouting, recovery, and service-specific policy decisions under changing network conditions.
C. A bird-eye view of the surveyed studies
The survey organizes optical-network ML studies by physical- and network-layer use cases, monitored data, learning paradigms, algorithms, and evaluation characteristics. QoT estimation is treated across prediction, monitoring, classification, database, spatial-inference, and regression approaches.
- Survey organization: The surveyed literature is divided into physical- and network-layer use cases and summarized in Tables I and II with approach characteristics and reference papers.The tables identify how supervised or unsupervised methods use data such as BER, OSNR, link load, and network alarms.
- QoT estimation: QoT estimation computes OSNR, BER, Q-factor, chromatic dispersion, or polarization-mode dispersion from field measurements or lightpath characteristics.It is applied both to unestablished lightpaths and to already deployed lightpaths for fault and malfunction identification.
- QoT estimation: Unestablished-lightpath QoT prediction is commonly formulated as binary classification against OSNR, BER, or Q-factor thresholds.Inputs can include lightpath length, traversed links, modulation format, and spectrum occupation.
- QoT estimation: A cognitive Case Based Reasoning approach stores measured lightpath data, retrieves the most similar case, and compares its Q-factor with a predefined threshold.The approach was reported to outperform Naive Bayes, J48 tree, and Random Forests, while database size trades off against computation time and classification effectiveness.
- QoT estimation: Other surveyed QoT approaches include controller-backed gradient-descent estimation, online neural networks, random-forest classification, network kriging, norm L2 minimization, and Gaussian-process regression.These methods respectively address parameter uncertainty, sequential model updates, candidate-route selection, spatially correlated measurements, and generalization across system configurations.
- QoT estimation: The survey also includes implementations such as an intelligent QoT-management architecture demonstrated on a real testbed.This connects algorithmic QoT estimation with control and management functions.
B. Optical amplifiers control
ML-based amplifier control addresses EDFA operating-point selection, interpolation of unmeasured characteristics, gain tuning, and power-excursion modeling. The surveyed methods combine characterization data, monitored states, and regression or database-based decision rules.
- Amplifier control: EDFA operating-point adjustment targets noise figure and gain flatness, which affect overall lightpath QoT.Existing studies first characterize amplifier metrics such as noise figure, gain flatness, and gain-control accuracy across the operating region.
- Amplifier control: ML interpolates EDFA gain-flatness and noise-figure mappings at points not measured during characterization.This addresses the time cost of producing fine-grained measurements when microcontrollers cannot directly obtain values for unmeasured operating points.
- Amplifier control: Interpolation with neural networks produced experimental errors below 0.5 dB for single and cascaded amplifiers.The neural networks use feed-forward and backward error propagation.
- Amplifier control: A cognitive methodology selects the amplifier-gain vector associated with the highest simulated OSNR when a new lightpath request arrives.The method stores measured gains of established lightpaths and estimates the OSNR produced by candidate gain vectors.
- Amplifier control: Real-time EDFA setpoint adjustment can be implemented through the GMPLS control plane using weighted-Euclidean-distance interpolation, including cascaded amplifiers.The implementation is described for dynamic amplifier control.
- Amplifier control: EDFA power-excursion modeling can be formulated as regression from binary WDM-channel-occupancy features to post-EDFA power discrepancy.Ridge regression and kernelized Bayesian regression were compared for configurations with two and three amplifier spans.
C. Modulation format recognition
ML-based modulation recognition uses clustering and neural-network methods to identify formats from received-signal representations. Across surveyed approaches, neural networks achieve zero reported error in one three-format experiment, while ML-based impairment mitigation improves BER/Q-factor by 0.5–3 dB.
- C. Modulation format recognition: Unsupervised clustering discriminates among BPSK, QPSK, 8-PSK, 8-QAM, and 16-QAM using True Positive Rate and running time across receiver OSNR.Silhouette coefficients can address the need to predetermine the number of clusters.
- C. Modulation format recognition: Neural networks trained on equalized eye-diagram amplitude-histogram features obtain 0% error when discriminating PM-QPSK, 16-QAM, and 64-QAM.A separate study reports an error-rate versus histogram-bin trade-off across six modulation formats.
- D. Nonlinearity mitigation: 0.5–3 dB improvement in BER/Q-factor is reported across the surveyed impairment-mitigation approaches.
- D. Nonlinearity mitigation: ML equalization approaches include Gaussian mixture models, clustering, artificial neural networks, k-nearest neighbors, and support vector machines.They are applied to decoding, predistortion, nonlinear equalization, and inter-subcarrier-intermixing mitigation.
- D. Nonlinearity mitigation: Received symbol samples can feed neural networks whose outputs are equalized signals with reduced inter-symbol interference.CNN outputs vary with PAM-4, PAM-8, or PAM-16 signal classes.
- D. Nonlinearity mitigation: Neural networks can efficiently approximate digital backpropagation and emulate receivers for nonlinear frequency-division multiplexing systems.The cited NFDM receiver outperforms nonlinear-Fourier-transform and minimum-distance receivers.
- D. Nonlinearity mitigation: A neural-network impairment-mitigation approach is also applied to radio-over-fiber signals from multiple users.
E. Optical performance monitoring
The surveyed optical-network applications combine monitoring and prediction with ML-based control decisions. Neural networks support optical-parameter estimation, while time-series models and deeper predictors support traffic forecasting and resource reconfiguration.
- E. Optical performance monitoring: Neural networks learn mappings from signal samples or extracted features to optical parameters including OSNR, PMD, PDL, baud rate, and CD.Features can be extracted from power eye diagrams and other signal representations.
- E. Optical performance monitoring: Gaussian process models outperform linear-regression prediction models for optical performance monitoring and can be robust under noise uncertainties.The cited work also describes them as easy to integrate into a network controller.
- A. Traffic prediction and virtual topology design: The NPDM module predicts traffic and interacts with other modules to perform virtual-topology reconfiguration.
- A. Traffic prediction and virtual topology design: ARIMA models use recent real-time traffic matrices as time-series input for traffic prediction and virtual-topology reconfiguration.
- A. Traffic prediction and virtual topology design: Neural-network predictors are also used for source-destination traffic matrices in virtual-topology reconfiguration.
- A. Traffic prediction and virtual topology design: Deep learning for intra-data-center traffic prediction and resource allocation outperforms conventional allocation algorithms and a single-layer neural network in blocking performance and resource occupation efficiency.An RNN is used for time-dependent BBU-pool traffic prediction in a 5G cloud radio access network.
- A. Traffic prediction and virtual topology design: Other surveyed frameworks use Bayesian inference, matrix factorization, clustering, or multi-objective genetic algorithms for traffic estimation and topology design.
B. Failure management
ML-based failure management spans localization, detection, diagnosis, and classification using network measurements such as BER, received power, and equipment telemetry. Surveyed methods include network kriging, Bayesian networks, regression, anomaly detection, and neural models.
- B. Failure management: Network kriging can localize failures along network links from failure counts observed at monitoring nodes on established lightpaths.Additional lightpath probing may provide information when localization is ambiguous, while monitoring-node requirements depend on network load.
- B. Failure management: 0.8% of tested instances were misclassified by a Bayesian classifier using BER and received-power time series to detect failures and identify causes.
- B. Failure management: Bayesian-network models are also applied to layered GPON/FTTH failure propagation and diagnosis.
- B. Failure management: A BER anomaly detector uses historical and monitored BER values to identify abrupt changes that may indicate failures.
- B. Failure management: LUCIDA combines historic BER, received power, and derived time-series features to map feature probabilities to failure-class probabilities.Its output is the most likely failure cause among candidate classes.
- B. Failure management: TISSUE compares estimated and measured BER across a lightpath, while FEELING transforms frequency-power signal pairs into features for failure-cause localization.
- B. Failure management: Support vector machines classify signals as Normal, FilterShift, or TightFiltering in the SSC module.
- B. Failure management: Failure-management studies also include optimized combinations of binary and multiclass SVMs, random forests, neural networks, regression, and anomaly detection.Another approach uses optical power, amplifier gain, temperature, and current measurements in SDN controllers to predict failures.
D. Path computation
Path computation is treated as a cross-layer ML problem that combines physical and network parameters to select routes and allocate resources. The survey also reviews learning-based control, evaluation metrics, and networking-specific deployment measures.
- D. Path computation: ML-based path computation uses parameters such as QoT, modulation format, and OSNR to select among alternative optical paths.
- D. Path computation: Q-learning formulates path and wavelength selection in optical burst-switched networks as a multi-arm bandit problem aimed at minimizing burst-loss probability.
- D. Path computation: Fuzzy C-Means clustering supports QoS-aware path computation in an SDON control plane using traffic, path length, modulation, OSNR, and BER inputs.Membership scores generate rules for real-time lightpath setup decisions.
- D. Path computation: A repository of ML algorithms and policies can act as a control-plane ‘brain’ that maps network-generated stimuli to actions such as path computation and topology reconfiguration.
- A. Performance metrics: The survey quantitatively compares selected ML applications using standard ML metrics and reports each paper’s main comparison outcome.
- A. Performance metrics: Classification evaluation commonly combines accuracy with algorithm complexity, often represented by training-phase duration.
- A. Performance metrics: Confusion matrices, TPR, FPR, ROC curves, AUC, and AIC provide complementary views of classifier or model performance.
- A. Performance metrics: Optical-network metrics include the number of monitors and BER sampling frequency needed to meet classification or failure-localization accuracy targets.
B. Quantitative algorithms comparison
The paper compares ML algorithms across selected optical-networking use cases using studies that applied multiple algorithms to the same dataset. The comparison reports the algorithms, evaluation metrics, and synthetic outcomes for each use case.
- Comparison scope: The comparison selects surveyed studies that evaluate different ML algorithms on the same dataset.The paper notes that quantitative comparisons across different papers are difficult because algorithms are designed for each paper’s available data.
- Comparison scope: Table III organizes the considered use cases by reference, ML algorithms, evaluation metrics, and synthetic study outcomes.
VII. DISCUSSION AND FUTURE DIRECTIONS
The discussion identifies evolving ML methodologies, limited and costly data, timescale and control requirements, and opportunities for integrated optical-network intelligence. It also highlights future directions including commercialization, failure recovery, visualization, and all-optical learning.
- ML methodologies: Most existing optical-network ML studies use offline supervised learning trained on historical data before field decisions.The paper proposes investigating semi-supervised, unsupervised, retraining, and active-learning approaches for changing network conditions.
- ML methodologies: Historical training assumptions can become unrealistic as traffic varies, components age, topologies change, and previously valid models become outdated.
- Data availability: Large field datasets remain undisclosed, while testbeds cannot reproduce the diversity of real-network scenarios.
- Data availability: Fault-condition datasets are difficult to collect because conservative network design makes failures negligible while under-utilizing resources.
- Timescales: Monitoring timescales affect both ML accuracy and the control bandwidth required to implement an ML-based system.A system requiring extremely frequent sampling may be impractical despite accurate predictions.
- A complete cognitive control system: Future unified control frameworks could coordinate ML applications across layers when routing lightpaths or modifying transmission parameters.The paper identifies QoT, failures, and traffic patterns as variables that may be jointly considered, although some are mutually dependent and others are not.
- Failure recovery: ML-based failure recovery could combine cross-layer data with probabilistic outputs that provide richer information than deterministic threshold-based tools.
- Visualization: Effective visualization remains necessary to make ML outputs accessible to optical-network management users, but design guidelines for different applications are still needed.
VIII. CONCLUSION
Optical networks are becoming more intelligent through advances in software-defined networking, coherent transmission, and flexible-grid technologies. The paper summarizes existing ML research and identifies potential directions for enabling smarter optical networks.
- VIII. CONCLUSION: Software-defined networking, coherent transmission, and flexible-grid technologies are contributing to the development of smarter optical networks.
- VIII. CONCLUSION: ML algorithms can use data from network-monitoring elements to learn from experience and make networks more agile and adaptive.
- VIII. CONCLUSION: The paper summarizes literature on ML for smart optical networks and provides insight into potential future research directions.
GLOSSARY
The glossary expands abbreviations used throughout the paper, covering optical networking, machine learning, monitoring, performance measures, transmission technologies, and control concepts.
- Machine learning: ANN means Artificial Neural Network, CNN means Convolutional Neural Network, MLP means Multi-layer perceptron, RNN means Recurrent Neural Network, and SVM means Support Vector Machine.
- Metrics: BER means Bit Error Rate, AUC means Area Under the ROC Curve, ROC means Receiver operating characteristic, and MTTR means Mean Time To Repair.
- Modulation: QAM, QPSK, and BPSK denote Quadrature Amplitude Modulation, Quadrature Phase Shift Keying, and Binary Phase Shift Keying, respectively.
- Machine learning: RL means Reinforcement Learning, and MDP means Markov decision processes.
- Transmission technologies: WDM means Wavelength Division Multiplexing, NWDM means Nyquist Wavelength Division Multiplexing, OFDM means Orthogonal Frequency Division Multiplexing, and NFDM means Nonlinear Frequency Division Multiplexing.
- Optical monitoring and performance: OSNR means Optical Signal-to-Noise Ratio, SNR means Signal-to-Noise Ratio, QoT means Quality of Transmission, and OPM means Optical Performance Monitoring.
- Networking and control: SDN means Software-defined Networking, SDON means Software-defined Optical Network, and SLA means Service Level Agreement.
- Optical transmission: SSFM means Split-Step Fourier Method, NFT means Nonlinear Fourier Transform, and NLI means Nonlinear Interference.