Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges

Motivation

• Growing network complexity makes manual administration less amenable to managing and optimizing modern networks.Internet and mobile telecom networks are described as increasingly dynamic, heterogeneous, and complex.Source: §I · p. 1
• Supervised ML methods are limited by their need for labeled data, which is expensive to curate and can become outdated quickly.Networking data is increasingly voluminous, with a disproportionate rise in unstructured unlabeled data.Source: §I · p. 1

Networking applications

• Unsupervised ML is presented as a complement to supervised ML for more efficient network management, monitoring, and self-optimization with administrators kept in the loop.Routing is one example where models can account for real-time congestion and application QoS requirements.Source: §I · p. 2

Unsupervised learning

• Unsupervised ML can analyze raw unlabeled network data for feature learning, clustering, dimensionality reduction, anomaly detection, and related tasks.The approach is motivated by advances in hierarchical learning, clustering, factor analysis, latent models, and outlier detection.Source: §I · p. 2, A. Hierarchical Learning · p. 3

Scope and organization

• The paper surveys hierarchical learning, clustering, latent variable models, outlier detection, and reinforcement learning, then reviews their networking applications.The survey also discusses future work and opportunities and identifies major pitfalls.Source: §I · pp. 2–3, §II · p. 3

Unsupervised Multilayer Feed Forward NN

• Unsupervised multilayer feed-forward neural networks use directed acyclic topologies to approximate nonlinear mappings between inputs and required outputs.Autoencoders are identified as prime examples of this network family.Source: 1) Unsupervised Multilayer Feed… · p. 4
• Autoencoders learn compressed, encoded representations by reconstructing input data, supporting dimensionality reduction and unsupervised pre-training.Variants include variational, sparse, and denoising autoencoders for clustering, visualization, feature extraction, manifold learning, and reconstruction.Source: 1) Unsupervised Multilayer Feed… · pp. 4–5

Unsupervised Competitive Learning NN

• Competitive-learning neural networks use winner-take-all competition to remove redundancies from unstructured data.Self-organizing maps and adaptive resonance theory networks are the two major techniques described.Source: 2) Unsupervised Competitive Learning NN · p. 5
• Self-organizing maps represent high-dimensional data in lower-dimensional topological spaces while preserving input-data topology.Their training proceeds through competition, cooperation among neighboring neurons, and adaptation.Source: 2) Unsupervised Competitive Learning NN · p. 6
• Adaptive resonance theory performs incremental clustering by automatically discriminating and creating new multidimensional categories as new information arrives.Its operation compares top-down expectations with bottom-up sensory information using a vigilance threshold and reset mechanism.Source: 2) Unsupervised Competitive Learning NN · pp. 6–7

Architectures

• Unsupervised deep neural networks include deep belief networks, deep autoencoders, and convolutional neural networks.Deep belief networks use hierarchical stochastic latent variables; autoencoders encode and reconstruct inputs; convolutional networks extract transformation-invariant features.Source: 3) Unsupervised Deep NN · pp. 7–8
• Deep autoencoders compress inputs into a central feature vector by encoding and then reconstructing the input.The encoder and decoder are formed from two symmetric deep belief networks.Source: 3) Unsupervised Deep NN · pp. 7–8

Networking applications

• Convolutional architectures can support QoE and QoS optimization for multimedia delivery, while scalable generative models learn hierarchical representations from high-dimensional unlabeled data.The cited CDBN combines properties of deep belief and convolutional networks and is presented as a promising scalable model.Source: 3) Unsupervised Deep NN · p. 8

Recurrent and recurrent-derived models

• Recurrent neural networks model sequential inputs using internal memory, enabling them to learn temporal behaviors.Unlike feed-forward networks, recurrent networks are stateful and incorporate prior events into subsequent processing.Source: 4) Unsupervised Recurrent NN · p. 8
• Unsupervised recurrent neural networks lack a single widely adopted training approach, motivating diverse training methods including LSTM-based approaches.One cited approach trains LSTM recurrent networks using Binary Information Gain Optimization and Non-Parametric Entropy.Source: 4) Unsupervised Recurrent NN · p. 8

Networking applications

• Hierarchical neural models are applied to traffic classification, anomaly and intrusion detection, DDoS detection, and cognitive-radio resource management.A hybrid ART–RNN model learns and predicts network traffic volume in real time, supporting adaptive flow control without retraining the entire network.Source: 4) Unsupervised Recurrent NN · p. 9

Clustering concepts

• Clustering organizes unlabeled data into natural groups with high intra-cluster and low intercluster similarity.The resulting structured representation is termed a data-concept.Source: B. Data Clustering · p. 9

Networking applications

• Clustering is widely used in networking for traffic analysis and anomaly detection across wireless sensor, mobile ad hoc, and other networks.These applications exploit clustering to identify structure in network data.Source: B. Data Clustering · p. 9
• Expectation-Maximization probabilistic clustering groups network flows into a small number of representative clusters for traffic analysis.The packet-tracing approach uses representative clusters to analyze network traffic efficiently.Source: B. Data Clustering · p. 9

Clustering taxonomy

• The taxonomy distinguishes hierarchical, Bayesian, and partitional clustering, with Bayesian methods modeling test-point assignments probabilistically.Hierarchical methods decompose data, whereas partitional methods construct multiple partitions evaluated by a criterion.Source: B. Data Clustering · p. 9
• Density-based clustering separates densely populated regions from low-density areas, and is treated separately because of its networking applications.Grid-based clustering is also identified as a distinct technique before the main taxonomy discussion.Source: B. Data Clustering · p. 9

Hierarchical clustering

• Hierarchical clustering avoids specifying the number of clusters beforehand but commonly has at least quadratic complexity and can mishandle messy high-dimensional data.Common agglomerative hierarchical algorithms are described as NP-hard, whereas K-means and EM have linear complexity.Source: 1) Hierarchical Clustering · p. 10

Bayesian clustering

• Bayesian clustering learns posterior data distributions from prior probabilities and is divided into parametric and non-parametric strategies.The distinction concerns whether the parameter space has finite or otherwise unrestricted dimensionality.Source: 2) Bayesian Clustering · p. 10

Partitional clustering

• Partitional clustering partitions n observations into k < n disjoint clusters and includes K-means clustering and mixture models.Each observation is assigned within a decomposition into separate groups.Source: 3) Partitional Clustering · p. 10

K-means clustering

• K-means assigns observations to the nearest cluster using cluster means as reference points.It takes statistical vectors as input to derive classification models or classifiers.Source: 3) Partitional Clustering · p. 10
• Detection rate stays above 96% while false alarm rate stays below 4% for a K-means intrusion-detection algorithm on a KDD-99 subset.The experiments also reported better ability to search clusters globally.Source: 3) Partitional Clustering · p. 10

K-medoids clustering

• K-medoids uses the most centrally located data point rather than the cluster mean as its reference point.The method has applications in anomaly detection.Source: 3) Partitional Clustering · p. 10

Mixture models

• Mixture models infer sub-population properties from pooled observations and naturally model data generated by alternative random sources.Finite mixtures infer source parameters from observations, producing clusters without relying solely on heuristic clustering choices.Source: 3) Partitional Clustering · pp. 10–11
• Mixture-model clustering addresses drawbacks of heuristic methods and can efficiently classify nodes in large-scale networks.K-means and hierarchical agglomerative methods rely on design decisions such as the number of clusters or model validity.Source: 3) Partitional Clustering · p. 11
• Combining EM with mixture models helps decipher the structure and topology of vertices connected through multidimensional networks.A Gaussian mixture model is reported to outperform signature-based anomaly detection in network traffic data.Source: 3) Partitional Clustering · p. 11

Blind Signal Separation

• Blind Signal Separation identifies and separates independent source signals from mixed inputs without or with very little mixing-process information.The process extracts source signals from a mixture and is used across multidimensional data processing.Source: 3) Blind Signal Separation · p. 11, Figure 7 · p. 12

PCA

• PCA orthogonally transforms n correlated variables into k uncorrelated principal components ordered by descending variability.The first component captures the greatest variability and the last captures the least.Source: 3) Blind Signal Separation · p. 11
• PCA reduces n-dimensional input data to k dimensions while retaining critical information, with one cited design retaining 99% of variance.The number of components is a design decision and may be selected using methods such as cross-validation approximations.Source: 3) Blind Signal Separation · pp. 11–12

ICA

• ICA separates multivariate inputs into additive components under the assumption that components are non-Gaussian and statistically independent.Unlike PCA’s variance-based approach, ICA exploits non-Gaussian features and maximizes the fourth moment of linear combinations.Source: 3) Blind Signal Separation · p. 12

NMF

• NMF factorizes a large nonnegative matrix into smaller nonnegative matrices whose product approximately reconstructs the original.This decomposition exposes hidden patterns and intrinsic features and supports exploratory analysis.Source: 4) Non-Negative Matrix Factorization · p. 12

Temporal and probabilistic models

• HMMs model temporal or dynamic processes when actual states are not directly observable, including primary-user arrivals in cognitive-radio networks.They represent Markovian stochastic processes without direct access to the underlying states.Source: 5) Hidden Markov Model · p. 13
• Bayesian learning estimates posterior distributions over parameter settings, while probabilistic graph models represent random variables and their dependencies through nodes and edges.These methods are applied to traffic classification, host-level intrusion detection, and anomaly-related feature selection.Source: 6) Bayesian Networks & Probabilistic… · p. 13, 7) Significant Applications of Latent… · p. 13

Networking applications

• Networking applications of PCA and NMF include Internet traffic classification, intrusion detection, anomaly detection, and wireless-sensor-network clustering.PCA is also used for probabilistic anomaly confidence, while NMF factorizes flow features before clustering anomalies and intrusions.Source: 7) Significant Applications of Latent… · p. 13

• Dimensionality reduction seeks essential patterns by extracting intrinsic dimensions from datasets with potentially thousands or millions of correlated dimensions.It creates representative functional dimensions rather than selecting a subset of original features.Source: D. Dimensionality Reduction · pp. 13–14
• Dimensionality reduction supports data modeling, compression, visualization, and learning by eliminating redundant dimensions and simplifying data representation.Source: D. Dimensionality Reduction · p. 14
• Linear methods such as PCA and ICA may miss nonlinear structure because they assume high-dimensional points lie on a linear subspace.Source: D. Dimensionality Reduction · p. 14
• Isomap combines PCA and MDS features to learn low-dimensional nonlinear manifold structure using geodesic distances along shortest paths.Source: 1) Isomap · p. 14
• Other nonlinear techniques include GTM, LLE, and principal curves for probabilistic mapping, manifold visualization, feature extraction, and dataset summarization.GTM models latent points through reference vectors, LLE preserves higher-dimensional embeddings, and principal curves summarize multidimensional data with smooth curves.Source: 2) Generative Topographic Model · pp. 14–15, 3) Locally Linear Embedding · p. 15, 4) Principal Curves · p. 15

5) Nonlinear Multi-dimensional Scaling:

• NMDS finds low-dimensional configurations that preserve similarities and dissimilarities without imposing factor analysis’s multivariate-normal assumption.Source: 5) Nonlinear Multi-dimensional Scaling · p. 15
• t-SNE represents high-dimensional data in 2 or 3 dimensions by minimizing the KL divergence between high- and low-dimensional probability distributions.Source: 6) t-Distributed Stochastic Neighbor… · p. 15

• Unsupervised outlier detection commonly uses nearest-neighbor or clustering-based techniques to identify samples distant from surrounding data.Source: E. Outlier Detection · p. 15, 1) Nearest Neighbor Based Outlier… · p. 15, 2) Cluster Based Outlier Detection · p. 15
• Outlier detection estimates distances and densities in non-annotated data and supports intrusion detection, fraud detection, leakage prevention, surveillance, and anomaly analysis.Source: 3) Significant Applications of Outlier… · p. 15
• Reinforcement learning learns actions for dynamic operating environments by having an agent observe states and rewards and progressively improve decisions.Source: F. Reinforcement Learning · pp. 15–16
• Deep reinforcement learning combines reinforcement learning with deep neural networks and can operate where handcrafted features are unavailable.Source: F. Reinforcement Learning · p. 16

1) Multi-agent Reinforcement Learning:

• MARL enables agents to exchange information such as Q-values and rewards, learning joint actions that maximize global rather than local performance.Source: 1) Multi-agent Reinforcement Learning · p. 16

• Tabular reinforcement learning becomes storage-intensive because the number of state-action pairs grows exponentially as states and actions increase.Source: 2) Reinforcement Learning with… · pp. 16–17
• Function approximation reduces this burden by representing state-action Q-values with features and a weight vector instead of a full table.Source: 2) Reinforcement Learning with… · p. 17
• Increasing the learning rate can accelerate convergence but may cause Q-value fluctuations when current rewards change substantially in dynamic environments.Source: 3) Model-based Reinforcement Learning · p. 17
• Q-learning is a model-free approach that maps environmental states to actions and incrementally improves Q-value estimates without constructing a stochastic environment model.Source: 4) Q-learning · p. 17
• Networking applications include multi-agent target tracking, channel-state representation, malicious-node detection, and fuzzy Q-learning for QoS-oriented cell association.Source: 5) Significant Applications of RL in… · p. 17
• The review identifies reinforcement learning for network management and optimization as a potential research area.Source: G. Lessons Learnt · p. 17

A. Internet Traffic Classification

• Internet traffic classification helps providers understand quality of service, user behavior, network security, and broader network characteristics.Source: A. Internet Traffic Classification · p. 17
• Unknown intrusions are increasingly difficult to address because evolving malware can use dynamic port negotiation, making traditional port-based classification obsolete.Source: A. Internet Traffic Classification · pp. 17–18
• Traffic classification recognizes packet destinations and enables flow control actions such as prioritizing one flow over another.Source: A. Internet Traffic Classification · p. 18
• Feature selection is a complex first step affected by large datasets and imbalanced traffic classes; proposed methods include ensemble and information-theoretic schemes.Source: A. Internet Traffic Classification · p. 18
• An unsupervised autoencoder learned latent feature representations that improved state-of-the-art results for malware classification and anomaly detection.Source: A. Internet Traffic Classification · p. 18
• Unsupervised traffic-classification approaches include EM clustering, first-five-packet analysis, correlation-based methods, K-means flow clustering, and neural generative models.Source: A. Internet Traffic Classification · pp. 18–19
• A deep belief network combined with a Gaussian model outperformed the previous maximum-likelihood estimation technique for wireless mesh backbone traffic prediction.Source: A. Internet Traffic Classification · p. 19

B. Anomaly/Intrusion Detection

• Signature-based detection identifies known attacks from stored signatures but cannot feasibly maintain signatures for all evolving attacks.Source: B. Anomaly/Intrusion Detection · p. 19

• Network management covers initializing, monitoring, and managing network functions, while optimization studies apply unsupervised learning to operational tasks.Source: C. Network Operations, Optimizations… · p. 21, Table IX · p. 22

QoS/QoE Optimization

• QoS measures quantitative service performance, whereas QoE captures qualitative end-user experience, making both important for Internet service optimization.Source: 1) QoS/ QoE Optimization · p. 21
• Unsupervised feature learning can be combined with supervised classification to adapt Internet services to changing real-time network conditions.Source: 1) QoS/ QoE Optimization · p. 22
• 43% of bit overhead on average can be reduced per web-delivered image using VoQS and unsupervised clustering of large image datasets.Source: 1) QoS/ QoE Optimization · p. 22

• Model-based and multi-agent reinforcement learning improve throughput by modeling environments, generating virtual rewards, and collaboratively selecting next-hop nodes.Source: 1) QoS/ QoE Optimization · pp. 22–23

TCP Optimization

• TCP congestion-control algorithms can be sub-optimal in hybrid wired/wireless networks, motivating optimization research.Source: 2) TCP Optimization · p. 23

Dimensionality Reduction & Visualization

• Dimensionality reduction reduces network variables to support anomaly detection, intrusion detection, reliability analysis, fault prediction, and visualization.Source: D. Dimensionality Reduction &… · p. 23
• Unsupervised autoencoders, PCA, multi-scale PCA, wavelet analysis, and robust PCA have been applied to extract features and separate anomalous traffic.Source: D. Dimensionality Reduction &… · p. 23
• Autoencoders and deep belief networks support cellular-network fault prediction and reliability analysis for self-organizing networks.Source: D. Dimensionality Reduction &… · p. 23
• PCA and t-SNE extract structure from high-dimensional networking data for lower-dimensional visualization and human decision-making.Source: D. Dimensionality Reduction &… · p. 23
• A deep-learning routing technique outperforms conventional OSPF in throughput and average delay per hop.Source: D. Dimensionality Reduction &… · p. 24

Software Defined Networks

• SDN simplifies network operation by making networks programmable and decoupling data-forwarding and control or decision planes.Source: 1) Software Defined Networks · pp. 24–25
• Unsupervised learning in SDN includes intrusion detection, security-attack mitigation, and prediction of unknown anomalies in cloud environments.Source: 1) Software Defined Networks · p. 25

Self Organizing Networks

• SONs learn from their surroundings to support self-configuration, self-healing, and self-optimization as connected devices and femtocells expand.Source: 2) Self Organizing Networks · p. 25

Internet of Things

• IoT requires machine learning to analyze data and make intelligent decisions at the extremely large scale of billions of future devices.Source: 3) Internet of Things · p. 25

Lessons Learnt

• The review recommends clustering with autoencoder-based latent representations for unsupervised Internet traffic classification and identifies generative and transfer learning as underexplored.Source: F. Lessons Learnt · pp. 25–26

Simplified Network Management

• The semantic gap between machine-learning results and actionable operator insights must be overcome to shift network management from reactive monitoring toward proactive reports and notifications.Source: A. Simplified Network Management · p. 26

Semi-Supervised Learning

• Semi-supervised learning combines unlabeled data with a small set of labeled examples and is used partly for anomaly detection and traffic classification.It is also identified as having potential with deep unsupervised architectures such as generative adversarial networks.Source: B. Semi-Supervised Learning for… · p. 26

Transfer Learning

• Transfer learning applies knowledge learned from one problem to a different but related problem and has potential in networking despite limited observed implementation.The passage gives wireless caching for backhaul offloading in 5G networks as an example.Source: C. Transfer Learning in Computer… · p. 26

Federated Learning

• Federated learning distributes processing across machines without centralized training data, with stated benefits for user-data privacy and reduced cloud computation.The section also cites applications involving network address management, federated healthcare, and federated cloud and IoT security.Source: D. Federated Learning in Computer… · pp. 26–27

Pitfalls and Caveats

• The section emphasizes that unsupervised learning has shortcomings, including technique-selection risks, feature or parameter-selection problems, and interpretability barriers.Deep neural networks may operate as black boxes, creating a trade-off between prediction accuracy and simplicity.Source: §V · p. 27, A. Inappropriate Technique Selection · p. 27, B. Lack of Interpretability of Some… · p. 27

Lack of Operational Success of ML in Networking

• Operational networks have relatively few practical ML solutions, particularly for intrusion detection, despite substantial academic research and applications in other fields.The cited challenges include high error costs, limited training data, semantic gaps, variable inputs, and difficult evaluations.Source: C. Lack of Operational Success of ML… · p. 27
• Researchers should verify that ML offers value beyond simpler non-ML solutions, because ignoring operational realities can undermine effectiveness.The paper frames ML solutions as augmenting and supplementing, rather than automatically replacing, existing approaches.Source: D. Ignoring Simple… · p. 27

Overfitting

• Overfitting occurs when an unsupervised model represents noise or random error instead of the actual data pattern.Performance is expected to improve with more data but deteriorate after model complexity becomes excessive; unoptimized out-of-the-box algorithms can also worsen with increasing data.Source: E. Overfitting · pp. 27–28, Figure 8 · p. 28

Data Dependence and Quality

• ML performance depends strongly on data nature, volume, quality, and representation, while class imbalance can seriously affect results.A dataset is imbalanced when one class has substantially fewer or more samples than other classes.Source: F. Data Quality Issues · p. 28

Model and Evaluation Challenges

• Accurate generic unsupervised models are difficult to build because algorithm suitability is application-specific and clustering often lacks known cluster counts or ground truth.Hierarchical clustering may also have an unclear stopping point, increasing process time complexity.Source: G. Inaccurate Model Building · p. 28

Adversarial Settings and Safeguards

• Adversarial anomaly-detection settings can defeat models trained on historical data when attackers deliberately craft inputs to circumvent previously learned schemes.The paper recommends context-appropriate safeguards including pipeline testing, visualization, regularization, dropout, and human sanity checks.Source: H. Machine Learning in Adversarial… · p. 28

Conclusions

• The survey covers major unsupervised networking applications and techniques, while acknowledging that evolving networks make exhaustive coverage impossible.It also discusses future work, open research areas, pitfalls, and challenges.Source: §VI · p. 28