Source-linked AI summary
Machine Learning for Networking: Workflow, Advances and Opportunities
Mowei Wang, Yong Cui, Xin Wang, Shihan Xiao, Junchen Jiang
TL;DR
Network diversity and complexity make efficient algorithms and systems for different network scenarios challenging. This article summarizes an MLN workflow and surveys representative advances, including DeepRM’s comparable performance at lower cost and Pytheas’s outperformance of state-of-the-art prediction-based systems.
Problem
Network diversity and complexity make developing efficient algorithms and systems for different network scenarios challenging.
Source: Introduction · p. 1Method
The article summarizes typical MLN workflows and surveys representative advances grouped by networking field, explaining how prior efforts use each workflow step.
Source: Introduction · p. 2Results
DeepRM performs comparably to state-of-the-art heuristics at lower cost, while Pytheas outperforms state-of-the-art prediction-based systems.
Source: Overview of Recent Advances · p. 5Takeaways & Limitations
The workflow and survey are intended as a practical starting guideline for researchers exploring machine learning in networking.
Source: Introduction · p. 2Robustness and generalization in dynamic real-world networking environments remain challenges for machine learning algorithms.
Source: Opportunities for MLN · p. 7
Abstract
from arXiv · showhide
Recently, machine learning has been used in every possible field to leverage its amazing power. For a long time, the net-working and distributed computing system is the key infrastructure to provide efficient computational resource for machine learning. Networking itself can also benefit from this promising technology. This article focuses on the application of Machine Learning techniques for Networking (MLN), which can not only help solve the intractable old network questions but also stimulate new network applications. In this article, we summarize the basic workflow to explain how to apply the machine learning technology in the networking domain. Then we provide a selective survey of the latest representative advances with explanations on their design principles and benefits. These advances are divided into several network design objectives and the detailed information of how they perform in each step of MLN workflow is presented. Finally, we shed light on the new opportunities on networking design and community building of this new inter-discipline. Our goal is to provide a broad research guideline on networking with machine learning to help and motivate researchers to develop innovative algorithms, standards and frameworks.
Introduction
The introduction presents machine learning for networking (MLN) as a way to address complex, diverse network problems and enable new applications. It outlines the article’s practical MLN workflow, selective survey of advances, and discussion of future opportunities.
Introduction
- Recent computing infrastructure, ML libraries, and distributed data-processing frameworks create opportunities to extend ML beyond traditional network uses such as prediction and classification [s1.p1#ec3541].The passage names GPUs, TPUs, TensorFlow, Scikit-Learn, Hadoop, and Spark as examples.Source: Introduction · p. 1
- Network diversity and changing application requirements make it challenging to develop efficient algorithms and systems for each scenario [s1.p2#b3c1d9].The passage gives wired and wireless networks, and security and live-streaming applications, as examples.Source: Introduction · p. 1
- MLN applies classification and prediction to intrusion detection and performance prediction, and decision-making to network scheduling and parameter adaptation [s1.p3#8e6d45].Learning can make these decisions based on current environmental states.Source: Introduction · p. 1
- ML can estimate complex network-system behavior when accurate analytic models are difficult to build, including CDN load patterns and throughput characteristics [s1.p3#8e6d45].A uniform training method may also support generalized models across network scenarios [s1.p9#3f7c23].Source: Introduction · pp. 1–2
- The article summarizes MLN workflows and requirements, surveys recent networking advances by workflow step, and discusses opportunities for future research [s1.p13#117c47].It aims to provide a practical starting guideline for researchers in MLN.Source: Introduction · p. 2
- MLN data collection should gather representative, unbiased network data offline for analysis and training and online for real-time inputs or feedback [s1.p14#5b8cae].New online data can also update the historical data pool for model adaptation.Source: Introduction · p. 2
Basic Workflow for MLN
The MLN workflow proceeds from problem formulation and data collection through analysis, model construction, validation, deployment, and inference; its stages are interrelated and may need to be revisited as errors are diagnosed.
Basic Workflow for MLN
- Data analysis cleans raw samples and extracts effective features, a domain-specific task that deep learning can sometimes automate.Preprocessing may include normalization, discretization, and missing-value completion.Source: Basic Workflow for MLN · p. 2
- The baseline MLN workflow links problem formulation, data collection, data analysis, model construction, validation, deployment, and inference rather than treating them as independent stages.Error analysis may require repeating earlier procedures.Source: Basic Workflow for MLN · pp. 2–3
- Problem formulation classifies the target as a learning task, such as classification, clustering, or decision making, guiding data collection and model choice.An unsuitable abstraction can lead to an inappropriate model and unsatisfactory performance.Source: Basic Workflow for MLN · p. 2
- Model construction selects, trains, and tunes a model according to the dataset, network scenario, and problem; hyperparameter tuning can be costly and lacks sufficient theoretical guidance.The tuning process may require searching a large parameter space or relying on experience.Source: Basic Workflow for MLN · pp. 2–3
- Offline validation uses cross-validation to assess model accuracy and detect overfitting or underfitting, while error analysis can guide model, feature, or data revisions.Overfitting may motivate increasing data volume or reducing model complexity.Source: Basic Workflow for MLN · p. 3
- Deployment must balance accuracy against operational computation, energy, and response-time limits, account for best-effort models without performance guarantees, and support real-time online inference.System designers should also consider fault tolerance.Source: Basic Workflow for MLN · pp. 3–4
Overview of Recent Advances
The overview organizes recent networking advances by their roles in the MLN workflow, showing how learning can extract fine-grained strategies from network data while facing practical feasibility challenges.
Overview of Recent Advances
- MLN methods can offer more fine-grained strategies and higher prediction accuracy than conventional analytics, but feasibility remains a challenge, including scarce labels, dynamic systems, and costly learning errors.The overview contrasts ML methods with time-series analytics, statistical methods, and rule-based heuristics, which are often more interpretable and easier to implement.Source: Overview of Recent Advances · pp. 4, 6
Traffic Prediction
- Traffic volume prediction supports congestion control, resource allocation, routing, and streaming, but direct measurement is costly; studies use domain knowledge or automatically extracted header features.The overview distinguishes time-series analysis using direct observations from network tomography, which predicts without them.Source: Overview of Recent Advances · p. 4
Information Cognition
- Limited measurement access makes network-state cognition difficult; Sibyl predicts unseen routes and assigns confidence, while the Knowledge Plane enables automatic information cognition.Sibyl uses supervised RuleFit and addresses limited vantage points and probing budgets.Source: Overview of Recent Advances · p. 4
Traffic Classification
- Statistical-feature learning addresses port-based classification’s unreliable assignments and payload inspection’s privacy and encryption limits, while unknown traffic and costly security misclassification complicate deployment.The roadmap extends from known-class classification toward unknown traffic, including zero-day applications.Source: Overview of Recent Advances · p. 4
Resource Management and Network Adaption
- For complex routing and congestion-control decisions, deep learning maps router traffic to next hops, while Remy generates TCP rules from prior assumptions and PCC learns online.Remy learns a network-state-to-cwnd mapping offline and applies it when ACKs arrive; these efforts are early steps toward automated protocol design.Source: Overview of Recent Advances · p. 5
Network Performance Prediction and Configuration Extrapolation
- Prediction supports QoE and configuration choices: CS2P clusters similar sessions, CFA uses domain knowledge, and cherrypick reduces costly pre-run tests; Pytheas instead uses exploration-exploitation.Pytheas outperforms prediction-based systems by reducing prediction bias and delayed response; DeepRM achieves performance comparable to state-of-the-art scheduling heuristics at lower cost.Source: Overview of Recent Advances · pp. 5–6
Opportunities for MLN
The section identifies opportunities for MLN in practical deployment, shared data and simulation resources, new network design methods, and advances to machine learning itself. It also emphasizes that feasibility, robustness, generalization, and interpretability remain important challenges.
Feasibility Discussion
- Training with global information and incrementally updating from local information can balance computation overhead against information staleness for delay-sensitive applications.Online systems commonly use a result table or a trained model for real-time decisions; processing times in selected advances suggest ML can be practical when the system is well designed.Source: Opportunities for MLN · p. 6
Open Datasets for the Networking Community
- Unified open datasets could enable standard benchmarks, reduce unrepresentative repeated experiments, and support research using fast, scalable, high-fidelity network simulators.Labeled network data are expensive and labor-intensive to collect, while limited access to large-scale systems makes testing costly; simulator-based learning has lower cost in RL scenarios.Source: Opportunities for MLN · pp. 6–7
Flexible Objectives and Cross-Layer Optimization
- Machine learning could optimize flexible objectives and enable cross-layer adaptation using application behavior or QoE as rewards, but imperfect rewards can produce faulty or dangerous behavior.Changing the reward function can change the optimization goal, unlike traditional heuristic algorithms; directly capturing the exact objective is often difficult.Source: Opportunities for MLN · pp. 6–7
Automated Network Protocol and Architecture Design
- Automated protocol and architecture design could improve network performance and efficiency, and generative or cooperative learning suggests potential for creating network components without human involvement.The section cautions that automatic protocol design remains difficult and that existing generated results are still far from that possibility.Source: Opportunities for MLN · p. 7
Improving the Comprehension of Network Systems
- Post hoc analysis of learning algorithms can reveal network behavior and inform high-performance designs, as DeepRM, CFA, and Remy illustrate.DeepRM reserves room for yet-to-arrive small jobs rather than being work-conserving, reducing job waiting time; CFA and Remy offer insights into video QoE and TCP congestion control.Source: Opportunities for MLN · p. 7
Automated Network Resource Scheduling and Decision Making
- Reinforcement learning could replace adaptive network algorithms or assign parameters according to network state, but adds complexity and uncertainty that challenge stability, reliability, and repeatability.The section notes that exploration-exploitation strategies are difficult to apply directly in highly varying network environments.Source: Opportunities for MLN · p. 7
Promoting the Development of Machine Learning
- Networking applications can expose machine-learning challenges in robustness, worst-case guarantees, generalization across dynamic traffic, accountability, and interpretability.Retraining whenever traffic characteristics change is unacceptable, and black-box models can obstruct understanding or intervention in their policies.Source: Opportunities for MLN · pp. 7–8
Conclusions
The supplied passages provide biographies of Mowei Wang and Yong Cui, covering their education, positions, research interests, and professional contributions.
Biographies
- Mowei Wang earned a 2017 B.Eng. from Beijing University of Posts and Telecommunications and is pursuing a Ph.D. at Tsinghua University.His research interests include data center networks and machine learning.Source: Biographies · p. 8
- Yong Cui earned B.E. and Ph.D. degrees from Tsinghua University in 1999 and 2004, respectively, and is a full professor there.He has published over 100 papers and co-authored seven Internet standard documents (RFC) on IPv6 technologies.Source: Biographies · p. 8
- Yong Cui’s research interests include mobile cloud computing and network architecture, and he serves as an IETF working group co-chair.He has served or serves on the editorial boards of IEEE TPDS, IEEE TCC, and IEEE Internet Computing.Source: Biographies · p. 8