Source-linked AI summary
How to Achieve High Classification Accuracy with Just a Few Labels: A Semi-supervised Approach Using Sampled Packets
Shahbaz Rezaei, Xin Liu
TL;DR
Large labeled datasets are cumbersome to capture for traffic classification. The paper pre-trains on unlabeled traffic using sampled packets, then retrains with a few labels; with 20 labeled examples per class, its QUIC model achieves high accuracy close to fully supervised models.
Problem
Method
The method pre-trains on unlabeled traffic to predict whole-flow statistical features from sampled packets, then transfers the learned model for retraining on a small labeled dataset.
Source: §4.1 · pp. 4–5Results
With 20 labeled data from each class, the QUIC model achieves high accuracy close to a fully supervised model.
Source: §7 · p. 14Takeaways & Limitations
Incremental sampling outperforms random and fixed sampling, with sampled flows capturing both short- and long-range patterns.
Source: §5.2 · p. 9On ISCX, the approach stayed below 68% accuracy with 20 flows per class when using statistical and time-series features; payload-based fully supervised learning exceeded 95% [10].
Source: §6 · pp. 13–14
Abstract
from arXiv · showhide
Network traffic classification, which has numerous applications from security to billing and network provisioning, has become a cornerstone of today's computer networks. Previous studies have developed traffic classification techniques using classical machine learning algorithms and deep learning methods when large quantities of labeled data are available. However, capturing large labeled datasets is a cumbersome and time-consuming process. In this paper, we propose a semi-supervised approach that obviates the need for large labeled datasets. We first pre-train a model on a large unlabeled dataset where the input is the time series features of a few sampled packets. Then the learned weights are transferred to a new model that is re-trained on a small labeled dataset. We show that our semi-supervised approach achieves almost the same accuracy as a fully-supervised method with a large labeled dataset, though we use only 20 samples per class. In tests based on a dataset generated from the more challenging QUIC protocol, our approach yields 98% accuracy. To show its efficacy, we also test our approach on two public datasets. Moreover, we study three different sampling techniques and demonstrate that sampling packets from an arbitrary portion of a flow is sufficient for classification.
1 Introduction
Large labeled traffic datasets are difficult to capture, while unlabeled traffic is readily available. The paper proposes pretraining with unlabeled data and sampled packets, then retraining with a few labeled examples.
1 Introduction
- Deep learning can capture complex traffic patterns, but its need for large labeled datasets makes accurate classifiers cumbersome to build.Labeling encrypted traffic is difficult, and common collection procedures require isolated or edge-network settings.Source: §1 · pp. 1–2
- Sampled packets replace entire-flow inputs, reducing memory and computation needs for time-series features.The authors frame this as practical for high-speed links and data centers.Source: §1 · p. 2
- The approach pre-trains on abundant unlabeled data, then retrains with a few labeled samples; the authors report near-upper-bound accuracy for some datasets and strong QUIC classification.They also test on public datasets, including pretraining on a dataset separate from the target dataset.Source: §1 · p. 2
2 Related Work
Prior traffic-classification studies use classical machine learning and deep learning, including CNNs and LSTMs. The related work identifies large labeled datasets and lack of packet sampling as gaps in the approaches reviewed.
2 Related Work
- Prior deep-learning classifiers use CNNs, LSTMs, or combinations on public and private datasets, with inputs including statistical features, time series, and raw header or payload bytes.The QUIC study cited captures five Google services and trains fully supervised on 150 GB of data.Source: §2 · pp. 2–3
- The reviewed traffic-classification framework outlines a seven-step process but still requires a sufficiently large training dataset.Its steps include data capture, preprocessing, model selection, and evaluation.Source: §2 · p. 3
- The authors characterize the reviewed work as assuming large labeled datasets and not considering packet sampling.Source: §2 · p. 3
3 Problem Statement
The problem is to train accurate traffic classifiers when only a few labeled examples are available. Labeling encrypted traffic is difficult, and collection settings or scripted traffic may not reflect operational traffic.
3 Problem Statement
- Deep-learning traffic classifiers can achieve high accuracy, but training them requires large labeled datasets that are cumbersome to capture and label.Source: §3 · p. 3
- Because encryption makes operational-network labeling nearly impossible without an accurate classifier, labels are often collected in isolated or edge settings whose traffic distributions may differ from core networks.The authors also state that scripted traffic may differ from human-generated traffic, causing poor performance on real traffic.Source: §3 · p. 3
- The objective is to use a large unlabeled dataset, D_u, alongside a few labeled examples per target class in D_l to train an accurate classifier.The unlabeled data may include classes absent from the labeled target dataset.Source: §3 · pp. 3–4
4 Methodology
The method pre-trains on unlabeled traffic by predicting whole-flow statistical features from sampled packets, then transfers learned weights and retrains with a small labeled dataset. It compares sampling strategies and evaluates on QUIC, Waikato, and Ariel datasets.
4.1 Key Components
- The pre-trained model is transferred into a new model and retrained with a small labeled dataset; the pre-trained model can also be reused for other classification tasks.The authors note that the pre-trained estimator need not initially be accurate, because it is further retrained for classification.Source: §4.1 · p. 4
- Pretraining estimates whole-flow statistical features, such as average packet length and inter-arrival time, from sampled packets’ length, direction, and relative time.The authors propose this as a way to learn from unlabeled flows without human labeling.Source: §4.1 · p. 4
- Time-series features are used as input and statistical features as pretraining targets, since whole-flow statistics require observing the full flow and QUIC exposes fewer unencrypted handshake fields.Source: §4.1 · p. 5
- Sampling a fixed number of packets from different flow portions avoids capturing or storing an entire flow and can expose patterns beyond the connection’s beginning.Repeated sampling from one flow can also serve as data augmentation.Source: §4.1 · p. 5
4.2 Semi-supervised Learning
- CNNs suit sampled-flow inputs because their shift-invariant filters can capture the same patterns when they appear in different input regions.Source: §4.2 · p. 5
- The model transfers convolutional-layer weights from unlabeled-data pretraining into a new model with more linear layers, then retrains it on labeled data.The input has two channels: sampled-packet inter-arrival time and signed packet length, with direction encoded by the sign.Source: §4.2 · pp. 5–6, Figure 1 · p. 6
4.3 Sampling Techniques
- Fixed-step sampling takes packets at a fixed interval, random sampling samples each packet with probability p, and incremental sampling increases the interval by multiplying it by α after every β sampled packets.Source: §4.3 · pp. 6–7
4.4 Datasets
- Evaluation uses a five-service QUIC dataset, unlabeled Waikato traces for pretraining, and a small portion of the Ariel dataset for retraining; flows under 100 packets are excluded.The Waikato traces predate QUIC, and the study uses about 4% of that dataset.Source: §4.4 · p. 7
5 Evaluation
The evaluation tests semi-supervised transfer and packet-sampling choices on QUIC and Ariel data. Incremental sampling approaches the fully supervised QUIC accuracy upper bound with few labeled flows, while human-generated traffic and alternative pre-training data expose important evaluation conditions.
5.1 Implementation Detail
- The QUIC experiments use 24 statistical-feature regression targets during pre-training, and macro-averaged accuracy for whole-model results.Pre-training used 6,439 unlabeled flows yielding 544,744 samples; supervised re-training used 30 captured flows per class, split into training and test sets.Source: §5.2 · p. 8, §5.1 · p. 8
5.2 QUIC Dataset
- Increasing labeled training-set size improves accuracy except with random sampling, which repeatedly samples from the start of a flow.Fixed-step and incremental sampling cover different flow portions, exposing the model to more patterns as training size increases.Source: §5.2 · p. 9, Figure 2 · p. 9
- Incremental sampling outperforms the other QUIC sampling methods, and pre-training improves fixed-step and incremental accuracy by around 10% over training without pre-training.Incremental sampling combines many packets from short ranges with some from long ranges.Source: §5.2 · p. 9, Figure 2 · p. 9
- With 20 flows per class for re-training, incremental sampling comes close to the fully supervised accuracy upper bound on QUIC.The authors identify the incremental-sampling model as their best model; fixed and random sampling require larger labeled sets to approach that bound.Source: §5.2 · pp. 9–10, Table 2 · p. 10
- On human-generated QUIC traffic, Google Search and Docs accuracy changes little, while Drive, YouTube, and Music accuracy drops by up to 7%.The authors attribute the class-dependent changes to differences in human interactions and traffic patterns.Source: §5.2 · p. 10, Figure 3 · p. 11
- Pre-training on Waikato data boosts QUIC accuracy even though QUIC traffic was absent, but the improvement is less than 10%.Waikato was captured before QUIC traffic was available; the authors describe the pre-trained model as a statistical-feature predictor.Source: §5.2 · p. 11, Table 3 · p. 11
5.3 Ariel Dataset
- Ariel pre-training that includes target-class flows improves accuracy over Waikato-only pre-training, although Ariel makes up only around 6% of the combined unlabeled data.The authors note that this resembles operational settings where target-task flows may be a small portion of captured traffic.Source: §5.3 · p. 12, Table 4 · p. 12
- Around 30 labeled flows per class can reach the fully supervised sampled-flow upper bound when pre-training includes the target classes.The reported result is for fixed-step sampling; the authors say trends for other sampling methods are similar.Source: §5.3 · p. 13, Figure 4 · p. 13
6 Discussion
The discussion identifies feature suitability and careful labeled-data coverage as boundaries on the approach, and notes that incremental sampling requires CNN-compatible parameter choices. It also distinguishes the method’s feature targets from alternatives reported in prior work.
6 Discussion
- 68% accuracy was the ceiling on ISCX with 20 flows per class, where statistical and time-series features had been reported to yield around 80% accuracy.The approach depends on those feature types being sufficiently effective for classification.Source: §6 · p. 13
- For a fair comparison, the Ariel experiment uses the same statistical features as pre-training regression targets, although other features have yielded better accuracy.Source: §6 · p. 13
- A CNN using payload information achieved above 95% accuracy with fully supervised learning [10].Source: §6 · p. 14
- The small labeled set must cover distinctive traffic patterns, such as different Android app actions, or missing patterns may degrade accuracy in real environments.The authors cite a missing Google-service action—renaming a file—as an example.Source: §6 · p. 14
- Incremental sampling outperforms the other sampling methods, but large α can make its widening packet gaps unsuitable for the CNN’s shared filters.The authors therefore advise against choosing a large α.Source: §6 · p. 14
7 Conclusion
The paper presents a semi-supervised traffic-classification approach that pre-trains on unlabeled sampled time-series data to predict flow statistics, then fine-tunes transferred weights with a small labeled set. On QUIC traffic, 20 labeled examples per class yield accuracy close to fully supervised training; tests on public datasets also show cross-dataset improvement.
7 Conclusion
- Pre-training predicts whole-flow statistical features from sampled time-series inputs, after which learned parameters are transferred to a model retrained on a small labeled set.Source: §7 · p. 14
- With 20 labeled data from each QUIC class, the approach achieves high accuracy close to a fully supervised model.Source: §7 · p. 14
- The paper evaluates fixed-step, random, and incremental sampling and reports that pre-training on an unlabeled public dataset can improve accuracy on another labeled dataset.Source: §7 · p. 14
- The authors conclude that transferred weights from a model pre-trained on a large unlabeled dataset may improve probably any traffic-classification task.Source: §7 · p. 14