Source-linked AI summary
New Directions in Automated Traffic Analysis
Jordan Holland, Paul Schmitt, Nick Feamster, Prateek Mittal
TL;DR
Network traffic analysis pipelines rely on painstaking manual feature engineering, model selection, and parameter tuning. The paper introduces nPrint, a unified packet representation, and nPrintML, its AutoML integration, then evaluates the system on eight tasks and releases the tools and datasets. The evaluation reports improved performance in many cases, while longer sequences and multi-flow temporal analysis remain open problems.
Problem
Traffic-analysis performance depends heavily on manually engineered feature representations, model choices, and parameter settings, making the pipeline painstaking and domain-intensive.
Method
nPrint transforms raw packets into a complete, normalized, aligned binary representation, while nPrintML integrates it with AutoML for automated model and hyperparameter selection.
Results
Across eight traffic-analysis tasks, nPrintML improved on the state of the art in many cases and outperformed the best hand-tuned extracted-feature models in all but one case.
Takeaways & Limitations
nPrint and nPrintML make many packet-based traffic-classification tasks amenable to largely automated analysis and provide released tools and datasets for extension.
Takeaways & Limitations
Capturing temporal relationships across multiple flows, analyzing longer traffic sequences, and applying the approach beyond supervised learning remain open directions.
Abstract
from arXiv · showhide
Despite the use of machine learning for many network traffic analysis tasks in security, from application identification to intrusion detection, the aspects of the machine learning pipeline that ultimately determine the performance of the model -- feature selection and representation, model selection, and parameter tuning -- remain manual and painstaking. This paper presents a method to automate many aspects of traffic analysis, making it easier to apply machine learning techniques to a wider variety of traffic analysis tasks. We introduce nPrint, a tool that generates a unified packet representation that is amenable for representation learning and model training. We integrate nPrint with automated machine learning (AutoML), resulting in nPrintML, a public system that largely eliminates feature extraction and model tuning for a wide variety of traffic analysis tasks. We have evaluated nPrintML on eight separate traffic analysis tasks and released nPrint and nPrintML to enable future work to extend these methods.
1 INTRODUCTION
Network traffic analysis depends heavily on manually engineering and representing features, selecting models, and tuning parameters. The paper introduces nPrint and nPrintML to automate this pipeline across diverse traffic-analysis tasks.
- Feature engineering, model selection, and parameter tuning remain iterative, painstaking processes requiring substantial specialized domain knowledge.
- nPrint encodes each packet in a normalized binary representation that lets models discover task-specific features without manual extraction.
- nPrintML combines nPrint with AutoML to automate model selection and hyperparameter tuning, often without writing code.
- Across eight case studies, nPrintML supports tasks including OS detection, device fingerprinting, application identification, mobile-trace country identification, and streaming-provider identification.
- The authors publicly released nPrint, nPrintML, and the datasets to support replication, extension, and broader application.
- Capturing temporal relationships across multiple flows and analyzing longer traffic sequences remain unsolved and unexplored.
2 DATA REPRESENTATION
The paper develops nPrint as a packet representation that combines binary data with semantic alignment. This design preserves packet information and ordering while avoiding the feature misalignment and manual decisions of alternative representations.
- Design requirements: The design aims for complete, constant-size, inherently normalized, and aligned representations so each feature has consistent meaning across packets.
- Alternative representations: Semantic representations lose ordering information, while naive binary representations can misalign features across protocols or packets.
- Alternative representations: 10% higher F1-score was achieved by binary TCP-option representations than semantic representations across a wide range of models.
- nPrint: nPrint is a hybrid representation that combines raw binary packet data with alignment based on packet semantic structure.
- nPrint: nPrint uses internal padding and reserves space for header types, preserving option ordering while mitigating misalignment and enabling bit-level interpretability.
- nPrint: nPrint is modular and extensible: single-packet fingerprints can be concatenated into multi-packet fingerprints, and additional protocols can be added.
3 NPRINT IMPLEMENTATION
The implementation supports common network protocols and offline or live packet processing. It processes packets efficiently with constant memory usage and can scale through parallel execution on high-rate links.
- nPrint supports Ethernet, IPv4, fixed IPv6 headers, UDP, TCP, ICMP, and corresponding packet payloads.
- The implementation processes PCAP and Zmap outputs offline, captures directly from live interfaces, reverses nPrint encoding to PCAP, and is publicly available.
- Over 1 million packets per minute were transformed, averaging about 1.5 million packets per minute with a single thread.
- 295 KB of constant memory was used for an IPv4-and-TCP configuration, while IPv4, UDP, ICMP, TCP, and 10 payload bytes used about 310 KB.
- Approximately 8 Gbps of live traffic was processed with near-zero loss using 16 queues and 16 parallel nPrint processes.
4 NPRINTML
nPrintML standardizes traffic-analysis pipelines by pairing nPrint’s packet representation with AutoML for feature selection, model search, and hyperparameter tuning. The implementation automates these steps across evaluated problems while exposing standardized metrics and reusable workflows.
- Pipeline design: nPrint standardizes packet feature development, while AutoML standardizes model selection and hyperparameter tuning for traffic-analysis problems.The design replaces expert-developed features and manually selected or tuned models with a unified representation and automated search.
- AutoGluon-Tabular AutoML: AutoGluon-Tabular performs feature selection, model search, and hyperparameter optimization across all eight evaluated problems.It trains, optimizes, and tests over 50 models from six base model classes, with the highest-performing model for each problem being an ensemble.
- AutoGluon-Tabular AutoML: The selected high-quality preset prioritizes high predictive accuracy and fast inference over the best-quality preset’s slightly higher accuracy and substantially greater inference time and disk usage.The best-quality preset incurs approximately 10x–200x slower inference and 10x–200x higher disk usage.
- AutoGluon-Tabular AutoML: Training uses an unconstrained time budget, a 75%/25% train-test split, and macro F1 optimization through hyperparameters and ensemble weights.Macro F1 is the unweighted mean of per-class F1 scores for multiclass classification.
- nPrintML Implementation: nPrintML combines nPrint and AutoGluon-Tabular in Python so researchers can create complete traffic-analysis pipelines with one program call.It can create pipelines from either a single traffic trace or an entire directory of traffic traces.
- Metrics: nPrintML reports balanced accuracy, ROC AUC, and F1 scores, using balanced accuracy for class imbalance and macro ROC AUC and macro F1 for multiclass evaluation.Macro ROC AUC uses a one-versus-rest calculation and averages the per-class scores without weighting.
5 CASE STUDIES
Across eight case studies, nPrintML applies a unified packet representation with automated model selection and tuning, matching or exceeding bespoke traffic-analysis approaches. The evaluations cover active device fingerprinting, passive OS fingerprinting, noisy DTLS application identification, and netML challenges.
- Case-study overview: nPrintML matches or beats existing bespoke approaches across eight traffic-analysis case studies.The case studies span multiple fingerprinting, identification, and challenge tasks.
- 5.1 Active Device Fingerprinting: Models trained on nPrint outperform Nmap’s hand-engineered features across device-fingerprinting metrics, without access to sent probes.The comparison includes average precision, other reported metrics, and runtime analyses.
- 5.2 Passive OS Fingerprinting: nPrintML achieves high-recall passive OS classification from few packets and identifies finer-grained OS differences than p0f.With one packet, nPrint models can outperform p0f using 100-packet samples; fine-grained distinctions include Windows Vista and Kali Linux.
- 5.2 Passive OS Fingerprinting: nPrintML separates Ubuntu and Windows with perfect balanced accuracy, ROC AUC, and F1 across device-pair training choices.The model learns the operating systems’ different initial IP time-to-live values rather than memorizing individual device characteristics.
- 5.3 DTLS Application Identification: For noisy DTLS handshakes, nPrintML achieves 99.8% accuracy and 99.8% F1 while avoiding manual feature engineering and model selection.The weighted ensemble also achieves a perfect ROC AUC and can perform the task using only UDP headers.
- 5.4 Additional Case Studies: In two of three netML challenge case studies, nPrintML outperforms the highest-performing hand-tuned leaderboard model without manual feature engineering.The supplied passage states that this occurs in every task but one.
6 RELATED WORK
Prior work addresses traffic classification, fingerprinting, deep learning, and AutoML, while nPrint combines a unified packet representation with automated model selection and tuning. The paper positions this combination as a new direction for network traffic analysis.
- Traffic analysis research has applied machine learning to classification, fingerprinting, obfuscated traffic detection, anomaly detection, and smart-home traffic identification.
- Deep learning studies have addressed intrusion detection and encrypted or Tor traffic, but this work examines whether models can automatically extract important features from network traffic.
- Host fingerprinting includes passive and active methods, with Nmap relying on probes and implementation differences while nPrint lets models learn from packet representations.
- AutoML research automates feature engineering, model architecture, and hyperparameter optimization, and this paper explores combining AutoML with network traffic classification.
- nPrintML uses AutoGluon-Tabular to search base models and create weighted ensembles while performing feature selection, model selection, and hyperparameter optimization.
7 CONCLUSION
The conclusion presents nPrint as a unified packet representation that supports automated traffic analysis through AutoML. Across eight tasks, nPrintML improved on the state of the art in many cases, while several multi-flow and time-series problems remain open.
- nPrint transforms raw network packets into a unified format suitable for representation learning and model training, automating part of a previously manual process.
- Applied to eight common traffic analysis tasks, nPrintML improved on the state of the art in many cases.
- Automated time-series analysis and classification involving multiple flows remain open problems, and the released tools and datasets provide a benchmark for extending the work.
A.1 Option Representation Evaluation
The evaluation tests whether preserving TCP option ordering matters by comparing nPrint with a semantic representation across device-fingerprint classification models. Losing ordering degrades performance broadly.
- Option ordering is treated as a representation requirement because semantic parsing does not preserve the order of TCP options.
- The evaluation compares nPrint and a semantic TCP-option representation using fingerprints from 15 device classes probed with Nmap.
- Table 9 reports that preserving option ordering increases performance across all evaluated models.
- Over 10% increase in F1 scores is observed for nPrint over the semantic representation across a wide array of models.
A.2 nPrintML Example
The nPrintML example demonstrates a reproducible workflow from repository and dataset setup through label generation and configurable traffic analysis runs. Existing labels can eliminate the need to write labeling code.
- The example connects nPrint with AutoML and is intended to reproduce results from scratch without writing code for the traffic-analysis pipeline.
- Labels are generated from dataset file paths and filenames, with label generation unnecessary when labels already exist.
- The workflow begins by cloning the repository and uncompressing the traffic traces.
- nPrintML can be configured for IPv4 and UDP traffic with selected payload lengths, including the first 10 or 100 payload bytes.
- The example also includes a UDP-headers-only configuration for running nPrintML on the labeled packet-capture directory.
- Table 10 documents Nmap’s highly complex device-detection tests used to generate each device fingerprint.
A.3 Nmap Tests
The nPrint command-line interface exposes options for selecting packet sources, protocol headers, timestamps, filtering, output indexing, and diagnostic behavior.
- Timestamp fields can be recorded as absolute or relative values, and nPrint bits can be filtered with a regular expression.
- Packet input can come from a CSV file of hexadecimal packets or a live-capture device, with an optional packet-count limit.
- Users can include Ethernet, IPv4, IPv6, ICMP, TCP, and UDP headers in the generated nPrint representation.
- Additional command-line options provide filter help, general help, usage information, and the program version.
- The tool can write an output-file index, print processing statistics, and display human-readable packets alongside nPrints.