Source-linked AI summary

NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generation

Xi Jiang, Shinan Liu, Aaron Gember-Jacobson, Arjun Nitin Bhagoji, Paul Schmitt, Francesco Bronzino, Nick Feamster

arXiv:2310.08543v1cs.NI

TL;DR

Labeled network datasets are scarce and often stale, while existing synthetic generators may omit packet details needed for realistic traces and downstream uses. NetDiffusion turns packet captures into images and uses controlled, fine-tuned diffusion with protocol-aware processing to generate synthetic packet captures. The paper reports greater similarity to real traffic, improved classification performance, and compatibility with network analysis and testing tools.

  • Problem

    Labeled network traces are scarce because collection and sharing raise cost and privacy concerns, while public datasets often remain static as network behavior changes.

    Source: §1 · p. 1
  • Method

    NetDiffusion converts packet captures into images and fine-tunes controlled Stable Diffusion with protocol-aware post-processing to generate synthetic packet captures.

    Source: §1 · p. 2, §3.2.2 · p. 9
  • Results

    NetDiffusion traces show higher statistical resemblance to real data and generally improve classification accuracy over the state-of-the-art generation method.

    Source: §1 · p. 2, §6 · p. 20
  • Takeaways & Limitations

    Generated traffic can be used as raw packet captures with common network analysis and testing tools, extending its use beyond ML tasks.

    Source: §1 · p. 2

    Generation is currently capped at 1,024 packets per flow sample.

    Source: §6 · p. 21

Abstract

from arXiv · show

Datasets of labeled network traces are essential for a multitude of machine learning (ML) tasks in networking, yet their availability is hindered by privacy and maintenance concerns, such as data staleness. To overcome this limitation, synthetic network traces can often augment existing datasets. Unfortunately, current synthetic trace generation methods, which typically produce only aggregated flow statistics or a few selected packet attributes, do not always suffice, especially when model training relies on having features that are only available from packet traces. This shortfall manifests in both insufficient statistical resemblance to real traces and suboptimal performance on ML tasks when employed for data augmentation. In this paper, we apply diffusion models to generate high-resolution synthetic network traffic traces. We present NetDiffusion, a tool that uses a finely-tuned, controlled variant of a Stable Diffusion model to generate synthetic network traffic that is high fidelity and conforms to protocol specifications. Our evaluation demonstrates that packet captures generated from NetDiffusion can achieve higher statistical similarity to real data and improved ML model performance than current state-of-the-art approaches (e.g., GAN-based approaches). Furthermore, our synthetic traces are compatible with common network analysis tools and support a myriad of network tasks, suggesting that NetDiffusion can serve a broader spectrum of network analysis and testing tasks, extending beyond ML-centric applications.

1 Introduction

Labeled network datasets are scarce and often stale, while existing synthetic traces may omit packet-level detail needed for realistic traffic generation and broader network use. NetDiffusion applies controlled diffusion to generate protocol-constrained packet captures, with reported gains in classification and compatibility with network analysis tools.

Motivation

  • Labeled network traces are difficult to collect and share because of cost and privacy concerns, and public datasets often remain unchanged as network behavior evolves.These constraints hinder training models that reflect evolving real-world conditions.Source: §1 · p. 1
  • Existing generators often target only selected attributes or flow statistics, limiting resemblance to real traces and performance when synthetic data augment classification datasets.Their focus on simplified attributes also leaves out protocol behaviors needed by conventional networking tools.Source: §1 · pp. 1–2
  • Diffusion models offer a route to generating detailed traffic patterns and full packet headers while conditioning generation on network properties and protocol dependencies.The paper contrasts these capabilities with GANs and describes more stable training gradients.Source: §1 · p. 2

Method

  • NetDiffusion converts packet captures to images, fine-tunes a text-to-image diffusion model, and uses controlled generation and protocol-aware heuristics to produce synthetic captures.The heuristics check and adjust generated fields for semantic correctness under transport- and network-layer rules.Source: §1 · p. 2

Results

  • NetDiffusion-generated traffic generally improves classification accuracy over the state-of-the-art generation method and shows promise for addressing class imbalance.The evaluation integrates generated traffic into a curated dataset at varying proportions.Source: §1 · p. 2

Applicability beyond ML

  • Generated traffic can be converted into raw packet captures for analysis and testing, including use with Wireshark, Scapy, and tcpreplay.The paper also reports that critical statistical features for network operations can be extracted from generated traffic.Source: §1 · p. 2

2 Motivation

The motivation traces a path from scarce and aging network datasets to synthetic augmentation, then identifies limits in NetFlow-style generators for classification and packet-level tasks. The motivating evaluation reports weaker accuracy with NetShare-generated data than with real NetFlow and the highest accuracy with raw traffic.

2.1 Network Data Scarcity

  • Network datasets support ML, analysis, and testing, but collection, privacy, labeling, and infrequent updates leave data stale or incomplete.The passage notes that even frequently updated MAWI data can contain missing or duplicate traces.Source: §2.1 · p. 3

2.2 Data Augmentation Using Synthetic Data

  • NetShare generates aggregated IPFIX/NetFlow-style statistics, but its limited attributes can miss patterns needed for accurate ML and cannot provide raw captures for other tasks.Template- or rule-based alternatives may also miss evolving traffic and complex protocol or application interactions.Source: §2.2 · pp. 3–4

2.3 Inadequate Performance from Existing Methods

  • The motivating NetShare case study compares real and synthetic NetFlow across training and testing conditions using RF, DT, and SVM on micro- and macro-level classification.The task covers 10 application classes at the micro level and three service categories at the macro level.Source: §2.3 · p. 4

2.3.1 Unsatisafactory ML Accuracy.

  • NetShare-generated NetFlow data lowers classification accuracy relative to real NetFlow, while raw traffic achieves the highest accuracy, with SVM near-perfect.The comparison attributes the raw-traffic advantage to richer information than the limited NetFlow feature set preserves.Source: §2.3.1 · p. 4
  • Raw network traffic preserves distinguishing details that NetFlow omits, motivating synthetic generation of raw traces rather than only aggregated statistics.The paper identifies preservation of critical distinguishing features as important for classification accuracy.Source: §2.3.1 · p. 4

2.3.2 Limited Applicability to Non-ML tasks.

  • NetFlow lacks packet-level details needed for packet analysis, replay, and deriving attributes, while raw traffic must also obey protocol rules for reliable tools and testing.Examples of relevant constraints include frame sizes, TCP sequence numbers, checksums, and flag settings.Source: §2.3.2 · pp. 4–5

3 Method

NetDiffusion adapts controlled text-to-image diffusion to generate synthetic raw network traffic that follows transport- and network-layer protocol rules. Its pipeline converts packet captures into images, fine-tunes a Stable Diffusion model for controlled generation, and applies domain-informed post-processing.

3 Method

  • NetDiffusion uses controlled text-to-image diffusion to generate synthetic raw traffic that complies with transport- and network-layer protocol rules.Text prompts characterize the traffic classes or types to be generated.Source: §3 · pp. 5–6
  • The pipeline has three components: converting pcaps to images, fine-tuning Stable Diffusion for controlled traffic generation, and applying domain-knowledge heuristics for protocol compliance.The framework presents these as sequential stages of the method.Source: §3 · p. 6

3.1 Network Traffic to Image Conversion

Image representations address the high dimensionality and sequential dependencies of packet traces while providing a consistent format for generative models. NetDiffusion encodes packet-header bits as standardized rows, groups packets into fixed-size images, and supports conversion back to pcaps.

3.1.1 Advantages of Using Image Representation of Network Traffic.

  • Network traces contain over a thousand attributes per packet in standardized formats such as nPrint, creating computational bottlenecks for generative models.Packets also have sequential dependencies that ordinary tabular formats may fail to preserve.Source: §3.1.1 · pp. 6–7
  • Image representations can capture local and global spatial patterns that reflect inter-packet and intra-packet dependencies, while drawing on mature image-generation tools.The paper also cites visualization and interpretability as advantages of image representations.Source: §3.1.1 · p. 7

3.1.2 Conversion Process.

  • NetDiffusion encodes pcaps with nPrint, maps set, unset, and vacant bits to green, red, and gray, and represents each packet as an image row.Packets are grouped in sets of 1,024; the resulting image is 1,088 by 1,024 pixels and can be converted back to pcaps.Source: §3.1.2 · p. 7

3.2 Fine-Tuning Diffusion Model and Controlled Generation

The section represents network flows as images, fine-tunes Stable Diffusion on network data, and combines class prompts with ControlNet to constrain synthetic traffic.

3.2 Fine-Tuning Diffusion Model and Controlled Generation

  • Fine-tuned diffusion models generate synthetic network traffic from image representations of real labeled flows.The traffic flows are first transformed into image representations, which are used to fine-tune the diffusion model.Source: §3.2 · p. 7

3.2.1 Advantages of Text-to-Image Diffusion Models for Network Traffic Generation.

  • Diffusion models are selected for their ability to capture intricate patterns, handle high-resolution images, and condition generation on text prompts.The paper also cites transparent generation and stable training behavior as advantages over GANs.Source: §3.2.1 · p. 8

3.2.2 Fine-Tuning a Stable Diffusion Base Model Using LoRa.

  • Stable Diffusion 1.5 is fine-tuned with LoRa, which adapts the model to network datasets while keeping model files compact.Class-specific encoded prompts are paired with traffic images during fine-tuning.Source: §3.2.2 · p. 9

3.2.3 Controlled Generation Via ControlNet.

  • Class-specific text prompts guide the reverse diffusion process to generate traffic images in the requested class distribution.The quantity of prompts supplied per class controls the generated dataset’s class counts and size.Source: §3.2.3 · p. 9
  • ControlNet uses detected field boundaries to constrain populated regions, while post-generation heuristics refine field details for protocol conformance.The paper uses M-LSD line detection to identify boundaries between fields that should be populated and those that should remain vacant.Source: Figure 2 · p. 10, §3.2.3 · p. 10

3.3 Improving Transport and Network Layer Protocol Compliance

The section explains why encoded traffic must be corrected before conversion to raw packet captures: generated values may not satisfy protocol rules within or across packets.

3.3 Improving Transport and Network Layer Protocol Compliance

  • Synthetic encoded traffic captures features for ML, but converting it to raw formats such as pcaps requires compliance with transport- and network-layer rules.Those rules govern packet formatting and sequencing, which network analysis and testing tools rely on.Source: §3.3 · p. 10, §3.3.1 · p. 11

3.3.1 Inter and Intra Packet Transport and Network Layer Protocol Rules.

  • Inter-packet rules govern field relationships and sequencing across a flow, while intra-packet rules constrain each packet’s contents, including checksums and port values.TCP sequence and acknowledgment numbers must align across packets; checksums must match packet contents.Source: §3.3.1 · p. 11

3.3.2 Post-Processing Heuristic for Protocol Rule Compliance.

  • Because ControlNet approximates field distributions but not all bit-level rules, the method post-processes critical fields using separate intra- and inter-packet dependency trees.The trees are traversed bottom-up, with intra-packet dependencies corrected before inter-packet dependencies; the accompanying figure gives TCP examples.Source: §3.3.2 · pp. 11–12, §3.3.1 · p. 11, Figure 3 · p. 12
  • The heuristic uses majority voting for selected uniform or initialization values and samples real-traffic time distributions for packet timestamps.The authors note that timestamp generation is not fully optimized because diffusion models may struggle with long-range temporal patterns; post-processing supports conversion to pcaps for non-ML tasks.Source: §3.3.2 · p. 12

4 Evaluation

NetDiffusion is evaluated on a curated network-traffic dataset for statistical similarity, ML classification, and compatibility with network analysis and testing tasks. Its generated traffic generally compares favorably with NetShare and supports packet-level uses beyond ML.

4.1 Dataset Overview and Synthetic Traffic Generation

  • The curated dataset contains nearly 20,000 flows from ten applications, and the authors randomly sampled 10% for evaluation after preprocessing and labeling flows.The applications span video streaming, video conferencing, and social media; the authors report comparable results on larger and smaller subsets.Source: §4.1 · p. 13, Table 2 · p. 13

4.2.1 Statistical Similarity Results.

  • Despite the complexity of pcap traffic, NetDiffusion's statistical distances from real traffic are lower than NetShare's distances for simpler NetFlow attributes.The evaluation compares normalized statistical differences across all generated fields and the shared IPv4 protocol field using JSD, TVD, and HD; lower values indicate greater similarity.Source: §4.2.1 · pp. 13–14, Table 3 · p. 14

4.2.2 ML Classification Results.

  • NetDiffusion generally achieves higher classification accuracy than NetShare across complete-synthetic and mixed-data evaluations, and across model choices.Table 4 reports higher accuracy across complete synthetic-data scenarios; the mixed-data evaluation also favors NetDiffusion, with the advantage generally holding across models.Source: Table 4 · p. 15, §4.2.2 · pp. 15–16
  • As the mixing rate rises, accuracy declines less sharply for NetDiffusion than for NetShare, but macro-level accuracy drops when the mixing rate exceeds 0.8.The mixing rate is the percentage of real training data replaced by synthetic data; the training-set size remains constant across rates.Source: §4.2.2 · pp. 15–16, Figure 4 · p. 16
  • Synthetic balancing matched or exceeded the original imbalanced dataset, with decision-tree accuracy increasing by 0.09 and Meet and Zoom accuracy increasing by 0.091 and 0.045, respectively.The authors attribute the gains mainly to improved performance on previously underrepresented classes.Source: §4.2.2 · pp. 17–18, Table 8 · p. 17

4.3 Extendability to Additional Network Anlysis and Testing Tasks

  • NetDiffusion traffic can be parsed by Wireshark, retransmitted with tcpreplay, and used to extract metrics for a broad range of network analysis tasks.All 1,024 packets were retransmitted without failures or truncations; reported applications include traffic and protocol analysis, performance assessment, device identification, routing, and error analysis.Source: §4.3 · pp. 18–19, Table 9 · p. 19, Table 10 · p. 19

5 Related Work

Prior work spans diffusion-based generation and several network-traffic generation families. Network traffic poses distinct protocol constraints, while existing methods differ in scalability, adaptability, and adherence.

  • Diffusion models have demonstrated generative capacity across image synthesis, video creation, and structured data generation; Stable Diffusion uses pretrained autoencoders.Source: §5 · p. 19
  • Traditional simulators emulate traffic from network topologies, structure-based methods capture patterns through heuristics and scale better, and GANs capture intricate temporal patterns.Source: §5 · p. 20
  • Traffic generation presents distinct protocol constraints; traditional methods may require substantial expertise and lack versatility, while GANs may fall short in protocol adherence.Source: §5 · p. 20
  • Diffusion controls seek to balance diversity and controllability, with approaches conditioning generation on inputs such as edge maps or identifiers.The passage describes ControlNet's task-centric conditions and DreamBooth's identifiers for personalized, diverse outputs.Source: §5 · p. 20

6 Conclusion and Future Work

NetDiffusion generates packet captures covering all packet headers that closely resemble authentic traffic and improve ML performance over current methods. The paper identifies protocol compliance, temporal modeling, longer flows, and meaningful payloads as future-work areas.

  • NetDiffusion produces packet captures covering all packet headers that closely resemble authentic data and improve ML performance, while supporting traditional network tools and retransmission.The paper identifies the traces as suitable for a broad range of network tasks.Source: §6 · p. 20
  • Protocol-rule compliance is currently post-generative; future work aims to embed those rules directly in the generation pipeline.Source: §6 · p. 21
  • Future work targets direct time-series generation for inter-packet dependencies and extending the current 1,024-packet-per-flow generation horizon.Source: §6 · p. 21
  • The paper also identifies network-specific diffusion foundation models and semantically meaningful payload generation as future directions.Meaningful payloads remain a challenge; autoencoders are suggested as a possible direction.Source: §6 · p. 21
Loading 2310.08543v1…