Source-linked AI summary
Matched-View Cross-Domain Evaluation of WireGuard VPN Traffic Classification Using Early-Flow Fingerprints
Yasameen Sajid Razooqi, Adrian Pekar
TL;DR
VPN traffic-classification benchmarks often confound encapsulation with session differences because they collect VPN and non-VPN traffic separately. This paper uses matched simultaneous WireGuard views to compare FlowFeatures and SPLT across tree models and CNN1D, finding that sequence-aware CNN1D transfers best without VPN training data.
Problem
Separate-session VPN benchmarks confound encapsulation effects with differences in user behavior, timing, and application mix.
Method
The paper trains models on non-VPN flows and tests them on matched VPN views using FlowFeatures or SPLT with Random Forest, XGBoost, and CNN1D.
Results
CNN1D achieves the strongest transfer with balanced accuracy 0.9753 and macro F1 0.8932 without VPN data during training.
Takeaways & Limitations
Cross-domain transfer depends jointly on feature representation and model architecture, with CNN1D exploiting SPLT sequence structure more effectively than flattened tree-based inputs.
Abstract
from arXiv · showhide
Classifying VPN-encrypted traffic by application category typically relies on datasets that collect non-VPN and VPN traffic in separate sessions, conflating encapsulation effects with session-level differences in user behavior, timing, and application mix. We use a recently published WireGuard tunnel dataset in which pre- and post-tunnel traffic is captured simultaneously, with a packet-level match ratio above 99.9%. This matched-capture design eliminates session-level confounds and enables a cross-domain benchmark: models are trained on non-VPN flows and tested on the VPN view of the same underlying flows. We compare whole-flow statistical aggregates (FlowFeatures) and Sequence of Packet Length and Time (SPLT) early-flow fingerprints across Random Forest, XGBoost, and a multi-scale CNN1D. Cross-domain transfer depends jointly on representation and model: tree ensembles achieve balanced accuracy of 0.84-0.93 with FlowFeatures but only 0.60-0.75 with flattened SPLT, whereas CNN1D processes the same SPLT fingerprint as a sequence and achieves the strongest transfer overall (balanced accuracy 0.98, macro F1 0.89) without any VPN data during training.
I. INTRODUCTION
The paper addresses confounded VPN-classification evaluation with matched pre- and post-tunnel captures, then shows that transfer depends jointly on representation and model architecture.
- VPN encapsulation removes payload cues and distorts packet sizes and inter-arrival times used by flow-level classifiers.
- Separate-session benchmarks confound encapsulation effects with differences in user behavior, timing, and application mix.
- A matched-view WireGuard benchmark pairs simultaneous pre- and post-tunnel traffic, eliminating session-level confounds and enabling direct evaluation of encapsulation effects.
- Tree ensembles transfer better with FlowFeatures than flattened SPLT, while CNN1D preserves SPLT sequence structure and achieves the strongest transfer.
II. RELATED WORK
Prior VPN-classification studies use varied representations, but widely used benchmarks collect VPN and non-VPN traffic in separate sessions, limiting attribution of transfer degradation to encapsulation.
- Prior methods include FlowPic packet-size and timing histograms designed to preserve category structure through VPN or Tor tunnels.
- ISCX-VPN-2016 and VNAT collect non-VPN and VPN traffic in separate sessions.
- Separate-session collection mixes encapsulation with differences in timing, user behavior, application versions, and network conditions.
III. DATASET AND MATCHED-CAPTURE DESIGN
The dataset provides simultaneous paired inner and outer WireGuard captures of the same residential flows, with packet-level matching and shared application labels.
- The dataset contains approximately 80 hours of residential WireGuard traffic from 10 end-user devices across two sessions.
- Simultaneous inner and outer captures record the same traffic at the WireGuard interface and through an inline network TAP.
- Inner packets match encrypted outer counterparts above a 99.9% packet-level ratio, enabling label transfer from DPI-visible inner traffic.
- After excluding 33 unmatched flows and categories with fewer than 200 samples, the dataset contains 226,281 flows per view across 14 categories.
IV. METHODOLOGY
The methodology compares whole-flow aggregates with early SPLT fingerprints, representing packet direction, size, and timing for the first 50 matched packet pairs.
- FlowFeatures represent each flow with 21 bidirectional statistical aggregates covering packets, bytes, sizes, timing, duration, and rates.
- SPLT encodes the first N=50 matched packet pairs as a 50 × 3 sequence.
- The three SPLT channels encode direction, packet size in bytes, and inter-arrival time in milliseconds.
B. Classification Models
The evaluation compares non-temporal tree baselines with a multi-scale CNN1D designed to process SPLT sequences. Tree models use aggregates directly or flattened SPLT, while CNN1D preserves multi-scale temporal structure.
- Random Forest and XGBoost serve as non-temporal baselines for FlowFeatures and flattened SPLT.Flattening converts the 50 × 3 SPLT tensor into 150 fixed feature positions, removing sequence-aware inductive bias.
- CNN1D applies parallel 1D convolutions with kernels k ∈{3, 7, 11} to detect temporal patterns at multiple scales.The branches target local bursts, medium-range request–response structures, and longer handshake sequences.
C. Evaluation Protocol
The protocol trains models on non-VPN views and evaluates them on matched VPN views of the same flows, isolating encapsulation-related representation shift. Balanced accuracy and macro F1 are prioritized because the test distribution is dominated by majority classes.
- Models are trained on non-VPN flows and evaluated on the corresponding VPN views of the same flows, with no VPN samples during training.This same-flow design measures transfer across encapsulation-induced representation shift rather than generalization to unseen flows.
- Balanced accuracy and macro F1 are the primary comparison metrics because the distribution has 86% majority-class dominance.Accuracy, weighted F1, and macro-averaged average precision are also reported.
- SPLT training uses random timestep masking and additive Gaussian noise, while CNN1D uses early stopping and learning-rate reduction on plateau.All models use a fixed random seed.
V. RESULTS
Matched-view results show that transfer success depends jointly on representation and model family. FlowFeatures support strong classical performance, flattened SPLT weakens tree models, and CNN1D achieves the strongest overall transfer with sequential SPLT processing.
- 0.9271 balanced accuracy is achieved by XGBoost with FlowFeatures, compared with 0.8427 for Random Forest on the same 21-dimensional representation.These results show that aggregate features retain useful cross-domain performance for classical models.
- CNN1D with SPLT achieves the best reported metrics: balanced accuracy 0.9753 and macro F1 0.8932.It uses no VPN data during training and exceeds the best classical configuration in balanced accuracy by 0.048.
- The results reveal a representation–model interaction rather than a transfer advantage determined by either representation or model family alone.The benchmark evaluates five cross-domain configurations trained on non-VPN views and tested on matched VPN views.
- CNN1D with SPLT reaches AP = 0.983, while classical configurations occupy the AP = 0.929–0.959 range.CNN1D sustains precision above 0.97 up to recall ≈0.86; XGBoost + SPLT drops earliest above recall ≈0.8.
A. Per-Class Cross-Domain Transfer
Per-class transfer is strongest and most consistent for the dominant Network and Web categories, while minority categories expose a pronounced interaction between representation and model. CNN1D with SPLT often helps temporally structured classes, but category-specific weaknesses remain.
- Network and Web account for 86.2% of the test set and achieve F1 above 0.94 under all three configurations.Their dominant volume and distinctive aggregate profiles provide sufficient signal across representations and models.
- CNN1D with SPLT leads on seven of twelve minority categories, with its largest margins on VoIP (+0.19 F1), Collaborative (+0.18), Download (+0.17), SoftwareUpdate (+0.16), and Chat (+0.13).These margins are reported against XGBoost with FlowFeatures.
- XGBoost with FlowFeatures remains stronger for statistically discriminative categories, including Database (F1 = 0.96), Unspecified (0.99), and Email (0.93).This shows that sequential modeling is not uniformly superior across minority categories.
- SoftwareUpdate exposes the starkest flattening failure: XGBoost with SPLT recalls 15% of flows (F1 = 0.26), whereas CNN1D reaches F1 = 0.96 on identical SPLT input.The difference isolates the effect of processing the same SPLT representation with or without sequence-aware modeling.
- CNN1D’s single weak category is Email (F1 = 0.61), where 99% recall is accompanied by 0.45 precision because Unspecified flows are over-predicted as Email.The Email precision loss is attributed almost entirely to Unspecified-to-Email confusion.
B. Representation–Model Interaction
Cross-domain transfer depends on the interaction between representation and model family: tree ensembles perform better with FlowFeatures, while CNN1D is strongest on sequential SPLT inputs. The matched-view evaluation indicates that sequence-aware processing can better preserve discriminative information under encapsulation-induced distortion.
- No feature representation dominates across model families: FlowFeatures transfer well with tree ensembles, while identical SPLT inputs span the worst and best configurations.
- Balanced accuracy exceeds macro F1 for every configuration, consistent with stronger minority recall accompanied by lower precision on overlapping categories.
- RF with FlowFeatures reaches balanced accuracy 0.8427 versus XGBoost’s 0.9271, but exceeds it in accuracy at 0.9680 versus 0.9421 and average precision at 0.959 versus 0.942.RF’s lower balanced accuracy is largely explained by System, where it achieves F1 = 0.03 versus XGBoost’s F1 = 0.99.
- CNN1D with SPLT reaches balanced accuracy 0.9753, compared with 0.6040 for RF and 0.7534 for XGBoost using the same representation.SoftwareUpdate and SocialNetwork achieve F1 values of 0.96 and 0.99 under CNN1D, versus lower values under flattened tree models.
- Flattened SPLT likely transfers poorly for tree models because position-specific thresholds are vulnerable to encapsulation shifts, whereas CNN1D filters operate on local neighborhoods.The proposed mechanism remains unverified because the paper calls for an ablation of SPLT design choices.
- The CNN1D balanced accuracy of 0.98 is best interpreted as robustness to encapsulation-induced distortion of early-flow temporal patterns, not generalization to unseen flows.
VII. CONCLUSION
The paper presents a matched-view WireGuard benchmark showing that transfer performance depends jointly on representation and model family. Its numerical conclusions are bounded by a narrow evaluation scope and by the benchmark’s focus on encapsulation-induced representation shift rather than unseen-flow deployment generalization.
- The benchmark uses simultaneous pre- and post-tunnel views of the same flows to eliminate session-level confounds in VPN classification evaluation.
- Transfer success depends jointly on feature representation and model family, with tree ensembles favoring FlowFeatures and CNN1D achieving the strongest result with SPLT.
- The evaluation covers one VPN protocol, one residential capture site, two sessions from 10 devices, and a heavily skewed class distribution.
- The matched-view benchmark isolates transfer across encapsulation-induced representation shift rather than deployment-style generalization to unseen flows.