PaperlayerView exact revision on arXiv

Source-linked AI summary

The Colossus with Feet of Clay: Debunking Encrypted Traffic Classifiers under PQC Evolution

Bingzhen Li, Lingjia Meng, Runhan Song, Chuanzhou Pan, Tongjun Pu, Ziqiang Ma, Yupeng Jiang, Lei Cui, Zhiyu Hao

arXiv:2608.22683v1cs.CR

TL;DR

Encrypted traffic classifiers are commonly evaluated under matched distributions, but TLS migration to hybrid PQC can reshape observable traffic while preserving application labels. This paper studies that PQC-induced protocol drift with a controlled Non-PQC/Hybrid-PQC benchmark and five classifiers across matched, cross-domain, and deployment-ratio settings. The experiments show that website information remains learnable, yet in-domain performance does not ensure reliable transfer across cryptographic environments.

  • Problem

    Matched-domain evaluation assumes deployment traffic remains representative, leaving the reliability of encrypted traffic classifiers under PQC-induced protocol drift insufficiently characterized.

  • Method

    The paper builds paired Non-PQC and Hybrid-PQC datasets with controlled Mixed test sets, then evaluates five classifiers and side-channel representations across cryptographic domains and deployment ratios.

  • Results

    Website identity remains learnable in matched domains, but models trained on Non-PQC traffic degrade on Hybrid-PQC traffic and generally decline as the Hybrid-PQC proportion increases.

  • Takeaways & Limitations

    Protocol configurations should be explicit evaluation dimensions, with cross-domain tests and deployment-representative datasets used to assess transfer across evolving cryptographic environments.

Abstract

from arXiv · show

Encrypted traffic classifiers often achieve high accuracy under matched training and testing conditions, implicitly assuming that deployment traffic follows the training distribution. TLS migration toward post-quantum cryptography (PQC) challenges this assumption because hybrid key establishment can reshape observable traffic without changing application labels. We frame this change as PQC-induced protocol drift and study its effects through closed-world HTTPS website fingerprinting using the deployed TLS~1.3 Hybrid-PQC group \texttt{\detokenize{X25519MLKEM768}}. We build a controlled, PQC-aware benchmark pairing Traditional (Non-PQC) and Hybrid-PQC traffic, then evaluate five representative classifiers and side-channel representations under matched-domain, cross-domain, and deployment-ratio settings. Collectively, the experiments show that PQC evolution does not remove learnable website information. Instead, it changes how that information appears in traffic, causing classifiers and feature combinations that perform well in-domain to lose reliability across cryptographic domains. By exposing the fragility of matched-domain evaluation, we offer strategic guidance, identify cross-domain robustness as a research priority, and recommend protocol-aware practices for dependable real-world encrypted traffic classification. The code is available at http://anonymous.4open.science/r/PQ-WF-Eval.

I. Introduction

PQC migration introduces protocol drift that reshapes observable TLS traffic while preserving website labels, exposing weaknesses in matched-domain evaluation. The paper constructs a controlled benchmark and evaluates classifier transfer across cryptographic domains and deployment mixtures.

  • Motivation: Hybrid-PQC key establishment adds larger key shares that can alter handshake sizes, packet segmentation, positions, and timing without changing website labels.The studied X25519MLKEM768 group combines X25519 with ML-KEM-768.
  • Motivation: Protocol drift occurs when evolving protocol mechanisms change the traffic-generation process under otherwise fixed experimental conditions.The transition from Traditional X25519 to Hybrid-PQC X25519MLKEM768 is the paper’s concrete PQC-induced instance.
  • Evaluation challenges: Traditional, Hybrid-PQC, and future Pure-PQC configurations may coexist, so matched-domain accuracy alone cannot characterize cross-domain mismatch or deployment-ratio sensitivity.This creates an evaluation setting in which training and deployment cryptographic domains can differ.
  • Results: Website identity remains learnable within either cryptographic domain, but cross-domain transfer degrades substantially for models trained on Non-PQC traffic.CUMUL, ET-BERT, and YaTC fall from 0.7731, 0.9316, and 1.0000 to 0.1919, 0.6255, and 0.3582, respectively, on Hybrid-PQC traffic.
  • Benchmark and evaluation: The benchmark pairs Non-PQC and Hybrid-PQC traffic while holding the label space and visit-level partitions fixed, enabling controlled cross-domain and deployment-ratio evaluation.It retains 195 websites and collects 120 complete visits per website and cryptographic configuration.
  • Results: Packet length is the strongest individual side-channel feature, yet no evaluated feature composition prevents cross-domain degradation.Direction and timing provide limited additional benefit, and packet-length advantage diminishes as Hybrid-PQC deployment increases.

B. Encrypted Traffic Classifiers: Representations and Learning Paradigms

Encrypted traffic classification uses observable traffic representations to infer labels without decrypting payloads, while distribution drift can arise from protocol evolution. Prior work spans statistical, sequence, byte-level, and learned representations, motivating explicit attention to protocol-induced shifts.

  • Classifier foundations: Encrypted traffic classifiers infer application, service, or activity labels from observable traffic without decrypting application payloads.Their effectiveness depends on both the learning architecture and the supplied representation.
  • Representations and learning paradigms: Existing methods use aggregate statistics, ordered packet sequences, raw bytes, or transferable self-supervised representations, trading off coverage, cost, interpretability, and data requirements.Pre-training is treated as a learning strategy layered onto byte-level or structured traffic inputs.
  • Distribution drift: Distribution drift is a mismatch between training and deployment traffic distributions that can arise from collection time, network paths, client platforms, services, data pipelines, or protocol stacks.The paper’s taxonomy distinguishes which sources are controlled or studied in its experiments.
  • Protocol drift: Protocol drift specifically changes the traffic-generation process through protocol evolution while labels, clients, vantage points, and collection pipelines remain fixed.Examples include TLS extensions, key-establishment algorithms, encrypted handshakes, and migration to QUIC or HTTP/3.

A. Comparison of TLS 1.3 Key-Establishment Modes

TLS 1.3 supports Traditional X25519, Hybrid-PQC X25519MLKEM768, and Pure-PQC MLKEM768 key establishment, which differ in exchanged key material and observable handshake effects.

  • Traditional (Non-PQC): Traditional TLS 1.3 uses ephemeral X25519 Diffie–Hellman, with 32-byte key shares exchanged in both directions.The resulting shared secret enters the TLS 1.3 key schedule.
  • Hybrid-PQC: Hybrid-PQC combines X25519 with ML-KEM-768, concatenating component key exchanges and shared secrets within one named group.X25519MLKEM768 uses 1216-byte client and 1120-byte server KeyShareEntries.
  • Pure-PQC: Pure-PQC removes the conventional Diffie–Hellman component and uses ML-KEM within the TLS 1.3 handshake flow.It remains a protocol-level comparison rather than part of the empirical evaluation.
  • Observable effects: 1216 and 1120 bytes are the Hybrid-PQC client and server KeyShareEntry sizes, compared with 32-byte Traditional entries.Larger handshake payloads can cross TLS-record or TCP-segment boundaries and increase observable packet counts.

B. PQC-Induced Changes Across Four Feature Dimensions

The paper represents each HTTPS visit with packet lengths, directions, timing, and selected packet bytes, then links Hybrid-PQC changes to shifts across these feature dimensions.

  • Traffic representation: Each website visit is represented as X = (L, D, T, P), separating compact side-channel sequences from selected byte-level packet representations.L denotes packet lengths, D directions, T timestamps or inter-arrival times, and P selected non-plaintext packet bytes.
  • Feature effects: The resulting shift is representation-level rather than semantic: encrypted application content retains its meaning while bytes may appear at new offsets or fall outside fixed windows.This can affect both compact sequence models and byte-oriented encoders.
  • Feature effects: Hybrid-PQC changes packet-length and byte representations directly through expanded key_share values, while direction and timing effects arise through segmentation, scheduling, and implementation.These effects can shift packet positions, burst structure, TLS-record boundaries, and byte alignment.

C. Hypotheses on Classifier Robustness under PQC-Induced Protocol Drift

The paper develops hypotheses that PQC-induced changes in packet lengths, directions, timing, and byte representations can challenge cross-domain classifier robustness during migration.

  • Robustness hypotheses: The hypotheses connect Hybrid-PQC handshake expansion to classifier degradation when models trained on Non-PQC traffic are evaluated on PQC traffic.Packet-length sequences and derived cumulative or statistical features are identified as primary inputs for this risk.
  • Robustness hypotheses: Hybrid-PQC may alter directional and temporal inputs through additional TCP segments, packet positions, acknowledgments, and ML-KEM processing or transmission overhead.The hypothesis concerns changes within the first N observations used by preprocessing.
  • Robustness hypotheses: Pre-trained byte-level representations may partially reduce cross-domain degradation by capturing reusable structures in P, but fixed input windows and preprocessing choices constrain that advantage.Expanded handshakes may cause ET-BERT or YaTC to omit contextual bytes within their input horizons.
  • Evaluation basis: The benchmark measures deployment and constructs paired traffic to study matched-domain, cross-domain, and deployment-ratio behavior under controlled conditions.Deployment analysis covers 2,060 Tranco-ranked websites, while the paired collection retains 195 common websites.

B. Constructing Paired Non-PQC and Hybrid-PQC Traffic Datasets

The benchmark pairs Traditional and Hybrid-PQC visits for common websites, validates negotiated groups, and keeps complete visits together across data splits.

  • Website selection: The study selected Top 200 websites supporting X25519MLKEM768, then retained 195 websites after validation for paired collection.The PQC dataset specifically uses Hybrid-PQC X25519MLKEM768 rather than Pure-PQC.
  • Paired collection: Firefox collected 120 complete page loads per website under each configuration by toggling Hybrid-PQC negotiation.This maintains balanced per-website sample counts across the two cryptographic domains.
  • Validation: Dataset membership was verified from the ServerHello key_share group: code point 4588 for Hybrid-PQC and 29 for Traditional X25519.This makes negotiated protocol state, rather than intended browser configuration alone, determine membership.
  • Data splitting: All TCP streams from a complete website visit were assigned to the same split, preventing related traffic from crossing partitions.The partitioning unit is the visit because page loads may span multiple TCP connections.

C. Constructing Deployment-Ratio Test Sets

The deployment-ratio test sets mix Non-PQC and Hybrid-PQC samples at controlled proportions while keeping labels and visit partitions fixed. This produces six settings for evaluating deployment composition without class-dependent skew.

  • The proportion α denotes the share of Hybrid-PQC samples in each test set, with the remaining share drawn from Non-PQC traffic.N_Hybrid-PQC and N_Non-PQC denote sample counts from the corresponding cryptographic domains.
  • Applying the same α to every website keeps cryptographic composition uniform across all 195 labels and prevents class-dependent deployment skew.Label space and visit-level partitioning remain fixed across settings, isolating test-domain composition effects.
  • The resulting test sets support the deployment-ratio and feature-control experiments in RQ3 and RQ4.

V. Evaluation

The evaluation studies five encrypted-traffic classifiers under PQC-induced protocol drift using a shared 195-website task, controlled observations, and four research questions. Models retain their native representations while excluding decrypted or out-of-band information.

  • Threat Model: Inputs are limited to passively observed encrypted packets, excluding DNS, decrypted handshake or application information, Tor, VPN tunneling, and active attacks.
  • Evaluated Classifiers: Five classifiers span aggregate statistics, packet sequences, timing-aware inputs, and pre-trained byte-level representations.The models cover packet length, direction, timing, and byte-level dimensions while preserving native inputs.
  • Evaluation Controls: All classifiers use the same closed-world HTTPS fingerprinting task over 195 websites and 120 complete visits per website and cryptographic domain.Visits are partitioned consistently at the complete-visit level, with identical assignments across models.
  • Evaluation Controls: The study preserves each model’s native feature extraction and preprocessing rather than imposing a common representation.RQ4 varies only DF’s input representation while holding its architecture and training procedure fixed.
  • Research Questions: The evaluation tests matched-domain learnability, bidirectional cross-domain transfer, deployment-ratio sensitivity, and side-channel feature contributions.

C. RQ1: Matched-Domain Learnability

Matched-domain evaluation finds that Hybrid-PQC traffic retains learnable website fingerprints. All five classifiers achieve comparable performance when training and testing remain within one cryptographic domain.

  • CUMUL records accuracies of 0.7731 and 0.7656, with F1 scores of 0.7598 and 0.7538, under N-N and P-P.
  • DF achieves accuracies of 0.5855 and 0.5900, while Tik-Tok achieves 0.5679 and 0.5737, under N-N and P-P.Their corresponding F1 scores are also similar across domains.
  • ET-BERT records accuracies of 0.9316 and 0.9332 under N-N and P-P, respectively.The pre-trained byte-level models attain the highest matched-domain scores within their native pipelines.
  • YaTC remains near perfect in matched-domain testing, decreasing only from 1.0000 under N-N to 0.9991 under P-P.
  • All five classifiers achieve comparable N-N and P-P performance, so Hybrid-PQC key establishment does not consistently reduce matched-domain learnability.Both settings use the same 195 labels, visit partitions, and model configurations.

D. RQ2: Bidirectional Cross-Domain Transfer

Cross-domain transfer is weaker than matched-domain performance and depends strongly on direction, while increasing Hybrid-PQC deployment generally reduces accuracy for models trained on Non-PQC traffic. Intermediate mixtures produce classifier-specific trajectories rather than uniform degradation.

  • Bidirectional Cross-Domain Transfer: Every classifier has lower cross-domain performance than its matched-domain baseline, and transfer degradation varies by classifier and direction.
  • Bidirectional Cross-Domain Transfer: Under N-P, CUMUL, DF, and Tik-Tok achieve accuracies of 0.1919, 0.3728, and 0.3873, respectively, below their N-N baselines.
  • Bidirectional Cross-Domain Transfer: Under N-P, YaTC declines from 1.0000 to 0.3582, while ET-BERT retains the highest accuracy at 0.6255.
  • Bidirectional Cross-Domain Transfer: P-N transfer is asymmetric: YaTC reaches 0.9709 accuracy, whereas CUMUL, DF, and Tik-Tok remain below 0.5000 accuracy.ET-BERT is the exception to the pattern that four classifiers perform better in P-N than N-P.
  • Deployment-Ratio Sensitivity: YaTC drops from 1.0000 at N-N to 0.6166 at N-M4, partially recovers to 0.7252 at N-M8, and falls to 0.3582 at N-P.
  • Deployment-Ratio Sensitivity: Increasing the Hybrid-PQC proportion generally reduces accuracy for Non-PQC-trained classifiers, although YaTC follows a non-monotonic trajectory.CUMUL, DF, Tik-Tok, and ET-BERT decline monotonically, while no classifier preserves its Non-PQC baseline across the full range.

F. RQ4: Side-Channel Feature Contributions

Packet length is the strongest individual side-channel across deployment ratios, but every tested representation degrades as Hybrid-PQC traffic increases. Combining direction, length, and timing does not provide cross-domain robustness.

  • Feature robustness under drift: All five representations show a monotonic accuracy decline from N-N to N-P as the test distribution shifts from Non-PQC to Hybrid-PQC.D falls from 0.5855 to 0.3728, L from 0.7944 to 0.4310, T from 0.5560 to 0.3766, DL from 0.8026 to 0.4186, and DLT from 0.7983 to 0.3899.
  • Individual side channels: Packet length is the strongest individual feature across all deployment ratios, although its advantage over direction narrows from 0.2089 under N-N to 0.0582 under N-P.Under N-N, L reaches 0.7944 versus 0.5855 for D and 0.5560 for T; under N-P, L reaches 0.4310.
  • Feature combinations: Feature combinations provide only marginal gains at low Hybrid-PQC proportions, and length alone outperforms DL and DLT from N-M4 onward.Under N-N, DL and DLT achieve 0.8026 and 0.7983, only slightly above L at 0.7944.
  • Feature combinations: Adding timing to the direction–length representation never improves accuracy, while direction and timing do not compensate for PQC-induced degradation.DLT never surpasses DL in any evaluated setting.
  • RQ4 conclusion: Feature fusion alone does not provide cross-domain robustness, despite packet length remaining the strongest individual signal.The RQ4 conclusion covers all deployment ratios and all five representations.

VI. Discussion

The discussion frames PQC-induced drift as a broader evaluation and modeling challenge: key-establishment-only experiments do not cover authentication-driven changes, and model scale alone may not solve cross-domain mismatch. Reliable assessment therefore requires protocol-aware cross-domain testing and attention to adaptive retraining and operational costs.

  • Beyond hybrid key establishment: The evaluation isolates PQC-induced drift at key establishment and does not empirically cover post-quantum signatures or certificate-chain effects.The paper notes that ML-DSA and Falcon can add kilobyte-scale authentication material that changes packetization, burst boundaries, and handshake timing.
  • Beyond hybrid key establishment: Authentication effects can create discontinuities in classifier inputs, motivating factorial evaluation that varies KEM and authentication mechanisms independently.TLS-record or TCP-segmentation thresholds can insert or reposition packets rather than merely increasing flow size.
  • Model scale and robustness: High matched-domain accuracy and cross-domain robustness can diverge substantially, so larger models may still rely on protocol-specific shortcuts.ET-BERT and YaTC illustrate this divergence; model capacity is useful only if representations remain stable across cryptographic environments.
  • Model scale and robustness: Fair model comparisons should include tokenization, pre-training corpora, truncation, adaptation data, bidirectional transfer, recovery costs, throughput, latency, and memory.The discussion treats these factors as part of the classifier and emphasizes the robustness–cost trade-off.
  • PQC-induced drift as incidental obfuscation: PQC evolution changes how website information appears rather than removing it, while fixed legacy classifiers may mistake model staleness for obfuscation.The paper identifies packet length as the dominant signal and limits this conclusion to the closed-world X25519MLKEM768 evaluation.
  • Evaluation implications: Classifier evaluation should treat protocol configurations as explicit dimensions, pair matched-domain accuracy with cross-domain tests, and use contemporary cryptographic deployments.The observed behavior may differ under other protocol stacks, configurations, or attack settings.
Loading 2608.22683v1…