Source-linked AI summary
Deep Fingerprinting: Undermining Website Fingerprinting Defenses with Deep Learning
Payap Sirinam, Mohsen Imani, Marc Juarez, Matthew Wright
TL;DR
Website fingerprinting can let local observers identify visited sites from encrypted Tor traffic, while lightweight defenses such as WTF-PAD and Walkie-Talkie were intended to reduce attack effectiveness. The paper develops and evaluates a CNN-based Deep Fingerprinting attack, finding strong performance against undefended traffic and WTF-PAD, while Walkie-Talkie remains more resistant.
Problem
Website fingerprinting threatens Tor’s protection against local observers, and the effectiveness of newer defenses against improved deep-learning attacks remained an open question.
Method
The paper develops Deep Fingerprinting, a CNN-based WF attack with a sophisticated architecture that avoids hand-crafted features, and evaluates it against Tor, WTF-PAD, and Walkie-Talkie.
Results
DF outperforms prior WF attacks, reaching over 90% accuracy against WTF-PAD, 49.7% against Walkie-Talkie, and 0.96 precision with 0.68 recall against WTF-PAD in the open world.
Takeaways & Limitations
The findings show that WTF-PAD is potentially vulnerable to deep-learning WF attacks, motivating further exploration of effective defenses deployable in Tor.
Takeaways & Limitations
Walkie-Talkie entails a 31% latency cost and requires browser context for selecting realistic cover pages, while adversarial defenses face live-trace manipulation constraints.
Abstract
from arXiv · showhide
Website fingerprinting enables a local eavesdropper to determine which websites a user is visiting over an encrypted connection. State-of-the-art website fingerprinting attacks have been shown to be effective even against Tor. Recently, lightweight website fingerprinting defenses for Tor have been proposed that substantially degrade existing attacks: WTF-PAD and Walkie-Talkie. In this work, we present Deep Fingerprinting (DF), a new website fingerprinting attack against Tor that leverages a type of deep learning called Convolutional Neural Networks (CNN) with a sophisticated architecture design, and we evaluate this attack against WTF-PAD and Walkie-Talkie. The DF attack attains over 98% accuracy on Tor traffic without defenses, better than all prior attacks, and it is also the only attack that is effective against WTF-PAD with over 90% accuracy. Walkie-Talkie remains effective, holding the attack to just 49.7% accuracy. In the more realistic open-world setting, our attack remains effective, with 0.99 precision and 0.94 recall on undefended traffic. Against traffic defended with WTF-PAD in this setting, the attack still can get 0.96 precision and 0.68 recall. These findings highlight the need for effective defenses that protect against this new attack and that could be deployed in Tor.
1 INTRODUCTION
Website fingerprinting lets a local passive adversary infer visited websites from encrypted Tor traffic. The paper introduces Deep Fingerprinting, a CNN-based attack that substantially improves performance against Tor and proposed defenses.
- Motivation: WF classifies websites by learning traffic features that distinguish encrypted traces.Prior attacks use hand-crafted features and classifiers such as SVM, k-NN, and random forests.
- Motivation: WTF-PAD and Walkie-Talkie add dummy packets or delays to hide traffic bursts and packet lengths.These defenses were proposed as practical, low-overhead countermeasures for Tor.
- Contributions: Deep Fingerprinting uses a CNN designed with deep-learning methods and does not require hand-crafted classification features.The evaluation uses a closed-world dataset containing 95 sites and 1,000 traces per site.
- Results: 98.3% accuracy makes DF more accurate against Tor than prior state-of-the-art attacks.This result concerns undefended Tor traffic in the closed-world setting.
- Results: 90% accuracy against WTF-PAD and 49.7% against Walkie-Talkie make DF the strongest evaluated attack in the defended closed-world experiments.Against Walkie-Talkie, 49.7% is described as nearly the theoretical maximum accuracy.
- Results: In the open world, DF achieves 0.99 precision and 0.94 recall on undefended traffic, and 0.95 precision and 0.70 recall against WTF-PAD.The study also examines attacks against weak Walkie-Talkie implementations and proposes directions for future attacks and defenses.
2 THREAT MODEL
The threat model considers a local, passive observer who records traffic between a Tor user and the entry node. The attacker trains on representative website traces and classifies new user traces, under both closed- and open-world assumptions.
- Attacker capabilities: A WF adversary captures packet sequences from representative websites, extracts distinguishing features, and trains a supervised classifier.The trained classifier maps features from new Tor traces to likely websites.
- Attacker capabilities: The attacker is local and passive, accessing only the link between the user and Tor’s entry node without modifying, delaying, dropping, or decrypting packets.Potential adversaries include local-network eavesdroppers, ISPs, autonomous systems, and entry-node operators.
- Attack scenario: The scenario assumes the adversary knows the client’s identity but aims only to identify the website.The client’s IP address can be obtained from the TLS connection to the entry node.
- Evaluation assumptions: Closed-world evaluation assumes a small site set for which the adversary has training samples, whereas open-world evaluation reflects the much larger space of potentially visited sites.The paper treats the open world as more realistic because collecting data for every possible site is infeasible.
- Evaluation assumptions: The evaluation focuses on homepage fingerprinting and assumes that visit traffic can be isolated from other traffic.The latter assumption may require an entry node under the adversary’s control or traffic-parsing techniques.
3 BACKGROUND AND RELATED WORK
Prior WF attacks progressed from low accuracy to roughly 90% using engineered features and increasingly practical classifiers, while lightweight defenses reduced attack accuracy. The paper positions DF as a deeper CNN intended to close this gap, especially against defenses and in open-world settings.
- WF attacks: Earlier WF attacks improved from 3% to 55% and then about 90% closed-world accuracy on Tor, though early high-performing methods were computationally costly.Later attacks maintained about 90% accuracy while reducing attack cost.
- WF attacks: CUMUL uses cumulative packet-length features with an SVM, while k-NN combines packet ordering, cell counts, and burst counts into a distance metric.These methods illustrate the field’s reliance on hand-crafted traffic representations.
- WF defenses: WF defenses add dummy packets or delays to make traffic features less distinctive, beginning with BuFLO’s constant-rate strategy.WTF-PAD and Walkie-Talkie were proposed as lightweight, low-latency defenses for Tor.
- WF defenses: WTF-PAD uses adaptive padding to mask traffic bursts, whereas Walkie-Talkie uses half-duplex communication, alternating bursts, dummy packets, and delays.Prior evaluations reduced attack accuracy against these defenses to below 30%.
- Deep-learning attacks: Deep-learning WF studies showed that SDAE and CNN models can automate feature extraction, with prior CNN results reaching 96.3% closed-world accuracy.Earlier reports did not achieve the accuracy reported here or demonstrate effectiveness against WTF-PAD.
- Deep-learning attacks: DF uses a more sophisticated CNN than AWF, including deeper convolutional processing, overfitting protections, depth-varying hyperparameters, tailored activations, and a two-layer classifier.The paper argues these architectural differences produce a deeper and more effective network, particularly against defenses and in open-world evaluation.
4 DATA COLLECTION
The study constructs closed-world, open-world, and defended Tor traffic datasets using realistic browser crawls, filtering corrupted visits before evaluation. The closed-world dataset contains 95 sites with 1,000 usable visits each, while the open-world collection contains 40,716 traces.
- Closed-world dataset: Closed-world traces were collected from top Alexa sites using tcpdump across ten low-end university machines.Visits were sequential and batched to control for long- and short-term time variance.
- Collection procedure: tor-browser-crawler drove Tor Browser visits to approximate real user browsing more closely than wget or curl.The authors note that the crawler did not model user browsing behavior because relevant Tor user statistics are unavailable.
- Closed-world dataset: 95 sites remained after removing corrupted traces and retaining only classes with at least 1,000 visits.Traces with no packets or fewer than 50 packets were discarded.
- Open-world dataset: The open-world crawl sampled sites from Alexa’s top 50,000, excluding the closed-world sites, and visited each remaining site once.Screenshots were used to remove blank, access-denied, CAPTCHA, and timeout pages; the final dataset had 40,716 traces.
- Defended datasets: Defended datasets were generated by applying BuFLO, Tamaraw, and WTF-PAD protocols or simulators, while Walkie-Talkie required a new half-duplex Tor Browser crawl.Walkie-Talkie could not be completely simulated because half-duplex communication is difficult to model.
5 EXPERIMENTAL EVALUATION
The evaluation compares DF with prior attacks across non-defended and defended Tor traffic, using a CNN architecture designed for traffic data. DF achieves strong classification performance, including 98.3% accuracy on non-defended traffic and improved performance over AWF.
- DF architecture: DF uses a one-dimensional CNN adapted to traffic vectors, including activation functions that preserve information from negative packet-direction values.Image filters and pooling are converted to one dimension because traffic data is represented as a vector.
- DF architecture: DF places batch normalization after convolutional layers and dropout after activation functions to mitigate overfitting.The authors report no evidence of overfitting in their experiments.
- DF architecture: DF adds fully connected layers between convolutional feature extraction and prediction, combining them with batch normalization and dropout.These layers are intended to support the learning and classification process while limiting overfitting.
- 98.3% accuracy: DF outperforms other evaluated attacks on the non-defended Tor dataset.The comparison includes AWF, CUMUL, k-FP, k-NN, and SDAE.
- Training effects: With 50 traces per site, DF and CUMUL achieve 90% accuracy, while k-NN, k-FP, and AWF require 250 traces.SDAE requires 750 traces to reach 90%, and accuracies mostly saturate after 550 traces except for SDAE.
5.5 Training Cost
GPU acceleration makes the deep-learning WF models’ training times relatively reasonable, while traditional attacks can require substantially longer training. The defended-traffic evaluation also shows DF’s strong performance against WTF-PAD and Walkie-Talkie, alongside substantial overhead for some defenses.
- Training Cost: 64 minutes for 30-epoch training, DF takes longer than SDAE’s 16 minutes and AWF’s 4 minutes with GPU acceleration.Without a GPU, DF requires approximately 10 hours, compared with 96 minutes for SDAE and 1 hour for AWF.
- Training Cost: 12.5 hours for k-NN and 57 hours for CUMUL exceed the training times of the deep-learning attacks.k-FP requires 1 hour, while GPU-accelerated SDAE, DF, and AWF require 16, 64, and 4 minutes, respectively.
- Defended-Dataset Evaluation: 90% accuracy against WTF-PAD makes DF substantially more effective than the other attacks evaluated on that defense.The evaluation reports at most 49.70% accuracy against Walkie-Talkie and over 90% accuracy for DF against WTF-PAD.
- Defended-Dataset Evaluation: 246% and 328% bandwidth overheads for BuFLO and Tamaraw, respectively, make both defenses impractical to deploy in Tor.The reported overheads are higher than in prior work, and Tamaraw’s overhead exceeds BuFLO’s in this dataset.
- Open-World Evaluation: 0.96 precision and 0.68 recall are achieved by DF against WTF-PAD when tuned for high precision in the open-world setting.When tuned for high recall, DF reaches 0.67 precision and 0.96 recall.
6 DISCUSSION
The discussion examines DF’s performance against defended Tor traffic, practical deployment trade-offs, limitations, and directions for stronger attacks and defenses.
- Improving Open-World Classification: Open-world DF performance may improve through architecture tuning, more training data, or timing features, but current design choices depend on environment and input data.The paper specifically suggests varying convolutional depth, filter sizes, dropout, and input statistics.
- Attack Costs: Large-scale attack data collection requires multiple PCs running for several days, while stale data and poor scalability constrain weaker attackers.The paper reports that WF accuracy can decline significantly after 10–14 days and that CUMUL took days to train because of hyperparameter search.
- WTF-PAD: DF can detect traffic patterns that remain after WTF-PAD padding, despite adaptive padding targeting timing gaps between bursts.The discussion attributes WTF-PAD’s failure against DF to residual patterns in defended traces.
- Walkie-Talkie: Walkie-Talkie performs better than WTF-PAD against DF because it directly targets website-fingerprinting features and explicitly seeks collisions.The paper presents Walkie-Talkie as a stronger defense candidate against DF, while noting deployment downsides.
- Walkie-Talkie: Walkie-Talkie requires a maintained website-pattern database, adds 31% fetching latency, and demands browser context for realistic cover-page selection.The system also risks exposure when sensitive pages lack suitable decoys, while high Top-2 accuracy can support auxiliary-information attacks.
- Alternative Defenses: Adversarial machine learning is a potential defense direction, but live traces and Tor’s inability to delete or accelerate packets make adaptation difficult.Tor can add dummy packets and delay packets, but cannot manipulate traces as freely as image-processing systems.
7 CONCLUSION
The paper evaluates deep-learning website fingerprinting in closed- and open-world settings and finds that DF remains effective against undefended and WTF-PAD-protected Tor traffic. The results motivate more robust defenses and further DF development.
- 7 CONCLUSION: DF uses a CNN-based architecture for feature extraction and classification in closed- and open-world website-fingerprinting experiments.The study reports a first open-world evaluation of deep-learning attacks against defended traffic.
- 7 CONCLUSION: DF outperforms prior website-fingerprinting attacks in closed-world experiments, including better than 90% accuracy against WTF-PAD.The conclusion also reports open-world metrics for undefended and WTF-PAD-protected traffic.
- 7 CONCLUSION: 0.99 precision and 0.94 recall on undefended traffic fall to 0.96 precision and 0.68 recall against WTF-PAD in the open world.These are the conclusion’s reported open-world results.
- 7 CONCLUSION: The study identifies a need for more robust website-fingerprinting defenses against deep-learning attacks and outlines directions for improving DF.The paper frames these as future research directions rather than completed solutions.
A DEEP FINGERPRINTING (DF) MODEL’S ARCHITECTURE FOR WF ATTACKS
The DF architecture uses a CNN with a simple one-dimensional packet-direction input and design choices intended to improve classification speed, generalization, and reproducibility.
- Architecture Overview: The paper provides technical details, hyperparameters, and implementation guidance for its CNN-based DF model to support future application and reproduction.Figure 8 presents the architecture referenced by these implementation details.
- Input Data: The model takes a 1 × 5,000 vector of packet directions as input rather than a two-dimensional image-like matrix.The authors report that one-dimensional input was slightly more accurate and significantly faster to train than two-dimensional input.
- Convolutional Layers: Each DF block combines convolutional layers, batch normalization, max pooling, and dropout for feature extraction.The first block is especially important because it is closest to the input.
- Activation Functions: ELU is used in the first two convolutional layers, followed by ReLU in the remaining convolutional layers, based on input characteristics and tuning results.The choice targets negative inputs and classification performance.
- Dropout Regularization: Dropout is placed after max pooling in feature-extraction blocks and between feature extraction and classification to mitigate overfitting.The architecture uses different dropout rates across these locations.
- Batch Normalization: Batch normalization and dropout together improved both performance and generalization while accelerating model learning.The paper notes that batch normalization also partially functions as a regularizer, with additional computational cost.
B ATTACK PERFORMANCE METRICS
Website-fingerprinting attacks are evaluated in two scenarios: closed world and open world.
- Evaluation Scenarios: Website-fingerprinting evaluation uses both closed-world and open-world scenarios.The section introduces these as the two settings under which attack success is evaluated.
- Evaluation Metrics: The metrics section defines the measures used to evaluate whether website-fingerprinting attacks succeed.The supplied passage introduces metric definitions without specifying their individual formulas.
- Evaluation Scenarios: Closed-world and open-world settings provide the evaluation framework for comparing website-fingerprinting attacks.The passage identifies both scenarios as the organizational basis for the evaluation.
B.1 Closed-world Evaluation
The closed-world evaluation assumes a fixed, known website set and measures attack success by the fraction of correctly classified traces. Correct predictions require matching each test trace to its website label.
- The attacker trains and evaluates on a fixed set of websites assumed to be known in advance.
- Accuracy is the ratio of correctly classified traces to the total number of test traces.
- A prediction is correct when the classifier’s output matches the website label of the test trace.
B.2 Open-world Evaluation
The open-world evaluation allows visits to many websites, so the attacker trains on a smaller monitored set and models the remainder as unmonitored. Precision and recall characterize performance in this setting.
- The attacker selects a relatively small monitored set because effective training across the full Web is impractical.
- Websites outside the monitored set form an unmonitored set that the attacker does not try to identify.
- Open-world performance is measured with precision and recall, using monitored and unmonitored classifications to define TP, TN, FP, and FN.
C OPEN-WORLD ROC CURVE
The open-world ROC evaluation compares website-fingerprinting attacks across non-defended, WTF-PAD, and Walkie-Talkie traffic. ROC curves expose the trade-off between true-positive and false-positive rates.
- ROC curves are plotted for all attacks across non-defended, WTF-PAD, and Walkie-Talkie datasets.
- The ROC curve evaluates classifiers by showing the trade-off between TPR and FPR.
- For DF on non-defended traffic, one operating point achieves 0.98 TPR and 0.03 FPR when optimizing for high TPR.
- For DF on non-defended traffic, another operating point achieves 0.94 TPR and 0.004 FPR when optimizing for low FPR.
- Figure 9 presents ROC curves for the open-world scenario.