PaperlayerView exact revision on arXiv

Source-linked AI summary

LUCID: A Practical, Lightweight Deep Learning Solution for DDoS Attack Detection

Roberto Doriguzzi-Corin, Stuart Millar, Sandra Scott-Hayward, Jesus Martinez-del-Rincon, Domenico Siracusa

arXiv:2002.04902v2cs.CRcs.NI

TL;DR

DDoS detection must handle diverse attacks and high-volume live traffic while supporting essential-service availability. LUCID uses a lightweight CNN with online-oriented, dataset-agnostic preprocessing and activation analysis, then evaluates the system on constrained hardware. It matches state-of-the-art detection accuracy, reduces processing time 40x, and demonstrates suitability for resource-constrained environments, although memory can become a bottleneck for large time windows.

  • Problem

    DDoS attacks threaten service availability, while detection must cope with diverse attack approaches and the volume of live traffic to analyze.

  • Method

    LUCID uses a CNN to classify traffic flows as malicious or benign, with dataset-agnostic preprocessing for online observations and activation analysis for explaining classifications.

  • Results

    LUCID matches existing state-of-the-art detection performance and demonstrates a 40x improvement in processing time over similar state-of-the-art solutions.

  • Takeaways & Limitations

    Evaluation on a resource-constrained device demonstrates LUCID's suitability for deployment in resource-constrained environments.

  • Takeaways & Limitations

    Memory resources can become the bottleneck in edge scenarios, with a 100 time window requiring approximately 20 GB RAM for the reported processing setup.

Abstract

from arXiv · show

Distributed Denial of Service (DDoS) attacks are one of the most harmful threats in today's Internet, disrupting the availability of essential services. The challenge of DDoS detection is the combination of attack approaches coupled with the volume of live traffic to be analysed. In this paper, we present a practical, lightweight deep learning DDoS detection system called LUCID, which exploits the properties of Convolutional Neural Networks (CNNs) to classify traffic flows as either malicious or benign. We make four main contributions; (1) an innovative application of a CNN to detect DDoS traffic with low processing overhead, (2) a dataset-agnostic preprocessing mechanism to produce traffic observations for online attack detection, (3) an activation analysis to explain LUCID's DDoS classification, and (4) an empirical validation of the solution on a resource-constrained hardware platform. Using the latest datasets, LUCID matches existing state-of-the-art detection accuracy whilst presenting a 40x reduction in processing time, as compared to the state-of-the-art. With our evaluation results, we prove that the proposed approach is suitable for effective DDoS detection in resource-constrained operational environments.

I. INTRODUCTION

DDoS attacks increasingly combine diverse techniques and threaten service availability, while existing defences remain inadequate. LUCID addresses this gap with a lightweight CNN, online-oriented preprocessing, interpretability analysis, and validation on constrained hardware.

  • DDoS attacks disrupt essential services, yet network defences remain inadequate despite decades of research.
  • Multi-vector attacks and changing techniques require more robust detection than signature-based and threshold-dependent statistical systems provide.
  • LUCID is a CNN-based architecture that learns malicious and benign traffic behaviour with low processing overhead and detection time.
  • Its dataset-agnostic preprocessing produces traffic observations matching online systems that analyse flow segments collected over predefined windows.
  • The paper analyzes CNN kernel activations for feature-level explanations and validates LUCID on resource-constrained hardware for edge-computing deployment.

A. Statistical approaches to DDoS detection

Statistical and machine-learning approaches address DDoS detection but face threshold-selection, online-observation, deployment, and stability challenges. Deep learning improves discrimination in some studies, while existing CNN and hybrid systems may remain resource-intensive or inconsistent across datasets.

  • Entropy-based detection relies on thresholds whose appropriate values vary across traffic types, volumes, networks, and attack scenarios.
  • Traffic-distribution monitoring alone may not sufficiently distinguish benign from malicious traffic.
  • Some flow-level statistics cannot be computed from traffic collected within observation windows, limiting suitability for online detection.
  • Machine-learning adoption in real-world network intrusion detection remains limited by high error costs and traffic variability.
  • Deep-learning studies report improved or higher detection performance, but prior systems include large resource-intensive models and CNN results with significant cross-dataset variance.
  • A DDoS-specific CNN feature-extraction approach achieved 87.35% accuracy on the CAIDA DDoS attack dataset.

III. METHODOLOGY

LUCID preprocesses live traffic into fixed-length, normalized packet-feature samples organized by time windows and bidirectional flows. This representation supports online detection while preserving flow labels and controlling sample size through the time-window and packet-count parameters.

  • A. Network Traffic preprocessing: Each sample contains 11 attributes, while IP addresses, transport ports, link-layer encapsulation, and application-layer attributes are excluded to improve generalization.The selected attributes include packet timing, protocol-layer information, and protocol representations.
  • A. Network Traffic preprocessing: LUCID converts packets from each bidirectional flow into [n, f] examples using 11 packet features collected within time windows.Flows longer than n packets are truncated, while shorter flows are zero-padded after normalization.
  • A. Network Traffic preprocessing: The preprocessing algorithm groups packets chronologically by flow identifier and time window, then outputs normalized, padded, labelled samples.Labels are assigned by matching each sample's flow identifier with the original dataset labels.
  • A. Network Traffic preprocessing: Shorter time windows create more samples by fragmenting flows into smaller pieces, whereas the value of n has negligible impact on the final sample count.The authors evaluate t and n as CNN hyper-parameters affecting online detection and available flow information.
  • A. Network Traffic preprocessing: The resulting array E organizes samples by time-window rows and bidirectional-flow columns, with empty elements indicating no captured packets.Non-empty samples contain packet attributes, and zero-padding supplies fixed length for CNN processing.

B. LUCID Model Architecture

LUCID uses a lightweight CNN that receives each traffic flow as an n × f matrix of packet features and classifies it as malicious or benign. Convolution and pooling learn compact flow representations before a sigmoid output produces the DDoS probability.

  • B. LUCID Model Architecture: LUCID classifies traffic flows as malicious DDoS or benign while minimizing CNN complexity and processing time for resource-constrained deployment.The architecture uses shared and reused convolutional weights to reduce model size and memory requirements.
  • B. LUCID Model Architecture: The input layer represents each flow as an n × f matrix F containing n packet vectors with f = 11 features.This spatial representation allows the CNN to learn correlations between packets belonging to the same flow.
  • B. LUCID Model Architecture: A single convolutional layer applies k filters of size h × f across F to learn local features useful for distinguishing DDoS and benign flows.Each filter uses stride 1 and generates an activation map of size n − h + 1.
  • B. LUCID Model Architecture: Max pooling retains the largest activations from each learned filter, discards positional information, and produces a compressed feature vector for classification.Pooling reduces network complexity by retaining stronger activations while removing less useful responses.
  • B. LUCID Model Architecture: The sigmoid output returns a malicious-flow probability p ∈ [0, 1], classifying the flow as DDoS when p > 0.5 and benign otherwise.The output node receives the flattened pooled representation through a fully connected layer.

C. The Learning Procedure

LUCID learns from labelled, balanced traffic examples by minimizing binary cross-entropy between predicted malicious-flow probabilities and ground-truth labels. This supervised procedure avoids requiring extensive expert-crafted feature engineering.

  • C. The Learning Procedure: LUCID minimizes a cost function by iteratively updating the model's trainable weights and biases to reduce prediction error.The cost compares each model prediction with the corresponding ground-truth label.
  • C. The Learning Procedure: Binary cross-entropy is used because the detection problem has two classes, DDoS and benign.The loss is computed over batches of samples using ground-truth labels and predicted malicious-flow probabilities.
  • C. The Learning Procedure: Training uses balanced datasets with equal numbers of malicious and benign flows to reduce bias in learning traffic representations.The labels provide supervised targets for distinguishing DDoS from benign traffic.
  • C. The Learning Procedure: The learned representation reduces the need for significant expert input to craft bespoke features and assess their statistical importance.Feature learning is encapsulated in the CNN training process.

IV. EXPERIMENTAL SETUP

The evaluation uses three publicly available UNB datasets containing normal and malicious traffic, including DDoS attacks. The experimental design balances flows, splits data per flow, and tests whether mixed-dataset training improves performance on unseen data.

  • IV. EXPERIMENTAL SETUP: LUCID is validated on the ISCX2012, CIC2017, and CSECIC2018 datasets from the Canadian Institute for Cybersecurity.The datasets contain several days of normal and malicious network activity and are publicly available as pcap traffic traces with labels and statistics.
  • IV. EXPERIMENTAL SETUP: The datasets model normal application and protocol activity alongside attack scenarios generated from recent security reports.Normal profiles cover HTTP, SMTP, SSH, IMAP, POP3, and FTP activities.
  • IV. EXPERIMENTAL SETUP: The selected traces contain IRC-botnet, LOIC-generated HTTP, and HOIC-generated HTTP DDoS attacks across ISCX2012, CIC2017, and CSECIC2018.The CIC2017 trace is restricted to the specified 3.30PM–5.00PM timeslot.
  • IV. EXPERIMENTAL SETUP: Training on ISCX2012 and testing on CIC2017 produced a false negative rate of about 17%, illustrating that cross-dataset performance can degrade for unseen attack distributions.The authors attribute this result to differences between the attacks represented in the datasets.
  • IV. EXPERIMENTAL SETUP: The combined dataset contains balanced benign and DDoS flows from all three datasets, with each dataset split into 90% training and 10% testing data.Ten percent of the training portion is used for validation, and splitting is performed per flow.
  • IV. EXPERIMENTAL SETUP: Shorter time windows produce larger sample counts because flows are divided into more fragments, while n has negligible impact on the final number of samples.Table IV summarizes the resulting UNB201X splits as a function of time-window duration t.

C. Evaluation methodology

LUCID is evaluated with standard classification metrics, F1-focused hyper-parameter tuning, and unseen test data. The CNN represents flows as packet arrays and balances accuracy against computational and memory demands.

  • Metrics: Accuracy, FPR, PPV, TPR, and F1 measure classification performance, with F1 receiving primary emphasis.F1 summarizes PPV and TPR; TP, TN, FP, and FN define the underlying outcomes.
  • Tuning: Hyper-parameters are selected through grid search using validation-set F1, while the test set remains completely unseen.The search explores 2835 combinations, and tuning avoids artificially improving test performance.
  • Traffic representation: Each sample records up to n packets chronologically within a time window, preserving packet attributes and relative timing.The representation captures a portion of a flow’s life and supports temporal-pattern characterization.
  • Hyper-parameter effects: F1 stabilizes once n ≥5 but continues increasing through n = 100, while smaller n reduces convolutions, storage, and RAM requirements.The selected setting balances high accuracy with low resource consumption.
  • Hyper-parameter effects: For n > 1, the CNN is relatively insensitive to time-window duration t; n = 1 instead depends more on training-sample count.With n = 1, each sample contains only one packet, preventing packet-attribute correlation within a sample.
  • Final configuration: The selected configuration is n = 100, t = 100, k = 64, h = 3, and m = 98, yielding 2241 trainable parameters.Using k = 32 or smaller n values can reduce resource requirements with a minimal F1 decrease.

V. RESULTS

The evaluation measures LUCID against state-of-the-art models using standard detection metrics. These metrics support both performance measurement and comparative analysis.

  • Evaluation metrics: LUCID is evaluated using Accuracy, False Positive Rate, Precision, Recall, and F1 Score.The same metrics are used to compare LUCID with state-of-the-art approaches.

A. Detection accuracy

LUCID maintains high performance on unseen traffic across multiple datasets and achieves F1 scores comparable to DeepDefense while processing samples substantially faster. Comparisons also examine other DDoS detection approaches on UNB datasets.

  • Detection accuracy: LUCID maintains very high detection performance across test datasets of unseen benign and DDoS flows.The results indicate robustness across the evaluated test sets.
  • Detection accuracy: CNN-based representations recognize malicious patterns regardless of their position within a traffic flow.The same output representation is produced whether the DDoS event appears at the start or end of the input.
  • State-of-the-art comparison: LUCID is compared with approaches validated on the UNB datasets, including methods evaluated on ISCX2012 and CIC2017.The paper notes that some prior results do not specify whether validation or test data were used.
  • DeepDefense comparison: LUCID and 3LSTM achieve comparable F1 scores across the evaluated test datasets.The comparison uses n = 100 and t = 100 for both deep-learning models.
  • DeepDefense comparison: More than 55000 samples/sec versus 1300 samples/sec: LUCID processes over 40 times as many samples as 3LSTM on average.LUCID’s smaller number of hidden units and trainable parameters contributes to lower computational complexity.
  • State-of-the-art comparison: LUCID’s ISCX2012 comparison includes 3LSTM and other approaches, while no previously validated DDoS detection solution is identified for CSECIC2018.The authors attribute differences between 3LSTM implementations to distinct pcap preprocessing mechanisms.

C. Discussion

The discussion attributes LUCID’s results to packet-level traffic attributes and notes that its combined-dataset tuning still transfers to unseen test data. Flow-level approaches are described as more prone to false negatives, with a CNN-RNN exception.

  • Discussion: Packet-level traffic attributes produce higher classification accuracy than flow-level features or entropy information in the discussed evaluations.The comparison includes LUCID, the authors’ DeepDefense implementation, and TR-IDS.
  • Discussion: Flow-level approaches such as E3ML, DeepGFL, and most models in are more prone to false negatives, except 1D-CNN+LSTM.The exception combines CNN and RNN layers and produces a high TPR.
  • Discussion: LUCID is tuned on a combined-dataset validation portion rather than individual datasets and still outperforms state-of-the-art approaches on unseen test data.This evaluation design tests transfer beyond dataset-specific tuning.

VI. ANALYSIS

LUCID’s activation analysis examines which traffic features drive its DDoS classifications and finds evidence that the CNN learns relevant domain information. The analysis also identifies possible extensions and adversarial-robustness concerns.

  • Kernel activations: Kernel activation analysis removes the classifier and measures convolutional activations to identify features emphasized by LUCID.Because each kernel matches the input width, activations can be captured per kernel and aggregated across kernels.
  • Kernel activations: Table X ranks 11 features by post-ReLU average column-wise activation sums and highlights the two most strongly activating features.The ranking is computed across all CNN kernels for the UNB201X dataset.
  • Highest Layer: The highest layer may indicate whether a flow is a network-, transport-, or application-layer DDoS attack.The authors propose extending dataset labels to predict attack type and support selection of a protection mechanism, leaving this for future work.
  • IP Flags: IP Flags rank second because attack and benign traffic exhibit different distributions of the Don’t Fragment flag.The passage reports 99.99% of DDoS packets with value 0x4000 versus about 92% of benign packets with Don’t Fragment set.
  • Future Directions: No definitive attack-agnostic feature list exists for direct comparison because salient features vary across attack classes and scenarios.The authors conclude that LUCID appears to learn relevant features, while noting that adversarially robust defenses remain an open problem.

VII. USE-CASE: DDOS DETECTION AT THE EDGE

The edge use case evaluates whether LUCID can operate on resource-constrained devices near IoT infrastructure. Results indicate usable processing and memory behavior, but memory can become the bottleneck as traffic-window demands rise.

  • Motivation: Edge nodes can detect and block DDoS traffic near compromised IoT devices, but their limited computing and memory resources constrain available defenses.LUCID is evaluated as an edge-deployable alternative on an NVIDIA Jetson TX2 board.
  • Detection: The experiments measure samples processed per second with and without GPU support on the UNB201X test set.Batch size affects detection speed by changing parallel processing and memory-read requirements.
  • Detection: Approximately 1.9 Mpps can be processed without GPU support using a batch size of 1024.Each sample contains up to 100 packets represented by 11 features.
  • Memory: Approximately 20 GB RAM would be required for a 100 s window at around 23K GPU-processed samples per second, exceeding typical edge-node memory.The authors identify memory resources as a potential bottleneck in edge computing scenarios.
  • Memory: With 1 GB RAM, approximately 100K samples can be stored, and setting t and n to 10 can reduce memory requirements by a factor of 100.The reported capacity excludes memory used by the operating system and applications; lower t and n may carry a detection-accuracy cost.
  • Conclusion: The measurements support LUCID’s usability on resource-constrained platforms, while requiring a trade-off between resource usage and detection accuracy.The overall resource balance includes traffic collection and preprocessing, whose speed was not optimized in these experiments.

B. Training time

Local retraining can avoid sending large volumes of edge data to remote facilities, but training time determines how long updated defenses remain unavailable. LUCID trains substantially faster than the compared DeepDefense implementation under the reported conditions.

  • Training time: Local edge retraining avoids the latency, bandwidth, and security concerns associated with sending training data to remote facilities.The practical challenge is controlling training time while the node remains exposed before updated parameters are available.
  • Training time: LUCID’s convergence training time is measured on the Jetson development board with and without GPU support and compared with a server.The experiments use UNB201X training and validation sets with validation-loss-based termination.
  • Training time: Around 2 hours without GPU support was approximately 4 times slower than server training but 40 times faster than DeepDefense 3LSTM under the same testing conditions.The reported board result corresponds to 184 epochs, while DeepDefense exceeded 1000 sec/epoch with GPU support.
  • Training time: Early termination or smaller training and validation sets can reduce convergence time, but may lower detection accuracy.The authors frame the deployment choice as a trade-off between convergence time and detection accuracy.
  • Conclusion: LUCID is designed as a lightweight CNN-based detector with low processing overhead and attack detection time.The conclusion also describes its practical deployment target and contrasts it with threshold-based statistical detection and feature-engineering-heavy alternatives.
  • Conclusion: LUCID matches state-of-the-art performance, shows consistent detection across datasets, and achieves a 40x processing-time improvement over similar solutions.The conclusion also reports resource-constrained-device suitability and activation analysis explaining how the model learns to detect DDoS traffic.
Loading 2002.04902v2…