Source-linked AI summary
Analyzing Adversarial Attacks Against Deep Learning for Intrusion Detection in IoT Networks
Olakunle Ibitoye, Omair Shafiq, Ashraf Matrawy
TL;DR
The paper addresses the limited evidence on adversarial robustness of deep-learning intrusion detection in IoT networks. It compares FNN and SNN detectors using IoT data and adversarial samples, finding that FNNs perform better on standard metrics while SNNs are more resilient to adversarial attacks.
Problem
The study addresses the unknown performance and adversarial resilience of SNNs for intrusion detection in IoT networks.
Method
The authors implement FNN- and SNN-based IoT intrusion detection systems and evaluate them against FGSM, BIM, and PGD white-box evasion samples.
Results
FNNs achieve better standard intrusion-detection performance, whereas SNNs show greater resilience to gradient-based adversarial samples.
Takeaways & Limitations
Adversarial samples are a real threat to deep-learning intrusion detection in IoT, while SNN self-normalization is associated with greater resilience.
Takeaways & Limitations
The study leaves the reason for SNNs' greater adversarial resilience for future investigation.
Abstract
from arXiv · showhide
Adversarial attacks have been widely studied in the field of computer vision but their impact on network security applications remains an area of open research. As IoT, 5G and AI continue to converge to realize the promise of the fourth industrial revolution (Industry 4.0), security incidents and events on IoT networks have increased. Deep learning techniques are being applied to detect and mitigate many of such security threats against IoT networks. Feedforward Neural Networks (FNN) have been widely used for classifying intrusion attacks in IoT networks. In this paper, we consider a variant of the FNN known as the Self-normalizing Neural Network (SNN) and compare its performance with the FNN for classifying intrusion attacks in an IoT network. Our analysis is performed using the BoT-IoT dataset from the Cyber Range Lab of the center of UNSW Canberra Cyber. In our experimental results, the FNN outperforms the SNN for intrusion detection in IoT networks based on multiple performance metrics such as accuracy, precision, and recall as well as multi-classification metrics such as Cohen's Kappa score. However, when tested for adversarial robustness, the SNN demonstrates better resilience against the adversarial samples from the IoT dataset, presenting a promising future in the quest for safer and more secure deep learning in IoT networks.
I. INTRODUCTION
The paper examines adversarial vulnerability in deep-learning intrusion detection for IoT networks, focusing on whether Self-normalizing Neural Networks can improve resilience relative to Feedforward Neural Networks.
- IoT security risks are expected to grow, while traditional cyber-threat protections may not address IoT's unique vulnerabilities.
- The study focuses on deep-learning intrusion detection systems for detecting and classifying network traffic in IoT environments.
- Deep-learning intrusion detection can reduce reliance on manual feature selection, but its security use is challenged by limited transparency and adversarial vulnerability.
- Adversarial examples intentionally perturb input features to confuse machine-learning models into producing incorrect predictions, including in intrusion detection.
- The paper introduces SNNs as stable during gradient descent and studies their previously unexamined resilience to adversarial attacks in IoT intrusion detection.
- The authors compare FNN and SNN intrusion detectors, finding stronger conventional performance for FNNs but greater adversarial robustness for SNNs.
- Input-feature normalization improves performance metrics but adversely affects deep-learning IDS resistance to adversarial attacks.
II. RELATED WORK
Prior work applied deep learning to intrusion detection and adversarial analysis, but did not establish SNN resilience for IoT intrusion detection. This study addresses that gap using a larger, more heterogeneous IoT setting.
- Earlier IoT dataset work used LSTM, SVM, and RNN models but did not evaluate adversarial robustness and performed only binary classification.
- A prior IoT intrusion study used only 2,313 training, 496 validation, and 496 test samples containing DDoS/DoS and normal traffic.
- The present study uses over 3.6 million records and a more heterogeneous attack profile than the limited dataset described in prior work.
- Prior adversarial analysis found an FNN intrusion detector's accuracy could decline from 93% to as low as 24% on adversarial samples, using the older NSL-KDD dataset.
- SNNs are FNN variants using SELU activation functions, proposed to address gradient decay in deep FNN architectures.
- The performance and adversarial resilience of SNNs for IoT intrusion detection had not previously been evaluated, defining this study's research gap.
IV. EXPERIMENTAL APPROACH
The experimental approach implements IoT intrusion detection systems, generates adversarial samples, and evaluates model resilience under white-box evasion attacks during prediction.
- The study implements deep-learning intrusion detection systems for an IoT dataset and tests their resilience to adversarial samples.
- Adversarial samples are generated from the dataset used to train the deep-learning models.
- The experiments use FGSM, BIM, and PGD to craft adversarial samples.
- The attacks are modeled as white-box evasion attacks launched during the model's prediction phase, assuming complete knowledge of the model.
- The implementation uses Python notebooks hosted in Google Colaboratory, which provides cloud execution and GPU features.
B. Dataset
The study uses the BoT-IoT dataset, a simulated IoT network collection with heterogeneous network profiles, scaled to approximately 3.6 million records. The task distinguishes normal traffic from four attack classes.
- The BoT-IoT dataset was created in a dedicated IoT environment to provide realistic network activity with heterogeneous profiles.
- Approximately 3.6 million records from a scaled-down version of the dataset were used for the study.The full dataset contains over 72 million network-activity records.
- The training and test sets use five output classes representing normal traffic and four IoT-network attack types.
C. Building the FNN and SNN deep learning based IDS
The paper builds parallel FNN and SNN intrusion-detection models with matching hidden-layer sizes but different normalization-oriented components. Adversarial examples are generated using gradient-based attack methods.
- Model architectures: Two intrusion-detection systems are implemented: a Feedforward Artificial Neural Network and a Self-normalizing Neural Network.
- Model architectures: Each neural network uses three hidden layers with 16 neurons per layer, totaling 48 hidden neurons.
- Self-normalization: The SNN aims to keep layer-wise mean near 0 and variance near 1 throughout the network.
- Model architectures: The FNN uses ReLU, standard dropout, and Glorot Uniform initialization, whereas the SNN uses SeLU, AlphaDropout, and Lecun Uniform initialization.
- Adversarial sample generation: FGSM performs a one-step gradient-sign update, while BIM applies finer FGSM-based optimization across multiple iterations with feature clipping.
V. RESULTS & EVALUATION
The evaluation examines adversarial degradation, FNN–SNN performance, adversarial resilience, and feature normalization. In the reported FNN experiment, FGSM samples sharply reduce intrusion-detection accuracy.
- The evaluation contains experiments on adversarial impact, FNN–SNN performance, adversarial resilience, and feature normalization.
- 95.1% initial FNN accuracy falls to 24% when evaluated on FGSM adversarial samples.
- Algorithm 1 iterates over FNN and SNN models, trains them while monitoring training and validation loss, and evaluates performance under adversarial attacks.
B. Performance Comparison of FNN and SNN IDS using the adversarial-free IoT dataset
On the adversarial-free IoT dataset, the FNN IDS outperforms the SNN IDS across classification and multiclassification metrics over multiple experiment runs.
- The FNN IDS consistently outperforms the SNN IDS in precision, recall, and F1-score across multiple experiment runs.
- The FNN also outperforms the SNN on Cohen's Kappa Score and MC Coefficient multiclassification metrics.
C. Comparison of Adversarial Resilience of FNN IDS and SNN IDS
Both FNN and SNN intrusion-detection models were degraded by adversarial samples, but the SNN was more resilient than the FNN.
- Both FNN and SNN IDS performance degraded when exposed to adversarial samples.
- The SNN IDS was more resilient to adversarial attacks than the FNN IDS.
D. Effect of Feature Normalization on a Deep Learning based IDS for IoT
Feature normalization improved adversarial-free prediction accuracy for both IDS models but reduced their resilience to adversarial samples.
- Without feature normalization, both IDS models had significantly lower prediction accuracy on the adversarial-free dataset.
- Feature normalization made both IDS models more vulnerable to adversarial samples.
- The experiments therefore show a trade-off between adversarial-free accuracy and adversarial resilience.
- Feature normalization improved both IDS models' performance metrics on adversarial-free data.
VI. CONCLUSION
Experiments on an IoT dataset found that adversarial samples threaten deep-learning IDS models, with SNNs more resilient than FNNs. Feature normalization improved ordinary performance but reduced adversarial resilience.
- Adversarial samples were a real threat to deep-learning intrusion-detection systems in IoT networks.
- The SNN's self-normalizing feature was associated with greater resilience to gradient-based adversarial samples than the FNN.
- Feature normalization negatively affected the adversarial resilience of both deep-learning IDS models.
- The authors identify investigating why SNN self-normalization improves resilience as future work.