Source-linked AI summary
Towards the Development of Realistic Botnet Dataset in the Internet of Things for Network Forensic Analytics: Bot-IoT Dataset
Nickolaos Koroniotis, Nour Moustafa, Elena Sitnikova, Benjamin Turnbull
TL;DR
IoT network-forensics research needs realistic datasets because existing datasets may lack IoT traffic, reliable labels, realistic testbeds, and diverse recent botnet scenarios. The paper develops and evaluates the labeled Bot-IoT dataset using statistical, machine-learning, and deep-learning analyses, reporting high classification accuracy overall while identifying weaker data-exfiltration multiclass performance and elevated binary fall-out in some models.
Problem
Existing network datasets have limitations including missing IoT-generated traffic, unreliable labeling, unrealistic testbeds, limited attack diversity, and absent new features.
Method
The paper constructs Bot-IoT on a realistic testbed with simulated IoT traffic, multiple botnet scenarios, organized labels, feature analysis, and machine- and deep-learning evaluation.
Results
High accuracy was observed in binary and multiclass classification, while data exfiltration had the worst multiclass metrics and some binary models had high fall-out.
Takeaways & Limitations
Bot-IoT provides a labeled dataset combining normal and attack traffic for evaluating network-forensic methods across IoT-specific scenarios.
Takeaways & Limitations
The dataset addresses identified shortcomings, but existing datasets still include scope differences such as anonymized header-only traffic or specific attack coverage.
Abstract
from arXiv · showhide
The proliferation of IoT systems, has seen them targeted by malicious third parties. To address this, realistic protection and investigation countermeasures need to be developed. Such countermeasures include network intrusion detection and network forensic systems. For that purpose, a well-structured and representative dataset is paramount for training and validating the credibility of the systems. Although there are several network, in most cases, not much information is given about the Botnet scenarios that were used. This paper, proposes a new dataset, Bot-IoT, which incorporates legitimate and simulated IoT network traffic, along with various types of attacks. We also present a realistic testbed environment for addressing the existing dataset drawbacks of capturing complete network information, accurate labeling, as well as recent and complex attack diversity. Finally, we evaluate the reliability of the BoT-IoT dataset using different statistical and machine learning methods for forensics purposes compared with the existing datasets. This work provides the baseline for allowing botnet identificaiton across IoT-specifc networks. The Bot-IoT dataset can be accessed at [1].
1. Introduction
The paper addresses shortcomings in existing IoT network datasets by developing Bot-IoT with realistic testbed traffic, diverse botnet scenarios, and organized labeled data. It evaluates the dataset through statistical analysis and machine- and deep-learning forensic methods.
- Existing datasets often lack reliable labels, diverse botnet scenarios, realistic traffic, and ground truth needed for forensic and intrusion-detection research.
- Bot-IoT organizes packet captures by attack type and uses multiple tools to generate several botnet scenarios.
- The paper designs a realistic Bot-IoT dataset and details the testbed configuration and simulated IoT sensors.
- The proposed dataset features are statistically analyzed using Correlation Coefficient and Joint Entropy techniques.
- Network forensic methods based on machine- and deep-learning algorithms are evaluated on Bot-IoT and compared with popular datasets.
- The paper proceeds from testbed design and feature extraction through benign and malicious scenarios, statistical and machine-learning analysis, and experimental results.
2. Background and Related Work
The paper reviews IoT, botnet, digital-forensics, and network-forensics concepts, then identifies limitations in existing datasets and testbeds that motivate Bot-IoT.
- IoT and botnets: IoT systems combine interconnected devices and cloud services, while botnets exploit their vulnerabilities through coordinated compromised machines and command-and-control infrastructure.Botnets can launch attacks including DDoS against IoT networks.
- Forensics analytics: Digital forensics analyzes digital evidence through identification, preservation, collection, examination, analysis, and presentation.Machine learning and big-data methods support forensic analytics during examination.
- Network-forensics applications: Network-forensics analytics processes logs, emails, and packets for applications including intrusion detection, honeypots, flow analysis, deep packet inspection, and email authorship.Deep packet inspection examines packet content with higher overhead, whereas flow analysis aggregates packets sharing network attributes.
- Comparison and novelty: Compared with prior approaches, the proposed environment combines virtualized deployment with a broader range of botnet activities and simulated IoT traffic.The reported activities include DDoS, DoS, and port scanning, while Node-red generates MQTT-based IoT traffic.
- Dataset limitations: Existing testbeds and datasets vary in attack scope, realism, labeling, IoT representation, and feature generation, leaving recent diverse IoT botnet traffic insufficiently covered.CAIDA lacked ground truth and new features, while UNSW-NB15 used a synthetic attack-generation environment.
3. The proposed IoT-Bot dataset
The proposed Bot-IoT testbed combines virtual network platforms, simulated IoT services, and feature extraction for forensic analytics. It generates benign and attack traffic through VMs, Node-red, MQTT brokers, and cloud connectivity.
- Testbed architecture: The testbed has three components: network platforms, simulated IoT services, and feature extraction with forensic analytics.Argus extracts features, followed by statistical and machine-learning analysis.
- Network platforms: Virtual machines, a firewall, and a tap provide network platforms containing normal and attacking systems for traffic generation and analysis.The environment includes LAN and WAN interfaces and uses packet filtering to monitor traffic between attack and normal platforms.
- Benign traffic: Ostinato generates large volumes of benign traffic while routine DNS, FTP, HTTP, and SSH service connections maintain normal VM activity.The traffic-generation process uses specified IPs and ports and periodically executes installed services.
- Simulated IoT services: Node-red simulates IoT sensors and services, including weather, refrigeration, lighting, garage-door, and thermostat scenarios.JavaScript code mimics sensor behavior such as temperature, pressure, humidity, motion, and probabilistic device actions.
- IoT communication: MQTT connects simulated devices to local and AWS brokers using publish/subscribe topics, producing periodic regular IoT traffic.The brokers act as intermediaries between smart devices and web or smartphone applications.
4. Extracting features and forensics analytics
The dataset pipeline converts captured packet traffic into labeled network flows and derives additional sliding-window features for forensic analysis.
- Network Flow Extraction: Captured normal and attack traffic was converted into Argus flow files, then imported into MySQL for processing.
- Network Flow Extraction: The conversion process produced a final CSV dataset from packet captures after flow extraction and database processing.
- Network Flow Extraction: Flow identifiers including saddr, sport, daddr, dport, and proto supported labeling of attack and normal instances.
- New Feature Generation: 100-connection sliding windows generated aggregate features intended to improve classifier predictive capabilities.
5. Benign and Botnet scenarios
The Bot-IoT testbed combines simulated smart-home traffic with synchronized botnet attack scenarios spanning probing, denial-of-service, and information-theft activities.
- Benign Scenarios: Five simulated smart-home devices generated benign traffic through Node-Red, AWS, MQTT, and additional Ostinato traffic.
- Botnet Scenarios: Four Kali Linux virtual machines launched attacks in parallel while normal background traffic continued.
- Botnet Scenarios: Probing scenarios included port scanning and OS fingerprinting using Nmap, Hping3, and Xprobe2.
- Botnet Scenarios: Denial-of-service scenarios used DDoS and DoS attacks over TCP, UDP, and HTTP, including Hping3 and GoldenEye tools.
- Botnet Scenarios: Information-theft scenarios covered data theft and keylogging using Metasploit-based exploitation and credential attacks.
6. Statistics and machine learning methods
The paper evaluates Bot-IoT features with correlation and entropy analysis, then assesses classifiers including SVM, RNN, and LSTM-RNN for forensic identification.
- Statistical Analysis Techniques: Pearson correlation measured linear relationships among Bot-IoT features, with outputs ranging from -1 to 1.
- Statistical Analysis Techniques: Joint entropy quantified uncertainty between feature pairs, with entropy values constrained to be greater than or equal to zero.
- Machine and Deep Learning Analysis: SVM, RNN, and LSTM-RNN models were trained to evaluate Bot-IoT quality for classification.
- Machine and Deep Learning Analysis: The linear-kernel SVM separates feature-space instances into classes by seeking a hyperplane that maximizes the margin.
- Machine and Deep Learning Analysis: RNNs use current inputs and previous hidden-state outputs, while LSTMs maintain memory for temporally distant associations.
7.1. Pre-processing steps of Bot-IoT dataset
Pre-processing converts the large labeled dataset into accessible CSV files, transforms feature representations, normalizes values, and prepares model-evaluation inputs.
- Dataset Extraction: The full dataset exceeded 72,000,000 records, with 16.7 GB in CSV form and 69.3 GB in PCAP form.
- Dataset Extraction: A 5% extract produced four files totaling approximately 0.78 GB and about 3 million records for training and testing.
- Feature Transformation: Numerical features were discretized into five equal-size bins, while categorical values were converted into consecutive numeric codes.
- Normalization: Min-Max normalization scaled values to [0,1] using the original minimum and maximum values and selected target bounds.
- Model Evaluation: The processed data supported evaluation through confusion matrices and a collection of machine-learning metrics.
7.2. Unsupervised Attribute evaluations
The study ranks features using correlation and joint entropy, preferring low redundancy and high disorder. It then selects a 10-feature subset through combined normalized scores.
- Statistical feature selection: Filter-based selection evaluates feature quality statistically to reduce dimensionality and improve machine-learning performance.The approach prioritizes statistical evaluation before model-based assessment.
- Entropy analysis: Joint entropy measures feature disorder, where higher average entropy indicates less shared information and lower ambiguity.The entropy calculation uses joint probabilities after feature discretization.
- Combined scoring: Features are considered ideal when they combine high entropy with low correlation after scores are normalized to [0,1].Correlation scores are inverted after Min-Max transformation so both measures use the same direction.
7.3. Supervised evaluation
The supervised evaluation compares SVM, RNN, and LSTM-RNN classifiers using the 10-best-feature and full-feature datasets. Results show high binary and multiclass accuracy, but generally high binary fall-out except for full-feature RNN and LSTM models.
- Evaluation design: Three predictors—SVM, RNN, and LSTM-RNN—assess whether the selected features retain sufficient classification quality.Each model is evaluated against a version containing all available features.
- SVM: The SVM uses a linear classifier with C=1, 4-fold cross-validation, and feature-dependent maximum iteration settings.The 10-best-feature model uses 100000 maximum iterations, while the full-feature model uses 1000.
- LSTM-RNN: The LSTM models use feature-count input layers, two hidden layers, and sigmoid binary-classification outputs for both feature configurations.The 10-best and full-feature models use 10 and 35 input neurons, respectively, with batch size 100.
- RNN: The RNN models use 10- or 35-neuron input layers, shared hidden-layer structures, and tanh hidden activations with sigmoid outputs.Both configurations are trained for 4 epochs with batch size 100.
- Results: High accuracy is reported for both binary and multiclass classification, while data exfiltration has the worst multiclass metrics.Training time is somewhat proportional to the number of records used.
- Results: The DoS UDP-Normal traffic model takes the longest training time among the reported models.The result is reported alongside the evaluation metrics and training-time comparison.
- Results: Binary-classification fall-out is generally high, except for RNN and LSTM models trained on the full-feature dataset.The authors suggest poor model optimization and relatively few epochs as possible explanations.
8. Conclusion
The conclusion presents Bot-IoT as a labeled dataset combining normal and attack traffic generated on a realistic testbed. It reports feature engineering and classifier-based validation, with model performance varying by feature set and metric.
- Dataset contribution: Bot-IoT combines normal IoT-related traffic, other network traffic, and diverse botnet attack traffic in a realistic testbed dataset.Labels identify attack flows and attack categories or subcategories for multiclass classification.
- Evaluation: Additional features and statistical analysis produce a 10-best-feature subset for evaluating classifier performance.The paper compares Accuracy, Precision, Recall, and Fall-out across models.
- Reported results: The full-feature SVM achieves the highest accuracy and recall, whereas the 10-best-feature SVM achieves the highest precision and lowest fall-out.The authors state that further model optimization could improve results.
- Future work: Future work will develop a deep-learning network forensic model and evaluate its reliability using the Bot-IoT dataset.This is identified as a planned extension of the present evaluation.