PaperlayerView exact revision on arXiv

Source-linked AI summary

IoT Security: Botnet detection in IoT using Machine learning

Satish Pokhrel, Robert Abbas, Bhulok Aryal

arXiv:2104.02231v1cs.LG

TL;DR

Imbalanced botnet datasets can make accuracy misleading and leave performance on unseen data unclear. The paper combines feature engineering and machine learning with SMOTE and evaluates models on imbalanced and balanced BoT-IoT data. KNN reports different accuracy and ROC AUC values across those datasets.

  • Problem

    Imbalanced botnet datasets can bias models toward majority traffic, making accuracy impractical, while training-set accuracy may not indicate performance on unseen data.

    Source: A. Research gap · p. 2

  • Method

    The study combines feature engineering and machine-learning classifiers with SMOTE, comparing training on imbalanced and balanced BoT-IoT data.

    Source: §I · p. 1, B. KNN · p. 4

  • Results

    KNN achieved 99.6% accuracy and 99.2% ROC AUC on imbalanced BoT-IoT data, versus 92.1% accuracy and 92.2% ROC AUC on balanced data.

    Source: 2) Results with KNN model · p. 9

  • Takeaways & Limitations

    The authors conclude that imbalanced data can yield illusory accuracy, while SMOTE produced more stable accuracy and ROC AUC in their comparison.

    Source: §VIII · p. 11

Abstract

from arXiv · show

The acceptance of Internet of Things (IoT) applications and services has seen an enormous rise of interest in IoT. Organizations have begun to create various IoT based gadgets ranging from small personal devices such as a smart watch to a whole network of smart grid, smart mining, smart manufacturing, and autonomous driver-less vehicles. The overwhelming amount and ubiquitous presence have attracted potential hackers for cyber-attacks and data theft. Security is considered as one of the prominent challenges in IoT. The key scope of this research work is to propose an innovative model using machine learning algorithm to detect and mitigate botnet-based distributed denial of service (DDoS) attack in IoT network. Our proposed model tackles the security issue concerning the threats from bots. Different machine learning algorithms such as K- Nearest Neighbour (KNN), Naive Bayes model and Multi-layer Perception Artificial Neural Network (MLP ANN) were used to develop a model where data are trained by BoT-IoT dataset. The best algorithm was selected by a reference point based on accuracy percentage and area under the receiver operating characteristics curve (ROC AUC) score. Feature engineering and Synthetic minority oversampling technique (SMOTE) were combined with machine learning algorithms (MLAs). Performance comparison of three algorithms used was done in class imbalance dataset and on the class balanced dataset.

I. INTRODUCTION

IoT’s expanding connected-device environment raises security concerns, including botnet-based DDoS attacks. The paper combines feature engineering, machine learning, and SMOTE, and examines how class imbalance affects botnet detection.

I. INTRODUCTION

  • IoT’s interconnected devices produce substantial data flows, and the paper frames security as a major concern as connected devices proliferate.The introduction identifies botnet-based DDoS attacks as a major IoT security challenge.Source: §I · p. 1

II. RELATED WORK

  • The related work includes botnet detection using BoT-IoT, SMOTE, and artificial neural networks, as well as studies combining feature engineering with multiple classifiers.Other cited approaches use honeypot-generated IoT data to capture malware activity and train detectors for new malware families.Source: §II · pp. 1–2

A. Research gap

  • The paper identifies feature engineering as a way to evaluate models more completely and address duplication and multicollinearity in large datasets.It cautions that using traditional data without feature engineering may introduce overfitting.Source: A. Research gap · p. 2
  • Imbalanced botnet datasets can make accuracy misleading because models may favor the majority class, while training-set accuracy does not establish performance on unseen data.The authors also identify overfitting and underfitting as concerns with such evaluation.Source: A. Research gap · p. 2
  • The study compares machine-learning performance on imbalanced and SMOTE-balanced BoT-IoT data, using cross-validation, accuracy, and ROC AUC to evaluate botnet detection.The authors use the comparison to assess performance on unforeseen data and the effect of imbalance.Source: A. Research gap · p. 2

A. Botnet

The section describes botnets and their command-and-control communication, explains how infected IoT devices can participate in DDoS attacks, and introduces machine learning and supervised classification as security tools.

A. Botnet

  • A botnet is a network of bots controlled by a botmaster through command-and-control communication to carry out malicious activity.Bots can remain hidden until commanded; the described lifecycle includes infection, connection, command and control, updating, and maintenance.Source: A. Botnet · pp. 2–3

B. DDoS attack

  • DDoS attacks flood a target with simultaneous malicious traffic, and infected IoT devices can act as bots in those attacks.The traffic can overwhelm server capacity and deny service to legitimate users.Source: B. DDoS attack · p. 3

C. Security vulnerabilities in IoT

  • The section describes low-end IoT devices as having weak security and as potential targets or participants in botnets.It presents machine learning as a way to model systems not readily represented by mathematical equations and to detect unusual traffic.Source: C. Security vulnerabilities in IoT · p. 3, §IV · p. 3

IV. MACHINE LEARNING

  • Machine learning uses examples to learn, and anomaly detection can identify unusual network traffic that might indicate intrusion.The section distinguishes supervised, unsupervised, semi-supervised, and reinforcement learning.Source: §IV · p. 3

A. Supervised learning:

  • Supervised learning uses labeled sample data to learn input-to-output mappings and support prediction; KNN, SVM, Naive Bayes, decision trees, and random forests are examples.Source: A. Supervised learning · p. 3

B. Unsupervised learning:

The section outlines several learning approaches before describing supervised algorithms used for botnet detection. Naïve Bayes uses Bayes’ theorem, while KNN classifies by the votes of nearby examples.

B. Unsupervised learning:

C. Semi-supervised learning:

  • Semi-supervised learning combines a limited amount of labeled data with a larger set of unlabeled data.The passage says these algorithms do better with large amounts of unlabeled data and fewer labeled examples.Source: C. Semi-supervised learning · p. 3

D. Re-enforcement learning:

A. Gaussian Naïve bayes machine learning algorithm

B. KNN

  • KNN categorizes an object by the majority vote of its K nearest neighbors using Euclidean distance.The choice of K affects accuracy; a small value can underfit and a larger value can overfit.Source: B. KNN · p. 4

C. MLP ANN

The research approach applies supervised machine-learning methods to BoT-IoT traffic, using preprocessing and feature analysis to prepare data for botnet detection. The selected dataset is highly imbalanced, and the authors also create a balanced version using SMOTE.

C. MLP ANN

  • MLP ANN uses input, hidden, and output layers with nonlinear activation functions and backpropagation.Variable weights connect neurons; weighted inputs are transformed through hidden and output neurons to yield an outcome.Source: C. MLP ANN · p. 4

VI. RESEARCH APPROACH

  • MLAs were benchmarked on imbalanced and class-balanced BoT-IoT data to select an algorithm for the botnet detection model.The study examined packet behavior and selected features through manual analysis and feature selection based on top F-scores.Source: §VI · p. 4, A. Experimental dataset (BoT-IoT) · p. 5

A. Experimental dataset (BoT-IoT)

  • The selected BoT-IoT file contains 999,610 records, including 994,828 botnet records, while the full dataset comprises 74 CSV files and about 72 million records.Source: A. Experimental dataset (BoT-IoT) · pp. 4–5
  • The authors created a second dataset by applying SMOTE to the selected BoT-IoT data, producing equal numbers of botnet and normal traffic records.The original selected data contains more than 99% botnet traffic and less than 1% normal traffic.Source: A. Experimental dataset (BoT-IoT) · p. 5

B. Data preprocessing

  • Preprocessing involved data cleansing, normalization, and transformation to improve data quality for model training.Source: B. Data preprocessing · p. 5

1) Data Cleansing:

2) Normalization:

The data preparation procedure scales and converts BoT-IoT features, selects features using chi-square scores, and uses SMOTE to address class imbalance.

2) Normalization:

  • Min-max feature scaling maps features with different ranges to [0,1] to help the model converge more quickly.Source: 2) Normalization · p. 5

3) Transformation:

  • Categorical BoT-IoT values, including protocol types, were converted to numeric values for use by the machine-learning algorithms.Source: 3) Transformation · p. 5

C. Feature Engineering

  • Chi-square feature scores were used to select the eight features whose scores exceeded the mean for training and testing.Feature engineering is described as reducing dimensionality, overfitting risk, and processing time while improving model performance.Source: C. Feature Engineering · p. 5

D. Synthetic Minority over-sampling technique (SMOTE)

  • SMOTE oversamples the minority class by using nearest neighbors, multiplying their differences by a random number between 0 and 1, and generating sample points.The paper notes that class imbalance may cause misclassification and affect algorithm performance.Source: D. Synthetic Minority over-sampling… · p. 6

E. Experimental Scenario

The study benchmarks KNN, MLP ANN, and Naive Bayes on the original and SMOTE-balanced BoT-IoT datasets. It evaluates classifiers with confusion-matrix-derived metrics and five-fold cross-validation.

E. Experimental Scenario

  • KNN, MLP ANN, and Naive Bayes were trained on BoT-IoT datasets D1 and D2, with accuracy benchmarked for algorithm selection; D1 was imbalanced, while SMOTE produced class-balanced D2.Source: E. Experimental Scenario · p. 6

F. Machine Learning Model Evaluation and Cross validation

  • Accuracy, precision, recall, F1-Score, and ROC AUC were used to evaluate classifier performance from confusion matrices.The authors note that accuracy can mislead on imbalanced data and prefer ROC AUC for effective evaluation.Source: F. Machine Learning Model Evaluation… · p. 6, 5) ROC AUC · p. 7
  • Five-fold cross-validation repeatedly tests on one data subset and trains on the remaining four to assess performance and mitigate overfitting.The authors describe the choice of k as a trade-off: higher values may increase computational overhead and time.Source: 5) ROC AUC · p. 7

VII. EXPERIMENT AND DISCUSSION

The experiment examines the highly imbalanced BoT-IoT dataset and describes preprocessing before transformation. The dataset’s protocol and state values are converted to numeric codes.

VII. EXPERIMENT AND DISCUSSION

  • 994,828 botnet records and 4,782 normal records make up the 999,610-row dataset, which has 29 columns.Fig. 7 presents the class sizes; the authors describe the dataset as highly imbalanced.Source: §VII · p. 7, Figure 7 · p. 7
  • Null-containing rows were dropped, and non-numeric values were converted to numeric values during preprocessing.Protocol values start at 1 and state values start at 10.Source: §VII · p. 7

A. Transformation

B. Analyzing the behavior of Normal and Botnet Traffic

The traffic analysis compares normal and botnet statistics and reports contrasting packet counts, rates, and durations. Several reported means show higher rates but lower packet counts and shorter duration for botnet traffic.

B. Analyzing the behavior of Normal and Botnet Traffic

C. Oversampling minority class

SMOTE is used to balance the BoT-IoT classes, and chi-square feature scores are used to select features for training. The dataset is split into training and test portions for evaluation.

C. Oversampling minority class

D. Feature Score

  • Eight chi-square-ranked features are bytes, sbytes, dbytes, rate, pkts, spkts, srate, and drate.The authors describe these as the top features used to train the model; Fig. 16 presents the feature scores.Source: D. Feature Score · p. 9, Figure 16 · p. 9

E. Train-Test Split

  • 80 percent of the data were used for training and 20 percent for testing.The split was used to evaluate model performance.Source: E. Train-Test Split · p. 9

F. Comparison of performance of Machine learning algorithms

The section compares Gaussian Naïve Bayes, KNN, and MLP ANN on imbalanced and SMOTE-balanced BoT-IoT data. Reported results show that accuracy alone can mislead on imbalanced data, while KNN performs well on both dataset conditions.

F. Comparison of performance of Machine learning algorithms

  • The evaluation uses cross-validation and compares classifiers on imbalanced data and data balanced with SMOTE, assessing true positive rate, false positive rate, precision, and recall.The dataset contains less than 1% normal traffic and more than 99% botnet traffic; the authors note that oversampling by duplication adds no new information.Source: F. Comparison of performance of… · p. 9

1) Results with Gaussian Naïve bayes model:

2) Results with KNN model:

  • KNN achieved 99.6% accuracy and 99.2% ROC AUC on imbalanced data, versus 92.1% accuracy and 92.2% ROC AUC on balanced data.The authors describe KNN as effective for botnet detection and say its ROC AUC is almost 1 on both datasets.Source: 2) Results with KNN model · pp. 9–10, Figure 19 · p. 10

3) MLP ANN model:

  • MLP ANN ROC AUC rose from 0.50 before SMOTE to 0.90 after SMOTE on BoT-IoT data.The authors report that SMOTE also improved precision, recall, and F1-score, while accuracy was 87.4% on imbalanced data and 85.8 after SMOTE.Source: Figure 22 · p. 10, 3) MLP ANN model · p. 10

G. Observations

The authors identify KNN as the most reliable algorithm in their BoT-IoT comparisons and report more stable performance after applying SMOTE. They propose real-time simulation and SDN implementation as future work.

G. Observations

  • Feature selection was used to reduce computational overhead, alongside SMOTE to address imbalance in the two evaluated datasets.The authors label the real-time BoT-IoT dataset D1 and the SMOTE-processed dataset D2.Source: G. Observations · p. 10
  • KNN was the most reliable algorithm in the overall comparison, with 92.1% accuracy and 92.2% ROC AUC reported for KNN.The authors also report that KNN works well on highly imbalanced real-time data.Source: G. Observations · p. 10, §VIII · p. 11

VIII. CONCLUSION

  • After SMOTE, the authors report more stable accuracy and ROC AUC, with precision, recall, and F1-score in a similar range.They caution that the high accuracy on imbalanced data may be illusory because recall and F1-score were low.Source: §VIII · p. 11

IX. FUTURE WORK

  • Future work proposes real-time simulation and an SDN implementation that could monitor traffic and block hosts sending botnet packets.The authors describe this as a possible way to mitigate botnet attacks in the server.Source: §IX · p. 11
Loading 2104.02231v1…