PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Stage Optimized Machine Learning Framework for Network Intrusion Detection

MohammadNoor Injadat, Abdallah Moubayed, Ali Bou Nassif, Abdallah Shami

arXiv:2008.03297v1cs.CRcs.LGcs.NI

TL;DR

Network intrusion detection remains challenged by high-volume, high-dimensional, and changing network data. The paper proposes and evaluates a multi-stage optimized ML-based framework using oversampling, feature selection, and hyper-parameter optimization across two intrusion-detection datasets, while related work has often lacked systematic sampling, multiple-dataset validation, and varied optimization methods.

  • Problem

    NIDS research still needs improvement for high-volume traffic, changing environments, high-dimensional datasets, and real-time detection, while prior work often lacks systematic sampling and multiple-dataset validation.

  • Method

    The paper proposes a multi-stage optimized ML-based NIDS framework that studies oversampling, compares information-gain and correlation-based feature selection, and investigates multiple hyper-parameter optimization techniques.

  • Results

    The framework is evaluated on the CICIDS 2017 and UNSW-NB 2015 datasets using accuracy, precision, recall, and false alarm rate.

  • Takeaways & Limitations

    The study provides a multi-dataset evaluation of an optimized NIDS framework designed to reduce computational complexity while maintaining detection performance.

  • Takeaways & Limitations

    The paper identifies related-work limitations including outdated datasets, random training-size selection, single-dataset evaluation, and limited hyper-parameter optimization.

Abstract

from arXiv · show

Cyber-security garnered significant attention due to the increased dependency of individuals and organizations on the Internet and their concern about the security and privacy of their online activities. Several previous machine learning (ML)-based network intrusion detection systems (NIDSs) have been developed to protect against malicious online behavior. This paper proposes a novel multi-stage optimized ML-based NIDS framework that reduces computational complexity while maintaining its detection performance. This work studies the impact of oversampling techniques on the models' training sample size and determines the minimal suitable training sample size. Furthermore, it compares between two feature selection techniques, information gain and correlation-based, and explores their effect on detection performance and time complexity. Moreover, different ML hyper-parameter optimization techniques are investigated to enhance the NIDS's performance. The performance of the proposed framework is evaluated using two recent intrusion detection datasets, the CICIDS 2017 and the UNSW-NB 2015 datasets. Experimental results show that the proposed model significantly reduces the required training sample size (up to 74%) and feature set size (up to 50%). Moreover, the model performance is enhanced with hyper-parameter optimization with detection accuracies over 99% for both datasets, outperforming recent literature works by 1-2% higher accuracy and 1-2% lower false alarm rate.

I. INTRODUCTION

The paper addresses computational and data challenges in ML-based network intrusion detection by proposing and evaluating a multi-stage optimized framework. It combines systematic preprocessing, feature selection, hyper-parameter optimization, and multi-dataset evaluation to improve efficiency and detection performance.

  • Motivation: High-volume traffic, evolving environments, high-dimensional datasets, and real-time requirements make feature selection and computational efficiency important NIDS challenges.Redundant or irrelevant features can slow model training and negatively affect detection capabilities.
  • Research Contributions: The proposed framework reduces computational complexity while maintaining detection performance through multiple optimized stages.The study examines preprocessing, feature selection, and hyper-parameter optimization as coordinated components of the framework.
  • Limitations of Related Work: Previous NIDS research often used imbalanced or outdated datasets, randomly selected training sizes, single-dataset validation, one optimization method, and limited time-complexity analysis.These limitations motivate a more systematic and comprehensive evaluation framework.
  • Research Contributions: The study determines a minimum suitable training size after evaluating oversampling techniques for effective intrusion detection.This directly addresses the contribution concerning systematic training-sample selection.
  • Research Contributions: Information gain and correlation-based feature selection are compared for their effects on detection performance and training and testing time complexity.The comparison evaluates both predictive performance and computational cost.
  • Research Contributions: Hyper-parameter optimization techniques are investigated, and the framework is evaluated on CICIDS 2017 and UNSW-NB 2015 using multiple detection metrics.The evaluation includes accuracy, precision, recall, and false alarm rate, alongside comparisons with recent literature.

IV. BACKGROUND AND PRELIMINARIES

The framework’s preprocessing stage prepares network data for machine learning by encoding categorical features, normalizing values with Z-score normalization, and oversampling minority classes with SMOTE.

  • Data Preprocessing: The data preprocessing stage uses Z-score normalization and SMOTE minority-class oversampling.These operations are described as the framework’s preprocessing components.
  • Z-Score Normalization: Categorical features are first transformed into numerical values using a label encoder before normalization.Encoding precedes the calculation of normalized sample values.
  • Z-Score Normalization: Z-score normalization calculates each normalized sample value from the sample, feature mean vector, and feature standard deviation.The paper states that normalized datasets can improve ML classification-model performance.

2) SMOTE Technique:

SMOTE addresses minority-class imbalance by synthesizing new instances, while feature selection reduces high-dimensional input using information gain or correlation-based criteria.

  • SMOTE Technique: SMOTE synthesizes minority-class instances to reduce class imbalance that can negatively affect ML classification performance.It is especially relevant for network traffic datasets, which typically suffer from class imbalance.
  • SMOTE Technique: SMOTE constructs new samples using the k-nearest neighbors of each minority-class instance.For each minority instance, a randomly selected neighbor contributes to the synthetic sample.
  • SMOTE Technique: Unlike replication-based oversampling, SMOTE generates new instances that statistically resemble minority-class samples.
  • Feature Selection: Information gain ranks features by the information they provide about the target class and selects those with the highest information.The evaluation uses entropy and mutual information concepts.
  • Feature Selection: Correlation-based feature selection ranks features by target-class correlation and retains features considered relevant or predictive.Its subset evaluation uses Pearson’s correlation coefficient, including class-feature and feature-feature correlations.

C. Hyper-parameter Optimization:

The framework investigates random search, meta-heuristic algorithms, and Bayesian optimization as alternative approaches for tuning ML hyperparameters.

  • Random Search: Random search tests a randomly selected subset of hyperparameter combinations rather than exhaustively evaluating every combination.Its objective function is typically model accuracy, with parm representing the parameters to tune.
  • Random Search: Random search can be parallelized, further reducing optimization computational complexity.
  • Meta-heuristic Optimization Algorithms: Meta-heuristic optimization algorithms seek sufficiently good solutions to combinatorial optimization problems with lower computational complexity.
  • Meta-heuristic Optimization Algorithms: PSO searches hyperparameter space with particles that iteratively update positions using their own best experience and the population’s best solution.The method models social behavior such as flocks of birds moving toward promising positions.
  • Meta-heuristic Optimization Algorithms: GA searches hyperparameter combinations through population initialization, fitness evaluation, ranking, crossover, mutation, and replacement.Chromosome fitness is typically measured using ML model accuracy.

3) Bayesian Optimization:

Bayesian optimization uses probabilistic surrogate modeling and acquisition functions to select promising evaluations for a scalar objective function.

  • Bayesian Optimization: Bayesian optimization minimizes a scalar objective by combining a surrogate model, Bayesian updates, and subsequent evaluation-point selection.
  • Bayesian Optimization: Gaussian Process modeling represents the objective using a mean and variance over the hyperparameter configuration space.The model is effective with few hyperparameters but ineffective for conditional hyperparameters.
  • Bayesian Optimization: TPE models the objective with density functions l(x) or g(x), selected according to a predefined threshold.TPE estimators follow a tree structure and can optimize all hyperparameter types.
  • Bayesian Optimization: The acquisition function selects the next evaluation point by maximizing expected improvement while avoiding values that increase the objective.xbest denotes the location of the lowest posterior mean.

V. PROPOSED MULTI-STAGE OPTIMIZED ML-BASED NIDS FRAMEWORK

The proposed NIDS framework combines preprocessing, feature selection, and hyperparameter optimization to target accurate, low-false-alarm, computationally efficient intrusion detection.

  • A. General Framework Description: The framework’s first stage applies Z-score normalization and SMOTE to improve training and reduce class imbalance.The paper states that sufficient samples for each class can reduce the required training sample size.
  • A. General Framework Description: The second stage selects features to reduce classification time complexity and training time without sacrificing performance.Two feature selection methods are compared in this stage.
  • A. General Framework Description: The third stage investigates random search, PSO and GA, and Bayesian optimization as three hyperparameter-tuning categories.These correspond to heuristic, meta-heuristic, and probabilistic global optimization models.
  • A. General Framework Description: The combined optimization stages produce an ML classification model that classifies new instances as normal or attack.Figure 1 illustrates the framework stages.

B. Security Considerations:

The framework combines staged preprocessing, feature selection, and hyper-parameter optimization to reduce computational complexity while preserving NIDS performance. Its overall complexity is dominated by correlation-based feature selection at O(MN^2).

  • Security considerations: The framework’s stages include preprocessing, feature selection, and hyper-parameter optimization, with each stage contributing to aggregate complexity.Overall complexity is determined by combining the highest aggregate complexities of the selected algorithms.
  • Security considerations: SMOTE has complexity O(M_min^2N), making the first preprocessing stage dependent on the minority-class sample count and feature count.Z-score normalization contributes O(N) in the stated formulation.
  • Security considerations: O(MN^2) characterizes correlation-based feature selection, whereas information gain-based selection has complexity O(MN).Correlation-based selection calculates class-feature and feature-feature correlations; information gain calculates joint probabilities.
  • Security considerations: O(N_parmlogN_parm) is the random-search complexity, while PSO and GA each have complexity O(N_parmN_pop).Random search tests a random subset rather than exhaustively evaluating all hyper-parameter combinations and can run in parallel.
  • Security considerations: O(MN^2) is the framework’s overall time complexity because correlation-based feature selection dominates while using the complete dataset.The ML classifier operates on reduced training samples and feature sets, so it does not dominate the aggregate complexity.

VI. DATASETS DESCRIPTION

The study evaluates its framework on CICIDS 2017 and UNSW-NB 2015, two recent intrusion datasets with nonlinear feature structure and varying class separability. Their attack labels and traffic compositions support binary attack detection across multiple attack types.

  • CICIDS 2017: CICIDS 2017 contains benign traffic and 14 current attack types collected across five days, with 3,119,345 instances and 83 features before unlabeled samples were discarded.Its traffic includes attacks such as DoS, port scanning, brute-force, web attacks, botnets, and infiltration.
  • CICIDS 2017: The CICIDS 2017 principal-component representation shows intertwined classes and nonlinear features, suggesting that a nonlinear kernel may classify instances better.The passage reports the observed structure and the paper’s expectation.
  • UNSW-NB 2015: UNSW-NB 2015 contains 2,540,044 instances and 49 features generated from real modern normal activities and synthetic attack behaviors.Its attack categories include fuzzers, analysis, backdoors, DoS, exploits, generic, reconnaissance, shellcode, and worms.
  • Labeling: The framework uses binary classification by labeling all attack types as “attack,” targeting detection of various attacks rather than only a finite common subset.The paper describes this training design as compatible with anomaly-based NIDS operation.

VII. EXPERIMENTAL PERFORMANCE EVALUATION

The experiments examine preprocessing, classifier setup, and learning-curve behavior on both datasets. SMOTE substantially reduces the training samples needed for convergence and shortens learning-curve construction without producing reported overfitting.

  • Experimental setup: The experiments use KNN and RF classifiers because they performed strongly previously and have lower computational complexity than alternatives such as SVM.The paper gives KNN complexity as O(MN) and discusses RF’s reduced training cost through multithreading.
  • Data preprocessing: The datasets are randomly split into training and testing portions using a 70%/30% criterion after normalization.Learning curves vary training accuracy and cross-validation accuracy as training sample size changes.
  • SMOTE: 1,818,477 benign and 1,800,000 attack CICIDS 2017 training instances are produced by SMOTE.The instances are denoted as classes 0 and 1, respectively.
  • Training sample size: 600,000 and 800,000 training samples are sufficient after SMOTE for CICIDS 2017 and UNSW-NB 2015, representing reductions of approximately 74% and 39%.Before SMOTE, convergence required close to 2.3 million and 1.3 million samples, respectively.
  • Runtime: 90 minutes are required to build the UNSW-NB 2015 learning curve after SMOTE, compared with close to 600 minutes before SMOTE.The paper presents this as a reduction in time complexity from oversampling.
  • Generalization: Small or zero training–cross-validation accuracy gaps indicate that the models do not suffer from overfitting before or after SMOTE.The paper interprets these gaps as evidence of suitability for testing and cross-validation datasets.

2) Impact of feature selection on feature set size and training sample size:

Feature selection reduces both feature-set and training-sample requirements. IGBFS generally selects fewer features and yields larger training-sample and runtime reductions than CBFS in the reported comparisons.

  • Information gain: Average packet size and packet length variance are among CICIDS 2017’s most informative features, while packet-size and time-to-live features rank highly for UNSW-NB 2015.The mutual-information rankings are shown in Figs. 8 and 9.
  • Feature-selection methods: IGBFS ranks features by information about the class, while CBFS selects features based on correlation with the target class.IGBFS does not consider mutual information between features; CBFS uses Pearson’s correlation coefficient.
  • Training sample size: 250,000 and 110,000 samples suffice after IGBFS for CICIDS 2017 and UNSW-NB 2015, reducing post-SMOTE requirements by 59% and 86%.These reductions are reported relative to the training sample sizes required after SMOTE.
  • Training sample size: 500,000 and 200,000 samples suffice after CBFS for CICIDS 2017 and UNSW-NB 2015, reducing post-SMOTE requirements by 17% and 75%.The paper attributes the reduction to selecting relevant features that positively affect learning.
  • Runtime: 21 minutes are required with IGBFS versus 25 minutes with CBFS to build the UNSW-NB 2015 learning curve.The paper notes that redundant selected features may negatively affect learning relative to IGBFS.
  • Generalization: Both feature-selection approaches show small or zero training–cross-validation accuracy gaps, indicating generalization without overfitting.This observation is reported for Figs. 10–13.

3) Impact of optimization methods on the ML models’ detection performance:

The framework evaluates classifier performance using accuracy, precision, recall/TPR, and FAR/FPR, comparing optimized and regular models across feature-selection settings and datasets.

  • Four metrics—accuracy, precision, recall/TPR, and FAR/FPR—evaluate classifier performance.Accuracy is defined using true positives, true negatives, false positives, and false negatives.
  • RS and PSO often select fewer KNN neighbors because their stopping criteria may miss optimal values.Their searches stop based on iterations or evaluations rather than testing all potential values.
  • RS and PSO tend to select fewer RF trees, whereas GA, BO-GP, and BO-TPE explore more values and select higher numbers.The text attributes the difference to premature stopping versus broader exploration.
  • CBFS uses the Gini splitting criterion because correlation-selected features may contain less class information and be overlooked by entropy splitting.This choice is linked to the information characteristics of features selected by CBFS.
  • Optimized models outperform recently reported regular models by 1-2% in accuracy and reduce FAR by 1-2% across both datasets.RF also outperforms KNN, while BO-TPE-RF achieves the highest detection accuracy across both datasets and feature-selection methods.

VIII. CONCLUSION

The paper frames network intrusion detection as an ongoing challenge involving high-volume, high-dimensional, and changing network data, then proposes a multi-stage framework to reduce computational complexity while maintaining detection performance. It evaluates this framework across two datasets and identifies future directions involving deep learning and combined supervised–unsupervised methods.

  • VIII. CONCLUSION: NIDS research still faces high-volume traffic, changing environments, high-dimensional datasets, and the need for real-time intrusion detection.These factors leave room for further improvement despite continuous performance gains.
  • VIII. CONCLUSION: The proposed multi-stage optimized ML-based NIDS framework targets lower computational complexity while maintaining detection performance.The framework combines training-sample analysis, feature selection, and model-parameter optimization.
  • VIII. CONCLUSION: The framework is evaluated on the CICIDS 2017 and UNSW-NB 2015 intrusion detection datasets.The evaluation uses two recent state-of-the-art datasets.
  • VIII. CONCLUSION: The study determines a minimum suitable training size by examining the impact of SMOTE oversampling on model training samples.The supplied conclusion passage reports that SMOTE can reduce the required training sample size, but the available passage ends before the numerical result.
  • VIII. CONCLUSION: Future work may explore deep learning classifiers for nonlinear, high-dimensional datasets and combine supervised with unsupervised methods to detect novel attacks.These directions are presented as possible extensions of the current work.
Loading 2008.03297v1…