PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction

Md. Alamin Talukder, Md. Manowarul Islam, Md Ashraf Uddin, Khondokar Fida Hasan, Selina Sharmin, Salem A. Alyami, Mohammad Ali Moni

arXiv:2401.12262v1cs.CRcs.LG

TL;DR

Traditional signature-based IDS methods struggle with evolving and sophisticated cyber threats, while ML-based IDS for large, imbalanced datasets faces dimensionality challenges. The paper combines random oversampling, clustering-based stacking feature embedding, and PCA, achieving high accuracy across three benchmark datasets and reporting improvements over existing methods.

  • Problem

    Traditional signature-based intrusion detection methods are limited against evolving and sophisticated cyber threats, motivating ML-based IDS for large, imbalanced datasets.

  • Method

    The model uses Random Oversampling for imbalance, clustering-based Stacking Feature Embedding to augment features, and PCA to reduce dimensionality to 10 features.

  • Results

    Accuracy improvements over existing methods range from 1.52% to 22.19% on UNSW-NB15, 0.12% to 2.99% on CIC-IDS2017, and 1.99% to 8.44% on CIC-IDS2018.

  • Takeaways & Limitations

    The proposed model provides a robust and adaptable intrusion-detection solution for varying data distributions and network environments.

  • Takeaways & Limitations

    Existing-work comparisons are limited by reliance on older datasets that omit recent attack scenarios, and by overlooking data balancing and computational demands.

Abstract

from arXiv · show

Cybersecurity has emerged as a critical global concern. Intrusion Detection Systems (IDS) play a critical role in protecting interconnected networks by detecting malicious actors and activities. Machine Learning (ML)-based behavior analysis within the IDS has considerable potential for detecting dynamic cyber threats, identifying abnormalities, and identifying malicious conduct within the network. However, as the number of data grows, dimension reduction becomes an increasingly difficult task when training ML models. Addressing this, our paper introduces a novel ML-based network intrusion detection model that uses Random Oversampling (RO) to address data imbalance and Stacking Feature Embedding based on clustering results, as well as Principal Component Analysis (PCA) for dimension reduction and is specifically designed for large and imbalanced datasets. This model's performance is carefully evaluated using three cutting-edge benchmark datasets: UNSW-NB15, CIC-IDS-2017, and CIC-IDS-2018. On the UNSW-NB15 dataset, our trials show that the RF and ET models achieve accuracy rates of 99.59% and 99.95%, respectively. Furthermore, using the CIC-IDS2017 dataset, DT, RF, and ET models reach 99.99% accuracy, while DT and RF models obtain 99.94% accuracy on CIC-IDS2018. These performance results continuously outperform the state-of-art, indicating significant progress in the field of network intrusion detection. This achievement demonstrates the efficacy of the suggested methodology, which can be used practically to accurately monitor and identify network traffic intrusions, thereby blocking possible threats.

1 Introduction

The paper motivates ML-based intrusion detection as a response to evolving cyber threats and the limitations of traditional methods, especially for large and imbalanced datasets. It proposes combining oversampling, stacking feature embedding, and PCA-based dimensionality reduction, then evaluates the approach with multiple classifiers and performance measures.

  • Conventional signature-based intrusion detection methods are inadequate for increasingly complex and evolving cyber threats.
  • ML-based IDS uses behavior analysis to detect anomalies and threats, but privacy concerns arise because effective algorithms often require sensitive data.
  • Many contemporary ML-IDS studies rely on small, outdated, or balanced datasets, motivating validation on large and imbalanced data containing diverse attack scenarios.
  • Random Oversampling balances minority and majority classes, Stacking Feature Embedded adds meta-features, and PCA reduces dimensionality while preserving vital information.
  • The model is evaluated with DT, RF, ET, and XGB using precision, recall, F1-score, confusion matrices, accuracy, and ROC curves; the classifiers exceed 99.9% accuracy.
  • The proposed framework combines preprocessing, oversampling, stacking feature embedding, and dimensionality reduction to improve intrusion detection robustness on substantial datasets.

2 Literature Review

The literature review surveys ML, feature-selection, clustering, and hybrid IDS approaches, while identifying persistent limitations in datasets, balancing, computational analysis, and feature embedding. These gaps motivate the paper’s focus on contemporary large-scale intrusion data and Stacking Feature Embedded methods.

  • Prior IDS research has used ML, hybrid, and deep-learning models to retrieve patterns distinguishing malicious from legitimate network traffic.
  • A feature-dropping approach increased UNSW-NB15 binary accuracy from 88.13% to 90.85%, while another reduced CIC-IDS2017 features from 77 to 24 and achieved 99.95% accuracy.
  • Reported prior results include 100% accuracy on UNSW-NB15, 99.78% on CIC-IDS2017, and 97.75% on CIC-IDS2018 across different proposed models.
  • 2.4 Limitations of the Existing Works: Existing works often rely on older datasets lacking recent attack scenarios, overlook data balancing, use complete feature sets, and omit time-complexity analysis.
  • 2.4 Limitations of the Existing Works: The reviewed literature does not explore Stacking Feature Embedded methods that incorporate meta-features to potentially improve intrusion detection accuracy.

3 Methodology

The methodology combines preprocessing, benchmark datasets, feature embedding, and dimensionality reduction to support intrusion detection on large datasets. It evaluates machine-learning classifiers using established datasets and cross-validation.

  • Framework: The proposed framework combines feature resampling, scaling, stacking feature embedding, and feature extraction for intrusion detection on big and imbalanced datasets.The framework is illustrated in Figure 1 and is intended to classify packets as normal or attack.
  • Framework: The workflow preprocesses data by cleaning missing values, removing duplicate rows, merging low-frequency similar classes, and reducing numeric data types.These operations are presented as steps for improving data quality and reducing dataset size.
  • Evaluation: The evaluation compares Decision Tree, Random Forest, Extra Trees, and XGBoost classifiers using k-fold cross-validation and multiple performance metrics.The workflow aims to identify suitable machine-learning models for intrusion detection.
  • Datasets: The study uses UNSW-NB15 and CIC-IDS2017 as realistic benchmark datasets containing contemporary attack categories.The supplied methodology passages also describe CIC-IDS2018 frequency distributions and a sampled experimental dataset.

3.2 Data preprocessing

The preprocessing stage prepares data for machine-learning analysis by addressing invalid values, column formatting, and duplicate records.

  • Data cleaning: Preprocessing removes rows containing null, -inf, or inf values, removes spaces from column names, and drops duplicate rows.The paper presents these operations as safeguards against invalid models, overfitting, errors, and low accuracy.

3.3 Feature Scaling to normalize the features

Feature scaling normalizes numerical inputs, while label encoding converts categorical values into numerical representations for machine-learning algorithms.

  • Standardization: Standardization subtracts each feature’s mean and divides by its standard deviation.The paper identifies this transformation as z-score normalization for features with substantially different values.
  • Standardization: After standardization, features have mean 0 and standard deviation 1.The paper states that Equation 1 represents this z-score normalization.
  • Label encoding: Label encoding replaces categorical values with integers from 0 to n-1, where n is the number of unique classes.This conversion enables categorical data to be used during model training.

3.4 Feature Resampling using Random Oversampling (RO)

Random oversampling addresses class imbalance by replicating minority-group instances in the training data to rebalance class distributions.

  • Random oversampling: Random oversampling arbitrarily replicates minority instances and incorporates them into the training portion of an imbalanced dataset.The paper presents RO as a naive method for rebalancing class spreading.
  • Random oversampling: The proposed framework uses RO for big imbalanced datasets to balance the data and improve performance without causing overfitting.The RO process is depicted in Figure 7.

3.5 Stacking Feature Embedded using Clustering with PCA

SFE-PCA combines clustering-based feature embedding with PCA to enrich network-traffic representations while reducing dimensionality. The framework targets big and imbalanced intrusion-detection datasets by balancing representation detail with computational efficiency.

  • SFE-PCA overview: SFE-PCA combines clustering and dimensionality reduction to improve machine-learning model performance.The approach is presented as a methodology for big and imbalanced network-intrusion datasets.
  • Stacking Feature Embedding: K-Means and Gaussian Mixture clustering group data points by intrinsic patterns before clustered results are embedded into the original feature space as meta-features.This augmentation is intended to capture finer data details than conventional feature representations.
  • Feature Extraction using PCA: PCA converts correlated variables into uncorrelated variables and reduces the feature space while preserving essential information.The paper uses PCA to reduce computation and retain important dataset information for intrusion detection.
  • Feature Extraction using PCA: PCA projects samples into a new k-dimensional subspace after centering, scaling, covariance calculation, and eigenvector selection.The projection is expressed as y = wT ∗x, where x is a d×1 sample vector and y is a k×1 converted vector.
  • Feature Extraction using PCA: The selected reduction ratios are 22.22% for UNSW-NB15 and 12.65% for CIC-IDS2017, supporting higher accuracy with a lower false rate.These ratios correspond to 10:45 and 10:79 output-to-input feature dimensions, respectively.

3.6 ML Algorithms

The paper evaluates several supervised tree-based machine-learning algorithms for intrusion detection, including DT, RF, ET, and XGB. These methods use decision rules, randomized ensembles, or gradient boosting to classify network data.

  • Decision Tree (DT): Decision Tree classification derives output predictions from decision rules learned from dataset features.Information gain and the Gini index support feature selection and node splitting, while pruning removes unnecessary nodes.
  • Random Forest (RF): Random Forest combines multiple decision-tree classifiers trained on diverse dataset subsets to improve predictive accuracy and mitigate overfitting.It operates as an ensemble meta-predictor using averaging.
  • Extra Tree (ET): Extra-Trees applies randomized decision trees to dataset subsamples and averages their outputs for classification or regression.The method builds unpruned trees and randomly selects node cut-points.
  • Extreme Gradient Boosting (XGB): Extreme Gradient Boosting uses gradient-boosted decision trees, residuals from earlier models, and gradient descent to reduce loss and improve performance.The associated training loss evaluates predictions on the training datasets, while regularization controls model complexity.

4 Experimental setup and Evaluations

The experimental setup evaluates DT, RF, ET, and XGB on binary and multilabel classification tasks using standard classification metrics. Ten-fold cross-validation divides each dataset into 90% training and 10% testing for every fold.

  • Experimental setup: The experiments use the CIC-IDS2017, CIC-IDS2018, and UNSW-NB15 benchmark datasets with four classifiers: DT, RF, ET, and XGB.Performance is examined for both binary and multilabel classification.
  • Evaluation measures: Accuracy, precision, recall, F1-score, ROC curves, and confusion matrices are used to evaluate the proposed model.The confusion matrix distinguishes true positives, true negatives, false positives, and false negatives.
  • Evaluation measures: ROC curves visualize the trade-off between classifier sensitivity and specificity across classification thresholds.This supports classifier selection under differing error costs and accuracy expectations.
  • Cross-validation: Ten-fold cross-validation trains each model on nine folds and tests it on the remaining fold, averaging the resulting performance measures.Each fold uses 90% of the data for training and 10% for testing.

5 Result Analysis

The proposed reduced-feature model improves intrusion-detection accuracy across UNSW-NB15, CIC-IDS2017, and CIC-IDS2018, with ensemble and tree models generally leading. Results also show strong class discrimination and low error rates.

  • UNSW-NB15: 99.59% accuracy was achieved by RF with the proposed feature set in UNSW-NB15 binary classification.
  • UNSW-NB15: 99.95% accuracy was achieved by both RF and ET in UNSW-NB15 multilabel classification with the proposed feature set.
  • UNSW-NB15: 99.98% AUC was achieved by both RF and ET in UNSW-NB15 binary classification, while RF and ET led the multiclass AUC results.
  • CIC-IDS2017: 0.0% false-positive rates were reported for DT, RF, and ET in CIC-IDS2017 binary classification, with ET recording a 0.05% false-negative rate.
  • CIC-IDS2018: 99.94% accuracy was achieved by both DT and ET on CIC-IDS2018 with the proposed feature set, exceeding RF at 99.93% and XGB at 98.87%.

6 Time Complexity

The paper evaluates time complexity as an efficiency consideration for IDS models and analyzes several commonly used classifiers. The reported formulations relate computational cost to data points, features, trees, weights, epochs, or tree depth.

  • Time complexity is treated as important for assessing efficient IDS operation.
  • DT has typical time complexity O(n · m · log(m)), where n represents data points and m represents features.
  • RF has time complexity O(t · n · m · log(m)), where t is the number of trees.
  • MLP has time complexity O(w · e · n), with w as weights, e as epochs, and n as data points.
  • KNN has training complexity O(n · m), while its prediction phase can be more computationally intensive.
  • LGB has time complexity O(t · n · m), whereas XGB generally has complexity O(t · d), where d is tree depth.

7 Conclusion

The study combines oversampling, clustering-based feature embedding, and PCA with multiple ML classifiers to address imbalanced data, feature representation, and dimensionality reduction in network intrusion detection. Evaluated across three benchmark datasets, the approach achieved high reported accuracies, while the authors identify the absence of deep learning and optimization techniques as a limitation.

  • The model combines Random Oversampling, Kmeans and GM clustering-based feature embedding, and PCA to address imbalanced data, feature embedding, and dimension reduction.
  • 99.95% accuracy was achieved by ET on UNSW-NB15, while RF achieved 99.59% accuracy on the same dataset.
  • 99.99% accuracy was achieved by DT, RF, and ET on CIC-IDS2017.
  • 99.94% accuracy was achieved by DT and RF on CIC-IDS2018.
  • Feature embedding captured nuanced patterns, while PCA reduced computational complexity and enhanced model interpretability.
  • The study did not employ deep learning models with optimization techniques, leaving those approaches for future work.

Declarations

The declarations report no relevant conflicts of interest, identify the datasets as free and open-access sources, and document the authors’ contributions across research activities.

  • The authors declare no conflicts of interest relevant to the article’s content.
  • The study uses free and open-access UNSW-NB15, CIC-IDS2017, and CIC-IDS2018 datasets.
  • Author contributions span conceptualization, methodology, software, validation, investigation, supervision, visualization, and writing.
Loading 2401.12262v1…