PaperlayerView exact revision on arXiv

Source-linked AI summary

Building an Efficient Intrusion Detection System Based on Feature Selection and Ensemble Classifier

Yuyang Zhou, Guang Cheng, Shanqing Jiang, Mian Dai

arXiv:1904.01352v4cs.CRcs.LG

TL;DR

Existing IDSs struggle with redundant features, imbalanced traffic, and classifiers that may not detect every attack type effectively. The paper proposes CFS-BA feature selection with a C4.5, Random Forest, and Forest PA ensemble using probability voting, and reports strong results across three datasets. The method also sharply reduces CIC-IDS2017 model-building time, while rare attacks remain a limitation.

  • Problem

    Redundant and irrelevant attributes, imbalanced traffic, and insufficient individual classifiers limit intrusion-detection performance.

  • Method

    CFS-BA selects correlation-based feature subsets, then C4.5, Random Forest, and Forest PA are combined with average-of-probabilities voting and 10-fold cross-validation.

  • Results

    The method reports strong Accuracy, Detection Rate, and False Alarm Rate results across NSL-KDD, AWID, and CIC-IDS2017, including 99.89% accuracy on CIC-IDS2017.

  • Takeaways & Limitations

    CFS-BA-Ensemble provides a competitive IDS approach that reduces feature dimensionality and improves efficiency while maintaining strong reported detection performance.

  • Takeaways & Limitations

    Performance is not guaranteed for every attack type, particularly rare classes such as Heartbleed and U2R.

Abstract

from arXiv · show

Intrusion detection system (IDS) is one of extensively used techniques in a network topology to safeguard the integrity and availability of sensitive assets in the protected systems. Although many supervised and unsupervised learning approaches from the field of machine learning have been used to increase the efficacy of IDSs, it is still a problem for existing intrusion detection algorithms to achieve good performance. First, lots of redundant and irrelevant data in high-dimensional datasets interfere with the classification process of an IDS. Second, an individual classifier may not perform well in the detection of each type of attacks. Third, many models are built for stale datasets, making them less adaptable for novel attacks. Thus, we propose a new intrusion detection framework in this paper, and this framework is based on the feature selection and ensemble learning techniques. In the first step, a heuristic algorithm called CFS-BA is proposed for dimensionality reduction, which selects the optimal subset based on the correlation between features. Then, we introduce an ensemble approach that combines C4.5, Random Forest (RF), and Forest by Penalizing Attributes (Forest PA) algorithms. Finally, voting technique is used to combine the probability distributions of the base learners for attack recognition. The experimental results, using NSL-KDD, AWID, and CIC-IDS2017 datasets, reveal that the proposed CFS-BA-Ensemble method is able to exhibit better performance than other related and state of the art approaches under several metrics.

1. Introduction

The paper motivates an IDS that addresses high-dimensional, imbalanced traffic and weaknesses of single classifiers by combining feature selection with ensemble learning. The proposed framework uses CFS-BA, multiple classifiers, and probability-based voting, then compares the approach across three datasets.

  • Research motivation: Imbalanced traffic makes it difficult to achieve a high Attack Detection Rate while maintaining a low False Alarm Rate.Only a fraction of network traffic may indicate malicious behavior while normal traffic dominates.
  • Research motivation: Single classifiers may be insufficient for intrusion detection, motivating ensemble models that combine outputs from multiple classifiers.Ensemble learning averages or combines decisions from individual classifiers to produce a classification decision.
  • Research motivation: High-dimensional traffic contains redundant and irrelevant attributes that interfere with IDS classification and increase computational complexity.Feature selection is presented as a way to identify relevant features while eliminating useless ones.
  • Proposed solution: The proposed methodology combines feature selection and ensemble classification to provide efficient and accurate intrusion detection.The paper identifies this combination as its central methodological contribution.
  • Proposed solution: CFS-BA selects features using correlation, while the ensemble combines C4.5, Random Forest, and Forest PA through average-of-probabilities voting.The voting classifier combines the probability distributions of the base learners for attack recognition.
  • Evaluation: Experiments on NSL-KDD, AWID, and CIC-IDS2017 report improvements over existing methods in Accuracy, F-Measure, and ADR while keeping FAR acceptable.The comparison uses an extensive three-dataset testbed.

2. Related work

Related work applies feature selection, ensemble learning, and hybrid combinations to improve IDS performance. Prior studies report accuracy, false-positive, detection-rate, and efficiency benefits across different models and datasets.

  • Scope of related work: Related IDS research considered machine-learning approaches that use feature selection, ensemble classifiers, or hybrid combinations.The review focuses especially on hybrid approaches combining the two techniques.
  • Feature selection: Feature-selection studies removed irrelevant attributes, with one NSL-KDD result reporting highest accuracy for C4.5 with information gain using only 17 features.The cited work illustrates dimensionality reduction with limited feature counts.
  • Ensemble learning: Bagging and cluster-based ensembles were reported to improve classification accuracy, false positives, or detection rate compared with other techniques.Examples used REPTree, ADTree, and k-nearest neighbors as ensemble components.
  • Hybrid approaches: Hybrid studies combined feature selection with Random Forest or bagging to obtain more appropriate features, higher accuracy, and lower false-positive rates.The reviewed methods used particle swarm optimization or gain ratio with tree-based ensembles.

3. Proposed methodology

The proposed IDS preprocesses traffic, reduces dimensionality with CFS-BA, trains three tree-based classifiers, and recognizes attacks using 10-fold validation and probability voting. The framework is intended to classify benign traffic and varied intrusive events accurately.

  • Framework phases: The framework begins by preprocessing original datasets into a format suitable for analysis.This is the first of four stated framework phases.
  • Framework phases: CFS-BA reduces dataset dimensionality and selects relevant features for each attack type.This stage addresses the high-dimensional dataset problem.
  • Framework phases: C4.5, Random Forest, and Forest PA are trained as base learners and combined into an ensemble classifier.The three classifiers are decision-tree-based components of the ensemble.
  • Framework phases: A 10-fold cross-validation test uses average-of-probabilities voting to combine base-learner probability distributions for classification decisions.The voting stage performs attack recognition from the ensemble outputs.
  • Framework outcome: The resulting ensemble detects and classifies benign traffic and various intrusive events with high classification accuracy.The framework description states this outcome after the ensemble-classifier stage.

3.1. Feature selection

CFS-BA combines correlation-based feature evaluation with bat-algorithm search to reduce redundant features and optimize a feature subset. The selected subset then reduces the training and testing dimensionality.

  • Feature-selection rationale: Feature selection seeks a representative subset of highly prediction-relevant attributes and is categorized as wrapper, filter, or embedded.The paper uses a hybrid approach rather than relying on a single feature-selection category.
  • Correlation-based feature selection: CFS favors feature subsets strongly correlated with the class but weakly inter-correlated with one another.This criterion addresses both relevance and redundancy.
  • Correlation-based feature selection: For a subset S with k features, CFS evaluates mean feature-class correlation and average inter-feature correlation using M_s.A larger r_cf or smaller r_ff produces a higher evaluation value, according to the supplied description.
  • Correlation-based feature selection: The subset with the highest CFS evaluation is used to reduce both the training and testing sets.The selected subset is the output of the correlation-based evaluation process.
  • CFS-BA integration: CFS-BA uses correlation-based fitness evaluation within BA to optimize the integrity of the reduced feature subset.The paper presents this hybrid as the feature-selection process used before classification.
  • Bat algorithm: BA searches the feature-space solution using bat locations, iterative movement, archived best solutions, and local random-walk search.The algorithm is inspired by microbat echolocation and includes population-based exploration.

3.2. Ensemble classification

The ensemble classification section describes C4.5, Random Forest, Forest PA, and probability-based voting as complementary components for multi-class intrusion detection.

  • Base classifiers: The proposed ensemble combines C4.5, RF, and Forest PA because their different mechanisms support multi-class intrusion detection.
  • C4.5: C4.5 selects decision-tree splits using gain ratio and can handle discrete, continuous, and missing attribute values.
  • Random Forest (RF): Random Forest constructs multiple decision trees from bootstrapped training subsets, with each tree contributing a class vote.
  • Forest PA: Forest PA builds highly accurate, diverse trees by exploiting all non-class attributes and adjusting attribute weights.
  • Vote: Average-of-probabilities voting assigns the class with the highest average predicted probability, supporting multi-class classification.

4. Evaluations and results

The evaluation trains and tests the CFS-BA-based ensemble classifier on three intrusion-detection datasets using the average-of-probabilities rule.

  • Evaluation setup: The hybrid CFS-BA method selects a feature subset, while the ensemble combines C4.5, RF, and Forest PA using AOP.
  • Evaluation setup: Experiments use three datasets and are conducted with Weka 3.8.3 on a desktop computer.

4.1. Description of the benchmark datasets

The benchmark comprises NSL-KDD, AWID, and CIC-IDS2017, spanning revised simulated data, real WiFi traces, and updated attack traffic.

  • Dataset motivation: Real-world intrusion-detection datasets are difficult to acquire because released network data are often anonymized or altered.
  • NSL-KDD: NSL-KDD revises KDDCup’99 by removing redundant records, rationalizing instance counts, and preserving sample diversity.
  • NSL-KDD: The experiments use KDDTrain+, KDDTest+, and KDDTest-21, with cross-validation and hold-out testing.
  • AWID: AWID contains real normal and intrusive 802.11 traffic, and the study evaluates its reduced four-class AWID-CLS-R-Tst set.
  • CIC-IDS2017: CIC-IDS2017 contains benign traffic and updated attacks, while experiments use its Wednesday-workingHours subset with six categories.

4.2. Dataset preprocessing

Preprocessing removes problematic records, converts symbolic values to integers, normalizes feature scales, and applies CFS-BA to select reduced feature subsets.

  • Preprocessing: Preprocessing includes data filtration, transformation, and normalization to make heterogeneous raw data suitable for analysis.
  • Data filtration: The study removes anomalous or redundant records, including duplicated features and Infinity or NaN values.
  • Data transformation: Symbolic feature values are replaced with integers because the datasets contain symbolic, continuous, and binary attributes.
  • Normalization: Minimum-maximum normalization maps features onto a normalized range to reduce scale-related effects on classification.

4.3. Results and discussion

Across repeated 10-fold validation and benchmark comparisons, CFS-BA-Ensemble generally improves classification performance while reducing model-building time. Its weaknesses are concentrated in attack classes with very few instances, and its superiority over prior methods cannot be claimed universally because comparison conditions are incomplete.

  • Feature selection and ensemble construction: CFS-BA drastically reduces dimensionality and removes irrelevant features before a voting ensemble of three decision-tree classifiers performs prediction.The ensemble combines C4.5, Random Forest, and Forest PA through a voting algorithm.
  • Limitations and class imbalance: Rare attacks remained difficult: CIC-IDS2017 contained only 11 Heartbleed instances, while KDDTrain+ contained 52 U2R instances.The authors state that selecting features for all classes cannot guarantee strong performance for attack types with very few examples.
  • Feature selection and ensemble construction: Feature selection increased average Acc, precision, DR, F-Measure, and ADR compared with using all features.The comparison attributes the improvement to selecting relevant features with CFS-BA.
  • Performance results: 0.998 accuracy, 0.998 F-Measure, 0.997 ADR, and 0.001 FAR were achieved on NSL-KDD by CFS-BA-Ensemble.The method performed best across the three reported classifier sets for this dataset.
  • Performance results: 0.999 accuracy, 0.999 DR, and 0.999 ADR were achieved on CIC-IDS2017 using only 13 features.The CFS-BA-Ensemble results exceeded the best accuracies reported for the individual C4.5, RF, and ForestPA classifiers with CFS-BA selection.
  • Computational efficiency: CFS-BA reduced CIC-IDS2017 ensemble model-building time from 977.94s to 98.42s, with all three datasets remaining below 100s after selection.The reported original-feature overhead was approximately 500s for AWID and 1000s for CIC-IDS2017.

5. Conclusions

The paper addresses high-dimensional and unbalanced network traffic with CFS-BA feature selection and an ensemble classifier, evaluated across three intrusion-detection datasets. The proposed method reports strong accuracy, detection, false-alarm, and efficiency results, while rare attacks remain a future improvement target.

  • CFS-BA selects an optimal feature subset based on feature correlation, while C4.5, RF, and ForestPA form the ensemble classifier.The IDS is evaluated using 10-fold cross-validation on three intrusion-detection datasets.
  • 99.81% accuracy, 99.8% DR, and 0.08% FAR are achieved on NSL-KDD using 10 features.
  • 99.52% accuracy and 0.15% FAR are achieved on AWID using 8 features, while CIC-IDS2017 reaches 99.89% accuracy and 99.9% DR using 13 features.
  • On CIC-IDS2017, feature selection reduces MBT from 977.94s to 98.42s compared with using no feature selection.
  • The method outperforms related feature-selection approaches in Acc, F-Measure, ADR, and efficiency while maintaining relatively low FAR.
  • Future work could improve the method's capability to handle rare attacks in massive network traffic.
Loading 1904.01352v4…