Source-linked AI summary
A Novel Hierarchical Intrusion Detection System based on Decision Tree and Rules-based Models
Ahmed Ahmim, Leandros Maglaras, Mohamed Amine Ferrag, Makhlouf Derdour, Helge Janicke
TL;DR
The paper addresses intrusion detection challenges involving rare attacks, attack misclassification, outdated datasets, and evaluation overhead. It proposes a three-classifier hierarchical IDS combining REP Tree, JRip, and Forest PA, with the third classifier receiving the original features and earlier outputs. On CICIDS2017, the model reports the highest overall detection rate and accuracy and the lowest false alarm rate among compared classifiers.
Problem
The paper targets low detection of rare attacks, attack misclassification, time overhead, and the limited practical value of commonly used outdated KDD and NSL-KDD datasets.
Method
The IDS combines REP Tree, JRip, and Forest PA in three classifiers, with the third using the original features plus the first two classifiers’ outputs.
Results
94.475% overall detection rate, 96.665% accuracy, and 1.145% false alarm rate are reported for the hierarchical model.
Takeaways & Limitations
The CICIDS2017 evaluation reports that the hierarchical model outperformed compared classifiers across detection rate, accuracy, false alarm rate, and computational time.
Abstract
from arXiv · showhide
This paper proposes a novel intrusion detection system (IDS) that combines different classifier approaches which are based on decision tree and rules-based concepts, namely, REP Tree, JRip algorithm and Forest PA. Specifically, the first and second method take as inputs features of the data set, and classify the network traffic as Attack/Benign. The third classifier uses features of the initial data set in addition to the outputs of the first and the second classifier as inputs. The experimental results obtained by analyzing the proposed IDS using the CICIDS2017 dataset, attest their superiority in terms of accuracy, detection rate, false alarm rate and time overhead as compared to state of the art existing schemes.
I. INTRODUCTION
The paper motivates a hierarchical IDS for evolving cyber attacks and the limitations of common benchmark datasets. It targets accurate attack-type classification with low false alarms and computational overhead, using CICIDS2017 for evaluation.
- Cyber attacks against systems processing sensitive information are becoming more sophisticated, increasing protection concerns for critical infrastructures.
- IDSs are categorized by classification model, monitoring mode, and, for industrial control systems, analysis basis.
- Accurate attack-type classification is needed because incorrect classifications can lead to countermeasures that harm system or network operation.
- Common KDD and NSL-KDD datasets are outdated because benign and malicious traffic has changed substantially since 1999.
- The proposed hybrid IDS combines REP Tree, JRip, and Random Forest to address rare-attack detection, attack misclassification, and time overhead.
III. PROPOSED MODEL
The proposed hierarchical model uses three classifiers to progressively refine network-traffic classification from benign-versus-attack detection to specific attack types.
- The model is designed to reduce classification errors caused by confusing attacks with normal behavior or with other attack categories.
- The first classifier labels each row as benign or attack using dataset features.
- The second classifier uses dataset features to classify each row as benign or one of several attack categories.
- The third classifier combines the original features with the first two classifiers’ outputs to identify benign traffic or a specific attack type.
A. Operation Mode
The proposed hierarchical model operates in two stages: training and testing.
- The model’s operation mode consists of training and testing steps.
1) Training Step:
Training proceeds hierarchically: the first two classifiers are trained first, and their outputs are then used to train the third classifier.
- 1) Training Step:: The first two classifiers are trained after relabeling rows as Attack or Benign and normalizing the dataset features.
- 1) Training Step:: The third classifier is trained with two added columns containing the first and second classifiers’ training outputs.
2) Test Step:
The test step processes each test row through two models, appends their outputs to the row’s features, and then classifies it using a third model.
- 2) Test Step:: Models 1 and 2 first process each test-data row independently.Their outputs are then added to the corresponding row’s feature set.
- 2) Test Step:: Model 3 receives the original features together with the outputs of models 1 and 2.It assigns each resulting row to Benign or a specific attack type.
IV. EXPERIMENTATION
The experimentation section describes the implementation environment used to evaluate the proposed intrusion detection model.
- IV. EXPERIMENTATION: Experiments were conducted on a Windows 10 64-bit PC with 8 GB RAM and a 2.7 GHz Intel Core i5 CPU.Weka Data Mining Tools and a MySQL database were used for implementation.
- IV. EXPERIMENTATION: The study covers the dataset, preprocessing procedure, performance metrics, model structure, and comparisons with different classifiers.These components define the paper’s experimental evaluation.
A. Data set and Data pre-processing
The study uses CICIDS2017, preprocesses its records and features, and organizes training and test subsets for evaluating attack classification.
- A. Data set and Data pre-processing: CICIDS2017 was selected because it satisfies eleven stated characteristics of a valid IDS dataset.These include attack diversity, complete capture, feature set, metadata, heterogeneity, and labelling.
- A. Data set and Data pre-processing: CICIDS2017 contains 2,830,743 rows across eight files, with 79 features per row.Each row is labelled Benign or one of fourteen attack types.
- A. Data set and Data pre-processing: Rows with Flow Packets/s equal to Infinity or NaN were removed before constant-valued features were eliminated.The removed constant-valued features include several bulk, PSH, and URG flag features.
- A. Data set and Data pre-processing: Training and test subsets were extracted after concatenating the eight files, ensuring that the same row did not appear in both subsets.Training rows were selected first by attack type, followed by random selection of remaining rows for testing.
- A. Data set and Data pre-processing: Feature values were normalized using their minimum and maximum values.The normalization equation is introduced for each feature value x_i of feature j.
B. Performance metrics
IDS performance is evaluated by whether network traffic is assigned the correct type, using attack-specific and global metrics derived from classification outcomes.
- B. Performance metrics: The confusion matrix enumerates the possible classification cases used to evaluate IDS performance.Performance is assessed according to the correctness of network-traffic type classification.
- B. Performance metrics: Attack-specific metrics include the detection rate for each attack type and the true negative rate.These metrics assess performance separately for individual attack categories.
- B. Performance metrics: Global metrics include overall detection rate, false alarm rate, and accuracy.The paper summarizes these metrics with corresponding calculation equations.
- B. Performance metrics: Overall detection rate is calculated from true positives and false negatives across each attack type.The displayed formula aggregates TP and FN terms for the attack types.
- B. Performance metrics: Accuracy combines classification outcomes across attack types and benign traffic.The displayed expression includes true-positive, false-negative, true-negative, and false-positive terms.
C. Practical structure of our model
The hierarchical model requires preprocessing and selects three classifiers through testing alternative compositions. Its third classifier adds the outputs of the first two as features.
- C. Practical structure of our model: The model requires preprocessing, attack-specific row labelling, and artificial features produced by two classifiers.The first classifier uses Attack/Benign labels, while the second uses more specific attack categories.
- C. Practical structure of our model: The authors tested several three-classifier compositions to identify the configuration with optimal performance.The paper states that the choice of classifiers was the most important and critical step.
D. Comparative Study
The study compares the hierarchical model with established and recent classifiers across attack-specific and global metrics. The model achieves the strongest overall reported detection, accuracy, and false-alarm results, with acceptable timing.
- D. Comparative Study: The hierarchical model achieves the highest true negative rate, 98.855%, in the comparative evaluation.The comparison includes well-known and recent classifiers across different attacks and benign traffic.
- D. Comparative Study: The model records the highest detection rates for seven listed attack types, including Heartbleed and Infiltration at 100%.It is also close to the highest detection rate for FTP Patator at 99.636% and SSH Patator at 99.909%.
- D. Comparative Study: 94.475% overall detection rate and 96.665% accuracy are the model’s highest reported global values.The same evaluation reports a lowest false alarm rate of 1.145%.
- D. Comparative Study: 1.145% is the model’s lowest reported false alarm rate in the global comparison.Its training time is 195.5 seconds and its test time is 2.27 seconds.
- D. Comparative Study: The model’s 195.5-second training time and 2.27-second test time are described as acceptable for a hybrid hierarchical model.The paper specifically compares these times with simpler models such as MLP and SVM.
V. CONCLUSIONS
The paper proposes a three-classifier hierarchical IDS whose later classifier uses outputs from earlier classifiers. On CICIDS2017, it reports leading overall detection, accuracy, and false-alarm results with low computational time.
- V. CONCLUSIONS: The proposed IDS combines REP Tree, JRip, and Forest PA in a three-classifier hierarchical model.Outputs from two classifiers are used as inputs for the third.
- V. CONCLUSIONS: Evaluation on CICIDS2017 reports the highest true negative rate and highest detection rate for seven attacks.The conclusion describes comparisons with well-known and recent machine-learning models.
- V. CONCLUSIONS: 94.457% overall detection rate, 96.665% accuracy, and 1.145% false alarm rate are reported for the hierarchical model.The conclusion also states that its low computational time supports incorporation into a soft real-time system.