A comparison of several intrusion detection methods using the NSL-KDD dataset

detection system to differentiate between normal and aberrant behavior. In order to successfully finish this learning stage, a dataset pertaining to security is necessary. If the training of the anomaly detection system is completed properly, it has the ability to identify and predict both new and zero-day threats. This is an opportunity for ICT to develop appropriate measures to protect its resources.


INTRODUCTION
Today, cybersecurity is considered an essential technology, as cyberattacks are increasing in frequency.Several different security protocols must be employed to safeguard assets from cyber intrusion...It is estimated that cybercrimes will result in a global economic loss of up to six trillion dollars by the end of 2021 [1].This represents a significant financial impact on both the economy and the technology sector.The intrusion detection system (IDS) proves its efficacy in thwarting intrusion attempts and serves as an important security component that cannot be overlooked.Firewalls and antivirus software have limited effectiveness against various network threats.It is important to detect cyberattacks and collect extensive data about them when they occur.These two essential and necessary advantages are implemented to protect the infrastructure of information and communications technology (ICT).The use of Intrusion Detection Systems (IDS) offers inherent benefits that stem from its operational mechanisms.Intrusion Detection Systems (IDSs) are classified according to their detection algorithms, which are used for identifying anomalies and detecting abusive behavior [2].Misuse detection relies on the presence of the security attack signature.In addition, due to the unavailability of its signature to Intrusion Detection Systems (IDSs), it is incapable of detecting new attacks.However, its distinctive characteristics enable accurate identification of familiar attacks.Anomaly detection systems in IDS use the attributes of network traffic to detect new attacks.To obtain optimal accuracy, it is necessary to train an anomaly detection system to differentiate between normal and aberrant behavior.In order to successfully finish this learning stage, a dataset pertaining to security is necessary.If the training of the anomaly detection system is completed properly, it has the ability to identify and predict both new and zero-day threats.This is an opportunity for ICT to develop appropriate measures to protect its resources.

Problem Statement
As previously said, researchers from all around the globe are working to build intrusion detection systems that may mitigate the issue of rising assaults.Here, we attempt to do a comparative analysis of the most recent anomaly IDS research conducted over the last three years, using machine learning and deep learning approaches on the NSL-KDD dataset.Since the NSL-KDD dataset is the most well-known cyber security benchmark, working on it is our main goal.

Contribution of Paper
This report presents a comparative analysis of many research publications utilizing the Network Security Laboratory Knowledge Discovery and Data Mining (NSL-KDD) Dataset that are linked to anomaly detection [3].This is how the rest of the paper is structured.Background information and related work are provided in Section 2. Section 3 contains the Subjects and Methods.A detailed presentation of the NSL-KDD-based intrusion detection systems is given in Section 4. The models' implementation, assessment metrics, findings, and discussion are given in Section 5. Section 6 offers a conclusion and recommendations for further study at the end of this essay.

Background and Related work
In this section we are discussing different types of IDSs depending on its deployment, response and detection; then related work is discussed.

Background
Intrusion detection systems are used to identify and accurately classify attacks and intrusion attempts occurring on a host or network.Host-based intrusion detection systems (HIDS) are specifically designed to monitor and identify intrusion attempts on a single host, while network intrusion detection systems (NIDS) are designed to monitor and detect intrusion attempts on an entire network [4].The aforementioned classification is based on the data source [5] [6].In addition, intrusion detection systems (IDS) can be divided into two types based on their operation method: active IDS (modifying the environment) and passive IDS (logging and alerting on unauthorized access) [7].IDS can be divided into three subcategories: stateful protocol analysis, anomaly-based, and signature-based IDS.The last knowledge-based approach is called stateful protocol analysis, which is a specification-based approach, sometimes also called signature-based.It can successfully defend against known attacks, detect unknown attacks using anomaly-based detection, and detect unknown attacks using stateful protocol analysis [8].In order to choose the right solution for each specific scenario, it is important to understand the goal of using an intrusion detection system (IDS).

Related work
A research study presented in [9] introduced a deep learning detection system using Deep Neural Network (DNN) for software-defined networks.This system only utilizes six fundamental network features (duration, protocol_type, src_bytes, dst_bytes, count, and srv_count) from the NSL-KDD dataset.The system achieved an accuracy of 75.75% for anomaly detection.The paper suggested the combination of Non-Symmetric Deep Auto-Encoder (NDAE) with Random Forest (RF) in order to increase the accuracy of current approaches for abnormality detection.The new method achieved an accuracy of 89.22% using the 13-class NSL-KDD dataset, which was the most increased reported accuracy until 2018.Additionally, the proposed method also resulted in a time saving of 98.81%.In research, the vector format raw traffic is converted into picture data format.Subsequently, the authors used a Convolutional Neural Network (CNN) intrusion detection model to enhance the accuracy, surpassing the performance of current Machine Learning based methods, achieving a precision of 79.48%.The study conducted in [12] introduced a method called Hierarchical Combining of Predictions of a Tree of Classifiers (HCPTC-IDS).The performance of this method was then compared to other approaches such as NB, FL, RIPPER, DT, ANN, and SVM.The model efficiently processes each record in 373 microseconds, demonstrating its rapid data traffic processing capabilities on the NSL-KDD Dataset, achieving an accuracy of 89.75%.The technique described in [13] combines Sparse Auto-Encoder (SAE) with Support Vector Machine (SVM).The Self-Taught Learning Intrusion Detection System (STLIDS) Utilizes Self-Taught Learning (STL) for data model and Support Vector Machines (SVM) for classification.This deep-learning intrusion detection system has an accuracy of 84.96%.The GRU-RNN, as introduced in reference [14], improves the efficiency of anomaly detection.They employed a Feedforward Deep Neural Network (FFDNN) for classification.Furthermore, studies have shown that the precision of the intrusion detection system is strongly influenced by the number of neurons used in the FFDDN classifier.This technique increases the precision to 87.74% by utilizing 30 nodes and 3 concealed layers.The study in [16] introduced a flexible ensemble learning framework that utilized the NSL-KDD dataset and included methods such as random forest, decision tree, and Deep Neural Network (DNN) to train the model.The proposed adaptive ensemble learning model is expected to have an accuracy rate of 85.2%.The paper [2] introduces the Scalable Hybrid Intrusion Detection Alertnet (SHIA) architecture, which examines network-level and host-level data to detect intrusions.In addition, it proposed utilizing a Deep Neural Network (DNN) to identify and detect cyber-attacks.The experimentation was conducted using NSL-KDD.The binary class classification achieved an accuracy of 80.1% using a one-layer deep neural network (DNN), while the multi-class classification achieved an accuracy of 78.5% using a five-layer DNN.The paper referenced as [17] presents the Improved Conditional Variational Auto-Encoder Deep Neural Network (ICVAE-DNN) model.The NSL-KDD dataset is used to evaluate this model.The accuracy, detection rate, and incorrect positive rate of the ICVAE-DNN model are then compared to six other classification algorithms: K-Nearest Neighbors (KNN), Multinomial Naive Bayes (NB), Random Forest (RF), Support Vector Machine (SVM), Deep Neural Network (DNN), and Deep Belief Network (DBN).It has been shown that its precision surpasses theirs.The accuracy achieved by (ICVAEDNN) is 85.97%.Self-adaptive and autonomous misuse the IDS (Intrusion Detection System) was introduced in [18] and relies on Self-taught learning combined with a technique called MAPE-K.Self-taught learning is an advanced method of deep learning that can effectively detect previously unknown assaults by reconstructing unlabeled data.When combined with the MAPE-K reference model, it achieves an impressive accuracy of 77.99% in identifying these unseen attacks.The Auto-Encoder (AE) and statistical analysis model are introduced, using the NSL-KDD dataset from reference [19].The results demonstrate the influence of adding or reducing Improving accuracy through hidden layers.Specifically, a single Hidden Layer (HL) consisting of 50 units achieves an accuracy of 84.24% for Binary classification and 87% for multi-Classification.A Deep learning system was introduced in [20] that demonstrated superior accuracy compared to previously existing systems, using a Spark Cluster setup.The system under consideration is referred to as DLS-IDS (Deep Learning Spark-IDS).The use of Long-Short Term Memory (LSTM) in conjunction with Synthetic Minority Over-Sampling Technique (SMOTE) resulted in a notable enhancement in the detection accuracy, achieving a commendable rate of 83.57%.The Difficult Set Sampling Technique (DSSTE) method [21] is proposed to enhance the classification model for learning unbalanced dataset data.The NSL-KDD dataset is utilized as a benchmark dataset.Their accuracy reached 82.84%.The study [22] shown a clear relationship between the accuracy of detection and the quality of data collection.To address this, the researchers suggested a 5-layer Auto Encoder model that improves the identification of anomalous network traffic with improved accuracy.The suggested methodology was evaluated using the NSL-KDD dataset, achieving a superior accuracy rate of 90.61%.In [23], a hybrid machine learning model and a feature selection strategy were proposed and deployed to the NSL-KDD dataset.The model picked seventeen characteristics for analysis.The suggested model achieved an accuracy result of 90.41%, surpassing other learning models by 11% in terms of accuracy and detection rate.The use of the ReLU activation function in deep neural networks, along with principal component analysis (PCA) as described in [24], results in a data processing acceleration and achieves an accuracy of 88.64%.The system proposed in [25], which combines an upgraded random forest with the synthetic minority oversampling technique (SMOTE), achieves an accuracy of 78.47%.The proposed system, GMM-WGAN, is a multimodule integrated intrusion detection system.It consists of three components: feature selection, imbalance processing, and classification.In a previous study [26], the system attained an accuracy of 86.59%.The proposed model is a deep neural network (DNN) that has been trained using 28 features from the NSL-KDD dataset.Additionally, a feature scaling technique described in [27] has been used.The model achieves an accuracy of 81.87%.

NSL-KDD Dataset
The KDD99 dataset, which was generated in 1999, became one of the most extensively used study datasets in the field of cyber security [28].After extensive investigation on KDD99, researchers have identified some drawbacks that need resolution, including redundancy and the excessive quantity of records in both the training and testing datasets.These issues pose challenges when working with the whole dataset in experiments.In order to address the aforementioned drawbacks, a more recent iteration called NSL-KDD was suggested in [3].Since 2009, NSL-KDD has been recognized as the primary dataset for cyber security research.The NSL-KDD dataset comprises of two subsets: KDDTrain+ with 80% or 125,973 records and KDDTest+ with 20% or 22,544 records.Each record consists of 41 characteristics, which are divided into four distinct categories: Basic features, time-based Traffic features, connection-based Traffic features, and Content features [29].There are twenty-one predicted label classes assigned to each record, representing both attack and normal records.Inside the cyber security field, every individual piece of data is regarded as a session, representing a link between two hosts inside a network.The probability distribution of KDDTrain+ differs from that of KDDTest+.The test dataset includes some assaults that are not included in the training data.The training dataset consists of 24 distinct kinds of assaults, whereas the testing dataset includes an additional 14 types of attacks that are not included in the training set.This is done to evaluate the classifier's capacity to recognize unknown attacks.NSL-KDD provides a novel approach that enhances the KDD99 dataset.For instance, the KDD99 dataset classifies probing as an attack, but the NSL-KDD dataset does not classify it as an attack until the number of rounds exceeds a certain threshold.Table 1 provides a comprehensive overview of the NSL-KDD record details.

Intrusion Detection Systems Based on NSL-KDD
As seen in Figure 1, Deep Learning (DL) is classified as a subset of Machine Learning (ML) methods.Machine Learning comprises a broad array of approaches that allow computers to learn from data.On the other hand, Figure 2 shows that Deep Learning primarily refers to methods that rely on neural networks with several layers, often known as deep neural networks [30].It can use a small or medium amount of data to build good models for its algorithm, which works better with organized and structured data.
Thanks to its multi-layer architecture, it works and excels at processing large amounts of unstructured data, such as images, text, and documents.

2
It divides the problem into sub-problems and solves them one by one.
It solves the problem in one step using its multiple layers. 3 Its algorithms have a relatively simple structure, such as linear regression or decision tree.
It is based on a complex, multi-layered, interconnected artificial neural network that mimics the structure of the human brain.
4 It can run on a central processing unit (CPU).It Requires a graphics processing unit (GPU) and more powerful hardware to function properly.

5
Its algorithms require greater human intervention to select, and process features to identify the correct input.
It can extract features automatically or with minimal intervention for the algorithm to learn and process from its errors and data. 6 In training, it takes a shorter time, But during testing, it becomes slow.
In training, it takes longer; However, during testing, it becomes faster.
7 Despite this, some of ML algorithms are still the fastest.
In this section we will give a simple summary about the interested IDS algorithms as follows:

Artificial Neural Network (ANN)
Artificial Neural Network is a specialized branch of machine learning.The Artificial Neural Network (ANN) seeks to develop a machine learning system that is modeled after the biological structure of the human brain.Artificial neural networks (ANNs) consist of many layers [32].The structure consists of an input layer, a hidden layer, and an output layer.In the context of the input layer, every feature in the dataset is represented by a neuron.The input is sent to the subsequent layer.The Hidden Layer is comprised of a collection of neurons, with each neuron being given a weight.Each layer receives information from the preceding layer.The final outcome is derived from the output layer.Neural Networks need a significant amount of additional computational power.There are three sequential operations that need to be carried out in any neural network, as seen in Figure 3: * Compute the predicted Y values (Ypred) by taking the input variables xi and utilizing the linear combination formula.Next, calculate the loss or error term.The error term is the difference between the observed values and the projected values Ypred.* Increase the loss function or error term in order to reduce its importance.

The CNN-LSTM Model
The proposed model by the authors is comprised of Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM).The primary design of the Convolutional Neural Network (CNN) involves executing convolutional and pooling operations on the input data, then passing it through a fully connected layer that carries out the classification task [33].The selected architecture, LSTM, is very consistent in translation and has a significant ability to predict the short term data sequence.In this study, the scientists chose to combine LSTM with CNN in order to recognize long-range dependencies in features.The authors employed the local nature of CNN to more comprehensively explore the data's properties.Later, they employed the sensitivity of the LSTM to the order of data characteristics in order to mitigate its effects.The The fusion approach attempts to take advantage of the diversity of learning methods to create a more powerful and accurate intrusion detection system.By combining methods like random forest, decision tree, and deep neural networks, the ensemble model can take into account different aspects of the data and have a more informed approach to potential cyber dangers.

The RECURRENT NEURAL NETWORK (RNN) Model
The RNN model is similar to the CNN model in terms of temporal structure.It's composed of three layers: input, a middle layer that has the ability to forward and backward propagate, and an output layer.RNNs have a superior capacity to deal with tabular data, as well as to classify, predict, and régress.The LSTM component of the RNN model helps solve problems associated with effectively predicting sequences.The structure of the RNN model is illustrated in Figure 5. [34] The RNN model has an input layer, an LSTM layer, hidden layers that are composed of a sigmoid function, and a final output layer that is dense.It employs the Adam optimizer and cross entropy loss.

MULTI-LAYER PERCEPTRON (MLP) Model
The Multilayer Perceptron model is sometimes also called the MLP model.It is a regression model that transforms input information into a complex structure.The structure of the data that is non-linear is inputted into the intermediate layer of the perceptron that is hidden, there, it is processed and then transmitted to the output layer.The middle hidden layer is composed of a Non-Linear function that performs regression predictions and addresses classification issues.The output is calculated by taking the weighted inputs and adding a bias to each layer's output.Figure 6 illustrates the Multilayer Perceptron (MLP) model.The MLP model is often considered to be a universal emulator because of its underlying design, which is based on XOR operations.The MLP, or MultipleLayer Perceptron, is a type of neural network that employs the backpropagation algorithm for the purpose of learning.This algorithm is intended to specifically teach the model how to handle increasing functions.[35]. .Choosing the appropriate number of epochs is essential to ensure that the model efficiently learns from the data without suffering from overfitting or underfitting.By monitoring the model's performance on a validation set and using strategies like as early halting, one may ascertain the ideal number of epochs.

Evaluation Metrics
• Accuracy is the percentage of correctly predicted incidents that are compared to the original (real) label.The more accurate the information, the more exact the created predictions become.Cybersecurity researchers attempt to improve the fidelity of their model to identify typical or unusual computer network incidents.Accuracy is determined by dividing the total number of accurate predictions by the total size of the dataset.The most accurate level is 1.Accuracy = (TP + TN) / (TP + FP + TN + FN) (1).TP is the number of true positive cases, TN is the number of true negative cases, FP is the number of false positive cases, and FN is the number of false negative cases.• Recall, also called sensitivity, is the percentage of violent incidents that are correctly classified.Recall is measured in terms of the fraction of accurate positive predictions among the total number of positive instances.The word "true positive rate" (TPR) is another term for the same.The optimal value for sensitivity is 1.Recall = TP/(TP + FN) (2).• Precision is the percentage of correctly predicted attacks that are actually committed.The math involves taking the number of accurate positive predictions and dividing it by the total number of positive predictions.It's occasionally called the positive predictive value (PPV).The optimal degree of accuracy is 1.Precision = TP / (TP + FP) (3).
• A greater number for both recall and precision indicates better performance.The F1-Score metric is used to capture the advantages of recall and accuracy in a single measure.The harmonic mean of accuracy and recall is determined as follows: F1-Score = 2TP / (2TP + FP + FN) (4).• To address the problem of typical occurrences being recognized as attacks, the False Positive Rate may be used.
It is calculated by dividing the count of false positive predictions by the total number of negatives.The optimal false positive rate is 0.0, FPR = FP / (FP + TN) (5).

Results and Discussion
• The RNN, MLP, CNN_LSTM, and ANN algorithms were applied to several trials, considering the limits indicated earlier.The number of epochs used in each trial varied, with options ranging from 10 to 100 epochs.• Table 3 presents a performance evaluation of the suggested Intrusion Detection Systems (IDSs).All of them have the similar requirements of utilizing the NSL-KDD dataset and evaluating accuracy using an appropriate number of epochs as a measure., and 10, it can be concluded that 10 is the best number of epochs.As a crucial part of this analysis, the optimal number of epochs to prevent divergence between the two curves was selected as 10 epochs.Figure 10 shows the accuracy training and validation curve for this experiment.

Conclusions and Future Work
• The Internet of Things (IoT) is an excellent platform for connecting consumers worldwide without the need for human intervention.These networks are susceptible to several types of assaults and abnormalities since they lack sensors for monitoring.Several Intrusion Detection Systems (IDSs) were suggested in order to safeguard IoT systems.Most of these methods have restricted scalability and precision.Current intrusion detection systems (IDSs) continue to encounter difficulties in enhancing detection accuracy, decreasing the proportion of false alarms, and identifying unknown assaults.• The objective of this research is to do a comparative analysis of intrusion detection systems as they are applied to the NSL-KDD dataset.This work is driven by two specific goals.Firstly, it is the dataset that is most often used.Furthermore, it is widely regarded as the fundamental and most often used dataset in cyber security research [36].• This comparison study will assist researchers in gaining a comprehensive grasp of the latest advancements in IDSs, which will ultimately enhance intrusion detection performance.The accuracy of the Table 3 indicates that the MLP and RNN models were more accurate than the CNN-LSTM and ANN models after 100 epochs.The combination of CNN and LSTM had a lower accuracy than the individual models, this is likely due to the higher complexity and the potential for overfitting.ANN had a simpler design than MLP, but it was less successful at recognizing intricate patterns.RNNs are more effective at sequentially processing data than basic ANNs.• Future planned work will concentrate on enhancing the dependability of IDS through extensive study.Exploring the recently proposed datasets, such as the CIC-IDS2018, can help with this.Additionally, to prevent unauthorized access and threats to the system, a mechanism for intrusion prevention will be studied.

FIGURE 1 .
FIGURE 1. -A Venn diagram showing how DL is considered a subset of ML

Table 1 . -NSL-KDD record Details
Table 3 displays the precision of the suggested models on NSL-KDD.MLP and RNN have the best accuracy, whilst ANN exhibits the lowest accuracy.• Table 4 presents a performance comparison of the IDSs indicated in section (2.2) in relation to the suggested IDSs.Each of the recommended Intrusion Detection Systems (IDSs) clearly demonstrates a greater accuracy value compared to the maximum value provided in section (2.2).The ANN IDS exhibits the lowest accuracy among the suggested IDSs, while its accuracy surpasses that of the 5LAE IDS mentioned in section (2.2). • The training accuracy and validation accuracy curves obtained from each experiment provide an analysis of the experiment results.These curves play a crucial role in assessing the performance and behavior of a deep learning model.The MLP and RNN curves exhibit a similar pattern, diverging at the desired point between 10 and 17 epochs.However, the CNN_LSTM and ANN curves do not show a clear determination of their epochs.Based on the analysis of Figures No. 7, 8, 9