1 Introduction

With the growth of the internet and the increasing use of technology in our daily lives, cybercrime has become a major concern for individuals, businesses, and governments alike. Cybercrime refers to criminal activities that are carried out using computers or the internet, such as hacking, phishing, identity theft, and malware attacks [1]. Many of the applications (like online banking, e-commerce, and healthcare services) which we use in our daily lives contain confidential and personal information that needs to be protected. To protect these applications, it is important to take a proactive approach to cybersecurity [2]. Furthermore, with the increasing number of connected smart devices in the IoT environment, there are also increasing security threats and vulnerabilities. Therefore, additional security considerations are necessary to safeguard these devices and the data they transmit. AI-based security solutions such as anomaly and intrusion detection and network traffic monitoring can be useful tools in enhancing IoT security [3, 4].

The concept of intrusion detection (ID) dates back to 1970 when it was extensively adopted to protect computer networks against both known and unknown attacks [5]. An intrusion detection system (IDS) is software that monitors a network for malicious activities and generates an alarm signal to be sent to the security team. Anomaly- and signature-based are the two main methods used in IDS. Signature-based IDS detects attacks based on matching input data with the signatures of known attacks. An anomaly-based IDS catches attacks by comparing abnormal behaviour to normal behaviour. Signature-based detection is unable to detect attacks that have not been seen before while anomaly-based detection often has high false positive rates [6].

Machine learning (ML) algorithms have been used for over 20 years to improve the performance of IDS [7]. Two types of ML have been for building anomaly detection models: shallow learning and deep learning. In general, shallow learning (Bayesian networks, support vector machines (SVMs), and artificial neural networks (ANNs)) depends on extracting features creating the prediction model [2] while deep learning has the ability to generate superior models by extracting better representations from the raw data [8]. Deep learning is a type of ML that uses artificial neural networks with multiple layers to learn hierarchical representations of data. DL can learn feature hierarchies based on massive amounts of unlabeled information, making it particularly useful for processing complex, high-dimensional data. Examples of DL algorithms include deep neural networks (DNNs) [9], convolutional neural networks (CNNs) [10], and recurrent neural networks (RNNs). One of the advantages of DL is that it can automatically learn relevant features from data, without the need for explicit feature engineering. This makes DL models more adaptable and flexible, as they can handle a wide range of input types and sizes. In the domain of intrusion detection, the most recent papers are using DL [8, 11–13]. RNN is one of the most popular deep learning algorithms for the classification of sequential data due to its recurrent (circular) manner of connections. Thus, RNN can recall all previous knowledge acquired from previous inputs during a training phase [14]. Recently, Long short-term memory (LSTM) has gained much attention due to its ability to solve the drawback of RNN in vanishing gradients by using a gating mechanism to learn long-term dependencies [15]. LSTM is also employed in attack detection, with the ability to detect unique attacks, as in [16].

Swarm intelligence (SI) algorithms (such as Butterfly optimization algorithm (BOA) [17], grey wolf optimizer [18], and particle swarm optimization (PSO) [19]) are widely used in global optimization and parameters tuning. BOA is inspired by the foraging behaviour of butterflies and it has the ability to find the optima in the hyper-search space [17]. Utilizing chaotic maps, a new version of BOA called chaotic Butterfly optimization algorithm (CBOA) has been proposed in [20]. This CBOA algorithm showed to be better than BOA in improving classification accuracy and reducing classification errors.

LSTM algorithm has been used in proposing different intrusion detection methods such as in [21] and in [22]. However, LSTM performance is impacted by extra problems with random weight initialization [23] and overfitting [24]. In other words, although LSTM has been used in many intrusion detection systems but it still suffers from two main limitations: (1) taking high numbers of iterations to find the best weight value of its network which affects the computational costs, and (2) its classification performance is still not high. The objective of this paper is to minimize the number of iterations needed to find the best weight values of LSTM network and improving the classification performance in intrusion detection systems.

To achieve this objective, an improved version of LSTM (i.e., ILSTM) was suggested. In the ILSTM, hybrid swarm algorithms, CBOA and PSO, were employed to optimize LSTM weights while using a fewer number of iterations. The ILSTM was then used for proposing an efficient and accurate intrusion detection system for two cases: binary (normal or abnormal) and multi-class (classifying many attacks) classification.

The contribution of this work can be summarised as follows:

1. Proposing a novel and improved version of LSTM called ILSTM in which hybrid swarm intelligence algorithms (i.e., CBOA and PSO) were employed to optimize the weights of the LSTM algorithm which led to better performance using a fewer iterations.
2. Building an efficient (i.e., fewer iterations) and accurate ILSTM-based intrusion detection system for binary (normal and abnormal) and multi-class classification (classifying more than attacks such as DoS, Prob, and U2R attacks).
3. Evaluating the performance of the new ILSTM and the intrusion detection system. Thorough evaluation was done using nine performance metrics (accuracy, detection rate, false alarm rate, precision, f–measure, false negative rate, mathew correlation coefficient and kappa coefficient) under two public datasets (NSL-KDD dataset and LITNET-2020). The ILSTM performed better than LSTM in both datasets.
4. Comparing the results of the proposed solution with various deep learning algorithms. The comparison demonstrated that the proposed ISTLM gave better results in binary and multi-classification of intrusion types such as DoS, Prob, and U2R attacks.
5. Conducing statistical analysis using Wilcoxon Signed-Rank test which showed that ILSTM is more statistically significant than LSTM.

The subsequent sections of this paper are as follows: Section 2 of the paper discusses some related works on swarm intelligence, deep learning, and network intrusion detection methods. Section reference 3 contains all implemented algorithms that were used in the development of the proposed algorithm. Section 4 includes the proposed algorithm (ILSTM). Section 5 provides an experimental setup for implementation the proposed algorithm, parameter setting, performance metrics, and preprocessing phase on the NSL-KDD and the LITNET-2020 datasets. Section 6 illustrates and discusses the performance of the proposed algorithm in binary and multi-class classification as well as comparisons with other deep learning and machine learning algorithms. Section 7 presents the conclusion of this work and future work.

2 Literature review

IDS are critical components of computer network defense. In prior studies, several approaches proposed intrusion detection based on deep learning. In [14], RNN classifier is proposed by using one hidden layer with eighty hidden nodes and 0.1 learning rate for binary and multi-class classification on the NSL-KDD dataset. However, applying RNN has the drawback of exploding and vanishing gradients, which this method does not solve.

In [8], an integrated intrusion detection model based on a staked denoising auto-encoder and deep belief network (SADE-ELM and DBN-SoftMax)is developed to overcome the shortcomings of existing deep neural network models, including their long learning times and poor classification accuracy, The proposed model only achieves 76.64% for accuracy in binary classification on the NSLKDD dataset.

The authors of [13] developed an intrusion detection model based on bidirectional long short-term memory (BiDLSTM) and convolution LSTM, and the results show that the proposed BiDLSTM is more effective than convolution LSTM. The accuracy of convolution LSTM is 89.81% but BiDLSTM reach to 94.26% in binary classification. Despite BidLSTM gives best result than convolution LSTM, it requires more training time than other compared algorithm.

The authors of [25] proposed a BAT-MC hybrid method of BLSTM and attention mechanism and compare it to other machine learning algorithms (J48, Naive Bay, NBTree, Random Forest, and SVM) using the NSL-KDD dataset in binary classification. The proposed method accomplishes 84.25% for accuracy in binary classification but has the lowest accuracy for U2R and R2L attacks in multi-class classification.

Jiang et al. [26] combined hybrid sampling techniques with deep learning networks (CNN) as a method for intrusion detection. They use one-side-selection (OSS) to reduce the noise samples in the majority categories and increase the minority categories by the synthetic minority over-sampling technique (SMOTE). The accuracy of this method is only 83.58% on the NSL-KDD dataset in binary classification and 82.74% in multi-class classification. However, while this method has a high detection rate for U2R attacks, it has a lower detection rate for other attacks such as (normal, Dos, Prob, R2L).

Chora and Pawlicki [27] studied ANN hyperparameters (activation, optimizers, batch size, epochs, layers, and neurons) for an intrusion detection model using NSL-KDD and CICIDS 2017. When using the parameters tanh, Adam, with 100, 300, 1, and 25, the accuracy was 99.9%. For the other parameters, accuracy dropped to 5.64 percent, demonstrating that the ANN model is sensitive to parameter values. They further did not consider multiclassification of intrusion types such as Dos, Prob, U2R, or R2L.

Multiple researchers have studied the use of swarm intelligence algorithms for machine learning algorithms. ELHasnony et.al [28] developed a hybrid swarm algorithm of BOA and PSO for selecting the best features. Selected features are applied for machine learning algorithm (KNN) with 5 K fold cross-validation for classification. 25 Datasets from UCI machine learning repository and COVID-19 dataset are used to evaluate the proposed algorithm, where proposed algorithm give better result than other swarm algorithms such as BOA, PSO, and GWO. ALsaleh et al. [29] investigated the impact of the salp sarm algorithm (SSA) for feature minimization on improving machine learning network-based anomaly detection classifiers such as XG Boost and Naive Bayes. Improved firefly algorithm is also proposed for optimizing parameters of XGBoost classifier for intrusion detection in [30], the proposed algorithm is tested on the NSL-KDD and UNSW-NB15 datasets. Firefly algorithm reduced the number of features to 19 from 42, where accuracy in binary classification is increased after selection features but other performance metrics are decreased, such as (precision and f-score). In multi-classification, most performance metrics give the best results after selection.

The use of swarm algorithms for deep learning networks was also investigated by researchers. As in [31], where the hybrid deep learning model CNN-OLSTM is used to detect DDos attacks and the grey wolf optimization method is present to choose the best features for detection, but it obtains a very low specificity of 51%. In [11], a feature reduction model based on correlation and information gain, followed by using a RNN classifier for the detection of attacks and non-attacks in a reduced-feature dataset, where 90% of the NSLKDD dataset is used for training. In [32] suggested that using the whale algorithm to optimize the weights of LSTM networks to develop an effective model is called WILS, the abbreviation for whale integrated long short term memory to detect a variety of threats on IoT networks. They used the same dataset for training and testing, using 70% of the NSL-KDD as training data and the remaining 30% for testing data in binary classification.

Some research papers use mathematics algorithms for optimizing weights of LSTM, such as [33] which uses four different optimizer (metaheuristic algorithms) such as harmony search (HS), grey wolf optimizer (GWO), sine cosine (SCA), and ant lion optimization algorithms (ALOA) to train LSTM for maximizing classification accuracy.

The authors in [34] developed a model (OCNN-HMLSTM) by using lion swarm optimization (LSO) for optimization hyperparameters of CNN (spatial features) and using HMLSTM for learning temporal features. The proposed model for NSL-KDD has a binary classification accuracy of 90%, while all attack types (Dos, U2R, Prob, and R2L) have higher false positive rates, reaching 9.92%. In the research paper [35], The authors proposed the firefly algorithm for feature selection of NSL-KDD and KDD Cup 99 datasets, then used DNN for the classification process. Despite the efficiency of the hybrid eFA-DNN framework, it is only proposed for binary classification algorithms.

The authors applied an evolutionary sparse convolution network (ESCNN) in [36] for identifying and tracking attacks in distributed denial of service (DDOS) in the IoT. A variety of DDoS attack-related feature analyses were used to design the technique to reduce network overhead. The proposed network achieves a 98.28% detection rate and 99.29% accuracy in binary classification. In [37], a new feature selection strategy has been proposed using bio-inspired algorithm GWO, in addition authors applied classification method (ELM) refer to extreme learning machine. Modified GWO was tested using the UNSW NB-15 dataset and achieved 78% accuracy. In order to boost the accuracy of a machine learning classifier for intrusion detection systems, relevant features from the UNSW-NB15 and CICIDS-2017 datasets are selected using the artificial bee colony (ABC) algorithm as described in [38]. According to [39], the Firefly algorithm is also used in network intrusion detection to choose features. The Firefly algorithm can choose 10 crucial features from the KDD CUP 99 dataset, which is applied to bayesian networks (BN) and C4.5 based classifiers for anomaly detection. Image recognition has also been applied lately in intrusion detection, as in [40] where a new approach has been proposed using multistage deep learning image recognition that transforms network features into four channel images (Red, Green, Blue, and Alpha) that are used in classification. Results reach 99.8% accuracy for the BOUN Ddos dataset.

From the above literature analysis and summarized in in Table 1, it could be concluded that the performance of the deep learning-based intrusion detection system could be still improved. Such improvement should cover two aspects: binary and multi-class classifications of attacks. It was also noticed that although LSTM has been used in many intrusion detection systems, such as [21, 22, 25] but it still suffers from two main limitations: (1) taking high numbers of iterations to find the best weight value of its network which affects the computational costs, and (2) its classification performance is still not high. In addition, LSTM performance is impacted by extra problems with random weight initialization [23].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of intrusion detection systems.

https://doi.org/10.1371/journal.pone.0284795.t001

3 Preliminary work

In this section, an overview of the algorithms used in our proposed algorithm and intrusion detection system is given.

3.5 Long short term memory (LSTM)

LSTM is an extension of RNN that able to learn long-term dependencies. The LSTM architecture is more complicated than the RNN architecture; it has four hidden layers that use gates to add and remove cell state information [46].

For one LSTM cell, at time step t, the forget, input and output gates are represented by i_t, O_t, f_t, respectively, as shown in Fig 1 which discussed before in [47]. Forget gate decides which information will be deleted from the cell state based on and x_t. The input gate determines which information from the current state will be stored in the cell state and updates it using the ‘tanh layer’ to generate a vector of new contender values. The final output gate decides how the output should look and passes it through the ‘tanh layer’ to the next neuron. The following equations mathematically describe the relationship between the inputs and outputs at time t and t − 1: (12) (13) (14) (15) (16) (17) Where C denotes the cell state The activation functions are defined by sigma (the sigmoid function) and tanh. x is the input vector, and h_t is the output vector. The weights and biases parameters are represented by W and b, respectively. A tanh layer generates a vector of new candidate values, g, which can be added to the state.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The architecture of LSTM cell.

https://doi.org/10.1371/journal.pone.0284795.g001

In this paper, we develop a deeper LSTM network with four hidden layers and two input and output layers. It starts by mapping inputs to their representations using the feature input layer. It then feeds the sequence to two double LSTM layers. LSTM outputs are then fed to two fully connected layers with the rectified linear unit (RELU) as an activation function. Finally, the fully connected layers learn and compile the extracted data from the LSTM layer to form a final output that passes through an output layer for classification. Fig 2 displays a summary of the LSTM network architecture with four hidden layers as a first phase in the proposed algorithm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The architecture of ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g002

4 The proposed ILSTM algorithm

The proposed algorithm (ILSTM) consists of a hybrid LSTM network described in section 3.5 and the hybrid swarm algorithm CBOA and PSO, as briefed in Sections 3.3 and 3.4, respectively. The hybrid CBOA+PSO was used for optimising weights of the LSTM network, which helps in improving the training of the LSTM network in a minimum number of iterations. In general, the proposed ILSTM consists of two main phases as described in Fig 2.In the first phase, the LSTM is traditionally trained to get the best parameters and weights of its internal network architecture. In second phase, the hybrid CBOA+PSO (see Algorithm (3)) was used for optimising weights of the trained LSTM network to further find the optimal weights which can improve the accuracy in both binary and multi-class classification1ion while taking a fewer iteration. More details are in the following sub-sections and in the ILSTM algorithm given in Algorithm (3).

5 Experimental setup

This section gives details about the experimental setup under which the experimental evaluations, in the next section, are conducted. Firstly, all experiments have been conducted on a laptop with an Intel(R) Core(TM) i5-6300U CPU@ 2.50 GHz and 8.00 GB of RAM and the proposed algorithms were implemented using Matlab R2020a running on Windows 10.

In Section (5.1), an overview of the performance metrics used to assess the quality of the proposed algorithm is given. The section then gives a description and the preprocessing of the two public datasets (NSL-KDD 2009, LITNET-2020) used for the evaluation process. Finally, we test our proposed algorithm on a modern dataset, LITNET-2020, to ensure its efficiency. On the other hand, Table 4 displays a summary of the LSTM network architecture. We compare the algorithm’s performance with state-of-the-art and deep learning methods trained and tested on the same dataset (i.e., the NSL-KDD dataset).

5.2 Dataset 1: NSL-KDD dataset

The NSL-KDD dataset is a refined version of the KDD cup [49]. It has a fair distribution of all types of attacks [50]. Many researchers employ the NSL-KDD to develop an effective intrusion detection algorithm, such as in [26, 34, 50]. The NSL-KDD includes 41 attributes that are classified as normal or attack traffic [49]. The NSL-KDD is divided into a training dataset (KDDTrain+) and two testing datasets, KDDTest+ and KDDTest-21. All of these datasets have normal records and four types of attack records, such as probe, remote to local (R2L), denial of service (Dos), and user to root (U2R). In this paper, all of the KDDTrain+ dataset is used for training, and all of two other datasets (i.e., KDDTest+ and KDDTest-21) are used for testing, where the training dataset represents 80% of the NSL KDD dataset and the testing dataset represents 20% of the NSL KDD dataset as shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. NSL-KDD dataset description.

https://doi.org/10.1371/journal.pone.0284795.t002

5.2.1 Dataset preprocessing.

KDDTrain+, KDDTest+, and KDDTest-21 datasets are preprocessed before being used for training and testing the LSTM network and the proposed ILSTM. We apply preprocessing step on raw dataset to better make full use of domain knowledge of network traffic. It contains three processes: (1) mapping symbolic features to numeric values; (2) sampling imbalanced classes in the dataset; and (3) normalizing features that have a large scale.

1. Data transformation The NSL-KDD dataset has 38 numeric features and 3 non-numeric features such as “protocol-type,” “service,” and “flag”. As LSTM classifier accepts only numeric values, we first convert non-numeric features, as in [51, 52], where we replace every single value with an integer in order to handle non-numeric features as in Table 3. One-hot encoding makes our training data more useful and expressive, and it can be rescaled easily. By using numeric values, we can more easily determine the probability of our values. In particular, one hot encoding is used for our output values since it provides more nuanced predictions than single labels. Each value is converted to binary code, so a protocol type with three values (tcp, udp, and icmp) becomes 1, 2, and 3, which are recognised as [1, 0, 0], [0, 1, 0], and [0, 0, 1]. Finally, NSLKDD has 122 features.
   In order to further analyse NSL-KDD, we used SHAP analysis for indication of structural predictors (inputs) that have the strongest influence on the particular output. This was done by evaluating the effect of each feature on the target variable and indicating the importance of each feature in determining the final predicted outputs. The outcome of the SHAP analysis is given in Figs 3 and 4.
2. Dataset balancing using hybrid sampling There are several aspects that might influence the performance of learning systems. One of these aspects is related to class imbalance, which occurs when training data have a larger number of examples for one class than other classes, such as in the NSL-KDD dataset. The classes in NSL-KDD network traffic data are not represented equally, where Normal and Dos have larger examples than other types of attacks such as U2R, R2L and Prob so these imbalanced data make a problem with classification as the prediction of the majority class is increased while the detection of the minority class is very low. Prior to [26], hybrid sampling was used, and the results were better than those of a standard dataset. Synthetic minority over-sampling technique (SMOTE) is an over-sampling method [53]. SMOTE forms new minority class examples by matching several minority class examples that lie together. SMOTE can avoid overfitting and make minority class boundaries spread through majority class space. To balance majority classes, the random under sampling (RUS) [54] technique is used to reduce the number of examples of the majority classes in the training dataset.
3. Normalization Some features in the NSL-KDD dataset, such as “duration,” “src-bytes”, and “dst-bytes” have a large scope between the minimum and maximum values, which can degrade the classification performance [55]. So, we applied the minimum-maximum normalization method [53] which maps features into the normalized range [0, 1]. This method can be defined as in Eq [32]. (32) Where X_min and X_max are the minimum and the maximum values of feature x.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Shap analysis for NSL-KDD in binary classification.

https://doi.org/10.1371/journal.pone.0284795.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Shap analysis for NSL-KDD in multi-class classification.

https://doi.org/10.1371/journal.pone.0284795.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Transformation of symbolic features in NSL-KDD.

https://doi.org/10.1371/journal.pone.0284795.t003

5.2.2 Parameter setting.

To determine the value of the parameters of the selected algorithms, we study the performance of LSTM network on NSL-KDD. Then the hybrid algorithms (i.e., CBOA+PSO) was used to optimize the weights of LSTM network and finally we evaluate the performance of the proposed algorithm ILSTM in binary classification (normal, anomaly) and five category classification (multi-classification) such as (Dos, Prope, R2L, and U2R). KDDTest+ dataset is used to determine the optimal parameters and network topology of the algorithm. These parameters and network topology are then applied to the KDDTest-21 and LTNET-2020 datasets. The parameters used in the simulation of the LSTM network are shown in Table 4, where:

1. The adaptive moment estimation (Adam) algorithm is used to update LSTM network’s parameters. For the binary classification, the loss function was cross-entropy while for multi-classification the categorical cross-entropy was used. We applied regularization in range [0.01, 0.001], which came down to adding a cost to the loss function for large weights to ensure that our network does not overfit the data.
2. When the learning rate of the network is too high, the loss function of networks will oscillate without convergence. If the learning rate is too low, the slow convergence rate will hinder the updating of networks. Therefore, choosing an appropriate learning rate is very important for network performance optimization. As in Fig 5 we studied the impact of a set of learning rates [0.1, 0.01, 0.001, 0.0001] in binary and multi-classification on the KDDTest+ dataset and selected the best learning rate that achieves high accuracy.
3. An essential component of choosing the overall neural network architecture is determining the number of neurons in the hidden layers. Applying too few neurons in the hidden layers will result in a problem called underfitting. When too many neurons are used in the hidden layers, a problem known as overfitting occurs and training time is increased. In this paper, we assumed that the number of hidden neurons should be between the size of the input layer and the output layer in a network model, so we applied Eq 33 as in [56] to get the best values. (33) Where N_i = number of input neurons, N_o = number of output neurons, N_s = number of samples in training data set, α = an arbitrary scaling factor usually be in the range [2, 10] and N_t = the number order for hidden layer.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Performance test on KDDTest+ with increasing learning rate.

https://doi.org/10.1371/journal.pone.0284795.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Parameter setting for LSTM.

https://doi.org/10.1371/journal.pone.0284795.t004

Table 5 studies parameters for the CBOA and PSO algorithms, which are applied in various research papers like as [17, 28].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Parameter setting for CBOA and PSO.

https://doi.org/10.1371/journal.pone.0284795.t005

5.3 Dataset 2: LITNET-2020 dataset

LITNET-2020 dataset is a relatively new dataset collected by LITNET (Lithuanian research and education network) academic network in Lithuania’s real-time network traffic. It is a real-world and up-to-date flow-based network dataset [57] which is developed to test IDS systems. In this dataset, there were 85 network flow features and 12 attack types, a summary of the attacks and their instances are given in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Summary of LITNET-2020 dataset.

https://doi.org/10.1371/journal.pone.0284795.t006

5.3.1 Dataset preprocessing.

By studying the LITNET-2020 dataset, it was found that it has many features, such as “fwd, opkt, and obyt,” which have only one unique value. Additionally, it contains source and destination IP and port numbers which are distinct features and could not be used in attack detection. Therefore, there were only 16 features available for attack classification. Further pre-processing was done where all categorical features were encoded using label encoding. It was also noticed that some features, such as “sp” and “dp”, have a large gap between the minimum and maximum values, which can degrade the classification performance. So, we applied the minimum-maximum normalization method 32 which maps features into the normalized range.

Further to that, we use SHAP analysis to explain the proposed algorithm’s prediction by calculating the contribution of each feature to the prediction, because SHAP analysis shows the importance of each feature on the target variable [58]. The results of the SHAP analysis is illustrated in Fig 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. SHAP analysis for LITNET-2020 dataset.

https://doi.org/10.1371/journal.pone.0284795.g006

5.3.2 Dataset balancing using hybrid sampling.

LITNET-2020 dataset suffers from imbalance problem in class distribution, where the number of normal instances (benign) reaches 3/4 of the size of the dataset, as shown in Table 6. To address this problem, hybrid sampling, as given in point 2 in subsection 5.2.1, was applied to produce a balanced the datasets.

5.3.3 Data splitting approach.

We divided the LITNET-2020 dataset to 60% for training process and 40% for testing and validation. We choose this approach after conducting a small experiment aiming to find out the best data-splitting approach. The results of this experiment are summarized in in Fig 7. As shown in this figure, we divided the LITNET-2020 dataset into 4 different training and testing sets. We then tested all of them and it was found that 60:40 set is the best approach.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. LITNET-2020 data splitting approach.

https://doi.org/10.1371/journal.pone.0284795.g007

6 Results and discussion

The section reports the experimental results and their discussion which were conducted on the two datasets described above. For each dataset, two main experiments are implemented to study the performance of the proposed algorithm, ILSTM. In the first experiment, the proposed algorithm is investigated for binary classification (i.e., normal or malicious traffic), while in the second experiment, ILSTM is evaluated against multi-class classification (i.e., to differentiate among normal, dos, prob, U2R, or R2L). Also, in each experiment, (1) a statistical analysis (Wilcoxon test) was performed to show the significance of the ILSTM algorithm, and (2) a comparison with other deep learning and machine learning methods was conducted to demonstrate the efficiency of the ILSTM algorithm.

6.1 Experiment 1: ILSTM performance for binary classification on NSL-KDD dataset

The aim of this experiment is to assess the performance of the proposed ILSTM for intrusion detection in the case of classifying network traffic into normal or abnormal (i.e., binary classification). This was done on KDDTest+ and KDDTest-21 datasets, as detailed below.

6.1.1 ILSTM Performance using KDDTest+ dataset.

To evaluate the performance of the proposed ILSTM, it is compared with the original LSTM and two optimized versions of LSTM using BOA and CBOA. A summary of the results of this experiment is given in Table 7. These results were recorded from an average of ten runs on the KDDTest+ dataset. It is clear from this table that the proposed ILSTM algorithm gave the best results achieving an accuracy of 91.31%, a specificity of 96.46%, and a FAR of 3.51% (which is a very important value for intrusion detection systems). Other best results are shown in bold text in this table. For detailed results of this experiment, the confusion matrix was reported in Fig 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Confusion matrices for KDDTest+ in binary classification.

https://doi.org/10.1371/journal.pone.0284795.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Comparison between LSTM, LSTM-BOA, LSTM-CBOA and ILSTM using KDDTest+ and KDDTest-21 in binary classification with average 10 runs.

https://doi.org/10.1371/journal.pone.0284795.t007

Another experiment was conducted on the KDDTest+ dataset to investigate the relationship between the accuracy and the number of iterations of the proposed ILSTM and original LSTM. The results of this experiment were plotted in Fig 9. From this figure, it can be noticed that the ILSTM took iterations less than the LSTM but the latter achieved a higher accuracy. In this Fig, two curves are represented as follows: (a) a conventional LSTM achieved an accuracy of 82.74% at iteration 70 and did not improve between 70-100 iterations while Fig 9b shows that after using the CBOA and PSO for LSTM weights (i.e., ILSTM) the accuracy improved to 92.41% at iteration 80 and only increased by 1% from iteration 80-100.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Accuracy and number of iterations for LSTM and ILSTM using KDDTest+ in binary classifications.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g009

6.1.2 ILSTM Performance using KDDTest-21 dataset.

The same above experiment was conducted but using the KDDTest-21 dataset. The aim is to compare the proposed ILSTM with the original LSTM, LSTM-BOA, and LSTM-CBOA. A summary of the results is given in Table 7. An average of ten runs were used to get these results. Also, the confusion matrix for all implemented algorithms in this experiment is shown in Fig 10. From these results, it could be concluded that the proposed ILSTM algorithm gave the best results in an accuracy of 86%, specificity 99%, precision 97.9%, and FAR 0.07 (which is a very important value for intrusion detection systems) and when it is small it means that the IDS is efficient. Other best results are shown in bold text in this table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Confusion matrices for KDDTest-21 in binary classification.

https://doi.org/10.1371/journal.pone.0284795.g010

Using the KDDTest-21 dataset, we also investigated the relationship between the accuracy and the number of iterations of the proposed ILSTM and original LSTM. The results of this experiment were plotted in Fig 11. From this figure, it can be noticed that after applying the optimization phase using CBOA and PSO on LSTM, the accuracy improved with 18% from iterations 72-76, see Fig 11B while it remained constant at 68.95% for LSTM without any optimization see Fig 11A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. LSTM vs ILSTM for KDDTest-21 for binary classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g011

6.1.3 Efficiency of ILSTM algorithm in binary classification.

Mean squared error (MSE) is used to calculate the difference between the actual value and the obtained value for measuring the performance in the optimization phase of LSTM. MSE is calculated using the following equation as in [59]. (34) Where x_i is a vector of actual data, y_i is a vector of predict data and n is the number of instances in the testing dataset.

Fig 12 summarizes the results of the optimized weights of ILSTM. It can be seen that accuracy is not only improving but also the lowest value of MSE is reached in binary classification (normal or abnormal traffic) using two datasets: KDDTest+ in Fig 12(a) and KDDTest-21 in Fig 12(b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. MSE for ILSTM algorithm in binary classification.

A: KDDTest+. B: KDDTest-21.

https://doi.org/10.1371/journal.pone.0284795.g012

6.1.4 Wilcoxon signed-rank test for binary classification on KDDTest+ and KDDTest-21 datasets.

In this section, we implement the Wilcoxon signed-rank test to demonstrate the effectiveness of the proposed algorithm in binary classification using two testing datasets. As in Table 8, statistics results from Wilcoxon are defined between the accuracy of conventional LSTM and the accuracy of the proposed algorithm (ILSTM), where the mean difference is -10.2 for KDDTest + and equal to -22.86 for KDDTest -21. The value of z is 2.8031, and the p-value is 0.00512 for the KDDTest+ and KDDTest-21 datasets. The results in Table 8 show that the p = 0.00512 which is lower than the significance level (0.05), so the null hypothesis should be rejected and it means that there is a significant difference between the proposed ILSTM algorithm and the other algorithms. This statistical analysis confirms the numerical results reported above.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Statistic Test for KDDTest+ and KDDTest-21 in binary classification.

https://doi.org/10.1371/journal.pone.0284795.t008

From Tables 7 and 8, three remarks can be noticed. Firstly, when using a chaotic map with BOA as in CBOA algorithm, the results of all evaluation metrics were improved. This is due to the fact that a chaotic map increases the search space for new solutions while avoiding local minima. Secondly, when using Eq 11 of PSO algorithm in local search instead of Eq 8, the search for new solutions is improved because the velocity helps in searching for local and global best solutions. So, using CBOA and PSO in our proposed ILSTM algorithm improved the results of intrusion detection in both binary classification in most performance metrics such as (Acc, Spc, Prec, FAR, f1-score, MCC, and Kapp) on KDDTest+ and KDDTest-21 datasets. Finally, the optimization process improved the LSTM network with better results in performance metrics and statistical tests than the conventional LSTM.

6.1.5 Comparison of the ILSTM algorithm with related methods.

In order to objectively evaluate the performance of ILSTM, we conducted a comparison with other deep and machine learning methods that were implemented in the intrusion detection literature. In this comparison, we used machine learning and deep learning methods reported in previous work such as [13, 14, 25]. The results of this comparison are reported in Fig 13 for machine learning methods and in Fig 14 for deep learning methods. From these figures, it can be concluded that our proposed algorithm outperformed all other algorithms in binary classification.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Comparison between ILSTM and machine learning-based algorithms in binary classification.

https://doi.org/10.1371/journal.pone.0284795.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Comparison between ILSTM and deep learning-based algorithms in binary classification.

https://doi.org/10.1371/journal.pone.0284795.g014

In addition, Table 9 shows the results of comparison with other methods suggested for binary classification. Those methods were used in [34]. From this table, it can be seen that the proposed ILSTM algorithm has achieved the best results in most measures, where the best values are shown in bold text.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Comparison between other methods in literature using KDDTest+ for binary classification.

https://doi.org/10.1371/journal.pone.0284795.t009

6.2 Experiment 2: ILSTM performance for multi- class classification on NSL-KDD dataset

The aim of this experiment is to assess the performance of the proposed ILSTM for intrusion detection in the case of classifying network traffic into different types of attacks (i.e., multi-classification) where there are 5 classes of data Normal and 4 types of attacks (Dos, Prob, U2R, R2L). Three sub-experiments are conducted. The first and second are designed for the performance evaluation of ILSTM under the KDDTest+ and KDDTest-21 datasets, respectively while comparing it with the most related work. The third experiment was for comparison with the other related work under eight performance metrics.

6.2.1 ILSTM performance using the KDDTest+ dataset.

This experiment aims to study the performance of ILSTM on accurately identifying four types of attacks (Dos, Prob, U2R, R2L) using KDDTest+ dataset. Also, the results of ILSTM were compared with conventional LSTM and other optimized versions of LSTM (i.e., LSTM-BOA, and LSTM-CBOA). All four algorithms (LSTM, LSTM-BOA, LSTM-CBOA, and ILSTM) were implemented and executed under the same environment to ensure a fair comparison. The results of these experiments are summarized in Table 10. In addition, the confusion matrix for all implemented algorithms is shown in Fig 15. From these results, it can be noticed that ILSTM outperformed almost all other algorithms under all evaluation metrics. This is due to the integration of CBOA with PSO in the proposed ILSTM algorithm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Confusion matrices of ILSTM using KDDTest+ for multi-classification.

https://doi.org/10.1371/journal.pone.0284795.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Comparison between LSTM, LSTM-BOA, LSTM-CBOA and ILSTM using KDDTest+ in Multi-class classification with average 10 runs.

https://doi.org/10.1371/journal.pone.0284795.t010

Also, the ILSTM was compared with the LSTM in terms of the number of iterations needed to achieve the highest accuracy and the results were plotted in Fig 16. From this figure, it can be seen that the proposed ILSTM algorithm achieved a higher accuracy with fewer iterations compared with the conventional LSTM, which reaches 79.72% with 100 iterations as shown in Fig 16(a); however, the ILSTM only needs 72 iterations to attain an accuracy of 88.17% as shown in Fig 16(b). So, the proposed ILSTM can improve intrusion detection performance and also save on computational costs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. LSTM vs ILSTM for KDDTest+ in multi-class classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g016

6.2.2 ILSTM performance using KDDTest-21.

To further evaluate the proposed ILSTM, we repeated the same experiment above but using a different dataset, namely KDDTest-21. The results of this experiment are summarized in Table 11. Also, confusion matrices of all compared algorithms in this experiment are plotted in Fig 17. From this table and the confusion matrix, it can be seen that the ILSTM achieved the best results when compared to other implemented algorithms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 17. Confusion matrices for KDDTest-21 in multi classification.

https://doi.org/10.1371/journal.pone.0284795.g017

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Comparison between LSTM, LSTM-BOA, LSTM-CBOA and ILSTM using KDDTes-21 in multi-class classification with average 10 runs.

https://doi.org/10.1371/journal.pone.0284795.t011

Under the multi-classification scenario, we also investigated the relationship between the accuracy of ILSTM and its number of iterations in comparison with the original LSTM. The results of this experiment were plotted in Fig 18. From this figure, it can be noticed that the optimization of LSTM using CBOA and PSO can boost the accuracy by 20% in 10 iterations, while the accuracy of conventional LSTM remains constant starting from iteration 60 to iteration 100. It can be seen that the conventional LSTM gave an accuracy of 52.54% with 100 iterations as illustrated in Fig 18(a), while ILSTM achieved an accuracy of 76.73% in the same number of iterations as illustrated in Fig 18(b), where optimization process started just after the LSTM accuracy becomes constant, i.e., at iteration 60.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 18. LSTM vs ILSTM for KDDTest-21 in multi-class classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g018

6.2.3 Efficiency of ILSTM algorithm in multi-class classification.

For a further thorough evaluation of the optimization of LSTM, the MSE was computed using Eq 34 in the context of multi-class classification. The results are summarized in Fig 19 which shows that optimizing the weights of a conventional LSTM network can enhance the multi-class accuracy (i.e., detecting different type of attacks) while also achieving the lowest MSE in multi-class classification for the KDDTest+ and KDDTest-21 datasets, where the KDDTest+ in Fig 19(a) provided the lowest MSE compared with the KDDTest-21 in Fig 19(b).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 19. MSE evaluation for the ILSTM algorithm in multi-class classification.

A: KDDTest+. B: KDDTest-21.

https://doi.org/10.1371/journal.pone.0284795.g019

6.2.4 Wilcoxon signed-rank test for multi-class classification on KDDTest+ and KDDTest-21 datasets.

To evaluate the significance of the ILSTM, we implemented the Wilcoxon signed-rank test in multi-class classification using two testing datasets, KDDTest+ and KDDTest-21 datasets. The Wilcoxon statistics results of the accuracy of the proposed ILSTM algorithm and the conventional LSTM are shown in Table 12. From this table, it can be seen that the value of z is 2.8031 and the p-value is.00512 for the KDDTest+ and KDDTest-21 datasets, and the mean difference for KDDTest+ is -7.99 and the mean difference for KDDTest-21 is -21.29. The results also show that the p = 0.00512 is lower than the significance level (0.05). This means that the null hypothesis should be rejected and it means that there is a significant difference between the proposed ILSTM algorithm and the other algorithms. This statistical analysis confirms the numerical results reported above.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 12. Statistic test for KDDTest+ and KDDTest-21 in multi-class classificatio.

https://doi.org/10.1371/journal.pone.0284795.t012

As a conclusion of the results given in Tables 10 and 11, ILSTM achieved better than LSTM, LSTM-BOA, and LSTM-CBOA in terms of DR, Spec, FNR, and MCC for all attack classes (Normal, Dos, Prob, R2l, U2R). This means using a hybrid optimization of CBOA and PSO helped in increasing search space for best solutions and finding global optimal solutions in all testing datasets.

6.2.5 Comparison of the proposed ILSTM algorithm and the other related algorithms.

As in the case of binary classification, we also compared ILSTM with other published work about multi-classification attacks. This comparison included deep and machine learning methods which have been published in the literature of intrusion detection context [13, 14, 25]. As shown in Figs 20 and 21, the proposed ILSTM algorithm outperformed other existing machine learning and deep learning methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 20. Comparison with machine learning methods in multi-class classification.

https://doi.org/10.1371/journal.pone.0284795.g020

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 21. Comparison with deep learning methods in multi-class classification.

https://doi.org/10.1371/journal.pone.0284795.g021

Given the comparison summarized in Table 13, it could be noticed that the proposed ILSTM algorithm achieved the lowest false alarm rate in all types of attacks. Also, the ILSTM can reach higher DR, Precision and f-measure in most types of attacks, the best results are written in bold text. In comparison with other methods, it can be seen that ILSTM gave superior results in Dos, Prob and R2l attacks in terms of Recall, Precision, F-score and FAR. Additionally, ILSTM can produce good results in normal but not as well under R2L attacks. This may be due to the limited number of instants in U2R and R2L attacks in the datasets. Hybrid sampling can help to resolve this issue by achieving results that are as excellent as those of R2L attacks. The best FAR results for U2R attacks, while also increasing accuracy to 31.88, but still not the best in most metrics for U2R attacks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 13. Comparison between other methods in literature using KDDTest+ for multi-class classification.

https://doi.org/10.1371/journal.pone.0284795.t013

6.3 Experiment 3: ILSTM performance for binary classification on LITNET-2020 dataset

Similarly to NSLKDD dataset, the ILSTM algorithm was evaluated on the LITNET-2020 dataset, described earlier. Also, the same nine performance metrics were used for evaluating the conventional LSTM and the proposed ILSTM algorithms. A summary of the results is given in Table 14. The confusion matrix for binary classification of LSTM before applying the optimization is given in Fig 22A). On the other hand, the results of applying optimization (i.e., ILSTM) is given in Fig 22B). Also, Fig 23 shows how the accuracy increased after using the ILSTM which starts the optimization process at iteration 58 to improve the accuracy of the original LSTM which was a constant at iteration 68. As shown in Fig 23B, ILSTM improved the accuracy from 92% to 94% at iteration 84 while the conventional LSTM gave an accuracy of 92.06% after 100 iterations as shown in Fig 23A. From Fig 23, it can be noticed that ILSTM has improved DR, Where ILSTM achieved 92.55% for DR but LSTM gave 87.46%.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 22. Confusion matrices for LITNET-2020 dataset in binary classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g022

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 23. LSTM vs ILSTM for LITNET-2020 in binary classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g023

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 14. Comparison between LSTM and ILSTM using LITNET-2020 in binary classification.

https://doi.org/10.1371/journal.pone.0284795.t014

6.4 Experiment 4: ILSTM performance for mulit-class classification on LITNET-2020 dataset

In this experiment, the proposed ILSTM is also compared with the original LSTM for a multi-classification scenario. In this experiment, the nine performance metrics were employed in this comparison and the results are summarised in Table 15. The bold text in this table indicates the best values of the results for the thirteen attack classes. These results showed that ILSTM can increase DR for the “fragmentation” attack, from 0% to 28.37% and also improved the DR for spam attack from 66.23% to 100%. Also, the confusion matrix of the multi-classification attack detection for the LSTM and ILSTM algorithms is given in Figs 24 and 25, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 24. Confusion matrix for LITNET-2020 dataset using LSTM algorithm.

https://doi.org/10.1371/journal.pone.0284795.g024

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 25. Confusion matrix for LITNET-2020 dataset using ILSTM algorithm.

https://doi.org/10.1371/journal.pone.0284795.g025

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 15. Comparison between LSTM, ILSTM using LITNET-2020 in multi-class classification.

https://doi.org/10.1371/journal.pone.0284795.t015

Using the LITNET-2020, we also investigated the relationship between the accuracy of ILSTM and the required number of iterations and compared it with the original LSTM. The results of this experiment were plotted in Fig 26. This figure shows that the proposed ILSTM algorithm achieved a higher accuracy rate (i.e., 95.77%) compared with the accuracy of the conventional LSTM, i.e., 91.04%. ILSTM results were achieved using 90 iterations after which the accuracy value became constant while LSTM achieved it accuracy results using 85 iterations after which the accuracy became constant.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 26. LSTM vs ILSTM for LITNET-2020 in multi-class classification.

A: LSTM. B: ILSTM.

https://doi.org/10.1371/journal.pone.0284795.g026

7 Conclusion and future work

In this paper, we developed an improved version of LSTM (called ILSTM) to improve the accuracy of LSTM-based intrusion detection system. The ILSTM made use of a combination of two swarm optimisation algorithms, CBOA and PSO, to determine the best weights for the LSTM network. The ILSTM consists of two phases: one for training the deeper LSTM network with the best parameters to get initial weights and another for optimizing these weights using CBOA and PSO. A comprehensive evaluation was conducted to assess the efficiecy of the proposed ILSTM algorithm for intrusion detection systems. Two public datasets (NSL-KDD and LITNET-2020) and nine evaluation metrics were used. The results showed that the proposed ILSTM algorithm is better than the orginal LSTM and two optimized versions of it (i.e., LSTM-PSO, and LSTM-CBOA) in two main cases: binary and multi-class classification. These results were also achieved using a few number of iterations and these were supported by confusion matrices for all the implemented algorithms. Additionally, by comparing the proposed ILSTM algorithm with published and related machine and deep learning methods, the ILSTM yields superior results in terms of accuracy, detection rate, precision, and f-measure when testing on KDDTest+ and KDDTest-21. It was noticed that our proposed algorithm accomplished excellent results when applying optimization, but it needs more time to optimize the population within large datasets. So in future work, it is planned to apply optimization with a faster algorithm. The limitations of the study are as follows: (1) problem of optimization size and computational effort where the proposed algorithm can take a long time when applying it on big data with millions of instances, as in the case of the LITNET-2020 dataset. (2) computational resources: if the problem size is too large, it might not be possible to store the processing data in the memory of the computer running this algorithm.