An Efficient Network Security Situation Assessment Method Based on AE and PMU

Network security situation assessment (NSSA) is an important and effective active defense technology in the field of network security situation awareness. By analyzing the historical network security situation awareness data, NSSA can evaluate the network security threat and analyze the network attack stage, thus fully grasping the overall network security situation. With the rapid development of 5G, cloud computing, and Internet of things, the network environment is increasingly complex, resulting in diversity and randomness of network threats, which directly determine the accuracy and the universality of NSSA methods. Meanwhile, the indicator data is characterized by large scale and heterogeneity, which seriously affect the efficiency of the NSSA methods. In this paper, we design a new NSSA method based on the autoencoder (AE) and parsimonious memory unit (PMU). In our novel method, we first utilize an AE-based data dimensionality reduction method to process the original indicator data, thus effectively removing the redundant part of the indicator data. Subsequently, we adopt a PMU deep neural network to achieve accurate and efficient NSSA. The experimental results demonstrate that the accuracy and efficiency of our novel method are both greatly improved.


Introduction
Network security situation assessment (NSSA) technology is one of the most effective active defense technologies to evaluate the threats of network security, by which the network administrators not only can comprehensively understand the security risk situation but also can understand the security threats which are faced by the current network and information system. Hence, the network administrators can manage the dynamic network security situation and judge the development trend of the network security situation [1,2]. As a result, NSSA attracts increasing attention.
Nowadays, plenty of NSSA methods have been proposed, while there still exist many inherent deficiencies in the existing methods, resulting in a lot of severe challenges that need to be solved solidly. Firstly, most of the existing NSSA methods pay too much attention to subjective judgment and prior knowledge. Meanwhile, they ignore plenty of other external factors and the time sequence property of the indicator data. As a result, they are not suitable for the long-term assessment over the network security situation. Secondly, the existing NSSA methods are not efficiency attractive. Specifically, the current indicator data is characterized by large scale, multifeature, heterogeneity, high dimensionality, and nonlinearity. Hence, the existing NSSA methods need to pay very expensive computational overhead to process these indicator data before evaluating the network security situation. Last but not least, some existing NSSA methods only consider part of the network security threats and attacks. However, the current network threats and attacks are characterized by diversity and randomness, resulting in very low accuracy in some existing methods. Therefore, how to design a novel method to effectively and accurately achieve NSSA has become one of the most important problems in the field of network security.

Contributions.
We study an essential but challenging problem in this paper, i.e., accurate and efficient NSSA in large-scale network environment. Then, we design a novel NSSA method based on the autoencoder (AE) and parsimonious memory unit (PMU), which can efficiently and accurately achieve NSSA. Therefore, the main contributions of this paper can be described as the following two aspects.
(1) In the current large-scale network environment, the indicator data is characterized by heterogeneity, large scale, multifeature, high dimensionality, and nonlinearity. Therefore, we design an AE-based data dimensionality reduction method to process the original indicator data, thus reducing the dimensions of the indicator data. Subsequently, we can efficiently extract the situation assessment elements on the premise of guaranteeing the integrity of the data features (2) We adopt PMU to design a novel NSSA method for the current large-scale network environment, in which PMU is utilized for feature representation and time-varying learning of situation assessment elements. Meanwhile, we offer the theoretical computational complexity comparison. Finally, we implement our novel method and provide the performance evaluation, which can intuitively demonstrate the high efficiency and accuracy of our novel method 1.2. Related Work. NSSA has been extensively studied in both academia and industry, resulting in a rich body of solutions. Generally speaking, the existing NSSA methods can be summarized into three categories: mathematical statisticsbased assessment method, knowledge reasoning-based assessment method [3], and machine learning-based assessment method [4,5]. Wang et al. [6] designed a hierarchical NSSA method based on an analytic hierarchy process (AHP), which used the hierarchical cyber threat situation assessment (CSA) indicator system constructed by AHP to decide the network threat weight values. Wang et al. [7] proposed an AHPbased NSSA and quantification method, which utilized the AHP and hierarchical situation assessment model to simplify the NSSA problem. Li et al. [8] adopted fuzzy optimal clustering criteria combined with c-means clustering to process the indicator data, thus getting the number of clusters and the optimal clustering center. Then, they got the final NSSA results by utilizing AHP to construct an assessment model. Zhang et al. [9] presented a distributed denial of service (DDoS) attack NSSA model based on fuzzy clustering algorithm fusion features, which can effectively evaluate the security status of DDoS attack. Although the above methods can effectively implement NSSA, they not only need a lot of subjective judgments but also are not conducive to long-term assessment.
Yi et al. [10] designed a NSSA method based on fuzzy theory. Their method used fuzzy theory to weaken the index factors with low credibility and eliminate the uncertainty, thus making the assessment results more accurate. Liu et al. [11] utilized D-S evidence theory to fuse the measured indexes for obtaining the device threat value. Then, they utilized AHP to calculate the weights for different devices and finally obtained the network threat situation value by the weighting method. Codetta-Raiteri et al. [12] designed a NSSA method based on decision networks (DN), which can achieve a reasonable tradeoff between computational complexity and analysis efficiency. To quantitatively assess the network security risk, Wang et al. [13] designed a NSSA model based on the Bayesian approach. Fan et al. [14] presented a security evaluation method based on a softwaredefined network (SDN), which used multiple observation hidden Markov model (HMM) to obtain the security evaluation value of SDN, by quantifying the network state. To more completely describe the network security situation, Liao et al. [15] designed a NSSA method based on the extended HMM. Although the above knowledge reasoningbased methods can improve the NSSA accuracy, they rely on too much prior knowledge and have no advantages in efficiency.
Nowadays, the support vector machine (SVM) [16] and neural network [17,18] are also widely used in NSSA. Chen et al. [19] adopted the SVM and gravitational search algorithm (GSA) to design a NSSA method, which has a better global optimization function. Qiang et al. [20] utilized an optimized cuckoo search back propagation neural network (BPNN) to design a new NSSA method. In their method, they used a cuckoo search (CS) algorithm based on conjugate gradient to optimize the initial parameters of BPNN and increase the training efficiency of the neural network. Shi and Chen [21] utilized a dual-SVM model for data learning and parameter estimation in command information system security situation samples, thus evaluating the command information system security situation. Gao et al. [22] designed the SVM information system security risk assessment model, which was optimized by an artificial fish swarm algorithm (AFSA). In their method, the AFSA was used to optimize the SVM, resulting in great accuracy and fast convergence. Han et al. [23] adopted convolutional neural networks (CNN) to design a quantitative network security situation evaluation method for an intelligent robot cluster under the wireless connection. Yang et al. [24] adopted a deep autoencoder (DAE) and deep neural networks (DNN) to study NSSA. Subsequently, they designed a new method to improve the network attack identification accuracy and the NSSA flexibility. Although the above methods can improve the accuracy, they cannot learn the correlation of time series. Therefore, they cannot be suitable for the NSSA over the indicator data which is characterized by the time sequence.
1.3. Organization. The rest of the structure of this paper is as follows. In Section 2, we describe the AE and PMU. In Section 3, we adopt AE and PMU to propose a novel NSSA method. Then, we implement the proposed method and 2 Wireless Communications and Mobile Computing provide the experiment results in Section 4. Finally, we simply summarize this paper in Section 5.

2.1.
Autoencoder. An autoencoder (AE) [25,26] is a common unsupervised learning algorithm, which utilizes the original input data as a reference for self-supervised learning to reduce dimension. AE generates information elements with more obvious features and lower dimensions than the original data element. AE maps the original data to the encoding layer to achieve encoding, then maps the encoded data to the decoding layer for decoding, and takes the final decoded data as the output data (see Figure 1).
where l represents the data dimensions. Note that the AE requires that the final input result is almost equal to the output result, that is, X = X ′ . The encoding process of the AE can be expressed as follows: where the bias of the encoding part can be represented as b e and the activation function of the encoding part can be represented as f e .
The decoding process of the AE decoding layer can be expressed as follows: where the bias of the decoding part can be represented as b d and the activation function of the decoding part can be represented as f d .
The MSE loss function is usually used in AE training, and it can be expressed as where X represents the input variable, X′ represents the output variable, LðX, X ′ Þ represents the loss function, and k represents the number of samples.

Parsimonious Memory Unit.
A parsimonious memory unit (PMU) is a new recurrent neural network, which can be viewed as an improved version of a gated recurrent unit (GRU) [27]. PMU is characterized by better managing the latent relations between short-and long-term dependencies [28]. Note that there are two gate structures in the GRU model, i.e., reset gate and update gate. However, there is only one gate structure in PMU, i.e., unit gate, as seen in Figure 2. Specifically, PMU integrates the update gate and the reset gate of GRU into a new unit gate, resulting in fewer parameters in PMU. Moreover, due to the fact that the PMU can better manage the latent relations between short-and longterm dependencies, PMU has better convergence and speed in training.
In Figure 2, we utilize U t to represent the unit gate, which is used to control the learning of long-term correlation and short-term correlation of the data. When U t is 1, PMU learns the long-term dependence of the data, while when U t is 0, PMU learns the short-term dependence of the data. The learning mode of PMU can be described as follows.
Firstly, the state of the unit gate is obtained by the evaluation state h t−1 transmitted from the previous node and the input x t of the current node: Secondly, the state of the current time memorized on the current candidate set e h t can be expressed as Thirdly, in the stage of updating memory, PMU updates h t through the following formula: Finally, the output of forward propagation is In the forward propagation process of PMU, we need to learn the following three parameters: W u , Wh, and W o , where Then, the output y t is the network domain security situation score value.

Method
In this section, we initially describe the system structure. Then, we introduce our AE-PMU-based NSSA method in detail. Internet of things [32,33], the modern network is characterized by new features, such as dynamic virtualized management methods and multilevel service models. As a result, the network threats have the characteristics of diversity and randomness. Meanwhile, the indicator data is largescale and has heterogeneity, resulting in plenty of new problems in NSSA. Specifically, the large volume of indicator data seriously affects the efficiency of the assessment method. Moreover, the diversity and randomness of network threats directly determine the accuracy and the universality of the assessment method. To handle the above challenges, we propose a novel NSSA method (as seen in Figure 3), which is mainly composed of AE-based data dimensionality reduction and PMU-based assessment method. Specifically, we first adopt AE to process the original indicator data to achieve data dimension reduction. Then, we extract the situation assessment elements efficiently. By taking AE, our method can greatly improve the efficiency and reduce the data loss. Then, we adopt PMU to design an efficient NSSA method. Compared with other deep neural networks (e.g., GRU), PMU is more suitable for managing the latent rela-tions between short-and long-term dependencies. Meanwhile, our PMU-based assessment method is more efficient than the GRU-based assessment method.
3.2. AE-PMU-Based NSSA Method. In this part, we provide the detailed description of our proposed AE-PMU-based NSSA method. The algorithm pseudocode is shown in Algorithm 1.
As described in Algorithm 1, M represents the dimension of data dimensionality reduction, n represents the training period, and C represents the situation value after the evaluation of the test set. The main processes are as follows.
(1) Initialize the data dimension reduction dimension M and the number of training period n (2) Use AE to extract the situation assessment elements from the initialized overall indicator dataset to achieve data dimensionality reduction

Time Complexity Analysis and Experiment
We initially analyze the time complexity in this section. Then, we implement our AE-PMU-based NSSA method and provide the experimental results, including the precision, the efficiency, and the fit between the assessed indicator value and the real indicator value.

Time Complexity Analysis.
Time complexity is a significant index to judge the merits of the algorithm. We will    Wireless Communications and Mobile Computing analyze and compare the forward propagation time complexity of the GRU-based NSSA method and the PMUbased NSSA method. We can assume that the dimension of the data input is m and the number of PMU hidden units is n. Firstly, according to formula (4), the number of operations for U t can be represented as Tðn × m + n 2 + nÞ. Secondly, according to formula (5), the number of operations for calculating the current state candidate set is Tðn × m + 2 × n 2 + nÞ. Thirdly, according to formula (6), the number of operations in the memory update phase is Tðn 2 + 2 × nÞ. Finally, the total number of operations of PMU is Tð2 × n × m + 4 × n 2 + 4 × nÞ. Overall, the time complexity is Oðn 2 Þ.
Compared with PMU, GRU has one more gate structure, which has the same number of operations as the U t gate of PMU. In addition, the memory update phase of the GRU is different from that of PMU. GRU uses Z t to control whether the candidate set state is added to the memory update phase in this state. Therefore, the number of operations in the memory update phase of the GRU is Tð2 × n 2 + nÞ. The operation time of other parts of GRU is the same as that of PMU. Therefore, the total number of GRU operations is Tð3 × n × m + 6 × n 2 + 4 × nÞ. In summary, the time complexity is Oðn 2 Þ.
As shown in Table 1, in general, although the time complexity of PMU is the same as that of the GRU, the total number of operations in PMU is much less than that in GRU. Therefore, the PMU-based NSSA is much more efficient.

Experimental Environment and Dataset.
In our simulation experiment, the public dataset UNSW-NB15 is utilized as the experimental dataset [34,35]. In UNSW-NB15, there are 9 different modern attacks. Meanwhile, every data record contains 43 elements and a corresponding label. UNSW-NB15 is divided into 4 different Comma-Separated Values (CSV) files, which contain a total of 2540044 data records. Moreover, there are 300000 abnormal traffic data records, as shown in Table 2.
As shown in Table 2, the dataset covers 9 different attack categories; the detailed categories are as follows: (1) Analysis: an intrusion method that penetrates web applications through email, web scripts, ports, etc.
(2) Backdoors: an intrusion method that bypasses the system security mechanism through technical secrets to evaluate the computer or its data

5
Wireless Communications and Mobile Computing using brute force to exhaust the resources of the attacked object, so as to achieve an attack that makes the target network unable to use services or resources (4) Exploits: a type of attack that exploits the attacker's knowledge of security vulnerabilities in the operating system or software (5) Fuzzers: an attack type in which an attacker provides a large number of random numbers to the program or the network to make it down (6) Generic: use hash functions for conflicts regardless of password configuration (7) Reconnaissance: attacks used to collect computer information, also called probes (8) Shellcode: the attacker uses shell commands and a small amount of code to control the attack mode of the attacked host (9) Worms: worm attack, a virus attack that can replicate itself to the control host without any operation For the convenience of experiment, we make statistics every ten minutes according to the time stamp in all the extracted dataset. A total of 144 sample data composed of the situation value is generated; among them, 100 are intercepted as the training set and 44 as the testing set. In addition, the network security situation value is 0-10. In the process of establishing the situation risk level, we will denote the situation value of 0-2 as safe, the situation value of 3-4 as low risk, and the situation value of 5-6 as medium risk. The situation value of 7-8 indicates a high risk, and the situation value of 9-10 indicates an emergency.

Experimental Criteria.
In this experiment, Accuracy, Precision, Recall, and F1 score are used to evaluate the effectiveness of our NSSA method and some of these concepts are defined as follows.
True We use a confusion matrix to represent TP, FP, TN, and FN, as shown in Table 3.
Then, the Accuracy, Precision, Recall, and F1 score are defined as follows: Accuracy represents the proportion of the number of correctly identified samples in the total sample. Precision represents the proportion of actual positive samples among the number of positive samples identified. Recall represents the percentage of positive examples in the sample that are predicted to be correct. However, it is unreasonable to evaluate the performance of the model only from Precision or Recall. To make the evaluation be more convincing, except for Precision and Recall, it is generally necessary to use F1 score as the model evaluation standard.

Evaluation Experimental Result.
In this part, we evaluate the proposed AE-PMU-based assessment method, the PMU-based assessment method, the GRU-based assessment method, and BPNN-based assessment method from the points of effectiveness, fitting degree, and efficiency.

Effectiveness Evaluation.
We compare the effectiveness of our AE-PMU-based assessment method, PMU-based assessment method, GRU-based assessment method, and BPNN-based assessment method, as shown in Figure 4. Figure 4 measures the effectiveness of four different assessment methods from the Accuracy rate, Precision rate, Recall rate, and F1 score. Among them, the AE-PMUbased assessment method has the best performance. This is   Although the GRUbased assessment method can consider the timing of the indicator data, compared with the PMU-based assessment method, GRU cannot effectively manage gates based on the latent relation between short-and long-term dependencies, so its effectiveness is inferior to that of the PMU-based assessment method. Because the data after AE dimensionality reduction removes the redundant part, the effectiveness of the AE-PMU-based assessment method is better than that of the PMU-based assessment method. This shows that the AE-based dimensionality reduction data fully retains the effectiveness of the indicator data, and the effectiveness of the PMU-based assessment method is better than that of the GRU-based assessment method.

Goodness of Fit.
We utilize a polyline graph to intuitively show the comparison of the fit between the assessment value and the real value, as shown in Figure 5. From Figure 5, we can see that when the sample numbers are 3, 13, 31, and 33, the network situation value fluctuates significantly, indicating that the network threats are relatively strong at these moments. In the third sample, a warning of "medium-risk" level appeared, indicating that the network is being threatened by a higher level attack, and security defense countermeasures should be taken. The "high-risk" level warnings appeared in the samples no. 13 and no. 31, indicating that the network suffers from extremely great security threats, and timely protection or rescue is required. According to the two fitting curves of the real value and the assessment value, it can be seen that the situation assessment result obtained by the proposed method basically fits the real security situation. Except for sample no. 31, which misjudged "high risk" as "safety," all other samples were correctly judged, which can more accurately fit the real security situation of the current network.
Among other methods, the PMU-based assessment method makes a mistake once, the GRU-based assessment method makes a mistake twice, and the BPNN-based assessment method makes a mistake three times.
Our analysis of the reasons is consistent with the above "effectiveness evaluation" reasons. According to the above experimental results, it can be shown that the AE-PMUbased NSSA method can adapt to the NSSA under the modern network environment and can more accurately fit the real network security situation changes.

Performance Evaluation.
Among the above four methods, although the network structure of BPNN is simple, it is rarely used in practical applications because it cannot fully characterize the data characteristics by using the time  7 Wireless Communications and Mobile Computing sequence of the data. Therefore, here we only compare the performance (assessment time) of NSSA methods based on PMU, GRU, and AE+PMU, as seen in Figure 6.
We can see from Figure 6 that the running time of the AE-PMU-based assessment method is the smallest, the running time of the PMU-based assessment method is the second, and the running time of the GRU-based assessment method is the longest. This is because the GRU has two gate structures. The reset gate helps the GRU decide which past information needs to be forgotten, and the update gate helps the GRU decide which past information needs to be passed to the future. However, PMU uses a gate to complete the calculation tasks of the GRU update gate and reset gate, thus reducing the amount of calculation. Meanwhile, AE reduces the dimension of the original indicator data. Hence, the computational efficiency of the AE-PMU-based assessment method is the best.

Conclusions
In large-scale network environment, the diversity of network threats and the high dimensionality of indicator data make the NSSA become more difficult. In this paper, we studied the NSSA in large-scale network environment and then proposed a novel NSSA method based on AE and PMU. Specifically, we first used AE for data dimensionality reduction to remove the redundant data. Then, we utilized PMU to achieve NSSA. By taking the advantage of PMU, the proposed method can effectively improve the performance of the model. Finally, we implemented the proposed method and provided the performance evaluation. The experimental results can show that compared with the existing methods, our method had significant advantages in efficiency, accuracy, and fit degree.