Data augmentation-based conditional Wasserstein generative adversarial network-gradient penalty for XSS attack detection system

GANs

GAN is recently introduced as a novel approach to train a generative model, which has achieved success in different fields, including images and natural language processing (Goodfellow et al., 2014). The network comprises two adversarial models: first, the generative model G for learning the distribution of data and, second, the discriminator D, which estimates the probability that a sample is from the real training data instead of G. Both models G and D in the network compete to outsmart the other where G and D can be a nonlinear mapping function, such as a multilayer perceptron. The generator G learns the distribution p_g over data x and constructs a mapping function from noise space of uniform dimension p_z(z) to data space as G(z, θ_g). The discriminator D(x, θ_d) returns a single scalar to estimate the probability that an instance x came from the real data distribution rather than p_g.

Both G and D are trained together, such that the parameters for G are adjusting to minimize log(1 − D(G(z))) and parameters for D are adjusting to minimize logD(X), similarly to the two-player minimax game accompanied by value function V(G, D) as in Eq. (1). (1)${\displaystyle \underset{G}{min}\underset{D}{max}V\left(G,D\right)={\mathrm{E}}_{x\sim P_r}\left[logD\left(x\right)\right]+{\mathrm{E}}_{\stackrel{\sim }{x}\sim P_g}\left[log\left(1-D\left(G\left(z\right)\right)\right)\right]}$

A CGAN extends GAN by adding space y to both the G and D to control data generation. The additional space y could be supplied from real data (class label) or data from other sources (Mirza & Osindero, 2014). The training phase of both CGAN and GAN is similar, and the minimax objective function of D and G is as shown in Eq. (2). (2)${\displaystyle \underset{G}{min}\underset{D}{max}V\left(G,D\right)={\mathrm{E}}_{x\sim P_r}\left[logD\left(x|y\right)\right]+{\mathrm{E}}_{\stackrel{\sim }{x}\sim P_g}\left[log\left(1-D\left(G\left(z|y\right),y\right)\right)\right]}$

where p_r is the real data distribution and p_g is the CGAN model distribution implicitly defined as $\tilde{x}=\left(z,y\right),z\sim p\left(z\right),y\sim p\left(y\right)$, where y and noise z are combined as input to the hidden layer.

The CGAN and GAN use Jensen–Shannon (JS) divergence shown in Eq. (3) to measure generative samples. (3)${\displaystyle JS\left(p_r,p_g\right)=KL\left(p_r\left|\right|p_m\right)+KL\left(p_g\left|\right|p_m\right),\left\{\begin{array}{c}klisullback-Leiblerdivergence\hfill \\ p_m=\left(p_r+p_g\right)∕2\hfill \end{array}\right.}$

However, both GAN and CGAN have unstable training (vanishing gradients) and mode collapse problems (Pan et al., 2019). To overcome these problems, the WGAN optimizes the original GAN objective using Wasserstein-1 distance, also known as the earth-mover distance (EMD) instead of JS (Arjovsky, Chintala & Bottou, 2017), where EMD measures the distance between the actual distribution of the data and the distribution of the generative model as in Eq. (4). (4)${\displaystyle W\left(p_r,p_g\right)=\begin{array}{c}inf\hfill \\ \gamma \in \prod \left(p_r,p_g\right)\hfill \end{array}{E}_{\left(x,y\right)\sim \gamma }\left[\left|\left|x-y\right|\right|\right]}$

where ∏(p_r, p_g) represents all possible joint distribution sets (x, y) of p_r and p_g of real and generated data distribution, respectively. Such that, for each feasible joint distribution γ, a real instance x and a generated instance y can be sampled, and the instance distance [||x − y||] is calculated. Therefore, the expected value γ $E_{\left(x,y\right)\sim \gamma }\left[\left|\right|x-y\left|\right|\right]$ of the instance to the distance under the joint distribution γ can be calculated.

The value function of WGAN was obtained by utilizing Kantorovich–Rubinstein duality (Villani, 2009), as shown in Eq. (5). (5)${\displaystyle \underset{G}{min}\underset{D\in \mathcal{F}}{max}V\left(G,D\right)=\underset{x\sim p_r}{\mathrm{E}}\left[D\left(x\right)\right]-\underset{\stackrel{\sim }{x}\sim p_g}{\mathrm{E}}\left[D\left(\stackrel{\sim }{x}\right)\right]}$

where $\mathcal{F}$ is the set of 1-Lipschitz functions restricted by k, $\left|\mathcal{F}\left(\mathrm{x}\right)-\mathcal{F}\left(\mathrm{y}\right)\le \mathrm{k}\right|\mathrm{x}-\mathrm{y}|$, and p_gi is the model distribution. The value function is minimized concerning G determined by $\tilde{x}=G\left(z\right),z\sim p\left(z\right)$. Therefore, the discriminator called a critic minimizes the W(p_r, p_g). Nevertheless, the WGAN still faces gradient extinction or gradient explosion because of weight clipping in the discriminator. The gradient penalty (GP) was added to the total loss function in the WGAN distance discriminator to achieve training stability (Gulrajani et al., 2017). The new objective value function is adjusted, as shown in Eq. (6). (6)${\displaystyle \underset{G}{min}\underset{D}{max}V\left(G,D\right)=\underset{x\sim p_r}{\mathrm{E}}\left[D\left(x\right)\right]-\underset{\stackrel{\sim }{x}\sim p_g}{\mathrm{E}}\left[D\left(\stackrel{\sim }{x}\right)\right]-\lambda E_{\hat{x}\sim p_{\hat{x}}}\left[{\left({\nabla }_{\hat{x}}D\left(\hat{x}\right)|{|}_2-1\right)}^2\right]}$

where $\hat{x}=\epsilon x+\left(1-\epsilon \right),\tilde{x}$ is a convex function combination of real data distribution p_r(x) and the model data distribution p_g(z), $\epsilon \sim unform\left[0,1\right]$, whereas λ is the gradient penalty parameter.

C-WGAN-GP model

This study proposes using data augmentation based on a GAN that takes real samples as inputs and outputs adversarial samples. The learning algorithm of our proposed model is based on CGAN (Mirza & Osindero, 2014) and WGAN-GP (Gulrajani et al., 2017), where both networks are integrated. Precisely, we used the WGAN-GP optimization approach to optimize CGAN. The integrated generative network is called C-WGAN-GP. Our goal is to generate synthetic samples of attack class (minority) with identical distribution to real XSS attack scenarios.

The primary idea is to use the learning of the joint probability distribution over x samples and y labels from the training data to perform data augmentation only if the generated sample $\tilde{x}$ satisfy the critic. The problem of unbalanced data can be mitigated by using an augmented data in the classification tasks, therefore improving the robustness and performance of the XSS attack detection for unbalanced data. A well-trained generator with joint distributions set (x, y) of p_r and p_g of real and generated data distribution optimized using GP should be able to generate $\left(\tilde{x},y\right)$ samples within the tolerated latent space and identical to the original data of (x, y), therefore providing valuable information to the detector as additional training data. To ensure that only useful instances are added to augment the training dataset, only generated cases that satisfy the critic are added to the original data.

The y labels of the minority class, which are XSS attacks in our case, are used as a conditional parameter. Passing the upper and lower real data space to the generator provides the generator with additional auxiliary information to define the latent space. The latent space establishes the scope of samples in the data variance. Therefore, the generator using the auxiliary latent space generates samples within the tolerated latent space identical to the real data. Consequently, the discriminator distinguishes the synthetic samples as real within small feedback loops needed to train the generator, reducing computational cost while providing high-quality generated data.

In the discriminator D, the p_r and p_g are linked with y in a joint hidden layer representation, whereas in generator G, the y is combined with p(z) in the same manner. The objective minimax function of models D and G is as shown in Eq. (7), whereas Eqs. (8) and (9) represent the loss reduction functions of D and G, respectively. (7)${\displaystyle \underset{G}{min}\underset{D\in \mathcal{F}}{max}V\left(G,D\right)=\underset{x\sim p_r}{\mathrm{E}}\left[D\left(x|y\right)\right]-\underset{\stackrel{\sim }{x}\sim p_g}{\mathrm{E}}\left[D\left(\stackrel{\sim }{x}|y\right)\right]-\lambda E_{\hat{x}\sim p_{\hat{x}}}\left[{\left({\nabla }_{\hat{x}}D\left(\hat{x}|y\right)|{|}_2-1\right)}^2\right]}$ (8)${\displaystyle \mathrm{min }L\left(D\right)=\underset{\stackrel{\sim }{x}\sim p_g}{\mathrm{E}}\left[D\left(\stackrel{\sim }{x}|y\right)\right]-\underset{x\sim p_r}{\mathrm{E}}\left[D\left(x|y\right)\right]+\lambda .E_{\hat{x}\sim p_{\hat{x}}}\left[{\left({\nabla }_{\hat{x}}D\left(\hat{x}|y\right)|{|}_2-1\right)}^2\right]}$ (9)${\displaystyle \mathrm{min }L\left(G\right)=-\underset{\stackrel{\sim }{x}\sim p_g}{\mathrm{E}}\left[D\left(\stackrel{\sim }{x}|y\right)\right]}$

Generative model design

Generative networks have gained popularity in image data; however, we are interested in digital datasets. Therefore, the technique used is similar but differs in design and implementation. In our model, we did not apply convolutional layers.

In the generator model G, the concatenate input layer equal to (z + c), where z is the vectors of noise set within the range of batch size and data dimension. The c is the conditioning variable’s dimension that equals 1. The model has three hidden layers of the neural network—the number of units in hidden layers equal to 128, 256, and 512, respectively. The output layer equal to z and the concatenate layer is equal to the input layer (z + c).

In the discriminator (critic) D, the same architecture is used but in descending order of hidden layers, which equal to 512, 256, 128, respectively. The third layer is a linear activation function that equals to 1. The batch size and numbers of epochs for the network are 128 and 4000, respectively. The activation function used for the generator and discriminator is the rectified linear unit. The C-WGAN-GP is fitted using Adam optimizer with α, β1, and β2 parameters calibrated to le-4, 0.5, and 0.99, respectively. The alpha or (α) for short refers to the learning rate, while beta1 (β1) and beta2 (β2) refer to the exponential decay rate for the first-moment and second-moment estimates, respectively. The value of the GP coefficient λ for C-WGAN-GP is set to 10. The parameter k of D is tuned to 4, whereas k of G is tuned to 1. A conditional critic neural network is trained to approximate the EMD using the minority class for control mode up to 4000 training steps. Note that the parameters are defined empirically, whereas the performance reduces significantly by changing the parameters.

The other different hyperparameters not mentioned are consistent with those originally reported. During the testing phase, the generated samples added to the real dataset are the samples approved by the critic. Algorithm 1 presents the generative approach for XSS attack data. Note that the other generator network architectures are similar, with negligible differences, which may be necessary for the implementation.

Experimental methodology

This study proposed generative model C-WGAN-GP, an oversampling solution of minority class to solve unbalanced XSS attack data. We trained the detector (XGBoost) using the real training dataset and test it using the test dataset before without augmented data, and then the results were recorded for comparison. Subsequently, we trained each of the GANs models using real training dataset to generate synthetic data. We repeated the training of the detector using the augmented data and test it using the test dataset. The results of each model were recorded for comparison. Similarly, traditional oversampling methods were trained and tested.

The C-WGAN-GP generator performance was evaluated in two directions. First, we assessed the performance of the C-WGAN-GP generative adversarial network against the other four GANs. Second, we compared our C-WGAN-GP model with two traditional oversampling methods include SMOTE (Han, Wang & Mao, 2002) and adaptive synthetic (ADASYN) (Bunkhumpornpat, Sinapiromsaran & Lursinsap, 2012). The systemic flowchart of the proposal is shown in Fig. 1.

Detector

We use an external model to evaluate the quality of the data generated by our proposed method and other methods. The XGBoost boosting model was used for all experiments to assess the augmented data’s quality on XSS attack detection performance. The XGBoost is a state-of-the-art boosting algorithm that is simple to apply and interpret, is highly efficient, does not require advanced data preparation, and has many advanced functions (Chen & Guestrin, 2016). The algorithm’s learning rate, tree size, and tree number hyperparameters tuned to 0.3, 4, and 100, respectively.

Datasets

To our knowledge, there is only one public dataset for intrusion detection that includes XSS attacks that we were able to find called CICIDS2017 designed by the Canadian Institute of Cybersecurity (Sharafaldin, HabibiLashkari & Ghorbani, 2018). The released CICIDS2017 dataset contains 80 features that include regular traffic and recent common attacks. To provide attack features, we used a CICFlowMeter tool to extract features from PCAPs files and used Selenium with Damn Vulnerable Web Application to run automatic XSS attacks. However, there are only 652 XSS attacks traffic, and over 140,008 regular traffic, making it a highly unbalanced dataset.

We have added another dataset proposed in our previous work (Mokbal et al., 2019). The dataset includes 67 features with labels that are categorized based on three groups, including HTML, JavaScript, and URL. We extracted 1,000 XSS attack samples and 100,000 benign samples as a second dataset.

We applied data preprocessing to each dataset. In both datasets, all samples have two classes, XSS attack and benign, which are set to [1, 0], respectively. The two datasets were split randomly into training and test sets with a 70% and 30% ratio, respectively, where data augmentation was performed only on the training dataset. Missing and infinite values in CICIDS2017 were updated using their features’ mean values, whereas the zero features and duplicate rows were omitted. The number of features with clean data in the CICIDS2017 dataset is 78, and the number of features with clean data in the second dataset is 67. Subsequently, the data’s scale within the range [0, 1] was applied using the minimax function for both datasets. The class-level distribution of datasets is shown in Table 1. The datasets are available at https://doi.org/10.6084/m9.figshare.13046138.v4.

Performance evaluation criteria

Although many performance metrics have been introduced, GANs do not have an objective measure of the generator model, as there is no consensus on the best metric that captures the strength/limitations of the model and should be used for a fair comparison between models (Borji, 2019). We used precision, $detectio{n}_{Rate}\left(DR\right)$/ recall, and F1 − score, which are proven and widely adopted methods for quantitatively estimating the quality of discriminative models suggested by Google Brain research (Lucic et al., 2018). Precision measures the generated instances similarity to the real ones on average. Whenever the instances generated are similar to the real instances, the precision is high. In GANs, the recall (detection rate) measures diversity. A high recall indicates the generator can generate any instances found in the training dataset. For cybersecurity detection sys, the recall/detection rate denotes the ability of sys to detect the real attacks. The F-score reflects the harmonic mean of precision and recall. Further, the area under the curve (AUC) measure that demonstrates a detector’s ability to distinguish between classes and summarizes the detector’s performance is also collected. The measurements are defined as follows. (10)${\displaystyle Precision=\frac{TP}{\left(TP+FP\right)}}$ (11)${\displaystyle DR=\frac{TP}{\left(TP+FN\right)}}$ (12)${\displaystyle F-Score=2\left(\frac{\left(Recall\times Precision\right)}{\left(Recall+Precision\right)}\right)}$ (13)${\displaystyle AUC=\frac{1}{2}\left(\frac{TP}{TP+FN}+\frac{TN}{TN+FP}\right)}$