Signal Modulation Recognition Algorithm Based on Improved Spatiotemporal Multi-Channel Network

Abstract

Automatic modulation recognition (AMR) plays an essential role in modern communication systems. In recent years, various modulation recognition algorithms based on deep learning have been emerging, but the problem of low recognition accuracy has not been solved well. To solve this problem, based on the existing MCLDNN algorithm, in this paper, we proposed an improved spatiotemporal multi-channel network (IQ-related features Multi-channel Convolutional Bi-LSTM with Gaussian noise, IQGMCL). Firstly, dividing the input IQ signals into three channels, time sequence feature extraction is carried out for route I, route Q, and route IQ, respectively. For route IQ, convolution kernel (2,1) is first used to extract relevant features. Two layers of the small convolution kernel (1,3) are used to extract time sequence features further, and the three channels are used to extract features further. Then, a two-layer short-length memory network is used to extract features from time and space more effectively. Through comparison experiments, Bi-LSTM is introduced to replace one layer of LSTM, and a fully connected layer is removed to prevent overfitting. Finally, multiplicative Gaussian noise is introduced to naturally corrode the feature parameters, further improving the robustness and accuracy of the model. Experiments are carried out on three public datasets RML2016.10a, RML2016.10b, and RML2016.04C. The experiments show that the IQGMCL network has higher recognition accuracies on all datasets, especially on the RML2016.10a dataset. When the SNR is 4 dB, the recognition accuracy reaches 93.52%. When the SNR is greater than 0 dB, the average recognition accuracy reaches 92.3%, 1.31%, and 1.2% higher than the original MCLDNN network, respectively.

1. Introduction

In recent years, the role of communication technology in war application has become more and more apparent. Satellite communication technologies are particularly important [1,2]. Among them, modulation mode recognition has an essential application in information warfare. Signal modulation is an important technology in modern communication systems [3]. The so-called signal modulation is a process of loading the low-frequency signal carrying the information to be transmitted onto the high-frequency signal. It is convenient for information propagation in an electromagnetic environment. However, in modern wireless communication, IQ signal modulation is standard. IQ signal is also called quadrature signal in the same direction [4]. I by in-phase(in-phase), Q by quadrature(orthogonal), and the phase difference between Q and I is 90 degrees. IQ signal is the mapping of continuous signal in a two-dimensional rectangular coordinate system, usually used for baseband signal conversion and reconstruction, and automatic modulation classification recognition plays an important role in modern wireless communication.

With the popularity of artificial intelligence technology, deep learning is applied more and more in modulation recognition algorithms [5]. Especially convolutional neural network has become a hot research topic. Traditional pattern recognition methods [6,7] require a lot of manual calculations when extracting the features of samples, and the recognition accuracy of multi-class modulation signals is not ideal. However, the neural network can automatically learn sample features from massive samples rather than manually designing expert features [8,9].

In 2016, O’Shea introduced a convolutional neural network (CNN) into the field of modulation signal recognition for the first time. When the SNR was greater than 0 dB, the average recognition accuracy reached 78% on the RML2016.10a dataset. In the same year, the CNN1 network [10] was proposed. On the RML2016.10a dataset, the recognition accuracy can reach 81%.

In 2017, Nathan E. West and O’Shea applied the CLDNN network model, which plays an outstanding role in the field of speech recognition, to signal modulation identification [11], and the recognition accuracy reached 83% on the publicly available RML2016.10a dataset. In the same year, Xiaoyu Liu proposed the ResNet network, DenseNet network, and CLDNN2 network [12]. These three networks achieved the highest recognition accuracy of 83.5%, 86.6%, and 88.5%, respectively, in the dataset containing ten types of modulated signals generated by GNURadio.

In 2020, Tekbiyik proposed the CNN2 network [13]. This network increased the number of convolution layers and the number of hidden nodes on the basis of CNN1, and the identification accuracy reached 84% on the RML2016.10a dataset. At the same time, a new dataset was introduced for testing. The experiment showed that this network had better recognition performance on the HisarMod2019.1 dataset than RML2016.10a. The same year, Ade Pitra Hermawan proposed the IC-AMCNet network [14]. Compared with the existing CNN architectures, it adjusts the number of layers and adds new layer types to meet the estimated delay standard beyond the fifth generation (B5G) communication. According to the simulation results, the proposed scheme has a better advantage in terms of calculation time, and the recognition accuracy reaches 84% on the RML2016.10a dataset and 92.6% on the RML2016.10b dataset.

In 2021, J. Njoku proposed the CGDNet network [15] and introduced GuassianDropout, which enhanced the feature extraction process and prevented the problem of gradient disappearance. In the same year, Fuxin Zhang proposed a PET-CGDNN network based on phase parameter estimation and transformation [16]. Compared with the CLDNN network and CLDNN2 network, this network uses a convolutional neural network (CNN) and gated cycle unit (GRU) as the feature extraction layer, which maintains a recognition accuracy rate higher than 90%. The number of parameters is less than 1/8. In 2022, Beiming Zhang verified the effectiveness of BiLSTM in literature [17]. Especially under the low signal-to-noise ratios, it had a better recognition effect than LSTM in open RML datasets.

In 2020, Jialang Xu proposed a spatiotemporal multi-channel learning framework (MCLDNN) for automatic modulation recognition [18]. This network could extract features more effectively from the perspective of time and space. Experiments on reference datasets showed that the proposed framework had high convergence speed and higher recognition accuracy.

Based on the existing modulation signal recognition network models, this paper proposes an improved spatiotemporal multi-channel network (IQGMCL) to solve the problem of low signal recognition accuracy. Firstly, the relevant characteristics of the IQ signal are added to the original MCLDNN network data input. After introducing the related features of IQ signals, drawing on the idea of a small convolution kernel in the VGG network, a small convolution kernel (1,3) is used to extract time series features further. These increase the number of model training features and improve the training accuracy and verification accuracy of the model. Secondly, LSTM is replaced with Bi-LSTM. On the basis of one-way flow from the past to the future of the LSTM network, data flow is increased from the future to the past. Bi-LSTM can better explore the temporal characteristics of data and improve the robustness of the model. Thirdly, a full connection layer is removed on the basis of the MCLDNN network. Due to the increase in training characteristics, it can not only prevent overfitting but also reduce model training parameters and also accelerate the convergence speed of the model. Finally, multiplicative Gaussian noise is introduced before classification to randomly erode features. It effectively alleviates the problem of training overfitting. It also improves the robustness of the model and the generalization ability of the network.

2. Dataset Introduction

This paper adopts three open-source modulation recognition datasets [10], which are RML2016.10a, RML2016.10b, and RML2016.04c. Three datasets are generated by O’Shea and T.J. using the GNURadio software platform.

RML2016.10a and RML2016.04c datasets include three types of digital modulation signals and eight types of analog modulation signals. RML2016.10b dataset includes three types of digital modulation signals and seven types of analog modulation signals. The SNR of various modulated signals ranges from −20 dB to 18 dB, with an interval of 2 dB. There are 20 kinds of SNRs in total. The size of each modulated signal is (2, 128); 128 corresponds to 128 sampling points per signal, and 2 corresponds to the input signal being a quadrature IQ two-way signal [19,20,21]. The RML2016.10a dataset has a total of 220,000 signals, the RML2016.10b dataset has a total of 1,200,000 signals, and the RML2016.04c dataset has a total of 162,060 signals. When preprocessing the dataset, the training set, validation set, and test set are divided according to the ratio of 6:2:2. In the specific division, random proportional sampling is carried out according to the number of samples of each modulation signal at each SNR. Taking the RML2016.10a dataset as an example, the number of each signal at each SNR is 1000. Randomly take 600 out of 1000 and put them in the training set, put 200 randomly into the validation set, and put the remaining 200 into the test set, so the dataset is organized in this way. Finally, the training set shape is [132000, 2,128]. The verification set shape is [44000, 2, 128]. The test set shape is [44000, 2, 128]. Similarly, in RML2016.10b, the dataset is also organized in this way. Finally, the training set shape is [720000, 2, 128]. The verification set shape is [240000, 2, 128]. The test set shape is [240000, 2, 128]. In RML2016.04c, the training set shape is [97160, 2, 128]. The verification set shape is [32460, 2, 128]. The test set shape is [32460, 2, 128].

The network training in this paper is supervised learning. Each sample signal has a corresponding data label, which corresponds to the category name of each signal. The dataset labeling process adopts the one-hot encoding method. Take the RML2016.10a dataset as an example. The dataset includes 11 types of modulated signals. The category of each signal can be obtained from the dataset. If a signal belongs to the first category, its one-hot encoding method is [1,0,0,0,0,0,0,0,0,0,0], and so on. Finally, The training set label shape is [132000, 11]. The verification set label shape is [44000, 11]. The test set label shape is [44000, 11]. Similarly, in RML2016.10b, the training set label shape is [720000, 10]. The verification set label shape is [240000, 10]. The test set label shape is [240000, 10]. In RML2016.04c, the training set label shape is [97160, 11]. The verification set label shape is [32460, 11]. The test set label shape is [32460, 11]. The three simulation datasets are highly similar to the real signals, so extending the model to the real signal datasets is of great significance in conducting experiments on these datasets.

Table 1. Open-source modulation recognition dataset description [10,22,23,24].

Dataset Name	Modulation Schemes	Sample Dimension	Dataset Size	SNR Range (dB)
RML 2016.10a	11 classes (8PSK, BPSK, CPFSK, GFSK, PAM, AM-DSB, AM-SSB, 16QAM, 64QAM, QPSK, WBFM)	2 × 128	220,000	−20:2:18
RML 2016.10b	10 classes (8PSK, BPSK, CPFSK, GFSK, PAM, AM-DSB, 16QAM, 64QAM, QPSK, WBFM)	2 × 128	1,200,000	−20:2:18
RML 2016.04c	11 classes (8PSK, BPSK, CPFSK, GFSK, PAM, AM-DSB, AM-SSB, 16QAM, 64QAM, QPSK, WBFM)	2 × 128	162,060	−20:2:18

describes the datasets.

The RML2016.10a dataset is taken as an example. Figure 1 shows the time domain waveforms of each modulation mode in the dataset, where each image is a randomly selected sample of the corresponding modulation mode.

In Figure 1, each type of signal is divided into IQ channels. The orange line is the I signal, and the blue line is the Q signal. The horizontal axis of each type of signal graph is the time axis, and the vertical axis is the amplitude of the signal. As can be seen from Figure 1, the time domain waveform of the signal varies with time. Therefore, such a signal is also called a time signal and has time sequence features. The so-called time sequence feature means that with time changing, the amplitude of each type of signal performs different regular and different high and low jumps. Thus, each type of signal has different characteristics. Using neural networks can automatically distinguish these signals after training. However, most of the current networks extract only the time sequence features of IQ signals. The relevant features are ignored. The so-called relevant features refer to the connection between the two paths of IQ signals. The connection features between I and Q are different for different signal types. By adding relevant features, more training features are added.

Although there are many similarities and differences among the above modulation waveforms, it is not easy to identify them visually due to the influence of noise, pulse shaping, distortion, etc. In particular, the time domain waveforms of the two signals, AM-DSB and WBFM, are very similar, so the time sequence features and input to the neural network are similar and not easily distinguishable.

3. Improved Spatiotemporal Multi-Channel Network (IQGMCL)

3.1. Relevant Features of IQ Signals

In literature [25], Dr. Cui Tianshu verified the validity of convolutional network structure with IQ-related features in automatic modulation recognition applications from the number, size, and depth of the convolutional layer and network operator. Based on the IQCLNet filter design method proposed by Dr. Cui, the relevant feature of IQ signals is introduced into the original MCLDNN network. According to the features of IQ signals with timing and phase features, a convolution layer with a convolution kernel size of (2,1) is first used to extract the relevant features between IQ signals. On the one hand, it reduces the dimension of data. On the other hand, it reduces the frequency of domain changes of data in subsequent processing. Then, referring to the small convolution kernel idea adopted in the VGG network, the (1,3) convolution layer is used to extract the time sequence features further. In this way, the features introduced into the training model are not only the time sequence features extracted from the spatiotemporal multi-channels but also the related feature information. The model training accuracy is better. The IQCLNet network structure principle is shown in Figure 2.

The data format of the original IQ sampling signals is N × 2, where N corresponds to the time length of the signal, and 2 corresponds to the in-phase component I and the orthogonal component Q. The two dimensions do not have the same properties, so symmetric operations cannot be carried out in the two dimensions like image processing. Therefore, at present, the convolution neural network mainly uses one-dimensional convolution to extract the time sequence features or uses two-dimensional convolution to extract the features vaguely in the processing of IQ signals. However, such operations ignore the relevant features between IQ, resulting in the loss of phase information and reducing the efficiency of information extraction. Introducing the relevant feature of the signals not only increases the number of training features but also improves the efficiency of information extraction.

3.2. LSTM and Bi-LSTM

3.2.1. LSTM

A recurrent neural network (RNN) is a kind of neural network with a good processing effect on temporal data, but it has a long-term dependence problem. That is, with the increase in input sequence length, the model cannot use the earlier data information in the sequence [26]. To solve this problem, Hochreiter et al. proposed an LSTM network in 1997.

LSTM network adds input gate $i_t$, forgetting gate $f_t$, and output gate $o_t$ three logical units to control the output of memory units. The input gate controls the current input state in the memory units. The forgetting door screens and preserves the processing results of the previous memory units. The output gate controls the output state of the memory units.

The LSTM network calculation process is as follows:

$$i_t=\sigma \left(W_{\mathrm{i}}{x}_t+U_{\mathrm{i}}{h}_{t-1}+b_i\right)$$

(1)

$$f_t=\sigma \left(W_f{x}_t+U_f{h}_{t-1}+b_f\right)$$

(2)

$$o_t=\sigma \left(W_{\mathrm{o}}{x}_t+U_{\mathrm{o}}{h}_{t-1}+b_{\mathrm{o}}\right)$$

(3)

$${\tilde{c}}_t=\tanh \left(W_{\mathrm{c}}{x}_t+U_{\mathrm{c}}{h}_{t-1}+b_{\mathrm{c}}\right)$$

(4)

$$c_t=i_t\cdot {\tilde{c}}_t+f_t\cdot c_{t-1}$$

(5)

$$h_t=o_t\cdot \tanh \left(c_t\right)$$

(6)

Formula: $i_t$ is determined by input $x_t$, output $h_{t-1}$ of the hidden layer at the previous moment and activation function σ. Input gate $i_t$ is a vector composed of real numbers between 0 and 1. $W_{\mathrm{i}}$, $U_{\mathrm{i}}$, $b_i$, $W_f$, $U_f$, $b_f$, $W_0$, $U_0$, and $b_0$ are door training parameters; tanh is the activation function.

3.2.2. Bi-LSTM

Because RNN has problems of long-term dependence, gradient disappearance, or gradient explosion in processing time series, so researchers propose LSTM to solve the issues of RNN. However, LSTM can only process forward information inputted into the neural network to obtain the predicted results. Bi-LSTM obtains the prediction results by processing the forward and backward information inputted into the neural network. The prediction result of Bi-LSTM is often better than that of LSTM [27]. The Bi-LSTM structure is shown in Figure 3.

For Bi-LSTM networks, the hidden layer state $h_t$ of each level is superimposed by three parts [28]. The output state $h_{t-1}$ of the hidden layer at the previous moment propagating in the forward direction along the time axis. The output state $h_{i-1}$ of the hidden layer at the previous moment propagating in the reverse direction along the time axis, and the input amount $x_t$ at the current moment. The combined process of each hidden layer state can be represented by Equations (7)–(9).

$$h_t=\mathrm{LSTM}\left(x_t,h_{t-1}\right)$$

(7)

$$h_i=\mathrm{LSTM}\left(x_t,h_{i-1}\right)$$

(8)

$${\mathrm{y}}_t=a_t{h}_t+b_t{h}_i+c_t$$

(9)

where LSTM refers to the operation process of a traditional LSTM network. $h_t$ is the forward hidden layer output at the current moment. $h_i$ is the backward hidden layer output at the current moment. $h_{t-1}$ is the forward hidden layer output at the previous moment. $h_{i-1}$ is the backward hidden layer output at the previous moment. $x_t$ is the input of the current time. Combined with Figure 3, it can be concluded that the LSTM layer is able to combine the current moment input $x_t$ with the previous moment output $h_{t-1}$, and the Bi-LSTM can combine the forward LSTM output $h_t$ with the reverse LSTM output $h_i$ to obtain $y_t$, which can better gain the data information. $a_t$ is the output weight of the hidden layer of the forward propagation unit. $b_t$ is the output weight of the hidden layer of the backward propagation unit. $c_t$ is the hidden layer bias optimization parameter at the current moment, and $y_t$ is the total output at the current moment.

Compared with traditional LSTM networks, the Bi-LSTM model is structurally a combination of forward and reverse propagation bidirectional cyclic structures. The data flow in the LSTM network is a one-way flow from the past to the future. On the basis of LSTM, the Bi-LSTM network increases the data flow from the future to the past, and there is no connection between the hidden layer used in the past and the hidden layer used in the future. Therefore, Bi-LSTM can better explore the temporal characteristics of the data [29,30,31].

3.3. Multiplicative Gaussian Noise

Multiplicative Gaussian noise is also called GuassianDropout in TensorFlow.Keras framework. Although there is Dropout, it is not Dropout in essence. The effect is to add multiplicative noise with a mean of 1 and a standard deviation of $\sigma$ to the results. It is often used as a stochastic regularization technique in deterministic neural network training [32,33,34]. Multiplicative Gaussian noise follows a normal distribution, and its probability density function expression is shown below.

$$f(x)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{(x-1)}^2}{2{\sigma }^2}},x\sim {\rm N}(1,\sigma )$$

(10)

where $x$ is the noise level, 1 is the mean of the noise level, $\sigma$ is the standard deviation of the noise level, and ${\sigma }^2$ is the variance.

Suppose the input variable is $\stackrel{\sim }{in}$, the formula is as follows:

$$\stackrel{\sim }{in}=\left[\begin{array}{c}I(t_1)\begin{array}{cc},I(t_2),\begin{array}{cc}I(t_3),& \dots \end{array}& ,\begin{array}{cc}I(t_i),& \begin{array}{cc}\dots & ,I(t_n)\end{array}\end{array}\end{array}\\ Q(t_1)\begin{array}{cc},Q(t_2),\begin{array}{cc}Q(t_3),& \dots \end{array}& ,\begin{array}{cc}Q(t_i),& \begin{array}{cc}\dots & ,Q(t_n)\end{array}\end{array}\end{array}\end{array}\right]$$

(11)

The output variable is: $\stackrel{\sim }{out}=\stackrel{\sim }{in}\cdot x$

The output expression with multiplicative Gaussian noise is obtained:

$$\stackrel{\sim }{out}=\left[\begin{array}{c}I(t_1)x_{I1}\begin{array}{cc},I(t_2)x_{I2},\begin{array}{cc}I(t_3)x_{I3},& \dots \end{array}& ,\begin{array}{cc}I(t_i)x_{Ii},& \begin{array}{cc}\dots & ,I(t_n)x_{In}\end{array}\end{array}\end{array}\\ Q(t_1)x_{Q1}\begin{array}{cc},Q(t_2)x_{Q2},\begin{array}{cc}Q(t_3)x_{Q3},& \dots \end{array}& ,\begin{array}{cc}Q(t_i)x_{Qi},& \begin{array}{cc}\dots & ,Q(t_n)x_{Qn}\end{array}\end{array}\end{array}\end{array}\right]$$

(12)

Noise is random [35]; in the neural network training process, adding Multiplicative Gaussian noise to corrode features randomly will effectively alleviate the training overfitting problem. It can eliminate the influence of isolated features on the training results during model training and improve the robustness of the model. To some extent, multiplicative Gaussian noise can improve the test accuracy of the model and improve the generalization ability of the network.

3.4. IQGMCL Framework

In 2020, Jialang Xu proposed a spatiotemporal multi-channel network, the MCLDNN network [18], which can extract features more effectively from the perspective of time and space. The network divides the input IQ signals into three channels and extracts the time sequence features of route I, route Q, and route IQ, respectively. The IQ data input form during model training is [Batchsize, 2, 128, channels], which belongs to four-dimensional data. MCLDNN networks are spliced in the channels dimension when merging channels. Then, the convolution kernel of (2,5) is used for further feature extraction. The extracted feature results are input into the two-layer LSTM to further extract the time sequence feature, and the two-layer DNN is connected for classification.

This paper proposes an IQGMCL network model, which divides the input IQ signals into three channels (I, Q, IQ). For the IQ path, the relevant features are first extracted with (2,1) convolution kernels with a quantity of 24. Then two layers of (1,3) convolution kernels with numbers 24 and 50 are used to further extract time series features. The I and Q paths are extracted with (1,8) convolution kernels with a quantity of 50, respectively. The number of convolution kernels of these two paths is consistent with the MCLDNN network, both of which are 50. Finally, the three channels are merged in the axis = 1 dimension, and the data form is [Batchsize, 3, 128, channels]. Further extraction of features with (3,5) convolution kernels with a quantity of 100 is conducted. Two-layer LSTM is used to extract features more effectively from time and space. Through comparison experiments, replaced LSTM with Bi-LSTM, and a full-connection layer is removed to prevent overfitting. Multiplicative Gaussian noise is introduced to naturally corrode the characteristic parameters, further alleviating the overfitting of the model and improving the robustness and accuracy of the model. The IQGMCL network model is shown in Figure 4.

4. Experiment and Analysis

4.1. Ablation Experiments

In the process of building the IQGMCL network, when the input data need to be further processed after features are extracted by the convolution layer, a series of models are established for the number of LSTM layers in the network, the number of fully connected layers, whether to replace LSTM as Bi-LSTM, and whether to introduce multiplicative Gaussian noise, etc. The purpose is to quantitatively determine the network structure for the increase in the number of training features to better complete the recognition task and improve the robustness and recognition accuracy of the model. These networks are represented as IQGMCL-1, IQGMCL-2, IQGMCL-3, IQGMCL-4, IQGMCL-5, and IQGMCL. The specific meaning is shown in

Table 2. Comparison table of network structure of each model.

Models	Network Structure
IQGMCL-1	Convolution + LSTM + LSTM + FC + FC
IQGMCL-2	Convolution + LSTM + LSTM + FC
IQGMCL-3	Convolution + LSTM + LSTM
IQGMCL-4	Convolution + LSTM + FC
IQGMCL-5	Convolution + LSTM + Bi-LSTM + FC
IQGMCL	Convolution + LSTM + Bi-LSTM + FC + Gaussian noise

.

The ablation experiment is conducted on the RML2016.10a dataset. The experiment randomly selects 60% of all data as the training set, the remaining 40% of the data, half as the verification set, and half as the test set. When training, the Epoch is 50, and the Batchsize is set to 128. Using the Adam optimizer based on stochastic gradient descent and the cross-entropy loss function. The initial learning rate is 0.001, and the learning rate is halved for every ten epochs trained. The experimental results are shown in Figure 5. In the figure, enlarging the blue dotted box and the contrast curve of the SNR from 0 dB to 18 dB is plotted separately for intuitive analysis.

The recognition accuracies obtained by the six networks are visually represented by a histogram, as shown in Figure 6. Macc (The maximum accuracy) in the table represents the maximum recognition accuracy of the networks in the SNR range. Aacc (The average accuracy) represents the average recognition accuracy of the networks when the SNR is greater than 0 dB.

Figure 5 and Figure 6 show the IQGMCL network has better recognition accuracy than the comparison networks. Comparing the two networks IQGMCL-1 and IQGMCL-2, the IQGMCL-2 recognition effect is better. Because of the introduction of signal-relevant features, the number of features increases. The model appears overfitting. Reducing a layer of FC can alleviate overfitting. Compared with IQGMCL-1, Macc and Aacc are increased by 0.41% and 0.4%, respectively. Comparing the three models IQGMCL-2, IQGMCL-3, and IQGMCL-4. IQGMCL-3 and IQGMCL-4 are based on IQGMCL-2 to remove the FC layer and a layer of LSTM, respectively. Both results are not as good as IQGMCL-2. The comparison of IQGMCL-2 and IQGMCL-5 shows that when Bi-LSTM is replaced on the basis of IQGMCL-2, the recognition effect of the IQGMCL-5 model will be improved. Compared with IQGMCL-2, IQGMCL-5 model’s Macc and Aacc are increased by 0.39% and 0.1%, respectively, because Bi-LSTM can better explore data features when the input features of the model increase, but IQGMCL-5 still has an overfitting phenomenon in the training process. To further alleviate overfitting and improve model accuracy, on this basis, the multiplicative Gaussian noise regularization layer is added. The mean of the noise level is set to 1, and the standard deviation is set to 0.5. The layer can only be mobilized when the network is trained. This layer can further eliminate the isolated features in training and improve the model’s generalization ability. Experiments show that the IQGMCL network has a better recognition effect. Compared with IQGMCL-5, Macc and Aacc are increased by 0.47% and 0.6%, respectively.

4.2. Comparative Experiments of Different Networks

In Section 4.1, the specific structure of the IQGMCL network is established through comparative experiments. In this section, the network is compared with the existing advanced modulated signal recognition networks, namely CNN1, CNN2, CLDNN, DenseNet, ResNet, CGDNet, GRU2 [36], ICAMCNET, LSTM2 [37], PETCGDNN, and MCLDNN. Three open-source datasets are compared: RML2016.10a, RML2016.10b, and RML2016.04c. Similarly, 60% of all data randomly selected on each dataset is the training set, 20% is the validation set, and 20% is the test set. When training, the Epoch is 40, and the Batchsize is set to 128, using the Adam optimizer based on stochastic gradient descent and the cross-entropy loss function. The initial learning rate is 0.001, and the learning rate is halved for every ten epochs trained. Network training in experiments is supervised learning. Each sample signal has a data label corresponding to the category name of each signal. The label adopts a one-hot encoding method. The construction of neural networks and the training of models use Tensorflow version 2.6.0 python 3.8. Use the model.fit() function for training, and pass in parameters, including training data and labels, validation data and labels, Batchsize, Epoch, and so on. The experimental results obtained are shown in Figure 7.

It can be seen from Figure 7a that on the RML2016.10b dataset, the recognition effect of the IQGMCL network and MCLDNN network is not much different when the SNR is high, and the IQGMCL network is slightly better. However, when the SNRs are between −10 dB and 6 dB, the IQGMCL network has obvious advantages. Figure 7b shows that compared with the existing modulation recognition network, the proposed improved network has a better recognition effect when the SNR is greater than 0 dB on the RML2016.04c dataset. From the experimental results of Figure 7c, it can be seen that on the RML2016.10a dataset, the IQGMCL network shows better recognition accuracy than other networks when the SNR is greater than −2 dB. The highest recognition accuracy of each network in the SNR range and the average recognition accuracy when the SNR is greater than 0 dB are shown in

Table 3. Macc and Aacc for each network.

		Datasets	RML2016.10a	RML2016.10b	RML2016.04c
	Macc/Aacc
Models
IQGMCL			0.9352/0.923	0.9374/0.933	0.9942/0.975
MCLDNN			0.9221/0.911	0.9363/0.930	0.9845/0.965
LSTM2			0.9204/0.908	0.9361/0.925	0.9502/0.905
PETCGDNN			0.9163/0.902	0.9322/0.926	0.9787/0.944
GRU2			0.9136/0.902	0.9335/0.923	0.9320/0.881
CLDNN			0.8472/0.835	0.9205/0.914	0.9322/0.918
ICAMCNET			0.8377/0.821	0.9286/0.917	0.9803/0.955
CNN1			0.8186/0.803	0.8476/0.842	0.9417/0.925
CNN2			0.8345/0.822	0.8690/0.858	0.9793/0.941
ResNet			0.8177/0.803	0.9081/0.896	0.9878/0.959
DenseNet			0.7891/0.769	0.9071/0.896	0.9803/0.939
CGDNet			0.8172/0.795	0.8971/0.889	0.9529/0.918

below. Macc (The maximum accuracy) in the table represents the maximum recognition accuracy of the networks in the SNR range. Aacc (The average accuracy) represents the average recognition accuracy of the networks when the SNR is greater than 0 dB.

Table 3 is represented graphically more intuitively, as shown in Figure 8.

Figure 8a shows the highest signal recognition accuracy of multiple networks in the SNR range on three public datasets. Figure 8b shows the average signal recognition accuracy of multiple networks when the SNR is greater than 0 dB on three public datasets. The recognition effect of each network model can be compared more intuitively through these two figures on three different datasets. Among the comparison networks, it can be found that the MCLDNN network has the best recognition accuracy. However, the IQGMCL network proposed in this paper has a higher recognition accuracy than MCLDNN. Therefore, MCLDNN is used as a reference object to present the results of further experiments.

According to Table 3, on the RML2016.10a dataset, the Macc and the Aacc of the IQGMCL network are 93.52% and 92.3%, respectively, which are 1.31% and 1.2% higher than the MCLDNN network. On the RML2016.10b and RML2016.04c datasets, the Macc and the Aacc of the IQGMCL network are 93.72%, 93.3%, 99.42%, and 97.5%, respectively, which are improved by 0.11%, 0.3%, 0.97%, and 1% compared with the MCLDNN network.

The network weights will be continuously updated in real time during the training process. We supervise val-accuracy to save the weights with the best verification accuracy for testing. Using the original spatiotemporal multi-channel network MCLDNN as the comparison model, the best verification accuracy and loss function of the IQGMCL network in the training process on three datasets are shown in Figure 9 below.

As can be seen from Figure 9a,b, the improved spatiotemporal multi-channel network on the three datasets is better than the original, both in the loss function and the verification accuracy.

In the specific training process, the comparison curves of training accuracy (train_accuracy), training loss (train_loss), verification accuracy (val_accuracy), and validation loss (val_loss) of the IQGMCL network and MCLDNN network are shown in Figure 10.

From Figure 10a,b, it can be found that the train_accuracy and train_loss of the IQGMCL network are significantly better than those of the MCLDNN network on the three publicly available datasets. This verifies the conclusion that the training accuracy is improved due to the increase in the number of features after the introduction of the relevant features of the IQ signal. From Figure 10c, it can be concluded that on each dataset, the val_accuracy of the IQGMCL network is better than that of the MCLDNN network as a whole. Especially on the RML2016.10a dataset, the IQGMCL network can achieve better recognition with fewer training times. After 20 epochs, the IQGMCL’s val_accuracy has reached a good level, indicating that the network converges faster, and IQGMCL’s highest val_ accuracy is higher than the original MCLDNN network. As can be seen from Figure 10d, on RML2016.10a, the IQGMCL network can achieve a lower level of val_loss with fewer training epochs. It also verifies that the network converges faster and shows that the val_loss is lower. On RML2016.10b and RML2016.04c, the IQGMCL’s val_loss is smaller than that of the MCLDNN network in general.

To more intuitively see the recognition accuracy of each type of signal in each SNR, the recognition accuracy diagrams on three datasets are drawn here, represented by the IQGMCL network, as shown in Figure 11.

Figure 12, Figure 13 and Figure 14 show several representative confusion matrices of IQGMCL and MCLDNN networks under three datasets. In a confusion matrix, the accuracy of the recognition result is expressed by the color depth of each square. The abscissa of the confusion matrix represents the results of classifying various modulated signals with networks. The ordinate represents the true modulation methods. If the diagonal of the matrix is darker, then the model’s prediction accuracy is better. If the color blocks are scattered throughout the matrix and are not concentrated at the diagonal, then the recognition effect is not ideal.

Figure 12 shows the confusion matrices in the RML2016.10a dataset when using IQGMCL and MCLDNN networks for signal modulation recognition, and SNRs of 0 dB, 4 dB, and 12 dB are selected. Where (a)(b)(c) are the results of the IQGMCL network, (d)(e)(f) are the results of the MCLDNN network. It can be concluded that the overall recognition effect of the IQGMCL network is better than that of MCLDNN. In particular, IQGMCL performs better in distinguishing between QAM16 and QAM64 signals. The improved network also improves the recognition accuracy of WBFM signals. Figure 13 shows the confusion matrices in the RML2016.10b dataset when using IQGMCL and MCLDNN networks for signal modulation recognition, and SNRs of 0 dB, 4 dB, and 12 dB are selected. It can be found that when the SNRs are 0 dB and 4 dB, the IQGMCL recognition effects are better. When the SNR is 12 dB, the two effects are comparable. Figure 14 shows the confusion matrices when the SNRs are 6 dB, 8 dB, and 14 dB on the RML2016.04c dataset. Similarly, (a)(b)(c) are the results of the IQGMCL network, (d)(e)(f) are the results of the MCLDNN network, and it can be derived that the recognition effect of the IQGMCL network is better than that of the MCLDNN network.

4.3. Experimental Conclusion

This chapter mainly constructs the IQGMCL network through ablation experiments and verifies its progressiveness in the field of signal modulation recognition. In the process of building the IQGMCL network, when the input data need to be further processed after features are extracted by the convolution layer, a series of models are established for the number of LSTM layers in the network, the number of fully connected layers, whether to replace LSTM as Bi-LSTM, and whether to introduce multiplicative Gaussian noise, etc. Then, the network is compared with other existing networks on three open-source modulated signal datasets. Various signals’ recognition accuracy contrasting curves are drawn. Meanwhile, the train_accuracy, train_loss, val_accuracy, and val_loss of IQGMCL and MCLDNN are compared, and the highest recognition accuracy of various models in the SNRs range and the average recognition rates when the SNR is greater than 0 dB are also compared. Finally, some confusion matrices are selected to observe the recognition effect of the networks more directly. Specifically, on the RML2016.10a dataset, the highest recognition accuracy of the IQGMCL network and the average recognition accuracy of the network when the SNR is greater than 0 dB are 93.52% and 92.3%, respectively, which are 1.31% and 1.2% higher than the MCLDNN network. The IQGMCL network has a faster convergence speed than the MCLDNN network. On the RML2016.10b dataset, the highest recognition accuracy of the IQGMCL network and the average recognition accuracy of the network when the SNR is greater than 0 dB are 93.72% and 93.3%, respectively, which are improved by 0.11% and 0.3% compared with the MCLDNN network. On the RML2016.04c dataset, the highest recognition accuracy of the IQGMCL network and the average recognition accuracy of the network when the SNR is greater than 0 dB are 99.42% and 97.5%, respectively, which are improved by 0.97% and 1% compared with MCLDNN network. Compared with other existing advanced networks, the IQGMCL network also has a better recognition effect on three datasets. It proves the progressiveness of the IQGMCL network in the current field of modulation recognition.

5. Conclusions

This paper proposes an improved spatiotemporal multi-channel network model (IQGMCL) based on the existing modulation signal recognition algorithms. The first chapter analyzes the research status at home and abroad in recent years and summarizes the existing advanced signal modulation identification networks. Chapter 2 describes the three open-source signal modulation identification datasets used in this article. Chapter 3 introduces the various improved modules in IQGMCL, which theoretically justifies the improvements. Referring to the multi-channel data input form of the MCLDNN network, the structure diagram of the IQGMCL model is drawn by fusing each module. Chapter 4 includes three parts: ablation experiment, comparative test, and experimental conclusion. The ablation experiment mainly determines the improved network structure quantitatively to achieve a better signal recognition effect for the increase in training features. The comparative experiment part mainly compares the IQGMCL structure determined in the ablation experiment with various existing advanced algorithms. It is found that the IQGMCL network has better recognition accuracy on three datasets. Among the comparison networks, it can be found that the MCLDNN network has the best recognition accuracy. Therefore, MCLDNN is used as a reference object for further experiments. The comparison curves of Train_accuracy, Train_loss, Val_accuracy, and Val_loss of IQGMCL and MCLDNN networks on three datasets are plotted, drawing the Val_accuracy and Val_loss comparison bar charts of the two networks under the optimal case. From the results, IQGMCL has higher Train_accuracy and Val_accuracy, lower Val_accuracy and Val_loss, and IQGMCL networks converge faster. Finally, by showing a part of the confusion matrices, the recognition effect of IQGMCL and MCLDNN is compared more directly. The experimental conclusion summarizes the advantages of IQGMCL network recognition accuracy and illustrates the progressiveness of the IQGMCL network in the current modulation recognition field.

The innovation points of the improved spatiotemporal multi-channel network proposed in this paper are as follows:

• Adding the relevant features of the IQ signal to the original MCLDNN network data input. After introducing the related features of IQ signals, drawing on the idea of a small convolution kernel in the VGG network, a small convolution kernel (1,3) is used to extract time series features further. These increase the number of model training features and improve the training accuracy and verification accuracy of the model.
• Due to the increase in training features, when the features are extracted by the convolutional layer and the data need to be further processed, it is necessary to quantitatively redesign the network structure on the basis of the MCLDNN network. Specifically, introducing Bi-LSTM to replace LSTM. On the basis of one-way flow from the past to the future of the LSTM network, increasing data flow from the future to the past. Bi-LSTM can better explore the timing characteristics of data and improve the robustness of the model. Then, on the basis of the MCLDNN network, removing a full-connection layer. It can not only prevent overfitting but also reduce the model training parameters and also speed up the convergence of the model.
• Multiplicative Gaussian noise is introduced before the classification after the full connection layer. It corrodes the features randomly, effectively alleviates the problem of training overfitting, improves the robustness of the model, and improves the generalization ability of the network. In this experiment, three open-source datasets are used for comparative experiments, which verifies the universal adaptability of the model and is convincing.

Combine the experimental results and conclusions to clarify the next step:

• It can be concluded from the experimental results that, no matter whether it is in the RML2016.10a or RML2016.10b dataset, it is not easy to improve the recognition accuracy of signals with high SNR. One important reason is the low recognition accuracy of WBFM signals, according to Figure 11. About half of WBFM signals are misclassified as AM-DSB signals. It can also be understood through signal simulation images that the IQ time-domain waveforms of these two signals are very similar, according to Figure 1. Therefore, the next step is how to distinguish these two signals more effectively to improve the overall accuracy of the model.
• Many existing algorithms generally have poor recognition performance at low SNRs. Improving the network recognition performance at low SNR is also a problem to be solved. In the next step, we consider adding an attention mechanism to build an improved network model to improve the signal recognition accuracy at low SNRs [27,38,39].