An automatic sleep staging method based on CRNN-HMM model

Sleep staging is an important process for detecting sleep quality and diagnosing sleep disorders. However, traditional sleep staging is a labor-intensive task, and it is prone to subjective errors. Therefore, this paper innovatively proposes an automatic sleep staging model based on single-channel EOG—CRNN-HMM. The CRNN-HMM in this paper combines Convolutional recursive neural networks(CRNN) and hidden Markov model(HMM). The main idea of this model is to use CRNN to automatically extract features from EOG, and send the feature signals to a variant of RNN, Bi-directional Long Short-Term Memory(BiLSTM), to mine the dependencies between sleep stages and realize automatic staging of sleep data. Finally, a Hidden Markov Model is used to convert the prior information of the sleep phase of the adjacent EOG cycle in order to improve the classification performance of S1, thereby improving the classification performance of CRNN. The simulation results show that the overall accuracy of the model on the CAP-Sleep data set reaches 95.0%, which proves that the model can provide a way for the evaluation of sleep quality.


Introduction
In the current society, sleep quality problems are becoming more and more serious. Research and surveys show that one-third of the world's people have sleep problems [1], so sleep staging has attracted more and more attention. Sleep staging can be used as the key basis for the diagnosis of sleep disorders, and the staging efficiency and accuracy are of great significance for the diagnosis and treatment of related diseases. Therefore, the study of sleep staging has been favored by more people.
The typical sleep analysis method is to use a traditional machine learning model, which requires different analysis methods to manually extract the characteristic parameters of the signal, and then use different classifiers to classify the signal [2][3]. In the past traditional algorithms, good results have been achieved, but due to the limitation of prior knowledge, no greater breakthroughs can be made in feature extraction and classification. In recent years, deep learning has developed rapidly and has achieved great success in various aspects. Compared with traditional algorithms, deep learning networks not only break the limitations of prior knowledge, but also make the staging effect more obvious. Therefore, deep learning networks are favored by more scholars. Obviously, the deep learning network has made significant progress in the sleep staging process, but it still has a lot of room for breakthroughs in the transition of sleep stages. Standards for dividing sleep stages and acquisition of experimental data.

Standards for dividing sleep stages
Polysomnography(PSG) is an important way to monitor sleep stages. In order to better analyze night polysomnography, the R&K rule is proposed. According to R&K rules [4], each 30-second epoch is marked as Wake, REM, and NREM, and is further divided into one of S1-S4 sleep stages, MOVEMENT, and UNKNOWN. Generally, MOVEMENT and UNKNOWN are excluded in the experiment. Later, the AASM guidelines were re-established, and the sleep relationship is shown in Figure 1.

Experimental data
The experimental research data uses a public data set: CAP-Sleep sleep database [5], which is provided by PhysioNet. The CAP-Sleep sleep database includes night polysomnograms of 108 subjects, including 9 patients with insomnia, 16 healthy subjects and other patients with sleep disorders, each subject includes 2 EOGs Signal channels, 3 EEG signal channels, 2 EMG signal channels and other physiological signals. Previous studies have shown that the EOG signal of the ROC-LOC channel can be used for effective sleep staging. Therefore, in this experiment, the EOG signal of the ROC-LOC channel of 9 subjects who had undergone pre-processing was selected for sleep staging research. Table  1 lists the classification results of sleep experts on the experimental data. Figure 2 shows the sleep stage records of the subjects throughout the night.  Figure 2. The subjects' sleep records for one night.

Introduction of CRNN model
CRNN is a combination of Convolutional Neural Network(CNN) and Recurrent neural network(RNN), which has the advantages of both CNN and RNN. CNN envelopes the convolutional layer, pooling layer and activation function. RNN is very easy to extract sequence features and has become one of the most important neural networks. The CRNN model designed in this paper uses CRNN to automatically extract features from the original electrooculogram signals, and sends the feature signals to the RNN variant BiLSTM to mine the dependencies between sleep stages and realize the automatic staging of sleep data. The module is composed of 4 convolutional layers, 4 pooling layers and a fully connected layer. Figure 3 shows the CRNN model architecture.

CNN module
CNN is an efficient recognition method, mainly composed of input layer, convolutional layer, convergence layer [6], fully connected layer and output layer. The convolution and pooling operations of CNN in the model are all one-dimensional. The one-dimensional convolution operation is defined as follows： Where X is the input feature map, and Y is the output feature map of the current layer. N is the total number of input feature maps. W and b are the convolution kernel and the deviation vector, respectively. * ( • ) and f( • )are vector convolution and activation functions, respectively. In this model, the ReLU function is used for all convolutional layers, and a batch normalization layer is added after each convolutional layer in the CNN model. Table 2 shows the main parameters in the CNN model.

BiLSTM module
LSTM is an extension of RNN network, its purpose is to solve the problem of gradient disappearance in RNN. The LSTM neural network includes three gates and a memory state unit.
In the method proposed in this paper, BiLSTM consists of two independent LSTMs [9], which can summarize information from forward and backward directions, and then merge information from both directions.

Fully connected classification layer
The output of BiLSTM is used as the input of the fully connected classification layer, and finally the sleep staging prediction is performed through the fully connected layer. The fully connected classification layer introduces the ReLU activation function to avoid the gradient explosion problem. Finally, the SoftMax function is used to output the probability prediction value of the correct classification for each sleep period, which indicates the credibility of the correct classification. The calculation formula is as follows: Where represents the output value of the i-th node, and c represents the category of sleep staging.

Model introduction
Hidden Markov Model is a generative model [10], which defines two random sequences, named state sequence and observation sequence. In HMM, the state transition matrix and the observation probability matrix define the initial probability distribution. In the CRNN-HMM model of this article, the observation sequence O and the state sequence I are the output of the CRNN and the actual sleep phase sequence, respectively. Both sets Q and V are defined as: Q = V = {Wake, S1, S2, S3, S4, REM}. In addition, the CRNN-HMM in this paper is trained by a two-step training algorithm. First, the CRNN model was trained. Subsequently, for each subject I, input the training data into the CRNN model, which has been trained and obtained a preliminary hypnotic map O i . The actual sleep stage annotation sequence of subject I is denoted as I i . Then the training set of the hidden Markov model can be constructed as D = {(I 1 , O 1 ), (I 2 , O 2 ),… ， (I L , O L )}. Then calculate the parameters through maximum likelihood estimation , the maximum likelihood estimation is defined as: Finally, use the Viterbi decoding algorithm to decode the hidden Markov model.

Simulation result analysis
In this experiment, in order to improve the efficiency of sleep staging, an innovative automatic sleep staging method based on the CRNN-HMM model of single-channel EOG signal is proposed. The experiment shows that the accuracy of staging is high, and the system is stable. , So the proposed CRNN-HMM model has certain reference value.
In this experiment, all feature data is Gaussian normalized to [0,1], and each type of sample in the data is scored 7, 3 points, of which 70% is used as the training set and 30% is used as the test set. So as to get a more balanced training sample and test sample. Then use the CRNN-HMM model to divide the sleep process into 6 stages (Wake, S1, S2, S3, S4, REM), and get the classification accuracy. In order to better express the classification effect of the model, the corresponding indicators are used for evaluation. The classification effect of the model proposed in this article is evaluated using different indicators, including overall accuracy (accuracy, ACC), precision (precision, PR), Specificity (SP) recall (recall, RE). The calculation methods of each indicator are as follows: Where TP means the number of positive classes predicted as positive classes, TN means the number of negative classes predicted as negative classes, FP means the number of negative classes predicted as positive classes, and FN means the number of positive classes predicted as negative classes Number. Since this article is studying a multi-classification problem, the category to be evaluated is regarded as the positive category, and the other categories are regarded as the inverse category to calculate each index.
The CRNN-HMM model proposed in this paper got rid of the limitation of prior knowledge and achieved certain results. It has been verified under the CAP-Sleep database, and the model has realized the improvement of classification accuracy.
From Table 3, we can see the classification accuracy, recall rate and accuracy of different sleep periods. In Table 3, it can be seen that the indicators of S2 period are slightly lower than those of other periods, but certain results have been achieved. In terms of accuracy, each period has very good results, so it can be seen that the model has a certain reference value.   Figure 4 that for the CRNN model, the awake period has a higher classification accuracy, followed by the S2 period, and the S1 period has the worst accuracy. For the CRNN-HMM model, it can be seen that the S1 stage is increased from 0.79 to 0.92, and the effect of other stages is not particularly obvious. It can be seen that the HMM model can improve the classification performance of S1, thereby improving the classification performance of CRNN.
As shown in Table 4, in past research experiments, many scholars have also proposed some sleep staging models, so they can be compared with this article. It can be seen from Table 4 that compared with other EOG experiments, the CRNN-HMM model has achieved a certain effect, and its accuracy has been improved to a certain extent. However, most of the current sleep staging studies are based on EEG signals or combined with EOG, EMG and other signals. Only a few scholars use EOG for sleep staging. As can be seen from Table 4, the effect of using EOG for sleep staging in this experiment is equivalent to EEG. A. Supratak, H. Dong, C. Wu and Y. Guo, "DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 11, pp. 199811, pp. -200811, pp. , Nov. 2017. Even better than the EEG signal, so it can be concluded that only using the EOG signal for sleep staging can also achieve a better classification effect, which provides experience for future research.

Conclusion
Taking the CRNN network as a prototype, the HMM model is added to change the sleep stage transition, and the EEG data of different subjects is used to verify the superiority of the proposed CRNN-HMM model. During the experiment, this model realized the automatic staging of sleep data, and its classification accuracy reached 95%, which is higher than most existing models, so this model can be applied to sleep staging.
In future sleep staging studies, different types of data and classifiers can be selected to further improve the staging efficiency.