Research on fault diagnosis method based on KPCA-SDAE

The complexity of modern chemical processes is increasing, and the degree of process automation is also improving. In order to extract further features and diagnose the faults of TE process accurately, steadily and quickly, this paper presents a method of using Kernel Principal Component Analysis (KPCA) and Stacked Denoising Auto-Encoder (SDAE) to extract the secondary features and using softmax to classify the faults. Firstly, the KPCA is used to reduce the dimensionality of the data, reduce the irrelevant feature information, and prepare for further feature extraction of SDAE. Then SDAE is used to extract features through unsupervised pre-training and supervised global fine-tuning, and effective feature information is obtained for classification. Finally, softmax classifier is used to classify the extracted features, and fault state recognition and classification of TE process are realized. Experiment results show that this method can extract feature information successfully, identify fault types accurately, and its accuracy and training speed are better than those using SDAE alone.


INTRODUCTION
Fault diagnosis technology includes monitoring the industrial process for a period of time, using the periodic sampling measurement results of the sensor, extracting the fault sensitive features from the measurement results, and treating the extracted sensitive features to judge the current health state and the fault type of the industrial process [1]. The fault diagnosis of the TE process is actually the process of pattern recognition, which mainly includes feature extraction and fault classification [2]. When the TE process fails, the variables of the process data will change. If the features of the fault data can be extracted, the fault can be classified. Therefore, the key to fault diagnosis is the extraction of fault features [3].
According to the actual chemical reaction process, Tennessee-Eastman Chemical Process (TE) is an open and challenging chemical model simulation platform developed by Eastman Chemical Company of the United States [4], which produces data of time varying, strong coupling and Nonlinear features. The data is widely used to test the control and fault diagnosis models for complex industrial processes. So far, domestic and foreign scholars have proposed some advanced methods for chemical process fault diagnosis. Danfeng Xie et al. [5] proposed an HDNN method to diagnose TE process failures, whose the average fault diagnosis accuracy is 80.5%. Feiya Lv et al. [6] proposed a stack-based sparse auto-encoder fault diagnosis model, which improves not only the separability between faults and normal processes, but also the chemical basis of the TE process data. The fault classification accuracy showed  [7] put forwarded a fault diagnosis model of DBN. The diagnosis and classification of 20 types of TE process fault types, the average fault diagnosis accuracy of this method reached 82.1%. Zhixin Hu et al. [8] proposed a new incremental imbalance correction depth neural network, which extended the fault diagnosis technology to the unbalanced data stream, and had significant robustness and adaptability in chemical fault diagnosis. Hao Wu et al [9] suggested a fault diagnosis algorithm for chemical process based on deep convolution neural network model. The average fault diagnosis rate of the 20 types of faults was as high as 88.2%, which was higher than the fault diagnosis methods mentioned in other papers. However, as the amount of data increases, the data is updated, and the method of fault diagnosis needs to be continuously updated and improved. Therefore, the TE process fault diagnosis still needs further research.
In summary, this paper proposes a fault diagnosis method for KPCA-SDAE-Softmax. Firstly, the data is subjected to KPCA dimensionality reduction, the extraneous components in the data are removed, and the dimension of SDAE feature extraction is reduced to improve the speed of SDAE feature extraction. Then, the dimensionally reduced data is input to the SDAE network for secondary feature extraction to extract deep abstract features of the fault data. Finally, the feature data is classified by the softmax classifier.
The main research contents of this paper are structured as follows. In the first part is about the research background, significance and research status of TE process fault diagnosis are expounded, and the proposed method is proposed. In Section 2, the principle and model structure of the proposed method are discussed in detail. In Section 3, the proposed method is experimentally verified in the process of Tennessee Eastman. Section 4 summarizes the method and experimental validations presented in this paper.

KPCA-SDAE-Softmax
Due to the high complexity of the TE process, multivariate, strong coupling, and nonlinearity make the traditional fault diagnosis method unable to meet the requirements of fault diagnosis [10]. Deep learning has a strong learning ability, and is widely used in image recognition. Fault diagnosis based on deep learning is also pattern recognition basically. However, the deep learning network is usually complicated in structure, and the processing speed of high-dimensional data is relatively slow. Therefore, KPCA is first used to reduce the original data, to speed up the learning of deep neural networks, and to improve the accuracy of fault diagnosis.

KPCA
KPCA is an improved method for PCA. The principle is to map n samples 12 n x ,x ,...,x of data X through non linear mapping  , from lower-dimensional input space n R to a higher-dimensional space, which is recorded as Then, to analyze the principal components of that has been mapped to the higher-dimensional space, thereby to reduce the dimension of the original sample data and to extract the feature information more effectively. The specific steps of the KPCA algorithm are as follows: Step 1. Preprocess raw data.
Step 2. The kernel matrix k k is computed and the preprocessed raw data is mapped from the low-dimensional data space to the high-dimensional feature space using kernel functions. The used kernel function is radial basis function, as shown in equation (1).
Step 3. The kernel matrix ce k is centered and used to modify the kernel matrix. As shown in equation ( , , , ) , that is, the larger the eigenvalue, the more useful information is contained, so the eigenvalues are sorted in descending order, and at the same time the eigenvectors are adjusted accordingly.
Step 5. Through the Schmidt orthogonalization method, orthogonalize and unitize the feature vector to get    1 2 n ( , , , ) . Step 6. The cumulative contribution rate of the eigenvalue    1 2 n ( , , , ) was calculated, and a threshold p . If   t p , choose the first t principal components as the data after dimension reduction.

DAE.
Auto-encoder (AE) is a 3-layer neural network ，whose structure can be separated into two phases: encoding phase and decoding phase. The input signal X is converted by the encoder into a signature which is then converted by the decoder into a reconstructed signal. Then calculate the reconstruction error between the reconstructed signal and the input signal and propagate the error. Reduce errors by adjusting the weight of connections between neurons in the AE network.
(1) Encoder: After normalizing the data set, calculate the high-dimensional input vector X using a nonlinear encoder function and map it to the hidden layer Y . The formula is expressed as follows equation (3).
θ Wb ; W is the weight matrix of the input layer to the hidden layer; b is the bias term; s is the activation function, and the sigmoid function is generally used.
(2) Decoder: The output vector Z is reconstructed byY according to equation (4). W is the weight matrix of the hidden layer to the output layer;  b is the bias term; s is the sigmoid activation function. The loss function is expressed as equation (5).
The Denoising Auto-encoder (DAE) is a valid improvement method for the Auto-encoder. The DAE improves the robustness of the training samples by randomly adding noise. Compared with the AE, the structure adds noise between the input layer and the hidden layer, so that the new processed noise has data, and then performs AE operation according to the new noise data. As shown in Figure 1.

SDAE. Stacked Denoising Auto-encoder (SDAE)
, which is a stack of several DAE units. The output of each layer is used as the input to the next layer of the DAE hidden layer, and the next hidden layer in the SDAE is initialized by the previous layer. The structure principle of the DSAE is shown in Figure 2.

Softmax
The Softmax model is a generalization of the logistic regression model and is mainly used on multi-classification problems. In the Softmax classification, the label y of the data has k different values.
Given a training data set   is estimated for each category j using a hypothesis function. That is, the probability that the category label is estimated for each of the k different possible values. Therefore, the hypothesis function will output a K dimension vector (the sum of its elements is 1) to represent the estimated probability of K classes. Assume that the function h ( x )  is as shown in equation (6).
The Softmax regression cost function is a negative likelihood logarithmic function, as shown in equation (7).
The feature vector obtained from SDAE self-learning is used as the input vector of the softmax algorithm to classify multiple fault modes.

KPCA-SDAE-Softmax
The TE process fault diagnosis process based on KPCA-SDAE-Softmax includes data dimensionality reduction, denoising, unsupervised pre-training, and supervised global fine-tuning. The specific steps are as follows: (1) Obtain raw data. (2) Data preprocessing. Perform kernel principal component analysis on the original data, map the sample data to high-dimensional space, and extract the feature information of effective nonlinear structure of sample data.
(3) Use SDAE for feature extraction to obtain effective deep feature information.
(4) Use the softmax classifier to classify the features extracted by the SDAE and output the fault diagnosis results.

CASE STUDY
In order to verify the validity of the method proposed in this paper, the Tennessee-Eastman process proposed by Downs and Vogel is used to verify the experiments of KPCA-SDAE-Softmax.

TE process
The entire process consists of five operating units: reactor, condenser, recycle compressor, separator and stripper. The process flow chart of the TE process is shown in the Figure 3. The process simulation of the TE process includes 21 types of faults and 1 normal data, for a total of 22 types of data. Each type of fault data contains 960 samples, each consisting of 52 process variable samples, so each type of fault data is a 960*52 matrix.

Experimental environment
First, the original input data is mapped to the high-dimensional feature space by the kernel function, and the PCA drop is performed in the mapped high-dimensional feature space. Since the 52 variables of the data sample contribute differently to the dimension reduction, a variable with a large contribution is selected as the principal. The contribution rate of each variable and the number of final principal components are shown in the Figure 4.   Figure 6. Training Accuracy Curve. It can be seen from the experimental comparison that after the KPCA dimensionality reduction data, the training time with SDAE network takes less time than that with SDAE alone, and the diagnostic accuracy with SDAE is higher than that of SDAE alone. The comparison results are shown in Table 1.

CONCLUSION
This paper presents an improved SDAE method based on KPCA-SDAE-Softmax. And the method is used for TE process troubleshooting. The experimental results show that this method has higher efficiency and higher precision for fault diagnosis of TE process.