Joint feature enhancement mapping and reservoir computing for improving fault diagnosis performance

Complicated industrial robot structure and harsh working conditions may cause signal features collected in the condition monitoring process to be seriously disturbed. In this paper, a joint feature enhancement mapping and reservoir computing (FEM-RC) method is presented to handle the industrial robot fault diagnosis problem. Firstly, a feature enhancement mapping (FEM) method is proposed to achieve intraclass distance minimization and interclass distance equalization to obtain an enhanced feature matrix. Then, the first reservoir computing (RC) network is adopted to map the original feature matrix to the feature enhancement matrix, and the second RC network is for fault type classification. The results of the experiment carried out on a six-axial industrial robot demonstrate that compared with other peer models, the present FEM-RC has better fault diagnosis performance and robustness.


Introduction
Industrial robots often have complex structures and extreme working environments, which are prone to various forms of failure and corresponding costs [1,2]. Therefore, it is of long-term significance to carry out condition monitoring and fault diagnosis on them. In recent years, the fault diagnosis of engineering system is mainly based on three ideas [3]: simulating the vibration response of the gearbox, fault simulation modelling under different loads; time-domain methods, frequency-domain methods, timefrequency domain methods, and other signal processing methods; data-driven intelligent diagnosis method. Specific signal processing analysis methods include short-time Fourier transform [4], wavelet analysis [5], wavelet packet transform [6], empirical mode decomposition [7], Winger-Ville distribution [8], matching pursuit method [9] and so on. However, for large-scale industrial rotating machinery systems (including industrial robots) with variable operating conditions, various fault types and unclear fault mechanisms, it is often difficult to ensure that the built diagnosis model can accurately extract features and mine rich potential information in big data. Thus, deep learning has become a research hotspot in the field of fault diagnosis. In actual work, feature selection and feature extraction [10,11] are both methods to find the most effective features from the original data. Guo et al. [12] proposed a novel hierarchical learning rate adaptive deep convolution neural network based on the improved algorithm to achieve bearing fault pattern recognition and fault degree evaluation. Tang et al. [13] used a new fault diagnosis method named Fisher discriminative sparse representation based on deep belief IOP Publishing doi:10.1088/1757-899X/1207/1/012020 2 network to obtain superior performance for feature extraction and classification in the field of complex system fault diagnosis. Shao et al. [14] proposed to construct a new deep auto encoder with noise reduction auto encoder and compressed auto encoder to enhance feature learning ability. He et al. [15] presented a new method that combines the convolutional neural network and long short-term memory network to perform a gradual changing fault classification.
Although these feature processing methods are powerful in some cases, there are still challenges in dealing with mechanical fault diagnosis. Especially deep learning models have the limitation of high computational complexity.
In this study, we proposed a feature enhancement mapping method to improve the performance of fault diagnosis. Firstly, a set of activation function mapping method is used between the original data and the class center to reduce the intraclass decision distance and form a new feature matrix. Then, each type of fault is distributed equally in the feature space to increase the decision distance between different categories to form a feature-enhanced matrix. Thirdly, reservoir computing (RC) is adopted to map the original feature matrix to the feature-enhanced matrix. Finally, put the feature-enhanced matrix into the other RC network for classification, which can achieve high class separability with a small computational complexity. In order to verify the performance of FEM-RC, it was applied to a six-axial industrial robot gearbox fault diagnosis experiment, and compared with the peer fault diagnosis models based on RC, stack autoencoder (SAE), and deep Boltzmann machine (DBM), respectively.

Model definition
This section describes the proposed FEM-RC model in detail. The detailed methods of the intraclass distance minimization and the interclass distance equalization are given in subsection 2.1. Mapping via RC network is realized in subsection 2.2. Application to intelligent fault diagnosis is implemented in subsection 2.3.

Intraclass distance minimization and interclass distance equalization
A feature enhancement mapping (FEM) approach is proposed for strengthening original characteristics expression. Establish feature enhancement mapping to reduce the intraclass distances of the same class in each dimension of the feature matrix to form a new class center matrix, and then adopt a balanced distribution algorithm to increase the distance between different classes in each dimension of the class center matrix. The transformation of training samples is shown in equation (1).
where 1 ∈ × represents the weight between raw sample data and the feature matrix reconstructed by FEM. G is the sigmoid function in activation functions shown in equation (2), ∈ ×1 is the output layer and ∈ ×1 is the input layer consisting of original data. To achieve feature enhancement mapping, the process of intraclass distance minimization and interclass distance equalization in each dimension is shown in equation (3) and (4) separately.
where is the center of the i-th category, l is the sample of the i-th category and suppose that the input layer contains c categories (i= 1, 2, …, c). ̅ ∈ ×1 represents the sample of reconstructed feature matrix with even distribution in vector space. To acquire the 1 , the raw sample data and the feature matrix reconstructed by FEM can be represents by equations (5) and (6). To avoid under fitting and over fitting, the regularization factor is added in equation (7). The regression model with different λ value is trained to control the fitting trend of the model.
where λ is the regularization factor and e is the unit diagonal matrix. Then, the feature matrix reconstructed by FEM can be calculated by equation (1).

Mapping via reservoir computing
The RC network is composed of input layer, reservoir layer and output layer. Figure 1 shows the network structure of the classic RC network. respectively. Correspondingly, the connection weights between the input layer and the reserve pool, the recursive connection weights inside the reservoir, and the connection weights between the reservoir and the output layer are represented by ∈ × , ∈ × and ∈ * ( + ) respectively. The state updating equation of the reservoir is shown in (8), and the output state updating equation of RC is shown in equation (9).
where the activation functions (·) and (·) are tanh and identity respectively. As shown in the optimization objective function of equation (10), the network output weight is obtained by minimizing the mean square error of the output of the RC network and the expected output. Suppose denotes the network state matrix and denotes the desired output sequence matrix.
where γ is regularization coefficient, ridge regression is used to solve for W out :

Application to intelligent fault diagnosis
The application procedure of the FEM-RC model for industrial fault diagnosis is shown in figure 2, and the detailed steps are as follows: Step1: Different data are collected according to the input variables of different fault categories to construct an input feature matrix. Step2: Divide data into a training set and a testing set.
Step3: Adopt FEM to enhance the features of the training set and test set.
Step4: Perform the first RC network to map the training set to a uniformly distributed feature matrix.
Step5: Perform the second RC network to map the uniformly distributed feature matrix to the corresponding label and evaluate the model's classification performance.
Step6: Fed testing data into the trained FEM-RC model to classify fault state mode.
Step7: Output the classification result. End.

Model evaluation
Accuracy/computing time

Experiments
To verify the performance of the proposed FEM-RC in fault diagnosis, it was applied to the health state diagnosis of a six-axial industrial robot and compared those with the other three models. Firstly, the confusion matrix is adopted to express the detailed classification accuracy of each model. Secondly, t-SNE (t-Distributed Stochastic Neighbor Embedding) is adopted to visualize the data distribution before and after FEM processing.

Dataset
The attitude signal of the industrial robot was collected through the WT901C-485 attitude sensor. The sampling frequency of the attitude sensor was 100 Hz, and the sampling time was 20s. Six failure state modes were defined in table 1. The data source information of every failure was detailed in table 2. Finally, 13500k data points were collected in total, and each sample length was 900 points. In this way, 15000 samples can be obtained and every 2500 samples correspond to one kind of fault. Among them, half of the acquired samples were randomly selected as the training samples and the rest were used as the test samples.

Model parameter settings
All the training and testing processes of the above models were programmed with matlab2018a and executed on a PC with Intel® Core (TM) i5-4590 CPU @ 3.3GHz processors and 16 GB RAM. The main parameter settings of different models are in detailed in table 3    For each model, twenty trials are carried out under the same parameters, and the classification results are detailed as follows. Table 4 shows the mean and standard deviation of classification accuracy and computing time. The fault diagnosis results given by the proposed FEM-RC model achieved the highest average accuracy with 94.09%. The average fault diagnosis accuracy is 81.7% by RC, 49.6% by SAE and 78.95% by DBM. Meanwhile, FEM-RC acquires the best robust with the lowest standard deviation among the five approaches. Table 4 Figure 3 shows the confusion matrix of each model at the highest accuracy in 20 experiments. Each row of the confusion matrix corresponds to the real value of the label, and each column corresponds to the predicted value of the model. When the ratio on the diagonal of the confusion matrix is higher, it means that the classification of each type of data is better. It can be seen from figure 3 that FEM-RC has a relatively high classification accuracy among five diagnosis models.  As shown in figure 4, t-SNE is used to analyze the feature enhancement effect of FEM on the original data more intuitively. Each color pattern in the picture represents a fault state mode (there are six different faults in this experiment). It can be seen that the original samples corresponding to each type of fault in figure 4(a) are irregularly mixed together. However, in figure 4(b), the data of the same data type begin to gather. Among them, samples of C0, C1, C4, C5 are obviously separated, and there is a clear boundary between each category. In conclusion, FEM can optimize the feature data and enhance the feature representation of the essential information of the data.

Conclusions
In this paper, a joint feature enhancement and reservoir computing (FEM-RC) approach has been proposed for industrial robot diagnosis. FEM was proposed to enhance the class features of the original data by minimizing the intraclass distance and equalizing the interclass distance, and then used the reserve pool to calculate the mapping relationship between the original data and the enhanced matrix. Finally, RC was used for simple and efficient classification to solve the problems of data redundancy and slow running speed caused by high-dimensional data.
Experiments have been conducted from two perspectives to evaluate the performance of the proposed FEM-RC. Firstly, three peer machine learning approaches were selected to compare and verify the fault diagnosis performance of FEM-RC. Secondly, t-SNE was adopted to evaluate the clustering effect that FEM could enhance the feature representation. Thirdly, confusion matrix detailed diagnostic accuracy to further validate the robustness of the proposed method. In these comparison experiments, the proposed FEM-RC approach achieved better performance.