Feature Extraction Strategy with Improved Permutation Entropy and Its Application in Fault Diagnosis of Bearings

Feature extraction is one of the most difficult aspects of mechanical fault diagnosis, and it is directly related to the accuracy of bearing fault diagnosis. In this study, improved permutation entropy (IPE) is defined as the feature for bearing fault diagnosis. In this method, ensemble empirical mode decomposition (EEMD), a self-adaptive time-frequency analysis method, is used to process the vibration signals, and a set of intrinsic mode functions (IMFs) can thus be obtained. A feature extraction strategy based on statistical analysis is then presented for IPE, where the so-called optimal number of permutation entropy (PE) values used for an IPE is adaptively selected. -e obtained IPE-based samples are then input to a support vector machine (SVM) model. Subsequently, a trained SVM can be constructed as the classifier for bearing fault diagnosis. Finally, experimental vibration signals are applied to validate the effectiveness of the proposedmethod, and the results show that the proposedmethod can effectively and accurately diagnose bearing faults, such as inner race faults, outer race faults, and ball faults.


Introduction
In the process of intelligent manufacturing, the reliability of the equipment determines the final product quality and operational safety.Meanwhile, rolling element bearings are commonly employed in rotary machinery, such as motors, mine hoists, and turbines, and their fault diagnoses will affect the normal operation of rotating machinery.Once a bearing fails, it will lead to performance degradation of the entire machine that could potentially lead to disastrous accidents [1,2].It is well known that bearing faults are one of the most common sources of machine failures, and recent studies show that more than 50% of machinery failures are related to various bearing defects [3].erefore, it is necessary to design an effective fault diagnosis method for rolling element bearings.A complete rolling element bearing usually comprises an inner race, outer race, a rolling element, and a retainer, and its major fault sources are defects on the inner race, outer race, or one of the rolling elements.Once any of the above components has a defect, various types of sensor data can be collected for fault diagnosis, such as temperature, acoustics, and vibration [4].As vibration data are easily collected, and also contain abundant information regarding bearing faults, they are extensively used to diagnose mechanical faults [5][6][7].
Suitable feature extraction can improve the diagnosis results of bearing faults, and numerous scholars have proposed a large number of effective methods to extract features from vibration signals, such as Fourier-based methods, Wigner-Ville distribution (WVD), wavelet transform-based (WT) methods, empirical mode decomposition (EMD), and EEMD.In [8], the zoom fast Fourier transform (ZFFT) technique was employed for fault diagnosis of three-phase induction machines that elicited an improvement in frequency resolution.Antoni and Randall [9] used short-time Fourier transforms (STFT) to conduct fault diagnosis of rotating machines.In [10], a cyclic spectral density-based WVD was proposed to diagnose faults of rolling element bearings.Kankar et al. [11] presented a method for rolling element bearing fault diagnosis using continuous wavelet transforms (CWT) and SVM classifiers, achieved a higher accuracy of fault diagnosis by extracting useful features from the original data, and removed irrelevant features.Zarei and Poshtan [12] used wavelet packet transforms (WPT) to develop a fault diagnosis method, and accurately detected the faults of two holes on the outer race and one hole on other bearing parts by analyzing the stator currents.In [13], the redundant second-generation wavelet packet transform (RSGWPT) was introduced to improve kyrtograms for weak fault-feature extraction in rolling element bearings.
Unlike the above traditional signal processing methods, EMD, a self-adaptive time-frequency analysis algorithm, was presented in [14], and it has been extensively used in mechanical fault diagnosis.In [15], a new fault diagnosis method was proposed based on EMD and yielded estimates of the parameters of the alpha stable distribution (ASD) using vibration analysis for fault diagnosis of low-speed rolling bearings by filtering the trend and noise components of EMD.Jiang et al. [16] developed a new fault diagnosis method based on EMD and singular value decomposition (SVD) to diagnose faults in a roller-bearing system.Ricci and Pennacchi employed EMD and Hilbert-Huang spectrum analyses to diagnose faults in gears [17].A fault detection method for roller-bearing systems was constructed using a wavelet denoising scheme and proper orthogonal values (POV) of an intrinsic mode function (IMF) covariance matrix [18].Although the EMD is extensively used for mechanical fault diagnosis, traditional EMD could suffer from mode-mixing problems when is used to analyze complex signals.To overcome this issue, EEMD was proposed by Wu and Huang [19] and is frequently used for mechanical fault diagnoses.For example, an EEMD based on a multiscale, independent component analysis (ICA), multivariate-monitoring approach, was proposed for slewing bearing fault detection and diagnosis [20].A similar fault diagnosis based on EEMD and principal component analysis (PCA)/kernel principal component analysis (KPCA) was presented in [21,22].A new improved version of a multifault diagnosis method for axle bearings was introduced by using EEMD and Hilbert marginal spectrum analysis [23].In this method, an IMF confidence index was designed to realize the aim of adaptive self-selection of the useful IMFs.Apart from the above studies, some other EEMD-based fault diagnosis methods can be found in [24][25][26].
In addition to the above signal processing method, other indices should be employed to characterize the collected vibration signal to complete the bearing fault diagnosis.As we know, when a bearing is under a failure state, its vibration signal will change in complexity synchronously.As a result, the vibration signals of various types of bearing fault types will have different complexities.If the complexity of the vibration signal can be effectively represented, then the bearing fault can be accurately diagnosed.Permutation entropy (PE), which was proposed by Bandt and Pompe [27], can analyze the complexity of signals in time series by utilizing the comparison of neighboring values.Since PE is simple and invariant with respect to nonlinear monotonous transformations, it has been introduced to mechanical fault diagnosis, signal complexity analysis, and other application fields [28,29].In [30], the PE was investigated as a tool to predict the absence of seizures of genetic absence epilepsy rats from Strasbourg (GAERS) by using electroencephalographic (EEG) recordings.Meanwhile, PE has also been employed to improve fetal behavioral state classification by heart rate analysis from biomagnetic recordings in near-term fetuses [31].In [32], the algorithm of PE was used to detect dynamic changes using a well-known nonlinear logistic map, and a nonlinear statistical measure method was proposed for status characterization of rotary machines.Yi et al. [33] presented a PE-based tensor singular spectrum decomposition algorithm and successfully applied it in bearing fault diagnosis.Shi et al. [34] designed an improved local mean decomposition (LMD) based on the selfsimilarity of vibration signals, and then combined PE and an optimized K-means clustering algorithm to diagnose bearing faults.Zhang et al. [28] integrated PE, EEMD, and optimized SVM, for bearing fault diagnosis.However, noise interference may also cause the complexity of vibration signal changes.at is to say, the PE values extracted at various time periods may exhibit random fluctuations that will ultimately reduce fault diagnostic accuracy [16].
In this study, a new diagnostic parameter, referred to as improved permutation entropy (IPE), is proposed for bearing fault diagnosis with support vector machines (SVM).First, a set of IMFs can be decomposed from vibration signals based on EEMD.Second, a feature extraction strategy based on statistical analysis is then presented for IPEs.ird, the obtained IPE-based samples are put into an SVM model, and a trained SVM can then be constructed as the classifier for bearing fault diagnosis.Finally, experiments are carried out on simulators to collect real vibration data to validate the effectiveness of the proposed method.e rest of this study is structured as follows: Section 2 introduces the basic theories that include the EEMD algorithm and the definition of traditional permutation entropy.e proposed method and experimental research are described in Sections 3 and 4, respectively.
e conclusions are presented in Section 5.

EEMD.
e traditional EMD method is associated with a mode-mixing problem when employed to analyze complex vibration signals.To overcome this, EEMD, an improved version of the EMD technique, was proposed by Wu and Huang in 2009 [19].e fundamental steps of EEMD are presented below [19][20][21][22][23][24][25][26]: (1) Determine the decomposition number N, and add noise with an amplitude A, thus yielding a new signal where e n (t) is white noise.

Shock and Vibration
(2) Decompose the noise-added vibration signal standard EMD and then obtain, where c n,m (t) and r n,M (t) are the mth IMF and residual signals of the nth EMD decomposition, respectively.(3) Repeat steps (1) to (3) N times, and obtain the final IMFs in accordance to (4) e original signal can then be decomposed as,  [28].erefore, it has been employed in numerous fields for evaluating the dynamic characteristics of various signals.e mathematical theory of PE is described briefly below [27][28][29][30][31][32][33][34].
For a time series x(n), n � 1, 2, 3, . . ., N { }, its m-dimensional embedding vector with time delay τ s can be written as All the elements in the embedding vector x m i can then be ranked from the smaller to the larger as where k i ≠ k j and the rank discussed above is the permutation of the embedding vector x m i , marked as π k 1 ,k 2 ,...,k m .If two or more elements of the embedding vector x m i are equal, we order them by their corresponding index value k i .When there is an equality case, e.g., x(i Herein, we can obtain the relative frequency of the permutation π k 1 ,k 2 ,...,k m using Subsequently, the PE with dimension m is defined as e maximum value of H PE (m) is ln(m!) when all possible permutations have the same probability.erefore, the normalized permutation entropy (NPE) can be obtained as According to the definition of PE and its calculation process, the selection of the time lag τ and the embedding dimension m influences the final result.
erefore, the method that is used to choose suitable parameters is one important step for PE.
e choice of excessively large τ values will make the correlation of all points to be too small and cannot reflect the dynamic characteristics of the measured object.When the embedding dimension m is set to take values that are too small, the reconstructed vector contains little information, and it may be difficult for the elicited PE to reveal the real features of a complex signal.Large m values will not only be time-consuming, but also difficult to reflect the subtle changes in the time series.In [27], Bandt and Pompe suggested the selection of an embedded dimension m � 3 − 7, whereby the value of the time lag was equal to unity.In this study, we mainly planned to use statistical averaging analysis to reduce the impact of noise on traditional PE features when PE was applied as a distinguishing feature for mechanical fault diagnosis.erefore, we set m � 3 and τ � 1.

Bearing Fault Diagnosis Based on IPE and SVM
In this study, a new diagnostic parameter, called improved permutation entropy (IPE), is proposed for bearing fault diagnosis.e proposed method includes three steps: vibration signals decomposed by EEMD, IPE feature extraction, and SVM-based fault diagnosis.Its implementation process is shown in Figure 1.

PE Calculation Based on EEMD.
In this study, EEMD is used to process the vibration signals to obtain IMFs for PE values and its two critical parameters need to be established: the ensemble number and the amplitude of the added noise.To make the EEMD more effective, Wu and Huang [19] suggested that the amplitude of the added noise is 0.2 times the standard deviation of the signal, and the value of the ensemble number is a few hundred units.Note that after the added noise amplitude is determined, the larger the ensemble number is, the smaller the decomposition error will be [19].However, increasing the ensemble number above 100 will marginally improve the error, with a large increase in the required computational time [35].erefore, in our study, the ensemble number and the amplitude of the added Shock and Vibration white noise are set to 100 and to 0.2 times the standard deviation of the signal, respectively.
In this study, only the vibration signals of one sensor are considered.When one data segment of the vibration signal is processed by EEMD, a set of IMF decomposition components can be obtained, and each one can be used to calculate a PE value.erefore, if there are type I data segments, we can obtain where F pe,i � [f pe,i,1 , f pe,i,2 , . . ., f pe,i,j , . . ., f pe,i,J ], which is defined as the PE set of the ith data segment, f pe,i,j is the PE value calculated from the jth IMF component of the ith data segment.

Definition and Extraction of IPE. Different types of bearing faults have vibration signals of differing complexity.
e traditional PE, proposed by Bandt and Pompe [27], can quickly show the complexities of vibration signals in the time domain by utilizing the comparison of neighboring values.However, noise interference causes the features extracted from the vibration signals over different time periods to display random fluctuations in some degree, which directly affects the accuracy of the bearing fault diagnosis.In other words, directly employing PE as a feature for fault diagnosis with an SVM-based classifier may result in a lower diagnostic accuracy.erefore, it is needed to develop a method to extract suitable features with a smaller wave range in the feature level.
e law of large numbers is a particularly important theory in mathematical statistics, and it shows that the mean of a random event is equal to the average value in experiments with a large number of measurements [36].In probability statistics, the central limit theorem (CLT) shows that for a large number of repeated measurements of a physical quantity, their arithmetic average value is distributed in accordance with the standard normal distribution [37].
erefore, in theory, using the mean forms of original features to construct a classifier may reduce the fluctuating range of the values of these features.
Although the averaged PE values can be used to reduce the fluctuating range of features to improve fault diagnosis effects, the optimal number of original PE values to obtain an accurate average PE remains to be solved.In this section, a solution is designed for this issue and its main steps are shown below: (1) Set i � 1 and the termination condition, such as the total number of iterations 4 Shock and Vibration for the averaged PE, and the calculated result is the IPE.Otherwise, i � i + 1, and the above steps are repeated until the termination condition is satisfied e above two stop conditions may also be set at the same time.If either is met, then the program terminates.Meanwhile, the search for the optimal m value can be carried out offline so that the termination conditions can be set with large values.As long as the optimal m is found, the IPE value of the j-th IMF of IVMD can then be calculated using f ipe,j � (1/m) k�m k�1 f pe,k,j , where the m PE values are randomly selected from the original feature set obtained using Equation (10).

SVM-Based Fault Diagnosis.
After extracting the IPE values, the next step is to construct a classifier.In this section, an SVM-based fault diagnosis method is introduced.First, considering the common faults in bearings, four bearing health conditions are considered, including the normal (S1), inner race fault (S2), ball fault (S3), and outer race fault (S4), and their goal outputs are set as "1," "2," "3," and "4."Other parameters of the SVM are set as the default values in [38]   As shown in this figure, this bench is consisted of a three-phase induction motor, a torque transducer, an encoder, shaft, a dynamometer, and rolling element bearings.e motor is used to drive the shaft on which a torque transducer and an encoder are installed.For conducted tests, single-point damages of the drive-end bearing (SKF deepgroove ball bearings: 6205-2RS JEM) were seeded on the rolling elements, inner race, and outer race, using an electrodischarge machining method.erefore, the considered operating states of the bearing are normal operation and three types of faults (0.014 inches), including the inner race fault, ball fault, and the outer race fault.e sampling frequency used was 12 kHz, and the corresponding vibration signals were collected by accelerometers and were attached to the housing with magnetic bases using a 16-channel data recorder when the bearings were rotated at speeds of 1797 rpm or less.Each feature was calculated from 2000 sampling points.Other information regarding the experimental instruments can be found in [39].Figure 3 shows the waves of the vibration signals based on the consideration of the four bearing conditions in the time and frequency domains.

Experimental Evaluation
Figure 4 shows the IMFs of these original vibration signals plotted in Figure 3.In this figure , (a), (b), (c), and (d), are plots of the IMFs of the bearing under normal, inner race, ball, and outer race faults, respectively.From these figures, it is difficult to diagnose the bearing conditions by directly using these plots without professional knowledge, even if some differences exist in the amplitudes and distributions of the IMFs at various health conditions.Conversely, there is noise in the collected vibration signals, and the IMF signals of the same bearing statue may also be different.erefore, these so-called differences may also be changed, or appear at different times.At the same time, each IMF contains many data points, and use of all the decomposed components to conduct fault diagnosis will increase the computational load of the model.For different bearing faults, the detail compositions of their vibration signals may also have some differences.at is to say, these decomposed components of EEMD have characteristics that can reflect the fault types of the rolling element bearing.
Figure 5 shows the comparative results of various features.In Figure 5(a), the original features are shown, while each of the features plotted in (b)-(d) correspond to the mean of three, five, and ten original PEs, respectively.To ensure the randomness of the original features, the data segments for the original features are randomly extracted from the vibration signals.In Figure 5(a), the amplitude values of these original features range from 0.985 to 0.995, and the major features appear to be around 0.989, i.e., these original features fluctuate within a certain range.Similar to the original features, the averaged features in Figure 5(b) fluctuate in the range of 0.988 to 0.991, and features also appear mainly around 0.989.Analyzing Figures 5(c) and 5(d) shows that these averaged features also appear around 0.989.Based on the comparisons and analyses of these feature values and their standard deviations plotted in Figure 5, it can be observed that the fluctuation range of Figure 5(d) is the smallest, and the ranking of the fluctuation ranges in accordance to the order of (d)<(c)<(b)<(a).Since each averaged PE value is the arithmetic mean value of a set number of original PEs, it will be closer to the real expectation of these features compared with that for most of the original features.erefore, as the number As it is known, these fluctuations of the same feature values will affect the accuracy of the bearing fault diagnosis, and it is difficult to use these features with large fluctuations to construct a stability classifier with increased accuracy of fault diagnosis.To overcome this issue, a feature extraction strategy based on statistical analysis is presented in subsection 3.2.at is, the averaged PEs will have less noise interference and can improve fault diagnosis.To ensure the randomness of original PE features, the corresponding data segments are extracted randomly from these vibration signals collected by accelerometers.Correspondingly, a feature dataset can be obtained for IPE.

Results and Discussion.
In the optimal number of the selection stage of the original PEs, various IPE feature sets calculated by different numbers of original PE values are used to obtain training and pretest sample sets.In this study, the number of samples in the training sample set and pretest sample set were 20 and 60, respectively.For the sake of the descriptions outlined herein, the ith set is defined as the averaged PE value estimated as the mean of ith original PE values and the diagnostic accuracy of the ith pretest sample set is marked as ρ i .In addition, the termination condition of iterations I, and the threshold value of diagnostic accuracy T, were preset to 25 and 100%, respectively.According to the principle defined in subsection 3.2, the termination condition is defined as either the iteration number I when the latter is larger than 25, or when the diagnostic accuracy values of the ith, (i − 1)th and (i − 2)th samples are all equal to 100%.
Figure 6 shows the diagnostic accuracy values of the pretest sample sets in the selection stage of the optimal number.In this figure, the x-axis is the serial number of the pretest sample set, which indicates the number of used original PE values for a specific IPE value.For example, "5" implies that each averaged PE value in the pretest sample is calculated using five original PE values.at is to say, when the serial number is "1," the feature is the original PE value.As observed in this figure, the diagnostic accuracy of the original PE values is clearly less than those of the averaged PE values.is shows that the proposed averaged PE values can improve the bearing fault diagnosis results without optimizing the classifier.Further analyses of these diagnostic results of the averaged PE values reveal that the diagnostic precision increases gradually as the serial number increases and that the diagnostic accuracy values, shown in green in Figure 6, become 100% when the serial numbers are equal to four and five.However, the diagnostic precision then decreases when the serial number becomes equal to six.In simpler term, four, five, and six are not the optimal numbers of the original PE values for IPE.Nevertheless, it can be found that the diagnostic accuracy values are 100% when the serial numbers become equal to seven, eight, and nine.According to the algorithm defined in subsection 3.2, the serial number seven is the optimal number for the original PE values for the averaged PE.In other words, the IPE in experiment 1 should be calculated using seven original PE values.
Based on the simulator shown in Figure 2, 100 IPE-based samples are extracted for each bearing condition, of which 20 samples are used to train the SVM model, and the remaining 80 ones are used to test the trained SVM model.In accordance with the principle of IPE, there are 700  As it can be observed in Figure 7(a), in the case of the test samples with inner race and outer race faults, the test outputs do not all match the corresponding goal outputs, i.e., the diagnostic accuracies of the comparison method "PE-based sample set (1) and SVM" are not 100%.e same  Shock and Vibration issue also occurs in Figure 7(b), and the test outputs of these test samples with sample number 561 to 1120 are not equal to the goal outputs.Additionally, some test samples with outer race faults also elicit erroneous test outputs.erefore, these two comparative methods all contain some misdiagnosed samples.When analyzing Figure 7(c), it can be observed that the outputs are all consistent with the goal outputs, that is, the diagnostic accuracy of the proposed method is 100%.
Table 1 shows the fault diagnosis results of the proposed and comparison methods, where "4" indicates the types of the bearing conditions considered in this study.From the test results in Table 1, the proposed method has the highest diagnostic accuracy, and a shorter test time.is proves that the proposed IPE can be used for bearing fault diagnosis with increased accuracy.

Experiment 2: Fault Diagnosis with the Bearing Datasets of Mine Hoist
Simulator.Mine hoist is an especially important equipment in vertical shaft and undertakes the transport task of personnel, materials, equipment, and coal, between underground and ground.Given the execution of frequent start-stop operations, high speeds, and heavy loads, the mine hoist will cause a fatal crash once the mine hoist experiences a bearing fault, such as an inner race, a ball, or outer race faults.Correspondingly, this will affect the safety of the mine and the efficiency of the production.erefore, it is significant to diagnose the bearing faults that occur in the working mine hoist.
e above experiment is conducted in an indoor laboratory, and its noise may not be as loud as that of an actual operating equipment.To further illustrate the effectiveness of the proposed method, an outdoor experiment was conducted, and the test bench was a simulated mine hoist, as shown in Figure 8. is bench mainly consists of derrick and driving systems.e derrick system is powered by a driving system.us, the driving system is important for a mine hoist.e driving system induces a motor, a gear box, two couplings, and two bearings (#1 and #2).As shown in Figure 9, in this experiment, the considered conditions of test bearing #2 are: normal (S1), inner race fault (S2), ball fault (S3), and outer race fault (S4).e sampling frequency used was 2 kHz, and the corresponding vibration signals were collected by an accelerometer, attached to the bearing   Shock and Vibration housing with magnetic base, when the shaft was rotating at a speed of 180 rpm.
Figure 10 shows the selection results of the optimal number of PEs for the IPE in experiment 2. It can be observed that the diagnostic accuracy values are 100% when the serial numbers become equal to 12, 13, and 14, as shown in these red dots plotted in Figure 10.According to the algorithm defined in Subsection 3.2, the optimal number of original PE values for IPE equals 12.In other words, the socalled IPE value of this experiment is calculated using 12 original PE values.From the comparison and analyses of Figures 6 and 10, it can be found that the IPE features of Figure 10 require an additional number of original PE values to realize the same diagnostic accuracy compared with that required by Figure 6.
erefore, more intense noise may need more original PE values to calculate one IPE feature.2. In conjunction with the diagnostic results shown in Table 2, the proposed method has the highest diagnostic accuracy, and is associated with a shorter test time.is proves that the proposed method is effective in diagnosing bearing faults.

Conclusions
Vibration signal analysis is an effective way of diagnosing rolling element bearing faults.However, noise interference causes the same features that are extracted from these collected vibration signals over different time periods to vary, which directly affects the accuracy of bearing fault diagnosis.To solve this problem, a new diagnostic parameter was designed for bearing fault diagnosis using SVM, referred to as the IPE.First, a set of IMFs was decomposed from vibration signals using EEMD.Second, a feature extraction strategy based on statistical analysis was designed for IPEs.
ird, the obtained IPE-based samples were input to an SVM model, and a trained SVM was then constructed as the classifier for bearing fault diagnosis.Finally, an experiment with real vibration signals was executed to validate the effectiveness of the proposed method.e experimental results showed that the proposed method can effectively and accurately diagnose bearing faults.However, the opposing fault diagnoses methods with traditional PE features and SVM not only have higher misdiagnosis rates but also consume more training and testing time than the proposed method.
erefore, this study provided a new diagnostic parameter for bearing fault diagnosis.
and part of them are (a) the kernel function is polynomial, (b) the punishment factor C is equal to 10, (c) the degree in kernel function is set to three, and (d) the type of SVM is C-SVC.e obtained IPEs are then input to an SVM model, and a trained SVM model can be constructed as the classifier for bearing fault diagnosis.Finally, experiments are carried out on simulators to collect real vibration signals to validate the effectiveness of the proposed method.

Figure 3 :
Figure 3: Waves of vibration signals in the (a) time and (b) frequency domains.

Figure 5 :
Figure 5: Comparative results of various features: (a) original features, (b)-(d) average features which are the means of three, five, and ten original features, respectively.e red solid lines are the mean values of these features plotted in the corresponding plots; std is the abbreviation of standard deviation.

Figure 6 :
Figure 6: Diagnosis accuracy values of pretest sample sets in experiment 1.

Figure 7 :
Figure 7: Fault diagnosis results of the proposed method and comparative methods: (a) PE-based sample set (1) with the use of an SVM.(b) PE-based sample set (2) with the use of an SVM, and (c) the proposed method.

10
Shock and VibrationBased on this simulation platform of mine hoist shown in Figure8, a total of 60 IPE-based samples are extracted for each bearing condition, of which 20 are used to train the SVM model.An SVM-based classifier is then constructed, and the rest of the samples are used to test the trained SVM model.e two comparative experiments shown in Figure7are also considered in this experiment.Since the optimal number of original PE values for IPE equals 12, each IPE value requires 12 PE values.erefore, each IPE-based sample can produce 12 PE-based samples.e fault diagnosis results of the proposed method and comparison methods are presented in Table

Table 1 :
Diagnosis results of comparison experiments.

Table 2 :
Diagnosis results of comparison experiments.