Attention Recurrent Neural Network-Based Severity Estimation Method for Early-Stage Fault Diagnosis in Robot Harness Cable

Cable is crucial to the control and instrumentation of machines and facilities. Therefore, early diagnosis of cable faults is the most effective approach to prevent system downtime and maximize productivity. We focused on a “soft fault state”, which is a transient state that eventually becomes a permanent fault —open-circuit and short-circuit. However, the issue of soft fault diagnosis has not been considered enough in previous research, which could not provide crucial information, such as fault severity, to support maintenance. In this study, we focused on solving soft fault problem by estimating fault severity to diagnose early-stage faults. The proposed diagnosis method comprised a novelty detection and severity estimation network. The novelty detection part is specially designed to deal with varying operating conditions of industrial applications. First, an autoencoder calculates anomaly scores to detect faults using three-phase currents. If a fault is detected, a fault severity estimation network, wherein long short-term memory and attention mechanisms are integrated, estimates the fault severity based on the time-dependent information of the input. Accordingly, no additional equipment, such as voltage sensors and signal generators, is required. The conducted experiments demonstrated that the proposed method successfully distinguishes seven different soft fault degrees.


Introduction
Cable is widely utilized because it is a crucial part of control and instrumentation in industrial fields, including power plants [1,2], vehicles [3], ship building [4], and factory automation [5]. To maintain the stability and reliability of the system operations, timely diagnosis of cable fault is vital. In particular, the early diagnosis of faults can ensure the maximization of productivity by reducing unwanted system downtime in fully automated processes. A single fault in an automated process can stop the related processes because the process controllers are interlocked with each other to prevent harmful effects to other facilities and dangerous hazards to workers when a fault is detected.
There are two states of cable faults: transient and permanent. The latter represents a permanent fault that results in the failure of machines and facilities without exception. This fault consists of open and short circuits and is referred to as a hard fault. The former is a transient state, which develops according to the external conditions, thereby resulting in the accumulation of small modifications in the cable. Because of harsh operating environments, the components of the cable, which include insulators, shields, and conductors, may experience partial damage. Therefore, this is referred to as a soft fault [6]. In reality, a soft fault is not an acute fault state and does not cause significant problems to machines and facilities at that moment. However, there is a high probability of it transforming into a hard fault, which deteriorates the operation of machines and facilities, i.e., causes unwanted system outage. In practical cases, the accumulation of damage to a limit close to a hard fault can yield intermittent symptoms or show no symptoms. The case of no symptoms is problematic because the latent faults cannot be detected through system diagnosis or identified by the maintenance staff. Thus, subsequent partial damage can accumulate in the cable if there is no maintenance. Consequently, soft faults often become hard faults. Therefore, early and accurate diagnosis of soft faults before they become uncontrollable is the most effective approach to deal with practical cable problems in the field.
Traditional cable diagnosis methods are largely categorized into electrical methods (partial discharge), chemical methods (oxidation induction time), and mechanical methods (elongation at break). Despite its wide usage in practice, these methods have limited applicability in on-site field applications when the machine is working because the online diagnostic ability is lacking or too sensitive to operating noise [1]. Moreover, access to the installed cable is largely limited in automated factories; thus, applying traditional techniques is not preferred in field conditions.
Recently, machine learning-based methods, including convolutional neural networks (CNN) and recurrent neural networks (RNN), have been studied in relation to fault diagnosis. These methods can automatically extract features from the input signal, which overcomes the problem of manual feature extraction requiring domain knowledge and being time consuming, particularly when a large amount of data are to be processed. Machine learning-based diagnosis methods can be categorized into two types: those that utilize the reference signal, which is to be injected into cables, and those that utilize the output signals from operating cables, such as the current signals flowing through the cable. Many studies have reported remarkable results in diagnosing faults using a reference signal. A regression-based multilayer perceptron method with clustering algorithms was developed to detect and classify hard faults of a multicore cable by analyzing the reflected signal of the reference signal in [7]. In [8], the reflected signals were converted into combined time-frequency images using Wigner-Ville distribution. Subsequently, the images were classified using a convolutional neural network (CNN). A 1D-CNN based diagnosis method was applied to diagnose soft faults in control cables of electric motors [9]. Although methods using reference signals have shown remarkable results, the reflected signal compensation and environment noise cancelation must be conducted prior to the diagnosis of faults. Additionally, there are physical limitations related to the irregular responsibility of propagating reference signals when they encounter fault locations [6]. Furthermore, additional equipment must be prepared to conduct the diagnosis owing to the requirement of a signal-injecting device and a signal generator. Several studies have used signals from operating cables to diagnose faults. In [10], a fast fault-detection method for a power transmission line was developed using the summation of the squared threephase currents. In [11], a novelty detection-based diagnosis method was introduced using an adversarial autoencoder and an anomaly scoring technique with three-phase current signals. The method is based on the concept that the data samples with high anomaly scores can be considered as faults [12][13][14][15][16][17][18]. The utilization of current signals for diagnosing faults is relatively reliable under environmental noise while also being cost effective, as the need to install additional equipment is eliminated. Although these methods have exhibited high accuracy in diagnosing cables, studies that focus on diagnosing the early stage of a fault-that is, a soft fault-are scarce. Moreover, fault severity estimation has not been studied in this regard. Such estimation results can provide informative guidelines when considering cable maintenance strategies in the field.
In the present study, we propose a soft fault diagnosis method that detects the presence of soft faults and estimates the fault severity in a cable using an autoencoder and an attention-based RNN. To the best of our knowledge, this is the first study to attempt to estimate soft fault severity in practical cable application using only current signals without any special equipment. First, the autoencoder network detects the novelty and calculates the anomaly score of the time-series input from the current signals. Subsequently, we construct an estimation algorithm based on a long short-term memory (LSTM) network to deal with time series data [19]. The estimation network comprises an encoder and a decoder. The decoder focuses on certain hidden states that significantly influence the output by adopting the attention mechanism (AM) [20]. The two networks for novelty detection and fault severity estimation are connected serially. The novelty detection network receives the zerosequence current (ZSC) and the squared current ratio (SCR) as inputs, and subsequently outputs the anomaly score that directly represents the novelty of the fault, whereas the fault severity estimation network receives ZSC, SCR, and the anomaly score as inputs and generates a fault indicator, which directly indicates the fault severity. Several experiments are conducted to verify the reliability of the proposed method, even at various operating points and fault severities. The major advantages of this study over previous methods are as follows.

1.
The method could diagnose an early-stage fault in a cable by estimating the soft fault severity before the fault become permanent state; that is, a hard fault.

2.
In contrast with reference signal-based methods, in this method, the need to design a reference signal and consider physical cable parameters is eliminated because only three-phase currents are required to conduct diagnosis.

3.
The method is reliable under various operating conditions that may result from irregular machine operation as well as various fault conditions that range from mild to severe.
The remainder of this paper is organized as follows. In Section 2, a soft fault and its effect on cables is described. In Section 3, the novelty detection and severity estimation method is proposed. In Section 4, the proposed method is demonstrated based on various experimental scenarios. Finally, Section 5 summarizes the study.

Soft Fault in Cable
Soft faults occur because of sequential stress from harsh environments. Local modifications can be made to parts of cables, such as conductors, insulators, and shields [6]. Among them, modifications or damage to the conductor can directly affect the health state of the cable. A damaged cable is presented in Figure 1. The cross-sectional area of the conductor is reduced because of the damage, as shown in the figure. Here, S o denotes the cross-sectional area of a healthy conductor, and S f denotes the cross-sectional area of a damaged conductor. To analyze the soft faults in the cable, an equivalent circuit model is presented in Figure 2. It is assumed that a soft fault occurs in phase a wire. As shown in figure, the soft fault is represented as an added resistance to the phase a wire. The resistance R f denotes the induced damage in the conductor because of the fact that R f and S f are inversely correlated. Further, R f , S f , and their relation to the normal condition can be expressed as follows: where S o is the normal cross-sectional area of the healthy conductor. R o denotes the resistance of the healthy conductor and ρ is the resistivity of the conducting material. The length of the conductor is denoted by l. α indicates the ratio of S f to S o . The reduced area S f can result in a phase imbalance in the current magnitude and negative effects, including heating and energy decrease in the system [21]. Further, an imbalance in the phase impedance could result in asymmetry of the three-phase system. For healthy and soft faulty cables, the value of α is one and less than one, respectively. Mainly soft faults are discussed in this study; therefore, hard fault cases including α = 0 (open-circuit) and α > 1 (short-circuit) were not addressed in the proposed method. According to the relation in (4), α directly provides information on the state of the conductor. Therefore, the fault indicator (FI) can be defined as In Figure 2, Z c is the characteristic impedance of normal wire. C i and R i in parallel denote the insulator, because the insulation layer can be represented by its capacitive and conductive characteristics [22]. i res is the leakage current, which is typically smaller than i cap in healthy wires. In case of the presence of faults in a wire, R f increases and R i decreases because the conductor and insulator are damaged. Consequently, i res increases due to the changed values of R f and R i according to the fault. In this case, the vector sum of the three-phase currents (i a , i b , i c -three sine waves separated by 120 degrees) will have a nonzero value owing to the leakage current. Therefore, the sum-that is, the ZSC-can be used as an input for the proposed network: where i ZS is the ZSC of the input currents. The amplitudes of each current are represented as I ma , I mb , and I mc . Its corresponding phase angles are represented as φ a , φ b , and φ c . The magnitude of i ZS is approximately zero or stable for healthy wires, whereas it fluctuates significantly when there is asymmetry in the phase wires. In case of a severe fault, the asymmetry worsens. Moreover, the magnitudes of the current of each phase differ from those of the normal condition because of the asymmetry in the three-phase system. Con-sequently, the ratio of each phase current to the total current immediately indicates an asymmetry in the system, as follows: where j denotes the phase a, b, and c. Each phase's SCR is defined as each squared phase current divided by the sum of squared phase currents (SSC) from all the phases. The SCR can be used for a comparison of the phase current with the total current. Therefore, it reflects a faulty three-phase network with R f and can be used as an input for the proposed network. The utilization of current signals originating from the operating machine with irregular motion is challenging because the amplitude, frequency, and phase sequence vary according to motion planning, as shown in Figure 3. For instance, an industrial robot installed in the production process can move its arm to perform the given motion commands, which allows the robot joint to rotate in the forward direction after rotating in the reverse direction. Even in this case, SCR and ZSC are not affected by the varying operating conditions because no phase rotation operator is involved, as shown in Figure 4. In the figure, SCRs and ZSC are stable when cable is healthy with the robot moving as mentioned above. Fluctuations appeared when varying degrees of fault were induced to the robot control cable. Still, we could not estimate the fault severity to diagnose a soft fault. The proposed diagnosis system in this paper will calculate an anomaly score of each SCR and ZSC and estimate fault severity to provide a guide for a proper maintenance plan. The analysis thus far assures that soft faults can be directly diagnosed by utilizing the ZSC and SCR.

Soft Fault Diagnosis System Structure
RNNs can effectively utilize the time series data because they have the ability to consider the time-dependent information of the time series input. In this section, we propose an attention-based RNN with an autoencoder network. The structure of the proposed network comprises two parts: anomaly detection and severity estimation. In the anomaly detection network, the encoder reduces the dimensions of the input data to generate latent code, and the decoder reconstructs the input as an output to compare the input and output, whereas, in the severity estimation network, an LSTM-based RNN structure was adopted to encode the input into the feature vector and decode the estimated FI as an output. An AM was integrated into the decoder to automatically select the related hidden states through all time steps. Figure 5 presents the graphical presentation of the proposed algorithm.

Anomaly Scoring with Autoencoder
An autoencoder can be constructed using an encoder and a decoder. The input dimension is reduced by the encoder, whereas the decoder reconstructs the input by using the learned latent code. Therefore, A = f • g can be considered for an autoencoder A, where f denotes the decoder and g denotes the encoder. Let n be the number of hidden layers in f and g where g = g n • · · · • g 2 g 1 and f = f 1 f 2 • · · · • f n . The partial computations of f and g for 1 ≤ i ≤ n can be g :i = g i • · · · • g 1 (12) The conventional anomaly scoring method compares the input and reconstructed data to detect a novel sample. Recently, it has been shown that comparing hidden spaces can generate more accurate anomaly scores [23,24]. This approach also compares the activations in the hidden space with the corresponding reconstructions in that hidden space to calculate the anomaly score. However, a hidden reconstruction cannot be directly computed. Instead, the hidden activations of the reconstructed input g :i (A(x)) can be used because it has been proven that hidden activations of the reconstructed input g :i (A(x)) are equivalent to the corresponding hidden reconstructions of the original input ( f n:i+1 • g)(x) [23]. This implies that hidden reconstruction values can be obtained by simply projecting the reconstructed input A(x) into the hidden spaces. If the input x and its reconstructed input A(x) are provided, their hidden activations are computed as To obtain the anomaly score, the normalized aggregation along the pathway (s N AP ) method was adopted in this study as follows: . . . ;ĥ n (x)]; h 0 (x) = x andĥ 0 (x) =x. X denotes the given training set, D denotes a matrix wherein the ith row matches d(x i ) for x i ∈ X, andD = UΣV T must be computed for normalization, whereD denotes the column-wise centered matrix of D. µ x denotes the column-wise mean of D. Consequently, this scoring method allows the evaluation of anomalies in the current signal when a soft fault exists.

Fault Severity Estimation with Attention Mechanism
The fault severity estimation part comprises encoder and decoder LSTMs with an AM. The encoder generates a hidden state h t that summarizes the information up to the current time step t. This can be represented as follows: where h t−1 is the output of the previous LSTM unit, x t is the current input, and f denotes the activation function, which is the LSTM [25]. LSTM can handle the time features of timeseries data. Further, LSTM has memory cells that store, retrieve, and maintain information. It comprises the input gate i t , output gate o t , and forgot gate f t . The abandoned information is controlled by the forgot gate f t as follows: where σ is a sigmoid function and W f and b f are the weight and bias parameters, respectively. The updating information is controlled by the input gate i t , which can be represented by where W i is the weight parameter of the input gate and b i is the offset term. Further, the long-term state is denoted by s t and o t is the output gate. These can be expressed as follows: where W o is the weight parameter of the output gate and b o is the offset term. Further, W s is the weight parameter of the state and b s is the bias parameter. The output h t of the LSTM cell can be represented as where denotes the Hadamard product. Hidden states h t are created across all time steps to summarize time-dependent information. The encoded input information was decoded by the decoder network to generate the estimated outputŷ T . LSTM is integrated into the decoder to estimate the output from time dependencies. Moreover, an AM is also integrated into the network to automatically select the important encoded hidden states through time steps. AM strengthens the influence of important and relevant information by focusing on output variables with a significant impact and ignoring uninterested ones. The attention weight can be computed as follows: where v d and W d are the weighting coefficients and U d denotes the bias coefficient. The attention weights α k t , which are obtained by normalizing the weights e k t using the softmax function, represent the importance of each hidden state from the network. The context vector c t can be denoted by This is provided to the decoder as input, and the hidden state of the decoder can be written as d t denotes the decoder hidden state at time step t, and f is a nonlinear activation function, which is the LSTM. The hidden state d t can be updated as explained previously in the encoder. Thus, the fault severity outputŷ T can be written aŝ where F denotes the approximate function for estimating fault severity from the given input vectors X = (x 1 , x 2 , · · · , x T ), and W y and b y are the weight matrix and bias terms, respectively. Consequently, the fault severity can be estimated by a linear function, where v T y and b v are the weight and bias, respectively.

Training Procedure
A novelty detection and fault severity estimation network was constructed using the Tensoflow framework [26]. The Adam optimizer [27] was used to train the model, and the sizes of the mini-batches were 128 and 20 for the novelty detection and fault severity estimation networks, respectively. Further, the sizes of the hidden states were 30 and 128 for the novelty detection and fault severity estimation networks, respectively. The time length T of the time-series input was 12. In addition, the l 2 loss function was used for the standard back propagation, and the objective functions for the two networks are as follows:

Structure of Diagnosis System
The structure of the diagnostic system is shown in Figure 6. It comprises signal processing, anomaly assessing, and fault severity estimation components. The data sample from the time-series input comprises SCR, ZSC, and s N AP , where the size of the input sample was a window of 9000 time steps. When computing SCR, ZSC, and s N AP using (11), (9), and (16), respectively, the raw input data were passed through a low-pass filter to remove unnecessary frequency components. The calculated fault indicator was used as the label of the network. Figure 6. The industrial robot system with the proposed diagnosis method. It comprises signal processing, anomaly detection, and fault severity estimation components. If the detected fault severity exceeds the threshold limit, a maintenance call is generated for the staff.

Experimental Setup
The experimental setup is illustrated in Figure 7. A HYUNDAI HH7 robot manipulator and its controller were installed to demonstrate the proposed method. The cable harness comprised control and signal cables, with the control cable targeted for conducting the fault diagnosis. The detailed characteristics of the target cable are represented in Table 1. The cable consisted of 32 wires, each of which consisted of an insulator and conductor. Further, each conductor was composed of 30 flexible strands, each measuring 0.25 mm in diameter. Further, a Tektronix DPO 4104 B phosphor oscilloscope was used to measure current signals at a sampling rate of 100 kHz. We simulated various faults and operating conditions in the third joint of the robot manipulator.

Experimental Scenarios
To simulate the fault conditions, soft faults were created manually in the wire of phase a, as shown in Figure 1. Artificial faults exhibit a range of fault severities, from mild to severe. The fault severity and corresponding scenarios are listed in Table 2. The datasets according to each fault severity were acquired as follows. First, a normal cable was damaged repeatedly according to the damage cases. There are eight cases according to the fault severity in the table. For example, 28 cutting is the most severe fault case, whose α value is small, and 93.4% of S o is damaged. However, the four cuts are mild faults, whose α value is large, and 13.4% of S o is damaged. Moreover, the experiments were conducted under actual operating conditions wherein the manipulator moved up and down according to the motion commands, as shown in Figure 7. In the figure, the robot joint rotated inversely following forward rotation. Consequently, the speed, rotational direction, and amplitude of the three-phase currents varied according to the robot movement, as shown in Figure 3. As mentioned previously, eight scenarios and acquired datasets were prepared to conduct experiments under these varying conditions.

Result and Analysis
The test results for the scenarios demonstrate that the proposed method can effectively diagnose soft faults. First, the autoencoder successfully detected the abnormal state and calculated anomaly scores, as shown in Figure 8. The results confirmed the suitability of the time-series input-that is, ZSC and SCR-for soft fault diagnosis by reflecting the power network imbalance. Second, the fault severities of the induced damages were correctly estimated by the attention-based RNN architecture, as shown in Figure 9. The result confirms that the RNN inputs-that is, ZSC, SCR, and s N AP -are appropriate for estimating the fault severity of a particular fault. As shown in Figure 9 (S f = 100.0%), the FI for healthy cable was approximately zero for varying operating conditions, such as speed, amplitude, and phase. The FI increased as the severity increased according to Scenario 7, as shown in Figure 9 (S f = 6.6%). Further, the faulty cable and healthy cable were clearly distinguished by the FI. Thus, it can be concluded that the FI is independent of the varying operating conditions resulting from the robot manipulator movements. Therefore, the proposed method is applicable to practical online diagnosis when automated machines operate in smart factories. Figure 9 (S f = 86.6%) shows the test result for Scenario 1. The fault condition was the least severe compared to the other scenarios, implying that diagnosing Scenario 1 was more difficult compared to other scenarios. Four strands of the 30 strands were damaged artificially to simulate very early soft fault conditions, where only 13.4% of the cross-sectional area of the conductor was damaged. In this case (early case even among the soft fault cases), the automated machines can work without any fault symptoms. Compared to the magnitude observed in Figure 9 (S f = 6.6%), the magnitude of the corresponding FI in Figure 9 (S f = 86.6%) was much smaller. Nevertheless, the early stages of soft faults can be diagnosed using the estimated FI, even under mild fault conditions. The FI estimation results for all scenarios are shown in Figure 9 (S f = 6.6%∼100.0%). The estimated FI tended to be more accurate for severe fault cases (Scenarios 4 to 7) because the magnitude of the simulated FI was much smaller in less severe scenarios (Scenario 1 to 3). Table 3 confirms this trend by showing mean square errors (MSE) of each scenario. Consequently, we can assert that the proposed method achieves a soft diagnosis capability under various fault and operating conditions, rendering this method applicable to practical problems.

Comparison with Other Studies
The proposed method was compared to other methods by considering several aspects, as listed in Table 4. Reflectometry-based methods [28][29][30][31] require additional equipment, such as arbitrary waveform generators, voltage sensors, and signal-coupling devices, to conduct the diagnosis. Thus, an extra budget is required to purchase the equipment. Space and accessibility for installing the equipment are also required. Furthermore, expert and cable parameters are required to analyze the returned reference signal, which must also be designed before conducting a diagnosis for each target cable. Moreover, only sstdr [30] among other reflectometry-based methods is capable of diagnosing cable faults in online conditions. The study by [9] overcame this problem by suggesting a method that is applicable to both online and offline diagnosis using a reference burst signal; however, it still requires a signal generator and a signal-coupling device. Several studies have utilized current signals when diagnosing cable faults [10,11], which are preferred over other methods because they do not require extra equipment. However, the study by [10] could not handle the varying conditions wherein the amplitude and phase angles varied according to the movements of automated machines. The study by [11] overcame this problem by adopting a novelty detection approach, even under varying operating conditions. However, the diagnostic purpose of this study was fast fault detection and isolation when an earlystage fault occurs in the cable; thus, severity estimation needs to be supported. The severity estimation capability is important for maintenance staff because they can assess the fault condition and consequently build a proper maintenance plan, such as cable replacement and overhaul. Thus, the fault severity estimation capability of the proposed method is the most significant difference compared to previous methods. Consequently, the comparison results show that the proposed method is an appropriate approach for addressing cable fault problems in industries.

Conclusions
This study proposed a new diagnostic approach for cable faults, which is particularly applicable in the field of industrial automation. The diagnosis method comprised a novelty detection and severity estimation network, wherein an attention-based RNN was integrated into the latter network to consider time-dependent information and overcome the longterm dependence problem. RNN is a type of neural network which utilizes sequential data. In this paper, RNN was adopted because the time series data obtained from the cable are sequential and time dependent. The inputs for the novelty detection network were ZSC and SCR, and an anomaly score was generated. The severity estimation network accepted the anomaly score and estimated the fault indicator, which directly represents fault severity. The proposed method is cost effective and saves space because it does not require additional equipment to conduct the diagnosis. Moreover, cable parameters and expert knowledge are not required. Furthermore, the proposed method can provide maintenance staff with a fault severity estimation capability to assess the fault situation. Experiments on an industrial application proved that the new approach works effectively and accurately under various faults and operating conditions. Future studies will focus on intermittent faults in flexible cables for moving applications using an explainable AI approach with a data fusion algorithm.