Multi-Level Federated Network Based on Interpretable Indicators for Ship Rolling Bearing Fault Diagnosis

Abstract

The federated learning network requires all the connection weights to be shared among the server and clients during training which increases the risk of data leakage. Meanwhile, the traditional federated learning method has a poor diagnostic effect for non-independently identically distributed data. In order to address these issues, a multi-level federated network based on interpretable indicators was proposed in this manuscript. Firstly, an interpretable adaptive sparse deep network is constructed based on the interpretability principle. Secondly, the relevance map of the network is constructed based on interpretable indicators. Based on this map, the contribution of the connection weights in the network is used to build a multi-level federated network. Finally, the effectiveness of the proposed algorithm has been proved through experimental validation in the paper.

1. Introduction

Ships’ powertrains are often required to operate for long periods of time or even overloaded during shipping, which can sometimes lead to powertrain failures. Rolling bearings, as a common component of the powertrain, are often damaged under some abnormal conditions [1]. People realize that it is important to conduct fault diagnosis of ship powertrain equipment. Based on the operating data of the equipment, faults can be detected and repaired in time, so that further damage can be prevented [2]. The powertrains units inside the ship include engines and a large number of rolling bearings where friction occurs continuously with the mechanical movements. The fault that occurs in these rolling bearings accounts for a significant proportion of all the faults of powertrains units [3].

Common fault diagnosis (classification) methods include analytical model-based models and knowledge-based and data-driven approaches. Analytical model-based observer methods are limited in their application in complex systems because of the difficulty of establishing an accurate observer model to describe the mechanism of the object [4,5,6]. In addition, the difficulty in acquiring complete knowledge of complex systems has limited the application of fault diagnosis methods based on knowledge acquisition and knowledge application of cognitive processes [7]. However, data-driven approaches are becoming increasingly popular with vast amounts of data [8,9]. Traditional data-driven methods, such as statistical analysis-based principal component analysis, are built on the necessary statistical properties of the obtained data. Nevertheless, it is a difficult situation for a system that has a large amount of data but hard to establish statistical properties [10]. A data-driven approach for fault diagnosis can effectively record operational information and fault states from a large amount of historical data [11]. Based on the obtained process information, data-driven diagnostic models can be built to describe industrial processes and provide valid diagnostic results [12]. The most popular data-driven fault diagnosis methods are artificial neural network (ANN) algorithms [13,14,15], autoencoders [16] and Bayesian networks [17]. In this manuscript, we propose a novel FL (Federated Learning) model to address the problem when data-driven diagnosis methods do not have enough data on a single client. The model provides good diagnostic results even when the FL dataset is under a non-independent identical distribution. Meanwhile, it can also effectively protect data privacy.

The subsequent sections of this manuscript are organized as follows: Section 2 presents the related work. Section 3 introduces preliminary theoretical preparations for the paper. Section 4 describes the method proposed in detail. Section 5 designs experimental validation for the proposed algorithms, and Section 6 ends with the conclusion and gives suggestions for future work.

2. Related Work

The data-based diagnostics methods require a large amount of historical data to train an effective model [18]. However, it is common that some equipment (whose data is required to be kept private) is confidential, such as military vessels and some commercial boats in specific industries, which makes it extremely difficult to collect large amounts of data. In recent years, Internet of Things (IoT) technology has achieved rapid development. To further explore the ubiquitously connected world, deep learning techniques are often used to analyze the data collected by the IoT. The sensors are constantly collecting data about the operational status of the equipment during the operation of the equipment. It is possible to analyze the condition of the equipment in depth by combining these collected data [19]. With the combination of IoT technology and FL [20], such situations have been effectively addressed. FL has now achieved considerable research results in the field of IoT. A typical federated learning base on the Internet of Things (FL-IOT) architecture network structure is shown in Figure 1.

There are many experts and scholars who have proposed various improvements and extension formats on the basis of FL. Zhang et al. [21] applied FL to short-circuit fault diagnosis of permanent magnet synchronous motors, which effectively solved the problem of lacking samples. FL is slow to converge because of the constant synthesis of each client’s upload network parameters during the learning process. To address this problem, Liu et al. [22] proposed a momentum federated learning (MFL) method, which uses the MFL method to perform gradient updates while conducting local training. Considering the problem that only some of the clients’ data are fused at a training epoch in federated learning, Chen et al. [23] proposed a probabilistic user selection scheme that connects the server to users whose local FL models have a significant impact on the global FL model with high probability, thus increasing the global convergence speed. Amiri et al. [24] found that most FLs did not study federated learning from the actual physical layer. This paper proposed a wireless multiple access channel (MAC) from the device to the server, through which the parameter server receives the gradient calculated by the device at each iteration of the distributed stochastic gradient descent algorithm, thus greatly increasing the convergence speed. The above research focus on improving the parameter integration algorithm to achieve higher accuracy without effectively considering the privacy of the data.

Cyber-attacks can also have a serious impact on the effectiveness of FL. For example, Saha R et al. [25] pointed out that although FL is a popular distributed learning approach, global aggregation in FL relies on a centralized server which is vulnerable to malicious attacks, resulting in inefficient training models. This paper added fog nodes under the cloud server according to the location of the client distribution, thus effectively avoiding too much reliance on a single cloud server. Hao et al. [26] proposed privacy-enhanced federated learning (PEFL) to address the phenomenon that an attacker can still perform attacks on the network in the FL framework. Compared to the existing solutions, PEFL is non-interactive and can prevent private data from being compromised even if multiple entities collude with each other. Li et al. [27] proposed a CNN gated recurrent unit (CNN-GRU)-based detection model to identify and isolate the attacks in federated networks. The above research mainly focus on improving the convergence speed of FL, and they do not fully consider how to deal with the problem of data leakage.

Improvements have also been proposed for the problem of whether the data for federated learning obeys the independently identically distributed (IID) data. Sattler et al. [28] addressed the situation where the samples are unevenly distributed across clients, i.e., the data is non-independently identically distributed (NOIID), and the network complexity is limited. A clustering federated learning algorithm is proposed. The clustering structure is derived from the cosine correlation metrics for the client data, and the gradient is updated by the clustering federation algorithm. The communication cost of a federated network is also an important limitation on its performance, so Chen et al. [29] propose an enhanced joint learning technique to address the dilemma. They took an asynchronous learning strategy for the client and a time-weighted aggregation strategy for the local model on the server. With the development of blockchain technology, researchers have started to consider the integration of blockchain technology and FL to implement a decentralized FL network [30]. However, these are all faced with the problem of a black box of machine learning neural network and lack of interpretability. They improve the federal aggregation algorithm to prevent cyber-attacks. However, no consideration has been given to the situation that there exists a hostile client and how to prevent data leakage if the hostile client knows the gradient parameters of the other clients.

In summary, it is worth pointing out that FL technology has some limitations: (1) Federated networks require fast and real-time communications capabilities; (2) the traditional FL method has poor diagnostic effect for data under NOIID distribution; (3) the risk of data leakage from federated networks exists, and some studies have shown that information about data can be reversed through the gradient of updates shared by federated networks [31]; (4) the federated network obtained by means of learning is a black box, and it is difficult to understand the internal mechanisms of the network. Moreover, the credibility of the diagnostic results given by the network is doubtful in practical applications. To address the above problems with the usage of FL, we design a multi-level shared federated network based on interpretable indicators (i-MFN).

The main contributions of this manuscript are as follows.

(1)

Construct the interpretable adaptive sparse depth networks;

(2)

Design a map of network correlations based on the network’s interpretability parameters;

(3)

Establish a multi-level FL network and design a corresponding gradient sharing mechanism.

3. Preliminaries

3.1. LRP Network Interpretability Principle

Neural networks have been widely used in various fields due to their extremely nonlinear fitting properties and powerful information extraction ability. Currently, people mostly use neural networks as black boxes, i.e., they do not care about their internal propagation mechanisms and logic, which leads to the results with low credibility in some areas to some degree. For example, in the field of medical diagnosis, doctors are unclear about the basis of the results given by neural networks, leading to limitations in their diffusion. The Layer-wise Relevance Propagation (LRP) [32] technique is a metric based on the importance of the correlation between layers of a neural network. The main principle can be summarized as follows: Firstly, starting from the last layer of the network and proceeding backward, the correlation index between the outputs of each neuron in the penultimate layer is calculated according to the output of the network. Then, according to the correlation index of the neurons in the penultimate layer, the correlation of each neuron in the preceding layer can be deduced backward. By parity of reasoning, calculating the correlation index of the outputs of all neurons in the network with the results is doable. The calculation formula is shown in Equations (1) and (2), where i is the input to the k-th neuron, ${\Sigma }_i$ is the input set of the k-th neuron, $w_{ik}$ is the connection value between the i-th neuron in layer l and the k-th neuron in layer l + 1, $a$ is the output value of the neuron, and $R_k^{(l)}$ denotes the correlation index of the kth neuron in the lth layer to the output result of the neural network.

$$R_k^{(l+1)}={\displaystyle \sum _i{R}_{i\leftarrow k}^{(l,l+1)}}$$

(1)

$$R_i^{(l)}={\displaystyle \sum _k{R}_{i\leftarrow k}^{(l,l+1)}}$$

(2)

$$R_{i\leftarrow k}^{(l,l+1)}=R_k^{(l+1)}\frac{a_i^{(l)}{w}_{ik}}{{\displaystyle {\sum }_i{a}_i^{(l)}{w}_{ik}}}$$

(3)

Taking a three-layer neural network as an example, its propagation calculation process is shown in Figure 2.

3.2. Introduction on Federated Learning Framework

Machine learning neural networks and some other learning algorithms require a large amount of data to train the network, and many scenarios require that data can only be used internally and are not available for sharing with others in practical applications, which makes it difficult to train a neural network with good performance. In order to solve this problem, the federated learning mechanism enables the neural network to effectively learn the internal features of each individual while effectively protecting the privacy of the user’s data. The client data can be non-independent and identically distributed.

As shown in Figure 1, a typical FL-IOT architecture network structure consists of different clients, servers, and communication networks. The local model of clients is trained on their own data, while the data keeps private. The model parameters are transmitted among the communication network, which is built on satellites or 5G communication technology.

Assuming that a certain federated network has $K$ clients, $X$ denotes the sample feature space, and $Y=\{1,\dots C\}$ denotes the label space. The federated learning loss function is shown in Equation (4).

$$\begin{array}{cc}\hfill L(w_t^f)& \triangleq {\displaystyle \sum _{k=1}^K\frac{n^k}{\left|D\right|}}{\displaystyle \sum _{i=1}^C{p}^k(y=i){\mathbb{E}}_{x|y=i}[\log {ℱ}_i(x,w_t^k)}]\hfill \\ & ={\mathbb{E}}_{(x,y)~p^k}[{\displaystyle \sum _{i=1}^C{1}_{y=i}\log {ℱ}_i(x,w_t^k)}]\hfill \end{array}$$

(4)

where $ℱ$ is the federated network model function, C represents the types of all fault outputs, $X$ is the feature space of the sample data, and ${ℱ}_i$ denotes the network model of the i-th client.

The training model for each client is shown in Equation (5).

$$\begin{array}{cc}\hfill L^k(w_t^k)& ={\mathbb{E}}_{(x,y)~p^k}[{\displaystyle \sum _{i=1}^C{1}_{y=i}\log {ℱ}_i(x,w_t^k)}]\hfill \\ & ={\displaystyle \sum _{i=1}^C{p}^k(y=i){\mathbb{E}}_{x|y=i}[\log {ℱ}_i(x,w_t^k)}]\hfill \end{array}$$

(5)

4. The Proposed Method

4.1. Interpretable Adaptive Sparse Depth Networks

The structure of the interpretable adaptive sparse depth networks constructed based on Layer-wise Relevance Propagation technique is shown in Figure 3. After normalizing the correlation values of each layer in the network, they are fed into the adaptive sparse layer, which performs adaptive sparse processing of the neurons. After the neural network has been trained, the sparse value of each neuron is the final sparse value.

The workflow of interpretable adaptive sparse depth networks is as follows.

Step 1: The network enters a fine-tuning phase after the coarse tuning of encoding and complete decoding.

Step 2: The interpretable correlation parameters for each neuron are derived after each forward training iteration is completed.

Step 3: The correlation parameters are normalized and used as the input value of the adaptive sparse layer neurons.

Step 4: The adaptive sparse layer is processed to obtain a sparse value which is used as a threshold for forward propagation sparsity of the network.

The literature [33] shows that, as the diagnostic capability of the network increases, the locations of the effective neurons in the network that extract the valuable signals will be fixed. Therefore, adding the interpretable term to the loss function of the network is expected, as shown in Equation (6), where λ is the coefficient of the interpretable loss function. A_t denotes the test accuracy of the test set, and $\varphi (x)$ represents the Softplus function after normalization [34]. $\varphi (x)$ is used as a nonlinear transformation function to measure the correlation between the output value of each neuron and the network outcome, and its expression is shown in Equation (7).

$$J_Q=\lambda (1-A_t){\displaystyle \sum _{l=1}^L{\displaystyle \sum _{i=1}^{nl}\varphi (R_i^{(l)})}}$$

(6)

$$\varphi (x)=\frac{softplus}{\max (softplus)}$$

(7)

$$softplus=\ln (1+e^x)$$

(8)

The interpretable correlation parameters for each neuron are used to train the adaptive sparse layer. Then the output value of the adaptive sparse layer is used for sparsing the network. From Equation (6), it can be seen that as the training accuracy continue improving, the neurons that play a positive correlation role will be gradually highlighted, while the negatively correlated neurons will be weakened. After such processing, the generalization ability of the network can be greatly improved. The adaptive sparse method is a shallow neural network consisting of neurons whose output values are processed by a sigmoid nonlinear function. Neurons will be inhibited, according to Equation (9), which will limit the output values to between (0, 1). This method is a soft sparse mechanism compared to the traditional 0 or 1 sparse method. Due to the neuron structure, the sparse metrics can be adaptively adjusted to the different network structures.

$$sigmoid(x)=\frac{1}{1+e^{-x}}$$

(9)

The main advantages of the interpretable adaptive sparse deep neural networks proposed in this manuscript are shown as follows when compared to traditional sparse methods.

(1)

The sparse approach is a soft-threshold sparse approach;

(2)

Sparse indicators are interpretable;

(3)

It can be adaptively adjusted according to the structure of the network or on the data set.

4.2. Construction of the Multi-Level Federated Learning Framework

4.2.1. Multi-Level Sharing Mechanism

Based on the LRP values of the network, an Rw map is constructed. Taking the neural network shown in Figure 4 as an example, the construction process of the Rw map is as follows:

(1)

The Rw map consists of $n_l\times n_{l+1}$. pixel points, where $n_l$ is the number of neurons in layer l, and $n_{l+1}$ is the number of neurons in layer l + 1;

(2)

The vertical axis corresponds to neurons in layer l, and the horizontal axis corresponds to neurons in layer l + 1. Each pixel $R_{i\leftarrow k}^{(l,l+1)}$ in the map corresponds to the cross-values of layers l and l + 1;

(3)

A Rw map can be made by coloring according to the red and blue heat map.

According to the definition of the LRP value, the deep red parts correspond to network nodes that have a significant contribution to the output, while the deep blue sections are negatively correlated. Figure 5 shows an example of the Rw map. The deep red part corresponds to a network node that has a significant contribution to the results, so it will be sent to the top level for parameter sharing. The middle-colored segment corresponds to the network parameters that contribute the second most to the network diagnostic results and are shared in the secondary sharing level. The network parameters corresponding to the deep blue section are negatively correlated to the diagnostic results and are thus reserved locally and not shared with other clients.

If $R_{i\leftarrow k}^{(l,l+1)}>R_{\theta +}$, the corresponding parameters of the net will be shared at the top sharing level. While it meets if $R_{\theta -}<R_{i\leftarrow k}^{(l,l+1)}<R_{\theta +}$, they are shared at the secondary level, and if $R_{i\leftarrow k}^{(l,l+1)}<R_{\theta -}$, they are not shared.

A multi-level federated learning framework is shown in Figure 6. The sharing method is divided into three levels. Network nodes can be classified according to their LRP values, and the importance of each federated network can be divided into three categories according to the node’s contribution to the prediction results during the design of the FL process. The first: if $R_{i\leftarrow k}^{(l,l+1)}>R_{\theta +}$, then its corresponding parameters are for the top sharing level. The second: if $R_{i\leftarrow k}^{(l,l+1)}<R_{\theta -}$, then its corresponding network connection parameters are not shared. The third: if $R_{\theta -}<R_{i\leftarrow k}^{(l,l+1)}<R_{\theta +}$, then its corresponding network parameters are shared in the secondary level. A multi-level parameter sharing mechanism is established.

Figure 7 shows the flow chart of the proposed i-MFN algorithm. Firstly, each client performs local training. Based on the network values, the LRP values of the network are calculated. Then the Rw map can be drawn. Finally, the parameters can be sent out for multi-level sharing based on the Rw map.

4.2.2. Network Parameter Update Strategy

Suppose that there is a set $X=\{X_1,X_2\dots X_L\}$, where $X_1=\{x_1^1,x_1^2\dots x_1^N\}$, $X_2=\{x_2^1,x_2^2\dots x_2^N\}\dots X_L=\{x_L^1,x_L^2\dots x_L^N\}$ reconstitute the elements at each position of $X_1,X_2\dots X_L$ into a new set, thus it obtains a new set $X^1=\{x_1^1,x_2^1\dots x_L^1\},$ $X^2=\{x_1^2,x_2^2\dots x_L^2\}\dots X^N=\{x_1^N,x_2^N\dots x_L^N\}$ Define mapping:

$$\mathcal{G}(X_1,X_2\dots X_L)=\{d^1,d^2\dots d^N\}$$

(10)

The following conditions are met:

(1)

The set of all non-zero elements in the set $X^n=\{x_1^n,x_2^n\dots x_L^n\}$ is $C^n=\{x_1^n,x_2^n\dots x_{L^{\prime }}^n|x_{l^{\prime }}^n\in X^n,x_{l^{\prime }}^n\ne 0\}$;

(2)

$d^n=\frac{1}{L^{\prime }}{\displaystyle {\sum }_{l=1}^{L^{\prime }}{x}_l^n}$.

Based on the parameter update strategy of the network, the pseudo-code of the i-MFN algorithm can be divided into two major parts: the server side and the client side.

Algorithm 1 shows the interpretable federated learning algorithm for the server side.

Algorithm 1:i-MFN algorithm for server side

1. Input: LRP threshold $R_{\theta }$, learning rate $l_r$, loss function $J$, maximum number of iterations $N_{train}$ 2. Cloud server side: 3. Cloud parameter initialization ${\theta }_0^{cloud}$ 4. for $i=1,2,\dots N_{train}$ do 5.   Send ${\theta }_{i-1}^{cloud}$ of round i − 1 to each secondary level 6.   Wait for each secondary level to upload its own network parameters ${\left\{{\theta }_i^{k\to cloud}\right\}}_{k=1}^{N_{edge}}$ 7.   Update ${\theta }_i^{cloud}\leftarrow \mathcal{G}({\theta }_i^{1\to cloud},{\theta }_i^{2\to cloud},\dots {\theta }_i^{N_{edge}\to cloud})$ 8. end for 9. Edge server side: 10. for $i=1,2,\dots N_{train}$ do 11.   for $k=1,2,\dots N_{edge}$ do 12.     Download ${\theta }_{i-1}^{cloud}$ from top level server 13.     Wait for each subordinate clients to upload its own network parameters ${\left\{{\theta }_i^j\right\}}_{j=1}^{N_{clientk}}$ and Rw-MAP 14.    Extract the secondary level parameters and share parameters for each client based on the uploaded Rw-MAP ${\left\{{\theta }_{i,cloud}^j\right\}}_{j=1}^{N_{clientk}}\leftarrow {\left\{{\theta }_i^j\right|R_w>R_{\theta +}\}}_{j=1}^{N_{clientk}}$, ${\left\{{\theta }_{i,edge}^j\right\}}_{j=1}^{N_{clientk}}\leftarrow {\left\{{\theta }_i^j\right|R_{\theta -}<R_w<R_{\theta +}\}}_{j=1}^{N_{clientk}}$, ${\left\{{\theta }_i^j\right\}}_{j=1}^{N_{clientk}}={\left\{{\theta }_{i,edge}^j\right\}}_{j=1}^{N_{clientk}}\cup {\left\{{\theta }_{i,Cloud}^j\right\}}_{j=1}^{N_{clientk}}$. Unshared positions in ${\theta }_i^j$ are filled with zeros ${\theta }_i^{k\to client}\leftarrow \mathcal{G}({\theta }_{i,edge}^1,{\theta }_{i,edge}^2\dots {\theta }_{i,edge}^{N_{clientk}},{\theta }_{i-1}^{cloud})$, ${\theta }_i^{k\to cloud}\leftarrow \mathcal{G}({\theta }_{i,cloud}^1,{\theta }_{i,cloud}^2\dots {\theta }_{i,cloud}^{N_{clientk}})$ 15.    Upload ${\theta }_i^{k\to cloud}$ to top level and send ${\theta }_i^{k\to client}$ to the secondary level 16.  end for 17. end for 18. Output: ${\theta }^{client}$

For better illustrating the parameter update process, the interpretable federated learning algorithm for the client side is shown as Algorithm 2.

Algorithm 2:i-MFN algorithm for client side

1. Client Side 2. for $i=1,2,\dots N_{train}$ do 3.   for $k=1,2,\dots N_{edge}$ do 4.     Download ${\theta }_{i-1}^{k\to client}$ 5.     for $j=1,2,\dots N_{clientk}$ do 6.     ${\theta }_i^{k,j}\leftarrow \mathcal{G}({\theta }_{i-1}^{k\to client},{\theta }_{i-1}^{k,j})$ as the network parameters for this round of training 7.        for each training batch do 8.         ${\theta }_i^{k,j}\leftarrow {\theta }_i^{k,j}-l_r\frac{\partial J}{\partial {\theta }_i^{k,j}}$ 9.        end for 10.     Upload client parameters ${\theta }_i^{j,k}$ 11.     end for 12.  end for 13. end for

5. Simulation Experiments

5.1. Dataset Preparation

In order to evaluate the effectiveness of the proposed method in fault diagnosis, some experiments were carried out for validation. The experimental data is a publicly available motor bearing dataset from Case Western Reserve University. We used the proposed interpretable federated learning model for fault diagnosis validation.

The rolling bearing dataset of Western Reserve University has nine fault types. They are: Ball Defect I (BDI), Ball Defect II (BDII), Ball Defect III (BDIII), Inner Ring Defect I (IR I), Inner Ring Defect II (IRII), Inner Ring Defect III (IRIII), Outer Ring Defect I (OR I), Outer Ring Defect II (ORII) and Outer Ring Defect III (ORIII). The sampling frequency of the vibration sensor is 12 kHz, and the motor speed is set to 1772 rpm. It can be figured out that there are about 400 sampling points per circle. Three hundred samples were collected for each fault type. The sample states are shown in

Table 1. Data information on nine types of fault states.

Fault Type	Fault Diameter (inch)	Label	Sample Size
BDI	0.007	1	300
BDII	0.014	2	300
BDIII	0.021	3	300
IRI	0.007	4	300
IRII	0.014	5	300
IRIII	0.021	6	300
ORI	0.007	7	300
ORII	0.014	8	300
ORIII	0.021	9	300

. The samples were processed using the k-fold cross-validation method to ensure that the experimental results are non-chance.

Figure 8 shows the waveform of the above fault samples on the drive side.

5.2. Experimental Settings

5.2.1. Feature Extraction

Effective fault feature extraction of the collected data is a key step toward effective fault diagnosis. We refer to the method in [35] to extract fault features in the frequency and time domains and feed the extracted feature information into the diagnosis network as the input to the network. We have listed the fault characteristics used in the text, as shown in

Table 2. Features used in the experiment.

Features	Expression
Absolute mean:	$\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\left|x_i\right|}$	Time domain
Variance:	$\frac{1}{n}{\displaystyle {\sum }_{i=1}^n(x_i}-x{)}^2$
Clearance factor:	$\max \left(\left|x_i\right|\right)/{\left(\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\sqrt{x_i^2}}\right)}^2$
Crest factor:	$\max \left(\left|x_i\right|\right)/\sqrt{\frac{1}{n}}{\displaystyle {\sum }_{i=1}^n{x}_i^2}$
Shape factor:	$\sqrt{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{x}_i^2}}/\left(\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\left|x_i\right|}\right)$
Average frequency:	$\left({\displaystyle {\sum }_{i=1}^n{\omega }_i{X}_i}\right)/{\displaystyle {\sum }_{i=1}^n{X}_i}$	Frequency domain
Crest:	$\max \left(\left|X_i\right|\right)$
Kurtosis:	$\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{X}_i^4}$
Mean energy:	$\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{X}_i}$
Variance:	$\frac{1}{n}{{\displaystyle {\sum }_{i=1}^n\left(X_i-\overline{X}\right)}}^2$

below.

5.2.2. Data IID Distribution

The data for each client is independently and identically distributed (IID). In practical experiments, it is assumed that each client contains the same fault types, and the dataset number is not significantly different.

5.2.3. Data NOIID Distribution

In this experiment, the case where each client has different types of fault data is called NOIID data distribution. The assignment of data between clients should ensure the NOIID distribution between clients while maintaining a balance of data across all clients. The actual assignment of fault types to individual clients is shown in

Table 3. Distribution of client data.

Client No.	Fault Type	Label
Client1	IRII IRIII	5,6
Client2	BDI BDII	1,2
Client3	BDII BDIII	2,3
Client4	BDII BDIII	2,3
Client5	BDIII IRI	3,4
Client6	BDIII IRI	3,4
Client7	BDI ORIII	1,9
Client8	ORI ORII	7,8

.

5.2.4. Rw Map

Based on the principles proposed in Chapter III, a schematic of the Rw map for a four-layer network is shown in Figure 9a, and a map decomposition of the Rw values at the corresponding level is shown in Figure 9b.

From Figure 9, the diagnosis net has four layers. According to the Rw map, the importance of each neuron is clearly demonstrated.

5.3. Experiment and Analysis

5.3.1. Description of Experimental Comparison Conditions

(1)

Upper Limit: All clients’ data is used to train a neural network, and the result is used as an upper limit for the diagnostic accuracy of the diagnosis network.

(2)

Distributed IID: All clients are trained only using their own private IID data. Distributed-own indicates a fully independent structure where the test dataset is also the client’s own data. The test dataset for Distributed-all is a centralized structure, indicating that the test dataset is the sum of all clients’ test sets.

(3)

Distributed NOIID: All clients are trained using only their own private NOIID data. Distributed-own indicates a fully independent structure where the test dataset is also the client’s own data. The test dataset for Distributed-all is a centralized structure, indicating that the test dataset is the sum of all clients’ test sets.

(4)

i-MFN: Denotes the test accuracy of the algorithm proposed in the manuscript, and the test set is centralized.

5.3.2. Experiment for IID Data Distribution

Figure 10 and

Table 4. Comparison of diagnostic accuracy under IID distribution.

IID
	Upper Limit	Distributed-Own	Distributed-All	i-MFN
C1	98.50%	94.00%	95.00%	98.00%
C2	98.00%	98.00%	96.88%	99.00%
C3	98.50%	97.00%	95.88%	98.00%
C4	98.75%	96.00%	96.88%	97.00%
C5	99.25%	95.00%	95.63%	97.00%
C6	99.00%	95.00%	96.13%	94.00%
C7	97.75%	99.00%	96.50%	99.00%
C8	100.00%	97.00%	97.38%	94.00%
Mean	98.72%	96.38%	96.28%	97.00%

show the experiment results for each experimental condition when the data is IID distributed.

When experiments were conducted with each client using the IID data distribution, the trend in accuracy rates for each client is shown in Figure 11. In the experiments, the test set is owned by the clients themselves, except for ClientX to All, whose test set is consolidated with all clients’ test sets merged.

The test accuracy of the individual client and server sides are shown in Figure 12 when network parameters are federated.

It can be seen from the experiment results that since the data obeys the IID distribution, each client’s data has almost the same features. As a result, the local clients can learn a strong global fault recognition capability without sharing data. This is precisely why Distributed-own and Distributed-all are of similar diagnostic accuracy. The Upper Limit method has all the datasets, and thus it can extract as much information as possible. So it is the most powerful method. The i-MFN method extracts the weights which are important to the network when sharing is performed. The information exchanging ability is thus effectively improved. The training efficiency of the network has been improved.

5.3.3. Experiment for NOIID Data Distribution

Figure 13 and

Table 5. Comparison of diagnostic accuracy under NOIID distribution.

NOIID
	Upper Limit	Distributed-Own	Distributed-All	i-MFN
C1	98.50%	100.00%	13.00%	99.50%
C2	98.00%	98.00%	31.00%	90.00%
C3	98.50%	99.00%	43.00%	98.00%
C4	98.75%	99.50%	43.00%	98.00%
C5	99.25%	100.00%	38.00%	99.00%
C6	99.00%	100.00%	38.00%	99.50%
C7	97.75%	100.00%	19.00%	89.50%
C8	100.00%	100.00%	13.00%	100.00%
Mean	98.72%	99.56%	29.50%	96.69%

show the experiment results for each condition when the data is under NOIID distribution.

The distribution of the data sets under NOIID distribution for each client is shown in Table 3. The server’s test set was composed of all clients’ test sets.

When experiments were conducted with each client using the NOIID data distribution with network parameters all-shared, the trend in accuracy rates for each client and server is shown in Figure 14.

When the experiments were conducted with each client using the NOIID data distribution with network parameters isolated, the trend in accuracy rates for each client and server is shown in Figure 15.

From the experimental results, it can be seen that, for NOIID distributed data, local clients can only learn the features of their own data without parameters sharing. As a result, the local client network has little diagnostic ability when new fault types occur. So we can find that the diagnosis results are better in the Distributed-own case but worse in the Distributed-all case. Similarly, federated + average (Ave-FL) [21], in the case of NOIID data distribution, does not extract valid information well due to gradient interference, making the server unable to perform gradient fusion, thus reducing the diagnostic ability. The i-MFN extracts some of the important network parameters for sharing and avoids the global sharing, effectively reducing the gradient interference from other clients in the global sharing, thus greatly improving the overall network diagnostic capability.

5.3.4. Privacy Analysis

The Deep Leakage from Gradients (DLG) method proposed by ZHU [30] is used to extract the original data information leaked by the shared gradients. Figure 16 shows the data information leaked by the full gradient information and the proposed i-MFN algorithm at the top sharing level, respectively.

We found that, as the gradient information decreases, the amount of information that can be extracted from the gradients will decrease until nothing can be extracted at all. With the proposed algorithm, the shared gradients are filtered, and data leakage from the client can be effectively avoided. The main advantages of the i-MFN are as follows.

(1)

The most valuable weights selected by the interpretable indicators ensure that the diagnostic capabilities are effectively transmitted between clients;

(2)

The gradients involved in sharing are not sufficient to cause a data leakage.

5.3.5. Experimental Analysis

A comparison of the performance of several algorithms is shown in

Table 6. Comparison of diagnostic accuracy of different algorithms.

Method	NOIID	Deep Leakage from Gradients
Upper Limit	98.72%	-
Distributed-own	99.56%	Yes
Distributed-all [36]	29.50%	No
Federated + SKF [36]	77.72%	Yes
Ave-FL [37]	27.67%	Yes
α-FedAvg(α = 25%) [38]	91.32%	No
CEC + Average [39]	92.89%	No
i-MFN	96.69%	No

Note: “-“: the dataset of the Upper Limit itself is shared, so there is no problem of “deep leakage from gradients”.

.

From the experimental results, the following conclusions can be drawn:

(1)

For data under IID data distribution. It can be seen from Table 4 that there is little difference between the average diagnosis of each client and the whole in the case of Distributed-own and Distributed-all. The main reason for this result can be summarized as follows: The data distribution is similar between the individual clients, and there is almost no difference between dataset distributions for Distributed-own and Distributed-all, so the federation results are not significantly different from those of the centralized one.

(2)

For data under NOIID data distribution. The test accuracy is 98.72% for Upper Limit, 99.56% for Distributed-own, 29.50% for Distributed-all, and 27.67% for Ave-FL. In addition, accuracy for α-FedAvg (α = 25%) is 91.32%, 92.89% for CEC + Average, 77.72% for Federated + SKF and 96.69% for i-MFN. It can be seen that Distributed-own and Distributed-all differ significantly due to the large change in data distribution between the individual clients. α-FedAvg and CEC + Average reduce parameter interference by reducing the numbers of sharing but have randomness and limitations. On the other hand, the i-MFN proposed in the paper is extremely effective in learning the characteristics of the data between NOIID data. Each client is well trained to learn the characteristics of other clients’ data. Thus, the overall diagnostic accuracy of the federated network can be improved.

(3)

For the protection of data privacy. The DLG method proposed by ZHU [30] is a kind of privacy data acquisition algorithm similar to an adversarial neural network. This method is significantly effective in obtaining complete raw data through the shared gradients. However, the stealing of private data by this method can be effectively blocked by the mechanism of multi-level sharing proposed in this paper. Because the unfriendly client only gets a small fraction of the gradient information at a single time, and the position of the gradient is uncertain. The main reasons for this can be summarized as follows: 1. We have experimentally verified that the DLG method needs to retain all gradient information in order to recover the original data effectively. This is the reason that both Ave-FL and Federated + SKF can lead to data leakage. They share all the gradient information during parameter aggregation. Even if randomly changing the value of a particular gradient to zero, it will significantly affect the effectiveness of the data recovery. 2. Based on the above analysis, we share the gradient parameters into multiple levels through the LRP interpretability metric. By disrupting the gradient information, the secrecy of the information is thus effectively guaranteed.

6. Conclusions

In this paper, an interpretable multi-level federated learning network is proposed to solve the black box problem of the general federated network. An interpretable multi-level federated network has also been constructed based on an interpretable index, which allows the dataset under NOIID distribution to maintain a good diagnostic accuracy in FL. The confidentiality of the network is enhanced. In addition, how to build a federal network structure in a multi-level federated center still needs to be further explored in future research.