A MEMS Gyroscope Noise Suppressing Method Using Neural Architecture Search Neural Network

,

According to the GNSS working principle, GNSS is the radio wireless signal-based navigation system. e receivers operate positioning function relying on receiving the wireless signals from the navigation satellites [8][9][10]. For GNSS, the navigation satellites in orbit emit the signals to the earth, and the receivers get the signal for obtaining distance between the user and satellite through measuring the transmitting time of the navigation signal. However, while the signal reaches the earth, it is quite weak and easy to be disturbed or interfered [8][9][10]. As illustrated in past publications, the following two reasons account for this phenomena: (1) the transmitting signal power is limited by the energy of the navigation satellite, saving the energy and for keeping the satellite life span, it is hard to enlarge the navigation signal strength will consume more energy; (2) for saving the cost of constructing the GNSS, while meeting the demand of covering the earth with fewer satellites, the orbit is far away from the earth and the signal transmits a long distance before reaching the earth [8][9][10][11]. Due to the above drawbacks, a standalone GNSS is usually unable to output seamless and ubiquitous navigation solutions. us, it is of significance for enhancing the performance of the GNSS in signal challenging environments.
In the past decades, researchers are devoted to improving the GNSS receiver performance under weak signals. e first approach is to extend the integration time, which is effective for weak signal tracking [12][13][14]. e second method is to utilize advanced signal tracking architecture, for instance, vector tracking loop (VTL) [15,16]. However, apart from the weak signals, the receiver cannot receive all the available satellite signals due to the signal blockage. Under this condition, extending the integration time is usually ineffective. New approaches are proposed to address this situation. According to past investigations, integrating GNSS with other available sensors or navigation system for improving the availability of navigation solutions is the most popular solution [15][16][17]. Among them, the inertial navigation system (INS)/GNSS integrated navigation system is the most popular, due to the highly complementary characteristics between GNSS and INS [18][19][20][21]. e INS is a famous self-contained navigation system, which produces navigation solutions from processing data from inertial measurement unit (IMU). Commonly, an IMU is composed of three orthogonal gyroscopes and accelerometers [18][19][20][21]. Gyroscopes measure angular velocity; the accelerometers sense the specific force. Measurements from the IMU are processed together for outputting navigation solution (position, velocity and attitude, PVA) continuously. Due to the random noise contained in the raw signals, the INS navigation solutions diverge dramatically over time. GNSS has precise navigation solutions, which could be integrated with INS for providing more reliable navigation solutions [18][19][20][21]. As mentioned above, the INS and GNSS are complementary. us, GNSS/INS integration system is always the preferable selection for long-term accurate navigation solutions. While GNSS signal is normal, the GNSS can calibrate the INS and compensate the INS errors. While the signal is abnormal, the INS could provide shortterm navigation solutions [18][19][20][21]. But, the errors diverge over time, which is determined by the noises contained in the IMU measurements. To further improve the performance of the GNSS/SINS integrated navigation system, it is critical to reducing the IMU measurement noise. Especially, for volume and cost-sensitive applications, Micromechanics Electronics Manufacturing System (MEMS) IMU is usually employed to construct the INS. In fact, there are several kinds of IMUs; among them, MEMS IMU is popular due to low cost and small size. MEMS IMU also performs the lowest accuracy. It is of great significance for reducing or suppressing MEMS IMU noises [18][19][20][21].
In this paper, inspired by the excellent capability of deep neural networks in sequence data processing, the advanced Neural Architecture Search Recurrent Neural Network (NAS-RNN) was employed and tested for reducing the MEMS gyroscope noise (only the gyroscope was tested using the proposed method, and the accelerometers processing was similar to that of gyroscope). A commercial MEMS IMU STIM300 was employed in the experiments [22][23][24][25][26][27][28][29][30][31]. e sampling frequency was 125 Hz, and the testing time was approximately 800 seconds.
Specially, Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and Simple Reduced Unit Recurrent Neural Network (SRU-RNN) were employed in MEMS gyroscope noise suppressing. Both LSTM-RNN and SRU-RNN were popular variants of RNN, and they both performed better than conventional machine learning or regression method [32,33]. However, it is interesting to explore a more feasible RNN structure in this application. NAS-RNN was able to automatically search for a more appropriate neural architecture for certain application. A popular commercial method was employed in this experiment for testing the NAS-RNN performance, and the LSTM-RNN was selected as the reference for demonstrating the NAS-RNN superiority compared with the LSTM-RNN.
Apart from the introduction section, the rest of the paper is organized as follows: (1) Section 2 introduces the model and the workflow of the employed NAS-RNN model; (2) Section 3 illustrates the NAS implementation, training, results, and the comparison details with the LSTM-RNN; (3) Section 4 concludes the paper and the discussion is added following this section.

Methods
In this section, we first introduce the basic architecture and the working flowchart of a popular LSTM-RNN, and then, an RNN generative method using NAS is presented. is section is divided into two subsections: (1) LSTM unit basic structure, relative mathematical equations, and workflow; (2) NAS-RNN mechanics and setting up, especially how NAS-RNN is trained and the logic.

Long Short-Term Memory Unit.
Before introducing the NAS-RNN, Figure 1 gives the basic architecture of the LSTM unit, which is composed of three "gates" including "forget gate," "input gate," and "output gate." Basically, the three different gates have its unique functions, and the details are as following: (1) "Forget gate" is the first gate, and its function is to decide what information will be thrown away from the previous cell state. As illustrated in Figure 1, the gate takes the variable h t− 1 and x t as the inputs of the function, which will generate values representing the forgetting degree of each number in the cell state. e operation equation f t is as follows: 2 Mathematical Problems in Engineering where σ(·) is a sigmoid function, W f is the updating weights, b f is the bias, h t− 1 is the hidden state, and x t is the input vector. (2) e next is the "input gate," which controls what new information should be inputted to the module. e relative equations are as follows: here σ(·) is a sigmoid function, W i is the updating weights, b i is the bias in the input gate, h t− 1 is the hidden state at time t − 1, W C is the updating weights, b C is the bias, x t is the input vector, and C t is the new cell candidate values. is gate has two functions.
(3) e last part is the "output gate," and by its nature, this gate is designed for managing the outputs of the unit. Two functions are employed in this gate, and the details are given in equations (3) and (4).
where W o is the updating weights, b o is the bias in the output gate, and C t is the cell state at time t.

Neural Architecture Searching Recurrent Neural Network.
e last subsection gives the basic LSTM unit computation in detail. In fact, it requires expert knowledge and takes time to design such basic recurrent unit. Recently, a neural architecture searching is proposed for generating neural networks with reinforcement learning [34-37]. Figure 2(a) is the overview of the NAS; the work is based on the observation that the structure and connectivity of a neural network can be typically specified by a variable-length string. It is therefore possible to use a recurrent network, the controller, to generate such string. Training the network specified by the string, the "child network," on the real data will result in accuracy on validation set. Using this accuracy as the renewal signal, we can compute the policy gradient to update the controller. As a result, in the next iteration, the controller will give higher probabilities to architectures that receive high accuracies.
In fact, the basic LSRN-RNN computations could be extended or illustrated as the tree of steps. Figure 3 shows an example of this tree structure, which is composed of two leaf nodes and one internal node. e leaf nodes are indexed by 0 and 1, and the internal node is indexed by 2. In this architecture, the controller RNN needs to predict 3 blocks first, each block specifying a combination method and activation function for each tree index. After this, it needs to predict the last 2 blocks that specify how to connect C t and C t− 1 to temporary variables inside the tree. Specifically, according to the predictions of the controller RNN in this example, the following computation steps will occur: (1) e controller predicts Add and tanh for tree index 0, and this means that we need to compute a 0 � tan h(W 1 * x t + W 2 * h t− 1 ); (2) e controller predicts ElemMult and ReLU for tree index 1, and this means that we need to compute a 1 � ReLU((W 3 * x t ) · (W 4 * h t− 1 )); (3) e controller predicts 0 for the second element of the "Cell Index"and Add and ReLU for elements in "Cell Inject," which means that we need to compute a new 0 � ReLU(a 0 + C t− 1 ). Notice that we do not have any learnable parameters for the internal nodes of the tree; (4) e controller RNN predicts 1 for the first element for the "Cell Index," and this means that we should set C t to the input of the tree at index 1 before the activation, C t � ((W 3 * x t ) · (W 4 * h t− 1 )).

Experiments and Results
In this section, field tests were carried out for evaluating the performance of the proposed method. Moreover, a common LSTM-RNN was employed and compared with the proposed NAS-RNN in terms of the standard deviation of the gyroscope signals and the attitude errors. In this experiment, a popular MEMS IMU STIM300 was adopted, which was manufactured by Sensors AS Company from Norway. e IMU was presented in Figure 4, the gyroscope's full measurement range was ± 400/s, the gyroscope's bias instability was 0.3°/h, the angle random walk was ≤0.15°/ � � h √ r, and the sampling frequency was 125 Hz [37].
Overall, this section is organized as follows: (1) First subpart presented the setting up of the NAS-RNN, and the results of the NAS-RNN, the training loss, the standard deviation values, and attitude errors were employed as indicators for evaluating the performance of the NAS-RNN denoising tanh tanh σ σ σ

Forget gate Input gate
Output gate was set 100. Input date length was 15, which means using the past 15 gyroscope measurements to predict the current gyroscope outputs. After this, the predicted outputs were employed for compensating the random noise contained in the raw signals. Before applying the NAS-RNN to predict noise, a training procedure was carried out for determining appropriate parameters and generating proper neural network architecture in NAS-RNN. Figures 3, 5, and 6 list the training loss for the three-axis gyroscope outputs. It could be seen that the NAS-RNN training loss converged during the training procedure. In Figures 5(b), 3(b), and 6(b), the red line represents the NAS-    Mathematical Problems in Engineering RNN denoised gyroscope signals and the green line represents the raw gyroscope signals. Furthermore, Table 1 lists the comparison of standard deviation values (SDV) between NAS-RNN and the raw gyroscope signals. Compared with the raw signals, the SDV of the NAS-RNN results decreased by 44.0%, 34.1%, and 39.3%, respectively. is result demonstrated the e ectiveness of the NAS-RNN in this MEMS gyroscope noise suppressing application.

NAS-RNN vs. LSTM-RNN.
For further analyzing the NAS-RNN performance in MEMS gyroscope denoising, and in this subsection, the LSTM-RNN was compared with NAS-RNN for exploring whether there was a more feasible architecture for this application. Table 2 lists the comparison results between raw signals and the NAS-RNN. As listed in Table 2, the SDV decreased by 60.0%, 36.6%, and 44.6% compared with the raw signals. Furthermore, Table 3 lists the comparison between LSTM-RNN and NAS-RNN. Compared with LSTM-RNN, the NAS-RNN realized a further decrease in SDV. More speci cally, the three-axis SDV decreased by 28.6%, 3.7%, and 8.8% compared with LSTM-RNN.
In addition, Figures 7-9 present the training loss and testing loss comparison between LSTM-RNN and NAS-RNN. It could be seen that NAS-RNN converged to smaller values, but NAS-RNN performed at slower convergence speed. Actually, the NAS-RNN was more complicated than LSTM-RNN, and NAS-RNN had a more heavy computation    load. More specifications were introduced in Section 2.2. Table 4 lists the comparison of attitude errors; the attitude errors obtained 26.5%, 20.8%, and 16.4% improvement in the three-axis attitude errors compared with LSTM-RNN.

Discussion
e motivation of this paper was to explore more feasible RNN for MEMS gyroscope noise modeling and compensation, and we think the following problems need more discussion: (1) In this paper, the training procedure or processing data was limited by the computation capacity, and this might influence the generation capacity of the neural networks. (2) Only a STIM300 was employed in testing the proposed method; it was of significance for evaluating the performance of the NAS-RNN in different MEMS IMUs since different IMUs performed different noise characteristics.

Conclusions
In this paper, an advanced variant of RNN was employed in MEMS gyroscope noise modeling and compensating. A commercial MEMS IMU product STIM300 was employed in this experiment for evaluating the performance of the proposed NAS-RNN-based method. Compared with the raw signals, the three-axis NAS-RNN denoised signals performed 60.0%, 36.6%, and 44.6%, decreasing in standard deviation individually. In addition, compared with popular LSTM-RNN, the NAS-RNN obtained 28.6%, 3.7%, and 8.8% further decreasing for the three-axis gyroscope signals. Basically, the NAS-RNN was effective for this application, and it performed better than LSTM-RNN. We hope these results might provide reference for embedding a deep learning module in MEMS gyroscope for improving the accuracy of the MEMS gyroscope.

Data Availability
e source codes used to support the findings of this study are available from the corresponding author upon request.