Reliability Assessment and Robustness Study for Key Navigation Components using Belief Rule Based System

The gyro device is the key navigation component for maritime tracking and control, and gyro shift is the key factor which influences the performance of the gyro device, which makes conducting the reliability analysis on the gyro device very important. For the gyro device reliability analysis, the residual life probability prediction plays an essential role although it requires a complex process adapted by existed studies. In this study the Belief Rule Base (BRB) system is applied to model the relationship between the time as the input and the residual life probability as the output. Two scenarios are designed to study the robustness of the proposed BRB prediction model. The comparative results show that the BRB prediction model performs better in Scenario II when new the referenced values are predictable.


Introduction
As is known that the gyro device is a critical component for maritime navigation, and its residual life probability prediction is the main subject of its reliability assessment. Traditional residual life probability follows similar means [1] . First, suppose that certain distribution, such as the Weibull distribution, is followed by the degradation process. Second, estimate the parameters to model the residual life prediction model of the gyro device, in which the Wiener process is mostly applied. Third, update the parameters for the prediction model proposed in the second step. Fourth, use the Expectation-Maximization (EM) algorithm as well as the prior degradation data to estimate the value of the parameters.
The biggest challenge is that the gyro device has a very high reliability degree, which makes it almost impossible to collect enough data to conduct aforementioned reliability analysis. With this, the Belief Rule Base (BRB) expert system is applied in this study to model the residual life probability prediction process since BRB can model complex system behaviors under uncertainty [2] [3] .
In order to improve the prediction precision, a corresponding BRB parameter learning approach is applied, which determines the best fit parameters of the BRB prediction model. In BRB parameter learning, the Evidential Reasoning (ER) algorithm is applied as the inference engine and the Differential Evolutionary (DE) algorithm is applied as the optimization means. Due to its excellent performance versus the more expensive leave-one-out cross-validation, the ten-folder crossvalidation [4] is applied to validate the efficiency and robustness of the proposed parameter learning approach.
The metrics to validate the efficiency and robustness of the proposed parameter learning approach are the average/minimum/variance of the error, the average variance among the results of the tenfolder cross-validation, and the running time. Furthermore, the robustness is studied when the datasets are systematically or randomly sampled.
The remainder of this study is organized as follows. The BRB prediction model is introduced in Section 2. Section 3 and Section 4 give the reliability assessment result and robustness study result of the case study, respectively. This study is concluded in Section 5.

The Belief Rule Base
The BRB system is comprised of a series of rules in the same belief structure. The kth rule is described as: represents the referenced values of the ith attribute, M represents the number of the attributes, N represents the number of the scales, L represents the number of the rules,  Fig. 1 gives the BRB prediction model. Steps are given as following:

The BRB prediction model
Step 1: Random or systematic sample from the original total dataset; Step 2: Divide the observation data into the training dataset and the testing dataset; Step 3: Identify key referenced values; Step 4: Construct the new BRB prediction model; Step 5: Validation. For Strategy I, only the data in the training dataset is available and the referenced values are "blindly" selected. After selecting the referenced values, the BRB prediction model is constructed. For the validation part, the testing dataset is compared with the predicted result of the constructed BRB prediction model, and then the errors are summed up.
For Strategy II, new referenced values are allowed to be predicted rather than just those from the training dataset. First, group certain data from the training dataset by the number of the referenced values as previously set. And then, fit the linear equation using the grouped data and calculate the intersecting points. At last, the BRB prediction model is constructed using the intersecting points. For the validation part, the testing dataset is compared with the predicted result of the constructed BRB prediction model, and then the errors are summed up.
Details of the Differential Evolutionary (DE) algorithm and the Evidential Reasoning algorithm is not introduced in this study, readers can refer to [3] [5] and referenced therein for more information.

Case study: Reliability Assessment
In this section, the residual life probability prediction of the gyro device is solved by a BRB prediction model using the proposed parameter learning approach. The input is the time and the output is the residual life probability.
The 500 sets of data derived from observation are divided into two dataset. The training dataset consists of 450 sets of data while the testing dataset consists of the left 50 sets of data.
The case study is implemented in Matlab R2010b on Core (TM) i5-2450M CPU 2.50GHz with Windows 7 system. For each instances, 50 independent runs are performed. The population size of the DE algorithm is set to be 40 and the generation number is set to be 100.

Preparation
In order to effectively construct the BRB prediction model, the starting and ending referenced values are pre-determined. Normally it would be the first and last referenced values in the training dataset. However, there are multiple sets of data close to zero and do not require to be treated differently. For this specific case, the referenced values whose probabilities are the first and last above 0.1E-5, 7100 and 44700, are taken as the two ending referenced values of the BRB prediction model. Any referenced value beyond this range is considered to be zero.

Scenario I: only the training dataset is available
For Scenario I, it is supposed that only the data in the training dataset are available.
When the number of the referenced values is below five, all the possible combinations of the possible referenced values are enumerated. When the number of the referenced values is beyond five, the running time is too long for a complete enumeration. For Scenario I with any number of referenced values, the DE algorithm is applied and calculated.
As previously discussed, the samples are systematically selected. The comparison on the minimum/average/maximum error for Scenario I with different number of referenced values ranging from three to seven is shown as in Fig. 2, in which the running time is also presented. Although all the possible combinations of the referenced values are enumerated when there are no more than five referenced values, the corresponding running time is not added in Fig. 2 because it is not in the same order of magnitude with the time consumed by the DE algorithm.

Scenario II: predicting new referenced values
For the second scenario, new referenced values are allowed to be predicted.
The comparison on the minimum/average/maximum error for Scenario II with different number of referenced values is shown as in Fig. 3, in which the running time is also presented. Fig. 3 shows that the minimum/average/maximum error decreases with the increase of the number of the referenced values. When there are four referenced values, compared with seven referenced values of Scenario I as shown in Fig. 2, the minimum/average error per each testing data is 0.08/0.104 (E-05), which is rather low and acceptable.

Case study: Robustness study
The robustness study is conducted in two stages. First, the average/minimum/variance of the errors in 50 runs are compared in Section 4.1 and some preliminary conclusions are drawn. And then a more comprehensive discussion is held based on more statistics including the average variance of the tenfolder cross-validation and the running time.  Fig. 4 shows the comparison on the error in 50 runs in both scenarios. For the purpose of visualization, the 50 runs are arranged by the values of the metrics in ascending order.

Robustness study on error for fifty runs
(1) For the average/minimum error, there shows similar trends between systematic and random sampling means in both scenarios. It is clear that when the numbers of referenced values are the same, the values of random sampling and systematic sampling in each scenario are very close. This trend is less clear with the variance of error.
(2) For the variance of error in both scenarios, the biggest variance is shown with random samples and three referenced values. When there are more than three referenced values, the variances in both scenarios are rather low and become acceptable. Taking one step backward, even for the biggest variance in both scenarios, it does not affect the ability of the DE algorithm in identifying the optimal referenced values, which can be proved by the fact that the average/minimum error as shown in Fig.  4(a) are rather stable and remain at low values. This is because the size of the training dataset is rather big and there are always the optimal referenced values in it. (3) For all the metrics, the values of either metric in Scenario II are smaller than those in Scenario I. This is because new referenced values are predicted in Scenario II while only the data in the training dataset is available in Scenario I. New referenced values can be recognized as information extracted from the parameter learning process, and adding new information means less ambiguity and more precision.

Robustness study on the statistics between Scenarios I/II
In this section, a more comprehensive compassion on multiple statistics is conducted. A graphic demonstration of this comparison is shown as in Fig. 5. In Fig. 5(a-c), the bars represent systematic samples while the dots represent random samples. In Fig. 5(d), the bars represent Scenario I while the dots represent Scenario II. (1) For the average/minimum error as in Fig. 5(a), it is observed that the results are quite consistent within and between different and different sampling means. Comparatively the average/minimum of the error in Scenario II is smaller than that in Scenario I. This is caused by new referenced values added in Scenario II.
For the values of the average/minimum error, when there are seven referenced values in Scenario I and four referenced values in Scenario II, the values of the two metrics are around 0.075/0.01 (E-05) per each set of data, which is fairly low and acceptable.
The consistency and the low values of the average/minimum error are the direct evidence for the robustness of the parameter learning approach.
(2) The variance of error and the average variance of ten-folder cross-validation as in Fig. 5(b/c) show the same trends. The following is explained from the perspective of different sampling means.
For systematic samples, both metrics remain rather low, below 1.5, yet shows no distinguished trends. Interestingly, there is an abnormal peak when there are five referenced values. This is because there are multiple local optimal combinations of five referenced values and the DE algorithm cannot escape from local optimality in order to identify the global optimal solution. When the samples are systematic, these local optimal solutions "randomly" show up and this would result in a peak in the variance.
For random samples, there is a decreasing trend with the increase of the number of referenced values. When there are over four referenced values, the two metrics would be within 3 and become acceptable. The advantage of random sampling is its ability to demonstrate the improvement of the proposed approach. This is reflected in the decreasing trend of the values of both metrics which is not observed when the samples are systematic.
Note that even for the peak of the systematic samples, the values of the two metrics are far less than those of the random samples. This is because systematic samples are equally distributed and tend to produced smaller errors.
(3) For the running time as shown in Fig. 5(d), the only decisive factor is the number of referenced values while the sampling strategy is not relevant. In both scenarios, the running time grows linearly along with the increase of the number of referenced values while they are not in the same order of magnitude.

Conclusion
The residual life probability prediction of the gyro device is crucial for the reliability analysis of the navigation system which is an important device for the thermonuclear fusion. The BRB prediction model is used to model this prediction process since the traditional means and approaches are rather (1) In Scenario I with no more than five referenced values, all the possible combinations of the referenced values are enumerated. The optimal referenced values of the enumeration are identical with the ones of the DE algorithm.
(2) In both scenarios, the average/minimum error decreases along with the increase of the number of referenced values.
(3) The running time in both Scenario I/II grows linearly along with the increase of the number of referenced values while Scenario II consumes much longer time than Scenario I.
The robustness of the proposed approach is studied as well. Certain conclusions are drawn as following: (1) For the average/minimum error, the consistency between both systematic/random samples and the two scenarios is observed.
(2) For the variance of the error and the average variance of ten-folder cross-validation, except when there are three referenced values with random samples, the values of both metrics are below 3 in Scenario II and below 1.5 in Scenario I.
Since the residual life probability prediction of the gyro device requires one and only one attribute/scale, it actually simplifies the parameter learning process. However it is believed that the principle of the proposed parameter learning approach is transferable when there are multiple attributes/scales. This leads to future work that needs to be done: the extension of the parameter learning approach of the BRB prediction model in other practical cases studies with multiple attributes/scales.