Developing a non-routine maintenance load forecasting procedure in maintenance, repair and overhaul (MRO) XYZ company: A case study of B737NG aircraft

Turn Around Time (TAT) is one of the most important performance indicators in a Maintenance, Repair, and Overhaul (MRO) company. MRO companies need a high percentage of on-time TAT to compete in the industry. In 2019, there is a 29% difference between the planned and the actual TAT in MRO XYZ. Based on observations in MRO XYZ, there is no planning to perform non-routine maintenance. 54% of the total maintenance loads is a non-routine maintenance, therefore, it needs to be planned. The first step of the research is to identify routine maintenance tasks that dominate the non-routine maintenance loads based on ATA Chapter and CAMP Number using Pareto Analysis. The next step is to develop a procedure which determines the variables and the mathematical model to estimate the Non-Routine Ratio (NRR) workloads. The last step is to implement the procedure to obtain the NRR estimation model, which is also as the validation of the developed procedure. Several routine maintenance tasks dominate Non-routine maintenance loads are categorized by ATA Chapter, CAMP Number, and task types. Dominant ATA Chapters are ATA 53, 25, and 57. Dominant CAMP numbers are 53-140-00, 53-800-00, and 53-866-00. Dominant task categories are DVI for the system, internal GVI for structure, and external inspection for zonal. The NRR forecasting model for B737NG C-Check is composed of C-Check number, aircraft’s age, the ratio of ATA 53, the ratio of ATA 25, and the ratio of ATA 57. The NRR forecast model can be improved by adding some variables, such as Flight Hours (FH) and Flight Cycles (FC).


Introduction
Turn Around Time (TAT) is an essential performance indicator in the Aircraft Maintenance, Repair, and Overhaul (MRO) industry. In this research, TAT's definition is the aircraft's period to enter the MRO facility until the aircraft returns to service. MRO is expected to have a high percentage of on-time TAT to compete in the industry and gain customer trust. In 2019, there are 918 days (28,8%) of differences in Planned TAT and Actual TAT at one of the MRO in Indonesia for B737NG. B737NG is the combined type of Boeing 737-700/-800/-900. There are additional costs for both airline and MRO if the on-time TAT isn't achieved. If there is an additional TAT, the airline cannot utilize the aircraft for commercial activities, and the MRO cannot use the slot to maintain another customer's aircraft. One of many causes of the difference TAT is non-routine maintenance works that are regularly found in routine maintenance. The total of non-routine maintenance works is as much as 54% of the routine maintenance. However, IOP Publishing doi:10.1088/1757-899X/1173/1/012059 2 in MRO XYZ, there is still no procedure yet to forecast the non-routine maintenance workload. This condition is causing the non-routine maintenance cannot be well-prepared such as the material, manpower, and equipment. Therefore, it is needed to develop a procedure to forecast the non-routine maintenance load based on historical data.
The purpose of this research is to identify the routine maintenance tasks that dominate the non-routine maintenance works for C-Check B737NG, is to develop a procedure which determines the variables and the mathematical model to estimate the Non-Routine Ratio (NRR) workloads for C-Check B737NG, and the last is to implement the procedure as a part of the validation of the developed procedure. The methodology used in this research is formulating the research background and the problem, conducting a literature review about data science related to the problem, then preparing the data with Pareto Analysis, developing a non-routine maintenance forecast procedure, and testing the forecast procedure.

Materials and Methods
In this research, several literatures will be used to develop a procedure for forecasting non-routine maintenance load. This section will discuss aircraft maintenance type, a regression model with Bayesian Inference, and the tests that will be used.

Aircraft Maintenance
Routine maintenance is performed at regular intervals based on the Continuous Airworthiness Maintenance Program (CAMP). A CAMP is a list of all maintenance tasks belongs to an airline which is developed based on the Maintenance Review Board Document (MRB Doc) and has been approved by the regulatory authority [1]. Aircraft maintenance program is divided into three groups, namely the Systems, Structures, and Zonal Inspections.
Aircraft maintenance program consists of several task's types, for example Lubrication, Servicing, General Visual Inspection (GVI), Detailed Visual Inspection (DVI), etc. The interval of the maintenance tasks is clustered for the easiness of execution. Common clusters are the Daily Check, Pre-flight Check, Weekly Check, A-Check, C-Check, and D-Check. C-Check is categorized into heavy maintenance that the aircraft needs to go into a hangar and typically scheduled every 24 months.

Machine Learning
Machine Learning (ML) is one of the applications of Artificial Intelligence (AI). ML is an intersection of many fields of study such as computer science, engineering, statistics and other disciplines. Based on the learning process, ML is divided into two categories namely the Supervised and Unsupervised Learning. There are seven steps of machine learning, namely Collecting Data, Preparing Input Data, Analysing Input Data, Validating Data, Algorithm Learning, Algorithm Testing, and Applying knowledge. [3].

Multiple Linear Regression
Multiple Linear Regression (MLR) model is used to modelling the non-routine maintenance load. Multiple Linear Regression is used to forecast the relation between the dependent variable with more than one independent variable [4]. There are at least three objectives of developing a regression model: a. Understanding the behaviour of y, given x, b. Predicting y, given x and c. Predicting how y would change if x were changed.

Bayesian Inference
Bayesian Inference is one of the statistical inference methods which combine the observation data with the sample information. There are three main parts of the Bayesian Inference model, namely the Prior, Likelihood, and Posterior model. The prior distribution is the initial information about the parameters, Likelihood distribution is the probability of observed data given the parameters, and Posterior distribution is the probability of the parameters given the data. From Eq. (1), the results of a Bayesian Inference are Posterior proportional to the Likelihood, multiplied by the Prior. Therefore, the results of a Bayesian Inference are not an exact value but in a form of distribution.

Tests Performed
A Normality test is used to determine if the normal distribution can model the sample data well. Two methods to perform the Normality Test are Kolmogorov-Smirnov and Chi-Square [5]. A model is considered has a normal distribution if the Kolmogorov-Smirnov or Chi-Square value less than a specific value. A Multicollinearity Test is used to evaluate whether the data have a strong correlation. The Multicollinearity Test is performed by checking the value of Variance Inflation Factors (VIF). If VIF equals 1, then there is no correlation, but if VIF's value is more than five, then there is a strong correlation between the dependent variables [6]. There are many effects due to multicollinearity, such as standard errors for the estimated coefficient will be extremely large and cannot measure the variable precisely. Information Criterion is a value to estimate the quality of the model. Widely Applicable Information Criterion (WAIC) is used in the Bayesian Analysis. The chosen model is the one with the lowest value of WAIC [4].

Markov Chain Monte Carlo (MCMC)
MCMC is a method to approximate the posterior distribution by random sampling in a probabilistic space. Monte Carlo method is an algorithm that uses random sampling to simulate a process. Markov Chain method is a stochastic process that assumes the future states depend only on the current state.

Results and Discussions
This section discusses the evaluation in the non-routine maintenance work loads, the development of the non-routine maintenance load forecasting procedure, and the results of applying the load forecasting procedure. Aircraft Systems, with a total frequency of 159 times (40%). Internal General Visual Inspection (GVI) was dominating in Aircraft Structure with a total frequency of 519 times (71%). External Inspections and Internal Inspections were dominating in Aircraft Zonal Inspections based on the total frequency and the total manhours, respectively.

NRR Forecasting Procedure
In this forecast procedure, the WAIC and Multicollinearity theory will be used to obtain a high accuracy and a high precision, respectively. Figure 1 shows the developed NRR forecast procedure.  Figure 1. NRR Forecast Procedure NRR Forecast Procedure starts with processing the maintenance data which come from various sources and in different formats. The process eliminates the missing data, sort out the inconsistent values, and give variable label. The normality and multicollinearity test are then conducted to evaluate whether the data can be modelled as normal distribution and check the correlation between the independent variables. If the data can be modelled as a normal distribution, and there is no strong correlation between independence variables, then the data divided into a training dataset and testing dataset. The ratio of the training and testing dataset is 9:1. Then, to the training dataset is implemented by the Bayesian Inference and MCMC sampling. Convergence tests are conducted to evaluate whether the result is convergence. If the results are already in the convergence state, then a model testing is conducted by using the testing dataset.

NRR Forecasting Procedure
From the 2019's maintenance database, the available variables are the C-Check Number, Additional Maintenance, Aircraft's Age, Ratio of ATA 53, 25, and 57 to the total maintenance. C-Check Number is the Number of C-Check maintenance from C01 until C07. The aircraft's age is the aircraft's length of time since its first flight; the ATA ratio is the total maintenance loads of the ATA Chapter divided by IOP Publishing doi:10.1088/1757-899X/1173/1/012059 5 the total routine maintenance loads. The statistical summary of the maintenance database in the year 2019 is shown in Table 1. Variables are selected from the model with the lowest WAIC. The model with the lowest WAIC is the one with C-Check Number, Aircraft's Age, Ratio of ATA 53, 25, and 57. From the normality test, all variables can be modelled as normal distribution except ATA 25 and ATA 57. However, in this case, ATA 25 and 57 still will be modelled as the normal distribution based on the Central Limit Theorem's assumption. There is also no VIF value larger than 5 in each variable, it means that there is no strong correlation between independent variables. The Bayesian Inference method is used to obtain the NRR forecast distribution. The Bayesian Inference implementation was carried out using the PyMC3 library in Python. In this research, the MCMC algorithm used 2000 samples and two chains. PyMC3 used the No-U-Turn-Sampler (NUTS), the developed Hamiltonian Monte Carlo, that is more efficient to use. From all of the convergence test, the results are all variables which are already in a convergence state. The convergence condition can also be seen from the trace plot. If the chains are mixed, then it can be considered that the model is already convergent. Figure 2 shows the trace plot example of the variable intercept, the ratio of ATA 57, and 53.

Conclusions
Several routine maintenance tasks are dominant source of non-routine maintenance loads and categorized as ATA Chapter: ATA 53 (Fuselage), 25 (Equipment & Furnishing), and 57 (Wings); by CAMP Number: 53-140-00, 53-800-00, and 53-866-00; and by task type: DVI for System, Interval GVI for Structural, and External Inspection for Zonal Inspection. NRR forecast procedure for C-Check uses variables: C-Check Number, Aircraft's Age, Ratio of ATA 53, ATA 25, and ATA 57. The mathematical model as the result of NRR forecast procedure implementation for 737NG C-Check is: NRR = 0,157 + 0,280 x (Ration of ATA 57) ± 0,064. It is recommended that for MRO XYZ is to standardize the maintenance report, especially the maintenance activity description, the CAMP number reference, and manhours data to perform the routine and non-routine maintenance. For future research, it is necessary to add some variables, such as Flight Hours (FH) and Flight Cycles (FC), to obtain a more accurate and precise NRR forecast model.