A Hybrid of NARX and Moving Average Structures for Exhaust Gas Temperature Prediction of Gas Turbine Engines

Abstract

Aiming at engine health management, a novel hybrid prediction method is proposed for exhaust gas temperature (EGT) prediction of gas turbine engines. This hybrid model combines a nonlinear autoregressive with exogenous input (NARX) model and a moving average (MA) model. A feature attention mechanism-enhanced long short-term memory network (FAE-LSTM) is first developed to construct the NARX model, which is used for identifying the aircraft engine using condition parameters and gas path measurement parameters that correlate to the EGT. A vanilla LSTM is then used for constructing the MA model, which is used for improving the difference between the actual EGT and the predicted EGT given by the NARX model. The proposed method is evaluated using real flight process data and compared to several dynamic prediction techniques. The results show that our hybrid model reduces the predicted RMSE and MAE by at least 13.23% and 18.47%, respectively. The developed FAE-LSTM network can effectively deal with dynamic data. Overall, the present work demonstrates a promising performance and provides a positive guide for predicting engine parameters.

1. Introduction

Gas turbine engines are utilized extensively in various industries, including commercial aviation, military fleets, land and marine propulsion, oil pumping stations, and power generation stations. Due to the non-stationary operating conditions, component degradation, and maintenance actions, gas turbine engines are highly complex and dynamic machines [1]. The engines require high levels of safety and reliability, which result in increased maintenance costs. An effective diagnostic and prognostic system can ensure engine safety, reduce maintenance costs, and minimize the risk of catastrophic events [2].

An engine gas path fault diagnosis typically determines engine health conditions based on the monitored gas path parameters, such as rotational speed, temperature, pressure, flow rate, and others. The core concept of fault diagnosis is to monitor the engine’s performance by comparing the current actual values of the monitored parameters with those values under healthy conditions. Therefore, the prediction models of gas path parameters under the engine health state are essential and widely investigated by many researchers. The construction of the prediction model can be divided into two classes: model-based and data-driven approaches [3,4,5,6].

The model-based methods involve building mechanical models that capture the physics of engines and engine failure, which do not rely on historical condition monitoring data. Thermodynamic models are critical for modern model-based control and engine health management of aircraft engines. Adaptive onboard engine models can effectively deal with engine degradation and have become one of the most popular engine control methods. For the prognostic problem, Pratt & Whitney proposed a diagnostic system called the enhanced self-tuning onboard real-time model, based on a modified Kalman filter and an adaptive onboard linear model, which has been applied to the PW6000 engine [7]. Additionally, many nonlinear methods have been proposed to address the nonlinear problems, including the extended Kalman filter (EKF) [8], unscented Kalman filter (UKF) [9], and particle filter algorithm [10,11]. Although model-based approaches offer flight-dependent model accuracy, most physical models can only cover a limited range of the engine’s operating conditions. Moreover, only a small portion of the degradation and failure modes of the engine are well understood, and most of the degradation and failure mechanisms cannot be fully understood by the physical models. Thus, the practical application of physical model-based methods is limited.

Data-driven methods rely on historical condition monitoring data, which means that the parameter prediction models are constructed based on the machine learning method. The increased availability of engine condition monitoring data has driven the broader use of data-driven approaches for the prognostics and health management (PHM) of aircraft engines. Due to the dynamic characteristics of the engines, dynamic neural networks are suitable for constructing engine parameter prediction models. Tayarani-Bathaie and Khorasani [12] proposed a fault detection and isolation (FDI) approach based on a dynamic neural model and time delay neural networks. The networks are used to learn the dynamics of the engine and predict the performance parameters. An FDI strategy for a nonlinear system based on a bank of recurrent neural networks (RNN) was proposed by Shahnazari [13]. The RNN is used as the plant model to produce deviations of the performance parameters with the actual measurement values. In his study, the FDI system can diagnose single, multiple, and simultaneous actuator and sensor faults. Bai et al. [14] proposed an anomaly detection method based on a nonlinear autoregressive with exogenous input (NARX) network. The NARX nets are used to extract the features of the engine’s normal pattern. Ibrahem et al. [15] proposed a real-time modeling method for a three-spool aero-engine based on an ensemble of RNNs. This ensemble technique can deal with the poor generalization problem of a single network. From the above research results, it can be seen that the data-driven modeling method is becoming increasingly useful.

Among all the parameters, exhaust gas temperature (EGT) is the key condition monitoring parameter for prediction, optimization, and condition monitoring. An increase in EGT is a typical sign of gas turbine engine deterioration. The change in EGT is widely used in deterioration detection, engine remaining useful life prediction, and the implementation of condition-based maintenance. Therefore, EGT prediction is very useful in engine PHM. Zhang and Dong [16] proposed an EGT prediction method based on autoregressive integrated moving average (ARIMA) models. The results show promising precision. A statistical and artificial intelligence approach was proposed for EGT prediction of a micro gas turbine by Koleini et al. [17]. The results show that both the artificial neural network (ANN) and multiple polynomial regression (MPR) approaches demonstrated good predicting capability for exhaust gas temperature using data gathered by an experimental setup of a micro gas turbine engine with a rotation speed range of 0~108,000. Zhou [18] used simple machine learning methods such as multilayer perception and support vector regression to predict EGT. The machine learning methods are optimized by an adaptive particle swarm algorithm, which can help select the hyper-parameters of the machine learning models, effectively. Ullah et al. [19] proposed an EGT prediction approach based on a long short-term memory (LSTM) network. The input features were recognized as a real-time series.

In recent years, LSTM network models based on attention mechanisms have been successfully applied in multivariate time series prediction [20]. This model embeds attention mechanisms into LSTM to enhance the prediction performance of LSTM. Qin et al. [21] developed dual attention-based recurrent neural networks (DA-RNN) to predict time series, which show a significant improvement over traditional RNNs. Liu et al. [22] proposed a dual-stage two-level attention cycle network (DSTP-RNN) to achieve long-term predictions of multivariate time series. It can be seen that RNN networks incorporating attention mechanisms can more effectively achieve time series prediction.

Hybrid model prediction methods, which are powerful tools for time-series prediction, have been investigated for some years. Pham et al. [23] presented an improvement of the hybrid NARX model and autoregressive moving average (ARMA) model for long-term machine state forecasting based on vibration data. Similarly, Cho et al. [24] proposed a hybrid attention-based LSTM and ARMA model for tomato yield forecasting. A hybrid ARIMA and NARX model for forecasting long-term daily inflows to the Dez reservoir using North Atlantic Oscillation and rainfall data was presented in [25]. A hybrid approach based on ARIMA and least-squares support vector machines (SVM) for long-term forecasting of net electricity consumption was presented in [26]. For these hybrid models, one part is used to forecast the deterministic component, and another part is used to predict the error component.

As evident from the overview presented above, data-driven parameter prediction or EGT prediction problems can be viewed as time-series prediction problems. Despite significant progress in the aforementioned research, several issues still require attention. Firstly, most of the experimental data reported in the published research is limited to simulated or steady-state data of gas turbine engines. Real flight data and flight process data have not been explored sufficiently. Secondly, the use of single machine learning models has its limitations in time series regression problems, particularly for long-term prediction problems. Aiming at these issues, this paper proposes a hybrid NARX and moving average (MA) structure method for EGT prediction of gas turbine engines, evaluated using real flight process data. The NARX structure is constructed based on a feature attention-enhanced LSTM network (FAE-LSTM) inspired by the attention mechanism, which is used for predicting long-term EGT. The MA structure is constructed based on a vanilla LSTM network, which gives us the prediction error of the NARX structure. The main contribution of this work can be summarized as follows:

• An improved LSTM network, i.e., FAE-LSTM, is developed to construct the NARX structure for the long-term prediction of EGT.
• A novel hybrid prediction model is developed by combining the NARX and moving average structures, for the first time in the literature, for EGT prediction of gas turbine engines.
• A real flight process dataset is used to evaluate the proposed method, which improves the high practical value of the proposed method.

2. Methodology

2.1. The Feature Attention-Enhanced-LSTM-Based NARX Structure

The NARX model is a type of artificial neural network used in time series analysis and prediction. It is a nonlinear extension of the classical autoregressive (AR) model that takes into account the effects of exogenous input variables. The NARX model was first introduced in the early 1990s as an extension of the linear AR model [27,28]. The main advantage of the NARX model over the AR model is its ability to model nonlinear relationships between input and output variables. One application scenario of the NARX model is in time series prediction, where the model is used to forecast the future values of a time series based on its past values and exogenous input variables. Another application of the NARX model is in system identification, where the model is used to estimate the parameters of a dynamic system based on its input–output data. Generally, NARX models are used both in parallel mode and series-parallel mode.

In the parallel mode, the delayed outputs of the network are fed back to the feed-forward network as part of the standard NARX model:

$${\widehat{y}}_t=f(u_{t-n+1},\dots ,u_t,{\widehat{y}}_{t-n},\dots ,{\widehat{y}}_{t-1})$$

(1)

In the series-parallel mode, the delayed outputs of the real system enter the input of the NARX model:

$${\widehat{y}}_t=f(u_{t-n+1},\dots ,u_t,y_{t-n},\dots ,y_{t-1})$$

(2)

Both models (1) and (2) are employed in this study. Model (2) is utilized to train the NARX model using training data collected from engine health states. Since NARX prediction models are primarily utilized in prognostic or fault diagnostic systems, the actual output of a real system may be influenced by faults or degradations, resulting in the delayed output of the real system being influenced by the faults to the degradations. Therefore, the series-parallel mode model (2) is unsuitable for prediction. Instead, for testing purposes, the parallel mode model (1) is used, which is referred to as long-term prediction in this study.

In this study, we develop a novel engine parameter prediction model, termed FAE-LSTM, which combines feature attention (FA) and LSTM networks to construct the NARX model. The FAE-LSTM model architecture is depicted in Figure 1a and consists of three main components: an encoder, a feature concatenate layer, and a decoder. Specifically, the encoder module focuses solely on encoding the exogenous features, u, outlined in model (2) while leveraging the FA structure to learn the intercorrelations between the characteristics of these exogenous features. By doing so, it effectively extracts the encoded feature sequence, h. Subsequently, the delayed target features $\{y_{t-n}^{},\dots ,y_{t-1}^{}\}$, the encoder outputs $\{h_{t-n+1}^{},\dots ,h_t^{}\}$, and the original exogenous features $\{u_{t-n+1}^{},\dots ,u_t^{}\}$ are concatenated along the feature dimension and fed into the decoder module through the concatenate layer. The decoder unit is made up of LSTM units that are capable of predicting the performance parameters by learning the temporal correlations in the input time series. The details of our FAE-LSTM will be presented in the following paragraphs.

For the encoder, the FA unit is a special attention mechanism module. Unlike the traditional attention mechanism, FA combines the recurrent structure of LSTM and can dynamically weight the entire input sequence along the time dimension, as shown in Figure 1b. In order to describe the calculation process of the FA unit more clearly, the update process of the LSTM unit is introduced briefly. For LSTM units, the updates of the hidden state at the current time, $h_t$, and the cell state at the current time, $c_t$, can be summarized as follows:

$$c_t=f_t\ast c_{t-1}+i_t\ast g_t$$

(3)

$$h_t=o_t\ast \tanh \left(c_t\right)$$

(4)

where $i_t=\sigma \left(W_i[x_t;h_{t-1}]+b_i\right)$, $f_t=\sigma \left(W_f[x_t;h_{t-1}]+b_f\right)$, $g_t=\tanh \left(W_g[x_t;h_{t-1}]+b_g\right)$, and $o_t=\sigma \left(W_o[x_t;h_{t-1}]+b_o\right)$ denote the input gate, forget gate, candidate gate, and output gate, respectively. The symbol * represents the element-wise multiplication. $W_i$, $W_f$, $W_g$, $W_o$, and $b_i$, $b_f$, $b_g$, $b_o$ are parameters to learn. For simplicity, the update process of (3) and (4) is expressed as:

$$[h_t,c_t]=\mathrm{lstm}(h_{t-1},x_t)$$

(5)

As mentioned earlier, the FAE-LSTM considers both the time dimension and the feature dimension. For multivariable input series (exogenous series), the k-th series of sequence length, T, is expressed as $u^{(k)}=(u_1^{(k)},u_2^{(k)},\cdots ,u_T^{(k)})$. Then, we can construct an attention mechanism via the following multiple-layer perception (MLP) model:

$$z_t^{\left(k\right)}=v_t^{\mathrm{T}}\tanh \left(W_z[h_{t-1}^{};c_{t-1}^{}]+V_z{u}^{(k)}+b_z\right)$$

(6)

where $W_z$, $V_z$, and $b_z$ represent the learnable parameters. $h_{t-1}^{}$ and $c_{t-1}^{}$ are the hidden state and cell state of the LSTM cell, respectively. In order to measure the importance of the k-th input feature, the attention weight, ${\alpha }_t^{(k)}$, is calculated. In this study, we use the Softmax function to get the attention weight, which can ensure the sum of the attention weights is 1. The Softmax function can be expressed as follows:

$${\alpha }_t^{\left(k\right)}=\exp \left(z_t^{\left(k\right)}\right)/\left({\displaystyle \sum _{i=1}^m\exp }\left(z_t^{\left(i\right)}\right)\right)$$

(7)

After the attention weights are obtained, the output of the attention mechanism is defined as follows:

$${\tilde{u}}_t={[{\alpha }_t^{(1)}{u}_t^{(1)},\cdots ,{\alpha }_t^{(m)}{u}_t^{(m)}]}^{\mathrm{T}}$$

(8)

Then, the output of the attention mechanism, ${\tilde{u}}_t$, is used as the input of the LSTM cell. The output of the FA module is the hidden state, $h_t$, according to:

$$[h_t,c_t]=\mathrm{lstm}(h_{t-1},{\tilde{u}}_t)$$

(9)

The final outputs of the encoder of FAE-LSTM are achieved by cycle calculating (6)–(9) along the time dimension of the input series.

The role of the concatenate layer is to fuse the output features of the encoder with other features and serve as the input of the decoder. In this study, the output of the encoder is fused with historical target parameters and raw exogenous features in the feature dimension through the concatenation layer to obtain a more enriched decoder input. The decoder is a traditional LSTM network that utilizes the long-term memory capability of the LSTM cell for time series and learns the fused features in the time dimension to obtain the final performance parameter prediction. Compared with advanced dynamic models such as DA-RNN and DSTP-RNN, FAE-LSTM is structurally simpler, and the network training is simpler and more stable due to the fusion of raw exogenous features through the concatenate layer. Compared with the traditional NARX model, FAE-LSTM adds FA units and enhances the input sequence in the feature dimension.

2.2. Vanilla LSTM-Based Moving Average Model

The MA model is a statistical time series model that is commonly used in analyzing and predicting trends in data [29]. It is a relatively simple model that assumes that the value of a time series at any point in time is a function of the average of past values of the series, with the weights of the past values determined by the model’s parameters. Unlike the AR model, the MA model does not consider the past values of the series themselves but, instead, uses their weighted average to model the current value. The MA model is commonly used in finance and economics to analyze and predict trends in stock prices, commodity prices, and other financial and economic data. It is also used in engineering and other scientific fields to analyze and predict trends in physical systems. The formula of the MA model is shown below:

$$y_t=f(u_{t-n},\dots ,u_t)$$

(10)

This study aims to develop an MA model to address the error parameter produced by the NARX model. Although NARX models trained under the series-parallel mode effectively predict EGT, significant testing errors still arise under the parallel mode, especially in the presence of sudden changes in operating conditions. This is due to insufficient sudden-change data in the training data, leading to larger testing errors. Furthermore, the autoregressive structure of the NARX model allows for error propagation under the parallel mode. As the prediction error of the NARX model is highly correlated with the variation of the condition parameters, the MA model inputs are selected as the difference of operating condition parameters. The formula can be expressed as follows:

$$y_t=f(\Delta u_{t-n},\dots ,\Delta u_t)$$

(11)

where $\Delta u_t=u_t-u_{t-1}$ denotes the difference value of the engine parameters.

The prediction problem using Equation (11) can be viewed as a time series regression problem. Similar to the NARX model, a dynamic neural network is used to construct the MA model. As the MA model is simpler and more stable than the NARX model, a vanilla LSTM is selected to construct the MA model in this paper.

2.3. Improved Hybrid Model for EGT Prediction

Due to insufficient training data, networks trained under the series-parallel mode often exhibit large errors in long-term prediction, particularly when there are sudden changes in the condition parameters. To address this issue, we propose a hybrid prediction method that combines the NARX and MA structures. Specifically, the FAE-LSTM model is utilized to construct the NARX model, which roughly predicts EGT. Then, the difference between the predicted value of FAE-LSTM and the actual observed EGT value can be corrected using a vanilla LSTM-based MA model. The hybrid prediction method includes the following five steps:

Step 1: Collect training data and select appropriate exogenous gas path parameters as the input and EGT as the output of the FAE-LSTM.

Step 2: Train the FAE-LSTM network using the collected training data under the series-parallel mode, ${\widehat{y}}_{t+1}=f(y_t,\dots ,y_{t-n},u_{t+1},\dots ,u_{t-n+1})$.

Step 3: Run the trained FAE-LSTM on the training data under the parallel mode and obtain long-term prediction results. Calculate the error between the predicted value and the actual value, i.e., $e=y-\widehat{y}$.

Step 4: Prepare the training data for the MA model. First, calculate the difference value of the input features as inputs of the MA model. Set $e=y-\widehat{y}$ as the output of the MA model. Construct the MA model using a vanilla LSTM network.

Step 5: Add the FAE-LSTM prediction results to the prediction results of the MA model to obtain the final prediction value.

Figure 2 illustrates the design procedure for the proposed hybrid model. As can be seen in this figure, the NARX model is first trained, and the MA model is trained based on the acceptable NARX model. After the NARX and MA models are trained, the hybrid model can be used to predict the long-term EGT values according to the testing procedure in Figure 2. It should be noted that the inputs of the MA model are the differential values of the raw operating parameters of the engine.

3. Experiment Settings

3.1. The Flight Dataset

The flight dataset used in this paper comes from the QAR (quick access recorder) data recorded by a commercial aircraft to evaluate the proposed method. QAR data are an essential component of modern aviation. They refer to the digital recordings of various flight parameters and system information generated by an aircraft’s sensors and systems during flight operations. These data are captured by a QAR device installed on the aircraft, and they can be used for various purposes, such as maintenance and safety analysis, flight performance monitoring, and incident investigation. QAR data are an essential tool for airlines, maintenance crews, and regulatory authorities to monitor the performance and safety of aircraft. They can help identify potential issues before they become significant problems and improve operational efficiency.

The dataset used in this study records the data of the aircraft engine during the climb, cruise, and landing phases of each flight mission. This dataset contains the continuous process of the engine in use, including the transient and steady-state processes. Compared with steady-state process prediction, the prediction of transient states is more complicated.

This QAR dataset records more than 200 different parameters, and the sampling frequency of the condition parameters is 4 Hz. An example of the engine operating parameters (scenario descriptors) of a flight cycle is shown in Figure 3. The scenario descriptors contain the flight altitude (ALT), Mach number (MN), the power lever angle (PLA), and the ambient temperature (T0). As seen in this figure, the operating parameters are not stable, especially the PLA.

Among more than 200 recorded parameters, we selected the parameters that can reflect the gas path performance of the engine according to [30,31]. The scenario descriptors, i.e., ALT, MN, PLA, and T0, determine the flight condition and are necessary for EGT prediction. Other gas path parameters, such as rotating speeds, temperatures, or pressures, can improve the prediction precision. The parameters used in this paper are shown in

Table 1. The selected parameters of the aircraft engine.

#	Symbol	Description	Unit
1	ALT	Flight altitude	Ft
2	MN	Mach Number	-
3	PLA	Power lever angle	%
4	T0	Ambient temperature	°R
5	Wf	Fuel flow	pps
6	N1	Fan speed	rpm
7	N2	Core speed	rpm
8	EGT	Exhaust gas temperature	°R

. The first seven parameters in Table 1 are thought to be correlated with EGT, and the last parameter, EGT, is the object parameter to be predicted.

Before constructing the prediction model, the origin flight data need to be preprocessed. We choose the flight data according to the flight mission time; the flight mission time from 1.5 h to 2 h is used in this study. There are a total of 46 flight cycles in this study, where 40 cycles are selected as the training dataset and 6 cycles are selected as the testing dataset. The details of the dataset are shown in

Table 2. The details of the studied datasets.

Dataset	No. of Cycles	No. of Samples
Training	40	299,144
Testing	6	41,036

.

Figure 4 shows Pearson’s linear correlation coefficient (PCC) of the selected measurement parameters [32]. PCC is the most commonly used linear correlation coefficient, which is calculated through ${\rho }_{XY}=\mathrm{cov}(X,Y)/({\sigma }_X{\sigma }_Y)$, where ${\sigma }_X$ and ${\sigma }_Y$ denote the standard deviations of X and Y, and $\mathrm{cov}(X,Y)={\displaystyle {\sum }_{i=1}^n(X_i-\overline{X})(Y_i-\overline{Y})}/(n-1)$ is the covariance of X and Y. As can be seen in this figure, PLA, Wf, N1, and N2 are strongly positively correlated to EGT, which means that they are likely to be very helpful for predicting EGT. ALT, MN, and T0 are the operating condition parameters, which are also important for EGT prediction, even though they are not as strongly correlated to EGT as the other parameters.

3.2. Network Settings

The hybrid prediction model utilized in this study requires two distinct networks, an FAE-LSTM, and a vanilla LSTM, to achieve optimal prediction performance. To evaluate the effectiveness of our hybrid model, we selected four up-to-date dynamic networks as the baseline models. All the baseline models employ the series-parallel structure for training and the parallel structure for testing. The following settings are used for the four baseline models:

NARX-NN: NARX neural network comprising two hidden layers, each containing 100 neuron cells. The activation function of each layer is the rectified linear unit (ReLU) function.

LSTM: LSTM network utilizing the NARX structure. The network comprises two LSTM layers, each containing 100 neuron cells.

DA-RNN: The dual-stage attention-based recurrent neural network. The RNN network is selected as the LSTM according to the original paper, and the number of neuron cells in the LSTM layer is 100.

DSTP-RNN: The dual-stage two-phase attention-based recurrent neural network. The RNN network is selected as the LSTM according to the original paper, and the number of neuron cells in the LSTM layer is 100.

For all the network models, the Adam optimizer is utilized to update the network parameters during the training process. The training mini-batch value is set to 1024, and the training period is 500 cycles. The initial learning rate for the network training process is 0.005, and the learning rate is halved every 100 training cycles. Prior to training, the data are normalized using the min-max normalization method to a range of [−1, 1]. These rigorous settings ensure a reliable and accurate comparison of the hybrid model with the four baseline models.

3.3. Evaluation Metrics

In this paper, root-mean-square error (RMSE) and mean absolute error (MAE) are used to evaluate the prediction accuracy of the model. These two metrics can be expressed as follows:

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{({\widehat{y}}_i-y_i)}^2}}$$

(12)

$$\mathrm{MAE}=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|{\widehat{y}}_i-y_i\right|}$$

(13)

where ${\widehat{y}}_i$ and $y_i$ represent the predicted value and the actual measured value, respectively, and N is the number of sample points.

4. Results and Analysis

4.1. Comparision Results

All the models are trained using the series-parallel structure on the training dataset and then tested using the parallel structure on the testing dataset. Since the network models may suffer high variance during the training procedure because of the initialization of the learnable parameters, all the models are trained over five different random seeds, and the best one on the training dataset is chosen. The sequence length of the network models is set to 16. The sequence length of the error regression model of our hybrid model is set to 20. The prediction results on the testing dataset of the six different models are shown in

Table 3. Prediction performance of all the models.

Models	RMSE	MAE
NARX-NN [15]	0.0709	0.0621
LSTM [19]	0.0451	0.0364
DA-RNN [22]	0.0516	0.0296
DSTP-RNN [21]	0.0469	0.0408
FAE-LSTM	0.0378	0.0287
Our hybrid model	0.0328	0.0234

. As can be seen in Table 3, our hybrid model has greatly improved the long-term prediction performance. Compared to the single models, the RMSE and MAE of the proposed hybrid model are reduced by at least 13.23% and 18.47%, respectively.

It can also be seen that the developed FAE-LSTM has the best testing performance among all the single network models. The prediction results of DA-RNN and DSTP-RNN using the dual-stage attention mechanism are not as good as those of the LSTM. The reason may be that the two-stage attention mechanism greatly increases the complexity of the network and is not conducive to network training. The feature attention module and the concatenate layer in our FAE-LSTM can improve the long-term prediction performance effectively.

Figure 5 illustrates the predicted error of all the models. There are six subgraphs in Figure 5, and each subgraph corresponds to a flight cycle in our testing dataset. The predicted errors are obtained by calculating the difference between the actual value and the predicted value. As can be seen in this Figure, NARX-NN shows the poorest performance; the reason is most likely that the NARX-NN is sensitive to the order of input and output delays. Our hybrid model has the best error performance, with no excessive prediction error in most flight phases. It can also be seen that our hybrid model doesn’t always perform best, as in the cruise phase in Figure 5f; the reason is that the predicted value of the MA model has a different sign to the real value, resulting in a larger error in the final results.

As mentioned before, the MA model is used to predict the prediction error of the NARX model. Figure 6 shows the prediction results of the MA model on the six flight cycles in the testing dataset. In Figure 6, the black line represents the real value, which is the prediction error of the FAE-LSTM model in Figure 5. The red lines represent the prediction results of the MA model. As can be seen in Figure 6, the MA model can predict the trend of the real error successfully.

Due to the good predictive performance of the NARX model, the true error value is small. As can be seen in Figure 6, most of the real errors are near zero. Therefore, if the prediction error of the MA model is slightly larger, it will cause the predicted error and the true error to have different signs, thereby decreasing the final EGT prediction performance. Due to the fact that the true error is calculated based on the NARX model, there are inevitably some uncertainties, resulting in unclear correspondence between the input and output of the MA model, which inevitably leads to inaccurate prediction results of the MA model. Based on the above analysis, it can be seen that when the true error of the MA model is small, it will not always improve the final prediction results and may even be worse than the prediction performance of a single model. The performance improvement is mainly reflected in the part of the NARX model with larger errors.

4.2. Analysis of the Network Structure of the Single FAE-LSTM Model

In order to verify the effectiveness of the proposed FAE-LSTM network, two experiments are carried out in this subsection.

In the first experiment, we explore the influence of various network structures on the RMSE performance of the FAE-LSTM model, including the sequence length and neuron cell number. Each network is trained over five different random seeds, and the mean and variance of the RMSE performance are demonstrated in Figure 7, representing the long-term performance on the test dataset. As illustrated in Figure 7a, the sequence length significantly affects the RMSE performance, where shorter sequence lengths result in higher mean and variance values. Notably, sequence lengths greater than 14 yield stable performance, with the optimal sequence length being 16. As presented in Figure 7b, the cell number of the attention layer in the FAE-LSTM has less impact on the long-term performance than the sequence length. The network performance is best when the number of neuron cells is 40, and, as the number of neurons increases, the variance of the network prediction results decreases.

From the observed results, it can be concluded that the FAE-LSTM is not sensitive to the input sequence length (order of the delayed input and output signals) and the number of neuron cells. These findings underscore the significance of selecting appropriate network structures to optimize the FAE-LSTM model’s performance, which is crucial for reliable and accurate predictions.

The second experiment is used to analyze the impact of the network modules on performance, including the FA module and the concatenate layer module. The network inputs without the FA module contain the original input features, $\{u_{t-n+1}^{},\dots ,u_t^{}\}$, and the delayed output features, $\{y_{t-n},\dots ,y_{t-1}\}$; the network is downgraded to an LSTM network. The network inputs without the concatenate layer module contain the outputs of the FA module, $\{h_{t-n+1}^{},\dots ,h_t^{}\}$ and $\{y_{t-n},\dots ,y_{t-1}\}$. FAE-LSTM, FAE-LSTM without FA module, and FAE-LSTM without the concatenate layer are trained over five different random seeds; the means and standard deviations of the testing performance are shown in

Table 4. Predicted results of different networks on testing dataset (mean ± σ).

Models	RMSE	MAE
Without FA	0.0513 ± 0.021	0.0388 ± 0.013
Without concatenate layer	0.0464 ± 0.023	0.0312 ± 0.019
FAE-LSTM	0.0457 ± 0.012	0.0292 ± 0.018

.

As can be seen in Table 4, the proposed FAE-LSTM has the best predictive performance. The network performance without the FA module is the worst, and without cascading layers; the network performance with the FA module is slightly inferior to FAE-LSTM. It can be concluded that the FA module and the concatenate layer can both improve the predictive performance of the network. By effectively obtaining information between features, the FA module has a greater improvement on the network than the concatenate layer.

4.3. Impact of the Input Features on the FAE-LSTM Performance

The results presented in the previous sections demonstrate that the proposed hybrid approach provides very good performance in terms of long-term EGT prediction. An ablation experiment is implemented to better understand how the different input features affect the prediction performance of FAE-LSTM, which can also provide some guidance for selecting sensors for different tasks. For this experiment, we referred to the analytical techniques in making aircraft engine health management sensor selection decisions provided in [33].

In this experiment, the scenario descriptors [ALT, MN, T0, PLA] are set to be the baseline features, which determine the flight operating conditions. The other sensors [Wf, N1, N2] are the additional input features. The input feature selection decisions for the prediction performance are shown below:

• Baseline,
• Baseline+ 1 additional sensor,
• Baseline+ 2 additional sensors,
• Baseline+ 3 additional sensors.

The presented results in

Table 5. EGT prediction performance of FAE-LSTM with different input features.

Inputs	RMSE
[ALT, MN, T0, PLA]	0.224 ± 0.143
[ALT, MN, T0, PLA, Wf]	0.174 ± 0.137
[ALT, MN, T0, PLA, N1]	0.174 ± 0.137
[ALT, MN, T0, PLA, N2]	0.214 ± 0.059
[ALT, MN, T0, PLA, Wf, N1]	0.047 ± 0.008
[ALT, MN, T0, PLA, Wf, N2]	0.180 ± 0.124
[ALT, MN, T0, PLA, N1, N2]	0.091 ± 0.019
[ALT, MN, T0, PLA, Wf, N1, N2]	0.046 ± 0.012

depict the evaluation of the proposed FAE-LSTM network’s performance on the testing dataset, where diverse input features have been employed. As shown in Table 5, utilizing only the baseline features leads to a considerable RMSE, indicating unsatisfactory prediction outcomes. However, the integration of Wf and N1 as input features yields a substantial improvement in the prediction performance, highlighting the importance of informative input features. Conversely, the incorporation of N2 as an input feature yields only a marginal improvement in performance. The reason is that N1 is the feedback signal, and Wf is the actuating signal of the engine control system, which can provide more information, while N2 is the output signal and has a strong correlation with PLA, as shown in Figure 4, resulting in feature redundancy. These findings underscore the critical role of appropriate input feature selection in achieving optimal predictive ability for the FAE-LSTM network.

5. Conclusions

In this paper, a novel prediction structure for EGT prediction of gas turbine engines is proposed. This prediction method combines a nonlinear autoregressive with exogenous inputs (NARX) structure and a moving average (MA) structure. A feature attention-enhanced LSTM (FAE-LSTM) is first proposed to construct the NARX model for a rough EGT prediction; then, a vanilla LSTM is used to construct the MA model for error prediction. The performance of the proposed method is evaluated using real flight process data, and the main conclusions are as follows:

(1)

N1 and Wf substantially improve the prediction performance among the measurement input features, while N2 yields only a marginal performance improvement.

(2)

Among all the single models, our FAE-LSTM performs best, which demonstrates that the feature attention module and the concatenate layer of our FAE-LSTM can improve the long-term prediction performance effectively. In addition, the FAE-LSTM is not sensitive to the input sequence length and the number of neuron cells.

(3)

Compared to the single model, the proposed hybrid can reduce the RMSE and MAE by at least 13.23% and 18.47%, respectively. The hybrid model has the best performance in all flight phases.

Further research will focus on the engine prognostic and diagnostic system based on the proposed hybrid model, aiming at the flight process data to further validate the practicality of the hybrid model.