Multi-step prediction of chaotic time-series with intermittent failures based on the generalized nonlinear filtering methods

doi:10.1016/j.amc.2013.02.071

Applied Mathematics and Computation

Volume 219, Issue 16, 15 April 2013, Pages 8584-8594

https://doi.org/10.1016/j.amc.2013.02.071 Get rights and content

Abstract

There are many practical situations that the chaotic signal appears in a random manner so that there are intermittent failures in the observation mechanism at certain times. These random interruptions, which are called as multiplicative noises, can be modeled by a sequence of independent Bernoulli random variables. Considering the observed chaotic signal perturbed by additive and multiplicative noises at the same time, this paper generalizes the original extended Kalman filtering (EKF), the Unscented Kalman filtering (UKF) and the Gaussian particle filtering (GPF) to the case in which there is a positive probability that the observation with intermittent failures in each time consists of additive noises alone. The shortened forms of these generalized new filtering algorithms are written as GEKF, GUKF and GGPF correspondingly. Using weights and network output of perceptron neural network to constitute state transition equation and observation equation, the input vector to the network is composed of predicted chaotic signal with given length (see Section 2 for details), and the multi-step prediction results are represented by the predicted observation value of nonlinear filtering methods. To show the advantage of these generalized new filtering algorithms, we applied them to the five-step prediction of Mackey–Glass time-series and equipment’s temperature (The corresponding time series can be found at http://robjhyndman.com/TSDL) with additive and multiplicative noises, respectively and compared them with the original EKF, UKF and GPF. Experimental results have demonstrated that the GEKF, GUKF and GGPF are proportionally superior to the original EKF, UKF and GPF. Moreover, GGPF is a better choice for multi-step prediction in comparison with GEKF and GUKF.

Introduction

It is well-known that long-term unpredictability is a generic feature of chaotic systems, so modeling and predicting the dynamics of chaotic systems are an important yet difficult problem in reality [1]. Several techniques have been used to deal with this challenging subject, such as radical basis function neural network [2], [3], multilayer perceptron neural networks [4], evolutionary recurrent neural network [5], wavelet process neural network [6], partial least squares regression [7] and support vector machine [8], [9], [10], which possess the abilities to approximate nonlinear systems.

Nonlinear filtering approaches have wide applications for their wide applicability and iterative algorithm structure. Moreover, chaotic signal may suffer from noises, so nonlinear filtering approaches can be applied to obtain more accurate prediction of chaotic time-series to handle these imperfections. Ma and Ten [11] and van der Merwe [12] use the EKF and UKF for one-step prediction of chaotic time series, Zhang and Chen [13], [14] propose the particle filtering (PF) algorithm to deal with the non-Gaussian noise in one and two-step prediction. However, all these proposed algorithms [11], [12], [13], [14] only consider the uncertain observations perturbed by additive noises.

In fact, there are many practical situations that the chaotic signal appears in the observation in a random manner so that there are intermittent failures in the observation mechanism at certain times. Under these situations, there is a positive probability called false alarm probability that only additive noise is observed at each time. To describe these uncertain observations, the observation equation is reformulated by multiplying the signal function by a binary random variable taking the values one and zero (Bernoulli random variable) at each sample time [15]. The value one indicates that the chaotic signal is present in the observation whereas the value zero reflects the fact that the observation is only noise. Thus, the observation equation involves both additive and multiplicative noises which model the uncertainty about the chaotic signal in each observation, and this is called the uncertainty of observation in this paper.

The multi-step prediction algorithm in nonlinear chaotic systems with such uncertain observations has not received much attention up to now. Therefore, the goal of this paper aims at obtaining the GEKF, GUKF and GGPF algorithms based on the EKF [16], UKF [17] and GPF [18] at first, then we use weights and network output of neural network as state equation and observation equation, and the prediction results are represented by the predicted observation value of nonlinear filtering algorithms. To prove the validity of our algorithms, we applied these generalized new filtering algorithms to the five-step prediction of Mackey–Glass time-series and equipment’s temperature with additive and multiplicative noises and compared them with the original EKF, UKF and GPF. Experimental results have demonstrated that the GGPF is a better choice among the six kinds of algorithms.

This paper consists of five sections. Section 2 presents the dynamical system model of chaotic time-series’ multi-step prediction. Section 3 is devoted to describe several kinds of generalized nonlinear filtering methods based on the statistical approximations of vector. Six kinds of nonlinear filtering algorithms based five-step prediction results of Mackey–Glass time-series and equipment’s temperature with additive and multiplicative noises have been obtained in Section 4, and the last section contains the conclusion.

Section snippets

Multi-layer perceptron neural network configuration

Our neural network model is a three-layer perceptron neural network with two hidden layers and one output layer, the configuration of feed-forward neural network is shown in Fig. 1.

According to this figure, the weights in paths are as follows:

(a)
[A_i,j]: weights from input neuron (i) to jth first hidden layer neuron.
(b)
[B_j,l]: weights from first hidden layer neuron (j) to kth second hidden layer.
(c)
[C_l]: weights from third hidden layer neuron (k) to output neuron.
(d)
b_1j, b_2l and b₃₁ denote the bias of each

Different generalized nonlinear filtering approaches

Due to the values of the variable γ_k, the density function of y_k = γ_kh(x_k, u_k) + w_k has the following mixture form $g (y_{k}) = p_{k} g (y_{k} | γ_{k} = 1) + (1 - p_{k}) g (y_{k} | γ_{k} = 0),$ where g(y_k|γ_k = 1) is the density of the vector h(x_k, u_k) + w_k and g(y_k|γ_k = 0) is the density of w_k. Hence, $\{\begin{matrix} E [y_{k}] = p_{k} \cdot E [h (x_{k}, u_{k})], \\ Cov [y_{k}] = p_{k} \cdot Cov [h (x_{k}, u_{k})] + p_{k} (1 - p_{k}) \cdot E [h (x_{k}, u_{k})] \cdot E [h^{T} (x_{k}, u_{k})] + R_{k}, \\ Cov [x_{k}, y_{k}] = p_{k} \cdot Cov [x_{k}, h (x_{k}, u_{k})] . \end{matrix}$

Therefore, one way to obtain approximations of the above statistics is to approximate E[h(x_k, u_k)], Cov[h(x_k, u_k)], Cov[x_k, h(x_k, u_k)],

Experimental setup and experimantal results

Two case studies are used to illustrate the effectiveness of the proposed approaches in this paper.

The first one is the Mackey–Glass time series’ five-step prediction, and this time series is produced by a time-delay difference system of the form [20] $\frac{dx (t)}{d (t)} = \frac{ax (t - τ)}{1 + x^{r} (t - τ)} - bx (t) .$

In order to be as standard as possible in the series we employ, the parameters are chosen as a = 0.2, b = 0.1, r = 10, τ = 17 and x(0) = 0.1 individually.

The second involves the equipment temperature’s (degree Celsius)

Conclusions

Considering the chaotic signal perturbed by additive and multiplicative noises and making full use of nonlinear filtering approaches’ wide applications, different nonlinear filtering methods are applied to the five-step prediction of Mackey–Glass time-series and equipment’s temperature with additive and multiplicative noises by using weights and network output of perceptron neural network to constitute state transition equation and observation equation. Experimental results have shown that the

Acknowledgements

The project was supported by the National Natural Science Foundation of China (No. 60774067) and the Natural Science Foundation of Jiangsu Province of China (No. BK2009727).

References (20)

K.W. Lau et al.
Local prediction of non-linear time series using support vector regression
Pattern Recognition
(2008)
B. Zhang et al.
Chaotic secure communication based on particle filtering
Chaos, Solitons & Fractals
(2006)
B. Zhang et al.
Particle-filter-based estimation and prediction of chaotic states
Chaos, Solitons & Fractals
(2007)
Z.K. Sun et al.
New scheme of anticipating synchronization for arbitrary anticipation time and its application to long-term prediction of chaotic states
Chinese Physics
(2007)
A.G. Fábio et al.
Multi-step ahead nonlinear identification of Lorenz’s chaotic system using radial basis neural network with learning by clustering and particle swarm optimization
Chaos, Solitons & Fractals
(2008)
X. Yan et al.
Research on chaotic time series prediction based on K-entropy and RBF neural networks
Journal of Systems Science and Information
(2006)
V.D. Sanjay
Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network
Applied Soft Computing
(2005)
Q.L. Ma et al.
Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network
Chinese Physics B
(2008)
G. Ding et al.
Time series prediction using wavelet process neural network
Chinese Physics B
(2008)
Z.X. Liu et al.
Chaotic time series multi-step direct prediction with partial least squares regression
Journal of Systems Engineering and Electronics
(2007)

There are more references available in the full text version of this article.

Cited by (14)

An encoder–decoder architecture with Fourier attention for chaotic time series multi-step prediction
2024, Applied Soft Computing
Multi-step prediction of chaotic time series has been a challenging problem. An encoder–decoder architecture based on novel Fourier attention was proposed, called FGNet, applied to chaotic time series multi-step prediction. Two perspectives were carefully re-considered in modeling, one was the integrated use of frequency and temporal domain information, the other was the fusion of channel information and series information. The encoder was a self-attention architecture with Fourier attention module for capturing dynamic information in time series. Correspondingly, the decoder used the Gated Recurrent Unit (GRU) module with identity mapping, and the decoder was used to model the representation based on temporal dependency. For the interpretability deficiency of Fourier attention in FNet, which was proposed by google, a novel Fourier attention module was proposed. Specifically, the frequency domain features were extracted using Fourier transform, series and channel features were integrated capture through channel swapping coupling. Then these features were fused in the frequency domain space to enhance feature representation and their interpretability is achieved through inverse Fourier transform. Simulated data (Mackey–Glass system and Lorenz) and real data (sunspot time series) were used to verify that FGNet proposed achieves pleasantly surprising multi-step prediction performance in practice.
An adaptive cubature Kalman filter for nonlinear systems against randomly occurring injection attacks
2022, Applied Mathematics and Computation
Citation Excerpt :
The renowned Kalman filter (KF) achieves the optimal estimation in the least mean square sense for linear Gaussian state-space models [6]. For decades, some extensions of the KF are proposed to seek suboptimal numerical solutions for nonlinear estimation since many practical problems do not satisfy the linear assumptions [2,3,7–10]. Among the existing nonlinear filters, the extended Kalman filter (EKF) is an important tool to tackle most of the filtering problems [7,11].
The problem of state estimation for nonlinear dynamic systems in the presence of randomly occurring injection attacks (ROIAs) is investigated. This paper requires no prior statistical information of the attacks, which relaxes the assumption of the existing result that the attack probability and the probability density function of attack signals need to be known. With the distribution of the attack probability and attack signals modeled as Beta distribution and Gaussian mixture distribution, a variational Bayesian based adaptive cubature Kalman filter is proposed to approximate the joint posterior distribution of the system state vector and unknown parameters. In addition, the update rules of the state and the statistical parameters of attacks are analytically derived by employing the fixed-point iteration approach. Finally, the effectiveness of the proposed filter is validated through numerical results.
Meta-cognitive recurrent kernel online sequential extreme learning machine with kernel adaptive filter for concept drift handling
2020, Engineering Applications of Artificial Intelligence
Citation Excerpt :
Lastly, there are lots of algorithms that are applied in multi-steps time series prediction. For example, the work in Wu and Song (2013) generalized the original extended Kalman filtering (EKF), the Unscented Kalman filtering (UKF) and the Gaussian particle filtering (GPF) to the benchmark data sets and compared the forecasting performance in the multi-step. At the same time, a reinforced The real-time recurrent learning (RTRL) algorithm for Two-step-ahead (2SA) forecasts using RNNs was proposed in the paper (Chang et al., 2012).
This paper proposes a multi-step prediction model for time series prediction, i.e. Meta-cognitive Recurrent Kernel Online Sequential Extreme Learning Machine with Drift Detector Mechanism (Meta-RKOS-ELM_ALD). Recurrent multi-step algorithm is applied to release the limitation in the number of prediction steps, and Drift Detector Mechanism (DDM) is used to overcome the problem of concept drift in the prediction model. The new meta-cognitive strategy decides the way of the incoming data during training, which decreases the training computation of prediction model and solves the parameter dependency. In our evaluation, we use a total of six artificial data sets and three real-world data sets (Standard & Poor’s 500 Index, Shanghai Stock Exchange Composite Index, and Ozone Concentration in Toronto) to prove the ability of kernel filters, the detecting ability of concept drift detector, and situation of applying meta-cognitive strategy in our proposed model. Experiments results indicate that the Meta-KOS-ELM_ALD with DDM has better forecasting ability in various predicting periods with the shortest learning time, as compared with other algorithms.
Co-evolutionary multi-task learning with predictive recurrence for multi-step chaotic time series prediction
2017, Neurocomputing
Citation Excerpt :
Note that * in Table 3 implies that the comparison was not very fair due to different embedding dimension reporting of results where it is not clear if the mean or best run has been reported. Further comparison for Mackey-Glass for 5-step-ahead has been shown using extended Kalman filtering (EKF), the unscented Kalman filtering (UKF) and the Gaussian particle filtering (GPF), along with their generalized versions G-EKF, G-UKF and G-GPF, respectively [84]. The Sunspot and Lazer series are compared with support vector regression, iterated (SVR-I), direct (SVR-D), and multiple models (M-SVR) methods [6].
Multi-task learning employs a shared representation of knowledge for learning several instances of the same problem. Multi-step time series problem is one of the most challenging problems for machine learning methods. The performance of a prediction model face challenges for higher prediction horizons due to the accumulation of errors. Cooperative coevolution employs in a divide and conquer approach for training neural networks and has been very promising for single step ahead time series prediction. Recently, co-evolutionary multi-task learning has been proposed for dynamic time series prediction. In this paper, we adapt co-evolutionary multi-task learning for multi-step prediction where predictive recurrence is developed to feature knowledge from previous states for future prediction horizon. The goal of the paper is to present a network architecture with predictive recurrence which is capable of multi-step prediction through a form of multi-task learning. We employ cooperative neuro-evolution and an evolutionary algorithm as baselines for comparison. The results show that the proposed method provides the best generalization performance in most cases. Comparison of results with the literature has shown to be promising which motivates further application of the approach for related real-world problems.
Moving Vehicle Tracking Under Measurement Uncertainties, Multi-Step Random Delays, and Packet Dropouts
2023, IEEE Access
On the Prediction of Chaotic Time Series using Neural Networks
2022, Chaos Theory and Applications

View all citing articles on Scopus

View full text

Multi-step prediction of chaotic time-series with intermittent failures based on the generalized nonlinear filtering methods

Abstract

Introduction

Section snippets

Multi-layer perceptron neural network configuration

Different generalized nonlinear filtering approaches

Experimental setup and experimantal results

Conclusions

Acknowledgements

Pattern Recognition

Chaos, Solitons & Fractals

Chaos, Solitons & Fractals

New scheme of anticipating synchronization for arbitrary anticipation time and its application to long-term prediction of chaotic states

Chinese Physics

Multi-step ahead nonlinear identification of Lorenz’s chaotic system using radial basis neural network with learning by clustering and particle swarm optimization

Chaos, Solitons & Fractals

Research on chaotic time series prediction based on K-entropy and RBF neural networks

Journal of Systems Science and Information

Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network

Applied Soft Computing

Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network

Chinese Physics B

Time series prediction using wavelet process neural network

Chinese Physics B

Chaotic time series multi-step direct prediction with partial least squares regression

Journal of Systems Engineering and Electronics