Sequential reconstruction of driving-forces from nonlinear nonstationary dynamics
Introduction
Many real-life dynamic systems generate nonstationary outputs due to the inevitable presence of measurement noise, dynamic noise and changes in the internal system parameters or environmental conditions in the course of time. Measurement noise implies stochasticity, whereas the perturbations may cause the system parameters to be time-varying, and the combined effects of these factors result in a nonstationary process arising from a perturbed dynamic system. The perturbation signals (driving-forces) can be viewed as unknown inputs to the unperturbed dynamics. They are hidden from the observer most of the time if not always, and estimating them can be of practical interest in many cases in physics and engineering.
Here we address some examples of how in many physical phenomena we have a dynamic system driven by a force that is not observable:
Sunspot time series: Sunspot is the cooler darker spot appearing on the sun’s photosphere whose mechanism is not exactly known. In 1848, Rudolph Wolf devised a daily method of estimating solar activity by counting the number of individual spots and groups of spots on the face of the sun [1]. The variation in the sun’s magnetic flux plays the role of a perturbation process impacting the evolution of the sunspot numbers’ time series. A mathematical model is developed in [2] to extract the information pertaining to sun’s magnetic flux.
Sea clutter: Sea clutter refers to the radar backscatter from an ocean surface. The study of sea clutter is not only of theoretical importance but also of practical importance because it places severe limitations on the detectability of point targets (e.g., low-flying aircraft, small marine vessels, navigation buoys, and small pieces of ice) on or near the sea surface. The two fundamental types of waves (i.e. gravity and capillary waves) dictate the roughness of the sea surface, and govern the clutter dynamics [3]. When a small target is embedded in the sea surface, the movement of the target will be also dominated by the governing sea waves. Then a small target can be modeled as an additional random perturbation in the sea surface, whose dynamics are closely coupled by those of the sea. Hence the estimation of a small random target’s signature can be casted as a driving-force estimation problem within the radar scene analysis.
Wireless communication channel: In a general wireless communications scenario, the transmit signal reaches the receiver by different transmission paths with different path lengths. This causes the received signal to be a superposition of different reflected signals, which have varying delays due to their path lengths. Since the receiver could be moving, fast fading occurs and the channel is therefore time-varying. In such situations, the physical and geometrical properties of the environment (buildings, trees, etc.) and movement of the receiver play the role of perturbation inputs.
Physiological processes: In living organisms, heart activity, breathing, muscle tremor and voice production are some examples of physiological processes. All these processes have some degree of nonstationarity due to the perturbations coming both from the environment and from different systems of the vital activity of the organism.
Seismic data: Earth’s deep interior has been largely influenced by earthquakes, most of which are caused by the sudden movement of rock masses along a fault. As these rocks grind together, energy is released and vibrations are produced, which we call seismic waves. The speed of seismic waves in rocks depends on several environmental variables, the most important of which are pressure and temperature [4]. Therefore, the seismic waves, being the reflection of the rock motions, are perturbed by those driving-forces.
In light of these physical examples, we can identify three estimation scenarios:
- 1.
The state and driving-force are both one-dimensional, as in the case of sunspot data.
- 2.
The state is multidimensional, and the driving-force is common to every element of the state, as in the case of sea clutter, where a population of the local scatterers are perturbed by the same small target. The target would play the role of a driving-force.
- 3.
The state and driving-force are both multidimensional; it is in such a situation where we need prior information on the evolution of state to estimate the driving-forces.
Consider a (possibly nonlinear) dynamic system as described in (1), whose state–space model consists of two parts:
- •
A process equation that describes the evolution of state under the action of process noise and a driving-force; the force is unknown.
- •
Measurement equation that defines the observables buried in measurement noise.
The continuous and measurable mapping defines the state transition, and defines the evolution of observables respectively. denotes the state, is the unknown input signal (driving-force), denotes the observable, is the dynamic noise, and is the measurement noise, all at discrete time , . denotes the real space. Note that (1) can be extended to cover the cases where the state is multidimensional without loss of generality.
In general terms, the problem that we are posing is similar to the state-estimation problem, where the requirement is to estimate the state . Our problem, however, is different: We have to estimate the driving-force without the knowledge of and given that is an invertible linear operator. If is a nonlinear mapping however, its knowledge is required a priori.
The paper is organized as follows: In Section 2, we present an overview of the related literature. In Section 3, we address the problem in a formal way, and present the derivation of the reconstruction algorithm. We reproduce the generalized posterior Cramer–Rao lower bound (PCRB) in Section 4. In Section 5, we present the results of controlled experiments, which are evaluated using PCRB. In Section 6, we apply the estimator to the real-life sunspot time series for the reconstruction of sun’s magnetic flux. We compare our results to two other models, and show that the proposed approach captures the essential dynamics of the sun’s magnetic flux. Section 7 concludes the paper with remarks on the future research.
Section snippets
Literature review
The hidden input estimation problem has received considerable attention in both physics and engineering disciplines. We can divide the numerous contributions in the literature to two main categories.
In the first category, an analytical model of the system model is readily available. The main contributions for linear stochastic systems within this category are based on the extension of Kalman filter for the estimation of hidden inputs [5], [6]. For nonlinear and stochastic systems, the
Regularized estimation and tracking of unobservable inputs
The basic idea behind the adaptive prediction error method is the following: Imagine that the observables from the unknown system in (1) are transformed into a time series model as given in (2) (see Appendix A for derivation), where the current observable is expressed as a function of the previous observable , and the unknown driving-force, . denotes the measurement noise. Next, consider that we design an online nonlinear predictor, which receives
PCRB
Error lower bounds provide performance limitations for the estimation algorithms, and also allow several methods to be compared against a reference. The Posterior Cramer–Rao Lower Bound (PCRB) is applicable to the estimation of dynamic parameters. In [32], the PCRB is derived for the generalized Markovian nonlinear systems, which is summarized below. Let represent a sample of measured data, let be an -dimensional estimated random parameter, let be the joint probability density of
Experiments on chaotic maps
The estimator’s performance on the chaotic dynamics will be illustrated on two different systems. The first system under consideration is a simple Markovian, yet extremely rich example, the logistic map as described in (25), where the chaotic state is linear in the driving-force, . is the measured sample at time . The second system under study is the Moran–Ricker map, which differs from the logistic map mainly in that the chaotic state
Application to sunspot time series
We finally illustrate the performance of the estimator on some real-life data, the sunspot time series, in which the hidden information exhibits an irregular behavior.
Sunspot is the cooler darker spot appearing on the sun’s photosphere whose mechanism is not exactly known. For sunspot time series, the dominating perturbation is the sun’s total magnetic flux, hence, the driving-force is one-dimensional. A mathematical model is developed in [2] to extract the information pertaining to sun’s
Conclusion
We described a sequential estimation approach building on the prediction error idea for the reconstruction of hidden inputs that perturb nonlinear dynamic systems. The proposed driving-force estimator owes its good performance to the selection of two robust systems; the ESN banks, and the regularized LMS adaptive filtering algorithm. Both the ESNs and the adaptive filter enjoy a common adaptation property. The ESNs keep learning from the environment by virtue of the continuous update of its
Acknowledgements
Dr. J. Reilly, Dr. H. deBruin, Dr. T. Kirubarajan and Dr. M. Grasselli of McMaster University are greatly acknowledged for their support and contributions to this work. Many thanks are owed to Dr. Solanki, Dr. Schuessler and Dr. Fligge, the authors of [2], for kindly permitting us to use the original version of the corresponding curve in Fig. 7. Dr. S. Haykin of McMaster University is also acknowledged for his involvement in the early stages of this work. The author is grateful to two anonymous
References (37)
Recurrence plots revisited
Physica D
(1997)Interdisciplinary application of nonlinear time series methods
Phys. Rep.
(1999)- et al.
Modeling nonstationary dynamics
Physica A
(2003) - Description of sunspot numbers, URL:...
- et al.
Secular variation of the sun’s magnetic flux
Astron. Astrophys.
(2002) - et al.
Uncovering nonlinear dynamics-the case study of sea clutter
Proc. IEEE
(2002) - Explanation of seismic waves, URL:...
- et al.
Unbiased minimum-variance input and state estimation for linear discrete-time systems
Automatica
(2006) - et al.
Optimal filtering for systems with unknown inputs
IEEE Trans. Automat. Control
(1998) - et al.
Maximum likelihood from incomplete data via the em algorithm
J. R. Stat. Soc. Ser. B
(1977)
A self-organizing state-space model
J. Amer. Statist. Assoc.
Monte carlo filter using the genetic algorithm operators
J. Stat. Comput. Simul.
Particle filters for state–space models with the presence of unknown static parameters
IEEE Trans. Signal Process.
Recurrence plots of dynamical systems
Europhys. Lett.
Detecting and analysing nonstationarity in a time series with nonlinear cross-predictions
Phys. Rev. Lett.
Nonstationary time-series analysis: accurate reconstruction of driving forces
Phys. Rev. Lett.
Cited by (16)
Reconstruction of driving forces from nonstationary time series including stationary regions and application to climate change
2017, Physica A: Statistical Mechanics and its ApplicationsShort-term wind speed forecasting based on spectral clustering and optimised echo state networks
2015, Renewable EnergyCitation Excerpt :The ESN model requires a simple training process and has short-term memory. Thus, this model has found widespread use and success in the field of time series forecasting for applications such as stock prices and power system loads [2,15–18]. An ESN can process multiple inputs and multiple outputs with such high efficiency that it is capable of meeting the requirements for real-time, multi-step, short-term forecasting of wind speeds.
Recursive hidden input estimation in nonlinear dynamic systems with varying amounts of a priori knowledge
2014, Signal ProcessingCitation Excerpt :The pseudocodes and computational complexities of the ADFE, the RBPF-AR(P) and the EM-KS-PF algorithms are presented in subsequent appendix sections. The basic idea underlying the adaptive driving-force estimator is to predict the observables one-step into the future using a bank of echo state networks (ESN) [1,2,39–41] in an online fashion, and then exploit the fact that the driving-force is not explicitly modeled by the predictor. Based on this simple idea, we relate the driving-force to the prediction error of the ESNs.
Review of force reconstruction techniques
2014, Journal of Sound and VibrationExploring the driving forces of long-term total ozone change: based on data from a ground based station at the northern mid-latitude over 1958–2018
2022, Theoretical and Applied ClimatologyExperimental study on prediction of nonlinear system based on causality test
2022, Wuli Xuebao/Acta Physica Sinica