The filtering based auxiliary model generalized extended stochastic gradient identification for a multivariate output-error system with autoregressive moving average noise using the multi-innovation theory

doi:10.1016/j.jfranklin.2020.03.028

Journal of the Franklin Institute

Volume 357, Issue 9, June 2020, Pages 5591-5609

https://doi.org/10.1016/j.jfranklin.2020.03.028 Get rights and content

Abstract

This paper studies the parameter estimation algorithms of multivariate output-error autoregressive moving average (M-OEARMA) systems. By means of the filtering technique and the auxiliary model identification idea, this paper gives an auxiliary model generalized extended stochastic gradient (AM-GESG) algorithm for identifying the M-OEARMA system as a comparison. In order to enhance the performance of the AM-GESG algorithm, a modified filtering based AM-GESG algorithm and a filtering based auxiliary model multi-innovation generalized extended stochastic gradient algorithm are proposed. Compared with the AM-GESG algorithm, the proposed two algorithms can generate highly accurate parameter estimates. The simulation examples demonstrate that the proposed algorithms are effective for identifying the M-OEARMA systems.

Introduction

System identification is the modeling theory and methods for researching linear systems [1], [2], [3] and nonlinear systems [4], [5], [6]. For system analysis and design with unknown parameters, the first step is to obtain the parameters of the system models through some identification methods [7]. The recursive identification and the iterative identification are two important types of parameter estimation methods [8]. The iterative algorithms use the batch data to update the parameter estimates and have many applications in off-line identification [9], [10]. In contrast to the iterative algorithms, the recursive algorithms can be used for on-line identification because they can update the parameters with new data as the time moves forward [11], [12]. The stochastic gradient algorithm belongs to the recursive algorithms which updates the estimates along the negative gradient and has been employed in many occasions [13], [14].

The modeling and identification of multivariable systems have received much attention in the field of industrial control [15], [16], [17]. It is worth pointing out that many multivariable systems possess various disturbances which contaminate the measurement outputs of the systems and therefore result in low identification accuracy [18]. To weaken the interference of the noises, one effective way is to apply the filtering technique to change the structure of the disturbance noise model and to improve the identification accuracy. Although the stochastic gradient algorithm can be used for system identification, it cannot achieve high estimation accuracy. In order to solve this issue, we use the multi-innovation identification method to enhance the performance of the stochastic gradient algorithm. The multi-innovation is a powerful approach to enhance the estimation accuracy of the gradient methods. The key is to expand the innovation and to make full use of input–output data. This paper investigates the parameter estimation problem of a multivariate output-error system with autoregressive moving average noise through using the multi-innovation theory and the filtering technique. For one thing, we use the filtering technique to process the noise model of the system; For another, we employ the multi-innovation theory to enhance the performance of the stochastic gradient algorithm. Besides, in order to make the stochastic gradient algorithm realizable, an auxiliary model is established to replace the unknown variables in the identification algorithm with the outputs of the auxiliary model. The main contributions of this paper are as follows.

•
A filtering based auxiliary model generalized extended stochastic gradient algorithm is proposed for the M-OEARMA systems by using the filtering technique and the auxiliary model. Compared with the auxiliary model based stochastic gradient algorithm, the proposed algorithm can generate more accurate estimates.
•
By expanding the identification innovation and introducing the convergence index, a modified filtering based auxiliary model generalized extended stochastic gradient algorithm and a filtering based auxiliary model multi-innovation generalized extended stochastic gradient algorithm are derived in order to improve the performance of the filtering based stochastic gradient algorithm.

The proposed identification algorithms are suitable for multivariate stochastic systems and can be extended to identify bilinear stochastic systems [19], [20], [21] and nonlinear stochastic systems [22] with colored noises, and can be applied to other fields [23], [24], [25], [26] such as engineering application systems [27], [28], [29], [30].

The rest of this paper is organized as follows. Section 2 describes the identification problems for multivariate output-error autoregressive moving average systems. Section 3 introduces an auxiliary model based stochastic gradient algorithm as a comparison. Section 4 presents a filtering based stochastic gradient algorithm and Section 5 proposes a filtering based multi-innovation stochastic gradient algorithm. Section 6 provides an illustrative example to indicate the effectiveness of the proposed algorithms. Finally, some concluding remarks are offered in Section 7.

Section snippets

The system description

Let us introduce some notations. ``A ≕ X” or ``X ≔ A” stands for ``A is defined as X”; the superscript T represents the vector/matrix transpose; the symbol I_n denotes an identity matrix of size n × n; $\hat{ϑ} (t)$ means the estimate of ϑ at time t; 1_n stands for a n-dimensional column vector whose elements are 1; the norm of a matrix (or a column vector) X is defined by ‖X‖² ≔ tr[XX^T].

Consider the following multivariate output-error autoregressive moving average (M-OEARMA) system: $y (t) = x (t) + w (t) = \frac{Φ_{1} (t) θ}{A}$

The auxiliary model based generalized extended stochastic gradient algorithm

This section extends the AM-GSG algorithm for M-OEAR systems in [31] to present an auxiliary model based generalized extended stochastic gradient algorithm for M-OEARMA systems in Eq. (1) as a comparison. According to the identification model in Eq. (8), we can derive the auxiliary model based generalized extended stochastic gradient (AM-GESG) algorithm: $\hat{ϑ} (t) = \hat{ϑ} (t - 1) + \frac{{\hat{Φ}}^{T} (t)}{r (t)} [y (t) - \hat{Φ} (t) \hat{ϑ} (t - 1)],$ $r (t) = r (t - 1) + ∥ \hat{Φ} {(t) ∥}^{2},$ $\hat{Φ} (t) = [Φ_{1} (t), {\hat{Φ}}_{x} (t), {\hat{Φ}}_{n} (t)],$ ${\hat{Φ}}_{x} (t) = [- \hat{x} (t - 1), - \hat{x} (t - 2), \dots, - \hat{x} (t - n_{a})],$ ${\hat{Φ}}_{n} (t) = [- \hat{w} ($

The filtering based auxiliary model generalized extended stochastic gradient algorithm

The stochastic gradient algorithm has a slow convergence rate and cannot reach a satisfactory estimation accuracy. In this section, a linear filter L(z) ≔ C(z) is introduced to deal with the colored noises and enhance the estimation accuracy of the AM-GESG algorithm.

Refer to the method in [31], and define the filtered output vector y_f(t) and the filtered information matrix $Φ_{f} (t)$ as $y_{f} (t) : = C (z) y (t) = y (t) + c_{1} y (t - 1) + c_{2} y (t - 2) + \dots + c_{n_{c}} y (t - n_{c}) \in R^{m},$ $Φ_{f} (t) : = C (z) Φ_{1} (t) = Φ_{1} (t) + c_{1} Φ_{1} (t - 1) + c_{2} Φ_{1} (t - 2) + \dots + c_{n_{c}} Φ_{1} (t - n_{c}) \in R^{m \times n}$

The filtering based multi-innovation generalized extended stochastic gradient algorithm

Based on the F-SG algorithm in Eqs. (35)–(47), we employ the multi-innovation theory to derive a filtering based multi-innovation generalized extended stochastic gradient algorithm to improve the parameter estimation accuracy in this section.

Define p as the innovation length. Based on the F-SG algorithm in Eqs. (35)–(47), expand the innovation vector $e (t) : = y (t) - \hat{Ψ} (t) \hat{ϑ} (t - 1)$ in Eq. (35) into a large innovation vector: $E (p, t) : = [\begin{matrix} y (t) - \hat{Ψ} (t) \hat{ϑ} (t - 1) \\ y (t - 1) - \hat{Ψ} (t - 1) \hat{ϑ} (t - 2) \\ ⋮ \\ y (t - p + 1) - \hat{Ψ} (t - p + 1) \hat{ϑ} (t - p) \end{matrix}] \in R^{m p} .$

Example

Consider the following M-OEARMA system: $\begin{matrix} y (t) & = & x (t) + w (t), \\ x (t) & = & \frac{Φ_{1} (t) θ}{A (z)}, \\ w (t) & = & \frac{D (z)}{C (z)} v (t), \\ Φ_{1} (t) & = & [\begin{matrix} y_{2} (t - 1) & u_{2} (t - 1) \cos (t / π) & u_{1} (t - 2) \\ u_{2} (t - 2) \sin (t / π) & u_{1} (t - 1) & y_{1} (t - 2) \end{matrix}], \\ A (z) & = & 1 + 0.12 z^{- 1} + 0.39 z^{- 2}, \\ C (z) & = & 1 - 0.58 z^{- 1} + 0.35 z^{- 2}, \\ D (z) & = & 1 + 0.09 z^{- 1} + 0.23 z^{- 2}, \\ θ & = & {[θ_{1}, θ_{2}, θ_{3}]}^{T} = {[0.42, 0.93, 0.09]}^{T}, \\ ϑ & = & {[0.42, 0.93, 0.09, 0.12, 0.39, - 0.58, 0.35, 0.09, 0.23]}^{T} . \end{matrix}$

In simulation, the inputs u₁(t) and u₂(t) are taken as two uncorrelated stochastic signal sequences with zero mean and unit variances; v₁(t) and v₂(t) are taken as two white noise sequences

Conclusions

This paper uses the filtering technique and the multi-innovation theory to investigate the identification problems of M-OEARMA systems and presents a filtering based auxiliary model generalized extended stochastic gradient algorithm and a filtering based auxiliary model multi-innovation generalized extended stochastic gradient algorithm. The simulation results indicate that the proposed algorithms have good performances after introducing the innovation length and the convergence index and the

Declaration of Competing Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

References (82)

J. Ding et al.
Particle filtering based parameter estimation for systems with output-error type model structures
J. Frankl. Inst.
(2019)
F. Ding et al.
Two-stage gradient-based iterative estimation methods for controlled autoregressive systems using the measurement data
Int. J. Control Autom. Syst.
(2020)
F. Ding et al.
Decomposition based least squares iterative identification algorithm for multivariate pseudo-linear ARMA systems using the data filtering
J. Frankl. Inst.
(2017)
F. Ding et al.
Gradient estimation algorithms for the parameter identification of bilinear systems using the auxiliary model
J. Comput. Appl. Math.
(2020)
X. Zhang et al.
State estimation for bilinear systems through minimizing the covariance matrix of the state estimation errors
Int. J. Adapt. Control Signal Process.
(2019)
L.H. Li et al.
Behavior of tire-geogrid-reinforced retaining wall system under dynamic vehicle load
Int. J. Geomech.
(2020)
C.C. Yin et al.
Exit problems for jump processes with applications to dividend problems
J. Comput. Appl. Math.
(2013)
C.C. Yin et al.
An extension of paulsen-gjessing’s risk model with stochastic return on investments
Insur. Math. Econ.
(2013)
C.C. Yin et al.
On the optimal dividend problem for a spectrally positive levy process
Astin Bull.
(2014)
C.C. Yin et al.
Exact joint laws associated with spectrally negative levy processes and applications to insurance risk theory
Front. Math. China
(2014)

P. Ma et al.

New gradient based identification methods for multivariate pseudo-linear systems using the multi-innovation and the data filtering

J. Frankl. Inst.

(2017)

L. He et al.

Accurate measurement of pavement deflection velocity under dynamic loads

Autom. Constr.

(2017)

M.Y. Liu et al.

Predicting fatigue life for finite line contact under starved elastohydrodynamic lubrication condition

Math. Probl. Eng.

(2020)

H.J. Wang et al.

Bifurcations, ultimate boundedness and singular orbits in a unified hyperchaotic lorenz-type system

Discrete and Continuous Dyn. Syst. - Series B

(2020)

L. Geng et al.

Control and backbone identification for the resilient recovery of a supply network utilizing outer synchronization

Appl. Sci.

(2020)

N. Zhao

Joint optimization of cooperative spectrum sensing and resource allocation in multi-channel cognitive radio sensor networks

Circuits Syst. Signal Process.

(2016)

L. Xu

The parameter estimation algorithms based on the dynamical response measurement data

Adv. Mech. Eng.

(2017)

J. Ding et al.

Particle filtering-based recursive identification for controlled auto-regressive systems with quantised output

IET Control Theory Appl.

(2019)

F. Ding et al.

The innovation algorithms for multivariable state-space models

Int. J. Adapt. Control Signal Process.

(2019)

G.Y. Chen et al.

A regularized variable projection algorithm for separable nonlinear least-squares problems

IEEE Trans. Autom. Control

(2019)

J. Ding et al.

Weighted parameter estimation for hammerstein nonlinear ARX systems

Circuits Syst. Signal Process.

(2020)

L.J. Liu et al.

Maximum likelihood recursive identification for the multivariate equation-error autoregressive moving average systems using the data filtering

IEEE Access

(2019)

L. Xu et al.

Parameter estimation and controller design for dynamic systems from the step responses based on the newton iteration

Nonlinear Dyn.

(2015)

L.J. Wan et al.

Decomposition- and gradient-based iterative identification algorithms for multivariable systems using the multi-innovation theory

Circuits Syst. Signal Process.

(2019)

L. Xu

The damping iterative parameter identification method for dynamical systems based on the sine signal measurement

Signal Process.

(2016)

J. Pan et al.

A filtering based multi-innovation extended stochastic gradient algorithm for multivariable control systems

Int. J. Control Autom. Syst.

(2017)

H. Ma et al.

Partially-coupled least squares based iterative parameter estimation for multi-variable output-error-like autoregressive moving average systems

IET Control Theory Appl.

(2019)

M. Gan et al.

A variable projection approach for efficient estimation of RBF-ARX model

IEEE Trans. Cybernet.

(2015)

S.Y. Liu et al.

Hierarchical principle-based iterative parameter estimation algorithm for dual-frequency signals

Circuits Syst. Signal Process.

(2019)

M. Gan et al.

On some separated algorithms for separable nonlinear squares problems

IEEE Trans. Cybernet.

(2018)

C.P. Yu et al.

Subspace identification of individual systems in a large-scale heterogeneous network

Automatica

(2019)

C.P. Yu et al.

Constrained subspace method for the identification of structured state-space models

IEEE Trans. Autom. Control

(2020)

M.H. Li et al.

The filtering-based maximum likelihood iterative estimation algorithms for a special class of nonlinear systems with autoregressive moving average noise using the hierarchical identification principle

Int. J. Adapt. Control Signal Process.

(2019)

J. Pan et al.

Control algorithms of magnetic suspension systems based on the improved double exponential reaching law of sliding mode control

Int. J. Control Autom. Syst.

(2018)

S.S. Tian et al.

Application of c6f12o/CO₂ mixture in 10 kv medium-voltage switchgear

IET Sci. Meas. Technol.

(2019)

G.Z. Zhang et al.

Ladder-wise calculation method for z-coordinate of transformer PD source based on planar layout UHF antenna sensors

IEEJ Trans. Electr. Electron. Eng.

(2020)

D.C. Chen et al.

A first-principles study of the SF6 decomposed products adsorbed over defective WS2 monolayer as promising gas sensing device

IEEE Trans. Device Mater. Reliab.

(2019)

Y.L. Li et al.

Experimental study on compatibility of eco-friendly insulating medium c5f10o/CO2 gas mixture with copper and aluminum

IEEE Access

(2019)

L.J. Wang et al.

Bidirectional short-circuit current blocker for DC microgrid based on solid-state circuit breaker

Electronics

(2020)

L.J. Wang et al.

Hybrid model predictive control strategy of supercapacitor energy storage system based on double active bridge

Energies

(2019)

Y.L. Zhang et al.

Signal reconstruction of compressed sensing based on alternating direction method of multipliers

Circuits Syst. Signal Process.

(2020)

Cited by (12)

A new adaptive identification framework for nonlinear multi-input multi-output systems under colored noise
2022, Applied Mathematical Modelling
In this paper, an adaptive identification scheme is proposed for nonlinear multi-input multi-output systems with colored noise based on a novel parameter update law. With the help of the hierarchical principle, the identification model is decomposed into three sub-models in which the computational burden is reduced. For each sub-model, the identification algorithm is proposed to estimate the sub-model parameters. In the process of the identification algorithm design, considering the system information corrupted by the noise, an adaptive filter gain is exploited to extract helpful identification data, in which a filter is designed using the system data instead of the independent design. Based on several auxiliary filtered variables, the estimation error data are obtained, and a new parameter adaptive law with a variable learning gain is proposed according to the estimation error data. Compared with the classic parameter update law, the parameter estimation update is derived based on the estimation error information instead of other error information, such as prediction error information. Under the persistent excitation condition, all the estimated parameters converge to the true parameters. an example is used and two experiments are conducted to test the outstanding identification performance of the proposed algorithm in terms of convergence rate and identification accuracy.
H<inf>−</inf> index for Itô stochastic systems with Possion jump
2021, Journal of the Franklin Institute
This paper studies the $H_{-}$ index problem for stochastic systems with Wiener processes and Poisson jumps. A necessary and sufficient condition is established for finite horizon Poisson jump stochastic systems to show that $H_{-}$ index lager than a given prescribed level $γ > 0$ implies the existence of the solution of the corresponding generalized differential Riccati equation (GDRE). Accordingly, the corresponding results are extended to infinite horizon $H_{-}$ index problems for time-invariant square systems. As the direct application of our results, the fault detection filter design is realized in evaluating the fault detection performance.
Auxiliary model-based multi-innovation PSO identification for Wiener–Hammerstein systems with scarce measurements
2021, Engineering Applications of Artificial Intelligence
Citation Excerpt :
At present, the application of the multi-innovation technology has been very extensive. Ding et al. exhibited the auxiliary model multi-innovation generalized extended stochastic gradient identification (Ding et al., 2020a); Sassi et al. used a multi-innovation technology to enhance the accuracy of the unscented Kalman filter (Sassi et al., 2020); and Cheng et al. proposed a multi-innovation fractional order gradient method for Hammerstein models (Cheng et al., 2018). The multi-innovation technology has important research significance because it is a powerful approach to improve the estimation accuracy of other identification methods, thus we introduce it into the PSO algorithm to achieve better estimation.
In many actual systems, it is often difficult to obtain complete input and output data. Thus, the problem of scarce measurements usually appears in the identification of these systems. This article investigates the parameter estimation of Wiener–Hammerstein systems with scarce measurements. A Wiener–Hammerstein system comprises an input linear unit, a nonlinear unit, and an output linear unit. The nonlinear unit in this paper is described by the saturation and dead-zone characteristics respectively. To solve the incomplete data problem caused by scarce measurements, the auxiliary model is applied. Then the auxiliary model-based improved particle swarm optimization (PSO) algorithm is derived. Furthermore, the multi-innovation technology is introduced to improve the convergence speed and estimation accuracy, and the auxiliary model-based multi-innovation improved PSO is proposed. Finally, the simulations of two numerical examples and the application of the turntable servo system indicate that the proposed multi-innovation method is applicable to Wiener–Hammerstein models with scarce measurements, the estimation accuracy and convergence speed are greatly improved.
Robust Gradient Estimation Algorithm for a Stochastic System with Colored Noise
2023, International Journal of Control, Automation and Systems
Bipartite Fixed-time Consensus of Multi-agents System with Disturbance via Event-triggered Control
2022, International Journal of Control, Automation and Systems
Two-stage gradient-based iterative algorithms for the fractional-order nonlinear systems by using the hierarchical identification principle
2022, International Journal of Adaptive Control and Signal Processing

View all citing articles on Scopus

^☆: This work was supported by the National Natural Science Foundation of China (No. 61472195) and by Key Program Special Fund in XJTLU (No. KSF-E-12).

View full text

The filtering based auxiliary model generalized extended stochastic gradient identification for a multivariate output-error system with autoregressive moving average noise using the multi-innovation theory☆

Abstract

Introduction

Section snippets

The system description

The auxiliary model based generalized extended stochastic gradient algorithm

The filtering based auxiliary model generalized extended stochastic gradient algorithm

The filtering based multi-innovation generalized extended stochastic gradient algorithm

Example

Conclusions

Declaration of Competing Interest

J. Frankl. Inst.

Int. J. Control Autom. Syst.

J. Frankl. Inst.

J. Comput. Appl. Math.

Int. J. Adapt. Control Signal Process.

Int. J. Geomech.

J. Comput. Appl. Math.

Insur. Math. Econ.

Astin Bull.

Front. Math. China

J. Frankl. Inst.

Autom. Constr.

Math. Probl. Eng.

Discrete and Continuous Dyn. Syst. - Series B

Appl. Sci.

Circuits Syst. Signal Process.

Adv. Mech. Eng.

Particle filtering-based recursive identification for controlled auto-regressive systems with quantised output

IET Control Theory Appl.

The innovation algorithms for multivariable state-space models

Int. J. Adapt. Control Signal Process.

A regularized variable projection algorithm for separable nonlinear least-squares problems

IEEE Trans. Autom. Control

Weighted parameter estimation for hammerstein nonlinear ARX systems

Circuits Syst. Signal Process.

Maximum likelihood recursive identification for the multivariate equation-error autoregressive moving average systems using the data filtering

IEEE Access

Parameter estimation and controller design for dynamic systems from the step responses based on the newton iteration

Nonlinear Dyn.

Decomposition- and gradient-based iterative identification algorithms for multivariable systems using the multi-innovation theory

Circuits Syst. Signal Process.

The damping iterative parameter identification method for dynamical systems based on the sine signal measurement

Signal Process.

A filtering based multi-innovation extended stochastic gradient algorithm for multivariable control systems

Int. J. Control Autom. Syst.

Partially-coupled least squares based iterative parameter estimation for multi-variable output-error-like autoregressive moving average systems

IET Control Theory Appl.

A variable projection approach for efficient estimation of RBF-ARX model

IEEE Trans. Cybernet.

Hierarchical principle-based iterative parameter estimation algorithm for dual-frequency signals

Circuits Syst. Signal Process.

On some separated algorithms for separable nonlinear squares problems

IEEE Trans. Cybernet.

Subspace identification of individual systems in a large-scale heterogeneous network

Automatica

Constrained subspace method for the identification of structured state-space models

IEEE Trans. Autom. Control

The filtering-based maximum likelihood iterative estimation algorithms for a special class of nonlinear systems with autoregressive moving average noise using the hierarchical identification principle

Int. J. Adapt. Control Signal Process.

Control algorithms of magnetic suspension systems based on the improved double exponential reaching law of sliding mode control

Int. J. Control Autom. Syst.

Application of c6f12o/CO2 mixture in 10 kv medium-voltage switchgear

IET Sci. Meas. Technol.

Ladder-wise calculation method for z-coordinate of transformer PD source based on planar layout UHF antenna sensors

IEEJ Trans. Electr. Electron. Eng.

A first-principles study of the SF6 decomposed products adsorbed over defective WS2 monolayer as promising gas sensing device

IEEE Trans. Device Mater. Reliab.

Experimental study on compatibility of eco-friendly insulating medium c5f10o/CO2 gas mixture with copper and aluminum

IEEE Access

Bidirectional short-circuit current blocker for DC microgrid based on solid-state circuit breaker

Electronics

Hybrid model predictive control strategy of supercapacitor energy storage system based on double active bridge

Energies

Signal reconstruction of compressed sensing based on alternating direction method of multipliers

Circuits Syst. Signal Process.

Application of c6f12o/CO₂ mixture in 10 kv medium-voltage switchgear