A Covariance Feedback Approach to Covariance Control of Nonlinear Stochastic Systems

,


Introduction
Most of the introduced approaches that analyze the dynamics of stochastic systems use a covariance control algorithm to design a control rule to provide limited desired covariance aims. e variance of a random variable is very important in most of the control researches such as predictive systems analysis and state estimation problems. Many researchers have done huge researches to reach and control the constrained variance objectives in related control goals to have a convergence in identification and estimation systems [1,2] and data filtering fault detection and diagnosis [3] or have a better convergence rate of some intelligent algorithms such as genetic algorithm [4] and neural network [5]. Choosing an incorrect variance can cause instability in the whole system and can reduce the performance of the system.
Most papers on covariance assignment focused on the state covariance assignment (SCA) theory for linear systems. In these studies, primary control purposes such as constraints of the variances of the states were assumed. Besides, secondary purposes could be assumed for the input variance values. By expanding the Skelton linear methods [6,7], some theories about the covariance control of perturbed, bilinear, or nonlinear stochastic systems have been presented and reviewed in [8][9][10][11][12][13].
By improving the SCA idea of a linear system, some theories about the determining the desired range of state covariance variations of nonlinear systems, multivariable, multisensor systems, or placing and setting the system covariance matrix when system parameters change have been presented [9][10][11][12][13][14]. For example, by expanding the linear methods, Kalandros [10] and Kalandros and Pao [14] deal with the problem of covariance control in multisensor systems and its application for reducing bias effects in interacting target scenarios is presented. Furthermore, in [8], [11], and [12], bilinear random systems have been controlled by the covariance control method.
Recently, finite-horizon covariance control problem for stochastic discrete-time linear systems with complete state information subject to input constraints has been considered in Bakolas [15]. Also, the optimal covariance control problem for stochastic discrete-time linear systems subject to probabilistic chance constraints has been studied by Okamoto et al. [16]. e practical covariance control theory incorporating the concept of variance-constrained control for the discrete TakagiSugeno (T-S) fuzzy stochastic systems has been applied in Chang et al. [17].
Chang et al. [18] presented a novel multiobjective sliding mode fuzzy control technique for a class of discrete-time nonlinear stochastic systems, such that the closed-loop system achieves passivity constraint and individual state variance constraints, simultaneously. Also, the problem of steering a linear dynamical system with complete state observation from an initial Gaussian distribution in state space to a final one with minimum energy control was addressed in Chen et al. [19]. e studies on linear stochastic systems are more than nonlinear random systems. Besides, in conventional covariance control methods, according to the Riccati equation limiting condition, it is not always possible to obtain covariance placement feedback for each desired covariance. Since the reports for nonlinear random systems are few, the researchers are interested in the application of linear systems theories for nonlinear cases as far as it is possible without wasting too much precision and without loss of performance.
In this paper, the covariance control algorithm for nonlinear systems using covariance feedback is investigated. e novel contributions of this paper are as follows: firstly, it is shown that, under the condition that the linear approximation of a nonlinear system is a good representation of a nonlinear system, covariance control of nonlinear systems scenario involves the theory of covariance control based on the idea of the covariance feedback. Since, in practice with real conditions, a linear approximation could not be used for any prespecified covariance matrix and numerous class of nonlinear systems; for nonlinear systems, we introduce the theory of covariance control using the feedback linearization approach developed in Khalil Hassan [20]. Also, according to the feedback linearization and covariance feedback idea stated in [12,21], it is proved that any desired covariance matrix can be placed to its corresponding nonlinear system. For this purpose, we try to linearize the nonlinear stochastic system via feedback linearization. en, the existing covariance control theory is extended to address the nonlinear stochastic control problems by the application of the linear system covariance control method. By changing the variables method, the nonlinear system is transformed into an equivalent linear system, and then, by using the state feedback controller approach, the closed-loop system will be stable and linear. According to the obtained covariance equations for the new transformed linear closed-loop system, the standard linear state-space model is presented for the covariance system. States of the new modelled system will be the covariance of the main nonlinear system states and also the input of the presented covariance system will be the covariance of the nonlinear system input. en, based on the proposed model, mathematical approaches are going to be derived for nonlinear stochastic systems by using a state feedback controller and a covariance feedback controller. Finally, the simulation study is used to investigate the effectiveness of the introduced approach in a nonlinear case study.

Preliminaries and Problem Formulation
In this section, to obtain the linearized approximation of the nonlinear systems, two linearization ideas are brought that include a linear approximation of nonlinear stochastic systems and exact feedback linearization of nonlinear stochastic systems.

Linear Approximation of Nonlinear Stochastic Systems.
Consider a general zero-mean nonlinear stochastic system as follows: where w(t) is a zero-mean white noise andw(t) has covariance Q and f(x(t)) is an odd function relative to x(t), thereby all of the mean values are zero. is system could be also considered as a mapping function such as Ω: w ⟶ x. Linear approximation of the general system, presented by Leithead [22,23] in equation (1), can be considered by the following form: and the corresponding mapping function is defined by ψ: υ ⟶ z. e relationship between Ω and ψ is defined as follows [22,23]: where h is a nonlinear coupling operator defined by the following function [23]: Leithead [22] proved that the linear system Ψ is an acceptable representation of a nonlinear system Ω when the error covariance matrix E is small, where where E is the expectation operator. To define "small", the evaluation of the accuracy of equation (2) from equation (1) is required. is assessment (a measure of the accuracy) can have provided by the following covariance matrix of the error [22,23]: For some suitable positive symmetric matrix Z since p is defined as system steady-state covariance matrix and C would be defined as follows: 2 Complexity where variable E can be used to determine the accuracy of the low-order expansion when it is authentic. erefore, the linear system Ψ is an appropriate representation of the system Ω only if E is a minimum value, which means that the filtering action of (Ψoh) must be strong. Also, Tr(C) represents the convergence rate of an asymptotic expansion, and if C is small, the expansions will be reliable, thus there will exist an estimation of nonlinear systems.
Remark 1. In [22], it was proved that the error measurement criterion C is a function of the optimal covariance matrix p. erefore, in the proposed covariance control procedure, the selection of the optimal control covariance matrix p should be done in a way that is compatible with keeping the matrix C with a small value. For practical applications in zero-mean type systems, the appropriate estimation error value of the variable C should be less than 25%, and in nonzero-mean systems, the allowable estimation error range is assumed to be about 10%.
In the following, a suitable optimal choice for the nonlinear system can be achieved when the system nonlinearities are substituted by their stochastic-input characterizing functions as follows: with nonlinear coupling function H(x(t)) such as where the matrix A could be calculated by the following equation: where E is the expectation operator and P is the steady-state covariance of the states. To estimate the precision of the loworder development of the system, the measurement of E can be used when C is small. e main problem of this method is that the obtained estimation of the nonlinear stochastic system with estimation accuracy parameters (C, E) is dependent on the system states' covariance matrix. Moreover, the provided estimation's compatibility conditions, which are minimizing the equations (6) and (7), must be satisfied simultaneously.
erefore, in practice with real conditions, this method could not be used for numerous class of nonlinear systems. us, with this method, we will not be able to place any desired covariance matrix with high accuracy and always there would be a considerable error value based on the system's final results.

Exact Feedback Linearization of Nonlinear Stochastic
Systems. Using state feedback control and change of variables method that transforms the nonlinear system into an equivalent linearized system is one of the most commonly used ideas of practical nonlinear control design [20].
Consider a more general form for zero-mean nonlinear stochastic system as follows: where f, c, and g are smooth vector fields. Also, x(t), u(t), w(t), and y(t) are state vector, input vector, white noise, and output vector of system, respectively. w(t) is a zero-mean noise with covariance Q and f(x(t)) is a function relative to x(t). Consequently, all of the mean values are zero.
In an input-state linearization technique, the problem of feedback linearization is going to be solved within two steps.
First, we consider a state transform z � z(x) and input transform u � u(x, v) such that the nonlinear dynamics of the system would be transferred into a linear time-invariant dynamic as the following form: en, linear standard control techniques (such as pole placement) are used to design v(t). So, by using the designed linearization feedback and the suitable state transform z � z(x), the problem of covariance control and stabilization of the nonlinear system (11) would become a problem of stabilization and covariance control with the new linear system dynamic (12a) with the new input v(t).
It should be noticed that the suggested covariance feedback idea Khaloozadeh and Baromand [21] is completely independent of the designed stabilizer state feedback in the tracking or stabilizer problem. So, we try to determine the statistical features of the control signal by considering states' covariance feedback of the system (x(t) or z(t)), and then, we try to control the covariance of the main system states as possible. Because of the independence of this method of the input signal, it enables us not to have some limitations for designing the control signal u(t).

Complexity
Unlike most of the presented common methods for placing the covariance matrix for linear or nonlinear systems, we can practically consider the issue of the system output tracking at the same time as the covariance control issue is considered as well. Another thing to pay attention to is that the presented idea of feedback linearization is obtained with a state transform and also an input transform, with this issue varying with the presented common linearization idea in Leithead [22] and Chung and Chang [9], that was determined and considered for the limited work areas, but in the presented feedback linearization idea, we will consider no limitations for this issue. Moreover, the idea of using linearization feedback for nonlinear systems covariance control issues could be taken within two steps.
In the first step, by using the state transform z � z(x), the nonlinear system (11) would become the linear standard form (12a). e condition of the existence of such a state transform will be mentioned in the following.
In the second step, by using the idea of covariance feedback, the covariance of the control signal would be designed in such a way that the system states' covariance will be placed to the desired preset values. erefore, as a summary, the linearization feedback and state variable change z � z(x) could be done by the following steps.
First, the vectorfield g, ad f g, . . . , ad n− 1 f g should be determined for the nonlinear system (11) where ad f g � [f, g] � ∇gf − ∇fg as a Lie Bracket operator. If we prove the existence of two conditions which are controllability and reversibility Khalil Hassan [20], then the first state z i (the output function that leads to input-output linearization with a relative degree of n) can be calculated as the following equation: where Also, the state transform z(x) and input transform u(x) can be calculated as follows: where , and L f h � ∇hf are determined as a Lie derivatives of the function h to the vector field f. By using the linearized new states and obtained control law in the previous step, the mentioned nonlinear system can be transferred into the standard linear system form (12a). Furthermore, the unstable nonlinear system can be stable by using the pole placement idea. Now, the covariance feedback idea could be implemented by following to place the desired preset covariance matrix to the new linearized system.
Given that most of the nonlinear systems can be transformed into the equivalent linear system by using linearizable feedback, the feedback covariance idea can be used for controlling most of the nonlinear systems.

Description of Covariance System and Design of Covariance Controller
Covariance control theory was described in references to find feedback gain sets that led to assign suitable state covariance by these sets. According to these mentioned references, the following variance limitations will be satisfied: where δ i determines the root-mean squared (RMS) constraints of the variances related to the system states. In this paper, to find the solution for zero-mean nonlinear stochastic system (ZMNSS), it can be simplified to find the covariance state feedback gain, that attains a determined value of p to comply with the variance limitations (equation (11)). If the small value of the measured matrix C is selected and the strong filtering action (6) in statistical linearization is assumed or by considering the controllability and involutivity conditions for input-state feedback linearization idea, finally, the linearized system ( (2) and (12a)) will be obtained. Besides, in both cases, the nonlinear system can be specified as where Az(t) is the describing function of f(x(t)) and u(t) is a function of x(t) and v(t) in any suitable forms, and the applied feedback gain G in the state feedback stabilizing signal v(t) � Gz(t) is calculated based on the system parameters in (16a) by using the pole-placement method. Also A ∈ R n×n , D ∈ R n×m , B ∈ R n×m , z(t) ∈ R n , v(t) ∈ R m , and w(t) ∈ R m . On the other hand, the input noise of the system w(t) in equation (16a) satisfies the following statistical specification: where Q ∈ R m×m is the process noise covariance matrix. Also, with the assumption of E[u(t)] � 0, the positive definite covariance matrix of final states, p(t) and p(t), can be defined as follows: where 4 Complexity e objective is to impose lim t⟶∞ E(x(t)) � 0 and assume a prespecified positive definite covariance matrix for the states, x(t). We name this problem as a covariance assignment problem.
In [21], to solve the covariance assignment problem, a method based on the covariance feedback technique is presented. In this research, we utilize the covariance assignment for nonlinear systems via the covariance feedback method.
In the following discussion, the aim is to reach a stochastic control signal v(t) with mean zero and the covariance matrix, V(t) ∈ R m×m , such that the steady-state covariance p(t) converges to a prespecified value p d (t).
en, an additional action input v(t) is determined in a way to assign the prespecified covariance to a nonlinear system based on the covariance feedback technique. Figure 1 demonstrates the structure of the overall control system. us, with the new additive control signal, v(t), (16a) can be rewritten by where and v(t) is an additive input noise signal which is specified by a method to assign the prespecified covariance matrix via covariance feedback technique.
At first, the case with a zero-mean control signal v(t) is considered. en, the problem is referred to as a regulator problem such that, for v(t), the following assumptions is satisfied: Also, for additional input signal v(t), covariance matrix V(t) is defined as where For a z(t) with mean zero, the covariance of the system states is represented by Using ((20)-(24)), the differential equation matrix of p(t) can be derived by dp(t) dt

Covariance feedback controller
Nonlinear stochastic system Input u (t)

Additive action input ṽ(t)
State covariance estimator State feedback linearization w (t) Figure 1: e block diagram of the overall control system.

Complexity 5
As a result, the dynamic covariance system (25) can be reduced to the standard state-space model by defining A cov , B cov , and C cov as follows [24]: where B cov � I and the elements of A cov ∈ R ((n(n+1)/2)×(n(n+1)/2)) and d(t) are the state and disturbance matrices of the proposed state-space covariance system, respectively, and computed based on the nonlinear mapping of A � (A + BG) and d(t). Also, d(t) is deterministic, and we can assume it as a disturbance term. Also C cov is arbitrary output matrix of the presented covariance system.
It should be noted that, although the model (20) is stochastic, the given covariance state-space model (29) is deterministic. Also, if A cov and B cov are controllable, then there exists a solution to the covariance assignment problem.

Design of Covariance Controller.
In the previous discussions, for the state covariance of a nonlinear stochastic system, a deterministic and linear model was introduced based on the combination of stochastic approximation and feedback linearization and rearrangement ideas. But, designing the covariance feedback V cov (t) � Kp cov (t) (that imposes the action input v(t)) to control the covariance assigned in the specified matrix p d is the main problem. By the linearity of the covariance system presented in (29) using the introduced technique in Khaloozadeh and Baromand [21], with the integral state _ q(t) � r − C cov p cov (t), the new augmented state space is defined by where r is defined as the desired covariance vector and C cov � I and us, the covariance control problem results in the stability of the augmented system (30). Controllability of the pair (A, B) is the necessary condition to exist as a solution to the covariance tracking problem. e state covariance feedback V cov (t) in (23a) is then given as follows: e controller gains K and thus V cov (t) is then obtained using the pole placement method. e elements of the vector V cov (t) determine the covariance matrix of the action input, v(t) (i.e.,

V(t) � E[v(t)v(t) T ] ). us, v(t) must be produced such
that v has the covariance matrix V. Because of the regulatory of the problem in the original state system, the control signal v(t) is a white noise with mean zero and the covariance matrix V. erefore, in each iteration, the input signal v(t) will be derived as follows: where e n m i�1 is denoted as white noise. us, with the presented idea, the control strategy is composed of two stages. e first, state feedback signal v(t) � Gz(t) (feedback from the mean value of state) stabilizes and linearized the system. e second, covariance feedback signal v(t) imposes the input strategy such that the system's states adopt the predefined covariance value.

Remark 2.
In the presented covariance feedback and linear approximation idea, unlike most of the derived strategies based on the state feedback that they could not be able to place any desired covariance matrices, with this mentioned covariance feedback idea and obtaining a distinct control signal from the stabilizer control signal, we can even place any covariance matrices. erefore, we can calculate the desired covariance matrix that satisfies the expected conditions of the control problem for representing a better estimation of the nonlinear system according to the minimum permitted value of the nonlinear error signal. Finally, we can achieve the control goal of the problem.
In summary, the presented covariance assignment algorithm is presented in Algorithm 1.

Illustrative Examples and Numerical Results
is section is devoted to studying the ability of the proposed control technique. For this purpose, four nonlinear stochastic systems are considered. e simulations are accomplished after 100 Monte Carlo runs using the MATLAB R2014a software on a PC (processor: Intel (R) Core i5-4570 CPU @ 3.2 GHz, RAM: 4.00 GB, System Type: 64 bit). In the simulation, the presented methods do not require any additional assumption on the model; however, the result becomes ill-conditioned if the sampling time is large or if the poles of the system are very fast. In all simulation results, the sampling time selected is 0.05 sec.
is section is divided into two parts. e first part considers two simple systems. e performance of the introduced covariance control technique is investigated, and the ability of the presented control strategy is also compared with other alternative methods presented by Chung and Chang [9]. e second part considers two complex systems. For these systems, the performance of the introduced covariance control technique is investigated.

Performance Compression.
ese examples involve two simple zero-mean and nonzero-mean nonlinear stochastic systems that may be found in [22,23]. 6 Complexity Example 1. Consider a zero-mean nonlinear stochastic system described by where d � b � 1 and w(t) is zero-mean white noise with variance value 2. Assume that the goal of tracking is to satisfy the variance condition p d (t) � var(x(t)) � 0.3. According to the linear approximation idea and from [21,22], the linearized gain statistical matrix can be easily calculated as A � − 3p � − 1.5. According to the A, the linearized system is stable and it is not necessary to use state feedback (A � (A)).
Estimation error between the nonlinear system and linearized system is in Figure 2. By substituting A and B � D � 1 in (12a) and assuming p cov (t) � p(t) � var(x(t)), the covariance system parameters will be described as A cov � 2 × (− 1.5), B cov � 1, and C cov � 1.
Using pole placement and the state-space model of the system (30), the feedback gain matrix is calculated and the covariance vector of additive input V cov (t) (equation (32)) is computed as e performance compression between our covariance feedback control design and state feedback controller approach [9]) is shown in Figure 3. e conventional state feedback only guarantees the upper band of the desired state variance and fails to track the desired trajectory. Compared to conventional state feedback, our proposed controller can track the desired trajectory. e mean value of CPU times for 100 runs (with sample time 0.05 sec) is 16.15 sec. Low CPU time also verified that this approach is not too time-consuming.
Example 2. Consider the nonzero-mean nonlinear stochastic system described by If the small value of the measured matrix c is selected and the strong filtering action (6) in statistical linearization is assumed or by input-state feedback linearization idea, finally, in both cases, the nonlinear system can be specified as follows (16a): Step 1: consider a state transform z � z(x) and feedback control input (16b) such that the nonlinear dynamic (2) or (11) would be transferred into the linear dynamic (16a).
Step 2: online state covariance estimation of the main system (20).
Step 3: rearrange and modify the covariance vector V cov and P cov from (26) and (27).
Step 5: modify the covariance matrix V(k) from V cov (k) as covariance of control signal v(k).
Step 6: generate the covariance control signal v(t) with covariance matrix V(k) from (33). where d � b � 1 and w(t) is the zero-mean white noise with variance value 0.5. Assume that the goal of tracking is to satisfy the variance condition p d (t) � var(x(t)) � 0.1. e gain statistical matrix can be easily calculated as follows [22]: According to the A, the linearized system is stable, and using pole placement and the state-space model of the system (29), the feedback gain matrix is calculated and the covariance vector of additive input V cov (t) (32) is computed. e performance compression between our covariance feedback control design and state feedback controller approach [9] is shown in Figure 4.
Compared to conventional state feedback, our proposed controller can track the desired trajectory. As can be seen, the variance of the state will converge to the desired covariance matrix value with a relatively good convergence rate. e mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 24.96 sec.

Examples for More Complex Nonlinear Systems.
In methods based on linear approximation idea, the feedback gain is dependent on estimation accuracy parameters (C, E), so in examples that are more complex, we do not have an acceptable estimation or the obtained feedback gains do not have an acceptable response. In the next examples, the simulation results are presented for a wide class of nonlinear systems with more complexity.

Example 3.
To introduce the idea of feedback linearization, consider a nonlinear stochastic continuous system (11) with where w(t) is the zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix: To remove the nonlinear term, we can choose a change of variables and state feedback control as which yields where and v(t) is zero-mean additive covariance controller input. us, the stabilization problem for the nonlinear system has been reduced to a stabilization problem for a controllable linear system with a proper linear state feedback control law where the poles of the closed-loop systems are assigned to the value [− 1.5 + 0.86j − 1.5 − 0.86j]. It is worth noting that the values of k 1 and k 2 are obtained by applying the poleplacement method stated in Chang et al. [25]. Assume By computing A cov , B cov , and C cov � I and considering that the poles of the closed-loop covariance systems were assigned to the value − 2 − 2 − 2 − 3 − 3 − 3 , the feedback gain is calculated and the covariance vector of additive input V cov (t) will be computed as   Figure 4: State variance and control signal of controlled system with proposed controller and state feedback approach [9]: (a) covariance tracking problem; (b) control signal. 8 Complexity e performance of the proposed controller for this system is illustrated in Figures 5-7.
Simulation results proposed that the controller can track the desired trajectory. As can be seen, the covariance and variance of the states are converged to the optimal covariance matrix value with a relatively good convergence rate. e mean value of CPU times for 100 runs (with sample time 0.05 sec) is also 11.56 sec.

Complexity
where w(t) is zero-mean white noise with variance value 1. In this example, the goal is to force the system states to track the desired covariance matrix: To remove the nonlinear term, we can choose the change of variables and state feedback control as which yields zero-mean additivecovariance controller input. e proper linear state feedback control law is assumed to be as v(t) � [0 − 5]z(t), where the poles were assigned to the value [− 2 − 5] using the developed pole-placement approach in [25].
By assuming p cov (t) and V cov (t) similar to Example 3 and _ q(t) � 0.3 − p cov (t), computing A cov , B cov , and C cov � [1 0 0], and supposing that the poles of the closedloop covariance systems assign to the − 10 − 1 − 2 − 9 , the feedback gain is calculated and the covariance vector of additive input V cov (t) is computed as Additionally, the elements of the vector V cov (t) determine the covariance matrix T ] is the covariance of additive controller input v(t).
e performance of the proposed controller applied to this system is illustrated in Figures 8 and 9.
Simulation results proposed that the controller can track the desired trajectory with a relatively good convergence rate. e mean value of CPU times for 100 runs (with sample time 0.05 sec) is 58.29 sec.

Conclusion
In this research, the covariance control algorithm for nonlinear systems using covariance feedback was investigated. At first, we showed that, under the condition that the linear approximation of a nonlinear system is a good representation of a nonlinear system, covariance control of nonlinear systems scenario involves the theory of covariance control based on the idea of the covariance feedback. Since in practice with the real condition, a linear approximation could not be used for any prespecified covariance matrix and numerous classes of nonlinear systems or we have poor linear approximation condition, we introduced the theory of covariance control of nonlinear systems based on the idea of feedback linearization. We showed that, with the feedback linearization idea and covariance feedback approach, we can be able to place the desired covariance matrix to a nonlinear system with almost high accuracy in steady state.

Future Recommendation
Because in some of practical applications, the actual structure of the system is unknown, nonlinear, or time varying, for a grope of time variant stochastic systems, we can introduce the adaptive nonlinear control theory for covariance control. Also, we can introduce presented covariance control theory for more complex systems such as underactuated systems with external disturbances [26] or chaotic systems with nonlinear functions under uncertainties and time-varying delays [27,28]. Besides, the use of the idea of the prespecified covariance matrix placement is not limited to industrial or applied systems, and for example, the proposed system can produce various types of cyber attacks.

Data Availability
e data used to support the study are included within the article.

Conflicts of Interest
e authors declare no conflicts of interest