Automatic Control for Time Delay Markov Jump Systems under Polytopic Uncertainties

The Markov jump systems (MJSs) are a special case of parametric switching system. However, we know that time delay inevitably exists in many practical systems, and is known as the main source of efficiency reduction, and even instability. In this paper, the stochastic stable control design is discussed for time delay MJSs. In this regard, first, the problem of modeling of MJSs and their stability analysis using Lyapunov-Krasovsky functions is studied. Then, a state-feedback controller (SFC) is designed and its stability is proved on the basis of the Lyapunov theorem and linear matrix inequalities (LMIs), in the presence of polytopic uncertainties and time delays. Finally, by various simulations, the accuracy and efficiency of the proposed methods for robust stabilization of MJSs are demonstrated.


Introduction
Markov Jump Systems (MJS) are a class of parametric switching systems (SSs). The MJSs are represented as a set of systems that are constantly in transition between models with a Markov process with a limited state number. These systems can also be classified as a case of hybrid SSs, in which switchings are managed by a Markov chain the Ref. [1]. Mathematically, Markov jump systems are classified as random systems, in which the system matrices are randomly jumped at a series of discrete times managed by the Markov process, and in the time between these random jumps, these matrices are time-invariant. Due to their widespread use in practical systems, significant attention has been paid to them in the last few decades. These systems are used to model a variety of processes, including sudden changes in structures, such as accidental breakdowns and sudden disturbances, environmental changes, and changes in the internal connections of subsystems, power systems and solar panels, flight controllers, modeling abrupt changes in economic systems and modeling and predicting network control systems. Many researchers have addressed the problem of modeling Markov jump systems, analyzing their stability and efficiency, designing a variety of controllers, including state feedback [2], robust H ∞ [3], output feedback [4], and sliding mode control [5].
MJSs were first introduced by Krasovsky and Lidsky in 1961, who studied the control process of a system subject to stochastic changes and expressed the style of this class of systems [6]. Vanham was then one of the pioneering researchers in the development of these systems, developing mathematical models and algorithms suitable for complex dynamic systems, which are under natural constraints and random perturbations [7]. In the model of economics studied in the Refs. [8,9], it is considered that the state of the economy can be described as one of three functional modes: "normal", "rapid growth", or "recession". Additionally, switching between these states can be modeled as a Markov-chain. In 1983, Aswarder et al. developed Markovian jump systems based on optimal control and random estimation [10]. Since then, the stability study of MJSs has been increased significantly. This is due to the fact that most practical systems are inherently exposed to stochastic changes and sudden environmental disturbances. Markov jump systems are also used to analyze the stability of controlled flight systems, which operates under various perturbations [11,12].
The Markov jumps are used in the control and dynamic identification of power systems [13], in which a switching mechanism is used to identify sudden load changes, production unit disconnection, and transmission line defects. The security criterion that is used to determine the vulnerability of the current state of the system and network topology can also be represented by a finite-time Markov process and a model-dependent jump. In network control systems, data latency and loss are the two main phenomena that occur due to the existence of the network, which must be considered in the design of the controller; otherwise, the efficiency drops sharply. One of the methods of modeling these two phenomena is as an MJS system [14].

Literature Review
in the Ref. [15], the H ∞ controller developed MJSs, and the improvement of the transient behavior was studied. In the Ref. [16], the feedback controller was formulated, and the time delay effect was investigated. The passivity analysis was presented in the Ref. [17], and the effect of deception attacks was studied by designing an asynchronous control scenario. In the Ref. [18], the application of the theory of MJSs in robotic problems was addressed, and the control scheme was developed for disturbance rejection. A quantized control scheme was designed in the Ref. [19] for Roesser MJS, and the feasibility of a designed controller in some applications was investigated. In the Ref. [20], by the use of neural networks (NN), the nonlinear dynamics in MJSs were eliminated, and the system dynamics were converted to the linear sub-systems, and then an optimal controller was designed for linearized models. The self-triggered scheme was developed in the Ref. [21], and by designing an H ∞ -based control system, the signal boundedness was ensured. The fuzzy controllers were investigated in the Ref. [22], and the problem of stochastic stability was analyzed.
The nonlinear MJSs under time delay has been rarely studied. For example, in the Ref. [23], the exponential stability was analyzed under time delay, and some stability conditions were derived via a LMI approach. The event-triggered scheme was studied in the Ref. [24], and a H ∞ -based controller was designed for fuzzy MJSs. The time-varying delay problem was addressed in the Ref. [25], and the output boundedness was ensured by the suggested LMI technique. In the Ref. [26], the time delay was studied by the use of fuzzy rules, and by the Lyapunov-Krasovskii approach, some stability conditions were suggested. In the Ref. [27], the discrete-time MJSs were taken into account, and the stochastic stability was proven. In this paper, some conditions were extracted to guarantee stability against time delay. The L1-gain analysis was developed in the Ref. [28] for MJS, and the stability against time delays was investigated. In the Ref. [29], the L ∞ scheme was developed for fuzzy MJSs, and the effect of transmission delay was studied. In the Refs. [30][31][32][33][34], the various versions of a sliding-mode controller (SMC), such as a simple SMC, integral SMC, backstepping SMC, and terminal were studied, and they were applied on an under-actuated quadcopter.

Novelties
The literature review shows that the state-feedback control system has been designed for the delayed MJSs. However, in most of them, the system equations are considered nominally, or structural uncertainties such as ∆A = EF(t)H are used in control design, in which ∆A is a real function, E and H are known matrices, and unknown F(t) satisfies F(t) T F(t) ≤ I. According to the literature review, in very few studies, the control design for MJSs under polytopic uncertainties was discussed. The main difficulties are the effect of polytopic uncertainties, time delays, and restrictions on LMI sets. The basic novelties are: • Introducing a suitable Lyapunov-Krasovsky function to analyze the stability of the time delayed MJSs, and extract a sufficient condition in the LMI form to find a higher delay bound; • Analyzing the stability of the MJSs in the presence of polytopic uncertainty and generalizing the obtained results; and • Designing the controller using a mode-dependent state feedback approach and finding appropriate control gains in LMI form.

Preliminaries
Definition 1. The polytopic-type uncertainties are defined as follows where ν i denotes the number of the vertices of Λ i , β ir (r = 1, . . . , ν i ) are the time-invariant uncertainties, A ir , A dir and B ir are known matrices and A i , A di and B i , are unknown matrices.

Definition 2.
For the finite Φ(t) ∈ R n and all initial modes r 0 ∈ S, the stochastic stability of MJS is ensured if the following inequality exists: where u(t) = 0, χ(q, Φ, r 0 ) represents a solution, and Φ and r 0 are initial conditions. Definition 3. The weak infinitesimal generator of a stochastic process (J ) is written as: Lemma 1. Finsler's Lemma: Consider real matrices Ω ∈ R n×n and the full row rank matrix F ∈ R m×n , then we can write: 1.
For a scalar µ ∈ R we can write µ + F T F < 0; 3.
The following condition holds: F ⊥T ΩF ⊥ < 0 Lemma 2. Consider matrices W, U, V, T 1 , T 2 and suppose that T 1 > 0, T 2 > 0, and that U/V have column/row full ranks. Then there exists a matrix Σ such that: if, and only if: where, i = 1, 2, . . . , N, and Note that F ⊥T i denotes the orthogonality complement of F i and Θ i (ι) = Θ i is defined.

Proof. The stochastic Lyapunov-Krasovskii functional is considered as
where P i = P(r(t)) when r(t) = i. Consider J for process {χ t , r t }, then, for each r t = i(i ∈ S), we have: From Jensen's inequality, one has One can achieve Therefore it is obvious that J V(χ(t), r(t)) < 0 holds if: Regarding the Newton-Leibniz formula, one has or equivalently, The equalities (20) and (17) hold, if and only if: This completes the proof.
Proof. According to Lemma 2, define T 1i = µ i I ∈ R 4n×4n with enough large scalars , with full row ranks, such that: Then, the LMIs (22) hold by using the change of variable Ψ i = µ i Σ i . Moreover, it is evident by the system (1) by (u(t) = 0) that in the presence of time delays it is stochastically stable, and this completes the proof.

Remark 1.
Before expressing the next Theorem, since LMIs (22) are affine in respect of A i and A i , sufficient conditions for investigating the robust stochastic stability in the presence of polytopic uncertainties can be acquired by considering the proposed LMIs only in polytopic uncertainty vertices for each mode. In other words, the corresponding system subject to polytopic uncertainties has robust stochastic stability if the following LMIs have feasible solutions: where: Proof. For each i, we multiply inequalities of (24) by scalars β ir , r = 1, . . . , N, where ∑ ν i r=1 β ir = 1, β ir > 0 and we sum up the achieved results with each other, and the LMIs of (22) are extracted.
Since Lyapunov matrices (P i , Q, Z) of LMIs (24) are independent of the system matrices, the suggested results are conservative. Therefore, the next Theorem will be proposed to reduce the conservatism of the robust stability analysis of the MJS. In this regard, the following matrices are defined where P Theorem 3. Consider MJS (1) with the system matrices belonging to polytopic uncertainty in the form of (3), the system has robust stochastic stability if, for ι > 0, there exist matrices Ψ i , P (r) i > 0, Q (r) > 0, Z (r) > 0 and scalars µ ( i r) > 0, such that: where where I ir indicates the N-tuple sets of the following form: Hence, for each π ir , there exists the following matrix: Proof. Considering polytopic uncertainties and multiplying LMIs of Theorem 2 by β jk , k = 1, . . . , ν j summing them up leads to where Now, multiplying (31) by scalars β ir , r = 1, 2, . . . , ν i and summing them up, we conclude: where P [π ir ] = P 1 (β 1 ) . . .

State Feedback Controller
Designing u(t) = K(r t )χ(t) and (1), one has Additionally, in the LMI extracted in Theorem 2, due to open-loop stability analysis, we should only replace A i with the A i + B i K i matrix. Thus, we will have: On the other hand, in Theorems 2 and 3, it was said that Ψ i = µ i Σ i , in which Σ i ∈ R 2n×4n is a matrix with an optional row rate, and the main structure of it will be chosen as below: In the following Theorem, the method of extracting interest of the state feedback controller due to nominal stability of the closed loop system is explained: Theorem 4. The system in relation (33) has the potential to change to random stability. If, for the given delay ι > 0, Σ 11i and Σ 14i matrices M 1i , M 2i , N i , P i > 0, Q > 0 and Z > 0, and scalar µ i > 0 exists in a way that the following LMIs can be established simultaneously: where : If the above LMI is affordable, controller K i is defined from the relation K i = µ −1 i N i .

Proof.
Where replacing Σ i is suggested from the relation in the F T i Ψ i matrix of LMI in relation (34), we will have Ψ i = µ i Σ i and Considering this point that choosing a Σ i matrix introduction is for the designer, Σ 11i and Σ 14i can be assumed as given (in other words, these two matrices are design parameters and their initialization data are defined by the designer). Additionally, Σ 21i & Σ 24i are undefined (these two matrices are free parameters). Now, applying the variables' changes N i = µ i K i , M 1i = µ i Σ 21i , M 2i = µ i Σ 24i , relation 31-3 will be a linear matrix according to parameters N i , M i1 , M i2 , and µ i , which is in fact the defined Ξ i in relation (36), and so the relation (36) is a type of LMI, and in the case of being affordable, the optimum values of anonymous parameters will be specified, and the interest of mode-dependent controllers will be defined from relation K i = µ −1 i N i . So far, we have managed to extract an optimal mode feedback controller to ensure closed loop stability for nominal systems mentioned in relation (33). In the next Theorem, the target is to generalize the obtained results for system equations in the presence of polytopic uncertainties. To gain this goal, assume A i , A di , B i are undefined matrices but are related to the given Λ i multidimensional. In Theorem 3, we have proved that an open loop system in the presence of ι > 0 delay has random stability if Ψ i , P (r) i > 0, Q (r) > 0, Z (r) > 0 matrices exist for i = 1, 2, . . . , N, in the way that the following LMIs are established simultaneously: To stabilize the closed loop system, it is sufficient to replace A ir + B ir K i instead of A ir in the F ir matrix, and the Theorem will be defined as below: Theorem 5. The system in relation (33) with a systemic matrix containing multidimensional uncertainty has randomly stable stability if, for ι > 0 delay Σ 11i , Σ 14i , the given M 1i , N i , P (r) i > 0, Q (r) > 0, Z (r) > 0, matrices and µ i > 0 scalar for i = 1, 2, ...N, r = 1, . . . , ν i LMIs are established simultaneously: . . , N, r = 1, . . . , ν i , ∀π ir ∈ I ir where : If the above LMI is feasible, the K i controller is determined from the K i = µ −1 i N i relation.
Proof. The proof procedure is exactly the same as Theorem 3. We know that relation (36) LMIs in Theorem 4 that have been achieved for designing a randomized stabilizer controller closed loop system in the presence of delay are affine to the Ξ ir matrix. Now, considering multidimensional uncertainty in systemic matrices and successive multiplication of relation (39) inequalities in β jk suitable scalars for their summation and repeating this method, this time-successive multiplication of the parties in β ir suitable scalars for r = 1, . . . , ν i and their summation will have: In relation (39), it is seen that for any (A i , A di , B i ) ∈ Λ i orΞ i ∈ Λ i and i = 1, ...N is confirmed in which a Lyapunov nominal matrix replaced with Lyapunov uncertainty matrices. Thus, in Theorem 5, we managed to achieve a sufficient condition with less conservation to relation (36) LMIs for designing a stable stabilizer controller due to ensuring randomized closed-loop stability under polytopic uncertainty in systemic matrices, and this section is thus also complete.

Simulations
Example 1. In this example, the stability of linear MJLS with nominal time delay is investigated.
In Theorem 2, we proved that system (1) is stable if, for delay ι > 0, there exists a full-rank matrix Ψ i and Laypunov matrices P i > 0, Q > 0, Z > 0 and scalar µ i > 0 such that the LMI (22) is satisfied. A system with the following matrices is considered: where, For ι = 0.756, we obtain:  It should be noted that, in comparison with the method of [37], the upper bound (UB) of the time delay is smaller than our method. In the LMI of [37], if m = 1, the UB of the time delay is obtained as ι = 0.377. The other advantage of our method is that, in our LMIs, the systematic matrices are separated from Lyapunov matrices. Then, the conservatism of our approach in the presence of polytopic uncertainties is reduced.

Example 2.
In this example, the effectiveness of state feedback is evaluated. In Theorem (4), we proved that for delay ι > 0, asymptotic stability is ensured if u(t) = K(r t )χ(t), and there exist M 1i , M 2i , N i , P i > 0, Q > 0, Z > 0 and scalar µ i > 0 such that LMIs (36) are satisfied. The system matrices are considered as: where, For ι = 0.1.05, we obtain: For the initial χ 0 = 1 0.5 T , the results are given in Figure 3. The Markovian changes are depicted in Figure 4. The control signal is given in Figure 5. It should be noted that, in comparison with the method of [35], the UB of the time delay is smaller than our method. In the LMI of [35], the stability is proved for ι = 0.47, while in our suggested approach, we obtain ι = 1.05. We see that states are converged compared to the larger time delay. Example 3. In this example, the stability is investigated in the presence of ploytopic uncertainties.
To consider ploytopic uncertainties, the system matrices are written as: The matrices A ir , A dir and B ir are known matrices. In Theorem 5, we proved that for delay ι > 0, the stability is ensured if there exist Ψ i , P > 0, such that the LMIs (39) are satisfied. For examination, the system matrices are considered as: where, For time delay ι = 0.62, and initial condition χ 0 = 1 0.5 T , the results are given in Figure 6.
The Markovian changes are depicted in Figure 7. The control signal is given in Figure 8. We see that χ 1 and χ 2 are converged compared to the time delay ι = 0.62 and polytopic uncertainties.  Example 4. In this example, the feedback controller for MJLSs in the presence of polytopic uncertainties and time delay is evaluated. The details of stability are given in Theorem 5. The system matrices are considered as: where, For ι = 0.74, we obtain: For initial condition χ 0 = 1.4 0.3 T , the results are given in Figure 9. The Markovian changes are depicted in Figure 10. The control signal is shown in Figure 11. We see that χ 1 and χ 2 are converged compared to time delay ι = 0.74 and polytopic uncertainties. Remark 2. The paper proposed a practical approach for MJSs, such that in addition to polytopic uncertainties, the effect of time delays was also considered. The conventional restrictions were made easier and the suggested controller can support more levels of time delays.

Conclusions
In this paper, the target was to analyze stochastic stability and design the stable stabilizer feedback controller for the delayed-MJLS systems in the presence of polytopic uncertainties. As it can be observed in the literature overview, systemic and polytopic matrices of the suggested Lyapunov-Krasovskii function exists in the extracted results of LMIs, and this causes more conservation for generalizing the polytopic uncertainty mode. For this target, in this paper, we managed to separate systemic matrices and Lyapunov matrices in the suggested Lyapunov-Krasovskii function to generalize the results of polytopic uncertainty with less conservation. The main results were divided into two main sections: stabilization analysis and feedback controller design for delayed MJSs. Four theorems were presented to give sufficient conditions to ensure stability in the presence of time delay and uncertainties. In simulation sections, four examples were presented to verify the obtained criteria. In the first example, the stability of a linear MJLS with nominal time delay was investigated. It is shown that the upper bound (UB) of time delay is bigger for our method in comparison with other conventional approaches. Additionally, in our LMIs, the systematic matrices were separated from Lyapunov matrices. Then, the conservatism of our approach in the presence of polytopic uncertainties was reduced. In the second example, the effectiveness of state feedback was evaluated. It was shown that the designed controller can support larger time delay. In the third example, the stability was investigated in the presence of polytopic uncertainties, and finally, in the last example, the feedback controller for MJLSs in the presence of polytopic uncertainties and time delay was evaluated. Simulation results show that the presented theorems are more effective, and in comparison with conventional methods, the stability was achieved in a higher upper bound of time delays. Additionally, it was shown that the conservative conditions became easier. For future studies, imperfections can be taken into consideration in switching with delay. Data Availability Statement: The study do not report any data.