Optimal Tracking for Periodic Linear Hybrid Systems

This article provides a comprehensive characterization of the quadratic optimal tracking problem for hybrid systems with linear dynamics undergoing periodic time-driven jumps. Solutions to such a problem are proposed for both the finite horizon and the periodic cases. Furthermore, it is shown that if the reference signals are not known in advance, then the best control strategy to deal with the worst case reference signals is to simply minimize the (scaled) outputs. Finally, the derived optimal solutions are used to solve two relevant control problems, which are the reconstruction of vector fields from noisy measurements of the corresponding flows and the estimation of the time derivatives of a periodic, sampled, and noisy signal.

and by the jump dynamics where x ∈ R n is the state, u ∈ R m is the flow input, v ∈ R p is the jump input, A, E ∈ R n×n , B ∈ R n×m , F ∈ R n×p , τ (0, 0) = 0, and x(0, 0) = x 0 ∈ R n . In (1), the constant τ M > 0 imposes a fixed dwell time constraint between two consecutive jumps, which occur every τ M time instants due to the dynamics of the timer variable τ . Namely, complete solutions to system (1) are defined on the hybrid time domain which is therefore a priori fixed. Given k ∈ N, the shortcut t k := kτ M is used to denote the jump times.
Letting φ(t, k, x 0 , u, v) denote the solution to system (1) at hybrid time (t, k) ∈ T with initial condition x(0, 0) = x 0 , flow input u, and jump input v, define the flow output y ∈ R s and the jump output z ∈ R of the hybrid system (1) as where C ∈ R s×n and G ∈ R ×n . By (2), it is assumed that no direct input-output connection is present, but the results given in this article can be easily extended to such a case at the expense of a much more complex notation. See [17,Remark 1] for a comparison between the lifting approach proposed in [19] and the framework reviewed in this section and [13] for the characterization of the structural properties of systems (1) and (2).

III. HYBRID LQ OPTIMAL TRACKING OVER FINITE HORIZON
The main goal of this section is to solve the LQ optimal tracking problem for systems (1) and (2) over a given finite horizon. Given the initial condition x 0 ∈ R n , a continuous referenceȳ ∈ R s , a discrete referencez ∈ R , a final hybrid time (T, K) ∈ T , T ∈ R, K ∈ N, and two positive-definite matrices: 1) R ∈ R m×m ; and 2) L ∈ R p×p , the hybrid LQ optimal tracking problem over finite horizon aims at determining the flow input u and the jump input v that minimize the cost functional as The cost functional J T,K x 0 is the extension of classical quadratic tracking cost functions for continuous-time and discrete-time linear systems [20]. In fact, it penalizes the deviation of the system outputs from the desired reference trajectories and the control effort, both in continuous-and discrete-time.
Theorem 1: Consider systems (1) and (2) and the cost functional (3). Let P , h, and c be the solutions defined on the hybrid time domain T to the hybrid system with flow dynamics and jump dynamics wherez + (t k , k − 1) denotesz(k), and final condition Then, the solution to the hybrid LQ optimal tracking problem over finite horizon is and the corresponding value of the cost J T,K The statement follows from the same reasoning used in [17,Sec. 3], but by using the cost-to-go function V (t, k, x) = x P (t, k)x + 2h (t, k)x + c(t, k) and considering the additional terms appearing in the continuous and discrete-time Hamilton-Jacobi equations due to the presence of the reference outputs y andz.
Remark 1: The results given in Theorem 1 hold, even if τ (0, 0) = 0 and τ (0, 0) = τ 0 ∈ (0, τ M ]. In fact, by construction, the function V (t, k, x) = x P (t, k)x + 2h (t, k)x + c(t, k) is the cost-togo function for all (t, k) ∈ T and all x ∈ R n . Therefore, even in τ (0, 0) = 0, the control inputs (5) solve the tracking problem and the corresponding value of the cost J T,K x 0 is given by

IV. PERIODIC HYBRID LQ OPTIMAL TRACKING
Suppose that the reference flow outputȳ is (Kτ M , K)-periodic, i.e., y(t, k) =ȳ(t + Kτ M , k + K) for all t 0 and all k ∈ N, and that the reference jump outputz if K-periodic, i.e.,z(k + K) =z(k) for all k ∈ N. In this case, rather than aiming at determining a solution to the finite horizon optimal control problem given by (1)-(3), one may envision a control policy that minimizes And, such that the closed-loop system admits a periodic trajectory, i.e., the closed-loop solution x(t, k) to the hybrid system (1) starting at x 0 satisfies x(Kτ M , K) = x 0 , for some x 0 ∈ R n . Note that this periodicity constraint was absent in the optimal tracking problem over finite horizon considered in Section III. As for the cost defined in (3), the one defined in (7) is the extension of classical quadratic tracking cost functionals for continuous-and discrete-time linear systems. This problem is usually referred to as periodic optimal tracking problem [21]. In view of the periodicity constraint, an optimal periodic control can obviously be obtained by periodic extension of any solution to problems (1), (2), and (7).
Lemma 1: Consider systems (1) and (2) and the cost (7). If there exists a solution to the boundary value problem defined over the hybrid time domain T , with flow dynamics and jump dynamics and the boundary conditions then the control inputs solve the periodic hybrid LQ optimal tracking problem.
Proof: Following the same reasoning used in [22,Th. 2.1], if there exists a function V (t, k, x) such that for some function σ : N → R, which satisfies and such that the inputs are such that x(t K , K) = x 0 for some initial condition x 0 ∈ R n , then u * and v * solve the hybrid periodic optimal control problem. Letting c(t, k) be the solution to (4c) and (4f) with arbitrary c(t K , K), consider the candidate solution Since P and h satisfy (8) and c satisfies (4c) and (4f), the function V satisfies (11). Moreover, since P (0, 0) = P (t K , K) and h(0, 0) = h(t K , K), the function V satisfies Finally, since the inputs given in (9) satisfy (12) and the corresponding solution x to system (1) from the initial condition x(0, 0) satisfies x(T, K) = x(0, 0), such inputs solve the periodic hybrid LQ optimal tracking problem.
Using the results given in Lemmas 1 and 2, the following theorem provides a constructive procedure to determine the solution to the periodic hybrid LQ optimal tracking problem.
Theorem 2: Consider systems (1) and (2) and the cost functional (7). If systems (1) and (2) are stabilizable and detectable, then there exists a solution to the periodic hybrid LQ optimal tracking problem.
Proof: If systems (1) and (2) are stabilizable and detectable, then the hybrid system governed by the flow dynamics iṡ Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
when (τ, x) ∈ [0, τ M ] × R n , and by the jump dynamics  σ). Therefore, also the eigenvalues of Φ(0, τ M )W = D are in C g , thus implying that the rank condition given in (14) holds. By Lemma 2, this implies that the boundary value problem given by (8a), (8b), (8d), (8e), (8g), and (8h) admits the solutions P ∞ and h ∞ , where, with a slight abuse of notation, P ∞ is the periodic continuation of the solution to (13) and h ∞ is the solution to the system defined over the hybrid time domain T with flow dynamics and jump dynamics where h 0 is the solution to the linear system of equalities Then, it remains to prove that there exists x 0 ∈ R n such that the solution to the hybrid system with flows dynamicṡ

and by the jump dynamics
Thus, by considering that the eigenvalues of D are in C g , the initial condition x 0 leading to periodic trajectories of (18) is Letting x ∞ be the solution to system (18) with x(0, 0) = x 0 , since P ∞ , h ∞ , and x ∞ solve the boundary value problem (8), the statement follows by Lemma 1.
The proof of Theorem 2 is constructive and allows one to determine a solution to the periodic hybrid optimal tracking problem by explicitly solving the problem given in (8) by determining the solution to a system of linear equations. The following proposition provides an alternative method to determine such a solution. (1) and (2) and the cost functional (7). Suppose that systems (1) and (2) are stabilizable and detectable. Let h N be the solution to system (16) with final condition h N (t NK , NK) =h, whereh is an arbitrary vector in R n . Then, one has that lim N →∞ h N (0, 0) = h 0 , where h 0 is the solution to (17).

Proposition 1: Consider systems
Proof: By the proofs of Lemma 2 and Theorem 2, sinceȳ is Therefore, since the eigenvalues of Φ(0, τ M )W are in C g , the statement holds.
By using Proposition 1 and [17, Lemma 1], an approximate solution to the boundary value problem (8) is given by the solution to system (4) over the hybrid time domain T with P (t NK , NK) = 0 and h(t NK , NK) = 0 for N → ∞, i.e., the solution to system (4) tends to one solution to the boundary value problem (8) as the final time goes to infinity. This additionally shows that the solution to the periodic optimal tracking problem is a good approximate of the solution to the optimal tracking problem over finite horizon, provided that the reference trajectories are periodic and the horizon is sufficiently large. The following corollary shows that such a solution has particularly desirable properties. In particular, if one applies the periodic extension of such inputs to the hybrid system (1), its solution tends to the periodic optimal one. Corollary 1: Let systems (1) and (2) be stabilizable and detectable and let P c , h c , and x c be the periodic continuations of the solution to the boundary value problem (8) given above. Then, letting x be the solution to system (1) with the control inputs from any initial condition x 0 ∈ R n , one has that Proof: Let ϕ(t, k, x 0 ,ū,v) be the solution at time (t, k) ∈ T to the hybrid system governed by the flow dynamicṡ and by the jump dynamics  (1) and (20) are given by ϕ(t, k, Thus, the statement follows by the fact that: by the same reasoning used in the proof of Theorem 2. Using the results stated in Corollary 1, the following remark highlights further properties of the determined optimal solution Remark 2: Let ϕ be defined as in the proof of Corollary 1, by [17,Th. 4] there exist c > 0 and ρ ∈ (0, 1) such that Therefore, a straightforward consequence of Corollary 1 is that the control inputs given in (20) actually minimize the average cost defined as lim K→∞J K , i.e., such inputs constitute a solution to the optimal tracking problem over infinite horizon for periodic reference signals.
The following remark compares the results given in this section with those given in Section III.
Remark 3: Although the problems considered in this and the previous sections seem similar, their solution is in fact rather different. First, in order to determine a solution to the optimal tracking problem over finite horizon defined by (1)-(3), no assumption is required about systems (1) and (2), whereas in order to determine a solution to the periodic optimal tracking problem defined by (1), (2), and (7), one has to assume that systems (1) and (2) are detectable and observable. Furthermore, while the solution of the former problem is given in terms of the solution to a hybrid dynamical equation to be solved backward in time, i.e., the hybrid Riccati equations (4), the solution of the latter is given in terms of the solution to a two-point boundary value problem, i.e., (8). These differences are due to the presence of an additional constraint in the periodic optimal tracking problem, which is the existence of an initial condition that makes the corresponding trajectory of the closed-loop system periodic.

V. CASE OF UNKNOWN REFERENCE OUTPUTS
In the case that the reference outputsȳ andz are not known a priori, a possible strategy to deal with the reference tracking problem is to recur to the framework of zero-sum hybrid differential games [16]. Namely, by defining the cost where r, l > 0, one may attempt at determining control inputs u and v and reference outputsȳ andz that constitute a saddle point of the cost functionalJ T,K x 0 , such that for all bounded control inputs u and v and for all bounded reference signalsȳ andz. If such inputs and reference signals exist, then u , v andȳ ,z constitute a Nash equilibrium of the zero-sum hybrid game defined by systems (1) and (2) and the cost functional (21). As for the costs defined in (3) and (7), the one defined in (21) is the extension of classical cost functionals arising in the zero-sum dynamic game for continuous-and discrete-time linear systems.
Theorem 3: Let systems (1) and (2) and the cost functional (21) be given, and suppose that r, l > 1. Then, letting Π be the solution defined on the hybrid time domain T to the hybrid system with flow dynamics and final condition the Nash equilibrium of the zero-sum hybrid game is Proof: Following [24, Sec. 6.2 and 6.5] and the reasoning used in the proof of Theorem 1, if there exists a function V (t, k, x), such that , then the solutions to the abovementioned min-max problems constitute a Nash equilibrium of the game. Since the argument of (24a) [respectively, (24b)] is convex in u (respectively, v) and concave inȳ (respectively,z), if Π satisfies the hybrid dynamics (22), then V (t, k, x) = x Π(t, k)x satisfies the Hamilton-Jacobi-Isaacs equations (24), and the corresponding saddle points are given by (23).
Since r, l > 1, both (r/[r − 1])C C and (l/[l − 1])G G are the positive semidefinite matrices. Therefore, the existence of a unique solution to (22) is guaranteed by classical results on the differential and discrete matrix Riccati equations.
The following remark discusses the case r, l 1.

Remark 4:
In the case that r 1 or l 1, by inspecting (24), it can be easily derived that there does not exist any Nash equilibrium for the considered zero-sum hybrid game. It can be easily observed that in these two cases, the player choosing the reference signalsȳ andz is incentivized to let them take arbitrarily large values.
The following remark frames the Nash equilibrium determined in Theorem 3 in terms of an optimal control problem.
Remark 5: By comparing the results stated in Theorem 3 with [17, Th. 1], it appears evident that the control inputs constituting the unique Nash equilibrium of the zero-sum hybrid game (1), (2), and (21) are those minimizing the cost and that the corresponding reference signals are given by the scaled outputs of the closed-loop system with these inputs.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
A similar result can be obtained in the case that the reference signalsȳ andz are (Kτ M , K)-periodic and K-periodic, respectively, by considering the cosť In this case, one aims at determining control inputs u • and v • and reference outputsȳ • andz • that constitute a saddle point of the cost functionalJ, such thať for all bounded control inputs u and v, such that the closedloop system admits a periodic trajectory for some initial condition x 0 ∈ R n , and for all bounded reference signalsȳ andz, which are extended periodically outside T ∩ ([0, Kτ M ] × {0, . . . , K}) by letting y(t, k) = y(t + Kτ M , k + K) for all t 0 and z(k + K) = z(k) for all k ∈ N. The solution to such a problem is given in the following theorem. Theorem 4: Let systems (1) and (2) and the cost functional (26) be given, and suppose that r, l > 1. Let systems (1) and (2) be stabilizable and detectable. Then, letting Π ∞ be the solution to the following twopoint boundary value problem: for all σ ∈ [0, τ M ], and The solution to the periodic zero-sum hybrid game is  (27). Furthermore, since systems (1) and (2) are also detectable, the control inputs u • and v • given in (28a) and (28b), respectively, are stabilizing. Thus, following the same reasoning used in [22,Th. 2.1], if there exists a function V (t, k, x) that satisfies (10) and (24), and such that the inputs attaining the minimum on the right-hand side of (24a) and (24b) are such that x(T, K) = x 0 for x(0, 0) = x 0 and some x 0 ∈ R n , then such inputs constitute a solution to the periodic zero-sum hybrid game. Note that the function V (t, k, x) = x Π ∞ (t − t k )x satisfies these properties with σ(K) = 0, and the corresponding inputs, given in (28a) and (28b), are such that if the closed-loop system is initialized at x 0 = 0, then x(T, K) = 0.
The following remark relates the Nash equilibrium determined in Theorem 4 to the solution to an optimal control problem over infinite horizon.
Remark 6: By comparing the results given in Theorem 4 and in [17,Th. 3], the control inputs constituting a solution to the periodic zerosum game given by (1), (2), and (26) are those minimizing the cost functional given in (25) for T + K → ∞. Furthermore, by [17,Th. 4], if systems (1) and (2) are stabilizable and detectable, then the periodic continuation of the control inputs u • and v • makes the closed-loop system asymptotically stable. Therefore, a direct consequence of Theorem 4 is that the control inputs and the reference signals given in (28) actually constitute a Nash equilibrium of the zero-sum hybrid game over infinite horizon given by (1) and (2), and lim K→∞J K ; see also Remark 2.

VI. APPLICATIONS TO VECTOR FIELD RECONSTRUCTION AND DERIVATIVE ESTIMATION
In this section, the results given in the previous sections are used to reconstruct vector fields from noisy measurements of the corresponding flows (see Section VI-A) and to estimate the time derivatives of a periodic signal (see Section VI-B).

A. Reconstruction of Vector Fields
Let f : R n → R n be a continuously differentiable function and assume that the systemξ where x denotes the closed-loop solution of the tracking problem initialized at x(0, 0) = [ ξ 0ξ 0 ] , withξ 0 given by the solution to the following quadratic problem: and using any function approximation technique. Example 1: In [25], a technique was proposed to design a vector field f so that the trajectories of system (29) tend to a given set with the aim of designing safe paths of motion for a mobile robot. The objective of this example is to solve a converse problem, i.e., given paths of motion of the mobile robot, find the vector field governing its dynamics. Following [25], 100 different robot trajectories have been simulated initializing ξ 0 at random using a Gaussian distribution and letting A zero-mean disturbance with standard deviation 0.01 has been added to the samples of the trajectories of system (29), so as to account for position measurement errors. Then, the technique outlined above, with τ M = 0.1, R = 10 −6 I, and L = I, has been used to gather pairs (ξ,f (ξ)). Such pairs have finally been used to fit a polynomial modelf of total degree 3 for the vector field f using classical linear regression [26]. Fig. 1 shows the stream plots of the vector fields f anď f . As shown by such a figure, the vector field obtained by using the proposed technique is a good approximation of the one governing the motion of the mobile robot.

B. Estimation of Time Derivatives
Let ς : R → R be a given T -periodic smooth signal. The solution to the periodic optimal tracking problem given in Section IV can be used to estimate the time derivatives of ς from its sampled measurements. Namely, letȳ(t, k) = 0 andz(k) = ς(kτ M ), k = 1, . . . , K with K = T /τ M , define and compute a solution to the periodic optimal tracking problem given by (1), (2), and (7). Then, the closed-loop solution of the tracking problem initialized at x(0, 0) = x 0 , where x 0 , given by (19), is an estimate ofς. Example 2: Consider the signal ς(t) = cos([π/5]t) − 0.5 cos ([2π/5]t) + 0.1 cos([3π/5]t). The objective of this example is to estimate the time derivative of this signal from noisy sampled measurements. Letting τ M = 0.2 and K = 50, the signalz and the hybrid systems (1) and (2) have been defined as above. A Gaussian noise with zero mean and standard deviation, i.e., 0.1, has been added to the samplesz, so as to account for measurement noise. Finally, the technique given in Section IV has been used to solve the periodic tracking problem, with R = 1 and L = 1. Fig. 2 shows the closed-loop trajectory of system (1), the signal ς, its time derivativeς, the noisy samplesz, the estimate ofς obtained as (z(k + 1) − z(k))/τ M , and the one gathered using an finite impulse response (FIR) differentiator of order 10. As shown by such a figure, despite the signal ς is sampled and the samples are affected by measurement noise, the proposed method is capable of determining a good estimate of the time derivatives of such a signal. In particular, the obtained estimates are more reliable than those gathered using dirty derivatives and FIR differentiators in the presence of measurement noise.

VII. CONCLUSION
A solution to the LQ optimal tracking problem was proposed for the hybrid system with linear dynamics and time-driven periodic jumps both in the finite horizon and periodic cases. By means of a game theoretic formulation, it was shown that if the reference signals are not known in advance, then the best control strategy that allows one to cope with the worst case references is to minimize the (scaled) outputs of the system. Finally, it was shown that how to use the derived solution to solve two relevant control tasks: 1) the reconstruction of vector fields from noisy measurements of the corresponding flows; and 2) the estimation of the time derivatives of a periodic, sampled, and noisy signal.
The results given in this article can be adapted to deal periodic optimal switching control problems [27], by adding an additional state, which is constant during flow and whose postjump values equal the discrete-time input v, and with impulsive optimal tracking problems [28], by letting B = 0.
Comparing the results given in this, not with those in [17], note that although the problem considered was different (namely, optimal tracking in the former and optimal regulation to zero in the latter), the solution to the LQ tracking problem inherits the feedback gain from the solution to the LQ regulation with an additional term depending on the reference signal. Similarly, comparing the results given in Section V with those given in [16], the feedback Nash equilibrium of the hybrid LQ zero-sum games defined by system (1) and the cost (21) or (26) have the same form of the solutions given in [16], despite the considered problems are different.
Following the ideas given in [18], the techniques given in Section III can be extended to deal with multimodal time-varying hybrid systems by letting the matrices A, B, E, F, R, and L in (4) be time-varying, and admitting that the dimension of the matrix P and the vector h has varying dimension, depending on the discrete time k.
As in classical LQ tracking, one of the drawbacks of the proposed approach is that all the reference signals have to be known in advance in order to determine the optimal control law. However, if such signals are actually known, then the proposed optimal control can be determined by means of simple linear algebra tools.
Future work will deal with the extension of the proposed results to singular, cheap, and constrained tracking problems and to the cases of hybrid systems with state-driven jumps and of nonperiodic references over infinite horizon.