On Hamilton-Jacobi Approaches to State Reconstruction for Dynamic Systems

We investigate the use of Hamilton-Jacobi approaches for the purpose of state reconstruction of dynamic systems. First, the classical formulation based on the minimization of an estimation functional is analyzed. Second, the structure of the resulting estimator is taken into account to study the global stability properties of the estimation error by relying on the notion of inputto-state stability. A condition based on the satisfaction of a Hamilton-Jacobi inequality is proposed to construct estimators with input-to-state stable dynamics of the estimation error, where the disturbances affecting such dynamics are regarded as input. Third, the so-developed general framework is applied to the special case of high-gain observers for a class of nonlinear systems.


Introduction
This paper investigates the problem of reconstructing the state of nonlinear dynamic systems based on limited output information by using a Hamilton-Jacobi theoretical framework. Toward this end, the state estimation problem is addressed by minimizing a quadratic cost functional in such a way as to find the structure of the estimator, which is a dynamic system forced by the output and able to generate an estimate of the system variables at each time instant. The stability properties of the estimation error for such an estimator are analyzed in the presence of disturbances acting on the dynamic and measurement equations by using the notion of input-to-state stability [1]. Based on this general setting, the analysis is conducted on a particular type of estimator, i.e., the high-gain observer, by showing that its estimation error is input-to-state stable under suitable conditions on the tuning law of the gain.
The first results concerning estimation for dynamic systems based on the solution of HJ equations are reported in [2]. In line with this approach, in [3], an estimator is proposed by devising a time-varying quadratic form in the estimation error as a value function of the HJ equation solving the problem. Later, in [4,5], the stability properties of the viscosity solutions of such equations in the presence of L 2 dis-turbances are analyzed. As a follow-up of this literature, in this paper, we investigate the input-to-state stability of the estimation error of such estimators and propose novel design conditions to ensure the desired performances. Specifically, we will focus on estimation for a class of nonlinear systems by using adaptive high-gain observers. Based on the pioneering work of Gauthier and Kupka [6], high-gain observers have achieved a lot of success (see, e.g., [7][8][9][10][11][12]).
There exists a vast literature on the use of HJ equations to solve optimal control problems (see [13] and the references therein). Verification theorems ensure that, if a unique viscosity solution exists to the HJ equation, this solution coincides with the value function, which enables finding the optimal control law [14]. Unfortunately, the application of this paradigm to optimal state reconstruction is not straightforward. The estimator equation can be derived only after assuming additional smoothness of the value function [2,3]. In [5], L 2 upper bounds on the estimation are established but a complete stability analysis is currently missing. This motivates the present work with the goal of ensuring stability, while relaxing the need of finding a solution to the HJ equation. Thus, in lieu of HJ equations, we will rely on HJ inequalities (see, e.g., [15]). Moreover, as a novelty w.r.t. the existing literature, we address the problem by using input-to-state stability in such a way as to treat noise-free estimators (usually called observers) and estimation in the presence of disturbances (performed by means of estimators denoted as filters) at the same time.
Generally speaking, the global stability of the estimation error is a strict requirement for state observers of noise-free dynamic systems, which is usually ensured by finding suitable Lyapunov functions. Instead, in this paper, the stability analysis of the error dynamics is conducted by using inputto-state stability [1], where the disturbances are regarded as an input and the role of the state is played by the estimation error incurred by the observer. An estimator (observer or filter) will be said to be input-to-state stable (ISS) if the norm of the estimation is bounded by the sum of two terms, with the first one depending on the magnitude of the initial error and decreasing with time, while the second one increases with the norm of the disturbances [16][17][18][19][20]. Input-to-state stability holds if and only if there exist some suitable Lyapunov functions, which are called ISS Lyapunov functions [21]. To overcome the issue that small noises may destabilize the estimation error [22] (see also [23]), we rely on input-to-state stability for the purpose of both analysis and design since it enables to quantify the effect of disturbances on the estimation error by using the notion of L 2 -gain. Finally, we address the problem for systems with Lipschitz nonlinearities by means of high-gain observers. Specifically, ISS design conditions to ensure a desired L 2 -gain are presented by relying on linear matrix inequalities (LMIs, see [24]), which are really well-suited to accounting for stability in a number of applications [25][26][27].
The paper is organized as follows. Section 2 presents the approach to estimation for dynamic systems based on HJ equations. In Section 3, we focus on the problem to find ISS estimators using HJ inequalities. Section 4 describes the application of what proposed in Section 3 to the specific case study of high-gain observers with numerical results shown in Section 5. Section 6 reports the conclusion and discusses the prospect of future works.
We conclude this section by presenting notation and definitions used throughout the paper. ℝ ≥0 denotes the set of the nonnegative real numbers. For a square matrix P ∈ ℝ n×n , P > 0ðP ≥ 0Þ means that this matrix is positive (semidefinite) definite and P < 0ðP ≤ 0Þ negative (semidefinite) definite; moreover, λ min ðPÞ and λ max ðPÞ are the minimum and maximum eigenvalues of the symmetric positive or negative definite matrix P, respectively. The norm of a real For the sake of brevity, we will adopt the notation jxj P ≔ ffiffiffiffiffiffiffiffiffiffi x ⊤ Px p , where P > 0. A continuous function α : ℝ ≥0 ⟶ ℝ ≥0 is said to belong to class K if it is strictly increasing and αð0Þ = 0. It is said to belong to class K ∞ if it is of class K and also lim r→+∞ αðrÞ = +∞.

Estimators Based on Hamilton-Jacobi Equations
Let us consider continuous-time nonlinear dynamic systems described by where t ≥ 0, the state is x ∈ X ⊆ ℝ n open, and the output is y ∈ ℝ m ; f : X ⟶ ℝ n and h : X ⟶ ℝ m are smooth functions. Let x : ½0,+∞Þ ⟶ ℝ n denote the solution of (1a) with the initial condition xð0Þ ∈ ℝ n . We search for state observers that usually take on the following structure: wherexðtÞ ∈ X, t ↦ kðtÞ ∈ K is called the "innovation function," and K is the set of admissible innovation functions, such as the set of ℒ 2 functions. Among the various possible choices, in line with [2] first of all, we will investigate optimal state observers, i.e., those resulting from the solution of the following problem: where x 0 ∈ ℝ n is a state prediction; P 0 , Q ∈ ℝ n×n , and R ∈ ℝ m×m are symmetric positive definite. In practice, the considered observers are designed by minimizing a least-squares performance, which is usually adopted as a construction criterion in many estimation problems. The solution of (3) can be addressed by using standard arguments from HJ theory. More specifically, let us denote the value function of (3) by vðt,xðtÞÞ, wherexðtÞ ∈ ℝ n is the state estimate at the final time instant t. If the value function satisfies in the viscosity sense the HJ equation given by with the initial condition vð0,xð0ÞÞ = jxð0Þ − x 0 j 2 P 0 , we obtain Advances in Mathematical Physics for all k ∈ K and t > 0, and, after integrating and hence (6) yields Since vð0,xð0ÞÞ = jxð0Þ − x 0 j 2 P 0 , the above inequality corresponds to the definition of the minimum over all the admissible innovation functions, and the satisfaction of (4) provides the minimum.
The maximum of the expression inside the curly brackets in (4) is accomplished on a function with a linear term and a positive-definite quadratic form in k. Thus, the sufficient and necessary condition for the existence of this maximum is ∂v ∂x with a negative definite Hessian equal to −2Q, and hence, the optimal innovation function is given by After replacing (9) in (3), we obtain The previous equation can be written as follows: where is the Hamiltonian function. Based on the aforementioned, an explicit expression of the observer structure can be derived according to [2] (see also [3]). Under additional assumptions on the smoothness of the value function, we selectxðtÞ as the minimum of the value function for t ≥ 0 and thus holds as a necessary condition. If we apply the time absolute derivative to (14), after a change of the derivation order, we obtain and, using (11), where ∇ 2 x vðt,xÞ is the Hessian of vðt,xÞ and Thus, using (14) with ∇ 2 x vðt,xÞ assumed to be nonsingular for all t ≥ 0 andx ∈ ℝ n , (16) and (17) yield the dynamics of the current optimal estimate as follows: Unfortunately, the proof of stability of the estimation error is not easy for such estimators owing the need to have at disposal an analytic expression of vðt,xÞ. In practice, under general assumptions, one can find only approximate solutions to the HJ equation (11) and hence derive only local stability results [28], whereas, in general, global properties are preferable [29]. This motivates the investigation of the global stability properties that can be ensured by estimators having a structure like that in (18), as shown in the next section.

Estimators Based on Hamilton-Jacobi Inequalities
Based on the results of the previous section, from now on we focus on observers for (1a) and (1b) described by wherexðtÞ ∈ ℝ n is the estimate of xðtÞ at time t and ℓ : ℝ n × ½0,+∞Þ ⟶ ℝ n×m is a smooth function. This function accounts for the second term in the r.h.s. of (18). The goal consists in selecting ℓ in such a way as to ensure that the 3 Advances in Mathematical Physics dynamics of the estimation error eðtÞ ≔ xðtÞ −xðtÞ given by admits the origin as a globally asymptotically stable equilibrium point irrespective of x. This problem is thus reduced to find a function ℓ and a smooth Lyapunov function e ↦ VðeÞ such that for all t ≥ 0, e ∈ ℝ n , and x ∈ ℝ n , where α 1 and α 2 of class K ∞ and α 3 is continuous positive definite.
In the presence of disturbances on the error dynamics, a graceful degradation of the performances with an increase of the uncertainty is expected in such a way that the lower the bound on the disturbances, the lower the bound on the estimation error. Anyway, the estimation error is required to be asymptotically stable in the absence of noise. Such a behavior can be expressed in terms of ISS properties. Toward this end, one may consider (1) subject to additive system and measurement disturbances, i.e., where w f ðtÞ ∈ ℝ n and w h ðtÞ ∈ ℝ m are the system and measurement noises, respectively. Indeed, more in general in lieu of (23), let us deal with such disturbances acting on the plant in an affine way, i.e., by using w = ðw f , w h Þ ∈ ℝ p and the smooth functions g 1 : ℝ n ⟶ ℝ n×p and g 2 : ℝ n ⟶ ℝ m×p with the system equation where wðtÞ ∈ ℝ p . Thus, from now on, we refer to the system description given by (24). Using (19) and (24), it follows that the dynamics of the estimation error is given by The observer (19) is said to be ISS if there exists a function β of class Kℒ and a function χ of class K ∞ such that where "ess sup" denotes the essential supremum (i.e., supremum except on sets of measure zero). The ISS property (26) can be equivalently stated by using an ISS Lyapunov function, as proved in [21]: (26) holds if and only if there exist functions α 1 and α 2 of class K ∞ and α 3 and η of class K such that for all t ≥ 0, e ∈ ℝ n , w ∈ ℝ p , and x ∈ ℝ n . An ISS formulation of the observer problem is preferable in general because of the issues occurring in the stability analysis with Lyapunov arguments for nonlinear systems. ISS and standard Lyapunov functions are different in general. Though the former is necessarily also the latter as ISS implies global asymptotic stability, the existence of a Lyapunov function with null disturbances does not allow for inference about ISS in general. Arbitrarily, small noises can cause instability and there may exist an observer with a globally asymptotically stable error in the absence of disturbances, while the error dynamics is not ISS, as detailed in the following example.
Example 1. Let us focus on the second-order system where wðtÞ ∈ ℝ is an unknown disturbance, and we choose x 1 ð0Þ > 0 and x 2 ð0Þ = x 1 ð0Þ 2 . For this system, let us consider the observer First, we notice and, after integrating and using the condition x 2 ð0Þ = x 1 ð0Þ 2 , The dynamics of e 2 = x 2 −x 2 is Advances in Mathematical Physics and hence is globally exponentially stable to zero irrespective of the disturbance. If w ≡ 0, we obtain y 1 ≡ 0 and thereby _ e 1 = _ x 1 − _ x 1 = −e 1 . Thus, the observer provides an exponentially stable error in the noise-free setting and a simple Lyapunov function VðeÞ = e 2 1 + e 2 2 satisfies (21) and (22). Let us focus on the noisy case: we have and hence, it is straightforward to derive and easy to verify that it is not ISS since a diverging behavior e 1 ðtÞ = ffiffiffiffiffiffiffiffiffiffiffi 2t + 2 p occurs with a converging disturbance input wðtÞ = 1/ ffiffiffiffiffiffiffiffiffiffiffi 2t + 2 p for t ≥ 0, as detailed in [30]. In principle, one can proceed with the observer design for a given system by devising a suitable ISS Lyapunov function. As a matter of fact, it would be preferable to find a particular function that ensures a desired attenuation w.r.t. the disturbances, as it will be clearer in the following. Toward this end, instead of (28), let us consider the equivalent inequality where α 3 and α 4 are of class K ∞ [21]. Equation (36) provides a different characterization of the ISS property in terms of dissipation. More specifically, from (36), we obtain the integral inequality for all t ≥ t 0 ≥ 0. Under the assumption that V is a C 1 function, (36) implies (37) (see [21] and the references therein). The function V is regarded as a storage function, whereas α 3 and α 4 are the dissipation and supply rates, respectively. If we cast the observer problem in an ℒ 2 setting (i.e., w, e ∈ ℒ 2 ), (37) allows to establish a connection between ISS and the H ∞ approach [31]. Basing on the aforementioned, we can reply on the use of a HJ inequality by proving the following.
Proof. From (25), it follows that and, by completing the squares, Thus, we obtain Advances in Mathematical Physics and hence, using (38), the previous inequality yields in such a way as to conclude and easily obtain (39) via integration.
If (39) holds, the observer is said to exhibit an ℒ 2 -gain μ w.r.t. w. Thus, after fixing μ, the problem becomes that of choosing the innovation function that satisfies (38). We will address this problem for a class of nonlinear systems by using an adaptive high-gain observer, as shown in the next section.

Adaptive High-Gain Observers
Let us consider the class of nonlinear noise-free continuoustime systems described by where t ≥ 0, xðtÞ ∈ ℝ n is the state vector and yðtÞ ∈ ℝ is a scalar measurement; A ∈ ℝ n×n , C ∈ ℝ 1×n , and the function f : ℝ n ⟶ ℝ n are defined as follows: , To estimate xðtÞ, we consider the full-order observer , ð47Þ with γ ≥ 1.
To establish the main results, we assume the following.
To derive more convenient stability conditions for the estimation error, we adopt a change of variables by replacingê ≔ x −x ∈ ℝ n with the new coordinates e ∈ ℝ n , wherê e = TðγÞe and with a suitable choice of the scalar parameter γ that will be detailed in the following. To establish the results presented in the sequel, together with Assumption 2, we need the following assumption, where we explicitly account for a time-varying choice of this parameter.
Thus, the solution of (46) with a measurement signal according to (44) is well-defined and unique on ℝ ≥0 . The error dynamics is derived from (44) and (46) as follows: Based on the aforementioned, we can recall the following lemma (see [32] for the proof).

Lemma 4.
Let t ↦ γðtÞ be such that γðtÞ ≥ 1 for all t ≥ 0. Then, the following facts hold: (i) e is asymptotically stable if and only ifê is asymptotically stable

Advances in Mathematical Physics
(ii) there exists k f > 0 such that for all t ≥ 0, and k f does not depend on γðtÞ and t.
Because of the particular observer structure, from (50), it follows that where : ð53Þ Theorem 5. Consider observer (46) for system (44); if we choose λ > 0, σ ≥ 0, a gain matrix K, and a symmetric positive-definite matrix P such that and t ↦ γðtÞ such that for all t ≥ 0, then the estimation error converges exponentially to zero.
Proof. Thanks to Lemma 4, we shall prove the stability of e and infer the same result forê. Toward this end, let us consider the Lyapunov function VðeÞ ≔ e ⊤ P e; its time derivative along (52) is The Schwarz inequality and (51) yield From (54) and (55), it follows Using (58) and (59) in (57), we obtain and hence, _ VðeÞ turns out to be a negative definite under condition (56), which completes the proof.
Let us now consider the dynamic system derived from (44) under the presence of additive disturbances: where t ≥ 0, G 1 ∈ ℝ n×m , G 2 ∈ ℝ 1×m , and wðtÞ ∈ ℝ m is a measurable function of time defined on ℝ ≥0 . If we consider the change of coordinate eðtÞ = TðγðtÞÞ −1 eðtÞ under the same assumptions made on γðtÞ in the noise-free setting, the input-to-state stability of eðtÞ implies that ofêðtÞ and vice versa. Since the conditions that ensure the existence of an ISS Lyapunov function are more conveniently expressed in terms of e instead ofê, we shall prove the input-to-state stability of the error dynamics of e, as follows.
Proof. It is straightforward to obtain This dynamics is ISS if it admits an ISS Lyapunov function VðeÞ that satisfies (27) and (28) [21]. Toward this end, let VðeÞ ≔ e ⊤ P e; thus, α 1 ðrÞ ≔ λ min ðPÞ r and α 2 ðrÞ ≔ λ max ðPÞ r. Using the same arguments in the proof of Theorem 5, from (63), we obtain 7 Advances in Mathematical Physics where γ inf ≔ inf t≥0 γðtÞ. Since as jTðγÞ −1 j = 1/γ < 1, it follows Let ε ∈ ð0, 1Þ, from the previous inequality, we have and thus with the obvious definitions of Concerning the design of a high-gain observer with a given ℒ 2 -gain, it is preferable to impose the attenuation with respect to the disturbances in terms of the original estimation errorê rather than of e, as detailed in the following theorem. Theorem 7. Consider observer (46) for system (61); if t ↦ γðtÞ is such that (55) sup hold and there exist λ > 0, σ ≥ 0, μ > 0, a vector Y ∈ ℝ n , and a symmetric positive-definite matrix P such that then such an observer with gain K = P −1 Y exhibits an ℒ 2 -gain μ w.r.t. w.
After replacing TðγðtÞÞ −1ê ðtÞ with eðtÞ, it follows that V 1 ðêÞ = γ 2n sup V 2 ðeÞ with V 2 ðeÞ ≔ e ⊤ Pe. Likewise, in the proof of Theorem 5, the time derivative of V 2 turns out to be upper bounded as follows: Using the well-known Young inequality, for any μ > 0 we have where Y = PK. Since the product of any square matrix for its transpose is positive semidefinite, we obtain and hence Since jTðγÞ −1 j = 1/γ < 1 < γ because of (70), from the previous inequality, it follows that 8 Advances in Mathematical Physics and similarly hold. Using (78), (79), and (80) in the r.h.s. of the inequality (75), we obtain and hence Thanks to (72), (82) yields sup e j j 2 + 2k f λ max P ð Þγ 2n sup e j j 2 + μ 2 2 w j j 2 , ð83Þ 9 Advances in Mathematical Physics and, using γðtÞ ≥ γ inf under condition (70) and jêj ≤ jTðγÞjjej ≤ γ n jej ≤ γ n sup jej because of (71), to conclude the proof.
As expected, the condition (72) for an observer to admit a given ℒ 2 -gain is more difficult to satisfy than that required for the sole ISS property (i.e., (54)). However, such a stricter condition can be treated via LMIs. Given a choice of σ ≥ 0, it is straightforward to verify that, using the Schur lemma, (72) is implied by the following condition: where Since, in addition, we need to comply with (70), instead of (85), a convenient procedure consists in solving the LMI in the unknown P > 0, Y, λ > 0, α 1 > 0, and α 2 > 0 and choosing μ > 0 and γ sup > max 1, 10 Advances in Mathematical Physics such that μ 2 = 2γ 2n sup max ðδ 1 α 1 , δ 2 α 2 Þ. Thus, a strict lower bound on μ is Theorems 6 and 7 allow to design time-varying highgain observers depending on a value of γðtÞ that can be tuned on line. To set such adaptive observers, let us suppose to have successfully solved the LMI design conditions for some σ > 0 and chosen γ inf and γ sup such that γ sup > γ inf and either (62) or (70) hold. Note that it can grow at an arbitrary high rate but has to decrease no faster than prescribed by (55). Clearly, an increase of γðtÞ is convenient in case of a large value of the resid-ual rðtÞ ≔ jyðtÞ − CxðtÞj, whereas it is preferable to reduce it if rðtÞ is small. Among the possible alternatives, let us focus on the following adaptive law: where, for example, γð0Þ = γ inf ; α and d are positive scalars to be suitably chosen. The selection of α depends on how much we require to speed up the growth of γðtÞ in the presence of a large residual; d can be updated according to the level of noise that affects the measurements. of observers and filters based on the solution of the HJ equations is not easy to apply owing to the additional assumptions required to get the structure of the estimator and the need to find the analytical solution of such equations. These difficulties have motivated the adoption of a different paradigm based on the satisfaction of HJ inequalities. The interest for this method is the possibility to ensure the input-to-state stability of the estimation error, i.e., to construct ISS estimators. We have successfully faced such a problem by dealing with estimation for a class of nonlinear systems by using adaptive high-gain observers. The current work is with regard to the extension of the proposed approach to a wider class of nonlinear systems and the investigation of input-to-state stability of the estimation error for observers having a saturated innovation function [33].

Data Availability
The simulation code is available upon an email request to alessandri@dime.unige.it.

Conflicts of Interest
The author declares that he/she has no conflicts of interest.