The stochastic Hamilton-Jacobi equation

We extend some aspects of the Hamilton-Jacobi theory to the category of stochastic Hamiltonian dynamical systems. More specifically, we show that the stochastic action satisfies the Hamilton-Jacobi equation when, as in the classical situation, it is written as a function of the configuration space using a regular Lagrangian submanifold. Additionally, we will use a variation of the Hamilton-Jacobi equation to characterize the generating functions of one-parameter groups of symplectomorphisms that allow to rewrite a given stochastic Hamiltonian system in a form whose solutions are very easy to find; this result recovers in the stochastic context the classical solution method by reduction to the equilibrium of a Hamiltonian system.


Introduction
Hamiltonian diffusions were introduced and studied by Bismut in the monograph [B81]. These systems were generalized in [LO07] to accommodate arbitrary Poisson manifolds as phase spaces and general continuous semimartingales as forcing noises. In that paper it was also shown that, when the phase space is an exact symplectic manifold, the stochastic Hamilton equations are fully characterized by a variational principle that generalizes the classical Hamilton's Principle. This circle of problems has also been treated in [BRO08] in the development of stochastic variational numerical integrators.
Hamilton-Jacobi theory is an important part of classical mechanics that provides a characterization of the generating functions of certain time-dependent canonical transformations that put a given Hamiltonian system in such a form that his solutions are extremely easy to find; this is the so called solution by reduction to the equilibrium. In this respect, the fact that the classical action satisfies the Hamilton-Jacobi equation is a very relevant result. Hamilton-Jacobi theory also plays a fundamental role in the study of the quantum-classical relationship, in integrable systems, or in the development of structure preserving numerical integrators. For all these reasons it is desirable to have at hand similar tools in the stochastic Hamiltonian context; this is the main goal of this work. The Hamilton-Jacobi equation was already studied by Bismut [B81] in the context of Hamiltonian diffusions and, as we will see, most of the ideas in that piece of work are still valid at our degree of generality; at some level, this paper can be seen as a completion of Bismut's work in which complete proofs are provided and where the results have been adapted to our framework using a more modern geometric language; this makes them more palatable to a growing community interested both in geometric mechanics and in stochastics.
The paper starts with a brief presentation in Section 2 of some basic facts about stochastic Hamiltonian systems and, more importantly, with the introduction of the stochastic Hamiltonian action. Sec-tion 3 is dedicated to showing that the stochastic action satisfies a generalized version of the Hamilton-Jacobi equation when written as a function of the configuration space using a Lagrangian submanifold (see Theorem 3.5). As an application of the results in this section we show in Example 3.7 how the exponential of the expectation of the so called projected stochastic action can be used to construct solutions of the heat equation corrected with a potential, in a way that strongly resembles the Feynman-Kac formula.
The paper concludes with a section on the relation between the solutions of the Hamilton-Jacobi equation and the generating functions of time dependent diffeomorphisms that allow the integration of the Hamiltonian stochastic differential equation in question in an easy manner. The natural framework for carrying this out is that of time-dependent Hamiltonian systems; that is why we have included a subsection that briefly recalls the classical theory of non-autonomous Hamiltonian systems and presents it in a form that is suitable for generalization in the stochastic context. Some of the statements in this section are either inspired or are a direct generalization of analogous results in [B81]; we have nevertheless included them in order to have a complete and self-contained presentation of the theory.
In this section we recall the basic facts about stochastic Hamiltonian dynamical systems and we fix the notation that we are going to use throughout the paper. The content of this section is extracted from [LO07], which the reader is encouraged to check with for a more comprehensive approach to stochastic Hamiltonian systems and their most relevant properties. Although such systems may be considered on any Poisson manifold (M, {·, ·}), we are only going to deal with the exact symplectic case, that is, stochastic Hamiltonian systems defined on exact symplectic manifolds.
The reason for such a restriction is that in that case there exists a stochastic action defined on the set of manifold valued semimartingales such that the stochastic Hamilton equations can be characterized using a critical action principle. In other words, taking suitable variations on the space of manifold valued semimartingales, a given semimartingale is critical for the stochastic action if and only if it is a solution of the stochastic Hamiltonian equations. This generalizes to the stochastic context the way in which the Hamilton equations are characterized in the classical deterministic contex. Like in that framework, the stochastic action plays a prominent role in the description of a Hamiltonian system and, as we will see later on, it also satisfies a stochastic analog of the Hamilton-Jacobi equation.
Let (M, ω) be a symplectic manifold. Using the nondegeneracy of the symplectic form ω, one can associate to each function h ∈ C ∞ (M ) a vector field X h ∈ X (M ) characterized by the equality We will say that X h is the Hamiltonian vector field associated to the Hamiltonian function h. The expression (2.1) is referred to as the Hamilton equations. Given h ∈ C ∞ (M ), solving the associated Hamilton equations amounts to finding the integral curves of X h . We now introduce the stochastic generalization of (2.1). Let V be a real finite dimensional vector space and let f : M → V be a differentiable function taking values in V . We define the differential df : T M → V as the map given by df = p 2 • T f , where T f : T M → T V = V × V is the tangent map of f and p 2 : V ×V → V is the projection onto the second factor. If V = R this definition coincides with the usual differential. If {e 1 , ..., e r } is a basis of V and is a linear mapping that depends smoothly on its two entries. The adjoint of (2.2) In this expression, {e 1 , ..., e r } is the dual basis of {e 1 , ..., e r }. It is easy to check that the adjoint The key to generalizing the Hamilton equations (2.1) to the stochastic context consists of realizing that they may be restated by saying that a smooth curve γ : [0, T ] → M is an integral curve of the Hamiltonian vector field X h , h ∈ C ∞ (M ), if and only if for any α ∈ Ω (M ) and for any t ∈ [0, T ] . Using this observation we will define the stochastic Hamilton equations by specifying the result of integrating an arbitrary one form α ∈ Ω (M ) along them. More especifically, let Ω, {F t } t∈R+ , P be a filtered probability space, X : R + × Ω → V a continuous semimartingale (that is, the paths X t (·) : Ω → V are continuous a.s. for any t ∈ R + ) that takes values on the vector space V with X 0 = 0, and h : M → V * a smooth function. We will say that a M -valued semimartingale Γ : R + × Ω → M is a solution of the stochastic Hamilton equations with stochastic component X and Hamiltonian function h if for any α ∈ Ω (M ) where the symbol δ denotes Stratonovich integration. In other words, the processes that solve the stochastic Hamilton equations are no longer driven by the deterministic time t but by the stochastic noise X. It can be shown ( [E89,Theorem 7.21]) that given a semimartingale X in V , a F 0 -measurable random variable Γ 0 , there are a maximal stopping time ζ > 0 and a continuous solution Γ of (2.4) with initial condition Γ 0 defined on the set {(t, η) ∈ R + × Ω | t ∈ [0, ζ(η))}. If ζ is finite, then Γ explodes at time ζ, that is, the path Γ t with t ∈ [0, ζ) is not contained in any compact subset of M . For the sake of simplicity, the stochastic Hamilton equations (2.4) will be symbolically denoted using the Stratonovich operator H as δΓ = H (X, Γ) δX. (2.5) We now state some of the basic properties of the flow defined by (2.4). Let ϕ (·, z) : [0, ζ(z)) ⊆ R + × Ω → M denote the unique solution of (2.5) with initial condition Γ 0 = z ∈ M a.s.. The map ϕ will be referred to as the stochastic flow associated to (2.5). For any (t, η) ∈ R + × Ω, let is an open set for any t ∈ R + a.s. and is a continuously differentiable diffeomorphism ( [K90,Theorem 4.8.4]). Additionally, is continuous and its partial derivatives with respect to z ∈ D t (η) are also continuous on [0, t] × D t (η). The local version of these results, that is, the case M = R 2n , can be also found in [P04,Chapter V Theorem 39]. Furthermore, the stochastic flow ϕ acts naturally on tensor fields and in particular on differential forms. Hence, by [K81,Theorem 3.3] and [K90,Section 4.9 In particular, if α = ω is the symplectic form, then £ X h i ω = 0 for any i = 1, ..., r and ϕ * ω = ω which is the stochastic version of the Liouville's Theorem (see [LO07,Theorem 2.1]). We conclude this brief summary on stochastic Hamiltonian systems by introducing the stochastic action and presenting how it characterizes the solutions of the Hamilton equations. As we already said, the stochastic action is only naturally defined for stochastic Hamiltonian systems occurring on exact symplectic manifolds: let (M, ω = −dθ) be an exact symplectic manifold, X : R + × Ω → V a semimartingale taking values on the vector space V , and h : M → V * a Hamiltonian function. We denote by S (M ) and S (R) the sets of M and real valued semimartingales, respectively. We define the stochastic action associated to h as the map S : S (M ) → S (R) given by where in the previous expression, the stochastic differential formĥ (Γ) : R + × Ω → V × V * over X is given byĥ (Γ) (t, ω) := (X t (ω) , h (Γ t (ω))). Given a M -valued semimartingale Γ and s 0 > 0, we say that the map Σ : (−s 0 , s 0 ) × R + × Ω → M is a pathwise variation of Γ whenever Σ s0=0 t = Γ t a.s.. We say that the pathwise variation Σ of Γ converges uniformly to Γ whenever the following properties are satisfied ( [LO07,Definition 4.4]): (i) For any f ∈ C ∞ (M ), f (Σ s ) → f (Γ) uniformly in compacts in probability (abbreviated in ucp) as s → 0. That is, for any ε > 0 and any t ∈ R + , (ii) There exists a process Y : R + × Ω → T M over Γ such that, for any f ∈ C ∞ (M ), the Stratonovich integral Y [f ]δX exists for any continuous real semimartingale X (this is for instance guaranteed if Y is a semimartingale) and, additionally, the increments (f (Σ s ) − f (Γ))/ s converge in ucp to Y [f ] as s → 0. We will call such a Y the infinitesimal generator of Σ.
We will say that Σ (respectively Y ) is bounded when its image lies in a compact set of M (respectively T M ). It can be shown that, given a M -valued semimartingale Γ, a compact set K ⊆ M , and a bounded process Y : R + × Ω → T M over Γ τK (the process Γ stopped at the first exit time τ K of Γ from K) such that Y [f ]δX exists for any continuous real semimartingale X and any f ∈ C ∞ (M ), there exists a bounded pathwise variation Σ that converges uniformly to Γ τK whose infinitesimal generator is Y ([LO07, Proposition 4.2]). Using these elements, a variational characterization of the stochastic Hamilton equations can be given ( [LO07,Theorem 4.2]): the semimartingale Γ satisfies the stochastic Hamilton equations (2.4) with initial condition Γ t=0 = m 0 ∈ M a.s. up to time τ K if and only if, for any bounded pathwise variation Σ : (s 0 , s 0 ) × R + × Ω → M with bounded infinitesimal generator which converges uniformly to Γ τK and such that Σ s 0 = m 0 and Σ s τK = Γ τ k a.s., s ∈ (−s 0 , s 0 ),

The stochastic action on Lagrangian submanifolds and the Hamilton-Jacobi equation
It is a classical result in mechanics that the action, when written as a function of the configuration space and time, satisfies the Hamilton-Jacobi equation (see for instance [A89]). The main goal of this section is showing that an analogous result holds for the stochastic action.
where θ is the one form of the exact symplectic manifold (M, ω = −dθ).
Proof. We will proceed by showing that for any pair of points x, y ∈ D t (η) we can write where γ : (a, b) ⊆ R → D t (η) is any smooth curve in D t (η) that links x and y. This expression immediately implies that R t has continuous directional derivatives and it is hence Fréchet differentiable. Indeed, using first (2.6), we have Given that γ : (a, b) → D t (η) and the points x, y ∈ D t (η) are arbitrary, the result follows.
Later on in this section we will need the composition of R with the inverse of the stochastic flow ϕ. More specifically, let (t, η) ∈ R + × Ω and let ϕ −1 In order to get closer to the classical deterministic result on the Hamilton-Jacobi equation we are first going to visualize it, using the map R, as a process depending on M through the initial condition of the flow ϕ generated by (2.5). Second, we will restrict R to a Lagrangian submanifold of M ; this encodes mathematically the writing of the action as a function of the configuration space. Recall that a submanifold ι : From now on we are going to assume that the underlying symplectic manifold (M, ω) is actually a cotangent bundle endowed with its canonical symplectic structure. More specifically, M = T * Q for some manifold Q. In this case, a point y ∈ L ⊂ T * Q in a Lagrangian submanifold L is said to be a regular point of L, if the restriction π| L : L → Q of the canonical projection π : T * Q → Q to L is a local diffeomorphism at y (that is, T y π| L : T y L → T π(y) Q is an isomorphism). In a neighborhood U ⊂ L of a regular point y ∈ L we can obviously describe the Lagrangian submanifold L using local coordinates on the base manifold Q, which we will generally denote by (q 1 , . . . , q n ). On the other hand, since ι * ω = d(ι * θ) = 0, there exists by the Poincaré lemma (shrinking U if necessary) a smooth function f ∈ C ∞ (U ) such that ι * θ = df . Conversely, if q 1 , . . . , q n , p 1 , . . . , p n are local Darboux coordinates in a neighborhood V ⊆ T * Q and f ∈ C ∞ (π(V )) is a function with no critical points, then the set is a local Lagrangian submanifold such that with ι f : L f ֒→ V the inclusion and π| L f : L f ⊂ T * Q → π(V ) the local diffeomorphism obtained by restriction of the canonical projection.
Theorem 3.2 Let Q be a manifold and let L ⊂ T * Q a Lagrangian submanifold. Let y 0 ∈ L be a regular point and let x 0 = π(y 0 ), where π : T * Q → Q is the canonical projection. Then, there exist two neighborhoods V y0 ⊆ L and V x0 ⊆ Q of y 0 and of x 0 , respectively and a map ξ : Ω × V x0 → R + with the property that ξ (x) : Ω → R + is a stopping time, such that the equation has a unique solution in V y0 ⊆ L for any η ∈ Ω, any x ∈ V x0 , and any s ∈ [0, ξ (η, x)]. We are going to denote this solution by ψ s (η, x).
) which depends continuously on s.
Proof. Let U y0 ⊂ L be an open neighborhood of y 0 ∈ L. We pick U y0 small enough so that π| Uy 0 is a diffeomorphism onto its image and a set of local coordinates q i ; i = 1, ..., n can be chosen on U x0 := π (U y0 ). Let y i = q i • π| L ; i = 1, ..., n be the corresponding induced coordinates on U y0 . Denote by q : U x0 → R n andŷ : U y0 → R n the local chart maps associated to these coordinates. For any y ∈ U y0 , let ∂y i (y) > 0 a.s., for any y ∈ V y0 . For any of these y ∈ V y0 , let which is a well defined and continuous semimartingale, by the continuity of the differential of the flow ϕ. Observe that Z 0 (y) > 0 for any y ∈ V y0 . Let T (y, η) := inf{τ Ux 0 (y) ≥ t > 0 | Z t (y, η) / ∈ R + }. Now, recall that we want to see that the equation π (ϕ s (η, y)) = x has a unique solution in y ∈ L, for any x ∈ V x0 in a suitable V x0 and up to a suitable stopping time ξ (x). Therefore, it suffices to solve the equation where ϕ T (y) (y) denotes the process ϕ (y) stopped at time T (y), that is, ϕ T (y) (y) (s, η) = ϕ T (y,η)∧s (η, y).
Observe that ϕ T (y) (y) is always in U y0 if y was already in V y0 . Consequently, ϕ T (y) (y) may be described using the local coordinates introduced above. Moreover, if we set ξ (x) := T π| −1 L (x) , V x0 := π(V y0 ), the equation (3.7) admits by construction a unique solution ψ s (η, x) via the Implicit Function Theorem. Additionally, if we apply the Stratonovich differentiation rules to we obtain that ψ s (η, x) satisfies up to time ξ (x) the Stratonovich differential equation with initial condition ψ s=0 (x) = y(x) ∈ V y0 a.s. such that π (y(x)) = x ∈ V x0 . That is, we can visualize ψ s (η, x) as the unique stochastic flow associated to the stochastic differential equation (3.8). This guarantees that the properties claimed in the statement hold.
We proceed now by considering the stochastic action R not as a semimartingale parametrized by T * Q through the initial condition of the stochastic flow ϕ defined by (2.5), but as a process depending on the base manifold Q. More specifically, we will restrict to the open neighborhood V x0 ⊂ Q introduced in the statement of Theorem 3.2 and which is mapped onto V y0 ⊂ L using the map ψ that solves (3.6). Furthermore, since we are always going to work around regular points of the Lagrangian submanifold, we will always consider Lagrangian submanifolds of the type L f (see (3.4)) for some f ∈ C ∞ (Q).
the open neighborhood of x 0 introduced in Theorem 3.2 and ψ(x) : [0, ξ(x)) → V y0 the semimartingale solution of (3.8) with initial condition x ∈ V x0 a.s.. We define the projected stochastic action S (x) : Notice that the differentiability properties of the maps R, f ∈ C ∞ (Q), and ψ imply that the map is continuously differentiable for any (t, η) ∈ R + × Ω such that t ∈ [0, ξ (x, η)). In this expression The following theorem provides an explicit expression for the spatial derivatives of the projected stochastic action S. (3.10) If q i , p i ; i = 1, ..., n are local Darboux coordinates of T * Q on an open neighborhood of a regular point y 0 ∈ L f , the expression (3.10) can be locally written as Proof. First of all observe that S t (η) can be expressed in terms ofR t (η) as follows: Then, for any smooth curve γ : Given that D ψ t (η) ⊂ L f , the curve γ takes values in the Lagrangian submanifold L f and hence (3.11) can be rewritten as ( 3.12) On the other hand, we saw in (3.3) that which substituted in (3.12) yields Since γ is an arbitrary smooth curve, we can conclude that We conclude this section by proving that the projected stochastic action S t satisfies a specific stochastic differential equation which generalizes the classical Hamilton-Jacobi equation. For obvious reasons, this equation will be referred to as the stochastic Hamilton-Jacobi equation.
Theorem 3.5 (Stochastic Hamilton-Jacobi equation) Using the same notation as in Theorem 3.2, the projected stochastic action S (q) : [0, ξ (q)) → R associated to the Lagrangian submanifold L f defined by the function f ∈ C ∞ (Q) satisfies In order to prove this theorem we need the following auxiliary result.
Proposition 3.6 ([K90, Theorem 3.3.2]) Let F (x) : R + ×Ω → R, x ∈ R n , be a family of continuous semimartingales parametrized by R n . Suppose that the dependence of this family on the R n parameter is at least three times differentiable. In addition, suppose that there exists a process f : R + × Ω × R n → R d that satisfies sufficient regularity conditions and a semimartingale X : Let g : R + × Ω → R n be a continuous R n -valued semimartingale. Then F (g) : Proof of Theorem 3.5. First of all observe that using the definition of the function R t the semimartingale S (q) : [0, ξ (q)) → R may be expressed as .
But, using the fact that ϕ is the flow of the stochastic Hamilton equations (2.5), by Proposition 3.6, we have that for any g ∈ C ∞ (M ) where y (q) ∈ L f is the unique point such that π| L (y(q)) = q. We claim that Indeed, since we are working at a local level we can use Darboux coordinates and we can replace θ by n i=1 p i dq i ; (3.17) is a straightforward consequence of (3.16). If we now plug (3.14), (3.15), and (3.17) into (3.13) we obtain Finally, we saw in Theorem 3.4 that For any η ∈ Ω, the time parameter s in the integrand of (3.18) is always smaller than ξ (q, η) and hence as ∂ e Ss ∂q i (q) and p i (ϕ s • ψ s (q)) coincide a.s. on [0, ξ(q)) for any i = 1, ..., n, the result follows.
Example 3.7 Let Q = R n and T * Q = R n × R n with global coordinates q i , p i ; i = 1, ..., n . Let f ∈ C ∞ (R n ), h 0 ∈ C ∞ R 2n , and h i = p i for any i = 1, ..., n. Consider the semimartingale X : where [·, ·] denotes the quadratic variation. Then, the projected stochastic action S : R + × Ω × R n → R built from the stochastic Hamiltonian system on R 2n with Hamiltonian fuction h = (h 0 , h 1 , ..., h n ) and stochastic component X satisfies by Theorem 3.5 If we transform the Itô integrals in this expression into Stratonovich integrals, (3.19) reads (3.20) It is not difficult to realize, though tedious to check, that δB r s so that, since h r = p r for any r = 1, ..., r, Therefore, disregarding all the finite variation terms in this last expression, we have where the property for arbitrary real semimartingales H, K, X, and Y ([P04, Chapter II Theorem 29]) has been used. Taking expectations in both sides of (3.20) and assuming that all the processes involved are regular enough so that Fubini's Theorem may be invoked, we obtain This shows that the projected stochastic action S t can be used to construct solutions of the heat equation modified with a potential term V , with initial condition given by the function exp(f ) ∈ C ∞ (R n ).

The deterministic case
We start by recalling the relation between the Hamilton-Jacobi equation and the generating functions for integrating canonical transformations in the classical deterministic case. In the next paragraphs we will write down some classical results in a form that is well adapted for the subsequent generalization to the stochastic case. All along this section we will consider Hamiltonian systems on cotangent bundles (T * Q, ω = −dθ) endowed with their canonical symplectic forms. Consider the manifold T * Q × T * Q endowed with the symplectic form Ω := τ * 1 ω − τ * 2 ω, where τ i : T * Q × T * Q → T * Q, i = 1, 2, denote the canonical projections onto the first and the second factors, respectively. Let now ψ : T * Q → T * Q be a smooth function. It is easy to verify that the map ψ is a symplectomorphism if and only if ι * ψ Ω = 0, where ι ψ : L ψ ֒→ T * Q × T * Q is the inclusion of the graph L ψ of ψ ([AM78, Proposition 5.2.1]), in which case is a Lagrangian submanifold of T * Q × T * Q. Given that Ω = −dΘ, with Θ = τ * 1 θ − τ * 2 θ, we have that 0 = ι * ψ Ω = −ι * ψ (dΘ) = −d(ι * ψ Θ) and hence by Poincaré's Lemma, we can locally write ι * ψ Θ = dS, for some function S ∈ C ∞ L ψ . We will say that S is a local generating function for the symplectic map ψ. In addition, suppose that with π : T * Q → Q the canonical projection, is a local diffeomorphism when restricted to L ψ and denote its (local) inverse by τ −1 : Q × Q → L ψ . We will suppose throughout this section that this is the case and we will think of the generating function S ∈ C ∞ L ψ as a function defined on Q × Q; that is, we will not distinguish between S and τ −1 * S. With this convention, we can write Let now {ψ t } t∈R be a family of symplectomorphisms depending smoothly on t ∈ R (for example {ψ t } t∈R could be the flow of a Hamiltonian vector field) and let S : R × Q × Q → R be the corresponding generating functions associated to this family. We will say that ψ t transforms a vector field X ∈ X(T * Q) to equilibrium if T ψ t (X) = 0 for any t ∈ R. For example, if X = X h is the Hamiltonian vector field associated to a Hamiltonian function h ∈ C ∞ (T * Q) and ψ t transforms X h to equilibrium, then the integral curve γ of X h with initial condition z is whereψ −1 is the inverse of the diffeomorphismψ : T * Q × R → T * Q × R given by (z, t) → (ψ t (z) , t). The main goal of the classical Hamilton-Jacobi theory in this context is proving that ψ transforms X h to equilibrium if, roughly speaking, its generating function S satisfies the (deterministic) Hamilton-Jacobi equation. As we deal with time-dependent transformations ψ t of the phase space, the time-dependent Hamiltonian formalism is more convenient.
Time-dependent Hamiltonian systems. Recall that, for time-dependent Hamiltonian systems, the phase space T * Q is replaced with the extended phase space R × T * Q. Given a time-dependent Hamiltonian function h ∈ C ∞ (R × T * Q), one introduces Ω h ∈ Ω 2 (R × T * Q) as Ω h = dh ∧ dt + ω, where ω ∈ Ω 2 (T * Q) is the canonical symplectic form and t denotes the global time coordinate in R. Observe that Ω h is exact, Ω h = −dθ h , where θ h = θ − hdt and θ is the canonical Liouville one form on the cotangent bundle. Then, the Hamiltonian vector field X h ∈ X (R × T * Q) is characterized by the two equations where π R : R × T * Q → R is the projection onto the first factor.
Sometimes it is more convenient to encode time-dependent Hamiltonian systems as autonomous Hamiltonian systems on the symplectic manifold E := T * (R × Q) = T * R × T * Q: let (t, u) be global coordinates for T * R, that is u is the conjugate momentum associated to the time t, and denote by π R×T * Q : T * R × T * Q → R × T * Q the projection ((t, u) , z) → (t, z), with z ∈ T * Q. It is straightforward to check that the Hamiltonian vector field X h ⋆ associated to the function h ⋆ := u+π * R×T * Q (h) ∈ C ∞ (E) is such that T π R×T * Q (X h ⋆ ) = X h . In other words, any time-dependent Hamiltonian system may be visualized as an autonomous Hamiltonian system by replacing R × T * Q by E and h by h ⋆ ; the integral curves of the original system X h are simply obtained form the integral curves of the autonomous system X h ⋆ by dropping the additional degree of freedom u, which is irrelevant as far as the dynamical description of the system is concerned. The following proposition deals with a time-dependent family of symplectomorphisms {ψ t } t∈R of T * Q in the enlarged phase space E and will be useful in order to transform time-dependent Hamiltonian systems.
Proposition 4.1 Let {ψ t } t∈R be a family of symplectomorphisms of T * Q and S ∈ C ∞ (R × Q × Q) its generating function. Defineψ Then, where ω E denotes the canonical symplectic two form of E = T * (R × Q).
Proposition 4.2 Let h ∈ C ∞ (R × T * Q). With the same notation as in Proposition 4.1, a curve γ : [0, T ] → R × T * Q is a solution of the Hamiltonian system defined by h if and only if, for any family of symplectomorphisms {ψ t } t∈R of T * Q, the curve ψ • γ : ) is a solution of a Hamiltonian system with Hamiltonian function where S ∈ C ∞ (R × Q × Q) is the generating function of {ψ t } t∈R .
Proof. Let γ : [0, T ] → R × T * Q be a solution of the time-dependent Hamiltonian system defined by h. Letγ : [0, T ] → E = T * (R × Q) be the curve such that γ = π * R×T * Q (γ) andu = ∂h ∂t (γ), u being the conjugate momenta of the time coordinate t. Then γ is a solution of the time-dependent Hamiltonian system defined by h ∈ C ∞ (R × T * Q) if and only ifγ is a solution of the autonomous Hamilton system on the phase space E with Hamiltonian function h ⋆ = u + π * R×T * Q (h). By (2.3), this means that for any β ∈ Ω (E), for any t ∈ [0, T ]. However, since we are not interested in the evolution of u, the conjugate momentum of the time, verifying that γ is a solution of the time-dependent Hamilton equations is equivalent to taking any curveγ such that γ = π * R×T * Q (γ) and checking that (4.8) holds for any differential form of the type π * R×T * Q (α), α ∈ Ω (R × T * Q). Let now {ψ t } t∈R be a time-dependent family of symplectomorphisms of T * Q and considerψ : R × T * Q → R × T * Q such thatψ (t, z) = (t, ψ t (z)), (t, z) ∈ R × T * Q, andψ : E → E such that ψ (t, u, z) = (t, u, ψ t (z)) as in Proposition 4.1. Letψ •γ : [0, T ] → E be defined as (ψ •γ)(s) :=ψ s (γ(s)). Then where Proposition 4.1 (ii) have been used in the last equality. Hence, we conclude that π R×T * Q (ψ •γ) = ψ • γ is a solution of the time-dependent Hamiltonian system given by (ψ −1 ) * (h + ∂S ∂t • J). The converse is left to the reader.
The content of Proposition 4.2 can be restated as follows. Given h ∈ C ∞ (R × T * Q) and a family of where Ω h ′ = dh ′ ∧ dt + ω and h ′ is given by (4.7) (see [MR99,Section 7.9]). Furthermore Tψ −1 (X h ) is the Hamiltonian vector field related to h ′ and the flow of X h ′ restricted to the phase space T * Q iŝ ϕ t = ψ −1 t • ϕ t • ψ 0 where, as usual, ϕ denotes the flow of symplectomorphisms of the Hamiltonian vector field X h ∈ X (T * Q). However, as we will be interested in transforming X h using T ψ rather than T ψ −1 we will rewrite (4.7) in the form h ′ (t, ψ t (z)) := h (z) + ∂S ∂t (t, J t • z) .
(4.9) Definition 4.3 Let h ∈ C ∞ (T * Q) be a Hamiltonian function and let q i , p i ; i = 1, ..., n be local Darboux coordinates on T * Q. Regarding h as a function of these coordinates, we will say that the generating function S : R × Q × Q → R satisfies the (deterministic) Hamilton-Jacobi equation if the function does not depend on the first entry q 1 ∈ Q.
Observe that in the right hand side of (4.10) we have carried out the substitution (p 1 ) i = ∂S ∂q i 1 (t, q 1 , q 2 ), i = 1, ..., n. We could also write (4.10) more intrinsically as where, for a fixed value (t, q 2 ) ∈ R × Q, we consider d Q1 S (t, q 1 , q 2 ) as an element in T * q1 Q. Notice that the map J t introduced in (4.3) is a local diffeomorphism for any t ∈ R because we required the projection τ defined in (4.1) to be a local diffeomorphism when restricted to the graph of ψ t . We may therefore (locally) write any z ∈ T * Q as z = J −1 t (q 1 , q 2 ) for some suitable (q 1 , q 2 ) ∈ Q × Q. The important point is that J −1 t (q 1 , q 2 ) = d Q1 S (t, q 1 , q 2 ) ([MR99, (7.9.1)]) and, consequently, the transformed Hamiltonian h ′ in (4.9) can be seen as a function on R × Q × Q. Explicitly, ifz = ψ t (z) ∈ T * Q, h ′ (t,z) = h (d Q1 S (t, q 1 , q 2 )) + ∂S ∂t (t, q 1 , q 2 ) , (4.11) so h ′ (t,z) equals the function K t (q 1 , q 2 ) introduced in Definition 4.3. Suppose now that S : R × Q × Q → R is a solution to the Hamilton-Jacobi equation. In other words, K t (q 1 , q 2 ) ≡ K t (q 2 ). Since q 2 = π (ψ t (z)) is the base point in the configuration space of the transformed point ψ t (z), z ∈ T * Q, we conclude that h ′ does not depend on the fiber coordinates. Hence, removing the subindices, the Hamilton equations associated to the new Hamiltonian h ′ arė q i = 0,ṗ i = − ∂K ∂q i (t, q) , i = 1, ..., n, which are easily integrable. In particular, if K is independent of both q 1 and q 2 , then ψ t transforms X h to equilibrium.

The stochastic case
We are now going to see that the classical Hamilton-Jacobi that we just outlined has a stochastic counterpart. More specifically, one may use a time-dependent family of symplectomorphisms and their generating function to transform a stochastic Hamiltonian system into another one in much the same fashion as in the deterministic case. The strategy consists of finding and characterizing a suitable generating function so that the new Hamiltonian system is easier to solve. Let T * Q be the cotangent bundle of the configuration space manifold Q and let {h 0 , h 1 , ..., h r } ⊂ C ∞ (T * Q) be a family of functions. Take a R r+1 -valued semimartingale X : R + × Ω → R r+1 such that X = X 0 , X 1 , ..., X r , with X 0 = t a.s., (4.12) and consider the stochastic Hamiltonian system on T * Q with Hamiltonian function h := (h 0 , h 1 , ..., h r ) and stochastic component X. If we want to remove the assumption that there is a Hamiltonian vector field, i.e. X h0 , playing the role of a deterministic drift, we may simply choose h 0 = 0. Using an approach similar to the one in Section 4.1, we will work in the extended phase space E := T * (R × Q). Indeed, it is easy to check that the solution semimartingales of the stochastic Hamiltonian system can be obtained out of the solutions of the stochastic Hamiltonian system on E with Hamiltonian functionh = (h ⋆ 0 , π * R×T * Q (h 1 ), ..., π * R×T * Q (h r )) and stochastic component X; notice that the functions h 0 , h 1 , ..., h r have already been considered as functions on R × T * Q instead of only T * Q. The solutions of the original system can be recovered by composing the solutions of the Hamiltonian system on E with π R×T * Q . When instead of working on the space E one uses directly R × T * Q instead of T * Q then a T * Q-valued semimartingale Γ is a solution of the corresponding stochastic Hamiltonian system when for any α ∈ Ω (T * Q), α, δΓ s = − dh τ * T * Q • ω # (α) (s, Γ s ) , δX s , where τ T * Q : R × T * Q → T * Q is the canonical projection onto the second factor.
Proposition 4.4 Let {ψ t } t∈R be a time-dependent family of symplectomorphisms of T * Q with generating function S ∈ C ∞ (R × Q × Q). Considerψ : R × T * Q → R × T * Q andψ : E → E the natural diffeomorphisms extending ψ to R×T * Q and E respectively. Then the semimartingale Γ : R + ×Ω → T * Q is a solution of the Hamiltonian system with Hamiltonian function h : T * Q → R r+1 , h = (h 0 , h 1 , ..., h r ),