Hopf-Lax formula for variational problems with non-constant discount

We provide a Hop-Lax formula for variational problems with non-constant discount and deduce a dynamic programming equation. We also study some regularity properties of the value function.


Introduction
We establish a Hopf-Lax formula for the Cauchy problem (1) − where ℓ is the convex conjugate of f , ι = (∇ℓ) −1 , and d t (s) = exp (− s t ρ(r) dr). The formula represents a Lipschitz solution that satisfies the Cauchy problem almost everywhere.
The classical Hopf-Lax formula applies to the case ρ ≡ 0 , and was given by Lax in [6], for n = 1. It was extended later to general n by Hopf in [5]. Further generalizations have maintained ρ = 0 but have considered functions f (t, −v x ) also depending on time, [9]; or functions f (v, −v x ) depending on both v and v x , [2], with some additional requirements.
The case we analyze in this paper is not covered in any of these previous works.
Actually, we find a Hopf-Lax formula that applies to more general Hamilton-Jacobi equations associated to the calculus of variations problems with variable discount.
These problems arise quite naturally in models of economics. Consider for instance the following problem: an agent optimally chooses a consumption path of a given good, with the aim of maximizing his/her satisfaction. This is measured by an utility function of consumption, ℓ(u), along a given time interval, [0, T ]. It is customary in the literature to Date: April 21, 2009. postulate concavity in the preferences of the agent, and to suppose that he/she is impatient, in the sense that the value of the utility attained today is higher than the utility attained tomorrow. This is the meaning of introducing a discount factor or impatience rate in the preferences of the agent.
Empirical studies suggest that people are more impatient about choices in the short run than in the long run, implying that the discount rate applied to current choices is higher than the one applied to far-in-the future choices. Thus, the discount factor should be taken to be non-constant. Several papers have considered the non-constant discount case: see e.g.
[1], [4] or [7] 1 . In [1], the optimal growth model with time-varying discount is considered, for a particular class of utility functions. A general problem, with infinite horizon, is analyzed in [4], whereas [7] considers the finite horizon case with fixed or variable terminal time.
The last two papers use discretization and passage to the limit to find a Hamilton-Jacobi equation that involves not only the unknown value function, but also a non-local term involving integration along the unknown optimal solution. We provide conditions so that the Hamilton-Jacobi equation involves only the derivatives of the value function and find the dynamic programming equation by direct methods.
Given the significance of the non-constant discount preference rate in economics, it is of interest to analyze in more detail this type of variational problems. First, deriving a Hopf-Lax formula for the solution of the variational problem (Section 3); second, establishing a modified dynamic programming equation, more amenable than the one found in previous papers (Section 4); and third, studying the regularity of the value function (Section 5).

Variational problem with discount
We follow the presentation in [3]. Let the value function where AC x,t = {y : [t, T ] −→ R n : y = y(s) absolutely continuous, y(t) = x}.
A typical element of this set will be called an arc. We will impose the following conditions. A1: ℓ : R n −→ R is C 2 , strictly convex, and lim |u|→∞ ℓ(u) |u| = ∞; A2: g is globally Lipschitz in R n ; 1], with a > 0, is Lipschitz continuous with d t (t) = 1 for each t. 1 We consider only papers on continuous time.
with time-varying preference rate, chooses optimally along time. In another reading, there is a continuum of agents, each one labelled by t ∈ [0, T ]; each agent (or generation t) applies a possibly different discount factor, d t , in the calculation of the utility flow from t onwards.
At time T the optimization process finishes and agent T derives utility g(y(T )) (or "scrap value"). The aim of each generation is to maximize the total discounted utility. In this process, the t-generation is not so much concerned with the consumption of the future generations as it is with respect its own consumption.
A common specification of d t (s) is where ρ ∈ L ∞ ([0, T ]). In this case d t (s) is Lipschitz in (t, s), which is the present value at time t of one unit of utility at time s ≥ t. The rate of discount is ρ, and most often it is considered constant. Other popular discount factors are those that depend only on the elapsed time, d t (s) = θ(s − t) for s ≥ t, through a scalar function θ, with θ(0) = 1. As will be seen in Section 4, the shape of the discount factor has a major effect in the structure of the dynamic programming equation.
Let us define ι = (∇ℓ) −1 , the inverse of ∇ℓ. Notice that by A1, both ∇ℓ and ι are continuous, and suprajective. We also consider ℓ * (p) = sup u∈R n {p · u − ℓ(u)}, the Legendre transform of ℓ. Finally, let the t-Hamiltonian Throughout the paper, ∇ denotes the gradient of a real function, and ∇ 2 the Hessian Matrix. For a vector function, ∇ denotes the Jacobian matrix.

Hopf-Lax formula
A Hopf-Lax formula describes an infinite dimensional variational problem as a finite dimensional one. In the present case, the formula is a bit more involved than in the nondiscounted case, due to the non-autonomous term d t (s). Notice also that the problem at hand is different from the one with a non-autonomous ℓ(s,ẏ(s)), because the current date t enters into the definition.
Notice that Y t,x,α (s) is absolutely continuous and Y x,t,α (t) = x, thus it is an admissible arc, i.e. it belongs to AC x,t . Observe also thatẎ x,t,α (s) = U t,α (s).
We establish the following lemma to facilitate posterior quotation. It is a consequence of assumption A1. (2) is given by

On the other hand, for an arbitrary function
This is possible by Lemma 3.1. For each t, s, p, the Hamiltonian H t (·, u, ·) is concave, thus for any α Let α = α defined above. Integrating (5) between t and T and rearranging terms we get to both terms of the above inequality we get that, for any arc y(s), there exist some α such Finally, observe that minimization with respect to α is equivalent of minimization with respect to p = ∇ℓ(α).
3. The infimum is in fact attained, since the function V (·, ·, α) is continuous and inf- valued and upper semicontinuous correspondence.
The following corollary is along the lines of the above proof.
Corollary 3.1. If x ∈ R n and 0 ≤ t < T , then for any selection α(x, t) ∈ A(x, t), the arc Y x,t,α(x,t) (s) is a solution of problem (2).
.  Proof. Let x,x ∈ K ⊆ R n with K compact, and t,t ∈ [0, T ) and α ∈ R n .
In the above, we have used the same C to denote several constants.
3. Choose α ∈ A(x, t). Then, by definition of v and estimate (6) v Reversing the role of (x,t) and (x, t) we get the desired Lipschitz property.
4. Now, let x ∈ R n , t < T and define δ t = T t d t (s) ds. Choose α ∈ R n such that by Jensen's inequality. Now, notice that for any α ∈ R n T t U t,α (s) ds thus, for every t close enough to T , there exists ǫ > 0 such that Thus, by (7) and (8) |v(x, t) − d t (T )g(x)| ≤ Cδ t for an appropriated constant C. Given that d t (T ) tends to 1 and δ t tends to 0 as t → T , we are done.

Dynamic programming equation
For any y ∈ AC x,t and t ≤ τ ≤ T , let Y τ denote an optimal arc from initial condition (y(τ ), τ ), that is, which exists by Corollary 3.1.
Then, by Corollary 3.1 Let us define the admissible arcỹ ∈ AC x,t bỹ Proof. In fact (9) is an equality since an optimal arc is attained for every initial condition x, t, by Corollary 3.1.

Now consider the function
The (generalized) dynamic programming equation is as follows. It could be obtained for a more general optimal control problem with some additional assumptions.
for any y ∈ AC x,t .

Taking limits in (13
for every y ∈ AC x,t . This yields for every u ∈ R n . Recalling the definition of ℓ * , this is equivalent to 4. To prove the equality, we use the same argument. Notice that equality holds in (13) for  (3) holds, since we are assuming d t (t) = 1 for each t.

This happens if and only if
In the general case, the dynamic programming equation (12) has a complicated structure.
Indeed, the optimal arc itself enters the formulation as a non-local term, thus the applicability of the equation should be taken with caution. In contrast, the solution given in (4) is simpler. This stresses the usefulness of having a Hopf-Lax formula at hand. Nevertheless, we can give a more amenable form to the dynamic programming equation, close to classical standards, when assuming that both the value function and function g are differentiable.
This is the content of the next theorem. is definite positive for every u ∈ R n and that g is differentiable; then, the dynamic programming equation (12) is Proof. Since we are supposing v is differentiable, the envelope theorem applied to (4) gives On the other hand, α is an irrestricted minimum of V , hence Since ∇ 2 ℓ has maximal rank and ∇ι(·) = (∇ 2 ℓ(·)) −1 , the gradient of V with respect to α is the null vector only if ∇ℓ(α) = −v x (x, t) and then α(x, t) = ι(−v x (x, t)) at points of differentiability of v (incidentally, this shows that α must be unique at points of differentiability of v). Plugging this value for α into w(x, t, α) we reach the expression for the dynamic programming equation asserted in the theorem.

Regularity of the value function
In the conditions of the above theorem, for the particular case of the Hamilton-Jacobi equation with dissipation we have  (1) If g is of class C 1 , then the value function v is differentiable in R n × (0, T ) and the minimizer α is continuous.
(2) If g is of class C 2 , then the value function v is also of class C 2 in R n × (0, T ) and the minimizer α is of class C 1 in R n × (0, T ).
Proof. The minimizers α in (4) satisfy 1. For x, t fixed but arbitrary, the mapping α → ∇ℓ(α) + d t (T )∇g (Y t,x,α (T )) is strictly monotone due to the convexity of g and the strict convexity of ℓ, thus α(x, t) is unique; as a correspondence, {α(x, t)} is upper semicontinuous thus, as a function it is continuous. The uniqueness of α leads to the differentiability of the value function, by Danskin's Theorem.
By the envelope theorem, v x (x, t) = ∇g(Y x,t,α(x,t) (T )) is of class C 1 hence, by the dynamic programming equation (14), v t is also of class C 1 .
Finally, a result concerning the monotonic behavior of α in the scalar case. (1) For each time 0 < t < T , there exists for all but at most countably many values of x ∈ R a unique point α(x, t) where the minimum in (4) is attained.
Since v is locally Lipschitz, it is differentiable almost everywhere, thus the minimum α is unique almost everywhere. On the other hand, the crossed derivative of V (x, t, α) with respect to x and α is Hence, (x, α) → V (x, t, α) is supermodular, and by Topkis' Theorem, [8], A(x, t) is a nonempty compact sublattice which admits a lowest element, which we denote again by α(x, t), satisfying α(x 2 , t) ≥ α(x 1 , t) whenever x 2 > x 1 . Then the mapping x → α(x, t) is non-decreasing and thus continuous for all but at most countably many x.