Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method

Abstract

We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov’s approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook’s model for gene expression.

Appendix

The Proof of Theorem 7 relies on the fact that the functions V ^ε defined by (15) are viscosity subsolutions of the Hamilton-Jacobi integro-differential equation (8). The proof adapts the arguments used in Barles, Jakobsen [2] Lemma 2.7. Following the proof of this Lemma, we introduce, for every h>0, u ²∈ℝ^N, $Q_{h}^{u^{2}}=u^{2}+ [ -\frac{h}{2},\frac{h}{2} )^{N}$, $\rho_{\varepsilon}^{h,u^{2}}=\int_{Q_{h}^{u^{2}}}\rho_{\varepsilon}(y)dy$, and $I_{h} ( x ) =\sum_{u^{2}\in h\mathbb{Z}^{N}}\rho_{\varepsilon}^{h,u^{2}}v^{\varepsilon} ( x-u^{2} )$. Thus, I _h is a convex combination of bounded, uniformly continuous viscosity subsolutions of (8). Moreover, by classical results, the discretization I _h converges uniformly to V ^ε. To conclude, we show that viscosity subsolutions are preserved by convex combination and uniform convergence.

Proposition 22

Given two bounded, uniformly continuous viscosity subsolutions v ₁ and v ₂ of (8) and two nonnegative real constants λ ₁,λ ₂∈ℝ₊ such that λ ₁+λ ₂=1, the convex combination λ ₁ v ₁+λ ₂ v ₂ is still a viscosity subsolution of (8).

Proof

The assertion is trivial when either λ ₁=0 or λ ₂=0. If λ ₁ λ ₂≠0, we let $\overline{x}\in \mathbb{R}^{N}$ and $\varphi\in C_{b}^{1} ( \mathcal{N}_{\overline{x}} )$ be a test function such that

(27)

for all y∈ℝ^N. We may assume, without loss of generality that φ∈C _b (ℝ^N). Indeed, whenever φ does not satisfy this assumption, one can replace it with some φ ⁰ defined as follows: First, notice that there exists some r>0 such that $B ( \overline{x},2r )\subset\mathcal{N}_{\overline{x}}$. We define

for all y∈ℝ^N, where χ is a smooth function such that 0≤χ≤1, χ(y)=1, if $y\in B ( \overline{x},r )$ and χ(y)=0, if $y\in \mathbb{R}^{N}\smallsetminus B ( \overline{x},2r )$. Then (27) holds true with φ ⁰ instead of φ. The new function φ ⁰ also satisfies

We introduce, for every ε>0

for all x,y∈ℝ^N. We recall that the functions v ₁,v ₂ and φ are bounded and continuous. This yields the existence of a global maximum (x _ε ,y _ε ) of Φ _ε . Moreover, by standard arguments,

(28)

We consider the test function ψ given by

for all x∈ℝ^N. We recall that the function v ₁ is a viscosity subsolution for (8). Then,

Standard estimates yield

(29)

In a similar way, we get

(30)

Finally, using (29), (30) and (28), and passing to the limit as ε→0, yields

 □

These arguments allow to obtain, by recurrence, that any convex combination of continuous, bounded viscosity subsolutions is still a subsolution for (8).

Proposition 23

(Stability)

Let (v _n ) _n be a sequence of continuous, uniformly bounded viscosity subsolutions of (8). Moreover, we suppose that v _n converges uniformly on compact sets to some continuous, bounded function v. Then the function v is a viscosity subsolution of (8).

Proof

We let x∈ℝ^N and $\varphi\in C_{b}^{1} ( \mathcal{N}_{x} )$ be a test function such that v−φ has a global maximum at x. As in the previous proposition, one can assume, without loss of generality, that φ∈C _b (ℝ^N). Classical arguments yield the existence of some point x _n ∈ℝ^N such that

for all y∈ℝ^N and

We assume, without loss of generality, that |x _n −x|≤1, and $x_{n}\in\mathcal{N}_{x}$, for all n≥1. Then,

(31)

We have

(32)

where C>0 is a generic constant independent of n≥1 and u∈U which may change from one line to another. We also get

(33)

Finally, for every m≥1,

(34)

We substitute (32)–(34) in (31) and allow n→∞ to have

(35)

for all m≥1. We conclude using the Assumption (A4b). □