Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics

In this paper we deﬁne an inﬁnite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with ﬁnite time horizon and unbounded cost. This process is a coupling between a continuous time Markov Chain and a set of semilinear parabolic partial di ﬀ erential equations, both processes depending on the control. We apply dynamic programming to the embedded Markov decision process to obtain existence of optimal relaxed controls and we give some su ﬃ cient conditions ensuring the existence of an optimal ordinary control. This study, which constitutes an extension of controlled PDMPs to inﬁnite dimension, is motivated by the control that provides Optogenetics on neuron models such as the Hodgkin-Huxley model. We deﬁne an inﬁnite-dimensional controlled Hodgkin-Huxley model as an inﬁnite-dimensional controlled piecewise deterministic Markov process and apply the previous results to prove the existence of optimal ordinary controls for a tracking problem.


Introduction
Optogenetics is a recent and innovative technique which allows to induce or prevent electric shocks in living tissues, by means of light stimulation.Successfully demonstrated in mammalian neurons in 2005 ( [8]), the technique relies on the genetic modification of cells to make them express particular ionic channels, called rhodopsins, whose opening and closing are directly triggered by light stimulation.One of these rhodopsins comes from an unicellular flagellate algae, Chlamydomonas reinhardtii, and has been baptized Channelrodhopsins-2 (ChR2).It is a cation channel that opens when illuminated with blue light.Since the field of Optogenetics is young, the mathematical modeling of the phenomenon is quite scarce.Some models have been proposed, based on the study of the photocycles initiated by the absorption of a photon.In 2009, Nikolic and al. [37] proposed two models for the ChR2 that are able to reproduce the photocurrents generated by the light stimulation of the channel.Those models are constituted of several states that can be either conductive (the channel is open) or non-conductive (the channel is closed).Transitions between those states are spontaneous, depend on the membrane potential or are triggered by the absorption of a photon.For example, the four-states model of Nikolic and al. [37] has two open states (o 1 and o 2 ) and two closed states (c 1 and c 2 ).Its transitions are represented on Figure 1  The purpose of this paper is to extend to infinite dimension the optimal control of Piecewise Deterministic Markov Processes (PDMPs) and to define an infinite-dimensional controlled Hodgkin-Huxley model, containing ChR2 channels, as an infinite-dimensional controlled PDMP and prove existence of optimal ordinary controls.We now give the definition of the model.We consider an axon, described as a 1-dimensional cable and we set I = [0, 1] (the more physical case I = [−l, l] with 2l > 0 the length of the axon is included here by a scaling argument).Let D ChR2 := {o 1 , o 2 , c 1 , c 2 }.Individually, a ChR2 features a stochastic evolution which can be properly described by a Markov Chain on the finite space constituted of the different states that the ChR2 can occupy.In the four-states model above, two of the transitions are triggered by light stimulation, in the form of a parameter u that can evolve in time.Here u(t) is physically proportional to the intensity of the light with which the protein is illuminated.For now, we will consider that when the control is on (i.e., when the light is on), the entire axon is uniformly illuminated.Hence for all t ≥ 0, u(t) features no spatial dependency.The deterministic Hodgkin-Huxley model was introduced in [33].A stochastic infinite-dimensional model was studied in [4], [10], [31] and [43].The Sodium (N a + ) channels and Potassium (K + ) channels are described by two pure jump processes with state spaces D 1 := {n 0 , n 1 , n 2 , n 3 , n 4 } and D 2 := {m 0 h 1 , m 1 h 1 , m 2 h 1 , m 3 h 1 , m 0 h 0 , m 1 h 0 , m 2 h 0 , m 3 h 0 }.For a given scale N ∈ N * , we consider that the axon is populated by N hh = N − 1 channels of type N a + , K + or ChR2, at positions 1 N (Z ∩ N I).In the sequel we will use the notation I N := Z ∩ N I.
We consider the Gelfand triple (V, H, V * ) with V := H 1 0 (I) and H := L 2 (I).The process we study is defined as a controlled infinite-dimensional Piecewise Deterministic Markov Process (PDMP).All constants and auxiliary functions in the next definition will be defined further in the paper.Definition 0.1.Stochastic controlled infinite-dimensional Hodgkin-Huxley-ChR2 model.Let N ∈ N * .We call N th stochastic controlled infinite-dimensional Hodgkin-Huxley-ChR2 model the controlled PDMP (v(t), d(t)) ∈ V × D N defined by the following characteristics: • A control space U = [0, u max ], u max > 0.
• A set of uncontrolled PDEs: For every d ∈ D N , (0.1) )) (0.2) with δ z ∈ V * the Dirac mass at z ∈ I.
• A jump rate function λ : with σ x,y : R × U → R * + smooth functions for all (x, y) ∈ D 2 .See Table 1 in Section 4.1 for the expression of those functions.
• A discrete transition measure Q : V ×D N ×U → P(D N ) defined for all (v, d, u) ∈ E×D N ×U and y ∈ D by where d i:y is obtained from d by putting its i th component equal to y.
From a biological point of view, the optimal control problem consists in mimicking an output signal that encodes a given biological behavior, while minimizing the intensity of the light applied to the neuron.For example, it can be a time-constant signal and in this case, we want to change the resting potential of the neuron to study its role on its general behavior.We can also think of pathological behaviors that would be fixed in this way.The minimization of light intensity is crucial because the range of intensity experimentally reachable is quite small and is always a matter of preoccupation for experimenters.These considerations lead us to formulate the following mathematical optimal control problem.Suppose we are given a reference signal V ref ∈ V .The control problem is then to find α ∈ A that minimizes the following expected cost (0.5) where A is the space of control strategies, Υ an auxiliary state space that comprises V × D N , X α • is the controlled PDMP and X α • (φ) its continuous component.We will prove the following result.
Theorem 0.1.Under the assumptions of Section 1.1, there exists an optimal control strategy α * ∈ A such that for all z ∈ Υ, and the value function z → inf α∈A J z (α) is continuous on Υ.
Piecewise Deterministic Markov Processes constitute a large class of Markov processes suited to describe a tremendous variety of phenomena such as the behavior of excitable cells ( [4], [10], [40]), the evolution of stocks in financial markets ( [11]) or the congestion of communication networks ( [23]), among many others.PDMPs can basically describe any non diffusive Markovian system.The general theory of PDMPs, and the tools to study them, were introduced by Davis ( [18]) in 1984, at a time when the theory of diffusion was already amply developed.Since then, they have been widely investigated in terms of asymptotic behavior, control, limit theorems and CLT, numerical methods, among others (see for instance [9], [14], [15], [17] and references therein).PDMPs are jump processes coupled with a deterministic evolution between the jumps.They are fully described by three local characteristics: the deterministic flow φ, the jump rate λ, and the transition measure Q.In [18], the temporal evolution of a PDMP between jumps (i.e. the flow φ) is governed by an Ordinary Differential Equation (ODE).For that matter, this kind of PDMPs will be referred to as finite-dimensional in the sequel.
Optimal control of such processes have been introduced by Vermes ( [44]) in finite dimension.In [44], the class of piecewise open-loop controls is introduced as the proper class to consider to obtain strongly Markovian processes.A Hamilton-Jabobi-Bellman equation is formulated and necessary and sufficient conditions are given for the existence of optimal controls.The standard broader class of so-called relaxed controls is considered and it plays a crucial role in getting the existence of optimal controls when no convexity assumption is imposed.This class of controls has been studied, in the finite-dimensional case, by Gamkrelidze ([29]), Warga ([47] and [46]) and Young ([50]).Relaxed controls provide a compact class that is adequate for studying optimization problems.Still in finite dimension, many control problems have been formulated and studied such as optimal control ([28]), optimal stopping ( [16]) or controllability ([32]).In infinite dimension, relaxed controls were introduced by Ahmed ([1], [2], [3]).They were also studied by Papageorgiou in [41] where the author shows the strong continuity of relaxed trajectories with respect to the relaxed control.This continuity result will be of great interest in this paper.
A formal infinite-dimensional PDMP was defined in [10] for the first time, the set of ODEs being replaced by a special set of Partial Differential Equations (PDE).The extended generator and its domain are provided and the model is used to define a stochastic spatial Hodgkin-Huxley model of neuron dynamics.The optimal control problem we have in mind here regards those Hodgkin-Huxley type models.Seminal work on an uncontrolled infinite-dimensional Hodgkin-Huxley model was conducted in [4] where the trajectory of the infinite-dimensional stochastic system is shown to converge to the deterministic one, in probability.This type of model has then been studied in [43] in terms of limit theorems and in [31] in terms of averaging.The extension to infinite dimension heavily relies on the fact that semilinear parabolic equations can be interpreted as ODEs in Hilbert spaces.
To give a sense to Definition 0.1 and to Theorem 0.1, we will define a controlled infinite-dimensional PDMP for which the control acts on the three local characteristics.We consider controlled semilinear parabolic PDEs, jump rates λ and transition measures Q depending on the control.This kind of PDE takes the form where L is the infinitesimal generator of a strongly continuous semigroup and f is some function (possibly nonlinear).The optimal control problem we address is the finite-time minimization of an unbounded expected cost functional along the trajectory of the form where x(•) is the continuous component of the PDMP, u(•) the control and T > 0 the finite time horizon, the cost function c(•, •) being potentially unbounded.
To address this optimal control problem, we use the fairly widespread approach that consists in studying the imbedded discrete-time Markov chain composed of the times and the locations of the jumps.Since the evolution between jumps is deterministic, there exists a one-to-one correspondence between the PDMP and a pure jump process that enable to define the imbedded Markov chain.The discrete-time Markov chain belongs to the class of Markov Decision Processes (MDPs).This kind of approach has been used in [28] and [12] (see also the book [34] for a self-contained presentation of MDPs).In these articles, the authors apply dynamic programming to the MDP derived from a PDMP, to prove the existence of optimal relaxed strategies.Some sufficient conditions are also given to get non-relaxed, also called ordinary, optimal strategies.However, in both articles, the PDMP is finite dimensional.To the best of our knowledge, the optimal control of infinite-dimensional PDMPs has not yet been treated and this is one of our main objectives here, along with its motivation, derived from the Optogenetics, to formulate and study infinite-dimensional controlled neuron models.
The paper is structured as follows.In Section 1 we adapt the definition of a standard infinitedimensional PDMP given in [10] in order to address control problems of such processes.To obtain a strongly Markovian process, we enlarge the state space and we prove an extension to controlled PDMPs of [10,Theorem 4].We also define in this section the MDP associated to our controlled PDMP and that we study later on.In Section 2 we use the results of [41] to define relaxed controlled PDMPs and relaxed MDPs in infinite dimension.Section 3 gathers the main results of the paper.We show that the optimal control problems of PDMPs and of MDPs are equivalent.
We build up a general framework in which the MDP is contracting.The value function is then shown to be continuous and existence of optimal relaxed control strategies is proved.We finally give in this section, some convexity assumptions under which an ordinary optimal control strategy can be retrieved.
The final Section 4 is devoted to showing that the previous theoretical results apply to the model of Optogenetics previously introduced.Several variants of the model are discussed, the scope of the theoretical results being much larger than the model of Definition 0.1.
1 Theoretical framework for the control of infinite-dimensional PDMPs

The enlarged process and assumptions
In the present section we define the infinite-dimensional controlled PDMPs that we consider in this paper in a way that enables us to formulate control problems in which the three characteristics of the PDMP depend on an additional variable that we call the control parameter.In particular we introduce the enlarged process which enable us to address optimization problems in the subsequent sections.
Let (Ω, F , (F t ) t≥0 , P) be a filtered probability space satisfying the usual conditions.We consider a Gelfand triple (V ⊂ H ⊂ V * ) such that H is a separable Hilbert space and V a separable, reflexive Banach space continuously and densely embedded in H.The pivot space H is identified with its dual H * , V * is the topological dual of V .H is then continuously and densely embedded in V * .We will denote by Let D be a finite set, the state space of the discrete variable and Z a compact Polish space, the control space.Let T > 0 be the finite time horizon.Intuitively a controlled PDMP (v t , d t ) t∈[0,T ] should be constructed on H × D from the space of ordinary control rules defined as where U , the action space, is a closed subset of Z. Elements of A are defined up to a set in [0, T ] of Lebesgue measure 0. The control rules introduced above are called ordinary in contrast with the relaxed ones that we will introduce and use in order to prove existence of optimal strategies.
For (v, d, a) ∈ H × D × A we will denote by φ a (v, d) the flow of (1.1).Let T n , n ∈ N be the jump times of the PDMP.Their distribution is then given by When a jump occurs, the distribution of the post jump state is given by The triple (λ, Q, φ) fully describes the process and is referred to as the local characteristics of the PDMP.
We will make the following assumptions on the local characteristics of the PDMP.

(H(λ))
For every d ∈ D, λ d : H × Z → R + is a function such that: 1.There exists M λ , δ > 0 such that: ) is locally Lipschitz continuous, uniformly in Z, that is, for every compact set K ⊂ H, there exists l λ (K) > 0 such that In particular, assumption (H(f )) implies (H(f))'2.and we will use the constants b 1 and b 2 further in this paper.Note that they can be chosen uniformly in D since it is a finite set.To see this, note that z → f d (0, z) is a weakly continuous on the compact space Z and thus weakly bounded.
It is then strongly bounded by the Uniform Boundedness Principle.Finally, assumptions (H(f ))3.and (H(L))4.will respectively ensure the existence of relaxed solutions of (1.1) and the strong continuity of theses solutions with regards to the relaxed control.
For that last matter, the compactness of Z is also required.The following theorem is a reminder that the assumption on the semigroup does not make the problem trivial since it implies that L is unbounded when H is infinite-dimensional.
(b) (T (t)) t≥0 is immediately norm continuous, and its generator has compact resolvent.
2. Let X be a Banach space.A bounded operator A ∈ L(X) has compact resolvent if and only if X is finite-dimensional.
Because of (H(L)) and (H(f )), for all a ∈ U ad ((0, T ), U ), (1.1) has a unique solution belonging to L 2 ((0, T ), V ) ∩ H 1 ((0, T ), V * ) and moreover, the solution belongs to C([0, T ], H) (see [41] for the construction of such a solution).We will make an extensive use of the mild formulation of the solution of (1.1), given by One of the keys in the construction of a controlled PDMP in finite or infinite dimension is to ensure that φ a enjoys the flow property φ a t+s (v, d) = φ a s (φ a t (v, d), d) for all (v, d, a) ∈ H ×D×U ad ((0, T ), U ) and (t, s) ∈ R + .It is the flow property that guarantees the Markov property for the process.Under the formulation (1.4), it is easy to see that the solution φ a cannot feature the flow property for any reasonable set of admissible rules.In particular, the jump process (d t , t ≥ 0) given by (1.2) and (1.3) is not Markovian.Moreover in control problems, and especially in Markovian control problems, we are generally looking for feedback controls which depend only on the current state variable so that at any time, the controller needs only to observe the current state to be able to take an action.Feedback controls would ensure the flow property.However they impose a huge restriction on the class of admissible controls.Indeed, feedback controls would be functions u : H × D → U and for the solution of (1.1) to be uniquely determined, the function x → f d (x, u(x, d)) needs to be Lipschitz continuous.It would automatically exclude discontinuous controls and therefore would not be adapted to control problems.To avoid this issue, Vermes introduced piecewise open-loop controls (see [44]): after a jump of the discrete component, the controller observes the location of the jump, say (v, d) ∈ H × D and chooses a control rule a ∈ U ad ((0, T ), U ) to be applied until the next jump.The time elapsed since the last jump must then be added to the state variable in order to see a control rule as a feedback control.While Vermes [44] and Davis [19] only add the last post jump location we also want to keep track of the time of the last jump in order to define proper controls for the Markov Decision Processes that we introduce in the next section, and to eventually obtain optimal feedback policies.According to these remarks, we now enlarge the state space and define control strategies for the enlarged process.We introduce first several sets that will be useful later on.b) On Ξ we define the enlarged controlled PDMP (X α t ) t≥0 = (v t , d t , τ t , h t , ν t ) t≥0 with strategy α ∈ A as follows: • τ t is the time elapsed since the last jump at time t, • h t is the time of the last jump before time t, • ν t is the post jump location right after the jump at time and its immediate extension χ α .(z) to A such that the process (X α t ) t≥0 starting at (v, d, 0, h, v) ∈ Ξ, admits χ α .as survival function: and φ α t (z) means Lemma 3], the set of admissible control strategies can be seen as a set of measurable feedback controls acting on Ξ and with values in U .The formulation of Definition 1.2 is adequate to address the associated discrete-time control problem in Section 1.3.ii) In view of Definition 1.2, given α ∈ A, the deterministic dynamics of the process (X α t ) t≥0 = (v t , d t , τ t , , h t , ν t ) t≥0 between two consecutive jumps obeys the initial value problem with s the last time of jump.The jump rate function and transition measure of the enlarged PDMP are straightforwardly given by the ones of the original process and will be denoted the same (see Appendix A for their expression).
iii) If the relation t = h t + τ t indicates that the variable h t might be redundant, recall that we keep track of it on purpose.Indeed, the optimal control will appear as a function of the jump times so that keeping them as a variable will make the control feedback.iv) Because of the special definition of the enlarged process, for every control strategy in A, the initial point of the process (X α t ) t≥0 cannot be any point of the enlarged state space Ξ.More precisely we introduce in Definition 1.3 below the space of coherent initial points.Definition 1.3.Space of coherent initial points.Take α ∈ A and x := (v 0 , d 0 , 0, h 0 , v 0 ) ∈ Ξ and extend the notation φ α t (x) of Definition 1.2 to Ξ by The set Ξ α ⊂ Ξ of coherent initial points is defined as follows Note that (X α t ) can be constructed like any PDMP by a classical iteration that we recall in Appendix A for the sake of completeness.

A probability space common to all strategies
Up to now thanks to Definition 1.2 we can formally associate the PDMP (X α t ) t∈R+ to a given strategy α ∈ A. However, we need to show that there exists a filtered probabily space satisfying the usual conditions under which, for every control strategy α ∈ A, the controlled PDMP (X α t ) t≥0 is a homogeneous strong Markov process.This is what we do in the next theorem which provides an extension of [10,Theorem 4] to controlled infinite-dimensional PDMPs and some estimates on the continuous component of the PDMP.

Theorem 1.2. Under assumptions (H(λ)), (H(Q)), (H(L)) and (H(f )) (or (H(f ))') are satisfied. a)
There exists a filtered probability space satisfying the usual conditions such that for every control strategy α ∈ A the process (X α t ) t≥0 introduced in Definition 1.2 is a homogeneous strong Markov process on Ξ with extended generator G α given in Appendix B. b) For every compact set K ⊂ H, there exists a deterministic constant c K > 0 such that for all control strategy α ∈ A and initial point The proof of Theorem 1.2 is given in Appendix B. In the next section, we introduce the MDP that will allow us to prove the existence of optimal strategies.

A Markov Decision Process (MDP)
Because of the particular definition of the state space Ξ, the state of the PDMP just after a jump is in fact fully determined by a point in Υ.In Appendix B we recall the one-to-one correspondence between the PDMP on Ξ and the included pure jump process (Z n ) n∈N with values in Υ.This pure jump process allows to define a Markov Decision Process , where ∆ ∞ is a cemetery state added to Υ to define a proper MDP.In order to lighten the notations, the dependence on a control strategy α ∈ A of both jump processes is implicit.The stochastic kernel and a ∈ U ad ([0, T ], U ), where The conditional jumps of the MDP (Z ′ n ) n∈N are then given by the kernel Q ′ (•|z, α(z)) for (z, α) ∈ Υ × A. Note that Z ′ n = Z n as long as T n ≤ T , where T n is the last component of Z n .Since we work with Borel state and control spaces, we will be able to apply techniques of [6] for discrete-time stochastic control problems, without being concerned by measurability matters.See [6, Section 1.2] for an illuminating discussion on these measurability questions.

Relaxed controls
Relaxed controls are constructed by enlarging the set of ordinary ones, in order to convexify the original system, and in such a way that it is possible to approximate relaxed strategies by ordinary ones.The difficulty in doing so is twofold.First, the set of relaxed trajectories should not be much larger than the original one.Second, the topology considered on the set of relaxed controls should make it a compact set and, at the same time, make the flow of the associated PDE continuous.Compactness and continuity are two notions in conflict so being able to achieve such a construction is crucial.Intuitively a relaxed control strategy on the action space U corresponds to randomizing the control action: at time t, instead of taking a predetermined action, the controller will take an action with some probability, making the control a transition probability.This has to be formalized mathematically.
Notation and reminder.Z is a compact Polish space, C(Z) denotes the set of all real-valued continuous, necessarily bounded, functions on Z, endowed with the supremum norm.Because Z is compact, by the Riesz Representation Theorem, the dual space [C(Z)] * of C(Z) is identified with the space M (Z) of Radon measures on B(Z), the Borel σ-field of Z.We will denote by M 1 + (Z) the space of probability measures on Z.The action space U is a closed subset of Z.We will use the notations

Relaxed controls for a PDE
Let B([0, T ]) denote the Borel σ-field of [0, T ] and Leb the Lebesgue measure.A transition probability from We will denote by R([0, T ], Z) the set of all transition probability measures from Recall that we consider the PDE (1.1): (2.1) The relaxed PDE is then of the form 2) is to be understood in the sense of Bochner-Lebesgue as we show now.The topology we consider on R([0, T ], U ) follows from [5] and because Z is a compact metric space, it coincides with the usual topology of relaxed control theory of [48].It is the coarsest topology that makes continuous all mappings for every Carathéodory integrand f : [0, T ] × Z → R, a Carathéodory integrand being such that This topology is called the weak topology on R([0, T ], Z) but we show now that it is in fact metrizable.Indeed, Carathéodory integrands f on [0, T ] × Z can be identified with the Lebesgue-Bochner space Hence, the weak topology on R([0, T ], Z) can be identified with the w * -topology in (L ∞ (M (Z)), L 1 (C(Z))), the latter being metrizable since L 1 (C(Z)) is a separable space (see [24, Theorem 1 p. 426]).This crucial property allows to work with sequences when dealing with continuity matters with regards to relaxed controls.Finally, by Alaoglu's Theorem, R([ , and the set of original admissible controls For the same reasons why (2.1) admits a unique solution, by setting fd (v, γ) : it is straightforward to see that (2.2) admits a unique solution.The following theorem gathers of [41, Theorems 3.2 and 4.1] and will be of paramount importance in the sequel.b) The mapping that maps a relaxed control to the solution of (2.2) is continuous from R([0, T ], U ) into C([0, T ], H).

Relaxed controls for infinite-dimensional PDMPs
First of all, note that since the control acts on all three characteristics of the PDMP, convexity assumptions on the fields f d (v, U ) would not necessarily ensure existence of optimal controls as it does for partial differential equations.Such assumptions should also be imposed on the rate function and the transition measure of the PDMP.For this reason, relaxed controls are even more important to prove existence of optimal controls for PDMP.For what has been done for PDE above, we are now able to define relaxed PDMPs.The next definition is the relaxed analogue of Definition 1.2.
Definition 2.1.Relaxed control strategies, relaxed local characteristics.a) The set A R of relaxed admissible control strategies for the PDMP is defined by Given a relaxed control strategy µ ∈ A R and z ∈ Υ, we will denote by µ z := µ(z) ∈ R([0, T ]; U ) and µ z t the corresponding probability measure on (Z, , we extend the jump rate function and transition measure as follows the expression for the enlarged process being straightforward.This allows us to give the relaxed survival function of the PDMP and the relaxed mild formulation of the solution of (2.2) ), we will also use the following notation The following proposition is a direct consequence of Theorem 1.2b).
Proposition 2.1.For every compact set K ⊂ H, there exists a deterministic constant c K > 0 such that for all control strategy µ ∈ A R and initial point The relaxed transition measure is given in the next section through the relaxed stochastic kernel of the MDP associated to our relaxed PDMP.

Relaxed associated MDP
Let z := (v, d, h) ∈ Υ and γ ∈ R([0, T ], U ).The relaxed stochastic kernel of the relaxed MDP satisfies where , with, as before, the conditional jumps of the MDP (Z ′ n ) n∈N given by the kernel

Main results
Here, we are interested in finding optimal controls for optimization problems involving infinitedimensional PDMPs.For instance, we may want to track a targeted "signal" (as a solution of a given PDE, see Section 4).To do so, we are going to study the optimal control problem of the imbedded MDP defined in Section 1.3.This strategy has been for example used in [12] in the particular setting of a decoupled finite-dimensional PDMP, the rate function being constant.

The optimal control problem
Thanks to the preceding sections we can consider ordinary or relaxed costs for the PDMP X α or the MDP and their corresponding value functions.For z := (v, d, h) ∈ Υ and α ∈ A we denote by E α z the conditional expectation given that X α h = (v, d, 0, h, v) and by X α s (φ) the first component of X α s .Furthermore, we denote by X α s := (v s , d s , τ s , h s , ν s ), then the shortened notation α(X α s ) will refer to α τs (ν s , d s , h s ).Theses notations are straightforwardly extended to A R .We introduce a running cost c : H × Z → R + and a terminal cost g : are nonnegative norm quadratic functions, that is there exists This assumption might seem a bit restrictive, but it falls within the framework of all the applications we have in mind.More importantly, it can be widely loosened if we slightly change the assumptions of Theorem 3.1.In particular, all the following results, up to Lemma 3.7, are true and proved for continuous functions c : H × Z → R + and g : H → R + .See Remark 3.4 below.
Definition 3.1.Ordinary value function for the PDMP X α .For α ∈ A , we define the ordinary expected total cost function V α : Υ → R and the corresponding value function V as follows: Assumption (H(c)) ensures that V α and V are properly defined.
Definition 3.2.Relaxed value function for the PDMP X µ .For µ ∈ A R we define the relaxed expected cost function V µ : Υ → R and the corresponding relaxed value function Ṽ as follows: We can now state the main result of this section.
Theorem 3.1.Under assumptions (H(λ)), (H(Q)), (H(L)), (H(f )) and (H(c)), the value function Ṽ of the relaxed optimal control problem on the PDMP is continuous on Υ and there exists an optimal relaxed control strategy µ * ∈ A R such that Remark 3.2.All the subsequent results that lead to Theorem 3.1 would be easily transposable to the case of a lower semicontinuous cost function.We would then obtain a lower semicontinuous value function.
The next section is dedicated to proving Theorem 3.1 via the optimal control of the MDP introduced before.Let us briefly sum up what we are going to do.We first show that the optimal control problem of the PDMP is equivalent to the optimal control problem of the MDP and that an optimal control for the latter gives an optimal control strategy for the original PDMP.We will then build up a framework, based on so called bounding functions (see [12]), in which the value function of the MDP is the fixed point of a contracting operator.Finally, we show that under the assumptions of Theorem 3.1, the relaxed PDMP X µ belongs to this framework.

Optimal control of the MDP
Let us define the ordinary cost and c ′ (∆ ∞ , a) := 0. Assumption (H(c)) allows c ′ to be properly extended to R([0, T ], U ) by the formula We can now define the expected cost function and value function for the MDP.
Definition 3.3.Cost and value functions for the MDP (Z ′ n ).For α ∈ A (resp.µ ∈ A R ), we define the total expected cost J α (resp.J µ ) and the value function The finiteness of theses sums will by justified later by Lemma 3.2.

The equivalence Theorem
In the following theorem we prove that the relaxed expected cost function of the PDMP equals the one of the associated MDP.Thus, the value functions also coincide.For the finite-dimensional case we refer the reader to [19] or [12] where the discrete component of the PDMP is a Poisson process and therefore the PDMP is entirely decoupled.The PDMPs that we consider are fully coupled.
Theorem 3.2.The relaxed expected costs for the PDMP and the MDP coincide: V µ (z) = J µ (z) for all z ∈ Υ and relaxed control µ ∈ A R .Thus, the value functions Ṽ and J ′ coincide on Υ.
Remark 3.3.Since we have A ⊂ A R , the value functions V α (z) and J α (z) also coincide for all z ∈ Υ and ordinary control strategy α ∈ A Proof.Let µ ∈ A R and z = (v, d, h) ∈ Υ and consider the PDMP X µ starting at (v, d, 0, h, v) ∈ Ξ µ .We drop the dependence in the control in the notation and denote by (T n ) n∈N the jump times, and Tn .Let H n = (Z 0 , . . ., Z n ), T n ≤ T .For a purpose of concision we will rewrite all quantities being non-negative.We want now to examine the two terms that we call I 1 and I 2 separately.For n ∈ N, we start with

the corresponding term vanishes).
Then By the strong Markov property and the flow property, the first term on the RHS is equal to Using the same arguments, the second term on the RHS of I 1 can be written as An integration by parts yields Moreover By definition of the Markov chain (Z ′ n ) n∈N and the function c ′ , we then obtain for the total expected cost of the PDMP,

Existence of optimal controls for the MDP
We now show existence of optimal relaxed controls under a contraction assumption.We use the notation R := R([0, T ]; U ) in the sequel.Let us also recall some notations regarding the different control sets we consider.
• u is an element of the control set U .
• a : [0, T ] → U is an element of the space of admissible control rules U ad ([0, T ], U ) ) is an element of the space of admissible strategies for the original PDMP.
• γ : [0, T ] → M 1 + (Z) is an element of the space of relaxed admissible control rules R.
• µ : Υ → R is an element of the space of relaxed admissible strategies for the relaxed PDMP.
The classical way to address the discrete-time stochastic control problem that we introduced in Definition 3.3 is to consider an additional control space that we will call the space of Markovian policies and denote by Π. Formally Π := A R N and a Markovian control policy for the MDP is a sequence of relaxed admissible strategies to be applied at each stage.The optimal control problem is to find π := (µ n ) n∈N ∈ Π that minimizes Now denote by J * (z) this infimum.We will in fact prove the existence of a stationary optimal control policy that will validate the equality Let us now define some operators that will be useful for our study and state the first theorem of this section.Let w : Υ → R a continuous function, (z, γ, µ) ∈ Υ × R × A R and define where , γ) which admits also the expression Theorem 3.3.Assume that there exists a subspace C of the space of continuous bounded functions from Υ to R such that the operator T : C → C is contracting and the zero function belongs to C. Assume furthermore that C is a Banach space.Then J ′ is the unique fixed point of T and there exists an optimal control µ * ∈ A R such that All the results needed to prove this Theorem can be found in [6].We break down the proof into the two following elementary propositions, suited to our specific problem.Before that, recall that from [6, Proposition 9.1 p.216], Π is the adequate control space to consider since history-dependent policies does not improve the value function.
Let us now consider the n-stages expected cost function and value function defined by for n ∈ N and π := (µ n ) n∈N ∈ Π.We also set J ∞ := lim We then get Since this last inequality is true for any ε > 0 we get

Proposition 3.2. There exists µ
Proof.By definition, for every π ∈ Π, J n ≤ J nπ , so that J ∞ ≤ J * .Now from the previous proposition, R is a compact space and LJ ∞ is a continuous function.We can thus find a measurable mapping µ * : Υ → R such that J ∞ = T µ * J ∞ .J ∞ ≥ 0 so from the first relation of the previous proposition, for all n ∈ N, J ∞ = T n µ * J ∞ ≥ T n µ * 0 and by taking the limit J ∞ ≥ J µ * .Since J µ * ≥ J * we get J ∞ = J µ * = J * .We conclude the proof by remarking that J * ≤ J ′ ≤ J µ * .
The next section is devoted to proving that the assumptions of Theorem 3.3 are satisfied for the MDP.

Bounding functions and contracting MDP
The concept of bounding function that we define below will ensure that the operator T is a contraction.The existence of the space C of Theorem 3.3 will mostly result from Theorem 2.1 and again from the concept of bounding function.d, h).Given a bounding function for the PDMP we can construct one for the MDP with or without relaxed controls, as shown in the next lemma (cf.[13, Definition 7.1.2p.195]).
Proof.Take (z, γ) ∈ Υ × R , z = (v, d, h).On the one hand from (3.6) and Definition 3.4 we obtain which immediately implies (3.7).On the other hand .Suppose now that it holds for k ∈ N. Then .
Using (3.8) and the definition of C, we conclude that Lemma 3.3.There exists κ > 0 such that for any (z, µ) Proof.The results follows from Lemma 3.2 and from the fact that We now state the result on the operator T .
Lemma 3.4.T is a contraction on L * : for any (v, w) Proof.We prove here the contraction property.The fact T : L * → L * is less straightforward and is addressed in the next section.Let z := (v, d, h) ∈ Υ.Let us recall that for functions f, g : R → R where

Continuity properties
Here we prove that the trajectories of the relaxed PDMP are continuous w.r.t. the control and that the operator R transforms continuous functions in continuous functions.
Lemma 3.5.Assume that (H(L)) and (H(f )) are satisfied.Then the mapping Proof.This proof is based on the result of Theorem 2.1.Here we add the joined continuity on Υ × R whereas the continuity is just on R in [41].Let t ∈ [0, T ] and let (z, γ) ∈ Υ × R. Assume that (z n , γ n ) → (z, γ).Since D is a finite set, we take the discrete topology on it and if we denote by z n = (v n , d n , h n ) and z = (v, d, h), we have the equality d n = d for n large enough.So for n large enough we have From(H(f ))1.we get By the Gronwall lemma we obtain a constant C > 0 such that Since lim n→+∞ ||v n − v|| H = 0, the proof is complete if we show that the sequence of functions Let us denote by x n (t) : Using the same argument as the proof of [41, Theorem 3.1], there is no difficulty in proving that (x n ) n∈N is compact in C([0, T ], H) so that, passing to a subsequence if necessary, we may assume that The next lemma establishes the continuity property of the operator R.
Proof.See Appendix C.
It now remains to show that there exists a bounding function for the PDMP.This is the result of the next lemma.
is a continuous bounding function for the PDMP.
Proof.For all (v, u) Remark 3.4.Lemma 3.7 ensures the existence of a bounding function for the PDMP.To broaden the class of cost functions considered, we could just assume the existence of a bounding for the PDMP in Theorem 3.1 and then, the assumption on c and g should just be the continuity.

Existence of an optimal ordinary strategy
Ordinary strategies are of crucial importance because they are the ones that the controller can implement in practice.Here we give convexity assumptions that ensure the existence of an ordinary optimal control strategy for the PDMP.Theorem 3.4.Suppose that assumptions (H(L)), (H(f )), (H(λ)), (H(Q)), (H(c)) and (A) are satisfied.If we consider µ * ∈ A R an optimal relaxed strategy for the PDMP, then the ordinary Proof.This result is based on the fact that for all (z, γ) ∈ Υ × R, (Lw)(z, γ) ≥ (Lw)(z, γ), with γ = Z uγ(du).Indeed, the fact that the function f d is linear in the control variable implies that for all (t, z, γ) The convexity assumptions (A) give the following inequalities , so that in particular χ γ t (z) ≥ χ γ t (z).We can now denote for all (z, γ) ∈ Υ × R and w : Υ → R + , Furthermore,

An elementary example
Here we treat an elementary example that satisfies the assumptions made in the previous two sections.
We consider the following PDE for the deterministic evolution between jumps with Dirichlet boundary conditions.We define the jump rate function for (v, u) ∈ H × U by and the transition measure by is a reference signal that we want to approach.
Lemma 3.8.The PDMP defined above admits the continuous bounding function Furthermore, the value function of the optimal control problem is continuous and there exists an optimal ordinary control strategy.
Proof.The proof consists in verifying that all assumptions of Theorem 3.4 are satisfied.Assumptions (H(Q)), (H(c)) and (A) are straightforward.
The continuity in the variable u is straightforward and the locally Lipschitz continuity comes from the fact that the functions v → 1/(e −||v|| 2 + 1), and v → e −β(v) , with and so This means that for every z 4 Application to the model in Optogenetics We begin this section by making some comments on Definition 0.1.In (0.1), C m > 0 is the membrane capacitance and V − and V + are constants defined by They represent the physiological domain of our process.In (0.2), the constants g x > 0 are the normalized conductances of the channels of type x and V x ∈ R are the driving potentials of the channels.The constant ρ > 0 is the relative conductance between the open states of the ChR2.For a matter of coherence with the theoretical framework presented in the paper, we will prove Theorem 0.1 for the mollification of the model that we define now.This model is very close to the one of Definition 0.1.It is obtained by replacing the Dirac masses δ z by their mollifications ξ N z that are defined as follows.Let ϕ be the function defined on R by For all z ∈ I N , ξ N z ∈ C ∞ ([0, 1]) and ξ N z → δ z almost everywhere in [0, 1] as N → +∞, so that (ξ N z , φ) H → φ(z), as N → ∞ for every φ ∈ C(I, R).The expressions v(i/N ) in Definition 0.1 are also replaced by (ξ N i/N , v) H .The decision to use the mollified Dirac mass over the Dirac mass can be motivated by two main reasons.First of all, as mentioned in [10], the concentration of ions is homogeneous in a spatially extended domain around an open channel so the current is modeled as being present not only at the point of a channel, but in a neighborhood of it.Second, the smooth mollified Dirac mass leads to smooth solutions of the PDE and we need at least continuity of the flow.Nevertheless, the results of Theorem 0.1 remain valid with the Dirac masses and we refer the reader to Section 4.2.The following lemma is a direct consequence of [10, Proposition 7] and will be very important for the model to fall within the theoretical framework of the previous sections.Lemma 4.1.For every y 0 ∈ V with y 0 (x) ∈ [V − , V + ] for all x ∈ I, the solution y of (0.1) is such that for t ∈ [0, T ], Physiologically speaking, we are only interested in the domain [V − , V + ].Since Lemma 4.1 shows that this domain is invariant for the controlled PDMP, we can modify the characteristics of the PDMP outside the domain [V − , V + ] without changing its dynamics.We will do so for the rate functions σ x,y of Table 1.From now on, consider a compact set K containing the closed ball of H, centered in zero and with radius max(V − , V + ).We will rewrite σ x,y the quantities modified outside K such that they all become bounded functions.This modification will enable assumption (H(λ))1.to be verified.The next lemma shows that the stochastic controlled infinite-dimensional Hodgkin-Huxley-ChR2 model defines a controlled infinite-dimensional PDMP as defined in Definition 1.2 and that Theorem 1.2 applies.Lemma 4.2.For N ∈ N * , the N th stochastic controlled infinite-dimensional Hodgkin-Huxley-ChR2 model satisfies assumptions (H(λ)), (H(Q)), (H(L)) and (H(f )).Moreover, for any control strategy α ∈ A, the membrane potential v α satisfies Proof.The local Lipschitz continuity of λ d from H × Z in R + comes from the local Lipschitz continuity of all the functions σ x,y of Table 1.2 and the inequality . By Lemma 4.1, the modified jump rates are bounded.Since they are positive, they are bounded away from zero, and then, Assumption (H(λ)) is satisfied.Assumption (H(Q)) is also easily satisfied.We showed in Section 3.4 that (H(L)) is satisfied.As for f d , the function does not depend on the control variable and is continuous from .
We then get Finally, since the continuous component v α t of the PDMP does not jump, the bounds are a direct consequence of Lemma 4.1.
Proof of Theorem 0.1.In Lemma 4.2 we already showed that assumptions (H(λ)), (H(Q)), (H(L)) and (H(f )) are satisfied.The cost function c is convex in the control variable and norm quadratic on H × Z.The flow does not depend on the control.The rate function λ is linear in the control.the function λQ is also linear in the control.We conclude that all the assumptions of Theorem 3.1 are satisfied and that an optimal ordinary strategy can be retrieved.
We end this section with an important remark that significantly extends the scope of this example.Up to now, we only considered stationary reference signals but nonautonomous ones can be studied as well, as long as they feature some properties.Indeed, it is only a matter of incorporating the signal reference V ref ∈ C([0, T ], H) in the process by adding a variable to the PDMP.Instead of considering H as the initial state space for the continuous component, we consider H := H × H.This way, the part on the control problem is not impacted at all and we consider the continuous cost function c defined for the result and proof of lemma 0.1 remaining unchanged with the continuous bounding function In the next section, we present some variants of the model and the corresponding results in terms of optimal control.
Table 1: Expression of the individual jump rate functions.

Variants of the model
We begin this section by giving arguments showing that the results of Theorem 3.1 remain valid for the model of Definition 0.1, which does not exactly fits into our theoretical framework.Then, the variations we present concern the model of ChR2, the addition of other light-sensitive ionic channels, the way the control acts on the three local characteristics and the control space.The optimal control problem itself will remain unchanged.First of all, let us mention that since the model of Definition 0.1 satisfies the convexity conditions (A), the theoretical part on relaxed controls is not necessary for this model.Nevertheless, the model of ChR2 presented on Figure 1 is only one among several others, some of which do not enjoy a linear, or even concave, rate function λ.For those models, that we present next, assumption (A) fails and the relaxed controls are essential.We will not present them here, but the previous results for the Hodgkin-Huxley model remain straightforwardly unchanged for other neuron models such as the FitzHugh-Nagumo model or the Morris-Lecar model.

Optimal control for the original model
In the original model, the function f d is defined from V to V * .Nevertheless, the semigroup of the Laplacian regularizes Dirac masses (see [4,Lemma 3.1]) and the uniform bound in Theorem 1.2 is in fact valid in V , the solution belonging to C([0, T ], V ).This is all we need since the control does not act on the PDE.This is why the domain of our process is V × D N and not just H × D N , and all computations of the proofs of the previous sections can be done in the Hilbert space V .From this consideration, and using the continuous embedding of H 1 0 (I) in C 0 (I) we can justify the local Lipschitz continuity of λ d from V × Z in R + .Indeed, it comes from the local Lipschitz continuity of all functions σ x,y of Table 1 and from the inequality Finally, [10, Proposition 5] states that the bounds of Lemma 4.2 remain valid with Dirac masses.

Modifications of the ChR2 model
We already mentioned the paper of Nikolic and al. [37] in which a three states model is presented.
It is a somehow simpler model that the four states model of Figure 1 but it gives good qualitative results on the photocurrents produced by the ChR2.In first approximation the model can be considered to depend linearly in the control as seen on Figure 2.
The first limit is not physical since rate jumps between states are positive numbers.The second limit is not physical either because it would mean that, in the dark, the proteins are trapped in the open state o, which is not the case.In the dark, when u = 0, the jump rates corresponding to the transition o → d and d → c are positive constants.For this reason, the functions σ o,d and σ d,c should be smooth functions such that they are equal to the rates of Figure 3 for large intensities, > 0 respectively, when u goes to 0. The resulting rate function λ is not concave and thus does not satisfy assumption (A) anymore.We can only affirm the existence of optimal relaxed strategies.The four states model of Figure 1 is also an approximation of a more accurate model that we represent on Figure 4 below.The transition rates can depend on either the membrane potential v or the irradiance u, which is the control variable.The details of the model and the numerical constants can be found in [49].Note that the model of Figure 4 is already an approximation of the model in [49] because the full model in [49] would not lead to a Markovian behavior for the ChR2 (the transition rates would depend on the time elapsed since the light was switched on).
d1 − K d1 , K d1 , e 12d , e 21d , c, c 1 and c 2 positive constants.As for the model of Figure 3, the mathematical definition of the function σ o1,c1 should be such that it is a positive smooth function and equals K d1 (v) in some subset of the physiological domain [V − , V + ].The resulting rate function λ will be concave but the function λQ will not be convex (it will be concave as well).Hence, Assumption (A) is not satisfied.

Addition of other light-sensitive ion channels
Channelrhodopsin-2 has a promoting role in eliciting action potentials.There also exists a chlorine pump, called Halorhodopsin (NpHR), that has an inhibitory action.NpHR can be used along with ChR2 to obtain a control in both directions.Its modelisation as a multistate model was considered in [38].The transition rates between the different states have the same shape that the ones of the ChR2 and the same simplifications are possible.This new light-sensitive channel can be easily incorporated in our stochastic model and we can state existence of optimal relaxed and/or ordinary control strategies depending on the level of complexity of the NpHR model we consider.It is here important to remark that since the two ionic channels do not react to the same wavelength of the light, the resulting control variable would be two-dimensional with values in [0, u max ] 2 .This would not change the qualitative results of the previous sections.

Modification of the way the control acts on the local characteristics
Up to now, the control acts only on the rate function, and also on the measure transition via its special definition from the rate function.Nevertheless, we can present here a modification of the model where the control acts linearly on the PDE.This modification amounts to considering that the control variable is directly the gating variable of the ChR2.Indeed, we show in [42] that the optimal control of the deterministic counterpart of the stochastic Hodgkin-Huxley-ChR2 model, in finite dimension and with the three states ChR2 model of Figure 2, is closely linked to the optimal control of where the control variable is the former gating variable o.Now the stochastic counterpart of the last model is such that the function f d is now linear in the control and the rate function λ and the transition measure function Q do not depend on the control any more.Finally, by adding NpHR channels to this model, we would obtain a fully controlled infinite-dimensional PDMP in the sense that the control would then act on the three local characteristics of the PDMP.Depending on the model of NpHR chosen, we would obtain relaxed or ordinary optimal control strategy.

Modification of the control space
In all models discussed previously, the control has no spatial dependence.Any light-stimulation device, such as a laser, has a spatial resolution and it is possible that we do not want or cannot stimulate the entire axon.For this reason, spatial dependence of the control should be considered.Now, as long as the control space remains a compact Polish space, spatial dependence of the control could be considered.We propose here a control space defined as a subspace of the Skorohod space D, constituted of the càdlàg functions from [0, 1] to R. This control space represents the aggregation of multiple laser beams that can be switched on and off.Suppose that each of theses beams produces on the axon a disc of light of diameter r > 0 that we call spatial resolution of the light.For an axon represented by the segment [0, 1], r is exactly the spatial domain illuminated.
We consider now two possibilities for the control space.Suppose first that the spatial resolution is fixed and define p := ⌊ 1 r ⌋ and Proof.We tackle this proof by remarking that U is in bijection with the finite dimensional compact space [0, u max ] p .
In this case, the introduction of the space D was quite artificial since the control space remains finite-dimensional.Nevertheless, the Skorohod space will be very useful for the other control space.Suppose now that the spatial resolution of the laser can evolve in [r min , r max ] with r min , r max > 0.
Let p ∈ N * the number of lasers used and define where γ 0 ∈ Υ is the initial position, and for i ∈ N * , s i is the time elapsed between the (i − 1) th and the i th jump while γ i is the location right after the i th jump.
i∈N * be a sequence of copies of the space Y Υ .We define Ω Υ := Υ × Π ∞ i=1 Y Υ i the canonical space of jump processes with values in Υ, endowed with its Borel σ-algebra F Υ and the coordinate mappings on Ω Υ as follows We also introduce ω i : Ω Υ → Ω Υ i for i ∈ N * , defined by for ω ∈ Ω Υ .Now for ω ∈ Ω Υ and i ∈ N * , let and the sample path (x t (ω)) t≥0 be defined by A relevant filtration for our problem is the natural filtration of the coordinate process (x t ) t≥0 on Ω Υ F Υ t := σ{x s |s ≤ t}, for all t ∈ R + .For given starting point γ 0 ∈ Υ and control strategy α ∈ A, a controlled probability measure, denoted P α γ0 , is defined on Ω Υ by the specification of a family of controlled conditional distribution functions as follows: µ 1 is a controlled probability measure on (Y Υ , B(Y Υ )) or equivalently a measurable mapping from U ad ([0, T ]; U ) to the set of probability measures on (Y Υ , B(Y Υ )), such that for all α ∈ A, We need to extend the definition of α ∈ A to the state (∞, ∆) by setting α(∆) := u ∆ where u ∆ is itself an isolated cemetery state and α takes in fact values in U ad ([0, T ]; U ∪ {u ∆ }).Now for a given control strategy α ∈ A, P α γ0 is the unique probability measure on (Ω Υ , T Υ ) such that for each i ∈ N * and bounded function f on Ω Υ  with α depending only on the variable in Υ when writing "α(y i−1 )" , y i−1 = (s i−1 , γ i−1 ).Let's now denote by F Υ γ,α and (F Υ,γ,α t ) t≥0 the completed σ-fields of F Υ and (F Υ t ) t≥0 with all the P α γnull sets of F Υ .We then rename the intersection of these σ-fields redefine F Υ and (F Υ t ) t≥0 so that we have t for all t ≥ 0.
Part 2. The canonical space of càdlàg functions with values in Ξ.Let Ω Ξ be the set of right-continuous functions with left limits (càdlàg functions), defined on R + with values in Ξ.
Analogously to what we have done in Part 1, we can construct a filtered space (Ω Ξ , F Ξ , (F Ξ t ) t≥0 ) with coordinate process (x Ξ t ) t≥0 and a probability P α on (Ω Ξ , F Ξ ) for every control strategy α ∈ A such that the infinite-dimensional PDMP is a P α -strong Markov process.For (t, y) ∈ R + × Ω Ξ , x Ξ t (y) = y(t).We start with the definition of F Ξ,0 t := σ{x Ξ s |s ≤ t} for t ∈ R + and F Ξ,0 := ∨ t≥0 F Ξ,0 t .In Davis [19] p 59, the construction of the PDMP is conducted on the Hilbert cube, the space of sequences of independent and uniformly distributed random variables in [0, 1].In the case of controlled PDMP, the survival function F (t, x) in [19] is replaced by the extension to ξ α of χ α defined in Definition 1.2 and the construction depends on the chosen control.This extension is defined for x := (v, d, τ, h, ν) ∈ Ξ α by χ α t (x) := exp − t 0 λ d (φ α s (x), α τ +s (ν, d, h))ds , such that for z := (v, d, h) ∈ Υ, χ α t (z) = χ α t (v, d, 0, h, v).This procedure thus provides for each control α ∈ A and starting point x ∈ Ξ α a measurable mapping ψ α x from the Hilbert cube to Ω Ξ .Let P α x := P (ψ α x ) −1 denote the image measure of the Hilbert cube probability P under ψ α x .Now for x ∈ Ξ α , let F x,α t be the completion of F Ξ,0 t with all P α x -null sets of F Ξ,0 , and define (B.6) The right-continuity of (F Ξ t ) t≥0 follows from the right-continuity of (F Υ t ) t≥0 and the one-to-one correspondence.The right-continuity of (F Υ t ) t≥0 is a classical result on right-constant processes.For theses reasons, we lose the superscripts Ξ and Υ consider the natural filtration (F t ) t≥0 in the sequel.
• Let f be continuously differentiable w.r.t.v ∈ V and τ ∈ R + .Define h v as the unique element of V * such that

Theorem 2 . 1 .
If assumptions (H(L)) and (H(f )) (or (H(f ))') hold, then a) the space of relaxed trajectories (i.e.solutions of 2.2) is a convex, compact set of C([0, T ], H).It is the closure in C([0, T ], H) of the space of original trajectories (i.e.solutions of 2.1).

π∈ΠT
µ0 T µ1 . . .T µn−1 I, and by definition of T , T I ≤ T µn−1 I. Using the first relation of the proposition we get T n I ≤ T µ0 T µ1 . . .T µn−1 I. Finally, T n I = inf π∈Π T µ0 T µ1 . . .T µn−1 I for all I ∈ C and n ∈ N. We deduce from the Banach fixed point theorem that J ∞ = lim n→∞ T n 0 belongs to C and is the only fixed point of T .

Definition 3 . 4 .
Bounding functions for a PDMP.Let c (resp.g) be a running (resp.terminal) cost as in Section 3.1.A measurable function b : H → R + is called a bounding function for the PDMP if there exist constants

Lemma 3 . 1 .
Let b is a bounding function for the PDMP.We keep the notations of Definition 3.4.Let ζ > 0. The function B ζ : Υ −→ R + defined by B ζ (z) := b(v)e ζ(T −h) for z = (v, d, h) is an upper bounding function for the MDP.The two inequalities below are satisfied for all (z, γ) ∈ Υ×R,

Lemma 3 . 7 .
Suppose assumptions (H(L)), (H(f )) and (H(c)) are satisfied.Now define c and g from c and g by taking the absolute value of the coefficients of these quadratic functions.Let M 2 > 0. Define M 3 := (M 2 + b 1 T )M S e MSb2T and b : H → R + by

(
A) (a) For all d ∈ D, the function f d : (y, u) ∈ H × U → E is linear in the control variable u.(b) For all d ∈ D, the functions λ d : (y, u) ∈ H × U → R + and λ d Q : (y, u) ∈ H × U → λ d (y, u)Q(•|y, d, u) are respectively concave and convexe in the control variable u.(c) The cost function c : (y, u) ∈ E × U → R + is convex in the control variable u.

Figure 2 :
Figure 2: Simplified ChR2 three states model This model features one open state o and two closed states, one light-adapted d and one darkadapted c.This model would lead to the same type of model as in the previous Section.In fact, the time constants 1/K d and 1/K r are also light dependent with a dependence in log(u).The corresponding model is represented on Figure 3 below

Figure 3 :
Figure 3: ChR2 three states model Some mathematical comments are needed here.On Figure 3, the control u represents the light intensity and c 1 , c 2 , K r and τ d are positive constants.This model of ChR2 is experimentally accurate for intensities between 10 8 and 10 10 µm 2 • s −1 approximately.We would then consider U := [0, u max ] with u max ≃ 10 10 µm 2 • s −1 .Furthermore,
c 2 > 0; 4. −L generates a strongly continuous semigroup (S(t)) t≥0 on H such that S(t) : H → H is compact for every t > 0. We will denote by M S a bound, for the operator norm, of the semigroup on [0, T ].
where H w denotes the space H endowed with the topology of weak convergence.
The jump times (t i ) i∈N are deduced from the sequence (s i ) i∈N * by t 0 = 0 and t i = t i−1 + s i for i ∈ N * and the jump process (J t ) t≥0 is given by J t := γ i for t ∈ [t i , t i+1 ) and J t = ∆ for t ≥ t ∞ := lim