Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter October 21, 2021

A fully backward representation of semilinear PDEs applied to the control of thermostatic loads in power systems

  • Lucas Izydorczyk , Nadia Oudjane and Francesco Russo EMAIL logo

Abstract

We propose a fully backward representation of semilinear PDEs with application to stochastic control. Based on this, we develop a fully backward Monte-Carlo scheme allowing to generate the regression grid, backwardly in time, as the value function is computed. This offers two key advantages in terms of computational efficiency and memory. First, the grid is generated adaptively in the areas of interest, and second, there is no need to store the entire grid. The performances of this technique are compared in simulations to the traditional Monte-Carlo forward-backward approach on a control problem of thermostatic loads.

Funding statement: The work was supported by a public grant as part of the Investissement d’avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH, in a joint call with Gaspard Monge Program for optimization, operations research and their interactions with data sciences.

A Appendix

A.1 A sufficient condition to obtain an equivalent probability

Lemma A.1.

We recall that b ~ was defined in (3.10). Let W be an ( F t ) t [ 0 , T ] -Brownian motion and let X be a solution of

X t = X 0 + 0 t b ~ ( s , X s ) 𝑑 s + 0 t σ ( s ) 𝑑 W s , t [ 0 , T ] ,

where X 0 is a Gaussian random vector independent of W. Set L t := σ ( t ) - 1 b ~ ( t , X t ) for all t [ 0 , T ] . Then the Doléans exponential

( - i = 1 d 0 L s i 𝑑 W s i ) := exp ( - 0 i = 1 d L s i d W s i - 1 2 0 | L s | 2 𝑑 s )

is an ( F t ) t [ 0 , T ] -martingale.

Proof.

By following [25, Corollary 5.14], it is sufficient to find a constant time step subdivision ( t n ) n of [ 0 , T ] such that, for all n ,

𝔼 ( exp ( 1 2 t n t n + 1 | L s | 2 𝑑 s ) ) < .

By combining Jensen’s inequality and Fubini’s theorem, this is fulfilled in particular if for all n ,

1 δ t n t n + 1 𝔼 ( exp ( δ | L s | 2 2 ) ) 𝑑 s < ,

where δ := t n + 1 - t n . Let s [ 0 , T ] . Then

| L s | 2 2 δ σ - 1 2 ( a 2 | X s | 2 + c 2 ) -a.s.,

since a , c are bounded and σ - 1 is also bounded being continuous on [ 0 , T ] . Furthermore, by Lemma 3.8 (i) and (3.14), X is a Gaussian process with mean function m X (resp. covariance function Q X ) solving the first line of equation (3.1) (resp. (3.2)) with initial condition 𝔼 ( X 0 ) (resp. Cov ( X 0 ) ).

By taking into account the fact that m X is bounded (since it is continuous), it suffices to find a subdivision such that

𝔼 ( exp ( 1 2 K δ | Z | 2 ) ) < ,

where Z 𝒩 ( 0 , I d ) and K := 4 σ - 1 2 a 2 Q X > 0 . This is the case in particular if K δ < 1 , which ends the proof. ∎

A.2 Proof of the local Lipschitz property of the cost functional J

Lemma A.2.

Suppose the validity of Assumption 4.1. Suppose in addition that the functions g and

x f ( t , x , α ) , ( t , α ) [ 0 , T ] × 𝒜 0 ,

are locally Lipschitz with polynomial growth gradient (uniformly in t and α). Then, for each ( t , α ) [ 0 , T ] × A 0 ,

x J ( t , x , α )

is locally Lipschitz, uniformly in t and α.

Proof.

We give here a proof of the local Lipschitz property for the term involving the function g since the other term can be treated in the same way.

Let ( t , α ) [ 0 , T ] × 𝒜 0 and let x , y be in a compact set of d . Let K be the Lipschitz constant of b. Using in particular the Cauchy–Schwarz inequality, we get

| 𝔼 ( g ( X T t , x , α ) - g ( X T t , y , α ) ) | 0 1 𝔼 ( | g ( a X T t , x , α + ( 1 - a ) X T t , y , α ) | | X T t , x , α - X T t , y , α | ) 𝑑 a
(A.1) e K T 0 1 𝔼 ( | x g ( a X T t , x , α + ( 1 - a ) X T t , y , α ) | ) 𝑑 a | x - y | ,

where we have used the estimate | X T t , x , α - X T t , y , α | e K T | x - y | , following from the identity

| X r t , x , α - X r t , y , α | | x - y | + K t r | X s t , x , α - X s t , y , α | 𝑑 s , r [ t , T ] ,

together with Gronwall’s lemma. In view of (A.1), the point is proved if

0 1 𝔼 ( | x g ( a X T t , x , α + ( 1 - a ) X T t , y , α ) | ) 𝑑 a

is bounded uniformly in t , x , y , α . This follows from polynomial growth of x g , classical moment estimates for sup s [ t , T ] | X s t , z , α | , z d (see for example [26, Corollary 2.5.12]), and the fact x , y lie in a compact set. ∎

A.3 A simplified version of the envelope theorem

Lemma A.3.

Let Λ be an arbitrary set and let O be an open subset of R d . Let x R d . Let F : O × Λ R such that, for all λ Λ , F ( , λ ) and V : x sup λ Λ F ( x , λ ) are differentiable at the point x. Suppose also that Λ * ( x ) = { λ Λ : V ( x ) = F ( x , λ ) } is not empty. Then

x V ( x ) = x F ( x , λ x * )

for every λ x * Λ * ( x ) .

Proof.

Let x be as in the proposition statement and let h d . Let λ x * Λ * ( x ) . Then, using in particular the differentiability of F ( , λ x * ) at the point x, we get

(A.2) V ( x + h ) - V ( x ) F ( x + h , λ x * ) - F ( x , λ x * ) = x F ( x , λ x * ) , h + o 0 ( | h | ) .

By the differentiability of V at the point x, (A.2) implies

(A.3) x V ( x ) - x F ( x , λ x * ) , h o 0 ( | h | ) .

Setting h to - h in (A.2) and proceeding as before, we obtain

(A.4) x V ( x ) - x F ( x , λ x * ) , h o 0 ( | h | ) .

Combining (A.3) and (A.4), we get

x V ( x ) - x F ( x , λ x * ) , h | h | h 0 0 ,

which forces x V ( x ) = x F ( x , λ x * ) . This ends the proof. ∎

References

[1] C. Bender and R. Denk, A forward scheme for backward SDEs, Stochastic Process. Appl. 117 (2007), no. 12, 1793–1812. 10.1016/j.spa.2007.03.005Search in Google Scholar

[2] C. Bender and T. Moseler, Importance sampling for backward SDEs, Stoch. Anal. Appl. 28 (2010), no. 2, 226–253. 10.1080/07362990903546405Search in Google Scholar

[3] C. Bender and J. Steiner, Least-squares Monte Carlo for backward SDEs, Numerical Methods in Finance, Springer Proc. Math. 12, Springer, Heidelberg (2012), 257–289. 10.1007/978-3-642-25746-9_8Search in Google Scholar

[4] B. Bouchard and N. Touzi, Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations, Stochastic Process. Appl. 111 (2004), no. 2, 175–206. 10.1016/j.spa.2004.01.001Search in Google Scholar

[5] R. Bronson and G. B. Costa, Matrix Methods: Applied Linear Algebra, Academic Press, New York, 2008. Search in Google Scholar

[6] D. S. Callaway and I. A. Hiskens, Achieving controllability of electric loads, Proc. IEEE 99 (2010), no. 1, 184–199. 10.1109/JPROC.2010.2081652Search in Google Scholar

[7] M. G. Crandall, H. Ishii and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N. S.) 27 (1992), no. 1, 1–67. 10.1090/S0273-0979-1992-00266-5Search in Google Scholar

[8] F. Delarue and S. Menozzi, An interpolated stochastic algorithm for quasi-linear PDEs, Math. Comp. 77 (2008), no. 261, 125–158. 10.1090/S0025-5718-07-02008-XSearch in Google Scholar

[9] C. Di Girolami and F. Russo, About classical solutions of the path-dependent heat equation, Random Oper. Stoch. Equ. 28 (2020), no. 1, 35–62. 10.1515/rose-2020-2028Search in Google Scholar

[10] E. Gobet and M. Grangereau, Federated stochastic control of numerous heterogeneous energy storage systems, preprint (2021), https://hal.archives-ouvertes.fr/hal-03108611. Search in Google Scholar

[11] I. Exarchos and E. A. Theodorou, Stochastic optimal control via forward and backward stochastic differential equations and importance sampling, Automatica J. IFAC 87 (2018), 159–165. 10.1016/j.automatica.2017.09.004Search in Google Scholar

[12] G. Fabbri, F. Gozzi and A. Świech, Stochastic Optimal Control in Infinite Dimension, Probab. Theory Stoch. Model. 82, Springer, Cham, 2017. 10.1007/978-3-319-53067-3Search in Google Scholar

[13] E. Gobet and C. Labart, Error expansion for the discretization of backward stochastic differential equations, Stochastic Process. Appl. 117 (2007), no. 7, 803–829. 10.1016/j.spa.2006.10.007Search in Google Scholar

[14] E. Gobet and C. Labart, Solving BSDE with adaptive control variate, SIAM J. Numer. Anal. 48 (2010), no. 1, 257–277. 10.1137/090755060Search in Google Scholar

[15] E. Gobet, J.-P. Lemor and X. Warin, A regression-based Monte Carlo method to solve backward stochastic differential equations, Ann. Appl. Probab. 15 (2005), no. 3, 2172–2202. 10.1214/105051605000000412Search in Google Scholar

[16] E. Gobet and P. Turkedjiev, Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions, Math. Comp. 85 (2016), no. 299, 1359–1391. 10.1090/mcom/3013Search in Google Scholar

[17] E. Gobet and P. Turkedjiev, Adaptive importance sampling in least-squares Monte Carlo algorithms for backward stochastic differential equations, Stochastic Process. Appl. 127 (2017), no. 4, 1171–1203. 10.1016/j.spa.2016.07.011Search in Google Scholar

[18] F. Gozzi and F. Russo, Verification theorems for stochastic optimal control problems via a time dependent Fukushima–Dirichlet decomposition, Stochastic Process. Appl. 116 (2006), no. 11, 1530–1562. 10.1016/j.spa.2006.04.008Search in Google Scholar

[19] F. Gozzi and F. Russo, Weak Dirichlet processes with a stochastic control perspective, Stochastic Process. Appl. 116 (2006), no. 11, 1563–1583. 10.1016/j.spa.2006.04.009Search in Google Scholar

[20] U. G. Haussmann and E. Pardoux, Time reversal of diffusions, Ann. Probab. 14 (1986), no. 4, 1188–1205. 10.1214/aop/1176992362Search in Google Scholar

[21] H. Ishii, On uniqueness and existence of viscosity solutions of fully nonlinear second-order elliptic PDEs, Comm. Pure Appl. Math. 42 (1989), no. 1, 15–45. 10.1002/cpa.3160420103Search in Google Scholar

[22] H. Ishii and K. Kobayasi, On the uniqueness and existence of solutions of fully nonlinear parabolic PDEs under the Osgood type condition, Differential Integral Equations 7 (1994), no. 3–4, 909–920. 10.57262/die/1370267713Search in Google Scholar

[23] L. Izydorczyk, N. Oudjane, F. Russo and G. Tessitore, Fokker–Planck equations with terminal condition and related McKean probabilistic representation, preprint (2020), https://hal.archives-ouvertes.fr/hal-02902615. 10.1007/s00030-021-00736-1Search in Google Scholar

[24] R. Jensen, P.-L. Lions and P. E. Souganidis, A uniqueness result for viscosity solutions of second order fully nonlinear partial differential equations, Proc. Amer. Math. Soc. 102 (1988), no. 4, 975–978. 10.1090/S0002-9939-1988-0934877-2Search in Google Scholar

[25] I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed., Grad. Texts in Math. 113, Springer, New York, 1991. Search in Google Scholar

[26] N. V. Krylov, Controlled Diffusion Processes, Stochastic Model. Appl. Probab. 14, Springer, Berlin, 2009. Search in Google Scholar

[27] C. Labart and J. Lelong, A parallel algorithm for solving BSDEs, Monte Carlo Methods Appl. 19 (2013), no. 1, 11–39. 10.1515/mcma-2013-0001Search in Google Scholar

[28] P.-L. Lions, Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness, Comm. Partial Differential Equations 8 (1983), no. 11, 1229–1276. 10.1080/03605308308820301Search in Google Scholar

[29] D. Nunziante, Existence and uniqueness of unbounded viscosity solutions of parabolic equations with discontinuous time-dependence, Nonlinear Anal. 18 (1992), no. 11, 1033–1062. 10.1016/0362-546X(92)90194-JSearch in Google Scholar

[30] E. Pardoux, Backward stochastic differential equations and viscosity solutions of systems of semilinear parabolic and elliptic PDEs of second order, Stochastic Analysis and Related Topics. VI (Geilo 1996), Progr. Probab. 42, Birkhäuser, Boston (1998), 79–127. 10.1007/978-1-4612-2022-0_2Search in Google Scholar

[31] E. Pardoux, F. Pradeilles and Z. Rao, Probabilistic interpretation of a system of semi-linear parabolic partial differential equations, Ann. Inst. Henri Poincaré Probab. Stat. 33 (1997), no. 4, 467–490. 10.1016/S0246-0203(97)80101-XSearch in Google Scholar

[32] H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Model. Appl. Probab. 61, Springer, Berlin, 2009. 10.1007/978-3-540-89500-8Search in Google Scholar

[33] D. Revuz and M. Yor, Continuous Martingales and Brownian Motion, 3rd ed., Grundlehren Math. Wiss. 293, Springer, Berlin, 1999. 10.1007/978-3-662-06400-9Search in Google Scholar

[34] C. Ribeiro and N. Webber, Valuing path-dependent options in the variance-gamma model by Monte Carlo with a gamma bridge, J. Comput. Finance 7 (2004), no. 2, 81–100. 10.21314/JCF.2003.110Search in Google Scholar

[35] P. Sabino, Forward or backward simulation? A comparative study, Quant. Finance 20 (2020), no. 7, 1213–1226. 10.1080/14697688.2020.1741668Search in Google Scholar

[36] A. Seguret, C. Alasseur, J. F. Bonnans, A. De Paola, N. Oudjane and V. Trovato, Decomposition of high dimensional aggregative stochastic control problems, preprint (2020), https://arxiv.org/abs/2008.09827. Search in Google Scholar

[37] N. Touzi, Optimal stochastic control, stochastic target problems, and backward SDE, Fields Inst. Monogr. 29, Springer, New York, 2013. 10.1007/978-1-4614-4286-8Search in Google Scholar

[38] A. Y. Veretennikov, Parabolic equations and Itô’s stochastic equations with coefficients discontinuous in the time variable, Math. Notes Acad. Sci. USSR 31 (1982), no. 4, 278–283. 10.1007/BF01138937Search in Google Scholar

Received: 2021-04-27
Revised: 2021-09-25
Accepted: 2021-09-26
Published Online: 2021-10-21
Published in Print: 2021-12-01

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 27.4.2024 from https://www.degruyter.com/document/doi/10.1515/mcma-2021-2095/html
Scroll to top button