Abstract
We propose a fully backward representation of semilinear PDEs with application to stochastic control. Based on this, we develop a fully backward Monte-Carlo scheme allowing to generate the regression grid, backwardly in time, as the value function is computed. This offers two key advantages in terms of computational efficiency and memory. First, the grid is generated adaptively in the areas of interest, and second, there is no need to store the entire grid. The performances of this technique are compared in simulations to the traditional Monte-Carlo forward-backward approach on a control problem of thermostatic loads.
Funding statement: The work was supported by a public grant as part of the Investissement d’avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH, in a joint call with Gaspard Monge Program for optimization, operations research and their interactions with data sciences.
A Appendix
A.1 A sufficient condition to obtain an equivalent probability
Lemma A.1.
We recall that
where
is an
Proof.
By following [25, Corollary 5.14], it is sufficient to find a constant time step subdivision
By combining Jensen’s inequality and Fubini’s theorem, this is fulfilled in particular if for all
where
since
By taking into account the fact that
where
A.2 Proof of the local Lipschitz property of the cost functional J
Lemma A.2.
Suppose the validity of Assumption 4.1. Suppose in addition that the functions g and
are locally Lipschitz with polynomial growth gradient (uniformly in t and α). Then, for each
is locally Lipschitz, uniformly in t and α.
Proof.
We give here a proof of the local Lipschitz property for the term involving the function g since the other term can be treated in the same way.
Let
where we have used the estimate
together with Gronwall’s lemma. In view of (A.1), the point is proved if
is bounded uniformly in
A.3 A simplified version of the envelope theorem
Lemma A.3.
Let Λ be an arbitrary set and let O be an open subset of
for every
Proof.
Let x be as in the proposition statement and let
By the differentiability of V at the point x, (A.2) implies
Setting h to
Combining (A.3) and (A.4), we get
which forces
References
[1] C. Bender and R. Denk, A forward scheme for backward SDEs, Stochastic Process. Appl. 117 (2007), no. 12, 1793–1812. 10.1016/j.spa.2007.03.005Search in Google Scholar
[2] C. Bender and T. Moseler, Importance sampling for backward SDEs, Stoch. Anal. Appl. 28 (2010), no. 2, 226–253. 10.1080/07362990903546405Search in Google Scholar
[3] C. Bender and J. Steiner, Least-squares Monte Carlo for backward SDEs, Numerical Methods in Finance, Springer Proc. Math. 12, Springer, Heidelberg (2012), 257–289. 10.1007/978-3-642-25746-9_8Search in Google Scholar
[4] B. Bouchard and N. Touzi, Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations, Stochastic Process. Appl. 111 (2004), no. 2, 175–206. 10.1016/j.spa.2004.01.001Search in Google Scholar
[5] R. Bronson and G. B. Costa, Matrix Methods: Applied Linear Algebra, Academic Press, New York, 2008. Search in Google Scholar
[6] D. S. Callaway and I. A. Hiskens, Achieving controllability of electric loads, Proc. IEEE 99 (2010), no. 1, 184–199. 10.1109/JPROC.2010.2081652Search in Google Scholar
[7] M. G. Crandall, H. Ishii and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N. S.) 27 (1992), no. 1, 1–67. 10.1090/S0273-0979-1992-00266-5Search in Google Scholar
[8] F. Delarue and S. Menozzi, An interpolated stochastic algorithm for quasi-linear PDEs, Math. Comp. 77 (2008), no. 261, 125–158. 10.1090/S0025-5718-07-02008-XSearch in Google Scholar
[9] C. Di Girolami and F. Russo, About classical solutions of the path-dependent heat equation, Random Oper. Stoch. Equ. 28 (2020), no. 1, 35–62. 10.1515/rose-2020-2028Search in Google Scholar
[10] E. Gobet and M. Grangereau, Federated stochastic control of numerous heterogeneous energy storage systems, preprint (2021), https://hal.archives-ouvertes.fr/hal-03108611. Search in Google Scholar
[11] I. Exarchos and E. A. Theodorou, Stochastic optimal control via forward and backward stochastic differential equations and importance sampling, Automatica J. IFAC 87 (2018), 159–165. 10.1016/j.automatica.2017.09.004Search in Google Scholar
[12] G. Fabbri, F. Gozzi and A. Świech, Stochastic Optimal Control in Infinite Dimension, Probab. Theory Stoch. Model. 82, Springer, Cham, 2017. 10.1007/978-3-319-53067-3Search in Google Scholar
[13] E. Gobet and C. Labart, Error expansion for the discretization of backward stochastic differential equations, Stochastic Process. Appl. 117 (2007), no. 7, 803–829. 10.1016/j.spa.2006.10.007Search in Google Scholar
[14] E. Gobet and C. Labart, Solving BSDE with adaptive control variate, SIAM J. Numer. Anal. 48 (2010), no. 1, 257–277. 10.1137/090755060Search in Google Scholar
[15] E. Gobet, J.-P. Lemor and X. Warin, A regression-based Monte Carlo method to solve backward stochastic differential equations, Ann. Appl. Probab. 15 (2005), no. 3, 2172–2202. 10.1214/105051605000000412Search in Google Scholar
[16] E. Gobet and P. Turkedjiev, Linear regression MDP scheme for discrete backward stochastic differential equations under general conditions, Math. Comp. 85 (2016), no. 299, 1359–1391. 10.1090/mcom/3013Search in Google Scholar
[17] E. Gobet and P. Turkedjiev, Adaptive importance sampling in least-squares Monte Carlo algorithms for backward stochastic differential equations, Stochastic Process. Appl. 127 (2017), no. 4, 1171–1203. 10.1016/j.spa.2016.07.011Search in Google Scholar
[18] F. Gozzi and F. Russo, Verification theorems for stochastic optimal control problems via a time dependent Fukushima–Dirichlet decomposition, Stochastic Process. Appl. 116 (2006), no. 11, 1530–1562. 10.1016/j.spa.2006.04.008Search in Google Scholar
[19] F. Gozzi and F. Russo, Weak Dirichlet processes with a stochastic control perspective, Stochastic Process. Appl. 116 (2006), no. 11, 1563–1583. 10.1016/j.spa.2006.04.009Search in Google Scholar
[20] U. G. Haussmann and E. Pardoux, Time reversal of diffusions, Ann. Probab. 14 (1986), no. 4, 1188–1205. 10.1214/aop/1176992362Search in Google Scholar
[21] H. Ishii, On uniqueness and existence of viscosity solutions of fully nonlinear second-order elliptic PDEs, Comm. Pure Appl. Math. 42 (1989), no. 1, 15–45. 10.1002/cpa.3160420103Search in Google Scholar
[22] H. Ishii and K. Kobayasi, On the uniqueness and existence of solutions of fully nonlinear parabolic PDEs under the Osgood type condition, Differential Integral Equations 7 (1994), no. 3–4, 909–920. 10.57262/die/1370267713Search in Google Scholar
[23] L. Izydorczyk, N. Oudjane, F. Russo and G. Tessitore, Fokker–Planck equations with terminal condition and related McKean probabilistic representation, preprint (2020), https://hal.archives-ouvertes.fr/hal-02902615. 10.1007/s00030-021-00736-1Search in Google Scholar
[24] R. Jensen, P.-L. Lions and P. E. Souganidis, A uniqueness result for viscosity solutions of second order fully nonlinear partial differential equations, Proc. Amer. Math. Soc. 102 (1988), no. 4, 975–978. 10.1090/S0002-9939-1988-0934877-2Search in Google Scholar
[25] I. Karatzas and S. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed., Grad. Texts in Math. 113, Springer, New York, 1991. Search in Google Scholar
[26] N. V. Krylov, Controlled Diffusion Processes, Stochastic Model. Appl. Probab. 14, Springer, Berlin, 2009. Search in Google Scholar
[27] C. Labart and J. Lelong, A parallel algorithm for solving BSDEs, Monte Carlo Methods Appl. 19 (2013), no. 1, 11–39. 10.1515/mcma-2013-0001Search in Google Scholar
[28] P.-L. Lions, Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness, Comm. Partial Differential Equations 8 (1983), no. 11, 1229–1276. 10.1080/03605308308820301Search in Google Scholar
[29] D. Nunziante, Existence and uniqueness of unbounded viscosity solutions of parabolic equations with discontinuous time-dependence, Nonlinear Anal. 18 (1992), no. 11, 1033–1062. 10.1016/0362-546X(92)90194-JSearch in Google Scholar
[30] E. Pardoux, Backward stochastic differential equations and viscosity solutions of systems of semilinear parabolic and elliptic PDEs of second order, Stochastic Analysis and Related Topics. VI (Geilo 1996), Progr. Probab. 42, Birkhäuser, Boston (1998), 79–127. 10.1007/978-1-4612-2022-0_2Search in Google Scholar
[31] E. Pardoux, F. Pradeilles and Z. Rao, Probabilistic interpretation of a system of semi-linear parabolic partial differential equations, Ann. Inst. Henri Poincaré Probab. Stat. 33 (1997), no. 4, 467–490. 10.1016/S0246-0203(97)80101-XSearch in Google Scholar
[32] H. Pham, Continuous-Time Stochastic Control and Optimization with Financial Applications, Stochastic Model. Appl. Probab. 61, Springer, Berlin, 2009. 10.1007/978-3-540-89500-8Search in Google Scholar
[33] D. Revuz and M. Yor, Continuous Martingales and Brownian Motion, 3rd ed., Grundlehren Math. Wiss. 293, Springer, Berlin, 1999. 10.1007/978-3-662-06400-9Search in Google Scholar
[34] C. Ribeiro and N. Webber, Valuing path-dependent options in the variance-gamma model by Monte Carlo with a gamma bridge, J. Comput. Finance 7 (2004), no. 2, 81–100. 10.21314/JCF.2003.110Search in Google Scholar
[35] P. Sabino, Forward or backward simulation? A comparative study, Quant. Finance 20 (2020), no. 7, 1213–1226. 10.1080/14697688.2020.1741668Search in Google Scholar
[36] A. Seguret, C. Alasseur, J. F. Bonnans, A. De Paola, N. Oudjane and V. Trovato, Decomposition of high dimensional aggregative stochastic control problems, preprint (2020), https://arxiv.org/abs/2008.09827. Search in Google Scholar
[37] N. Touzi, Optimal stochastic control, stochastic target problems, and backward SDE, Fields Inst. Monogr. 29, Springer, New York, 2013. 10.1007/978-1-4614-4286-8Search in Google Scholar
[38] A. Y. Veretennikov, Parabolic equations and Itô’s stochastic equations with coefficients discontinuous in the time variable, Math. Notes Acad. Sci. USSR 31 (1982), no. 4, 278–283. 10.1007/BF01138937Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston