A fast proximal gradient method and convergence analysis for dynamic mean field planning

Yu, Jiajia; Lai, Rongjie; Li, Wuchen; Osher, Stanley

doi:10.1090/mcom/3879

A fast proximal gradient method and convergence analysis for dynamic mean field planning
HTML articles powered by AMS MathViewer

by Jiajia Yu, Rongjie Lai, Wuchen Li and Stanley Osher

Math. Comp. 93 (2024), 603-642

DOI: https://doi.org/10.1090/mcom/3879

Published electronically: July 24, 2023

HTML | PDF | Request permission

Abstract:

In this paper, we propose an efficient and flexible algorithm to solve dynamic mean-field planning problems based on an accelerated proximal gradient method. Besides an easy-to-implement gradient descent step in this algorithm, a crucial projection step becomes solving an elliptic equation whose solution can be obtained by conventional methods efficiently. By induction on iterations used in the algorithm, we theoretically show that the proposed discrete solution converges to the underlying continuous solution as the grid becomes finer. Furthermore, we generalize our algorithm to mean-field game problems and accelerate it using multilevel and multigrid strategies. We conduct comprehensive numerical experiments to confirm the convergence analysis of the proposed algorithm, to show its efficiency and mass preservation property by comparing it with state-of-the-art methods, and to illustrate its flexibility for handling various mean-field variational problems.

References

Yves Achdou, Francisco J. Buera, Jean-Michel Lasry, Pierre-Louis Lions, and Benjamin Moll, Partial differential equation models in macroeconomics, Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 372 (2014), no. 2028, 20130397, 19. MR 3268061, DOI 10.1098/rsta.2013.0397
Yves Achdou, Fabio Camilli, and Italo Capuzzo-Dolcetta, Mean field games: numerical methods for the planning problem, SIAM J. Control Optim. 50 (2012), no. 1, 77–109. MR 2888257, DOI 10.1137/100790069
Yves Achdou, Fabio Camilli, and Italo Capuzzo-Dolcetta, Mean field games: convergence of a finite difference method, SIAM J. Numer. Anal. 51 (2013), no. 5, 2585–2612. MR 3097034, DOI 10.1137/120882421
Yves Achdou and Italo Capuzzo-Dolcetta, Mean field games: numerical methods, SIAM J. Numer. Anal. 48 (2010), no. 3, 1136–1162. MR 2679575, DOI 10.1137/090758477
Y. Achdou, Jiequn Han, Jean-Michel Lasry, Pierre-Louis Lions, and Benjamin Moll, Income and wealth distribution in macroeconomics: a continuous-time approach, Technical Report, National Bureau of Economic Research, 2017.
Yves Achdou and Mathieu Laurière, Mean field games and applications: numerical aspects, Mean field games, Lecture Notes in Math., vol. 2281, Springer, Cham, [2020] ©2020, pp. 249–307. MR 4214777, DOI 10.1007/978-3-030-59837-2_{4}
Yves Achdou and Victor Perez, Iterative strategies for solving linearized discrete mean field games systems, Netw. Heterog. Media 7 (2012), no. 2, 197–217. MR 2928376, DOI 10.3934/nhm.2012.7.197
Martin Arjovsky, Soumith Chintala, and Léon Bottou, Wasserstein GAN, Preprint, arXiv:1701.07875, 2017.
Heinz H. Bauschke, Regina S. Burachik, Patrick L. Combettes, Veit Elser, D. Russell Luke, and Henry Wolkowicz (eds.), Fixed-point algorithms for inverse problems in science and engineering, Springer Optimization and Its Applications, vol. 49, Springer, New York, 2011. MR 2858828, DOI 10.1007/978-1-4419-9569-8
Amir Beck and Marc Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci. 2 (2009), no. 1, 183–202. MR 2486527, DOI 10.1137/080716542
Jean-David Benamou and Yann Brenier, A numerical method for the optimal time-continuous mass transport problem and related problems, Monge Ampère equation: applications to geometry and optimization (Deerfield Beach, FL, 1997) Contemp. Math., vol. 226, Amer. Math. Soc., Providence, RI, 1999, pp. 1–11. MR 1660739, DOI 10.1090/conm/226/03232
Jean-David Benamou and Yann Brenier, A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem, Numer. Math. 84 (2000), no. 3, 375–393. MR 1738163, DOI 10.1007/s002110050002
Jean-David Benamou and Guillaume Carlier, Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, J. Optim. Theory Appl. 167 (2015), no. 1, 1–26. MR 3395203, DOI 10.1007/s10957-015-0725-9
Jean-David Benamou and Guillaume Carlier, Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, J. Optim. Theory Appl. 167 (2015), no. 1, 1–26. MR 3395203, DOI 10.1007/s10957-015-0725-9
Jean-David Benamou, Guillaume Carlier, and Maxime Laborde, An augmented Lagrangian approach to Wasserstein gradient flows and applications, Gradient flows: from theory to application, ESAIM Proc. Surveys, vol. 54, EDP Sci., Les Ulis, 2016, pp. 1–17 (English, with English and French summaries). MR 3565819, DOI 10.1051/proc/201654001
Jean-David Benamou, Guillaume Carlier, and Filippo Santambrogio, Variational mean field games, Active particles. Vol. 1. Advances in theory, models, and applications, Model. Simul. Sci. Eng. Technol., Birkhäuser/Springer, Cham, 2017, pp. 141–171. MR 3644590
Jean-David Benamou, Brittany D. Froese, and Adam M. Oberman, Numerical solution of the optimal transportation problem using the Monge-Ampère equation, J. Comput. Phys. 260 (2014), 107–126. MR 3151832, DOI 10.1016/j.jcp.2013.12.015
Giuseppe Buttazzo, Luigi De Pascale, and Paola Gori-Giorgi, Optimal-transport formulation of electronic density-functional theory, Phys. Rev. A 85 (2012), no. 6, 062502.
Pierre Cardaliaguet, Weak solutions for first order mean field games with local coupling, Analysis and geometry in control theory and its applications, Springer INdAM Ser., vol. 11, Springer, Cham, 2015, pp. 111–158. MR 3408214, DOI 10.1007/978-3-319-06917-3_{5}
P. Cardaliaguet, G. Carlier, and B. Nazaret, Geodesics for a class of distances in the space of probability measures, Calc. Var. Partial Differential Equations 48 (2013), no. 3-4, 395–420. MR 3116016, DOI 10.1007/s00526-012-0555-7
Pierre Cardaliaguet and P. Jameson Graber, Mean field games systems of first order, ESAIM Control Optim. Calc. Var. 21 (2015), no. 3, 690–722. MR 3358627, DOI 10.1051/cocv/2014044
Pierre Cardaliaguet, Alpár R. Mészáros, and Filippo Santambrogio, First order mean field games with density constraints: pressure equals price, SIAM J. Control Optim. 54 (2016), no. 5, 2672–2709. MR 3556062, DOI 10.1137/15M1029849
Codina Cotar, Gero Friesecke, and Claudia Klüppelberg, Density functional theory and optimal transportation with Coulomb cost, Comm. Pure Appl. Math. 66 (2013), no. 4, 548–599. MR 3020313, DOI 10.1002/cpa.21437
Antonio De Paola, Vincenzo Trovato, David Angeli, and Goran Strbac, A mean field game approach for distributed control of thermostatic loads acting in simultaneous energy-frequency response markets, IEEE Trans. Smart Grid 10 (2019), no. 6, 5987–5999.
Alfred Galichon, Optimal transport methods in economics, Princeton University Press, Princeton, NJ, 2016. MR 3586373, DOI 10.1515/9781400883592
Diogo Gomes and João Saúde, A mean-field game approach to price formation in electricity markets, Preprint, arXiv:1807.07088, 2018.
Diogo A. Gomes and Tommaso Seneci, Displacement convexity for first-order mean-field games, Minimax Theory Appl. 3 (2018), no. 2, 261–284. MR 3882532
Diogo A. Gomes and João Saúde, Mean field games models—a brief survey, Dyn. Games Appl. 4 (2014), no. 2, 110–154. MR 3195844, DOI 10.1007/s13235-013-0099-2
P. Jameson Graber and Alpár R. Mészáros, Sobolev regularity for first order mean field games, Ann. Inst. H. Poincaré C Anal. Non Linéaire 35 (2018), no. 6, 1557–1576. MR 3846236, DOI 10.1016/j.anihpc.2018.01.002
Olivier Guéant, Jean-Michel Lasry, and Pierre-Louis Lions, Mean field games and applications, Paris-Princeton Lectures on Mathematical Finance 2010, Lecture Notes in Math., vol. 2003, Springer, Berlin, 2011, pp. 205–266. MR 2762362, DOI 10.1007/978-3-642-14660-2_{3}
Steven Haker, Lei Zhu, Allen Tannenbaum, and Sigurd Angenent, Optimal mass transport for registration and warping, Int. J. Comput. Vis. 60 (2004), no. 3, 225–240.
Minyi Huang, Peter E. Caines, and Roland P. Malhamé, Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\epsilon$-Nash equilibria, IEEE Trans. Automat. Control 52 (2007), no. 9, 1560–1571. MR 2352434, DOI 10.1109/TAC.2007.904450
Minyi Huang, Roland P. Malhamé, and Peter E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle, Commun. Inf. Syst. 6 (2006), no. 3, 221–251. MR 2346927, DOI 10.4310/CIS.2006.v6.n3.a5
Matt Jacobs, Flavien Léger, Wuchen Li, and Stanley Osher, Solving large-scale optimization problems with a convergence rate independent of grid size, SIAM J. Numer. Anal. 57 (2019), no. 3, 1100–1123. MR 3950682, DOI 10.1137/18M118640X
Jean-Michel Lasry and Pierre-Louis Lions, Mean field games, Jpn. J. Math. 2 (2007), no. 1, 229–260. MR 2295621, DOI 10.1007/s11537-007-0657-8
Wonjun Lee, Rongjie Lai, Wuchen Li, and Stanley Osher, Generalized unnormalized optimal transport and its fast algorithms, J. Comput. Phys. 436 (2021), Paper No. 110041, 24. MR 4236013, DOI 10.1016/j.jcp.2020.110041
Robert Michael Lewis and Stephen G. Nash, Model problems for the multigrid optimization of systems governed by differential equations, SIAM J. Sci. Comput. 26 (2005), no. 6, 1811–1837. MR 2196577, DOI 10.1137/S1064827502407792
Haoya Li, Yuwei Fan, and Lexing Ying, A simple multiscale method for mean field games, J. Comput. Phys. 439 (2021), Paper No. 110385, 18. MR 4253925, DOI 10.1016/j.jcp.2021.110385
Jialin Liu, Wotao Yin, Wuchen Li, and Yat Tin Chow, Multilevel optimal transport: a fast approximation of Wasserstein-1 distances, SIAM J. Sci. Comput. 43 (2021), no. 1, A193–A220. MR 4198574, DOI 10.1137/18M1219813
Carlo Orrieri, Alessio Porretta, and Giuseppe Savaré, A variational approach to the mean field planning problem, J. Funct. Anal. 277 (2019), no. 6, 1868–1957. MR 3985520, DOI 10.1016/j.jfa.2019.04.011
Nicolas Papadakis, Optimal transport for image processing, Ph.D. Thesis, 2015.
Nicolas Papadakis, Gabriel Peyré, and Edouard Oudet, Optimal transport with proximal splitting, SIAM J. Imaging Sci. 7 (2014), no. 1, 212–238. MR 3158785, DOI 10.1137/130920058
Gabriel Peyré, Marco Cuturi, et al., Computational optimal transport, Found. Trends Mach. Learn. 11 (2019), no. 5–6, 355–607.
Alessio Porretta, On the planning problem for a class of mean field games, C. R. Math. Acad. Sci. Paris 351 (2013), no. 11-12, 457–462 (English, with English and French summaries). MR 3090129, DOI 10.1016/j.crma.2013.07.004
Alessio Porretta, On the planning problem for the mean field games system, Dyn. Games Appl. 4 (2014), no. 2, 231–256. MR 3195848, DOI 10.1007/s13235-013-0080-0
R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, 1970. MR 274683, DOI 10.1515/9781400873173
Lars Ruthotto, Stanley J. Osher, Wuchen Li, Levon Nurbekyan, and Samy Wu Fung, A machine learning framework for solving high-dimensional mean field game and mean field control problems, Proc. Natl. Acad. Sci. USA 117 (2020), no. 17, 9183–9193. MR 4236167, DOI 10.1073/pnas.1922204117
Cédric Villani, Optimal transport, Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 338, Springer-Verlag, Berlin, 2009. Old and new. MR 2459454, DOI 10.1007/978-3-540-71050-9
Weinan E, Jiequn Han, and Qianxiao Li, A mean-field optimal control formulation of deep learning, Res. Math. Sci. 6 (2019), no. 1, Paper No. 10, 41. MR 3891852, DOI 10.1007/s40687-018-0172-y
Chungang Yang, Jiandong Li, Min Sheng, Alagan Anpalagan, and Jia Xiao, Mean field game-theoretic framework for interference and energy-aware control in 5G ultra-dense networks, IEEE Wirel. Commun. 25 (2017), no. 1, 114–121.
Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Weinan Zhang, and Jun Wang, Mean field multi-agent reinforcement learning, International Conference on Machine Learning, PMLR, 2018, pp. 5571–5580.