Abstract
We review an approach for discretizing Bellman’s optimality principle based on piecewise constant functions. By applying this ansatz to a suitable dynamic game, a discrete feedback can be constructed which robustly stabilizes a given nonlinear control system. Hybrid, event and quantized systems can be naturally handled by this construction.
An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision.
Richard Bellman, 1957
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This property can be ensured by suitable asymptotic controllability properties and bounds on g.
- 2.
Available at http://www.github.com/gaioguy/gaio.
- 3.
The subsequent statements remain true if we replace \(\tilde{U}\) by any set \(\widehat{U}\subset U\) with \(\tilde{U} \subset \widehat{U}\) for which the argmin in (18) exists.
References
Anta, A., Tabuada, P.: To sample or not to sample: self-triggered control for nonlinear systems. IEEE Trans. Autom. Control 55(9), 2030–2042 (2010)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Dellnitz, M., Froyland, G., Junge, O.: The algorithms behind GAIO-set oriented numerical methods for dynamical systems. In: Ergodic Theory, Analysis, and Efficient Simulation of Dynamical Systems, pp. 145–174, 805–807. Springer, Berlin (2001)
Dellnitz, M., Hohmann, A.: A subdivision algorithm for the computation of unstable manifolds and global attractors. Numerische Mathematik 75(3), 293–317 (1997)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1, 269–271 (1959)
Fleming, W.H.: The convergence problem for differential games. J. Math. Anal. Appl. 3, 102–116 (1961)
Grüne, L., Junge, O.: A set oriented approach to optimal feedback stabilization. Syst. Control Lett. 54(2), 169–180 (2005)
Grüne, L., Junge, O.: Approximately optimal nonlinear stabilization with preservation of the Lyapunov function property. In: Proceedings of the 46th IEEE Conference on Decision and Control, pp. 702–707 (2007)
Grüne, L., Junge, O.: Global optimal control of perturbed systems. J. Optim. Theory Appl. 136(3), 411–429 (2008)
Grüne, L., Müller, F.: An algorithm for event-based optimal feedback control. In: Proceedings of the 48th IEEE Conference on Decision and Control, Shanghai, China, pp. 5311–5316 (2009)
Grüne, L., Müller, F.: Global optimal control of quantized systems. In: Proceedings of the 18th International Symposium on Mathematical Theory of Networks and Systems — MTNS2010, Budapest, Hungary, pp. 1231–1237 (2010)
Grüne, L., Nešić, D.: Optimization-based stabilization of sampled-data nonlinear systems via their approximate discrete-time models. SIAM J. Control Optim. 42(1), 98–122 (2003)
Junge, O.: Rigorous discretization of subdivision techniques. In: International Conference on Differential Equations, vol. 1, 2 (Berlin, 1999), pp. 916–918. World Scientific Publishing, River Edge (2000)
Junge, O., Osinga, H.M.: A set oriented approach to global optimal control. ESAIM Control Optim. Calc. Var. 10(2), 259–270 (2004)
Lincoln, B., Rantzer, A.: Relaxing dynamic programming. IEEE Trans. Autom. Control 51(8), 1249–1260 (2006)
Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Natl. Acad. Sci. U.S.A. 93(4), 1591–1595 (1996)
Sethian, J.A., Vladimirsky, A.: Ordered upwind methods for static Hamilton-Jacobi equations. Proc. Natl. Acad. Sci. U.S.A. 98(20), 11069–11074 (2001)
Tsitsiklis, J.N.: Efficient algorithms for globally optimal trajectories. IEEE Trans. Autom. Control 40(9), 1528–1538 (1995)
Tucker, W.: Validated Numerics: A Short Introduction to Rigorous Computations. Princeton University Press, Princeton (2011)
von Lossow, M.: A min-man version of Dijkstra’s algorithm with application to perturbed optimal control problems. In: Proceedings of the GAMM Annual Meeting, Zürich, Switzerland (2007)
Acknowledgements
OJ thanks Michael Dellnitz for being his mentor, colleague and friend since more than 25 years. OJ and LG gratefully acknowledge the support through the Priority Programme SPP 1305 Control Theory of Digitally Networked Dynamic Systems of the German Research Foundation. OJ additionally acknowledges support through a travel grant by DAAD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Grüne, L., Junge, O. (2020). From Bellman to Dijkstra: Set-Oriented Construction of Globally Optimal Controllers. In: Junge, O., Schütze, O., Froyland, G., Ober-Blöbaum, S., Padberg-Gehle, K. (eds) Advances in Dynamics, Optimization and Computation. SON 2020. Studies in Systems, Decision and Control, vol 304. Springer, Cham. https://doi.org/10.1007/978-3-030-51264-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-51264-4_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51263-7
Online ISBN: 978-3-030-51264-4
eBook Packages: EngineeringEngineering (R0)