Abstract
Neural networks are widely applied in control applications, yet providing safety guarantees for neural networks is challenging due to their highly nonlinear nature. We provide a comprehensive introduction to the analysis of recurrent neural networks (RNNs) using robust control and dissipativity theory. Specifically, we consider
Zusammenfassung
Neuronale Netze sind in regelungstechnischen Anwendungen weit verbreitet, doch die Gewährleistung von Sicherheitsgarantien für neuronale Netze ist aufgrund ihrer hochgradig nichtlinearen Natur schwierig. Wir bieten eine umfassende Einführung in die Analyse von rekurrenten neuronalen Netzen (RNNs) mit Hilfe der robusten Regelung und Dissipativitätstheorie. Um das spezifische Problem der Robustheit gegenüber Eingangsstörungen zu addressieren, betrachten wir die
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: 390740016
Award Identifier / Grant number: 468094890
Funding statement: This work was funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2075 – 390740016 and under grant 468094890. We acknowledge the support by the Stuttgart Center for Simulation Science (SimTech). The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Patricia Pauli and Julian Berberich.
About the authors
Patricia Pauli received the Master’s degree in Mechanical Engineering and Computational Engineering from the Technical University of Darmstadt, Germany, in 2019. She has since been a Ph. D. student with the Institute for Systems Theory and Automatic Control under supervision of Prof. Frank Allgöwer and a member of the International Max Planck Research School for Intelligent Systems (IMPRS-IS). Her research interests are in the area of robust machine learning and learning based control.
Julian Berberich received the Master’s degree in Engineering Cybernetics from the University of Stuttgart, Germany, in 2018. Since 2018, he has been a Ph. D. student at the Institute for Systems Theory and Automatic Control under supervision of Prof. Frank Allgöwer and a member of the International Max Planck Research School (IMPRS). He received the Outstanding Student Paper Award at the 59th Conference on Decision and Control in 2020. His research interests are in the area of data-driven system analysis and control.
Frank Allgöwer is professor of mechanical engineering at the University of Stuttgart, Germany, and Director of the Institute for Systems Theory and Automatic Control (IST) there. Frank is active in serving the community in several roles: Among others he was President of the International Federation of Automatic Control (IFAC) for the years 2017–2020, Vice-president for Technical Activities of the IEEE Control Systems Society for 2013/14, and Editor of the journal Automatica from 2001 until 2015. From 2012 until 2020 Frank served in addition as Vice-president for the German Research Foundation (DFG), which is Germany’s most important research funding organization. His research interests include predictive control, data-based control, networked control, cooperative control, and nonlinear control with application to a wide range of fields including systems biology.
Acknowledgment
The authors thank Dennis Gramlich for helpful discussions.
References
1. Delgado, A., C. Kambhampati and K. Warwick. 1995. Dynamic recurrent neural network for system identification and control. IEE Proceedings-Control Theory and Applications 142(4): 307–314.10.1049/ip-cta:19951873Search in Google Scholar
2. Gu, F., H. Yin, L.E. Ghaoui, M. Arcak, P. Seiler and M. Jin. 2021. Recurrent neural network controllers synthesis with stability guarantees for partially observed systems. arXiv preprint arXiv:2109.03861.10.1609/aaai.v36i5.20476Search in Google Scholar
3. Nikolakopoulou, A., M.S. Hong and R.D. Braatz. 2020. Feedback control of dynamic artificial neural networks using linear matrix inequalities. In: 2020 59th IEEE Conference on Decision and Control (CDC). IEEE, pp. 2210–2215.10.1109/CDC42340.2020.9303770Search in Google Scholar
4. Fazlyab, M., M. Morari and G.J. Pappas. 2020. Safety verification and robustness analysis of neural networks via quadratic constraints and semidefinite programming. IEEE Transactions on Automatic Control.10.1109/TAC.2020.3046193Search in Google Scholar
5. Yin, H., P. Seiler and M. Arcak. 2021. Stability analysis using quadratic constraints for systems with neural network controllers. IEEE Transactions on Automatic Control.10.1109/TAC.2021.3069388Search in Google Scholar
6. Pauli, P., D. Gramlich, J. Berberich and F. Allgöwer. 2021. Linear systems with neural network nonlinearities: Improved stability analysis via acausal Zames-Falb multipliers. arXiv preprint arXiv:2103.17106.10.1109/CDC45484.2021.9683341Search in Google Scholar
7. Manchester, I.R., M. Revay and R. Wang. 2021. Contraction-based methods for stable identification and robust machine learning: a tutorial. arXiv preprint arXiv:2110.00207.10.1109/CDC45484.2021.9683128Search in Google Scholar
8. Aquino, B., A. Rahnama, P. Seiler, L. Lin and V. Gupta. 2021. Robustness against adversarial attacks in neural networks using incremental dissipativity. arXiv preprint arXiv:2111.12906.10.1109/LCSYS.2022.3150719Search in Google Scholar
9. Hill, D. and P. Moylan. 1976. The stability of nonlinear dissipative systems. IEEE transactions on automatic control 21(5): 708–711.10.1109/TAC.1976.1101352Search in Google Scholar
10. Veenman, J., C.W. Scherer and H. Köroğlu. 2016. Robust stability and performance analysis based on integral quadratic constraints. European Journal of Control 31: 1–32.10.1016/j.ejcon.2016.04.004Search in Google Scholar
11. Paganini, F. 1995. Necessary and sufficient conditions for robust H2 performance. In: Proceedings of 1995 34th IEEE Conference on Decision and Control, vol. 2. IEEE, pp. 1970–1975.10.1109/CDC.1995.480635Search in Google Scholar
12. Stoorvogel, A.A. 1993. The robust H2 control problem: a worst-case design. IEEE Transactions on Automatic Control 38(9): 1358–1371.10.1109/9.237647Search in Google Scholar
13. Revay, M., R. Wang and I.R. Manchester. 2020. A convex parameterization of robust recurrent neural networks. IEEE Control Systems Letters 5(4): 1363–1368.10.23919/ACC50511.2021.9482874Search in Google Scholar
14. Szegedy, C., W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow and R. Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.Search in Google Scholar
15. Paganini, F. and E. Feron. 2000. Linear matrix inequality methods for robust H2 analysis: A survey with comparisons. In: Advances in linear matrix inequality methods in control. SIAM, pp. 129–151.10.1137/1.9780898719833.ch7Search in Google Scholar
16. Scherer, C. and S. Weiland. 2000. Linear matrix inequalities in control. Lecture Notes, Dutch Institute for Systems and Control, Delft, The Netherlands 3(2).Search in Google Scholar
17. Byrnes, C.I. and W. Lin. 1994. Losslessness, feedback equivalence, and the global stabilization of discrete-time nonlinear systems. IEEE Transactions on automatic control 39(1): 83–98.10.1109/9.273341Search in Google Scholar
18. Zhang, J., J. Carrasco and W.P. Heath. 2021. Duality bounds for discrete-time Zames-Falb multipliers. IEEE Transactions on Automatic Control.10.1109/TAC.2021.3095418Search in Google Scholar
19. Turner, M.C. 2021. Zames-Falb multipliers: don’t panic. arXiv preprint arXiv:2106.15913.Search in Google Scholar
20. Willems, J.C. 1972. Dissipative dynamical systems part ii: Linear systems with quadratic supply rates. Archive for rational mechanics and analysis 45(5): 352–393.10.1007/BF00276494Search in Google Scholar
21. Paganini, F. 1999. Convex methods for robust H2 analysis of continuous-time systems. IEEE Transactions on Automatic Control 44(2): 239–252.10.1109/9.746251Search in Google Scholar
22. Paganini, F. 1999. Frequency domain conditions for robust H2 performance. IEEE Transactions on Automatic Control 44(1): 38–49.10.1109/9.739065Search in Google Scholar
23. Pauli, P., N. Funcke, D. Gramlich, M.A. Msalmi and F. Allgöwer. 2022. Neural network training under semidefinite constraints. arXiv preprint arXiv:2201.00632.10.1109/CDC51059.2022.9992331Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
