Abstract
Autonomous driving can possibly facilitate the load and change the methods for transportation in our everyday life. Work is being done to create algorithms of decision making and motion control in Autonomous driving. Recently, reinforcement learning has been a predominant strategy applied for this purpose. But, problems of using reinforcement learning for autonomous driving is that the actions taken while exploration can be unsafe, and the convergence can be too slow. Therefore, before making an actual vehicle learn driving through reinforcement learning, there is an urgent need to solve the safety issue. The significance of this paper is that, it introduces Safe Reinforcement Learning (SRL) into the field of autonomous driving. Safe reinforcement learning is the method of adding constraints to ensure the safe exploration. This paper explores the Constrained Policy Optimization (CPO) algorithm. The principle is to introduce constraints in the cost function. CPO is based on the framework of the Actor-Critic algorithm where the space that is explored during the policy update process is enforced by setting tough constraints which reduces the size of policy update. A comparison is also made with typical reinforcement learning algorithms to prove its advantages in learning efficiency and safety.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Achiam, J., Held, D., Tamar, A., Abbeel, P.: Constrained policy optimization. In: International Conference on Machine Learning, pp. 22–31. PMLR (2017)
Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Baheri, A., Nageshrao, S., Kolmanovsky, I., Girard, A., Tseng, H.E., Filev, D.: Deep q-learning with dynamically-learned safety module: a case study in autonomous driving. In: Neural Information Processing Systems (NeurIPS 2019) (2019)
Baheri, A., Nageshrao, S., Tseng, H.E., Kolmanovsky, I., Girard, A., Filev, D.: Deep reinforcement learning with enhanced safety for autonomous highway driving. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1550–1555. IEEE (2019)
Chen, J., Tang, C., Xin, L., Li, S.E., Tomizuka, M.: Continuous decision making for on-road autonomous driving under uncertain and interactive environments. In: 2018 IEEE Intelligent Vehicles Symposium (IV) (2018)
Driessens, K., Džeroski, S.: Integrating guidance into relational reinforcement learning. Mach. Learn. 57(3), 271–304 (2004)
Garcıa, J., Fernández, F.: A comprehensive survey on safe reinforcement learning. J. Mach. Learn. Res. 16(1), 1437–1480 (2015)
Gehring, C., Precup, D.: Smart exploration in reinforcement learning using absolute temporal difference errors. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, pp. 1037–1044 (2013)
Geibel, P., Wysotzki, F.: Risk-sensitive reinforcement learning applied to control under constraints. J. Artif. Intell. Res. 24, 81–108 (2005)
Hafner, R., Riedmiller, M.: Reinforcement learning in feedback control. Mach. Learn. 84, 137–169 (2011)
Hoel, C.J., Wolff, K., Laine, L.: Tactical decision-making in autonomous driving by reinforcement learning with uncertainty estimation. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 1563–1569. IEEE (2020)
Howard, R.A., Matheson, J.E.: Risk-sensitive Markov decision processes. Manag. Sci. 18(7), 356–369 (1972)
Konda, V.R., Tsitsiklis, J.N.: Actor-critic algorithms. In: Advances in Neural Information Processing Systems, pp. 1008–1014. Citeseer (2000)
Li, M.G., et al.: DBUS: human driving behavior understanding system. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 2436–2444. IEEE (2019)
Thananjeyan, B., et al.: Recovery RL: safe reinforcement learning with learned recovery zones. IEEE Robot. Autom. Lett. 6(3), 4915–4922 (2021)
Wen, L., Duan, J., Li, S.E., Xu, S., Peng, H.: Safe reinforcement learning for autonomous vehicles through parallel constrained policy optimization. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–7. IEEE (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kong, Q., Zhang, L., Xu, X. (2021). Lane Keeping Algorithm for Autonomous Driving via Safe Reinforcement Learning. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12817. Springer, Cham. https://doi.org/10.1007/978-3-030-82153-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-82153-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82152-4
Online ISBN: 978-3-030-82153-1
eBook Packages: Computer ScienceComputer Science (R0)