Abstract
This paper addresses the challenging task of developing an autonomous chase protocol. First, training of an autonomous vehicle capable of driving autonomously from point A to B was developed to proceed with a chase protocol as a second step. A dedicated driving setup, based on a discrete action space and a single RGB camera, was developed through a series of experiments. A dedicated curriculum learning agenda allowed to train the model capable of performing all fundamental road maneuvers. Several reward functions were proposed, which enabled effective training of the agent. In the subsequent experiments, we selected the reward function and model that produced the most significant outcome, guaranteeing that the chasing car was within 25 m of a runaway car for 63% of the episode duration. To the best of our knowledge, this work is the first one that addressed the task of the chase in urban driving using the Reinforcement Learning approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Core concepts - carla simulator
Introduction - carla simulator
Brackstone, M., McDonald, M.: Car-following: a historical review. Transport. Res. F Traffic Psychol. Behav. 2(4), 181–196 (1999)
Chu, T., Wang, J., Codecà, L., Li, Z.: Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Trans. Intell. Transp. Syst. 21(3), 1086–1095 (2019)
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: Carla: an open urban driving simulator. In: Conference on Robot Learning, pp. 1–16. PMLR (2017)
Fagnant, D., Kockelman, K.: Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transp. Res. Part A Policy Pract. 77, 167–181 (2015)
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An introduction to deep reinforcement learning (2018)
Inagaki, T., Sheridan, T.B.: A critique of the SAE conditional driving automation definition, and analyses of options for improvement. Cogn. Technol. Work 21(4), 569–578 (2019)
Jahoda, P., Cech, J., Matas, J.: Autonomous car chasing. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12538, pp. 337–352. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66823-5_20
Jo, K., Kim, J., Kim, D., Jang, C., Sunwoo, M.: Development of autonomous car-Part I: distributed system architecture and development process. IEEE Trans. Industr. Electron. 61(12), 7131–7140 (2014)
Liang, X., Wang, T., Yang, L., Xing, E.: CIRL: controllable imitative reinforcement learning for vision-based self-driving. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 604–620. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_36
Moreno-Noguer, F., Lepetit, V., Fua, P.: Accurate non-iterative o(n) solution to the PNP problem. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)
Olstam, J.J., Tapani, A.: Comparison of Car-Following Models, vol. 960. Swedish National Road and Transport Research Institute Linköping (2004)
Palanisamy, P.: Hands-On Intelligent Agents with OpenAI Gym: Your Guide to Developing AI Agents Using Deep Reinforcement Learning. Packt Publishing, Birmingham (2018)
Rojas-Rueda, D., Nieuwenhuijsen, M.J., Khreis, H., Frumkin, H.: Autonomous vehicles and public health. Annual Rev. Public Health 41, 329–345 (2020)
SAE International: J3016. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, April 2021
Schwarting, W., Alonso-Mora, J., Rus, D.: Planning and decision-making for autonomous vehicles. Annu. Rev. Control Robot. Auton. Syst. 1(1), 187–210 (2018)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Synopsys, Inc.: The 6 levels of vehicle autonomy explained
United Nations, Department of Economic and Social Affairs, Population Division. World Urbanization Prospects: The 2018 Revision (ST/ESA/SER.A/420). United Nations (2019)
Weng, L.: Curriculum for Reinforcement Learning, January 2020
Wu, Q., et al.: A Unified Cognitive Learning Framework for Adapting to Dynamic Environment and Tasks (2021)
Acknowledgment
We would like to thank the AGH Institute of Electronics which financially supported publishing of this paper with subvention 16.16.230.434.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kołomański, M., Sakhai, M., Nowak, J., Wielgosz, M. (2023). Towards End-to-End Chase in Urban Autonomous Driving Using Reinforcement Learning. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 544. Springer, Cham. https://doi.org/10.1007/978-3-031-16075-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-16075-2_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16074-5
Online ISBN: 978-3-031-16075-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)