Abstract
We propose a novel efficient method for autonomous navigation of nonholonomic mobile robots within complex indoor environments, with application to Electric Powered Wheelchairs (EPW). It is designed and developed using the Deep Reinforcement Learning (RL) framework. Specifically, an end-to-end navigation model is devised utilizing an “off-policy” RL algorithm, which fully exploits information from a realistic setup to perform map-less, collision-free navigation. The model takes as input noisy sensor readings and produces moving commands for the mobile robot, thus, mapping a flow of environmental observations to a continuous space of driving actions, to reach a desired target. The effectiveness and efficiency of the proposed approach is tested through experiments in simulation, in both seen and unseen environments, and is compared to human performance, as well as state-of-the-art motion planners. The results show that our trained planner is not only able to navigate the nonholonomic mobile robot (EPW) through the challenging scenarios with significantly high success rates, but also, it outperforms the baseline methods in a range of performance aspects.
This work is supported by the Assistive Devices for empowering dis-Abled People through robotic Technologies (ADAPT) project. ADAPT is selected for funding by the INTERREG VA France (Channel) England Programme which is co-financed by the European Regional Development Fund (ERDF).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, C., Seff, A., Kornhauser, A., Xiao, J.: DeepDriving: learning affordance for direct perception in autonomous driving. In: Proceedings of the IEEE International Conference on Computer Vision 2015 Inter(Figure 1), pp. 2722–2730 (2015). https://doi.org/10.1109/ICCV.2015.312
Dieter, F., Wolfram, B., Sebastian, T.: The Dynamic Window Approach to Collision Avoidance, pp. 137–146 (1997). https://www.ri.cmu.edu/pub_files/pub1/fox_dieter_1997_1/fox_dieter_1997_1.pdf
Gao, C., Sands, M., Spletzer, J.: Towards autonomous wheelchair systems in urban environments. In: Howard, A., Iagnemma, K., Kelly, A. (eds.) Field and Service Robotics. Springer Tracts in Advanced Robotics, vol. 62, pp. 13–23. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-13408-1_2
Grewal, H.S., Thotappala Jayaprakash, N., Matthews, A., Shrivastav, C., George, K.: PCL-based autonomous wheelchair navigation in unmapped indoor environments. In: 2018 9th IEEE Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, UEMCON 2018, pp. 291–296 (2018). https://doi.org/10.1109/UEMCON.2018.8796660
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: 35th International Conference on Machine Learning, ICML 2018, vol. 5, pp. 2976–2989 (2018)
Kirby, R.L., Swuste, J., Dupuis, D.J., MacLeod, D.A., Monroe, R.: The Wheelchair Skills Test: a pilot study of a new outcome measure. Arch. Phys. Med. Rehabil. 83(1), 10–18 (2002). https://doi.org/10.1053/apmr.2002.26823
Kretzschmar, H., Spies, M., Sprunk, C., Burgard, W.: Socially compliant mobile robot navigation via inverse reinforcement learning. Int. J. Robot. Res. 35, 1289–1307 (2016). https://doi.org/10.1177/0278364915619772
Li, R., Wei, L., Gu, D., Hu, H., McDonald-Maier, K.D.: Multi-layered map based navigation and interaction for an intelligent wheelchair. In: 2013 IEEE International Conference on Robotics and Biomimetics, ROBIO 2013, pp. 115–120, December 2013. https://doi.org/10.1109/ROBIO.2013.6739445
Mirowski, P., et al.: Learning to navigate in complex environments. In: 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings (2019)
Morales, Y., Kallakuri, N., Shinozawa, K., Miyashita, T., Hagita, N.: Human-comfortable navigation for an autonomous robotic wheelchair. In: IEEE International Conference on Intelligent Robots and Systems, pp. 2737–2743 (2013). https://doi.org/10.1109/IROS.2013.6696743
Pfeiffer, M., Schaeuble, M., Nieto, J., Siegwart, R., Cadena, C.: From perception to decision: a data-driven approach to end-to-end motion planning for autonomous ground robots. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 1527–1533 (2017). https://doi.org/10.1109/ICRA.2017.7989182
Rösmann, C., Feiten, W., Wösch, T., Hoffmann, F., Bertram, T.: Efficient trajectory optimization using a sparse model. In: 2013 European Conference on Mobile Robots, pp. 138–143 (2013). https://doi.org/10.1109/ECMR.2013.6698833
Sinyukov, D., Desmond, R., Dickerman, M., Fleming, J., Schaufeld, J., Padir, T.: Multi-modal control framework for a semi-autonomous wheelchair using modular sensor designs. Intel. Serv. Robot. 7(3), 145–155 (2014). https://doi.org/10.1007/s11370-014-0149-7
Tai, L., Paolo, G., Liu, M.: Virtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation. In: IEEE International Conference on Intelligent Robots and Systems, September 2017. https://doi.org/10.1109/IROS.2017.8202134
Yayan, U., Akar, B., Inan, F., Yazici, A.: Development of indoor navigation software for intelligent wheelchair. In: INISTA 2014 - IEEE International Symposium on Innovations in Intelligent Systems and Applications, Proceedings, pp. 325–329 (2014). https://doi.org/10.1109/INISTA.2014.6873639
Zhou, X., Gao, Y., Guan, L.: Towards goal-directed navigation through combining learning based global and local planners. Sensors (Switzerland) 19(1) (2019). https://doi.org/10.3390/s19010176
Zhu, Y., et al.: Target-driven visual navigation in indoor scenes using deep reinforcement learning. In: Proceedings - IEEE International Conference on Robotics and Automation, pp. 3357–3364 (2017). https://doi.org/10.1109/ICRA.2017.7989381
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Chatzidimitriadis, S., Sirlantzis, K. (2022). Deep Reinforcement Learning for Autonomous Navigation in Robotic Wheelchairs. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13364. Springer, Cham. https://doi.org/10.1007/978-3-031-09282-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-09282-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09281-7
Online ISBN: 978-3-031-09282-4
eBook Packages: Computer ScienceComputer Science (R0)