Abstract
Modular robotic systems are a group of identical robots capable of reconfiguring depending on the assigned tasks. The task of reconfiguring the kinematic structure is the construction of a trajectory connecting the initial and target positions for each module of the system. This paper presents a method for reconfiguring the kinematic structure of a modular robotic system using deep reinforcement learning. This method is based on the deep Q-learning algorithm. In addition, a method for the formation of Q-tables has been developed, which makes it possible to effectively scale the system, as well as to obtain the most complete information about the joint position of the system and transfer it to the neural network, a mechanism for forming the observation vector has been developed. The aim of the training is to build a reward-maximizing control algorithm. To assess the effectiveness of the proposed method, a computer simulation of a modular robotic system consisting of 5, 10 and 15 modules was created. The percentage of test episodes completed without collision was about 94% at 5 modules, 89% at 10 and 82% at 15, and the percentage of successfully completed episodes was about 87%, 64% and 0%, respectively. In addition, in most of the episodes that were completed without a collision, most of the modules were in target positions. In this connection, we can conclude that for the agent was successfully trained to reconfigure 5 and 10 modules, and for 15 a partial solution of the reconfiguration problem was obtained.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yim, M., Duff, D.G., Roufas, K.: Modular reconfigurable robots, an approach to urban search and rescue. In: 1st International Workshop on Human-friendly Welfare Robotics Systems, pp. 69–76 (2000)
Yim, M., et al.: Modular self-reconfigurable robot systems [grand challenges of robotics]. IEEE Robot. Autom. Mag. 14(1), 43–52 (2007)
Yim, M., White, P., Park, M., Sastra, J.: Modular self-reconfigurable robots. In: Encyclopedia of Complexity and Systems Science, pp. 5618–5631 (2009)
Feczko, J., Manka, M., Krol, P., Giergiel, M., Uhl, T., Pietrzyk, A.: Review of the modular self reconfigurable robotic systems. In: 2015 10th International Workshop on Robot Motion and Control (RoMoCo), pp. 182–187 (2015)
Yim, M., Duff, D.G., Roufas, K.D.: PolyBot: a modular reconfigurable robot. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings, pp. 514–520. IEEE (2000)
Mezenceva, O.S., Petrenko, V.I., Zhilina, E., Pavlov, A.S., Apurin, A.A.: Developing a concept of available multi-functional modular robot for education and research. In: CEUR Workshop Proceedings SLET (2019)
Stoy, K., Brandt, D., Christensen, D.J., Brandt, D.: Self-reconfigurable Robots: An Introduction. The MIT Press, London (2010)
Kovács, G., Yusupova, N., Smetanina, O., Rassadnikova, E.: Methods and algorithms to solve the vehicle routing problem with time windows and further conditions. Pollack Periodica 13(1), 65–76 (2018)
Lynch, K.M., Park, F.C.: Modern Robotics. Cambridge University Press, Cambridge (2017)
Liu, J., Zhang, X., Hao, G.: Survey on research and development of reconfigurable modular robots. Adv. Mech. Eng. 8(8), 1–21 (2016)
Ababsa, T., Djedl, N., Duthen, Y.: Genetic programming-based self-reconfiguration planning for metamorphic robot. Int. J. Autom. Comput. 15(4), 431–442 (2017). https://doi.org/10.1007/s11633-016-1049-4
Nguyen, T.L., Nguyen, D.V., Le, T.H.: Reinforcement learning based navigation with semantic knowledge of indoor environments. In: 2019 11th International Conference on Knowledge and Systems Engineering (KSE), pp. 1–7. IEEE (2019)
Katyal, K., Wang, I., Burlina, P.: Leveraging deep reinforcement learning for reaching robotic tasks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 18–19 (2017)
Han, Y., Guo, Y., Mi, Z., Yang, Y.: Robot path planning in dynamic environments based on deep reinforcement learning. In: Ning, H. (ed.) CyberDI/CyberLife -2019. CCIS, vol. 1137, pp. 265–283. Springer, Singapore (2019). https://doi.org/10.1007/978-981-15-1922-2_19
Petrenko, V.I., Tebuyeva, F.B., Pavlov, A.S., Gurchinskiy, M.M.: Method for reconfiguring the kinematic structure of a mechatronic-modular robot in non-deterministic conditions. Syst. Eng. Inf. Technol. 2(4), 57–56 (2020). (in Russian)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Blinov, D., Vatamaniuk, I., Saveliev, A. (2021). Method for Reconfiguring Kinematic Structure of Modular Robots Using Deep Reinforcement Learning. In: Silhavy, R., Silhavy, P., Prokopova, Z. (eds) Data Science and Intelligent Systems. CoMeSySo 2021. Lecture Notes in Networks and Systems, vol 231. Springer, Cham. https://doi.org/10.1007/978-3-030-90321-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-90321-3_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90320-6
Online ISBN: 978-3-030-90321-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)