Skip to main content
SpringerLink
Log in

Two-Legged Robot Motion Control With Recurrent Neural Networks

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Legged locomotion is a desirable ability for robotic systems thanks to its agile mobility and wide range of motions that it provides. In this paper, the use of neural network-based nonlinear controller structures which consist of recurrent and feedforward layers have been examined in the dynamically stable walking problem of two-legged robots. In detail, hybrid neural controllers, in which long short-term memory type of neuron models employed at recurrent layers, are utilized in the feedback and feedforward paths. To train these neural networks, supervised learning data sets are created by using a biped robot platform which is controlled by a central pattern generator. Then, the ability of the neural networks to perform stable gait by controlling the robot platform is examined under various ground conditions in the simulation environment. After that, the stable walking generation capacity of the neural networks and the central pattern generators are compared with each other. It is shown that the inclusion of recurrent layer provides smooth transition and control between stance and flight motion phases and \(L_2\) regularization is beneficial for walking performance. Finally, the proposed hybrid neural network models are found to be more successful gait controllers than the central pattern generator, which is employed to generate data sets used in training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code Availibility

In this study, datasets are artificially generated in the simulation environment. After that, neural networks are trained and tested with these datasets. Codes that are used to generate datasets and trained neural networks will be published upon acceptance. Biped locomotion videos can be seen from https://figshare.com/s/77c93575fc72aa26f367 link.

References

  1. Feng, S., Xinjilefu, X., Atkeson, C.G., Kim, J.: Optimization based controller design and implementation for the atlas robot in the darpa robotics challenge finals. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 1028–1035. IEEE (2015)

  2. Guizzo, E.: By leaps and bounds: an exclusive look at how Boston dynamics is redefining robot agility. IEEE Spectrum 56(12), 34–39 (2019)

    Article  Google Scholar 

  3. Holmes, P., Full, R.J., Koditschek, D., Guckenheimer, J.: The dynamics of legged locomotion: models, analyses, and challenges. SIAM Review 48(2), 207–304 (2006). https://doi.org/10.1137/S0036144504445133

    Article  MathSciNet  MATH  Google Scholar 

  4. Uyanık, İ., Saranlı, U., Morgül, Ö.: Adaptive control of a spring-mass hopper. In: 2011 IEEE International Conference on Robotics and Automation, pp. 2138–2143. IEEE (2011). https://doi.org/10.1109/ICRA.2011.5979726

  5. Chignoli, M., Kim, D., Stanger-Jones, E., Kim, S.: The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors. arXiv:2104.09025 (2021)

  6. Schwind, W.J.: Spring Loaded Inverted Pendulum Running: a Plant Model. Ph.D. thesis, University of Michigan, USA (1998). http://hdl.handle.net/2027.42/131537

  7. Uyanık, I., Ankaralı, M.M., Cowan, N.J., Saranlı, U., Morgül, Ö.: Identification of a vertical hopping robot model via harmonic transfer functions. Transactions of the Institute of Measurement and Control 38(5), 501–511 (2016). https://doi.org/10.1177/2F0142331215583327

    Article  Google Scholar 

  8. Shih, C.L.: Ascending and descending stairs for a biped robot. IEEE Transactions on Systems, Man and Cybernetics-Part A: Systems and Humans 29(3), 255–268 (1999). https://doi.org/10.1109/3468.759271

    Article  Google Scholar 

  9. Nishiwaki, K., Kagami, S., Kuniyoshi, Y., Inaba, M., Inoue, H.: Online generation of humanoid walking motion based on a fast generation method of motion pattern that follows desired zmp. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2684–2689 (2002). https://doi.org/10.1109/IRDS.2002.1041675

  10. Goswami, A.: Foot rotation indicator (fri) point: A new gait planning tool to evaluate postural stability of biped robots. In: Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), vol. 1, pp. 47–52. IEEE (1999). https://doi.org/10.1109/ROBOT.1999.769929

  11. Popovic, M.B., Goswami, A., Herr, H.: Ground reference points in legged locomotion: definitions, biological trajectories and control implications. The International Journal of Robotics Research 24(12), 1013–1032 (2005). https://doi.org/10.1177/2F0278364905058363

    Article  Google Scholar 

  12. Ankarali, M.M., Saranli, U.: Stride-to-stride energy regulation for robust self-stability of a torque-actuated dissipative spring-mass hopper. Chaos: An Interdisciplinary Journal of Nonlinear Science 20(3), 033121 (2010). https://doi.org/10.1063/1.3486803

    Article  MATH  Google Scholar 

  13. Kerimoğlu, D., Morgül, Ö., Saranli, U.: Stability and control of planar compass gait walking with series-elastic ankle actuation. Transactions of the Institute of Measurement and Control 39(3), 312–323 (2017). https://doi.org/10.1177/2F0142331216663823

    Article  Google Scholar 

  14. Spröwitz, A., Tuleu, A., Vespignani, M., Ajallooeian, M., Badri, E., Ijspeert, A.J.: Towards dynamic trot gait locomotion: design, control, and experiments with cheetah-cub, a compliant quadruped robot. The International Journal of Robotics Research 32(8), 932–950 (2013). https://doi.org/10.1177/2F0278364913489205

    Article  Google Scholar 

  15. Sproewitz, A., Moeckel, R., Maye, J., Ijspeert, A.J.: Learning to move in modular robots using central pattern generators and online optimization. The International Journal of Robotics Research 27(3–4), 423–443 (2008). https://doi.org/10.1177/2F0278364907088401

    Article  Google Scholar 

  16. Crespi, A., Ijspeert, A.J.: Online optimization of swimming and crawling in an amphibious snake robot. IEEE Transactions on Robotics 24(1), 75–87 (2008). https://doi.org/10.1109/TRO.2008.915426

    Article  Google Scholar 

  17. Aoi, S., Tsuchiya, K.: Stability analysis of a simple walking model driven by an oscillator with a phase reset using sensory feedback. IEEE Transactions on Robotics 22(2), 391–397 (2006). https://doi.org/10.1109/TRO.2006.870671

    Article  Google Scholar 

  18. Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Networks 21(4), 642–653 (2008). https://doi.org/10.1016/j.neunet.2008.03.014

    Article  Google Scholar 

  19. André, J., Teixeira, C., Santos, C.P., Costa, L.: Adapting biped locomotion to sloped environments. Journal of Intelligent & Robotic Systems 80(3–4), 625–640 (2015). https://doi.org/10.1007/s10846-015-0196-0

    Article  Google Scholar 

  20. Santos, C.P., Alves, N., Moreno, J.C.: Biped locomotion control through a biomimetic cpg-based controller. Journal of Intelligent & Robotic Systems 85(1), 47–70 (2017). https://doi.org/10.1007/s10846-016-0407-3

    Article  Google Scholar 

  21. Liu, C., Yang, J., An, K., Chen, Q.: Rhythmic-reflex hybrid adaptive walking control of biped robot. Journal of Intelligent & Robotic Systems 94(3–4), 603–619 (2019). https://doi.org/10.1007/s10846-018-0889-2

    Article  Google Scholar 

  22. Ijspeert, A.J., Kodjabachian, J.: Evolution and development of a central pattern generator for the swimming of a lamprey. Artificial Life 5(3), 247–269 (1999). https://doi.org/10.1162/106454699568773

    Article  Google Scholar 

  23. Nakamura, Y., Mori, T., Sato, M.A., Ishii, S.: Reinforcement learning for a biped robot based on a cpg-actor-critic method. Neural Networks 20(6), 723–735 (2007). https://doi.org/10.1016/j.neunet.2007.01.002

    Article  MATH  Google Scholar 

  24. Matsuoka, K.: Mechanisms of frequency and pattern control in the neural rhythm generators. Biological Cybernetics 56(5), 345–353 (1987). https://doi.org/10.1007/BF00319514

    Article  Google Scholar 

  25. Maeda, Y., Ito, A., Ito, H.: Central pattern generator and its learning via simultaneous perturbation method. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2012). https://doi.org/10.1109/IJCNN.2012.6252803

  26. Park, C.S., Hong, Y.D., Kim, J.H.: Evolutionary-optimized central pattern generator for stable modifiable bipedal walking. IEEE/ASME Transactions on Mechatronics 19(4), 1374–1383 (2013). https://doi.org/10.1109/TMECH.2013.2281193

    Article  Google Scholar 

  27. Taga, G., Yamaguchi, Y., Shimizu, H.: Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biological Cybernetics 65(3), 147–159 (1991). https://doi.org/10.1007/BF00198086

    Article  MATH  Google Scholar 

  28. Lu, Q., Tian, J.: Research on walking gait of biped robot based on a modified cpg model. Mathematical Problems in Engineering 2015, 9 (2015). https://doi.org/10.1155/2015/793208

    Article  MathSciNet  MATH  Google Scholar 

  29. Lewis, F.L., Jagannathan, S., Yesildirak, A.: Neural Network Control of Robot Manipulators and Non-linear Systems. CRC Press (1998)

  30. Zurada, J.M.: Introduction to artificial neural systems, vol. 8. West Publishing Company St, Paul (1992)

    Google Scholar 

  31. Haykin, S.: Neural Networks and Learning Machines, 3/E. Pearson Education India (2010)

  32. Pearlmutter, B.A.: Gradient calculations for dynamic recurrent neural networks: a survey. IEEE Transactions on Neural networks 6(5), 1212–1228 (1995). https://doi.org/10.1109/72.410363

    Article  Google Scholar 

  33. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  34. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, vol. 25, pp. 1097–1105 (2012). http://kr.nvidia.com/content/tesla/pdf/machine-learning/imagenet-classification-with-deep-convolutional-nn.pdf

  35. Çatalbaş, B., Çatalbaş, B., Morgül, Ö.: Human activity recognition with different artificial neural network based classifiers. In: 2017 25th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE (2017). https://doi.org/10.1109/SIU.2017.7960559

  36. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017). https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf

  37. Mesnil, G., He, X., Deng, L., Bengio, Y.: Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. In: Interspeech, pp. 3771–3775 (2013). https://www.isca-speech.org/archive/archive_papers/interspeech_2013/i13_3771.pdf

  38. Sutskever, I., Martens, J., Hinton, G.E.: Generating text with recurrent neural networks. In: Proceedings of the 28th International Conference on Machine Learning, pp. 1017–1024 (2011). https://icml.cc/2011/papers/524_icmlpaper.pdf

  39. Hénaff, P., Scesa, V., Ouezdou, F.B., Bruneau, O.: Real time implementation of ctrnn and bptt algorithm to learn on-line biped robot balance: experiments on the standing posture. Control Engineering Practice 19(1), 89–99 (2011). https://doi.org/10.1016/j.conengprac.2010.10.002

    Article  Google Scholar 

  40. Çatalbaş, B.: Recurrent Neural Network Learning with an Application to the Control of Legged Locomotion. Master’s thesis, Bilkent University, Turkey (2015). http://hdl.handle.net/11693/30072

  41. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529 (2015). https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  42. Kingma, D.P., Ba, J.: Adam: a Method for Stochastic Optimization. arXiv:1412.6980 (2014)

  43. Çatalbaş, B., Morgül, Ö.: A new learning algorithm: sinadamax. In: 2019 27th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE (2019). https://doi.org/10.1109/SIU.2019.8806259

  44. Rostro-Gonzalez, H., Cerna-Garcia, P.A., Trejo-Caballero, G., Garcia-Capulin, C.H., Ibarra-Manzano, M.A., Avina-Cervantes, J.G., Torres-Huitzil, C.: A cpg system based on spiking neurons for hexapod robot locomotion. Neurocomputing 170, 47–54 (2015). https://doi.org/10.1016/j.neucom.2015.03.090

    Article  Google Scholar 

  45. Jaramillo-Avila, U., Rostro-Gonzalez, H., Camuñas-Mesa, L.A., Romero-Troncoso, Rd.J., Linares-Barranco, B.: An address event representation-based processing system for a biped robot. International Journal of Advanced Robotic Systems 13(1), 39 (2016). https://doi.org/10.5772/2F62321

    Article  Google Scholar 

  46. Guerra-Hernandez, E.I., Espinal, A., Batres-Mendoza, P., Garcia-Capulin, C.H., Romero-Troncoso, R.D.J., Rostro-Gonzalez, H.: A fpga-based neuromorphic locomotion system for multi-legged robots. IEEE Access 5, 8301–8312 (2017). https://doi.org/10.1109/ACCESS.2017.2696985

    Article  Google Scholar 

  47. Gutierrez-Galan, D., Dominguez-Morales, J.P., Perez-Peña, F., Jimenez-Fernandez, A., Linares-Barranco, A.: Neuropod: a real-time neuromorphic spiking cpg applied to robotics. Neurocomputing 381, 10–19 (2020). https://doi.org/10.1016/j.neucom.2019.11.007

    Article  Google Scholar 

  48. Wright, J., Jordanov, I.: Intelligent approaches in locomotion-a review. Journal of Intelligent & Robotic Systems 80(2), 255–277 (2015). https://doi.org/10.1007/s10846-014-0149-z

    Article  Google Scholar 

  49. Auddy, S., Magg, S., Wermter, S.: Hierarchical control for bipedal locomotion using central pattern generators and neural networks. In: 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pp. 13–18. IEEE, Oslo, Norway (2019). https://doi.org/10.1109/DEVLRN.2019.8850683

  50. Mandava, R.K., Vundavilli, P.R.: An adaptive pid control algorithm for the two-legged robot walking on a slope. Neural Computing and Applications 32, 3407–3421 (2020). https://doi.org/10.1007/s00521-019-04326-2

    Article  Google Scholar 

  51. Janczak, A.: Identification of Nonlinear Systems using Neural Networks and Polynomial Models: a Block-Oriented Approach, vol. 310. Springer Science & Business Media (2004)

  52. Çatalbaş, B., Çatalbaş, B., Morgül, Ö.: Two-legged robot system identification with artificial neural networks. In: 2020 28th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE (2020). https://doi.org/10.1109/SIU49456.2020.9302094

  53. Choi, H., Crump, C., Duriez, C., Elmquist, A., Hager, G., Han, D., Hearl, F., Hodgins, J., Jain, A., Leve, F., et al.: On the use of simulation in robotics: opportunities, challenges, and suggestions for moving forward. Proceedings of the National Academy of Sciences 118(1) (2021). https://doi.org/10.1073/pnas.1907856118

  54. Olaru, A.D., Olaru, S.A., Mihai, N.F., Smidova, N.M.: Animation in robotics with labview instrumentation. International Journal of Modeling and Optimization 9, 34–40 (2019). http://www.ijmo.org/vol9/680-RA05.pdf

  55. Liu, C.K., Negrut, D.: The role of physics-based simulators in robotics. Annual Review of Control, Robotics, and Autonomous Systems 4 (2020). https://doi.org/10.1146/annurev-control-072220-093055

  56. Rudenko, A., Palmieri, L., Herman, M., Kitani, K.M., Gavrila, D.M., Arras, K.O.: Human motion trajectory prediction: a survey. The International Journal of Robotics Research 39(8), 895–935 (2020). https://doi.org/10.1177/0278364920917446

    Article  Google Scholar 

  57. Xu, T., An, D., Jia, Y., Yue, Y.: A review: Point cloud-based 3d human joints estimation. Sensors 21(5), 1684 (2021). https://doi.org/10.3390/s21051684

    Article  Google Scholar 

  58. Bledt, G., Powell, M.J., Katz, B., Di Carlo, J., Wensing, P.M., Kim, S.: Mit cheetah 3: Design and control of a robust, dynamic quadruped robot. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2245–2252. IEEE (2018). https://doi.org/10.1109/IROS.2018.8593885

  59. Focchi, M., Del Prete, A., Havoutis, I., Featherstone, R., Caldwell, D.G., Semini, C.: High-slope terrain locomotion for torque-controlled quadruped robots. Autonomous Robots 41(1), 259–272 (2017). https://doi.org/10.1007/s10514-016-9573-1

    Article  Google Scholar 

  60. Nguyen, Q., Powell, M.J., Katz, B., Di Carlo, J., Kim, S.: Optimized jumping on the mit cheetah 3 robot. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 7448–7454. IEEE (2019). https://doi.org/10.1109/ICRA.2019.8794449

  61. Li, J., Nguyen, Q.: Force-and-moment-based model predictive control for achieving highly dynamic locomotion on bipedal robots. arXiv:2104.00065 (2021)

  62. Plestan, F., Grizzle, J.W., Westervelt, E.R., Abba, G.: Stable walking of a 7-DOF biped robot. IEEE Transactions on Robotics and Automation 19(4), 653–668 (2003)

    Article  Google Scholar 

  63. Vazquez, JA and Velasco-Villa, Martín: Numerical analysis of the sliding effects of a 5-DOF biped robot. In: 2011 8th International Conference on Electrical Engineering, Computing Science and Automatic Control, pp. 1–6. IEEE (2011)

  64. You, Z., Zhang, Z.: An overview of the underactuated biped robots. In: 2011 IEEE International Conference on Information and Automation, pp. 772–776. IEEE (2011)

  65. Barron-Zambrano, J.H., Torres-Huitzil, C.: Cpg implementations for robot locomotion: analysis and design. In: Robotic Systems-Applications, Control and Programming. IntechOpen (2012)

  66. Efe, M.Ö.: Neural network assisted computationally simple pid control of a quadrotor uav. IEEE Transactions on Industrial Informatics 7(2), 354–361 (2011). https://doi.org/10.1109/TII.2011.2123906

    Article  Google Scholar 

  67. Olah, C.: Understanding lstm networks (2015). http://colah.github.io/posts/2015-08-Understanding-LSTMs/. Accessed 17 Nov 2019

  68. Pineda, F.J.: Generalization of back-propagation to recurrent neural networks. Physical Review Letters 59(19), 2229 (1987). https://doi.org/10.1103/PhysRevLett.59.2229

    Article  MathSciNet  Google Scholar 

  69. Gomez, A.: Backpropogating an lstm: a numerical example (2016). https://medium.com/@aidangomez/let-s-do-this-f9b699de31d9. Accessed 17 Nov 2019

  70. Krogh, A., Hertz, J.A.: A simple weight decay can improve generalization. In: Advances in Neural Information Processing Systems, pp. 950–957 (1992). https://proceedings.neurips.cc/paper/1991/file/8eefcfdf5990e441f0fb6f3fad709e21-Paper.pdf

  71. Hobbelen, D.G.E., Wisse, M.: Limit cycle walking. In: Humanoid Robots, Human-like Machines. IntechOpen (2007)

  72. Hamzaçebi, H.: Analysis and Control of Periodic Gaits in Legged Robots. Ph.D. thesis, Bilkent University (2017)

  73. Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6(02), 107–116 (1998). https://doi.org/10.1142/S0218488598000094

    Article  MATH  Google Scholar 

  74. Meyes, R., Lu, M., de Puiseau, C.W., Meisen, T.: Ablation studies in artificial neural networks. arXiv:1901.08644 (2019)

Download references

Acknowledgements

We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research.

Funding

This work was supported by the Scientific and Technological Research Council of Turkey (TUBITAK) through project 120E104.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey

    Bahadır Çatalbaş & Ömer Morgül

Authors
  1. Bahadır Çatalbaş
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ömer Morgül
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Bahadır Çatalbaş: Conceptualization, Software, Data curation, Formal analysis, Investigation, Visualization, Writing - original draft, Writing - review editing. Ömer Morgül: Conceptualization, Methodology, Formal analysis, Resources, Writing - original draft, Writing - review editing, Supervision.

Corresponding author

Correspondence to Bahadır Çatalbaş.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Çatalbaş, B., Morgül, Ö. Two-Legged Robot Motion Control With Recurrent Neural Networks. J Intell Robot Syst 104, 59 (2022). https://doi.org/10.1007/s10846-021-01553-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01553-5

Keywords

Search

Navigation