Hybrid autonomous controller for bipedal robot balance with deep reinforcement learning and pattern generators

Highlights

• Bipedal Robots face the real-world problem of recovering from an abrupt push.

• Asynchronous combination of pattern generators with deep reinforcement learning for recovering after a push.

• Offline and onboard training capabilities of the controller.

• Ability to utilize the pattern generators after disconnecting the deep reinforcement network.

Abstract

Recovering after an abrupt push is essential for bipedal robots in real-world applications within environments where humans must collaborate closely with robots. There are several balancing algorithms for bipedal robots in the literature, however most of them either rely on hard coding or power-hungry algorithms. We propose a hybrid autonomous controller that hierarchically combines two separate, efficient systems, to address this problem. The lower-level system is a reliable, high-speed, full state controller that was hardcoded on a microcontroller to be power efficient. The higher-level system is a low-speed reinforcement learning controller implemented on a low-power onboard computer. While one controller offers speed, the other provides trainability and adaptability. An efficient control is then formed without sacrificing adaptability to new dynamic environments. Additionally, as the higher-level system is trained via deep reinforcement learning, the robot could learn after deployment, which is ideal for real-world applications. The system’s performance is validated with a real robot recovering after a random push in less than 5 s, with minimal steps from its initial positions. The training was conducted using simulated data.