Transformed Successor Features for Transfer Reinforcement Learning

Abstract

Reinforcement learning algorithms require an extensive number of samples to perform a specific task. To achieve the same performance on a new task, the agent must learn from scratch. Transfer reinforcement learning is an emerging solution that aims to improve sample efficiency by reusing previously learnt knowledge in new tasks. Successor feature is a technique aiming to reuse representations to leverage that knowledge in unseen tasks. Successor feature has achieved outstanding results on the assumption that the transition dynamics must remain across tasks. Initial Successor feature approach omits settings with different environment dynamics, common among real-life tasks in reinforcement learning problems. Our approach transformed successor feature projects a set of diverse dynamics into a common dynamic distribution. Hence, it is an initial solution to relax the restriction of transference across fixed environment dynamics. Experimental results indicate that the transformed successor feature improves the transfer of knowledge in environments with fixed and diverse dynamics under the control of a simulated robotic arm, a robotic leg, and the cartpole environment.