Generalization of Reinforcement Learning Agents for Production Control

Abstract

In times of rapidly changing markets and increasing complexity the fast and precise adaption of production systems to new circumstances is key for the economic success of manufacturing companies. Given highly adaptable production systems, the control of these systems still has to be optimized for each configuration. One critical aspect is worker control. To enable an automatic control logic which is capable of adapting autonomously and anticipatory to new configurations of the production system, a combination of reinforcement learning (RL) with simulation is promising. Key to a successful implementation of RL in such a dynamic production system is that the RL agent is able to perform well independently of the present system configuration and not just in the one system he is originally trained in. This paper presents an approach to develop such generalized production control RL agents, which was tested and evaluated using a discrete event simulation of a real-world production system. The approach defines a methodology of hyperparameter tuning for generalization including training, agent selection and testing of the RL in independent configurations of the production system. The results indicate that the approach is very successful in creating a generalized production control RL agent, which is able to control the workers efficiently in various configurations of a production system and adapt rapidly to new circumstances.