Evolving complex robot behaviors

Communicated by Kaoru Hirota
https://doi.org/10.1016/S0020-0255(99)00078-XGet rights and content

Abstract

Building robots is a tough job because the designer has to predict the interactions between the robot and the environment as well as to deal with them. One solution to such difficulties in designing robots is to adopt learning methods. The evolution-based approach is a special method of machine learning and it has been advocated to automate the design of robots. Yet, the tasks achieved so far are fairly simple. In this work, we first analyze the difficulties of applying evolutionary approaches to synthesize robot controllers for complicated tasks, and then suggest an approach to resolve them. Instead of directly evolving a monolithic control system, we propose to decompose the overall task to fit in the behavior-based control architecture, and then to evolve the separate behavior modules and arbitrators using an evolutionary approach. Consequently, the job of defining fitness functions becomes more straightforward and the tasks easier to achieve. To assess the performance of the developed approach, we evolve a control system to achieve an application task of box-pushing as an example. Experimental results show the promise and efficiency of the presented approach.

Introduction

Building behavior-based control systems for robots has become a major alternative to the traditional robot design nowadays. Many behavior-based robots have been built in recent years, and this approach has been proven successful [1], [2], [3]. In contrast to the classical robot control (which divides the task of implementing intelligent behaviors along functional lines to form a sense–think–act cycle), behavior-based control performs task decomposition to structure the overall control mechanism as layers of behavior modules in an incremental manner. A behavior module is a task-achieving controller; it is a simple but complete computational mechanism which only uses sensory information related to this partial task to compute actuator outputs. To build a behavior-based system is to develop the behavior modules first, and then to design mediators to combine the individual results from different behavior modules into an unified command for actuators. By this design methodology, researchers are able to break the computational bottleneck, and successfully build robots acting in the real world in real time.

However, the extension of this approach to design complete autonomous robots for more complex tasks still present some challenges, as its initiator Brooks has already informed us [4]. The first concern is about how to code a single behavior module always capable of dealing with the information in the uncompletely known world to achieve a specific task. The second one is actually the action selection problem; it concerns how to decide, for a system including multiple behaviors to handle a variety of situations, which behavior or behaviors should be active at any particular time to achieve various tasks. All these problems will become even more challenging when the number of behavior modules involved increases to follow the increase in task complexity.

Both of the above problems are in fact caused by the difficulties in predicting complicated robot–environment interactions and in dealing with them. One solution to these problems is to adopt learning methods by that the robot is expected to self-improve its own behaviors to adapt to the different environmental situations it experiences, without explicit programming. Evolution-based algorithms, inspired by the Darwinian principle of selective reproduction of the fittest, are a specific kind of machine learning approach. They simulate the natural evolution process in which members of populations change their structures to adapt to the environment, in order to survive. The evolutionary algorithms have been proposed to synthesize (evolve) robot control systems [5], [6] and there are some successful results [6], [8], [9]. As can be seen, researchers in this application field have mainly focused themselves on exploring the power of evolutionary algorithms and on analyzing the dynamics of evolutionary mechanisms, so their experiments always tried to use the least built-in knowledge to evolve a monolithic control systems from the scratch (i.e., evolving the controller as a whole). Admittedly, the tasks achieved so far, such as obstacle avoidance or light seeking, are relatively simple.

To investigate how the evolutionary computation technique can be used to develop robot controllers for more complicated tasks, in this paper we first analyze the problems of direct use of this technique to solve complicated control tasks, and then present a practical approach at the intermediate level: instead of hand-coding a behavior-based system or insisting on a strong learning principle as others do, we take a distributed control architecture similar to the behavior-based system, meanwhile, use the evolutionary technique to learn the individual behavior modules and the coordination strategies between them. This will take the advantages of both approaches and mediate between the two to overcome their own disadvantages. To demonstrate the proposed approach can speed up the learning process and can be scaled up to develop control systems for more complex tasks, we use it to undertake an application task of box-pushing as an example. In addition, we also use the GP system to evolve monolithic controllers to solve the same task in comparison to our approach. The results prove that only our approach can successfully evolve controllers to fulfill the target task.

Section snippets

Related work

In general, the process of evolving control systems is similar to that of the traditional evolution-based work; but the major difference lies in that evolving controllers involves a robot (simulated or real). Evaluating a controller is to execute the corresponding control code on a robot in the experimental environment for a period of time and then to measure its performance. Based on the evolutionary technique used, research work in this field can be categorized into genetic algorithm

The difficulties

Generally speaking, the evolutionary approach is a kind of search-based method in which genetic operators are used in the hope of finding a satisfactory solution in a space; and the dimensionality of this space is determined by the number of basic elements constituting a controller. When the size of the controller is reasonably increased to match the increase in task complexity, the solution space will grow exponentially and then leads the search to be more and more difficult. This is

Evolving distributed task-achieving controllers

As described above, to evolve distributed control systems to achieve complex tasks, we intend to use the technique of task decomposition to break the overall tasks and use the GP techniques to evolve separate behavior controllers and coordinators for integration. In this section, we will describe the general structure of the control architecture corresponding to the task decomposition, and then present the genetic representation of the controller to be evolved.

The robot and simulator

The robot used in this work is the miniature mobile robot, Khepera [26] (Fig. 6). It has two wheels driven by two DC motors and both motors can revolve forward and backward independently. It is originally equipped with eight infra-red proximity sensors which can also be used in a different mode to measure ambient light around the robot. For the purpose of identification, the sensors are numbered from 0 to 7 as illustrated in Fig. 6.

The main goal of this paper is to demonstrate how our approach

Experiments and results

In our experiments, a fitness function is in fact a penalty function. This differs from the traditional GAs, but is often used in GP-based work. As we emphasized previously, in our approach, the fitness function for a lower-level task should be defined in terms of sensor and motor activities; but for a complex task, the fitness function can be directly defined at the high level without describing all of the subgoals involved. The experiments below will show how fitness functions at different

Conclusion

In this paper, we have indicated the challenges of extending the behavior-based approach to design control systems and have analyzed, from different points of view, the main difficulties in applying evolutionary techniques to synthesize robot controllers for complicated tasks. None of them are trivials, nor are they easy to achieve. In order to reduce the load of robot programmers, and to evolve distributed control systems efficiently, we proposed our approach at the intermediate level. Instead

References (27)

  • M. Dorigo et al.

    Robot shaping: developing autonomous agents through learning

    Artificial Intelligence

    (1994)
  • R.A. Brooks

    A robot that walks: emergent behaviors from a carefully evolved network

    Neural Computation

    (1989)
  • R. Pfeifer et al.

    Sensory–motor coordination: the metaphor and beyond

    Robotics and Autonomous Systems

    (1997)
  • L. Steels

    Building agents out of autonomous behavior systems

  • R.A. Brooks, Challenges for complete creature architectures, in: From animals to animats: Proceedings of the First...
  • R.A. Brooks, Artificial life and real robots, in: Proceedings of the First European Conference on Artificial Life,...
  • D. Cliff et al.

    Explorations in evolutionary robotics

    Adaptive Behavior

    (1993)
  • D. Floreano et al.

    Evolution of homing and navigation in a real robot

    IEEE Transactions on Systems, Man and Cybernetics

    (1996)
  • T. Gomi, A. Griffith, Evolutionary robotics – an Overview, in: Proceedings of IEEE International Conference on...
  • J.R. Koza, J.P. Rice, Automatic programming of robots using genetic programming, in: Proceedings of AAAI-92, 1992, pp....
  • C.W. Reynolds, Evolution of corrider following behavior in a noisy world, in: From animals to animats 3: Proceedings of...
  • S.J. Ross, J.M. Daida, C.M. Doan, T.F. Bersano-Begey, J.J. McClain, Variations in evolution of subsumption...
  • J.R. Koza

    Genetic Programming: on the Programming of Computers by Means of Natural Selection

    (1992)
  • Cited by (36)

    • An approach to evolve and exploit repertoires of general robot behaviours

      2018, Swarm and Evolutionary Computation
      Citation Excerpt :

      The path planner can choose the sequence of behaviours to execute based on the expected trajectory of each locomotion behaviour. These forms of control can be seen as hierarchical control, in which the goal task (such as navigating towards the target) is decomposed into simpler sub-tasks (for example, walking forwards, turning right) [32–34]. Duarte et al. [35–37] have extensively studied this form of control, and developed the hierarchical control synthesis approach, where control is evolved for each sub-task, and the final controller is then evolved hierarchically in a bottom-up fashion.

    • Modeling and control of flexible context-dependent manufacturing systems

      2017, Information Sciences
      Citation Excerpt :

      In fact, the programming task depends on so many factors, such as the number of components, environment size, concurrency, parallelism, behavioral shifting, etc., so that traditional paradigms for software development are usually unsuitable. The literature provides some alternatives, such as the use of concurrent and dynamic programming [10,11], interfaces [12], general-purpose control languages [13,14], computational intelligence [15,16], etc. Another alternative is to adopt high level model-driven strategies in order to express system behaviors and requirements [17–19].

    View all citing articles on Scopus
    View full text