A probabilistic finite state machine based strategy for multi-target search using swarm robotics

Highlights

• A mathematical model is established for the multi-target search in swarm robotics.

• A novel probabilistic finite state machine based search strategy (PFSMS) is proposed.

• The PFSMS has the highest efficiency and best stability in all presented strategies.

• With lots of robots, the efficiency of the PFSMS is close to the optimal value.

• The PFSMS strikes a good balance between parallelism and cooperative capability.

Abstract

As a distributed system, swarm robotics is well suited for the multi-target search task where a single robot is rather inefficient. In this paper, a model of the multi-target search problem in swarm robotics and its approximate mathematical representation are given, based on which a lower bound of the expected number of iterations is drawn. Two categories of behavior-based strategies for target search are introduced: one is inspired from swarm intelligence optimization while the other from random walk. A novel search strategy based on probabilistic finite state machine is put forward, showing the highest efficiency in all presented algorithms, which is very close to the optimal value in situations with a large number of robots. It has been demonstrated by extensive experiments that the novel strategy has excellent stability, striking a good balance between exploration and exploitation, as well as a good trade-off between parallelism and cooperative capability.

Introduction

Swarm robotics is a field inspired by the self-organized behaviors of social animals [1], aiming at designing a large number of simple robots to complete some complex tasks in a low-cost way with high reliability and efficiency [2], [3] or to simulate some expected collective behaviors [4], through local interactions among robots and between the robots and environment [5], in which people have done lots of various research and survey work [6], [7], [8], [9]. As a distributed system, swarm robotic system is well-suited for tasks involving area coverage [5], such as searching for multiple targets in a large space. Strategies for multi-target search have a broad prospect of application, such as hunting submarines [10], searching for victims and wreckage after air crash or shipwreck, monitoring the leak water quality [5], exploring and destroying battlefield targets, and so on.

Similar to the general foraging task, the search space in the multi-target search problem is very large. The large space emphasizes the importance of good exploration ability or high diffusion rate of the swarm. In addition, the influence scope of targets is also wide and robots can only perceive the local fitness information generated by targets (similar to radiation intensity but non-directional). The wide scope implies the necessity of good exploitation ability of robots [11]. For simplicity, the targets are static, and the fitness information will remain unchanged in the collecting process of the corresponding target, but will disappear immediately once the collection is finished. We may consider a more realistic scenario, such as the salvage task at sea. The people waiting for rescue are scattered in different areas, and a swarm of robots (ships or aircrafts) equipped with specific sensors (for life sign detection or other signals) are launched and instructed to search a designated region and find the people as soon as possible, and the signals will disappear once the people are saved.

The methods for designing collaborative mechanisms of swarm robotics can be divided into two categories: behavior-based design and automatic design [8]. Generally, behavior-based design is a bottom-up process, in which individual behaviors of robots are iteratively adjusted and tuned until the desired collective behavior is obtained. Automatic design methods mainly include reinforcement learning and evolutionary robotics, which can be considered as top-down approaches and generate behaviors automatically without the explicit intervention of the developer. Basically, the design methods of current strategies for multi-target search task can be classified as the behavior-based category [12], [13], such as methods based on artificial potential functions [14], [15], and methods adapted from some swarm intelligence optimization algorithms, such as Glowworm Swarm Optimization (GSO) [16], Particle Swarm Optimization (PSO) [17], Bee Swarm Optimization (BSO) [18], Fireworks Algorithm (FWA) [19], and Differential Evolution (DE) [20], [21].

In addition to swarm intelligence optimization, another important behavior-based perspective is using mathematical physics methods to model and analyze the foraging and migratory behaviors of animals, which is often referred to as “random search” [22] or “stochastic optimal foraging theory” [23]. Lévy flight [24] is a typical random search strategy, which can also be called Lévy walk [25] except for a negligible difference (In Lévy walk models, “jumps are not instantaneous but a time interval related to a finite velocity to complete the jump is involved” [23]). Random walk strategies can find many applications in swarm robotics [3], and we put forward a benchmark algorithm combining linear ballistic motion with triangle estimation technology [26]. Random walk strategy such as Lévy flight, bears excellent exploration ability (i.e. high diffusion rate), which is important for searching tasks in large space. Furthermore, if the targets are distributed sparsely and can be located directly by robots in their influence scope, then random walk strategy may be the only reasonable solution. In the problem scenario of this article, random walk strategies are generally used in areas without fitness information to help robots move quickly to other areas.

The probabilistic finite state machine (PFSM) is also a behavior-based design method for swarm robotics [8], which is convenient for describing various kinds of algorithms in an intuitive way. In this paper, the PFSM is used as an approach to describe, analyze and design the strategy. The motivation is from the previous work about triangular formation search [26], where robots in a team can roughly maintain a triangular formation to improve the exploitation ability of robots. However, the algorithm is rather complex, causing difficulties in maintenance and expansion, thus the disbandment and reorganization of the formation are not considered. In addition, to a certain extent, the triangular formation leads to excessive concentration of resources, limiting the exploration ability of robots. Therefore, we want to design a self-organized triangular formation strategy, in which the formation can be disbanded or reorganized freely. However, it is found unnecessary to maintain a formation explicitly, which will increase the computation and communication load. In the new strategy, there is no grouping operation nor rigid triangular formation, and only the triangle estimation technology is adopted. The results in [26] are probably not easy to relate to those in the current paper for two reasons: the parameters of RPSO strategy in [26] are not well tuned, and the IS strategy in [26] is enhanced with the inertia mechanism and corresponds to the BMS in current paper.

This article mainly includes three contributions. Firstly, an approximate mathematical model is established for the multi-target search problem in swarm robotics, based on which a lower bound of the expected number of iterations is derived. Secondly, three kinds of independent search strategies are introduced, each of which is combined by a kind of random walk strategy, the triangle estimation technology, and the inertia mechanism. Finally, a novel search strategy based on PFSM is proposed, showing the highest efficiency and the best stability in all presented algorithms. With a large population, the efficiency of the PFSMS is close to the theoretical lower bound of iterations for the problem. The PFSMS also strikes a good balance between exploration and exploitation, and a good trade-off between parallelism and cooperative capability.

The rest of the paper is organized as follows. In Section 2, both of two categories of searching algorithms and a formation search strategy are introduced. In Section 3, the model and analysis of the multi-target search problem in swarm robotics are stated. In Section 4, the behavior-based strategies proposed in this paper are described in detail. In Section 5, experimental results and discussions are presented. Finally, the work is concluded in Section 6.