A Discrete Event Formulation for Multi-Robot Collision Avoidance on Pre-Planned Trajectories

,


I. INTRODUCTION
Multi-Robot Path Planning (MRPP) is a fundamental problem in robotics, whose objective is to move all agents to their respective goal through the same environment while taking into account safety constraints, such as collisions avoidance [1]- [4], and performance constraints, related to the travelling time [5]- [7]. The focus of this work is on centralized MRPP problems where the environment is discretized and the objective is to minimize the maximum traveling time among the robots. As this problem is PSPACE-complete [8] it is generally solved by heuristic approaches giving not optimal solutions. A solution assigns a trajectory to each robot, i.e., a set of movements and a time schedule. Thus, the non optimality of a solution may depend on the robots' movements and on the associated time schedule.
We address the problem of improving a given solution by computing new time schedules for the trajectories, which reduce the maximum traveling time among the robots while not changing the robots' movements and avoiding collisions.
The associate editor coordinating the review of this manuscript and approving it for publication was Shouguang Wang . Given a solution computed from an external path planner, we provide two approaches to compute an improved time schedule. The first approach is optimal with an high computational complexity; the second approach is heuristic with a polynomial complexity. A preliminary version of such a heuristic method was introduced in [9], while here the method is formalized and a proof is provided. To evaluate the proposed optimal and heuristic approaches, we use as reference path planner the algorithm we proposed in [10].
The main contributions of this paper are: (i) a Mixed Integer Linear Programming (MILP) problem formulation and (ii) a heuristic approach to minimize the maximum traveling time among robots; (iii) a characterization of the complexity of the proposed heuristic; (iv) numerical results which show the effectiveness of the proposed approaches.
After a brief review of related works in Section II, we present in Section III all relevant and necessary preliminaries to our discrete event formulation of the MRPP problem [9]. In Section IV the main problem under consideration is stated. In Sections V-VI we propose, respectively, a MILP formulation and a heuristic algorithm, whose complexity is discussed in Section VII. Finally, after the presentation of VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ numerical results in Section VIII, we give concluding remarks in Section IX.

II. RELATED WORKS
In the current literature, a wide variety of MRPP problem formulations are proposed under different working assumptions and by means of different mathematical tools. For background and theory on motion planning we refer the reader to [11] and to [12], [13] for comprehensive reviews. The focus of this paper is on centralized approaches, which can be divided into three classes: (i) translating the MRPP problem to other problems that are well studied in computer science; (ii) heuristic solvers; (iii) optimal solvers. In the following, we include a brief review of heuristic solvers belonging to the search-based and rule-based families of solvers having similar formulations as ours, to which the approaches proposed in this work can be possibly applied.
A notable example of search-based solvers is the Hieratchical Cooperative A * (HCA * ) [14]. Such solvers reduce the problem complexity by computing, for each robot, independent paths [15] [16] which are then coordinated to avoid all possible collisions. However, they cannot guarantee finding a solution in all cases [17], except under certain conditions [18], and cannot guarantee an optimal solution.
Graph-based MRPP problem formulations and corresponding rule-based MRPP solver [6], [19]- [22] can be traced back to [23]. As highlighted in [24] and more recently in [25], such solvers are guaranteed to return a feasible solution if there is one and have polynomial time complexity. In this formulation, the robots are confined to an arbitrary connected graph, where nodes model the partitioning of the environment and edges the allowed movements, and collisions arise if two robots move to the same vertex or along the same edge. In two of the most recent and relevant works [6], [25], it is shown that the problem of minimizing the makespan is NP-hard and several heuristics are introduced.
The main difference between these works and the one presented here is that they allow cyclic rotations of robots along fully occupied cycles since a robot can enter a position while another one is leaving it.

III. PRELIMINARIES
Event sequences and languages are a simple mean to describe the behaviour of a discrete event system. Their basic building blocks are events σ , which belong to an alphabet .
A string is a sequence of events s = σ 1 · · · σ m . The set of all strings of events in is denoted * : this set also includes the empty string ε. A language is a set of strings L ⊆ * . The concatenation of two strings s, t ∈ * is written as st.
A substring s of s is a string contained in s. Further, s[p : q] denotes the substring of s = σ 1 · · · σ m having σ p ∈ s as the first event and σ q ∈ s as the last event. A superstring s of s is a string which contains s . We denote the set of substrings of s as sub(s) and the set of superstrings of s as sup(s). Prefixes and suffixes are special cases of substring.
A prefix p of a string s is a substring of s that occurs at the beginning of s, while a suffix q of a string s is a substring that occurs at the end of s. We denote the set of prefixes of s as pre(s) and the set of superstrings of s as suf (s). It holds that pre(s) ⊂ sub(s) and suf (s) ⊂ sub(s).
System behaviors are modelled using automata. A finitestate automaton is a 5-tuple G = (X , , δ, x 0 , X m ) where X is a finite set of states, is a finite alphabet of events, δ : X × → X is the (partial) transition function, x 0 ∈ X is the initial state, and X m ⊆ X is the set of marked (or final) states. The transition function is extended to strings in * by letting, for all x ∈ X , δ(x, ε) = x and, for all s ∈ * and σ ∈ , δ(x, sσ ) = δ(δ(x, s), σ ). Furthermore, given a string s ∈ * , δ(x, s)! means that there exists a state y such that δ(x, s) = y, i.e., δ(x, s) is defined.
The language generated by an automaton G, or the closed behaviour of G, is L(G) = {s ∈ * |δ(x o , s)!}. The language accepted by an automaton G, or the marked behaviour of G,

IV. DISCRETE EVENT PROBLEM FORMULATION
We consider a set of n mobile robots positioned in a planar environment. The environment where the robots operate is partitioned into a square grid, each square cell is denoted as a position. As shown in Figure 1, Each robot is an automaton G g = (X g , g , δ g , x 0,g , X m,g ), defined as follows.
• X g is the set of states: a state x g,i belongs to this set if position p i can be visited by robot r g .
• g is the alphabet: an event σ = (g, p i , p j ) belongs to this set if there exists an available transition of robot g from position p i to position p j .
• δ g : X g × g → X g is the transition function: for all available transitions σ = (g, p i , p j ) we define x g,j = δ(x g,i , σ ).
• x g,0 is the initial state of robot g corresponding to a generic initial position.
• X g,m is the set of final states of robot g corresponding to generic final positions.
The overall system can be completely described by the 3-tuple (G, f , R), which we call System of Time-weighted Automata with Resources (STAR). First we introduce the concept of resource.
Definition 1 (Standard Set of Resources): Consider a set of n robots described by automata G g = (X g , g , δ g , x 0,g , X m,g ) for g ∈ {1, . . . , n} and let = n g=1 g be set of all events across all robots.
A resource is a set of events r ⊆ g which can not occur simultaneously.
The standard set of resources R of G = G 1 × · · · × G n contains: • A resource R g for each robot r g with g = 1, . . . , n such that all events of g are in that resource; • A resource P i for each position p i with i ∈ 1, . . . , N p such that all events (across all robots) which involve the position p i are in that resource.

Definition 2 (System of Time-weighted Automata with Resources): A System of Time-weighted Automata with
finite-state automaton obtained by parallel composition of n automata, each one describing a single robot; • f : → R + is a weight function which assigns to each event σ ∈ a positive real number f (σ ) which denotes the time required to execute that event; • R denotes the standard set of resources, as in Definition 1. We now give an example of two robots in an environment, their automata and their STAR.
Example 1: In Figure 1 it is shown an environment where the initial and final positions of robots r 1 and r 2 are denoted, respectively, as I1, I2 and F1, F2. On the sides, the automata G 1 and G 2 corresponding to robots 1 and 2 are depicted. We now show how to construct the corresponding STAR G = (G, f , R): composition of the automata where the single final state x m ∈ X m is the cartesian product to the single final states of automata G 1 and G 2 .
• f is a function which assigns a weight to all events σ ∈ corresponding to the time required by the robot to execute the movement. In this example robots always require exactly one time unit to move between two squares, thus it holds f (σ ) = 1, ∀σ ∈ .
• R is a set of resources defined as follows. Each robot r g is modeled as a resource R g such that for each feasible transition of the robot, its corresponding event is in that resource. Each position p j is also modeled as a resource P j such that for each feasible transition (across all robots) which involves position p j , its corresponding event is in that resource. In this example, the set of resources is Given a STAR G = (G, f , R), a string s ∈ L m (G) denotes a sequence of movements which leads the robots from their initial position to their final position.
The projection π r (s) of string s over r ∈ R is a new string where events not belonging to resource r are removed.
Thus, π R g (s) denotes the sequence of movements of robot g. Events in a string are executed depending on their order. The main problem addressed by this paper is to determine a schedule for the events in s (allowing events to be executed at the same time) such that robots do not collide.
A schedule of s for G is an ascending ordered list of nonnegative real numbers ρ = [t 1 , . . . , t m ] ⊂ R m ≥0 , with t k establishing the start of the execution of the event σ k . We denote P(s) the set of schedules of s.
A schedule is a sequential schedule if for all k = 1, . . . , m it holds that t 1 = 0 and t k+1 = t k + f (σ k ).
Thus, when a schedule ρ = [t 1 , . . . , t m ] is associated to a string s = σ 1 · · · σ m , each event σ k ∈ s, for k = 1, . . . , m, has a starting time t k , an event execution time f (σ k ) and a completion time t k +f (σ k ). The events are executed following the order in the string s but their execution can be overlapped in time. The sequential schedule represents the classic way to execute events in a discrete event system, i.e., no two events are executed simultaneously. In this context, the sequential schedule implies that each robot moves while the others are standing. In this work we deal with strings for which the sequential schedule does not give a collision, i.e., no two robots occupy the same poisition a the same time. We call them valid strings, as follow. VOLUME 8, 2020 Definition 5 (Valid String): Let G = (G, f , R) be a STAR with G = G 1 × · · · G n being the concurrent composition of n robots and x 0 = (x 1,0 , · · · , x n,0 ) be the initial states.
The string s ∈ L(G) is a valid string if for all prefixesŝ of s, and for all couples g, h ∈ {1, . . . , n}, it holds that i.e., no two robots occupy the same position after the firing of any prefix of s. The set of all valid strings of G is denoted Given a valid string, for all other schedules which are not the sequential one, one wants to characterize the ones which do not give rise to collision. We call them collisionfree schedules.
Definition 6 (Collision-Free Schedule): For any valid string there exist infinitely many collisionfree schedules: first, there exists at least one, which is the sequential one; second, delaying the firing time of all events in the sequential schedule by the same quantity results in a new valid schedule. When a collision-free schedule is associated with a string, we can define the makespan of a string, which is the time needed to execute all events in the string with respect to the given schedule.
The makespan of a string s given a schedule ρ is the largest completion time among all events in the string If s = ε, i.e., the empty string, we let τ ρ (ε) = 0.
Finally, it is of interest defining the best collision-free schedule with respect to the resulting makespan of a given string. The smallest makespan among all possible collisionfree schedules is called strict makespan.
The strict makespan of the string s is If s = ε, i.e., the empty string, we let υ(ε) = 0. Given robots' trajectories s g ∈ V(G g ), there can exists several strings s ∈ V(G) that give trajectories s g , i.e., π R g (s) = s g for all g ∈ {1, . . . , n}. We thus define the trajectory invariant set of a string.
Definition 9 (Trajectory Invariant Set): Let G = (G, f , R) be a STAR and s ∈ V(G) a valid string.
The trajectory invariant set of the string s is defined as Given a STAR G = (G, f , R), we point out that two valid strings s 1 , s 2 ∈ V(G) such that s 2 ∈ S(s 1 ), can have different strict makespans since the events in a string can be fired only following their order in the string (see Example 2). Because of this fact, given a valid string s 1 , we address the problem of computing a new valid string s 2 obtained by shuffling events of s 1 while preserving robots' trajectories, i.e., s 2 ∈ S(s 1 ), such that s 2 has the lowest strict makespan among all strings in S(s 1 ). In other words, the trajectory of each robot g is fixed as π R g (s 1 ), but the order of events of different robots are adjustable via shuffling, aiming for a minimum strict makespan while avoiding collisions.
Compute among all valid strings in S(s), the one whose strict makespan is minimum, i.e.,

V. OPTIMAL SOLUTION
To solve Problem 1, given a valid string s, one can directly compute the earliest time each event in s can be fired and then construct the new valid string based on this schedule. This consideration translates the problem into Problem 2. Problem 2 (Optimal Schedule): Let G = (G, f , R) be a STAR, let s = σ 1 · · · σ m ∈ V(G) ⊂ L m (G) be a valid string and let s g = π R g (s) = σ 1 g σ 2 g · · · σ m g g , for all g = 1, . . . , n. Compute schedules ρ g = [t 1 g , t 2 g · · · t m g g ] for all robots g = 1, . . . , n such that, defined (i) A scheduleρ by sorting elements of ρ g for g = 1, . . . , n in ascending order; (ii) A strings by sorting elements of s g for g = 1, . . . , n with respect toρ, it holds that 1) The strings is a valid string, i.e.,s ∈ V(G); 2) The makespan τρ(s) ofs givenρ is equal to the lowest stric makespan among all the valid shufflings of s, i.e., Proposition 1: Solutions to Problem 1 and Problem 2 are equivalent.
Proof: In Problem 1 the solution is a string s * while in Problem 2 the solution is a set of schedules ρ g for g = 1, . . . , n. They are equivalent in the sense that the strinḡ s, obtained as explained in Problem 2, has the same optimal makespan as the string s * , i.e., υ(s) ≡ υ(s * ).
An optimal solution to Problem 2 (and consequently to Problem 1 by Proposition 1) can be computed by the MILP Model in Proposition 2. Note that the constraints considered in equation (2) are defined in following, within the proof of the proposition..

Proposition 2 (MILP Model):
Let (G, f , R) be a STAR, s = σ 1 · · · σ m ∈ V(G) be a valid string and let us denote the projection of s on robot g as s g = π R g (s) = σ 1 g σ 2 g · · · σ m g g , for all g = 1, . . . , n. Consistently, for each event σ k g in the string s we define a variable t k g which refers to its firing time. allows to compute schedules ρ g = [t 1 g , t 2 g · · · t m g g ] which are solution to Problem 2.
Proof: We will go through a detailed explanation of all the constraints and the objective function, with the aim of showing that constraints set (2) defines the set of collision free schedules for the considered problem and that the objective function is optimized by the optimal schedule.
Objective function: The objective function of (2) is the makespan of the string infered from the firings time, consistently to contidion 2) of Problem 2.
Positivity constraints: We consider the first moment a robot starts moving as t = 0, then all starting times are in R ≥0 . Thus, we have the following set of constraints (3) Robot constraints: Events belonging to the same robot must be executed subsequently with respect to their order in s. Thus, we have the following set of constraints i.e., event σ k+1 g has to start after the completion of event σ k g . Position constraints: Each position occupied by a robot can be the initial, final or intermediate. For each category, we define a set of constraints.
• Initial: Each robot has to leave its initial position before all other robots enter it. This means that the first event of each robot must be executed (and terminated) before all events of other robots which involve that position. Thus, call P g i the resource of initial position of robot g, we have the following set of constraints ∀g ∈ [1, n], ∀σ k j ∈ s such that σ ∈ P by linear constraints we refer to [26]. Let δ 1 , δ 2 be two logical variables associated to literals X 1 , X 2 , i.e., X 1 ⇔ [δ 1 = 1] and X 2 ⇔ [δ 2 = 1]. Therefore, require X 1 ⊕ X 2 is equivalent to require → R is a linear function, and assume that x ∈ X with X a given bounded set and define U = max x∈X f (x), L = min x∈X f (x) and ε is a small tolerance, e.g. the machine precision. Thus, and L = −|s|+1. Furthermore, we name δ 1 = δ h,g and δ 2 = δ l,k h,g . Thus, we have the following set of constraints for each position and each couple of robots g, h ∈ [1, n] passing through that position Algorithm 1 : HeuristicShuffle(s, f , R, N) Input : A valid string s = σ 1 . . . σ m ∈ V (G), a transition weight function f : → R, a set of resources R ⊆ 2 an integer number N ∈ N. Output: A valid string s Set (H , s,ŝ,q)

VI. HEURISTIC SOLUTION
On the other hand, to solve Problem 1 one can compute all the strings which can be obtained by shuffling events in s, compute their minimal schedule and finally select the one with the smallest strict makespan. This approach would be much more complex because of its combinatorial nature. The main idea underlying the proposed heuristic approach, given in Algorithm 1, is to shuffle events in the string s in a smart way, selecting only shuffles which result in a new valid string.
A previous approach called CompressTrace [27] -used in our algorithm -operates an optimal rearrangement without swapping two events involving the same resource, i.e., if s = CompressTrace(s) for each resource r ∈ R it holds that π r (s) = π r (s ). Our algorithm relaxes this constraint for resources associated to positions. Thus, it allows π P j (s) = π P j (s ) for a generic position p j with j = 1, . . . , N p and associated resource P j . This relaxation allows robots having a common path to change their priority of access to it, which is the intrinsic idea of the proposed approach.

A. GENERAL IDEA
Given a valid string s, Algorithm 1 allows to identify all subsequencesŝ containing N events which can be shifted backwards in positionq while resulting in a new valid string. This is done at lines 1 − 7. A subsequence which can be possibly shifted backward in the string is called a dense subsequence.
A dense subsequence of s is any substringŝ of π R g (s), where R g is the resource of robot r g .
After identifying all subsequence-index couples, Algorithm 1 performs all the shifts and, if there is, returns the one with the smallest strict makespan. This is done at lines 8 − 12. In order to better understand which kind of couples subsequence-index the algorithm is able to find and how a shuffle can effectively improve a string, an example is given.

B. CORRECTNESS OF THE APPROACH
In the following we first state and prove conditions for a dense subsequence to be shifted in a specific position and then prove that Algorithm 1 outputs a valid string based on these conditions. Proposition 3 (Shift of a Dense Subsequence Within a String): Let G = (G, f , R) be a STAR, let x 0 = (x 1,0 , · · · , x n,0 ) be the initial state of all robots, let s = σ 1 . . . σ m ∈ V(G) be a valid string, letŝ = σ p . . . σ q be a dense subsequence of s of robot g, letq be an index such that q < p. Define such that the string obtained by the shift is s = s iŝs . Three conditions are defined: (1) there exists an event σ ∈s such that σ ∈ R g .
Then s ∈ V (G). Proof: Given a dense subsequenceŝ = σ p · · · σ q of the string s, Algorithm 1 selects an indexq by using functions First and Last at line 5 and then validates it by function Middle R . It is necessary to ensure that such indexes are feasible, in the sense that the new string obtained by shifting backŝ after σq results in a valid string. By Proposition 3 we know that if a couple (ŝ,q) satisfies conditions (1), (2a) and (2b), then the new string it is a valid string. We point out that:  HeuristicShuffle(s, f , R, N ) is a valid string, i.e. shiftinĝ s in positionq + 1 leads to a valid string.

VII. COMPLEXITY ANALYSIS
In this section the complexity of Algorithm 1 is analyzed.
The first for-loop is executed m times, where m is total number of robots' movements. We approximate each robots' journey with the maximum shortest distance between any two positions in the environment, which, considering a square grid without obstacles with side length N p , leads to m ≈ n 2N p . The projection operated at line 2 requires q iterations, while the slicing operated at line 4 requires N iterations. At line 5 the maximum between two values is taken, which has constant time complexity, while the two called functions First and Last have both O(m) complexity. Then, at line 6, the if-condition calls function Middle whose complexity is O(m·n). Since all these operations are executed serially, the overall loop complexity is equal to O(m · n) since   q ≤ m and N ≤ m. Second for-loop is executed m times because in the worst case tests at lines 6 and 9 succeed and exactly m tuples are added in S. The most complex operation is CompressTrace whose complexity is O(m · (n + N p )), see [27]. The overall loop complexity is equal to O(m 2 ·(n+N p )) ≈ O(m 2 ·N p ). The slowest for-loop is the second because N p n, i.e., O(m 2 · N p ) ≈ O(n 2 N p ). Finally, considering the number of robots n be upper-bounded by the number of positions N p , the time complexity of Algorithm 1 is O(N 3 p ). As mentioned in Section II, most of rule-based algorithms to solve MRPP problems do not have optimality guarantees but their solution can be provided in O(N 3 p ) time, bound given by [23], equal to the time complexity of the proposed approach.

VIII. SIMULATION RESULTS
In order to test the effectiveness of the presented algorithm we generated random MRPP problems in three different environment: Cyclic Corridor, Closed Corridor, and Extensible Grid depicted in Figures 3, 4 and 5.
To generate a random problem instance with n robots we selected a random start and goal position amongst all the possible free positions in the environment, such that no two robot share the same start position, nor do they share the same goal position.  It is possible for a robot to have its start position as its goal position and another robot's start position as its goal position. Velocity of robots is constant but two robots may have different velocities. In particular, time required two move between two cell may vary between 1 second and 2 seconds, i.e., one robot can move at most twice as fast as another robot. For each setup (number of robots and environment) 50 different problems were generated and a solution to them is computed by [10]. For each problem to which a solution was found: 1) We solve Problem 2 with a MILP problem formulation in Proposition 2. When the complexity of the problem becomes computationally intractable, we exploit a lower bound for the optimal solution by choosing the maximum traveling time among all robots. 2) We applied Algorithm 1 to find a sub-optimal solution to Problem 1 and evaluated the distance to the optimal solution or to the lower bound for large problems. Average results of these simulations are depicted in Figures 6-7, where the x-axis represents the number of robots being coordinated and y-axis represents the average ratio of the execution time compared to our lower bound (or optimal solution when available).
Furthermore, we analyzed the time required by the optimal and sub-optimal approach (see Figure 8) by fixing the ratio between number of positions and robots at 20% and enlarging the size of the grid environment up to 11 × 11.

IX. CONCLUSIONS
Optimal and heuristic approaches to minimize the makespan of pre-planned robot's trajectories are proposed and proved to be guaranteed to return a feasible solution if there is one. The improvement is achieved through the computation of FIGURE 8. Chart showing average time required to compute an optimal solution through MILP in Proposition 2 and a sub-optimal through Algorithm 1 vs size of the environment whit a congestion equal to to 20%. We considered a failure when comptuing the optimal solution took more than 5 minutes: it happened when we tried to coordinate more than 10 robots. new time schedules by addressing the collision avoidance problem in a discrete event formulation [9], [10] of the MRPP problem, which makes use of time-weighted automata.
The heuristic algorithm was tested on a variety of problems and it was shown, both theoretically and by simulations, that its use always leads to a lower makespan, when it is possible, getting very close to the optimal value. When the optimal solution was not available, due to the complexity of the problem, we compared heuristic solutions to a specific lower bound. Such an heuristic approach does not require a significant amount of time with respect to the time required to compute an optimal or sub-otpimal solution.
The speed of the proposed heuristic strategy leads us to regard potential its use in a dynamic context, which would be a future prosecution of this work.