Abstract

The decentralized model predictive control (DMPC) of multiple cooperative vehicles with the possibility of communication loss/delay is investigated. The neighboring vehicles exchange their predicted trajectories at every sample time to maintain the cooperation objectives. In the event of a communication loss (packet dropout), the most recent available information, which is potentially delayed, is used. Then the communication loss problem changes to a cooperative problem when random large communication delays are present. Such large communication delays can lead to poor cooperation performance and unsafe behaviors such as collisions. A new DMPC approach is developed to improve the cooperation performance and achieve safety in the presence of the large communication delays. The proposed DMPC architecture estimates the tail of neighbor's trajectory which is not available due to the large communication delays for improving the performance. The concept of the tube MPC is also employed to provide the safety of the fleet against collisions, in the presence of large intervehicle communication delays. In this approach, a tube shaped trajectory set is assumed around the trajectory of the neighboring vehicles whose trajectory is delayed/lost. The radius of tube is a function of the communication delay and vehicle's maneuverability (in the absence of model uncertainty). The simulation of formation problem of multiple vehicles is employed to illustrate the effectiveness of the proposed approach.

1. Introduction

To address the intervehicle communication loss/delay in a network of cooperative multiple vehicles, a previously developed delay-dependent decentralized model predictive control (DMPC) architecture in [1] is modified and extended to subsystems with generic class of dynamics, subject to intervehicle communication packet loss. The proposed decentralized model predictive control (DMPC) features two main contributions. The first contribution is the development of a new DMPC approach that estimates the trajectory of the neighboring vehicles for the tail of the prediction horizon which would otherwise not be available due to the communication losses leading to random communication delays. In this approach, the tail of the cost function is estimated by adding extra decision variables in the cost function.

A relatively small amount of existing works have investigated the implementation issues such as communication loss associated with the exchange of trajectory information, but so far a few works have proposed a systematic tail estimation process to compensate for large delays. For example, in [2–5], it is assumed that the neighboring vehicles remain stationary at the last delayed states broadcasted by them. Such assumptions may yield poor performance for large communication delays, because the stationary state vector is not necessarily an accurate estimate of a time-varying state vector. Similar issues are investigated in [6], where for the hardware implementation of a robust DMPC to wheeled robots, both computation and communication delays are considered. To account explicitly for the time delays a model of the vehicle is used to estimate their state vector when required. Then the uncertainty arising from this estimation is taken into account by accommodating that into the effective disturbance force used for constraint tightening. Also, in [7], the stability, feasibility, and computation issues of a distributed MPC are investigated. The main practical attraction of the proposed architecture in [7] is that it allows terminating the computations at the end of each sampling period even if convergence is not achieved; this feature facilitates the real-time implementation of DMPC.

The second contribution of this paper is an extension of the tube-based model predictive control (MPC) approach [8, 9] for the case of the large communication delays in order to guarantee the safety of the fleet against possible collisions during formation control problems. The concept of the tube MPC in the existing works [8, 9] is normally used to calculate a robust bound on the states due to model uncertainty, whereas in this paper the approach is used to calculated bounds on the exchanged neighbor trajectories when large intervehicle communication delays exist.

In [10], the wireless communication packet loss/delay is considered; once the packet loss/delay occurs, the previous available trajectory of the faulty vehicle—whose trajectory is delayed—is extrapolated to predict the future reference trajectory. Also, in [11], the communication failure in formation flight of multiple vehicles leads to break in the communicated messages that force the fleet to redefine the communication graph.

This paper is organized as follows. Section 2 presents a general formulation of the delay-free decentralized model predictive controller and the corresponding algorithm. In Section 3 a new delay-compensated DMPC (DC-DMPC) is developed to address the communication delays arising from communication losses. A safety guarantee method is also developed based on the concept of tube MPC. In Section 4 the proposed algorithms are tested through simulation of a leader-follower formation problem for a fleet of unmanned vehicles.

2. Decentralized Model Predictive Control

Consider a team of cooperative vehicles with uncoupled dynamics. Each vehicle is equipped with three main components: (1) measurement sensors, (2) communication device, and (3) computation resource. The measurement sensors of each vehicle measure its own states assuming full state measurement is available. The communication device is used to gather the information from the neighboring vehicles and communicate with human operators. Using the computation resource, each vehicle solves a decentralized optimization problem at each sampling time based on its current measured states (from sensors) and the neighbor’s predicted state trajectories (provided through communication). It is further assumed that there is no sensor error, model uncertainty, or communication noise. These assumptions allow focusing on the main problem concerning intervehicle communication loss/delay. The developed algorithms do not rely on such assumptions; in fact, these assumptions are not restrictive for extending the proposed algorithms to nonideal situations. It is worth mentioning that even if a perfect model of neighboring vehicle is available still there is a possibility of mismatch between the predicted neighboring vehicle’s trajectory and the actual ones, because the neighboring vehicles may use a particular communication topology which leads to different DRHC optimization problems and hence different solutions. This problem can be overcome only by a centralized approach which is beyond the approach of this paper, because in a centralized approach only one communication topology which connects all the team members exists. Then, it is intended to focus on such mismatches arising from decentralized nature of the problem rather than mismatches coming from model uncertainty, and so forth.

Moreover, the terms agent, vehicle, team, member, and subsystem bear the same meaning. And so do the terms group, team, fleet, and network.

2.1. Interaction Graph Topologies

Graph theory is a relevant mathematical tool for modeling and analyzing the interactions among the cooperative vehicles (see for instance, [12, 13]). Graph topology is described by two basic elements: nodes and edges. Nodes represent the vehicles, and an edge between two nodes denotes the interaction between those vehicles. The interaction graph is denoted by and represented as follows: where is the set of nodes (vehicles) and is the set of edges , with . Hence, considering a set of vehicles cooperating to perform a common mission, the th vehicle in the team is assigned to the th node of the graph. If an edge connecting the th and th node is present in , it means that the th and th vehicles have an interaction. This relation is termed as neighborhood for th and th vehicles, and it is said that(i)th and th vehicles are neighboring vehicles; (ii)th vehicle is a follower of the th vehicle; (iii)th vehicle is a leader of the th vehicle.

Also, it is assumed that the interaction graph is directed, that is, does not necessarily imply , unless it appears explicitly in . Further, let and denote the number of the leaders and followers of vehicle , respectively.

In this paper, the information exchange structure coincides the control structure; in fact, an edge indicates that (1) vehicle sends information on its planned trajectory to and (2) the cost function of vehicle is coupled to states of vehicle . Also, it is assumed that the interaction graphs have a particular fixed structure and is set by the human operator prior to the mission.

2.2. DMPC Notation and Terminology

With model predictive control, a cost function is optimized over a finite time window called the prediction horizon, . The first of the computed optimal inputs is applied to the plant. The reader is referred to [14] for a comprehensive review of MPC schemes.

In this paper, the DMPC architecture is formulated based on the quasi-infinite-RHC formulation [15] where the key is to tune the terminal cost so that the closed-loop MPC has the property of an infinite horizon controller. In this approach the states are driven to a neighborhood of the origin where it is a positively invariant set under a feedback terminal controller for linearized system.

The discrete timing is then shown by where . As mentioned for the DMPC scheme presented in this paper the neighboring vehicles exchange their predicted state trajectories to provide cooperation. However, as we will see later for the time periods when the updated leader’s state trajectory is not available, it is estimated by the followers. Hence, considering the trajectory of vehicles, those calculated by themselves and those estimated by followers, an appropriate notation is required to present all these possible trajectories (actual, predicted, and estimated). The possible state vectors are introduced as follows:(i): the actual state vector of the th vehicle at time step ;(ii): the state vector of the th vehicle at time , predicted by the th vehicle at time step , where ;(iii): the state vector of the th vehicle at time step , estimated by the th vehicle at time step where .

Further, the sequence of states over the prediction horizon is called the state trajectory of vehicle calculated by itself and is represented by : Then let the following represent the concatenated state and input trajectories of the leader(s) of the th vehicle at time : If the trajectory is defined on an interval which is different from by other vehicles, then the beginning and end times are indicated as , that is, where is the interval on which the trajectory is defined.

2.3. Delay-Free DMPC Formulation

In this section, a brief overview of a delay-free DMPC problem and its implementation are described. More details can be obtained from [6, 12, 16]. For the DMPC scheme presented in this paper, the predicted trajectories are exchanged instead of being estimated, thereby reducing the online computational time. Figure 1 shows the delay-free intervehicle communication between two neighboring vehicles and the information exchange at time step .

Figure 1 

The delay-free intervehicle communication between two neighbors.

In Figure 1, it is assumed that the current leader’s predicted trajectories are always available instantly to their followers. However, at least one step delay has to be considered as the current predicted trajectories are not available instantly, even if an infinite communication bandwidth is used. Then, the information set of the th vehicle for the case of communication delay-free DMPC is introduced as follows: where set contains the updated information available to the th vehicle at time step and is referred to the information set in this paper. This collects (1) the instant state vector of the th vehicle and (2) the concatenated state trajectory of neighbors calculated at the previous time step, .

For the particular case of formation control, the delay-free decentralized cost function for the th vehicle in the team at time is defined as follows: where and , , and are positive definite and symmetric matrices, is the state vector of target (waypoint) of vehicle , and is the vector of desired relative position between vehicles and .

2.3.1. Delay-Free DMPC Problem

Assume that the following describes the discrete-time dynamics of the (homogeneous) vehicle : Then, the delay-free DMPC problem is defined for any vehicle at time step as follows.

Problem 1. Delay-Free DMPC Problem . Calculate subject to (for ) where , , and denote the set of admissible states, inputs, and final states (terminal region), respectively, for the th vehicle.

2.3.2. Delay-Free DMPC Algorithm

Each vehicle at each sampling period solves the decentralized problem . The output of the optimization problem is the predicted input and state trajectories of vehicle . The generated state trajectory is sent to followers for cooperation purposes, and the first optimal input is applied to the individual vehicle . The following algorithm is presented for the online implementation of the delay-free DMPC problem above. The algorithm is formulated for the th vehicle as in Algorithm 1.

Initialization
For at steps 2 and 3, assume (because no information from leader(s) is available).

Algorithm 1 is repeated until the assigned targets are reached. The targets are assumed to be known and assigned to each agent a priori.

3. Delay-Compensated DMPC (DC-DMPC)

This section develops a new delay-compensated DMPC (DC-DMPC) approach. The safety issues are also discussed.

3.1. Delayed Cost Function and Information Set

Assume at time step the vehicle receives the information from neighbor with time-delay , where and , see Figure 2. Then, the term large communication delays denotes the time delays which are larger than the sampling period, that is, .

Figure 2 

Delayed intervehicle communication between two neighboring vehicles.

When a vehicle receives the delayed information from neighbors the information set is updated as follows (compare with (5)): The information set represents updated information available to the th vehicle at time step . It implies at time step that each vehicle has access to its own delay-free information and delayed information from neighbors. Then, the decentralized delayed cost function for all at time step is defined as follows (compare with (6)):

The communication delay affects the cooperation cost for the periods when the leader’s trajectory is not available. In fact, for the tail of the cost function during the leader’s trajectory is not available due to communication delay to calculate the cost function. More precisely, assume at time step the vehicle receives the information from neighbor with time-delay , where and . Then, the trajectory of neighbor for only the time interval is available to vehicle , while according to cost function of (6) vehicle needs the trajectory of neighbor for the entire interval . Hence for the portion the trajectory of is not available due to the delay (see Figure 3). When the time delay is small this lack of information is not important, but for large communication delays the tail of cost function during becomes large and as shown by simulations in Section 4 it can lead to poor performance and unsafe behaviors (see also [13, 17]). One remedy to this problem is proposed here by estimating the tail of the cost function by including extra decision variables in the cost function.

Figure 3 

Tail of the leader’s trajectory in the presence of large communication delay .

3.2. Delay-Compensated Cost Function

The main idea with the proposed cost function is that it replaces the missing information due to communication loss, with the decision variables. Then, the decision variables are used to approximate the missing information. More specifically, communication losses prevent access to updated leader’s trajectory, and then the available delayed neighbor’s trajectory is utilized. However, the delayed leader’s trajectories do not cover the entire time slot required by follower to calculate its cost function; hence, a delay-compensated cost function is proposed as follows (for the th vehicle in the team at time step ): where is calculated from (11) and The subscript “D” stands for “Delay”. The delay-compensated decentralized cost function includes two main parts.(1)The delayed cost which includes the cost of local vehicle and the cost of neighboring vehicles calculated from the delayed information. Literally, this part is calculated using the information available from communication (and measurement sensors for local vehicle).(2)The tail cost (13) which calculates the cost associated with the neighbors over the tail of the cost function during where information is not available due to the communication loss/delay. In fact, the added extra decision variables are used to calculate this part.

Although it is assumed that there is model uncertainty and communication noise, the lack of updated information due to communication loss will impose an uncertainty on the updated leader’s trajectory, because, in the event of communication loss, the followers will utilize the leader’s delayed trajectory, which may differ from the updated unavailable trajectories. Such uncertainty which is derived from the mismatch between updated and delayed trajectories may lead to poor performance of the decentralized DMPC.

3.3. Safety Guarantee Using Tube DMPC

For the particular case of formation control, consider a group of vehicles that are required (1) to keep certain relative positions (local objective) and (2) to visit a set of waypoints (global objective). In general, incorporating the collision avoidance constraint into the optimization problem of DMPC is not trivial because of the nonconvexity nature of the distance constraint. To address this problem, in [18] a hybrid rule-based extension of the decentralized model predictive control (DMPC) is proposed to avoid possible collisions. Also, in [19] a mixed integer linear programming (MLIP) approach is utilized to handle the nonconvex collision avoidance constraint using a DMPC architecture.

The problem becomes even more serious when the large communication delays exist because communication delay leads to a lack of updated information on the trajectory of neighboring vehicles and this can make the formation unsafe and put the team in jeopardy. Hence, a discussion on the collision avoidance is required.

In practice, normally the desired distance in formation control is chosen large enough to ensure safety against overshoots and uncertainties. In this paper another approach which is applicable to both delay-free and delayed situations is proposed. In this approach, instead of using an assumed trajectory for neighboring vehicles, a tube-shaped trajectory is assumed around the trajectory of leader, where the tube radius is the radius of the reachable set of leaders. This way, the leader is put in a protection zone where the followers cannot enter. The radius of tube is a function of maneuverability and communication delay; the smaller the communication delays the thinner the tube. Note that at the end the desired relative distance in the formation is chosen to be larger than the tube radius. This is the main distinction between this work and the cooperative or noncooperative approaches aiming at online estimation of the reachable set. In contrast, here the tube is calculated offline and is used for the online purposes.

The idea of tube MPC was first used to calculate a robust bound on the states for uncertain systems [8, 9].

To avoid conservative reachability sets (thick tubes) the maneuverability of each vehicle is restricted, and then the reachable set of each vehicle is calculated; in fact, the leader imposes an input constraint to its optimization problem, that is, at any time instant the computed input trajectories of leader do not deviate too far from the previous one.

Note that, although restricting the maneuverability can lead to less conservative results, it can be restrictive to application where an agile vehicle is needed. Hence, for applications where, for example, slow moving ground vehicles are used such as automated highway systems [20] or mobile robots [21, 22], this approach can be beneficial. In such application since aggressive maneuvers are not required, restricting the maneuverability is not prohibitive, and hence the proposed approach is well suited.

3.3.1. Tube Formulation and Calculation

This section presents a numerical method for offline calculation of the tube for subsystems with generic dynamics. Any other computational or analytical reachability set calculation method may be used for this purpose. The main advantage of the presented tube calculation algorithm is that it can be computed offline and does not impose significant online computation load. Besides, the available analytical bounds are usually too conservative and are not in the desired format for tube analysis (often a bound on the norm of states is available).

Figure 4 shows the tube ℍ around a nominal trajectory. The tube ℍ is formulated as follows (see also [23]): where represents the component-wise absolute value of vector and denotes the radius of tube around the trajectory of vehicle at time step .

Figure 4 

A tube around a nominal trajectory, of vehicle i.

The following theorem represents a method for calculating the tube for a generic class of systems. Roughly speaking, tube is a generalization of reachability set; the reachability set is normally calculated around an initial state; however, tube is calculated around a trajectory of state vector. In other words, the deviation from some nominal trajectory over a given time is called tube.

Theorem 1. Assume that at time step the step delayed trajectory of leader is available to vehicle . If then the trajectory of vehicle at time step belongs to the tube around ; the tube is formulated as: where represents the tube around the trajectory of vehicle , calculated by vehicle at time step . Also, denotes the tube radius and is the solution of the following maximization problem.

Problem 2. Given , calculate for all

Proof. At time step vehicle uses the input trajectory which yields the state trajectory , then the solution of the differential equation (8) is calculated as: But if vehicle uses the assumed input , then the trajectory will be as follows: Subtracting (18) from (19) yields

By finding the tube is determined. is the upper bound on and then is calculated by solving the following maximization problem.

Problem 3. Given , and calculate for all

In optimization Problem 3 a nominal trajectory is given as input. In other words the optimal value depends on any nominal trajectory. On the other hand, at any time step DMPC generates a new trajectory which is served as nominal trajectory in Problem 3. Hence, Problem 3 should be modified to be independent of any nominal trajectory and be applicable for a general trajectory . Hence, considering and as new decision variables in the optimization Problem 3 and shifting the time from to will lead to Problem 2. This way, the tube can be computed offline and used for online applications, and no online computation is required.

3.3.2. Tube Calculation Algorithm

The application of Theorem 1 allows calculating the tube around the trajectory of each neighboring vehicle. Assume that the control input at each sample time for vehicle is bounded as follows: where is a vector with appropriate dimension and is called the maneuverability vector. It allows restricting the maneuverability of each vehicle during the time segments when the actual predicted trajectory of vehicle may not be available to followers. The neighboring vehicles by considering such restrictions can compute a bound on the trajectories. Then, if at any time step vehicle receives the information from leader with steps time delay, that is, , then the results of Theorem 1 are applicable by finding . To find the input constraint (22) can be used sequentially as follows (the superscript j is dropped temporarily): Hence, Therefore, we have

Then , the radius of tube , is calculated from Problem 2 after calculating from (25). The output of Problem 2 is the trajectory of vector and will be used in the online DMPC algorithm.

The main benefit of using this procedure for tube calculation is that calculation of does not impose any online computation load as it can be computed offline. The only parameter which may be unknown prior to mission is the time delay . Hence, the tube can be computed for different delays, and the results are given to the DMPC controller as some tabulated data, to be used in online applications. In this way no online computation is imposed.

3.3.3. Nonconvexity Avoidance

Using a tube instead of a trajectory in the cost function (12) can lead to nonconvexity for the optimization problem due to the nonconvex nature of the tube. Thus, in order to avoid the nonconvexity, in the cost function (12), is not modified (or replaced by tube ), instead the desired relative position will be modified as follows: where is the tube radius at time step and is calculated from Problem 2. In fact, the margin is added to the desired distance to ensure the safety. Since is the relative position vector, is multiplied by the diagonal matrix to ensure that adding increases the magnitude of ; in fact is preserved, and extra margin is added.

3.4. Delay-Compensated DMPC Problem

The delay-compensated DMPC (DC-DMPC) problem is defined below at time step k for any th vehicle. The outputs of this decentralized optimization problem are (1) the input and (2) the state trajectory of the local vehicle over the prediction horizon and (3) the trajectory of neighboring vehicles during the tail of the cost function.

Problem 4. DC-DMPC Problem . Given calculate subject to (i)for : (9a)–(9c) (ii)and for and :(iii)and for : where constraints (9a)–(9c) are the same as in the delay-free DMPC problem and correspond to the trajectory for calculating the delayed cost function . Constraints (28a) and (28b) correspond to the tail cost function (13). Constraint (28c) is the terminal constraint and is the same as (9c) in . Moreover, constraint (28d) is added for safety guarantee purpose, where represents the component-wise absolute value of vector and is a vector with appropriate dimension and called the maneuverability vector. Constraint (28d) is called maneuverability constraints (such constraint is also known as compatibility constraints in similar literatures, [24, 25]). It allows restricting the maneuverability of each vehicle and hence calculating the tube as explained in Theorem 1. Roughly speaking, constraints (28a)–(28c) are added for cooperation performance improvement, and constraint (28d) is added for safety purpose.

Remark 2. Expressing equations (28d) in the norm form instead of component-wise absolute value leads to more conservative results in practice.

3.5. DC-DMPC Algorithm

Algorithm 2 is presented for the online implementation of the proposed DC-DMPC problem . The algorithm is formulated for the th vehicle; in fact, all vehicles run this algorithm during the mission simultaneously.

Initialization
For to at steps 2, 3, and 4 assume that . For to at step 7 assume (because no information from leader(s) is available).

This algorithm is a modified version of Algorithm 1 and handles the large communication delays; it also provides the safety guarantee by executing the step 3 using the tube DMPC approach.

Remark 3. In step  2 of Algorithm 2, to measure the time delays, it is assumed that all the vehicles are equipped with synchronous clocks. Then at each sampling time, each vehicle, in addition to its predicted trajectory, broadcasts the time it is calculated the trajectory. Hence, the knowledge on the amount of delay is not required a priori.

4. Simulation Results

In this section, the proposed approach is tested on the formation problem of a fleet of unmanned vehicles with the following 2DOF dynamics. This dynamics can capture the motion of a mini-hovercraft equipped with two sets of thrusters to control the side and forward motions; also, a small damping term is added as the typical resistance of water to hovercraft motion is small: where and represent the components of position vector in coordinate and and are their corresponding velocity components. The input vector is given by . This dynamics is discretized by sampling time . The MPC problem is solved using optimization toolbox of MATLAB. When Algorithm 1 is used a set of decision variables are used for each vehicle to model the control inputs (because each vehicle has only two control inputs). Once Algorithm 2 is used, two extra sets of decision variables are used; decision variables are used. It means the computation grows with communication delay.

In the first simulation example, it is desired to examine the effect of the tail cost added to the cost function. The simulation was run for two cases.(1)Using the cost function without the tail cost. In this case the control input is set to for the tail of the cost function. The extra decision variables for tail cost estimation are not included in the optimization problem.(2)Using the cost function with the tail cost. In this case the tail of the cost function (12) is estimated using the extra decision variables in the optimization problem .

The matrix penalties in the cost function are chosen as follows: (where is identity matrix), and then and. The final penalty matrix is calculated from the Lyapunov equation [15]. The optimization horizon is chosen to be. In all cases no disturbances, sensor noise, or model uncertainty is considered in the simulations in order to focus on the effect of the communication delay.