Control of multi-agent systems: Results, open problems, and applications

2.1 Vehicular traffic

Vehicular traffic modeling started around 100 years ago with pioneering work such as that of Greenshields [10]. The complexity of the problem is often underestimated. In fact, traffic systems include important components to be considered:

1. Infrastructure, usually a road network.

2. Infrastructure regulating technology, e.g., traffic lights.

3. Physical agents, i.e., the vehicles traveling on the infrastructure.

4. Human agents, i.e., the drivers.

2.1.1 Microscopic models

Microscopic models are based on set of difference or differential equations for each vehicle in the traffic stream. Among the most popular models, we discuss the Follow-the-Leader (FtL) models [13], the intelligent driver model (IDM) [14], and the optimal velocity or Bando model [15].

Let x i and v i as the position and velocity of the ith driver, respectively, the general form of an model is given as follows:

(1) x ˙ i = v i , v ˙ i = ϕ ( v i , v i + 1 − v i ) ( x i + 1 − x i − l v ) γ ,

where ϕ is a modeling function, l v is the vehicle length, and γ is an integer exponent. The simplest choice is given by ϕ ( x , y ) = y and γ = 1 . Nice features of FtL models include their intuitive formulation and the avoidance of conflicting trajectories. On the other side, the follower’s acceleration will be zero as long as its speed is the same as the leader but independent from the headway. Thus, an extremely small headway may occur.

In the FtL family, the IDM can be formulated as follows:

(2) x ˙ i = v i , v ˙ i = a 1 − v i v 0 4 − s ( v i , v i + 1 − v i ) x i + 1 − x i − l v 2 ,

where a is the maximal acceleration, v 0 is the maximal speed, and s is a modeling function. IDM has been widely used in engineering work, but it presents a number of mathematical drawbacks, as illustrated in [16].

The Bando model has a different formulation focusing on the attempt of the driver to achieve an ideal velocity, which depends on the headway:

(3) x ˙ i = v i , v ˙ i = ( V ( Δ x i ) − v i ) ,

where the modeling function V ( ⋅ ) is called the optimal velocity function and, in the original model, was chosen as follows:

(4) V ( x ) = V max tanh ( x − l v d 0 − 2 ) + tanh ( 2 ) 1 + tanh ( 2 ) ,

where d 0 is the safety distance and V max is the maximum velocity.

The combination of Bando and FtL models [17] is amenable to reproducing the emergence of traffic waves observed in real traffic [11]. To illustrate this fact, we considered 22 vehicles on a ring road with dynamics given by a combination of the FtL model (1) and the Bando ones (3) with weights α = 0.5 m/s 2 for FtL and β = 20 s − 1 for Bando. To use realistic parameters, the maximal speed is set to 9.75 m/s and the vehicle length to l v = 4.5 m . The obtained simulation is reported in Figure 1 with one trajectory highlighted in red. One can observe the appearance and persistence of so-called stop-and-go waves in the famous experiment [18] performed in Japan.

Figure 1

A simulation of Bando-FtL combined model exhibiting the appearance and persistence of stop-and-go waves.

2.2 Crowd dynamics

The term crowd is used in different ways in the literature, including specifically referring to pedestrians, or more generally to groups of agents having social behavior. The latter includes decision-making processes taking into account an energy functional, as well as other criteria dictated by the interaction with other agents. To unify the perspective, here we first present this section with some models for pedestrians, then turn attention to animal groups.

2.2.2 Animal group models

Several models were proposed in the specialized literature on animal groups. Indicating again by x i the position of the ith agent, a general model takes the form:

(9) x ˙ i = ∑ j ∈ A i ϕ a ( x i , x j ) x j − x i ‖ x j − x i ‖ + ∑ j ∈ R i ϕ r ( x i , x j ) x i − x j ‖ x j − x i ‖ ,

where A i is the set of agents interacting with the ith agent by attraction, ϕ a is a modeling function depending on agent positions, and similarly for R i , ϕ r , and repulsion. Note that this model is first order since the velocity is prescribed directly. The choice is justified by the fact that, despite being called social forces, the agent is able to modify its speed almost instantaneously. The functions ϕ a and ϕ r may contain terms that depend on the angle between agents x i and x j (in two or three dimensions) and are related to the visual field of the animal under consideration. Such models are able to reproduce various self-organization patterns observed in nature, such as lines, crystal clusters, V shapes, and others (Figure 2). For example, elephants form lines, as well as lobsters on the ocean floor, while ducks form crystal clusters on rivers and cranes V shapes when migrating. We refer the reader to [35] and references therein for a more extensive discussion.

Figure 2

V-like and echelon formations obtained with model (9): large group traveling left to right (left) and zoom on a V-like formation (right).

While many models were considered by the biology community, a specific model attracted the attention of many applied mathematicians and physicists: the CS model [36]. Interestingly, the model was originally designed for the evolution of languages, but it reproduces a specific dynamic feature, called alignment, which is also typical of birds. The model for N agents with positions x i and velocities v i reads as follows:

(10) x ˙ i ( t ) = v i ( t ) , v ˙ i ( t ) = 1 N ∑ j = 1 N ϕ ( ‖ x j ( t ) − x i ( t ) ‖ ) ( v j ( t ) − v i ( t ) ) ,

where ϕ is modeling the dependence of attraction on the distance among agents. Usually, ϕ decreases as the distance among agents increases, as in the original choice ϕ ( r ) = 1 ( 1 + r 2 ) β with β > 0 . For β < 1 2 , the agents asymptotically align their velocity (Figure 3).

Figure 3

Left: time evolution of the CS model (10) with a ( s ) = 1 ( 1 + s 2 ) 1 3 in R 3 . Right: time evolution of D = ∑ i j ‖ x i − x j ‖ 2 and V = ∑ i j ‖ v i − v j ‖ 2 measuring position and velocity dispersion of the group.

2.3 Social dynamics

Social dynamics is a fairly general term, which may include crowd dynamics. Here, we are interested in presenting a simple example of opinion dynamics. The celebrated HK model [37] models the evolution of opinions, represented by a single parameter, in a group of agents interacting only with proximal neighbors. This characteristic is called bounded confidence and mathematically corresponds to restricting interactions of the ith agents only with those within a given radius, e.g., on the set { y : ‖ y − x i ‖ ≤ 1 } . This results in a dynamic that asymptotically converges to a finite number of clusters at a distance greater than or equal to 1. In formula:

The set of interacting agents changes with evolution, so the problem is indeed nonlinear and even discontinuous in time and space. Given the discontinuity in spaces, one has to carefully choose the appropriate concept of solution. The most common approach is to use Caratheodory solutions, but this implies non-uniqueness, and statements about the asymptotic states have to be formulated meticulously. We refer the reader to [38] for an extensive discussion and examples. The HK model gave rise to many variants and generalizations [39–41].

3.1 ODE-PDE models for moving bottlenecks

In vehicular traffic, a moving bottleneck can model a large bus or truck or a vehicle driving differently from an autonomous one. Some mixed-scale models coupling ODEs and PDEs were proposed, see [42,43], together with numerical schemes [44–46] and various extensions [47,48]. We focus on the approach proposed in [42,49,50] given as follows:

where f ( ρ ) = ρ v ( ρ ) , the control ω ∈ [ 0 , U ] indicates a speed chosen by the bottleneck, say an AV or a vehicle with autonomous cruise control, v ( ρ ( t , y ( t ) + ) ) = lim h → 0 , h ≥ 0 v ( ρ ( t , y ( t ) + h ) ) , and F α is given as follows:

The function F α measures the capacity drop given by the presence of the AV driving at a different speed, usually slower. To compute the value of the maximum in the definition of F α , we first consider the concave function f ( ρ / α ) = ρ α v ( ρ α ) (Figure 4). For every ω , let u ˜ ω be such that f ′ α ( ρ ) = ω , and set ϕ ω ( ρ ) = f α ( u ˜ ω ) + ω ( ρ − u ˜ ω ) . If y ˙ = ω , i.e., the AV drives slower than the traffic average speed, then F α = ϕ ω ( 0 ) = f α ( u ˜ ω ) − ω u ˜ ω . Otherwise, if y ˙ ( t ) = v ( ρ ( t , y ( t ) + ) ) , then the last inequality in (12) is obvious since the right-hand side vanishes. This model produces nonclassical shocks corresponding to values u ˆ ω and u ˇ ω defined as follows. Set I ω = { ρ : f ( ρ ) = ϕ ω ( ρ ) } , then u ˇ ω = min I ω and u ˆ ω = max I ω . We refer again to Figure 4 and [49,51] for details. Moreover, continuous dependence results are available for such a system [52,53].

Figure 4

Graphical representation of F α , u ˜ ω , u ˇ ω , and u ˆ ω .

3.2 Hybrid model for multilane traffic

Many of the classical models for traffic, as in Section 2.1, deal with single-lane roads or averages over different lanes. However, lane-changing occurs on highways, and other multilane roads, and is a big component of traffic phenomena. For instance, lane-changing is suspected to be a trigger of traffic waves, and is a cause of capacity drop effect [54].

Lane-changing maneuvers are usually either voluntary or mandatory, see [55–57] for details. We focus on the former type of lane change and consider two conditions for its occurrence:

• Safety: There should be enough space in the new lane to allow the maneuver, also considering the velocities of vehicles;

• Incentive: The new lane is less crowded or, in any case, has better traffic conditions.

Following [58], we focus on modeling the two conditions via constraints on acceleration. For a fixed lane k and vehicle i , we indicate by a i k ( t ) its acceleration at time t , and by a ¯ i k ′ ( t ) the expected acceleration if the lane is changed to k ′ (assuming k and k ′ are adjacent). The conditions can be stated as follows:

• Incentive condition: a ¯ i k ′ ( t ) ≥ a i k ( t ) + Δ ,

• Safety condition: a ¯ i k ′ ( t ) ≥ − Δ and a ¯ j ( i , k ′ ) k ′ ( t ) ≥ − Δ ,

The model obtained requires nontrivial notation to be detailed. In particular, a complete state of the system requires the use of mixed variables ( k , x k , v k ) ( t ) for each vehicle, where k is the lane and ( x k , v k ) , the position and velocity on lane k . Note that to specify an FtL-type dynamics, one has to reconstruct for each vehicle the position of the leader in the current lane. Finally, we obtain a hybrid stochastic system and refer the reader to [59] for details.

3.3 State space and asymptotic states in social dynamics

As mentioned in Section 2.3, the HK model was generalized in various ways. Since the model represents opinions, one may argue that more than one variable should be used to represent opinions on different issues. At the same time, opinions should belong to a compact set to be meaningful. Finally, a model with both features is obtained by projecting the HK model in dimension d + 1 to the d -dimensional sphere S d :

where a i j are interaction weights: positive for attraction and negative for repulsion. This model presents interesting dynamic features, including a new type of equilibria w.r.t. to the clusters of the HK model and new asymptotic states called dancing equilibria. To describe the latter, we introduce a piece of notation. From (14), one can compute

where ⟨ ⋅ , ⋅ ⟩ is the scalar product in R d + 1 . A trajectory along which the quantity (15) vanishes is called a dancing equilibrium. The vanishing of (15) ensures that the relative position of agents on the sphere does not change. Therefore, the trajectory could be obtained by fixing the agents at their initial position and rotating the whole sphere underneath. This behavior reminds us of cyclicity in many social systems, and the visualization of such phenomenon on a two-dimensional sphere is provided in Figure 5. We refer the reader to [60] for details. Extensions to general manifolds are discussed in [61,62].

Figure 5

The trajectories of 25 agents with a randomly generated sign-symmetric interaction matrix starting from t = 0 (left) and t = 0.03 (right).

We note that other interesting asymptotic behaviors emerge in systems with many agents interacting in a social fashion. For instance, deep analysis was provided for specific configurations, such as consensus (or unique cluster), flocking (or alignment of velocities), and milling [40,63–65].

3.4 Time-evolving measures

Many of the patterns observed in pedestrian dynamics are significantly complex, such as, for example, the lane formation in opposite flows. The latter appears without initial self-organization and emerges as a form of symmetry breaking after the two flows start to interact. This happens ubiquitously at pedestrian crossings, see [30] for details.

The attempt to capture this complexity motivated the use of measure solutions instead of classical function solutions to dynamics modeling pedestrians [66–68]. In simple terms, a group of pedestrians occupying an environment Ω is represented by a measure μ , which assigns to each (measurable) set E ⊂ Ω the number μ ( E ) quantifying pedestrians in E . This simple idea allows both continuum and discrete modeling, using either a μ that is absolutely continuous w.r.t. the Lebesgue measure (assuming Ω ⊂ R n for some n ) or an atomic measure. Usually, one assumed μ to be a Radon measure, but σ -additivity is the only necessary assumption to reflect the principle of additivity of the mass. For μ to evolve in time, one may use standard transport-type equations in the Eulerian setting, such as:

where v [ 0 , + ∞ ) × Ω → R n is the velocity at which the mass of pedestrians travels. As usual for a measure, the equation has to be interpreted in the weak sense, i.e., for every test function ϕ and a.e. t , we require

This model is first-order rather than second-order as the social force one, and it is convenient to represent the velocity v as the sum of two terms as follows:

where the dependence v [ μ t ] is of functional type, i.e., v is a map from the space of measures to the space of vector fields. The term v d represents a desired velocity, which depends only on the single pedestrian, and v i is an interaction term depending on the whole measure μ .

This framework is well connected to mean-field approaches [69], and it is convenient to use the Wasserstein distance [70], which is defined as follows. Denote by P ( R n ) the space of probability measures, then, for μ , ν ∈ R n , we set

where Π ( μ , ν ) indicates the set of transference plans between μ and ν , i.e., the measures on R n × R n such that:

The problem of finding the transference plan in (19) is called the optimal transport problem. We refer the reader to [70,71] for a complete treatment of the subject and historical remarks. In Figure 6, we show a simulation of crossing flows with a multiscale measure model. The presence of both the discrete and continuous parts allows a symmetry breaking, thus generating the lanes observed in a pedestrian crossing.

Figure 6

Simulation of a multiscale model for crossing flows.

4.1 Sparse controls

As mentioned earlier, the control of large systems requires the use of controls affecting a small number of agents. A useful concept is that of sparsity. In information theory [72], one looks for the most economical way to represent data, which corresponds to the use of minimal symbols from a dictionary, while, in compressed sensing [73], one looks for the representation of signals using the minimal number of coefficients of a fixed base. Similarly, we define sparse controls as the one using only a small number of agents.

A convenient way to promote sparsity is using ℓ 1 -type norms. More precisely, consider N agents, each evolving in R d , with an alignment dynamics as in (10). Define for every u , v ∈ R N d :

where ⟨ ⋅ , ⋅ ⟩ is the scalar product in R d . The form B is bilinear and can be used to define a variational problem to promote sparsity in the following way. First, fix a bound M for the control action and ( x , v ) ∈ R N d × R N d (providing the positions and velocities of the agents). Then, we define U ( x , v ) as the set of solutions to the following:

where

The coefficient γ measures the strength of future interaction, and if B ( v , v ) < γ , then the group tends to alignment without external intervention [74]. Thus, the controls in U ( x , v ) promote alignment, due to the term B ( u , v ) , and sparsity at the same time, due to the ℓ 1 terms in the cost and the constraint. Note that U ( t , x ) may be multivalued, thus generating a differential inclusion, but sample-and-hold techniques can be used to select solutions with controls that are piecewise constant in time. We refer the reader to [75,76] for details.

4.2 Controlling across scales and by leaders

A classical tool in gas dynamics is passing to the mean-field limit in microscopic models [69]. The limit consists of partial differential equations of Vlasov or Boltzmann-type [77]:

where H is the interaction kernel, so the microscopic model reads x ˙ i = v i , v ˙ i = H ⋆ μ N , with ⋆ the convolution and μ N = ∑ i δ ( x i , v i ) the empirical measure. See [78] for applications of mean-field techniques to multi-agent systems.

In the controlled case, one would obtain the following equation:

but the limit of control policies ν may be a singular measure w.r.t. to μ , thus having no effect on the dynamics. Imaginatively, it is like trying to steer a river by means of toothpicks!

A possible solution is to restrict the control policies so that in the limit one obtains an equation of the following type:

i.e., with ν absolutely continuous w.r.t. μ with Radon-Nikodym derivative d ν d μ = f . This approach was used in [79], assuming f to be Lipschitz continuous in ( x , v ) . While this is a severe restriction, it is possible to obtain a simultaneous Γ -limit and mean-field limit of the optimal control problems.

An alternative idea is to split the population into two groups, leaders and followers, and assume that only the former can be controlled [80]. The complete microscopic dynamics reads as follows:

where y k and w k are the positions and velocities, respectively, of the controlled leaders; u k are the controls; x i and v i , are the positions and velocities, respectively, of the followers; and μ m is the empirical measure of followers and leaders. We expect m to be much smaller than N , so every control policy will be essentially sparse by design. In Section 5, we will consider specifically the control of traffic via AVs. In that case, the ratio m N is called the penetration rate, and effective strategies should deal with the case of the penetration rate between 2 and 5%.

Optimal control problems for (24) can be stated as follows:

where U is the control set and L is the running cost or Lagrangian. The last term of ℓ 1 -type further promotes sparsity for controls already acting only on leaders [81–83].

In this setting, it is possible to define the mean-field limit of the controlled system by letting N → ∞ while keeping m fixed. The resulting system is of mixed type with fully coupled controlled ODEs and a Vlasov-type PDE:

Moreover, one can prove the Γ -convergence of the optimal control problem, so that optimal controls of (24) and (25) converge to solutions to the optimal control problem for (26) stated as follows:

These results can be used in two different ways: designing or computing optimal controls at the microscopic scale (equations (24) and (25)) and proving properties of the limiting system, or using numerical methods for the mean-field scale (equations (26) and (27)) and project over the microscopic one for real applications.

4.3 Avoiding black swans

In Section 3.3, we showed how the behavior of large groups of agents may result in rich asymptotic regimes. At the same time, the classical stabilization problem at the origin may be replaced with stabilization problems at a periodic orbit or another limit set representing the asymptotic behavior of the group. Often times, such limit sets are locally attractive, and Lyapunov-type techniques have been used [84]. Here, we present another innovative aspect: the avoidance of a limit set, called Black Swan. The latter, in the socioeconomic systems, represents a low-probability event with significant consequences for the system’s dynamics [85].

Following [86], we focus on the general HK-type system:

where N is the number of agents and u i are the controls. To fix the ideas, assume that a : ] 0 , + ∞ [ → ] 0 , + ∞ [ is locally Lipschitz with sublinear growth at infinity, so that solutions to (28) always exist (interpreting the case x i = x j for some i ≠ j appropriately). The interest is to understand if there exist controls, possibly sparse, that may avoid consensus, i.e., the manifold ℳ = { x 1 = ⋯ = x N } , which represents the Black Swan. For instance, in a stock market consensus to sell may generate a financial collapse. Four main results are stated in [86]:

Result 1. If lim s → 0 s a ( s ) = 0 , i.e., a does not blow up too fast for s → 0 , then for every M and initial condition x ( 0 ) , there exists (sparse) control strategy so that x ( t ) ∉ ℳ for every t > 0 . In simple words, using appropriate controls, one can prevent the Black Swan to occur.

Result 2. If lim s → 0 s a ( s ) = + ∞ , i.e., a blows up fast enough, then, for every M , there exists a Black Hole, i.e., a neighborhood N of ℳ so that if x ( 0 ) ∈ N , then the corresponding solution converges to ℳ in finite time. In simple words, independently of the strength of the controls, if the system gets close enough to a consensus then the Black Swan is unavoidable.

Result 3. If lim s → + ∞ s a ( s ) = 0 , i.e., a decreases sufficiently fast to 0 for s → 0 , then for every M , there exists a safety region, i.e., a neighborhood S of infinity so that if x ( 0 ) ∈ S , then there exists a (sparse) control keeping, the evolution uniformly far away from the Black Hole.

Result 4. Finally, if lim s → + ∞ s a ( s ) = + ∞ , i.e., a does not decrease to 0 fast enough for s → 0 , then for every M , there exists basin of attraction, i.e., a neighborhood ℬ of ℳ that attracts trajectories from every initial data.

The results stated above were also generalized to the mean-field case, i.e., for the equation:

where χ ω is the indicator function of the set ω , u = u ( x ) , ‖ u ‖ L ∞ ≤ M , and sparsity is modeled as ∫ ω ( t ) 1 ≤ c for some fixed c > 0 .

4.4 Bounded variation controls

Another innovative approach to design parsimonious controls is that of preventing chattering phenomena, i.e., the presence of an infinite number of switchings in optimal control problems. In [87], the author considered the simple optimal control problem:

and showed that the unique optimal control satisfies

where ( t k ) is the increasing sequence of switching times, from u = + 1 to u = − 1 or vice versa. This phenomenon was shown to be generic for single-input systems in higher dimensions [88].

It is desirable to design control policies that achieve costs close to the optimal one while having a finite number of switchings. To do so, consider the general optimal control problem:

where U is the set of admissible controls steering the system to 0 in time t ( u ) , L is the Lagrangian, and f is the dynamics. We assume that f and L are smooth, then we approximate the optimal control problems with the family:

where T V ( u ) indicates the total variation of u (in time). Therefore, if a control has a finite cost, it must have a finite total variation so a finite number of switching. To state the first result, we need some more notation. First, as usual, we indicate by Lie f the Lie algebra generated by the vector field { f ( ⋅ , u ) : u ∈ U } (where U is the control set), and by Lie f ( x ) its evaluation at the point x . We also recall that the system x ˙ = f ( x , u ) is small-time locally controllable at 0 if for any δ , the set of points reachable from 0 within time δ is a neighborhood of 0. We are ready to state the following:

In simple words, Theorem 1 states that every optimal control problem, which is sufficiently regular, admits approximations by control problems with a finite number of switchings. Under further regularity (see [89] for details), one can achieve an estimate of the convergence rate:

where x ∗ is an optimal trajectory for (31) corresponding to control u ∗ . These techniques are also useful to approximate optimal controls for hybrid systems and for problems with state constraints, see [89] for a detailed discussion and statements.

4.5 Control of time-evolving measures

As mentioned in Section 3.4, some problems are conveniently represented by time-evolving measures. The resulting transport-type equations have been studied in the framework of Wasserstein spaces, i.e., measure spaces endowed with Wassertein-type metrics. For instance, consider the nonlinear transport equation:

where v [ ⋅ ] highlights the functional dependence of the velocity field v on the measure μ , thus v : P → C ( R n , R n ) is a map from the space of probability measures on R n , endowed with the Wasserstein distance W (19), to the space of continuous vector fields on R n . In this setting, it is natural to state the following conditions:

1. For every μ , the vector field v [ μ ] is globally Lipschitz and bounded.

2. The map μ → v [ μ ] is Lipschitz for the metric W and the uniform norm ‖ ⋅ ‖ ∞ .

Extending this framework, one may consider control problems as follows. First, consider a set-valued map V : P ⇉ C ( R n , R n ) associating every probability measure with a set of locally Lipschitz vector fields. Then, one can define the differential inclusions in Wassertein space:

As usual for differential inclusion problems, a solution to (35) is a curve t → μ ( t ) for which there exists a measurable selection t → w ( t ) ∈ V ( μ ( t ) ) , so that (34) is satisfied with v [ μ ( t ) ] = w ( t ) . We refer the reader to [91] for the general theory and to [92] for a version of Pontryagin Maximum Principle in this setting. Along the same line of thought, some authors investigated the properties of value functions for optimal control problems [93,94]. A different route to control problems for measures was taken in [95–99], where the authors considered an alternative formulation as follows. Starting from a classical controlled dynamics x ˙ = f ( x , u ) , one can model a more general nonlocal controlled system as follows:

where v ( t , x ) ∈ { f ( x , u ) : u ∈ U } . This approach is more inherently connected to the microscopic dynamics x ˙ = f ( x , u ) than to the continuity equation (34).

Going back to (34), it is quite straightforward to generalize the approach to measures with a total mass different from one (as for probability measures). However, some problems require the addition of source terms, which would modify the total mass in time, as in:

Then, one has to modify the Wasserstein metric to compare measures with different masses. A possibility is to allow the addition/subtraction of mass at a fixed cost, which gives the metric:

where μ ˜ ≤ μ means μ ˜ ( A ) ≤ μ ( A ) for every measurable set A and ∣ ⋅ ∣ is the total variation norm (or total mass). In simple words, the distance W a , b considers all possible ways to arrange the mass of μ to the mass of ν by canceling mass with cost a per unit of mass or transporting mass with cost b per unit of mass. This distance allows the development of a theory for equations with sources or sinks [100,101]. Alternative approaches based on optimal transport were proposed, and we refer the reader to [102] for a discussion including applications to image processing and artificial intelligence.

5.1 A general view on equations for measures and limits

Building upon the modeling effort illustrated in Section 4.5, a new type of equation was recently proposed, called MDEs [104]. The main idea is to generalize the concept of ordinary differential equations on a manifold to measures. Let us start by noting that an ODE is defined by assigning a vector field V : M → T M , x → ( x , v ( x ) ) , and assigning to every point x ∈ M a tangent vector of the tangent bundle T M . Specifically, the map definition x → ( x , v ( x ) ) forces the tangent vector associated with x to belong to the tangent space T x M of M at the point x , i.e., to the fiber of T M over x . An alternative way to state this is to require that π ( v ( x ) ) = x , where π : T M → M is the projection over the base, i.e., π ( x , v ) = x . The natural immersion of M into P ( M ) , the space of probability measures over M , is given by x → δ x associating to every point the Dirac delta centered at the point. This immersion can be defined for every manifold, thus can be used to extend the concept of vector field to that of probability vector field (PVF) on M , which is a map V : P ( M ) → P ( T M ) associating with every probability measure on M , a probability measure on T M , with the only requirement π # V ( μ ) = μ , where # is the push-forward of measures. Given a PVF V , we can define an MDE as follows:

where a solution is interpreted in weak sense. This means that for every test function f , we have

Existence of solutions can be proved using standard assumptions on V of sublinear growth (of measure support) and continuity w.r.t. to Wasserstein metrics on M and T M . However, uniqueness can be recovered only at the level of semigroup using Lipschitz-type assumptions based on the functional:

where V , W ∈ P ( T M ) , Π is the set of transference plans and π 13 : T M × T M → M × M is the projection over the first and third components. In simple words, we consider all transference plans between measures on T M , with the requirement that the projection on the bases to be optimal (in the sense of optimal transport). This approach was recently generalized considering the one-sided Lipschitz condition for W and making ingenious connections with evolution variational inequalities [105]. We note that this approach allows for uniqueness at the level of solutions, thus opening the door to control problems. On the other side, if one wants to use a non-variational interpretation, then the setting of MDEs suffers from the fact that PVFs enjoy only a structure of abelian monoid and not of Lie algebra. A possibility is to explore control properties at the level of the flows, instead of solutions, taking advantage of the geometric control techniques [106].

Another intriguing vision was given by the recent preprint [107], where multiple scales and concepts of limits were discussed. With permission from the authors, we report a figure representing the general picture of limiting processes (Figure 7). These approaches are related to the solution of Hilbert sixth problem, i.e., axiomatization of mathematical physics [108]. The authors start with a general microscopic system of the type:

where x i represent parameters describing the ith agent, while ξ is its state. Comparing with systems presented in Section 2, the main difference is the presence of the variable x i , which is assumed to be constant in time, thus x ˙ i = 0 . This allows the interaction function G to depend on the specific particle bypassing the classical limitation of indistinguishability of particles assumed in mean-field theories [69].

Figure 7

Relationships between particle (microscopic) system, Liouville (probabilistic) equation, Vlasov (mesoscopic, mean-field) equation, the Euler (macroscopic, graph limit) equation (from [107] with permission from the authors).

An alternative way to send the number of agents N to infinity is via the so-called graph-limit by interpreting the right-hand side of (42) as a Riemann sum. Another interpretation is to look at the graph of interacting agents as a graphon, i.e., a function from [ 0 , 1 ] 2 to [ 0 , 1 ] , and passing to the limit in the cut norm [109]. For recent results, see also [110,111]. In [107], the authors show other connections via mean-field, graph-limit, and embeddings. A future direction of great interest will be to understand how control problems behave with respect to such limits, as in the spirit of Section 4.2.

Along the same notes, we want to point to another recent contribution [112], where the authors, besides the classical Lagrangian and Eulerian point of view, consider the Kantorovich formulation. The latter is based on representing solutions to equations as measures over the space of continuous curves. Using the superposition principle [113], one is able to connect the Lagrangian and Eulerian formulation to the Kantorovich one. In this framework, the authors present various equivalence and Γ -convergence results, but it would be interesting to understand sparsity, parsimonious controls, and necessary and sufficient conditions for optimality.

Finally, we want to point out the reader’s attention to the general framework of mutations [114], a generalization of the concept of derivatives. In [115], the authors proved an extension of the Filippov theorem to the metric space setting including evolutions in Wasserstein spaces.

5.2 Real-world applications

In this section, we report experimental results concerning the control of traffic via AVs. More precisely, we first discuss the problem of stop-and-go waves and their impact on traffic. Then, we show how a single AV can dissipate waves on a ring road with other 20 cars. Finally, we describe a recent traffic experiment with 100 AVs on an open highway: the I-24 in Nashville, TN. The latter is, at the present time, the largest traffic experiment ever performed with AVs on an open highway.

5.2.1 Stop-and-go waves and fuel consumption

As recalled in Section 2.1, stop-and-go waves may appear ubiquitously due to differences in human driving, also in a confined setting [18,116]. They can be explained in terms of instabilities of traffic flow [117], and their impact includes increasing fuel consumption and increased breaking [118]. Several studies proposed strategies to smooth traffic via AVs in simulation [119–122].

A first set of experiments to dissipate waves via a single AV was performed on a ring-road [123–127], which is the same setting as the seminal experiment [18]. A single AV with level 2 autonomy (according to the society of automobile engineers taxonomy) out of 22 cars was able to dissipate waves, improve fuel economy, and reduce significant breaking events. The experiment started with the same setting as the Japanese one, with the driver instructed only to keep a safe distance and the AV being human-driven. Figure 8 reports the velocities of all human-driven vehicles (in gray) and the AV one (in blue). One can note the appearance of a wave, which, after around 90 s, becomes stable causing all vehicles to accelerate and brake periodically. The velocity oscillations are significant ranging between 0 and 11 m/s. Then, the AV automatic control is activated (the human commands only the steering wheel), and the wave is dissipated in a short time. At the end of the experiment, the AV turns back to human-driven mode and the wave reappears. The final result is a reduction of fuel consumption of 43% (measured with on-board diagnostic-II on all vehicles) and a reduction of heavy breaking events (decrease of more than 1 m/s in 1 s) by 98%.

Figure 8

Trajectories of the field experiment on a ring-road: AV trajectory in blue, human-driven vehicles’ trajectories in gray.

5.2.2 The I-24 MOTION testbed

A sizeable testbed was developed on the interstate I-24 in Nashville, TN, called I-24 MOTION [128,129]. This testbed was completed in October 2022 by the Tennessee Department of Transportation and Vanderbilt University with almost 300 4K cameras mounted on more than forty 110 ft roadside poles. Each pole hosts six cameras to allow complete coverage of 500 linear feet of highway. The system captures more than 150,000 vehicles daily [130]. For comparison, each day the system captures data that are three orders of magnitude larger than the celebrated next generation simulation data set [131]. Figure 9 shows the construction site for the installation of one of the poles.

Figure 9

Left: Installation of a roadside pole. Right: aerial view of the installed pole with 8 cameras and I-24 traffic underneath.

5.2.3 The 100-AV experiment

In November 2022, the CIRCLES consortium [103] performed a 100 AV experiment on the I-24 testbed. The research team was comprised of more than 60 researchers from the CIRCLES main partners (UC Berkeley, Rutgers-Camden, Temple, and Vanderbilt) as well as others (Arizona, ENS, and others). Moreover, three OEMs participated: Nissan, GM, and Toyota. The test was prepared through extensive work, including designing a holistic approach and model-based controls [132], using simulation architectures [133], modeling energy consumptions [134], designing AI-based algorithms [135,136], and developing hardware to control cars [137–139]. From November 14th to18th, 2022, the team deployed on the I-24 more than 100 autonomous vehicles, with level 2 autonomy, during the morning rush hour testing several controls developed with mathematical model techniques as well as AI tools. More than 150 drivers were trained and sent to the highways on two fixed routes around the portion of the highway monitored by cameras. The experiment was planned for several months to allow the safe deployment of vehicles and an appropriate penetration (percentage of controlled vehicles on overall traffic). Figure 10 depicts the planned operations on the parking lot, from which cars left to reach the I-24, and shows a picture from one day of the experiment.

Figure 10

Top: A plan of the parking lot from which the cars left to reach the I-24. Orange and yellow routes are highlighted. Bottom: A photo of the lot during the experiment with the two groups of cars (corresponding to the orange and yellow routes).

5.3 Future experiments

The 100 AVs experiment represented the largest experiment worldwide of its kind. As we are writing, the massive experimental data are analyzed to quantify the impact of AV regulation on traffic. Preliminary evidence showed a significant impact.

The experiment showed evidence of the feasibility of interventions at scale on real traffic. Future development may include the following:

• Traffic regulation via fleets, e.g., buses, taxis, and others.

• Recruitment of community stakeholders, such as commuters, to reduce congestion and fuel consumption.

• Exportation of the ideas to other areas, including pedestrian crowds, robotic formations, and socio-economic systems.