Optimization of time‐varying feedback controller parameters for freeway networks

In this work, a freeway traffic control scheme is proposed to regulate traffic in freeway networks via multi‐class ramp metering. Specifically, each controlled on‐ramp of the system adopts a feedback regulator of proportional‐integral type and is able to compute the traffic flows entering the freeway mainstream, differentiated for each class of vehicles, e.g., cars and trucks. Since a crucial aspect of proportional‐integral controllers is the tuning of parameters, in this work an optimization‐based procedure is devised to determine the controller parameters to be used in the different on‐ramps of the network, according to the detected traffic conditions. To this end, a set of representative traffic scenarios is defined and, for each scenario, an offline optimization procedure is applied to determine the optimal parameters. Then, in real time, a controller parameter selector identifies the scenario associated with the current system conditions and communicates to the local controllers the optimal parameters to be applied. The effectiveness of the proposed control scheme is tested in a simulative example and the main results are discussed in the paper.


INTRODUCTION
In a world in which individual mobility needs are becoming more and more pressing and road transport, both for freight and for passengers, keeps increasing, the efficient management of traffic networks is a fundamental topic to which researchers continue to devote important efforts to find solutions that combine economic needs, users' levels of service, efficiency, and environmental protection. Freeway systems, although designed to handle high volumes of traffic, are the infrastructures that have been mostly affected by the increase of traffic flows. The difficulty in managing the current traffic volumes, of both cars and trucks, is often due to structural shortcomings that cannot be easily solved, both for economic constraints and for physical limitations. In these circumstances, the development and the adoption of specific traffic controllers is the most effective way to improve the performance of freeway traffic systems.
In the past years, various control measures have been developed in order to allow for the efficient exploitation of transport infrastructures by making use of modern information and communication systems. 1 Among these, one of the most widespread is ramp metering which, in real time and on the basis of the state measurements in the mainstream, modulates the traffic flows entering the freeway with the aim of reducing congestion, or with other objectives, such as minimizing emissions and crash risks, among others. In practical terms, applying ramp metering means defining the traffic volumes that can enter the on-ramps and translating them into appropriate traffic light cycles (an overview of the most common ramp metering implementation policies is given in Reference 2).
From a methodological point of view, the determination of the on-ramp flows can be carried out through different methodologies. Several approaches can be found in the literature, ranging from the application of feedback regulators to more sophisticated control strategies involving the solution of optimization problems, the definition of heuristic algorithms, or the development of hierarchical control schemes (for a complete overview see Reference 3). All these methods have proven to be capable of counteracting the onset of congestion with different levels of effectiveness by showing advantages and disadvantages, as briefly outlined below.
The most sophisticated control strategies are those based on the adoption of optimal control schemes. These frameworks appear in various forms within the literature where they are used based on the solution of finite horizon optimization problems [4][5][6] or defined as Model Predictive Control schemes. [7][8][9][10] In the freeway traffic control domain, the optimal control problems are generally characterized by a non-linear and non-convex structure which, combined with the large-scale nature of the system to be controlled, makes such problems generally difficult to be solved. One of the most effective methodologies for solving optimal control problems arising in the context of freeway traffic systems is the feasible direction algorithm, based on gradient calculation. 11 In case the calculation of the gradient becomes too demanding or even impossible, derivative-free algorithms can be applied to find the numerical solutions of the optimal control problems. 12 Although very effective in theory, the application of optimization-based methodologies in real freeway traffic systems is still rather limited, and this is mainly due to the high computational effort associated with the solution of these optimal control problems. To address this issue, several optimization-based control schemes have been specifically designed to reduce the computational effort, leading for instance to hierarchical control schemes 13 or distributed control schemes. 14 Yet, in the perspective of reducing the computational effort, other approaches involve the transformation of nonlinear traffic models into linear parameter varying models. 15,16 As an alternative to optimization-based approaches, feedback regulators are often applied to compute ramp metering flows. Compared with optimization-based control schemes, they generally provide a less performing solution but present fewer limitations of practical applicability. These feedback control strategies are generally based on the definition of the desired traffic conditions and the implementation of control actions to keep the current traffic states close to predefined target values. Differently from optimal controllers, feedback regulators normally have a local nature, that is, the control actions are calculated on the basis of traffic measurements that are detected in an area close to the controlled on-ramps. The most famous feedback regulator for freeway systems is ALINEA, 17 which is an integral controller designed to improve the freeway throughput. An extension of ALINEA is given by PI-ALINEA, 18 including a proportional term in the control law, that has been proven to be more effective than ALINEA in stabilizing traffic conditions in presence of bottlenecks located far downstream the on-ramps. 19 Although the effectiveness of ALINEA and PI-ALINEA has been demonstrated in many real applications, 18 their ability to cope with congestion is inevitably affected by the typical limits of local control schemes in which the control action depends exclusively on the state measured in the immediate vicinity of the on-ramps. To overcome the local nature of classical feedback controllers, a proportional-integral feedback controller called Extended Multi-class PI-ALINEA (EMC-PI-ALINEA) has been proposed in References 20,21. As deeply analyzed in Reference 21, these types of controllers, which consider traffic measurements in wide road portions associated with each on-ramp, are more effective than traditional local PI regulators. However, a relevant issue associated with all feedback controllers concerns the tuning of the parameters characterizing the control law, that is, the gains and the reference value. Typically, these parameters are defined for a standard traffic scenario for which they prove to be effective, but, since in freeway networks the traffic conditions can change as a consequence of a wide combination of events, these standard parameters can result not to be completely effective in many cases.
In this paper, we propose a ramp metering control scheme designed to meet the real-time control requirements for large-scale traffic networks, in which the flows entering the on-ramps are computed through PI controllers whose control parameters are chosen according to the occurrence of different traffic scenarios and defined using an optimization-based procedure. Moreover, in this scheme the ramp metering controllers employed in each on-ramp have a multi-class nature and, therefore, specific control actions and control parameters are defined for different classes of vehicles (e.g., cars and trucks, for which the entering ramps are equipped with two separate lanes and signals). Some examples of multi-class ramp metering can be found in References 6,[22][23][24][25] However, as further clarified later on, the procedure here proposed has a general nature and can also be adopted to define the parameters of single-class feedback controllers.
In particular, the main features of the proposed control scheme are: • the inflows, for each on-ramp and for each class of users, are computed with PI controllers; • the parameters of the PI controllers can be different for each on-ramp and depend on the traffic conditions of the system; in particular, an optimization procedure is proposed to determine the parameters of the PI controllers according to a set of traffic scenarios; such optimization is realized offline for each scenario and a library of scenarios, each one characterized by a set of optimal control parameters, is created; • a controller parameter selector is used to online identify the present scenario and, consequently, to select the optimal parameters for the PI controllers.
This scheme, with a few modifications, can be applied to optimally define the parameters of any PI feedback controller, in some way adapting such parameters to the traffic conditions in which they operate. For simplicity, in this paper we will explain the functioning of the control scheme using the EMC-PI-ALINEA controller as a PI controller. Through the simulation results we will demonstrate the versatility of the proposed scheme by applying it to optimally decide the parameters not only of EMC-PI-ALINEA controllers but also of Multi-class PI-ALINEA controllers. 26 It is worth noting that the idea of optimizing the controller parameters on the basis of real-time traffic conditions has already been applied in the context of traffic control systems. For example, in Reference 27, the parameters of the perimeter-control regulator are updated in real-time based on traffic measurements, by exploiting machine learning techniques and adaptive optimization principles. In Reference 28, a modification of ALINEA is proposed in which the density set-point is time-varying and computed online on the basis of measurements of speeds and flows upstream the bottleneck. In Reference 29, a receding-horizon parametrized traffic control approach is considered, in which parametrized control laws are identified and the controller optimizes the parameters of such control law. Parametrized control laws are also considered in Reference 30, where the receding horizon control approach is scenario-based, that is, it optimizes the worst case among a set of scenarios in order to handle uncertainties. Compared with such approaches, in which the parameter optimization is realized in real time, the present work aims at reducing the computational effort by computing the controller parameters completely offline for each scenario, so that the only effort to be made online corresponds to choose the present scenario and the associated set of parameters. Indeed, one of the purposes of this work is to demonstrate the convenience of using optimization-based approaches within a scheme specifically developed to meet practical applicability requirements. As mentioned above, optimal control schemes often do not find application in real-world contexts both because of the high computational burden and because of the nonlinearity of the problems to be solved that prevents from easily finding a global optimal solution. The simulation results reported in this paper show that, by applying the optimization-based procedure to define the control parameters, the performance of the controllers is significantly increased. Moreover, the results produced by the application of the control scheme proposed in this paper are compared with those resulting from the solution of a finite-horizon optimal control problem. From this comparison, it can be observed that the application of the proposed control scheme, both using EMC-PI-ALINEA controllers and Multi-class PI-ALINEA controllers, produces results slightly worse than those obtained by solving the optimal control problem but with a computational effort that is much lower.
The paper is organized as follows. Section 2 describes the multi-class traffic flow model used in the optimization module. Section 3 presents the proposed control scheme, which is composed of the EMC-PI-ALINEA controllers, the optimization-based module and the controller parameter selector. In Section 4, the proposed control scheme is assessed in a simulation environment, considering the application to a test case. Conclusive remarks and some insights are finally given in Section 5.

THE MULTI-CLASS METANET TRAFFIC FLOW MODEL
The optimization module included in the control scheme proposed in this paper is based on the predictive capabilities of a traffic flow model. In particular, the dynamic behavior of traffic is represented with a macroscopic and multi-class traffic The two dynamic equations for the freeway links are referred to the two system state variables, that are the partial traffic density and the mean speed, that is, The steady-state speed-density relation V c ( m,i (k)) and the traffic volume q c m,i (k) in (2) are respectively obtained as The dynamic equation for the origin links is a conservation equation for the queue lengths, which are other state variables, given by , that is, the traffic flow of class c leaving origin link o, having m as downstream link, is computed in a different way if the origin link o is a controlled on-ramp or not. Hence, it is possible to write: where q max,c Moreover, the incoming traffic flow in a given node and the traffic flow entering the first section of a link exiting a node are computed, respectively, as

THE PROPOSED CONTROL FRAMEWORK
The traffic control scheme proposed in this paper aims at regulating a freeway network via multi-class ramp metering in order to counteract the traffic congestion that may arise as a consequence of high traffic volumes. This control scheme The proposed control framework is designed to combine the requirements of practical applicability (low computational effort and simple controller structure) with the need to ensure high performance in the presence of different traffic scenarios, due to both recurrent and non-recurrent traffic conditions. As specified in the Introduction, this control scheme can include several types of PI controllers. In this section, the scheme in which each controlled ramp is equipped with an EMC-PI-ALINEA controller is presented. Specifically, as sketched in Figure 2, this control framework is composed of three main components, the EMC-PI-ALINEA controllers and the control parameter selector, which act online, and the optimization-based calibration of parameters, which is a procedure applied completely offline. The main features of these components are briefly sketched below and, then, described in the following subsections: • the EMC-PI-ALINEA controllers are present in each controlled on-ramp and compute the control actions on the basis of the measurements collected in a neighborhood of the on-ramp itself: the parameters of each controller are time-varying, that is, they change according to the current scenario detected in real time by the control parameter selector; • the control parameter selector is the module of the control scheme which decides the parameters to be transmitted to each EMC-PI-ALINEA controller; this block receives the state measurements from the real system and other information from the traffic manager and, on the basis of this real-time knowledge, chooses from the library of traffic scenarios the most suitable one and transmits to the EMC-PI-ALINEA controllers the associated parameters; • the optimization-based calibration of parameters is an optimization procedure applied offline which computes, for each traffic scenario, the corresponding set of optimal values of the EMC-PI-ALINEA parameters.

The EMC-PI-ALINEA controllers
In each controlled on-ramp, an EMC-PI-ALINEA controller operates referring to a neighborhood of the on-ramp, called action area of the on-ramp. This area corresponds to a set of road sections and links which are located between two consecutive on-ramps. In particular, referring to a generic controlled origin An example to clarify the concept of action area is reported in Figure 3, where the action area for origin links 1, 2, and 3 are shown with reference to a simple freeway network. It is worth noting that the size of each action area depends on the geometry of the freeway network and, as in the case of origin links 1 and 2 in this example, it may happen that two action areas overlap. Also, in case of a bifurcation in the road network, the action area of an on-ramp can include a set of road sections that are not contiguous but correspond to the diverging paths.
Considering a generic origin link o ∈ O, the EMC-PI-ALINEA controller computes the flow of each class c of vehicles that should enter in the mainstream at time step k from link o according to the following control law: where: where c o (k) is given by • act o (k) is the highest total density detected in the action area associated with on-ramp o, that is, considering the highest value of the density in the action area to be compared with the set-point corresponds to refer to the worst case and, then, to apply a ramp-metering controller that is able to react immediately when a congestion appears; • K c, P,o , K c, R,o , and̂o are the parameters of the controller and represent, respectively, the proportional/integral gains and the total density set-point for the action area associated with the current scenario .
As specified earlier, the controller parameters K c, P,o , K c, R,o and̂o are time-varying and communicated by the control parameter selector that chooses the optimal values of these parameters associated with the current scenario . Note that the gains K c, P,o and K c, R,o are different for each origin link o and for each class c, while the density set-point̂o refers to the overall traffic density and only varies with the origin link o.
Moreover, the control actions (10) are referred to the specific classes of vehicles but are computed by measuring the overall density inside road sections, thus considering the contemporary presence in the freeway links of vehicles belonging to different categories. Finally, it is worth noting that both f c,act o (k) in (11) with (12) and act o (k) in (13) are computed considering the measurements in the overall action area, in this way the ramp metering control law is able to react more in advance compared to the case of local controllers based only on one measurement close to the on-ramp.

Optimization-based calibration of parameters
In the proposed control scheme, the controller parameters vary for each origin link and, also, depending on the considered traffic conditions. The need to consider different parameters for different origin links is related to the fact that, in a large scale network (e.g., containing links with different length and slope or with multiple merging zones or bifurcations), the characteristics of the road may differ significantly in different areas as well as some local events can have impact only in specific areas. Moreover, recurrent or non-recurrent fluctuations in the traffic demand and the occurrence of exceptional events such as road accidents, road works, or bad weather conditions may require the change of the controller parameters over time. These different traffic conditions are captured in the proposed control scheme in terms of traffic scenarios. Specifically, let Σ denote the set (library) of considered traffic scenarios. Each traffic scenario ∈ Σ is characterized by a set P of data and a set K of controller parameters. The set P includes the model parameters and the features of the scenario, that are used in the optimization procedure. On the other hand, the set K includes the controller parameters that are obtained as a result of the optimization procedure and then used by the EMC-PI-ALINEA controllers.
The considered scenarios are associated with both recurrent traffic conditions and non-recurrent situations: • in recurrent conditions, each scenario refers to a given time window (e.g., morning peak hour) together with the corresponding features, such as the traffic demand; note that different scenarios are considered for working and non-working days; • non-recurrent conditions correspond either to capacity reductions caused by accidents/road works or to traffic peaks due to exceptional events; such peaks can be referred to a specific vehicle class or the total traffic flow; non-recurrent conditions are differentiated depending on time windows and type of day, analogously to recurrent conditions. Moreover, it is assumed that there are not capacity reductions/traffic peaks simultaneously happening in different action areas of the freeway network.
The set of traffic scenarios must be customized for the considered traffic network on the basis of historical data; moreover, the library of scenarios can be updated from time to time when new traffic conditions become relevant enough to be classified as a new scenario. Then, it is necessary to start with a base library that may also contain a limited number of scenarios including at least the recurrent conditions of a representative working day and of a representative non-working day. The duration of the time windows, and thus the number of scenarios to be included in the library, depends on the characteristics of the traffic demand of the considered network. In particular, the duration can vary from 2 to 4 h for daytime scenarios, while it can be more extended for night scenarios, ranging from 6 to 8 h, yielding about 6-10 scenarios per day. As for non-recurring scenarios, these can be defined based on historical data or reproduced using a dynamic traffic model. For instance, in the case of planned roadworks in which the capacity of a freeway stretch is expected to be temporarily reduced, the model presented in Section 2 can be used to reproduce the scenarios associated with this event.
The set P of data associated with scenario is given by where k 0 is the starting time step, H is the length of the time horizon, The set K is given by that is, it is composed of the controller parameters present in the control law defined (10). The parameters gathered in K and associated with each scenario are found by offline running an optimization-based procedure in which they are the decision variables to be found, while the objective function corresponds to the minimization of travel times for travelers. The problem constraints are given by the multi-class METANET traffic flow model described in Section 2. Hence, the optimization-based procedure exploits the predictive capabilities of the traffic flow model in order to determine the controller parameters that yield the optimal predicted dynamic behavior of the system during the whole time horizon in each scenario. The general formulation of the optimization problem, referred to scenario , based on the set P and aimed at the optimal calibration of controller parameters in K , is the following. Problem 1. Given the system initial conditions I , the demands D and the set M , find K that minimizes subject to (1)-(9), (10)-(13) and with K P , K R , and K P , K R , , respectively, lower and upper bounds for the decision variables.
The objective function to be minimized in Problem 1 is the Total Time Spent, denoted also with TTS and measured in [PCE⋅h]. This index is a very common objective in traffic control problems and gives an indication of the time spent by all vehicles in the freeway system. As shown in (16), the Total Time Spent is obtained as the sum of the so-called Total Travel Time, that is, the time spent by vehicles traveling along the freeway and denoted with TTT, and the so-called Total Waiting Time, that is, the time spent by vehicles waiting in the origin links and denoted with TWT. Note that minimizing TTS means maximizing the system throughput and thus maximizing the system performance.
The constraints in Problem 1 are related to the dynamics of the controlled system (1)-(9), (10)- (13) and to lower and upper bounds for the values of the gains and set-point, as reported in (17)- (19). Note that the lower and upper bounds of the decision variables must be properly defined since their values can affect the duration of the traffic light cycle.
Besides computing the parameters in K , the solution of Problem 1 also provides the optimal values of the state variables for the considered scenario . For such variables, the average values in time intervals of equal length (in which the horizon H is discretized) are computed. More precisely, P intervals are defined, with P < H , and the following values are obtained and used by the controller parameter selector, as it will be described in Section 3.3. It is worth noting that the dynamics of the traffic system described by the problem constraints (1)-(9) and (10)-(13) is nonlinear, hence Problem 1 is a nonlinear optimization problem. For the solution of this problem, we have adopted a gradient-free solution algorithm and, specifically, Simulated Annealing. Simulated Annealing is an iterative stochastic algorithm for global nonlinear optimization, first introduced in Reference 32, then extended to combinatorial optimization, 33 and to continuous optimization. 34 To perform a random search in the solution space, the Simulated Annealing algorithm is initialized with a feasible starting solution s 0 that is assigned to the current solution s, that is, s = s 0 . At each iteration of the algorithm: • a tentative solution s (tentative values of the gain parameters and of the density set-points) in the neighborhood of the current solution s is generated; • if the tentative solution is better than the current one, that is, obj(s ) < obj(s) (where obj(⋅) is the objective function associated with the solution), the tentative solution becomes the current solution, that is, s = s ; • if the tentative solution is worse than the current one, that is, obj(s ) ≥ obj(s), the tentative solution is accepted as new current solution according to a probability given by exp . The parameter Γ is called "temperature" by analogy with the annealing process and is updated with the iterations according to a suitable cooling schedule, so that the probability of accepting a worse solution decreases with the algorithm iterations. In particular, the cooling scheme proposed in Reference 35 is applied, that is, the temperature at iteration is obtained as Γ = | | | obj(s) log(P ) | | | , where represents the worsening percentage at each iteration and P is the worsening reference acceptance probability, with P = ⋅ P −1 ; • the algorithm stops when = I max or = I ni,max , with I max being the maximum number of total iterations and I ni,max the maximum number of consecutive non-improving iterations.

The controller parameter selector
The controller parameter selector acts online and chooses the set of parameters K to be adopted by the local controllers on the basis of the present scenario . At the beginning of each time window for each type of day, this module changes the scenario referred to that time window in recurrent conditions. On the other hand, non-recurrent situations must be detected online depending on the type of condition: • capacity reductions are communicated to the selector by the traffic manager; this communication can occur in advance, for example, in case of pre-scheduled road works, or in real time, for example, in case of traffic accidents; • traffic peaks due to exceptional events are detected on the basis of the measured traffic state, compared with the average values of the state variables associated with the recurrent scenario of the same time window and type of day; in particular, at each time step k the following conditions are verified: with Δ , Δ v and Δ l suitable tolerance values. If one of these conditions is verified for at least one vehicle class c, a section i of link m or an origin link o, a different scenario must be chosen. This scenario is the one corresponding to the present time window and type of day but with an increased demand profile. In case the detected scenario is completely new, that is, the corresponding parameters have not been computed yet, the present scenario corresponding to recurrent conditions is maintained and the parameters associated with the new scenario are computed off-line, so that they will be used in the future when this scenario will happen again.
Note that, alternatively to the procedure proposed in this article, the most suitable scenario can be also selected with more sophisticated methods, for example, based on classification techniques or clustering methods (see .

SIMULATIVE RESULTS
This section reports the results obtained by applying the control scheme proposed in Section 3. More in detail, the objective of this section is twofold: on the one hand, we want to show that the optimal setting of the controller parameters significantly affects the performance of the controllers and, on the other hand, we want to demonstrate the versatility of the proposed scheme by adopting two different PI controllers, namely the Multi-class PI-ALINEA and the EMC-PI-ALINEA, within the control framework introduced in Section 3. In addition, an optimization-based control approach is considered to further assess the effectiveness of the proposed control framework. Specifically, this section is further divided into three subsections: Section 4.1 describes the characteristics of the freeway network in use and the settings for creating the libraries of scenarios related to the two controllers. Section 4.2 analyzes in depth the traffic behavior in absence of control, while in Section 4.3 the results obtained with the application of several control strategies are illustrated and discussed. All results shown in this section were obtained using MATLABR2017b software on a MacBook Pro with a 2 GHz Intel Core i5 quad-core processor.

Settings of the case study
Let us first describe the structure of the freeway network considered in this work and depicted in Figure 4. Such network consists of six freeway links, indicated with m1 to m6. Each link is composed of three lanes and a variable number of sections, each of which has a length of 800 [m]. The network also includes five origin links denoted with o1 to o5 and one exiting branch located between the origin links o2 and o3. Note that the origin links o2, o3, o5 are controlled and equipped with dedicated traffic lights for each class of vehicles, that is, o2, o3, o5 ∈ O. Then, for each controlled origin link the corresponding action area has been individuated, as shown in Figure 4. On the basis of such network, a library of traffic scenarios representing recurrent and non-recurrent traffic conditions has been created. For the purposes of this work, a working day has been considered in which eight time windows lasting 3 hours have been identified. This implies that each scenario, whether representing recurring or non-recurring conditions, has this duration. The behavior of each scenario has been simulated with the multi-class METANET model described in Section 2 considering a sample time T corresponding to 10 [s] that, for a total time horizon of 3 hours, yields K = 1080 time steps. The main model parameters common to each traffic scenario are reported in Table 1.
The library associated with the control scheme adopting EMC-PI-ALINEA controllers has been created for each scenario by solving Problem 1. To create the library in case Multi-class PI-ALINEA controllers are used, a slightly different F I G U R E 4 Layout of the freeway stretch with action areas (red dashed line)

Parameter Value Range
Multi-class traffic flow model control problem has been solved, that is, Problem 1 in which (10)- (13) are replaced by the Multi-class PI-ALINEA control law, given by: where the weighted ratio f c o (k) is computed as For each scenario belonging to each library, the Simulated Annealing algorithm has been initialized with different initial solutions. Furthermore, since the solution search procedure of the algorithm is randomized, the solution algorithm has been run several times for each instance, in order to choose the best solution to be included in the traffic scenario libraries. According to our results it seems that, for this optimization problem, the choice of the initial solution does not have a significant influence on the solution produced by the algorithm that always provides a set of parameters improving the objective function. In Figure 5 two examples of objective function trends during the optimization runs F I G U R E 5 Examples of objective function trends in two optimization runs of the Simulated Annealing algorithm with different initial solutions for the parameter definition of EMC-PI-ALINEA controllers are shown. Similar trends have been obtained for optimization problems in which the Multi-class PI-ALINEA control law has been applied. Note that the two trends in Figure 5 are characterized by two different initial solutions, but still reach a similar local minimum. As it can also be seen in Figure 5, the Simulated Annealing algorithm, by its intrinsic nature, admits the acceptance of worse solutions and it can, in some cases, increase the value of the objective function. However, in this work, the solution corresponding to the lowest value of the cost function is stored in memory and therefore it is used if, at the end of the solution procedure, the current solution is not the minimum one. Finally, note that the computational time to solve Problem 1 for this setting is of about 1 minute on a standard PC, hence absolutely acceptable taking into account that these optimization problems are solved offline.

The uncontrolled case
Let us now introduce a case study concerning the freeway network previously described and specifically developed to test the capabilities of the proposed control scheme. In this section, only the behavior of the system without the application of any control measures is analyzed. This case study starts at 6 a.m. and lasts until 1 p.m. for a total duration of 7 [h] (corresponding to K = 2500 time steps), including both recurrent and non-recurrent traffic scenarios. Specifically, a data set that considers two types of vehicles, that is, cars and trucks, has been considered.
The profiles of traffic demand at the controlled on-ramps are reported in Figure 6 in which the blue line indicates the traffic demand of cars and the red line refers to the traffic demand of trucks. The first three hours represent a typical rush hour situation, while at 10:30 a.m. a traffic accident occurs causing the capacity reduction of a portion of freeway for about one hour. The freeway sections affected by the capacity reduction are located in the action area referred to origin link o2, immediately upstream of the freeway exit branch.
With these traffic inputs and considering the model settings previously introduced, the multi-class METANET model reproduces the traffic behavior of the freeway network, which can be summarized in the mainstream density evolution reported in Figure 7. Analyzing this figure, it is possible to observe that the uncontrolled case is characterized by the presence of two main congestions. The former involves the first three hours of the simulation and is due to a high transport demand of commuters, the freeway links more affected by this congestion being m3, m4, and m5. The latter congestion appears as a consequence of the traffic accident in the freeway link m2 and propagates backwards until link m1. The capacity reduction induced by the traffic accident has two main effects, namely obstructing the flow that wants to reach its destination far downstream and blocking the exit branch. Without applying any traffic control, the Total Time Spent TTS is equal to 10489 [PCE⋅h].

The controlled cases
In this section the results obtained by applying four control strategies to the case study introduced in Section 4.2 are shown and discussed. More in detail, the adopted control schemes are: Control scheme 1: standard Multi-class PI-ALINEA controllers are adopted with standard controller parameters taken from the literature; Control scheme 2: Multi-class PI-ALINEA controllers are adopted with controller parameters optimized according to the framework proposed in Section 3; Control scheme 3: EMC-PI-ALINEA controllers are adopted with controller parameters optimized according to the framework proposed in Section 3 (this is exactly the control scheme described in this article); Control scheme 4: a finite-horizon optimal control problem is solved to compute the optimal inflows at the entering ramps.
The overall framework corresponding to Control scheme 2 is shown in Figure 8. Note that, in such scheme, the Controller Parameter Selector receives all available state measurements, while the control actions for each controlled on-ramp are computed according to (24), that is, they are based on the measurements detected immediately downstream the entering ramp.
Before comparing the performance of the four control schemes, let us further detail such schemes. In Control scheme 1 the following control parameters are adopted for the Multi-class PI-ALINEA controller in each controlled ramp o ∈ O: K 1 P,o = 100, K 2 P,o = 50, K 1 R,o = 33, K 2 R,o = 6,̂o =140. Regarding Control scheme 2 and Control scheme 3, the following considerations hold. Since the simulation starts at 6:00 a.m. and ends at 1:00 p.m., the simulation includes three time windows. Each time window is associated with a recurrent traffic scenario and the presence of the traffic accident requires a non-recurrent traffic scenario associated with the action area and the time window in which the capacity reduction occurs. This leads to the selection of four traffic scenarios that, for the sake of simplicity, for both control schemes are indicated with = 1, 2, 3, 4, as better specified in Table 2. These traffic scenarios are associated with different sets of parameters K , = 1, 2, 3, 4, the values of which are depicted in Figure 9 for Control scheme 2 and in Figure 10 for Control scheme 3.
Finally, in Control scheme 4 the flows entering the on-ramps are computed by defining and solving a finite-horizon optimal control problem. In this optimal control problem the objective function J to minimize is given by where TTS is the Total Time Spent, while the second term, weighed with , is used to suppress the variations of the control actions over the time horizon. Furthermore, the traffic flow model presented in Section 2 is included by means of constraints in the optimal control problem. The resulting optimization problem is very complex since it is a highly nonlinear problem with large dimensions. Furthermore, due to the presence of many local minima, the numerical solution of the optimal control problem has been obtained by first performing a global search and then a local search starting the optimal solution obtained with the global search. Both searches have been conducted using the Simulated Annealing algorithm described in Section 3.2. Note that the solution of this optimal control problem for the test case here considered takes several hours, hence it is not applicable in real time but considered only for benchmark purposes. Alternatively to the solution of a finite-horizon optimal control problem, receding-horizon optimal control problems or model predictive control schemes can be used to optimally define the entering flows at the on-ramps. However, it should be noted that the application of such approaches on large-scale freeway networks, although very effective, is still a challenging problem because of their computational burden. Recent works in the literature, such as those proposed in References 40, 41 have recognized that the limits to the applicability of these approaches for real-time control purposes are due, among others, to the complexity of these problems and their computation time that increases rapidly as the size of the freeway network increases. The application of the four control schemes leads to a substantial improvement in traffic conditions compared to the uncontrolled case. This observation stems from the comparison of the traffic density evolution in the uncontrolled case ( Figure 7) with the density evolution for the four controlled cases displayed in Figure 11. By observing these profiles, it is possible to state that the first congestion created by commuter movements during peak hours is completely solved with the application of all control schemes, while the second congestion caused by the traffic accident is partially reduced with a level of effectiveness that depends on the implemented control scheme.
To better analyze the effectiveness of each control scheme, Table 3 shows a comparison in terms of their capability to improve the traffic conditions. Specifically, the columns of the upper part of Table 3 report the values of the Total Time Spent TTS, the Total Travel Time TTT, the Total Waiting Time TWT and the percentage of reduction of TTS compared with the uncontrolled case R TTS . The lower part of Table 3 reports the Total Time Spent TTS ′ computed in the first time window and for the whole network (i.e., during the first congestion), the respective percentage of reduction compared with the uncontrolled case R TTS ′ , the Total Time Spent TTS ′′ computed for the links and for the time periods involved in the traffic accidents and the corresponding percentage of reduction compared with the uncontrolled case R TTS ′′ .
Analyzing these indicators in Table 3, and especially comparing those obtained from the application of Control scheme 1 and Control scheme 2, it can be observed that the Multi-class PI-ALINEA controllers with optimized parameters cause a travel time reduction that is higher than the one obtained with the same controllers adopting standard (not optimized) parameters. It is worth noting that the performance produced by Control scheme 1 could be improved by introducing some  constraints on the queue lengths at the entering ramps. However, this aspect is not investigated in this article since it is out of the scope of this work. Moreover, in Table 3 it can also be observed that the application of Control scheme 3, in which all local controllers are of EMC-PI-ALINEA type, leads to better performance than Control scheme 1 and Control scheme 2. However, the best performance is achieved by applying Control scheme 4 in which the optimization-based approach is performed, even if this performance is only slightly better than the one obtained with Control scheme 3. Note that the parameters adopted in Control scheme 2 and Control scheme 3 have been obtained on the basis of scenarios with characteristics that are similar but not identical to the traffic conditions of the case study, making these results even more valuable.
To further compare the four control schemes, it can be useful to analyse the queue lengths at the origin links, as reported in Figure 12. Looking at this figure, it is possible to consider that the Multi-class PI-ALINEA controllers with standard parameters in Control scheme 1 lead to very long queues at the entering on-ramps, corresponding to high waiting times, represented by TWT, as reported in Table 3. Again in Figure 12 it is possible to notice that the results obtained from the application of Control scheme 3 are the closest to the results obtained by solving the optimization problem in Control scheme 4.
Specifically, when Control scheme 1 and Control scheme 2 are applied, the ramp metering strategy is actuated mainly at origin link o5 creating queues on that on-ramp (see Figure 12a and Figure 12b), hence entailing an overall improvement in the mainstream, as also shown in Figure 13, at the expense of users aiming to access from ramp o5. The Control scheme 3, on the other hand, activates ramp metering on both the controlled on-ramps o3 and o5 (see Figure 12c), producing a more effective and fair solution. Even observing the trend of the queues produced in dealing with the congestion created by the traffic accident, it can be noted that Control scheme 1 and Control scheme 2 act later than Control scheme 3, which turns out to be more performing also in this circumstance. This effectiveness is motivated by the fact that in Control scheme 3 the control actions are calculated on the basis of traffic conditions detected over the entire action area associated with each controlled origin link and not only on the basis of the measurements taken downstream the on-ramps as in Multi-class PI-ALINEA. Of course, the best control actions are those obtained by applying Control scheme 4. Analyzing the trends of the queues in Figure 12d it is possible to notice that this control scheme acts in strong F I G U R E 12 Queue length at the controlled origin links in the four control schemes: blue line for cars, red line for trucks F I G U R E 13 Traffic density evolution downstream the origin link o5 in the four control schemes advance in comparison with the other control schemes producing, at the origin link o5, a queue of shorter length but lasting longer.
Finally, it can be observed that the control scheme proposed in the present article (Control scheme 3) provides a performance that is very close to the optimal one (Control scheme 4) and, also, is a very effective tool for large-scale and real-time applications, since all the optimization procedures are applied completely offline and, hence, the computational burden does not represent a limitation for the real applicability. Another aspect to consider for the optimal calibration approach is that all the scenarios and the corresponding controller parameters must be saved in a database but the associated memory requirement is not critical for real applications since the size of data to be saved is absolutely compatible with the standard databases used by freeway traffic managers to save historical traffic data.

CONCLUSIONS
This paper proposes an optimization-based calibration approach to define the control parameters of PI controllers adopted to compute the ramp metering flows. The control framework proposed in this work has a general nature and can be applied to different PI regulators. The main peculiarities of this scheme concerns the optimal determination of the controller parameters, since the tuning of the control parameters is particularly crucial to ensure an effective control. The methodology proposed in this paper is based on offline optimization of these parameters on the basis of recurring or non-recurring traffic scenarios that may occur on the network under consideration. This allows to build a library of parameters from which the most appropriate ones are chosen online on the basis of the traffic conditions actually occurring in the network. As shown in the section dedicated to the description of the simulative results, the adoption of a specific version of the Simulated Annealing algorithm allows to find optimal solutions that are not very sensitive to the choice of the initial solutions adopted and in a rather short computation time. Despite these characteristics, it is not advisable to use this optimization procedure in real time both because it is a stochastic procedure that for some runs could provide inadequate solutions, and because it is not always possible to obtain from the real system all the information needed to solve the optimization problem.
The application of this scheme to a case study has shown that the optimal setting of the controller parameters produces very satisfactory results. In addition, these results have been compared with those obtained from solving an optimal ramp metering control problem. From these experiments it is possible to conclude that the rather simple implementation of the offline controller parameter library and the selection of these parameters online makes this approach workable in real control applications.

DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available by contacting the corresponding author.