FEEDFORWARD FEEDBACK ITERATIVE LEARNING CONTROL METHOD FOR THE MULTILAYER BOUNDARIES OF OVERSATURATED INTERSECTIONS BASED ON THE MACROSCOPIC FUNDAMENTAL DIAGRAM

The feedback control based on the model and method of iterative learning control, which in turn is based on the macroscopic fundamental diagram (MFD), mostly belongs to the classification of single-layer boundary control method. However, the feedback control method has the problem of time delay. Therefore, a feedforward feedback iterative learning control (FFILC) method based on MFD of the multi-layer boundary of single-area oversaturated intersections is proposed. The FFILC method can improve the effectiveness of boundary control and avoid the time-delay problem of feedback control. Firstly, MFD theory is used to determine the MFD of the control area; the congestion zone and the transition zone of the control area are identified; and the two-layer boundary of the control area is determined. Then, the FFILC controllers are established at the two-layer boundary of the control area. When the control area enters into a congestion state, the control ratio of traffic flow in and out of the two-layer boundary is adjusted. The cumulative number of vehicles in the control area continues to approach the optimal cumulative number of vehicles, and it maintains high traffic efficiency with high flow rates. Finally, The actual road network is taken as the experimental area, and the road network simulation platform is built. The controller of the feedforward iterative learning control (FILC) is selected as the comparative controller and used to analyse the iterative effect of FFILC. Improvements in the use of traffic signal control indicators for the control area are analysed after the implementation of the FFILC method. Results show that the FFILC method considerably reduces the number of iterations, and it can effectively improve convergence speed and the use of traffic signal evaluation indicators for the control area.


Introduction
Alleviating urban traffic congestion is a difficult problem in the development of major cities (Zhao et al., 2018;Klos and Sobota, 2019), especially in the traffic control of oversaturated intersections. Recent studies have shown that the macroscopic fundamental diagram (MFD) can effectively manage and control inflow traffic to a target area (network). The concept of MFD was first proposed by Godfrey in 1969 (Godfrey, 1969), but its theoretical principle has not been elaborated until 2007 by Daganzo and Geroliminis et al. (Daganzo,2007;Geroliminis and Sun, 2011). MFD, which is believed to be an inherent attribute of road networks, objectively reflects the relationship between aggregated traffic variables such as the weighted traffic flow and the weighted traffic density, or the number of vehicles and network throughput. Since the discovery of MFD, some scholars have studied the application of MFD to road network boundary control, such as the optimal boundary control method (Haddad et (Haddad and Mirkin, 2016) and so on. A comprehensive analysis showed that all the boundary control methods mentioned above belong to the general classification of model-based feedback control method, which have the problem of time delay. That is, a time delay may occur between the identification of control parameter deviation and the implementation of corrective measures, and the expected output value of the control system may not be tracked accurately. In view of the shortcoming of feedback control, scholars have proposed the iterative learning control (ILC) method based on MFD to track completely and accurately the expected output value of the system by taking advantage of the characteristics of ILC. The ILC concept was first proposed by the Japanese scholar Uchiyama in 1978 (Uchiyama, 1978). Subsequently, Arimoto et al. (1984) continuously improved the ILC method and formally proposed ILC theory in 2010. ILC can correct the control law according to the output result error and enable the output approach to gradually reach the desired trajectory by using the repetitive operation input. ILC has been applied to ramp control (Chi et al., 2013), speed control (Hou et al., 2007 ) and traffic signal control in urban areas , but the application of ILC to boundary control is only at its infancy.  proposed an urban road traffic signal with the ILC strategy, and its impact on a road network with MFD was evaluated by considering the repeatability of traffic flow. Jin et al. (2018) proposed a design method for feedforward feedback iterative learning control (FFILC) in urban-traffic areas by using MFD theory. At present, the feedback control method based on the model and method of ILC, which in turn is based on MFD, mostly belong to the classification of single-layer boundary control method. If congestion in a control area is unevenly distributed (i.e. a heterogeneous network exists), then the effect of the control and management strategies will be affected. In other words, the single-layer boundary control will not work effectively under the traffic conditions of large networks. Therefore, a multi-layer boundary FFILC method based on the MFD of single-area oversaturated intersections is proposed to improve the effect of boundary control and avoid the time-delay problem of feedback control. The actual road network is taken as the experimental area, and the road network simulation platform is built, and the FILC controller is selected as the contrasting controller to analyse the iterative effect of FFILC. The changes in the use of traffic signal control indicators for the control area before and after the implementation of the FFILC method are compared.

Methodology 2.1. MFD estimation method
Accurately estimating the MFD of road network is the premise of its application. At present, MFD estimation methods include the Loop detector data estimation (LDD estimation method) and the floating car data estimation (FCD estimation method) . LDD estimation method uses the traffic data collected by the loop detectors such as video detector, and then estimates the road network MFD according to the related theory of road network MFD. FCD estimation method uses the traffic data collected by the floating cars(such as taxis, buses, etc.) and then uses the driving trajectory estimation method to estimate the MFD of the road network.
However, there are some limitations in the application of the two methods. For example, LDD method can only obtain the traffic data of some main road sections with loop detectors, and can not obtain the traffic data of the road sections without loop detectors. And low coverage of floating cars will affect the accuracy of FCD method. In order to estimate MFD accurately and conveniently, we assume that there is a high proportion of floating cars in the road network, and use FCD method to estimate MFD. Nagle and Gayah (2014) assumed that a floating car is evenly distributed in a road network and its distribution proportion is known; subsequently, FCD estimation method was proposed to obtain the road network's MFD. The formulas are as follows: where k w and q w are the weighted traffic density (veh/km) and the weighted traffic flow (veh/h), respectively; ρ is the ratio of floating cars; T is the acquisition cycle (s); n is total number of sections in the road network; li is the length of the i-th road section (km); is the number of floating cars during the acquisition cycle T (veh); t'j is the driving time of the j-th floating car during the acquisition cycle T (s); and d'j is the driving distance of the j-th floating car during the acquisition cycle T (m).

Design of feedforward feedback iterative learning controller
In the set of sub-areas denoted by PN, where PN＝ {1,2,3}, sub-area 1 is the congestion zone, sub-area 2 is the transition zone and sub-area 3 is the normal zone ( Figure 1). The sub-area variables x,y∈PN, qxy(t) denote the transfer traffic flow from area x to area y at t time; Nxy(t) is the cumulative number of vehicles from area x to area y at t time; () xy t  is the control ratio from area x to area y at t time; areas 1 and 2 are the control objects; According to research results of Daganzo and Geroliminis et al., the MFD of the homogeneous road network is expressed as qc is the maximum weighted traffic flow; and The number of vehicles corresponding to qc is called the critical cumulative number of vehicles, which is expressed as Nc.
The boundary control is used to monitor the cumulative number of vehicles entering PN1 and PN2 in real time. The control ratio is determined according to MFD theory. The inflow control ratio (such as ) of the boundary of the control area is adjusted by the boundary controller, in which the aim is for the cumulative number of vehicles in sub-areas PN1 and PN2 to have values close to the optimal cumulative number of vehicles (Nm) at t time. In this manner, the ILC can operate rapidly and effectively and avoid the situation in which the cumulative number of vehicles in the road network approaches or even exceeds the critical value; these scenarios may cause the road network to enter a state of extreme and uncontrollable congestion. the Nm is taken as 90% of Nc. In the whole control process, the most critical problem is knowing how to determine the control ratio of the inflow and the outflow at the boundary of the control area when this area enters the congestion state. In this study, two FFILC controllers are designed to determine the real-time traffic flow control ratios at the boundary of the control area. The actual cumulative number of vehicles in the control area should not exceed the optimal cumulative number of vehicles (N1,d(t) and N2,d(t)). The design process is explained below (Jin et al., 2018

Design of FFILC for the boundary of subarea PN1
The cumulative number of vehicles N1(t) has the following relationship in sub-area PN1: where N1(t) is the cumulative number of vehicles in sub-area PN1 at t time (veh); N11(t) is the cumulative number of vehicles transferred within sub-area PN1 at t time (veh); and N12(t) is the cumulative number of vehicles transferred from sub-area PN1 to subarea PN2 at t time (veh).
The completed travel-weighted traffic flow of subarea PN1 includes two parts: internal flow and transfer flow. The relationship between these two flows is as follows: where 12 21 ( If the ratio between the number of vehicles transferred from sub-area PN1 to sub-area PN2 (the number of vehicles in sub-area PN1 is defined as the external transfer ratio coefficient of sub-area PN1, which is expressed as α1), then the following expression can be obtained: Therefore, by synthesising Eqs. (3)- (7), the traffic flow conservation of the entry-exit traffic flow at the boundary of sub-area PN1 can be rewritten as follows: If the influence of accident factors, such as traffic accidents, road construction and weather conditions, are neglected in the urban traffic system, then the change in the daily traffic flow process can be regarded a repetitive process, and the corresponding boundary control problem of the oversaturated road network can be regarded a repetitive task. The index i is subsequently introduced, and its value is iterated daily according to ILC theory. Then, the traffic flow conservation of the inbound-outbound traffic flow at the PN1 boundary of the sub-area can be rewritten as follows: where traffic flow is regarded repetitive. Adding an iteration index to q11(t), q12(t), q21(t) is not needed in this case.
According to ILC theory, if the initial parameters are given, such as with the traffic flow equilibrium equation, real-time traffic demand, {q11(t), q12(t), q21(t)}, initial cumulative number of vehicles (N1(0)),optimal cumulative number of vehicles (Nm1) and outward transfer cumulative number of vehicles (N12(t)), then the core problem of ILC at the boundary of the oversaturated intersections can serve as basis for the constant adjustment of the control ratio (μ12(t) and μ21(t)) of the entry-exit traffic flow at the boundary, and the number of vehicles in the control area can meet the expected number of vehicles. In this manner, the cumulative number of vehicles in the control area can also meet the desired cumulative number of vehicles (N1,d(t)) in the near future.
The tracking error of the cumulative number of vehicles in the control system is expressed as e1,i(t).
The curve change of the desired cumulative number of vehicles (N1,d(t)) can be expressed as the initial cumulative number of vehicles (N1(0)) gradually approaching the optimal cumulative number of vehicles. The change follows a certain growth (θ1), as shown in Figure 2. The expression of the desired cumulative vehicle number curve is as follows: 1, where θ1 is the growth of the desired cumulative number of vehicles in sub-area PN1 with acquisition interval time (veh), and t1,0 is the initial time for subarea PN1 to start the ILC controller (s). The FFILC consists of a feedforward ILC loop and a feedback control loop. Its control law is constructed as follows: where u1,i(t) is the control ratio matrix of the inbound-outbound traffic flow of sub-area PN1 boundary at t time in the i-th iteration expressed as, , is a feedforward iterative learning gain matrix expressed as ; and K is the feedback control gain matrix expressed as .
Therefore, the state equation and related parameters of the FFILC controller for the PN1 boundary can be established as follows: The iterative process of the FFILC controller is shown in Figure 3. In the iteration process, the average number of vehicles with errors in each iteration can be used to evaluate the effectiveness of the ILC. The formula is as follows: where E ILC is the average vehicle number error in each iteration (veh); total E is the sum of the absolute vehicle number error in each iteration sampling point (veh); and m is the number of sampling points. (1), 67-87, 2020 · · The traffic system of the controlled PN1 The feedback gain The ILC gain The traffic system of the controlled PN1 The feedback gain The ILC gain If the average vehicle number error in each iteration is small, and if the actual cumulative vehicle number of the road network is somewhat similar the desired cumulative vehicle number of the road network, then the road network operation state is similar to the expected operation state.

Design of FFILC for the boundary of subarea PN2
Similar to the design process of FFILC for the boundary of sub-area PN1, the state equation and the related parameters of the FFILC for the boundary of sub-area PN2 can be obtained as follows: where N2,i(t) is the cumulative number of vehicles in sub-region PN2 in the i-th iteration at t time (veh); N23,i(t) is the cumulative number of vehicles transferred from sub-area PN2 to sub-area PN3 in the ith iteration at t time (veh); q 22 (t) is the internal transfer traffic flow of sub-area PN2 at t time (veh/h); q 23 (t) is the traffic flow from sub-region PN2 to subregion PN3 at t time (veh/h); q 32 (t) is the traffic flow from sub-region PN3 to sub-region PN2 at t time (veh/h); is the travel-weighted traffic flow of sub-area PN2 in the i-th iteration at t time (veh/h); is the ratio coefficient of the external transfer of traffic flow in sub-area PN2 in the i-th iteration at t time ( , where denotes the control ratio of the incoming-outgoing traffic flow of the sub-area PN2 boundary in the i-th iteration at t time); is the tracking error of the accumulated number of vehicles inputted by the controller of the sub-area PN2 boundary in the i-th iteration at t time is the desired cumulative vehicle number of sub-area PN2 at t time (veh); is the initial cumulative number of vehicles in sub-area PN2 (veh); is the best cumulative number of vehicles in sub-region PN2 whose value is 90% of the critical cumulative number of vehicles (veh); is the growth of the desired cumulative number of vehicles in sub-area PN2 with acquisition interval time (veh); and is the initial time to start ILC in sub-area PN2 (s). Moreover, is the control ratio matrix of the entry-exit traffic flow of the subarea PN2 boundary in the i-th iteration at t time expressed as ; is a feedforward iterative learning gain matrix expressed as ; and K is the feedback control gain matrix expressed as .

FFILC method for the multilayer boundaries
The core function of the multi-layer boundary FFILC method emphasises the adjustment of the control ratio of the two-layer boundary of the control area in real time. In this manner, the cumulative number of vehicles in the control area can approximate the optimal cumulative number of vehicles. If this method is not carefully considered, then the control effect of the whole road network will be affected. Therefore, a new traffic assignment method, which monitors the traffic status of the control area and the queue length of the boundary connection section, is proposed. Two FFILCs are used in the proposed scheme. The basic idea is to apply MFD theory to determine the MFD of the control area. Then, the congestion zone and the transition zone of the control area are identified; the two-layer boundary of the control area is determined; the FFILCs at the twolayer boundary of the control area are established; and real-time traffic data, such as the cumulative number of vehicles and the weighted traffic flow, are acquired and then used to generate the traffic demand information of the controller. The traffic status of the road network is also monitored. The control process of the two FFILCs are discussed in the subsections in detail.

Determination of the MFD of the control area
According to MFD theory, the MFD of a control area can be represented as an asymmetric single-peak parabolic curve. This curve reflects the relationship between the cumulative number of vehicles (N) and the weighted traffic flow (q w ). In vehicle network environments, the FCD estimation method can be used to transform Eqs. (1)-(2) into Eqs. (17)- (18), in which the abovementioned control areas are combined.
where a is the control area (a = 1 represents the core region of PN1, a = 2 represents the transition region of PN2 and a = 3 represents the normal region of PN3); Na(t) is the cumulative number of vehicles in the a-th control area at t time (veh); is the weighted traffic flow at t time in the a-th control area (veh/h); PNa is the a-th control region; n is the number of vehicles in the acquisition period T (veh); tij is the driving time of the j-th vehicle on the i-th section in the acquisition period T (s); li is the length of the i-th section (m); T is the acquisition period (s); and dij is the driving distance of the j-th vehicle on the ith section in the acquisition period T (m).

Implementation of FFILC with multilayer boundaries
The congestion zone and the transition zone of the control area are initially identified, and the control boundary of the congestion zone (PN1) and the transition zone (PN2) is determined. The FFILCs are established for the two boundaries of the control area, and the FCD estimation method is used to obtain real-time traffic data, such as the cumulative number of vehicles and the weighted traffic flow in the control area. These data are then used to generate the traffic demand information of the controller and subsequently used to monitor the traffic state of the road network. When PN1 or PN2 enter the oversaturated state, their respective iterative learning controllers will be activated. The multi-layer boundary FFILC method is implemented to adjust the control ratios of the PN1 and PN2 boundaries in real time at each unit time (T). After several iterations, the cumulative number of vehicles in the PN1 and PN2 control areas will continue to approach their corresponding desired cumulative number of vehicles. The values should not exceed the optimal cumulative number of vehicles in the control area. As a manner of simplifying the calculation, the approximate value of the green-light time at the controlled boundary entrance and exit in the control process can be represented by the control ratio multiplied by the original green-light time at the boundary entrance, i.e. . At the same time, the shortest green-light time at the border entrance is set to 10 s to avoid calculation issues related to traffic accidents caused by the short green-light time at the border entrance after the adjustment. A detector is set up on the boundary section of the control area to monitor the maximum queue length of the boundary section of the control area in real time. If the maximum queue length of a boundary section exceeds the safe queue length, the boundary control ratio is adjusted according to the improved single-layer boundary control algorithm(Lin et al., 2017)(i.e. the algorithm considers the effect of queue length). If the maximum queue lengths of all boundary sections of PN1 in the congestion area exceed the safe queue length (i.e. a scenario indicating the absence of extra queuing space in the boundary section), then the second controller needs to be started compulsively. The flowchart of the method is shown in Figure 4. The algorithm sets 90% of the critical cumulative number of vehicles in the road network as the optimal cumulative number of vehicles. In this manner, the ILC can operate rapidly and effectively and avoid the situation in which the cumulative number of vehicles in the road network approaches or even exceeds the critical value; these scenarios may cause the road network to enter a state of extreme and uncontrollable congestion.

Experimental Analysis
A core intersection group in Tianhe District in Guangzhou is taken as an example (Lin et al., 2019). A vehicle network simulation platform is established to verify the effect of implementing the FFILC method with multi-layer boundaries by using the Vissim traffic simulation software. According to the experimental results of Lin et al., 2019，the intersection group is divided into the congestion zone, the transition zone and the normal zone, as shown in Figure 5. The cumulative vehicle number (N) and the weighted traffic flow (q w ) of the congestion zone and the transition zone of the control area are estimated by the FCD estimation method. The MFD of the congestion zone and the transition zone (including the congestion zone) are plotted, as shown in Figures 6 and 7. The scatter points of MFD for the control area is fitted to derive the fitting function. Then, the critical cumulative vehicle number (Nc) and critical weighted traffic flow (qc) of each fitting function are calculated, as shown in Table 1.
Here, 90% of the critical cumulative vehicle number (Nc) is defined as the optimal cumulative vehicle number (Nm). Therefore, the optimal cumulative vehicle number in the congestion zone and the transition zone (including the congestion zone) of the control area are 671 veh and 778 veh, respectively. The cumulative number of vehicles in the control area is divided by the spectral clustering algorithm (Shang et al., 2017), and the results are shown in Figures 8 and 9. As shown in Figures 8 and 9, the congestion zone enters the oversaturated state at 26400 s, and its initial cumulative number of vehicles is 652 veh. Meanwhile, the transition zone (including the congestion zone) enters the oversaturated state at 26640 s, and its initial cumulative number of vehicles is 713 veh. The traffic demand data of the congestion zone and the transition zone (including the congestion zone) for the time range of 26400 to 32400 s are obtained. The data include internal transfer traffic flow, external transfer to internal traffic flow, internal transfer to external traffic flow, cumulative number of internal transfer to external traffic flow and cumulative number of internal transfer to external traffic flow. The cumulative vehicle number ratio and the weighted traffic flow from the inside to the outside are also calculated, as shown in Tables 2 and  3.   The low peak period The flat peak period The peak period The oversaturation period The low peak period The flat peak period The peak period The oversaturation period This study assumes that the changes in traffic in the control area are repetitive, and the effects of weather, traffic accidents and other contingency factors are ignored. In this manner, the multi-layer boundary FFILC method can be implemented. The initial state of the control ratio of the traffic flow in and out of the control area is set to 1. The increments of the desired cumulative number of vehicles are set to 0.04. The iterative learning gain is taken as , and the feedback control gain matrix is taken as . The simulation data acquisition interval is 120 s. The initial state parameters of ILC in the congestion zone and the transition zone (including the congestion zone) are obtained, as shown in Tables 4 and 5. Two feedforward iterative learning control (FILC) controllers are designed as contrasting controllers for the congestion and transition zone boundaries to analyse the iterative effect of FFILC. The difference between FILC and FFILC is based on their respective control laws. The control law of FILC is constructed as follows: where is the control ratio matrix of the inbound-outbound traffic flow of the sub-area PN1 boundary in the i-th iteration at t time expressed as , and is a feedforward iterative learning gain matrix expressed as , in which the value is .
The FILC and FFILC methods are programmed by Matlab. The iteration of the cumulative vehicle number for the congestion zone and the transition zone The results are shown in Figures 10-17. After implementing the FILC method, the optimal control ratio for the boundary of the congestion zone is obtained after 119 iterations, and the optimal control ratio for the boundary of the transition zone (including the congestion zone) is obtained after 107 iterations. Similarly, after implementing the FFILC method, the optimal control ratio for the boundary of the congestion zone is obtained after 19 iterations, and the optimal control ratio for the boundary of the transition zone (including the congestion zone) is obtained after 17 iterations. The FFILC method can considerably reduce the number of iterations and effectively improve convergence speed. Furthermore, after implementing the FFILC method, the final control ratios for the boundary entries and exits of the congestion zone and the transition zone (including the congestion zone) are obtained, as shown in Figures 18 and 19, respectively.
Lin, X., Xu, J., Archives of Transport, 53(1), 67-87, 2020   The FFILC method is implemented when the road network simulation reaches 26400 s, as shown in Figures 18 and 19. When the simulation time is 29400 s, the control ratio of the road network boundary entry will become extremely small. This scenario indicates that the traffic flow of the road network seriously exceeds the road network capacity. Then, the simulation data for the time range of 26400-29400 s are taken to analyse further the changes in the use of traffic signal control indicators for the control area after the implementation of the FFILC method. If the boundary exit control ratio is less than 1, as in the case for some time periods, then the traffic flow out of the boundary does not need to be restricted. VB language in Vissim is used to further develop the COM programming interface and verify the control effect of the multi-layer boundary FFILC method for the control area. The multi-layer boundary FFILC method is applied to the control area according to the abovementioned boundary control ratios after 26400 s, and the maximum of 29400 s is simulated. All boundary intersections are replaced by single intersection phase sequences to simplify the implementation. The length of green light in the controlled direction is equal to the original green-light time multiplied by the control ratio, whilst the lengths of green light in the other directions remain unchanged.
Temporary pedestrian traffic lights are set up to regulate the control effects of the individual boundaries without signal controls. That is, if ( ) 100% ut  , and if the change rate of () ut is higher than 5%, then the green-light durations of the control boundary entrance and exit are adjusted to  whilst the green-light durations of the other directions are maintained. At the same time, the shortest green-light time at the border entrance is set to 10 s to avoid issues related to traffic accidents caused by the short green-light time at the border entrance after adjustment. The statistical analytical results shown in Table 6 indicate that the use of traffic signal control indicators can be improved after implementing the multi-layer boundary FFILC method for the control area at the time range of 26400-29400 s. When the control area is saturated, the FFILC with multi-layer boundaries is activated to regulate the traffic flow at the boundary of the congestion zone and the transition zone of the control area. The traffic signal evaluation indicators of the congestion zone and the transition zone (including the congestion zone) of the control area are significantly improved, and the control area is maintained in a saturated state with high flow. The overall traffic operation efficiency of the control area is subsequently improved.

Conclusions
As traffic flow increases, the traffic pressure on intersections in the core area of a road network will also increase. This study proposes a multi-layer boundary FFILC method based on MFD for oversaturated intersections to slow down the spread of traffic congestion, improve the effect of boundary control and avoid the delay of feedback control. The following conclusions can be drawn from the results of our empirical analysis: 1) Traffic flow can gather towards the centre of a sub-area, and congestion can spread to the periphery. In this scenario, the multi-layer boundary FFILC method is applied when the control region enters the oversaturated state. Compared with the FILC method, the FFILC method considerably reduces the number of iterations and effectively improves convergence speed. 2) When the control area is in a saturated state, the FFILC with multi-layer boundaries is activated to regulate the traffic flow at the boundary of the congestion zone and the transition zone of the control area. Consequently, the use of traffic signal evaluation indexes for the congestion zone and the transition zone (including the congestion zone) is significantly improved, and the control area is kept in a high flow saturation state. The overall traffic operation efficiency of the control area is also improved. 3) This study assumes that changes in traffic flow in the control area are repetitive, and the effects of weather, traffic accidents and other contingency factors are ignored. At the same time, the sample data are used as the simulation data of the vehicle network simulation platform, but they cannot be regarded the actual data. Therefore, in our future work, the impact of contingency factors on the control method will be considered, and actual data on road network traffic will be used to verify and analyse the algorithm.