Modeling the Optimal Maintenance Scheduling Strategy for Bridge Networks

An optimal maintenance scheduling strategy for bridge networks can generate an efficient allocation of resources with budget limits and mitigate the perturbations caused by maintenance activities to the traffic flows. This research formulates the optimal maintenance scheduling problem as a bi-level programming model. The upper-level model is a multi-objective nonlinear programming model, which minimizes the total traffic delays during the maintenance period and maximizes the number of bridges to be maintained subject to the budget limit and the number of crews. In the lower-level, the users’ route choice following the upper-level decision is simulated using a modified user equilibrium model. Then, the proposed bi-level model is transformed into an equivalent single-level model that is solved by the simulated annealing algorithm. Finally, the model and algorithm are tested using a highway bridge network. The results show that the proposed method has an advantage in saving maintenance costs, reducing traffic delays, minimizing makespan compared with two empirical maintenance strategies. The sensitivity analysis reveals that traffic demand, number of crews, availability of budget, and decision maker’s preference all have significant effects on the optimal maintenance scheduling scheme for bridges including time sequence and job sequence.


Introduction
Bridges are the most vulnerable elements in highway networks, which play an important role in economic growth and social development [1]. Due to the increasing heavier truck-loads and exposure to wet environments, bridges are inclined to face rapid performance deterioration. Routine maintenance is necessary to guarantee the bridge network at a high service level and extend the service life of bridges. However, the available budget cannot cover the cost of maintaining all bridges simultaneously in a network. Hence, there is a growing demand for developing an efficient maintenance scheduling strategy for bridge networks.
In the literature, a large body of studies were carried out focusing on the optimization of bridge maintenance strategy, which is defined as an optimal resource allocation problem by most Appl. Sci. 2020, 10, 498 2 of 16 of the researchers [2,3]. In summary, there are two categories of the existing approaches dealing with this problem. The first category is the mathematical programming method [4] and the second category is the simulation-based method [5]. These methods can identify which bridges to maintain, but cannot determine the time sequence of maintenance activities for the bridges. Additionally, the interaction between the maintenance activities and the traffic delays in the network has not been completely investigated.
To this end, this paper aims to concentrate on the optimal maintenance scheduling strategy (OMSS) for bridge networks. Our work makes the following contributions. (1) We propose a nonlinear bi-level model to formulate the OMSS problem, which aims to minimize the total traffic delays during the maintenance period and ensures that the available maintenance budget covers as many bridges as possible.
(2) We transform the bi-level model into an equivalent single-level model and employ the simulated annealing algorithm (SAA) as the model solution. (3) We use a highway bridge network to test the effectiveness of the proposed model and algorithm and compare the OMSS generated by our model with two commonly used empirical strategies.
The remainder of this paper is organized as follows. Section 2 presents a literature review of the existing studies of maintenance optimization for bridge networks. Section 3 formulates the problem and develops the solution algorithm. Section 4 illustrates the proposed model by a case study. Section 5 analyzes the sensitivity of relevant parameters. Conclusions and future work are discussed in Section 6.

Literature Review
The studies of network-level bridge maintenance optimization can be classified into two groups, in terms of the length of time horizon: single-stage maintenance problem and multi-stage maintenance problem [6][7][8]. This research falls in the first group.
For the single-stage maintenance problem, it aims to identify the prioritization of a set of bridges to be maintained in a network under budget limits. Combinatorial optimization theory is the most used tool in dealing with this problem, which formulates the problem as linear or nonlinear integer programming or mixed-integer programming models. For instance, Liu and Frangopol solve the bridge annual maintenance prioritization using multi-objective combinatorial optimization, which minimizes the largest condition index and the present value of maintenance cost as well as maximizes the smallest safety index [6]. Additionally, the maximization of bridge quality [9], maximization of tolerable condition index [10], maximization of service life [7,11], minimum user cost in the bridge network [12], minimum cumulative expected failure cost [13], maximum structural reliability [14], etc. are also considered to be objective functions in the literature. Recently, the concept of resilience was highlighted in the studies of infrastructure systems, and the maximization of the bridge network resilience was also taken into consideration. For example, Bocchini and Frangopol define bridge network resilience as the integral in time of the network functionality and present a linear programming model with two conflicting objectives, i.e., maximum resilience and minimum maintenance cost to generate an optimal bridge maintenance schedule [15]. In general, two approaches exist to solve the single-stage maintenance problem: the top-down approach and the bottom-up approach [16,17]. The top-down approach assumes that all the facilities in the transportation network have homogeneous features, e.g., traffic patterns, properties of performance deterioration [18], while the bottom-up approach assumes that the facilities have different characteristics, which can capture facility-specific features [19,20]. Kuhn and Madanat apply a top-down approach to solve selecting and scheduling maintenance activities on networks of transportation facilities under uncertainty [21]. Robelin and Madanat deal with the problem of optimizing bridge maintenance and replacement decisions over a finite planning horizon using a bottom-up solution method [22].
The multi-stage maintenance problem is usually formulated as dynamic programming models. Numerous studies employ Markov decision processes (MDP) and its variants to generate a long-term maintenance plan [23,24]. MDP simulates the discrete performance conditions of transportation facilities with the state transition equation. O'Connor et al. propose a Markov chain model to simulate the propagation phase of performance deterioration over time, which is incorporated into modeling the optimal maintenance strategy [25]. Schöbi and Chatzi put forward a partially observable Markov decision processes model combined with a non-linear action model for the optimal decision making of transportation infrastructure [26]. Additionally, Monte Carlo simulation also has been applied to solve the multi-stage maintenance problem [27][28][29]. However, the maintenance plans obtained by the above methods are randomized policies that are probabilistic in nature so that they cannot determine the specific maintenance strategy of every single facility in practice. To fill this gap, some other researchers adopt the approximate dynamic programming (ADP) models. For example, Kuhn introduces an ADP-based model to overcome the network-level infrastructure maintenance problem under budgetary restrictions and resource constraints [30]. The comparison between the ADP-based model and the MDP-based model carried out by Medury and Madanat shows that using ADP has an advantage in incorporating a variety of modeling assumptions within the decision-making process of transportation infrastructure maintenance [31].
Despite the wide range of studies in this realm, it is rare in the literature to investigate the interaction between the maintenance activities and the traffic delays in the bridge network or determine the time sequence and job sequence of the maintenance activities. To this end, this work aims to develop a state-of-the-art method framework to fill the abovementioned gaps.

Assumption
This research is carried out based on the following assumptions: (1) Every single bridge is closed when its maintenance activity is ongoing, i.e., the traffic capacity of the bridge that is being maintained drops to zero. (2) Users can obtain the maintenance schedule of the bridge network in advance according to which users choose their routes. (3) All the traffic demands are fixed during the period of maintenance activities.

Notation
The notations used in this study are given as follows:

Sets and indices
A Set of all links in the highway network, indexed by a ∈ A. I Set of bridges to be maintained or set of maintenance activities, indexed by i ∈ I. S Set of all crews, indexed by s ∈ S. W Set of origin-destination (OD) pairs, indexed by w ∈ W. K w Set of paths that connect OD pair w, indexed by k ∈ K w . Traffic capacity of link a at time t. If the bridge i maintained at time t lies on link a, C t a = 0; otherwise, C t a = C 0a . u t a (q) Travel time of link a at time t. h w,t k Traffic flow on the path k that connects OD pair w at time t.

Parameters
x ist Binary variable, which means if the maintenance activity of bridge i is implemented by crew s and starts at time t, Binary variable, which is defined as δ w,t a,k = 1, if link a lies on path k that connects OD pair w; otherwise, δ w,t a,k = 0.

Modeling
In this subsection, we propose a bi-level programming model to formulate the OMSS for bridge networks considering the dynamic traffic evolution. The upper-level model determines the OMSS to minimize the cumulative traffic delays during the makespan of the maintenance scheduling scheme subject to the budget limits and the number of crews. In the lower-level model, dynamic traffic assignment is simulated following the maintenance scheduling strategy obtained in the upper-level model to ensure the minimum travel time.

of 16
where Equation (1) is the objective function, which guarantees the minimum cumulative traffic delays during the makespan M. M t = 1 U t is the total travel time during the makespan M. The other objective function in Equation (2) aims to cover as many bridges to be maintained as possible with the budget limit. Equation (3) defines the maintenance makespan as the maximum end time for all maintenance activities max i∈I,s∈S (5) is denoted as the number of ongoing maintenance activities carried out by crew s at time t, i.e., each crew can maintain at most one bridge at time t. Equation (6) represents that the amount of simultaneous maintenance activities cannot exceed the number of crews at time t. Equation (7) defines the types of decision variables.
The lower-level model Subject to where Equations (8)-(11) define the Beckmann's transformation [32] to simulate the traffic assignment under UE state at time t. Herein, the objective function in Equation (8) minimizes the total travel time in the network. Equation (9) denotes that the traffic demands of any OD pair are the sum of traffic flows on all paths linking the OD pair. Equation (10) indicates that the traffic volume on every single link is the sum of traffic volume on all paths on which the link lies. Equation (11) sets the traffic volume on each path as non-negative. Equation (12) formulates the free-flow travel time u 0a on link a, which is defined as the ratio of the length l a to the speed limit V 0a . Equation (13) is the classical BPR function, which is used to estimate the travel time, where α and β are function parameters. Equation (14) defines the types of decision variables.

Model Solution
The model solution is divided into two phases, namely the bi-level model is transformed into a single-level model in the first phase and in the second phase, SAA is employed to solve the single-level model.

Single-Level Model
Denote ρ as the decision maker's preference so that the two objective functions in the upper-level model can be normalized as where x is a vector of the binary decision variable x ist , q is a vector of the decision variable q t a . Z min is the minimum value of objective Z. M max is the maximum value of objective M.
We define e as a vector of ones so that the discrete variable x ist (∀i ∈ I, ∀s ∈ S, ∀t ∈ T) is equivalent to x T (e − x) = 0 when 0 ≤ x ≤ 1 [33]. Hence, Equation (7) is equivalent to Equations (16) and (17).
Then, denote Ω as the set of the feasible link flows for the lower-level model, i.e., Equations (8)- (14). Ω is formulated as Hence, the optimal-value function of the lower-level model is defined as Apparently, for any feasible solution, it has U t (x, q) − ω(x) ≥ 0 . Since the traffic flow on each link is a unique value, U t (x, q) − ω(x) = 0 holds if and only if q as the unique solution given fixed x. Therefore, the bi-level model can be transformed into its equivalent single-level model as follows: Subject to

Simulated Annealing Algorithm
The optimal maintenance scheduling strategy is a combinatorial optimization problem in nature, which can be solved by a set of algorithms including genetic algorithm, ant colony algorithm, tabu search algorithm, artificial immune algorithm, etc. Due to the advantages of obtaining the global optimal solution and fast convergence [34], SAA is an efficient algorithm for solving combinatorial optimization problems, which accepts the optimal solution with a certain probability [35][36][37]. Hence, we employ SAA to solve the transformed single-level model. The iterative steps are as follows: Step 1: Initialization. Denote T 0 and T f as the initial temperature and termination temperature respectively and calculate the value of objective function F 0 = F(x 0 ist based on the initial solution x 0 ist that is generated randomly. The initial number of iterations is set as k = 0.
Step 2: If the temperature in the kth iteration T k reaches the termination criterion of the inner loop, the procedure goes to Step 3; otherwise, randomly select a feasible solution from the neighborhood and calculate F k using the following formulation.
where x k ist is the solution in the kth iteration, x * ist is a random feasible solution from the neighborhood. If F k < 0, x * ist is accepted as the new solution completely, otherwise, x * ist is accepted with the probability P(T k ) when P(T k ) > ε, ε ∈ (0, 1), if P(T k ) < ε, repeat Step 2.
Step 3: Denote k = k + 1. If T k ≤ T f , output the optimal solution x * ist , update the temperature by the following strategy and return to Step 2.
Step 4: Output the optimal solution and stop the procedure.

Testing Network and Basic Data
We use the highway bridge network shown in Figure 1 to demonstrate the effectiveness of the proposed method and algorithm. The network consists of 14 nodes and 13 bridges. There are 21 bidirectional links among nodes and 42 unidirectional links. The characteristics of all the 42 unidirectional links including length, traffic capacity, and speed limit are reported in Table 1. The daily traffic demands of the 20 OD pairs in the pre-maintenance period, which are assumed to be fixed are listed in Table 2 Assume that the transportation agency plans to maintain the bridge network at t = 0 day. The available budget is 1500 fund-units. The maintenance duration and maintenance cost of each bridge are given in Table 3.     Assume that the transportation agency plans to maintain the bridge network at t = 0 day. The available budget is 1500 fund-units. The maintenance duration and maintenance cost of each bridge are given in Table 3. The other parameters are valued as follows: The procedure is implemented using the commercial solver CPLEX (version 12.7.1), which was sourced in Waterloo, Canada from International Business Machines Corporation (IBM). All experiments are conducted on a Windows 10 PC with an Intel Core i7-9700 CPU (4.6 GHz) and 16.0 GB DDR4 (2400 MHz), which was sourced in Waterloo, Canada from Lenovo.

Results
The OMSS generated by the proposed model is demonstrated in Figure 2. Figure 2a shows the start and end time of the maintenance activity for each bridge, where the number on each bar is the duration of each maintenance activity. Figure 2b indicates the assignment of all the maintenance activities, where the letter and number on each bar are the bridge ID and maintenance duration, respectively, and the job sequence of each crew is ordered from left to right. Specifically, crew 1 undertakes the maintenance activities of bridge B and H, crew 2 is responsible for the maintenance activities of bridge L, E, and G, and bridge N and J are assigned to crew 3. The entire makespan of the maintenance schedule is 25 days. The procedure is implemented using the commercial solver CPLEX (version 12.7.1), which was sourced in Waterloo, Canada from International Business Machines Corporation (IBM). All experiments are conducted on a Windows 10 PC with an Intel Core i7-9700 CPU (4.6 GHz) and 16.0 GB DDR4 (2400 MHz), which was sourced in Waterloo, Canada from Lenovo.

Results
The OMSS generated by the proposed model is demonstrated in Figure 2. Figure 2a shows the start and end time of the maintenance activity for each bridge, where the number on each bar is the duration of each maintenance activity. Figure 2b indicates the assignment of all the maintenance activities, where the letter and number on each bar are the bridge ID and maintenance duration, respectively, and the job sequence of each crew is ordered from left to right. Specifically, crew 1 undertakes the maintenance activities of bridge B and H, crew 2 is responsible for the maintenance activities of bridge L, E, and G, and bridge N and J are assigned to crew 3. The entire makespan of the maintenance schedule is 25 days. From the simulation of traffic evolution during the makespan, we know that the travel time in the network has a significant day-to-day fluctuation, which is shown in Figure 3. Apparently, the travel time changes at the start and end time of every single maintenance activity. Compared to the travel time under UE state in the pre-maintenance period, the maintenance activities add up to 22,610 h of traffic delays. From the simulation of traffic evolution during the makespan, we know that the travel time in the network has a significant day-to-day fluctuation, which is shown in Figure 3. Apparently, the travel time changes at the start and end time of every single maintenance activity. Compared to the travel time under UE state in the pre-maintenance period, the maintenance activities add up to 22,610 h of traffic delays. In the bridge maintenance practice, two empirical strategies, i.e., flow-first scheduling strategy (FFSS) [38] and worst-first scheduling strategy (WFSS) [39] are two commonly adopted strategies. FFSS determines the time sequence of bridge maintenance activities by traffic flow from highest to lowest. WFSS determines the time sequence of maintenance activities by bridge performance from lowest to highest. Additionally, both the two strategies try to cover as many bridges as possible under the budget limit. Table 4 presents the traffic flow and performance ranking of each bridge in the premaintenance period. According to Table 4, FFSS and WFSS generate the maintenance schedules as in Figures 4 and 5.  In the bridge maintenance practice, two empirical strategies, i.e., flow-first scheduling strategy (FFSS) [38] and worst-first scheduling strategy (WFSS) [39] are two commonly adopted strategies. FFSS determines the time sequence of bridge maintenance activities by traffic flow from highest to lowest. WFSS determines the time sequence of maintenance activities by bridge performance from lowest to highest. Additionally, both the two strategies try to cover as many bridges as possible under the budget limit. Table 4 presents the traffic flow and performance ranking of each bridge in the pre-maintenance period. According to Table 4, FFSS and WFSS generate the maintenance schedules as in Figures 4 and 5.   Table 5 lists the results of the OMSS, FFSS and WFSS respectively. OMSS covers maintenance activities of seven bridges with the lowest cost, i.e., 1450 fund-units generating the minimum traffic delays during the makespan of 25 days. WFSS exhausts the largest portion of the budget, which can only cover six bridges. FFSS has the maximum makespan and brings about the maximum traffic delays. Obviously, OMSS outperforms the two empirical strategies, which illustrates that the proposed method can make the bridge maintenance decision with lower cost, fewer traffic delays and shorter maintenance period.

Discussion
In this subsection, we discuss the sensitivities of traffic demand, number of crews, availability of budget and decision maker's preference to illustrate the effects of these parameters on the results of OMSS. The sensitivity of every single parameter is analyzed by keeping all the other parameters constant.   Table 5 lists the results of the OMSS, FFSS and WFSS respectively. OMSS covers maintenance activities of seven bridges with the lowest cost, i.e., 1450 fund-units generating the minimum traffic delays during the makespan of 25 days. WFSS exhausts the largest portion of the budget, which can only cover six bridges. FFSS has the maximum makespan and brings about the maximum traffic delays. Obviously, OMSS outperforms the two empirical strategies, which illustrates that the proposed method can make the bridge maintenance decision with lower cost, fewer traffic delays and shorter maintenance period.

Discussion
In this subsection, we discuss the sensitivities of traffic demand, number of crews, availability of budget and decision maker's preference to illustrate the effects of these parameters on the results of OMSS. The sensitivity of every single parameter is analyzed by keeping all the other parameters constant.  Table 5 lists the results of the OMSS, FFSS and WFSS respectively. OMSS covers maintenance activities of seven bridges with the lowest cost, i.e., 1450 fund-units generating the minimum traffic delays during the makespan of 25 days. WFSS exhausts the largest portion of the budget, which can only cover six bridges. FFSS has the maximum makespan and brings about the maximum traffic delays. Obviously, OMSS outperforms the two empirical strategies, which illustrates that the proposed method can make the bridge maintenance decision with lower cost, fewer traffic delays and shorter maintenance period.

Discussion
In this subsection, we discuss the sensitivities of traffic demand, number of crews, availability of budget and decision maker's preference to illustrate the effects of these parameters on the results of OMSS. The sensitivity of every single parameter is analyzed by keeping all the other parameters constant.

Traffic Demand
We define τ (−∞< τ < +∞) as the growth rate of traffic demands. Table 6 presents different results of the OMSS for different τ. It is found that traffic delays increase exponentially with the growth of τ. For τ < 62%, the results of OMSS keep constant that do not differ with the variation of τ. When τ is 62%, the OMSS will change the time sequence of the maintenance activities to reduce traffic delays by avoiding maintaining bridge G and bridge H simultaneously. Hence, for the bridge network with heavy traffic, it is beneficial to avoid simultaneous maintenance activities of bridges on two parallel links. For τ ≥ 86%, the OMSS will maintain fewer bridges and change the maintenance time sequence to shorten the makespan, which can prevent more traffic congestion in the network.

Number of Crews
The number of crews means the maximum number of simultaneous bridge maintenance activities allowed by the available manpower. Table 7 shows three maintenance scheduling results for different numbers of crews. It is found that changes in the number of crews will affect the maintenance time sequence but not the bridges to be maintained. Obviously, the more the crews, the shorter the makespan. However, the relationship between the number of crews and the makespan is nonlinear. It has the maximum makespan, i.e., 73 days when there is only one crew, which produces more traffic delays. Although the makespan is the minimum, i.e., 16 days when there are five crews, there are still more traffic delays. Additional experiments show that |S| = 3 can minimize traffic delays in this case study.

Availability of Budget
We obtain three results of the OMSS when the availability of budget is 1500, 2000 and 2500 fund-units respectively, which are shown in Table 8. Compared with B = 1500, the maintenance result under B = 2000 includes three more bridges to be maintained but has a longer maintenance period generating more traffic delays. Please note that the maintenance scheduling scheme keeps constant when B increases from 2000 to 2500, i.e., additional investment will not change the maintenance result because unilaterally increasing the capital investment but ignoring the increase of manpower cannot improve the OMSS.   Table 9 indicates three results of OMSS for three different ρ. The three results of OMSS are different in both maintenance time sequence and bridges to be maintained. Compared with ρ = 0.5, for ρ = 0.3, the OMSS generates more traffic delays, while for ρ = 0.7, fewer bridges are involved with maintenance activities. Hence, the OMSS is a tradeoff between the traffic delays and the number of bridges to be maintained based on the decision maker's preference.

Conclusions
Because of climate change and increasing truck-loads, bridges are more inclined to suffer from rapid performance deterioration. Hence, periodic maintenance activities are highly needed to improve the bearing capacity and extend the service life of bridges. However, the available budget cannot cover all the bridges in a network. In this situation, it is necessary for transportation agencies to develop an optimal maintenance scheduling strategy with limited available resources. Most of the existing studies define this problem as optimal allocation of the limited funds, which can identify which bridges to maintain but cannot determine the time sequence of maintenance activities for the bridges accurately.
This research focuses on the optimal maintenance scheduling strategy for bridge networks. We formulate this problem as a bi-level model. The upper-level model is a multi-objective nonlinear programming model, which minimizes the total traffic delays during the maintenance period and maximizes the number of bridges to be maintained subject to the budget limit and the number of crews. In the lower-level, the users' route choice following the upper-level decision is simulated using a modified user equilibrium model. Then, a SAA is employed to solve the model. Finally, the proposed method is illustrated on a highway bridge network to generate an optimal maintenance scheduling strategy. The key findings are summarized as follows: (1) Compared with the two empirical strategies, i.e., FFSS and WFSS, the optimal maintenance scheduling strategy generated by the proposed method has an advantage in saving maintenance cost, reducing traffic delays, minimizing makespan and can provide a potential aid for transportation agencies in making efficient maintenance decisions. (2) Traffic demand has a significant impact on the optimal maintenance scheduling strategy including time sequence and job sequence. For the bridge network with heavy traffic, avoiding simultaneous maintenance activities of bridges on two parallel links can reduce traffic delays during the maintenance period. (3) More crews can shorten the entire maintenance period apparently, but cannot always guarantee minimal traffic delays. Because more crews mean more bridges being maintained simultaneously, which aggravate traffic congestion. Only when the manpower and funds match can the optimal maintenance scheduling strategy be improved. (4) Decision maker's preference also affects both the time sequence and job sequence. Hence, decision makers should consider a reasonable tradeoff between the traffic delays and the maximum number of bridges to be maintained.
The results of the case study show that the proposed model can generate the optimal maintenance scheduling strategy for a bridge network taking both the traffic delays and the maximum number of bridges to be maintained into account, which provides a reference for transportation agencies in bridge management. However, there are still some limitations of this study. Future work will include the following two aspects. (1) Take uncertainties, e.g., duration of maintenance activities, traffic demands between OD pairs into consideration during the scheduling decision process. (2) Develop more algorithms and compare the efficiency of these algorithms to find a more suitable model solution.