Simulation-Based Framework for Estimating Crash Modification Factors (CMFs): A Case Study for ITS Countermeasures

,


Introduction
The traffic fatality rate on United States highways and freeways presented a declining trend from 2006 to 2014 after deploying safety initiatives, such as mandatory seat belt use and strict law imposition against driving under influence of drugs.Furthermore, advance technologies installed on vehicles (e.g., air bags, lane departure warning, blind spot warning, and adaptive cruise control) had contributed to decreasing the traffic fatalities [1].Fatality rate has increased 5.6 percentages to 37,461 people in 2016, after exhibiting the highest increase rate of last three decades in 2015 which was 8.4 percentages increase [2].To date, several researchers are working on developing crash prediction models and finding countermeasures to eradicate the traffic crashes on our roads [3,4].However, an appropriate scientific method is essential to evaluate these proposed countermeasures prior to the actual implementation that usually requires significant budget and time.
To introduce a science-based technical approach that can assess safety performance of proposed countermeasures in planning stage, the Highway Safety Manual (HSM) was published by the cooperative efforts of the American Association of State Highway and Transportation Officials (AASHTO), Federal Highway Administration (FHWA), and Transportation Research Board [5].HSM allows users to prioritize the countermeasures based on the changes in crash frequency and severity.In HSM, a catalog of safety performance functions (SPFs) and crash modification functions (CMFs) for different geometric and operational countermeasure types is included to predict safety performance of any facility [5].In addition to planning, the HSM can provide quantitative safety performance and assistance to select countermeasures for design, and operational phases of the project.However, the successes in safety performance predictions depend on the methodological and statistical validation of CMF values and functions [6].Many potential CMFs are excluded from first HSM edition due to the failure of implicating proper validations and justifications [7].While researchers are focsuing on parameters such as crash types and severity [8], roadway types [9], different socioeconomic conditions [10], and time changes [11] to investigate the variability of CMFs, variables such as vehicle types, driver ages and characteristics, and weather conditions have not been examined so far [12].On the other hand, estimating combined safety impacts of multiple countermeasures remain an unresolved key issue in HSM and just multiplication of the CMFs may lead to an over-or underestimation of combined effects [13].In addition, AASHTO has addressed the failure in considering the local impact of different factors in HSM [14].The duration of study for developing CMFs is another drawback, in that the research process requires data of traffic crashes before and after implementation of countermeasures.The further discussion regarding study duration continues, as discussed in literature review.Finally and most importantly, the implementation of countermeasures could demand a huge amount of investment, and once it is built, additional money could be wasted to undo in case of errors that may increase the number of traffic crashes.For example, installation of ramp meter on multilane highway could cost approximately 1 million excluding right of way acquisition.
The objectives of this study are to (i) develop a stepby-step traffic microsimulation-based method for developing CMFs which could be used to predict potential crash reduction benefit of proposed countermeasures, and (ii) provide a proof of concept of proposed method.Additionally, this proposed simulation-based method could be utilized to improve the local representation of existing CMFs.The remaining of this study is organized as follows.Section 2 discusses the current practices and their shortcomings, and Section 3 presents proposed method for developing CMFs.A case study is presented in Section 4 to investigate the proposed method with Section 5 focuses on the results of the case study and statistical analysis, and Section 6 concentrates on validation of the method.Then concluding remarks and future research direction are presented in Section 7.

Literature Review
The before-after, cross-sectional, and case-control studies are widely used methods to estimate/develop CMS [15].In observational before-after study, it is required to implement a countermeasure at crash prone region and then over a period of time, crash frequency at that location is recorded.The comparison between before and after implementation provides the observed CMF value of that countermeasure.There are many approaches to perform observational beforeafter evaluations including but not limited to (a) naïve before-after, (b) before-after with comparison group, (c) empirical Bayes, (d) full Bayes.
Among the above listed approaches, naïve before-after is the simplest method, although this method fails to contemplate "regression toward the mean" effects [16].As a result, this method overestimates the effect of countermeasure.Beforeafter method with comparison group compares the after implementation crash frequency with similar untreated locations [17].Empirical Bayes method to calculate CMFs has been the most common and rigorous approach in last ten years [18].This method overcomes the regression toward the mean effect and also considers the effect of the change in traffic volume over the period of study.Recently, empirical Bayes has been applied using negative binomial regression models to overcome the challenge of heterogeneity in traffic crash data [19,20].The full Bayes is a statistical inference method which is similar to empirical Bayes.However, this method uses the expected value and its variance to generate a predictive distribution of crash frequency [21].Another advantage of this method is that CMFs can be determined using small sample size.
One of the disadvantages of the carefully designed observational before-after study method is that it fails to identify confounding factors [18].Furthermore, the collection of traffic crash data after implementation could be time consuming and expensive [16].The cross-sectional method is an observational study which isolates the magnitude of implementing a selected countermeasure upon crash frequencies from the effect of other treatments applied at the specific study regions in a prescribed time period.Researchers applied this method to calculate CMFs for the effects of lane width, shoulder width, and presence of edge-line marking for frontage roads, and median width for freeways and rural multilane highways [22].However, the cross-sectional method sometimes overestimates the effect of the countermeasure due to the presence of confounding variables, whereas case-control method estimates the casual-effects while controlling impacts of confounding variables [23].Although this method can be used to investigate the effect of multiple countermeasures, the data collection and sample selection become very complex in case of multiple countermeasures [24].However, all these observational studies require high quality crash data of before and after implementation of countermeasures.To prove statistical significance, this data collection sometimes continues over multiple years.Table 1 presents the study duration of previous studies for developing different countermeasures' CMFs.
Recently, Banihashemi used four years rural highway crash data including about 5000 miles road geometry data and annual average daily traffic (AADT) and then proposed a heuristic method to calculate CMFs [33].Researchers investigated the safety effectiveness of the seatbelt and driver training using cohort method [34].The change in probability of crash occurring after implementation of countermeasure is estimated through the steps of this method.Meta-analysis  method and expert panel survey were also proposed to estimate CMFs [35].Furthermore, researchers have been exploring the global application of the HSM crash prediction algorithm, hence transferability of CMFs to different road networks in other countries, such as Italy, Canada, Denmark, Germany, and New Zealand [36].However, Gettman and Head found that traffic simulation platforms could be potential resources for evaluating vehicle interactions regarding safety aspects [37].Meanwhile, other researchers criticized this simulation-based approach arguing that built-in evasive algorithms in simulation platforms prevent modeling crash scenarios, and this can lead to the failure of developing the relation between risk behaviors and traffic crashes [38].However, Sacchi et al. [39] utilized video based traffic conflict analysis and found similarity in results with previous observation study.Recently, Shahdah et al. [40] developed an integrated method by combining observational before-after method and simulation-based method and drew a conclusion that the estimated CMFs by using simulation could match with the outcomes of observational studies.However, this study failed to develop an independent system for estimating CMFs using simulation.Table 2 summarizes the methods used in previous studies and their limitations.This paper aims to provide detailed step-by-step procedures for calculating CMFs of ITS countermeasures using simulation-based method and demonstrate the proof of concept in applying this method in real-world scenarios.

Proposed Method
A unique approach is proposed in this study, where estimation of CMFs could be conducted through four interconnected but distinct steps.The steps are (a) identification of countermeasures, (b) traffic simulation modeling, (c) conflict analysis, and (d) factor calculation.The overall method proposed in this research is illustrated in Figure 1.
To improve the safety aspects of a study location, the first step is to identify possible implementable countermeasures.
Previous studies on similar locations and experts' opinion can be used to identify countermeasures for the study location.Then the study site is modeled in traffic microsimulation software.It is important to note that several studies were conducted to evaluate the performances of traffic simulation modeling and study results showed that the simulation outputs were a statistically significant representation of the real-world [44].However, the traffic simulation models need to be calibrated and validated using collected actual traffic data to mimic the real-world traffic operation [45].In this study, the calibration is performed using real-world data and then the outputs from traffic microsimulation are considered as a satisfactory representation of the real-world.After developing base model, the necessary algorithms for simulation representation of proposed countermeasures are developed and coded.The simulations are run multiple times with different seed values to capture variability and initialize randomness in simulation traffic patterns [46].The vehicle trajectory files, which is a binary file containing the course of vehicle positions in the simulation network, are generated while running the simulation model.The next step is conflict analysis, where improvements in safety after implementing countermeasures can be inspected.Researchers defined conflict as an intersection of the trajectories of two or more vehicles and collision can happen if their movements remain unchanged [47].After a thorough research on the relation between crash and conflicts, a conflict analysis tool called "Surrogate Safety Assessment Model (SSAM)" was developed by FHWA [48].This conflict analysis tool uses the trajectory files imported from the runs of traffic microsimulation models.Five parameters are considered in SSAM tool to estimate the frequency of simulated conflicts; they are time-to-collision (TTC), postencroachment time (PET), deceleration rate (DR), maximum speed (MaxS), and speed difference (DeltaS).The threshold values of these parameters need to be adjusted with the driving behaviors of study locations.Then the number of conflicts is estimated by analyzing the trajectory files in SSAM tool. Figure 2 shows  the integration platform of traffic simulation software and the conflict analysis tool.
Finally, the numbers of conflicts calculated for base model (i.e., existing condition) and alternative models (i.e., proposed countermeasures) are compared to calculate the change in conflict frequency, i.e., CMFs after implementation of countermeasures.The following equation, proposed by [18], is utilized to estimate the CMFs of proposed countermeasures.A statistical analysis, Student's t-test, is preformed to establish the statistical validation of calculated CMFs.

Case Study for ITS Countermeasures
A case study was performed to validate the proposed simulation-based method and also to compute the values of CMFs when Intelligent Transportation Systems (ITS) countermeasures are being considered to lower crash severity and improve safety.Transportation professionals, automotive industry, and decision-makers throughout the world consider ITS measures as the viable solutions for traffic congestion reduction and safety improvement [49].Researchers categorized different ITS measures into six major categories: advanced traffic management systems, advanced travelers information systems, commercial vehicles operation, advanced public transportation systems, advanced vehicles control systems, and advanced rural transports systems [49].
Examples of advanced traffic management systems include, but not limited to, ramp metering (RM), variable speed limit (VSL) assignments, junction controls (JC), dynamic lane assignment (DLA), automated warning system (AWS), arterial management (AM), traffic signal monitoring (TSM), road weather information system (RWIS), and incident monitoring (IM).In this research, RM, VSL, JC, and DLA were contemplated.The descriptions of these countermeasures are provided in Table 3.
There are multiple traffic simulation tools (e.g., VISSIM, Paramics, CORSIM, SimTraffic, and AIMSUN) currently available, which can be integrated with SSAM tool for safety evaluation.In this study, VISSIM was selected for modeling ITS countermeasures as the corresponding software allows users to simulate user-defined driving behavior for modeling ITS equipment representations in simulation environment [50].
. .Traffic Simulation.In this study, a roadway segment of Interstate-76 (I-76), also known as Schuylkill Expressway, was modeled in microsimulation environment.This expressway has been experiencing significant traffic congestion since the traffic demand has almost doubled after the completion of this highway in 1960.A 15 mile long segment of I-76 (from the intersection of Schuylkill Expressway (I-76) and Pennsylvania Turnpike (I-276) to the intersection of Schuylkill Expressway (I-76) and U.S. Route-1) was coded in VISSIM simulation environment.In this network segment, 43% of total length had 2 lanes in both directions: 26% had 3 or more lanes and the rest had single lane roadway.Based on DVRPC's (Delaware Valley Regional Planning Commission) 2016 traffic counts dataset, the study site carried approximately 163,705 average annual daily traffic (AADT) in both directions [51].In this study, morning rush hours (6:30 AM to 8:30 AM) traffic volumes were projected as simulation traffic volume over a period of two simulation hours.Researchers have utilized the peak hour volume(s) of study region for predicting crash frequency due to time consuming nature of the traffic simulation runs [52].The results of simulation crash analysis using peak hour volume were utilized to develop statistically significant prediction models [53].The traffic model was calibrated using observed speed distributions and travel time data collected from field visits.A map presenting the study area is shown in Figure 3.
After developing base model of the study area, ITS countermeasures were coded in simulation environment.As mentioned earlier, RM, VSL, JC, and DLA were contemplated in this study and programmed in four separate model files using VISSIM VAP (Vehicle Actuated Programming) platform.The authors proposed to replace 22 static speed limit signs in the study region by a variable speed limit system.Based on the downstream congestion level, the upstream speed limit will be adjusted to improve the safety of travelers.The speed limit values displayed in a variable speed limit system were assigned as speed distribution to grasp drivers' stochastic behaviors in simulation.For example, assume, due to downstream congestions, the speed limit for upstream segment was estimated to 45 mph.Then 22% of all vehicles crossing the speed limit display will maintain a speed between 38 and 42.5 mph.Another 42% will drive at a speed between 42.5 and 47 mph, and the remaining will continue between 47 and 55 mph.Three ramp meters were recommended to implement on three entrance ramps along the study corridor.
For each simulation model scenario (base model and four alternatives' models), 10 simulation runs were generated with linearly incremented random seed values starting from 5 and ending at 45.As a result, each simulation model yielded 10 trajectory files to be exported into SSAM tool.
. .Conflict Simulation.Calibration of SSAM model is essential for integration of the real-world time-space distributions for the safety evaluation.This calibration task is conducted by adjusting its five parameters (i.e., time-tocollision (TTC), postencroachment time (PET), deceleration rate (DR), maximum speed (MaxS), and speed difference (DeltaS)) based on field collected data.Researchers found that the threshold values of SSAM parameters could vary depending on many factors, i.e., time of day, highway geometry, driving behavior, and drivers' age [54].For example, TTC threshold could be lower when significant numbers of the drivers in study area are more likely to drive aggressively, i.e., urban highways.However, due to data unavailability, SSAM model was not calibrated in this study.The default values of TTC and PET are 1.5 sec and 5.0 sec, respectively, which means when TTC ≤ 1.5 sec and PET ≤ 5.0 sec, this tool considers the events as the possibility of potential conflicts.These default values of TTC and PET are utilized by many other researchers [55] and thus applied in this study.Furthermore, there are three types of conflicts considered in SSAM tool.These types of conflicts are separated based on the conflict angles between the vehicles.They are (a) crossing collisions (when conflict angle between vehicles > 85 ∘ ), (b) rear end collisions (when conflict angle < 30 ∘ ), and (c) lanechanging conflicts (when conflict angle ≥ 30 ∘ and conflict angle ≤ 85 ∘ ).
The trajectory files imported from five VISSIM simulation models were analyzed in SSAM tool.The estimated total conflicts for each model were distinguished into three conflicts types, i.e., crossing collisions, rear end collisions, and lane-changing conflicts.Each type of conflicts was averaged over 10 runs for base model and four alternatives.The comparisons of the estimated conflicts between base model and different ITS alternatives are shown in Figure 4.

Results and Discussion
There were 40,332 total conflicts (i.e., traffic crashes) identified after analyzing base model.Each conflict was then categorized based on its conflict angle between vehicles.It was found that among the total of 40,332 conflicts there were 7,304 crossing conflicts, 12,914 lane change conflicts, and 20,114 rear end conflicts.However, after implementing Ramp Metering, the crossing conflicts increased by about 1,300, while other two conflict types were decreased from base scenario.Total 39,045 conflicts were found after implementing RM.The deployment of JC reduced the total number of conflicts from base condition by around 250 conflicts.Other ITS countermeasure, DLA reduced all types of conflicts from the existing conditions.However, the number of identified conflicts after implementation of VSL was found higher than base model, since lane change and rear end conflicts were increased by around 3,300 conflicts.Dynamic changes of speeds, i.e., speed limit of road segment, within a short time interval (each 15-minute interval) could be the reason of increasing conflict frequencies in simulation models.This sudden change provoked the drivers in simulation to perform lane change more frequently than before.As a result, the  number of lanes change and rear end conflicts were increased after DLS implementation.The total conflicts after implementation of these four ITS countermeasures were compared with base model.The percentage changes of conflicts from base model are tabulated in Table 4.
It is found that the number of injurious crashes could be reduced by 3.20%, 0.60%, and 5.10% from the base scenario after implementing RM, JC, and DLA, respectively.However, conflict frequencies were increased by 7.90% after implementing VSL.Finally the CMF values for these proposed ITS countermeasures were calculated.The calculated CMFs would be 0.97, 0.99, 0.95, and 1.08 after accomplishing RM, JC, DLA, and VSL on the roadways in selected study region, respectively.

Validation of Proposed Method
It is recommended that the proposed method needs to be validated either qualitatively or quantitatively.Qualitative validation examines the relationship between output(s) and variables, where quantitative validation compares the predicted values of output(s) against the values of similar output(s) calculated using a well-established method.In this study, a quantitative approach was applied during the validation process by comparing the calculated CMFs with the values estimated in previous research studies.Though, the technological feasibility of DLA and JC is under thorough investigations till today and they are not installed in different areas.So, there is limitation of crash data availability to validate the calculated CMFs of these two countermeasures.But the validation of other two countermeasures, i.e., RM and VSL, was performed in this study.
Researchers evaluated the safety effects of ramp meter implementation on a 9.2 miles segment of I-880 in Hayward, California [56].They utilized a crash prediction model developed by Lee et al. (2002) [57] where the crash frequency was estimated using traffic flow characteristics of the study region.It was found that 5% of total crashes could be eliminated after implementation of ramp metering on the segment of I-880.Kansas Department of Transportation (KDOT) conducted a safety evaluation of its ramp meters in 2010, and it was found that the crash rate dropped by 24% on roads where ramp meter was implemented earlier on I-435 [58].While they performed an observational before-after study to evaluate the ramp meters, there was no statistical evidence to validate the result of the study.Recently Chen et al. (2016) utilized a hypothetical traffic network to inspect the influence of ramp metering on safety improvement and observed 1.65% of the total crashes reduced after application of ramp metering [59].Another research [60] was conducted to investigate the application of variable speed limits using the same crash prediction model developed by Lee et al. (2002).After analysis, the researchers concluded that the variable speed limit could increase the crash potential when duration of intervention is low; i.e., frequency of speed limit changes.They found that variable speed limit could increase traffic crash by 3.14 to 4.94% depending on variations in road geometry.Saha et al. (2015) examined variable speed limits implemented in Wyoming and suggested that these variable speed limits could reduce crashes by minimum 24% [61].However the scope of this study was limited, and researchers only considered crashes that occurred during adverse weathers.These results were compared with the estimated CMFs using proposed simulation-based method.For this purpose, the percentage error was calculated using the equation given below: Table 5 represents the comparison between proposed method and crash prediction model.For calculating percentage error, the CMFs estimated using proposed simulationbased method were considered as "estimated value".On the other hand, the similar values calculated using crash prediction model, i.e., developed by Lee et al. (2002), were tagged as "theoretical value", since the authors utilized these values as references for validation.In Table 5, it is shown that CMFs calculated using proposed simulation-based method were slightly different than the same values calculated using crash prediction model, except study 4 cited for variable speed limit implementation.The percentage error for ramp meter was found to be 2.1% and 1.0% based on the data collected from study 1 and study 2, respectively.On the other hand, this percentage error for variable speed limit was 3.8% based on study 3.However, an error value of 42% was estimated in case of study 4.This significant difference between CMF values estimated using simulation-based method and crash prediction model could happen due to the limited number of crashes examined in study 4, since that study considered only crashes that occurred in adverse weathers.

Conclusion
Development of CMFs could be useful for the practitioners to perform safety evaluation, since significant statistical knowledge is required for utilizing HSM practically.The proposed simulation-based approach for estimating CMFs will provide a vital tool to them and assist them in traffic safety management.The application of this method could reduce the time dependency of developing CMFs over conventional observational method.Another benefit of using the proposed method is to achieve the ability of investigating the combined effect of multiple countermeasures.At present, the expected combined effects, i.e., combined CMF, are calculated by combining (for example: multiplication) the individual CMFs of proposed multiple countermeasures.Though, the validity of this method has not rigorously investigated.The simulation-based method proposed in this study can be used to model multiple countermeasures in simulation environment and evaluate their combined safety effect.Furthermore, traditional method of safety evaluation requires installation of countermeasures at crash prone locations, and the installation of ITS countermeasures could demand a "significant" investment.But the proposed simulation-based method could be used for evaluating ITS countermeasures before implementation at crash prone locations.As a result, potential errors in design could be avoided during planning stage.Additionally the impacts of geometric changes, for example, implementation of bus priority lanes [62], could be evaluated using proposed microsimulation method.Also, the proposed method could be utilized to validate existing CMFs for local representation, a validation process recommended by AASHTO [36].As a result, the practitioners can comfortably transfer the prior developed CMFs and apply those at different problematic regions.
In this study, the proposed simulation-based approach was used to calculate CMFs for four ITS countermeasures: ramp metering, variable speed limit assignment, junction control, and dynamic lane assignment.These countermeasures were coded in traffic simulation environment.The simulation models were run to generate the trajectory files including binary information of the course of vehicles in simulation.Then these trajectory files were imported to SSAM tool.In SSAM tool, the default values of TTC (= 1.5 sec) and PET (= 5.0 sec) were utilized to analyze the trajectory files.After analyzing, it is found that except variable speed limit assignment the proposed ITS countermeasures could reduce the number of crashes at crash prone location.Finally the CMFs for ramp metering, variable speed limit assignment, junction control, and dynamic lane assignment were estimated as 0.97, 1.08, 0.99, and 0.95, respectively.
. .Limitations of is Research.Even though the proposed approach could be performed in less time duration than current practices, the success of estimating CMFs using this method depends on the calibration of both traffic simulation and SSAM models.Another limitation of this method was lack of crash severity prediction.SSAM tool cannot draw a relation between trajectory information and severity of crashes due to evasive measures of traffic simulation.In this study, the traffic simulation model was calibrated, though SSAM model was not calibrated due to data unavailability.Furthermore, default thresholds of parameters were utilized.It is recommended to evaluate the change in conflict frequency with respect to change in these parameter values.The outcomes of the proposed method were compared with the CMF values developed for the proposed ITS countermeasures implemented in different study sites.However, the proposed method could not be validated with real-world traffic crash data of the study location, since the proposed countermeasures were in concept development stage at the time of this paper publication.As a result, the spatial dependency could not be evaluated in this study, and furthermore different time frames, i.e., midday rush hours, afternoon rush hours, and weekends traffic, were not modeled other than morning rush hours.
In future, the crash frequencies estimated using SSAM will be compared with available real-world crash records to validate the implementation of selected ITS countermeasures in the study region.Furthermore, a methodology will be developed to determine crash severity based on TTC, DR, and DeltaS thresholds.

Figure 2 :
Figure 2: Integration platform of traffic simulation tool and SSAM.

Figure 4 :
Figure 4: Conflict frequency for base model and proposed ITS countermeasures.

Table 2 :
Summarization of methods used to estimate CMFs.

Table 4 :
Conflict reduction percentage after implementation of ITS countermeasures.Negative percentage of conflict represents the increase of conflicts after countermeasure implementation.

Table 5 :
Validation of proposed simulation-based method for estimating CMFs.