Comparative study for safety performance measures of signalized intersection sites

The decision on which safety performance measure to use for diagnosis of sites with probable to profit from safety improvements is a major factor in making the right decision for the guidance of resources. This study concentrated on introducing a quantitative comparison of sites according to four crash-based methods suggested in the Highway Safety Manual (HSM). Archived crash data for three years are obtained to conduct a comparison course. As a case study, the analysis and measure of safety for 9 four-leg signalized intersections in Baghdad city have been done independently using Empirical Bayes method (EB-method), observed Crash Frequency (CF), Crash Rate (CR) and Empirical Bayes Adjustment (EB-adj.) as safety performance measures in HSM. In this paper the EB-method is used as a benchmark for comparison. The safety measures are evaluated through rank correlation analysis while hazard location identification results are compared through the use of values rank-based mean absolute error. Quantitative evaluation tests showed that each of EB-adj and observed CF correlated well with EB-method while CR method exhibited poor performance in comparison with EB-method and was the worst in hazard location identification. This result is quite confusing since many agencies still depend on the CR method in traffic safety analyses.


Introduction
Intersections' safety has always been a critical concern to traffic engineers since the traffic crashes often occurred at intersections due to multiple conflicts created by users travelling through. When studying intersection crashes, crash analysis on why they occur and how to reduce them is always a main priority. Signalized intersections should be continuously screened and systematically monitored during their cycle life. Overall, intersections known with the highest number of crashes were identified by safety practitioners and they focused their efforts and resources for studies, analyses and development in different countries [1].
In 2010, AASHTO published the Highway Safety Manual (HSM) as a huge tool for safety analyses [2]. Quantitative methods have been mostly collected in HSM. Before the first edition of HSM, transportation engineers did not have a single national resource of measuring information about analysis and evaluation of crash data.
There are several approaches to measure safety ranging from using crash data to crash prediction models which relate the expected crash frequency to traffic volume and geometric characteristics. Historically, safety analysts used (and many still use) traditional methods to identify and evaluate the safety of a site [3]. On the other hand, EB-method performs better than other safety measures and is considered the most suitable and dependable method for recognizing priority investigation sites and has the capabilities to account for the shortcomings of traditional methods. However, most highway agencies continue to use less effective methods [4,5]. For example, in the survey of hotspot identification methods in 8 European countries, it was found that most of the methods are primitive and are probable to include substantial inaccuracies [6,7]. Therefore, there is a critical need for research studies to compare different direct safety methods [4]. Also, only a few researchers have compared between different Safety Performance Measures (SPMs) and it is not clear which measure is the most consistent in the high-risk identification.
The main purpose of this study is to explore the performance of three crash-based methods in HSM with EB-method as the best available methods for identifying site (or sites) that probable to profit from safety improvements. Quantitative evaluation test for each of CF, CR and EB-adj. is based on measuring the safety performance of these methods for study sites with the corresponding performance measure of EB-method, the evaluation method is based on the ranking comparison that is derived from measuring safety for sites data using different crash-based methods for the same data.

Safety performance measures in HSM
HSM provides a set of thirteen SPMs that can be applied either isolated or in conjunction with the network screening process. The selection of performance methods depends on the availability of the required data from each SPM and Regression-to-the-Mean (RTM). Table 1 presents the stability of SPMs, the last three methods in the table below are more consistent and dependable methods to measure safety but require more data for analysis based on CF and CR. The safety performance metrics are described in section 4.2.3 from part B in HSM along with their strengths and limitations. The selected SPMs can be applied to facilities like intersections and roadway segments using different screening methods. Only a simple ranking method can be selected to screen intersections [2,8].

HSM predictive method in safety assessment
HSM 2010 provides guidance as to best practices that allow for prediction of the safety performance of road facilities with specific site conditions. The HSM also provides a series of commonly used tool that facilitates the understanding of the relationships between roadway characteristics and crashes. A series of crash prediction models are available in HSM (Part C) commonly mentioned to as safety performance functions (SPFs), which can be utilized to estimate the CF on particular site as a function of traffic volumes, geometry of roadway, traffic control type, and other factors. SPFs can be valuable for estimating the safety impacts of specific sites.
The predictive models for Four-Leg Signalized Intersections (4SG) in HSM (Part C) are composed of three basic elements: SPFs, Crash Modification Factors (CMFs), and a calibration factor. SPFs establish a basis for predicted crashes for 4 SG in consideration of the effects of major and minor traffic volume, and other factors. SPFs for intersections take the following general form given in Equation (1): Where: Nspf is the predicted average CF for a site with base conditions, AADT +,-./ is the annual average daily traffic for major road, AADT +23./ is the annual average daily traffic for minor road and β0 β1β2 are the estimated parameters.
In order to account for the unique characteristics of a specific site, SPF for 4SG is multiplied by the CMFs. Appraisal of the effect of multiple characteristics or treatment HSM presume that CMFs can be multiplied together. A list of CMFs for a variety of geometric and operational treatment types for urban signalized intersection, backed by robust scientific evidence available in HSM and other references is given [9,2].

EB-method in the HSM
The EB-method in HSM combines an estimation of the observed crash data of the study site with characteristics of similar sites using SPFs to predict the expected number of crashes. The EB-method in the HSM is used as part of the predictive method.
EB-method performs better than other safety measures and is considered as the most suitable and dependable method for recognizing priority investigation sites [4,5]. The weighted adjustment factor in EB-method is used to determine how much "weight" is given to the two estimate methods: the estimate derived using SPFs based on site (intersection or roadway segment) with similar feature and observed CF on the site of interest. The over dispersion parameter (k) that coincides with SPF is used to define the value of the weighted adjustment factor. Eq. (2) shows how the site-specific crashes according to the EB-method are calculated; Eq. (3) is for obtaining weighting adjustment factor [8]: Where Nexpected is the estimate of expected average CF for the study period, Npredicted is the predictive model estimate of crashes for the study priod according to the SPF; Nobserved is the observed crashes at the site over the study period, w is the weighted adjustment to be placed on the SPF prediction, and k is the over dispersion parameter from the associated SPF.

Observed crashes and crash rate methods in safety assessment (traditional methods).
Oftentimes, observed CF and CR are used as a measure to recognize and prioritize locations as in need of treatments and for appraisal of the usefulness of countermeasures. In the CF and CR methods, the study period is often three to five years in safety analyses. Relatively short periods of time should not be used for analyzing and assessing safety at site due to effect Regression-to-the-Mean (RTM) bias [8,1].
In order to address some limitations of the CF and CR methods many agencies developed and applied statistical models using regression analysis. These models address RTM bias and also provide the capability to reliably estimate of crashes for not only existing roadway conditions but also for various alternatives design for the site prior to its building and use. When historic crash data for a specific site or facility can be combined with the predicted crashes from model, the reliability of crash estimation is improved due to accounting RTM bias [10].
A crash frequency is obtained by counting the number of crashes at an intersection or a roadway segment, over a certain period of time, while the crash rate normalizes the number of crashes relative to exposure (traffic volume) by dividing the total number of crashes by the traffic volume, the traffic volume for intersection includes the total number of vehicles entering it (the total entering vehicles, TEV, is a sum of the AADT for major and minor street), measured as million entering vehicles (MEV).

2.4-Expected average CF with EB-adjustment method (EB-adj).
This method of measuring safety is similar to the EB-method but the version of the EB-method implemented here uses yearly correction factors for consistency. This method considers for RTM bias and has greater reliability than observed crash frequency and crash rate [8].
Explanation of steps to calculate the expected average CF with EB-adjustment is presented in part B, Ch.4 from HSM (HSM p.4-58).

Study sites
A sample of 9 sites is studied in view of the research requirements according to the criteria of: 1-All study sites are signalized intersections and located in the urban area surrounding the CBD of Baghdad city. 2-No major changes in geometric design and surrounding area of sites during the study period (2015-2017), and 3-Availability of crash data for the corresponding period. Figure 1 shows the locations and names of study intersections in Baghdad city.

Crash data
For purpose of this study, two sources have been used to collect crash data, the first source from traffic department using the crash investigation reports that have been prepared by a traffic-policeman who is responsible about reporting crashes. The second source is the police stations according to the geographical locations of the study sites. Crash data have been collected for three years from 2015-2017 for 9 intersections. According to crash data there are 143 crashes that occurred in 9 sites during 3 years, the average crashes for intersections was 5.29 crashes per year. In this study, crash data are represented for the crashes between vehicles only.

Traffic volume
By the aid of positioning of surveillance cameras at each study site as well as field observation, necessary input data of traffic volume was collected. AADTmajor and AADTminior are estimated from accounting peak hour volume (PHV) for each site by using conversion factor (K) in Highway Capacity Manual (11) for the urbanized area.
AADTmajor and AADTminior were estimated from accounting peak hourly volume (PHV) at field for each site and using converting rules to reconcile PHV as vehicle per hour to AADT as vehicle per day. The AADT is calculated based on traffic survey in 2017, AADT for 2015 and 2016 which have been estimated based on used growth factor for traffic that has been obtained from the relevant authority of urban transportation study in Baghdad city.

1-Test (1): comparative analysis based on spearman rank correlation coefficient (ρs)
In the field of test and comparison the performance of two safety measures for crashes, several studies used ρs for this purpose [12][13][14]. This correlation coefficient is predominantly used as a nonparametric alternative to a traditional coefficient of correlation and can be utilized under general conditions. ρs is calculated to determine the level of agreement between each pair of rankings and calculated as shown in equation (4). A score of 1.0 characterizes complete correlation and a score of 0 reveals no correlation. A benefit of using ρs is that when examining for correlation between two sets of data, it is not required to make assumptions about the nature of the populations sampled [13].
Where: n is the number of items ranked and di is the differences between two rankings for item i.
The ordered data pairs are randomly matched, under a null hypothesis of no correlation. Equation (5) is used to test the null hypothesis. Z-value calculated can be compared to a critical Z-value at 95% level of significance. For this study, a Z-value of 1.96 is identified, representing a 95% level of significance. Significance levels of 95 % would be fulfilled by the ρs values in excess of 0.69.

2-Test 2: comparative analysis based on rank positions
Many studies performed a comparative analysis between two SPMs based on rank-based mean absolute error (rank-based MAE) [4,15]. The rank-base MAE quantified how close one set of ranks (e.g., the ranks in the subject SPM) was to the other set of ranks (e.g., the ranks for reference SPM); lower MAE value indicated that the two sets had a less relative error. The comparison between rank positions of two sets of data is achieved according to the equation (6): Where: i is the location index (i = 1 . . . n), n is the locations number, rank (xi) is the rank of location i on the basis of reference SPM and rank (yi) is the rank of location i on the basis of the other performance measures that will be compared. The following steps are performed in order to analyse data and detect the results:

Data Analysis and Results
Step A:measure safety according to crash-based methods In this study, measure of safety is calculated as follows: 1-Expected average CF is obtained based on EB-method according to the procedure of HSM predictive method for 4SG in the urban area (The details steps, functions, tables and figure used in a predictive method for intersections are described in HSM, part C, chapter 12, section 12.6.2). 2-Observed CF represents the total number of crashes for the three-year study period at each site. 3-CR for the site is estimated for each intersection by dividing the total number of crashes by MEV for the three-year study period. 4-Expected average CF is obtained based on EB-adj. method according to the procedure listed in HSM (HSM part B, Ch.4, pp.58-65).

Step B: ranking intersections based on outputs.
In this step each intersection will be ranked based on their outputs as follows: Each intersection will be ranked based on the observed CF for the 3-year study period for CF method, crash/MEV for CR and expected average CF /year for EB-method and EB-adj.
In the process of screening sites for further details of evaluation to identify proposed improvements and countermeasure, ranking is achieved from highest to lowest values represented by studied safety performance measures (EB-method, CF, CR and EB.adj). In this study, it is intended to conduct a comparison between the output of simple ranking according to the value of safety measure associated in each site based on SPMs used. Table 2 shows that site No.5 is ranked 1 due to EB (7.669 crash / year), CF (21crash for three years) and EB.adj. (7.53 crash/year) while site No.6 is ranked 1 due to CR (0.304 crash/MEV). Although there is similarity in ranking of site No.5 (14 th Ramadhan intersection) and site No.9 (Al-Saylow intersection) due to EB-method, CF and EBadj. Methods, there is no such similarity for other sites and further statistical analysis and priority optimization are appreciated. Table 3 presents a result of the comparison of the ranking of intersections based on CF, CR and EBadj. methods as subject SPMs versus the ranking of intersections based EB-method as reference SPM according to the ρs and rank-base MAE values. The results showed that the highest rank correlation coefficient and lowest MAE value is obtained by EB-adj method as compared with EB-method. The second higher rank correlation coefficient and lowest MAE are obtained by the CF method. The rank correlation coefficient values of these ranking comparisons are 0.899 for EB-adj. and 0.791 for CF, which are significant at 95% level of confidence. Further, a comparison between methods based on value on rank-based MAE revealed that the EB-adj is very close to the EB-method with lowest relative error (MAE value is 0.888 based on comparison ranking sites according to the EB-method and ranking sites according to the EB-adj).

6.3-Comparative analysis
As shown in the result in Table 3, analyses showed a convergence in ranking intersections based on EB-method and CF method. On the other hand, it can be noted that the CR method presented the insignificant rank correlation coefficient and highest value of rank-based MAE with EB-method.
As known, the CR method assumed a linear relationship between crashes and traffic volume. Based on results, it is clearly the effect of traffic volume of biasing in results of identifying prioritizing of sites, since the CR method identified the sites with the lowest volume at the highest rank (site No 6,7, and 8). Further, the rank order of sites based on the CR method is completely different to EB-method, it is near to the ranking sites based on traffic volume (from the lowest value to higher value).

Conclusion
Safety assessment tools are a key component in the process of identifying potential sites for safety improvement in many countries. The judgment on which safety performance measure to employ for diagnosis of locations with probable to profit for safety improvements is the most important factor in the evaluation of intersections. This study concentrated on introducing a quantitative comparison of site ranking among four SPM suggested by the HSM when implemented to a set of signalized intersections in the urban area of Baghdad city.
Four crash-based methods are used in this study to distinguish potential site (or sites) for safety improvements. Measure of safety based on the results of three SPMs (EB-method, CF and EB-adj.) identified 14 th Ramadhan intersection (site No.5) as a critical site for safety improvement.
Further, in this study, tests for consistency and performance of each measure to rank the sample sites were found through the rank difference between each safety performance measure and EBmethod as a benchmark. The results have suggested an acceptable agreement between the most dependable safety performance measure (EB-method) and each of EB-adj and CF while the CR method performed poorly in comparison with the EB-method and was the worst in terms of MAE, the poor performance of the CR method in this the study showed that simply dividing the CF by the traffic volume does not correctly account for the volume difference across locations from a safety evaluation viewpoint and leads to erroneous identification results.
In spite of this, many agencies still depend on CR method in the analyses of the crash data. The result of this study showed that CR method failed to take account of differences in traffic volume across intersections from a safety assessment viewpoint and that led to false identification results in comparison with EB-method. On the other hand, the CF method as a SPM performed better than the