On random-parameter count models for out-of-sample crash prediction Accounting for the variances of random-parameter distributions

One challenge faced by the random-parameter count models for crash prediction is the unavailability of unique coefficients for out-of-sample observations. The means of the random-parameter distributions are typically used without explicit consideration of the variances. In this study, by virtue of the Taylor series expansion, we proposed a straightforward yet analytic solution to include both the means and variances of random parameters for unbiased prediction. We then theoretically quantified the systematic bias arising from the omission of the variances of random parameters. Our numerical experiment further demonstrated that simply using the means of random parameters to predict the number of crashes for out-of-sample observations is fundamentally incorrect, which necessarily results in the underprediction of crash counts. Given the widespread use and ongoing prevalence of the random-parameter approach in crash analysis, special caution should be taken to avoid this silent pitfall when applying it for predictive purposes.


Introduction
Since its introduction by Milton et al. (2008) and Anastasopoulos and Mannering (2009), the random-parameter approach has attracted considerable research interest and has become the benchmark for crash analysis (Mannering and Bhat, 2014).Numerous studies have demonstrated its superiority in interpretability, with a substantial improvement in goodness-of-fit and the ability to account for unobserved heterogeneity (Mannering et al., 2016).Recent studies reported that the approach empirically underperformed in out-of-sample crash prediction compared with its fixed-parameter counterpart (Tang et al., 2019;Hou et al., 2020).However, this underperformance can be readily attributed to using predictions that are based only on the means of the random parameters while ignoring their variances.Our study therefore attempts to demonstrate such a bias by providing a straightforward yet analytic solution to incorporate both the means and variances of random parameters for unbiased prediction.

Methods
Let Y i denote the number of crashes for observation i(i = 1, 2, ..., n) during a certain period.Z i and X ij represent the exposure and the jth explanatory variable, respectively.When developing the randomparameter approach to model crash frequencies, without loss of generality, we formulate the following functional form given the random, non-negative, and integral nature of crash counts: where λ i is the parameter of Poisson distribution (i.e., the expected number of crashes), β 0 is the intercept, β ji (j = 1, 2, ..., k) refers to the jth regression coefficients, and u i denotes the unstructured effect.Compared with the negative binomial model (also known as the Poissongamma model in which exp(u i ) follows a Gamma distribution with mean 1 and variance α; Lord and Miranda-Moreno, 2008), a more flexible alternative is to assume u i as a normal distribution as suggested by Lord and Mannering (2010): Unlike the conventional Poisson-lognormal model with fixed parameters, by specifyingβ ji as a predefined distribution with mean β j and variance σ 2 j , each observation now has its individual coefficients.In theory, β ji can be assumed to follow any distributions, but the normal distribution is mostly used, given its better statistical fit (Anastasopoulos and Mannering, 2009;El-Basyouny and Sayed, 2009;Ukkusuri et al., 2011;Venkataraman et al., 2013;Chen and Tarko, 2014;Xu and Huang, 2015;Amoh-Gyimah et al., 2016;Meng et al., 2017;Cai et al., 2018;Hou et al., 2018Hou et al., , 2021;;Tang et al., 2019).
A challenge arises when applying the estimated random-parameter model to predict the number of crashes for out-of-sample observations, as in reality, it appears to be impossible to know the exact value of If β j is used as the default to estimate the number of crashes (i.e., λ [i] ; Mannering et al., 2016;Tang et al., 2019;Hou et al., 2020), we then have: However, the expectation of λ [i] is not only dependent on β j but also on the variances σ 2 j and σ 2 u .Indeed, if we take a Taylor series expansion near the point (β can be closely approximated using a quadratic polynomial as follows (Wong and Wong, 2019): where ∂β j [i] and ∂u [i] refer to the first-order partial derivative of λ [i] to β j [i] and u [i] , respectively.
are the corresponding second-order partial derivatives.
Taking the expectation on both sides of Eq. ( 4), we have: The difference between Eq. ( 5) and (3) can then be quantified as: That is, using λ [i] as the prediction will always result in a downward bias, given the convex property of the link function in Eq. ( 1).This silent pitfall has been ignored by safety analysts when applying randomparameter models for predictive purposes (Tang et al., 2019;Hou et al., 2020).
To validate, we conducted the following numerical experiment, given its advantages in maneuverability and reproducibility.We set n = 5000, Z i ∼ Uniform (200,20000), X i ∼ Uniform(0, 20), β 0 = 0.004, β 1i ∼ Normal(0.70,0.06 2 ), β 2i ∼ Normal(0.03,0.03 2 ), and u i ∼ Normal(0,0.60 2 ).Only one explanatory variable was considered here for simplicity.We then sequentially sampled the values of Z i , X i , β 1i , β 2i , and u i from the aforementioned distributions, calculated λ i , and generated Y i using the freeware RStudio (R Core Team, 2013).Such a simulated dataset matched well with the empirical data collected by Tang et al. (2019) for rural two-lane roads in Pennsylvania.Detailed descriptive statistics of our data under investigation are listed in Table 1.
To evaluate the out-of-sample predictive performance of the randomparameter count model, we performed a 10-fold cross-validation (James et al., 2013;Hou et al., 2021), in which the original 5000 observations were randomly partitioned into 10 mutually exclusive subsets of equal size.Each time, one subset was left out as the validation while the remaining subsets were combined for model estimation.This process was repeated 10 times.We repeatedly predicted the crash frequencies for validation samples as λ [i] and E(λ [i] ) using Eq. ( 3) and Eq. ( 5), respectively.In each round, in addition to the random-parameter model, we also developed its fixed-parameter counterpart for comparison.To characterize their overall predictive accuracy, since each metric only reflects partial aspects of error characteristics (Chai and Draxler, 2014), we computed three commonly used measures namely the mean bias error (MBE), the mean absolute error (MAE), and the root-mean-square error (RMSE).We then plotted the cumulative residuals (CURE; Elvik et al., 2013;Hauer, 2015;Srinivasan et al., 2016;Tang et al., 2019;Hou et al., 2020) to visualize any anomalies.Models with a CURE curve oscillating around zero within the 95% confidence interval (CI) are regarded as reliable in predictive performance (Hauer, 2015).Consistent with Tang et al. (2019) and Hou et al. (2020), we also used the latest econometric software LIMDEP 11 (Greene, 2016) to estimate our models.Reliable parameters for the random-parameter count models could be estimated via 600 Halton draws based on the simulated maximum likelihood method (Tang et al., 2019;Hou et al., 2020).The results are presented in Table 2 and Fig. 1.

Results
As Table 2 shows, although the predictive accuracy of E(λ [i] ) was slightly lower than that of λ [i] in terms of the average magnitude of P. Xu et al. absolute values according to the MAE results, E(λ [i] ) was less sensitive and more robust to outliers, given a lower RMSE value.It is also interesting to find that when E(λ [i] ) was used as the predictive value, the predictive performance of the random-parameter model was comparable with that of its fixed-parameter counterpart, as both models produced similar MAE and RMSE values. However, the MBE value of E(λ [i] ) was the closest to zero, suggesting that E(λ [i] ) is least likely to be affected by systematic over-or under-prediction bias.This finding becomes even more plausible when comparing the MBE values of λ [i] and E (λ [i] ).
More importantly, as illustrated in Fig. 1, when plotting the CURE curves over the entire range of a variable of interest, only λ [i] consistently exhibited a monotonically increasing trend, resulting in a substantial deviation from zero in the end.By comparison, although the CURE curve for E(λ [i] ) was not always close to zero, it oscillated broadly within the 95% CI.A similar fashion was also observed for the fixed-parameter model.

Discussion
A major challenge faced by the random-parameter count models in transferability and extrapolation is the unavailability of unique coefficients for out-of-sample observations.By virtue of the Taylor series expansion, we provided an analytic solution that includes both the , respectively.Here, Y [i] and Ŷ [i] represent the observed and predicted number of crashes for validation sample [i], respectively.† Ŷ ), where β 0 , β 1 , β 2 and σ u are the parameters estimated by fixed-parameter model.
P. Xu et al. means and variances of the random parameters for unbiased crash prediction.Compared with the simulation method 1 (Alogaili and Mannering, 2020;Islam et al., 2020;Hou et al., 2021), our approach is not only theoretically sound but also readily applicable without much extra computational cost.
Based on a well-designed numerical experiment, we further demonstrated that simply using the means of random parameters to estimate the number of crashes for out-of-sample observations is essentially incorrect, which necessarily results in an underprediction of crash counts in the long run.This finding should be robust, particularly after adjusting for confounders arising from small sample sizes (Lord and Miranda-Moreno, 2008), misspecification of functional forms (Wang et al., 2020), underreporting of crashes (Imprialou and Quddus, 2019;Zeng et al., 2020), excess zeros (Lord et al., 2007;Liu et al., 2018), measurement errors in exposure (Xie et al., 2018;Kamed and Sayed, 2020), omitted variables (Mitra and Washington, 2012), spatial correlation (Xu et al., 2017;Dong et al., 2020;Ziakopoulos and Yannis, 2020;Cui and Xie, 2021), and temporal instability (Mannering, 2018;Alnawmasi and Mannering, 2019;Behnood and Mannering, 2019;Islam et al., 2020), all of which are frequently encountered in empirical studies.Accordingly, the underperformance of random-parameter count models reported in previous studies (Tang et al., 2019;Hou et al., 2020) arises very likely from the convenient use of λ [i] as the prediction.That is, the conclusion that random-parameter count models were less reliable and inaccurate for out-of-sample crash prediction reached by Tang et al. (2019) and Hou et al. (2020) seems implausible and misleading, which should not be attributed to the modeling technique itself, but rather to the artificial and incorrect use of model estimates by analysts.Given the widespread use and ongoing prevalence of the random-parameter approach in crash analysis, special attention should be paid to this fundamental issue when applying it to predict crash frequencies for outof-sample observations.
For future research, since the data structures in empirical settings are much more complicated and intractable than anticipated (Mannering et al., 2020), a thorough comparison of the predictive performance of random-parameter count models with that of other techniques, such as the emerging deep learning methods (Bao et al., 2019;Cai et al., 2019), is highly recommended.

Table 1
Descriptive statistics for the simulated data.

Table 2
Predictive accuracy metrics for out-of-sample observations based on 10-fold cross-validation.MAE, and RMSE refer to the mean bias error, mean absolute error, and root-mean-square error, which are calculated as 1 500