Introduction

The purpose of the current paper is to present a useful multi-level approach to estimating the road safety impacts of street lighting when relighting a conurbation is carried out and apply it to the lighting change carried out in the city of Leeds.

The study reported here concerns the relighting of the city of Leeds, UK, which changed its predominantly orange light lamps to white light ones. The number of replacements was nearly 80 thousand. This report largely follows the methods given in the open access paper, Marchant et al. (2020), concerning the relighting of Birmingham. As remarked in that paper, Road Traffic Collisions (RTCs) are a major global health issue, see World Health Organization (2018) with approximately 1.3 million people killed each year. A sizeable proportion of the world’s RTCs occur in the hours of darkness. It would be beneficial if a guaranteed way of reducing the darkness RTC toll of death and injury could be found. Increasing road lighting in terms of brightening and whitening is believed by some to be very effective in achieving the desired aim.

In terms of background history of the issue of RTCs and road lighting, Boyce (2008: p.6), in the book ‘Lighting for Driving’, makes the point that “… the fundamental purpose of lighting on vehicles and roads is to enhance the safety of road users by increasing the visibility of the road ahead …”. The book gives a positive view of lighting and accepts at face-value the evidence of TRL 929 (Scott, 1980) that brighter lighting caused an observed reduction of the night to day accident ratio, i.e., the daylight adjusted darkness RTC rate, with increasing road surface luminance.

In the next year, the systematic review of Beyer and Ker (2009) on street lighting for preventing road traffic injuries, carried out under the auspices of the Cochrane Collaboration, was published. (The protocol for it is Beyer et al., 2005.) The review’s authors identified 145 potentially relevant studies, of which just 17 met their inclusion criteria, specified in their protocol. In addition, the review points out that ‘the methodological quality of the included studies was generally poor’. Of those 17 included studies there were no Randomised Controlled Trials discovered. All 17 studies were controlled before-after studies. On the roads receiving the new lighting, the numbers of RTCs, in the individual studies tended to be small, mostly less than one hundred during the before and after periods. Also, certainly none were at the scale of a whole city. Note the Cochrane Review did not include the study report 929 (Scott, 1980), used to justify the Leeds relighting scheme, because of its poor methodological design.

It is also worth noting that although it is obligatory to have a protocol for any Cochrane Review, (Higgins et al., 2022), it is not known whether any of the lighting studies included by Beyer and Ker had one and it seems there was no protection against the possibility of the decision to write a Lighting and RTCs report only being made after the effect of the lighting change had become known. This would open up the possibility of publication bias. The conclusion of the Cochrane Review that road lighting may improve road safety has been criticised, see the Comments on the Review https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD004728.pub2/read-comments and also independently by the University of Cambridge team led by Prof. Sir David Spiegelhalter, FRS at https://understandinguncertainty.org/node/231. Two of the issues raised are publication bias and the potential for a type of bias known as Regression Towards the Mean (RTM). The latter arises if the reason that the roads in these studies received the new lighting was because they had an unexpectedly high collision rate. However the problem is that the high collision rate, bringing about the relighting, could be just a statistical fluctuation away from its usual more average level, to which it subsequently relaxes. This then is likely to lead to a spurious claim that it was the lighting that brought about the relaxation, rather than just natural statistical variation. RTM was first recognised by Galton (1886) and is described by Bland and Altman (1994) and Marchant (2008).

Some further problematic matters with lighting research are discussed in Marchant (2017). Faults in more general research are discussed by Ritchie (2020).

The National Institute for Health Research (NIHR) funded ‘Local Authority collaborators’ National Evaluation of Reduced Night-time Streetlight’ (LANTERNS) project could not detect an effect on RTC-rate (using 95% confidence limits) through any change to lighting, including relighting with white light, despite its extensive data, see the report of Perkins et al. (2015). An abbreviated version was published by Steinbach et al. (2015). The latter says it involved changes to lighting using data from 62 of the 174 local authorities in England and Wales and although originally designed only to detect the effect of reduced lighting on roads, in a variation to protocol, it also examined the effect of changing to white light, on 15,833 km of roads, such as has been done in Leeds. The 95% confidence interval for the day-adjusted night collision rate ratio (CRR) of changing roads to white light from the previous lighting was found to be (0.93, 1.09). (The CRR is the ratio, after to before, of the ratio of night-time collisions to those in day-time.) The conclusion of the study is therefore that no effect of changing lighting on collisions was detected.

A further view on the safety benefit of road lighting is given by Fotios and Gibbons (2018) who state “… there appears to be little, if any, credible empirical support for light levels recommended in much current road lighting guidance …”. It therefore seems worthwhile checking what the result of large and costly lighting projects do achieve in terms of safety.

Leeds City Council made a successful bid to the UK Government, to acquire Private Finance Initiative funds to go ahead with the relighting of the city with white lamps to replace predominantly orange ones. The bid made several references to claims of the effectiveness of new lighting in reducing the RTC-rate. For example, “Nationally, there is evidence that good street lighting can reduce night-time road accidents by between 15 and 35% (source – Institution of Lighting Engineers, also TRL reports 586 and 929).”, Leeds City Council (2004). The Leeds document claims a great benefit to cost ratio from the scheme. Leeds started the programme of replacement of old lamps with new white-light ones in mid-2006.

The study report 929 (Scott, 1980, also reported in Hargroves & Scott, 1979) cited by Leeds City Council does indeed suggest that brightening roads does give lower daylight adjusted darkness collision rates, implying a safety benefit. However, the design of the study is poor because it is ‘cross sectional’ such that different roads with different lighting levels are compared. Such a design allows many confounding factors to mislead, through the difference in the nature of the roads and the traffic on them. Therefore, in such a study ‘like’ is not being compared with ‘like’ and it may well be that there are other differences between the roads, correlated with brightness, which cause the observed safety differences in safety. Another study, Jackett and Frith (2013), of the same cross sectional design is criticised by Marchant (2019), and more fully in Marchant (2022). These critiques give some examples of how confounding variables might operate and suggests longitudinal studies are superior.

The second report (586) quoted by the Leeds bid (Sabey & Johnson, 1973) was a controlled before and after study on trunk road sites where lamps had been changed (“improved”), using other trunk roads as comparators. It is surprising that this study was referenced by Leeds in its bid to acquire new street lighting, as it showed variable effect on safety and none were statistically significant for the roads on which there were 30, 40 and 50 miles per hour (mph) speed limits. (Only 70 mph roads showed a statistically significant effect.) Roads with limits of 50mph or below form the bulk of those in Leeds and indeed other UK cities.

It seems highly likely that studies of the kind that Sabey and Johnson (1973) performed will suffer from the statistical bias of Regression Towards the Mean (RTM), described above.

In studies of the Sabey and Johnson type, controlled before and after studies, it is important that the comparators providing the control are very similar to those receiving the intervention.

It therefore seems that the evidence adduced by LCC for the bid for new lighting is weak, so it is worthwhile checking what the result of the large public funding for the lighting achieved in terms of road safety. It is the aim of the present confirmatory study to see if the relighting programme resulted in its objective of reduced RTCs. The differences between a confirmatory study and an exploratory one are explained in Schwab and Held (2020). The use of multilevel models is not unknown in RTC research, see for example Jones and Jorgensen (2003), but we are unaware of this method being used to estimate the impact of any large-scale implementation.

Materials and Methods

Using data on the fitting of new white lamps and on road traffic collisions, this present study modelled the impact of relighting on Leeds RTCs. The weekly numbers of personal injury RTCs in different areas were modelled as a function of time, on the increasing number of white, broad-spectrum, lamps installed and operating, while comparing with other areas where lighting is changed at different times and by different amounts. The underlying trend in collisions in the absence of changed lighting is fitted by a polynomial in time, with indicator variables for month of the year to account for seasonality and another set to account for differences in traffic in weeks containing a public holiday.

The analysis used a multilevel modelling approach, (Goldstein, 2010; Snijders & Bosker, 2012) which is appropriate for the structure of the data, as the RTCs and lamp changes, implemented in a ‘stepped wedge’ fashion over the time period, are nested within the 107 Middle Layer Super Output Areas (MSOAs), of the city. (MSOAs are geographical units used in England and Wales to disseminate neighbourhood statistics providing a balance between geographical granularity and data reliability.) The stepped wedged introduction of the replacement lamps, the ‘dose’ of lighting, constitutes a sporadically interrupted time series. To maximise transparency and help guard against reporting bias, the protocol for the study was sent to independent custodians on 5 August 2016, see the Acknowledgements section. (The protocol is given in the Supplementary Materials provided in the open access repository https://osf.io/kfg64/https://doi.org/10.17605/OSF.IO/KFG64.) The computation of the length of darkness, and its complement, the length of daylight, used the same code as for the Birmingham study, (Marchant et al., 2020, Supplemental Material 002) but was changed for the latitude of Leeds, 53.8 degrees North. (The analysis dataset is available to download from the public, open access repository https://osf.io/kfg64/https://doi.org/10.17605/OSF.IO/KFG64.)

The key aim, as for the LANTERNS study noted above, is to determine how the daylight adjusted darkness collision rate (that is the ratio of RTC-rates occurring in darkness to those occurring in daylight) changes when the new lighting is introduced. This change factor, for a given amount of relighting, is the collision rate ratio (CRR), as it is the ratio of the collision rate with the increase in relighting to that without. This method of checking the impact of an implementation could apply to topics other than lighting, where an intervention is incrementally introduced.

Lamp data

A full street lamp inventory for the city was made available, by Leeds City Council, for our analysis. Although the Leeds inventory was given in 2016, it was subsequently found that Part Night Lighting (PNL), whereby some lamps are switched off for the early hours of the morning, had been gradually introduced by the City Council on some roads, in various areas, commencing 1 Oct 2013. Therefore, in order to avoid the confusion of some areas having PNL, the time series used for analysis ended on Sunday 29 Sept 2013 and ran from Monday 03 Jan 2005 until then.

Each new installed lamp was assigned to the appropriate one of the 107 MSOAs for Leeds, in the data file, on the basis of its geographical coordinates (Easting and Northing). The date that the new lamp was installed allowed the cumulative number of new white lamps, operating at a given time (week) in a given MSOA, to be known. By this means, the file for analysis contained the number of new lamps operating in each week in each MSOA, between week 1 (starting Monday 03 Jan 2005) and week 456 (starting Monday 23 Sep. 2013) (inclusive). A total of 78,189 lamps were introduced up to 29 Sept 2013. Had the time series analysed continued to the end of 2015 there would have been 79,729 lamps. Thus, there was only a small reduction, of 1.9%, by reducing the time series by two and a quarter years of data. Therefore, the data series used comprises virtually all the lamp changes.

Collision data

STATS19 data (personal injury collisions reported to the Police) were sourced from the UK Department for Transport (http://data.gov.uk/dataset/road-accidents-safety-data). The STATS19 data contains a ‘lighting code’ for the reported state of natural lighting at the time of the collision. Collisions were assigned a binary code of either daylight or darkness. Daylight is defined as starting 30 min before sunrise and ending 30 min after sunset; otherwise, it is darkness. The Birmingham data of the previous study, Marchant et al. (2020), showed that there was a small percentage of inconsistencies between the reported state of daylight or darkness and the solar position at the reported time of occurrence, but this was considered not to materially affect the conclusion of that research.

Each RTC was assigned to its MSOA, through the geographical coordinates (Easting and Northing) given in the STATS19 data and to its appropriate week, from the date of its occurrence. The numbers of RTCs occurring in darkness and daylight in each of the 107 MSOAs, in each of the 456 weeks were included in the analysis data file. The unit of ‘Week’ was chosen for the analysis in order to balance the varying traffic flows within a week. A total of 13,977 daylight collisions and 5,302 darkness collisions were recorded over the study period of 8 years 9 months. The 24 h weekly RTC-rate was simply constructed by summing the rates of RTCs assigned to daylight and to darkness. A suspect pattern was noticed in this official STATS19 RTC data in the years 2012 and 2013. This led to an additional analyses, excluding those two years. In this truncated series, there were 11,953 daylight collisions and 4,078 darkness collisions while 76,288 replacement lamps were installed.

The variables in the analytical dataset are listed in Table 1.

Table 1 The variables in the dataset
Full size table

Multilevel modelling

It is the daylight adjusted darkness RTC rate, (= the ratio of darkness to daylight RTC rates), the conventional measure used in studies of lighting effectiveness e.g., Perkins et al. (2015), that is the focus of this work. The estimate of the alteration of daylight adjusted darkness RTC rate, from relighting, by a given increase in the number of white lamps, can be obtained directly by fitting a binomial logistic model. This gives the daylight adjusted darkness collision rate ratio (CRR), that is the ratio of the daylight adjusted darkness RTC rates with a given increase in the number of new lamps to that without the increase. Using this daylight adjusted measure is to compensate for changes in both daylight and darkness RTC-rates due to changes in other features of the areas involved, such as how busy they are. (The focus on the daylight adjusted darkness RTC rate in this research is a variation from those outcomes given in the protocol. However, the originally specified protocol outcomes are also reported.)

Weekly numbers of RTCs in each MSOA were analysed as a multilevel model; time points at level 1 and MSOA at level 2, for the natural logarithm of the response variable of interest, e.g., the daylight adjusted darkness collision rate. The model used a polynomial for the underlying time trend, that is the trend for when there is no lighting change. We included a measure of the amount of new white lighting introduced in the model, which in the principal analysis was simply the number of new lamps operating, each week in an MSOA. Indicator variables were included to reduce background effects on the RTC-rate from seasonality (months) and weeks containing a public holiday.

The progress of the relighting was denoted as the difference in the number of white lamps within each MSOA from the MSOA’s white lamp mean over the series. The models also included a second lighting term for the difference of a MSOA’s mean number of white lamps from the (grand) mean number across all MSOAs giving the between area mean lighting difference. The two terms (‘within’ and ‘between’) for the build-up of white lighting were thus ‘centred’. The aim of the modelling was to separate the underlying temporal change in RTC-rate from the change associated with the relighting of the roads with white lamps.

The final form of the models used had the usual link functions given below for the response variable at time point i and MSOA j:

For the daylight adjusted darkness RTC-rate we use a Binomial model at the heart of which is the Binomial distribution, Binomial(n, π), (Forbes et al., 2011).

In our case, the n-parameter of the Binomial is the total number of collisions for the week, that is the sum of RTCs in darkness and daylight. (It can be shown that the ratio of two independent Poisson distributions, here those for darkness and daylight, is Binomial, conditional on their sum.)

The link function is logit:

$$\mathrm{logit}({\uppi }_{\mathrm{ij}})=\mathrm{log}({\uppi }_{\mathrm{ij}} /(1-{\uppi }_{\mathrm{ij}}))$$

where i = level 1 index and j = level 2 index and πij = the proportion of RTCs occurring in darkness = μdark ij / (μdark ij + μdaylight ij).

so 1- πij = the proportion of RTCs occurring in daylight = μdaylight ij / (μdark ij + μdaylight ij).

This leads to:

logit(πij) = log(μdark ij / μdaylight ij) that is the logarithm of the darkness RTC-rate divided by the daylight.

RTC-rate. This ratio we call the daylight adjusted darkness collision rate.

and.

For the Poisson models for the mean weekly RTC rates, 24 h, darkness and daylight, the link is log, = log(μij) = log(mean weekly RTC-rate).

The linear combination of predictors in all cases was, as specified in the protocol, of the form:

$$={\beta }_{0}+{\upbeta }_{1}t+{\upbeta }_{2}{\mathrm{t}}^{2}+\dots +{\upbeta }_{\mathrm{Mk}}{\mathrm{Month}}_{\mathrm{k}}+{\upbeta }_{\mathrm{HI}}{\mathrm{PubHol}}_{\mathrm{l}}+{\upbeta }_{\mathrm{W}}({\mathrm{L}}_{\mathrm{ij}}-{\langle {\mathrm{L}}_{\mathrm{ij}}\rangle }_{\mathrm{j}})+{\upbeta }_{\mathrm{B}}({\langle {\mathrm{L}}_{\mathrm{ij}}\rangle }_{\mathrm{j}}-\langle \langle {\mathrm{L}}_{\mathrm{ij}}\rangle \rangle )$$

where <  > j denotes the mean with respect to week i in area j, <  <  >  > the mean of the area means. t = the time that the midweek is from the start of the series.

The β0 + β1t + β2t2 + … polynomial, with a degree to be determined, represents the underlying secular time trend. The β0 term, the intercept coefficient, was modelled as a random term because different areas will be differentially busy. Other polynomial coefficients, e.g., β1, might also be expected to be random because of different underlying temporal trends between different MSOAs. The βW term represents the effect of the deviation of the number of white lamps, Lij, from its mean < Lij > j, over the time series duration, in an area, giving the within-area effect of lighting change. This coefficient enables the effect of changing lighting on road safety within an area to be measured. The βB term is the between-area effect term; the effect of the deviation of the mean number of white lamps in an area, < Lij > j, over the series, from the mean of the MSOA means <  < Lij >  > .

The βMk term represents the effect of the k = 1 to 11 Month indicator variables (reference = January) to account for seasonality in RTC rates.

βHl that of the 7 public holiday weeks per year l = 1 to 7 (reference = weeks which are not public holiday weeks) to account for the fact that traffic in a public holiday week may be different.

The models for the separate darkness only RTCs and daylight only RTCs incorporated an offset in each; the logarithm of time-exposure. That is, the logarithm of the fraction of the 24 h period when darkness or daylight applied, because, for example in winter, there is more darkness and so more opportunity for an RTC to occur in darkness. The binomial (logit) model for the darkness to daylight collision ratio incorporated the two offsets.

The predictor variables, i.e., time and number of lights, used in the model fitting were scaled, in order to ensure that all coefficient values were of a convenient size (neither too big nor too small) in the output produced. The time variable, the time that the midweek is from the start of the series, was scaled to use the unit of ten years so the coefficient gives the effect on the RTC-measure over a decade. The number of new replacement white lamps was scaled to be in units of one hundred, so the coefficient gives the effect on the RTC-measure by relighting by 100 lamps.

Estimation was done using Maximum Likelihood Estimation (MLE). This was used rather than Markov Chain Monte Carlo (MCMC) (Browne, 2009) also available in the modelling package MLwiN 3.01 (Rasbash et al., 2009). MCMC can be used to improve estimates and also to obtain the information criterion (IC) in order to help select an appropriate model. However previous experience revealed that the MCMC approach took many hours to run and yet the results were almost identical with those from MLE, presumably because of the large data size.

The selection of the most appropriate model was judged by the statistical significance of the coefficients of additional temporal terms in the polynomial for the secular time trend. Whether a coefficient was made random, or just left as fixed, was judged by the statistical significance of the variance estimate when a coefficient was made random. The logistic structure was anticipated to be satisfactory, however extra binomial variation was also investigated. The Poisson distribution for the examination of 24 h, darkness and daylight was anticipated to be satisfactory. However, over-dispersion was investigated, as was using the alternative, Negative Binomial distribution. The previous work, on relighting in Birmingham, (Marchant et al., 2020) indicated that overdispersion was not an issue for weekly MSOA RTC rates in that city.

The change over time in the prediction of the lighting effect from the fitted models, of the mean rates, between two time points i = a (after) from i = b (before) is found by differencing, ‘after’ minus ‘before’. The contribution to the change in the prediction on the log scale due to the change in the number of new lamps installed within an area j between those time points, b and a, is therefore given by:

$${\upbeta }_{\mathrm{W}}\left({\mathrm{L}}_{\mathrm{aj}}-{<{\mathrm{L}}_{\mathrm{ij}}>}_{\mathrm{j}}\right)-{\upbeta }_{\mathrm{W}}({\mathrm{L}}_{\mathrm{bj}}-{<{\mathrm{L}}_{\mathrm{ij}}>}_{\mathrm{j}})={\upbeta }_{\mathrm{W}}({\mathrm{L}}_{\mathrm{aj}}-{\mathrm{L}}_{\mathrm{bj}}).$$

The difference between logarithms is equal to the logarithm of the ratio of the individual arguments (i.e., either the daylight adjusted darkness RTC rate (μdark ij / μdaylight ij), a binomial model or the mean RTC rate (μij), a Poisson model).

So βW(Laj – Lbj) gives the relighting effect on log(RTC-ratea / RTC-rateb).

We want to have the estimate of the change in the daylight adjusted darkness RTC rate (μdark ij / μdaylight ij) and also the changes in the mean RTC rates (μij) for a given lighting change, in their un-logged state. Therefore, we exponentiate, which causes the right hand side of the linear model equation to go from a sum of terms to a product of exponentiated terms. This enables determining the result we require, that is the estimate of the collision rate ratio (CRR), the factor by which the specific quantity of interest, e.g., the daylight adjusted darkness collision rate (μdark ij / μdaylight ij) (or the mean RTC rate) is multiplied on increasing an area’s lighting by a certain number of lamps, that is from Lbj to Laj from time point i = b to i = a. The factor is exp(βW(Laj – Lbj)). Therefore, in the case of an additional 100 new lamps being installed, this is simply exp(βW), as units of one hundred lamps are used in the computations.

As stated above, the main focus is on the daylight adjusted darkness RTC rate and is done to avoid confounding with other factors contemporaneous with lamp installation. The result sought is obtained directly using the binomial logit model. An additional estimate of the lighting effect on the daylight adjusted darkness RTC-rate can also be obtained by differencing the separate fitted darkness and daylight Poisson models, because again the difference in the logarithms equals the logarithm of the ratio of their arguments, the numerator being the darkness effect and the denominator being the daylight effect. The result from the direct binomial model and that through differencing can be compared and the degree of agreement observed.

Note: the analysis at the smaller spatial scale mentioned in the protocol could not be done because of the pressure of time.

Results

The analysis dataset

The data in the analysis consisted of the number of RTCs occurring each week in darkness and daylight, in each of the 107 MSOAs, together with the number of replacement white lamps operating for that week and MSOA. The time series ran from the week commencing Mon. 03 Jan 2005 until Sun. 29 Sept. 2013. There were no missing data. The MSOAs form level 2 and the 456 weeks form level 1 of the multilevel analysis.

Descriptive statistics for the lamp data

The time series of the count of newly installed lamps across the whole of Leeds, exhibits a steady increase over the period of study (Fig. 1). This broad pattern of the rise in the number of new lamps is also evident at the level of individual MSOAs (Fig. 2). The increase in the numbers of new white lamps within the MSOAs over the analysis time period up to 29 Sept 2013 had the following statistics for the 107 areas: minimum = 275, maximum = 1,291, mean = 730.74, standard deviation = 173.505.

Fig. 1
figure 1figure 1figure 1

Leeds-wide Road Traffic Collisions and the build-up of lamps over the period up to Part Night Lighting, Leeds UK. Note: An Epanechnikov (40%) smoothed line is shown for the RTC component of the graph

Full size image
Fig. 2
figure 2

Cumulative numbers of new lamps operating in each week by Middle Layer Super Output Area, Leeds UK

Full size image

Descriptive statistics for the RTC-rates

The mean weekly RTC-rates, over the whole period, were calculated for each MSOA (i.e., 107 mean rates). A five number summary for these mean RTC-rates is given in Table 2.

Table 2 Statistics for mean weekly RTC-rate at the MSOA level, for the whole analysis time period
Full size table

Time series graphs for RTCs and new lamps for the whole of Leeds

The graphs (Fig. 1) of the weekly RTC time series aggregated to cover all of Leeds exhibit a general decline for both 24 h and daylight over the period of study, whereas RTCs in darkness stay roughly constant with a rise in the last couple of years. Additionally, graphs of the proportion of the RTCs occurring in darkness to those occurring at any time of day or night (24 h) and the darkness to daylight RTC ratio (that is the daylight adjusted darkness collision rate) were constructed.

As mentioned above in the ‘Collision data’ section, although constructed from the official STATS19 Personal Injury accident data file, the time series graphs for the aggregate of Leeds RTCs, (Fig. 1) show seemingly the anomalous behaviour after 2011. The fact that there is something different about the data in 2012 and possibly 2013 is clear in Table 3 which reports yearly RTCs from STATS19 for the whole of Leeds.

Table 3 Yearly RTCs from STATS19 for the whole of Leeds
Full size table

The decision was taken to proceed with the series up to the start of Part Night Lighting (PNL) because of the official nature of the data, despite the anomalous pattern. However, subsequent analyses using the truncated series of data, only up until the end 2011, were performed (see later) to compare with the results obtained using the longer series of data. Fortunately, most of the changes to lamps had occurred before the anomalous RTC pattern occurred, so although the truncated series involves a 20% reduction in length of the time series, there is only a 2.6% reduction in the number of lamp changes.

Note: Confidence limits of 95% are used throughout this work.

Modelling the daylight adjusted darkness collision rate on the number of new lamps operating in an area (using the series to 29 Sept 2013)

The best logit model for the daylight adjusted darkness collision rate had a random intercept, fixed linear temporal linear term with a positive coefficient indicating an increasing underlying darkness to daylight crash ratio through the time period. The appropriate darkness and daylight duration offsets were incorporated in the model. The estimate relating to the coefficient of the key within-area effect of the number of white lamps, along with its standard error (SE) are in Table 4.

Table 4 The within-area effect (coefficient) of relighting for the binomial logistic model analysis of the Darkness to Daylight RTC ratio, using the number of white lamps installed in units of one hundred
Full size table

The estimate and confidence limits are for βW, the effect on the natural log of the daylight adjusted darkness collision rate of relighting by 100 white lamps. Clearly, no statistically significant within-area lighting effect could be detected, as the standard error is of similar magnitude to that of the point estimate.

Exponentiating the figures from the within area table, to obtain the daylight adjusted darkness collision rate ratio rather than its log, gives a point estimate of 0.990 and a 95% CI of (0.971, 1.010) for 100 replaced lamps. The confidence interval giving the daylight adjusted darkness collision rate spans a reduction of 3% to an increase of 1% around zero change. The confidence interval for other than 100 lamps will, of course, also include zero change. To obtain the interval for an arbitrary number of lamp changes one needs to multiply the confidence limits in the above table by the number of new white lamps expressed in units of 100 lamps and then exponentiate. Note if the magnitude of the exponent is considerably less than one the exponentiation of the within area lighting term yields approximately 1 + βW(Laj – Lbj), given the number of new lights installed going from Lbj to Laj, expressed in units of 100 lamps. For a typical area relit with 700 lamps the point estimate of the collision rate ratio is 0.934 within a 95% CI of (0.816, 1.070), that is an 18% reduction to a 7% increase in RTCs.

The between area estimate of the lighting coefficient, βB, obtained from the model was also indistinguishable from zero with the values, estimate (standard error) of -0.02231 (0.01701). Therefore, there is no good evidence that areas receiving different mean numbers of white lamps, in the time period, differ in their effect on the daylight adjusted darkness collision rate.

A caterpillar plot of the level-2 (MSOA) residuals by rank, of the random intercept term (cons), of the logit model. together with the confidence intervals, is shown in Fig. 3. The overlap of the confidence intervals indicates that the MSOA mean daylight adjusted collision rates do not differ much between the areas.

Fig. 3
figure 3

The caterpillar plot of the level-2 (MSOA) residuals by rank, of the random intercept term (cons) from the binomial model, together with the confidence intervals

Full size image

Analysis using an overdispersed logistic model showed that any overdispersion was very small and not statistically significant. The coefficient estimates were nearly identical to the model without overdispersion, indicating overdispersion is not an issue.

Modelling the daylight adjusted darkness collision rate on the fraction of the full implementation completed in an area

Another way of estimating the impact of the lighting programme on road safety is to use the fraction of the full implementation in place in a given MSOA in a given week as the regressor instead of the number of lamps. Although constituting a variation to the protocol, this seems to be an appropriate way of tackling the problem. This is because the design of the programme was asserted to enhance road safety to all areas once completed. The implementation of the programme had effectively been completed by the time Part Night Lighting (PNL) was begun. A new variable for the fraction of the implementation completed was created by dividing the number of new lamps operating in an area in a given week, by the number operating in the week before PNL started. Again, the within MSOA lighting term was centred on each MSOA’s mean fraction and the between MSOA term was centred on the mean of the MSOA means. The estimates in Table 5 give the values for the impact of completing the relighting programme.

Table 5 The within-area effect (coefficient) of relighting for the binomial logistic model analysis of the Darkness to Daylight RTC ratio, using the fraction of the full implementation in place
Full size table

The estimate of the between MSOA coefficient, βB,was found to be -0.19342 with a standard error of 0.18998. Both estimates are indistinguishable from zero at our level of confidence. Exponentiating the values in the table of the within area estimates gives a point estimate for the daylight adjusted darkness collision rate ratio (CRR) due to full implementation as, 0.920 in a 95% CI of (0.776, 1.09), which means a 22% reduction to a 9% increase in RTCs.

The analyses of the Poisson 24 h, darkness and daylight models

The best models for 24 h and daylight log RTC-rate had the intercept as random, a random linear temporal coefficient, with no higher order terms needed. The point estimates of both the 24 h and daylight linear underlying temporal terms were negative, indicating that the underlying RTC rate was decreasing, as anticipated. For the darkness model the intercept was random, and the linear temporal was a fixed effect, with latter being positive, indicating that the underlying weekly darkness RTC-rate tended to increase during the period of study. Point estimates for the lighting coefficients were negative for all three outcome measures, 24 h, darkness, and daylight. The estimates relating to the key within-area effects of the number of white lamps are given in Table 6, with their standard errors (SE) and 95% confidence limits.

Table 6 The within-area effects (coefficients) of relighting for the Poisson model analyses of 24 h, Darkness and Daylight RTC rates when modelled using the number of white lamps installed in units of one hundred
Full size table

The final row of the table of results is from estimating the log daylight adjusted darkness collision rate by subtracting the daylight coefficient from that of darkness. The appropriate standard error is obtained assuming statistical independence and therefore calculating the result by root sum square of the separate standard errors. The lamp effect for log (μdark / μdaylight) = -0.01947 – -0.01265 = -0.00682. The associated standard error is given by (0.008092 + 0.00530 2)0.5 = 0.00967. These values of estimate and standard error are similar to the logistic binomial ones. The magnitude of the estimate is smaller than its standard error, therefore, like the direct binomial approach, it shows that no detectable effect was found for relighting with white lamps on the daylight adjusted darkness to RTC-rate.

Repeating the analyses using the data truncated at the end of 2011

The data were also analysed only using information up until the end of 2011 because of the peculiarities of the RTC rates observed in the later period. The number of weeks in the truncated series, running until the end of 2011, was 365. Exponentiation of the result of the binomial model gives a point estimate of 0.993 within a 95% CI (0.971, 1.015) for the daylight adjusted darkness collision rate ratio (CRR) for 100 replaced lamps. The underlying linear time trend slope was positive and statistically significant but its magnitude was about half of that from modelling the longer time-series. (Poisson models were also fitted to the truncated data series.)

Using the truncated data, all of the results in Tables 7 and 8 are similar to those for the longer series extending to 29 Sept 2013. The point estimate results are very similar because the vast majority of the installation had been completed by the end of 2011. The standard errors are a little wider because of the reduction of sample size in the shorter series. When scaled up to 700 lamps the binomial model gives a point estimate for the CRR of 0.950 and CI (0.816, 1.107), that is a reduction of 18% to an increase of 11% for the daylight adjusted darkness collision rate.

Table 7 The results for the within area lighting effect from analysis for the Darkness to Daylight collision ratio using the Binomial logistic model using the truncated series
Full size table
Table 8 For the Poisson models using the truncated series
Full size table

Further modelling

Results from further modelling are given in the Supplementary Materials provided in the open access repository https://osf.io/kfg64/https://doi.org/10.17605/OSF.IO/KFG64. These results are consistent with those above.

Further details of the three fitted binomial models

Because the focus of the study is on the daylight adjusted darkness RTC rate, the binomial modelling is key. Therefore, all the estimates and their standard errors of the model parameters obtained from the three binomial analyses are given side by side in Table 9. This allows easy comparison and shows consistency in the lighting effect. (It is important to remember that the Model Table 5 estimates are for the full relighting implantation, rather than just for 100 lamps). There are however differences in the temporal terms. The estimates given under Model Table 7 are delivered by the model which excludes the dubious data beyond 2011 whereas the other two sets displayed, consistent between themselves, do not exclude it.

Table 9 All the estimates and standard errors for the parameters of the logistic binomial models for the daylight adjusted darkness RTC rate. Table 4 per 100 lamps in the MSOA, Table 5 for a full implementation & Table 7 per 100 lamps in the MSOA, using series truncated to end 2011
Full size table

Discussion

As shown above the effect of lighting change on the daylight adjusted darkness collision rate for relighting by 100 white lamps is given by the CRR which has a point estimate of 0.990 within a 95% CI of (0.971, 1.010). A typical Leeds MSOA with 700 lamp changes yields a CRR point estimate of 0.934 within a 95% CI (0.816, 1.070), a reduction of 18% to an increase of 7%, for the change from predominantly orange lamps to white lamps. The model using the fraction of the relighting programme completed yields, a result of the CI residing between a 22% reduction to a 9% increase for an area being fully relit.

The result based on the more cautious truncated series of Leeds data gives a CRR point estimate of 0.993 within a 95% CI (0.971, 1.015) which when scaled up to 700 lamps gives a point estimate of 0.920 in a CI (0.816, 1.107), a reduction of 18% to an increase of 11%. The daylight adjusted darkness collision rate ratio, whether using the series to 29 Sept 2013 or that to the end of 2011, either from the logistic model or from subtraction of the daylight model from that for darkness, shows no detectable effect resulting from relighting. In rough terms one might say therefore that the plausible range of impact resides between a 20% reduction to a 10% increase.

The daylight adjusted darkness collision rate approach is preferred in assessing the impact of the lighting, here and in other research, because of the problem of confounding for count (Poisson) models for the weekly RTC rates. The problem is, if there are other changes occurring, which affect both the darkness and daylight RTC rates, at the same time as lamps are changed, this will confound the individual darkness and daylight effects of lighting. One example of confounding might be when the installation of new lamps is associated with increased activity day and night, and therefore traffic flows, thereby increasing the number exposed to risk, and thereby leading to an increase in the RTC rate, both in darkness and in daylight. Another example could be when lighting is changed at the same time as other alterations to roads which lead to safety improvements, therefore reducing the RTC rates in both darkness and daylight. However, if such changes influence the darkness and daylight RTC rates in a similar fashion the daylight adjusted darkness collision rate will not be so affected by the confounding and so makes this a better, more valuable outcome measure. The daylight adjusted darkness collision rate ratio (CRR) is what has been directly extracted by using the binomial logistic models. (The lighting effect derived from differencing the daylight model from that for darkness is consistent with result of the binomial model.)

Note that any biasing effects of Regression Towards the Mean (RTM) (Bland & Altman, 1994; Galton, 1886; Marchant, 2008) are likely to be negligible in affecting the result. This is because the data analysed comprised almost all of the roads of the city rather than those roads which have come to attention through a poor safety record.

Interestingly, as an aside, although the impact of relighting on the daylight adjusted darkness collision rate is indistinguishable from zero, the underlying trend in the absence of lighting change is for the rate to increase, even when modelling the shorter series. This could perhaps be because of an increase in night time road use and/or through an increase of risky behaviour during night time, e.g., alcohol and substance use. From the graphs given, one can see that, while daylight RTCs across the whole city decrease markedly over time, those occurring in darkness stay roughly constant and therefore the overall darkness to daylight RTC rate, that is the daylight adjusted darkness RTC rate increases with time.

Comparison and combination of the Leeds results with those from Birmingham

As stated above, the present study followed a similar one (Marchant et al., 2020) carried out on the UK city of Birmingham. The Birmingham data 2005–2013 comprises fewer lamp changes (36,123) but more RTCs, (20,282 in daylight and 8,085 in darkness) than for Leeds. The time series for both studies started at the same time point. The Birmingham study series ended on 29 Dec. 2013 whereas the Leeds one finished just three months earlier, thereby meaning that the Leeds series is only 3% shorter. The variables and the modelling are the same in both studies. The Birmingham result from the binomial model for the log mean daylight adjusted darkness collision rate ratio was 0.0162 with a standard error of 0.0287, again for relighting by 100 white lamps (compared with Leeds vales of -0.00974 and 0.00987). Using differencing of the daylight value from that of darkness for Birmingham gives 0.0154 with a standard error of 0.0310. So, it can clearly be seen, as in Leeds, that no effect of relighting on the daylight adjusted darkness collision rate was detected. The underlying time trends in the log models for darkness and daylight were more complicated for Birmingham than for Leeds, requiring going beyond the first order in the polynomial for time. Interestingly, in the Birmingham case these underlying trends for darkness and daylight were very similar to each other, indicating that the time trend in the logistic model for the underlying darkness to daylight collision ratio should be flat which it was, unlike the case for Leeds, where it rose.

Clearly the results for the effect of relighting, over a similar time period, on the daylight adjusted darkness collision rate, derived in the same way, for the same amount of relighting in an MSOA, for the two cities are comparable and consistent. Therefore, these can be combined in a two sample fixed effect, inverse variance meta-analysis. We use the results from the binomial models. Doing so with the estimate from the data up until 29 Sept 2013 for Leeds yields a pooled estimate of -0.00700, with a standard error of 0.00934, giving a 95% CI of (-0.02529, 0.01130) for the logarithm of the daylight to darkness RTC ratio (i.e., log CRR) for relighting by 100 lamps. Exponentiating gives the factor by which the daylight adjusted darkness RTC rate is multiplied (the daylight adjusted darkness collision rate ratio (CRR) as a result of relighting by 100 white lamps in a MSOA as 0.993 within a CI ( 0.975, 1.011). For a typical Leeds MSOA with 700 lamps this translates to a CRR point estimate of 0.952 within a CI (0.838, 1.082), suggestion a 16% reduction to an 8% increase in RTCs.

The equivalent combined figures, using the more cautious truncated data series for Leeds, are similar. The pooled estimate is -0.00421 with a standard error of 0.01036 giving a 95% CI of (-0.02452, 0.01609). Exponentiating gives the CRR, the factor by which the daylight adjusted darkness collision rate is multiplied for relighting, by 100 white lamps as 0.996 within a 95% CI (0.976, 1.016). For a typical Leeds MSOA with 700 lamps this translates to a point estimate of 0.971 in a CI (0.842, 1.119), that is a 16% reduction to a 12% increase in RTCs.

Comparison of the Leeds result with those from the NIHR LANTERNS study

The Leeds results given above of being unable to detect any change in road safety from relighting with white light, echo the null result from the extensive LANTERNS study on changing to white light (Perkins et al, 2015; Steinbach et al, 2015). The LANTERNS white light conversion was on 15,833 km of roads compared with around 2,800 km in Leeds, see Leeds City Council (2018). The LANTERNS overall result for effect of relighting, was a point estimate for the CRR of 1.01 within a 95% CI (0.93, 1.09). That is the factor by which the daylight adjusted darkness collision rate changed for a road segment being relit with white light. Their result for just the Yorkshire and Humber Region, of which Leeds is the largest conurbation, was 0.96 in a 95%CI (0.81, 1.15), a19% reduction to a 15% increase. This is similar to our Leeds range given above of roughly a 20% reduction to a 10% increase.

As discussed in the report of the Birmingham study (Marchant et al., 2020) there are similarities between the multilevel approach used for Leeds and the conditional Poisson method of LANTERNS, both operate on the log-scale of response. A direct comparison of results is not possible, however, as the lighting data from the NIHR LANTERNS study is not available.

Conclusion

The conclusion of the study is that the evidential basis, for the claim that changing street lighting would improve road safety, was weak and that changes to road safety, positive or negative, could not be detected following relighting. This null result was despite the large dataset which essentially comprises the relighting of the whole city of Leeds. In rough terms one might say that the study gives a plausible range of impact, on the daylight adjusted darkness collision rate, which typically resides somewhere between a 20% reduction to a 10% increase. The null result is consistent with those from other recent studies. It remains an open question whether relighting with white light gives any improvement (or detriment) to road safety.