Evaluation of an operational air quality model using large-eddy simulation

The large-eddy simulation (LES) model uDALES is used to evaluate the predictive skill of the operational air quality model SIRANE. The use of LES in this study presents a novel approach to air quality model evaluation, avoiding sources of uncertainty and providing numerical control that permits systematic analysis of targeted parametrisations and assumptions. A case study is conducted over South Kensington, London with the morphology, emissions, meteorological conditions and boundary conditions carefully matched in both models. The dispersion of both inert (NOx) and reactive (NO, NO2 and O3) pollutants under neutral, steady-state conditions is simulated for a south-westerly and westerly wind direction. A quantitative comparison between the two models is performed using statistical indices (the fractional bias, FB, the normalised mean squared error, NMSE, and the fraction in a factor of 2, FAC2). SIRANE is shown to successfully capture the dominant trends with respect to canyon-averaged concentrations of inert NOx (FB = −0.08, NMSE = 0.08 and FAC2 = 1.0). The prediction of along-canyon velocities is shown to exhibit sources of systematic error dependant on the angle of incidence of the mean wind (FB = −0.18). The assumption of photostationarity within SIRANE (deviations from equilibrium of up to 170% exist close to busy roads) is also identified as a significant source of systematic bias resulting in overand underpredictions of NO2 (FB = −0.18) and O3 (FB = 0.14) respectively. The validity of the assumed uniform in-canyon concentration is assessed by analysing the pedestrian, leeward and windward concentrations resolved in uDALES. The use of canyon-averaged concentrations to predict pedestrian level exposure is shown to result in significant underestimations. Linear regression is used to effectively capture the relationship between pedestrianand canyonaveraged concentrations in uDALES. Correction factors are derived (m 1.62 and = R 0.92 2 for inert NOx) that are capable of significantly improving SIRANE's ability to assess pedestrian level exposure for the conditions investigated in this study. The prevalence of intersections and advective nature of the shear layer are highlighted as important differences between modelling real heterogeneous urban morphology and idealised infinite can-

The large-eddy simulation (LES) model uDALES is used to evaluate the predictive skill of the operational air quality model SIRANE. The use of LES in this study presents a novel approach to air quality model evaluation, avoiding sources of uncertainty and providing numerical control that permits systematic analysis of targeted parametrisations and assumptions.
A case study is conducted over South Kensington, London with the morphology, emissions, meteorological conditions and boundary conditions carefully matched in both models. The dispersion of both inert (NO x ) and reactive (NO,NO 2 and O 3 ) pollutants under neutral, steady-state conditions is simulated for a south-westerly and westerly wind direction. A quantitative comparison between the two models is performed using statistical indices (the fractional bias, FB, the normalised mean squared error, NMSE, and the fraction in a factor of 2, FAC2).
SIRANE is shown to successfully capture the dominant trends with respect to canyon-averaged concentrations of inert NO x (FB = −0.08, NMSE = 0.08 and FAC2 = 1.0). The prediction of along-canyon velocities is shown to exhibit sources of systematic error dependant on the angle of incidence of the mean wind (FB = −0.18). The assumption of photostationarity within SIRANE (deviations from equilibrium of up to 170% exist close to busy roads) is also identified as a significant source of systematic bias resulting in over-and underpredictions of NO 2 (FB = −0.18) and O 3 (FB = 0.14) respectively. The validity of the assumed uniform in-canyon concentration is assessed by analysing the pedestrian, leeward and windward concentrations resolved in uDALES. The use of canyon-averaged concentrations to predict pedestrian level exposure is shown to result in significant underestimations. Linear regression is used to effectively capture the relationship between pedestrian-and canyonaveraged concentrations in uDALES. Correction factors are derived (m 1.62 and = R 0.92 2 for inert NO x ) that are capable of significantly improving SIRANE's ability to assess pedestrian level exposure for the conditions investigated in this study. The prevalence of intersections and advective nature of the shear layer are highlighted as important differences between modelling real heterogeneous urban morphology and idealised infinite canyons.
NO 2 is related to cardiovascular diseases, decreased lung functionality, aggravated asthma, low birth weights and premature deaths (Andersen et al., 2012;Vineis et al., 2006). NO 2 and ozone (O 3 ) are precursors of acid rain, particulate matter and are partially responsible for the eutrophication and acidification of soil and water. Despite stringent regulations on both emissions and air quality, NO 2 concentrations in many European cities exceed regulations (EEA, 2017). The problems associated with urban air quality are even more severe in low-to middleincome countries.
There are numerous sources of NO x into the atmosphere (Wallace and Hobbs, 2006), but harmful concentrations in cities are principally a result of fossil fuel combustion (Palmgren et al., 1996). The chemistry associated with tropospheric NO x is complex, however considering that the timescale at which pollutant dispersion occurs in the urban canopy is typically a few minutes (Soulhac et al., 2011), it is often simplified to the reactions of the null cycle (which have no net chemical effect; Crutzen, 1979). NO x emissions primarily consist of NO that, during the day, is oxidized by O 3 to NO 2 . NO then rapidly establishes an equilibrium with NO 2 as a result of photodissociation (Wallace and Hobbs, 2006).
Urban pollution dispersion models provide information on the temporal and spatial evolution of air pollutants in and above the urban canopy. They are thus an integral tool for researchers, urban planners and public authorities to understand, estimate and subsequently combat the challenges associated with outdoor air quality. A range of dispersion models exist that consider this problem at different levels of abstraction, with an inherent trade off between the provided resolution and accuracy against the costs and speed of computation.
The most computationally demanding aspect of this problem is resolving the unsteady, heterogeneous and turbulent flows that occur within and above the urban canopy. Operational models typically parametrise the dominant dispersive exchanges and flows of the urban environment to produce a bulk approximation of this complex flow field (Baklanov et al., 2009). This simplified flow representation (O (100 m)) is combined with detailed data (e.g. traffic flows, background concentrations, meteorological conditions and street morphologies) and physico-chemical schemes (e.g. chemistry and deposition) to provide estimates of pollution levels at a city or neighbourhood scale over days or longer time-periods with limited computational cost. Computational fluid dynamics (CFD) models present alternative tools through which the flow field and subsequent dispersion is resolved (Di Sabatino et al., 2007;Blocken, 2015). Operational CFD models typically parametrise the role of turbulence resulting in (quasi-)steady solutions (e.g. Reynolds-averaged Navier-Stokes (RANS) models). Large-eddy simulation (LES) models are capable of resolving the unsteady turbulent three-dimensional flow field to the highest possible resolutions (O(1 m, 0.1 s); Letzel et al., 2008;Tominaga and Stathopoulos, 2013;Salim et al., 2011). Typically LES models have been used in intensive, generalised studies to formulate the parametrisations that define operational models. Although LES is computationally expensive, increasing computational capabilities mean that it is now possible to produce realistic urban LES simulations at a comparable scale to operational models (neighbourhood scales over time periods of hours).
Operational dispersion models have been validated extensively against both experimental and field data (Soulhac et al., , 2017 and comparative studies between operational and RANS models exist (Di Sabatino et al., 2007), but to the authors' knowledge this study presents the first evaluation of an operational model using LES. The use of LES in this analysis has the advantage of simulating real-time urban pollution dispersion (as would be captured in field or experimental studies) but with the computational advantages of 1) high-resolution mean and turbulent statistics over the entire domain and study period, 2) the numerical control to conduct systematic, targeted simulations (e.g. with or without chemistry/buoyancy effects) and 3) capturing the same fundamental simulation conditions as the operational model and therefore vastly reducing the associated uncertainty (e.g. carefully matching the modelled morphology, emissions and meteorology but at a hugely enhanced resolution; Chang and Hanna, 2004). This methodology does not present a further validation of an operational model (due to the associated modelling and numerical uncertainty; Stern et al., 2001), but it does therefore provide the opportunity to systematically analyse the assumptions, simplifications and parametrisations used by operational models against an "ideal" dataset.
In this study a comparison is conducted between the operational air quality model SIRANE (Soulhac et al., 2011), and the LES code uDALES (Heus et al., 2010;Tomas et al., 2015Tomas et al., , 2016. The evaluation objective is to assess the predictive skill of SIRANE over its quasi-steady hourly timesteps and to systematically analyse the applicability of SIRANE's parametrisations and assumptions using the high-resolution capabilities of uDALES. Details of the two models are outlined in Section 2. The setup of the case study is described in Section 3, detailing how the morphology, emissions and meteorology of South Kensington, London were systematically matched for both SIRANE and uDALES. The results for inert and reactive simulations alongside the sensitivity to wind direction are discussed in Sections 4.1 and 4.2 respectively. In order to assess the validity of SIRANE's uniform in-canyon concentration assumption, the pedestrian, leeward and windward concentrations from uDALES are analysed. The role of intersections within the case study area and other impacts of resolving the heterogeneity of real urban morphology are discussed. Finally conclusions are drawn upon the strengths and shortcomings of SIRANE and comments are made on the developing role of LES in this area.

Models
SIRANE and uDALES are both physically-based numerical models that differ in the scale at which they resolve the flow. SIRANE operates at street scale, and therefore relies on flow and turbulence parametrisations to estimate the dispersion of pollutants through a defined street network, whereas uDALES operates at metre-scale, and thus resolves the turbulent and unsteady 3-D flow field around buildings and the subsequent dispersion of emissions within and above the urban canopy. A description of both models is given below. The implementation of the null cycle in both models is highlighted.

SIRANE
SIRANE is an operational urban dispersion model (Soulhac et al., 2011). The domain is divided into two sub-domains: the urban canopy and the external atmosphere. The urban canopy is represented by street canyons (spaces between the buildings) which are modelled as a series of connected boxes (a street network). The pollutant concentration in each box is assumed to be uniform and is computed as a mass balance which includes the advective transfer along the street, turbulent transfer at roof level, chemical transformations occurring inside the canyon and dispersion at intersections . The pollutant dispersion above the canopy is represented by a Gaussian model. The mesoscale meteorological properties (boundary layer height, friction velocity, surface heat flux etc.) are calculated through parametrisations based upon observational data such as wind speed, wind direction, cloud cover, precipitation and surface albedo (Soulhac et al., 2008(Soulhac et al., , 2013. The chemical equilibrium between NO, NO 2 and O 3 is assumed to follow the null cycle. SIRANE accounts for these reactions by first simulating the passive dispersion of NO x through and above the street network and then uses the resulting NO x concentration field and the background concentrations of NO, NO 2 and O 3 to solve the resulting system of equations by satisfying the Leighton relationship (Leighton, 2012): [NO ] , (1) T. Grylls, et al. Atmospheric Environment: X 3 (2019) 100041 where J NO2 is the photolysis rate for NO 2 and k 3 is the + NO O 3 reaction rate. This methodology therefore follows the assumption that the pollutants reach a photostationary steady-state (PSS) within each canyon over SIRANE's hourly time steps. SIRANE has been validated against wind tunnel experiments and real measurements in Lyon, France (Soulhac et al., , 2017Carpentieri et al., 2012;Salem et al., 2015).

uDALES
uDALES is a high-resolution, flow-resolving atmospheric LES model (Heus et al., 2010;Tomas et al., 2015Tomas et al., , 2016 that solves the equations of motion at the energetically dominant scales of turbulent motion. The three-dimensional flow field is modelled at resolutions of O(1m) and O (0.1s), satisfying the filtered Navier-Stokes equations throughout the domain. The Vreman subgrid model is used to parametrise the dynamics at the subgrid scale (SGS; Vreman, 2004). The scalar fields are modelled following the governing equation where . denotes the filtered part, q i is the SGS scalar flux vector, is an additional term that performs chemical transformations and S is a source term for volumetric emissions. The numerical scheme consists of a spatial discretisation on an Arakawa C-grid, utilising a second-order central-differencing scheme for all field variables except for the pollution concentration, which employs a κ scheme to ensure positivity. A third-order Runge-Kutta time integration scheme is applied. An immersed boundary method (IBM) has been incorporated into uDALES in order to provide the capability to place objects in the domain (Tomas et al., 2015). The SGS roughness of the buildings and dynamics close to the wall are modelled by log-law wall functions (Uno et al., 1995;Suter, 2018). uDALES has been validated against wind and water tunnel data Tomas, 2016) and the dynamic core, DALES, has been validated extensively and has been part of several atmospheric intercomparison studies (Heus et al., 2010).
The chemical processes of the null cycle were added into uDALES for the purposes of this study. The relatively high reactivity of O• permits a simplification of the null cycle leading to the following prognostic relationships (Zhong et al., 2015): The chemistry is implemented following a fully implicit time-integration scheme. A validation of the chemical scheme is provided in Appendix B.

Case study
The case study region is situated in South Kensington, a district that is located centrally within London, UK. The study area spans 672 × 1344 m and contains a range of morphological features (street lengths, building heights, open spaces etc.) and street types (pedestrianised, high streets, a-roads etc.). High resolution emissions data was available through the augmentation of traffic counts with the traffic microsimulation model VISSIM (PTV AG, 2017). SIRANE and uDALES operate at different scales and are based on fundamentally different numerical representations of the urban environment. These differences mean that consideration is required in terms of matching the simulation properties; e.g. the morphology, emissions, meteorological conditions (wind, chemistry) and boundary conditions. A detailed description of the methodology used in producing consistent simulation properties is given below. An outline of the case study properties of both models is provided in Appendix A.

Morphology
The representation of the urban morphology in both uDALES and SIRANE was obtained using one metre resolution, vector-based LIDAR data with an applied building mask (Environmental Agency, 2017). For use within uDALES, the mean building heights were first rasterised to the defined grid resolution. The resulting set of buildings were then processed to work optimally with the implemented IBM. Morphological details that resulted in small gaps along building walls were filled, T. Grylls, et al. Atmospheric Environment: X 3 (2019) 100041 isolated protruding cells were removed and enclosed features (e.g. courtyards) below a threshold area (<100 m 2 ) were filled. These alterations avoid small spatial features where the number of cells can be insufficient to effectively resolve the fluid flow. A representation of the resultant buildings that are resolved within uDALES is shown in Fig. 1a. Contrary to uDALES, SIRANE does not explicitly model the buildings within the domain. Streets are instead represented as boxes which are positioned within the urban canopy. By overlaying the street network of the case study area onto the buildings processed for use within uDALES, the network of SIRANE's boxes could be defined. First, street canyons were assumed to be straight, with their beginning and end points at the centre point of the connecting intersections. Second, the average heights and widths on either side of the box centre line were calculated from the height of and respective distance to the surrounding buildings. The construction of the network follows the guidelines outlined by Soulhac et al. (2011) and results in the street network shown in Fig. 1a.

Traffic and emissions
High quality traffic and emissions data is imperative to obtaining useful results from pollution dispersion models. In this study, high-resolution emissions data was obtained through the coupling of the traffic microsimulation and emissions models outlined below. The resulting second-by-second emissions data over the case study area was then processed for use within both SIRANE and uDALES. Only emissions due to road transport are considered in this study.
A representation of the traffic flow in South Kensington is first necessary to estimate the vehicular emissions. The traffic flow characteristics were obtained using a VISSIM traffic microsimulation. VISSIM (version 9.00-12 is used here) is a traffic, timestep and behaviour microsimulation model (Bloomberg and Dale, 2000), based on both traffic flow and light signal control models (PTV AG, 2017). The traffic flow model is derived from a psycho-physical approach which states that the behaviour of each driver can be classified into four different modes (free driving, approaching, following and braking), responding to the perceived relative speed and distance to the preceding vehicle (Gomes et al., 2004). The software also accounts for driving styles via a distribution of speed and distance behaviour (PTV AG, 2017). Fixed traffic lights were implemented throughout the network and manual traffic count records of the local rush hour were used to calibrate the model. The simulation was run for one hour in order to match SIRANE's timestep; instantaneous data of location, speed and acceleration was given for every vehicle in the network on a second-bysecond basis. An appropriate emissions model is essential to transform the instantaneous data into estimations of NO x for use within SIRANE and uDALES. Average-speed models (e.g. COPERT) are known to systematically underestimate NO x and overestimate CO 2 emissions (Achour et al., 2011), meaning that an instantaneous emissions model was required. These instantaneous models, which are usually based on various parameters (e.g. instantaneous speed and acceleration, road grade, etc.), were developed following the observation that exhaust emissions are directly related to instantaneous driving behaviour (smooth versus aggressive; Tzirakis et al., 2007)  It should be noted that, as the primary aim of this work is to compare two dispersion models and not to provide the most realistic representation of the real world, this assumption does not limit the validity of the evaluation.
The resultant second-by-second emissions data was processed for use within both uDALES and SIRANE. In uDALES the data was time averaged and rasterised to the defined grid size, resulting in a network of point sources which capture the spatial variability of emissions to a high resolution (shown in Fig. 1b). These emissions were then spatially averaged over the boxes defined in SIRANE to produce uniform emissions within each box. An NO 2 /NO x emissions ratio of 0.2 was assigned to the emissions in both models (Carslaw et al., 2016;O'Driscoll et al., 2016; UK National Atmospheric Emission Inventory, 2018).

Meteorological conditions
As an operational model, simulations within SIRANE are set up to utilise field data of the designated study area. For this reason, typical meteorological conditions of South Kensington, London were applied within SIRANE: a summer's day (July, 1 p.m.) with neutral stability (enforced by setting an albedo of 1), partial cloud cover, no precipitation, a wind speed of 2.5 m/s at 30 m and a south-westerly wind direction.
SIRANE utilises parametrisations to produce a boundary layer height ( = h 412.6 m) and friction velocity ( = u 0.33 * m/s) from the assigned conditions. These were matched in uDALES by setting the domain height equal to h and enforcing constant uniform pressure gradients, m/s 2 (for the southwesterly wind direction). Neutral conditions were produced within uDALES by neglecting buoyancy, moisture and thermal forcings.
The photolysis rate ( = J 0.0063 NO2 s −1 ) and reaction rate ( = k 10041.8 3 m 3 /mole s) were also automatically assigned through parametrisations within SIRANE, as a function of the solar elevation and cloud cover and air temperature respectively. Background concentrations were obtained from the hourly data available from the Greater London Authority (King's College London, 2017). The background concentrations were averaged over the specified date and time across a 10 year data set. The resulting concentrations were relatively close to satisfying the PSS condition with the defined reaction rates. In order to enforce photostationarity, the concentrations were equilibrated to the corresponding PSS. The final background concentrations were x; = 28.59 µg/m 3 (Baukal and Eleazer, 1998).

Boundary conditions
The differences between the two models also required consideration when defining the boundary conditions. SIRANE applies the background concentrations at the upwind sides of the domain, with the Gaussian model capturing the downwind dispersion of pollutants across the domain. It also assumes that the form of the boundary layer is spatially and temporally steady over each timestep.
The flow characteristics were reproduced in uDALES by applying periodic horizontal boundary conditions and a free slip condition at the top of the domain for velocity. The simulation was run up until a steady-state, fully developed boundary layer had developed matching the friction velocity and boundary layer height discussed in Section 3.3. The upwind scalar concentration profiles were set to background levels and outflow conditions were set for the scalars at the downstream boundaries. In order to avoid direct interaction between morphological features on the upwind and downwind edges of the domain (due to the periodic horizontal boundary conditions for velocity) and to ensure that the scalar boundary conditions do not have a direct impact on the concentration field in the canopy layer, a 12 m buffer region was defined at the domain's edge where neither emissions nor buildings were modelled.
The case study comparison is conducted for a single hourly timestep within SIRANE. In order to produce converged statistics from uDALES the simulation was run for 10800 s ( 8.5h u / * , where h is the domain height and u * is the friction velocity) with a sampling time of 5 s.
T. Grylls, et al. Atmospheric Environment: X 3 (2019) 100041 Criteria for the convergence of the obtained statistics was satisfied through analysis of the time and horizontally-averaged momentum flux profiles. The dispersive momentum fluxes reduced to negligible values and the turbulent momentum fluxes exhibited a linear profile above the roughness sublayer (Coceal et al., 2006;Vinuesa et al., 2016). The computational requirements for the simulations were 35 h on 192 cores for uDALES and 1 s on 16 cores for SIRANE.

Results
In order to isolate the dispersive performance of the models from their respective chemistry schemes, a comparison is first conducted between inert simulations where NO x emissions are dispersed passively with no chemical reactions. The comparison is then broadened by switching on the respective chemistry schemes of the two models. The sensitivity of the results to wind direction is discussed by comparing the findings of the south-westerly case study to an equivalent study with a defined westerly wind. Altering the wind angle by 45 ∘ produces significantly different angles of incidence over the canyons, with many more streets either perpendicular or parallel to the wind for the westerly case. The results for the westerly case are shown alongside those of the south-westerly case where they have been discussed in detail within the text.
The evaluation of SIRANE is conducted with two main points of focus. Firstly, the predictive skill of SIRANE in modelling canyonaveraged properties is analysed by temporally and spatially averaging uDALES's three-dimensional dataset to produce directly comparable results (denoted by DA-s). Secondly, the high-resolution mean and turbulent statistics of uDALES are used to directly analyse the strengths and shortcomings of SIRANE's parametrisations. For example, the incanyon variability is assessed to understand the impact of the uniform concentration assumption and the fluxes at the canyon tops within uDALES are discussed to assess the parametrisation of vertical exchange within SIRANE.
Box and scatter plots are used as tools to analyse both the bulk and street-by-street properties of the resulting datasets. Quantitative analysis of SIRANE's predictive skill is provided using the normalised mean square error (NMSE, a metric quantifying the overall deviation or scatter between the two sets of results), fractional bias (FB, a metric quantifying mean bias and therefore identifying potential sources of systematic error) and the fraction in a factor of two (FAC2, a more robust metric of model performance as it is not overly influenced by outliers; Chang and Hanna, 2004). These statistical indices are displayed in Table 1. The same metrics have been applied to previous experimental and field validations of SIRANE (Soulhac et al., , 2017 alongside the criteria for « good » model performance (|FB| 0.3, NMSE 4 and FAC2 0.5; Chang and Hanna, 2004;Hanna and Chang, 2012). These criteria are not directly applicable to this study because of the reduction in uncertainty associated with the use of LES as opposed to field or experimental measurements. The limits for « good » model performance must be redefined for application to evaluations of this form; however, as this study is the first to utilise LES for this purpose, there are no other benchmarks with which to refine these limits (Chang and Hanna, 2004). The statistical indices are therefore interpreted in the context of the original limits with the understanding that more stringent criteria are necessary. Following future similar evaluations, redefined limits can be developed to indicate the requirements for « good » model performance in this context.
There is a monitoring station situated on the junction of Cromwell Rd and Queen's Gate in the case study region (Air Quality England, 2019). The obtained concentrations of inert NO x and reactive NO and NO 2 in this study (both from SIRANE and uDALES) are within the observed ranges of the corresponding hourly means measured at this site. Whilst the aim of the study is not to reproduce measured concentrations in the field, this finding indicates that the modelled emissions, background concentrations and reaction rates are realistic. Fig. 2 shows three NO x concentration fields, providing a visual comparison between the nature of the data produced by SIRANE and uDALES. An instantaneous field of NO x at pedestrian level (1.8 m above ground) from uDALES (Fig. 2a) is presented illustrating the turbulent and spatially heterogeneous nature of pollution dispersion within the urban canopy. This concentration field contrasts with the coarser results of SIRANE (SI; Fig. 2b) that display the canyon-averaged concentrations over the street network and the external concentrations elsewhere for the hourly timestep. A qualitative comparison can be drawn between this field and the processed results of uDALES (DA-s; Fig. 2c). DA-s was calculated by temporally and spatially averaging the data of uDALES to form a 2-D field that is comparable to the results of SIRANE. In the positions of SIRANE's boxes, the canyon-averaged concentration is taken and elsewhere the external concentrations were taken at a height of 20 m (as defined within SIRANE; if the building height exceeded 20 m then the first cell above the building was used in DA-s). Qualitatively it can be seen that both the canyon-averaged and external concentrations of SIRANE and uDALES are in reasonably good agreement. In particular, the vertical exchange at the intersection and subsequent downwind dispersion looks to have been captured effectively.

Inert case
The predictive skill of SIRANE is assessed in Fig. 3a through the comparison of the canyon-averaged concentrations of both models. The box plot for canyon-averaged NO x indicates that the bulk statistics of SI and DA-s are matched well with a very similar median and upper quartile. This performance is confirmed by the statistical indices shown in Table 1 for both wind directions. The scatter plot presents a more detailed analysis in which the performance can be broken down into a range of street types. The parametrisations defined for the boxes in SIRANE are based upon studies of idealised infinite canyons. Three street types were identified that deviate from these simplified conditions: • Short streets, L W (square markers). Short streets are dominated by secondary flows at intersections and other heterogeneity that SIRANE does not account for.
• Streets adjacent to open spaces (unfilled markers). These streets will not exhibit the classical canyon vortex, channelling or shear layer of an infinite canyon. The shortcoming of dealing with open spaces has been identified within SIRANE's literature (Soulhac et al., 2011). • Upwind streets (triangular markers). The boundary conditions as defined in both SIRANE and uDALES do not model the region Table 1 Overview of the statistical indices used to quantify the performance of SIRANE's canyon-averaged predictions against the results of uDALES for a south-westerly (SW) and westerly (W) wind direction. . where X DA and X SI are the canyon-averaged results of uDALES and SIRANE respectively. The scatter plot in Fig. 3a indicates that the most erroneous concentrations can be attributed to these street types. This finding illustrates the challenge in representing highly heterogeneous real urban morphologies through idealised parametrisations. The predictive skill of SIRANE was shown to be robust against the range of angles of incidence and street aspect ratios across the case study region. On the whole, the findings for canyon-averaged inert NO x indicate that the operational model has successfully captured the dominant processes of pollution dispersion through its parametrisations, as can be seen by the low NMSE values (0.08 and 0.09 for south-westerly and westerly wind  The flow within the street network can be analysed by comparing the canyon-averaged velocity along the street axis, U, in both models. The along-canyon velocity is a key variable in defining the exchange that occurs at SIRANE's intersections. It is parametrised as a function of the entrainment into the canyon from the external flow and the drag induced by the building walls (Soulhac et al., 2008). Fig. 3b compares the values of U for both models, indicating that SIRANE overpredicts the along-canyon velocities for the south-westerly case. The corresponding fractional bias (FB = 0.18) indicates a source of systematic bias within SIRANE. The overestimation can also be attributed to the assumption in SIRANE that all streets are infinite canyons with a defined surface roughness. The parametrisation for U neglects the role of secondary flows at intersections (which are captured within uDALES) that act to reduce along-canyon velocities. The fractional bias for the westerly wind case (FB = 0.02) does not indicate the same tendency for systematic overestimation. However the corresponding fraction in a factor of 2 is significantly lower (FAC2 = 0.68). It can be seen from Fig. 3b that this discrepancy occurs due to the set of streets that are underestimated with low along-canyon velocities in SIRANE. The other streets in the westerly dataset illustrate the same systematic overestimation as in the south-westerly case. The selection of streets which exhibit an underprediction all have angles of incidence exceeding 77 ∘ . Analysis of the time averaged flow field within the urban canopy layer from uDALES illustrates that in these high-angle-of-incidence streets, the exchanges at the corresponding intersections can induce alongcanyon flow (both through imposed pressure gradients and the presence of morphological features such as changing road widths at these intersections). SIRANE therefore has a tendency to overestimate U in streets with angles of incidence lower than 77 ∘ and potentially underestimate those with angles that exceed this threshold value. Despite this, the results for canyon-averaged NO x indicate that the flow within the street network is captured sufficiently well to predict the canyonaveraged concentrations.
Coupling SIRANE with a detailed street canyon model has been discussed as future work within the literature as a method to account for the in-canyon variability of the concentration field and potentially increase the complexity of other parametrisations (e.g. additional scaling of the exchange velocity with the canyon aspect ratio; Soulhac et al., 2011). The high-resolution mean and turbulent statistics of uDALES were used to analyse 1) the importance of accounting for the spatial variation of pollutants within the canyons (pedestrian level concentrations and asymmetry) and 2) the nature of vertical exchange occurring at the canyon tops.
The NO x concentration fields of uDALES are spatially averaged over each box's horizontal extent at a height of 1.8 m and time averaged over the simulation period to provide pedestrian-averaged concentrations (DA-p). This data is analysed against DA-s in Fig. 4a to study the relationship between canyon-and pedestrian-averaged concentrations. The scatter plot displays a strong quasi-linear relationship indicating that the pedestrian-averaged concentrations are proportionally larger than the canyon-averaged values, a result of both the proximity to the emissions source and the mixing induced by the canyon vortex. The increased magnitude of concentrations at pedestrian level is also evident in the box plots in Fig. 4c where the data for DA-p is comparatively skewed towards higher concentrations. Whilst this relationship between pedestrian-and canyon-averaged concentrations may be selfevident, it highlights a fundamental shortcoming in that the use of SIRANE's canyon-averaged concentrations to assess pedestrian level exposure will lead to significant underestimations of inert NO x . The linearity of this relationship suggests that despite the range of e.g. emissions, angles of incidence, aspect ratios and street types, the spatial distribution of the pollutant concentrations in this study can be reasonably represented through the application of a simple correction factor. Applying a linear least-squares regression that intercepts the background concentration ( where m is a correction factor and b is the background concentration) permits the formation of a formula that therefore describes a simplified relationship between pedestrian-and canyon-averaged concentrations. Table 2 displays the resulting correction factors and corresponding R-squared values. The linear relationship as calculated for the south-westerly wind direction ( = m 1.62) is overlaid onto Fig. 4a. Considering the complexity and variability of the dispersion processes that are being captured through this linear regression, the corresponding R-squared value ( = R 0.92 2 ) is remarkably high and indicates a strong correlation. The westerly case exhibits a similar correction factor ( = m 1.57) with a lower R-squared value ( = R 0.75 2 ), the increased spread can be seen in Fig. 4a and is a result of the greater range of angles of incidence.
A corrected dataset (SI-c) was calculated by applying this relationship to SIRANE's canyon-averaged concentrations ) and is compared to the pedestrian-averaged concentrations of uDALES in Fig. 4b. The agreement between SI-c and DA-p illustrates that the application of the correction factor has achieved a similar level of performance in predicting pedestrian-averaged quantities as was provided by the canyon-averaged quantities in Fig. 3a. The applied correction factor is evidently a simplified solution. Substantial further work would be necessary to fully assess its sensitivity to what is a large parameter space (e.g. different atmospheric stabilities and wind speeds). However, it is clear that such a correction (or a more comprehensive coupling with a street canyon model) is necessary when considering using SIRANE to evaluate pedestrian level exposure and that in this case study a simple linear regression provides an effective tool to account for this correction. This finding also agrees with results from the field study of Xie et al. (2003) where similar vertical gradients were observed in the daily NO (and NO x ) and CO concentration profiles within a street canyon.
The results of Xie et al. (2003) also indicate the role of asymmetry within the street canyon, the effect where concentrations on the leeward side of the canyon are greater than the windward side due to the dispersion induced by the canyon vortex. Other operational models have utilised parametrisations to account for this difference in concentrations. The Operational Street Pollution Model (OSPM) splits the street canyon into lee and windward receptors, using the assumption of a helical structure within the canyon (accounting for the wind direction) to differentiate between the fluxes of pollutant received in opposing sides of the canyon (Berkowicz, 2000). The advection of the emissions towards the leeward side of the canyon is evident in the pedestrian level instantaneous concentration field in Fig. 2a. The NO x concentration fields of uDALES were processed to calculate pedestrianaveraged concentrations on the lee-(DA-pl) and windward (DA-pw) sides of each box in order to assess the prevalence of asymmetry over the case study area. These results are included in the box plot in Fig. 4c. The quartiles illustrate a bulk trend of higher leeward concentrations, although there are street segments where the asymmetry effect was not exhibited and the magnitude of the difference between DA-pl and DApw scales in comparison to the difference between DA-s and DA-p. The asymmetry effect is less pronounced because the exchanges at intersections complicate the idealised helical flow representation, producing recirculation zones and drawing in fluxes of cleaner or more polluted air into the canyon. The asymmetry effect is therefore most dominant downwind from intersections. No dominant scaling was found between the asymmetry ratio (DA-pl/DA-pw) over the street network for factors such as angle of incidence or aspect ratio. SIRANE utilises the friction velocity of the external flow to define a uniform exchange velocity across the street network and therefore the vertical flux of pollutants at the top face of each box . The vertical fluxes of NO x from uDALES, spatially averaged over the top face of each box and time averaged over the simulation time, were analysed to investigate how this idealised parametrisation compares to the exchange occurring within a realistic urban morphology. The predominant finding was that the heterogeneity of the morphology over the case study area challenges the typical shear layer as defined in idealised studies. Uneven building heights and secondary flows at intersections induce flow structures that cause advective as opposed to turbulent flux components to dominate. The heterogeneity of the urban morphology means that the tops of the defined boxes reside in the roughness sublayer where the flow is highly three-dimensional and the turbulent footprint is an amalgamation of the upwind microclimatic features. Applying the same averaging process to the vertical velocities in uDALES illustrated that there was an effective bulk transport in and out of the canopy in different locations. Typical values of exchange velocity were therefore challenging to extract from the 3-D roughness sublayer, however the overall vertical exchange from the canopy layer (canyons and intersections) must be captured well within SIRANE in order for there to be little fractional bias in the canyon-averaged NO x concentrations. As the friction velocity and boundary layer height were matched within uDALES this bulk parametrisation is shown to provide a good approximation of the bulk exchange out of the defined street network. These findings were reproduced within the westerly simulations.   The markers are consistent with Fig. 3.

Reactive case
In this section both models are run under the same conditions but with the respective chemistry schemes active. As previously discussed, the reactions of the null cycle act to equilibrate the concentrations of NO, NO 2 and O 3 towards a photostationary steady-state (PSS) at a timescale of minutes during the day. The NO x emissions result in a local chemical imbalance from the background PSS state which SIRANE solves by calculating an in-canyon equilibrium and which uDALES resolves in real-time alongside the dispersion of the emissions through the domain. Fig. 5 provides a comparison between the canyon-averaged NO, NO 2 and O 3 concentrations from SIRANE and uDALES. The results for NO exhibit a similar trend to that of inert NO x in Section 4.1. The clear shift between the quartiles of SI and DA-s in the box plots of Fig. 5b and c indicate that there is a significant over-(FB = −0.18) and underprediction (FB = 0.14) of NO 2 and O 3 respectively by SIRANE. The reason for this source of systematic bias is the assumption of a PSS in SIRANE. Whilst the assumption of a quasi-steady chemical state over the hourly time period holds due to the timescale over which the null cycle acts, it is not assured that photostationarity would be achieved throughout the urban canyon. Vehicular NO x emissions are NO rich and cause a local deviation from the PSS. The timescale at which an equilibrium is reached is of similar order of magnitude to the local dispersion meaning that in the proximity of emission sources a deviation from the PSS is expected. The chemical imbalance can be quantified by the PSS defect, ps , given by: ). In these locations the net effect of the null cycle is to deplete the concentrations of NO and O 3 and produce NO 2 . In uDALES these reactions are modelled in real time as the pollutants are dispersed. SIRANE's PSS assumption considers that the concentrations equilibrate throughout the defined domain but the PSS defects in Fig. 6b illustrate that this is not the case. The PSS defects calculated here concur with magnitudes found in the literature (100-200% near busy roads; Tong and Leung, 2012;Chate et al., 2014;Zhong et al., 2016). The trends exhibited in Fig. 5 indicate that the concentration of O 3 is the limiting factor in terms of the production of NO 2 in this case study. This state is a result of the background concentrations, reaction rates, magnitude of the emissions and NO 2 /NO x emissions ratio but is consistent with other research findings (Palmgren et al., 1996). The similarities between the results for reactive NO and inert NO x indicate that the emission of NO into the canyon is dominating over its depletion due to the chemical cycle. PSS defects in the westerly case exhibit the same trends.
The PSS defect also scales approximately with the residency time of the pollutant within the canyon. This effect has been captured in street canyon models where the emissions, exchange rate and chemical reactions can be solved analytically under an assumed steady-state (Palmgren et al., 1996). Whilst this can be implemented relatively simply for a single infinite canyon, the chemistry scheme in SIRANE would have to be directly coupled to the dispersion model to account for the exchange of pollutants throughout the street network and above (Soulhac et al., 2011).
The in-canyon variability of the concentration fields as defined in Section 4.1 is further complicated by the non-linear interaction between the simultaneous reaction and dispersion of the pollutants. The null T. Grylls, et al. Atmospheric Environment: X 3 (2019) 100041 cycle acts to produce and deplete the pollutants temporally and spatially throughout the domain as a function of the local chemical imbalance. Fig. 7 provides box plots comparing the bulk statistics of canyon-and pedestrian-averaged concentrations and Table 2 presents the linear regression analysis for the reactive pollutants (following the same process as in Section 4.1). Fig. 7a reiterates that the canyonaveraged NO concentrations follow a similar trend to that of inert NO x . The role of chemistry is evident however in the increased correction factor calculated for NO ( = m 1.75). The correction factor for NO was applied to produce the improved estimation of pedestrian level concentrations SI-c in Fig. 7a. The difference between pedestrian and canyon-averaged concentrations for NO 2 and O 3 is less pronounced ( = m 1.39 and 1.16 respectively), which can be expected as secondary pollutants exist more spatially homogeneously (Breuer and Bower, 1999). This finding once again agrees with the field study of Xie et al. (2003) where the vertical distributions of NO 2 and O 3 exhibit an increase and decrease respectively but to a lesser extent than the primary pollutants (NO and CO). The correction factors were not applied to NO 2 SI and O 3 SI as the systematic error introduced to these by the assumption of photostationarity outweighs the corresponding improvement. The linear regression analysis for the reactive pollutants indicates again that a simple linear model is capable of accounting for pedestrian level concentrations for the conditions investigated in this case study. Fig. 7 also presents the lee-and windward-averaged concentrations at pedestrian level. The same general trend of higher concentrations (lower for O 3 ) on the leeward side of the street canyon is exhibited. In a domain with longer streets the role of asymmetry is likely to be of greater importance, particularly in terms of the variability of the secondary pollutants.
The results presented in this section are consistent with field studies (Derwent et al., 1995;Palmgren et al., 1996;Chate et al., 2014) and a monitoring station that is present in the case study region (Air Quality England, 2019). However it is important to note the sensitivity of these results to the chemical state of the system (emissions ratio and magnitude, background concentrations and reaction rates). The magnitude and spatial distribution of the PSS defect captures the relationship between the dispersion and chemical transformation of pollutants within the street canyon. An understanding of the sensitivity of these results can be obtained by considering the expected change in this term. For example, a decrease in the magnitude of emissions would result in lower PSS defects and therefore a reduction in the error caused by the assumption of photostationarity in SIRANE. The emissions used in this study were relatively high and therefore SIRANE would be expected to perform better under lower emissions scenarios. The emissions ratio has been highlighted as a sensitive variable in the study of reactive dispersion models (Vardoulakis et al., 2007). One would expect that a reduction in the emissions ratio would result in larger PSS defects, particularly close to the emissions sources. The bias caused by the PSS assumption would therefore be more pronounced and e.g. the correction factor for NO 2 would be decreased (as a result of the reduced role of the primary emissions of NO 2 and increased role of secondary formation). Further work is required to quantify these sensitivities.

Conclusion
An evaluation of the predictive skill of the operational pollution dispersion model SIRANE was conducted using the LES model uDALES. A case study over South Kensington, London under neutral conditions was systematically set up to produce comparable results between the two models. The use of LES as a tool to perform this evaluation provided an element of numerical control (e.g. with or without chemistry), thereby negating typical sources of uncertainty, and facilitated analysis against an "ideal" dataset. This methodology enabled a targeted analysis of SIRANE's simplifications, assumptions and parametrisations over a realistic urban morphology. The study outlines a novel approach to the evaluation of operational pollution dispersion models that can be used alongside necessary validations against field and experimental data.
Considering the differences in resolution and computational cost, SIRANE was shown to qualitatively and quantitatively capture the dominant trends in canyon-averaged and above-canyon concentrations and velocities. Streets segments that deviated away from idealised, infinite canyons (short streets and streets next to open spaces) were shown to result in the most erroneous predictions by SIRANE across the evaluation. The most significant source of systematic bias resulted from the assumption of photostationarity in SIRANE. PSS defects as high as 170% were exhibited by uDALES close to busy roads. The PSS assumption was shown to lead to over-(FB = −0.18) and underpredictions (FB = 0.14) of NO 2 and O 3 respectively. SIRANE was shown to have a general tendency to overpredict (FB = −0.18) along-canyon velocities because it does not capture the inhibiting role of secondary flows at intersections. Although an underprediction was exhibited in streets with high angles of incidence (>77 ) where along-canyon flow was induced in uDALES due to morphological details within the street network.
The high-resolution capabilities of uDALES were used to investigate the applicability of SIRANE's uniform in-canyon concentration T. Grylls, et al. Atmospheric Environment: X 3 (2019) 100041 assumption. The use of canyon-averaged concentrations to assess pedestrian level exposure was shown to result in a significant underestimation. It was shown that a linear correction factor was capable of accounting for the difference between pedestrian-and canyon-averaged concentrations. The application of this correction factor to inert NO x ( = m 1.62) and reactive NO ( = m 1.75) was shown to significantly improve SIRANE's capability to predict pedestrian level exposure. The high corresponding R-squared values indicated that the linear trend was robust to the range of e.g. angles of incidence, wind directions, aspect ratios and emissions present in this study. The sensitivity of these correction factors to conditions not present in this case study was not investigated, however this finding highlights the need to conduct such a study or to couple SIRANE with a more detailed street canyon model in the future. Analysis of the vertical exchange out of the urban canopy layer highlighted the significant differences between the shear layers present over realistic heterogeneous urban morphologies and over classical infinite canyons. Despite these differences the bulk exchange out of the street network in SIRANE was proven to be captured with a reasonable agreement.
The explicit definition of street canyons within the LES domain allowed for direct comparison with SIRANE, but more importantly served to reduce the inherent complexity of the flows within the streets to its essential features (mean concentrations, pedestrian exposure, wind effects, exchange with the atmosphere). This approach could be of benefit more generally for urban pollution dispersion studies of realistic morphologies, as additional sophistication (e.g. by introducing separate control volumes at the intersections and higher order statistics) can be incorporated straightforwardly .  Fig. B.8shows the wind-and leeward profiles of NO x (= NO + NO 2 ; a passive pollutant following Equations (3) and (4)) and the passive scalar of Pavageau and Schatzmann (1999). The agreement shown in this figure indicates that uDALES captures the dispersion of a passive pollutant in an idealised urban canyon and that the applied chemistry scheme satisfies the assertion that the null cycle has no net chemical effect on the NO x concentration. The incorporation of the chemistry scheme as described in Section 2.2 has therefore been shown to be capable of modelling reactive pollutant dispersion (Han et al., 2018).

Figure B.8.
Vertical profiles of the spanwise-and time averaged NO x (NO + NO 2 of a reactive simulation following Han et al. (2018)) and passive scalar (experimental results of Pavageau and Schatzmann (1999)). Profiles were taken at a distance of 0.03H from each wall (where H is the building height) and normalised by the concentration on the leeward profile at = z H 0.08 .