Elsevier

Atmospheric Environment

Volume 53, June 2012, Pages 51-59
Atmospheric Environment

ENSEMBLE and AMET: Two systems and approaches to a harmonized, simplified and efficient facility for air quality models development and evaluation

https://doi.org/10.1016/j.atmosenv.2011.08.076Get rights and content

Abstract

The complexity of air quality modeling systems, air quality monitoring data make ad-hoc systems for model evaluation important aids to the modeling community. Among those are the ENSEMBLE system developed by the EC-Joint Research Center, and the AMET software developed by the US-EPA. These independent systems provide two examples of state of the art tools to support model evaluation. The two systems are described here mostly from the point of view of the support to air quality model users or developers rather than the technological point of view. While ENSEMBLE is a web based platform for model evaluation that allows the collection, share and treatment of model results as well as monitoring data, AMET is a standalone tool that works directly on single model data. The complementarity of the two approaches makes the two systems optimal for operational, diagnostic and probabilistic evaluations. ENSEMBLE and AMET have been extended in occasion of the AQMEII two-continent exercise and the new developments are described in this paper, together with those foreseen for the future.

Introduction

In atmospheric dispersion and air quality modeling, the evaluation of the model performance has long been a topic of research activities and establishing best practices (e.g. Oreskes et al., 1994, Steyn and Galmarini, 2008). Before we tackle the model evaluation topic per-se, it is probably appropriate to give a couple of definitions. We will refer here to atmospheric dispersion models as the models that simulate the dispersion of a passive, decaying or reactive species released from a point source, be it a stack or a limited-area source. No limit exists to the scale of application which can extend as far as global. These are the models normally used for emergency preparedness and response applications. With air quality model we refer however to a model which deals with distributed sources of various chemical precursors and that treats the chemical transformations occurring in a volume of air that can range from the meso- to the global-scale in a three-dimensional domain. In the two cases the dynamic fields are acquired from meteorological models. The final application of these models is in the realm of support to air quality policy planning and regulation.

The verification of the model capability to adhere to experimental evidence has occupied atmospheric scientists in recent decades. The complexity of the atmospheric system in terms of spatial and temporal variability, stochasticity and scales, makes the task of evaluating a model particularly complicated. The collection of experimental evidence, representative of space and time scales, which could be used for evaluation of a time- and space-averaged model result, has always constituted the first burden in the practice of model evaluation. To date, operational networks of instruments provide organized, quality-checked information useful in this context for both atmospheric dispersion and air quality models. Routine monitoring data however are often insufficient to assess the performance of a model in depth, as they may only provide evidence of the general correspondence of model results with surface-based point measurements that are typical in such network data. These types of evaluations do not provide clear indications of the veracity of the modeled-chain of processes that leads to the model result. In other words, using a common expression, we would not know whether the right model results were obtained for the right reasons.

The problem becomes particularly complicated in the case of predictions where data are not sufficiently available. This is the case, for example, in atmospheric dispersion modeling or scenario studies in air quality. In such cases as well as in the case of a total absence of experimental evidence, a reasonable alternative to model evaluation is model inter-comparison, meaning that models are applied to a common case study and inter-compared. The relative differences in model results are useful indications of uncertainties in modeling science or data that need attention.

The Joint Research Center of the European Commission and the US-EPA over the past two decades have played an important role in promoting model evaluation and bringing together the European, US and international model communities in promoting best practices in model evaluation. These activities were intended to develop and promote standards in model evaluation (e.g. Gamarini et al., 2010). Over the last decades this practice in particular has been extremely important in bringing together communities of model developers and users and has led to very important developments beyond the singular practice of model evaluation.

Given the complexity of air quality modelling systems and diversity of information necessary to run them and to evaluate them, the availability of tools that could assist modelers in the latter is very important. On the one hand one wants to guarantee that model data produced by different sources are handled in the same way. At the same time one should consider that diverse monitoring information is assembled, organized and harmonized so that all users could use the same source of information efficiently. In this context diversity pertains not only to the nature of the information (meteorology, micro-meteorology, atmospheric chemistry), but also to the fact that same atmospheric species could be measured differently by different networks. In parallel when evaluating a model one would like to use similar metrics and a standard set of tools so that all model performances are assessed against the same ruler and terms of comparison.

Toward these ends, information technology has provided us with useful means for an easier, consistent, standardized and practical way of evaluating models. Over the last decade the European Commission Joint Research Center and US-EPA have independently invested resources in the development of two distinct systems for model evaluation, namely, the ENSEMBLE system (Galmarini et al., 2004a, Galmarini et al., 2004b) and AMET (Appel et al., 2011). While they both work toward the same goals they are different in philosophy and approaches and cover, in a complementary way, different aspects of the problem.

Similar systems exist for more specialized applications such as AeroCom (Textor et al., 2006; http://nansen.ipsl.jussieu.fr/AEROCOM), which provides an online access to consultation of global model predictions observations of various kinds relating to aerosols and aerosols components. Off line model evaluation tools were developed over the years on the footprints of AMET such as BOOT (Chang, 2002; Chang and Hanna, 2004) specialized in atmospheric dispersion models and the CITY-DELTA tool still from the Joint Research Center (Thunis et al., 2008) specialized in the assessment of differences in air quality scenarios at the mesoscale models.

ENSEMBLE and AMET have been used extensively for the AQMEII activity and without them a project of this scale could not happen in the time scale in which it was performed. In this paper the two independent systems are presented in terms of their usefulness for both model evaluation and inter-comparison application, including their future developments and synergies. The perspective presented is that of the model developer/user rather than the information technology side to give an idea on the amount of resources and rapid advancements to the atmospheric science discipline such systems can provide. A model evaluation framework adopted by the AQMEII activity (Dennis et al., 2010) that considers four types of evaluation: operational, diagnostic, dynamic, and probabilistic. Operational is intended as direct comparison of model results with observations which may lead to assessment pass or fail; diagnostic is intended as investigation on the reasons for a pass or a fail; diagnostic would lead to determining the sensitivity of model results to input data, whereas the probabilistic evaluation aims at providing an assessment of confidence in model results and uncertainty. We will show that the two system serve all these type of evaluation thus allowing a thorough and complete analysis of model performance.

Section snippets

Evaluating models through a web-distributed service: the ENSEMBLE system

ENSEMBLE was developed in 2000 as a system for the real-time acquisition and consultation of the results of several models produced by different groups on the same case study. The field of application was long range atmospheric dispersion forecasting. Details on that application can be found in Galmarini et al. (2001), Bianconi et al., 2004, Galmarini et al., 2004a, Galmarini et al., 2004b, Potempski et al., 2008, Galmarini et al., 2008.

Fig. 1 shows schematics of the concept behind the ENSEMBLE

Evaluating models through a standalone service: the AMET system

The Atmospheric Model Evaluation Tool (AMET; Appel et al., 2011) was designed to aid in evaluation of meteorological and air quality model output, and has been utilized by a number of the AQMEII modeling groups to aid in the evaluation of their model output.

The AMET software is a collection of several software packages integrated together to provide a system for storing and analyzing meteorological and air quality observations and model output. AMET uses FORTRAN and Perl code to pair model

Conclusions

Over the past couple of decades a number of modeling consortia have been established where a large number of institutions have come together with the scope of comparing the results of their models with others in the scientific community. It is from discussions within these consortia that activities of such nature can only be beneficial to promoting progress in modeling science and application, in particular in the case of air quality modeling. The progress produced by these communities in terms

Acknowledgements

The authors would like to thank Dr C. Hogrefe (Bureau of Air Quality Analysis and Research, New York State Department of Environmental Conservation) for his constant support and advice on the collection and harmonization of monitoring information and Drs R. Gilliam (US-EPA) and J. Zhang (Environment Canada) for their contribution in data preparation. The authors also acknowledge for the strong support of the European Commission, Airbus, and the Airlines (Lufthansa, Austrian, Air France) who

References (20)

There are more references available in the full text version of this article.

Cited by (53)

  • Joint Nordic nuclear research to strengthen nuclear emergency preparedness after the Fukushima accident

    2018, Journal of Environmental Radioactivity
    Citation Excerpt :

    Among other things the group performs real-time exercises on atmospheric dispersion of radioactive matter. It maintains liaison with relevant international activities, including the EU ENSEMBLE platform (Galmarini et al., 2012) that can collect dispersion forecasts provided by more than 20 operational models. On the research and development level, Nordic organisations naturally also participate in wider international networks in European collaborative research and development projects, particularly under the EURATOM framework.

  • Theoretical Predictive Air Quality Models

    2016, Comprehensive Analytical Chemistry
View all citing articles on Scopus
View full text