Guidelines on validation procedures for meteorological data from automatic weather stations

Summary

Quality control is a major prerequisite for using meteorological information. High quality data sources are vital to scientists, engineers and decision makers alike. Validation of meteorological data ensures that the information needed has been properly generated and that it identifies incorrect values and detects problems that require immediate maintenance attention. In this work, several quality assurance procedures based on different criteria are proposed and applied to meteorological data from the Agroclimatic Information Network of Andalusia (Southern Spain) to assess their integrity and quality. The procedures include validations of record structure data, range/limits, time and internal consistency, persistence and spatial consistency tests. Quality assurance tests consist of procedures or rules against which data are tested, setting data flags to provide guidance to end users. The proposed system is capable of identifying several types of errors and is used as a tool that allows one to make decisions such as sensor replacement and to remove data prior to their application.

Introduction

Meteorological observations and related environmental and geophysical measurements are necessary for a real-time preparation for weather analyses, forecasts and severe weather warnings, for the study of climate, for local weather-dependent operations, for hydrology and agricultural meteorology, and for research in meteorology and climatology (WMO, 2008). Some of the applications for meteorological data include: risk assessments, hydraulic structures design, crop water-use estimates, irrigation scheduling, input variables of climate change and hydrological models, active and passive renewable energy uses, etc. (Weiss and Robb, 1986, Meyer and Hubbard, 1992, Del Greco et al., 2005, Flores et al., 2005, Younes et al., 2005).

During the last two decades, the number of automated weather station networks has greatly increased throughout the world. This rapid development has been the consequence of the need to provide meteorological data in near-real time and the great evolution of automatic data acquisition systems (Miller and Barth, 2003). Irrigation Agroclimatic Information System (SIAR in Spanish language) is the most important automated meteorological network in Spain. It was installed in the period 1999–2000, covering the majority of irrigated areas of Spain. SIAR was established across all the Spanish regions for agronomic purposes (Pérez de los Cobos et al., 2003). The climate networks of many National Meteorological Services only provide temperature and precipitation data (Gázquez et al., 2003). Nevertheless, these data are not easily available with the frequency required (Meyer and Hubbard, 1992), in spite of World Meteorological Organization (WMO) resolutions requesting free and unrestricted exchange of climate information between National Organizations including the private sector (Cuadrat et al., 2002). Nowadays, the improvement and strengthening of meteorological observation networks to support data collection for operational applications and applied research in hydrological processes, agricultural meteorology or climate change models are required (WMO, 2006).

For purposes of scientific research and resource management, meteorological data from weather stations are recorded in large databases. All these efforts are enhanced by quality checking the data; however, no consistent methodologies are employed in many cases. Questionable results have been attributed to poor data quality as a consequence of non-existing or mixed quality control methods (Meek and Hatfield, 1994). There are three important reasons for applying quality control procedures to meteorological data: (i) to ensure that meteorological information is properly generated; (ii) to identify erroneous data involving inadequate decision making; and (iii) to detect and solve problems for a correct maintenance of the stations and periodic calibration of sensors (Doraiswamy et al., 2000).

Quality control of data is the best known component of quality management systems. It consists of the examination of data at weather stations and data centers with the aim of detecting errors (WMO, 1993). Different methods can be applied to ensure quality of meteorological data: periodic maintenance of stations and field sensor checks, validation of data using statistic procedures and, finally, sensor calibration. O’Brien and Keefer (1985) proposed a set of three computer-based rules, which were applied by Meek and Hatfield (1994) to validate meteorological data. These rules include computation of fixed or dynamic high/low bounds for each variable, use of fixed or dynamic rate of change limits for each variable and a continual no-observed-change in time limit. In the validation process, data of a doubtful quality must be detected and appropriately flagged. These quality control flags, which supplement but do not alter the data, are employed to describe which test the data failed. All archived meteorological data must be coupled with flags (“good”, “suspect”, “warning” or “failure”) that indicate the level of confidence that network managers place upon their observations (Fiebrich and Crawford, 2001). Quality control flags are grouped into two main categories, “informative” and “severe”, such as the California Irrigation Management Information System (CIMIS) (Snyder and Pruitt, 1992). In some cases, algorithms can be applied to correct erroneous data or to fill gaps, but both original and corrected data should always be archived in the databases (Reek et al., 1992). Finally, meteorological data considered as potentially erroneous should be verified and manually inspected by qualified personnel.

This paper presents some guidelines for applying quality control procedures to meteorological measurements according to several tests that use data from a single site (Meek and Hatfield, 1994, Shafer et al., 2000) and others that use data from multiple sites, comparing a station’s data against neighboring stations (Reek et al., 1992, Hubbard, 2001). These validation methods have been applied to the Agroclimatic Information Network of Andalusia (RIAA), belonging to Spanish SIAR Network, using data from 1999 to 2006. The impact of error measurements in climate data on reference evapotranspiration (ET₀) has been recently studied for this network (Estévez et al., 2009). In general, valid meteorological data are required to make climate assessments and to make climate-related decisions. In semiarid regions, with a structural water deficit, the integrity and quality of these data are crucial to improving ET₀ estimates and precipitation, ensuring an adequate irrigation water management.