Technical NoteGuidelines on validation procedures for meteorological data from automatic weather stations
Highlights
► We propose several quality assurance methods for meteorological information. ► The procedures include validation of structure data, and range, step, persistence and spatial consistence tests. ► The developed system is an useful tool for providing guidance to end users.
Introduction
Meteorological observations and related environmental and geophysical measurements are necessary for a real-time preparation for weather analyses, forecasts and severe weather warnings, for the study of climate, for local weather-dependent operations, for hydrology and agricultural meteorology, and for research in meteorology and climatology (WMO, 2008). Some of the applications for meteorological data include: risk assessments, hydraulic structures design, crop water-use estimates, irrigation scheduling, input variables of climate change and hydrological models, active and passive renewable energy uses, etc. (Weiss and Robb, 1986, Meyer and Hubbard, 1992, Del Greco et al., 2005, Flores et al., 2005, Younes et al., 2005).
During the last two decades, the number of automated weather station networks has greatly increased throughout the world. This rapid development has been the consequence of the need to provide meteorological data in near-real time and the great evolution of automatic data acquisition systems (Miller and Barth, 2003). Irrigation Agroclimatic Information System (SIAR in Spanish language) is the most important automated meteorological network in Spain. It was installed in the period 1999–2000, covering the majority of irrigated areas of Spain. SIAR was established across all the Spanish regions for agronomic purposes (Pérez de los Cobos et al., 2003). The climate networks of many National Meteorological Services only provide temperature and precipitation data (Gázquez et al., 2003). Nevertheless, these data are not easily available with the frequency required (Meyer and Hubbard, 1992), in spite of World Meteorological Organization (WMO) resolutions requesting free and unrestricted exchange of climate information between National Organizations including the private sector (Cuadrat et al., 2002). Nowadays, the improvement and strengthening of meteorological observation networks to support data collection for operational applications and applied research in hydrological processes, agricultural meteorology or climate change models are required (WMO, 2006).
For purposes of scientific research and resource management, meteorological data from weather stations are recorded in large databases. All these efforts are enhanced by quality checking the data; however, no consistent methodologies are employed in many cases. Questionable results have been attributed to poor data quality as a consequence of non-existing or mixed quality control methods (Meek and Hatfield, 1994). There are three important reasons for applying quality control procedures to meteorological data: (i) to ensure that meteorological information is properly generated; (ii) to identify erroneous data involving inadequate decision making; and (iii) to detect and solve problems for a correct maintenance of the stations and periodic calibration of sensors (Doraiswamy et al., 2000).
Quality control of data is the best known component of quality management systems. It consists of the examination of data at weather stations and data centers with the aim of detecting errors (WMO, 1993). Different methods can be applied to ensure quality of meteorological data: periodic maintenance of stations and field sensor checks, validation of data using statistic procedures and, finally, sensor calibration. O’Brien and Keefer (1985) proposed a set of three computer-based rules, which were applied by Meek and Hatfield (1994) to validate meteorological data. These rules include computation of fixed or dynamic high/low bounds for each variable, use of fixed or dynamic rate of change limits for each variable and a continual no-observed-change in time limit. In the validation process, data of a doubtful quality must be detected and appropriately flagged. These quality control flags, which supplement but do not alter the data, are employed to describe which test the data failed. All archived meteorological data must be coupled with flags (“good”, “suspect”, “warning” or “failure”) that indicate the level of confidence that network managers place upon their observations (Fiebrich and Crawford, 2001). Quality control flags are grouped into two main categories, “informative” and “severe”, such as the California Irrigation Management Information System (CIMIS) (Snyder and Pruitt, 1992). In some cases, algorithms can be applied to correct erroneous data or to fill gaps, but both original and corrected data should always be archived in the databases (Reek et al., 1992). Finally, meteorological data considered as potentially erroneous should be verified and manually inspected by qualified personnel.
This paper presents some guidelines for applying quality control procedures to meteorological measurements according to several tests that use data from a single site (Meek and Hatfield, 1994, Shafer et al., 2000) and others that use data from multiple sites, comparing a station’s data against neighboring stations (Reek et al., 1992, Hubbard, 2001). These validation methods have been applied to the Agroclimatic Information Network of Andalusia (RIAA), belonging to Spanish SIAR Network, using data from 1999 to 2006. The impact of error measurements in climate data on reference evapotranspiration (ET0) has been recently studied for this network (Estévez et al., 2009). In general, valid meteorological data are required to make climate assessments and to make climate-related decisions. In semiarid regions, with a structural water deficit, the integrity and quality of these data are crucial to improving ET0 estimates and precipitation, ensuring an adequate irrigation water management.
Section snippets
Data
Meteorological data used in the analysis were obtained from the RIAA. This network is currently composed of 99 automatic weather stations with the aim of providing ET0 values and other meteorological data to improve irrigation water management (Gavilán et al., 2008). Daily data can be obtained from the Web at www.juntadeandalucia.es/agriculturaypesca/ifapa/ria. Automatic weather stations are controlled by a programmable CR10X datalogger (Campbell Scientific Inc., CSI, Logan, Utah) and are
Quality control
The results of running the procedures on the dataset for each climate variable are reported below and they are summarized in Table 4. A previous analysis of meteorological database for detecting gaps was necessary before applying quality control tests. In this sense, no trend is observed, 2005 and 2006 being the years with fewer gaps.
Summary and conclusions
A growing number of climate change and variability studies, as well as applied research towards improving engineering design climatographies, require high-quality meteorological data. It is known that long-term and extreme-value climate data sets are necessary for accurate and reliable estimates and assessments, being the validation of these data necessary for present and future uses. Several quality assurance procedures have been introduced for assessing integrity of meteorological data from
Acknowledgments
This work was supported by the Spanish National Institute of Agricultural Research (INIA) under Project RTA04-063. The authors are grateful to Mr. Juan de Haro for his valuable work in the Agroclimatic Information Network of Andalusia.
References (58)
Using the FAO-56 dual crop coefficient method over an irrigated regions as part of an evapotranspiration intercomparison study
Journal of Hydrology
(2000)- et al.
Techniques for methods of collection, database management and distribution of agrometeorological data
Agricultural and Forest Meteorology
(2000) - et al.
Application of a control algorithm for wind speed prediction and active power generation
Renewable Energy
(2005) - et al.
Regional calibration of Hargreaves equation for estimating reference ET0 in a semiarid environment
Agricultural Water Management
(2006) - et al.
A web service for controlling the quality of measurements of global solar irradiation
Solar Energy
(2002) - et al.
Data quality checking for single station meteorological databases
Agricultural and Forest Meteorology
(1994) - et al.
Quality control of solar radiation data: present status and proposed new approaches
Energy
(2005) - AEMET, 2008. Resumen de extremos climatológicos en España....
Assessing integrity of weather data for reference evapotranspiration estimation
Journal of Irrigation and Drainage Engineering ASCE
(1996)- Allen, R.G., Pereira, L.S., Raes, D., Smith, M., 1998. Crop Evapotranspiration: Guidelines for computing Crop Water...
An update for the calculation of reference evapotranspiration
ICID Bulletin
A technique for maximizing details in numerical weather map analysis
Journal of Applied Meteorology
A quality control procedure for hourly wind data
Journal of Atmospheric and Oceanic Technology
Automated Validation Precipitation Data Using GIS and Spatial Interpolation Schemes. General Assembly European Geosciences Union, 24–29 April 2005
The quality control of long-term climatological data using objective data analysis
Journal of Applied Meteorology
A comprehensive aerological reference data set (CARDS): rough and systematic errors
Bulletin of the American Meteorological Society
Sensitivity analysis of a Penman-Monteith type equation to estimate reference evapotranspiration in southern Spain
Hydrological Processes
Quality control of daily meteorological data in China, 1951–2000: a new dataset
International Journal of Climatology
The impact of unique meteorological phenomena detected by the Oklahoma Mesonet and ARS Micronet on automated quality control
Bulletin of the American Meteorological Society
Complex quality control of meteorological data
Monthly Weather Review
Comparison of standardized reference evapotranspiration equations in Southern Spain
Journal of Irrigation and Drainage Engineering ASCE
Relationships among daily mean and maximum wind speeds with application to data quality assurance
International Journal of Climatology
Improved quality assurance for historical hourly temperature and humidity: development and application to environmental analysis
Journal of Applied Meteorology
Complex quality assurance of historical hourly surface airways meteorological data
Journal of Atmospheric and Oceanic Technology
Cited by (136)
Automatic quality control of weather data for timely decisions in agriculture
2024, Smart Agricultural TechnologyA multifactorial study of mass movement in the hilly and gully Loess Plateau based on intensive field surveys and remote sensing techniques
2024, Science of the Total EnvironmentEvaluating the freeze-thaw vulnerability of soapstone monuments and geoheritage sites in the Parco del Paradiso (Chiavenna, Italy).
2024, Procedia Structural IntegrityReference evapotranspiration projections in Southern Spain (until 2100) using temperature-based machine learning models
2023, Computers and Electronics in AgricultureA regional machine learning method to outperform temperature-based reference evapotranspiration estimations in Southern Spain
2022, Agricultural Water Management