Impact of measurement error and limited data frequency on parameter estimation and uncertainty quantification

Highlights

• Historical observed data is required for calibration.

• Measurement error and limited data frequency result in parameter uncertainty.

• The results highlight the critical roles of measurement error and frequency in the calibration.

• The effect of the measurement uncertainty is significant when the calibrated data are limited.

• The research findings can be used to support measurement prioritization and resource allocation.

Abstract

Parameter estimation, using historical observed data, is an important part of the environmental modeling. The uncertainty in the parameter estimation limits the applications of environmental models.

In this paper, the influence of limited and uncertain calibrated data on the performance of the parameter estimation are systematically investigated. For this purpose, synthetic observations with a given uncertainty and frequency are used to estimate the model parameters of a conceptual water quality (WQ) model of the River Zenne, Belgium. Bayesian inference using Markov Chain Monte Carlo sampling is adopted to simultaneously perform the automatic calibration and the uncertainty analysis. The results highlight the critical roles of measurement frequency and uncertainty in the model calibration. We found that the effect of the measurement uncertainty on the parameter estimation is significant when the calibrated data points are limited (e.g. monthly data). The research findings can be used to support measurement prioritization and resource allocation.

Introduction

In order to use environmental models for different tasks, such as predictions, scenario analysis, and setting up regulations, they should represent the reality adequately. Moreover, they should be scientifically sound, robust and defensible (U.S. EPA. 2002). In general, model results are affected by the model structure (i.e. the model assumptions), the model inputs, the boundary conditions, and the model parameters (van Griensven and Meixner, 2006; Rode et al., 2010). The underlying assumptions of the model are often fixed, and therefore, the model structure is not changed during the modeling processes. Moreover, the input data and the boundary conditions, obtained through measuring campaigns or provided by responsible authorities, are not altered by the modeler (Nossent, 2012). On the other hand, most of the model parameters, representing some properties of the system, cannot be measured directly (Vrugt et al., 2003). As a consequence, the model parameters should be set to appropriate values in order to increase the agreement between the model results and the real system. The parameters adjustment is based on the reduction of the difference between the model results and historical measurements of the system response (Laloy et al., 2010; Vrugt et al., 2013; Leta et al., 2015). This procedure is referred to as parameter estimation, parameter optimization, model calibration or inverse modeling (Raat et al., 2004).

Parameter estimation using historical observed data is an important part of the environmental modeling practice which has been the focus of many researches and studies (e.g. Duan et al., 2006). However, because the models are only an approximation of the real system and the observed data used for the calibration contain error (i.e. measurement uncertainty), parameter estimation is error-prone (Vrugt et al., 2002). As a result, it is difficult to well identify the parameters and the parameter uncertainty is caused. In addition, in some fields, such as water quality modeling, the data collection is resource-intensive (Mannina, and Viviani, 2010), consequently, the available data has a limited frequency, e.g. biweekly or even monthly intervals (Zheng and Keller, 2007a). In these cases, a serious complication for the model calibration is the lack of reliable calibration data which results in parameter uncertainty (Raat et al., 2004; Franceschini, and Tsai, 2010a,b). The ambiguity in the parameter estimation has considerable impact on the model simulation uncertainty, and, therefore, limits the applications of environmental models, such as water quality models (Wagener et al., 2003).

Despite the critical role of amount and reliability of calibration data on the performance of the parameter estimation, to the best of the authors’ knowledge, the literature contains very few studies on quantifying the impact of these two aspects of the measurements (i.e. amount and reliability) on the parameter uncertainty of water quality models. For example, for a catchment nitrogen modeling, Raat et al. (2004) explored the relationship between the quality of the calibration data (i.e. measurement uncertainty) and the uncertainty associated with the final parameter estimates (i.e. parameter uncertainty), using virtual data. Wang et al. (2017) used synthetic data to explore how the number of tracer (i.e. isotope) data samples (i.e. measurement amount) affect model calibration. However, the effect of measurement errors of the tracer data on the parameter estimation was not studies. Therefore, it is needed to investigate the effect of both amount and reliability of the calibration data on the performance of the calibration. Neglecting the other source of uncertainties (e.g. model structural uncertainty, input data uncertainty), a modeler should first investigate that it is feasible to reach a pre-defined model performance with a given amount of uncertain measured data.

As the impact of these critical characteristics of calibration data, amount and reliability, on the model calibration has not fully addressed in literature, in this study, we focus on assessing the influence of limited uncertain calibrated data on the performance of the parameter estimation and on the parameter uncertainty intervals of water quality models. For this purpose, the following questions are formulated:

• What is the effect of increasing the measurement frequency on the water quality parameter estimation?

• What is the effect of reducing the measurement error of the observed data on the water quality parameter estimation?

To address the research questions, synthetic observations with a given uncertainty and frequency are used to estimate the model parameters of a conceptual water quality (WQ) model of the River Zenne in Belgium, for simulation dissolved oxygen (O2) and biological oxygen demand (BOD). As an optimization tool, Bayesian inference using Markov Chain Monte Carlo (MCMC) sampling (Bates and Campbell, 2001; Kuczera and Parent, 1998), is adopted to simultaneously perform the automatic calibration and the uncertainty analysis. The synthetic data series are generated by running a WQ model, with a given set of model parameter values as ‘true’ values. The model outputs are then perturbed with a pre-specified random error, as measurement error, and sampled with a given frequency to mimic discrete measurements. The sampled data are then considered as if they were observed data and used to calibrate the model parameters, using the MCMC algorithm. Finally, it is verified if the model parameters can be identified using the limited and uncertain data. To evaluate the relationship between the amount and reliability of the calibration data and the parameter uncertainty, nine different synthetic data sets are generated by increasing the measurement error and decreasing the measurement frequency. Then, the generated synthetic data are used as calibration data in subsequent optimization runs.