How model and input uncertainty impact maize yield simulations in West Africa

Crop models are common tools for simulating crop yields and crop production in studies on food security and global change. Various uncertainties however exist, not only in the model design and model parameters, but also and maybe even more important in soil, climate and management input data. We analyze the performance of the point-scale crop model APSIM and the global scale crop model LPJmL with different climate and soil conditions under different agricultural management in the low-input maize-growing areas of Burkina Faso, West Africa. We test the models’ response to different levels of input information from little to detailed information on soil, climate (1961–2000) and agricultural management and compare the models’ ability to represent the observed spatial (between locations) and temporal variability (between years) in crop yields. We found that the resolution of different soil, climate and management information influences the simulated crop yields in both models. However, the difference between models is larger than between input data and larger between simulations with different climate and management information than between simulations with different soil information. The observed spatial variability can be represented well from both models even with little information on soils and management but APSIM simulates a higher variation between single locations than LPJmL. The agreement of simulated and observed temporal variability is lower due to non-climatic factors e.g. investment in agricultural research and development between 1987 and 1991 in Burkina Faso which resulted in a doubling of maize yields. The findings of our study highlight the importance of scale and model choice and show that the most detailed input data does not necessarily improve model performance.


Introduction
Importance of input data and scale for processbased crop modeling (1) Modeling agro-ecosystems aims at describing and understanding relevant plant processes and their interactions with abiotic and biotic factors and as well as future behavior of the system e.g. in the face of global climate change. While on the field scale it is possible to study crop growth and yield going to the regional/national scale opens the possibility to also analyze crop production, crop-climate interactions and land use and cropping pattern simulations for a country or larger region (Yu et al 2012). Rötter et al (2011) identified ways of getting crop models to account for the variable landscape and environmental conditions across larger areas and estimate larger-scale regional productivity as a main future research activity. Studies on the up-scaling of indicators of agricultural productivity should therefore consider indicators of temporal, spatial and the interaction of temporal and spatial variation (Olesen et al 2000). (2) Using appropriate input data is one requirement for this task. For larger scales input data is less frequently available and more uncertain due to interpolation. Large-scale data sets are available for soils, weather or climate and management (incl. crop type, planting dates, cultivar) that is the typical input for a crop model and also varies spatially (Hansen and Jones 2000). The quality of this data is different in world regions. Especially in the tropics the quality of climate data is low or data is scarce. The two largest continents together covering about half of the Earth's land surface, Africa and Asia are the regions most weakly covered with only about 5.000-6000 stations each (11-12% of total), reporting precipitation between 1950 and 2012 for the Global Precipitation Climatology Centre database which challenge the efforts to achieve a highly reliable gridded climate product (Becker et al 2013). Crop modelers are concerned about input data accuracy as this together with an adequate representation of plant physiology processes and choice of model parameters are the key factors for a reliable simulation. (3) The spatial scale and aggregation level of the input data influences simulation results. Mearns et al (2001), for a modeling study in Central Great Plains of the United States, found that the spatial scale of the climate scenario had a large effect on the mean yields. At the same time the spatial scale of soils had a larger effect on the spatial variability of yields than did the spatial scale of the climate scenarios. In contrast Van Bussel et al (2011) concluded that aggregating weather information had only little effect on predicting wheat phenology in a modeling study in Germany but point out that the aggregation error might be larger in regions with large spatial heterogeneity in weather conditions. De Wit et al (2005) showed that aggregated winter wheat yields on a national scale are independent on the accuracy of precipitation and radiation data in France and Germany and concluded that the uncertainty in input data strongly decreases when simulation results are spatially aggregated to regional or national scale. In agreement with these finding Olesen et al (2000) showed that for estimating the aggregated effects of climate change on national productivity of winter wheat in Denmark it is not necessary to apply climate information on a fine resolution. (4) For West Africa, the influence of uncertainty in weather information can be very different. For example, Ramarohetra et al (2013) showed that uncertainty in rainfall can introduce large biases in simulated maize and millet yields in Benin and Niger. Also Baron et al (2005) showed that aggregating weather information, in particular rainfall, produced biases in yield simulations which needs to be considered in drought-related studies in semi-arid Niger. (5) Simulated crop yields will differ depending on the crop model used. Several crop model intercomparison studies showed that simulated crop yields largely depend on the crop model applied (Porter et al 1993, Singh et al 2008, Palosuo et al 2011, Eitzinger et al 2012, Rötter et al 2012, Asseng et al 2013, Rosenzweig et al 2013 and that differences in modeling approaches affect the simulation results as shown e.g. for cassava growth and development (Gray 2000), and for the effect of increased atmospheric CO 2 on grain yield (Tubiello and Ewert 2002). (6) However the differences in simulated crop yield have not been explicitly attributed to different assumptions, processes included, training and validation of the model, or the scale a crop model is usually applied at, although Rosenzweig et al (2013) distinguish between crop yield changes on a global scale from site-based models and from ecosystem models. (7) Except for Angulo et al (2013) (four barley models, Finland, weather data) and Asseng et al (2013) (27 wheat models, global, soil, climate, management data) there is no dynamic crop model by input data comparison study in a larger region which limits the robustness of conclusions in studies which only analyze the effects of different input data because crop models differ in their sensitivity to input data.

Research questions
The overarching research question is if more detailed input data provided for crop modeling improves the simulations of two different crop models. We postulate that input data resolution and the level of information will matter more or less depending on the crop model used and its model-specific sensitivity to soil parameters, climate variables and management information. While one might argue that the higher the resolution of the model input the better this might not be true considering the scale the model was developed for and the aggregation level of the model output. We seek to answer the following question. (1) How does the level of information of different soil, climate and management data sets influence the crop yields simulated from two crop models, and how well do they compare with agricultural statistics? Regarding the importance of model output aggregation for comparison among the two models and with agricultural statistics we further aim at answering the questions. (2) Are there important differences in the simulated spatial variability of simulated crop yields on the grid-cell level and (3) are there important differences in the simulated temporal variability of simulated crop yields on the aggregated national level? (figure 1).

Study area
(1) Mean annual rainfall in Burkina Faso follows a gradient from wetter southwest with up to 1200 mm rainfall (Guinean zone) to drier Northeast with 300-600 mm (Sahelo-Sudanian zone). The central region is characterized by 600-1000 mm rainfall (Sudanian zone) (figure 2). The climate follows the seasonal movement of the intertropical convergence zone which leads to strong influence of the northern dry, high pressure system from October to April and from tropical maritime masses between May and September.
(2) About 43% of the land area is covered with agricultural land (annual and perennial crops and pastures) and almost all of it is rainfed; the area under full or partial irrigation control was reported to be 18 600 ha in 2001 (0.15% of land area), mostly in the regions Hauts Bassins and Boucle du Mouhoun in the Central-South part (FAO 2014). Maize is the third most important crop in Burkina Faso in terms of harvested crop area (950 500 ha in 2012) after sorghum (180 000 ha) and millet (1 300 000 ha) (FAO 2013).

Methods
The crop models APSIM and LPJmL (1) Process-based crop models simulate crop development and growth. We compare two models which were developed for different scales: the point-scale model APSIM (Keating et al 2003) and the global model LPJmL (Bondeau et al 2007). APSIM is a modeling framework influenced by models like CERES while LPJmL is based on the dynamic global vegetation model LPJ that was developed to assess changes in terrestrial vegetation structure and biomass, carbon stored and water fluxes. The crop routines of LPJmL are largely based on EPIC and SWAT/SWIM models. (2) Both models aim at reflecting the management as well as actual and possible crop yields properly at a certain location. For APSIM this location is mostly a paddock, a field but also, in a multi-point simulation a sub-national unit or a country. For LPJmL this location is a grid cell but usually model outputs for grid cells are aggregated to sub-national units, countries, continents or world regions. Both types of models differ in their input data, their model components and approaches to simulate soil, water, nitrogen and crop growth processes (table 1)   daily time steps and uses daily meteorological data. It is frequently applied for one or several locations within a country or across countries on the micro (1-20 km 2 ) -to meso-scale (20-50 km 2 ). Daily input data can be created from nearby weather stations, grids or climate surfaces. Soil data is taken from a soil database comprising individual soil profiles or representative soils for a location are created by the user. (4) The global agricultural model LPJmL runs on daily time steps as well and with monthly or daily meteorological data. If monthly data is provided, monthly temperature, precipitation and fractional cloud coverage/ radiation are downscaled to daily resolution via a semi-stochastic weather generator. The model runs for all 0.5 × 0.5°grid cells on the Earth's land surface (macro-scale, 50-5000 km 2 ) and results are usually aggregated to the national or world region level. A grid-cell model output can be interpreted as a point (center of the grid cell) or mean value for that grid cell. By default, soil texture classes are taken from the Harmonized World Soil Database version 1.2 from FAO-IIASA (Nachtergaele et al 2012) and soil water dynamics are simulated with the multi-bucket approach described in Schaphoff et al (2013). Sowing dates in both models can be prescribed or calculated internally from a weather dependent sowing day rule.

Input data and nomenclature for model simulations
We simulate grid-cell and national maize yields between 1961 and 2000 with APSIM and LPJmL. We compare simulated maize yields with different input data for climate, soil and sowing dates (table 2). Up to eight combinations of different input settings are possible for the two crop models. Even though some settings might not be very practicable for an actual model application they represent the upper and lower level of information/resolution of input data available for the study area. The eight combinations that are investigated in this study are labeled: (1) CRU-FAO-MIRCA, (2) CRU-FAO-Variable, (3) CRU-AfSIS-MIRCA, (4) CRU-AfSIS-Variable, (5) WFD-FAO-MIRCA, (6) WFD-FAO-Variable, (7) WFD-AfSIS-MIRCA and (8) WFD-AfSIS-Variable. Using this setup we can assess both, input and model structure uncertainty.

Model settings
(1) Prior to the simulation we have to set a management intensity factor (1)-(7) as it is an unknown Table 1. Basic model components and processes, partly as described in Wang et al (2002), Tubiello and Ewert (2002) and Nair et al (2012  AfSIS; multiple, grid-cell specific soils from African soil database (Leenaars 2012) Variable; multiple, grid-cell specific sowing date from a climatic rule based on rainfall (Dodd and Jolliffe 2001) parameter in the LPJmL model. In LPJmL the management intensity factor i.e. the degree and frequency of crop production control and input application (fertilizer, technology, labor, weed and diseases control etc) is represented by three parameters: the maximal attainable leaf area index LAImax, the harvest index HImax, and the parameter α-a, scaling leaf-level biomass to field level. Comparing the factor to industrial fertilizer application rates (N, K 2 O, P 2 O 5 ) showed that there is a positive relationship especially with N but with some exceptions as manure application strongly determines the intensity level in many countries (Fader et al 2010). We set the management intensity factor in LPJmL to 2 which is the default value for Burkina Faso. The simulated mean national maize yields in 2000 (1.35-1.68 t ha −1 ) are in a similar range but slightly lower than FAO statistics in the year 2000 and the detrended maize yields in 1996-2000 (1.75 t ha −1 ). The model is initialized with this parameter which is kept constant over the simulation period and for different input data.
(2) For APSIM simulations, the nitrogen fertilizer application rate (kgN ha −1 ) is required. We set it to 5 kg N ha −1 according to an estimate from FAO for the N fertilizer application rate per hectare arable land in Burkina Faso (FAO 2013). The simulated maize yields in 2000 (0.53-1.64 t ha −1 ) are slightly lower than the FAO statistics but mostly also in a similar range. (3) The objective of the crop model simulations is to assess the influence of input data on the simulated crop yields and production and to understand the propagation of uncertainty in input data to uncertainty in simulation results but not to adjust the models to perfectly simulate observed yield levels. We use standard model parameters or available data from literature or data bases instead of calibrated parameters and therefore accept over-or underestimation of absolute yield levels. We later discuss how absolute yield levels and trends in yield over time and space are simulated compared to observations.

Metrics used to describe differences between yields
The level of agreement in simulated and observed grid-cell yields for one year indicates how well the spatial variability in crop yields can be simulated over a country. The level of agreement in simulated and observed mean national yields over time indicates how well the temporal variability in yields can be simulated. The agreement is measured by comparing the standard deviations in simulated and observed grid-cell yields in one year and in mean national yields over time as well as correlation coefficients and root mean square errors between simulated and observed yields. All three statistics are related and can therefore be represented by one point simultaneously in the twodimensional Taylor where R is the correlation coefficient, rms is the centered root mean square error and σ y and σ o the standard deviations.
where again R is the correlation coefficient, rms is the centered root mean square error and σ y and σ o the standard deviations. y c n , is the simulated nationalmean maize yield in t ha −1 in year n with N = 40 years, o c n , is the observed national-mean maize yield in t ha −1 in year n (see supplementary material S2). The simulated national-mean yield y c n , in one year is calculated as the crop area-weighted mean yield from grid-cell yield y n and crop area in grid cell n and year 2000 a n (see supplementary material S2):  The correlation in APSIM simulations with uniform sowing dates and monthly climate is slightly larger than in APSIM simulations with uniform sowing dates and daily climate, however also the standard deviation is larger (>0.70 t ha −1 ). (3) APSIM simulations with daily climate and uniform sowing dates are closest to the observations. In LPJmL the differences between these four simulations are smaller and simulated standard deviations are closer to the observed standard deviation in simulations with monthly climate. LPJmL simulations with monthly climate and uniform sowing dates are closest to the observations. (4) The observed spatial variation in maize yields is overestimated in all APSIM simulations, except the simulations with daily climate and variable sowing dates and underestimated in all LPJmL simulations (figures 3 and 4). (5) Some triangles/ points are grouped together in particular points representing LPJmL simulations. These are simulations with the same crop model, climate and sowing dates but different soil input data. They have similar statistics and therefore similar skill to reproduce the observed spatial variability. Soil data is less important for the skill of the two crop models to reproduce the observed spatial variability even though the spatial variation in the two soil data sets differs to a larger extent than the spatial variation in the two climate data sets and is of similar magnitude as in the two sowing date settings (figure 3(B) and table S5). The difference between crop models is larger than between input data settings and APSIM is more sensitive to input data, in particular to soil data than LPJmL. (6) The difference between APSIM and LPJmL simulated maize yields and between yields from the different climate data sets can be partly explained by different soil processes in the models in particular NO 3 leaching. NO 3 leaching out of the root zone is simulated in APSIM as a function of rainfall and soil texture. Crop yields in APSIM increase with rainfall but drop after about 600 mm in-crop rainfall whereas crop yields in LPJmL increase linearly with increasing in-crop rainfall ( figure 5, left). This is because the soil texture type in APSIM determines not only the water holding capacity but also the soil fertility level which decreases with increased NO 3 leaching rates (figure 5, right) whereas in LPJmL soils only differ in their water holding capacities. Therefore maize yields simulated for different soil texture types differ considerably in APSIM leading to a higher spatial variability in yields while soil texture has less impact on yields in LPJmL. Rainfall and thus NO 3 leaching in simulations with daily (WFD) climate data is higher than with monthly (CRU) climate data. (7) Mean NO 3 leaching rates in APSIM from all simulation years in individual grid cells are between 0 and 2.9 kg N ha −1 with a maximum of 17.1 kg N ha −1 in clay, clay-loam and loam soils and 0 and 7.8 kg N ha −1 with a maximum of 2 kg N ha −1 in sandy-clay-loam, sandy-loam and loamy-sand soils. Similar N losses from leaching are simulated with EPIC-maize for Benin (Ramarohetra et al 2013) and summarized by Smaling et al (1993) from several studies. Depending on the clay content, rainfall and N application type 18%-53% N leaching is reported while mean N leaching relative to available N in this study is between 0.4% and 21%.

Effect of soil, climate and sowing dates on temporal variability in maize yields
(1) The agreement in simulated and observed temporal variability is smaller than in spatial variability indicated by smaller correlation coefficients for most of the simulations (figure 6). The two LPJmL simulations with monthly climate and uniform sowing dates have a correlation coefficient of more than 0.25 and are significantly correlated with observations at the 0.05 level (N = 40). The agreement with observations improves in 8 out of 12 simulations displayed in figure 6 if only the time period 1961-1987 is analyzed. This is because national mean maize yields doubled in the 5 years period from 1987 to 1991 which is not simulated from the crop models (figure 7). The doubling of maize yields is most likely a result from an increase of research expenditures in this time period in Burkina Faso (Stads and Kaboré 2010). This is related to a World Bank funded National Agricultural Research Project which amounted to a five billion CFA loan (∼ten million US $) for the period 1989-1994 (Mazzucato 1994 1961-2000 and 0.65 in 1961-1987 (similar for WFD annual rainfall) so higher than the correlation of any modeled yields with observations. (4) For estimating the long-term mean of national maize yields information on sowing dates becomes less important. The 40 year national mean maize yields simulated from LPJmL vary between 1.24 and 1.61 t ha −1 and between 0.86 and 1.55 t ha −1 as simulated from APSIM (see supplementary material S4). Yields simulated with LPJmL, monthly climate and global soil information and simulated with APSIM, monthly climate and local soil information are closest to statistics.
Propagation of uncertainty and importance of input and model uncertainty (1) The importance of sowing dates for simulating mean national maize yields has been shown before for Burkina Faso from Waongo et al (2014). The study found a deviation of potential maize yields in 2000-2010 between −10% and +60% simulated with the crop model GLAM with sowing dates varying between −20 and +12 days. In this study similar deviations in maize yields in 2000 of −32% and +47% result from variations in sowing dates between −27 and +10 days for e.g. LPJmL simulations with CRU climate and global soil information.
(2) Depending on the model and input setting Burkina Faso's maize production in e.g. 1999 of about 469 000 tones is underestimated by 1-17% in LPJmL simulations. Maize production in 1999 is underestimated in most APSIM simulations by 3-46% but overestimated by 15% in APSIM simulations with daily climate, local   soil information and variable sowing dates. The same simulation with global soil information result in exactly the same maize production as observed. (3) Similarly to Baron et al (2005) and Ramarohetra et al (2013) for simulation studies in Benin and Niger we found that uncertainty in weather data can introduce large biases in simulated crop yields and production.
(4) However model uncertainty can be larger than input uncertainty. Similarly Rosenzweig et al (2013) found that including ecosystem-based models like LPJmL increases the range of uncertainty in model intercomparison studies when simulating climate change impacts on crop yields. This is for a different reason than in our study namely because of different approaches to model effects of elevated atmospheric CO 2 concentrations on crop yields in site-based and ecosystem-based crop models. In a crop model comparison study for wheat models at four locations with temperate climate Asseng et al (2013) also concluded that model uncertainty is more important than uncertainty from different climate projections.

Summary and conclusions
(1) We found that the level of information of different soil, climate and management data sets influences the simulated crop yields in both models. The uncertainty in input data propagates to uncertainty in simulated maize yields and production in which, for the present modeling setup and study area information on soil parameters is less important than information on sowing dates and climate. However, the difference between models can be larger than between input data in particular when assessing the spatial variability of crop yields.
(2) Further, the agreement between simulated and observed spatial variability is higher than between simulated and observed temporal variability due to abrupt changes in national mean yields from 1987 to 1991 in Burkina Faso which cannot be explained by rainfall variability like in the previous decades and therefore cannot be simulated from the two crop models used. The most accurate estimation of spatial variability in maize yields with APSIM is possible with daily climate information and uniform sowing dates i.e. with detailed information on climate data but little information on sowing dates (R = 0.65).
In contrast the most accurate estimation of spatial variability in maize yields with LPJmL is possible with monthly climate information and uniform sowing dates i.e. with little information on both, climate and sowing dates (R = 0.80). APSIM and LPJmL tend to overestimate and underestimate, respectively the spatial and temporal variability of maize yields.
(3) Soil data that determines water holding capacities is less important for the skill of the two crop models to reproduce the observed spatial variability even though the spatial variation in the two soil data sets differs to a larger extent than the spatial variation in the two climate data sets and is of similar magnitude as in the two sowing date settings. However soil fertility levels and soil processes in the crop models such as NO 3 leaching from the root zone are important and partly explain the deviations between both models and between simulations with monthly and climate data.
(4) Our results and conclusions are valid for the lowinput agricultural systems in Burkina Faso and other parts of West Africa with low yield levels compared to other world regions and they depend on the limitations to crop growth specific to this study area. We expect changes in spatial and temporal variability with increasing yield levels which might lead to different conclusions on the ability of the two crop models to simulate observed yield levels. (5) However the findings of our study highlight the importance of scale, model choice and aggregation level and show that using the most detailed input data or crop model does not necessarily increase the agreement between simulations and observations. Therefore we suggest that not only the resolution of input data but rather its appropriateness regarding the study area, the model used and the aggregation level of the model output should be assessed before using it. Both models are able to represent the national mean maize production with an appropriate set of input data. Our findings inform about the magnitude of uncertainty in simulated maize yields and production arising from different input data and crop models which will assist interpretation of results in future modeling studies in West Africa. (6) Small scale crop models such as APSIM respond to little changes in input data and simulate much more crop and soil processes than large scale models such as LPJmL leading to higher variation in simulated crop yields (temporal and spatial). This strength gives them a potential weakness because uncertainty in input data will affect model outcomes much more than from large scale models. We suggest that a careful analysis of drivers of changes in historic crop yield patterns should be compulsory and precede the choice of input data and model, simulation settings, and model projections. For developing both types of models, modelers can build on detailed knowledge and dynamics in small scale models and together think about smart ways of incorporating them in a simple way into large scale models.