Evaluation of WaPOR V2.0 evapotranspiration products across Africa

,

and the NASA/EOS MODIS evapotranspiration team for the MOD16 global evapotranspiration product.We would also like to thank Karin M. Viergever for being our key correspondence with the data provider.

Introduction
Actual evapotranspiration (ETa) is the second-largest process and flow in the terrestrial water budget after precipitation (PCP).ETa is also an essential component of plant growth and, therefore, the carbon cycle.Available water resources are becoming, or are already scarce, in many basins worldwide (Degefu et al. , 2018).The acceleration of the water cycle from a climate change perspective will further influence water availability not only for human consumption but also our food sources (Rockström, Falkenmark, Lannerstad, & Karlberg, 2012).For this purpose, accurate estimates of ETa are required for several management tasks, including, but not limited to, water accounting, water footprint, basin-wide water balances, irrigation, crop management and monitoring of climate change and its impact on crop production.These activities require ETa at varying extents and spatio-temporal resolutions.
Validation of these remote sensing products is an essential step in understanding their applicability.Validation is essential to understand and characterise uncertainty.This uncertainty can guide if the ETa product is suitable as input into different water management activities along with the associated risk when making a decision based on the product.Many studies exist that attempt to validate large remote sensing-based ETa datasets.Most studies are focused on one or two validation methods at one scale.The most common validation methods are either point or pixel scale against ground-truth data, like eddy covariance measurements (e.g., Mu, Zhao & Running , 2011), or spatial inter-comparison of a product over regions, land classes, biomes (e.g., Mueller et al. , 2011).Some authors validate multiple products against each other for spatial and temporal patterns and against ground-truth data (e.g., Hu, Jia & Menenti, 2015;Nouriet al. , 2016).Recently, Weerasinghe, Van Griensven, Bastiaanssen, Mul, & Jia, (2019) compared multiple ETa products at the basin scale to the long term water balance utilising other global models on precipitation and run-off while Liu et al. , (2016) evaluation of basin-scale evapotranspiration estimates against the water balance method.However, these validation efforts often fail to evaluate the product at multi-scale, from pixel to basin or region.
The best-practice validation strategies of big remote sensing datasets have been proposed by (Zeng et al., 2019;2015).They recommend multi-stage validation activities that include combinations of direct validation, physical validation and cross-comparisons.In practice, many developers of remote sensing products include all or at least a combination of these activities during their validation.To name a few, these include the MODIS MODLAND product (Morisette, Privette, & Justice, 2002;Morisette, Privette, Justice, & Running, 1998); Copernicus Global Land Service products Dry Matter Productivity (Swinnen, Van Hoolst, & Toté, 2015); and ASTER land surface temperature (Schneider, Ghent, Prata, Corlett, & Remedios, 2012).
In regions such as Africa, where little observational data is available, validation should utilise all available avenues for ascertaining product quality, with a multi-step and -phase validation strategy that includes direct validation (with ground measurements), physical consistency check and cross-comparisons.As such, the limitations due to the sparseness of available data are reduced, and the product quality is understood from a multi-scale perspective, by using validation best-practice and combining multiple validation techniques.
The latest available database of continental products, released in 2019, for Africa and the Middle East, is now available on The Food and Agricultural Organization (FAO) portal to monitor Water Productivity through Open access of Remotely sensed derived data (WaPOR) (https://wapor.apps.fao.org/home/WAPOR_2/2).It provides the highest available spatial resolution for an operational open-access actual evapotranspiration and interception (ETIa-WPR) product at the continental scale.This paper presents a multi-scale validation of the version 2 (V2.0)ETIa-WPR.The results from each validation procedure were analysed individually and then as a whole to determine trends and draw conclusions of the product quality.

The dataset
The analysis dataset is the ETIa-WPR V2.0 products available on the WaPOR portal.The ETIa-WPR is based on a modified version of the ETLook model described in Bastiaanssen, Cheema, Immerzeel, Miltenburg & Pelgrum (2012).The ETLook model uses Penman-Monteith (PM) to estimate ETa adapted to remote sensing input data (FAO, 2018(FAO, , 2020a)).The PM approach uses the combined approaches of the energy balance equation and the aerodynamic equation and is described in the FAO-56 drainage paper (Allen, Pereira, Raes & Smith, 1998).The ETIa-WPR defines soil evaporation and transpiration separately using Equation 1and Equation 2. The interception is a function of the vegetation cover, leaf area index (LAI) and precipitation (PCP).The ETI-WaPOR is then calculated as the sum of evaporation, transpiration and interception.Where E and T (mm/day) are the evaporation and transpiration respectively and λ is the latent heat of vaporisation.Rn (MJ/m2/day) of the soil (Rn,soil) and canopy (Rm, canopy) is the net radiation and G (MJ/m2/day) is the ground heat flux.ρ air (kg/m3) is the density of air, C P (MJ/kg/°C) is the specific heat of air,( e sat − e a ) (kPa) is the vapour pressure deficit (VPD), r a (s/m) is the aerodynamic resistance, r s (s/m) is the soil resistance, or canopy resistance when using the PM-model to estimate evaporation or transpiration respectively.Δ =d (e sat )/d T (kPa/°C) is the slope of the curve relating saturated water vapour pressure to the air temperature, and γ is the psychometric constant (kPa/°C).This approach partitions the ETIa-WPR to evaporation and transpiration using the modified versions of PM, which differentiate the net available radiation and resistance formulas based on the vegetation cover according to the ETLook model (Bastiaanssen et al., 2012).A major difference between the WAPOR model and the ETLook model is the source of remote sensing data for the soil moisture.In the original ETLook soil moisture is derived from passive microwave, and in the WAPOR approach soil moisture is derived from Land Surface Temperature (LST).The WaPOR database provides ETIa-WPR in three spatial resolutions dependent on the location and extent.The products available specifically for Africa are shown in Table 1.
Datasets (including intermediate datasets) available for the validation include soil moisture content (SMC), normalised difference vegetation index (NDVI), solar radiation (SR), NDVI quality layer, land surface temperature (LST) quality layer, PCP and reference evapotranspiration (RET) (Table 2).The producer provided the SMC and NDVI layers for the validation.All other layers are available on the WaPOR portal.The NDVI quality layer and the LST quality layer are indicators of the quality of the input satellite data.The NDVI quality layer provides the gap, in days, to the nearest valid observation for that variable.The LST quality layer provides the number of the days between the date of the data file and the earlier remote sensing observation on which the data is based.
WaPOR further relies on input from weather data, air temperature, relative humidity wind speed, which are obtained from MERRA up to the start of 21-02-2014 and GEOS-5 after 21-02-2014 (Rienecker et al. , 2011).The weather data is resampled using a bilinear interpolation method to the 250m resolution.The temperature is also resampled based on elevation data (FAO, 2018).

Validation approach and workflow
The validation approach comprises three components, physical validation, direct validation and level consistency (Figure 1).The physical validation and direct validation were undertaken on the L1 product for the period 2009-2018.The physical validation (section 2.3) includes an assessment of the water balance and water availability (2.3.1) and a spatial and temporal consistency check (2.3.2) for the extent of Africa.The water balance utilises other existing continental datasets to complete the water balance and is therefore also considered cross-validation.The spatial and temporal consistency checks if spatial and temporal patterns were being captured.The direct validation (section 2.4) involves a comparison to ETa estimations from EC stations.The level consistency (section 2.5) checks for the consistency between levels and therefore indicates if the quality of the L1 product is representative of the L2 and L3 products.

Water balance and water availability
The basin-scale performance of ETIa-WPR is analysed for 22 major hydrological basins of Africa (Lehner & Grill, 2013) through three approaches (Figure 2).First, the ETIa-WPR was compared to the PCP on an annual basis to analyse the water consumed through ETIa to the water available from PCP.
Second, the basin-scale water balance approach compared the long term ETIa-WPR product to the long term ETa derived from the water balance (ETa-WB).In many studies, the long term water balance (>1 year) for large basins assume a negligible change in storage (Hobbins, Ramírez, & Brown, 2001;Wang & Alimohammadi, 2012;Zhang et al. , 2012).The long term water balance, taken from 2009-2018 in this case, is therefore defined using equation 2.
1. ETa-WB (mm/yr) = PCP (m/yr) -Q (mm/yr) Where PCP is the long term precipitation and Q is the long term basin run-off or streamflow, and the ETa-WB is the long term ETa derived from the water balance.The PCP product found in the WaPOR portal was obtained from the Climate Hazards Group Infrared Precipitation with Stations (CHIRPS) dataset (Funk et al. , 2015).The long term Q was obtained from the Global Streamflow Characteristics Dataset (GSCD) (Beck, De Roo, Van Dijk, 2015).The GSCD consists of global streamflow maps, including percentile and mean Q, providing information about runoff behaviour for the entire land surface including ungauged regions.
Third, the ETIa-WPR and PCP annual values were compared to the ETa from MODIS Global Evapotranspiration Project (ETa-MOD16) for the period 2000-2013 (Mu Heinsch, Zhao, & Running, 2007;Mu, Zhao, & Running, 2013) and to values from the literature for basins where data is available.The ETa-MOD16 product is also based on the PM equation and considers the surface energy partitioning process and environmental constraints on ETa.The algorithm uses both ground-based meteorological observations and remote sensing observations from MODIS.Basins were excluded in the ETa-MOD16 comparison missing data on an annual level exceeded 20%.

Spatial and temporal consistency
The temporal and spatial trends were observed over the African continent in space and time by observing mean ETIa-WPR, SMC and NDVI for all climate zones during the study period on a dekadal basis.The Koppen-Geiger classification (Figure 2) is used to consider the mean dekadal values for the main climatic zones in Africa (Kottek, Grieser, Beck, Rudolf, & Rubel, 2006).A sample size of 30,000 stratified random pixels is used to represent the continental.This corresponds to less than 0.01% of the total image, however, is considered suitable to represent seasonal trends for the major climate zones.The arid or desert class -B -dominates Africa (57.2%), followed by the tropical class -A (31%) and then warm temperate -C (11.8%).
The largest sample count corresponds to the largest climatic zones, with a linear 1:1 line representing area to count.The data is further disaggregated based on the northern and southern hemispheres to account for opposite seasonal patterns.

Direct validation
The ETIa-WPR is compared to the in-situ ETa from eddy covariance (EC) fluxes (ETa-EC) at a dekadal scale using 14 locations (13 across Africa and 1 in the Spain extension area) (Figure 2).The country, station code, vegetation, climate zones and available data for comparison -for both WaPOR and the local site, are shown in Table 3.The majority of EC sites are in shrubland or savannas.Egypt stations (EG), the NG-WAM station and GH-ANK station which are located in an irrigated area, agricultural land and forested areas respectively.Center, Giza, Egypt in the Nile Delta.These irrigated sites in the Nile Delta, were under rotation with three major summer crops -rice, maize and cotton -and four major winter crops -wheat, berseem, fava beans and sugar beet.
ETIa-WPR for L1 (250m) were spatially averaged over a 3x3 pixel window surrounding the EC station, based on the assumption that the window represents the measurement footprint of the EC station.The ETa-EC data was derived from LE flux and then aggregated temporally to dekadal averages to match the temporal resolution of the ETIa-WPR products.Intermediate products, including WaPOR NDVI, SMC and the NDVI and LST quality layers were analysed along with the ETa trends to identify possible sources of error.Reworking the LE flux data to daily values was done (accounting for NaN, non-removed spikes, early morning (dawn) and evening (day-night inversions), dew spiking, etc.) which are not necessarily removed by the standard Eddy Covariance pre-processing software's (converting very high frequency sonic 30-sec and gas analyzer measurements to 30-minute interval fluxes).

Level consistency
L3 and L2 ETIa-WPR were compared to the L1 data for the period of 2009-2018 on a dekadal basis.A bilinear resampling method was used to spatially aggregate the high-resolution L3 and L2 layers to the resolution of the coarse L1 layer.A random stratified sample of 30,000 points over the entire L2 extent is used for the comparison of the L1 and L2.The L1 and L3 were compared over the entire L3 extent of the Awash, Zankalon, ODN and Koga L3 irrigation areas for all pixels.Table 4 shows the description of each L3 irrigated area.The EC station at Zankalon is located in a L3 area.Therefore, as part of the level consistency, all three levels were also compared to the ETa-EC at this station.The method described in section 2.4 was used to extract the L3 and L3 ETIa-WPR at the station.

Water balance and water availability
The annual ETIa-WPR divided by the annual PCP (ETIa/PCP) during 2009-2018 for Africa is shown in Figure 3.The annual ETIa-WPR exceeds the annual PCP (ETIa/PCP >1) on 55% occasions for all basins over the ten years study period.The highest number of exceedances occur in 2014 and 2016 (64%), and the lowest number of exceedances occur in 2018 (27%).The majority of these exceedances, 66%, are by less than 10%.The average ETIa-WPR to PCP ratio for the continent of Africa is 0.93.The lowest ratio is in 2010, 0.87, and the highest is in 2015, 0.97.These ratios are significantly higher than the suggested average, 0.65, of evapotranspiration to precipitation ratio over the global terrestrial surfaces (McDonald, 1961).This ratio is expected to be lower in dry regions or parts of the continent .Except for Lake Chad Basin, basins in the Central, North and West of Africa have ETIa-WPR less than PCP.Most of the exceedances (ETIa>PCP) occur in the South of Africa and on the Horn of Africa.
The basins have the highest ETIa-WPR/PCP ratio in 2015, particularly in Southern Africa.All basins south of Zambezi Basin show a significant decrease in PCP from 2014 to 2015, including a 246, 98 and 238 mm/year drop in Limpopo, Orange and the South Interior respectively.In the same timeframe, the largest ETIa-WPR change is in Limpopo, with a 17mm/year increase, followed by the South Atlantic Coast with a 35mm/year decrease.The decrease in PCP is due to the drought in this region during this period as a result of the El Nino climatic event (USAID, 2016).However, ETIa-WPR does not seem to respond appropriately to these extreme drops in PCP.
The average (av.), minimum (min) and maximum (max) annual ETIa-WPR and PCP values for the 2009-2018 period are shown in Table 5.Where literature values were available, annual estimates of ETIa-WPR and PCP are compared with historical estimates on annual ETa and PCP, with ETa from MODIS Global Evapotranspiration Project (ETa-MOD16) and with the ETa-WB.In most cases, the ETIa-WPR is larger than the ETa values in literature, from the water balance and from MOD16.The PCP falls within the range of literature for all but three basins.The average PCP in the database is higher than that in literature for the Congo.The PCP is less than that found in literature in the Limpopo and Orange Basin, which is also likely due to the drought in this region which occurred after the estimates as reported in the literature.
It is also important to note that the Congo River Basin, Central West Coast and west coast basins have vast areas of low-quality NDVI and LST layers for much of the year.They are making the annual mean ETIa-WPR values derived from remote sensing much less reliable in these basins.
The ETIa-WPR and ETa-MOD16 are plotted against the ETa-WB in Figure 4.The relationship between both the ETIa-WPR and ETa-MOD16 products show strong linear relationships with ETa-WB.While the ETa-WPR product has a better R2, the ETa-MOD16 has a lower bias.The ETIa-WPR shows a slightly positive bias, which is increasing with increasing ETa-WB.The absolute difference between the ETIa-WPR and the ETa-WB is typically increasing with increasing ETa-WB.The relative differences between ETIa-WPR and ETa-WB are lower at high ETa values.The absolute difference and relative difference between ETIa-WPR and ETa-MOD16 were greater at lower ETa-MOD16.The absolute relative difference, between ETIa-WPR and ETa-WB typically decreased with increasing PCP.The long term ETIa-WPR is larger than the ETa-WB on 13 out of 22 basins.The Q represented from 4.4% (South Interior) up to 47.0% (Central West Coast), with a median of 18.6%, of the long term PCP.The Q is greater in basins with greater ETIa-WPR and PCP.In basins where the long term average Q is less than 150mm/year (18 basins), the relative difference between ETa estimates ranged from -20% to +70%.When the long term average Q is greater than 200mm/year the relative difference ranged from -12% to +20%.

Spatial and temporal consistency
The mean ETIa-WPR, SMC and NDVI were plotted for all climate zones for the northern and southern hemisphere.Figure 6 shows some examples of the largest sub-zones per main climate; wet tropical-savanna (Aw), arid-desert-hot (Bwh) and temperate-dry winter-warm summer (Cwb).The average ETIa-WPR (yaxis on the left), and SMC and NDVI (y-axis on the right) are reported from dekad 0901 (2009 -dekad 1) to 1836 (2018 -dekad 36).
The temporal trend for each climate zone is inversed between hemispheres, reflecting the opposite seasons between hemispheres.For example, peak ETIa-WPR values occur around dekad 19 and trough values occur around dekad 01 in the northern hemisphere.Conversely, in the southern hemisphere, peak ETIa-WPR values occur around dekad 01 and trough values occur around dekad 19.The inverse pattern highlights the need to separate climate zones based on hemisphere, as these trends would otherwise cancel out and flatten out temporal trends.
The Aw zones are maintaining the highest ETIa-WPR values and shows the lowest relative variability throughout the year.The BWh zones consistently have lower ETIa-WPR values.The BWh in the southern hemisphere is higher than in the northern hemisphere, and the relative intra-annual variation is greater.
The ETIa-WPR in these zones follows a clear seasonal pattern, that is not evident from the NDVI or the SMC.The ETIa-WPR is predominantly governed by evaporation in these arid zones, which is indicated by the low NDVI all-year-round.The temperate zone, Cwb, shows the greatest intra-annual variability in ETIa-WPR, which reflects the more dramatic climatic seasonal variation in these years.ETIa-WPR in Cwb in the northern hemisphere shows two peaks per year.The two seasons are consistent with the zones' location in the Rift Valley of Eastern Africa.The Rift Valley experiences two wet seasons as influenced by the intertropical convergence zone (Hills, 1978) and the longer wet season.
ETa is either controlled by available energy or available water.All zones, other than BWh and Aw in the northern hemisphere, show a clear relationship between the ETIa-WPR and the NDVI and SMC.The Aw zone in the southern hemisphere, shows two ETIa-WPR peaks a year in the northern hemisphere, while, SMC and NDVI show one.Therefore it is related to net radiation.Although not shown here -ETIa-WPR in BWh in the northern hemisphere follows the same seasonal trend as radiation.In the Aw zone in the northern hemisphere, the net radiation peaks several dekads before the NDVI and SMC, resulting in a double-peaked ETIa-WPR.The ETIa-WPR in BWh zone shows a clear seasonal trend, despite no clear seasonal NDVI or SMC trend.Therefore it is governed by the amount of solar radiation which has a clear yearly trend at the latitudes within the BWh zone.

Direct validation
The agreement between ETIa-WPR and ETa-EC is shown in Figure 7 and Table 6. Figure 7 shows the time series of ETIa-WPR and ETa-EC for all available in-situ data from all EC stations.Table 6 shows the corresponding metrics for each station, including r, RMSE, bias, the R2 and the average NDVI and LST quality for the comparison period.A good overall correlation (r=0.75) is found between all sites and observations.However, substantial variations existed between sites.Consistency in results is seen between years for most sites.The ETIa-WPR typically captured seasonality well at most sites.
The best-performing sites are SN-DHR and SD-DEM.The SN-DHR and SD-DEM sites are characterised by arid or semi-arid climates and short vegetation.The ETIa-WPR closely follows the ETa-EC at the SN-DHR and SD-DEM site, and both respond quickly to rainfall events.At these sites, the WaPOR SMC and NDVI are well related to both the ETa-EC and ETIa-WPR.For example, the R2 for the SMC or NDVI and ETa-EC or ETIa-WPR ranges between 0.82-0.87 at SN-DHR and 0.69-0.86 at SD-DEM.SD-DEM does overestimate ETIa-WPR when ETa-EC is low and NDVI is low.
ETIa-WPR is also performing well at ES-SCL, ZM-MON, CG-TCH, EG-ZAN, EG-SAA, EG-SAB and SA-SKU.Excluding CG-TCH, these sites have high-quality LST and NDVI layers (the average LST quality for the comparison period is equal to or less than 1).The good performance at this site may be because the variation in CG-TCH station ETa-EC and ETIa-WPR is strongly related to the VDP derived from the EC station and RET, with R2=0.62 and 0.66 respectively.The VDP and RET are derived from GEOS-5 (VDP and RET) and MSG (RET only), as compared to being derived from satellite images.GEOS-5 and MSG are available daily and satellite image gaps do not influence the quality of the VDP and RET quality.
The ETIa-WPR frequently overestimates ETa-EC show good correlations and R2 between ETa-EC and ETIa-WPR at the irrigated agriculture sites, EG-ZAN, EG-SAA and EG-SAB.However, the ETIa-WPR is systematically larger than the ETa-EC during both high and low ETa-EC, as indicated by the average daily bias (Table 6).The seasonal values ETIa-WPR and ETa-EC for the summer maize 2012 crop at EG-ZAN are 682 mm and 424 mm, respectively.Compared to ETa from a lysimeter (ETa-lys), 543mm, as cited in literature (Atta et al. , 2015), at EG-ZAN for the same crop and period.It, therefore, suggests that the ETa at the irrigated sites fall somewhere between the ETa-EC and L1 ETIa-WPR.The overestimation is likely directly related to the net radiation difference between the EC and WaPOR datasets as inferred from the RET estimated from the EC data and compared to the WaPOR RET.The WaPOR RET has a high linear agreement with the EC RET (R2=0.93).However, the bias of WaPOR RET is consistently 50% greater than the EC RET.
ETIa-WPR and ETa-EC show a weak correlation at NE-WAF and NE-WAM.The ETIa-WPR begins increasing earlier in the season, particularly at NE-WAM, and although the ETIa-WPR is capturing the seasonal trend, it is not capturing the magnitude of the ETa-EC summer values.The difference is likely related to the low-quality NDVI and LST layers during the summer (average annual values LST and NDVI gaps appear low in Table 6, however major gaps are concentrated in the summer season).These sites are not highly correlated with the site VDP or RET and therefore the lower quality LST and NDVI is expected to have a great impact on the quality of ETIa-WPR here.The ETIa-WPR is strongly related to the SMC at these sites (e.g.R2=0.73 at NE-WAM); however, the ETa-EC shows no relationship with the WaPOR SMC (R2=0.37 at NE-WAM).Both of these sites are dominated by evaporation (in WaPOR) for most of the year -as indicated by low NDVI all year.
The ETIa-WPR performance at BN-NAL is not capturing the site seasonality well.BN-NAL ETIa-WPR and ETa-EC show annual values ranging from 1.4-4.5mm/dayand 0.6-6.9mm/dayrespectively.The ETIa-WPR at BN-NAL does not appear to capture the rainy period in July-September where the highest gaps in the NDVI exist (low NDVI quality).At this site, the WaPOR SMC and NDVI layers have a stronger relationship with the ETa-EC than the ETIa-WPR.For example, the R2 between the WaPOR NDVI and the ETa-EC and the WaPOR NDVI and the ETIa-WPR are 0.87 and 0.56 respectively.This is, therefore, pointing to an overestimation of the evaporation component when NDVI is low and an underestimation of the transpiration component when the transpiration is high.
The ETIa-WPR has the lowest performance at the GH-ANK and KWSTI in terms of both the regression and the temporal trends.The GH-ANK site is characterised by a tropical climate and high vegetation height (evergreen forest).Further, the ETa-EC is not strongly related to the VDP or the RET.The VDP at this site ranges from 0.07-0.81with high relative humidity.The KWSTI site is located in the Rift Valley, between the Aberdares Ranges to the east and the Mau escarpment to the west.This setting creates a complex micro-climate with significant diurnal variation in temperature and wind speed, among other meteorological variables.This site has an inferior NDVI quality layer and a very low correlation with VDP.As a result, errors in the input meteorological data may highly influence ETa-EC estimates at the site.
The results improve slightly for all sites on a monthly scale.The Monthly mean daily ETIa-WPR plotted against monthly mean daily ETa-EC is shown in Figure 8.The R2 metric improves the most.The RMSE improves at all stations except EG-SAA, where the RMSE increases by 63%.The correlation and R2 improved slightly at all stations.The correlation and R2 increase on average, across stations -not weight, by 9% and 8% respectively.The absolute bias increases slightly at 5 of the 14 stations.

Level consistency
The consistency between the evaporation and transpiration data products for the L1 and L2 data products is high.The ETIa-WPR RMSE, between L1 and L2, for each dekad for the 2009-2018 period ranged from 0.01 to 0.11mm/day with a median of 0.03mm/day, while the correlation ranged from 0.95 to 1.00 with a median of 0.98.The median R2 over the period is 0.96 while the median bias is 7%.The consistency between layers dropped slightly after 2014.In 2014 the PROBA-V was introduced for L2, as compared to resampling of MODIS to 100m before 2014.The median correlation dropped from 1.0 to 0.96, and the median RMSE increased from 0.01 mm/day to 0.04 mm/day.A slight positive systematic bias, in favour of L2, is evident after 2013, with median bias increased from 4% to 9%.
The L1 and L3 ETIa-WPR products have a lower consistency as compared to the L1 and L2 products in the four irrigation areas.The mean ETIa-WPR values for all dekads in the Zankalon and Awash schemes are shown in Figure 9.The Awash area has the highest consistency of all scheme areas, reflected in the highest correlation, R2.The ETIa-WPR RMSE between L1 and L3 in the Wonji ranges from 0.42-1.01mm/day,while the correlation ranges from 0.63-0.92.The median correlation for all dekads in the study period is 0.84, and the median R2 is 0.84.The RMSE is highest when the ETIa-WPR is highest.The RMSE temporal trend is in line with the seasonal trend in the Awash and displays the two seasons associated with the intertropical convergence zone.The correlation is above 0.73 on 95% of dekads, and lowest on dekads when the mean ETIa-WPR is highest.
The Koga has the lowest consistency of the schemes.Although the RMSE between L1 and L3 is lower, ranging from 0.26-0.71mm/day, the median correlation is 0.67, and the median R2 is 0.45.Zankalon performed slightly better, with a median correlation of 0.71 and a median R2 of 0.51.The RMSE is higher in Zankalon than the Koga, but this reflects the higher ETIa-WPR values found in the area.The ODN had the same RMSE (0.64mm/day) as Zankalon and the highest range of RMSE (0.15-1.62mm/day).The correlation and R2 are also similar, with median values of 0.73 and 0.53 respectively.All schemes show similar per cent bias medians (9-12%).The only scheme that shows a systematic bias is ZAN, where the L1 is consistently higher ETIa-WPR values than L3.
The 10-daily average ETa-EC and ETIa-WPR for all three spatial resolutions at EG-ZAN are shown in Figure 10.The L1 and L2 ETIa-WPR show high consistency with each other.The L3 ETIa-WPR is consistently sitting between the ETa-EC and the L1 and L2 ETIa-WPR.All levels capture the overall ETa-EC seasonal trends.The L3 data shows a slightly lower R2 (L3=0.66 and L1=0.69) and correlation (L3=0.53 and L1=0.68), but a much lower bias (L3=1.06mm/dayand L1=1.68mm/day) and a lower RMSE (L3=0.99mm/dayand L1=2.19mm/day) when compared with ETa-EC.The better R2 and correlation reflect the L1 and L2 ETIa-WPR ability to capture the temporal fluctuations of ETa-EC better than L3 ETIa-WPR.An example of this is at dekad 1117, where L1 and L2 ETIa-WPR capture the ETa-EC dip, whereas L3 ETIa-WPR stays flat.The L3 ETIa-WPR have a better seasonal agreement with the ETa-lys for the summer maize crop in 2012 (L3=487mm, L1=682mm and ETa-lys=543mm).
The NDVI and ETIa-WPR for the 250m buffer are shown in Figure 11 for the three spatial resolutions.The 30m level is picking up more spatial variation (standard deviations: L3=0.05,L2=0.02;L1=0.02) at the site and has a lower mean NDVI for the site as compared to L2 and L1 (mean: L3=0.74;L2=0.82 and L1=0.83).This reflects the lower ETIa value for this dekad, which is more similar to the EC -as seen in Figure 10.

Product accuracy
The ETIa-WPR results are comparable the improved MODIS global terrestrial evapotranspiration algorithm, MAPE of 24.6% as compared to EC measurement, when driven by driven by the tower meteorological data (Mu, Zhao & Running 2011).The ETIa-WPR error estimates, on average, are also close the average errors in EC measurements (20-30%) (Allen, Pereira, Howell, & Jensen, 2011;Blatchford, Mannaerts, Zeng, Nouri, & Karimi, 2019), however, it appears that the ETIa-WPR is regularly overestimating ETIa, which is evident at local to basin level.Figure 12 shows the bias and number of observations between ETIa-WPR and ETa-EC for all EC observations disaggregated based on 0.5mm/day ETa-EC increments.The results are further defined based on non-irrigated sites, irrigated agriculture and all stations.For non-irrigated sites, there is a positive bias (ETIa-WPR>ETa-EC) when the ETa-EC is less than 2.5mm/day and becomes negative when the ETa-EC is greater than 2.5mm/day.This bias increases, both positive and negative, as the ETa-EC deviates from 2.5mm/day.The underestimation is further exacerbated by the fact that ETa-EC estimations can lead to underestimation of the latent energy or ETa-EC by 20% (Wilson et al. , 2002;Glenn et al. , 2007).Underestimation bias is larger than overestimation bias and increases with increasing ETIa-WPR.However, Africa as a continent is dry with long term (2010-2015) average daily ETIa-WPR for the continent being 1.5mm/day.Therefore, the ETIa-WPR frequently overestimates at the annual, basin scale.The irrigated sites (EG-SAA, EG-SAB and EG-ZAN) are overestimated for nearly all ETa-EC.The irrigated sites strongly influenced the overall bias, as these sites have many observation points.When irrigated and non-irrigated results are combined, the changing point where ETIa-WPR is greater than ETa-EC occurs when ETa-EC exceeds 3.5mm/day.

Why is WaPOR overestimating when ETIa is low?
ETIa-WPR is overestimating ETa in dry, hot, water-stressed conditions (e.g., water-limited).The ETIa-WPR estimates for prolonged dry weather and the dry seasons of WaPOR are usually higher than the observed values (flux towers, field).These overestimations are small in terms of absolute values (mm/day) but can lead to overestimation of results in higher annual ETIa-WPR when compared to water mass balance checks of river basins.The overestimation in dry regions is likely to be primarily due to the functioning of the SMC constraint or the too high SMC in dry regions.
The WaPOR SMC is considered, on average, high in arid regions (e.g., Figure 6) and therefore, ETIa-WPR is likely not effectively accounting for soil moisture limitations.The high SMC is resulting in an overestimation of the evaporation component in particular, as NDVI is low and therefore the region is dominated by the evaporation component of ETIa-WPR.Arid regions should be largely regulated by water availability rather than energy.Conversely, under well-water conditions, the PM method is primarily driven by Rn (e.g.energy limited) (Rana & Katerji, 1998).As PM is a linearised approximate solution, problems may occur in extreme conditions and errors in the soil evaporative term (Leca, Parisi, Lacointe, & Saudreu 2011).Majozi et al., (2017b) noted that PM methods need to include a SMC constraint.Though the ETIa-WPR methodology does include a SMC constraint, overestimations in SMC are reducing its functionality.The SMC is estimated using the trapezoidal method (function of NDVI and LST) (FAO, 2018).Where the NDVI is low, the LST component could be the primary contributing factor to SMC errors.
For water-stressed crops, crop resistance errors can attribute to the large error in ETa estimations, while for tall crops, the VPD can have a large influence on the error (Rana & Katerji, 1998).Extreme conditions include when aerodynamic resistance is high, >50m/s (Paw, 1992).High aerodynamic resistance can occur in sparse vegetation, when surface temperature is much greater than air temperature (e.g.water-stressed conditions) and when wind speed is very low (Paw, 1992;Dhungel, Allen, Trezza, & Robison, 2014).Cleverly et al., (2013) and Steduto, Todorovic, Caliandro, & Rubino, (2003) found when the standard aerodynamic resistance values were used the PM method over-and underestimated RET when RET is low and high respectively and suggested the aerodynamic resistance should vary with climatic variables as it is responsive to relative humidity gradients.
It is recommended to further verify the behaviour of the SMC (soil moisture content index).The SMC relative moisture index is derived from land surface temperature and vegetation cover (NDVI) data.Therefore, verification against highest available physically-based satellite soil moisture data (e.g., active microwave sensors onboard Sentinel-1A, Metop, etc.) is advised.It may be helpful to use SMC for transpiration and passive microwave sensors for evaporation.
The main source of error in the ET-WB method is the uncertainty in PCP.Studies on the CHIRPS PCP product shows high correlations, at monthly and regional scales, in Eastern Africa (r = 0.7-0.93)(Dinku et al. , 2018;Gebrechorkos, Hulsmann, & Bernhofer, 2018) and Burkino Faso (r = 0.95) (Dembele and Zwart, 2016) with little to no bias.Muthoni et al., (2018) reported that CHIRPS v2 slightly over-estimated lowintensity rainfall below 100 mm and slightly under-estimated high-intensity rainfall above 100 mm compared in Eastern and Southern Africa.On an annual, basin-scale, the CHIRPS PCP product does not show significant bias, except for in largely ungauged tropical basins (e.g.Congo) (Liu et al. , 2016).Weergeshi et al., (2019) compared terrestrial water storage by Rodell et al. , (2018) and found they represented a maximum of 2.3% of long term basin ETa for basins in Africa.Therefore the large overestimations of ETIa-WPR should not be attributed to the simplified water balance approach.
Why is WaPOR overestimating ETIa in irrigated fields?
ETIa-WPR is overestimating ETa dry, hot, non-water-stressed conditions (e.g., irrigated fields).These errors might lie in the FAO-PM method's and may be associated with local advection effects.Local advection may increase ETa over a water-limited field by up to 30% (De Bruin, Trigo, Bosveld, & Meirink, 2016;Trigo et al. , 2018).There is an underlying assumption of no advection in the RET definition for a reference grass field (Allen et al. , 1998).However, in small fields, under arid conditions with high temperatures, local advection effects may occur when warm, dry air formed over an upwind, adjacent field is advected horizontally over the well-watered fields (De Bruin & Trigo, 2019).This horizontal advection of sensible heat increases the evapotranspiration of water from well-watered areas but will result in the overestimation of evapotranspiration in water-limited fields or areas.The Zankalon irrigated area, where EG-ZAN is located, has small fields, ˜0.2ha (Table 4), as does the EG-SAA and EG-SAB.Therefore these sites may be particularly influenced by this effect as 0.2ha is 3% of an L1 -250m pixel, 20% of an L2 -100m pixel and 200% of an L3 -30m pixel (e.g., see Figure 10).
Why is WaPOR misrepresenting ETIa when ETIa is high in humid conditions?
ETIa-WPR is not representing ETa well in non-water limited conditions with high humidity.The PM method is not suitable for very low VPD (or high humidity) (Paw & Gao, 1988).Further, for tall crops, the VPD can have a considerable influence on the error (Rana & Katerji, 1998).It is not suitable in these conditions because of the linear assumption of saturated vapour pressure and air temperature.Paw, (1992) advised that the use of non-linear equations should be used in extreme conditions to maintain errors of less than 10-15%.
Quality of input data is likely affecting the quality of the ETIa-WPR in these regions.Low-quality data or missing RH data means VPD is calculated from Tmin.In humid climates condensation occurs during the night, which leads to an overestimation of VPD (Allen et al. , 1998), which is found when PM is applied without RH data in humid regions of Ecuador (Cordova, Carrillo-Rojas, Crespo, Wilcox, & Celleri, 2015).
In non-water limited regions, the overestimation of VPD can lead to higher ETa, as it is easier for the flux to occur when there is less moisture in the air.Further, these regions frequently contain low-quality NDVI and LST layers in these regions.This is resulting for example, in overestimation of radiation at GH-ANK skewing results at this location.The NDVI and LST-quality layers are therefore a good indicator of the quality of the ETIa in these regions.

Product consistency
There is very high consistency between L1 and L2 products.The high consistency is partly explained by the SMC component, which is based on MODIS for both L1 and L2.The consistency between the L1 and L3 products is mixed.The Awash and ODN L3 areas show high consistency between L1 and L3.In the Koga, there is a strong positive bias for L1 ETIa-WPR, while the agreement between L1 and L3 in the Koga and in Zankalon is lower.These errors are likely largely attributed to the different input temporal and spatial resolutions available from the satellite platform combined with high spatial and temporal heterogeneity in the area (e.g.Koga and Zankalon have much smaller irrigated fields and higher crop diversity than the Awash and ODN-see Table 4).All levels have a dekadal time-step.However, the satellite revisit period varies; having revisits of 1-day, 2-days and 16 days for MODIS (L1), Proba-V (L2) and Landsat (L3), respectively, with daily meteorological data input.The variation in the revisit period can lead to differences when interpolating images to a dekadal timescale, particularly in rainy periods and during the growing season (Gao, Masek, Schwaller, & Hall, 2006).Uncertainty of up to 40% has attributed to the difference in a 16-day revisit as compared to 4-day revisit, depending on climate and season (Guillevic et al. , 2019), though this was without daily meteorological data as a tool for interpolation.Conversely, the L3 dataset can capture more spatial variability for a given image as compared to the L1 and L2 data, which is highly important when using non-linear models.Therefore the L3 dataset is expected to perform better in areas of higher spatial heterogeneity (Sharma, Kilic, & Irmak, 2016).

Conclusions
The WaPOR products for Africa and the Middle East provide the highest resolution continuous near-realtime products available so far to monitor ETIa.Current validation efforts need to be continued and intensified to confirm the suitability of these products for various uses.However, significant issues with the sparseness of available ground-truth measurements make direct validation to in-situ, insufficient as a sole means to validate the ETIa product over continental Africa.To compensate for insufficient ground-truth locations, we added physical consistency and level consistency checks as part of the validation analyses.Results suggest that the ETIa-WPR are responsive to general trends associated with climate types.In dry irrigated areas, WaPOR appears to be overestimating ETa, particularly the coarse resolution.Analysis of the intermediate data components provide insights into some of the possible causes of the over-and underestimation of ETI-WPR, which appear to be primarily driven by an overestimation of the SMC which is driving overestimation of evaporation.The WaPOR database shows promising results, with an overall MAPE of 29-31% from local to basin scale, particularly considering it presents a continental almost near real time open-access dataset.
Further validation activities are suggested as new ground-data become available, particularly in cropped and irrigated areas.
The SA-SKU, SNDHR, GH-ANK, SD-DEM, CG-TCH, ZM-MON and ES-SCL EC sites were obtained from the global Fluxes Database Cluster Dataset (FLUXNET).The FLUXNET 2015 (https://fluxnet.fluxdata.org/)dataset consist of open-source high-quality data products collected from multiple regional networks.The NE-WAM, NE-WAF and BN-NAL sites were obtained from the African Monsoon Multidisciplinary Analysis-Coupling the Tropical Atmosphere and the Hydrological Cycle (AMMA-CATCH) project, aiming at establishing long term observations on the climate and the environment over Western Africa.KWSTI is operated by the International Institute for Geo-Information Science and Earth Observation at the University of Twente (ITC-UTWENTE) in partnership with Water Resources Management Authority (WRMA), the Kenya Wildlife Services (KWS) and Egerton University.The EG-ZAN, EG-SAA and EG-SAB sites were operated through the University of Tsukuba, in partnership with Cairo University, National Water Research Center, Delta Barrage, Qalubia, Egypt and the Agriculture Research