Absolute Surface Elevations Accuracies Assessment of Different DEMs Using Ground Truth Data Over Kingdom of Bahrain

For small islands, accurate digital elevation model (DEM) can help to understand the sea level rise prediction and scenarios impact on coastal zones, flooding risks assessment, flood inundation modelling, erosion and landslide, and environmental disaster process management. Currently, DEMs are available from several different sources using space borne systems, photogrammetry, surveying, topographic contour lines, etc. The aim of this study focuses on a comparison of absolute surface heights accuracies of four independent DEMs datasets over small island as Kingdom of Bahrain. The first two DEMs were acquired with space borne, Shuttle Radar Topographic Mission (SRTM-V4.1) and Advanced Space borne Thermal Emission and Reflection Radiometer (ASTER-V2.1) with 30 m pixel size. The second two DEMs with 2.5 m (DEM-2.5) and 5 m (DEM-5) spatial resolutions were derived from two different topographic contour lines maps at scales, respectively, 1:5000 and 1:25000 using inverse distance weighted (IDW) interpolation method. For validation purposes, a datasets of 400 ground control points uniformly distributed over the study site were used. They were measured using a Differential Global Position System (DGPS) assuring ± 1 and ± 2 cm accuracies, respectively, for planimetry and altimetry. The obtained results show that the derived DEM-2.5 exhibit the best accuracy ± 0.55 m which is excellent by reference to the tolerance or maximum error ± 0.78 m calculated based on errors sources propagation. As well, the DEM-5 shows very good accuracy ± 1.37 m by reference to the calculated tolerance ± 1.54 m. Then, SRTM shows a satisfactory performance with ± 3.00 m accuracy which is less than the absolute vertical height accuracy (± 5.6 m) advocated by NASA for African continent and Middle-East regions. Finally, the achieved ASTER accuracy ± 8.40 m is better than the estimated error (± 17.01 m) by USGS and JAXA.


Introduction
The Kingdom of Bahrain is a small island developing country in the Arabian Gulf. It has a limited capacity to adapt to sea level rise (SLR) and climate change impact. Indeed, SLR and extreme water levels are important manifestations of climate change impacts causing major threats to human beings around the world particularly in low lying coastal zones. Obviously, if SLR accelerates considerably, coastal environments and human populations will be affected significantly. According to Intergovernmental Panel on Climate Change Report [1], the global mean sea level has been rising during the last century an average rate of 1.7 ± 0.5 mm/year. By the end of this century, global climate models have predicted a global SLR of between 0.18 and 0.59 m [2]. Other approaches estimated higher rises of 0.8 m [3] and 0.5 to 1.4 m [4]. However, other scientists included the contribution of rapid dynamic effects to ice sheets for SLR by 2100, concluding that 0.8 m SLR is "likely", but 2.0 m is "plausible" if the highest reasonable rates of acceleration are included in the model [3]. Of course, to be 1 or 2 m SLR, the potential impacts increase significantly when populations and their related economic activities are highly concentrated along the coastal zones [5].
The low-lying nature of the coastal zones of Bahrain islands, coupled with significant land reclamation investments and extensive industrial, commercial, and residential activity, emphasize the country's critical vulnerability to SLR. The global warming, climate change and probable resulting accelerated SLR are among the hardest impacts that fall upon those coastal zones. The major impacts are increased flooding and inundation of low lying areas, shoreline retreat and loss of land [6]. These will produce geomorphological, ecological and socio-economic sever impacts [7]. In fact, tAcronyms Termshe increasing coastal inundation vulnerability may lead to considerable socio-economic losses such as the loss of coastal structures construction, damage to buildings and settlements, displacement of the population, and the loss of the agricultural production [8].
Furthermore, digital elevation model (DEM) data used to evaluate coastal zones vulnerability to SLR and flooding are available at various spatial resolutions and from several sources [9]. However, the quality of used DEM in such environmental assessment can significantly affect the detection of topographic features, the magnitude of hydrological processes and, consequently, affect the accuracy of the phenomenon evaluation or prediction [10]. Indeed, the DEMs can be generated using different methods that depend on acquisition procedures and techniques, such as photogrammetric methods, satellite-based techniques, field surveying and existing topographic maps [11]. Advances in elevation data acquisition allow for rapid data collection with acceptable altimetric accuracy. For instance, several DEMs such as Shuttle Radar Topography Mission (SRTM) and Advanced Space borne Thermal Emission and Reflection Radiometer (ASTER) are freely available today on the web. However, choosing the appropriate data for a specific project remains a difficult decision [12]. The aim of this study is the comparison of absolute surface heights accuracies of four DEMs datasets. Two acquired with space borne (ASTER-V2.1 and SRTM-V4.1) with 30 m pixel size, and two DEMs with 2.5 m (DEM-2.5) and 5 m (DEM-5) spatial resolutions were derived from topographic contour lines maps at scales, respectively, 1:5000 and 1:25000 using inverse distance weighted (IDW) interpolation method. For validation purposes, a datasets of 400 ground control points (GCPs) uniformly distributed over the Bahrain territory were used. They were measured using a Differential Global Position System (DGPS) assuring ± 1 and ± 2 cm accuracies, respectively, for planimetry and altimetry.

Study site
The Kingdom of Bahrain (26° 00' N, 50° 33' E) is a group of islands located in the Arabian Gulf (Figure 1), east of Saudi Arabia and west of Qatar. The archipelago comprises 33 islands, with a total land area of about 770.34 Km 2 in 2013 [13]. According to the aridity criteria and to great variations in climatic conditions, Bahrain has an arid to extremely arid environment [14]. Bahrain is characterized by high summer temperatures around 45°C in (June-September) and an average of 17°C approximately in winter (December-March). The rainy season runs from November to April, with an annual average of 72 mm. Mean annual relative humidity is over 70% due to the surrounding Arabian Gulf waters, and the annual average potential evapotranspiration rates is 2099 mm [15,16].

ASTER DEM data
The ASTER GDEM is a joint product developed and made available to the public by the Ministry of Economy, Trade, and Industry (METI) of Japan and the United States National Aeronautics and Space Administration (NASA). It is generated from data collected from the optical instrument ASTER onboard TERRA spacecraft [17]. This instrument was built in December 1999 with an along-track stereoscopic capability using its nadir-viewing and backward-viewing telescopes to acquire stereo image data with a base-to-height ratio of 0.6 [18]. Since 2001, these stereo pairs have been used to produce single-scene (60 × 60 km) DEM based on stereo-correlation matching technique using WGS84 geodetic reference [19]. In 2011, the validation and the accuracies assessment of ASTER GDEM products (version-2) were made jointly by NASA and Japanese partners [20,21]. The results of this study showed that the absolute geometrical rectification accuracies, expressed as a linear error at the 95% confidence level, are ± 8.68 and ± 17.01 meters for planimetry and altimetry, respectively [22]. The GDEM over our study region was downloaded from USGS data explorer gate [23], and it was preprocessed in GIS environment.

SRTM DEM data
The Shuttle Radar Topography Mission (SRTM) is an international project managed by Jet Propulsion Laboratory (JPL) and sponsored by NASA, National Geospatial-Intelligence Agency (NGIA) of the US Department of Defense, German Aerospace Center (DLR) and Italian Space Agency (ISA). It collect the most complete high-resolution digital topographic database over 80% of the Earth's land surface from 60° north to 56° south during 11-day mission; which was flown aboard the space shuttle Endeavour in February 11-22, 2000 [24]. The used radar systems are the C-band (5.6 cm) Space borne Imaging Radar (SIR-C) developed by NASA and the X-Band Synthetic Aperture Radar (X-SAR, 3.1 cm) developed by DLR with ASI participation. They have been flown for tests on two Endeavour missions in April and October 1994, and then modified for the SRTM mission to collect single-pass interferometry (InSAR) data using two signals at the same time from two different radar antennas. The first one was located on board the space shuttle and used as a transmitter and receiver, and the second receiver antenna at the end of a 60-meter (baseline) extended from the payload bay [25]. Obviously, the differences between the two signals allowed for the calculation of surface elevation using stereophotogrammetry methods [26].
Since 2000, the SRTM data were provided in 30 m pixel size only within USA territory, while for the rest of the world the data were available for public use at 90 m pixel size. On September 23, 2014, the American government announced that the highest resolution elevation data generated from NASA's SRTM in 2000 will be released globally over the next-short future with the full resolution of the original measurements, 30 m pixel size. Data for most of Africa and its surrounding areas were released with the September 2014 announcement. Then, in November 2014, the data were released for south and North America, most of Europe, and islands in the eastern Pacific Ocean. The most recent release, in January 2015 includes most of continental Asia, the East Indies, Australia, New Zealand, and islands of the western Pacific [27]. The data are projected in geographic coordinates system using WGS-84 geodetic reference and EGM-96 (Earth Gravitational Model 1996) vertical datum. According to USGS [28], at 90% confidence, the absolute vertical height accuracy is equal or less than ± 16 m, relative vertical height accuracy of less than ± 10 m, circular absolute planimetric error of less than ± 20 m, and circular relative planimetric error of less than ± 15 m [29]. These data have been planned to meet the needs of the scientific applications, civilian applications, and military applications. The used SRTM over our study region was downloaded from USGS data explorer gate [28] and it was preprocessed in GIS environment.

Topographic contours lines preprocessing
The topographic contour lines maps were obtained from the Survey and Land Registration Bureau, Topographic survey Directorate (Kingdom of Bahrain). Both maps were established from photogrammetric stereo-preparation and restitution exploiting optico-mechanic stereo-plotter and aerial photographs at 1:10000 scale. The contour lines intervals (equidistance) are 2.5 m and 5 m, respectively, for the very large (1:5000) and medium scales (1:25000) maps. These data maps were stored in Auto-CAD format and they were converted to shape file using spatial analyst module of ArcGIS 10.2. This conversion generally requires additional filtering and preprocessing. Indeed, the contour lines representing the real altitude value were unfortunately aggregated with other information layers ( Figure 2) that may cause background noise during the transformation process. Consequently, it was necessary to remove these background noises in ArcGIS. In the first step, the contour lines and their attributes (elevation values) were converted to shape file. Then, vectors were edited for error commissions' verification, control and correction. Moreover, topology building was used to re-inspect if some errors remain and to build spatial relationships. Then, the data were retroprojected using Universal Transverse Mercator (UTM) and World Geodetic System 1984 (WGS-84).

Topographic contours data and DEMs derivation
The derivation of DEM-2.5 and DEM-5 was based on contour lines extracted from topographic maps, respectively, with a very large (1:5000) and medium (1:25000) scales. As we discussed before, both contour lines maps were established from photogrammetric stereopreparation and restitution exploiting optico-mechanic stereo-plotter. According to the photogrammetric theory, the vertical accuracy of contour lines depends on the base-height ratio, and the relationship of the ground distance between successive exposures of photographs to the flying height [30]. It depends also on the precision of GCPs incorporated in stereo preparation and stereo-pairs model calibration (internal and external orientations). Thus, the vertical RMSE (RMSEVertical) of generating contour lines from photogrammetry can be computed using the following equation [31]: Since the used contours intervals are 2.5 m and 5 m, the calculated contours lines elevations accuracies (RMSEVertical) are ± 0.76 m and ± 1.52 m, respectively, for 1:5000 and 1:25000 maps. Nevertheless, based on error propagation theory, we must consider the digitalization and interpolation method errors. As well, the output pixel size specifications which determine the derived DEM details must be taken in consideration depending on the richness of contour lines and their spatial distribution [32,33]. However, this error is insignificant since the contour lines incorporated in the interpolation process are very dense with excellent spatial distribution over Bahrain. Indeed, the statistical nearest neighbor analysis regarding the density and distribution showed an excellent precision (RMSE=0.1%). Moreover, the contour lines elevation values have been introduced manually in the attributes table immediately after the digitalization of each vector, thus eliminating the probable altimetry error. But for plan metric coordinates position error, based on the maps scales and the digitizing table characteristics (± 0.249 mm accuracy), it has been estimated at ± 12.5 cm for 1:5000 map and ± 25 cm for the 1:25 000 map. These plan metric errors are insignificant vis-a-vis the desired output pixel sizes (2.5 m and 5 m) after interpolation process.
Likewise, it has been demonstrated that DEM accuracy can vary to a certain degree with different interpolation algorithms and interpolation parameters [33]. Several interpolation methods existent in ArcGIS and other mapping software's, and the best and appropriate DEM interpolation method must reproduce as close as possible the terrain shape [34][35][36][37]. The IDW method is a local deterministic technique that estimates the unknown value as a distance-weighted average of known points in a defined neighborhood [37]. It considers that points closer to the query location will have more influence, and weights the sample points with inverse of their distance from the required point. Compared to other methods, IDW has been found to adjust themselves to the topographic variations and more appropriate for geo-morphologically smooth areas [38] without significant topographic variation as Bahrain. Consequently, the IDW approach was considered in this study to generate the contours DEMs with 2.5 m output pixel size from 1:5000 maps (DEM-2.5) and 5 m output pixel sizes based on 1:25 000 maps (DEM-5). As stated by Gao [39], the accuracy of a derived raster DEM using interpolation method (RMSE Interpolation ) is related to the contour density and the DEM output pixel size, and it was formulated as follows: RMSE Interpolation = ± (7.274+1.666 S) D/(1000+ɛ) (2) Where, S stand for resolution in meters; D stands for contour density expressed as Km/ Km 2 ; ε is an error term related to D. Contour density was calculated by dividing the total length of contour by the size of the study area. Based on these research variables, these accuracies were estimated at ± 16.0 cm for DEM-2.5 m and at ± 31.1 cm for DEM-5 m. Therefore, the total DEM elevation error in terms of RMSE can be formulated as follow: Finally, considering all error sources propagation, the total obtained RMSE (RMSE Total) is ± 0.79 m on the DEM-2.5 and ± 1.54 m on the DEM-5. In other word, these are the tolerances or the maximal errors which must not be exceeded in comparison with the reference points for validation i.e., DGPS, GPS, geodetic or cadastral surveying points.

Validation and accuracy assessment
In the section above, we calculated the accuracies of the derived DEMs based on topographic contour lines and IDW interpolation method. In this section we present the mathematical relation to calculate the accuracy for each DEM independently by reference to DGPS in situ measurements assuming that these errors are normally distributed. The different DEMs were validated with reference to elevation data acquired with DGPS which are the truth elevation, and they were uniformly distributed across Bahrain. According to American Society for Photogrammetry and Remote Sensing (1990), the height precision of each DEM should be expressed by the root mean square error (RMSE DEM-j) given by the following relation: Where H Ref is the reference DGPS elevation data (in situ measurements), HDEM-i is the elevation data from each considered source (SRTM, ASTER and topographic contour maps), and "n" corresponds to the total number of DGPS points used for validation. As recommended by USGS [40], DEM error estimation is usually made with a minimum of 28 GCPs. However, Li [41] reported that many GCPs are needed to achieve a consistency closer to what is accepted in most statistical tests. Indeed, the number of validation GCPs is an important factor in consistency because it conditions the range of stochastic variations on the RMSE and standard deviation values [41]. If we consider only 28 GCPs, as recommended by USGS, the confidence level value is around 85%. Therefore, to reach 95% confidence level we must consider approximately two hundred (200) GCPs. To guarantee the error estimation stability in this research, 400 GCPs have been considered for validation.

Results
The four considered DEMs (2.5 m, 5.0 m, SRTM and ASTER) are obtained from different and independent data source, and their accuracies varies as a function of many source errors. Their visual interpretation shows globally similar characteristics (Figure 3) which are composed of five distinctive physiographic zones [42]. The first region is the coastal lowlands with elevation less than 10 m above mean sea level and slopes less than 0.5%. The second region is the upper Dammam back-slope which reflects the general asymmetrical shape of the main Bahrain dome whit elevation between 10 and 20 m, and slopes less than 5.4%. The third region is the multiple escarpment zones surrounding the interior basin of the island; it is a continuous belts of low multiple enfacing escarpments. From the north-west to the south-west of this region, the elevation and slopes varies significantly, respectively, from 20 to 34 m and from 5.4 to 14%. The fourth region is the interior basin which looks as an asymmetrical ring of lowlands surrounds the central plateau region (fifth region) whit relatively height elevation and strong slopes classes, respectively, 34 to 51 m and 14 to 29.5%. Finally, the fifth region is central plateau with upstanding residual hills and mountain. In this region, the elevations and slopes varies significantly between 51 and 134 m for Jabal Dukhan (the highest point in Bahrain) and 30 to 81%, respectively.   Table 1: Altitudes statistics of 400 validation GCPs measured by DGPS and their homologues in the considered DEMs. Figure 4 illustrate a statistical correlation between the 400 DGPS validation points and their homologous in each DEM. For the contours derived DEMs, we observe that the validation DGPS GCPs correlate perfectly with their homologous (R 2 =0.99 for DEM-2.5 and R 2 =0.97 for DEM-5). This result was expected because the topographic contours lines were plotted based on accurate stereo-photogrammetry and surveying methods. For the space-born DEMs, the correlation coefficients are 0.96 and 0.92, respectively, for SRTM and ASTER. These correlations indicated that SRTM perform slightly better than ASTER by reference to validation points. This slight performance is also expressed by the Figure 3c which depicted the distribution of validation points around the fitting axe (line 1:1). We observe that the SRTM cluster points fall very close to the one-to-one line axe better than ASTER. In addition, for ASTER DEM, Figure 4d show that the majority of cloud points are above the line 1:1 confirming that it overestimated altitudes significantly. These scatter-plots corroborate the summarized statistics in Table 1. Furthermore, Table 2 summarizes the statistics of absolutes altitudes differences (Δh) between the 400 GCPs measured by DGPS and their corresponding homologous in each considered DEM. These altitudes differences are also illustrated by the scatter plot in Figure 5. In this Figure 5, the "X" axis of this scatter plot at the coordinate (0, 0), named also zero error axis, is a hypothetical "ideal" line which around it theoretically must gravitate the cloud points showing the perfect concordance between the ground truth (DGPS-GCPs) and their homologous. More the points are closer to this hypothetical line more the accuracy is better and vice-versa. However, most likely this assumption is not always validated because the altitude estimation accuracy is often influenced by the nature of the land use classes, nature of targets, various terrain slopes, several terrain morphology, and several errors sources [43,44]. Globally, we see that the DEM-2.5 altitudes differences points are well distributed around the "X" axis showing minimum and maximum difference variations between -1.98 and +2.61 m, with a SD of ± 1.03. Although the DEM-5 indicated a minimum value of -2.12 m and a maximum of +5.83 m, the majority of differences altitudes points correctly concur with the hypothetical line with satisfactory SD (± 1.27). The difference altitude distribution of SRTM versus DGPS-GCPs vary between -13.00 m and +10.79 m, with a large number of points relatively distant from the theoretical line between -5 m and +5 m; which means a relatively significant error compared to contour lines DEMs. While, the ASTER altitude difference diverge between -20.86 m and 17.16 m and a greater number of validation points are very scattered with respect to the hypothetical line. Moreover, the large standard deviation (SD=± 5.08) state a large variance and, consequently, a large variability of altitudes differences. Obviously, this broad divergence expresses a significant error of ASTER to estimate accurately the altitude of small island. In addition, Figure 5 indicates that the altitudes differences between validation reference points and their homologous are sometimes negative and sometime positive and there is no clear and logic relationship behind this random variability.   Finally, the global surface heights accuracies expressed with RMSE were calculated using the equation 4. As was expected, the derived DEM-2.5 from topographic contours map at 1:5000 exhibit the best accuracy ± 0.55 m compared to the tolerance or the total error (± 0.79 m) which is calculated based on errors sources propagation (equation 3). As well, the generated DEM-5 from 1:25000 maps shows very good accuracy ± 1.37 m by reference to the tolerance (± 1.54 m). Then, the results shows good performance of SRTM with an accuracy of ± 3.00 m which is less than the absolute vertical height accuracy (± 5.6 m) advocated by NASA for African continent and Middle-East regions. Finally, the achieved ASTER accuracy ± 8.40 m is better than the estimated error (± 17.01 m) by USGS and JAXA. This finding concur with those published by Hirano et al. [19] who estimated an RMSE in elevation between ± 7 and ± 15 m; and those published by EDC [43] which yield an RMSE of ± 8.6 m. Nevertheless, these accuracies are significantly influenced by the nature of the land use classes and slopes as discussed above. Accuracies are relatively influenced by the variation of the topography; errors are relatively larger for high to medium altitude with relative strong slopes, while they are smaller in the low relief areas with low slopes. These results are in agreement with other previously published results [44][45][46]. Furthermore, SRTM perform better than ASTER because radar wavelength ranges furnish good signal returns from the Earth's surface. Moreover, the almost total absence of vegetation cover in the study area helps the radar system to characterize the surface topography since its signal adheres very well to the micro-topography and determines the intensity and type of the backscattered signal [47]. Indeed, radar is sensitive to surface roughness because shorter radar wavelengths (X-band) are most sensitive to micro-topography, while long wavelengths (C-band) are sensitive to macro-topography. Obviously, these characteristics advantage SRTM compared to ASTER. Nevertheless, both DEMs datasets characterized equally the macro-topography, geological structures, erosion features and terrain geomorphology (Figure 3).

Conclusions
The aim of this study was the comparison of absolute surface heights accuracies of four DEMs datasets over Kingdom of Bahrain. The SRTM-V4.1 DEM derived from radar interferometry and ASTER-V2.1 DEM derived from digital photogrammetry was used. In addition, two DEMs with 2.5 m (DEM-2.5) and 5 m (DEM-5) spatial resolutions were derived from topographic contour lines maps at scales, respectively, 1:5000 and 1:25000 using IDW interpolation method. For validation purposes, a datasets of 400 GCPs uniformly distributed over the study site were used. They were measured using a DGPS assuring ± 1 and ± 2 cm accuracies, respectively, for planimetry and altimetry. The obtained results show that over small island with topographic features higher than 100 m, except ASTER DEM, the three other tested DEMs are found to be consistent. Indeed, the derived DEM-2.5 exhibit the best accuracy ± 0.55 m which is excellent by reference to the tolerance or maximum error ± 0.78 m. As well, the DEM-5 shows very good accuracy ± 1.37 m by reference to the calculated tolerance ± 1.54 m. Decidedly, these two DEMs are more accurate to evaluate coastal zones vulnerability to SLR, flooding and the detection of topographic features and the magnitude of hydrological processes. The only problem is the availability of this type of data and the economic factors which is significant.
Furthermore, the SRTM shows a satisfactory performance with ± 3.00 m accuracy which is less than the absolute vertical height accuracy (± 5.6 m) advocated by NASA for African continent and Middle-East regions. Although this acceptable result it still subject of several errors sources which are propagated in the measurements acquired by SRTM mission. These included to the shuttle position, astronauts activities, uncertainty of the baseline (the length and orientation of mast) which is the most significant error source, timing error, multipath, phase measurement error, thermal distortions and noise of the radar system as the Shuttle moves around the Earth in orbit, and going in and out of sunlight [48,49]. Farr et al. [50] elaborated all these error sources in detail with their mathematical equations and they quantified the effects of each one individually. However, according to this good enough accuracy and the ready-to-be-used, SRTM-V4.1 DEM is of great interest for morphological studies of small islands located especially in regions with frequent cloud coverage.
Finally, the achieved ASTER-V2.1 DEM accuracy is ± 8.40 m, and it is better than the estimated error ± 17.01 m by USGS and JAXA. This large error can be related to many anomalies and artifacts, due to sensor radiometric sensitivity, atmospheric variability, clouds, stereopairs images geometry, and the automated algorithm used to generate the final DEM based on stereo correlation procedures. Moreover, other scientists believe that orbital parameters of the TERRA-Platform might have an impact on ASTER DEMs data acquisition [51]. However, although these errors sources and its limited accuracy, ASTER provides globally an acceptable representation of the overall island morphology and macro-topography. But it is not providing a suitable and accurate DEM to simulate the impact of SLR scenarios on small islands or to analyze the vulnerability of low lying areas to inundation and flooding.