Vertical accuracy comparison of multi-source Digital Elevation Model (DEM) with Airborne Light Detection and Ranging (LiDAR)

Digital Elevation Model (DEM) is a digital representation of ground surface topography or terrain. There are many freely available DEM data with a spatial resolution of 30 m to 90 m. Nevertheless, their vertical accuracy may vary, depending on the vegetation cover and terrain characteristics. This study examined the vertical accuracy of open-access global DEMs (ALOS PALSAR, ASTER GDEM3, SRTM, TanDEM-X) and fused DEM (EarthEnv-DEM90, MERIT DEM). Their performances were assessed using a Digital Terrain Model (DTM) generated using airborne LiDAR data that had an outstanding absolute vertical accuracy (mean error (ME) = 0.24 m; root mean square error (RMSEz) = 1.20 m). Height differences between the global DEMs and the LiDAR DTM were calculated and examined their performances by forested vs. non-forested, slope, and elevation classes. The results showed the MERIT DEM was superior to other DEMs in most of the testing methods. It outperformed other DEMs with an RMSEz value of 3.02 m in the forested areas, followed by ALOS PALSAR (9.29 m), EarthEnv-DEM90 (9.40 m), SRTM (9.80 m), TanDEM-X (10.41 m), and ASTER GDEM3 (12.57 m). The MERIT DEM also had the best accuracy in the higher elevation areas. Overall, the ASTER GDEM3 had the worst accuracies, with relatively large over-estimations compared to other DEMs. Despite its low spatial resolution, the MERIT DEM was the best for representing terrain elevation for applications over a large area.


Introduction
Digital Elevation Model (DEM) can be defined as a 3D projection that generally serves as bare ground elevation data (DTM -Digital Terrain Model) or Earth's surface, including natural and man-made features (DSM -Digital Surface Model). Compared with labor-intensive traditional ground methods, utilization of spaceborne and airborne remote sensing systems show significant recognition in collecting a vast amount of data in a short period time at regional and global levels. Remote sensing systems can be divided into two main categories, passive optical sensors that measure reflected natural energy emitted from the sun. In contrast, active sensors emit their own energy and illumination.
Satellite-borne imagery (e.g., SPOT and ASTER) and aerial photogrammetry methods on a flying platform such as Unmanned Aerial Vehicle are used for terrain mapping at the global profile and local scale, respectively. However, photogrammetric DEM commonly characterizes top of terrain features due to its inability to penetrate the vegetation canopy and reach the ground. Radar-based technology utilizes radio waves which have advantages over photogrammetry; it can penetrate vegetation partially and is weather independence. For instance, commercial airborne radar imaging technology (NEXTMAP IFSAR) and spaceborne radar-based satellite (SRTM, ALOS PALSAR, TanDEM-X) are the most widely used RADAR data in various spatial analysis. Nevertheless, the backscatter radar intensity depends on the polarization and the forest structure, such as size, density, and distribution of the branches and leaves [1]. SRTM elevation value is between DSM and DTM, whereby it recorded IOP Publishing doi:10.1088/1755-1315/1053/1/012025 2 scattering in the upper part of the canopy before reaching the bare ground. Laser-based sensor (airborne LiDAR) has advantages over photogrammetry and radar-based by producing high-density and high accuracy 3-Dimensional (3D) point clouds. Phua et al. [2] demonstrated its ability to penetrate the dense forest canopy by recording multiple returns pulses that provide information on the bare ground elevation and vegetation height.
The potential errors and inconsistencies identified in DEM products usually arise from distortions caused by atmospheric, terrain, and sensor conditions during data acquisition [3]. In recent years, few researchers produced fused DEM from multi-source to resolve voids, noise, absolute bias, or tree height bias [4][5][6][7]. Several studies were performed to assess the global DEMs accuracy using various sets of reference data or high accuracy DEM [8][9]; however, most of the previous studies evaluated without taking into consideration of time gap between DEMs acquisition date and reference data. Thus, this study examines the vertical accuracy of six open-access global DEMs (ALOS PALSAR, ASTER GDEM3, SRTM, TanDEM-X) and fused DEM (EarthEnv-DEM90, MERIT). Besides, investigate the influence of topographic characteristics and land cover conditions on the accuracy of DEMs using LiDAR.

Study area
In this study, the validation areas are situated at Sulaman and Tamparuli in the district of Tuaran, Sabah,

DEM data
The LiDAR flight mission was conducted on 24 September 2017, with Optech Orion-C200 attached to a fixed-wing aircraft. LiDAR DTM with a pixel size of 1 m × 1 m was generated using the classified ground points. Six open-source DEMs are employed in this study. As the DEMs initially projected in the different coordinate system, all the datasets were projected in a Universal Transverse Mercator (UTM) 50 North projection, with the horizontal and vertical datum of WGS84 and EGM96 respectively; then resampled to 1 m pixel resolution using nearest-neighbor resampling method for elevation extraction and comparison. Table 1 shows the specifications of DEMs used in this study.

Vertical error assessment
DEMs' performance was assessed using ground control points (GCPs) acquired from DGNSS, and secondly, LiDAR DTM as reference data with respect to a function of vegetation cover, slope, and elevation. In each GCP point, calculated the difference by subtracting the DGNSS elevation from the corresponding DEMs pixel. The elevation differences (denoted as ) were then subjected to statistical analysis, mean error (ME) standard deviation (STD) and root mean square error (RMSEz), to evaluate the DEMs' vertical accuracy.
In the second approach, high accuracy LiDAR DTM was used as the reference model to analyze impacts of terrain characteristics and land cover conditions in general and calculate the same accuracy metrics used in the first approach. The slope classified into five classes (0° to 5°, 5° to 10°, 10° to 15°, 15° to 20°, and 20° and above), while the elevation was divided into four classes at 50 m intervals (0 to 50 m, 50 m to 100 m. 100 m to 150 m, and 150 m and above). It is necessary to find stable regions within the study region to obtain reliable evaluations from the different DEM datasets due to the acquisition time variation. Land cover effect on DEMs' vertical height error was assessed by grouping the random points according to unchanged land cover sources of forested areas and bare land.

Results and discussion
DEMs' performance was first assessed by calculating statistical metrics (ME, STD, and RMSEz) of the elevation difference between DEMs and GCPs ( Table 2). ME value refers to overestimation or underestimation from reference data. STD measures the dispersion of a dataset from another, and RMSEz can be explained as the dispersion of density distribution deviations between the reference dataset and DEMs data. ASTER DEM has the lowest accuracy among the six datasets with ME (11.19 m) and RMSEz (12.43 m). This study's finding was similar to a previous research [3], where ASTER had an overall RMSEz value of 9.22 m to 13.52 m. LiDAR DTM showed the best performance with ME (-0.24 m) and RMSEz (1.2 m), followed by fused DEM, MERIT with ME (1.79 m), and RMSEz (3.12 m). Thus, LiDAR DTM was selected as the reference model to examine the vertical accuracy of global DEMs, regarding land cover types and terrain characteristics for the whole study area. Figure 2 presents the correlation between global DEMs and LiDAR DTM on a scatter plot. In general, all DEMs revealed a strong correlation that fitted the 1:1 line. The result showed that ALOS PALSAR and MERIT have the best fit of the 1:1 line with r = 0.9940 and r = 0.9942, respectively.  Open-access global DEMs' performance on the land cover conditions was evaluated using statistical metrics as tabulated in Table 3. In every DEM, the STD value increases as the RMSEz value increases, indicating that the error magnitude variation was directly proportional to the RMSEz. MERIT DEM provided the best accuracy in both forested areas (RMSEz = 6.26 m) and barren land (RMSEz = 5.74 m) with a minor overestimation. However, the DEMs' vertical accuracy was significantly lower in the forested areas compared to barren land. ALOS PALSAR recorded RMSEz value 6.88 m in barren land but increased to 10.70 m in the forested areas. TanDEM-X acquired with single-pass SAR interferometry reported a RMSEz value of 14.41 m and ME of 11.50 m in the forested areas is close to the error reported by [10] even though they used a higher spatial resolution commercial TanDEM-X of 12 m.  [11] reported MERIT DEM showed promising performance in sites with urban, hilly forest, and rugged terrain (ME 3.17 m and RMSEz 7.82 m). Both ASTER and EarthEnv-DEM90 recorded their highest RMSEz in elevation 100 to 150 m and decreased to 16.25 m and 17.60 m respectively in elevation above 150 m. The DEMs RMSEz value was affected by slope; all DEMs showed higher error in a high steep slope, as shown in Figure 3(b). MERIT and EarthEnv-DEM90 exhibited the same pattern whereby RMSEz gradually increased as the slope increase. Overall, TanDEM-X provided the worst accuracy in high elevation areas and every slope class. According to the result, MERIT DEM outperformed the other five DEMs; it showed the lowest RMSEz value in every testing method. MERIT DEM was developed by separating and removing the four major error components in space-borne DEMs, stripe noise, absolute bias, tree height bias, and speckle noise [6] from existing DEMs (SRTM3 DEM, AW3D DEM and Viewfinder Panoramas' (VFP) DEM and ICESat/GLAS GLA14 data, U-Maryland Landsat forest cover data, NASA Global Forest Height Data, and JAMSTEC/U-Tokyo G3WBM waterbody data as supplementary data. Liu et al. [12] stated that both VFP-DEM and MERIT DEM show higher accuracy than the original SRTM, confirming the improved quality version of MERIT DEM. Another fused DEM, EarthEnv-DEM90, generated by filling input DEM data (ASTER and SRTM) voids and smooths the global surface to produce a nearlyglobal fused DEM. However, according to the result, its vertical accuracy was lower than MERIT. ASTER's uncertainty might be due to the input data of ASTER in EarthEnv-DEM90. Uuemma et al.   [3] also concluded that ASTER had the worst accuracy and highest uncertainty across all terrain and land cover types.

Conclusion
In conclusion, this study assessed DEMs' performance based on two approaches; (i) GCPs obtained from DGNSS and (ii) high accuracy LiDAR data as the reference model to examine the effect of terrain characteristics and land cover types on the DEMs vertical error. From the analysis, ASTER GDEM3 showed the least accuracy with a high RMSEz value and STD (high uncertainty) in forested areas and bare land. Terrain characteristics showed a significant impact on coarse resolution DEM. Despite MERIT DEM having a spatial resolution of 90 m, this newly released fused DEM outperformed other widely used DEMs. Thus, it is recommended that further processing and correction of these open-access global DEMs are necessary and beneficial for local or regional scale analysis.