DSM Accuracy Evaluation for the ISPRS Commission I Image Matching Benchmark

Abstract. To improve the quality of algorithms for automatic generation of Digital Surface Models (DSM) from optical stereo data in the remote sensing community, the Working Group 4 of Commission I: Geometric and Radiometric Modeling of Optical Airborne and Spaceborne Sensors provides on its website http://www2.isprs.org/commissions/comm1/wg4/benchmark-test.html a benchmark dataset for measuring and comparing the accuracy of dense stereo algorithms. The data provided consists of several optical spaceborne stereo images together with ground truth data produced by aerial laser scanning. In this paper we present our latest work on this benchmark, based upon previous work.

As a first point, we noticed that providing the abovementioned test data as geo-referenced satellite images together with their corresponding RPC camera model seems too high a burden for being used widely by other researchers, as a considerable effort still has to be made to integrate the test datas camera model into the researchers local stereo reconstruction framework. To bypass this problem, we now also provide additional rectified input images, which enable stereo algorithms to work out of the box without the need for implementing special camera models. Care was taken to minimize the errors resulting from the rectification transformation and the involved image resampling.

We further improved the robustness of the evaluation method against errors in the orientation of the satellite images (with respect to the LiDAR ground truth). To this end we implemented a point cloud alignment of the DSM and the LiDAR reference points using an Iterative Closest Point (ICP) algorithm and an estimation of the best fitting transformation. This way, we concentrate on the errors from the stereo reconstruction and make sure that the result is not biased by errors in the absolute orientation of the satellite images. The evaluation of the stereo algorithms is done by triangulating the resulting (filled) DSMs and computing for each LiDAR point the nearest Euclidean distance to the DSM surface. We implemented an adaptive triangulation method minimizing the second order derivative of the surface in a local neighborhood, which captures the real surface more accurate than a fixed triangulation. As a further advantage, using our point-to-surface evaluation, we are also able to evaluate non-uniformly sampled DSMs or triangulated 3D models in general. The latter is for example needed when evaluating building extraction and data reduction algorithms.

As practical example we compare results from three different matching methods applied to the data available within the benchmark data sets. These results are analyzed using the above mentioned methodology and show advantages and disadvantages of the different methods, also depending on the land cover classes.