ACCURACY IMPROVEMENT OF DEM FOR GENERATING ORTHOPHOTO BY REVERSE PROJECTION METHOD

The generation of orthophotos using digital aerial photography is performed for the maintenance of spa tial information data for wide areas at the mid-scale.For orthophoto generation, a digital elevation model (DEM) based on stereo matc hing methods is widely used, the accuracy of which should be effectively improve d by removing mismatched points created during the s ereo matching process for efficient orthophoto generation.With this motive in mind, a new method called the “Reverse-Projection M ethod” is proposed to remove mismatched points from DEM generation. The m ost remarkable points of this method are its abilit y to detect mismatched points and improve the accuracy of a DEM using only stereo models on the previous and following strips and restricting the higherresolution stage in the coarse-to-fine approach usi ng the Digital Terrain Model as a reference. The ef f ctiveness of the method is discussed in this paper.


INTRODUCTION
Digital orthophotos are widely generated using digital photogrammetry, for the preparation of geospatial information over wide areas.Digital orthophotos are created by projecting aerial photo images orthometrically using a digital elevation model (DEM).A DEM is an intermediate product of digital photogrametry and is extracted through a matching process.When a mismatch occurs during the matching process, a DEM with erroneous altitude information is extracted and a degradation of the accuracy of the orthophoto occurs.Checking and modifying a digital orthophoto requires a lot of manpower.Thus, the reduction of mismatches during DEM extraction directly affects the efficiency of digital orthophoto generation.There are two approaches for a reduction of mismatches during DEM extraction: a restriction of the search range and an improvement in the similarity of the texture pattern.A restriction of the search range has been adopted for epipolar image creation (Schenk 1999), the vertical line locus method (Bethel 1986), and the coarse-to-fine method (Shecnk 1999), while an improvement in the similarity of the texture patterns has been adopted for least square matching (Foerstner 1982;Ackermann 1984;Gruen 1985, et.al).However, even with these approaches, a perfect prevention of mismatches has yet to be achieved (Bethmann et al. 2010).While the following three methods have been proposed to judge and remove mismatched points, a problem remains in each method: Foerstner's method (Foerstner 1984) also incorrectly removes matched points in steep areas.The Back-Matching method (Zhang et al. 2006) may not remove mismatched points in areas with a continuously similar texture pattern.Finally, the multi-lay stereo model method (Zhang et al. 2005) requires additional time during the processing by creating stereo models between the base images and neighboring images (Takeda et al, 2008).In traditional aerial photogrammetry, stereo models for each image along the direction of every strip are created.Therefore, it is examined that the reliability of DEM points from multiple directions has been positively evaluated, and the accuracy of the DEM has been improved using the stereo models described above along and across the strip, without the need to create new stereo models.In this research, a Reverse Projection Method (RPM), which uses neighbor stereo models for evaluating the reliability of the DEM points, is proposed.Next, the propriety of this method for an improvement in the accuracy of the DEM is discussed.Furthermore, the best way to apply the RPM for optimizing the creation of the DEM is investigated.

THEORY AND ALGORITHM
The RPM proposed in this research is performed as follows.(1) Project DEM points to a neighbor stereo model after the matching process.(2) Evaluate the reliability of the DEM points using the similarity of texture patterns from the neighbor stereo model.(3) Finally, DEM points with low reliability are removed.
Figure 1 The flow for DEM extraction using the RPM Using these steps, the RPM prevents any mismatches, which are a problem for the efficiency of digital orthophoto generation, and maintains or improves the accuracy of the DEM.The flow of DEM extraction using the RPM is shown in Figure 1, and further details on this process are given in the following section.The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.34, Part XXX

RPM
Figure 2 shows the overall concept of the RPM.In Figure 2(a), a pair-wise matching point is extracted on master stereo model A through a matching process, and its ground coordinate is calculated using a forward colinearity equation (Schenk 1999).
In Figure 2(b), the ground coordinate of a DEM point is projected to neighbor stereo model B using back projection based on a colinearity equation.The reliability is then evaluated based on the similarity of the texture pattern at the projected point on stereo model B. Here, points with high reliability are adopted, while those with low reliability are rejected.

Figure 2 Matching Process and RPM
In this research, the normalized correlation value, which is stable against the tone or contrast of the image, is adopted as an indicator of the similarity.In addition, the threshold is set to 0.4, which is generally assumed to be a fairy strong correlation.Further, when multiple stereo models are available for a projection, the reliability can be evaluated as follows: ・ The DEM point is projected to each neighbor stereo model, and the similarity of each model is evaluated.
・ When the similarity is in the upper threshold, one vote is given to each stereo model.
・ When the similarity is in the upper threshold, one vote is given to each stereo model When a point is rejected, its altitude is interpolated from the adopted points of its neighbor by creating a triangle network (TIN), after the RPM is completed for all points, and the process moves to the next stage.

Coarse-to-fine approach
The coarse-to-fine approach enables a fast point search by creating a few stages of gradually scaled images and restricting the search areas for matching.However, a problem exists in that, when a mismatched point is generated during the middle of the process, the mismatch cannot be recovered until the end of the matching process is reached to the end of the matching process.Therefore, mismatched points need to be filtered during the middle of the process.Here, to remove mismatched points, the RPM filters out points with low reliability during each stage of the coarse-to-fine approach.
In this research, the number of stages for the coarse-to-fine approach is set to 6, which means creating gradually scaled images of Stage 6 (1:32) to Stage 1 (1:1).The search range used during the matching process begins with the lowest resolution of 1:25, with a substitution of the values in equation ( 1 H D , the height to ground;∆Z, the uncertainty of the elevation; and Z p , the approximate elevation. (1)

Reference of the Digital Terrain Model
During the matching process for the lowest resolution stage in the coarse-to-fine approach described in Section 2.2, mismatched points with a large gap from the correct altitude are often created.It has been speculated that these mismatched points are caused from a search of the entire range, as shown in Figure 3 (a).In this case, the uncertainty of the altitude is significant.On the other hand, as shown in Figure 3(b), when the approximate altitudes of the DEM points are known, large mismatches are prevented by restricting the search range around each altitude.
Figure 3. Matching process on the lowest resolution stage in the coarse-to-fine approach Digital terrain models (DTM) have been recently prepared for use at the global scale.Thus, the approximate altitude for every point is acquired in advance for stereo matching, based on DTM as a reference.
In this research, during the lowest resolution stage of the coarse-to-fine approach, the approximate altitude for the points are acquired from the "Digital Map 50 m Grid (Elevation)" published by GSI Japan.With this information, the search ranges are restricted around the DTM altitude and large mismatches are prevented prior to the matching process.In this research, the number of resolution stages for the coarse-to-fine approach is set to 4 when using the "Digital Map 50 m Grid (Elevation)" as a reference, which means creating lowresolution images of Stage 4 (1:8) to Stage 1(1:1).With a substitution of the values in equation ( 1), and under the same conditions described in Section 2.2, the uncertainty of an elevation is minimized to 20 m, which is the accuracy of the "Digital Map 50 m Grid (Elevation)," at the lowest resolution of 1:5.The search range around the DTM, as shown in Figure 3 (b), is set to 20 m, which is the same as the elevation uncertainty of the elevation.

RESULTS AND EXAMINATION
In this chapter, to confirm the effectiveness of the RPM proposed in this research, a DEM is extracted from aerial photo images using a matching process, and then qualitatively and quantitatively evaluated.

Various Factors
The aerial photo images used for the examination in this research were taken over five different areas.Each of these areas contains mountains, which are easily mismatched.The aerial photo images were also taken under the conditions shown in Table 1.When analog cameras were used, digital image files were created from the film using the pixel sizes shown in this table.
Here, the theoretical values for the horizontal (σ XY0 ) and vertical (σ Z0 ) directions are induced from parallax equation ( 2).where σ XY0 is the theoretical value for horizontal accuracy; σ Z0 the theoretical value for vertical accuracy; H, the height to ground; f, the focal length; B, the baseline length; and σ P , the Read Delta (1.0 pixel in this research). (2)

Accuracy Estimation
In this section, for an examination of the improvement in accuracy of the DEM extracted from an aerial photograph when applying the RPM, quantitative and qualitative evaluations are first performed.
Next, from a fear of expanding the processing cost of the RPM, as an increasing number of stereo models are used, the best combination of accuracy and processing cost was determined by adapting the stereo models for use with the RPM, as shown in Figure 4: (a) all stereo models of neighbors of the master image, (b) stereo models in the current strip, and (c) stereo models in the previous and following strips.Furthermore, under the best combination of stereo models, the restriction of the application of the RPM in coarse-to-fine approach is varied as shown in Figure 5. Here, the RPM is performed for (1) all resolution stages, (2) low-resolution stages only, (3) high-resolution stages only, and (4) with the DTM as a reference in the lowest-resolution stage and the RPM in the high-resolution stages only.In addition, the best method is investigated from by comparing the accuracy and processing time.
Figure 4 Combination of the stereo models for use of the RPM Figure 5 Restriction of the application of RPM in coarse-to-fine approach.Aerial photo by GSI Japan (2008)

Quantitative Evaluation
A quantitative evaluation of the RPM is performed by calculating the elevation gap between the matched DEM and the airborne lidar data treated as the most probable value.The elevation gaps are calculated at the check points where the edges of the features on the digital ortho photographs can be clearly distinguished.Here, points with elevation gaps over the theoretical value for the vertical direction, as shown in Table 1, are regarded as mismatched points.The airborne lidar data satisfies the accuracy of map level 2500, with a density of 2.8 points per square meter.

3.2.1.1
Comparison of variations in the stereo model combinations for the RPM Based on the variations in the stereo model combinations for the RPM, the mean squares of the elevation gap change, as shown in Table 2, as do the number of mismatched points, as shown in Table 3. From Tables 2 and 3, for the RPM with all stereo models or with stereo models of the previous and following strips, it is confirmed that both the mean square of the elevation gap and the number of mismatched points are improved as compared to a case without the use of the RPM.Here, the gap in the value between the use of the RPM with all stereo models, and with the models of the previous and following strips, is slight.The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.34, Part XXX Therefore, the stereo models of the previous and following strips are determined to be the best combination for the RPM from the viewpoint of processing cost.
Unit : m Table 2 The mean square of the elevation gap (combinations of stereo models) Table 3 The number of mismatched points (combinations of stereo models)

3.2.1.2
Comparison of the restricted application of the RPM in the coarse-to-fine approach For the use of the RPM with the stereo models of the previous and following strips, the changes in the accuracy of the DEM and in the processing times are examined using a restricted application of the RPM in the coarse-to-fine approach.The mean squares of the elevation gaps are shown in Table 4, and the numbers of mismatched points are shown in Table 5.
Table 4 Mean square of the elevation gap (Restriction in the resolution stage in the coarse-to-fine approach) Unit : m Table 5 The number of mismatched points (Restriction in the resolution stage in the coarse-to-fine approach) Based on Tables 4 and 5, compared to the cases without the use of the RPM, the accuracy of the DEM is improved with the application of the RPM for (1) all resolution stages, (3) highresolution stages, or (4) when using the DTM as a reference.
Table 6 shows the changes in the processing time with variations in the application of the RPM in the coarse-to-fine approach.Case (4) was confirmed to perform the processing within the same or shorter amount of time than when the RPM is not used.Based on this improvement, case (4) using the RPM and DTM as a reference, with only a high resolution in the coarse-to-fine approach, is deemed the most applicable method from a quantitative examination.
Unit : Second Table 6 Comparison of the processing times

Qualitative Evaluation
An examination of the effectiveness of the RPM was performed qualitatively by checking the distribution of the mismatched points on a color-shaded map created from the DEM extracted using the matching process.

Comparison based on the variations in the stereo model combinations for the RPM
The changes in the color-shaded maps with variations in the stereo model combinations for the RPM are shown in Figure 6.The squares in the figure indicated mismatched points in a mountainous area and rice fields generated without the use of the RPM.These mismatched points are removed by the RPM with a combination of all stereo models and only the stereo models of the previous and following strips.From a qualitative evaluation using color-shaded maps, the accuracy was confirmed to be improved from the use of the RPM when using combinations of all stereo models and only stereo models of the previous and following strips, which is the same result as from the quantitative evaluation.Also, for the distribution of mismatches, there are slight gaps between the two cases.The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.34, Part XXX Thus, as in the quantitative evaluation, the stereo models of the previous and following strips are judged to be the best combination for the RPM from the viewpoint of processing cost.

3.2.2.2
Comparison of the restricted application of the RPM in the coarse-to-fine approach For the RPM using stereo models of the previous and following strips, changes in the accuracy of the DEM are examined based on the restricted application of the RPM in the coarse-to-fine approach.The qualitative evaluation of the DEM accuracy is performed by checking the distribution of mismatched points on the color-shaded maps.Figure 7 shows the changes in the color-shaded maps for areas A and B, with variations in the restricted use of the RPM in the coarse-to-fine approach.Figures 7(a) and 7(b) show color-shaded maps for mountainous areas and rice fields, respectively.Mismatched points are generated in the squared areas without the use of the RPM.Additionally, it was confirmed that these mismatched points are removed for both areas when (1) the RPM is used with all resolution stages, (3) the RPM is used with a high-resolution stage, and (4) the RPM is used with DTM as a reference.Here, as in the quantitative evaluation, from the qualitative evaluation of the restricted application of the RPM, it was confirmed that these same cases improve the accuracy of the DEM.Based on the processing time for each case shown in Table 6, from a qualitative examination, it was judged that (4) the RPM with DTM as a reference using only high-resolution stages in the coarse-to-fine approach is the most applicable method.

Examination
In this section, changes in the accuracy of the DEM are examined, with variations in the stereo model combinations for the RPM, and with variations in the application of the RPM in the coarse-to-fine approach.

Changing the stereo model combinations for the RPM
From the quantitative and qualitative evaluations in section 3.2, the accuracy of the DEM was confirmed to be better when the RPM is used with all stereo models, or with only stereo models of the previous and following strips, as compared with stereo models of only the current strip.This result is speculated to have been caused from changes in the usable area of the projection based on variations in the stereo model combinations for the RPM, as the RPM, from its mechanization, cannot evaluate the reliability of the matched points when there is no stereo model to project, and because the reliability of the matched points depends on the amount of usable area for the projection.Table 7 shows the amount of usable area for the RPM and the reduction in the number of mismatched points for area D, where aerial photos are taken under the normal condition of a 60% overlap and 30% sidelap.It was confirmed that the amount of usable area for the RPM is proportionally related with the reduction in the number of mismatched points.
Table7 Amount of usable area for the RPM and the reduction in the number of mismatched points ( Area D ) From this result, the reduction in the number of mismatched points is the highest when the RPM is used with all stereo models.However, a few gaps exist in the rate of reduction of mismatched points when comparing the use of the RPM with all stereo models and with stereo models of only the previous and following strips.In addition, based on the processing cost with an increasing number of stereo models for the RPM, it was determined that the use of the RPM with the stereo models of the previous and following strips is the most effective method.

Restriction of the RPM in the coarse-to-fine approach
From the evaluation described in section 3.2, it was confirmed that the RPM with a restriction in the high-resolution stages in the coarse-to-fine approach, and the RPM using the DTM as a reference, both improved the accuracy.Figure 8 shows the changes in the color-shaded maps under the coarse-to-fine approach for each use of the RPM.From this figure, it can be confirmed that the mismatched points generated in the square areas are not filtered during the resolution stage when the RPM is not used.Furthermore, when the RPM is used with DTM as a reference during the lowest-resolution stage, large mismatched points are removed during stage 4 prior to the matching process during the higher resolution stage.Therefore, it is speculated that unnecessary searches are prevented and that the processing time is drastically reduced.

CONCLUSION
In this research, an RPM is proposed for an improvement in the accuracy of the DEM.The RPM is a method for evaluating the reliability of DEM points by projecting to neighbor stereo models and removing the mismatched points generated during the matching process.Evaluations of the RPM were performed for five areas under various photo conditions.A quantitative evaluation was performed by comparing the airborne lidar data and the DEM extracted through stereo matching, while a qualitative evaluation was performed using a color-shaded relief map.The results confirm the effectiveness of the RPM.In particular, it was confirmed that the use of the RPM with all stereo models, or with stereo models of the previous and following strips only, is effective for an improvement in accuracy.Furthermore, the RPM was successful in solving the problem of non-removed mismatched points generated during half of the processing when using the coarse-to-fine.The success of the RPM in improving the reliability of the matching process, while maintaining a fast search, shows the merit of the coarse-to-fine approach.The RPM achieved these results by re-using the stereo models generated by traditional aerial photogrammetry along and across the strip.That is, these results are achieved without generating further processing times for creating new stereo models, which is a problem in multiple viewpoint matching.In particular, this research offered the following original ideas to optimize this process.
(1) The use of stereo models of the previous and following for evaluating the reliability of the DEM points.
(2) Restricting the process during the high-resolution stage in the coarse-to-fine approach, and referring to the DTM to remove large mismatched points prior to the matching process.Based on the originality of this method, the accuracy of the DEM is expected to improve.This method is also expected to contribute toward optimizing the preparation of special information, with a tendency toward higher accuracy, higher density, and a wider area.
) : s = 6.0 pixels, Z p = 0 m, ∆Z = ±100 m, b 0 = 600 m, H D = 1920 m (b 0 and H D are calculated at a photo scale of 1:16000, with an overlap rate of 60% and a camera focal length of 120 mm), where s is the length of the search area; b 0 , the baseline length; Matching point on master stereo model (b) Reverse projection on neighbor stereo model B Aero photo by GSI Japan (2008) the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B4, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia

Figure 6
Figure 6 Comparison of elevation with RPM changing stereo model Rice fields(area B) International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B4, 2012 XXII ISPRS Congress, 25 August -01 September 2012, Melbourne, Australia

Figure 7
Figure 7 Comparison of elevation with the RPM Changing Resolution Stage in coarse-to-fine approach

Figure 8
Figure 8 Changes of the DEM under the coarse-to-fine approach.Aerial photo by GSI Japan (2008).