REPROCESSING CLOSE RANGE TERRESTRIAL AND UAV PHOTOGRAMMETRIC PROJECTS WITH THE DBAT TOOLBOX FOR INDEPENDENT VERIFICATION AND QUALITY CONTROL

: Photogrammetry has recently seen a rapid increase in many applications, thanks to developments in computing power and algorithms. Furthermore with the democratisation of UAVs (Unmanned Aerial Vehicles), close range photogrammetry has seen more and more use due to the easier capability to acquire aerial close range images. In terms of photogrammetric processing, many commercial software solutions exist in the market that offer results from user-friendly environments. However, in most commercial solutions, a black-box approach to photogrammetric calculations is often used. This is understandable in light of the proprietary nature of the algorithms, but it may pose a problem if the results need to be validated in an independent manner. In this paper, the Damped Bundle Adjustment Toolbox (DBAT) developed for Matlab was used to reprocess some photogrammetric projects that were processed using the commercial software Agisoft Photoscan. Several scenarios were experimented on in order to see the performance of DBAT in reprocessing terrestrial and UAV close range photogrammetric projects in several configurations of self-calibration setting. Results show that DBAT managed to reprocess PS projects and generate metrics which can be useful for project verification.


INTRODUCTION
Close range photogrammetry has often been used to acquire 3D data (e.g.shape, position, and size) from images (Grussenmeyer et al., 2002).The rise in the use of UAVs (Unmanned Aerial Vehicles) and rapid developments in imaging technology and image processing have increased the use of close range photogrammetry for mapping purposes (Murtiyoso and Grussenmeyer, 2017).This relatively low cost solution (Barsanti et al., 2014) for mapping and reality-based 3D modelling is often complemented by commercial, easy-to-use photogrammetric and/or SfM (Structure-from-Motion) software packages.Although some open source software alternatives exist (Pierrot-Deseilligny and Clery, 2012;González-Aguilera et al., 2016), commercial software such as Eos System's Photomodeler, Pix4D, and Agisoft Photoscan remain very popular, especially outside the photogrammetric community due to their simplicity in creating fairly accurate results (Grussenmeyer et al., 2002;Remondino et al., 2014;Burns and Delparte, 2017).Commercial solutions typically hide the algorithms and show a simplified interface to the user in order to make it easy to generate the desired result.This is an advantage for many users, especially those that are not used to the classical photogrammetric workflow.At the same time it complicates a transparent and independent check of the result of each stage of the workflow.
One main and important aspect of the photogrammetric workflow is the external orientation or camera pose estimation step, in which the positions and rotational attitudes of each of the camera stations are determined.The camera pose estimation problem is often resolved using a bundle adjustment process with initial values coming from several possible approaches such as relative orientation, spatial resection, Direct Linear Transformation (DLT), etc. (Luhmann et al., 2014).The primary aim of this paper is to test whether the open source toolbox DBAT (Damped Bundle Adjustment Toolbox) (Börlin and Grussenmeyer, 2013) can be used to perform the bundle adjustment reprocessing of terrestrial and UAV photogrammetric projects that were previously processed using the commercial software Agisoft Photoscan (PS).
Compared to classical aerial photography, images provided by terrestrial and UAV close range acquisitions present a particular problem absent in traditional aerial photography, in that the image and control point configuration is often irregular.It is therefore in the interest of some users to understand the results in a more detailed manner.In addition, the projects used in this paper contain up to a few hundred images, which is higher than the number of images tested for DBAT in previously published studies.A secondary aim of this paper is thus to present a larger case study for DBAT.

SOFTWARE AND RELATED WORK
The UAV was originally designed for military purposes, but has since seen many civilian uses in recent years.Photogrammetry by UAV opens many possibilities for its application in closerange situations.It complements terrestrial acquisition of 3D information (Nex and Remondino, 2014).In its role as an aid to photogrammetric work, the UAV has seen many applications in various fields, such as disaster management (Achille et al., 2015;Baiocchi et al., 2013), 3D building reconstruction (Roca et al., 2013), surveying/mapping (Cramer, 2013), and heritage documentation (Chiabrando et al., 2015;Murtiyoso and Grussenmeyer, 2017).
The Damped Bundle Adjustment Tools (DBAT) is a series of functions developed in the Matlab language that enables the user to reprocess bundle adjustment projects generated by Photomodeler or Photoscan.While the toolbox was originally developed to test various techniques ("damping") to improve the convergence radius for the bundle adjustment process (Börlin and Grussenmeyer, 2013), DBAT also provide comprehensive statistics once converged such as posterior variance and correlations between estimated parameters.
DBAT has been tested for several cases, including camera calibration (field calibration and laboratory-based, coded-target plate calibration) (Börlin and Grussenmeyer, 2014), large scale aerial photographs (small image sample) (Börlin and Grussenmeyer, 2016), as well as several tests in close range configurations (Dall'Asta et al., 2015).In this paper, the use of DBAT on PS projects is emphasized, since PS generates few statistics and is therefore more difficult to verify on its own.This paper uses DBAT version 0.6.5.5.
Agisoft Photoscan (PS) is a 3D reconstruction software which may be used for both aerial and terrestrial images.PS offers an easy-to-use graphical interface and workflow, with further possibilities to perform automation using Python scripts.PS employs a computer vision-leaning approach to generate 3D models.This presents a particular challenge to compare it to photogrammetric conventions since some terms (Granshaw, 2016) and definitions (Hastedt and Luhmann, 2015;Murtiyoso et al., 2017a) are different.DBAT has recently been developed to accommodate these differences between photogrammetry and computer vision, particularly in terms of lens distortion parameters.This paper uses the PS version 1.3.4build 5067.

Data sets
Two data sets were used in this paper; a UAV data set and a terrestrial close range data set.The UAV data set was of the St-Pierre-le-Jeune church which has previously been modelled using several software solutions (Murtiyoso et al., 2017a) and will serve as a basis for the reprocessing using DBAT.The St-Pierre-le-Jeune church is situated at the recently enlisted UNESCO World Heritage Site of Neustadt, in the city of Strasbourg, France.Although the church in its entirety has been documented in 3D, for the purposes of this research only the principal façade will be reprocessed in DBAT.The St-Pierre data set consisted of 239 images (Figure 1) each with a 38 MP resolution.Among these images, 67 were taken from a perpendicular point of view while the rest were oblique images taken with the sensor oriented upwards, downwards, to the left, and to the right.This configuration was used in order to take into account the geometric requirements of a convergent photogrammetric block, as well as to cover difficult parts of the object during the dense matching step.The used UAV was the Sensefly Albris, which has the capability to maintain an approximate distance to the object.This enabled the data set to have a roughly constant camera-to-object distance and therefore constant theoretical Ground Sampling Distance (GSD).In this case, the theoretical GSD is 1.4 mm for a distance of 8 meters.
A total of 9 ground control points (GCPs) were measured on the façade, with a precision of 5 mm.From these 9 GCPs, 3 were selected as check points (CPs).The choice of GCPs and CPs follows the convention usually used in classical aerial photogrammetry (Kraus and Waldhäusl, 1998).
In addition to this field acquisition, the Albris sensor was calibrated beforehand using a set of coded targets that was put in a dedicated room.The coded-targets were measured using a total station in order to give a rigorous setup for the calibration.The sensor was then calibrated in PS.The precalibrated values were used in one of the scenarios tested in this paper, while tests using approximate values derived from the images' EXIF file were also performed.
The second data set, "Lacey", was a terrestrial close range acquisition of several World War I graffiti in the Maison Blanche underground site in Neuville-Saint-Vaast, northern France.The graffitis have a special historical interest as they were made by Canadian soldiers several days prior to the Battle of Vimy Ridge on April 1917.The entire Maison Blanche has similarly been modelled in a previous research (Murtiyoso et al., 2017b), and in this paper only a segment will be reprocessed in DBAT.The Lacey comprises of 346 images at 50 MP acquired by a Canon EOS 5DR camera with a 28 mm lens.Five coded-target GCPs were measured on this data set.In order to perform verifications for the results, two additional check points were measured indirectly from the laser scanning point cloud of the same site.The laser scanner used to this end was a FARO Focus X330.The Lacey data set was taken from an average camera-to-object distance of 2 meters, giving a theoretical GSD of 0.3 mm.The main difference between the scenarios lies in the self-calibration parameter configuration.The scenarios marked beginning with an "S" signify the processing of the St-Pierre data set, while those marked with "L" at the beginning signify the processing of the Lacey data set (see Table 1).All scenarios were recreated in DBAT and the results were compared.The quality criteria of interest were chosen to be the RMS values of the GCP errors, and the RMS values of the CP errors.In addition, the estimated calibration parameters were also analysed in order to see if DBAT could recreate the project.The GCP RMS may be seen as a measure of internal bundle adjustment precision of the respective algorithms, while the CP RMS may give an idea on the accuracy of the solution compared to ground truth data.

Experiments
The precision of the GCP measurements was taken into account during the bundle adjustment in PS and DBAT as weighting factors.In addition, the a priori marking precision for both manual and automatic object points (OPs) in both data sets were fixed at 1 pixel.The choice of this value was done in order to facilitate the comparison between PS and DBAT.

RESULTS AND DISCUSSIONS
An illustration of the result of the bundle adjustment in the two algorithms tested is shown by Figure 2. In general, all algorithms managed to reach convergence in their computation and orient all images in all of the proposed scenarios.Several analyses regarding the results of the self-calibration process and the performance of each algorithm will be described in this section.The self-calibration analysis of the UAV sensor will be emphasized in this paper, while the GCP and CP analysis will be performed for both data sets.The UAV sensor selfcalibration is of particular interest since it consists of a small low-cost sensor in a parallel geometry acquisition, compared to the convergent geometry of the DSLR camera in the Lacey data set.

Self-calibration results
Detailed results of the self-calibration for the St-Pierre data set can be seen in Table 2.As a comparison, a column containing the precalibrated values was also added to  Differences in terms of the distortion parameters were more difficult to ascertain.To this end, the radial distortion curves have been plotted in Figure 3.In the cases of S1, S2, and S4, DBAT managed to generate a similar distortion profile as that of PS, with small differences beginning at the radial distance of 4.25 mm from the projective centre.These minor differences may come from slight errors due to the conversion from PS to DBAT distortion coefficient format.The DBAT format follows the Photomodeler convention in presenting distortion parameters as polynomial coefficients scaled by the focal length, while PS calculates the normalised value of these parameters.Differences with the precalibrated values are to be The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W8, 2017 5th International Workshop LowCost 3D -Sensors, Algorithms, Applications, 28-29 November 2017, Hamburg, Germany expected since the conditions during the calibration are not exactly the same as the conditions during the real acquisition.Furthermore, some difference between PS and DBAT may be expected because PS most probably performs a free network adjustment followed by a conformal 3D transformation, whereas DBAT includes GCPs in its bundle adjustment computation.
It is also interesting to note that PS and DBAT arrived at the same calibration values in S1 (with EXIF initial values) and S4 (with precalibrated values as initial values).However, it should be noted that the case of the St-Pierre data set presents a particular case where oblique photos were also included in the bundle adjustment process; this increases the strength of the acquisition network geometry.S3 presents an interesting observation on its distortion curve.By not calculating K 3 in the self-calibration process, DBAT and PS's curve diverge almost from the 1 mm radial distance.Furthermore, the σ 0 value of S3 in DBAT gives a value of 2.070 which presents an anomaly compared to the other cases (see also Table 3).PS also gave a reprojection error of 2.700 pixels.Even though the fact that K 3 is not calculated suppresses the correlation (see Table 4) between the estimated calibration parameters, this may indicate that for this particular sensor K 3 is nevertheless an important factor.Comparison of the GCP and CP RMS for the different scenarios tested in this experiment can be seen in Table 3.It should be noted that in this experiment, in order to compare both algorithms, the GCPs were weighted using their precision of 5 mm, while all markings whether automatic or manual were weighted using a uniform marking precision of 1 pixel.

GCP and CP verification
Results of the bundle adjustment show that for the UAV St-Pierre data set a maximum difference of 0.3 mm for the GCP RMS between DBAT and PS were observed.The maximum difference of CP RMS for the same data set was also 0.3 mm, for a theoretical GSD of 1.4 mm.For the Lacey data set, the maximum GCP RMS difference was 0.5 mm and 0.7 mm for CP RMS for a theoretical GSD of 0.3 mm.
In the Lacey data set, a higher value of RMS can be explained by the fact that the GCPs were measured using a coded-target, which would have had a very small marking precision (the default PS value was 0.1 pixels).This influences the weighting in the bundle adjustment, as shown also on the value of the σ 0 .Indeed, the σ 0 value of 4.5 on average indicates that one of the a priori standard deviation was heavily underestimated, which is true in this case.Furthermore, the even higher values for the CP RMS in the Lacey data set is due to the fact that the CPs were indirectly measured from the laser scanner data set, and concern points of interest rather than clear artificial marks or coded targets.The centimetric result for the CP RMS is therefore expected.
In the St-Pierre data set, several other factors also contribute to the final RMS result.The GCPs were distributed evenly on the façade; however the lack of depth variation between the GCPs may contribute to the final RMS.Furthermore, the noise present on the images also generates another source of error.However, the main objective of the experiment was to compare the performance of PS and DBAT.In this regard, DBAT has managed to reprocess PS projects under approximately the same conditions and weighting, although a slight difference is always to be expected when dealing with a black-box solution.It may then be used as a tool to verify PS's results and perform a quality control on it.The experiments and analyses in sections 4.1 and 4.2 indicate that DBAT may be used to reprocess photogrammetric project in the case of UAV data (St-Pierre) and also classical terrestrial close range data (Lacey).One advantage of DBAT lies in the metrics that it provides the user at the end of the bundle adjustment process.Several metrics can therefore be used to assess the quality of the St-Pierre and Lacey projects, and eventually to determine if in some way their quality can be improved.

Quality control using DBAT
In terms of correlation values, Table 4 shows the high correlation values between the different calibration parameters as well as the number of automatic tie points with high correlation values in all the scenarios tested.In the cases where K 3 is calculated in both data sets, the results show a strong correlation between the radial distortion coefficients.The standard deviation values given by DBAT for the calibration parameters are also useful to assess the quality of the selfcalibration process.As regards to the automatic tie points, the St-Pierre data set shows that over a quarter of its tie points have a strong correlation of more than 95%.The Lacey data set only has 0.24% of its tie points which has a correlation of more than 95%.A strong correlation may mean that the parameters in the bundle adjustment were not solved correctly.This can therefore be an indication to the quality of the image matching and feature detection process in PS for this data set, or the presence of tie points with few rays at very small angles.Based on this information, a filtering of the automatic tie points for the St-Pierre data set could be performed in order to increase the quality of the bundle adjustment.Indeed, by performing this filtering in DBAT, the high correlations disappeared.Another example of metrics which can be derived from DBAT includes the standard deviations for the external orientation parameters.This may be useful in some cases to help users in sorting images which may worsen the results of the bundle adjustment.In the case of the data sets tested, these metrics are shown in histogram form in Figure 4.In the case of S4, a slight increase in rotational standard deviation can be seen for the images numbered around 100 and 120.Whereas for L1, a significant spike can be observed in the rotational standard deviation histogram for the images numbered around 320 and 330.This indicates that the orientation of these images were not precise, and may be a clue to reassess these images in PS and in the worst case suppress them from the project altogether.

CONCLUSIONS
This paper aims to reprocess terrestrial and UAV-based photogrammetric projects which have been processed in PS using DBAT.The purpose of such reprocessing is to recreate PS's results as closely as possible, and thereafter derive various statistics which can then be used as a quality control.In this paper, two data sets (one UAV and another terrestrial) have been tested.The experiments have shown that DBAT can be used to reprocess PS projects.DBAT has managed to perform well in terms of self-calibration as well as bundle adjustment, as shown by the experiments comparing DBAT and PS's calibration, CP, and GCP results.Although the algorithm behind PS remains hidden, this method of "dissection" permits users to have an idea on the results that they receive from PS as well as their quality.
The paper has also shown how DBAT was used as a quality control tool for PS.Correlations and exterior orientation standard deviations are only two among other metrics which may be interesting to PS users in order to assess their photogrammetric project.Based on this extra information, that was otherwise minimal in PS, the user may take decisions to perform modifications on the project as it fits the requirements of the project.
The use of black-box like software solutions presents a big advantage to many users as they eliminate processing parameters to the bare necessities.This fact, together with developments in photogrammetric and computer vision algorithms, has largely supported the growth of more commercial and user-friendly software.However, in some cases where precision and robustness is required, a black-box solution may not be enough.It is therefore important to have an open tool which enables users to look into their photogrammetric projects in more detail and to perform quality control and assessment.

Figure 2 .
Figure 1.(a) One image of the main façade in the St-Pierre data set.(b) An orthophoto of the façade.Red triangles denote GCPs.Green triangles denote CPs.(c) The site for the Lacey data set

Table 3
. In general DBAT has successfully reprocessed the PS projects in terms of camera calibration values.For the focal length, DBAT has managed to calculate values with an average difference against PS of 6.25 μm.As for the principal point offset, DBAT's results were virtually the same as PS's, within 3 significant numbers.

Table 2 .
The estimated parameters and standard deviations for four scenarios for the St-Pierre data set

Table 3 .
Results for the different scenarios, showing the σ 0 , reprojection error RMS, GCP error RMS, and CP error RMS

Table 4 .
Correlations in the processed projects in DBAT.For the automatic points, the value denotes the percentage of automatic tie points with high correlation values