Evaluation of Photogrammetry and Inclusion of Control Points: Significance for Infrastructure Monitoring

Structure from Motion (SfM)/Photogrammetry is a powerful mapping tool in extracting three-dimensional (3D) models from photographs. This method has been applied to a range of applications, including monitoring of infrastructure systems. This technique could potentially become a substitute, or at least a complement, for costlier approaches such as laser scanning for infrastructure monitoring. This study expands on previous investigations, which utilize photogrammetry point cloud data to measure failure mode behavior of a retaining wall model, emphasizing further robust spatial testing. In this study, a comparison of two commonly used photogrammetry software packages was implemented to assess the computing performance of the method and the significance of control points in this approach. The impact of control point selection, as part of the photogrammetric modeling processes, was also evaluated. Comparisons between the two software tools reveal similar performances in capturing quantitative changes of a retaining wall structure. Results also demonstrate that increasing the number of control points above a certain number does not, necessarily, increase 3D modeling accuracies, but, in some cases, their spatial distribution can be more critical. Furthermore, errors in model reproducibility, when compared with total station measurements, were found to be spatially correlated with the arrangement of control points.


Introduction
Structure from Motion (SfM)/Digital Photogrammetry provides the ability to build three-dimensional (3D) models from two-dimensional (2D) images [1,2]. Comparing 3D models built from images acquired at different times, can then be used to analyze deformation or displacement of an object over that time frame [3]. In this regard, imaging technologies can also enable readily-accessible processed spatial data for assessment of infrastructure systems [4][5][6][7].
Incorporating effective monitoring techniques for infrastructure systems is imperative when diagnosing performance and ensuring the structural integrity of systems. Moreover, ensuring structural safety for geotechnical applications is a critical component for transportation asset management systems [8,9]. The demand for an effective asset management protocol is growing for geotechnical infrastructure, such as retaining walls, for improving operations and enhancing safety at a minimal cost, while enabling conformity assessment results for structures [10][11][12][13].
With advancing technology, it is not a surprise that innovative, sensing-based techniques have been documented for the assessment of infrastructure systems, with comparable accuracies to traditional methods [9]. Furthermore, photogrammetry enables both a qualitative and a quantitative evaluation approach for monitoring changes or performances, and would be advantageous for an asset management system, easily data-logging its condition over time [9]. With researchers becoming more interested in using photogrammetry, comparative experimental studies between photogrammetry and Terrestrial Laser Scanning (TLS), for 3D reconstruction of civil and geotechnical infrastructure systems, have been documented [4][5][6][14][15][16][17][18][19][20][21][22][23]. While TLS has been implemented in areas of geotechnical engineering, because of improvements in scanning speed and mobility as well as capabilities for building management [24][25][26][27], few efforts emphasize photogrammetric techniques [9]. Photogrammetric 3D reconstruction provides a low-cost, comparative alternative to TLS, in terms of the point cloud density of the model and the geometrical accuracy [2,6,28].
Accuracies within the photogrammetry dense point cloud reconstruction can be impacted by different variables, such as the complexities in the scene or environment, and noise along imperfectly textured or flat surfaces [2,29]. The location and characteristics of the ground control points (GCP) could be reflective of the scene complexities. Result accuracy of photogrammetric methods was also found to be dependent on the functionality of the image processing algorithms [2]. The impact of the GCP should be further investigated, as this is often deemed as a limitation in the method [30][31][32]. Several researchers, who deployed unmanned aerial vehicle photogrammetric techniques for assessment, have noted concerns in control points impacts in rectifying images in larger scale applications [28,[33][34][35][36]. The GCP has an important impact in remote sensing rectification, including quality of selection, distribution and, even, quantity, although few research studies have collectively assessed these influences [34,35].
This study focuses more inclusively on control point evaluation in close-range, terrestrial photogrammetric applications for monitoring infrastructure systems. Enhancing close-range photogrammetry measurability improves both accuracy and the method's capability in further mobile mapping, or other non-traditional applications, as documented [37,38]. The authors previously highlighted photogrammetry's ability to quantitatively capture changes of a retaining wall model application within a few millimeters, and largely within 2-3 cm, when compared to a traditional method [9]. In this study, the significance of the control point configuration, for detecting spatial changes or failure modes of a simulated infrastructure system, is investigated further. Two commonly used photogrammetry software packages are utilized to illustrate the comparability in modeling and processing functionalities for capturing displacement changes in controlled structural failure simulations. In addition to spatial distribution, the capability of a close-range photogrammetry approach to detect precise spatial changes of a structure and its displacement measurability, with and without control points, is also evaluated.

Methods and Experimental Data Setup
To deploy this photogrammetric approach, a novel observation of displacement and spatial changes along the surface of a retaining wall model was performed [9]. A 2.43 m × 2.43 m (8 ft × 8 ft) retaining wall model, consisting of two 1.22 m × 2.43 m (4ft × 8ft) panels, A and B, was subjected to four common displacement failure modes [39][40][41]. These failure modes were categorized into five testing scenarios: forward translation of wall panel B (scenario H), forward tilted rotation of wall panel B (scenario I), deep seated overturned rotation of wall panel B (scenario J), forward translation of wall panel A (scenario K) and flexural bending of wall panel B (scenario L) [9]. A brief schematic of the top of the retaining wall panels, providing a plan view for each of the five simulated failure scenarios, is shown in Figure 1. Fourteen reference control points were placed in the test setup to infer ground location for georeferencing. Ten of these control points were placed on the individual retaining wall panels, with five on each panel, and the remaining four were placed on neighboring stationary objects positioned at different elevations and depths from the wall model. The stationary control points, those placed on static surfaces, assisted in the co-registration of the point clouds in a common coordinate system for the different scenarios [9]. For each of the five scenarios, two images were collected, one kneeling and one standing, to produce vertical parallax from at least ten different positions from a 25-foot perpendicular line of 97 sight [9]. A Nikon D5100 was used to collect images with a 55-mm focal length lens. At the end of 98 each testing scenario configuration, a Trimble S3 Robotic total station (TS) surveying device, of 0.91 99 mm precision, was used to capture the location of the control points under failure simulation [9].  For each of the five scenarios, two images were collected, one kneeling and one standing, to produce vertical parallax from at least ten different positions from a 25-foot perpendicular line of sight [9]. A Nikon D5100 was used to collect images with a 55-mm focal length lens. At the end of each testing scenario configuration, a Trimble S3 Robotic total station (TS) surveying device, of 0.91 mm precision, was used to capture the location of the control points under failure simulation [9]. Figure 2a shows the setup of the wall model, with reference control points (CP) identified on the panels, as well as the static locations, such as the neighboring building wall and concrete platform. The photos taken from each scenario were placed into two commonly-used, commercially-available software packages, Agisoft Photoscan and Pix4D.

127
For each of the testing scenarios, the control point measurements captured from the 128 photogrammetric models (shown in Figure 2b) were compared with the total station measurements, 129 and corresponding differences or errors in the measurements were produced.  The software uses the control points to best align all images in the dataset. The photogrammetry point cloud models consist of numerous sets of points in 3D, on the order of several million. This magnitude of points results in point densities of tens-to hundreds-of-thousands of points per m2 [9]. Figure 2b provides an illustration of the point cloud model, including the flagged referenced control points within the testing environment. Control points on the moving wall panels were compared between scenarios, but also with total station measurements of those same points. The displacements were calculated by comparing the positions of the control points along the surfaces, for different scenarios. The relative space coordinate locations of the control points were determined in each of these scenarios, which were compared to the ground truth total station measurements.
A focused evaluation, considering different amounts of control points, was also performed for each of the five failure mode scenarios, specifically using the Photoscan software's 3D model. Three sets of data, i.e., point cloud models, were created by aligning the collected images: (1) without any of the reference control points, (2) with two of the stationary control points and (3) with all four stationary control points. This strategy was implemented to consider the impact of control points for the model creation, varying from zero to four control points. For all three datasets, the locations of the wall panel control points, and two stationary points, were captured in the point cloud models and compared to the same total station measured locations. A rescaling adjustment factor was also included to present a best-fit evaluation, by knowing the distances of the control points and alignment of the points in space.

Results and Discussion
For each of the testing scenarios, the control point measurements captured from the photogrammetric models (shown in Figure 2b) were compared with the total station measurements, and corresponding differences or errors in the measurements were produced. Figure 3 reveals a comparison of the control point displacement errors for both the Photoscan and Pix4D software. The x-axis indicates the testing scenario evaluated and corresponding control points on the panel. Each of the scenarios are highlighted, along with the numbered control points (labeled from 1-5), along the particular panel's A or B surface. The maximum displacement error is 35 mm with an average mean error of all control points determined as 10.5 mm for Photoscan and 9.8 mm for Pix4D ( Figure 3). Scenario K presents the smallest errors (mean error of 4.1 mm and 1.5 mm, respectively, for Photoscan and Pix4D) across the five control point locations, when compared to the total station measurements. The largest errors are shown in scenario J, with a mean error of 14.3 mm and 23.9 mm, respectively, for Photoscan and Pix4D. On average, there is nearly a 10 mm difference between the two software programs within scenario J, which could be due to the complexity in modeling the imposed deep-seated failure on the wall within each software's design.      Scenarios H, I and L experience unique translational, rotational and bending simulation, with average errors ranging between the largest (scenario J) to the smallest (scenario K). When comparing the displacement values captured with the total station, and those with the Pix4D software, percent errors are as small as 0.6% for control point location a4, in scenario K, and even smaller in scenario H, for control point b2 (less than 0.0049%). Percent errors with Photoscan were slightly larger, with the smallest, comparing the total station and software's outputs, capturing a 3.4% error. These accuracy values are comparable with other researchers' documenting accuracies, within an average range of 3.3-20.9 mm, when deploying traditional TLS techniques for infrastructure systems [42,43]. Digital Image Correlation (DIC) was implemented for measuring in situ deformation of the retaining wall, obtaining an accuracy of 3 mm, which further advocated for development of 3D model inclusion [44].
Several photogrammetric, unmanned aerial applications indicated 0.03-0.25 m range accuracy in assessing digital surface models [29], and accuracies near 4 cm with an aerial multi stereo approach [45].
Further assessment of the data outputs was performed beyond the confined point locations with the wall's displacement contour plots. Figure 4 reveals the plane-perpendicular displacement, shown as a color range with equal value contours, between the two software's outputs for scenario L, the flexural bending failure model. The left panel remained stationary, and this is shown by the near-zero displacement. The right panel shows the simulated bending failure behavior displacements up to −6 cm (or 60 mm) for both Photoscan and Pix4D (Figure 4). The displacement contour along the surface plane shows little differences between the Photoscan (Figure 4a) and Pix4D (Figure 4b) software outputs. However, more detailed artifacts are visible along the Pix4D plane contour (Figure 4b), which is more evident on the right panel contour lines from imposed bending. in assessing digital surface models [29], and accuracies near 4 cm with an aerial multi stereo approach 154 [45].

155
Further assessment of the data outputs was performed beyond the confined point locations with 156 the wall's displacement contour plots. Figure 4    An analysis of the differences between the Photoscan and Pix4D results is presented in Figure 5 173 as a frequency polygon plot of the differences in the model outputs for the scenarios H through L.
Scenario H and J present the widest distribution of differences (ranging from zero to slightly past −15 mm). In this regard, Figure 5a also aligns with the Figure 3 output, with the greatest discrepancy 176 between the two software programs revealed in scenario H and J along the right panel or also referred 177 as panel B. The other scenarios reveal a close-to-zero difference between the two software programs 178 for the majority of the pixels on the model's image. Overall, the agreement between the results from 179 both software packages is high, and we can consider the differences to be less than 1 cm in general.  Distinctions in processing and modeling features, including noise rectifications between the two software programs, could have attributed to the variation in outputs. Nonetheless, this figure illustrates the feasibility of photogrammetric analysis for capturing changes along the surface of the infrastructure system, providing more descriptive data. This would enable a current, detailed performance or condition monitoring system, for infrastructure deformation, to effectively manage systems over a series of events (or time).
An analysis of the differences between the Photoscan and Pix4D results is presented in Figure 5 as a frequency polygon plot of the differences in the model outputs for the scenarios H through L. Scenario H and J present the widest distribution of differences (ranging from zero to slightly past −15 mm). In this regard, Figure 5a also aligns with the Figure 3 output, with the greatest discrepancy between the two software programs revealed in scenario H and J along the right panel or also referred as panel B. The other scenarios reveal a close-to-zero difference between the two software programs for the majority of the pixels on the model's image. Overall, the agreement between the results from both software packages is high, and we can consider the differences to be less than 1 cm in general.  These results revealed comparable, and, even more accurate measures, when compared with other photogrammetric aerial investigations. Bakula et al. (2016), incorporating photogrammetry for measuring infrastructures as a part of unmanned aerial systems, obtained proven accuracies within 10 cm [46]. A greater optimal accuracy, near 5 cm, was captured when an iterative control point method was deployed [46]. Udin and Ahmad (2014) also presented submeter accuracies using UAV photogrammetry for large-scale stream mapping [36]. Additionally, the impact of choosing different configurations and numbers of control points was investigated for three different point cloud models using zero, two, and four control points for georeferencing. Figure 6 shows a comparison of the control point distributions when evaluated with the Photoscan software. Error ranges are significantly larger for scenarios H, I and J, than for the rest of the scenarios. The results shown in Figure 6 suggests that increasing the amount of control points does not, necessarily, lead to smaller errors. Therefore, a minimal implementation of control points, depending on the project, could be adequate for accurate 3D modeling. As the control points used in georeferencing increased from two to four, the spatial position of the points was also increased. This was shown to have a different impact in the scenarios H-J and scenarios K-L. Figure 7a. presents the results for scenario I, with the control points on the wall panels themselves and with the stationary control points. Scenario I presents a unique illustration in which the control point's inclusion actually increased errors, particularly as they increased from two to four control points, unlike many of the other scenarios. In this scenario, as with others, the control points on each of the wall panels, and the four stationary points positioned on objects (two of which can be noted on the building wall and concrete platform in Figure 2b) near to the retaining wall model, at varying elevations, were used for referencing. The inclusion of the four control points produced larger errors.
This error, or change in results, may be related to the distribution of the additional control points incorporated in the analysis; the additional points were much farther away from the rest of the points and could, therefore, carry larger uncertainties, affecting the overall error values ( Figure 6). When scenario I was evaluated using only the control points on the wall panels, A and B (and not the two stationary control points, as previously), a smaller error range was obtained (see left side of Figure 7a). The values along the x-axis in Figure 7. indicate the number of control points used, and a large increase in the error is observed when the number of control points used changes from 9 to 10. On the other hand, scenario K doesn't show such an increase in the error with the different selection of control points (Figure 7b).
With distribution concerns, Gindraux et al. (2017) also assessed the accuracy of DSMs using UAV photogrammetry for glacier movement detection, in which the study emphasized accuracy decreases with closer distances to the GCP [29]. Liew et al. (2012) data highlighted that the center distribution of the control points presented the lowest total RMSE, lower than 5 pixels in aerial geometric distribution [35]. This may not, necessarily, be true, as our Figure 7b indicates reduction of error when control points beyond the center distribution were considered. Nonetheless, our results, and others overall, encourage uniformity in distributing control points.    To the contrary, when considering the amount of control points, researchers have presented varying findings. Shahbazi et al. (2015) suggests that, to reach the highest accuracy, a large number of well-distributed GCPs could be provided for a UAV-photogrammetry, DSM-georeferenced application of a gravel pit landscape, which may not, necessarily, be regarded as a minimum, but rather, a maximum amount [47]. Similarly, Tahar (2013), Rosnell and Hankavaara (2012) also promoted an increase in control points for improved DSM accuracy [33,48]. However, our study aligns with the notion of installing them at height variations, and even along different sides of the imaging zone, as Shahbazi documents, as well [46]. Gindraux et al. (2017), and other researchers, [49,50] also determined a common analysis, indicating accuracy increases with increased GCPs until a certain density is reached [29].
From this evaluation, a few considerations can be gathered. 3D modeling software is able to capture failure displacement, even with complex rotational movements with small errors. Test results from photogrammetry methods compared well with traditional measurements. The spatial distribution of control points may influence the quality of the results and could provide a considerable impact when detecting changes of an infrastructure system. Well-distributed control points are expected to produce better results, and minimal control points may enable an optimal density accuracy. Great precision between comparable modeling software further advocates the capability of photogrammetric processing for retaining wall performance monitoring.
The study also advocates the use of photogrammetry for, not only qualitative, but quantitative, data management of buildings and structures. Moreover, this study promotes the need for effective tools to manage risks and to ensure that performance-based requirements are met, as the research notes [13,26]. Maintaining performance data for transportation systems has been noted as a key element for transportation asset operation and planning [51], which photogrammetry methods could supplement and expand. The use of photogrammetry technologies for asset and, even, system-risk management, could be further considered to provide up-to-date measurements for a range of infrastructure corridors, whether transportation, civil or geotechnical. Additional investigation for obtaining accurate 3D models in photogrammetry software, considering testing setup or space configuration, anticipated displacement behavior, software specifications, camera settings, and increased or varied control point designs, should be pursued in future work.

Conclusions
Photogrammetry has the ability to generate 3D models of an object and enables a detailed description of changes in position or displacements over time. This study presents the evaluation of two widely-used photogrammetry software packages for capturing spatial and displacement changes of a retaining wall model. This study also shows the significance of the control points in evaluations and influences that may need to be considered for infrastructure monitoring. Implications of increasing control points for model reconstruction demonstrated that the inclusion of control points does not, necessarily, enhance model quality in georeferencing, but the position and distribution of the control points can influence model accuracy. It is recognized that spatial arrangement of control points needs considerable attention to adequately scale the 3D point cloud models and obtain optimal accuracy.