Abstract
In the last two decades, the integration of a terrestrial laser scanner (TLS) and digital photogrammetry, besides other sensors integration, has received considerable attention for deformation monitoring of natural or man-made structures. Typically, a TLS is used for an area-based deformation analysis. A high-resolution digital camera may be attached on top of the TLS to increase the accuracy and completeness of deformation analysis by optimally combining points or line features extracted both from three-dimensional (3D) point clouds and captured images at different epochs of time. For this purpose, the external calibration parameters between the TLS and digital camera needs to be determined precisely. The camera calibration and internal TLS calibration are commonly carried out in advance in the laboratory environments. The focus of this research is to highly accurately and robustly estimate the external calibration parameters between the fused sensors using signalised target points. The observables are the image measurements, the 3D point clouds, and the horizontal angle reading of a TLS. In addition, laser tracker observations are used for the purpose of validation. The functional models are determined based on the space resection in photogrammetry using the collinearity condition equations, the 3D Helmert transformation and the constraint equation, which are solved in a rigorous bundle adjustment procedure. Three different adjustment procedures are developed and implemented: (1) an expectation maximization (EM) algorithm to solve a Gauss-Helmert model (GHM) with grouped t-distributed random deviations, (2) a novel EM algorithm to solve a corresponding quasi-Gauss-Markov model (qGMM) with t-distributed pseudo-misclosures, and (3) a classical least-squares procedure to solve the GHM with variance components and outlier removal. The comparison of the results demonstrates the precise, reliable, accurate and robust estimation of the parameters in particular by the second and third procedures in comparison to the first one. In addition, the results show that the second procedure is computationally more efficient than the other two.
Funding source: Bundesministerium für Wirtschaft und Energie
Award Identifier / Grant number: ZF4081803DB6
Funding statement: The research presented was partly carried out within the scope of the collaborative project “Spatio-temporal monitoring of bridge structures using low cost sensors” with ALLSAT GmbH, which was supported by the German Federal Ministry for Economic Affairs and Energy (BMWi) and the Central Innovation Programme for SMEs (Grant ZIM Kooperationsprojekt, ZF4081803DB6).
Acknowledgment
The authors would like to acknowledge Dr.-Ing. Manfred Wiggenhagen from the Institute of Photogrammetry and Geoinformation of Leibniz Universität Hannover for his invaluable advice and kind supports concerning the camera settings and calibration.
References
[1] A. Abellán, M. Jaboyedoff, T. Oppikofer, and J. M. Vilaplana (2009). Detection of millimetric deformation using a terrestrial laser scanner: experiment and application to a rockfall event. Natural Hazards and Earth System Sciences, 9(2), 365–372.10.5194/nhess-9-365-2009Search in Google Scholar
[2] F. Buill, M. A. Núñez-Andrés, N. Lantada, and A. Prades (2016). Comparison of photogrammetric techniques for rockfalls monitoring. In IOP Conference Series: Earth and Environmental Science, 44(4), 042023, IOP Publishing.10.1088/1755-1315/44/4/042023Search in Google Scholar
[3] A. Ebeling (2014). Ground-based deformation monitoring. Ph. D. thesis, University of Calgary.Search in Google Scholar
[4] D. Wujanz (2016). Terrestrial laser scanning for geodetic deformation monitoring. Ph. D. thesis, Technischen Universität Berlin.Search in Google Scholar
[5] M. Alba, L. Fregonese, F. Prandi, M. Scaioni, and P. Valgoi (2006). Structural monitoring of a large dam by terrestrial laser scanning. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 36(5), 6.Search in Google Scholar
[6] S. Gamse (2018). Dynamic modelling of displacements on an embankment dam using the Kalman filter. Journal of Spatial Science, 63(1), 3–21.10.1080/14498596.2017.1330711Search in Google Scholar
[7] W. Li and C. Wang (2011). GPS in the tailings dam deformation monitoring. Procedia Engineering, 26, 1648–1657.10.1016/j.proeng.2011.11.2350Search in Google Scholar
[8] H. Lõhmus, A. Ellmann, S. Märdla, and S. Idnurm (2017). Terrestrial laser scanning for the monitoring of bridge load tests–two case studies. Survey Review, 1–15.10.1080/00396265.2016.1266117Search in Google Scholar
[9] J. W. Lovse, W. F. Teskey, G. Lachapelle, and M. E. Cannon (1995). Dynamic deformation monitoring of tall structure using GPS technology. Journal of Surveying Engineering, 121(1), 35–40.10.1061/(ASCE)0733-9453(1995)121:1(35)Search in Google Scholar
[10] D. Reagan, A. Sabato, and C. Niezrecki (2017). Feasibility of using digital image correlation for unmanned aerial vehicle structural health monitoring of bridges. Structural Health Monitoring, 1475921717735326.10.1177/1475921717735326Search in Google Scholar
[11] M. Scaioni and J. Wang (2016). Technologies for Dam Deformation Measurement: Recent Trends and Future Challenges. In 3rd Joint Int. Symp. on Deformation Monitoring (JISDM 2016), 1–8.Search in Google Scholar
[12] D. Schneider (2006). Terrestrial laser scanning for area based deformation analysis of towers and water dams. In Proc. of 3rd IAG/12th FIG Symp., Baden, Austria, 22–24.Search in Google Scholar
[13] L. Truong-Hong, H. Falter, D. Lennon, and D. F. Laefer (2016). Framework for bridge inspection with laser scanning. In EASEC-14 Structural Engineering and Construction, Ho Chi Minh City, Vietnam, 6–8.Search in Google Scholar
[14] J.-A. Paffenholz, J. Huge, and U. Stenz (2018). Integration von Lasertracking und Laserscanning zur optimalen Bestimmung von lastinduzierten Gewölbeverformungen. Allgemeine Vermessungs-Nachrichten (avn), 125(4), 73–88.Search in Google Scholar
[15] N. Haala, R. Reulke, M. Thies, and T. Aschoff (2004). Combination of terrestrial laser scanning with high resolution panoramic images for investigations in forest applications and tree species recognition. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 34(5/W16).Search in Google Scholar
[16] J. A. Beraldin (2004). Integration of laser scanning and close-range photogrammetry—The last decade and beyond. In Proceedings of the XXth ISPRS Congress, Commission VII, Istanbul, Turkey, 972–983.Search in Google Scholar
[17] H. J. Przybilla (2006). Fusion of terrestrial laser scanning and digital photogrammetry. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 36, 5.Search in Google Scholar
[18] M. N. S. Sayyad (2016). Joint use and mutual control of terrestrial laser scans and digital images for accurate 3D measurements. Ph. D. thesis, Fachrichtung Geodäsie und Geoinformatik, Univ.Search in Google Scholar
[19] H. Yang, M. Omidalizarandi, X. Xu, and I. Neumann (2017). Terrestrial laser scanning technology for deformation monitoring and surface modeling of arch structures. Composite Structures, 169, 173–179.10.1016/j.compstruct.2016.10.095Search in Google Scholar
[20] J. Albert, H. G. Maas, A. Schade, and W. Schwarz (2002). Pilot studies on photogrammetric bridge deformation measurement. In Proceedings of the 2nd IAG Commission IV Symposium on Geodesy for Geotechnical and Structural Engineering, 21–24.Search in Google Scholar
[21] Ö. Avsar, D. Akca, and O. Altan (2014). Photogrammetric deformation monitoring of the second Bosphorus Bridge in Istanbul. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(5), 71.10.5194/isprsarchives-XL-5-71-2014Search in Google Scholar
[22] I. Detchev, A. Habib, and M. El-Badry (2011) Case study of beam deformation monitoring using conventional close range photogrammetry. In ASPRS 2011 Annual Conference, ASPRS, Milwaukee, Wisconsin, USA.Search in Google Scholar
[23] U. Hampel and H. G. Maas (2003). Application of digital photogrammetry for measuring deformation and cracks during load tests in civil engineering material testing. Optical 3-D Measurement Techniques VI, 2, 80–88.Search in Google Scholar
[24] H. G. Maas (1998). Photogrammetric techniques for deformation measurements on reservoir walls. In The Proceedings Of The IAG Symposium On Geodesy For Geotechnical And Structural Engineering, Eisenstadt, Austria, 319–324.Search in Google Scholar
[25] H. G. Maas and U. Hampel (2006). Photogrammetric techniques in civil engineering material testing and structure monitoring. Photogrammetric Engineering & Remote Sensing, 72(1), 39–45.10.14358/PERS.72.1.39Search in Google Scholar
[26] W. Niemeier, B. Riedel, C. Fraser, H. Neuss, R. Stratmann, and E. Ziem (2008). New digital crack monitoring system for measuring and documentation of width of cracks in concrete structures. In Proc. of 13th FIG Symp. on Deformation Measurement and Analysis and 14th IAG Symp. on Geodesy for Geotechnical and Structural Engineering, Lisbon, 12–15.Search in Google Scholar
[27] T. Whiteman, D. D. Lichti, and I. Chandler (2002). Measurement of deflections in concrete beams by close-range digital photogrammetry. In Proceedings of the Symposium on Geospatial Theory, Processing and Applications, 9–12.Search in Google Scholar
[28] P. J. Besl and N. D. McKay (1992). Method for registration of 3-D shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, 1611, 586–607, International Society for Optics and Photonics.10.1117/12.57955Search in Google Scholar
[29] A. Wendt and C. Dold (2005). Estimation of interior orientation and eccentricity parameters of a hybrid imaging and laser scanning sensor. Proceedings of the ISPRS Working Group, 5, 1682–1750.Search in Google Scholar
[30] D. D. Lichti, S. J. Gordon, and T. Tipdecho (2005). Error models and propagation in directly georeferenced terrestrial laser scanner networks. Journal of Surveying Engineering, 131(4), 135–142.10.1061/(ASCE)0733-9453(2005)131:4(135)Search in Google Scholar
[31] M. Zámečníková, H. Neuner, S. Pegritz, and R. Sonnleitner (2015). Investigation on the influence of the incidence angle on the reflectorless distance measurement of a terrestrial laser scanner. Vermessung & Geoinformation, 2(3), 208–218.Search in Google Scholar
[32] D. Wujanz, M. Burger, M. Mettenleiter, and F. Neitzel (2017). An intensity-based stochastic model for terrestrial laser scanners. ISPRS Journal of Photogrammetry and Remote Sensing, 125, 146–155.10.1016/j.isprsjprs.2016.12.006Search in Google Scholar
[33] X. Zhao, H. Alkhatib, B. Kargoll, and I. Neumann (2017). Statistical evaluation of the influence of the uncertainty budget on B-spline curve approximation. Journal of Applied Geodesy, 11(4), 215–230.10.1515/jag-2017-0018Search in Google Scholar
[34] D. Schneider and H. G. Maas (2007). Integrated bundle adjustment of terrestrial laser scanner data and image data with variance component estimation. The Photogrammetric Journal of Finland, 20, 5–15.Search in Google Scholar
[35] K. R. Koch (2014). Robust estimations for the nonlinear Gauss Helmert model by the expectation maximization algorithm. Journal of Geodesy, 88(3), 263–271.10.1007/s00190-013-0681-9Search in Google Scholar
[36] Q. Zhang and R. Pless (2004). Extrinsic calibration of a camera and laser range finder (improves camera calibration). In Intelligent Robots and Systems, 2004 (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on IEEE, 3, 2301–2306.Search in Google Scholar
[37] G. Pandey, J. McBride, S. Savarese, and R. Eustice (2010). Extrinsic calibration of a 3d laser scanner and an omnidirectional camera. IFAC Proceedings Volumes, 43(16), 336–341.10.3182/20100906-3-IT-2019.00059Search in Google Scholar
[38] R. Unnikrishnan and M. Hebert (2005). Fast extrinsic calibration of a laser rangefinder to a camera. Carnegie Mellon University.Search in Google Scholar
[39] D. D. Lichti, C. Kim, and S. Jamtsho (2010). An integrated bundle adjustment approach to range camera geometric self-calibration. ISPRS Journal of Photogrammetry and Remote Sensing, 65(4), 360–368.10.1016/j.isprsjprs.2010.04.002Search in Google Scholar
[40] G. Pandey, J. R. McBride, S. Savarese, and R. M. Eustice (2012). Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information. In Proceedings of the AAAI National Conference on Artificial Intelligence, 2054–2056.Search in Google Scholar
[41] M. Omidalizarandi and I. Neumann (2015). Comparison of target-and mutual information based calibration of terrestrial laser scanner and digital camera for deformation monitoring. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(1), 559–564.10.5194/isprsarchives-XL-1-W5-559-2015Search in Google Scholar
[42] M. Omidalizarandi, J.-A. Paffenholz, U. Stenz, and I. Neumann (2016). Highly accurate extrinsic calibration of terrestrial laser scanner and digital camera for structural monitoring applications. In: Proceedings of the 3rd Joint International Symposium on Deformation Monitoring (JISDM), Vienna, 30 March–1 April, 2016, p. 8, CD Proceedings.Search in Google Scholar
[43] F. M. Mirzaei, D. G. Kottas, and S. I. Roumeliotis (2012). 3D LIDAR—camera intrinsic and extrinsic calibration: Identifiability and analytical least-squares-based initialization. The International Journal of Robotics Research, 31(4), 452–467.10.1177/0278364911435689Search in Google Scholar
[44] E. K. Forkuo and B. King (2004). Automatic fusion of photogrammetric imagery and laser scanner point clouds. International Archives of Photogrammetry and Remote Sensing, 35(2004), 921–926.Search in Google Scholar
[45] M. A. Fischler and R. C. Bolles (1987). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Readings in Computer Vision, 726–740.10.1145/358669.358692Search in Google Scholar
[46] A. Habib, M. Ghanma, M. Morgan, and R. Al-Ruzouq (2005). Photogrammetric and LiDAR data registration using linear features. Photogrammetric Engineering & Remote Sensing, 71(6), 699–707.10.14358/PERS.71.6.699Search in Google Scholar
[47] M. Omidalizarandi, B. Kargoll, J. -A. Paffenholz, and I. Neumann (2018). Accurate vision-based displacement and vibration analysis of bridge structures by means of an image-assisted total station. Advances in Mechanical Engineering, 10(6), 1687814018780052.10.1177/1687814018780052Search in Google Scholar
[48] Zoller+Fröhlich GmbH (2007). Technical Data IMAGER 5006; Version 1.0.5; Zoller+Fröhlich GmbH: Wangen im Allgäu, Germany (in German).Search in Google Scholar
[49] U. Stenz, J. Hartmann, J.-A. Paffenholz, and I. Neumann (2017). A framework based on reference data with superordinate accuracy for the quality analysis of terrestrial laser scanning-based multi-sensor-systems. Sensors, 17(8), 1886.10.3390/s17081886Search in Google Scholar PubMed PubMed Central
[50] Photographylife. https://photographylife.com (Accessed 27 November 2018)..Search in Google Scholar
[51] M. Omidalizarandi, J.-A. Paffenholz, and I. Neumann. Automatic and accurate passive target centroid detection for applications in engineering geodesy. Survey Review, 1–16.10.1080/00396265.2018.1456001Search in Google Scholar
[52] Hexagon Metrology (2015). Leica Absolute Tracker AT960 Brochure. Available online: http://www.hexagonmi.com/products/laser-tracker-systems/leica-absolute-tracker-at960#loren (Accessed on 2015).Search in Google Scholar
[53] C. B. Duane (1971). Close-range camera calibration. Photogram. Eng. Remote Sens., 37, 855–866.Search in Google Scholar
[54] K. Al-Manasir and C. S. Fraser (2006). Registration of terrestrial laser scanner data using imagery. The Photogrammetric Record, 21(115), 255–268.10.1111/j.1477-9730.2006.00379.xSearch in Google Scholar
[55] T. Luhmann, S. Robson, S. Kyle, and J. Boehm (2015). Close-Range Photogrammetry and 3D Imaging. 2nd ed. de Gruyter, Berlin.Search in Google Scholar
[56] B. K. Horn (1987). Closed-form solution of absolute orientation using unit quaternions. JOSA A, 4(4), 629–642.10.1364/JOSAA.4.000629Search in Google Scholar
[57] K. R. Koch (2014). Outlier detection for the nonlinear Gauss Helmert model with variance components by the expectation maximization algorithm. Journal of Applied Geodesy, 8(3), 185–194.10.1515/jag-2014-0004Search in Google Scholar
[58] E. Parzen (1979). A density-quantile function perspective on robust estimation. In: L. Launer, G. N. Wilkinson (eds.) Robustness in Statistics, pp. 237–258, Academic Press.10.1016/B978-0-12-438150-6.50019-4Search in Google Scholar
[59] Z. Wiśniewski (2014). M-estimation with probabilistic models of geodetic observations. Journal of Geodesy, 88(10), 941–957.10.1007/s00190-014-0735-7Search in Google Scholar
[60] ISO/IEC (2008). JCGM 100:2008 Evaluation of measurement data — Guide to the expression of uncertainty in measurement. First edition 2008, corrected version 2010. International Organization for Standardization (ISO), Geneva.Search in Google Scholar
[61] K. D. Sommer, B. R. L. Siebert (2004). Praxisgerechtes Bestimmen der Messunsicherheit nach GUM. Technisches Messen, 71, 52–66.10.1524/teme.71.2.52.27068Search in Google Scholar
[62] K. R. Koch and B. Kargoll (2013). Expectation maximization algorithm for the variance-inflation model by applying the t-distribution. Journal of Applied Geodesy, 7(3), 217–225.10.1515/jag-2013-0007Search in Google Scholar
[63] K. Takai (2012). Constrained EM algorithm with projection method. Computational Statistics, 27, 701–714.10.1007/s00180-011-0285-xSearch in Google Scholar
[64] H. Alkhatib, B. Kargoll, and J.-A. Paffenholz (2017). Robust multivariate time series analysis in nonlinear models with autoregressive and t-distributed errors. In O. Valenzuela, F. Rojas, H. Pomares, I. Rojas (eds.), Proceedings ITISE 2017 – International work-conference on Time Series, 1, 23–36.Search in Google Scholar
[65] G. I. Hargreaves (2002). Interval Analysis in MATLAB. Numerical Analysis Report, No. 416, Manchester Centre for Computational Mathematics, The University of Manchester, ISSN 1360-1725.Search in Google Scholar
[66] E. M. Mikhail and F. E. Ackermann (1976). Observations and least squares. Dun-Donelly, New York.Search in Google Scholar
[67] W. Niemeier (2008). Ausgleichungsrechnung: eine Einführung für Studierende und Praktiker des Vermessungs- und Geoinformationswesens, 2nd ed. Walter de Gruyter, Berlin (in German).10.1515/9783110206784Search in Google Scholar
[68] A. R. Amiri-Simkooei and S. Jazaeri (2013). Data-snooping procedure applied to errors-in-variables models. Studia Geophysica et Geodaetica, 57(3), 426–441.10.1007/s11200-012-0474-2Search in Google Scholar
[69] P. J. G. Teunissen (2006). Testing Theory: an Introduction. Series on Mathematical Geodesy and Positioning. Delft University Press, Delft University of Technology, Delft, The Netherlands.Search in Google Scholar
[70] K. R. Koch (2013). Parameter estimation and hypothesis testing in linear models. 2nd ed. Springer Science and Business Media, Berlin, Germany.Search in Google Scholar
[71] F. Neitzel (2010). Ausgleichungsrechnung–Modellbildung, Auswertung, Qualitätsbeurteilung. Qualitätsmanagement geodätischer Mess- und Auswerteverfahren, Beiträge zum, 93, 95–127 (in German).Search in Google Scholar
[72] W. Baarda (1968). A testing procedure for use in geodetic networks. Delft, Kanaalweg 4, Rijkscommissie voor Geodesie, 1968, 1.10.54419/t8w4sgSearch in Google Scholar
[73] D. Schneider (2008). Geometrische und stochastische Modelle für die integrierte Auswertung terrestrischer Laserscannerdaten und photogrammetrischer Bilddaten. Ph. D. thesis, Deutsche Geodätische Kommission, Reihe C, Nr. 642, Technische Universität Dresden, Dresden, Germany. Available from: http://dgk.badw.de/fileadmin/docs/c-642.pdf.Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
