Bridge Structural Deformation Monitoring Using Digital Camera

Burgeoning off-the-selves Digital Single Lens Reflector (DSLR) cameras have been gaining attentions as a fast and affordable tool for conducting deformation monitoring of man-made engineering structures. When a sub millimetre of accuracy is sought, deliberate concerns of their usage must be considered since lingering systematic errors in the imaging process plaque such non metric cameras. This paper discusses a close range photogrammetric method to conduct structure deformation monitoring of the bridge using the digital DSLR camera. The bridge is located in Malang Municipality, East Java province, Indonesia. There are more than 100 images of the bridge’s concrete pillars were photographed using convergent photogrammetric network at distance variations between 5m to 30m long on each epoch. Then, the coordinates of around 550 captured retro-reflective markers attached on the pillars facade are calculated using self-calibrating bundle adjustment method. The coordinate differences of the markers from the two consecutive epochs are detected with a magnitude between 0.03 mm to 6 mm with a sub-millimetre precision measurement level. However, by using global congruency testing and a localization of deformation testing, it is confirmed that the bridge pillar’s structures are remain stable between those epochs.


Introduction
Deformation monitoring of the bridge's structures has been disseminated in wide range multi discipline literatures [1][2][3]. A state of the art of spatially driven information in localizing deformations of manmade bridge structures is categorized into two approaches namely: contact-based and non-contact-based methods [4]. The contact-based method is achieved by utilizing a single or multi sensor of measurement tools attached on the bridge [5,6]. For examples, Global Navigation Satellite Systems (GNSS) technology has been employed to monitor bridge deformation [7,8]. Albeit it offers some advantages such as weather proof continuous operability and a provision of instantaneous 3D absolute displacements [7], but its high observation noise limits an attainable precision displacement extraction [8]. Furthermore, multipath effects could downgrade precision since the GNSS receivers are stationed along the bridge [9].
On the other hand, the non-contact based method provides more advantages [10] such that it cannot destruct the bridge surface by equipment [11], it has high precision, high efficient and high flexibility characteristics [12], and it can be operated in real time [13]. The non-contact method usually utilizes optical centric devices such as laser beam [14], radar [15,16], acoustic [17], thermal model [18], and image-based measurements [19,20]. Furthermore, the image-based methods are mainly grouped into two approaches: computer vision approach [21] and photogrammetric restitution approach [22]. This paper discusses the close-range photogrammetric restitution approach for processing images to calculate the coordinates of observation points of retro reflective markers. Any 3D structure displacements can be analyzed through coordinates differences of the markers from different epochs of image acquisition. A workflow of the restitution starts from image registration process and followed by self-calibrating bundle adjustment process to produce 3D coordinates of the markers. Then, a set statistical tests are conducted to ascertain stability of some or all markers points by utilizing congruency testing and deformation localization testing. This procedures are elaborated as follows. Figure 1 depicts a general methodology to deformation monitoring of the bridge using two epoch analysis. The method is generally separate into two stages. The first stage is an image acquisition processes which aims to determine the object points coordinates. The next one is the deformation analysis itself which aims to test a stability of the point network. Those methods are elaborated as follows.

Self-Calibrating Bundle Adjustment
The photogrammetric restitution begins with by selecting two arbitrary overlapped images to determine its relative orientation parameters [23] and datum of a reference frame coordinate. Then, estimated orientation parameters of each processed images using a sequence of resection [24,25] and intersection [26] methods iteratively. Once the approximate values of each image's exterior orientation parameters and each marker's 3D coordinates of the chosen datum are obtained, these parameters are entered into the least squares adjustment process which known as self-calibrating bundle adjustment method rooted from photogrammetric collinearity condition [27]. The method generates a refined values of aforementioned parameters as well as the camera lens distortions parameters. Equation (1) compactly illustrates the method.
Equation (1) is a hyper matrix of the method's normal equation. Subscripts of i, j and p represent information pertaining to the i th image of m images and j th point of n object points (i.e. markers), while p contains the number of lens distortion parameters. The matrix P is formed by inversing the covariance matrix of the measured image points. The ̇,̈, ⃛ submatrices are symmetric, block diagonal, with each block on the diagonal referring to the particular exterior orientation parameters in ̇, the object point coordinates in ̈, and lens distortion parameters in ⃛ respectively. These matrices are formed by a summation process as illustrated in equation (2a).
Since the image measurements are assumed to be independent of each other, the contributions to the normal equations from each set of the collinearity equation can be summed. The total ̇ matrix is block diagonal, with 6 by 6 blocks on the diagonal, each referring to a separate image. Each ̇ is the sum of the ̇ submatrices, formed by the A1ij and Pij matrices from each set of the collinearity equations that refer to the image i. Also, the ̈ has 3 by 3 blocks on the diagonal, each referring to the coordinates of the individual point marker. Each ̈ is formed from the A2ij and Pij of the collinearity equations referring to the point marker j. The ̅ , ̃, ̂ submatrices in equation (2b) are generated based upon a point by point basis, not a summations. Their compositions are determined by which point marker occurs on which images.
In the application of deformation analysis, each epoch of image acquisition is adjusted separately as a free network bundle adjustment for obtaining markers' 3D coordinates as well as their covariant matrices. Since our prime focus is to identify and discern displacements of suspected point markers with highly confidence, a removal systematic image errors is deemed necessary. A viable solution until now is to utilizing the self-calibrating bundle adjustment method. Additional parameters p are introduced into the bundle adjustment method to model the behavior of the systematic error in the form of ⃛ submatrix. The estimation and solution of the additional parameters are determined from equation (1). In the last iteration, the covariance matrix of the solution ( = 0 2 − ) and the adjusted markers coordinates are then used to analyze the occurrence of possible point displacements.

Deformation analysis
Deformation analysis aims to detect the smallest possible displacements which are of the same order of magnitude as the precision of the measurement from which they are derived. The analysis process involve identification and quantification of the displacements, as well as ensuring that the measured IOP Publishing doi:10.1088/1755-1315/1051/1/012009 4 displacements were indeed not the result of random or systematic observations errors. Statistical testing of estimated displacements between two epochs is necessary to analyze whether significant movements have occurred. An acceptance of the test indicates that no significant displacement was occurred, otherwise the point movements were implied. The deformation analysis in this research is consisted of two interrelated phases: congruency test of the photogrammetric network between two epochs and localization of deformations test in Euclidian space and time.

Congruency Testing
The congruency test detects a stability and consistency of networks of a set of point markers between any two epochs. The set of points can either be all common points (i.e. a global congruency test) or few selected common points (i.e. a partial congruency test) suitable for datum definitions [28]. The testing procedure was initiated by performing the global congruency test. When the significant movements were indicated, the localization test is conducted then followed by some more partial congruency tests using reduced common points. These processes were repeated until the congruency test is pass and the remaining points were set as stable datum points. The global congruency test examines null hypothesis 0 (i.e. no significant displacements) of all points of markers over two epochs which can be formulated as: Where x1 , x2 are the vector of 3D coordinates of common point markers in both epochs in the same datum, d is a displacement vector with its cofactor matrix: Where Qx1 and Qx2 are the cofactor matrix of computed coordinates of x1 and x2 respectively. The test value is expressed as [29]: Where h is a rank of the cofactor matrix of of coordinate differences, i.e. (3n -7) for a 3D spatial network of n number of point markers. The common variance factor of ŝ0 2 is estimated from ̂0 2 = ( 1̂01 2 + 2̂02 2 )⁄ ; and = 1 + 2 Where r1 and r2 being the degrees of freedom, together with their corresponding variance factor in the estimation of x1 and x2. The test of is against the Fisher's distribution ℎ, ,1−∝ , and usual significant level chosen for the test is ∝= 0.05. If the is less than this critical value, the null hypothesis 0 is accepted. It means that the points of network at the second epoch must be congruent (i.e. same shapes) with that at the first one. On the other hand, if the null hypothesis of global congruency is rejected, it indicates a significant change of movements. Also, the + is the Moore-Penrose pseudo inverse of together with its inner constraint matrix such that [28]: = [ However, in the photogrammetric network whose coordinate points of x1 , x2 are calculated using a free network adjustments, the congruency test can be simplified into [30]: The next step is to identify point or points in the network of point markers whose displacements cause a change in shape.

Localization of Deformation Test
When the congruency test fails, it indicates significant displacement. A non-congruency of the network between the two epochs is encoded in the quadratic form of  which possible to measure the contribution of each point displacement di of each Ω . The point which has highest value of Ω is likely to be a significant displacement, and it needs to be removed from the network by using partitioning method [30]: Where di is the vector of eliminated point and dr are the retain datum points. S is implied similarity transformation when equation (9) is used to perform congruency test. Once the localization test was conducted, a verification of each stable point is confirmed using: If is less than 3, ,1−∝ , point i is considered as stable. The next section will discuss a process and result of the aforementioned general methodology.

Results and Discussion
A two series of photogrammetric campaigns were conducted to monitor a bridge located in Pandansari village, Malang Municipality, East Java province, Indonesia. The retro reflective markers attached on the bridge's concrete pillars facades were photographed by using a DSLR camera as seen in figure 2. Approximately, more than 500 markers were observed as object points of the deformation monitoring network ( figure 3). The self-calibrating bundle adjustment outlined in equation (1)  The three parameters of p are interior orientation parameters which consist of calibrated focal length (c), and the camera's coordinates of principal point (xp, yp). The next three parameters in table 1 are the lens radial distortion parameters (K1, K2, K3), and followed by the lens decentering distortion parameters (P1, P2) and the sensor camera's affinity (B1, B2). It can be noticed that a relatively insignificant perturbations of theses parameters between epochs still could degrade precisions of the obtained 3D coordinate of point markers.  The values of the 2 parameters comprise 555 point markers computed using free network datum on each epoch. Coordinates of these points and their cofactor matrices of 1 and 2 in equation (4) on each epoch are generated using the self-calibrating bundle adjustment outlined in equation (1). Rigorous statistical testing were conducted on each epoch to ensure that all measurements are free of systematic errors and meet reliability criteria for deformation measurements. Some of the point coordinates as well as its variance components presented in standard deviations are illustrated in table 2 and table 3. The sign of (⋮ ) that appear in all tables is indicated that not all data are presented. For a clarity of the discussion only few data are selected as an illustration purpose.   The differences of points of coordinates between two epochs of d as illustrated in figure 4 are not necessarily indicated as displacements. In order to check the integrity of the network between epochs, the congruency test was conducted using the Fisher's distribution with the significant level of ∝ = 0.05, and it gives the value of 2.61 at the maximum boundary. Table 5 shows the result of the test which implies that the value in equation (9) of each point displacement are none surpass the threshold. It indicated that there were no significant movements occurred between measurements. Hence, the localization of deformation test was not necessarily conducted. Although the averaged movement is about 0.534 mm with a magnitude between 0.026 mm -5.867 mm, the congruency of the network shape is still valid in all epochs.  Table 5. Global congruency test using Fisher's distribution according to equation (9). The sign of (⋮ ) means that not all data are presented.