MONITORING DYNAMIC GLOBAL DEFLECTION OF A BRIDGE BY MONOCULAR DIGITAL PHOTOGRAPHY

This study uses MDP (monocular digital photography) to monitor the dynamic global deflection of a bridge with the PST-TBP (Photographing scale transformation-time baseline parallax) method in which the reference system set near the camera is perpendicular to the photographing direction and does not need parallel to the bridge plane. A SONY350 camera was used to shoot the bridge every two seconds when the excavator was moving on the bridge and produced ten image sequences. Results show that the PST-TBP method is effective in solving the problem of the photographing direction being perpendicular to the bridge plane in monitoring the bridge by MDP. The PST-TBP method can achieve sub-pixel matching accuracy (0.3 pixels). The maximal deflection of the bridge is 55.34 mm which is within the bridge’s allowed value of 75mm. The MDPS (monocular digital photography system) depicts deflection trends of the bridge in real time, which can warn the possible danger of the bridge in time. It provides key information to assess the bridge health on site and to study the dynamic global deformation mechanism of a bridge caused by dynamic vehicle load. MDP is expected to be applied to monitor the dynamic global deflection of a bridge.


INTRODUCTION
Bridge deflection is an important basis to evaluate bridge health as it directly reveals the rigidity, stability, bearing capability and earthquake resistance of the bridge [1].Bridge dynamic deflection can reflect the load impact coefficient and the internal force distribution of the structure [2].It is therefore important to monitor the dynamic deflection of a bridge with a traffic light as it confronts heavier vehicle dynamic loads than others [3].
At present, there are many methods to monitor bridge deflection such as the precise levelling, the total station, the measurement robot, the laser speckle, the photographic imaging, the inclinometer method, GPS (Global Positioning System), the inertial measurement, the tension line method, the pipe deflection monitoring method and the photogrammetric techniques.The precise levelling can monitor the static bridge deflection with high precision, it is however out of monitoring the dynamic bridge deflection [4].The total station can be used to monitor the deflection of a longspan bridge.It is however challenging to monitor the bridge dynamic deflection [5].Although the measurement robot can monitor the instantaneous coordinates of a single point, it only can monitor the short periodic deflection of the bridge as it continuously monitors the same point two times with a periodic [6].The laser speckle and the photographic imaging can monitor the dynamic deflection of a point on the bridge without monitoring the dynamic global deflection of a bridge [7,8].The inclinometer can monitor the bridge dynamic deflection with high precision, but it requires the installation shaft parallel to the bridge axis which is difficult to carry out in the field [9].GPS can be used to monitor a long-span bridge as it has a low accuracy in monitoring the dynamic deflection of a bridge [10].The inertial measurement has a high-resolution ability, but it is ineffective in low frequency [11].The tension line method cannot monitor the global dynamic deflection of a bridge as one tension line only can monitor the deflection of one point [12].The pipe method can monitor the static global deflection of the bridge, but it is not flexible in monitoring the dynamic deflection of a bridge yet [13].Photogrammetric techniques [14,15] can monitor the global deflection of a bridge, and it has advantage in non-contact measuring.But it cannot monitor the dynamic global deflection of a bridge as it uses two cameras or takes images from at least two different positions by one camera [16].
As such, a specific method is required to monitor the dynamic global deflection of a bridge and warn the possible danger of the bridge in time.MDP offers the potential to solve this problem [17,18].MDP, combining close range photogrammetric technique [19][20][21] with information technology [22,23], can monitor dynamic global deflection of a bridge and obtain image sequences of a bridge as it continuously monitors a bridge by a single digital camera.Although it has not been as popular in bridge structures as the photogrammetric techniques, many pioneering applications in this field have proved its increasing capability [24,25].
It is clear that in most of the previous studies MDP had been used in monitoring bridge deflection.However, they did not consider problems such as digital camera parallaxes caused by the environment and the photographing direction being un-perpendicular to the bridge plan.
The aim of this study is to propose the PST-TBP (photographing scale transformation-time baseline parallax) method in MDP to monitor the dynamic global deflection of a bridge to grasp the deflection characteristics of the bridge caused by vehicle dynamic load and MDPS (monocular digital photography system) is used to depict the deflection trend curves of the bridge in real time to assess bridge health on site and warn the possible danger.

THE CIVIL ENGINEERING JOURNAL 2-2018
where P1 and P2 are the weight matrices of image point observations and the reference point observations, respectively, ∆ is the correction matrix of the reference points, N0 is the coefficient matrix of ∆，M is the coefficient matrix of ∆, and  1 is the constant matrix of image point observations.
In Table 1, the calculation distances of U0-U2, U1-U3, and U2-U4 were obtained by the DLT method.Measurement distances of U0-U2, U1-U3, and U2-U4 were seen as precise values.The measurement errors were obtained by differencing the calculation distance with the corresponding measurement distance.The maximal measurement error of the digital camera is within 1mm which meets the accuracy requirements of deformation monitoring [27].

A principle of photographic scale transformation
The photographic scale of somewhere always changes along the photographing distance (from the position to the photography centre) [28,29].Figure 1 shows a schematic diagram of a CCD (Charge Coupled Device) camera capturing images at different photographing distances H3 and H4.H1 is the focal length of a CCD camera, H2 is the distance between the optical origin (o) and the front end of CCD camera, D1 on reference plane and D2 on object plane are the real-world length formed by the view field of the CCD camera at photographing distances H3 and H4 respectively, and N is the maximal pixel number in a horizontal scan line of an image plane, which is fixed and known as a priori irrelevant of the photographing distances.

Fig. 1 -Schematic diagram of photographing scale transformation
Based on Figure 1, the relationship between pixel counts and distances can be described by: In general, H3 and H4 are meter-sized, while H2 is centimetre-sized.Assume that H2 can be ignored when the camera is far from the bridge, Equation (3) can be expressed as: From Equation (4), we have: Assume that M1 and M2 are the photographing scale of the reference plane and the object plane, respectively.According to Equation (5), we have: Namely, where ∆ is the photographing scale transformation coefficient, and ∆ = 4 3

Photographing scale transformation-time baseline parallax method
Assume that there are no errors in the measurement, the horizontal and vertical displacements of deformation point on object plane based on the time baseline method are given by: where   and   are the horizontal and vertical deformation of a deformation point on the object plane, ∆  and ∆  are the horizontal and vertical displacements of the corresponding deformation point on the image plane, M is the photographic scale on the reference plane, and   is the photographing scale on the object plane.Note that ∆  and ∆  are with parallax errors.
However, a time baseline parallax method requires the photographing direction perpendicular to the bridge plane and the camera constantly when a single-digital camera is used to monitor the bridge.It is difficult to carry out [30].This study proposes the PST-TBP (photographing scale transformation-time baseline parallax) method to solve these problems.
The PST-TBP method consists of three steps.Firstly, the TBP method is used to get the displacements on the reference plane of deformation points.These displacements are with the parallax errors caused by the change of intrinsic and extrinsic parameters of a digital camera.Secondly, reference points are used to match a zero image with the successive images to eliminate the parallax errors.And the corrected displacements on the reference plane are obtained.Lastly, the real displacements on the object plane of deformation points are equal to the corrected displacements on the reference plane multiplied by the photographing scale transformation coefficient.The details are as follows: The first step, when a point on an object plane moves from A to B (Figure 2), its horizontal and vertical displacements on reference plane are given by: where ∆ and ∆ are the horizontal and vertical displacements on reference plane of a deformation point, ∆  and ∆  are the horizontal and vertical displacements on image plane of deformation point, and M is the photographing scale on the reference plane.Note that ∆  and ∆  are with parallax errors.
In the second step, some points are laid at a stable position around the camera to form a reference plane which is perpendicular to the photographing direction, and the parallax is therefore eliminated through differencing a zero image with the successive images based on reference plane, respectively.The reference plane (Figure 2) consists of six reference points labeled as C0-C5 (at least three reference points), and the reference plane equation can be expressed as : where (x, z) and (  ,   ) are the image plane coordinates and the image plane parallaxes of a reference point, respectively.(  ,   ) and (  ,   ) are the parallax coefficients in x and z-direction respectively.(c, d) are the constant parallax coefficients in x and z-direction, respectively.
After correcting the displacements on the image plane of a deformation point based on the parallaxes of the reference plane, the corrected displacements on the image plane of a deformation point is obtained: where (∆′  , ∆′  ) and (∆  , ∆  ) are the corrected displacements and the measured displacements on the image plane of a deformation point, respectively.Thus, we obtained the corrected deformation values on the reference plane of a deformation point: where(∆′, ∆′) are the corrected deformation values on the reference plane of a deformation point.
The last step is obtaining the corrected displacements on the object plane of a deformation point: where (∆  )′and (∆  )′ are the corrected displacements on the object plane of a deformation point.
In order to perform the PST-TBP method, a data processing toolkit has been developed in the environment of Microsoft Visual C++ 6.0.It allows synchronization of measuring pixel coordinates of deformation points and data processing.The monocular digital photography system consists of functions of solving data, storing data and displaying deformation curves, etc.The procedure of the toolkit is detailed in Figure 3. Before the test, we set the total station and three digital cameras on the specified position of the south side of the Xiaoqing river.To facilitate the stress analysis of this bridge and the field condition, six deformation point targets labelled as U0-U5 were set on the bridge's superstructure, and seven deformation point targets labelled as U6-U12 were evenly set on the bridge's major deck.Reference points labelled as C0-C1, forming the reference system, were set near the digital camera whose view is in Figure 4.The reference system, used to match a zero image with the successive image, is perpendicular to the photographing direction.July 16, 2009) In the bridge test, the SONY350 cameras were held constantly as much as possible.The cameras used in this test can capture the instantaneous deflection of a bridge in 1‰ second and shoot this bridge seven times in one second.The test process was described as follows:

Fig. 4 -Field test on the Fenghuangshan road bridge (taken on
(1) In the test, one excavator moved on the deck at a speed of 20km/h from the north to the south.The cameras were used to shoot the bridge to produce the zero images before the excavator on the bridge.
(2) The cameras were used to shoot the bridge every two seconds when the excavator was moving on the bridge.Finally, ten successive images were produced.
(3) The total station was used to obtain the spatial coordinates of reference points and deformation points after the test.
Table 2 shows the two-dimension coordinates of selected monitoring points and camera as the elevation can be ignored in the study.

Photographing scales of deformation points
Based on the photographing scale transformation, the photographing scales of U6, U9, and U12 can be expressed as: where  6  ， 9  and  12  are the photographing scales of U6, U9 and U12 after the photographing scale transformation，respectively;  0 is the photographing scale on the reference plane; ∆ 6 ，∆ 9  ∆ 12 are the photographing scale transformation coefficients of U6, U9, and U12, relative to the reference plane.∆ 6 ，∆ 9  ∆ 12 can be expressed as: (15) where OA, OB, OC, and OD are the photographing distances of the reference plane, U12, U9, and U6, relative to the camera.They are detailed in Figure 5.The photographing direction is perpendicular to the reference system, Line4, Line5, and Line6.Point A is the projection of Line 2 in reference system.Points B, C, and D are the intersections of Line 4 and Line 2, Line 5 and Line 2, Line 6 and Line 2, respectively.OB is the projection of Line1 in Line2 and OD is the projection of Line 3 in Line2.
Similarly, the approximate photographing scales of C0, C1, U7, U8, U10, and U11 were obtained.Table 3 shows the photographing scales of these reference points and deformation points on the bridge deck.

Measurement accuracy
In theory, reference points did not move during the test and their displacements were zero.However, the displacements of these reference points obtained by the PST-TBP method were not zero.These displacement values of these reference points can therefore be used to represent the measurement accuracy.Table 4 shows that the maximal error is 1.65mm in monitoring the reference points.
As image matching is the key to this monocular digital photography, we measured the image coordinates of these deformation points on the zero image times to assess its image matching accuracy.Table 5 shows that the maximal error was 1 pixel, and the minimal error was 0 pixels.The average errors of U6-U12 were 0.1 pixels, 0.1 pixels, 0 pixels, 0.3 pixels, 0 pixels, 0.3 pixels and 0.2 pixels, respectively.This method reached a sub-pixel Image matching accuracy.
Based on Table 3, it was obtained that the deflection errors of U6-U12 were 2.89 mm, 2.59 mm, 0 mm, 5.91 mm, 0 mm, 4.60 mm, 2.63 mm.The maximal and average deflection errors were 5.91 mm and 1.86 mm, respectively.

Analysis of bridge deflection trends
We calculated the pixel displacements (Table 6) and space displacements of some deformation points (Table 7).The positive and negative values (Tables 6 and 7) represent the deformation point moving up and down, respectively.The deformation points on the bridge deck (U6-U12) were chosen as the aim of this paper is to study the bridge deflection trends caused by vehicle dynamic load.Table 7 shows that in Test 6 deformation point U7 developed the maximal deflection-55.34 mm which was within the allowed value-75mm (the allowed value = the bridge span/800, where L = 60 m in this study).6 and 7) were also depicted in order to visually analyse the bridge deflection law caused by vehicle dynamic load.Figure 6 shows that the deflection of every position on the bridge's major structure was inelastic, all of which was within the bridge allowed value.Figure 7 shows that vehicle dynamic load results in the bridge moving down in spite of U7 moving up in test 5 and 7.In addition, the bridge global deflection curves fluctuated up and down like some sinusoidal-cosinusoidal curves.The bridge deflection was a parabola when the excavator moved to the bridge centre (Test 6).Especially, the deflection of every position on the bridge almost reached its maximal when the excavator moved to the bridge centre, which conforms to the deformation characteristics of the bridge caused by the external load.

Fig. 7 -Global deflection curves of the bridge
Note that it is impossible to monitor the bridge deflection with high accuracy when the camera is far from the bridge.But it can provide data support to study the deflection deformation law of the bridge caused by vehicle dynamic load.Although we cannot get the precise deflection of the bridge, the deflection trend of the bridge is also an important indicator to assess the bridge health.In the future, MDP & MR (monocular digital photography and measurement robot) would be a useful method to monitor the global deflection of the bridge as the measurement robot can monitor precise short periodic deflection of the bridge, and MDP can monitor the global deflection trend of the bridge.

CONCLUSION
This study used MDP based on the PST-TBP method to monitor the dynamic global deflection of a bridge.Before the bridge test, we first set SONY350 cameras in proper place and levelled them.Then, the reference system, formed by reference points, perpendicular to the photographing direction, was set near the selected camera.We produced a zero image using the cameras to shoot the bridge before the test and produced image sequences by shooting the bridge every two seconds when the excavator was moving on the bridge.Through the results, the following conclusions are obtained: x and z are image plane coordinates of deformation points without errors, X, Y and Z are the space coordinates of the correspondence deformation points, and Li (i =1,2,3,…,11) are the functions of the exterior and interior parameters of a digital camera.

Fig. 5 -
Fig. 5 -Illustration of the photographing scale transformation in the bridge test.
-Measurement errors of reference points/mm