A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation

According to the American Society of Civil Engineers' (ASCE) report card for America's infrastructure, $2.2 trillion is needed to uplift the nation's infrastructure to 'good' condition. This budget needs to be spent over a five-year period. The current national infrastructure grade is 'D', which is close to failing [1]. There is an urgent need to develop effective approaches for the inspection and evaluation of these infrastructure systems. In addition, periodical inspections and targeted maintenance of structures will prolong their service life [2].

Despite the development of several non-destructive evaluation (NDE) techniques, visual inspection remains the predominant method currently used for the crack inspection of almost all infrastructure systems [3]. In this approach, the crack width, which provides important information about the degradation level of the component under inspection, is estimated by a trained inspector using a crack scale and a loupe. It is a subjective, tedious, time-consuming and, most importantly, highly qualitative approach. Due to these shortcomings, the need and importance of autonomous, or semi-autonomous, vision-based systems that can accurately quantify crack width have steadily increased.

The crack quantification methodology proposed in this study was developed for condition assessment of nuclear power plants. The outer layer of a nuclear power plant dome is made of concrete. This layer contains cracks that might not be due to structural deficiency (e.g., shear cracks); however, it is critical to monitor these tiny cracks to ensure that water and air cannot infiltrate into the underlying layers and cause material degradation (e.g., concrete corrosion), which leads to structural deficiencies in the long term.

In the nuclear power plant condition assessment project, the thickness threshold for crack rehabilitations was 0.25 mm (i.e., cracks thicker than 0.25 mm had to be sealed). On the other hand, due to safety constraints, the data acquisition system had to be placed 20 m away from the dome. Due to this constraint, NDE techniques that are not contact-less could not be used. Many non-contact NDE techniques had resolution issues. For instance, highly expensive laser scanners were not able to quantify cracks smaller than 1 mm from the distance of 20 m due to the large working distance. A vision-based crack detection algorithm, previously proposed by the authors, however, was capable of detecting cracks of 0.1 mm from the distance of 20 m. For this purpose, a super telephoto lens with 600 mm focal length was used. Having a system that is fully autonomous, fully contact-less, and can quantify crack thicknesses accurately and reliably, enhances the rehabilitation process, which is currently based on qualitative visual inspections, and improves the structural safety of the whole system.

Previously, the authors developed a vision-based crack detection approach that could segment the whole crack from its background [4]. In the same study, a fully contact-less crack quantification methodology was introduced. In this study, a new contact-less crack quantification approach, which can quantify crack width in unit length (e.g., a millimeter), is introduced and evaluated. The results clearly indicate that the new proposed methodology in this study is more robust than the previously developed one. In both methodologies, the 3D depth perception of the scene is used to quantify cracks. To the best of the authors' knowledge, these two methodologies are the only fully contact-less, fully autonomous crack width quantification approaches that do not need any scale attachment to the area under inspection, nor prior knowledge about the dimensions of the object under inspection. Furthermore, these methodologies are developed for field applications where several parameters, such as the camera–object distance, are uncontrollable. Here, the terminology 'cracks' is meant to refer to cracks that are dark, spatially narrow and elongated objects that have contrast with the background.

In the past twenty years, several image-based approaches have been developed for autonomous crack detection [4–24]; however, there are few studies that address the crack width quantification based on photogrammetry.

Dare et al [25] developed a crack detection methodology based on semiautomatic feature extraction, and used bilinear interpolation of pixel values to compute crack width. The measurement unit for this approach was pixels, not the unit length (e.g., a millimeter). Schutter [26] used a video microscope to monitor cracked structures. The small inspection range, and the use of a special hardware system, are among the limitations of this approach.

Ito et al [27] introduced an approach to quantify the area of fine concrete cracks. To enhance the measurement accuracy, a sub-pixel interpolation method based on the total brightness of a grayscale image was proposed. In order to estimate a crack area in millimeter squared units, a crack scale has to be attached to the surface under inspection. Yamaguchi and Hashimoto [22] improved this approach by utilizing a crack detection approach based on the percolation model and edge information. In the new approach, the crack width measurement was performed during the morphological thinning operation. This approach still requires a crack-scale attachment to estimate the crack width in millimeters. Furthermore, both of the above approaches require prior calibrations.

Benning et al [28] used photogrammetry to measure the deformations of reinforced concrete structures. A grid of circular targets is established on the testing surface. Up to three cameras capture images of the surface simultaneously. The relative distances between the centers of adjacent targets make it possible to monitor the evolution of cracks.

Sohn et al [29] developed a system to monitor crack growth. The focus of that study was to detect newly generated cracks using spatiotemporal images. The crack quantification approach was not evaluated. Furthermore, the orientation of the crack was not considered for accurate width measurement. This approach needs control targets (i.e., black–white cross marks attached on the specimen) for image registration. The experiments are conducted using a test specimen where the size of the specimen is known.

Chen et al [19] developed a semiautomatic crack measuring system using multitemporal images by utilizing the first derivative of a Gaussian filter. Quadratic curve-fitting was used to achieve sub-pixel accuracy. Their proposed system computed the crack width in millimeters in a controlled environment, assuming that the image resolution in the object space was available (i.e., the physical size of each pixel).

Chen and Hutchinson [30] developed an image-based framework for crack width quantification by computing the thickness, at each crack boundary pixel, as the shortest distance between the two edges of a given crack. In this approach, the image resolution was predetermined by controlling the image acquisition setup. To address this limitation for field applications, the use of a scale was proposed.

Lee et al [31] incorporated the thinned and boundary images of cracks to compute the crack width, at each centerline pixel, as the summation of the minimum distances between the centerline pixel and the two closest pixels on each of the crack edges. The results were promising, although this definition is not physically an accurate definition for crack width. Similarly to the current study, a pixel-to-millimeter conversion technique was used.

Zhu et al [24] developed a system to retrieve concrete crack properties, such as length, orientation and width, for automated post-earthquake structural condition assessment. These properties were reported as relative measurements related to the dimensions of the structural elements.

Jahanshahi et al [4] introduced a contact-less crack quantification approach to estimate crack thicknesses in unit length. In this approach, the orientation perpendicular to the crack centerline (i.e., thinned crack image) at each centerline pixel was defined as the crack width.

As can be seen from the above, there are few studies regarding the vision-based quantification of crack thicknesses. Furthermore, most of the previous studies used a scale, which was attached to the region under inspection, to convert the quantified crack thicknesses from pixels to millimeters. In this study, however, a fully autonomous, fully contact-less crack quantification approach is introduced. The need for scale attachment is compensated for by developing a pixel-to-millimeter approach inspired by the human vision system that incorporates several depth parameters. In the proposed approach, a segmented crack is correlated with a set of proposed strip kernels. For each centerline pixel, the minimum computed correlation value is divided by the width of the corresponding strip kernel to compute the effective crack thickness. In order to obtain more accurate results, an algorithm is proposed to compensate for perspective errors. Validation tests are performed to evaluate the capabilities, as well as the limitations, of the proposed methodology.

Section 2 introduces the proposed approach. The detailed mathematical representation and the implementation of the proposed approach are described in sections 2.1 and 2.2, respectively. Experimental results and their discussion are presented in section 3. Section 4 includes the conclusions and future work.

Suppose Ω represents the binary image of a segmented crack. Let ${\Omega }_{u,v}^{\prime}$ be a k × k subset of Ω that centers at (u,v). We define k × k strip kernels, Γ_γ, that are formed by vertically arranging n orientational kernels of γ^∘. γ is the angle between the orientational kernel and the horizontal direction. The correlation between ${\Omega }_{u,v}^{\prime}$ and Γ_γ represents the area of the crack that is bounded between the strip edges. Consequently, the thickness orientation, θ, at (u,v) is computed as the orientation of the strip kernel which minimizes the correlation value,

$$\begin{eqnarray} &&\theta (u,v)=\arg \mathop{\min }\limits _{\gamma }(\Omega _{u,v}^{\prime}\odot {\Gamma }_{\gamma }), \end{eqnarray} \tag{ 1 }$$

where ⊙ is the correlation operator.

If n is set to be small (e.g., 11), the crack thickness does not change dramatically in the small intersection region between ${\Omega }_{u,v}^{\prime}$ and Γ_γ, and consequently the areas bounded by the strips are all assumed to be trapezoids for different γ values. Furthermore, the minimum correlation value corresponds to the area of the trapezoid that is approximately a rectangle. The width of this rectangle is the projection of the n-pixel length on a line which is perpendicular to the thickness orientation (i.e., the tangent at the centerline pixel). This width is ncosθ(u,v) (see figure 2). Thus, the crack thickness at (u,v) is defined as

$$\begin{eqnarray} &&t(u,v)=\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n\cos \theta (u,v)}. \end{eqnarray} \tag{ 2 }$$

Zoom In Zoom Out Reset image size

The above approach is accurate if the camera orientation is perpendicular to the plane of the object under inspection (i.e., the region under inspection in the image is flat). In many applications, such as inspection of nuclear power plant domes, the area captured by an image can be considered as flat, without introducing too much error, since the local curvature is small (i.e., the diameter of the dome is large). Note that by using the structure from motion (SfM) problem (section 2.3), the lens distortion parameters for each camera is retrieved, and each view is undistorted prior to the quantification phase so as to improve the accuracy of the proposed approach. However, if the region of interest in the image has a significant curvature, such as a pipeline, the distorted pixels may yield poor quantification.

If the plane of the object under inspection is not perpendicular to the camera orientation (i.e., the projection surface and the object plane are not parallel), a perspective error will occur. In order to overcome the perspective error, the camera orientation vector and the normal vector of the object plane are needed. In this study, the camera orientation vector had already been retrieved using the SfM problem (section 2.3), and the normal plane could be computed by fitting a plane to the reconstructed 3D points, seen in the corresponding view, by excluding outliers using the random sample consensus (RANSAC) algorithm. In the RANSAC algorithm, a number of the 3D reconstructed points are randomly chosen and a plane is fitted to these points. Then, for each of the 3D reconstructed points, the error is defined as the distance between the point and the fitted plane. Points with errors greater than a threshold are detected as outliers. This procedure is repeated several times until the least amount of total error is computed, and the minimum number of outliers is detected.

For each centerline pixel, the numbers of pixels that are aligned with the corresponding thickness orientation are computed in the horizontal and vertical directions as follows:

$$\begin{eqnarray} &&{t}_{x}(u,v)=\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n\cos {\alpha }_{x}} \end{eqnarray} \tag{ 3 }$$

and

$$\begin{eqnarray} &&{t}_{y}(u,v)=\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n\cos {\alpha }_{y}}\tan \theta (u,v), \end{eqnarray} \tag{ 4 }$$

where t_x and t_y are the perspective-free components of the crack thicknesses (for each centerline pixel) in the x and y directions, α_x and α_y are the angles between the camera orientation vector and the fitted plane's normal vector in the x and y directions (see figure 3). In other words, α_x and α_y are the angles between the fitted plane and the image plane in the x and y directions, respectively. The first two rows of the camera orientation matrix, retrieved from SfM, represent the x any y axes on the image plane. To compute α_x, the angle between the normal vector of the fitted plane and the x axis (i.e., first row of the orientation matrix) is computed, which is the cosine inverse of the dot product of the two vectors divided by their magnitudes, and subtracted from 90°. α_y is computed similarly. For each centerline pixel, the resultant of the two perspective-free components is the crack thickness,

$$\begin{eqnarray} &&{t}_{p}(u,v)=\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n}\sqrt{\frac{{\tan }^{2}\theta (u,v)}{{\cos }^{2}{\alpha }_{y}}+\frac{1}{{\cos }^{2}{\alpha }_{x}}}. \end{eqnarray} \tag{ 5 }$$

Zoom In Zoom Out Reset image size

Note that a similar method can be used for the kernels that are formed by horizontal arrangement of the orientational kernels. In this case, the regular and perspective-free crack thicknesses are respectively

$$\begin{eqnarray} &&t(u,v)=\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n\sin \theta (u,v)} \end{eqnarray} \tag{ 6 }$$

and

$$\begin{eqnarray} {t}_{p}(u,v)&=&\frac{{\Omega }_{u,v}^{\prime}\odot {\Gamma }_{\theta (u,v)}}{n}\nonumber\\ &&\times \ \sqrt{\frac{1}{{\tan }^{2}\theta (u,v){\cos }^{2}{\alpha }_{y}}+\frac{1}{{\cos }^{2}{\alpha }_{x}}}. \end{eqnarray} \tag{ 7 }$$

In this study, the segmented crack was correlated with 35 kernels, where these kernels represented equally-incremented strips from 0° to 175°. For the 0°–45° and 135°–175° kernels, each column consisted of 11 non-zero values (see figure 4). For the 50°–130° kernels, each row consisted of 11 non-zero values. Figure 4 shows the strip kernels from 0° to 45°. The size of these kernels is 71 × 71 pixels.

Zoom In Zoom Out Reset image size

For each centerline pixel, obtained from morphological thinning, the strip kernel corresponding to the minimum correlation value represents the thickness orientation. Each correlation value is the area of the crack that is bounded between the strip edges. Since an 11-pixel length is relatively small, the crack thickness does not dramatically change in such a small region, and consequently the areas bounded by the strips are all assumed to be trapezoids. Furthermore, the minimum correlation value corresponds to the area of the trapezoid that is approximately a rectangle. The width of this rectangle is the projection of the 11-pixel length on a line which is perpendicular to the thickness orientation (i.e., the tangent at the centerline pixel). Finally, the crack thickness at each centerline pixel is estimated by dividing the corresponding minimum correlation value by this length. A user can interactively select a portion of a crack, and the proposed system will average the crack thicknesses for that region. This will improve the robustness of the system in the presence of noise.

Figure 5 shows an example of the proposed thickness quantification method described above. The white squares are crack pixels of a larger crack image. The blue (dark gray) and yellow (light gray) squares represent the strip kernel, centered at the red square (the dark pixel in the middle of the image), that has the minimum correlation value at the centerline pixel (red square). The kernel orientation in this example is 135°. The number of blue (dark gray) squares (i.e., 66) is the correlation value for the red centerline pixel (i.e., center pixel of this image). Consequently, the thickness for this centerline pixel is 66/(11 × cos45°) = 8.5 pixels. This thickness has to be scaled to obtain the thickness in unit length.

Zoom In Zoom Out Reset image size

Using a simple pinhole camera model, the crack size represented by n pixels can be converted to unit length by the following expression:

$$\begin{eqnarray} &&\mathit{CS}=\frac{\mathit{WD}}{\mathit{FL}}\times \frac{\mathit{SS}}{\mathit{SR}}\times n, \end{eqnarray} \tag{ 8 }$$

where CS (mm) is the crack thickness represented by n pixels in an image, WD (mm) is the working distance, FL (mm) is the camera focal length, SS (mm) is the camera sensor size and SR (pixels) is the camera sensor resolution. The geometric relation between some of these parameters is shown in figure 6.

Zoom In Zoom Out Reset image size

The sensor size and resolution are provided by the camera manufacturer. The approximate focal length can be extracted from the image exchangeable image file format (EXIF) file. The working distance can be measured using a tape; however, in order to develop a fully contact-less system, the working distance should be obtained by using a laser range finder, a stereo vision system [32], etc. In this study, the reconstructed 3D cloud and camera locations from the SfM problem are used to estimate the working distance and the focal length. To reach this goal, first, several overlapping images of the object are captured from different views. The SfM approach aims to optimize a 3D sparse point cloud and viewing parameters simultaneously from a set of geometrically matched keypoints taken from multiple views. The SfM system developed by Snavely et al [33] is used in this study. In this system, scale-invariant feature transform (SIFT) keypoints [34] are detected in each image and then matched between all pairs of images. The RANSAC algorithm [35] is used to exclude outliers. These matches are used to recover the focal length, camera center and orientation, and radial lens distortion parameters (two parameters corresponding to a fourth-order radial distortion model [36] are estimated) for each view, as well as the 3D structure of a scene. This huge optimization process is called bundle adjustment.

To obtain the camera–object distance, a plane is fitted to the 3D points seen in the view of interest. This can be carried out by using the RANSAC algorithm to exclude the outlier points. By retrieving the equation of the fitted plane, one can find the intersection between the camera orientation line passing through the camera center and the fitted plane. The distance between this intersection point and the camera center is computed as the relative working distance. Note that the SfM problem estimates the relative 3D point coordinates and camera locations. By knowing how much the camera center has moved between just two of the views, the computed working distance can be scaled to obtain the absolute working distance.