3D reconstruction of laser projective point with projection invariant generated from five points on 2D target

Vision measurement on the basis of structured light plays a significant role in the optical inspection research. The 2D target fixed with a line laser projector is designed to realize the transformations among the world coordinate system, the camera coordinate system and the image coordinate system. The laser projective point and five non-collinear points that are randomly selected from the target are adopted to construct a projection invariant. The closed form solutions of the 3D laser points are solved by the homogeneous linear equations generated from the projection invariants. The optimization function is created by the parameterized re-projection errors of the laser points and the target points in the image coordinate system. Furthermore, the nonlinear optimization solutions of the world coordinates of the projection points, the camera parameters and the lens distortion coefficients are contributed by minimizing the optimization function. The accuracy of the 3D reconstruction is evaluated by comparing the displacements of the reconstructed laser points with the actual displacements. The effects of the image quantity, the lens distortion and the noises are investigated in the experiments, which demonstrate that the reconstruction approach is effective to contribute the accurate test in the measurement system.

In recent years, 3D reconstruction based on camera has attracted wide attention of researchers and has been applied in various research fields. E.g. Mian 1 extracts features from the different angular face images that are acquired by using the computer screen as a kind of coded illumination and reconstructs the 3D profile models by using a new efficient algorithm. Kim 2 presents a linear layered method including the affine recovery and the metric recovery. The camera calibration and 3D reconstruction are performed by utilizing the scene geometry.
Vision measurement on the basis of structured light is a very significant way of reconstructing 3D object surfaces due to non-contact test, simple operation, high efficiency and good accuracy 3-6 . Nguyen 7 presents a real-time measurement method of the object shape with the advantages of high speed and accuracy. A multithread parallel algorithm is adopted to deal with the acquired images by a camera and a programmable projector. A process is provided by Qin 8 to detect the terrestrial change at street level by combining point clouds and terrestrial pictures. It inspects the consistency of point clouds and stereo images by rectifying and re-projecting stereo portions of the terrestrial pictures. A method using structured light for the 3D shape reconstruction is proposed by Dipanda 9 for automatically obtaining information of the objects. The correspondence procedure in real-time is realized by the cell algorithm. Xu 10 describes the solution method of the planar structured light in the vision-based inspection by employing the Plücker matrices that shows a precise calibration of the planar structured light under the impact factors of the image quantity and the noise. Saeed 11 introduces a method of extracting the weld pool surface information from the images. The calibrated charge-coupled sensor is used to capture the mirrored laser beam to constitute the images. The depth of the weld pool surface can be calculated by utilizing the information of the charge-coupled device sensor and the location of the laser. Chen 12 proposes a method to reconstruct the surface of a wheel with the data from the vision sensor. The wheel surfaces are obtained from two structured-light-sensors which are calibrated by the iteration of the closest point. Li 13 outlines a method to improve the Fourier transform profilometry (FTP). According to the geometrical conditions, the map of the light phase is recovered by a gray-scale image of the fringe. Yun 14 presents a framework to improve the FTP method by retrieving the absolute phase pixel-by-pixel. Two images with different frequencies are adopted in the approach. The high-frequency phase is generated from the low-frequency phase. An accurate and effective camera calibration technology is particularly important for the 3D reconstructions of objects. Three kinds of the calibration systems for the camera are recorded in the previous works: the 3D calibration system [15][16][17] , the 2D planar calibration system [18][19][20][21] and the 1D linear calibration system [22][23][24] . 3D calibration system obtains the parameters of a camera by only one captured image. Meanwhile, high accuracy is achieved by the 3D calibration target. However, in order to extract enough feature points, the 3D target needs to be manufactured to a certain volume and the three faces of the volume must be accurately vertical to each other. Therefore, the process and transportation of the 3D target are time-consuming and inconvenient. Compared with the 3D target, the 2D planar calibration system has plenty of advantages. The 2D target calibration system is easy to be manufactured and suitable for the on-site calibration. In addition, the 2D planar calibration system provides sufficient calibration information in a convenient way. The 1D target is also easy to be made than the 2D target and 3D target, however, the measurement accuracy of 1D target is lower than the 2D calibration system and the 3D calibration system because of the lack of information. Thus, the 2D planar calibration system is adopted in this study.
A reconstruction method of the 3D laser projective point is proposed for the measurement adopting the line-structured-light. The 3D reconstruction method of the laser projective point is cataloged into four parts. Firstly, the laser line of the projector lies on the plane that is coincident with the 2D calibration target plane. Therefore, on the same plane, non-collinear points can be extracted to construct a projection invariant. Secondly, a laser projective point and five non-collinear points on the target are randomly extracted from the captured image. Similarly, the invariant with the same points is calculated in the camera coordinate system. For the same points, the invariant in the image coordinate system is equal to the one in the camera coordinate system. According to the idea above, homogeneous linear equations are constructed and the closed form solution is determined by the decompositions of singular values. Thirdly, the optimization function is designed to promote the reconstruction accuracy by minimizing the parameterized re-projection errors of the laser point and the target points. The 3D laser points are reconstructed in the world coordinate system. Finally, the effects of the image quantity, the lens distortion and noises are experimentally evaluated by comparing the differences between the reconstruction displacements of the laser points and the real displacements.
The following paper is outlined as follows: Section 2, Methods, present the reconstruction approach consisting of the closed form solution and optimization solution. Section 3, Results, provide the experimentation to verify the reconstruction method. The factors, lens distortion, noises and application cases are considered in the section. Section 4, Discussion, evaluates the reconstruction method according to the absolute and relative errors. Section 5, Summary, concludes the paper.

Methods
The reconstruction method of the projective point of the laser line includes two main components. The first component is to calibrate the camera, which has been accurately solved by Zhang 21 . The intrinsic and extrinsic parameters of a camera, including rotation matrix and translation vector, are provided by the calibration results. We consequently focus on the second component, to construct the projection invariant according to the laser projective point and five target points. The closed form solution of the laser projective point is achieved in this section and further improved by the optimization function. The reconstruction model of the laser projective point on an object is illustrated in Fig. 1. The reconstruction approach is interpreted by the block diagram in Fig. 2. The camera, the target and the image plane define the three coordinate systems (W) and O (I) -X (I) Y (I) , respectively. In the camera-laser-line-based measurement system, a 2D target is employed to perform the two aims, the camera calibration and the projective point reconstruction of a laser line. A laser projector is connected to the target. The laser line of the projector lies on the O (W) -X (W) Y (W) plane that is the identical one of the target. Considering the coplanarity of the laser projective point, the laser line, the target plane and the target feature points, there is a projective invariant derived from the laser projective point on the measured object and the five points on the target. The laser projective point and the five points on the target should be non-collinear to avoid the linear dependence. The projective invariant in the world coordinate system is identical to the projective invariant of the laser projective point and the five mapping target points in the image coordinate system. The laser projective points and the mapping target feature points are extracted by the Harris method 25 .  The laser line projector is fixed on a planar target attached with the world coordinate system. The laser line is located on the O (W) -X (W) Y (W) plane of the world coordinate system. Thus, the projective point of the laser line is also placed in the O (W) -X (W) Y (W) plane. In the projection geometry, the projective point x (I) in the image coordinate system and the projective line l (I) in the image coordinate system can be given by ref. 26 where x p (W) and x q (W) are two 2D points on the line l (W) . The projective line l (I) in the image coordinate system is given by ref. 26 where x m (I) and x n (I) are two 2D points on the line l (I) . A projection variable I i (I) generated from five 2D points on the 2D target and a laser projective point is constructed by are the five point coordinates on the 2D target in the image coordinate system, i is the group number of the five points on the target in an captured image.
Stacking Eqs (1)-(5), we have the projection invariant The projection invariant I i (I) in the image coordinate system is identical to the one in the world coordinate system, I i For different point combinations in the image, Eq. (7) is represented by a matrix form as where 0is the zero vector, The unknown projective point x 1 (W) in Eq. (8) is solved by the singular value decomposition 27 .
in the world coordinate system.
In order to transform the 3D projective point from the free world coordinate system to the stationary camera coordinate system, the 3D projective point B j (W) is left multiplied by the calibrated rotation matrix R j (WC) and the translation vector T j (WC) as ref. 26 where B j (C) is the 3D projective point of the laser line in the camera coordinate system. Considering the lens distortion coefficients k 1 and k 2 , the ideal image points and the distortion image points are expressed by ref. 21 The re-projected points should approach the ideal points in the image. Therefore, the differences of the re-projected points and the ideal points in the image coordinate system are contributed by minimizing the optimization function arg min K R ,

Results
The 3D reconstruction precision of the projective point of the laser line cannot be directly evaluated as there is no benchmark of the point in the camera coordinate system. However, the displacement between two projective points in the world coordinate system is identical to the one in the camera coordinate system. Thus, the displacement of two projective points is considered as the relative benchmark for the reconstruction. The verification principle of the method is illustrated in Fig. 3. The ruler with the checkerboard pattern is chosen as the benchmark of the test. The feature point coordinates on the benchmark ruler can be reconstructed by the laser projector system. The projective laser point B j (W) in the world coordinate system is achieved by the optimization process. It is transformed to the projective laser point B j (C) in the camera coordinate system. The reconstruction displacement error of two projective laser points is  Fig. 4. The blue points indicate the recognition results of the 2D target points. In addition, there are six green points in every image. Five of them are non-collinear points that are randomly selected in the 2D target. The other one is the laser projective point. Figure 5 shows the reconstruction errors of laser points of the nonlinear optimization solution (NOS) and the close form solution (CFS), without considering the distortion in the four groups of experiments. Figure 5     It is apparent from the experimental results in Fig. 5 that the error gradually increases with the increasing displacement with the same image quantity. The displacement of the proposed optimization method with lens  distortion is closer to the real displacement than the optimization method without considering the distortion. When the image number is 24, the reconstruction error of the displacement approaches its minimum. As the image number increases, the camera calibration is more accurate. Accordingly, the difference between the reconstructed displacement of the laser points and the actual displacement declines with the increasing image number. Therefore, in the following experiments, 24 images are adopted to calibrate the camera and analyze the test errors.
In the reconstruction of the displacement between the projection laser points, noise is an indispensable factor to be evaluated. The noise effects on the displacement reconstructions are investigated in the experiments. Four kinds of Gaussian noises are added to test the influences on the reconstruction errors. The experimental results of NOS and CFS without considering the lens distortion in the four groups are illustrated in Fig. 6. Four groups of experiments are performed to compare the effects of the noises and displacements. The statistical data are shown in Table 1. It is evident that the reconstruction error shows a stable growing trend with the increasing noise in the above experimental results. The errors of NOS are smaller than the errors of CFS under the same noise level. Therefore, NOS provides a higher accuracy. Under the same noise and the same optimization condition, the error values increase with the increasing displacement.
In the 3D reconstruction process of the displacement between the laser points, the lens distortion is the other important factor to be evaluated. The effects of the lens distortion on the displacement reconstructions are investigated in the experiments. The experimental results of NOS and CFS with regard to the distortion effects are shown in Fig. 7. Four groups of experiments with regard to the lens distortion are performed to compare the effects of the noises and displacements. The statistical results are listed in Table 2. In the experiments above, the results reflect that the error variances of CFS and NOS show the growing trend with the increasing noise level. The means of errors based on CFS and NOS rise as the displacement increases. The error variations of NOS method are closer to the real displacements than CFS method under the identical noise and displacement. The errors prove that NOS provides high accuracy for the 3D reconstructions of the laser projective points.
Taking all the factors into consideration, we get the conclusion that the average errors of CFS and NOS grow with the increase of noise. Moreover, the reconstruction results of NOS considering the lens distortion contribute a higher accuracy and are closer to the true values. In addition, the average errors of both CFS and NOS decline with the decreasing displacement.
Three application cases are provided to further show the applications of the reconstruction method. The wheelbase of a car model, the distance between two holes on the circuit board and the diameter of a coin are chosen as the objects to be measured. Every measured value is tested by the reconstruction method twenty times. The benchmarks of the tests are determined by the measurement outputs of a vernier caliper. The benchmark values are 64.00 mm, 41.07 mm and 19.08 mm, respectively. The measurement results are shown in Fig. 8. Four kinds of the reconstruction errors are considered in the three cases. The means of the reconstruction errors, CFS without respect to the lens distortion, are 1.10 mm, 0.61 mm and 0.40 mm in the three measurements. Then, the means of the reconstruction errors, CFS with respect to the lens distortion, are 0.95 mm, 0.53 mm and 0.35 mm in the three measurements. Furthermore, the means of the reconstruction errors, NOS without respect to the lens distortion, are 0.84 mm, 0.45 mm and 0.27 mm in the three measurements. However, the means of the reconstruction errors, NOS with respect to the lens distortion, are 0.68 mm, 0.40 mm and 0.20 mm, respectively. The NOS with respect to the lens distortion demostrates the best performance in the three applications. Its average error is 0.43 mm, which proves that the reconstruction method can be used in some non-contact measurements and inspections.

Discussion
A method to reconstruct the 3D laser point is proposed in the paper and realized by the project invariant generated from five points on a target plane. A laser projector is fixed on the 2D target in order to locate the laser point and the points of the 2D target on the same plane. The projective invariants are obtained from the laser projective point and the five non-collinear points that are randomly selected from the 2D target. The coordinates of the laser points are initially determined by the singular value decomposition. The optimization function is constructed by the differences between the re-projection points and the ideal points of the 2D target points and the laser points that are extracted in the image coordinate system. The 3D reconstructions of the laser projective points are finally obtained by minimizing the parameterized re-projection errors in the optimization function. The effects of the image quantity, the lens distortion and the displacement are analyzed by the experiments. In the first group of experiments, the error means are 0.30 mm, 0.40 mm, 0.54 mm and 0.68 mm on the condition that the displacement grows from 10 mm to 40 mm. In the second group of experiments, the error means are 0.29 mm, 0.39 mm, 0.49 mm and 0.64 mm. In the third group of experiments, the error means are 0.28 mm, 0.37 mm, 0.49 mm and 0.69 mm. In the fourth group of experiments, the error means are 0.28 mm, 0.41 mm, 0.55 mm and 0.75 mm. The error mean is 0.47 mm in the four groups of experiments. Considering the reasonable measurement scope in the experiments, it is necessary to evaluate the relative errors of the reconstruction method in the experiments. The relative errors of the first group are 3.03%, 2.02%, 1.82% and 1.69%. The relative errors of the second group are 2.94%, 1.96%, 1.63% and 1.61%. The relative errors of the third group are 2.77%, 1.86%, 1.65% and 1.72%. The relative errors of the fourth group are 2.82%, 2.06%, 1.82% and 1.88%. The largest relative error is 3.03%, which is less than 5% of the normal measurement instruments, in the four groups of experiments. The error variation grows with the increasing displacement. The projection invariant realizes the 3D reconstructions of the laser projective points. Moreover, the optimization function considering the lens distortion effectively promotes the reconstruction accuracy of the 3D laser projective points, which has the applicable potential to detect and reconstruct the 3D feature points or profile of objects.

Summary
In summary, a reconstruction method for a laser projective point is discussed by the invariant determined by the five points on a 2D target. The close form solution is achieved by the differences between the image-coordinate-invariants and world-coordinate-invariants. The nonlinear solution is conducted by minimizing the parameterized re-projection errors. The experiment results verify that the mean of the measurement errors is 0.47 mm and the largest measurement error is 0.75 mm. The mean of the relative errors is 2.08% and the largest relative error is 3.03%. It is proved that the reconstruction method can support the applications in more general systems of the non-contact measurements and inspections. In future works, the improvement method to enlarge the measurement scope should be investigated for the wide applications.