Integrated calibration of a 3D attitude sensor in large-scale metrology

A novel calibration method is presented for a multi-sensor fusion system in large-scale metrology, which improves the calibration efficiency and reliability. The attitude sensor is composed of a pinhole prism, a converging lens, an area-array camera and a biaxial inclinometer. A mathematical model is established to determine its 3D attitude relative to a cooperative total station by using two vector observations from the imaging system and the inclinometer. There are two areas of unknown parameters in the measurement model that should be calibrated: the intrinsic parameters of the imaging model, and the transformation matrix between the camera and the inclinometer. An integrated calibration method using a three-axis rotary table and a total station is proposed. A single mounting position of the attitude sensor on the rotary table is sufficient to solve for all parameters of the measurement model. A correction technique for the reference laser beam of the total station is also presented to remove the need for accurate positioning of the sensor on the rotary table. Experimental verification has proved the practicality and accuracy of this calibration method. Results show that the mean deviations of attitude angles using the proposed method are less than 0.01°.


Introduction
The problem of accurate six-degree-of-freedom (6DOF) measurement for rigid body objects is important in large-scale equipment manufacturing and engineering in several domains including aircraft and spacecraft, ships, tunnel boring machines, and cranes [1][2][3][4]. Field measurements commonly require a 6DOF measurement system to have long range, high accuracy, to be robust in harsh environments and portable. For example, the underground guidance of tunnel boring machines requires measurement of real-time position and orientation to within 10 mm and 1 mrad (about 0.057°) at a range of more than 100 m.
The 6DOF of a rigid body in space are composed of 3D coordinates and 3D attitude. In large-scale metrology, 3D coordinate instruments are increasingly available, including visionbased measurement systems [5], laser trackers [6], indoor GPS [7], total stations [8], etc. Among these, the total station is a popular tool in industrial and external engineering environ ments, offering the advantages of flexibility of use, efficient measurement, long range (at least hundreds of meters), high accuracy (1 mm level) especially for angle measurement (1 arcsec level), and reasonable cost ($2000 to $20 000). However, the total station is a 3D coordinate measurement instrument, and it generally achieves 3D coordinate measurement by cooperation with an ordinary cube-corner prism. The total station cannot survey orientation directly, which limits its applications.

Integrated calibration of a 3D attitude sensor in large-scale metrology
Recently, an attitude sensor for atotal station (TS-attitude sensor) [9] has been proposed. The TS-attitude sensor is mainly composed of a pinhole prism, a converging lens, an area-array camera and a biaxial inclinometer. By cooperating with this single sensor, which replaces the traditional prism, a total station can realize 3D attitude measurement with high accuracy. For sufficient attitude measurement accuracy, a significant task for the TS-attitude sensor is its calibration before measurement, mainly involving two parts: the intrinsic param eters of the camera model and the relationship between the camera and the inclinometer. The calibration normally utilizes two calibration setups in these different steps [9]. However, the calibration needs a large number of manual operations and adjustments, which is unreliable, time-consuming and inefficient.
In the open literature on related calibration methods for attitude sensors for inertial systems [10] and star trackers [11], multi-position tests are common methods that mount the sensor unit on a precise multi-axis rotary table. Motivated by the aforementioned studies, this paper proposes a novel calibration method for a TS-attitude sensor which uses a three-axis rotary table and a total station as calibration tools to provide the reference standard. After fixing a TS-attitude sensor on the three-axis rotary table just once, all the unknown parameters in the measurement model can be calibrated using the different steps of multi-position tests.
The main contributions of this paper are as follows.
(1) Establishment of a precise measurement model for a TS-attitude sensor. (2) Proposal of a practical calibration method that enables different types of unknown parameters in the model to be calibrated in a single setup. (3) Presentation of a technique that makes the calibration simpler, with no need for complex installation of the sensor in the calibration platform.
This paper is organized as follows. In section 2, the configuration and the mathematical model of the attitude sensor are described. Section 3 introduces the integrated calibration method. The experimental validations are described in section 4. Finally, concluding remarks and a brief overview of further work are presented.

System composition and working principle
The attitude measurement system in this paper has two main units: a total station which acts as a base station, and a TS-attitude sensor which acts as a cooperative target. The function of the system is to determine the 6DOF (including 3D position and 3D attitude) of the target sensor with respect to the stationary total station. The system prototype and composition are illustrated in figure 1.
The TS-attitude sensor is very small and portable, being housed in an aluminum casing. The only exposed surface is the reflecting surface of the prism. The pinhole prism is adapted from a Leica standard circular prism whose vertex is cut in a small area to form a light channel. A converging lens and an area-array camera are designed behind the pinhole prism inside the sensor. The converging lens focuses at infinity and its aperture diaphragm is at the pinhole of the prism. During measurement, the total station emits an approximately collimated measuring beam on the pinhole prism. Most of the light is reflected back by the prism which enables cooperative tracking as usual. The portion of light which passes through the pinhole is converged by the subsequent lens and finally produces a concentrated light spot on the imaging plane of the camera. The center of gravity method [12] is used to find a sub-pixel centroid of the spot. The location of the spot on the image plane gives the direction of the laser source and hence the orientation of the sensor in two degrees of freedom. The remaining degree of freedom, the rotation about the axis of the laser beam, is determined using a biaxial inclinometer located inside the sensor. Further, by the fusion of the detecting information from the imaging system and the inclinometer, the TS-attitude sensor finally obtains its 3D attitude in the coordinate system of the total station.

Measurement model
The measurement model involves the following conventions.
• A single superscript to the left of a vector denotes its coordinate system; for example, a vector v T , is in frame T. • The use of both a subscript and superscript denotes a transformation between two coordinate systems; for example, a rotation matrix R T C means its transformation is from frame C to frame T. • A single subscript character following a matrix represents its state or sequence in a series of data; for example, a rotation matrix R i ( ) means it is the ith matrix in a series of matrices (i = 1, 2, …, n).
In the attitude measurement system, three different coordinate systems are defined: the total station frame (O T X T Y T Z Tcoordinate, frame T) as the reference coordinate system; the camera frame (O C X C Y C Z C -coordinate, frame C); and the inclinometer frame (O I X I Y I Z I -coordinate, frame I) in the sensor, as shown in figure 2.
A total station is a spherical coordinate measuring system; the origin O T is the starting point of the ranging measurement. Therefore, the unit vector of the measuring beam v T is obtained easily according to the measurement value of the total station. The Z T -axis is perpendicular with the horizontal plane and therefore the gravity vector in frame T is = g Since frame T is a left-handed coordinate system (inherent in the definition of a total station), all the coordinate systems in this paper are defined as left-handed systems.
Unlike a traditional imaging system, which maps 3D points in the world coordinate system to 2D image points, the imaging system in the sensor instead maps the beam direction v T . Taking all the components in the light path including the prism into account, the reflection center of the prism where each incident ray intersects is the real optical center. Therefore, in frame C, the reflection center of the prism is defined as the origin O C .
This imaging system conforms to the pinhole model with lens distortion. As shown in the camera frame definition part of figure 3, the unit vector v C denotes the vector of the laser beam in frame C, (u′, v′) denoting the real (distorted) image pixel coordinates, and (u, v) denoting ideal image pixel coordinates. According to the pinhole imaging model, the fourparameter model of a camera is given by (1) where (u 0 , v 0 ) denote the image coordinates of the camera's principal point, (x, y) are ideal normalized image coordinates, and a x and a y are scale factors. Then, optical lens distortion is added to the ideal coordinates to obtain a precise model and, in this paper, only the first two terms of radial distortion are considered [13]. Let (x′, y′) be the real (distorted) normalized image coordinates. Now where k 1 and k 2 are the coefficients of the radial distortion.
The imaging model has been established from laser beam vectors to the corresponding centroid of the spots. Also, as an inverse, the laser beam vector v C can also be obtained from the corresponding centroid of the real spot (u′, v′) according to equations (1)-(3) with all the camera parameters calibrated. The geometrical measurement model of a biaxial inclinometer is also shown in figure 2. Observed data (η, µ) indicate the inclined angles between the X I -axis, the Y I -axis of the inclinometer and the horizontal plane. This time, the unit vector of gravity direction in frame I is calculated by Now that the measurement model in each coordinate system has been described, suppose the attitude matrix from frame I to frame C is R C I , and that from frame C to frame T is R T C , which is the final goal. Since the gravity vector in frame C is obtained from that in frame I: Thus, the calculation of the attitude between the total station and the sensor has become a problem of attitude determination using two vector observations: the laser beam vector and the gravity vector. According to Wahba's study [14], this problem is described as finding the proper orthogonal matrix R T C that minimizes the least-squares loss function (7) where p 1 and p 2 are weighting coefficients of each observation vector. Their values are determined based on the angle measuring uncertainties of the imaging system and the inclinometer, respectively. This problem can be solved by the singular value decomposition (SVD) method that is presented by Markley in [15]. Define the matrix Then R T C is solved by the SVD method: where V and U are right and left singular matrices of B. The detailed derivation process is explained in [15]. According to the measurement model, the 3D attitude measurement accuracy of the TS-attitude sensor is mainly determined by the locating accuracy of the light spot on the image plane, the angular accuracy of the inclinometer, and the calibration accuracy.
The measurement uncertainty of the inclinometer is determined completely by its manufacturer. As for the locating accuracy of the light spot, many factors impact on it: the design and selection of the imaging system including the pixel resolution of the camera and the focal length of the converging lens; the quality of the light source (the laser beam from total station); the quality of the optical lens; the size of the pinhole (if the size is too large, the image is easily overexposed, which enlarges the spot locating uncertainty); and the spot locating algorithm. Except for the spot locating algorithm, other factors are all determined in the stage of designing and manufacturing. Once the TS-attitude sensor is produced, the measurement uncertainty influenced by these factors is fixed. Therefore, improving the calibration accuracy is a major approach to improving the measurement accuracy of the TS-attitude sensor after it is manufactured.

Calibration setups
In the measurement model of the system, two aspects should be calibrated in order to achieve measurement: the unknown intrinsic parameters a a u v k k , , , , ,  the calibration includes two steps, and finally they are integrated together.
The calibration setup is shown in figure 3(a). The three-axis rotary table used has four main components: the fixed base, the external frame which rotates around the horizontal-rotation axis with respect to the fixed base, the middle frame which rotates around the vertical-rotation axis with respect to the external frame, and the internal frame which rotates around the roll-rotation axis with respect to the middle frame. Notice that the rotary table used is levelled, which means that the horizontal-rotation axis is the same as the direction of gravity. The TS-attitude sensor is mounted on the internal frame of the rotary table with the prism pointing towards the stationary total station, and the measurement laser beam of the total station is locked to the prism.
A rotary table coordinate frame (O R X R Y R Z R -coordinate, frame R) is defined on the internal frame; its origin is on the rotary center, and in the default position, the Z R -axis, Y R -axis and X R -axis are the same as the horizontal-rotation, verticalrotation and roll-rotation axes. At this position, the default rotation is at horizontal angle α = 0°, vertical angle β = 0° and roll angle γ = 0°. A rotating sketch of all three axes is plotted in figures 3(b)-(d).

Calibration of the imaging model
The external frame rotates around the horizontal-rotation axis of the rotary table, and the middle frame rotates around the vertical-rotation axis to calibrate the imaging model. By changing multiple angular positions of these two frames, for the sensor on the rotary table in front of the reference laser beam from the total station, an accurate control field of the laser beam can be constructed.
The reflection center of the prism should be adjusted to the rotary center of the rotary table in order to make sure that during rotation, the laser beam vector from the total station to the prism remains stationary. However, in order to avoid blocking the light path by the rotary table, the TS-attitude sensor should be mounted offset from the rotary center and above the X R Y R -plane. As a result, the reflection center of the sensor varies in both vertical and horizontal directions relative to the fixed base during the rotating around all the axes, as shown in figure 4.
Since the sensor is tracked by the total station continuously in lock mode, by reading the deviation values of the horizontal and vertical angles from the total station, the reference laser beam can be corrected. Actually, because of the utilization of the total station, the demand for sensor installation on the rotary table is not strict. The installation of the total station should be the same height as the TS-attitude sensor in the default position of the rotary table. In this situation, the laser beam vector in frame R can be expressed as follows: where (ϕ, ψ) are yaw and pitch angles, respectively, which are illustrated in figure 4(a), and (δϕ, δψ) are the corresponding deviation angles. The value of ϕ is unknown, and since the rotary table and total station are both levelled, ψ = 0°. The values of (δϕ, δψ) are directly obtained by the total station as illustrated in figure 4. During the camera calibration, no motion around the rollrotation axis is maintained. In a position where the internal frame and the sensor are rotated by the horizontal angle α (i) (ith position) and vertical angle β ( j ) ( jth position), the relative attitude of frame R with respect to the fixed base or its default position has changed, and the laser beam vector in frame R has been transformed as  where the definition of R R i j , , 0 ( ) is the same as in equation (10).
Suppose the rotation matrix from frame R to frame C is R T C , then the laser beam vector in frame C is expressed as cos c os sin c os sin Equation (13) establishes the exact relationship between the laser beam vector and the camera coordinate system via the 2D rotary transformation of the rotary table. Equation (13) represents an extrinsic parameter model in which the parameters are called extrinsic parameters. Further, suppose This nonlinear minimization problem can be solved using optimization techniques such as the Levenberg-Marquardt algorithm [16]. Note that reasonable initial values should be given for a global optimal solution.

Calibration between the inclinometer and the rotary table
Since the rotation matrix between the coordinate systems of the camera and rotary table has been calibrated, by calibrating the transformation between the inclinometer and the rotary table, the final rotation R C I can be obtained. A practical and high-precision method is presented by changing multiple angular positions of the rotary table's middle frame and internal frame. Because the rotary table is levelled, in the default position the unit vector of the gravity direction in frame R is During calibration, no motion is maintained around the horizontal-rotation axis. In a position where the internal frame and the sensor are rotated by the vertical angle β ( j ) (jth position) and roll angle γ (k) (kth position), the relative attitude of frame R with respect to the fixed base or its default position changes, hence the gravity vector in frame R is transformed as where the definition of R R j k 0, , ( ) is the same as in equation (10). Substituting for R R j, k 0, ( ) by (10) in (15), and simplifying, results in Suppose the rotation matrix from frame R to frame I is R I R ; the gravity vector in frame I can be calculated from that in frame R as Since the gravity vector in frame I is measured directly according to equation (4), by substituting for g where V and U are right and left singular matrices of AD T .

Calibration summary
According to the results of sections 3.2 and 3.3, the final rotation between the coordinate systems of the inclinometer and the camera is obtained as and to summarize, the basic block diagram of the whole calibration process is shown in figure 5.

Experiments
The camera used in the TS-attitude sensor had 1280 × 1024 pixels, each of 5.3 µm × 5.3 µm in size. The converging lens designed had a 12 mm focal length, which determines the field of view to be approximately 33° × 27°. The inclinometer in the sensor had a measurement range of ±15° in both axes, and its measurement maximum permissible error was 0.005°. After the camera and the inclinometer were installed in the sensor, the X I -axis was nearly parallel to the Z C -axis, and the Y I -axis was nearly parallel to the X C -axis due to the precision of the machining. The three-axis rotary table used had an angle measurement accuracy of 1 arcsec. A Leica TS15 total station with angle measurement uncertainty of 1 arcsec was used for these experiments. The calibration platform is shown in figure 6.
In the ideal case, the values of the intrinsic parameters of the imaging model are as listed in table 1: the scale factor is the quotient of the designed focal length divided by the pixel size, the principal point values are half of the pixel numbers, and all distortion coefficients are set to zero. The value of the rotation matrix R C I in the ideal situation is also listed in table 1, and was calculated according to the installation relationship between the camera and the inclinometer.
During the calibration of imaging model, the horizontalrotation axis was rotated from −15° to 15°, and the verticalrotation axis from −12° to 12°, both with a 2° step, which results in 16 × 13 = 208 groups of calibration data. During the calibration for R I R , the vertical-rotation axis and rollrotation axis were both rotated from −10° to 10° with a 5° step, which results in 5 × 5 = 25 groups of calibration data. By the control of an automated program, the whole calibration process was completed in less than 30 min without human intervention.
The calibration results are also listed in table 1, and they are evidently different from the ideal values. In addition, the root mean square (rms) residual distance error of the laser spot center is 0.23 pixel, which corresponds to equivalent rms angle error of 0.0059° according to the imaging model. Also, the rms residual error of the inclinometer calibration is 0.0025°. These results verify the feasibility and effectiveness of the calibration approach.
The final performance of the TS-attitude sensor was also evaluated using the rotary table as an angle standard.
For a direct comparison, the attitude matrix of the sensor was parameterized by three Euler angles: yaw, pitch and roll, which can be evaluated directly by comparison with the horizontal-rotation, vertical-rotation and roll-rotation angles of the rotary table. The attitude measurement system was inspected at 27 different random positions of the rotary table with the proposed method, and measurement values were compared with the ground reference obtained directly from the rotary table. In addition, the experiment data were processed by the model with uncalibrated parameters which were set by ideal values. The results of the 3D angle deviation are plotted in figure 7, showing that by using the proposed method the mean deviation of these 27 sets of result is (0.0066°, 0.0018°, 0.0023°). As a comparison, figure 8 shows the mean deviation processed by the model without calibration as (0.0193°, 0.0119°, 0.0163°). Figure 7 shows that the variability in the yaw angle measurement is larger than that of the pitch and roll angles; this is because the yaw angle is almost completely determined by the imaging system whose measurement accuracy is lower than that of the inclinometer. This phenomenon is not clear in figure 8 because the effect  of uncalibrated parameters is much larger than the random effects in the measuring units.

Conclusions
This paper has proposed a novel calibration method for an accurate 3D attitude sensor to improve calibration efficiency and reliability. The configuration and measurement principle of the sensor were introduced, and a mathematical model was also developed to obtain the 3D attitude of the sensor with respect to the cooperative total station. The new calibration method enables all the unknown parameters in the measurement model (the intrinsic parameters of the imaging model, and the transformation matrix between the camera and the inclinometer) to be calibrated with a single placement on a three-axis rotary table without requiring an accurate location of the installed sensor. Therefore, the method is very significant for engineering applications.
In the practical experiment, by control of an automated program, the whole calibration process was completed in less than 30 min without human intervention. For validation, the performance of the calibrated sensor system was compared with that without calibration. The evaluation experiments showed that the mean deviation of yaw, pitch and roll angles using the proposed method are (0.0066°, 0.0018°, 0.0023°) compared with (0.0193°, 0.0119°, 0.0163°) using uncalibrated parameters.
The calibration method described here could be applied in other attitude measurement systems based on sensor fusion. Future work may benefit greatly by studying a more complicated camera model.

Acknowledgments
This work was funded by the National Natural Science Foundation of China (Grant No. 51405338, 51305297, 51225505) and the Natural Science Foundation of Tianjin (Grant No. 15JCQNJC04600). The support of the EPSRC through the Light Controlled Factory project, EP/K018124/1, is also acknowledged.