Phase-stepped fringe projection by rotation about the camera's perspective center

: A technique to produce phase steps in a fringe projection system for shape measurement is presented. Phase steps are produced by introducing relative rotation between the object and the fringe projection probe (comprising a projector and camera) about the camera’s perspective center. Relative motion of the object in the camera image can be compensated, because it is independent of the distance of the object from the camera, whilst the phase of the projected fringes is stepped due to the motion of the projector with respect to the object. The technique was validated with a static fringe projection system by moving an object on a coordinate measuring machine (CMM). The alternative approach, of rotating a lightweight and robust CMM-mounted fringe projection probe, is discussed. An experimental accuracy of approximately 1.5% of the projected fringe pitch was achieved, limited by the standard phase-stepping algorithms used rather than by the accuracy of the phase steps produced by the new technique.


Introduction
Precision measurement of manufactured parts commonly uses contact measurement methods. A probe mounted to a coordinate measuring machine (CMM) touches the surface of the part, recording the probe's tip position at each contact. Devices have been developed recently that continuously scan the probe tip across the surface, enabling quicker measurement for shapes that are easily parameterized such as a sphere or a plane. However, contact measurement remains slow and requires considerable user input for more general objects such as those with free-form surfaces. There is a requirement for a CMM-mounted optical probe that can provide the preliminary shape of a free-form surface to enable faster subsequent measurements with a contact probe. Full-field optical measurement is preferred to single point measurement (such as laser spot triangulation or time-of-flight) for speed of data acquisition. The optical probe should be compact, lightweight and contain no moving or delicate parts due to the high accelerations (up to 20ms −2 ) experienced in a CMM.
Fringe projection produces high resolution, full-field shape measurements. In phase stepping fringe projection, a series of sinusoidal patterns, with a phase step between each, is projected on to an object. A camera records a corresponding sequence of images. The phase of the pattern at each imaged point is calculated and converted to a 3D representation of the object's surface. The phase step is often produced by a relative motion of components within the projector, such as the light source and a mirror, or a slide and a projection lens [1]. Programmable data projectors can project different patterns to resolve step-height ambiguities, but are too heavy to be attached to a CMM and require considerable time to reach the thermal stability needed for accurate shape measurement [2]. Miniature data projectors do not currently have the resolution or brightness required and the high CMM accelerations could have an effect on the internal moving parts (digital micro-mirror device array) [3]. Kranz et al. presented an approach that avoided internal moving projector parts by translating the object with respect to a fixed camera and fringe projector [4]. The object was moved in a direction parallel to the camera image plane and perpendicular to the projected fringes by a distance equivalent to an integral number of pixels at the camera image plane. The image translation was compensated so that each pixel imaged the same point on the object before and after the motion, but the phase of the projected fringes at that point was stepped. This system relied on telecentric imaging optics, or a long standoff compared to the depth of field, so that the magnification remained constant throughout the measurement volume.
In this paper, we present a technique to produce phase steps by relative rotation between the fringe projection probe and the object, about the perspective center of the camera.
Compensation of the image motion is independent of the distance to the object, enabling a short standoff distance to be used. A single fringe pattern is projected, removing the need for any moving internal parts in the projector. The theory of the new technique is described in the next section. The technique is then validated experimentally, using a static fringe projection system and moving the object on a coordinate measuring machine (CMM) in an arc about the camera's perspective center. The technique is suitable for a lightweight, compact, rugged CMM-mounted probe that is rotated about the camera perspective centre in order to measure a static object. The design and implementation of such a probe is beyond the scope of the present paper, although a prototype system is discussed briefly at the end. Figure 1 shows a schematic of the fringe projection system. The camera and projector are described using the central perspective projection model [5]. The model consists of a bounded image plane that is normal to the optic axis and a "pinhole" or perspective center through which light is considered to pass in a straight line. Lens distortions are incorporated into the model through subsequent calibration of the system [6,7]. The global coordinate system X R X X = + , and R is the rotation matrix between the two coordinate systems. Although Fig. 1 shows the camera and projector coordinate Y-axes parallel to each other (i.e. coplanar optic axes) it should be emphasized that it is not a requirement: the mathematical development below does not require it and experimentally the axes are determined by the system calibration. The projector and camera optic axes intersect with included angle α, at standoff distance S from the camera's perspective center.

Theory
A point x p in the projector image plane is projected to ( ) , , and c p is the principle distance of the projector (the distance between the perspective center and the image plane along the optic axis). Without loss of generality, the projector is assumed to project fringes extending in the ˆP Y direction with period p in its 'image' plane. The phase at any point ( ) in the projector's pixel array is given by 2 x p π . Then the phase at the point P X due to the projected fringes is 2 .  The period of the fringes at this point normal to the optical axis of the projector is P P Z p c . Substituting for X P and Z P gives an expression for phase at a point X in the camera coordinate system.
Consider the rotation of the camera about its own perspective center. An object point that is imaged at the camera image plane at ( ) , , x x y c ′ = − (in camera coordinates) prior to the rotation, will be imaged after the rotation at ( ) 1 , , represents the original image point x in the coordinate system of the rotated camera and c is the principal distance of the camera. That is, the image point after rotation is the projection of the original image point on to the image plane of the rotated camera. Importantly, the relationship between new and old image coordinates for a given point is not dependent on the position of the point in object space. Therefore the image point from the rotated camera can be translated by 1 x x − to return it to its original position in the image plane, independent of the distance to the object point, i.e. the object's shape. The phase change due to the relative motion of the projector with respect to the object can be calculated using a small angle approximation. A point X following the rotation may be expressed in the rotated camera coordinate system relative to the camera perspective center as where ω is the small rotation angle and V is the unit vector in the direction of the effective motion of the object point relative to the camera. From Eq. (3), the phase change at X due to the motion of the projector is . 2ˆ.
Provided that the projector to object distance is large compared to the motion, Equation (6) separates into a number of factors: P P pZ c represents the period of the fringes at distance Z P from the projector; ˆˆ. P corresponds to the component of the motion contributing to the phase change; and X ω is the effective motion of the object relative to the camera. Equation (6) shows that the phase change varies linearly with the angle of rotation of the fringe projection probe, to a first order approximation. Errors introduced by higher order terms, i.e. non-linear phase steps, are discussed later. The magnitude of the phase change for a given angle of rotation varies with distance from the projector and is therefore not constant with position in the image. Standard algorithms which are valid for a range of phase step sizes, e.g. Carré's algorithm [8] and the algorithms described by Novak [9], should therefore be used.
The phase change is maximized if the rotation produces a motion that is perpendicular to the fringes, i.e. the camera is rotated about the Y-axis in Fig. 1. For the particular case where the camera and projector Y-axes are parallel, the phase change at the intersection of the optical axes from Eq. (6) simplifies to 2 cos P P c S pZ The term P P pZ c represents the period of the fringes at distance Z P from the projector and cos Sω α is the component of the object motion that contributes to the phase change. This simple expression enables the rotation angle required to achieve a required nominal phase step at distance S from the camera and Z P from the projector to be calculated.

Experimental demonstration
A shape measurement system was constructed, comprising a PointGrey FLEA-HIBW camera and a Hewlett-Packard VP6311 digital video projector. Calibration targets and test objects were fixed to a two-axis Renishaw Revo articulating head mounted on a three-axis Mitutoyo Crysta Apex 9106 CMM. The CMM was driven using a Renishaw UCC2 controller. Objects were moved in an arc about the perspective center of the camera to simulate rotation of the camera and projector about the camera's perspective center. There are several advantages in choosing to validate the new technique with a static fringe projection probe (fringe projector and camera) and by rotating the object about the camera's perspective center. Firstly, the technique could be demonstrated with a readily available data projector and camera prior to the construction of a suitably compact and light CMM-mounted probe. Secondly, the data projector could project phase-stepped fringes in the traditional way for a direct comparison with the new technique. Clearly, the data projector chosen would not be suitable for a compact and light CMM-mounted probe. The design and implementation of such a light probe is beyond the scope of the present paper, although a prototype system is discussed briefly later.
The optic axes of the projector and camera were nominally coplanar and intersected at an angle α of ~27°. The axes intersected approximately 230 mm from the perspective center of the camera. The exact angles and positions (including the off-axis projection typically found in a commercial data projector) do not need to be known for accurate shape measurement due to the calibration process described below. Straight parallel fringes with a sinusoidal intensity profile were projected into the 100×100×100 mm 3 measurement volume. The period of the projected fringes at the intersection of the camera and projector axes was approximately 4 mm (approximately 30 pixels in the camera image). For the system geometry used, a nominal phase step of 90° in the projected fringe pattern required a rotation of about the camera's perspective centre of 0.25°. The motion required by the CMM to effect this rotation about the camera's perspective centre comprised a rotation about the Revo head's centre of rotation of 0.25° and a translation in an arc of 300µm. The head and CMM motors are capable of positioning correct to 0.5 arc sec (or 1.4×10 −4 degrees) and 5µm respectively. From Eq. (6) the phase step variation throughout the measurement volume varied between 0.8 and 1.4 times the nominal value for a given rotation about the camera perspective center.
The set-up procedure first required the CMM to be calibrated. The standard procedure ISO 10360 has various stages and resulted in the accurate and traceable position of the tip of a calibrated touch probe attached to the two-axis head on the three-axis CMM to the order of 1µm in the measurement volume. Next, the intrinsic calibration parameters of the camera were calculated by taking multiple images of a calibration target (Edmund Optics part number NT46-250) [10]. Intrinsic parameters include the camera's principal distance and terms describing lens distortions. Finally the extrinsic parameters (position and orientation of the camera) were found in CMM coordinates by recording images of a custom-made calibrated touch probe (comprising a white spherical stylus ball and black shaft) placed at different positions throughout the camera's field of view. The image coordinates of the center of the probe tip and the known position of the probe tip in CMM coordinates were input to a least squares fit to determine the position of the perspective center and the orientation of the camera axes relative to the CMM coordinate system. The root mean square (rms) error between the measured position of the probe tip in the camera image and the position calculated from its CMM coordinates and the camera calibration parameters was 0.4 pixels for the measurement volume.

Rotation about the camera perspective center
The accuracy with which the position of points in the camera image could be calculated for a known rotation about the camera's perspective center was determined experimentally. Pairs of images were recorded, between which the CMM touch probe was moved along a circular arc centered at the camera's perspective center. The expected position of the tip in the second image of each pair was calculated from the initial tip position in the first image and the applied rotation about the camera's perspective center. Differences between the calculated and measured positions in the second image are indicated in Fig. 2, for a rotation of 1 degree (nominally equivalent to one fringe period) at one plane in the measurement volume. The arrows indicating the difference at each point are scaled to make them visible. The measurement was repeated at various planes within the measurement volume and an rms error of 0.1 pixels (maximum error of 0.5 pixels) was achieved.

Phase to height calibration
Fringe projection shape measurement systems commonly use a calibration between the optical phase measured and the object height, in order to accommodate the variation of the fringe period in the measurement volume (due to the relative geometry of the camera and projector) and the lens distortions in the projector [11]. A matte white plane calibration surface was attached to the CMM and placed in the measurement volume of the fringe projection system, approximately normal to the camera's optical axis. The optical phase on the plane was measured at each of 14 distances from the camera equally spaced through the measurement volume. The phase steps were introduced by rotating the plane in an arc about the camera Yaxis (i.e. about the camera perspective center) using the 5-axis CMM. At each distance of the calibration plane, data sets were recorded with four images (with a nominal phase step of 110° between each) and with five images (nominal phase step of 90°). Wrapped phase maps were calculated using Carré's algorithm [8] and Novak's algorithm A1 [9], for the four and five frame data sets, respectively. As mentioned previously, these algorithms were chosen because they are valid for a range of linear phase step values. The phase maps for each distance of the calibration plane were unwrapped from the 'zero order' projected fringe, in order to provide an absolute phase to height correspondence. It is straightforward to identify a zero order fringe, for example by projecting a spot or stripe with the same data projector. A cubic polynomial was fitted at each pixel relating the phase to height relative to a reference plane.
The shape measurement error for a plane surface positioned in the measurement volume is shown in Fig. 3. The plots show the error for a typical line across the surface and the caption shows the rms error for the whole plane (expressed as a fraction of the projected fringe period of 4 mm at the intersection of the projector and camera axes). Figure 3(a) and (b) show the result for the new technique for four frame (error 3.1% rms) and five frame data sets (error 1.6% rms), respectively. The error is due to Carré's and Novak's linear phase step invariant algorithms, rather than to the accuracy of the phase steps produced by the new technique. To demonstrate this claim, a data set was also recorded using phase stepped fringe patterns directly from the data projector (i.e. standard phase stepping with no rotation about the camera perspective center). Using Novak's algorithm to process these images produced an error of 1.5% rms, Fig. 3(c), which is comparable to the new technique. Using Bruning's five frame algorithm [12] to process the same images produced an error of 0.6%, Fig. 3(d), which is comparable to the performance of other phase-stepped fringe projection systems reported in the literature [13,14]. Hence the accuracy achieved is limited by the linear phase step invariant algorithms, rather than errors in the phase steps produced by the new technique. Fig. 3. Effect of phase stepping algorithm on the shape measurement errors for a plane surface. Graphs show a single line across the plane. Error for phase-stepped images recorded with rotation about the camera perspective center for (a) four frames (Carré's algorithm, rms 3.1% of the projected fringe period for the entire plane) and (b) five frames (Novak's algorithm, rms 1.6%). Error for standard phase-stepped images for (c) Novak's algorithm (rms 1.5%) and (d) Bruning's algorithm (rms 0.6%).

Object measurement
An object with a free-form surface shape with overall depth of approximately 40 mm was mounted to the CMM and measured using the new phase step technique. Five frames were recorded and used to calculate a wrapped phase map, Fig. 4(a), which was unwrapped from the zero order fringe. The unwrapped phase was converted to height above the reference plane using the phase to height calibration described above. The height for the horizontal line marked in Fig. 4(a) is shown in Fig. 4(b). Figure 4(b) also shows the height recorded for a data set using phase-stepped fringe patterns directly from the data projector (i.e. standard phase stepping with no rotation about the camera perspective center) and processed using Bruning's algorithm. This second measurement was used as the baseline to evaluate the performance of the new technique. The rms difference over an object area of approximately 100×100 mm 2 was 60µm, equivalent to approximately 1.5% of a fringe period

Discussion
The new technique to produce phase steps in a shape measurement system, by relative rotation between the fringe projection probe and the object about the perspective center of the camera, has been successfully demonstrated. The experiments used a static fringe projection probe and rotated the object on a coordinate measuring machine (CMM) about the camera's perspective center. Validating the new technique with a static fringe projector probe enabled a standard data projector to be used and the results to be compared directly with phase-stepped fringes projected in the traditional way. The intended implementation of the new technique is in, for example, a compact and light CMM-mounted fringe projection probe (projector and camera) that is rotated about the camera's perspective centre with the object stationary. Clearly a data projector such as the one used in our experiments would not be suitable for a CMM-mounted probe, for the reasons described in the Introduction. It is in a CMM-mounted probe, where traditional techniques to introduce phase steps are impractical, that the new technique will find application.
The new technique is well-suited for use in a compact, lightweight and robust fringe projection probe with no moving internal parts, mounted on a coordinate measuring machine [15]. A CMM-mounted probe demonstrator has been developed, comprising a commerciallyavailable camera and imaging lens, and a bespoke LED projector containing a fixed, amplitude mask with a sinusoidal pattern. The probe is compact (less than 100 mm in each dimension) and weighs less than 300g. It mounts directly to the Renishaw Revo articulating head using the standard connector used for interchangeable touch probes, and is powered through the same connector. Measurements from the optical probe, which is interchangeable with standard touch probes and is able to withstand the high accelerations encountered in a CMM, will enable fast preliminary measurement of a free-form surface to guide subsequent touch probe measurements. The optical probe projects a single fringe pattern which means that the zero order fringe for phase unwrapping cannot be easily identified by projecting a spot or stripe. Alternative approaches to resolve step height and hidden fringe ambiguities can be used.
The height error for the plane calibration surface was 1.6% of the projected fringe period and the phase difference for the free-form surface was 1.5%. It was shown that these errors arise due to the linear phase step invariant algorithms used. The errors do not originate from the size of the phase steps produced by the new technique, and so could be reduced by using an alternative phase step algorithm (probably involving more phase stepped images). However, the small increase in error introduced by the linear phase step invariant algorithms with only four or five phase steps is acceptable for our intended application.
The calibration plane and the free-form surface were both matte white. Metallic or non-Lambertian surfaces could result in a less accurate measurement from the new technique compared to standard phase stepping. In particular, speckle noise, inter-frame intensity variations and non-linear phase step errors could potentially affect the new technique in a different way to standard phase stepping. The effect of these error sources will be discussed in turn.

Speckle noise
Speckle, which arises from the roughness of the surface being measured, produces a multiplicative intensity fluctuation in the image of the projected fringe pattern. In standard phase stepping, speckle reduces the modulation amplitude at individual pixels in the camera image, and therefore increases the sensitivity of the measured phase to camera intensity noise [15,17]. However with the phase step created by relative motion of the object and camera the speckle noise could vary between phase-stepped images. The effect of speckle noise on the Carré and Novak algorithms was simulated for zero-mean Gaussian distributed multiplicative intensity noise, and the resulting rms error for a range of contrast values are shown in Fig.  5(a). The rms value was obtained over one period of phase for 1000 repetitions of the simulation. The mean phase error was zero.
For the experimental results presented, the speckle contrast for the surfaces and illumination used was less than 1% of the mean intensity, and the effect on the measurements presented was negligible.

Inter-frame intensity variation
For a non-Lambertian surface, the reflectance properties vary with viewing and illumination direction which could cause a change in the mean intensity and fringe modulation amplitude between successive images for the proposed technique [18,19]. The reflectance changes rapidly with illumination and observation angles close to the specular direction, but more slowly for other angles. It is therefore expected that phase measurement would be less accurate close to the specular direction, but that the reflectance would not change significantly with the small angular rotation required for the phase for diffusely reflecting angles away from the specular direction. The response of Carré's and Novak's algorithms to a linear variation in background intensity and fringe amplitude across the set of four or five phase stepped images was simulated. The mean error over one fringe period dominated the rms error, and is shown in Fig. 5(b). A straightforward way to reduce such errors with a CMMmounted probe would be to choose a measurement position away from the specular direction for the surface patch under inspection.
For the experimental results presented, the object surfaces were diffuse with no significant specular component, and the effect on the measurements was negligible.

Non-linear phase step error
Equation (5) showed that the phase step produced by the new technique is proportional to the angle of rotation (to a first order approximation) and that its magnitude varies with position in the measurement volume. In order to accommodate the variation in magnitude across the image, Carré's and Novak's linear phase step invariant algorithms were used. Non-linearity in the phase step will result in an error in the calculated phase for these two algorithms. By considering the second order approximation of Eq. (5) the non-linearity resulting from the rotation about the perspective center can be estimated. The non-linearity in phase step for a point X is proportional to the fringe width at that point. The factor of proportionality is at most of order 1/|X| for a practical system and varies with position within the measurement volume. Therefore the non-linear phase step error is small provided that the fringe pitch is small compared to the standoff distance. This condition is desirable for fringe projection systems because height resolution increases with decreasing fringe period. The response of Carré's and Novak's algorithms to non-linear phase steps was simulated. The mean error over one fringe period is shown in Fig. 5(c), and again dominated the rms error. For the measurements presented, the non-linear phase step error was calculated to be less than 1% and its contribution to the phase error was therefore insignificant. This was experimentally verified by the insignificant mean error in either the phase to height calibration (Fig. 3) or the object measurement (Fig. 4). If non-linearity were more significant, a calibration procedure or an alternative phase step algorithm could be considered.