Profilometry without phase unwrapping using multi-frequency and four-step phase-shift sinusoidal fringe projection

A three-dimensional (3D) profilometry method without phase unwrapping is proposed. The key factors of the proposed profilometry are the use of composite projection of multi-frequency and four-step phase-shift sinusoidal fringes and its geometric analysis, which enable the proposed method to extract the depth information of even largely separated discontinuous objects as well as lumped continuous objects. In particular, the geometric analysis of the multi-frequency sinusoidal fringe projection identifies the shape and position of target objects in absolute coordinate system. In the paper, the depth extraction resolution of the proposed method is analyzed and experimental results are presented. 2009 Optical Society of America OCIS codes: (100.3175) Interferometric imaging; (110.6880) Three-dimensional image acquisition; (120.4120) Moiré techniques. References and links 1. E. Stoykova, J. Harizanova, and V. Sainov, “Pattern projection profilometry for 3D coordinates measurement of dynamic scenes,” in Three-Dimensional Television: Capture, Transmission, Display, H. M. Ozaktas and L. Onural, ed. (Springer, 2008), pp. 85-164. 2. J. Salvi, J. Pagès, and J. Batlle, “Pattern codification strategies in structured light systems,” Pattern Recogn. 37, 827-849 (2004). 3. D. M. Meadows, W. O. Johnson, and J. B. Allen, “Generation of surface contours by Moiré patterns,” Appl. Opt. 9, 942-947 (1970). 4. H. Takasaki, “Moiré topography,” Appl. Opt. 9, 1467-1472 (1970). 5. P. Picart, E. Moisson, and D. Mounier, “Twin sensitivity measurement by spatial multiplexing of digitally recorded holograms,” Appl. Opt. 42, 1947-1957 (2003). 6. E. A. Barbosa, E. A. Lima, M. R. R. Gesualdi, and M. Muramatsu, “Enhanced multiwavelength holographic profilometry by laser mode selection,” Opt. Eng. 46, 075601-075607 (2007). 7. G. Mauvoisin, F. Brémand, and A. Lagarde, “Three-dimensional shape reconstruction by phase-shifting shadow Moiré,” Appl. Opt. 33, 2163-2169 (1994). 8. X. F. Meng, L. Z. Cai, X. F. Xu, X. L. Yang, X. X. Shen, G. Y. Dong, and Y. R. Wang, “Two-step phaseshifting interferometry and its application in image encryption,” Opt. Lett. 31, 1414-1416 (2006). 9. I. Yamaguchi, “Phase-shifting digital holography,” in Digital Holography and Three-Dimensional Display, T.-C. Poon, ed. (Springer, 2007), pp. 145-172. 10. J. Hahn, H. Kim, S.-W. Cho, and B. Lee, “Phase-shifting interferometry with genetic algorithm-based twin image noise elimination,” Appl. Opt. 47, 4068-4076 (2008). 11. D. Kim and Y. J. Cho, “3-D surface profile measurement using an acousto-optic tunable filter based spectral phase shifting technique,” J. Opt. Soc. Korea 12, 281-287 (2008). 12. M. Chang and C. S. Ho, “Phase measuring profilometry using sinusoidal grating,” Exp. Mech. 33, 117-122 (1993). 13. R. Zheng, Y. Wang, X. Zhang, and Y. Song, “Two-dimensional phase-measuring profilometry,” Appl. Opt. 44, 954-958 (2005). 14. M. Takeda, Q. Gu, M. Kinoshita, H. Takai, and Y. Takahashi, “Frequency-multiplex Fourier-transform profilometry: a single-shot three-dimensional shape measurement of objects with large height discontinuities and/or surface isolations,” Appl. Opt. 36, 5347-5354 (1997). 15. W. Su and H. Liu, “Calibration-based two-frequency projected fringe profilometry: a robust, accurate, and single-shot measurement for objects with large depth discontinuities,” Opt. Express 14, 9178-9187 (2006). #109366 $15.00 USD Received 27 Mar 2009; revised 18 Apr 2009; accepted 23 Apr 2009; published 27 Apr 2009 (C) 2009 OSA 11 May 2009 / Vol. 17, No. 10 / OPTICS EXPRESS 7818 16. J.-L. Li, H.-J. Su, and X.-Y. Su, “Two-frequency grating used in phase-measuring profilometry,” Appl. Opt. 36, 277-280 (1997). 17. C. E. Towers, D. P. Towers, and J. D. C. Jones, “Optimum frequency selection in multifrequency interferometry,” Opt. Lett. 28, 887-889 (2003). 18. J. Li, L. G. Hassebrook, and C. Guan, “Optimized two-frequency phase-measuring-profilometry lightsensor temporal-noise sensitivity,” J. Opt. Soc. Am. A 20, 106-115 (2003). 19. J. Ryu, S. S. Hong, B. K. P. Horn, D. M. Freeman, and M. S. Mermelstein, “Multibeam interferometric illumination as the primary source of resolution in optical microscopy,” Appl. Phys. Lett. 88, 171112 (2006). 20. W.-J. Ryu, Y.-J. Kang, S.-H. Baik, and S.-J. Kang, “A study on the 3-D measurement by using digital projection Moiré method,” Opt. Int. J. Light Electron. Opt. 119, 453-458 (2007). 21. J. A. N. Buytaert and J. J. J. Dirckx, “Moiré profilometry using liquid crystals for projection and demodulation,” Opt. Express 16, 179-193 (2008).


Introduction
3D profilometry is a technology for extracting the position and depth information of 3D objects and has been researched intensively due to its importance in space recognition [1][2][3][4].In profilometry, specific patterns are designed and projected on the target object surfaces (this process is referred to as space coding), and then, from the images of the projected patterns on 3D object surfaces, the shapes and other information related to depth and position are analyzed (this process is referred to as space decoding).
In general, profilometry is comparable with coherent interferometry.Some coherent interferometric techniques use multi-frequency patterns synthesized with a few laser sources with different wavelengths or several different carrier frequencies in order to reduce the phase ambiguity that usually appears in single frequency interferometric techniques [5,6].One of the disadvantages of coherent interferometric techniques is that it is difficult to obtain a degree of freedom associated with changing or controlling carrier frequencies.On the contrary, the profilometry uses the composition of projectors with incoherent light sources and a charge-coupled device (CCD) camera.The fringe patterns for 3D profiling of target objects are directly generated by the projector and the projected images of the fringe patterns on target objects captured by the CCD camera are used as measurement fringe patterns.Thus it can be said that profilometry has an advantage in easily controlling and changing the spatial frequencies of fringe patterns in comparison with coherent interferometric techniques.
In some profilometry methods, phase-shifting technique is used for reducing background noise and solving phase ambiguity problems [7,8].In the use of the phase-shifting technique, at least three steps of phase-shifting are necessary, which gives two relative phase bases.However, four steps of phase-shifting are usually preferred due to accuracy and reliability [9][10][11].Thus, in this paper, we use four-step phase shifting method for synthesizing multifrequency composite fringe patterns and performing the depth extraction.
On the other hand, conventional phase-shifting profilometry methods use a single spatial frequency fringe pattern [12,13].The spatial frequency is smaller than the maximum frequency that can be recorded on a CCD camera.In this case, phase unwrapping is indispensable for depth extraction of target objects [14].Also, single frequency profilometry has a limitation in the dynamic range of depth extraction.If separations or discontinuities in target objects are larger than the values that can be managed by the spatial frequency of used fringe pattern, the depth extraction of the objects cannot be successful.Also if the spatial frequency is much higher than that of the prominence of target object surface, it is hard to extract fine depth map of the target object surface.
Therefore, profilometry techniques using multi-frequency fringe patterns have been actively researched.In the multi-frequency profilometry techniques, both relatively lower and relatively higher frequency fringes are used to analyze relatively large separations or discontinuities and detail depth map, respectively [15][16][17][18].
However, even in previous multi-frequency profilometry techniques, the need of phase unwrapping processes still remains although the multi-frequency profilometry technique was successful in some applications for profiling surface morphology without phase unwrapping [19].
In this paper, a novel 3D profilometry method without phase unwrapping is proposed.The key factors of the proposed profilometry method are the use of composite projection of multi-frequency and four-step phase-shift sinusoidal fringe patterns and its geometrical analysis.In most conventional profilometry techniques, only relative depth between separate objects can be measured [20,21].However, without phase unwrapping process, the proposed method provides the depth and position information of target objects in the absolute coordinate system with the geometric analysis regarding the positions and field-of-views of a projector and a CCD camera.
In Section 2, the proposed system geometry for depth extraction in the absolute coordinate system is described.The resolution of depth extraction of the system with a projector and a CCD camera with finite resolution is analyzed.In Section 3, the proposed depth extraction method is elucidated.The construction of multi-frequency and four-step phase-shift sinusoidal fringe patterns and its use for practical depth extraction are addressed in detail.In Section 4, experimental results are presented.The feasibility of extracting the depth of multiple objects with large discontinuity is shown.In Section 5, concluding remarks are provided.

System geometry for depth extraction in the absolute coordinate system
In this section, system geometry for depth extraction in the absolute coordinate system is addressed.Figure 1 shows the geometry of the profilometry system represented in twodimensional x-z coordinate system.Basically the profilometry system is composed of a projector, a CCD camera and target objects.Sinusoidal fringe pattern with a single frequency is projected by the projector to the target objects.And the CCD camera captures and saves the image of the target objects with fringe pattern on their surfaces.This projection and capture process is repeated for several different frequency fringe patterns.The meaning and objective of this measurement process will be detailed in next section.The center of imaging lenses of the projector and the CCD camera are referred to as the reference points of the projector and CCD camera, respectively.In Fig. 1, the reference points of the projector and CCD camera are located on ( ) The projector and the CCD camera have finite angular fields of view.Let the angular field of views of the projector and the CCD camera be denoted by PRJ Θ and CCD Θ , respectively.As indicated in Fig. 1, it is not necessary that the image planes of the projector and the CCD camera should be placed on the same plane.The normal vectors of the image planes of the projector and the CCD camera are respectively rotated by specific angles so that the regions inside the fields of view of the projector and the CCD camera have a intersection region and this intersection region covers the target objects to be observed.In  ) A point ( ) ′ ′ in the rotated coordinate system is transformed into ( ) , x z in the absolute coordinate system by rotation transform by the tile angle CCD ϕ as cos sin sin cos With all geometric parameters, including the reference points ( ) ( ) , known, the position ( ) x z can be calculated.Therefore, OBJ x and OBJ z have the relation as From Eqs. (3a) and (3b), OBJ x and OBJ z are obtained as The absolute coordinate position of a point, ( ) , on the surface of target object is evaluated from the geometric parameters in absolute coordinates.In the above analysis, it is assumed that the point on the object surface is indicated by the bright line produced by the , , , , The resolution of depth extraction in this profilometry scheme is analyzed.Figure 2 shows the intersectional region of the fields of view of a projector and a CCD camera.The depth of objects placed in this region can be extracted with the stated method.In practice, the projector and the CCD camera have finite resolutions of 1024 768 × pixels and 1280 960 × pixels, respectively.Accordingly, the field of view is quantized as shown in Fig. 2. In Fig. 3(a), the relation between spatial resolution and angular resolutions of both CCD camera and projector is shown.Since the projector is positioned apart from the CCD camera, the minimum resolvable voxel (volume pixel) has a rhombic shape approximately.Let the vertices of the ( ) , th m n voxel are indexed by ( ) , , m n m n x z + + and ( ) . The area of this voxel is given by ( ) ( ) Here, ( ) S is Heron's formula representing the triangular area with three points given by

y x y x y x y x y x y x y x y x y
The contour map of a resolvable voxel in the intersectional region is shown in Fig. 3(b).The minimum resolution is about 2 0.7mm around the point 1500mm away from the reference point of the projector.The 3D discontinuous objects used in the first experiment are placed from 2300mm to 2800mm from the reference point of the projector.Hence, the resolution ranges of the first experiment are from 2 4mm to 2 8mm approximately.

Depth extraction using multi-frequency and four-step phase-shift sinusoidal fringe composite
In Section 2, it is shown that if 3D objects can be scanned by a bright line, we conduct 3D profiling of the target objects with simultaneous analysis of depth and position.In practice, the direct scanning of bright line over target objects is inefficient since moving picture of line image scanning over the target objects is necessary.
In this paper, an efficient method for realizing the above principle with multi-frequency sinusoidal fringe patterns is devised.In this section, the devised depth extraction analysis method that is effectively equivalent to the above stated line image scanning but more efficient than it is described.
Figure 4 shows the schematic of the devised 3D profilimetry method.Within the setup depicted in Fig. 1, a sinusoidal fringe pattern, , ( )   n p P ξ , with a specific spatial frequency, n f , and a specific phase shift, p , is generated as given by ( ) , ( ) 1 cos 2 where ξ is the lateral axis of the projector image plane.The phase-shift, p , is given by one of four quadrate phase-shift values as 0, 2, , and 3 2 p In the proposed method, we should take a bundle of images of distorted sinusoidal fringes with several spatial frequencies n f .Also, it is important that for a spatial frequency Before this process, let us address a mathematical property of superposition of the raw data pictures , n p I .Basically, the objective of using several multi-frequency fringe patterns is to equivalently realize the bright line scanning as stated in Section 2. According to the Fourier transform, a sharp bright line image can be represented by a superposition of several weighted sinusoidal fringe patterns.The reason of the need of four fringe patterns with quadrate phaseshifts is to construct an exponential Fourier basis.Since the sinusoidal fringe patterns that can be generated by the projector are inevitably positive value functions, the necessary exponential Fourier basis for a spatial frequency n f is constructed as follows.
( ) sin 2 2 As a result, the exponential Fourier basis can be obtained as n n n n n j f P P j P P Then we set a shifted pulse-shaped function ( ) where s is the lateral translation of the centered pulse-shaped function ( ) h ξ , and find the Fourier coefficients of ( ) real t is the real part of a complex number t .For convenience, we adopt the pulse-shaped function by a delta function with the peak position s given by ( ) ( ) As a result, the local illumination image function, ( ) For a projector with finite resolution of 1024 768 × pixels, the maximum spatial frequency of expressible sinusoidal fringe pattern is 10  2 − .Thus, the available spatial frequencies for the projector with 1024 768 × resolution are in the range of 10 As a result, without actual scanning action, with just 4N pictures, 1024 pictures of line scanning images of the target objects are obtained.This is a main meaning of using multi- frequency and four-step phase-shift sinusoidal fringe projection in the proposed profilometry method.
In every charts, the ξ -axis indicates the lateral pixel index of the 1024 768 × projector.The η -axis is the light intensity of the composite local illumination image.

Conclusion
In conclusion, 3D profilometry without phase unwrapping using multi-frequency and fourstep phase-shift sinusoidal fringe projection has been developed.The scanning of local illumination pattern necessary for the depth extraction without phase unwrapping is performed by composition of multi-frequency sinusoidal fringes with four-step phase shifting method.In this proposed profilometry, the ambiguity in depth extraction of conventional profilometry methods associated with phase unwrapping is removed and the feasibility is demonstrated with experimental results.

Fig. 1 ,
the amount of rotation tilt angle of the projector and the CCD camera are indicated by PRJ ϕ and CCD ϕ , respectively.PRJ ϕ ( CCD ϕ ) is the angle between the left boundary of the field of view of the projector (CCD camera) and the z-axis.Let us assume that the projector illuminates a single bright line image focused on a point on the object surface ( ) of the line in the projection image is straightforwardly converted to the local illumination angle, Local θ , and the position of the line in the image captured by the CCD camera is also corresponding to the detection angle, Detect θ .These geometric parameters are used in extracting the depth of objects in the absolute coordinate system.In the rotated coordinates, where the axis of z′ is parallel to the left boundary of the viewing region of the CCD camera, the bright line position, ( ) surface, seen by the CCD camera is located on the line give by

Fig. 2 .
Fig. 2. Intersectional region with field of views by both a CCD camera and a projector.

Fig. 3 .
Fig. 3. Spatial resolution in the intersectional region: (a) the relation between spatial resolution and angular resolutions of both a CCD camera and a projector and (b) the contour map of resolvable voxel in the intersectional region.

nf
, four distorted fringe images of four sinusoidal fringe patterns with quadrate phase-shift values of 0, 2, , and 3 2 p π π π = have to be reserved.Then if we use N spatial frequencies, we have to get 4N pictures.The next step is numerical depth extraction of target objects from the obtained 4N pictures.

#
109366 -$15.00USD Received 27 Mar 2009; revised 18 Apr 2009; accepted 23 Apr 2009; published 27 Apr 2009 (C) 2009 OSA we use a finite number of Fourier harmonics, the Fourier series cannot represent the original shifted pulse-shaped function, but it is enough to represent local illumination pattern wherein brightness around the center of the original pulse-shaped function is relatively stronger than that of other parts.Let this partially truncated Fourier series of the original pulse-shaped function ( ) important property of Eq. (12) associated with profilometry is that the peak shift of the local illumination function can be simply controlled by a change of s , where s is in the range of 0 1023 s ≤ ≤.Therefore, we can perform bright line scanning action with just raw images of multi-frequency sinusoidal fringes effectively by repeatedly calculating Eq. (12) with change of s .Without actual scanning action of bright line illumination over target objects, we can numerically calculate the pictures of object surface illuminated by shifting bright line images,

Fig. 5 .Fig. 6 .
Fig. 5. Composite local illumination functions of multi-frequency sinusoidal fringe patterns of (a) a single spatial frequency of n = 10, (b) two spatial frequencies of n = 5 and 10, (c) two spatial frequencies of n = 9 and 10, and (d) nine spatial frequencies of n = 2, 3,...,and 10.The shift parameter s is set to 512 s = .

Table 1 .
Comparison between the actual depth of the center points of 20 balls and the extracted depth of the center points.