Profilometry with digital fringe-projection at the spatial and temporal Nyquist frequencies

A phase-demodulation method for digital fringe-projection profilometry using the spatial and temporal Nyquist frequencies is presented. It allows us to digitize tridimensional surfaces using the highest spatial frequency ({\pi} radians per pixel) and consequently with the highest sensitivity for a given digital fringe projector. Working with the highest temporal frequency ({\pi} radians per temporal sample), the proposed method rejects the DC component and all even-order distorting harmonics using a bare-minimum 2-step phase shift. The proposed method is suitable for digitization of piece-wise continuous surfaces because it does not require spatial low-pass filtering. Gamma calibration is unnecessary because the projected fringes are binary, and the harmonics produced by the binary profile can be easily attenuated with a slight defocusing on the digital projector. Viability of the proposed method is supported by experimental results showing complete agreement with the predicted behavior.


Introduction
Nowadays, structured-light projection profilometry is arguably the most popular technique for tri-dimensional (3D) shape measurement [1][2][3]. Particularly the phase-demodulation methods working with sinusoidal fringe patterns provide the highest-quality dense estimations of the digitized surface [1][2][3][4]. A typical setup for fringe-projection profilometry (FPP) requires an image projector, a camera, and a computing device for control and data processing. The sensitivity in FPP is proportional to the spatial frequency of the projected fringes, and to the tangent of the angle between the projection and observation directions [5]. However, using a large angle is generally unadvisable because extensive self-occluding shadows are produced over the 3D solid under study. Likewise, the highest spatial frequency we can work with is limited by the discretization of the projected fringes, and by the lateral-spatial resolution of the camera and the projector. Also, we must remark, modern digital cameras have much higher spatial-resolution in comparison to digital image projectors. Therefore, being able to work with the highest spatial frequency achievable by the digital projector is of great interest.
According to the Nyquist-Shannon sampling theorem, the maximum frequency of a sampled signal must be strictly lower than half the sampling frequency in order to avoid information loss due to aliasing [6]. Half the sampling frequency of a discrete system is called the Nyquist frequency. Digital projector and cameras have a Nyquist frequency 0.5 samples per pixel; thus, a spatial period longer than 2 pixels is necessary when projecting/recording sinusoidal fringes in order to avoid aliasing. Nevertheless, as we will show in this paper, we can work at the spatial Nyquist frequency for the fringe-projection stage (spatial frequency for the recording stage must remain below Nyquist). Describing each spatial period with two discrete samples, the fringes become binary and an analog low-pass filter (defocused projection) must be applied in order to recover a sinusoidal profile [7].
Defocused projection of binary fringes removes the need to calibrate non-linear gamma encoding of most projectors, which is one of the main difficulties for high-quality phase estimations in digital FPP [1][2][3][4][5][6][7][8]. It also impedes us from using many-step phase-shifting algorithms (PSAs) for robust quadrature filtering. Nevertheless, as we will show in this paper, working with the spatial and temporal Nyquist frequencies we derived a phase-demodulation algorithm with reasonable high signal-to-noise ratio (SNR) and as robust against distorting harmonics as the popular 4-step least-squares phase-shifting algorithm (LS-PSA).
The rest of the paper is divided as follows. Section 2 presents the theoretical framework for the proposed method. In section 3 we derive the figures of merit for our proposed method: SNR and robustness against distorting harmonics using the frequency transfer function (FTF) formalism. Section 4 shows some experimental results consistent with the predicted behavior. Finally, section 5 contains our conclusions and some perspectives for future work.

Theoretical framework
Consider the single-projector single-camera setup for digital fringe-projection profilometry illustrated in Fig. 1 The normalized intensity is assumed to be discretized on at least 8 2 levels for negligible discretization distortion. Finally, the label "p" stands for projection. When projected over a 3D object, the phase-shifted open fringes in Eq. (1) become phasemodulated proportionally to the height profile of the surface under test [1][2][3][4][5]. Our ideal mathematical model for the sinusoidal phase-modulated fringe pattern is given by:   remains true if the camera sees just a portion of the region illuminated by the digital projector [9]. Rewriting ( , , ) I x y t using Euler's formula, we have: Both spatial and temporal carriers are assumed to be known ( ' u is easily calibrated using a reference plane), so we can estimate ( , ) c x y or ( , ) c x y  using synchronous methods. By convention we choose ( , ), c x y from where the searched phase-map and the fringes' contrast function are computed as its argument and amplitude, respectively [1][2][3][4][5]. For the spatial synchronous method, having 0 (0, ), where 1 ( , ) ( , ), r x y b x y  and ( , ) h x y is real-valued impulse response with a narrowbandwidth FTF that isolates the signal of interest around the spectral origin [1][2][3][4][5]. This approach is suitable for dynamic measurements, but the spatial low-pass filtering implies the demodulated phase 1 ( , ) xy  will be distorted around any discontinuity on the surface. Alternatively, for the temporal synchronous method, the searched analytic signal is given by Here 2 ( , ) ( , ) r x y b x y  and M represents the number of temporal phase-shifted samples. The complex-valued weight factors n w define the impulse response function and its FTF as Since no spatial low-pass filter is involved, 2 ( , ) xy  is undistorted around discontinuities on the surface. However, requiring several phase-shifted fringe patterns makes this approach unsuitable (in principle) for the study of fast-changing phenomena [1][2][3][4][5].

Aliasing at the Nyquist frequencies
Substituting Eq. (7) in Eq. (1), it is evident that digital projection working at the spatial Nyquist frequency produces standing fringes with fluctuating-contrast instead of phaseshifted fringes with constant-amplitude. This is illustrated in Fig. 2. From Eq. (7) and Fig. 2, it is clear that the only linearly-independent fringe patterns with the highest-possible spatial frequency and contrast are obtained when 0 {0, }. t

 
Since the alternative is working with low-contrast fringes containing the same information and the fringes' contrast is essential for a high-quality phase-demodulation [4,5], the logical choice is to project only two fringe patterns using both Nyquist frequencies, from where we have: H   due to the 2  periodicity of the discrete-time Fourier transform [4]. This is illustrated in Fig. 3. Fig. 3. Temporal quadrature filtering below and exactly at the Nyquist frequency. The vertical arrows represent the discrete-time Fourier transform of I(x,y,t). The blue line is |H3(ω)| for tunable a 3-step PSA having spectral zeroes at ω={0, -ω0}. Below Nyquist we are able to isolate c = (b/2)exp(iφ). On the other hand, when working at the temporal Nyquist frequency, the spectral zero at ω = -ω0 would filter-out both analytic signals simultaneously.
Recall that defocused projection is assumed for the high-frequency binary fringes in order to obtain the sinusoidal profile modeled in Eq. (8). For a detailed discussion, see [7].

Proposed phase-demodulation algorithm
Here we deduce a phase-demodulation algorithm suitable for the spatial and temporal Nyquist frequencies in FPP. Operating on the phase-shifted fringe patterns on Eq. .
The subtraction in '( , ) I x y describes the following impulse response and FTF: Since 0 1, applying a one-dimensional spatial Hilbert filter (OSHF) for the horizontal frequencies, which is defined by [10,11]: Finally, back on the spatial domain, the searched the analytic signal is given by: Here 3 ( , ) 2 ( , ). r x y b x y  Please note that spatial low-pass filtering was not required for the phase demodulation, and that Hilbert filtering preserves all the spectral information for realvalued input data [11]. In other words, this approach is suitable to work with discontinuous surfaces, similarly to the temporal synchronous demodulation method.

Signal-to-noise ratio (SNR)
The analysis in the previous section was restricted to ideal conditions for ease of exposition. Now let's assume the phase-modulated fringe patterns in Eq. H   that condition was replaced by the one-dimensional Hilbert filtering.
Continuing the analysis of the general case for M-step LS-PSAs filtering, the signal-tonoise ratio (SNR), computed over an area  where we have valid data, is given by: M is the number of phase-shifted temporal samples, and  is the number of pixels on a spatial period of the projected fringes. Note that the ratio / M   is the only free-parameter that allows us to increase the SNR (an excessive increase of  would produce large selfoccluding shadows where ( , ) 0 b x y  ). Moreover, using    is recommendable for short spatial periods in order to avoid changes on the profile of the projected fringes. See Fig. 4. Equation (18) assumes high-frequency spatial carriers. The variations on the fringes' profile are less significant for low-frequency spatial carriers, leaving / M   unrestricted. As usual, the SNR for this case is improved using as many phase-shifted temporal samples as possible.

Robustness against distorting harmonics
Harmonic distortion is to be expected under saturation on the image sensor and/or due to residual distortion after the defocused projection of the binary fringes [1][2][3][4][5]7,8]. Therefore, a more realistic model for our phase-modulated fringe patterns is given by: Here k b is the contrast of the k-th order distorting harmonic, satisfying , K is the order of the distorting harmonic with still-significant energy.
The background signal or DC-offset is given by 0 , b and all other terms remains as previously defined. As before, the ideal result would mean to isolate 11  This result shows that the 2-step temporal filter eliminates not only the background signal ( 0) k  but also filters-out all distorting harmonics of even order. For completeness (we will revisit this result at the end of this section), we have in the discrete-time Fourier domain: Note that some even-order harmonics are aliased to the spectral origin and filtered-out by The Hilbert filtering at Eq. (13) erases any spectral component within the negative-half of the spatial spectrum, but only one component of every k-th odd-order harmonic lies within the positive-half of the spectral domain, where 0 u  , and they remain unfiltered by the OSHF. Therefore, we conclude from Eqs. (13)-(22) that the analytic signal distorted by harmonics will be given by: The plus-or-minus signs depend on the actual value of .  As shown in Fig. 5, the robustness against harmonics of our 2-step phase-demodulation algorithm is superior to the 3-step LS-PSA (particularly because the second harmonic is usually the one with highest distorting energy) and equivalent the robustness of the popular 4-step LS-PSA.

Experimental results
Feasibility of the proposed method was confirmed with experimental results using a 1-camera 1-projector profilometer (as shown in Fig. 1). We used a digital light processing (DLP) projector with 1024 768  pixels resolution and 8-bit color depth per channel, working only with the green channel for monochromatic operation. The images were recorded using an 8bit grayscale digital camera with 0 1600 120  pixels resolution. The camera explored a small fraction of the region being illuminated by the DLP projector. Calibrating the effective carrier with help of the reference plane we found 0.2167   for our particular setup. Two 3D solids were digitized: a continuous surface as calibration object (showing the many-wavelengths sensitivity achieved) and a piece-wise continuous object (illustrating the applicability of the proposed method for the study of discontinuous surfaces). to see at this small scale but satisfactory contrast was achieved on the raw data (see Fig. 6(b)). In Fig. 7 intensity is in logarithmic scale with the spectral origin at the center for ease of visualization. Note from Fig. 8 the many-wavelength sensitivity achieved projecting at the spatial Nyquist frequency.    Figures 9-11 show the main steps on the digitization of a discontinuous (piece-wise continuous) solid: a combination-square tool equipped with the standard head. As before, we worked at the Nyquist frequencies to obtain a high-sensitivity highly wrapped phase map as the argument of Eq. (13). However, the surface steps and the self-occluding shadows made impossible to unwrap the highly modulated phase-map using spatial methods. Figure 9(c) shows one of four low spatial-frequency fringe patterns processed additionally to obtain a non-wrapped low-sensitivity estimation (having 0 /10 u   and 0 2 / 4   ), which was used as stepping stone for dual-sensitivity temporal phase unwrapping of Fig. 11(b) [13]. Fig. 9. Digitization of a combination-square tool. (a) One of two phase-modulated fringe patterns obtained with digital fringe projection using both Nyquist frequencies. (b) Zoomed-in section of (a). (c) One of four low-frequency carrier fringes (using u0=π/10) to be used as stepping stone on dual-sensitivity temporal phase unwrapping [13].

Conclusions and perspectives
We proposed a robust and easy to implement phase-demodulation method for digital FPP suitable for the spatial and temporal Nyquist frequencies during the projection stage. We remark, once again, that the Nyquist-criterion must be satisfied during the recording stage (having 1   in our notation). Using the spatial Nyquist frequency ensures the highest depth-sensitivity for any given angle between the projection and recording directions. The proposed method does not require any low-pass filter for the phase-demodulation, so it can be applied to the digitization of discontinuous (piece-wise continuous) surfaces. Among the many phase-demodulation algorithms based on DC bias removal plus Hilbert transform/filtering, the only one we found to be equivalent to Eq. (13) was published by Zheng and Da in 2013 [14]. However, its applicability to the Nyquist frequencies was not even mentioned. Instead they discussed a method for gamma compensation, because highorder distorting harmonics (due to non-linear gamma encoding) usually require many-step PSAs for high-quality estimations [1][2][3][4][5]. Nevertheless, we demonstrated in this paper that the proposed algorithm (Eq. (13)) is as robust against distorting harmonics as the popular 4-step LS-PSA despite using half the number of phase-shifted temporal samples. Moreover, nonlinear gamma calibration is unnecessary for our method because the projected fringes become binary at the spatial Nyquist frequency.
The proposed method is convenient for high-speed implementations because it requires a single phase-step (two frames) and, as illustrated in Fig. 2, the on/off pixels alternate between each phase-shifted fringe pattern. This makes them ideal for color-encoded single-image FPP. For instance, one might encode the first frame in the red channel and the one shifted by  radians (equivalently to a 1-pixel lateral shift) in the blue channel. The resulting full-color image will contain both phase-shifted fringes patterns with essentially null-crosstalk and without spatial overlap (the later meaning the 3D test surface will be fully illuminated the entire time, except of course for self-occluding shadows).