Fourier analysis of RGB fringe-projection profilometry and robust phase-demodulation methods against crosstalk distortion

In this paper we apply the frequency transfer function (FTF) formalism to analyze the red, green and blue (RGB) phase-shifting fringe-projection profilometry technique. The phase-shifted fringe patterns in RGB fringe projection are typically corrupted by crosstalk because the sensitivity curves of most projection-recording systems overlap. Crosstalk distortion needs to be compensated in order to obtain high quality measurements. We study phase-demodulation methods for null/mild, moderate, and severe levels of RGB crosstalk. For null/mild crosstalk, we can estimate the searched phase-map using Bruning's 3-step phase-shifting algorithm (PSA). For moderate crosstalk, the RGB recorded data is usually preprocessed before feeding it into Bruning's PSA; alternatively, in this paper we propose a computationally more efficient approach, which combines crosstalk compensation and phase-demodulation into a single process. For severe RGB crosstalk, we expect non-sinusoidal fringes' profiles (distorting harmonics) and significant uncertainties on the crosstalk calibration (which produces pseudo-detuning error). Analyzing these distorting phenomena, we conclude that squeezing interferometry is the most robust demodulation method for RGB fringe-projection techniques. We support our conclusions with numerical simulations and experimental results.


Introduction
Multichannel red, green and blue (RGB) color encoding is a very useful approach in fringe projection profilometry, which allows simultaneous acquisition of three linearly independent phase-shifted fringe patterns for single-shot measurements [1][2][3][4][5][6]. Pioneer applications of RGB fringe-projection are due to Wust and Capson [1], who in turn used the phase-demodulation methods by Bruning et al. [7] and Takeda et al. [8]. In RGB fringe-projection profilometry, a digital projector casts simultaneously up-to three sinusoidal phase-shifted fringe patterns over the tridimensional (3D) object under test, and these fringe patterns became phase-modulated proportionally to the height profile of the object [1 -6]. Under ideal conditions, we can estimate the searched modulating phase from the RGB encoded data using Bruning's 3-step phase-shifting algorithm (PSA) [1,7]. However, most color cameras and digital projectors have overlapping spectral sensitivities, which produce RGB-crosstalk distortion [9]. Phaseshifted fringe patterns distorted by RGB crosstalk have non-sinusoidal profiles and their phase-steps differ from the nominal value, which is particularly challenging for few-step algorithms [10][11][12]. In order to improve the quality of the measured phase proportional to the object's profile under study, crosstalk compensation is required before feeding the data into the 3-step PSA [1][2][3][4][5][6]. Other common practices in fringe projection include gamma correction and projection defocusing in order to obtain more sinusoidal profiles [1-6, [13][14][15].
In this paper, we employ the frequency transfer function (FTF) formalism to analyze the robustness of Bruning's 3-step PSA when applied to fringe patterns distorted by RGB crosstalk. From our analysis, we deduce an equivalent but computationally more efficient method to perform the quadrature filtering in fringe patterns with mild to moderate RGB crosstalk. We also show that Bruning's 3-step PSA is unreliable for severe RGB crosstalk, since it introduces pseudo-detuning distortion and preserves too many distorting harmonics. When working under these conditions we strongly recommend working with the squeezing interferometry technique (which we also analyze briefly in this paper) for robust quadrature filtering. Finally, we support our conclusions with numerical simulations and by digitizing a static 3D surface (for repetitively, in order to compare all methods discussed in this paper).

The FTF formalism and Bruning's 3-step PSAs.
For ease of exposition and the reader's convenience, let us begin with a brief review of the FTF formalism applied to phase-shifting algorithms [12]. Our standard mathematical model for a temporal-carrier fringe pattern is given by: where ( , ) a x y represents the background signals, ( , ) b x y is the fringes' contrast function, and ( , ) x y  is the searched modulating phase. Finally, the temporal carrier frequency 0 (0, )    is assumed to be known. Taking the temporal Fourier transform of Eq. (1), we have: We need to isolate one of the spectrally displaced analytic signals (without loss of generality we choose the one at 0    ) in order to demodulate the searched phase. This is done using an N -step frequency transfer function ( ) H  that fulfills the quadrature conditions: From Eq. (2) and (3), taking ( , , ) ( ) I x y H   results: Back on the temporal domain we evaluate at 1 t N   in order to use all available data [12], and using the compact notation ( , ) ( , , ) ( ) n I x y I x y t t n    , we have: and we use a "hat" to label estimated values. Once we isolate this analytic signal, it is easy to solve for all unknowns in Eq. (1) as: Since the estimated phase is wrapped within ( , ]    , we generally apply a phase unwrapper as last step of the demodulation process [12,16].
Particularly, in this paper we will work with the 3-step PSA by Bruning et al. [7,12], for which 0 Since the coefficients n c also define the FTF, we have after some algebraic manipulation: As illustrated in Fig. (1

RGB fringe projection profilometry with 3-step phase shifting.
Three-step phase-shifted fringe projection with a single-frequency spatial carrier is arguably the most commonly used approach in RGB fringe-projection profilometry (for alternative approaches see, for instance, Zhang's review [5] and its references). Our mathematical model for these spatial-carrier open fringes is given by: Here {·} LPF represents an analog low-pass filter (usually defocusing), 0 u is the frequency of the spatial carrier (in radians per pixel), and the numerical values for  are within the range (1.6, 2.4) . The low-pass filtering and the non-linear gamma encoding allow us to attenuate nonlinear intensity distortions during the fringe-projection process [14,15]. Considering the profilometer setup in Fig Here ( , ) a x y and ( , ) b x y represent the background illumination signal and the fringes' contrast function, respectively. The searched modulating phase is proportional to the profile of the 3D object under study plus a known term; therefore, it is trivial to solve for ( , ) p x y once we find ( , ) x y  using Eq. (6) and (7). Using RGB color encoding (see Fig. 3), we can project a single full-color image and generate simultaneously the three phase-shifted fringe patterns in Eq. (10). Without loss of generality we assume 0 ( , ) f x y , 1 ( , ) f x y , and 2 ( , ) f x y being projected in the red, green and blue channel, respectively. In absence of crosstalk distortion (which is feasible using dichromic filters and multiple sensors such as the ones inside of a 3-CCD camera), we can recover the searched fringe patterns from the recorded data in the RGB channels as [1,9]: It is a well-known fact that most single-sensor color cameras have overlapping sensibilities [9]. Therefore, when using this kind of cameras for fringe projection profilometry, RGB crosstalk distorts the recorded data and some compensation algorithm is required [2][3][4][5][6]. In practice, assuming the crosstalk phenomenon to be linear, we have: The real-valued matrix elements mn A represent the camera's sensibility on the m-th channel Huang et al. [2], proposed a calibration method for matrix A , which was later improved by Zhang et al. [3]. This method consists in generating only red, only green, and only blue phase-shifted open fringes and project them sequentially (instead of simultaneously) over a white reference plane. Due to crosstalk, the CCD camera records these single-color projected fringes on all three RGB channels. Analyzing separately each channel of the recorded data, we can easily calibrate all nine crosstalk coefficients mn A proportionally to the average of the fringes' contrast function [2,3]. Once we calibrate matrix A , we can easily solve Eq. (12) for the crosstalk-compensated phase-shifted fringe-patterns as: We can introduce these crosstalk-compensated fringe patterns into Bruning's 3-step PSA: However, from Eq. (13)- (14), it is easy to see that an equivalent but computationally more efficient approach to demodulate the searched analytic signal is given by: The complex-valued coefficients n d need to be computed only once for fixed experimental conditions. The above equation requires 4 operations per pixel and it works directly over the raw data { , , } , , } I I I . Since RGB fringe-projection profilometry is meant for dynamic measurements [1-6], this represents a substantial reduction on the computational workload. As far as we know, Eq. (15) has never been reported before so this is our first contribution on this paper.

Influence of systematic errors on the phase-demodulation algorithm
Here we illustrate how systematic errors produce low-quality phase estimations when applying Bruning's 3-step PSA under non-ideal conditions, such as residual errors during the calibration of the crosstalk coefficients and distorting harmonics.
We start by modeling our estimation for the crosstalk matrix as ˆ  where   ε Bδ ; therefore mn  are unknown, and | | 1 . We must remark that these plots still correspond to Bruning's 3-step PSA but being applied to fringe patterns distorted by RGB crosstalk. Right side shows the Fourier spectrum of the analytic signal estimated using one of these FTFs.
From Fig. 4 Carefully controlled experimental conditions and/or fringe pattern normalization allow us to eliminate the background signal term [13]. Thus, taking ( , ) 0 a x y  , we have: For the trained eye, Eq. (18) predicts that the wrapped phase will be distorted by a doublefrequency spurious signal, which is identical to the well-known distortion due to detuning error in phase-shifting interferometry [12]. We performed a numeric simulation to assess this pseudo-detuning distortion (see Fig. 6). First, we modulated 3 phase-shifted fringe patterns with a known phase. Next we estimated the analytic signal according to Eq. (16) using random values for the crosstalk calibration uncertainty, such as | | {0.05, 0.10, 0.15} mn   . Finally we computed the wrapped difference between the actual phase and the estimated one, revealing the predicted double-frequency spurious signal. For these simulation, we used a low-frequency spatial carrier for ease of observation. Here N represents the order of the highest harmonic with still-significant distorting energy, ( , ) n b x y is the contrast function of the n -th harmonic, and the wrapped location of the Dirac deltas is due the 2  periodicity of the discrete-time Fourier transform [12]. When the fringe pattern is distorted by harmonics, we want to isolate 1 exp( ) b i in order to find the best- It is easy to see from the above equation that back on the temporal domain we will have an analytic signal distorted by pseudo-detuning (because 1 exp( ) b i  remains unfiltered), and corrupted by the distorting harmonics with the highest energy (both components of the second one, both components of the third one, and so on). Therefore, it is clear that we require a more robust approach to cope with non-sinusoidal fringes and severe RGB crosstalk.
Comparing Eq. (20) and (24), it is clear that the temporal and spatial carriers produce similar spectral distributions, but there are significant differences. First, the energy of { e } ; thus, the n -th harmonic has much lower spectral density than the searched analytic signal. Also, processing in the spatial Fourier domain is immune to detuning error because we can filter-out all signals outside the area of interest, instead of just the components exactly at ( 2 / 3,0)   and (0,0) . As illustrated in Fig. 5, this means we can achieve robust quadrature filtering with the following synchronous demodulation formula [10,11]: n I x y , and finally we compute its argument and unwrap the modulating phase, as usual [10,11].

Experimental results
In our experiment, we worked with a static test subject for repeatability: digitizing a humanface model made of expanded polystyrene (see Fig. 7). This allows us to take a crosstalk-free measurement using sequential grayscale-based fringe projection as null test. We used this null test to assess the quality of the recovered phases from single-shot RGB fringe projection using Bruning's 3-step PSA and squeezing interferometry. The open fringes were projected using a 1024 768  digital light projector, with defocusing and pre-encoding gamma [14,15]. The fringe patterns were captured using a single-sensor digital color camera that performs RGB color separation using a Bayer filter mosaic with a 1280 768  resolution and 8-bit color depth per channel. We calibrated the crosstalk matrix using 6-step least-squares phase-shifting measurements of the reference plane for increased reliability (to generalize from [2,3] is straightforward). Our numerical values for the crosstalk calibration matrix Â (see Eq. (12)- (13)) and for the corresponding phase demodulation coefficients n d , required to work directly with raw data (see Eq. (15) According to this numerical values, we faced severe RGB-crosstalk distortion in comparison to other calibration matrices reported in the literature [2,3,6].   The next step is to compute   ( ', ) S I x y  , from where we isolate the searched analytic signal by means of a narrow one-sided bandpass filter. Back on the spatial domain, we squeezed the analytic signal back to the original size of ( , ) n I x y , and then we estimated the wrapped phase from its argument. For ease of visualization, at this point we removed the low-frequency modulation 0 tan( ) u x  , previously calibrated from the reference plane. See Fig. 9.  Figure 10 shows the recovered profiles from the RGB raw data, the crosstalk compensated fringe patterns, squeezing interferometry, and our null-test for illustrative purposes. These plots use the amplitude of the analytic signal as texture in order to visualize fine details. Figure 10(a) shows the 3D profile recovered from the raw camera data, where crosstalkdistortion was so severe that the phase unwrapper failed to recover even the overall shape of the test subject. Figure 10(b) shows the result of processing the same raw data using our proposed algorithm (Eq. (15)); this result shows a drastic improvement respect to Fig. 10(a) but still exhibits the ripple pattern due to pseudo-detuning error and distorting harmonics. Figure 10(c) shows our best quality result, which we obtained with the combination of crosstalk-compensation and squeezing interferometry. Finally, Fig. 10(d) shows the result of our crosstalk-free null test for illustrative purposes; this result was obtained by means of sequential gray-scale based fringe projection. Figure 11 shows horizontal slices of the unwrapped phase for qualitative comparison of the results estimated with Bruning's 3-step PSA and with squeezing interferometry. This figure also demonstrates that the ripple pattern due to pseudo-detuning and distorting harmonics is observed on the modulated phase. Fig. 11. Horizontal slices of the unwrapped phase estimated with Burning's 3-step PSA, the squeezing interferometry method, and the crosstalk-free null test. A global piston was included between each plot for ease of visualization.

Conclusions
We analyzed the most commonly used phase-demodulation approaches for RGB fringeprojection profilometry by means of the FTF formalism. We proposed a computationally more efficient algorithm applicable for mild to moderate RGB-crosstalk conditions, which combines crosstalk compensation and phase demodulation, and that works directly with the raw data (Eq. (15)). We show that relying on Bruning's 3-step PSA for severe crosstalk conditions (when modeling this phenomenon using linear combinations no longer applies) translates into a pseudo-detuning distortion and high-energy distorting harmonics. The pseudo-detuning error was somehow unexpected because the phase-step is a controlled parameter in fringe projection techniques (see Eq. (9)). We also analyzed the squeezing interferometry phase-demodulation method to show that it allows us to recover high-quality results even under severe crosstalk distortion. Finally, we supported our claims with numerical simulations and experimental results.