Spatiotemporal phase-shifting method for accurate phase analysis of fringe pattern

High accurate phase analysis of waveforms as fringe patterns is essential for a wide range of scientific and engineering disciplines. However, precise phase analysis under extremely low signal-to-noise conditions is a challenging task for conventional phase-shifting methods. Here, a novel accurate phase recovering technique, called the spatiotemporal phase-shifting method (ST-PSM), is developed to measure the phase information robustly by utilizing two-dimensional intensity data in spatial- and temporal-domains simultaneously. Our simulation results indicated that ST-PSM had strong tolerance to random noise, and a self-neutralizing function to eliminate the periodical phase error due to the nonlinearity of detector, intensity saturation, vibration or phase-shifting error. The effectiveness was demonstrated experimentally from a non-contact shape measurement in fringe projection profilometry under extreme underexposure and overexposure recording conditions. Furthermore, by incorporating modern GPU parallel computing technology, a 4-step phase-shifted fringe pattern with 8 K image size can be calculated within one second. Due to its robustness and high accuracy with a fast calculation, therefore, we believe this technique has a significant impact on a variety of research and scientific fields.


Introduction
Nowadays, the phase-shifting method (PSM) [1,2], also known as the phase-stepping method, is one of the most powerful tools in optical metrology and is suited for measuring the phase information of the fringe pattern or interference image. The PSM was firstly reported by Carré in 1966 [3] and by Bruning et al in 1974 [4]. The basic idea of the PSM is to introduce several 'phase shifts' and record multiple phase-shifted fringe patterns comprising four essential parameters including background (A), amplitude or modulation (B), fringe pitch (P), and position-dependent modulating phase value (j). By utilizing discrete Fourier transform (DFT), as one kind of least square fitting in mathematics, A, B, and j can be determined simultaneously if the phase-shifting number is larger than 3.
To detect the phase information more accurately, various phase-shifting methods (PSMs) have been developed by many researchers over the last half-century and more than 80 algorithms have been reported [5]. Several unique phaseshifting algorithms also have been reported by only use of two-step phase-shifting [6,7], or with an unknown relative step to obtain the phase distribution [8].
In optical methodology, the PSM can be applied to various optical metrologies, such as Moiré interferometry [9], electronic speckle pattern interferometry (ESPI) [10], phaseshifting digital holography (PS-DH) [11] and fringe projection method [12,13] for full-field measurements. However, the quality and contrast of the acquired fringe image could be reduced due to various factors. For instance, the nonlinearity of the detector, random noise or intensity saturation when the exposure time is too short or too long, and very large or small reflectance of the object to be measured, which results in a significant error (random or periodical distributed) in the analysis result of the phase distribution, or a large measurement error occurs when a phase-shifting error is included due to an environmental vibration during measurement or the performance of the phase-shift device.
The random and periodical phase error can be improved by increasing the number of phase-shifted fringe patterns [14,15]. However, there is a dilemma that a large number of image recodings could significantly reduce the measurement speed. The trade-off between measurement accuracy and speed limits its application. Therefore, there is a demand for novel techniques which can dramatically improve the analysis accuracy of the phase distribution, without increasing the measurement time as the most challenging task.
In this study, we present a phase retrieval method called the spatiotemporal phase-shifting method (ST-PSM) that utilizes two-dimensional intensity data in spatial-and temporaldomains simultaneously by extending the idea of the temporal phase-shifting technique and the spatial sampling Moiré (SM) method [16,17]. The ST-PSM overcomes the challenges of the conventional phase analysis methods using the phaseshifting technique, by balancing the measurement accuracy and speed. The rest of this paper is organized as follows. Section 2 introduces the outline of the conventional PSM and the SM algorithm and then the novel principle of the ST-PSM is presented. In section 3, a series of simulation results of the comparison between the PSM, the SM and the ST-PSM are presented. In section 4, the effectiveness of the ST-PSM is validated from a non-contact shape measurement experiment under three different conditions. Section 5 provides further discussions, and finally, section 6 concludes the article.

Principle
The spatiotemporal phase-shifting method is based on the N-step phase-shifting method-a technique commonly used for phase extraction of fringe pattern-combined with the sampling Moiré method. Before explaining the fundamental principle of the ST-PSM, we briefly review the outline and issue of the PSM and the SM first.

Outline of the phase-shifting method and the sampling Moiré method
In the well-known N-step phase-shifting method [6], the acquired intensity f (x, y; n) for the nth image frame at the pixel (x, y) in the sensor plane is given by where A(x, y) and B(x, y) represent the background and amplitude intensities of the fringe pattern, respectively. P is the fringe pitch in the x-direction, and ( ) j x y , 0 is the positiondependent phase value according to the shape of the object; and ( ) j x y , is the desired phase value to be determined. By solving equation (1) with a 1D discrete Fourier transform (DFT) algorithm, the desired phase information is obtained as follows: where {·} arg is the operator of argument function for complex numbers calculation, and W is the twiddle factor in the DFT algorithm, and is defines as The background and amplitude intensities are determined by Compared with the PSM using temporal multiple phaseshifted fringe patterns, one of the authors also developed a single-shot phase analysis technique called the sampling Moiré (SM) method [16] using spatial phase-shifted Moiré fringe patterns.
In the SM, for a single fringe pattern is as expressed by equation (1) when n=0, after performing down-sampling (image processing of thin out) with a constant T-pixel (an integer nearly equal to P) and intensity interpolation, multiple phase-shifted Moiré fringe with a low spatial frequency can be generated, expressed as follows: Finally, the desired phase of the original fringe pattern is simply obtained by adding the sampling phase ( ) j x y , . Besides, the fringe pitch distribution also can be calculated from the differential of the Moiré phase distribution [17].
Thus, we can use a suitable sampling pitch T close to the fringe pitch P for accurate analysis in the SM.

Principle of the spatiotemporal phase-shifting method
To solve the above mentioned problems, we have developed an accurate phase analysis technique, namely the spatiotemporal phase-shifting method (ST-PSM). The key point is that both the PSM and the SM utilize the same phase-shifting calculation algorithm but different phase-shifted intensity information: the former is in the temporal domain and the latter is in the spatial domain. That is to say, the phase at any pixel can be determined using either the temporal information or spatial information. Figure 1 illustrates the fundamental idea and concept of the ST-PSM behind the basic principles of the PSM ( figure 1(a)) and the SM ( figure 1(b)). In the ST-PSM, the phase information at any pixel is determined from a twodimensional (2D) cosinusoidal wave by use of the spatial-and temporal-intensities data from a local space of N×T pixels, as shown in figure 1(c). Figure 2 represents the fundamental principle and the image processing procedure of the ST-PSM algorithm, including five steps. In the first step, the background A(x, y) and the amplitude B(x, y) intensities are initially determined by the N-step PSM. In the second step, to apply arbitrary surface reflectivity of the measured object, normalized fringe patterns ( ) f x y n , ; are obtained by substituting the background and amplitude intensities to the recorded N-step phase-shifted fringe patterns, as denoted in equation (10).
Then, in step 3, by performing down-sampling and intensity interpolation with a sampling pitch T (T near equal to P) in the x-direction to all of the N-step phase-shifted fringe images, N×T-step phase-shifted Moiré fringe patterns can be generated using the SM algorithm: It should be noted that 'two kinds' of the phase shifting term can be obtained in equation (11). One is the / p T 2 in the spatial-domain and the other is / p N 2 in the temporal-domain. This part is most important in the ST-PSM algorithm, which is useful to eliminate any periodical phase error due to nonlinearity, intensity saturation, and vibration (phase-shifting error), which will be demonstrated in the simulation part in detail.
In step 4, the background, the amplitude and the phase distributions of the Moiré fringe are determined by using the 2D DFT algorithm: , can be obtained more accurately compared with the PSM and SM methods. Thus, the ST-PSM has high potential ability to improve the phase information as well as the image quality of background and amplitude distributions. Interestingly, similar spatiotemporal analysis employed a 3D-FFT approach [18] was reported to retrieve the phase information of interferometry [19].

Implementation of the ST-PSM for parallel computing
In the ST-PSM, after a sampling pitch T (an integer close to the fringe pitch P) is chosen, all the procedures from step 1 to step 5 can be automatically performed for each pixel independently without any iterative or subset searching processes, which enables fast calculation and is suitable to parallel computing using modern computer with multi-core CPU and GPU.
In the implementation, the input (fringe patterns) and the output (phase image) are loaded and saved by using a singlecore CPU, and the algorithms of the PSM and the ST-PSM for parallel computing are implemented by using open multi-processing (OpenMP) for multi-core CPU, and C++ accelerated massive parallelism (AMP) for GPU by C/C++ language (Visual Studio 2013).

Simulation
In phase-shifting methods, the random noise, the nonsinusoidal waveform error and the phase-shifting error are major error sources [20]. Any random noise and nonlinear effect deforms the ideal sinusoidal fringe pattern in nonsinusoidal shape and introduces random or periodical phase error. Therefore, in this section, with the help of simulated fringe patterns, we compare the performances of the conventional N-step PSM, the SM and the proposed ST-PSM. Since the 4-step phase-shifting algorithm is more extensively used for high-speed applications, we provide detailed analysis of phase error for the 4-step PSM, the single-shot SM, and the 4-step ST-PSM for all simulations.

Simulation condition of single error source
To understand the nature of phase error that appears in the conventional PSM and the effectiveness of our proposed ST-PSM, first, we carried out a set of primary computer simulation to investigate four factors of single error source: (i) random noise, (ii) nonlinearity (gamma distortion effect) of the detector or projector, (iii) intensity saturation, and (iv) mismatch between the fringe pitch and the sampling pitch, respectively.
(i) In the first simulation of the error source for random noise, 4-step cosinusoidal fringes with 10-pixel fringe pitch (P=10) are generated according to equation (15).
The simulated image type is 'float' in C/C++ programming data types. The image size is 256×256 pixels, and the root-mean-square (RMS) phase error, E , RMS in the central area of 200×200 pixels was evaluated according to equations (16) and (17).

theory
The phase error , for each method is calculated from the wrapped phase difference between the ideal phase value 2 , theory 0 and the analyzed phase value ( ) j x y , simu obtained by the PSM, the SM, and the ST-PSM algorithms. We fixed the nonlinearity factor g=1 as an ideal cosinusoidal wave, and added an additive random noise s , noise as indicated in equation (18): , ; . 18 We assume the noise has a mean of zero and a standard deviation ranging from 0% (noiseless) to 100% at 10% increment step, i.e.

noise
(ii) In the second simulation of the error source for the nonlinearity of the camera/projector, we fixed the random noise level at 2%, and changed the parameter of nonlinearity factor γ at nine different values [ ] g = 0.2, 0.4, 0.6, 0.8, 1, 2, 3, 4, 5 , in equation (18). (iii) In the third simulation of the error source for the intensity saturation (the maximum intensity limitation of the camera sensor), the saturated fringe pattern is generated by equation (20) under assumption of an 8-bit camera usage.
 g x y n g x y n g x y n , ; , ; , ; 255 255 otherwise . 20 We fixed the random noise level as 2% and γ=1, and then changed the saturation level S ranging from 0 to 0.95, i.e.
[ ] = S 0: 0.05: 0.95 . When S=0, the fringe pattern is an ideal cosinusoidal wave. When S increases, the saturation area also increase, and when S reaches to 1, all the intensity will become to the maximum value (255), which cannot be analyzed. Besides, the initial phase j 0 is set to be ( ) x y 0.5peaks , , using the MATLAB 'peaks' function together with a carrier to generate nonuniform fringe patterns.
(iv) In the fourth simulation of the error source for the mismatch of the fringe pitch and the sampling pitch, regarding one of the error source for the SM, we investigated the RMS phase error when the sampling pitch was changed from 6-pixel to 16-pixel at one pixel increment step for uniform 10-pixel pitch fringe patterns without random noise, nonlinearity and saturation. Figure 3 shows the simulation results of the influence of the respective single noise on RMS phase error.

Simulation results of single error source
3.2.1. Random noise. Since noise is unavoidable in practical phase measurement, therefore, we compare the sensitivity of three algorithms towards the noise in more extensive optical methodology such as ESPI and PS-DH techniques. Figure 3(a) shows the simulation result for different random noise level and gives a comparison of the RMS phase error between the PSM (blue line), the SM (black dotted line) and the ST-PSM (red line), when random noise varies from 0% to 100% at 10% step. As expected, the ST-PSM method can effectively alleviate the influence of random noise error and greatly improve the phase measurement accuracy, since more intensity information in both spatial-and temporal-domains was used to determine the fringe phase distribution. In the N-step PSM algorithm, the relationship between a variation in the phase error s j noise and the number of phase shift N has been expressed as equation (10) [21]: where s noise is a standard deviation of random noise and ( )/s B x y , noise is the signal-to-noise ratio (SNR). The variation in the phase error is inversely proportional to the square root of the number of recorded images and SNR, and the measurement accuracy can be improved if a large number of phase-shifting fringe patterns are used. This is the main reason why the ST-PSM can improve the phase measurement accuracy since N×T-step phase-shifted Moiré fringes were used for the spatio-temporal analysis approach.
Generally, the intensity values of an 8-bit (256 graylevel) image taken with commercial low-cost CCD/CMOS cameras fluctuate within a range between two or three, corresponding nearly to 1% noise level. Besides, the projector also vary with a similar noise level. For this reason, we chose 2% random noise in the following simulations of nonlinearity and saturation, to consider for a practical situation. Figure 3(b) shows how the RMS phase error changes with nonlinearity (gamma distortion of detector and/or projector) for a linear phase fringe pattern with 2% random noise. When the gamma value is different from an ideal condition (γ=1), the RMS phase error substantially increases in the PSM. On the other hand, the SM and the ST-PSM are more robust to the nonlinearity. Besides, we also confirmed the same trend of RMS phase error is the same for three methods under 5% random noise. Figure 3(c) shows how RMS phase error changes with saturation level (S) in the range of 0 to 0.95 for a distorted fringe pattern with 2% random noise. When the saturation level exceeds 0.5, the RMS phase error suddenly increases in the PSM. Compared with the PSM, the SM and the ST-PSM are almost unsusceptible to saturation even when the saturation level is very high. Besides, we also confirmed the same trend of RMS phase error is the same for three methods under 5% random noise.   16-pixel when the fringe pitch is 10-pixel without noise. When the sampling pitch T is matched to the fringe pitch P, i.e. P=T=10 pixels, the phase error is zero. In other cases, the SM has a small (periodical) phase error.

Saturation.
According to our earlier study on the theoretical error analysis of the SM [22], we found that the phase distribution  figure 3(d). Besides, the frequency of the phase error is twice the frequency of the fringe pattern, as denoted in equation (22). Here, the On the other hand, ST-PSM can analyze the phase distribution without error even the sampling pitch is not matched to the fringe pattern, as shown in the red line in figure 3(d). In the SM, the periodical phase error due to mismatch of pitch strongly depends on the phase of the original fringe pattern, as denoted in equation (22). If the fringe pattern is shifted, the phase error of the fringe pattern is also shifted. Since all the phase-shifted fringe patterns are used in the ST-PSM, the periodical phase error is averaged in one-period and eventually neutralized. This is why the ST-PSM can automatically remove the phase error after 2D DFT calculation. This characteristic of the ST-PSM, which the SM does not possess, is crucial to practical application.

Simulation condition of multiple error sources
We also compare the measurement accuracy for the phase error mixed with more than two error source factors, since the measurement error is practically induced from the combination of various noise factors. To simulate an irregular shape target with a boundary under multiple error source factors, for the second set of simulation, four phase-shifted circular fringes with a diameter of 100 pixels were analyzed to visually evaluate the difference between the PSM and the ST-PSM. The image size is 100×100 pixels and the fringe pitch is 20-pixel to visualize the relationship between the phase error frequency and the fringe number (five fringes in the image). For the circular fringe patterns, the upper half image was analyzed by the PSM and the lower half image was analyzed by the ST-PSM. The RMS phase error was compared for ten different cases: Here, with an additional error factor (case D), we also considered the phase-shifting error (or vibration) since phaseshifting errors are always introduced in the phase-shifted images due to environmental disturbances. The phase-shifting errors for the third and the forth fringe images were set to / j p D = -15 3 and / j p D = 15, 4 respectively. Figure 4 shows the comparative results of RMS phase error for different conditions including single error factor (random noise, nonlinearity, saturation, and vibration) and the mixed factors of error sources. In this figure, we only show the first one of the 4-step phase-shifted fringe patterns. From these simulation results, the following points are important to mention:

Simulation results of multiple error sources
(1) Random noise: The measurement accuracy can be improved with an increase of the number of phaseshifting. In this simulation, the numbers of phaseshifting are N=4 for the PSM, T=20 for the SM, and N×T=80 for the ST-PSM, respectively. It can be found from [A] in figure 4 that the proposed method can effectively reduce the RMS phase error from 0.0698 rad to 0.0126 rad when the random noise was 10%.  figure 4, a more complex phase error appears in the results from the PSM when three or four noise factors are mixed. This is the reason why conventional filtering cannot eliminate the periodical phaser error. In contrast, the ST-PSM can directly and dramatically reduce the phase error without any additional filtering.

Discussion on simulation results
According to the simulation results of a single error source, we found that the phase error due to random noise can be dramatically reduced, by incorporating multiple phase-shifted Moiré fringes from the spatial and temporal domains simultaneously and the fringe phase can be accurately determined by utilizing the 2D DFT algorithm. The influence of other factors affecting the phase accuracy, such as nonlinearity, intensity saturation, and vibration, can be wholly eliminated, which is the most attractive advantage of the ST-PSM. Next, from the simulation results of multiple error sources, we can recognize the disadvantages of the conventional PSM algorithm that many researchers have attempted to reduce the error over past half century. That is to say, a precise optical measurement needs a high-performance camera and a good phase-shifter without vibration environment. Otherwise, a complicated periodical phase error could be appeared in the measurement result. On the other hand, the results from the ST-PSM algorithm show the highest measurement accuracy compared with the other two methods. These results demonstrate the ST-PSM algorithm can extract the phase information with high accuracy from a small number of phase-shifting fringe patterns.

Comparison of calculation speed
The computational efficiency is also an essential aspect of fringe pattern analysis. We compared the calculation time for the PSM and the ST-PSM using the same fringe images and the same computer environment. Table 1 indicates the comparison of computation time using the simulated fringe patterns (N=4 step; P=T= 10 pixels; image size=7680×4320 pixels). The calculation time is the average of 10 runs result. We used a Windows 7 Pro PC (Intel, Xeon ® CPU E5-2690 v2 @ 3.00 GHz, 10 of cores and 20 of threads; 64 GB RAM memory; 64-bit) to test the computation time using multi-core CPU. In addition, we also used a latest Windows 10 Pro PC (Intel, Core ™ i9-9980XE CPU @ 3.00 GHz; 128 GB RAM memory; 64-bit) with a newest graphics processing unit (GPU) board (GeForce RTX 2080Ti; CUDA Core=4352) to test the computation time using GPU.
The ST-PSM is 7.8 times slower than the PSM when a single-core CPU is used, since 2D DFT calculation uses 10 times (T=10 pixels) fringe image data. However, the calculation time can be shortened by utilizing multi-core CPU and GPU parallel computing. Compared with a single CPU, the speedup of 4-core CPU, 20-core CPU and GPU are 2.58×, 2.98×, and 17.94× for the PSM algorithm; and  The calculation time (s) is the average of 10 results for 4-step fringe patterns with 8 K image size (7680×4320 pixels); the sampling pitch is 10-pixel for the ST-PSM. 4.30×, 12.41 ×, and 93.14×for the ST-PSM algorithm. If a 20-core CPU or a GPU are used, the difference of calculation time is less than twice between the PSM and the ST-PSM. Furthermore, it should be noticed that 4-step phase-shifted fringe patterns with 8 K image size can be calculated within one second by recent GPU parallel computation.

Experiment
We further evaluated the performance of our proposed method through experiments. Since fringe projection profilometry (FPP) using a phase-shifting method has been extensively studied and recognized as one of the most effective techniques for noncontact 3D shape and deformation measurement in various engineering field [23][24][25]. The dominant phase error source in the FPP measurement are random noise, nonlinearity, and saturation. So, we measured a complex plaster model under three different exposure times including 0.001 s, 0.1 s, and 0.3 s to demonstrate the effectiveness of the developed method as a practical application.

Experimental setup
Here, a typical setup for non-contact 3D shape measurement by fringe projection is used, as shown in figure 5(a). A LED projector (NEC, ViewLight NP-L50WJD, 1280×800 pixels, 500 lm) projected a cosinesoidal pattern with 4-step phase-shifting onto a plaster model. A CMOS camera (The Imaging Source, DMK 33UX178, 3072×2048 pixels, 8-bit gray image) with a 16 mm focal lens (PENTAX, TV LENS C1614-M) recorded the projected fringe pattern from a different angle with nearly 20-degree. The distances between the target and the camera is 750 mm and between the target and the projector is 700 mm. A two-frequency temporal phaseunwrapping algorithm, i.e. Gushov-Solodkin phase unwrapping algorithm [26] was adopted for absolute phase retrieval. In our experiment, the relative prime numbers or grating pitch ratio are m 1 =8 pixels and m 2 =9 pixels. The unwrapped phase can be used for 3D reconstruction once the system is calibrated. Thus, a linear stage (OptoSigma, HST-100, positioning accuracy is 12 μm) controlled by a motion controller (OptoSigma, HSC-103) was used to move a flat plane as the reference plane in the position of 0 mm and 40 mm to calibrate the phase-to-height relationship.
The target object of a complex plaster statue is shown in figures 5(a) and (b) shows one example of the captured fringe pattern image in the central 1246×2048 pixel when the fringe pitch is eight pixels in the projector system. Figure 5(c) represents the fringe pattern recording by changing three different exposure times, i.e. 0.001 s, 0.1 s, and 0.3 s, in a zoomed area with 600×1000 pixels. The actual intensities of a cross-section in horizontal direction is plotted in the diagram below. Clearly, we can see the shutter speed of 0.1 s is an appropriate exposure for fringe analysis, and the shutter speed of 0.001 s is an underexposure and the random noise is conspicuously observed. Besides, the shutter speed of 0.3 s is an overexposure and saturation that could induce large phase error is observed in the image. Obviously, both the exposure times of 0.001 s and 0.3 s are very challenging measurement conditions.

Experimental results
Since the average fringe pitch for the captured image in the central area nearly equal to 30-pixel for 8-pixel fringe projection, and 34-pixel for 9-pixel fringe projection, the sampling pitches of the ST and the ST-PSM analyses were set to 30-pixel and 34-pixel, respectively. Then, we measure the wrapped phase distribution, and unwrapped these results by two-frequency temporal phase-unwrapping algorithm. The height distribution is determined from the phase difference between the object phase and the reference phase of a flat plane and a linear phase-to-height coefficient obtained by the calibration.  height distribution, we also compared with the phase gradient results since the accuracy of phase gradient is more sensitive to the phase distribution. Phase gradient is simply obtained by differential with the phase in the horizontal direction as 23 First, for an appropriate measurement condition, both the measured height distributions by the PSM and the ST-PSM seem good, as shown in figure 6. The gray tone scale indicates the height value in mm, and the color scale gives the measured phase gradient. The height error near the hollow of the eye by the SM is caused by the mismatch of the fringe pitch and the sampling pitch since such an area has a high curvature gradient.
Next, when we carefully compare the phase gradient distribution for three methods, the ST-PSM can completely remove the periodical phase error due to nonlinearity intensity response characteristic of the FPP system without using any additional filters (such as low-pass filter, sine/cosine filter, and other Gaussian filters), which indicates the outperforming of robust to nonlinearity and other error sources. Conversely, the spatial resolution of the ST-PSM could be slightly decreased than the PSM.
Second, for the underexposure condition, where the dominant error source is random noise, figures 7(a) and (b) show the comparison of height and phase gradient distributions obtained by the PSM, the SM, and the ST-PSM, respectively. Since the intensity of the recorded four-step phase-shifted fringe patterns are extremely low, as shown in figure 5(c), the measured result of the PSM includes a large error.
The measurement result of the SM is better than the PSM. However, there are some large measurement error in the hollow of the eye. On the other hand, both the height and phase gradient distributions can be nicely measured by the ST-PSM even the exposure time was extremely short (100 times shorter than the first measurement), since the intensity information in 2D domains is much richer than conventional 1D phase-shifting approach, which utilizes a 2D phase-shifting determination algorithm.
Similarly, figures 8(a) and (b) represent the results of the image with saturation owing to the excessive exposure time. The PSM cannot analyze accurately for a heavily saturated fringe pattern, as predicted from the simulation result of saturation ( figure 3(c)). The SM can analyze the height and phase gradient distributions better than the PSM. However, there are large errors in the hollows near the eye in the height distribution and some periodical error in the phase gradient distribution, due to the mismatch of fringe pitch and sampling  These experimental results confirm that the ST-PSM algorithm successfully improved the accuracy of measurement significantly, especially for the underexposure (0.001 s) and overexposure (0.3 s) conditions. Notably, the ST-PSM algorithm can completely remove the periodical phase error due to nonlinearity, saturation, and vibration, which has been an unsolved problem in the phase-shifting technique for a long time. It should be noted that the use a low-pass filter, sine/cosine filter and other Gaussian filters for denoising of fringe patterns are also useful for the ST-PSM to reduce the noise components of the captured fringe patterns. However, our experimental results indicated that an accurate measurement could be performed by the ST-PSM even without any filtering process, as shown in figures 6-8.

Discussions
We compared the measurement accuracy and speed of the ST-PSM with a conventional PSM by numerical simulations and experiments. These results indicated that our method allows the analysis of fringe patterns under extremely low SNR or extreme saturation conditions. The ST-PSM provided the robust phase retrieval ability of phase-shifted fringe pattern, showing the high potential and feasibility of optical metrology such as 3D shape and deformation measurement. Especially, the proposed ST-PSM using two-dimensional intensity data presented in this article has the following advantages: (1) Accurate and robust: The periodical phase error, due to nonlinearity, intensity saturation, and phase-shifting error, can be eliminated as well as the random noise. (2) Simple and fast: The phase analysis algorithm is simple since the down-sampling and intensity interpolation, and DFT calculation is a typical image processing, and the same image processing procedure is carried out for each pixel. There is no need to iterate and search calculation, which enables easy implementation for parallel computing using a multi-core CPU or GPU to accelerate the calculation speed. (3) Wide application: Since phase analysis can be applied to various applications such as high-sensitivity and fullfield deformation measurement by the Moiré technique [27], noncontact reflective-index measurement by phase-shift laser microscopes [28], 3D reconstruction [29] or nano-order deformation measurement by the phase-shifting digital holography [30], and x-ray interferometer [31] for non-destructive testing. Thus, the ST-PSM could be a useful measurement tool for such applications. For instance, in the biomedical x-ray imaging [32], the exposure time of x-ray possibly can be reduced to more than 10 times if the ST-PSM algorithm is adopted.
On the other hand, the phase-shifting method has the advantages of high resolution because it can obtain the coordinates of the object pixel-by-pixel. For the ST-PSM, the multiple phase-shifted Moiré fringes are generated by the image processing of down-sampling and intensity interpolation, thus the spatial resolution is slightly lower than the PSM. If the target with high curvature, a fringe projection with small grating pitch is preferred to be used for the ST-PSM. However, in recent years, the resolution of both the camera and the projector are dramatically enhanced. The spatial resolution is not a severe problem for the most case, except for the measurement of very sharp shape. Actually, in our experiment, the results obtained by the PSM and the ST-PSM seem almost the same, as shown in figure 6(a).

Conclusion
A novel spatiotemporal phase-shifting method was proposed for accurately measuring the phase distribution of fringe patterns. In the proposed method, multiple phase-shifted Moiré fringes from the spatial and temporal domains are generated by incorporating down-sampling technique. This method can drastically reduce phase error resulting from random noise. Besides, simulation and experimental results verified that other factors affecting the phase accuracy, such as nonlinearity, saturation, and vibration, can be wholly eliminated, which is considered to be the most attractive advantage compared with existing phase-shifting methods.
As a simple demonstration, we applied our method to a non-contact shape measurement in fringe projection profilometry. Experimental results indicated that the periodical phase error due to nonlinearity of the projector and intensity saturation of the camera could be eliminated by applying the proposed method. Furthermore, the method could analyze the fringe patterns captured under extremely low SNR conditions. Due to its robust, accuracy and fast calculation, this technique could have a significant impact on a variety of research and scientific fields, ranging from nano-scale to geometric-scale applications.