Formation mechanism and a universal period formula for the CCD moiré

Moiré technique is often used to measure surface morphology and deformation fields. CCD moiré is a special kind of moiré and is produced when a digital camera is used to capture periodic grid structures, like gratings. Different from the ordinary moiré setups with two gratings, however, CCD moiré requires only one grating. But the formation mechanism is not fully understood and also, a high-quality CCD moiré pattern is hard to achieve. In this paper, the formation mechanism of a CCD moiré pattern, based on the imaging principle of a digital camera, is analyzed and a way of simulating the pattern is proposed. A universal period formula is also proposed and the validity of the simulation and formula is verified by experiments. The proposed model is shown to be an efficient guide for obtaining high-quality CCD moiré patterns. ©2014 Optical Society of America OCIS codes: (050.0050) Diffraction and gratings; (050.2770) Gratings; (110.2650) Fringe analysis. References and links 1. D. M. Meadows, W. O. Johnson, and J. B. Allen, “Generation of surface contours by moiré patterns,” Appl. Opt. 9(4), 942–947 (1970). 2. D. Post, B. Han, and P. Ifju, High Sensitivity Moiré: Experimental Analysis for Mechanics and Materials (Springer, 1994). 3. P. K. Rastogi and E. Hack, Optical Methods for Solid Mechanics: A Full-Field Approach (Wiley-VCH, 2012) 4. W. N. Sharpe, Handbook of Experimental Solid Mechanics (Springer, 2007). 5. J. A. Buytaert and J. J. Dirckx, “Moiré profilometry using liquid crystals for projection and demodulation,” Opt. Express 16(1), 179–193 (2008). 6. X. Xiao, Y. Kang, Z. Hou, W. Qiu, X. Li, and X. Li, “Displacement and strain measurement by circular and radial gratings moire method,” Exp. Mech. 50(2), 239–244 (2010). 7. C. Quan, Y. Fu, and C. J. Tay, “Determination of surface contour by temporal analysis of shadow moiré fringes,” Opt. Commun. 230(1–3), 23–33 (2004). 8. T. Tu and W. B. Goh, “Moiré pattern from a CCD camera – are they annoying artifacts or can they be useful,” in Proceedings of International Conference on Computer Vision Theory and Applications (2009). 9. R. S. Chang, J. Y. Sheu, C. H. Lin, and H. C. Liu, “Analysis of CCD moiré pattern for micro-range measurements using the wavelet transform,” Opt. Laser Technol. 35(1), 43–47 (2003). 10. T. Tu and W. B. Goh, Using CCD Moiré Pattern Analysis to Implement Pressure-Sensitive Touch Surfaces (Springer, 2009).


Introduction
A moiré pattern is produced by superimposing two sets of gratings whose frequencies and structures satisfy certain conditions [1][2][3][4]. It is often used to measure deformation fields and surface morphology via different optical paths and using various kinds of data processing methods [5][6][7].
The two gratings that are used to produce a moiré pattern are referred to as a reference grating and a specimen grating. Moiré-like schlieren are often observed when periodic structures, such as striped clothes and buildings, are captured by a digital camera. Analogical to the moiré pattern formation principle, the schlieren in digital images are generated by the interaction of a periodical structure (specimen grating) and the periodical imaging cells on the CCD device (reference grating). This special kind of moiré pattern is referred to as a CCD moiré pattern [8]. For most imaging cases, this kind of moiré pattern should be avoided as it will seriously impair image quality. However, geometrical information of a captured structure can be extracted from a moiré pattern and hence it can be used to measure the deformation field and surface morphology. Chang, et al. [9] first suggested a CCD moiré method and measured the micro-range distance between the CCD chip and a two-dimensional grating by analyzing the moiré fringe using a wavelet transform algorithm. Tu and Goh [8,10] proposed a subsampling method to analyze the CCD moiré pattern and reconstruct a curved surface. Compared with the ordinary moiré method, a CCD moiré pattern, with the reference grating omitted, is more convenient to the implementation of measurements and is especially suitable for measuring the deformation and morphology of some large outdoor structures. On the other hand, compared with ordinary image analysis based measurements, the CCD moiré method shares the same optical path but can achieve a higher space resolution.
To realize the measurement, one should first analyze the relationship between the pitch (or frequency) of the moiré pattern and the deformation of the specimen grating. Chang, et al. [9] adopted the formula of an ordinary moiré method, using a specimen grating of 300 lines/in, so that the ratio of the specimen grating and reference grating p s /p r is close to 1. Tu and Goh [8,10] expanded the formula to satisfy the condition where p s /p r is much bigger than 1; this permits the use of a printed grating in the experiment, in which case p s /p r ≈4. But the analysis of a CCD moiré pattern is still based on the framework of an ordinary moiré method [8,10], assuming that the periodical arrangement of the CCD chip is merely a reference grating in the optical path of the CCD moiré pattern; the imaging principle of the CCD chip was not taken into account. In comparison to the ordinary moiré method in which p s /p r is always controlled to be close to 1, the value of p s /p r in the CCD moiré method can vary across a much wider range. Extensive analysis shows that it is hard for the expanded formula to depict the CCD moiré method when p s /p r <1. Furthermore, the formula can only satisfy half of the cases where p s /p r is much bigger than 1. In this paper, the formation mechanism of the CCD moiré method based on the imaging principle of a digital camera is explored and an explanation that the CCD moiré pattern is formed by a periodic variation of digital sampling results when capturing a periodic structure is studied. Based on the explanation above, a method to simulate the CCD moiré method was developed, the disciplines of the periodic sampling results in a CCD moiré pattern is summarized and a universal formula that satisfies all the p s /p r cases is proposed. The formula was verified by simulation and experiments, and factors influencing the quality of the moiré pattern are discussed. This paper is organized as follows: in the second section the principles and simulation of the CCD moiré method is introduced. In the third section, the formula of a moiré pattern is derived and the factors that influence the quality of a moiré pattern are discussed. In the fourth section, the proposed formula is verified with experiments.

Principles and simulation of CCD moiré patterns
Similar to the ordinary moiré method, the formation of a CCD moiré pattern can be explained as illustrated in Fig. 1. When a periodical grating is captured by a camera, a virtual specimen grating (real image formed by lens) is formed on the CCD target. Given the periodic arrangement of the CCD target, the light intensity sensed by the CCD cells will vary periodically. A moiré pattern is formed when the grating and CCD target satisfy certain conditions. The CCD moiré formation process was simulated based on the imaging principle of a CCD camera. As is shown in Fig. 2 (1) A virtual grating, whose pitch is p, is projected on the CCD target. An imaging unit is shown in Fig. 2(b). For a single CCD unit, b is the width of the white lines (regions that are illuminated) that are projected on a valid photosensitive region; and I is the light intensity of the white lines. In normal CCD working conditions without any noise, the gray scale value G output by the imaging unit and the average irradiation intensity on it can be expressed as a linear relationship: where C is a constant related to the physical parameters of the chip and camera. Thus, all of the outlet gray scale values of the CCD units can be calculated. Assuming that CI = 1,q = 1,and d v = 0.7, the simulated CCD moiré patterns with different values of p are shown in Fig.  3. From this, we can see that the CCD moiré patterns are of different shapes and clarity against varying p. Generally, the moiré patterns are trapezoidal waves. When p/q is close to 1, the moiré pattern is clearer and the waveform is close to a sinusoidal wave. When p/q is much smaller than 1, the moiré pattern becomes obscure. When p/q is much larger than 1 (for example, p/q = 1.94) the moiré pattern contains high frequency artifacts. By fitting the data obtained from a 1D cross-section of the image using a trigonometric function, the pitch of the moiré pattern can be obtained, as shown in Fig. 3(d) (the noise was first removed using a subsampling method [8] if it contained high frequency artifacts).

Period and clarity of a CCD moiré pattern
For the data analysis of a CCD moiré pattern, it is most important to study the relationship between the pitch of the moiré pattern against that of the specimen grating. With this relationship, the specimen deformation can be calculated from the moiré fringe and a measurement can thus be realized.

Existing theories and formulas
The present study applies theory of geometric moiré patterns, in which the pitches of the reference and specimen gratings are p s = p and p r = q respectively. Since the CCD moiré itself is a digital image, we can define q as the unit of length and the period of the moiré pattern T can be expressed in pixel coordinates as; where p m is the period of the moiré pattern. The period of the ordinary moiré pattern (also used in CCD moiré method by Chang, et al. [9]) can be written as Similarly, the period of the moiré pattern (given by Tu and Goh) when p is much larger than q can be written as: where   is the flooring function. Equation (5) expands the application scope of the moiré formula.
Let n be the ratio of p and pixel size q: Transforming the classic Eq. (4) with Eq. (7) gives; Similarly, simplifying Eq. (5) gives 1 n T m n = − + According to the above simulation method, the moiré patterns were obtained with n varying from 0 to 3.5 using a step size of 0.001. The period T of the simulated moiré pattern under these conditions was then calculated, thus obtaining the curve of pitch n and moiré pattern period T. The simulated value of T and theoretical values calculated by Eq. (8) and Eq. (9) were compared as shown in Fig. 4. As can be seen in Fig. 4, the classic geometric moiré formula is valid in the vicinity of n = 1 (light gray region), while the expanded formula proposed by Tu and Goh [8] is valid when k-0.5<n<k (k is integer no less than 2) (dark gray region). However, no formula is proposed for the white region. In view of this, the existing formulas cannot describe all of the CCD moiré phenomena. Therefore, a universal moiré period formula is needed.

More universal theories and formulas
First, it is noted that the existing formula does not hold when n<1. This is because the pixel cells (the basic imaging units) can only output in gray scale representing the average irradiation intensity illuminated on them. If the CCD target array is considered as a reference grating and the formed moiré pattern is simply captured by CCD itself, then the moiré pattern will contain high frequency artifacts in the case where n<0.5. However, results of the simulation show that there are no high frequency artifacts when n<1, as shown in Fig. 3(a) (n = 0.33). This indicates that the CCD array cannot act as a reference grating, and the CCD moiré pattern is formed because of the periodic gray scale variation of an image (caused by discontinuity of the CCD target).
When n≥1, the difference between n and its nearest integer is |[n]-n|. After k cycles, the relative position of the grid lines and the unit will be repeated.
It is noted that in the cases where n is an integer, T tends to be infinite and moiré phenomenon disappears. When n<1, several whole lines should be visible making it closer to 1 (the cell size). The difference between the combined 'line' and 1 is |n × [1/n]-1|. In this case, Thus the moiré period is To sum up, a moiré period formula that is suitable in all cases is given as: From Eq. (14), it can be seen that when n is around 1, the formula degenerates to the wellknown Eq. (8). This formula indicates that the exact range where the geometric moiré formula can be used is 2/3<n<3/2. When k-0.5<n<k (k is integer no less than 2), it degenerates to the formula proposed by Tu and Goh [8].
Comparing calculation results from Eq. (14) with simulation results (Fig. 5(a) and (b)), it can be seen that the results of the two are consistent.
From Fig. 5 and Eq. (14), it was found that when n is near k or 1/k (k is an integer), the CCD moiré period tends to be infinite, indicating a constant gray scale distribution in the image. Yet it can also be seen from Fig. 5 that the moiré period is also infinite when n = 0.7 (star in Fig. 5). This is because the pitch of the grating equals the length of a CCD cell (q = 1, d v = 0.7, n = 0.7) in the simulation, such that all the pixels are illuminated equally, resulting in equality of gray scales within the image and a disappearance of the moiré pattern accordingly.

Clarity of a CCD moiré pattern
Comparing Fig. 3(a) (n = 0.33) and Fig. 3(b) (n = 1.1019), it can be seen that the moiré pattern in Fig. 3(b) is much clearer. This indicates that the variation of n may affect the clarity of the CCD moiré pattern. Considering that the gray scale in the simulation varies from 0 to 1, the contrast of the CCD moiré pattern G r can be expressed by the difference between the maximum and minimum gray scale in the image: Figure 6 shows the curve of the gray scale contrast G r varies with n (d v = 0.7). From Fig.  6, it can be seen that when n>2d v , G r will always be 1, therefore high frequency artifacts become the main factor that affects the clarity of a moiré pattern; when n<2d v , contrast G r varies with n, and the clarity of a CCD moiré pattern can be represented by G r . In the case of n<2d v , G r has a trend of decreasing with the decline of n, and the moiré pattern becomes obscure. Also, it occurs that G r = 0 at n = d v /k (k is an integer). This is because each CCD unit is illuminated equally when n = d v /k (k is an integer), thus the outlet gray scales are equal and the moiré period becomes infinite. The model described above can explain the existence of high frequency artifacts in a CCD moiré pattern. In the cases where n>2d v , some units are fully lit while some are not illuminated at all, therefore the moiré pattern produced contains high frequency artifacts, whereas in the case of n<2d v , the units are only be partly illuminated thus the moiré pattern is just a periodic variation of the gray scale without high frequency artifacts. Also, the model gives guidance on how to get a high quality CCD moiré pattern. The main factors that affect the quality of the CCD moiré pattern are the moiré period and clarity. For display devices, several pixels can be difficult to be distinguished with naked eye, therefore the moiré phenomenon is almost invisible in the vast majority of cases, and is only obvious when the moiré period is relatively large. From the figures it can be seen that the moiré pattern is more likely to be observed only when n is near k or 1/k (k is an integer) because the moiré period is relatively large. Meanwhile, since the moiré pattern itself is a kind of amplification, a larger moiré period denotes a larger amplification, thus giving a higher sensitivity. In the cases where n<2d v , the clarity can be represented by contrast, while in the cases where n>2d v , high frequency artifacts should be considered.

Experimental verification
The moiré period formula proposed above was verified by experiments, and the experimental arrangement is shown in Fig. 7. An image was captured using a digital camera (Basler A641f, whose resolution is 1624 × 1236, pixel size q is 4.4μm) with a zoom lens (AVENIR, focal length 11-110mm) of a grating whose pitch p, was known. Two dots were printed on the grating separated by a distance d. If the coordinates of the center of the two dots are (x 1 , y 1 ) and (x 2 , y 2 ) respectively, the pitch of the image grating p projected on CCD target can be calculated. The ratio of p and q, i.e. n can be obtained as, By adjusting focal length, moiré images with different n can be obtained. The moiré patterns obtained are shown in Fig. 8. Using the method proposed in section 2, the moiré period T can be calculated.  The captured image (n = 0.52, 1.10, 2.18 respectively) is shown in Fig. 8. The experimental results agreed with the theoretical calculations obtained by Eq. (14) to a great extent, as is shown in Fig. 9. Thus the effectiveness of the universal formula given above was verified. Fig. 9. A comparison between experimental and theoretical results.

Conclusion
The CCD moiré method can be observed when the periodic structures are captured by digital camera, which can be used in measurements. The study shows that the CCD moiré pattern is not formed by superimposing two sets of 'gratings', as is widely accepted, but by periodic variation of the digital sampling results when it captures a periodic structure. The mechanisms are different when the relation between the pixel size of a CCD camera q and the pitch of the image projected on the CCD target p varies. A universal period formula was also proposed based on these mechanisms. According to the universal formula proposed in this paper, when p is close to q (more specifically, 2/3<p/q<3/2), the moiré period formula is identical to the well-known formula; when p/q>3/2, the CCD units should be taken as a whole to make pitch of p close to q; and in the case of p/q<2/3, the grating lines should be taken as a whole. The proposed formula is applicable on the whole range, and also, it can degenerate to the existing formula in the certain ranges. On the other hand, artifacts are determined by the valid photosensitive width of each unit (d v ). In the cases where n>2d v , the CCD moiré pattern contains high frequency artifacts; while in the cases where n<2d v , it is just a periodic variation of the gray scale without any high frequency artifacts.
The main factors that affect quality of a CCD moiré pattern are the moiré period and the clarity. In respect of the moiré period, when p/q is near k or 1/k (k is an integer), it may not be possible to observe the moiré pattern because its period is relatively large. In respect of the clarity, when n<2d v , the moiré pattern with higher contrast will be clearer, and in the cases where n>2d v , high frequency artifacts should be considered. It can be said that, as long as a digital camera is used to capture a periodical structure, a CCD moiré pattern occurs. Under most other circumstances, however, the moiré phenomenon can hardly be seen because it is so compacted or indistinct. If good use is made of the CCD moiré method given in this paper, a high-quality moiré pattern can be achieved by adjusting the imaging parameters. This is very meaningful for measurement of CCD moiré patterns.
In actual measurements, illumination, noise and the CCD device itself will make the moiré pattern worse than the simulation. It will be greatly convenient to get a moiré pattern with a higher quality with the model.