Characteristic and optimization of the effective perspective images’ segmentation and mosaicking (EPISM) based holographic stereogram: an optical transfer function approach

Based on our proposed method for holographic stereogram printing using effective perspective images’ segmentation and mosaicking (EPISM), we analyze the reconstructed wavefront errors, and establish the exit pupil function model of proposed printing system. To evaluate the imaging quality, the optical transfer function (OTF) of the holographic stereogram is modelled from the aspect of frequency response. The characteristic of the OTF with respect to the exit pupil size and the aberration are investigated in detail. We also consider the flipping effect in spatial domain. The optimization of hogel sizes, i.e., the sampling interval of original perspective images and the printing interval of synthetic effective perspective images, are given for the optimized reconstruction. Numerical simulations and optical experiments are implemented, and the results demonstrate the validity of our analysis, and the optimized parameters of hogel sizes can improve the imaging quality of full parallax holographic stereogram effectively.


Methods
Exit pupil function of EPISM based holographic stereogram. The exit pupil function model of EPISM based holographic stereogram is shown in Fig. 1. The virtual H 1 plate, LCD panel and H 2 plate are placed parallelly along the z-axis, and the distances among them are z 1 and z 2 , respectively. Point O on LCD panel is located on the center of hogels in virtual H 1 plate and H 2 plate, and light rays emitted by point O are diffused to the hogels with frustum patterns. The diameter of human pupil is assumed as about l e = 3~5 mm. The virtual hogels in virtual H 1 plate and the hogels in H 2 plate are all with square and hard exit pupils, whose sizes are l 1 and l 2 respectively, and they are supposed to be smaller than the size of human pupil, i.e., l e > l 1 and l e > l 2 .
For most holographic stereograms, there is some distance between the viewing plane and the exit pupil plane (virtual H 1 plate). For simplicity, the viewing plane is set at the exit pupil plane, and only one-dimension case is analyzed. The deduced conclusions can be extended to two-dimension case easily. Exit pupil is a virtual aperture 1 2 1 Optical transfer function with defocusing aberrations. In a conventional hologram, the wavefront of the 3D scene is reconstructed by the complete amplitude and phase information, where the amplitude and phase present the luminance and depth of any object point respectively. However, in the holographic stereogram, the accurate phase information isn't recorded when achieving a sequence of two-dimensional perspective images, as a series of wavefront segments are used to approximate the true wavefront of the 3D scene 44 . Many researchers are studying the accommodation cues and continuous motion parallax in holographic stereogram, and have proposed many solutions to solve the defocusing errors, such as reconfigurable image projection (RIP) algorithm 45,46 and diffraction specific coherent panoramagram (DSCP) algorithm 47 . For simplification of analysis in our manuscript, when observing the holographic stereogram, the perceived curvature radius of wavefront is only a measurement of the distance between the LCD panel and the observer. Obviously, there will be reconstructed wavefront errors during the hologram reconstruction when the object points aren't located at the LCD panel, and they are mainly expressed as defocusing aberrations. The defocusing aberrations will lead to the decline of amplitude and the variation of phase of the wavefront, reducing the imaging quality.   For an isolated point in 3D space, its wavefront can be considered as a spherical pattern, and the reconstructed wavefront error with respect to the original scene depth is shown in Fig. 3. The object point is assumed to be at the position (x 0 , z 0 ) in space, and the observer at the center of virtual hogel in virtual H 1 plate will locate the point at (x 0 , z 1 ), where x 0 is the central coordinate of the exit pupil in the viewing plane. The origin point of z-axis is fixed at the viewing plane.
When the pixel of point O on LCD panel is diffused to the exit pupil, with a Fresnel approximation, the curvature error of wavefront can be expressed as It is supposed that x 0 = 0, and the first term z 0 − z 1 will be omitted in the future analysis as it only adds a constant phase factor.
Then the defocusing aberrations can be expressed as 1 0 0 1 2 where k = 2/λ is the wavenumber and λ is the wavelength of the laser source. For an optics system with a square and hard exit pupil, the generalized exit pupil function P(x) can be written as x l ( ) rect( / ), is the rectangle window function. The generalized exit pupil function P(x) contains not only the limitations of the size and the shape of the exit pupil, but also the function of the system aberrations. The diffraction limited system can be usually regarded as a linear shift-invariant system. With the knowledge of Fourier transform (FT) and the point spread function (PSF), the impulse response of the system is proportional to the FT of the exit pupil function. Then the coherent transfer function λz f P( ) x 1 of the system can be expressed as The optical transfer function OTF (f x ) can be calculated by the normalized autocorrelation function of its coherent transfer function, and it has been sufficiently studied by Goodman 48 . The result is shown as follows, where sinc(x) = sin(πx)/πx, and the triangular function is defined as Similarly, the OTF of a full parallax holographic stereogram is x y x y x Moreover, when the viewing plane is not coincident with the virtual H 1 plane, we suppose the coordinate of viewing plane is z v on the z-axis (z v < 0). Then the reconstructed wavefront error in holographic stereogram is shown in Fig. 4.
, where l is shown in Eq. (1). If l v ≥ l e , the exit pupil size of the system is l e , otherwise, the exit pupil size of the system is l v . Then the OTF of a full parallax holographic stereogram can be expressed as

Results and Discussions
Numerical simulations. From the analysis mentioned above, there are defocusing aberrations during the reconstruction of holographic stereogram, and the defocusing aberration is determined by the distance between the object point in space and the LCD panel, and the farther the distance is, the greater the aberration will be. To illustrate the influence of exit pupil size on the OTF, the parameters of printing system in our previous work 37 are taken as an example. The field of view (FOV) is supposed as θ = 30°, and the corresponding parameters are z 0 = 186 mm and λ = 639 nm. The LCD panel is located at the mid-plane of the 3D scene. The depth distribution of the 3D scene is about Δz = −50 mm ~50 mm, where it is supposed that Δz = 0 at the LCD panel. For different aberration conditions, the optimized sizes of exit pupil functions will not be the same.
We choose three different planes, which are non-aberration plane at Δz = 0 mm, middle aberration plane at Δz = −20 mm and high aberration plane at Δz = −50 mm, and calculate the OTF of each plane against exit pupil size and spatial frequency. The results are shown in Fig. 5. For simplicity, the results only show the variations of OTF based on spatial frequency f x and exit pupil size l, and they are only the cases of positive spatial frequency, i.e., f x > 0.
The max spatial frequency is set as 5 cycles/mm according to the characteristic of the LCD panel we used. In our experiment, a LCD panel (VVX09F035M20) produced by Panasonic is used. It is 8.9 inches with 1920 × 1080 pixels, and the pixel interval is about Δl = 0.1 mm. Therefore, the maximum spatial frequency of image displayed by the LCD panel is calculated as m ax 1 2 = 5 cycles/mm. As shown in Fig. 5, the OTF drops towards zero rapidly at relatively small spatial frequency for very small values of l. When there is a defocusing aberration, the OTF decreases more quickly, and there occurs oscillations on the edges, and the value of OTF turns to be negative. That is to say, the sign reversal of the OTF occurs, which means the contrast reversal, and will reduce the imaging quality seriously. Meanwhile, the greater the defocusing aberration is, the more rapid the descent and the more obvious the oscillations of the OTF will be, which will lead to a reduction of reconstruction quality.
For a certain defocusing aberration of the 3D scene plane, to investigate the global behavior of full parallax holographic stereogram on all spatial frequencies, the optimized exit pupil size is determined by the corresponding OTF which has the highest average integral value across the passband, which can be calculated as A larger ϒ means a higher modulated ability for the whole passband. Considering the holographic stereogram printing system which we have given, the variations of ϒ with respect to exit pupil size in different aberration planes are shown in Fig. 6.
As shown in Fig. 6, the optimized exit pupil sizes l opt of holographic stereograms with non-aberration, middle aberration and high aberration are 5 mm, 1.2 mm and 0.7 mm, respectively. A larger exit pupil size is better when there is no aberration, and the holographic stereogram has the highest average integral value of the OTF. Because of the limited human pupil size, the maximum exit pupil size cannot exceed 5 mm. For the plane with aberration, the greater the aberration is, the smaller the optimized exit pupil size will be.
The behavior of OTF with the optimized exit pupil sizes l opt under different aberration conditions are shown in Fig. 7. Obviously, the greater the aberration is, the more rapidly the OTF decreases, and there are no oscillations occoured in Fig. 7 as the optimized exit pupil sizes are used under different aberration conditions.
For the more general case, optimized exit pupil size under different aberration conditions are given in Fig. 8, and Δz is changing from −50 mm to 50 mm. As shown in Fig. 7, for the same 3D scene depth (the absolute values of aberrations are identical), the optimized exit pupil sizes are slightly different when the value of aberration is positive or negative. However, the variation trend is consistent, namely, the higher the 3D scene plane deviating from the LCD panel (zero error plane) is, the smaller the corresponding optimized exit pupil size will be.
In order to verify the relationship between the optimized exit pupil size and the aberration, we use single perspective image of a resolution test target model as the original input image to conduct the simulation. For the three cases of non-aberration plane, middle aberration plane and high aberration plane, after passing through different pupils, the reconstructed images are shown in Fig. 9. The values of peak signal to noise ratio (PSRN) of different reconstructed images are also calculated as shown in Table 1.
It can be seen that the simulation results are consistent with the theoretical analyses above. The aberrations will degrade the imaging quality, and the greater the aberration is, the worse the imaging quality will be. The optimized exit pupil size is 5 mm when there is no aberration, and the value of PSRN is the highest, while the optimized exit pupil sizes are 1.2 mm and 0.7 mm in middle-aberration plane and high-aberration plane respectively.
From the analysis above, we can get the optimized exit pupil size corresponding to a certain depth plane of the 3D scene. When considering the complete 3D scene, the variation of aberration is added to the evaluation index ϒ tot , namely, the depth of the 3D scene is also considered to be integrated. A larger ϒ tot means a higher modulated ability for the whole passband of the complete 3D scene.  The variations of ϒ tot with respect to exit pupil size in different depth distributions of the 3D scene are shown in Fig. 10. The optimized values of exit pupil size are 2.55 mm, 1.75 mm and 1.1 mm respectively for 3D scenes with different depths. The results indicate that there is an optimized value of the exit pupil size with a certain depth distribution of the 3D scene, and this optimized value is smaller when the depth distribution of the 3D scene is broader.
Optical experiments. To demonstrate the validity of our analysis in numerical simulations when the reconstructed image is under a certain aberration condition, an optical experiment is designed. The experimental model with resolution test target under different aberration conditions is shown in Fig. 11. The distance between the camera sampling plane and the LCD panel is 11.4 cm, while the resolution test target plane is 0 cm, 2 cm, or 5 cm distant from the LCD panel, to express non-aberration aberration, middle aberration or high conditions, respectively. The sampling grid is 6 × 6 or 15 × 15, while the sampling interval is 0.5 cm or 0.2 cm correspondingly. The resolution of sampling perspective images is 600 pixel × 600 pixel. The hogel sizes of holographic stereogram printing are 0.5 cm or 0.2 cm in practice.
The optical setup of the holographic stereogram printing system is shown in Fig. 12. A 400 mW 639 nm single longitudinal mode and linear polarization solid-state red laser (CNI MSL-FN-639) is used as the laser source, and an electric shutter (Sigma Koki SSH-C2B) is used to control the exposure time. The laser beam passes a λ/2 wave-plate and a polarizing beam splitter (PBS), then divides into two beams, i.e., the signal beam and the reference beam. Intensity ratio of the signal beam and the reference beam is adjusted by the first λ/2 wave-plate, and the other λ/2 waveplate is used to adjust the polarization state of the reference beam, to keep the polarization state consistent between the two beams. On the signal beam path, a series of images are displayed on the LCD panel with a diffuser, then the light rays with image information are diffused onto the holographic plate. Holographic plate is sandwiched between two apertures to ensure only a square area of the holographic plate (i.e., the hogel) exposed, and it is installed on a motorized KSA300 X-Y stage which is driven by a programmable MC600 controller. The distance between the LCD panel and the holographic plate is 11.4 cm. The reference beam passes through a spatial filter comprised of a 40× objective and a 15 μm pin-hole to filter out the higher spatial frequency, then modulated by a collimating lens to get a uniform plane wave. The reference beam is about 40° off from the normal axis of the holographic plate. The signal beam and the reference beam are interfered from different sides, and the interference fringes are recorded on the holographic plate.  In our EPISM based holographic stereogram printing system, we don't adopt a more usual printer configuration with a FT high numerical aperture (NA) lens system which is referred to as a lens-based printer. The reasons are as follows. When utilizing the NA objective lens, the FOV of the hologram will be fixed, however the value of FOV can be variable in EPISM based holographic stereogram. Moreover, the objective lens will bring in the image distortion more or less. It is such a principle verification experiment that we don't care much about the printing efficiency, so a LCD panel and a diffuser are used for printing, not a lens-based configuration. Nevertheless, the EPISM based holographic stereogram can be also applied to the lens-based printers, as long as the field angle between the real image of SLM and the hogel keeps the same with the field angle between the LCD panel and the hogel.   A Canon EOS 5D camera with a 100-mm focus lens is put about 40 cm in front of the holographic plate to capture the reconstructed images. Optical reconstruction images are shown in Fig. 13 under different aberration conditions, and they are all virtual images inside of the holographic plate.
As shown in Fig. 13, when it is under the non-aberration condition, the reconstructed effects are both well whether the hogel size is 2 mm or 5 mm. When there exists an aberration, the reconstructed effect with 2 mm hogel size is much better than that of 5 mm hogel size, especially under a higher aberration condition. If the aberration is −50 mm and the hogel size is 5 mm, the effect is so bad that we can hardly focus on the reconstructed image when capturing photos. It should be pointed out that, since the camera with a macro lens is used, the view field size could be captured by the camera lens is limited. When the aberration is higher, the reconstructed image is closer to the camera, so the range of the image captured by the camera is smaller.
Take the middle aberration condition as an example, to express the location of reconstructed image. One ruler is placed parallel to the holographic plate, and the other one is placed 9.4 cm behind. The images captured at different focus depths are shown in Fig. 14. The spatial position relation is shown in Fig. 14(a). In Fig. 14(b), both the reconstructed image and ruler1 are clear simultaneously while ruler2 is blurred. In Fig. 14(c), both the holographic plate and ruler2 are clear simultaneously, and the printed hogels can be observed clearly, while ruler1 is blurred.
We have discussed the effect of the exit pupil size on the imaging quality from the aspect of frequency domain above. Now we will take the spatial domain analysis into consideration, and consider whether the information can transmit completely during the simulated reproduction process of master hologram (virtual H 1 plate) to transfer hologram (H 2 plate), to find the optimized hogel sizes for EPISM based holographic stereogram printing system, i.e., the optimized size of virtual hogel (l 1 ) in virtual H 1 plate and the optimized size (l 2 ) of hogel in H 2 plate. Based on the depth distribution of the 3D scene, and the virtual or real reconstructed effect finally, we can determine the parameter z 2 of the EPISM based holographic stereogram printing system.
Considering the flipping effect when observing the holographic stereogram, Yatagai analyzed the maximum value of 3D scene depth range Δz range in spatial domain as 49  ensure that there is no information lost during the reproduction. Consequently, the exit pupil is determined by the hogel in H 2 plate as shown in Fig. 2. Furthermore, as shown in Fig. 10, when the depth distribution of the 3D scene is determined, the hogel size can be optimized from the frequency domain analysis to achieve a higher modulated ability of the hologram.
More specifically, the reconstructed 3D scene is supposed to be 114 mm outside of the holographic medium, and the parameters z 1 = 186 mm, z 1 = 114 mm, λ = 639 nm and Δz range = 2 × 10 = 20 mm are taken into equations above, finally the optimized hogel sizes for EPISM based holographic stereogram printing system are In EPISM based holographic stereogram printing system with above conditions, the ideal value of l 1 is 1.64 mm, and it represents the sampling interval of the original perspective images. However, this value is too small to resulted in a serious time cost for sampling, and it is the most extreme condition to avoid flipping effect in terms of the farthest object point deviating away from the LCD panel. During the experiment, we find that when l 1 = 3.5 mm, it is enough to get a good effect for synthetic effective perspective images' segmentation and mosaicking. Then, < l l z z 2 1 2 1 = 2.14 mm. Referring to Fig. 10, the optimized exit pupil size is 2.55 mm. Since the exit  mm. Considering the practical situation, the value of l 2 is chosen as 1.5 mm in optical holographic stereogram printing eventually.
A dinosaur model with 6.2-cm width, 2.8-cm height and 2-cm depth is utilized as the 3D scene. The sampling interval of original perspective images is 3.5 mm, and the sampling count is 63 × 63. The printing interval of synthetic effective perspective images is 1.5 mm, and the number of hogels is 40 × 40 = 1600. The photographs of optical reconstruction from different perspectives are shown in Fig. 15, and the parallax can be reflected by the shape of the cube obviously. It can be seen that the total reconstruction quality of full parallax holographic stereogram is well when adopting the optimized parameters.
According to the analysis above, we can also get the optimized values for other parameter sets, such as different hologram sizes, different viewing distances, and different camera positions, applying the optimized procedure illustrated from Eq. (1) to Eq. (14). Therefore, the conclusions can be extended to different and more general situations.

Conclusions
In this paper, the exit pupil function model of EPISM based holographic stereogram printing system is established, and reconstructed wavefront errors of the system are investigated and expressed as defocusing aberration. In practice, the square and hard window function is used as exit pupil function. To evaluate the imaging quality, the OTF of holographic stereogram is analyzed. Frequency responses indicate that there is an optimized value of the exit pupil size with a certain depth distribution of the 3D scene, and it is smaller when the depth distribution of the 3D scene is broader. In addition, considering the flipping effect in spatial domain, the maximum value of hogel size in virtual H 1 plate can be achieved. Considering the complete transitivity of the image information during the simulated reproduction process of virtual H 1 plate to H 2 plate, the relationship between their hogel sizes is achieved. Therefore, the optimized hogel sizes in EPISM based holographic stereogram printing system can be achieved, i.e., the optimized sampling interval of original perspective images and the optimized printing interval of synthetic effective perspective images. The theoretical and experimental results indicate that the modelling agrees well with the experiments, and our OTF method will be helpful to improve the imaging quality of EPISM based holographic stereogram.