Holographic display having a wide viewing zone using a MEMS SLM without pixel pitch reduction

A one-micron pixel pitch is believed to be required for spatial light modulators (SLMs) to realize holographic displays possessing a wide viewing zone. This study proposes the use of a microelectromechanical systems (MEMS) SLM for not only displaying holographic patterns but also scanning laser beam. During the rotation of MEMS mirrors in the MEMS SLM, the timing of laser pulses illuminating the MEMS SLM is controlled to change the reflection direction of light modulated by the MEMS SLM in order to enlarge the viewing zone. In this technique, the width of the viewing zone depends on the rotation angle of MEMS mirrors, and not on the pitch of pixels (MEMS mirrors). We experimentally demonstrated the enlargement of the viewing zone angle to ∼40° using the MEMS SLM with a pixel pitch of 13.68 μm. © 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement


Introduction
Holography [1] can reconstruct the wavefront of light emitted from three-dimensional (3D) objects. This enables all physiological factors of human depth perception [2] to accurately function in the case of 3D images generated by holography. Therefore, holography provides ideal 3D images that are free from the visual fatigue caused by vergence-accommodation conflict [3]. Visual fatigue prevents the widespread use of current 3D displays. The electronic implementation of holography is difficult because conventional holographic display techniques require a one-micron pixel pitch for spatial light modulators (SLMs), which are used to display hologram patterns, to obtain a wide viewing zone (the pixel pitch requirement is explained in the next paragraph). This study proposes a technique for enlarging the viewing zone of electronic holographic displays using a microelectromechanical systems (MEMS) SLM illuminated by pulse-modulated laser light. The proposed technique does not require the reduction of the pixel pitch of SLMs.
An SLM with a narrower pixel pitch diffracts light at a wider angle so that the viewing zone of 3D images is enlarged. When the pixel pitch of the SLM is denoted by p and the resolution is denoted by N × M, the viewing zone angle is given by 2 sin −1 (λ / 2p) [4], where λ is the wavelength of light. The display screen size is given by Np × Mp. For example, to obtain a viewing zone angle of 30°and a screen size of 40 inches, the pixel pitch should be 0.97 µm and the resolution should be approximately 886,600 × 498,000, when λ = 0.5 µm.
The minimum pixel pitch of present SLMs is approximately 4 µm, which provides a viewing zone angle of approximately 7°when λ = 0.5 µm. Many efforts have been conducted to develop ultra-high-resolution SLMs with a one-micron pixel pitch [5][6][7][8]. The use of multiple SLMs has also been proposed to increase the viewing zone angle and screen size of holographic display systems [9][10][11][12]. To increase the viewing zone, a different approach that does not require pixel pitch reduction has been proposed. The combination of an acousto-optic modulator (AOM) and two-dimensional (2D) light scanning was developed [13,14], where the one-dimensional (1D) high-resolution hologram distribution generated by the AOM is scanned two-dimensionally on the display screen. Because the AOM modulates light only in a horizontal direction, this system provides only horizontal parallax.
Our research group proposed the combination of 2D light modulation by the MEMS SLM and 1D light scanning by a horizontal scanner [15][16][17][18][19][20], as shown in Fig. 1. This technique can enlarge both the screen size and the viewing zone of electronic holographic displays. The screen size of the MEMS SLM is enlarged to increase the screen size of the display using a magnifying imaging system. In this case, the pixel pitch increases so that the viewing zone reduces. The reduced viewing zone is scanned by the horizontal scanner to enlarge the viewing zone. Therefore, both the viewing zone and the screen size are enlarged. Using this viewing-zone scanning holography, the viewing zone can be effectively enlarged, although the width of the reduced viewing zone should be larger than the pupil diameter of the human eye to ensure wavefront reconstruction. A holographic display system with a viewing zone angle of 40°has been developed [15], along with a color display system [16], a multi-channel system [17], and a 360°display system [18]. In this manuscript, "viewing zone" is used as a general definition associated with display systems, that indicates the region from where the image can be observed. "Reduced viewing zone" and "enlarged viewing zone," instead, are both referred to the particular cases of scanning display systems and denote the viewing zones corresponding to, respectively, a single scanning point and all the scanning points. Moreover, for the scanning display systems, "enlarged viewing zone" is utilized as the equivalent of "viewing zone" for the general ones. In addition, "viewing zone angle" is an angle indicating the viewing zone width measured from the screen center; in horizontal scanning systems, the viewing zone angles are measured along the horizontal direction.
In this study, we propose a new viewing-zone scanning holography technique that can remove the horizontal scanner from the display system. The MEMS SLM is illuminated by a pulsemodulated laser to scan light modulated by the MEMS SLM. The beam scanning technique using the MEMS SLM was recently proposed for the LIDAR system [21]. This scanning technique was applied in the current research to the viewing-zone scanning holography. Thus, the MEMS SLM was employed for both displaying hologram patterns and scanning the viewing zone. The removal of the horizontal scanner from the display system also reduces system complexity.
MEMS SLMs have been used for electronic holographic display systems due to their highframe-rate image generation. The combination of the MEMS SLM and the 2D scanner has also been developed using two Galvano mirrors for the 2D scanning [22,23]. Additionally, 360°h olographic display systems using MEMS SLMs have also been developed [24][25][26][27].
Recently, a beam scanning technique using the MEMS SLM was adopted to light field display [28]. This technique is based on ray reconstruction and, as such, the number of scanning directions was limited to seven. The technique proposed in this study is based on wavefront reconstruction; as such the number of scanning directions can be increased to approximately 200 as the complex amplitude distribution can be manipulated on the Fourier plane.
In section two below, the proposed viewing-zone scanning holography technique using the MEMS SLM and a pulse-modulated laser is explained. The experimental verification of the proposed technique is presented in section three. A discussion of results follows in section four, and conclusions are provided in section five. Figure 2 shows a schematic diagram of the holographic display technique proposed in this study. The MEMS SLM is illuminated by short laser pulses. The screen of the MEMS SLM comprises a 2D array of MEMS mirrors that rotate to modulate light two-dimensionally. We considered that the MEMS mirrors had on/off states, and that all mirrors changed their states simultaneously. This study employed a rotating state for the MEMS mirrors; in this state, the MEMS mirrors rotates to change from being in an on state to an off state, and vice versa. Conventionally, the rotating state was not used for image generation. The pulse width of the illumination laser was made shorter than the duration time of the rotating state. The MEMS SLM displays holographic patterns sequentially and at a high frame rate. When a short laser pulse illuminates the MEMS mirrors in the rotating state, the light reflection direction depends on the illumination timing of the laser pulse. Thus, light two-dimensionally modulated by the MEMS mirrors is scanned by controlling the pulse generation timing. As a result, the viewing zone of holographic images is enlarged by the time-multiplexing technique. The viewing-zone scanning scheme is depicted in Fig. 3. The laser light illuminates the screen of the MEMS SLM from the direction normal to the screen. In Fig. 2, the illumination direction is depicted as not being normal to the screen, to support ease of drawing. The rotation angles of the MEMS mirrors to the normal for on and off states are +α and −α, respectively. When a pulse laser illuminates the MEMS mirrors at a rotation angle of θ, the reflection angle is given by 2θ. Thus, the maximum scan angle is ±2α and the viewing zone angle is enlarged to 4α. Black images are inserted between hologram patterns to reset all MEMS mirrors. During the rotating state, when the MEMS mirrors change from the off to the on state, the laser pulses illuminate the MEMS SLM. When the frame rate of the MEMS SLM is denoted by f MEMS , the frame rate of the hologram pattern generation is given by f MEMS /2 because a black image is inserted between each two consecutive hologram patterns. Therefore, when 3D images are displayed at a frame rate of 60 Hz, the number of scanning points, i.e., the number of the reduced viewing zones generated during a single scan, is given by f MEMS /120.  Figure 4 illustrates the holographic display system based on the proposed technique. The laser diode is used to generate short laser pulses, which illuminate the screen of the MEMS SLM from the normal direction via the half mirror. The pulse generator outputs the image update signals fed to the driver of the MEMS SLM, and the modulation signals fed to the driver of the laser diode. Since black images are displayed between the hologram patterns by the MEMS SLM, the latter pulses are outputted once every two former pulses, with appropriate delays for scanning. In the viewing-zone scanning system, as shown in Fig. 1, the light modulated by the MEMS SLM is converged, so that the viewing zone is localized at the light converging point [15]. In Fig. 4, a light emitted from the laser diode is converged by a lens to illuminate the MEMS SLM. The light modulated and reflected by the MEMS SLM generates the reduced viewing zone at the light convergence point, which is scanned to enlarge the viewing zone.

Proposed technique
Next, the diffraction and the reflection of laser pulses by the 2D MEMS mirror array are explained. We considered that square MEMS mirrors with a width of a were aligned twodimensionally with a pitch of p and MEMS mirrors rotated around their diagonal lines. As shown in Fig. 5(a), the 2D MEMS mirror array was rotated 45°, so that the rotation axes of the MEMS mirrors were aligned vertically and the light reflection direction changed horizontally. The distribution produced by the MEMS mirror array is represented by the convolution of the obliquely sampled hologram pattern and one MEMS mirror, as shown in Fig. 5(b). Thus, the distribution obtained on the scanning plane is given by the Fourier transform of the distribution displayed by the MEMS mirror array, which represents the multiplication of (A) the Fourier transform of the obliquely sampled hologram pattern, and (B) the Fourier transform of one MEMS mirror, as shown in Fig. 6. The coordinate of the scanning plane is denoted by (x s , y s ).  In the former distribution, the Fourier transform of the obliquely sampled hologram pattern occupies a 45°rotated square region and is repeated two-dimensionally on the scanning plane with an interval of λl/p, where l is the distance between the MEMS mirror array and the scanning plane. The latter distribution is given by the 45°rotated 2D sinc function shifted by l tan2θ in the horizontal direction. The side lobe intensities of the 2D sinc function normalized by the main lobe intensity are also indicated in Fig. 6. When the laser pulse generation timing is changed to alter the reflection direction, the latter distribution moves along the horizontal axis, while the former distribution remains fixed.
On the scanning plane, the reduced viewing zone (represented as a blue rectangle in Fig. 7(a)) should move along the horizontal axis in accordance with the change in the reflection direction which corresponds to the horizontal position of the distribution (B) in Fig. 6. The horizontal center of the reduced viewing zone should coincide with that of the distribution (B) (green in Fig. 7(a)). The region of the reduced viewing zone must be equal to or less than a half of the 45°r otated square region of the Fourier transform in order to enable the elimination of the conjugate image (as explained later on). In this study, the reduced viewing zone was determined so that its upper corners touched the borders of this 45°rotated square region as shown in Fig. 7 As shown in Fig. 7(a), when the reduced viewing zone is scanned horizontally, it lies on plural Fourier transform regions, unless its horizontal center coincides with that of the Fourier transform regions; in other words, each Fourier transform region contains divided reduced viewing zones as shown in Fig. 7(c). As shown in Fig. 7(a), the reduced viewing zone is also repeated two-dimensionally as the Fourier transform is repeated. The repeated viewing zones (represented as dotted blue rectangles) existing along the horizontal scan line are unwanted and their intensities are determined by the side lobes of the 2D sinc function (the distribution (B)). The maximum intensity ratio between the side lobes along the horizontal scan line and the main lobe is given by (sinc ν x ) 2 (sinc ν y ) 2 νx=3π / 2 , νy=3π / 2 = [sinc(3π / 2)] 4 = 16/81π 4 . Therefore, the influence of the repeated reduced viewing zones is negligible. Figure 8 schematizes the calculation of the hologram pattern displayed on the MEMS SLM. First, the object wave on the MEMS SLM screen is calculated based on the point-based or polygon-based method. Then, it is Fourier transformed to obtain the distribution on the scanning plane. The area outside the divided reduced viewing zones (blue) is masked (black) and the complex-conjugate symmetric distribution (yellow) is successively added to render the inverse Fourier transform real-valued [29]. The inverse Fourier transform is performed to obtain the real-valued distribution that, finally, is binarized to be displayed on the MEMS SLM screen. When the size of the MEMS mirrors decreases, the brightness of the 3D images decreases. In addition, the distance among the main and side lobes of the distribution (B) shown in Fig. 6 increases. When the width of the MEMS mirrors is half of the pitch of the MEMS mirrors, the intensities of the unwanted reduced viewing zones become minimum. When the width is less than half of the pitch, the intensities of the unwanted reduced viewing zones increase so the image quality of the 3D images degrades. In addition, the structures in the gaps between the MEMS mirrors diffuse light. Therefore, the gaps should be reduced.

Scanning of the reduced viewing zone
First, scanning of the reduced viewing zone by the MEMS SLM was verified. In this study, a digital micromirror device (DMD) was used as the MEMS SLM. The DMD used for the experiments was the Discovery TM 4100 (Texas Instruments, Inc.) The DMD screen consisted of MEMS mirrors with a square shape that were aligned two-dimensionally along x and y-axes, and the MEMS mirrors rotated around their diagonal lines. The pitch of the MEMS mirrors (pixel pitch) was p = 13.68 µm and their width was a = 13.12 µm. The number of MEMS mirrors (resolution) was 1,024 × 768. Their rotation angle was ±12°(α = 12°) and rotation time was 3.83 µs. The frame rate was 22,727 Hz and the frame time was 44.0 µs. Thus, the maximum scan angle was ±24°and the possible maximum viewing zone angle was 48°(4α). The DMD was rotated by 45°to enable viewing zone scanning in the horizontal direction, as shown in Fig. 9.
A laser diode with a wavelength of λ = 488 nm and a maximum output power of 100 mW was used. An FPGA was used for pulse signal generation. The pulse width was 20 ns. Thus, the number of scanning points was 192 and the frame rate for the 3D image generation was 59.2 Hz.
Since the FPGA clock was 50 MHz, it could generate a sequence of pulses with a time interval of 20 ns. Figure 10 shows the image update pulses sent to the DMD driver and the optical power of the laser pulses, which was measured by an optical power meter. The laser pulses were  The focal length of the converging lens was 100 mm and the distance between the DMD screen and the light convergence point was l = 300 mm. Thus, the side length of the Fourier transform was 10.7 mm. The size of the reduced viewing zone shown in Fig. 7 was 7.57×3.78 mm 2 . The reduced viewing zones were generated at an interval of 1.31 mm.
The experimental results are shown in Fig. 11. A tracing paper was placed on the scanning plane and the light distribution on the paper was captured by a video camera. In this experiment, to clearly show the scanning effected by the MEMS SLM, the reduced viewing zone was located at the center of the Fourier transform in the horizontal direction, so that square patterns were observed in the Fourier transform. As shown in Fig. 11, the square pattern was scanned on the scanning plane in the horizontal direction. However, undesirable light distributions appeared in the left-end scan region. These distributions were generated by the MEMS mirrors, which were in the off state. These mirrors vibrated at several degrees when the other MEMS mirrors rotated from the off state to the on state [30]. This small-angle vibration by the off-state mirrors caused the undesirable distributions in the left-end region. Thus, the effective scan angle was from −18°t o +24°and the viewing zone angle could decrease from 48°to 42°. Next, we verified the movement of the reduced viewing zones across the repeated Fourier transforms, as shown in Fig. 7. The scanning pitch was 0.25°. Figure 12 shows the five consecutive reduced viewing zones captured at scan angles of −0.50°, −0.25°, 0°, +0.25°, and +0.50°, respectively; the images are magnified at the center of the scanning plane to show the generation of the reduced viewing zone and the complex-conjugate symmetric distribution. The corresponding arrangements of the reduced viewing zone and the complex-conjugate component (see Fig. 8) are also illustrated in Fig. 12. The movement of the reduced viewing zone with a scanning pitch of 0.25°was verified.
The scanning angles were derived from the abovementioned experiments about the viewingzone scanning. Figure 13 shows their relationship with the delay in the pulse sent to the laser driver; the scanning angle almost linearly changed with the pulse delay.

Generation of holographic images
In this stage, the holographic images were generated. The point-based method was used to represent 3D objects, that is, 3D objects were represented by an aggregate of object points. Spherical waves generated by the object points were calculated on the DMD screen and were added to obtain the object wave. Then, the diffraction calculation was performed to obtain distribution at the reduced viewing zone using the Fourier transform. The hologram patterns  were calculated following the process illustrated in Fig. 8. This calculation was performed for all reduced viewing zones.
The reconstructed images are shown in Fig. 14. The number of object points were 848, 48, and 1,200 for Figs. 14(a), 14(b), and 14(c), respectively. The reconstructed images were captured by a video camera that was moved in the horizontal direction on the scanning plane. A slit was attached to the camera lens to remove the complex conjugate component. As shown in Fig. 14(c), the image distortion was not observed in the reconstructed image.
As shown in Fig. 14, the left and right sides of the 3D objects could be observed when the camera position was changed horizontally. The viewing zone angle was enlarged to 41°. The screen size was equal to the DMD screen size, i.e., 0.69 inches. The reconstructed image could be observed by both eyes and had enough brightness for observation in the laboratory room. Following on, the depth representation of the 3D images generated by the proposed technique was verified. Two words were generated at different depth positions, where the distances from the DMD screen to the words "TUAT" and "3D" were 20 and 50 mm, respectively. Figure 15 shows the captured images when the video camera focused on the two words. The word being focused on appeared sharp while the other looked blurred. A movie of the reconstructed image is provided (Visualization 1).

Elimination of conjugate image
Next, the optical system was added to the experimental system to remove the complex conjugate component. Figure 16 depicts the modified holographic display system. The single sideband (SSB) filter placed on the scanning plane removed the complex conjugate component [29]. Lens 2 projected the screen of the MEMS SLM onto Lens 3, which was the display screen of the modified system. Lens 3 projected the scanning plane onto the viewing zone plane of the modified system. As the SSB filter, a slit was used instead of an edge to block the undesired repeated Fourier transforms. As shown in Fig. 11, (a) peak existed at the center of each Fourier transform, which was caused by unmodulated light reflected by the MEMS SLM. The SSB filter was shifted slightly upward to block these peak distributions.  Figure 17 shows the constructed experimental system. The focal lengths of Lens 1, 2, and 3 were 80.0, 38.1, and 80.0 mm, respectively. The distance between the DMD screen and Lens 2 was 63.5 mm and the height of the SSB filter was 0.80 mm. The magnification of Lens 2 was 1.5, so that the screen size was enlarged to 1.0 inches. Because the diameter of Lens 2 was 25 mm, the scan angle by DMD was limited to 19.8°. Because the screen size was magnified, the viewing zone angle of the modified display system was reduced to 13.2°. The reduced viewing zone was generated at a distance of 500 mm from the display screen. The pixel pitch on the display screen was 20.5 µm, and thus the size of the reduced viewing zone was 8.40 × 4.20 mm 2 .
Because the scan angle of the DMD was reduced to 19.8°, the number of hologram patterns displayed by the DMD for a single scan was reduced to 79. Therefore, the frame rate of the 3D image generation was increased to 144 Hz. Figure 18 shows the reconstructed images generated by the modified experimental system. The conjugate images were not observed. The line profiles were provided to show the removal of the conjugate images. The reconstructed images could be observed by both eyes and the brightness of the reconstructed images was sufficient for observation in the laboratory room environment.

Discussion
The effectiveness of the proposed holographic display technique was verified by experiments using a DMD, a device developed for 2D image generation. MEMS SLMs specialized for holographic displays should be developed to increase the viewing zone and the screen size of holographic displays. The mirror rotation time should be increased and the duration of the on/off states should be reduced to increase the number of scanning points and the brightness of the 3D images. The rotation time was roughly one tenth of the frame time for the DMD used for the experiments. The 3D images could be observed in an indoor-lighting environment, but may not be observable in an outdoor environment. An increase in laser power will also increase the brightness of 3D images.
Several reduced viewing zones entered the eyes because they were substantially overlapped, as explained later on. However, since the laser pulse width was only 20 ns, the brightness of the 3D images is discussed. To evaluate the light power entering human eyes with an average pupil diameter of 5 mm, a mask having a 5.0 mm × 3.5 mm aperture was placed at the center of the enlarged viewing zone and the optical power of the light passing through it was measured; a value of 2.4 nW was obtained for reconstructed images of random patterns. For comparison, the DMD was then replaced with a smartphone whose screen was covered by a mask with an aperture whose size was equal to the DMD screen. The measured optical power was 19.7 nW when white images were displayed; when the smartphone brightness was set to the minimum and the image could still be seen in an indoor lighting environment, its value decreased down to 1.6 nW. This indicated that our experimental system can generate 3D images brighter than those obtained with the smartphone operated at the minimum brightness level. The use of a multi-mode laser diode and the use of multiple laser diodes could increase the laser power illuminating the DMD and, thus, also the brightness of the 3D images generated.
The viewing zone angle of the experimental system was enlarged to 41°, which can be achieved using a SLM with a pixel pitch of 0.7 µm, as per conventional holographic display techniques (λ = 0.5 µm). Enlargement of the rotation angle of the MEMS mirrors to 25°was reported in [31]. Using this technique, the viewing zone angle could be increased to 100°in the present study.
The viewing zone angle of the modified experimental system was limited by the diameter of Lens 2. As Lens 2 was used for screen size magnification, its focal length was short; hence, combined lenses should be used for Lens 2 to increase its diameter.
As we experimentally verified, the proposed technique does not require the reduction of pixel pitch to increase the viewing zone angle. Thus, the screen size can be increased by increasing the MEMS mirror pitch. However, the width of the reduced viewing zone depends on the MEMS mirror pitch. Because the width of the reduced viewing zone should be larger than the pupil diameter of the human eye, the MEMS mirror pitch should be less than 20.7 µm to obtain a viewing zone width of 5 mm (the average pupil diameter) when λ = 0.5 mm and l = 1 m. Since the screen size is limited by the screen size of the MEMS SLM, a tiling technique should be developed to additionally increase screen size. The display system described in Fig. 16 provides a larger screen size than that of a DMD because of the magnifying imaging system. This technique can therefore be used for tiling the screens of multiple display systems.
The viewing zone angle of the reduced viewing zone, which is given by λ/p, was 2.04°for the experimental system. Because the scan angle was 48°and the number of scanning points was 192, the scan pitch was 0.25°. Therefore, there was substantial overlap among the reduced viewing zones. The overlapping of the reduced viewing zones had the effect of reducing the speckles in the reconstructed images.
Since the scan angle of 48°could be covered with 24 reduced viewing zones, the total of 192 reduced viewing zones could be divided into eight subgroups. When red, green, and blue lasers are used for the pulse generation and each reduced viewing-zone subgroup is illuminated with a different color and intensity, the color 3D image generation can be achieved; this time-multiplexing technique for color image generation has been previously developed for viewing-zone scanning systems [16]. Color image generation can also be attained by using three DMDs [25].
In [28], light scanning by the MEMS SLM was applied to the light field display. The number of scanning points was seven only. Because light field displays are based on ray reconstruction, and as one of the repeated Fourier transforms (distribution (A) shown in Fig. 6) should be selected for the image projection, the Fourier transform of the single MEMS mirror (distribution (B) shown in Fig. 6) should be matched to the selected Fourier transform. This means that the images can be displayed only in the direction of diffraction orders. Therefore, the number of scan points was determined by the MEMS mirror pitch. For wavefront reconstruction, as the complex amplitude distribution can be manipulated and the reduced viewing zone moved continuously across the repeated Fourier transforms (as shown in Fig. 7), this study increased the number of scanning points to ∼200. With the proposed technique, the number of scanning points was determined according to the rotating time of the MEMS mirrors and the laser pulse width.

Conclusion
This study proposed the use of the MEMS SLM and laser pulse illumination to widen the viewing zone of holographic displays without using optical scanners. The MEMS SLM was used for both the hologram pattern generation and laser beam scanning. The proposed technique was experimentally verified using the MEMS SLM with a pixel pitch of 13.68 µm operating at a frame rate of 22,727 Hz. During the rotation of the MEMS mirrors, laser pulses with 20 ns duration illuminated the MEMS SLM to scan the viewing zone into 192 directions. Enlargement of the viewing zone angle to 41°was demonstrated. The frame rate for 3D image generation was 59.2 Hz. The conjugate image was removed by adding the SSB filter to the experimental system. In this case, the viewing zone angle decreased to 13°due to the lens diameter used in the experimental system and the frame rate for 3D image generation became 144 Hz.

Funding
Japan Society for the Promotion of Science (KAKENHI Grant Number 19H02189).

Disclosures
The authors declare no conflicts of interest.