Large field-of-view microlens array with low crosstalk and uniform angular resolution for tabletop integral imaging display

Tabletop autostereoscopic displays have the advantage of displaying realistic three-dimensional (3D) content. In particular, the Integral imaging (InIm) method is superior in tabletop displays, as it is possible to make the system small using microlens arrays (MLA). Off-the-shelf MLAs have a small field of view (FOV) and are unsuitable for tabletop displays where the half viewing angle is typically around 45 degrees. The MLA for tabletop display also needs to have uniform angular resolution and low crosstalk to display 3D content. In this paper, we designed a new MLA with a multilayer structure and a reversed lens shape. This lens array has a wide viewing angle of −58 degrees to +58 degrees, with uniform angular resolution and low crosstalk of less than 0.2. We implemented a tabletop display using our MLA and verified its properties experimentally. The misallocation error of the elemental image is experimentally measured and compensated for as well.


Introduction
The autostereoscopic 3D display has the potential to provide realistic 3D content, and this technology has been actively studied [1][2][3]. The tabletop application of autostereoscopic displays particularly has the advantage of presenting realistic 3D content on a table [4][5][6][7][8], which is very useful for multiple users to work on a 3D model or review a design together. Tabletop displays require a viewing angle of 45 degrees or higher because the user watches the content on the table from about 45 degrees. Thus, most tabletop displays require much larger optics than 3D content to project the images. On the other hand, the 3D display in the InIm method uses an MLA to display 3D contents, and each lens projects a part of the 3D contents. Therefore, the distance from the lens to the image plane is very short, and the size of the 3D image and the optical system are similar [7,8]. This feature is a considerable advantage in tabletop applications since the size of the InIm display is significantly small compared to other 3D displays. Also, it is possible to expand the display size by increasing the number of lenses.
There are general configurations in the InIm method. One is the resolution priority Inim, and the other is the depth priority InIm [9][10][11]. In the resolution priority InIm, the gap between the element image and the lens is larger than the focal length of the MLA. In the depth priority InIm, the gap is the same as the focal length of the MLA. Since the FOV in InIm is determined by the gap and the width of the elemental image, the FOV of the depth priority InIm is larger than resolution priority InIm. So, the depth-priority InIm is suitable for the tabletop application, which needs a large viewing angle.
Most of the MLAs used in InIm displays have a small FOV. Off-the-shelf MLAs are primarily utilized in TVtype displays [12][13][14][15]. In this case, the FOV of each MLA lens is just 30 degrees or less because the user watches 3D contents at a distance with a small viewing angle from the display. Over 30 degrees, the aberration becomes too severe and the crosstalk between viewpoints becomes critical because users watch 3D content at about 45 degrees on the tabletop display. Moreover, the required viewing angle is much larger than 45 degrees because of the close distance between the user and the tabletop display. Therefore, an MLA with a large FOV is essential for the tabletop application.
The MLA applied to the tabletop display should have a large FOV and a uniform angular resolution within the total FOV. Uniform angular resolution indicates a situation in which the inclination angle of the wave vector in the image domain is linear according to the distance away from the optical axis in the object plane. The MLA is generally made of a spherical lens array and has a projection formula as a multiplication of the focal length and tangent of the output angle. So, it has non-linear characteristics when the projection angle increases. Such uniform angular resolution is important because the elemental image in the InIm display is expressed as pixels with equal spacing.
Most MLAs for the InIm display are spherical lens arrays, so their crosstalk is severe at large projection angles. To our knowledge, in most InIm, 3D content is presented within a small FOV where the crosstalk effect is negligible. Also, the only chief ray passing through the center of the optical aperture is considered to calculate the elemental image. Therefore, the FOV appears to be determined by the gap between the element image and the lens [16,17]. However, in the calculation method that only considers the chief rays, there is a limitation because the image quality deterioration is hard to estimate since the marginal rays also determine the actual overall optical performance. As a result, using an optical design tool is necessary to characterize the effects of chief and marginal rays for calculating crosstalk in MLA and designing an improved MLA.
This paper presents a novel design for a large viewing angle tabletop display with uniform angular resolution and low crosstalk. It is achieved by using a multilayer structure and a reverse-shaped lens design. The parameters of each layer are optimized to realize uniform angular resolution and to minimize crosstalk. The performance of the proposed MLA is experimentally verified, and a scalable tabletop display is implemented. Figure 1 shows the viewing angle of the tabletop display with the MLA having a large projection angle. There are two important angles: the projection angle and the viewing angle of the tabletop display. The projection angle is the FOV of the microlens without significant crosstalk problem. The viewing angle refers to the maximum angle where the observer can watch 3D contents without any clipping problem. The relative position of the microlens on the tabletop is different and the combination of the projection angles of the whole microlenses determines the viewing angle of the tabletop display. The viewing angle is smaller than the projection angle.

Large field-of-view MLA design with uniform angular resolution
For the camera lens, the transformation relationship between the input and the output is described as the projection formula [18]. The input of the projection formula is the incident angle from a real scene, while the output is the position on the camera's sensor plane. Among the projection formulas, the rectilinear and equidistant lens are the two representative formulas, and they are respectively expressed as where h is the position on the camera's sensor plane, f is the focal length, and θ is the input angle. In the rectilinear lens formula, there is a problem that the output position on the sensor plane excessively increases at a large input angle. On the other hand, in the equidistant lens formula, the output position linearly increases even at large input angles. The projection formula of the individual lens in the MLA is simply understood as an inverse function by exchanging the input and output of the previously mentioned camera lens. In this case, the input is the position on the elemental image plane, while the output is the emission angle from the lens to the observer. The most off-the-shelf MLA projection formula is the inverse of the rectilinear lens formula. For achieving the large emission angle, the position of the elemental image plane should be too large a value. So, it is difficult to satisfy the target viewing angle of the tabletop display. The off-the-shelf MLA has a plano-convex surface where the flat surface faces the input plane, the same as the elemental image plane. In this case, total internal reflection happens when the position on the elemental image plane is far from the center of the elemental image plane. In other words, this fact may bring about a significant problem in the uniformity of the angular resolution. The angular resolution in both meridional and circumferential directions becomes non-uniform at the large projection angle. Therefore, the off-the-shelf MLA is not suitable for the tabletop display application since most observers watch the contents on the tabletop from a large viewing angle.
The equidistant lens is a more suitable projection formula for the tabletop application than the rectilinear lens because its output angle linearly increases according to the position at the elemental image plane. This linearity provides uniform angular resolution over the FOV of the MLA. From the optical design perspective, a multilayer structure helps MLA to have an equidistant lens formula because this type of lens has a barrel distortion compared to a rectilinear lens. In this paper, the proposed MLA has a multilayer structure and reversed lens and is called a multilayer reversed lens MLA (MLR-MLA). Figure 2 shows a schematic of the MLR-MLA for uniform angular resolution. The MLR-MLA has a multilayered structure, and its shape looks like a reversed lens compared to the off-the-shelf MLA. The reversed lens has a spherical surface toward the elemental image plane and a flat surface toward the observer. It makes it possible to increase the FOV without the total internal reflection. Therefore, the spacer is important in providing a stable and proper gap between the spherical surface and the elemental image plane. An optical plastic plate made of polycarbonate is used as the spacer. The MLR-MLA has four layers: spacer, lenses, optical stop, and cover glass. For simply deriving the projection formula, only the chief ray is involved as this passes through the center of the optical stop. The required conditions are given by Here, θ 1 is the incident angle from the elemental image plane, while θ 2 is the first refracted angle of the chief ray at the boundary of the spacer. Then, each n spacer and n air are the refractive index of the spacer and the air, respectively. The refractive index of the spacer and the air is different because h is a specific position on the elemental image where the optical ray starting from the h position with angle θ 1 passes through the center of the stop. t spacer is the thickness of the spacer, and t lens is the central thickness of the lens. After the lens, the refracted angles are obtained by where θ 3 is the refracted angle from the flat surface of the lens and θ 4 is the output emission angle of the chief ray. Then, n cover is the refractive index of the cover glass.
From Equations (5) and (6), the output emission angle, θ 4 , is expressed as The projection angle is plotted according to the normalized position. The normalized position is the ratio of the position, h, to the semi-diameter of the lens, r, as shown in Figure 3(a). The black line is the projection angle of the MLR-MLA, while the green line is an ideal straight line. The blue line is that of the off-the-shelf MLA, where the inclination at the origin equals the MLR-MLA's inclination. The MLR-MLA has a much larger FOV than the off-the-shelf MLA and has good linearity when the normalized position is less than two. Here, the linearity error is approximately 6.6%, and the projection angle is +58.0 degrees. Figure 3(b) shows the angular distribution of the output rays from uniformly sampled input positions on the square-shaped elemental image where the sampling number is 20 × 20. These black dots express the angle of the direction cosine of the output ray from the sampled position. Each angular diameter of the green circle is the output emission angle when the sampling is the input position on the elemental image at twice the sampling interval from the second sampling point to the tenth point from the center. With this setup, about 284 pixels are placed in the largest green circle with an angular diameter of 116 degrees.

Large field-of-view MLA with low crosstalk for tabletop display
The concept of the MLR-MLA is introduced using only the chief ray, and one of the surfaces of the lens is supposed to be a spherical surface. This approach is insufficient for the crosstalk analysis in the tabletop 3D display because the crosstalk is calculated from the visibility of each view represented as the intensity distribution. The crosstalk is related to the collimation ability of the lens over the FOV [19][20][21]. Therefore, it is essential to consider the entire ray bundles between the marginal and the chief rays to analyze the collimation ability over the FOV. In this paper, we design an aspherical surface to reduce the crosstalk. The sag of the aspheric surface is generally represented as the extended polynomial [22], defined as where c is the curvature of the surface. E i (u, v) is the i-th extended polynomial term in u and v, which is the coordinate at the stop, and A i is the coefficient of the extended  polynomial. N is the number of polynomial terms in the series. The design parameters of the MLR-MLA are listed in Table 1. Figure 4 shows the layouts of the MLR-MLA and offthe-shelf MLA. The MLR-MLA has an aspherical surface lens array for low crosstalk even though the off-the-shelf MLA has a spherical surface. Each color represents the emission angle of the chief rays from the elemental image plane, while the emission angles are selected at 0, + 14.5, + 29, + 33.5, and +58 degrees. The two cases have the same aperture size and FOV of 116 degrees for comparison. The MLR-MLA's collimation ability is  definitely better than off-the-shelf MLA. In the MLR-MLA in Figure 4(a), emission rays hardly diverge even when the projection angle is +58 degrees. However, as shown in Figure 4(b), emission rays are slightly gathered when the projection angle is zero in the off-the-shelf MLA, but the emission rays excessively diverge at a large projection angle. Therefore, crosstalk is expected to be significantly lower in MLR-MLA. The projection angles are calculated using the Zemax programmable language (ZPL) to analyze two cases. In the ZPL, the optical ray is determined by two positions: the pixel position on the elemental image and the pupil position on the stop. The ZPL traces the ray, which passes through two points. The pupil position on the stop is selected as a hexapolar pattern in the simulation. The hexapolar pattern of the stop for ray tracing is defined by which N s is the number of concentric circles within the diameter of the optical stop, i is the order of these circles where the range of i is a natural number less than or equal to N s . j is the number of positions on each circle. The j range is from one to six time i. When the coordinate is (0, 0), the optical ray passes through the center of the stop, which is the chief ray. The projection angle from a specific position (x, y) on the elemental image is then given by where − → k output is the wave vector of the output ray and λ is the wavelength. Figure 5(a) and (b) show the normalized visibility of MLR-MLA and off-the-shelf MLA as a function of the projection angle. ZPL's ray tracing is used to calculate the projection angle distribution of the input rays at a given pixel position. The FOV is set the same in both cases, considering only the chief rays. In Figure 5(a), the maximum normalized position of the input ray is used within a half FOV, which is from zero projection angle to a maximum projection angle. The normalized position ranges from −1.9 to +1.9, while 20 points are sampled at equal intervals. On the other hand, in Figure 5(b), the maximum normalized position of the input ray with a maximum viewing angle of +58 degrees is +2.85. This value is very large compared to MLR-MLA because the off-the-shelf MLA follows the rectilinear lens formula. In the same way, 20 points are sampled at equal intervals within the range from minimum to maximum normalized positions. For the off-the-shelf MLA, the distribution of projection angles at a large-normalized position severely leans towards zero degrees. For example, in the distribution at the end where the projection angle of the chief ray is +58 degrees, the peak of the distribution is only +44 degrees.  In general, the angle of the viewpoint is determined by the peak position of the projection angle distribution, while the crosstalk is also calculated using the peak value from the viewpoint angle [19][20][21]. Therefore, the crosstalk's projection angle differs from the chief ray's projection angle. In Figure 5(c), the MLR-MLA has low crosstalk values of 0.18 or less. On the other hand, in Figure 5(d), the distribution of normalized visibility of the off-the-shelf MLA is skewed to zero degrees. The peak position of the normalized visibility distribution is shifted to zero degrees, while the crosstalk value is found to be seriously high at 0.68. Our designed MLR-MLA is verified to have obviously low crosstalk values with sufficiently large FOV that is suitable for the tabletop display.

Evaluation of the misallocation from image distortion
The performance of our MLR-MLA is demonstrated through the experiment by embodying a 3D tabletop display requiring a large viewing angle. Tabletop displays should provide a large viewing angle because the observer watches the contents around the table. The optical system is designed to be scalable by tiling the units consisting of a projector and a corresponding MLR-MLA. Figure 6 shows the structure of a tabletop display system composed of an array of units. The projector made by Cremotech has a resolution of 1,366 × 768 and a throw distance of about 4 m. In order to reduce the size of the system and increase the pixel density in the elemental image, a projection lens is inserted in front of the projector to reduce the throw distance to 188.9 mm. At this time, the pixel density of the elemental image is 317.5 pixels per inch. In the MLR-MLA, each microlens has a lens pitch of 0.8 mm, and the number of lens arrays in a unit is 125 × 70. As previously mentioned, when the normalized position by the radius of the lens is two, this means that one microlens covers 20 pixels. But physically, the pitch of one microlens is equal to the interval of 10 pixels of the elemental image. As a result, 1,250 × 700 pixels are used as elemental images in the projector, and this system is configured with a 2 × 2 projector array to verify scalability. Therefore, the total resolution of the elemental image is 2,500 × 1,400. The projector's spare pixels are used to compensate for misallocation caused by optical distortion and misalignment.
In the InIm method, a misallocation error between the elemental image and MLA is a common and annoying problem. This problem is particularly critical when several MLAs are tiled. In this paper, the misallocation error is experimentally measured to address this problem. Figure 7 shows some examples of the misallocation problem caused by the distortion of an elemental image. The elemental image misallocation in the system is caused by the misalignment between optical components and incorrect magnification of each projector. Figure 7(a) shows the view when the elemental image is correctly       allocated. Here, the center of the individual lens and the center of the elemental image coincide so that the elemental image forms a correctly allocated view over the total FOV. Figure 7(b) shows the case where the center of the elemental image is shifted from the center of each lens. The elemental image spans only a part of the total FOV and is also shifted. The FOV, which is not covered by the elemental image, has inappropriate information from the adjacent elemental image. In order to correct this problem, the elemental image needs to be shifted in the opposite direction of the error. Figure 7(c) shows the case where magnification of the elemental image is smaller than the designed value. The view by elemental image does not cover the total FOV, and inappropriate views from the left and right adjacent elemental images cover the rest. In order to correct this problem, it is necessary to increase the number of pixels constituting the elemental image. In most cases, misallocation problems from the shift and shrinkage occur together. Thus, experimentally measuring the misallocation map is necessary to reallocate the elemental image. Figure 8 shows the experimental setup for measuring the misallocation map and the tabletop display structure using the MLR-MLA. A misallocation map is obtained by measuring the viewing angle generated by each pixel in the elemental image from the captured image. As shown in Figure 8(a), the setup for measuring the misallocation map is composed of a CCD camera and a rotation stage for measuring the intensity of the tabletop display screen according to the change in the viewing angle. The camera measures the brightness of the emission ray coming through each microlens array. A misallocation map is obtained by determining the position of the brightest pixel at a specific viewing angle. Figure 8(b) shows a tabletop display system with the MLR-MLA. As shown in Figure 8(c), each unit has a 6-axis mount for projector alignment. It is designed to enable shifting and tilting about the x, y, and z axes using ten bolts. This mount significantly reduces alignment errors but does not compensate for internal alignment errors or distortions in the optics. Figure 8(d) shows a 2 × 2 MLR-MLA applied to a tabletop display system. Figure 9 shows misallocation maps measured at two different viewing angles. Figure 9(a) shows the misallocation map at −53.3 degrees, half the maximum viewing angle. Figure 9(b) is the misallocation map when the viewing angle is 0 degrees in front of the display. The MLR-MLA applied to the tabletop display consists of 125 × 70 lenses, and these MLR-MLAs are in 2 × 2 tiled position. Therefore, the size of the misallocation map is 250 × 140, where one pixel corresponds to the measurement value of one lens array. The number of pixels on the horizontal axis of the elemental image corresponding to one lens is 20, so the total number of viewpoints also becomes 20. Therefore, the misallocation map is marked in different colors according to the 20 viewpoints. The color is determined according to the degree of shift in the misallocation error. For example, two other points where the misallocation value differs by five are understood as an approximately 5-pixel difference in shift error. Since four projectors generate elemental images for each MLR-MLA, the misallocation map appears as if it were divided into quadrants. Here, the shrinkage of the elemental image is induced from the gradient of the misallocation value. If the size of the elemental image is the same and there is only a shift error, the gradient becomes 0. However, when the size is small, the degree of shift differs between adjacent lenses, resulting in gradients of the misallocation. The misallocation maps measured at two different viewing angles also have slight differences. It is necessary to fine-tune the misallocation correction value according to the viewing angle.

Discussion
Experimental measurements obtained via the MLR-MLA are similar to simulation results. Figure 10 shows the normalized visibility and crosstalk of an actual fabricated MLR-MLA. As expected in Section 3, the visibility distribution of MLR-MLA is very narrow and has a leftright symmetric distribution. The crosstalk also shows excellent characteristics as low as 0.2 or less. Figure 11 shows the experimental results of the tabletop display measured from a viewing angle ranging from −53.3 degrees to +53.3 degrees. As previously mentioned, the viewing angle needs to be greater than 45 degrees for use in a tabletop display. Therefore, a viewing angle of ±53.3 degrees is considered sufficient. So, if an observer is positioned at one edge of the display, the projection angle at the center lens of the display is equal to the viewing angle. But the part closer to the observer has a projection angle smaller than the viewing angle, and the part farther from the observer has a larger projection angle than the viewing angle. In this experiment, when the viewing angle is +53.3 degrees, the projection angle of the edge of the display opposite the observer is +58 degrees. Figure 12 shows the captured images in two cases: one with both real and virtual contents are displayed, and another that displays 3D contents from capturing the real human head. In the case of the former, the contents consist of two red squares, a white circle, and a white cross as shown in Figure 12(a)-(c). The red squares are located at the MLR-MLA surface at D = 0 mm, and D is along the surface normal direction of the tabletop display to the observer. The white circle and cross are located in the virtual and real fields, respectively. From the MLR-MLA plane, the white circle is positioned at D = −15mm, while the white cross is positioned at D = 15 mm. Three real items are placed on the center of the display. The relative positions of the contents are distinguished by comparing the captured images with different viewing angles. In Figure 12(a), the red square obscures the left side of the white circle even though the right side of the white cross covers the red square. In Figure 12(c), the red square obscures the right side of the white circle even though the left side of the white cross covers the red square. Figure 12(d)-(f) present the 3D contents of a human head. The elemental images for this 3D contents are obtained from plural pick-up images of the head from different angles. In Figure 12(d) and (f), the seams between the adjacent MLR-MLAs are distinct. This is mainly because the incident angle on the screen changes abruptly at the boundary where two images come from different projectors. We expect that this seam problem can be solved by increasing the screen's diffusion angle. However, in case the diffusion angle is too high, the light efficiency is lowered. By analyzing this trade-off, we plan to optimize the screen's diffusion angle in the future.
The lateral resolution and depth resolution are important properties for InIm. Since our MLR-MLA is similar to an equidistant lens, the lateral resolution, R L , is determined by where d is the optical stop size, and p is the pixel size of the elemental image. f is the focal length of the MLR-MLA.
In our system, the optical stop size, d and the pixel pitch, p are 0.4 mm and 80 μm, respectively. The focal length is 0.86 mm as calculated from the graph in Figure 3(a). The depth resolution is the reciprocal of the average interval of the image planes [11] and its value is about 40 planes per millimeter. The depth of focus of this tabletop display is limited by the large projection angle of the MLR-MLA and the number of the microlenses. It is widely known that there is a trade-off relationship between the viewing angle and the depth of focus. Since the FOV of our system is very large, the depth of focus is as short as 62.2 mm.

Conclusion
Tabletop displays require a large viewing angle because of the observer's position. For the InIm tabletop display, an MLA must have uniform angular resolution and low crosstalk to improve the quality of 3D content. In this paper, we propose a new MLA with uniform angular resolution and low crosstalk within a large FOV. This MLA has a multilayer structure and a reverse lens shape called MLR-MLA. Compared with off-the-shelf MLA, the MLR-MLA has significantly superior properties. It has good linearity in projection angles from −58 degrees to +58 degrees and has low crosstalk of less than 0.2. The properties of the MLR-MLA are also experimentally confirmed by realizing the tabletop display. In addition, the misallocation map is quantitatively acquired by measuring the misalignment. The elemental image is reallocated using the misallocation map. We expect that the MLR-MLA will be useful for other systems that require a large viewing angle, such as near-eye displays and wavefront sensors. In the future, we plan to optimize the diffusing angle of the screen to solve the seam problem that occurs when tilting projection units for scalability.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Daerak Heo received his B.S. and M.S. degrees in Electronic Engineering from Kyungpook National University in 2017 and 2019, respectively. He is currently a graduate student at the School of Electronic and Electrical Engineering of the same university. His research interests include the optimization of optomechanical design, temporal multiplexing, and spatial multiplexing techniques for 3D display systems.
Beomjun Kim received his B.S. and M.S. degrees in Electronic Engineering from Kyungpook National University in 2019 and 2021, respectively. His research interests include dual-depth AR head-up displays using holographic optical elements (HOE) and eye-box expansion using the waveguide, as well as absolute depth estimation using a single snapshot with an asymmetric aperture.