Holographic pancake optics for thin and lightweight optical see-through augmented reality

Holographic pancake optics have been designed and fabricated in eyewear display optics literature dating back to 1985, however, a see-through pancake optic solution has not been demonstrated to date. The key contribution here is the first full-color volume holographic pancake optic in an optical see-through configuration for applications in mobile augmented reality. Specifically, the full-color volume holographic pancake is combined with a flat lightguide in order to achieve the optical see-through property. The fabricated hardware optics has a measured field of view of 29 degrees (horizontal) by 12 degrees (vertical) and a measured large eyebox that allows a ±10 mm horizontal motion and ∼±3 mm vertical motion for a 4 mm diameter pupil. The measured modulation transfer function (average orientation) is 10% contrast at 10 lp/deg. Three holograms were characterized with respect to their diffraction efficiency, angular bandwidth, focal length, haze, and thickness parameters. The phase function in the reflection mode hologram implements a spherical mirror that has a relatively simple recording geometry. © 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement


Introduction
Optical see-through augmented reality (OST-AR) systems offer unprecedented user experiences that are expected to have a transformative impact on communication, remote work, entertainment, healthcare, education, simulation and training, among other applications. These new user experiences seamlessly superimpose digital images on physical environments around a user. In these systems, a near-eye display provides the primary visual interface between users and digitally overlaid content. It is therefore of crucial importance to engineer OST-AR displays that deliver visually compelling results while offering comfortable experiences that are accessible to a diverse population of users. For this purpose, OST-AR displays must be able to display high-quality digital images over a field of view that optimally serves the target application, with a large eyebox and a high degree of color and luminance uniformity, while maintaining a thin device form factor and low weight. No existing OST-AR system is capable of achieving all of these goals simultaneously [1].
Conventional OST-AR display designs are reviewed by Kress [2] and suffer from different tradeoffs. Organized spatially, we have three choices when designing optics for eyewear form factor: place the optics and the light source 1) in front of a prescription lens, 2) behind a prescription lens, or 3) either use the existing surfaces of a prescription lens as a lightguide or some flat lightguide and prescription lens arrangement to meet prescription specifications.
Optical design examples for cases of placing the optics in front of or behind the prescription lens are reviewed by the first author [3]. The optical design presented in this paper relates to the third category of options, namely, flat lightguides. We next briefly discuss the challenges and limitations involving the first two options and turn our attention to flat lightguides in the next section.
Classical examples of placing the optics in front of a prescription lens include the "Birdbath" optical design form or a scanner source combined with a magnifier optic. The Birdbath design form, as for example used by Google Glass, offers a simple and robust design but it trades off field of view with device thickness. Virtual retina displays (i.e., laser-scanned OST-AR systems) have been designed but suffer from small eyebox sizes [4], as explained by the Lagrange invariant, which severely limits the usability of a display and does not support a wide range of users with varying interpupillary distances.
Placing the optics behind the prescription lens is commonly implemented by placing the field lenses on the same side of the head as the combiner. This approach suffers from an undesired wrap angle, even for fresnel combiners that flatten the sag of an otherwise highly curved combiner, deviating from the eyewear look [5]. A positive wrap angle can be achieved, restoring a continous combiner curve for both eyes, by placing the field lenses and the combiner on the opposite sides of the head [6], however, fitting such a design across a population of users remains challenging. In the category of placing the optics and the light source behind the eyeglass lens, free space holograms for augmented reality displays have been proposed in the literature, however, the eyebox is too small (around 1 mm) to be considered practical in any real-world application [7,8].

Related work on flat lightguides and holographic pancake optics
In our work, we focus on the optical see-through and flat lightguide portion of the optical design space. Thus, in this section, we review optical design work in the area of flat lightguides, pupil expansion for meeting eyebox specifications and its consequences, as well as holographic pancake optics to provide an overview and put our contribution in context.

Flat lightguides and pupil expansion
Flat [9] and curved lightguide [10] based architectures are strong contenders towards solving the eyewear form factor optical see-through augmented reality display problem. We use the term lightguide to refer to a guide where there are tens of thousands of modes (within the visible spectral range) as a consequence of guide thickness such that the resolution is not limited by modes of the guiding structure. In thin guides on the order of <1 mm, there would be a few thousand modes (within the visible spectral range) and this has to be evaluated with respect to resolution targets of the display.
A significant distinguishing property between flat and curved lightguides is that most (but not all) flat lightguide architectures require a collimator optic to inject light into total internal reflection for light propagation to the outcoupler. In flat lightguides, typically the collimator optic is placed between the microdisplay or laser and the incoupler to inject light into total internal reflection (TIR), and this light in TIR propagates down the length of the lightguide all the way towards the outcoupler. Therefore, the light reaching the eye will have propagated through the lightguide in addition to the eyerelief. This longer light propagation distance (on the order of 2x eyerelief) has vertical eyebox consequences.
Even though the pupil and field specification may be consistent with the source area and emission cone, from the point of view of Lagrange invariant, the desire for thin systems gives rise to the concept of pupil expansion and places demands on the in/outcoupler implementation. Pupil expansion can be implemented in 1D or 2D, each choice yielding a number of tradeoffs, most significantly vertical eyebox, optical efficiency, luminance, and color uniformity.
One-dimensional horizontal pupil expansion, for example using partial mirrors [9], is limited by the eyebox in the vertical dimension. The vertical eyebox limitation is because of the longer propopagation distance from the collimator to the users eye as mentioned earlier. The vertical eyebox reduces as a function of propogation distance (eye relief (ER)), diameter of the collimator optic (DO), and field of view (i.e., DO − 2 · tan(θ) · ER) [11]. Furthermore, OST-AR systems based on this principle either offer a low degree of luminance uniformity or compensate that using optical coatings or gratings that cause "cosmetic" problems for the user by partially occluding the users' eyes. Alternative pupil expander implementation choices, such as holographic or diffractive outcouplers, do not resolve this vertical eyebox and optical efficiency tradeoff [12]. Additional product design considerations include unwanted reflections and the perception of the glasses as viewed from the worldside.
Two-dimensional reflective exit pupil expansion are being explored [13] and usually results in color or luminance uniformity issues, corrupting the perceived image quality. One of the most common optical designs for OST-AR displays are diffractive waveguides with 2D exit pupil expansion. However, these systems are inefficient (≪ 1%), limiting the usability of OST-AR systems in outdoor scenarios and requiring high luminance light engines, which significantly reduces the all-day use potential of wearable computing systems and imposes other challenges, such as thermal management.

Polarization folded pancake optics
Optical "pancake" lens designs [14] have been demonstrated to overcome many of the challenges of conventional optics using a polarization folding mechanism that enables a wide field of view with a large eyebox in a compact device form factor. Note that in the classical pancake design, the microdisplay is in-line with the eye, therefore, the classical pancake configuration is not see-through. Most recently, such designs have enabled thin and lightweight virtual reality display systems [15]. However, these systems are not able to operate in an optical see-through mode as currently known in the state-of-the-art.
Prior to our discussion of our see-through holographic pancake, we first review the polarization management aspects of the classical pancake design [14], as illustrated in Fig. 1. The light from the display first passes through a circular polarizer (assuming RCP) and a 50/50 partial reflective mirror, then transmit through a QWP that changes polarization of the light to p-polarization. Following that QWP is a PBS which is configured to reflect the p-polarized light. Then the light passes through the QWP again and becomes RCP. After it reflects from the 50/50 mirror, it changes handedness and becomes LCP. The LCP light will pass though the QWP and PBS, and the transmission axis of the linear polarizer is aligned with that of the PBS. Therefore the light exiting the pancake is s-polarized light. Note that in transmission a classical pancake design can achieve ∼ 12.5% efficiency with an unpolarized display and ∼ 25% efficiency with a polarized display. From an aberration correction point of the view, the classical pancake benefits from rotational symmetry in achieving high performance, compared to curved lightguides where freeform surfaces (or phase profiles) have to be used to correct astigmatism. The classical pancake design has been extended to use holographic optical elements as we discuss in the next section.

Holographic pancake optics
Holographic architectures may make use of holographic optical elements (HOEs) or spatial light modulators (SLM). Full-color holographic mirrors in reflection mode written into dichromate gelatin date back to 1985 [16]. Margarinos et al. wrote their holographic mirrors in gelatin film to replace the spherical mirror in the classical pancake optics with a holographic optical element mirror. They report spectral bandwidths of 10 nm to 200 nm. Furthermore, monochromatically, diffraction efficiencies >90% are reported for a 20 µm film. Recent work in the area of implementing holographic optical elements written into photopolymers as pancake optic magnifiers has been in the context of virtual reality (not see-through) [15]. Cholesteric liquid crystals have been used to flatten the optical power from a double-pass catadioptric pancake configuration [17] for VR optics.

Optical systems combining flat lightguides and holographic optical element outcouplers
There has been work in the area of using reflection mode holograms in combination with flat lightguides to implement optical see-through magnifiers by Lee [18] and Kasai [19]. The work by Kasai is monochromatic (green only) with a 3 mm exit pupil (very small eyebox). The work by Lee et al. is a flat lightguide with a variation on a birdbath outcoupler, where the optical mirror phase function of a classical birdbath is written into a volume hologram, and it is thicker at 10 mm of lightguide thickness and longer by 12 mm, which would make it challenging it to integrate into an eyeglass formfactor. Furthermore, the efficiency of the Lee approach is lower, when compared to the approach presented in this paper, due to the double-pass through the beamsplitter. Lee reports the optical efficiency at 12.5% excluding the hologram efficiency. The hologram efficiency was not reported in the Lee paper, however, assuming 30% RGB hologram efficiency, the Lee approach would have a ∼3.8% total optical efficiency. Finally, the eyebox of the Lee approach is 2x smaller compared to the design presented in this paper.

Our contribution
The main contribution of this paper is the extension of an existing optical design form (polarization folded pancake optics) into a new mode of operation (i.e., optical see-through), therefore, we consider a comprehensive comparison to existing systems out-of-scope in this paper, however, we do organize the key optical design parameters of interest to be compared against the papers we cite to put our work in context in Table 1. Specifically, these parameters include see-through capability, field of view, eyebox, thickness, collimator optic requirement, efficiency, and image quality. Note that these are relatively complicated optical systems and exact numbers involve construction details, materials, coatings, and in some cases diffraction efficiencies. There is no standard definition for eyebox in industry. The eyebox values listed use a mixture of exit pupil diameters, range of motion for a fixed pupil, as well as illumination/imaging criteria, most authors do not specify which definition in the papers. Furthermore, the vertical eyebox in the case of flat lightguides with partial mirror outcoupler (1D) is limited by the colllimator vertical aperture. The exact vertical eyebox number in that particular case drives the temple height dimension. Therefore, the exact number for vertical eyebox is function of acceptable industrical design (i.e., subjective). Our paper is see-through, has the largest eyebox, highest efficiency, it does not require a collimator optic, full-color, and has a reasonable thickness compared to systems delivering this level of image quality. Note that designs that are limited with coating reflectivities in terms of cosmetics will start becoming visible and noticeable, due to color change with angle in the coatings, for combiner reflectivities ∼ >10%. Not all authors publish design or measured MTF curves, therefore, we are listing the image quality column in a qualitative way.

Optical see-through pancake: results and discussion
Here, we demonstrate the first OST-AR system based on full-color holographic polarization folded pancake optics. Our fabricated prototype AR display is thin and it offers full-color images over a field of view of 29 • by 12 • and a large eyebox that allows about ±10 mm horizontal and ±3 mm vertical ocular decentering motion for a 4 mm diameter pupil. These remarkable capabilities are achieved using an optical design that combines an emissive microdisplay, a flat light guide, and a holographic pancake optic (see Fig. 2(A)). The full specifications of our optical design are summarized in Table 2.  We propose the combination of a multiplexed RGB see-through holographic pancake and a flat lightguide to optimize for eyebox, image quality, efficiency, worldside cosmetics, and compactness. The optical layout of the flat lightguide with an RGB multiplexed hologram pancake is shown in Fig. 2(B). The optic has potential to look like eyewear as it can be shaped to fit into an eyeglass frame as shown in Figs. 2(A), 3(A). The light couples from the microdisplay into the flat lightguide refractively through the flat incoupling surface facing the microdisplay. There is a linear polarizer laminated to the incoupler surface. The incoupler tilt and the microdisplay are designed such that the incoupled light total internally reflects on the eyeside following the incoupling refraction. Then the light total internally reflects on the worldside surface of the flat lightguide followed by a third bounce that is the second eyeside total internal reflection. Then a flat polarizing beamsplitter outcouples the light towards the see-through holographic pancake. The light interacts with the optically flat surfaces on the lightguide for a total of 6 times. These optical flat surfaces are easy to manufacture and measure interferometrically.
In our prototype, there are two interactions with the hologram as illustrated in Fig. 2(C). The output from linearly polarized light from the outcoupler is converted to circularly polarized light using a quarter-wave plate. The input polarization into the holographic pancake is circularly polarized light. The light from the lightguide interacts with the hologram first in transmission.
In HOEs, the angle of incidence is measured from the fringe planes (not the substrate surface normal). Most of this light is transmitted due to the large angle of incidence rays make with the curved fringe planes of the holographic spherical mirror. Then the transmitted light from the hologram is reflected by a flat polarization film. The flat polarization film maintains the handedness of the polarization. This reflected light is directed back to the holographic mirror. However, during this second interaction, the rays make a small angle of incidence with the fringe planes. Therefore, the holographic mirror diffracts this light, acting like a mirror for the rays during the second interaction. The diffracted light from the hologram changes the handedness of the circularly polarized light. Then this diffracted light is focused and transmitted through the polarization film. These interactions are summarized visually in Fig. 2. The key to high efficiency is the single interaction within the angular bandwidth of the hologram. In see-through, world light is off-bragg, therefore, we observe good see-through quality without artifacts. The see-through transmission was measured with an unpolarized source using an NDK haze meter to be 27.9% in the combiner region and 88.4 % outside of the combiner region of the optic. The measured transmission as a function of wavelength through the see-through holographic pancake combiner region and the lightguide data is provided in Supplement 1.
In contrast to the classical flat lightguide and collimator architecture, where the vertical eyebox reduces during light propagation in the lightguide and along eyerelief, in the optical architecture proposed here, placing the holographic pancake optic on the eyeside does not incur this vertical eyebox reduction penalty. Therefore, we place the holographic pancake on the eyeside surface of the lightguide. As a consequence of this optical design choice, our solution can achieve a large eyebox without sacrificing image quality and optical efficiency.
We choose to implement the holographic mirror with a holographic optical element written into a photopolymer due to its physical compactness and no electrical power requirements in contrast to SLMs. Recent advances in photopolymers allow environmentally stable holograms getting closer to be included in products. Holographic optical elements have the benefits of acceptable cosmetics when viewed from the worldside.
Our choice of a holographic pancake yields a much thinner solution (4 mm) compared to classical collimators used in flat lightguides. The majority of the holographic pancake thickness comes from the coverglass sandwich on both sides (2 mm per side for prototyping purposes). Reconstructing the supporting substrate with an off-the-shelf readily available gorilla glass at 300 microns of thickness will immediately reduce the total thickness of the holographic pancake assembly by ∼6x. For the same microdisplay active area, a refractive single-pass (i.e., each ray sees each surface only once) collimator would be on the order of ∼8× longer. The construction parameters of the lightguide for the purpose of reproducibility are 40 mm long, 25 mm wide, 6 mm thick, with an incoupler angle of 30 • , and an outcoupler angle of 60 • . The lightguide material is common N-BK7 glass. The weight of the lightguide alone is 14.7 grams. The weight of the lightguide can be reduced with a low-birefringence optical grade plastic. Optical glass was used in this first prototype to minimize the birefringence effects. The weight of the holographic pancake assembly is 5.9 grams.
The holographic mirror is constructed by interfering an on-axis collimated plane wave and an on-axis spherical beam with a radius of curvature of 60 mm. These beams illuminate the photopholymer from opposing sides to generate a reflection mode hologram. The recording beams were generated at 639 nm, 532 nm, and 457 nm for RGB multiplexing. Wavelength multiplexing of volume holograms is well-known in the art and several simultaneous and sequential writing techniques have been proposed [22]. The holographic mirror was written using a simultaneous exposure setup [22]. The HOE active is 40 mm in diameter and substrate dimensions are 40 mm × 60 mm × 1 mm. The holographic mirror is written into a 16 µm thick photopolymer. At this thickness, the hologram is in the thick hologram regime [23] and would follow the Kogelnik model [24]. In a traditional holographic outcoupler used in a flat lightguide, the field of view uses the entire available angular bandwidth. In contrast, in our geometry, the eyerelief (pupil location) is close to the radius of curvature of the spherical mirror, and this yields an efficient use of the angular bandwidth.
A virtual image illustrating indoor (B) and outdoor (C) use case are shown in Fig. 3. The prototype combining the housing, drive electronics, and the optics are shown in Fig. 3(A,D). A world view as seen through the optics with the display turned on is shown in Fig. 3(C). Notice that we do not see any artifacts such as rainbows or back reflections from the optic as would occur in alternative diffractive outcouplers.
A total of three holograms were fabricated and characterized. Specifically, the diffraction efficiency is measured at 457 nm, 532 nm, and 639 nm at the left edge, center, and right edge of each hologram. The holographic mirror measurements of diffraction efficiency, angular bandwidth, focus, haze, and thickness are documented in Table 3. The measured polychromatic nominal modulation transfer function of the on-axis as well as an off-axis field point for both vertical and horizontal bar targets are shown in Fig. 4(A). The see-through pancake is limited by axial color on-axis and variation of astigmatism with color off-axis. The ability of humans to refocus allows for reasonable field curvature residuals on the order of 0.25 diopters. The modulation transfer (MTF) function was measured in visual space with a Gamma Scientific NED-LMD E101. The eyerelief was set to 20 mm during the MTF measurement. The camera pupil was set to 4 mm in diameter. The eyebox volume is a key design parameter in the optical design of augmented and virtual reality optics. The extent of the eyebox volume determines the experience of a user in seeing the entire virtual magnified image. Furthermore, a 3D description of eyebox facilitates the design of augmented and virtual reality products for a population of users. The 3D eyebox can be calculated by combining several criteria such as nonuniformity and resolution [11]. We measured the eyebox based on the center pixel luminance criteria using the Gamma Scientific NED-LMD E101 instrument. The 3D eyebox was measured at 3 discrete eyerelief planes ±4 mm away from the nominal eyerelief. The horizontal and vertical eyebox does not change significantly across the 8 mm of eyerelief depth of the measurement. The measured horizontal eyebox is approximately 20 mm, the measured vertical eyebox is approximately 5.7 mm.

Conclusion
In conclusion, the key contribution here is a breakthrough optical see-through geometry extending the classical pancake optic for applications in mobile see-through augmented reality. In particular, a flat lightguide with a refractive incoupler and a flat fold mirror outcoupler with a linear polarizer and a polarizing beam splitter is used. Following the outcoupler, the light interacts with the RGB holographic pancake optic. The choice of placing the holographic pancake on the eyeside provides a significant vertical eyebox advantage when compared to a flat lightguide with a traditional 1D horizontal partial mirror pupil expander, and an optical efficiency advantage when compared to a 2D expander. In addition, our prototype demonstrates excellent image quality, eyebox, and optical efficiency in hardware. Our new optical architecture is a first step towards realizing the thin device form factors required to enable future eyeglasses-like optical see-through augmented reality displays.