Design of an off-axis see-through display based on a dynamic phase correction approach

: The design and the analysis of an off-axis (50°) diffractive imaging optical system is presented in this paper. A 10°x15° field of view is considered. The optical system is composed of two diffractive optical elements. A static diffractive optical element having a frozen phase transfer function is used to perform a virtual point in the considered field of view. A dynamic diffractive optical element having an adapted calculated phase transfer function is used to compensate for aberrations of the static element. Using a sequential creation of virtual image points and considering human eye characteristics, it is shown that a nine points virtual image can be obtained with current technology. Moreover, it is presented that aberrations can be compensated whatever the position of the virtual point in the 10°x15° field of view. Finally, using rigorous coupled wave analysis, it is shown that an average diffraction efficiency of 79% can be reached across the considered field of view with a standard deviation of nearly 5%.


Introduction
Augmented reality (AR) supplements the real world with computer-generated objects that appear to coexist in the same space as the real world. AR technology has been used in fields as varied as medicine, robotics, manufacturing, machine repair, aircraft simulations, personal information systems, entertainment, gaming and education [1]. AR can be potentially applied to all senses, but only sight is considered in this paper. An AR scene can be visualized by using projection displays [2], handheld displays [3] or head-worn displays (HWDs) [4,5].
The present paper deals with HWD technology. An AR scene can be created with an HWD by using a see-through approach (video/optical) [6] or a projection approach [7]. In the video see-through approach the virtual environment is replaced by a video feed of reality; the AR is integrated into the digitized images. In the optical see-through approach, AR information is directly superimposed with a combiner onto the real-world perception. In the projection approach, an AR image is projected onto real objects and retro-reflected back to the eye of the user.
The present paper deals with optical see-through HWDs. They generally have off-axis configurations and therefore suffer for high geometrical aberrations as Gauss conditions are not respected. The compensation for their aberrations is thus fundamental to obtain high quality images. Several designs applying aspheric, diffractive, holographic or free-form surfaces have been proposed to improve compactness and imaging quality of see-through HWDs. For instance, Cakmacki [8] realized an off-axis magnifier using a free-form mirror as a combiner and a diffractive optical element in relay optics. Nagahara [9] developed a wide field of view catadioptrical HWD achieving a horizontal view angle of 180° to cover the human peripheral field of vision. Zheng [10] designed and realized an off-axis see-through HWD combining a tilted reflective combiner with an x-y polynomial surface in relay lenses to compensate for aberrations. Cheng [11] used a free-form prism to design a see-through HWD with a low f-number and a wide field of view. Ando [12] developed a stereoscopic HWD using an holographic optical element (HOE) as a combiner. Kasai [13] proposed a practical method for the design and the fabrication of HOEs as off-axis imaging systems. Amitai [14] proposed a HWD having an holographic combiner recorded with complex wavefronts to compensate for aberrations. In all these approaches, an image created by a microdisplay is relayed by lenses and superposed on the real environment by the combiner. The optical system is optimized to simultaneously minimize the aberrations for the entire image. However, with such a simultaneous approach, aberration compensation is not in general uniform and/or very close to the diffraction limit over the whole field of view.
In this paper, the design of an optical see-through HWD associating a static diffractive optical element (DOE) and a dynamic DOE (Fig. 1) is proposed. These DOEs are used to construct point-by-point a nine points virtual image in a sequential way over a 10°x15° field of view. The virtual image is displayed at a finite distance. The sequential construction of the virtual image has the advantage to perform an optimized correction of aberrations for each and every point of the nine points image by adapting the phase function of the dynamic DOE. The proposed optical system and the principles of the sequential creation of the virtual image are introduced in section 2. A discussion is done in section 3 to evaluate the feasibility of the proposed setup considering current technology. A method to determine and calculate the optimized PTF of the dynamic DOE is developed in section 4; the resulting aberrations compensation is also presented. The performances of the system in term of diffraction efficiency are discussed in section 5.

General description
An incident monochromatic plane wave Φ C1 of wavelength λ 0 is diffracted, at the first order of diffraction, and modulated by a dynamic DOE labeled 1 in order to generate a complex wave Φ D1 (Fig. 2). This complex wave acts as a reading wave Φ C2 for a static DOE labeled 2. The direction of the optical axis of the system is modified at the surface of DOE 2. Its normal is at an angle α 1 to the optical axis before its break and at an angle α 2 after its break. The reading wave Φ C2 is backward-diffracted, at the first order of diffraction, by the DOE 2 into a diverging wave Φ D2 until the eye of the user. The user will thus see, through DOE 2, a virtual point I superposed into the real environment. According to the description, DOE 2 is an imaging element. It is a reflective analogical element having a frozen phase transfer function (PTF) Φ PTF2 . This PTF has a simple form to ease the fabrication. The position and the aberrations of the image point I, especially coma and astigmatism [15][16][17], depend on the reading wave Φ C2 . DOE 1 is a dynamic computer-generated diffractive optical element. It is used to generate at the first order of diffraction a complex wavefront that compensates for the aberrations. Its PTF is dynamic. It is optimized for each position of the image point I to minimize aberrations. The method of calculation is developed in section 4. Fig. 2. Top view of the optical system (not to scale). Q is a point of the plane of DOE 1, P is point of the plane of DOE 2 and I is a point of the plane of the virtual image. α1 is the angle between the normal to DOE 2 and the direction (O1O2). α2 is the angle between the normal to DOE 2 and the direction (OO2). The thickness of DOEs is not drawn.

Specifications of the virtual image
When the reading wave Φ C2 is modified, the form and the position of I in the virtual space are modified. It is thus possible to construct an image sequentially "point by point" by controlling the reading wave. This sequential approach presents advantages as it allows an adapted correction of aberrations for each and every point of the image scene rather than a global and simultaneous optimization for the entire image. The properties of the virtual image are defined from the physiological characteristics of the eye. The sequential creation of the virtual image uses the afterimage effect which is around 50ms. As a consequence, all the points constituting the virtual image have to be displayed in a shorter time. According to the chosen technology for the dynamic DOE, this specification limits the number of points of the image (see section 3). The smallest distance separating two side-by-side points of the image can be defined thanks to the resolving power of the eye which is around 1 arc minute. At a display distance D of one meter, the separation distance is approximately 0.3mm. As a consequence, considering a 10°x15° field of view, nearly 600x900 points can be potentially displayed in a 180mm x 270mm area. It is therefore possible to display a virtual image with a limited number of points at different positions over the 10°x15° field of view. The studied case is limited to a virtual image with nine points.

Technological discussion
From a practical point of view, spatial light modulators (SLMs) can be used as dynamic elements. Development of SLMs has been an active research field with numerous potential applications [18]. Usually, SLMs are based on liquid crystal displays (LCD) and digital micromirrors devices. In this paper, only LCDs are considered as they can be used in reflection or in transmission. Nematic (twisted or parallel) and ferroelectric (smectic C* or smectic A*) phases are commonly used. They have different physical properties and offer therefore different performances. A brief description of the characteristics of these technologies is presented. Current technology allows the implementation of the proposed setup. Moreover, some practical limitations are discussed.

Modulation mode
Each liquid crystal allows light modulation because of their birefringence properties. When a voltage is applied, the internal structure of the liquid crystal is modified and a modulation is obtained (amplitude or phase). Theory and experimental characterization of twisted-nematic LCD (TN-LCD) have shown that amplitude and phase modulations are generally coupled. It is possible to obtain an amplitude-mostly or a phase-mostly modulation mode (2π or greater) by adding adjusted polarization elements [19,20]. Parallel-nematic LCD (PAL-LCD) behaves as a tunable retardation plate and a pure dephasing of 2π is possible [21]. Ferroelectric LCDs (F-LCD) are more suitable for amplitude modulation with generally a binary phase modulation (0π) [22]. Multiple-level phase-only modulation mode obtained by cascading of F-LCDs has been shown [23]. A continuous 0-2π phase modulation concept with F-LCD has also been proposed [24].

Response time
Whatever the technology used for the dynamic element, the number of points of the virtual image will be limited. As discussed in section 2, the display time must be less than 50ms.  The maximal number of points contained in the virtual image as a function of the frequency rate is presented in Fig. 3. The different technologies are shown in relation with their speed limits. With TN-LCD, the virtual image is limited to 10 points. The number of points that can be displayed with smectic C* F-LCD varies between 100 and 2000. Finally, smectic A* F-LCD can potentially display the greatest number of points, up to 250 000.

Pixel pitch
The SLM is a matrix of separated pixels on which a voltage is applied to perform a local modulation (see Fig. 4). The pixel pitch p of the SLM is an important parameter. Indeed, it defines the resolution limit R = (2p) −1 of the dynamic element and thus the practical implementation of the corrective phase transfer function which calculation is presented in section 4. Among commercially available LCD SLMs, pixel pitch is in a range from 5µm to 32µm. As a consequence, the resolution of the dynamic element is currently limited from 15lp/mm to 100lp/mm. Table 1 summarizes the previous discussed points for each liquid crystal type. In our case, three requirements are needed for the optical system. First, nine points have to be displayed in a time shorter than 50ms. Then, a continuous phase only modulation is necessary for the dynamic element. Finally, the resolution of the dynamic element must be as high as possible. Such requirements can be obtained with a nematic LCD. Their slow frequency rate is enough for sequential generation of nine points in a time shorter than 50ms. Moreover, they can perform a nearly continuous phase-only modulation (more than 2π). As it is shown in section 4.4, resolution is sufficient to perform the expected correction. For applications requiring a higher number of points, a solution based on ferroelectric LCDs would have to be considered due to their high frequency rate.

Determination of the PTFs of the dynamic reconfigurable element
In this part, the method used to determine and calculate the theoretical corrective PTF of the dynamic DOE for a given point I of the virtual image is developed. Results obtained with and without calculated aberration compensation are discussed.

Assumptions
Assumptions done in this section are: (1) DOEs are pure phase elements, absorption is not considered; (2) DOEs are considered infinitely thin as corresponding thicknesses are very small comparing to the displaying distance of virtual points; (3) each image wavefront is assumed to be aberration-free, i.e. a spherical diverging wave; (4) wavefronts are described locally by a set of rays normal to the local tangent planes according to the eikonal description. It is therefore possible to link the phase Φ and the unit propagation vector u of the local ray by the following relation: where n is the average refractive index of the modulated medium and λ 0 the recording and readout wavelength in vacuum. This relation is valid only for slowly varying wavefronts.

Phase transfer function of the static element
The static element is obtained by the recording of a superposition of two counter-propagating laser beams (Fig. 5a): a collimated plane wave, as reference beam and a diverging spherical wave, as object beam. The reference beam is directed to the recording hologram plate with an incidence angle θ R while the object wave is centered at point S. The corresponding phase function in the plane of the static element is described by where Φ O and Φ R are respectively the phase distributions in the plane of the static element of the object wave and the reference wave. Inserting the various geometrical parameters and taking the constant phase term in such way that Φ PTF2 (O) = 0, the recording FTP can be written as where k 0 = 2π/λ 0 , P is one point at the surface of static element and d(A,B) is the distance between two points A and B. The restitution geometry is presented on Fig. 5b. The diffracted wave phase at the first diffraction order is given by where Φ C2 and Φ D2 are respectively the phase distributions of the reading wave and the diffracted wave in the plane of the static element. A reconstructed wave deviates from the desired spherical one excepted in the case where restitution and reference recording waves are identical. Strong primary aberrations i.e. coma and astigmatism are present.

Method of determination of the correcting phase transfer functions
In this section the different steps for the determination of the ideal PTF of the dynamic DOE for a given point I kl of the virtual image are presented. Obviously, it is the same approach for each point of the virtual image. A reverse approach has been used. Light is propagated into the optical system from a virtual point I kl to the plane of the dynamic DOE, where we want to determine the two-dimensional phase distribution (Fig. 6). The first step consists in defining by ray-tracing a sampling grid of N points in the plane of each DOE. These two grids are used to obtain the phase of the different waves and thus the PTF of the dynamic DOE. The surface of the static DOE is uniformly sampled into N P ij points. N rays originating from the point I kl are traced to the points P ij . These N rays are locally orthogonal to the ideal spherical diverging diffracted wave. Their associated unit propagation vectors u D2 are given by Ray-tracing formula in vectorial form for a diffractive element is then applied to obtain the unit propagation vectors u C2 of the rays locally orthogonal to the ideal reading wave of the static DOE: where sin( ) and 0 , cos( ) (7) are respectively the unit propagation vector of the rays locally orthogonal to the object and the reference waves used to define the PTF of the static DOE (Fig. 5a). At this stage, the sampling grid in the plane of the dynamic DOE can be defined. Indeed, its sampled points Q ij are at the intersection of the rays directed by u C2 with the plane of the dynamic DOE. Phase distributions can be expressed into the coordinate system (Oxyz) at points P ij (x ij We can thus calculate the phase of the ideal reading wave in the plane of the dynamic DOE: Equation (10) is available with the local planar approximation. The ideal PTF of the dynamic DOE can finally be numerically expressed at points Q N : where Φ D1 = Φ C2 and Φ C1 is an incident monochromatic plane wave. We change the reference coordinate system to express Φ PTF1 in the coordinate system (O 1 x 1 y 1 z 1 ). Then we realize a X-Y polynomial interpolation to obtain an analytical form of the PTF of the dynamic DOE: ( ) x y a x y π φ λ = ∑∑ (12) With this analytical form, a particular set of coefficients a kl is obtained for each position of the virtual image point I. Moreover, this description is relevant to determine the local spatial frequencies ν X and ν Y along the direction (O 1 x 1 ) and (O 1 y 1 ). As discussed in section 3.3, the practical realization of the PTF is limited by the SLM pixel pitch. The absolute values of these spatial frequencies are defined as the following: These spatial frequencies depend on the incidence (θ 1 , δ 1 ) (Fig. 7) of the incident monochromatic wave in Eq. (11). Fig. 7. Three-dimensional representation of the oblique incidence (θ1, δ1) of the incident plane wave at the surface of the dynamic DOE. θ1is the angle between the wave normal uC1 and the z1 axis. δ1 is the angle between the plane of incidence and the x1 axis.

Results
A monochromatic light wave with λ 0 = 514.5nm is considered for the recording of the static element as well as for its reconstruction. The incidence of the recording reference plane is θ R = 50°. The source point S of the recording object wave is at the position (−421.2mm, 0mm, 903.3mm). As it is shown on Fig. 2, the angle α 2 between the normal of the static element and the normal of the dynamic element is 50°. Their centers O 1 and O 2 are distant of 50 mm.
The image used for simulation is presented in Fig. 8. It is a 3x3 matrix centered in point I 22 which is located at the same place than the point S used for the recording geometry. The eight other points are located at peripheral positions of the 10°x15° field of view. The coordinates of these nine points are listed in Table 2. Aberrations are maximal at the peripheral points. As a consequence, if the aberrations compensation is obtained for these points it can also be achieved for the points inside the 10°x15° field of view since they introduce fewer aberrations.
The method developed in section 4.3 has been used to calculate corrective PTF of the dynamic DOE for the nine points of the image. The analytical form (Eq. (12)) has been obtained by applying a 3rd order polynomial interpolation. Two configurations for the incident monochromatic wave have been considered for the calculation of spatial frequencies ( Fig. 9): (a) the incident wave used to lighten the dynamic DOE is a plane wave having a normal incidence whatever the position of the virtual image point I, the beam is both deflected   and modulated by the dynamic DOE; (b) the incident wave is a plane wave with an oblique incidence (θ 1 , δ 1 ) (Fig. 7) that depends on the position of the point I of the virtual image. In this case, the unit propagation vector u C1 is assumed to be identical to the main unit propagation vector u D1 of the wave diffracted by the dynamic DOE. The deflection is assumed to be done before the dynamic DOE by a system which is not studied in the present paper. The calculated spatial frequencies (Eqs. (13), (14)) are presented in Table 3. The technological limitation of 100lp/mm (see section 3.3) is only respected by configuration (b).
Modulation transfer functions (MTFs) with and without aberration compensation (Fig. 10) have been obtained using the CodeV® software. As expected, the MTFs of the uncorrected Table 3. Spatial frequencies in (O1x1) and (O1y1) direction of the optimized PTFs of the SLM for the nine points of the grid defined in Fig. 8 Configuration (a)  Point  I11  I12  I13  I21  I22  I23  I31  I32  I33  setup (blue curves) are far from the limit of diffraction excepted for the central point of the image. On the other hand, the MTFs of the optimized system (red curves) are very close to the limit of diffraction whatever the position of the point in the 10°x15° field of view.

Diffraction efficiency
In order to present a complete description of the performance of the optimized studied system, the diffraction efficiency (DE) of the proposed setup has been calculated over the 10°x15° field of view.

Theory and assumptions
Considering the geometry of the setup, the diffraction is conic and Kogelnik's approximated coupled wave theory cannot be used [26]. DE calculation has been done using the rigorous coupled wave analysis (RCWA) theory [27]. It is valid for a planar grating lit by an obliquely incident plane wave. The DE of the whole setup is obtained by multiplying the dynamic element DE with the static element one. In this paper, only static element DE is considered, that is to say dynamic element DE is assumed to be constant (100%) whatever the position of the image point in the 10°x15° field of view. The static element is assumed to be a volume hologram, i.e. its thickness is not neglected.
Here, RCWA cannot be used directly for two reasons. The static hologram is not a planar grating and reading waves have a complex form. A local description is therefore used (Fig.  11). Fig. 11. Zoom to the localized planar grating description of the static DOE. Λ is the local grating period, Φ is the local slanted angle, (K) is the local grating vector, uC2 is the local unit propagation vector of the reading wave, θ2 is the local incidence, δ2 is the angle between the plane of incidence and the x-axis, e is the thickness of the element, ψ is the arbitrary polarization angle.
The static element is divided into N elementary rectangles each one centered on a point P ij . Because of the slow variation of the phase function, each area is considered as a localized planar grating having a local period Λ and a local slanted angle Φ [28]. In order to have a plane wave description, reading wave are described locally by a set of rays normal to the local tangent planes according to the eikonal description. A local grating vector K and a local slanted angle Φ can thus be calculated. RCWA can therefore be used to calculate local DE η ij as defined in [27]. The total diffraction efficiency η for a given position of the virtual point in the 10°x15° field of view is then obtained using , ij i j N η η = ∑∑ (15) where N is the total number of P ij points considered.

Results
Using the description presented in section 5.1, the diffraction efficiency of the optimized system has been calculated over the 10°x15° field of view (Fig. 12). It has been sampled into 81x81 uniformly distributed virtual image points. A transverse electric polarization state (ψ = 90°) has been considered. The static hologram of 16.2mm x 17.2mm dimension has been divided into 100 elementary rectangles. By optimizing the index modulation and the thickness of the hologram, an average diffraction efficiency of 79% can be reached over the considered field of view with a standard deviation close to 5%.

Conclusion
The design of an off-axis see-through display associating two diffractive optical elements has been presented in this paper. A dynamic method to display sequentially a virtual image having nine points in a 10°x15° field of view has been proposed. The feasibility of this approach has been discussed considering the current state of the technology. A nematic LCD SLM can be used to perform the dynamic setup. It has been shown that aberrations compensation of the virtual image can be done whatever the position of its points into the 10°x15° field of view. These compensations allow to obtain MTFs close to the limit of diffraction. With an optimal analog holographic recording for the static element, relatively high diffraction efficiency (79%) with good uniformity (5% standard deviation) has been obtained across the considered field of view.