Optimizing the display performance for virtual reality systems

: We propose a systematic optimization method for two commonly used display devices, organic light-emitting diode (OLED) display and liquid crystal display (LCD), for virtual reality (VR) headsets. An optical simulation model for the VR system is established, and three performance metrics, namely total light efficiency (TLE), field color gamut coverage, and field color non-uniformity, are proposed as the optimization objectives. For the RGB (red, green, and blue) OLED display, the microcavity structure is optimized to suppress the field color unevenness while maintaining a high TLE and large field color gamut coverage. For the direct-lit LCD, the optimization is from the viewpoint of entire VR system. A two-dimensional patterned prism film is implemented in the backlight unit to locally modulate the radiation pattern of the LCD. Thus, the vignetting effect in the VR system is alleviated and the TLE is further enhanced by 40%. After optimizing the OLED display and LCD, their pros and cons in the VR systems are analyzed. Our optimization method is proven to be effective for designing a proper display panel for VR systems.


Introduction
The fierce competition between organic light-emitting diode (OLED) displays and liquid crystal displays (LCDs) has a long history [1,2].As an emissive display, OLED exhibits a true dark state, fast response time, and thin form factor [3].However, to improve color purity and out-coupling efficiency, the embedded microcavity in a multi-layer OLED structure causes a nonnegligible angular color shift.Therefore, to optimize the microcavity of an RGB (red, green, blue) OLED display, the tradeoff between color gamut, angular color shift, and out-coupling efficiency should be considered [4][5][6].On the other hand, the mature manufacturing process of LCD panels provides an important cost advantage.However, the light leakage in an LCD panel severely limits its contrast ratio (CR) [7,8].To reduce the dark state light leakage and color shift, and to widen the viewing angle, film-compensated multi-domain LCD structures have been commonly used [9][10].A typical CR of an LCD varies from 1000:1 to 5000:1, depending on the LC mode employed.Recently, by using a two-dimensional (2D) mini-LED array with local dimming control, a thin-profile, high-dynamic-range LCD with CR=10 6 :1 has been achieved [11][12][13].In addition, the high peak brightness of 2D mini-LED backlit LCDs can achieve a higher ambient contrast ratio than OLED displays under strong ambient lighting environments, such as automotive displays and outdoor displays [14].Therefore, another round of competitions between mini-LED backlit LCDs and OLED displays in televisions, tablets, notebook computers, gaming monitors, and vehicle displays have just begun.
Unlike direct-view displays, such as computers and TVs, the immersive virtual reality (VR) displays use a lens to magnify the images [15][16][17].Therefore, the display performance analysis and optimization processes in a VR system are quite different from those of a direct-view display.Figure 1(a) depicts a direct-view display.The viewers may see such a display from different angles.Thus, a lot of efforts are focused on minimizing the unevenness of color, brightness, and contrast at different viewing angles.On the other hand, Fig. 1(b) depicts the schematic of a VR display system.The light emitted from the display panel is first refracted by a magnifying eyepiece, and then reaches the eyebox.Because of its small etendue, only a small portion of the display light can reach the eyebox (marked as green color), while the rest is either wasted or becomes stray light in the optical system (marked as gray color) [18].In addition, those pixels near the edge of the display panel may impinge on the edge of the lens, leading to a larger refraction angle from the lens.In contrast, those pixels near the center of the display panel will hit the center of the lens where the refraction angle is small.More specifically, different display pixels will have different emission cone angles that can reach the eye pupil through the imaging lens.The abovementioned differences between VR and direct-view displays bring up some unique requirements to optimize the display panel for a VR system [19].For example, resulting from the small etendue of the eye pupil, a directional display can improve the optical efficiency of the VR system [20].In addition, as shown in Fig. 2(a) and 2(b), because different emission pixels of the display have different emission cone angles that can reach the eye pupil, the angular dependent light leakage, intensity variation, and emission spectrum of the display may cause the field non-uniformity (vignetting effect) in both brightness and color.Here, to separate the parameters used for describing the display panel itself and the virtual images in a VR system, we use field brightness and field color to characterize the properties of the virtual images, while using brightness and emission spectrum to describe the image properties of the display panel.In this paper, we first build a raytracing optical simulation model to determine the angular power collection efficiency (ratio of the power received by the eye pupil to the power emitted from the display pixel) of each display pixel in the VR system.After that, three VR system metrics (total light efficiency (TLE), field color gamut, and field color non-uniformity) are defined and implemented in the display panel design.Finally, the microcavity structure of the RGB OLED display and the backlight structure of the LCD are analyzed and optimized based on the above three metrics.

Device structure
Here, we focus on the lightweight VR system with a Fresnel lens [21].The system consists of a flat panel display (OLED display or LCD), a Fresnel lens, and a receiver (circular eye pupil with a diameter of 4 mm).The raytracing model in the LightTools software is illustrated in Fig. 3(a).In the model, the Fresnel lens diameter is ∼45 mm, the eye relief is ∼15 mm, and the distance from display to the lens is ∼35 mm.For the display, point sources are used to represent the pixels in the display panel.In addition, according to the circular symmetry of the VR system, we can simplify the display system in one direction.Therefore, 28-point sources with 1-mm pitch are built along the y-direction to represent the entire display system.In each point source, the total emission cone (polar angle (θ): 0 °to 90 °; azimuthal angle (φ): 0 °to 360 °) is divided into 91 units by the polar angle, with an interval of 1 °.Then, we sweep the polar angle of each point source from 0 °to 90 °to define the angular power collection efficiency of each cone unit.Results are plotted in Fig. 3(b).Again, the angular power collection efficiency (C (θ, y)) is the ratio of power emitted from the point source to the power received by the receiver (eye pupil).As Fig. 3(b) shows, when the point source is in the central area (y-position = 0), only the emission cone with a small polar angle can reach the eye pupil.On the other hand, the emitted light from edge pixels reaches the eye pupil through the emission cone with a larger polar angle.Based on the radiation pattern of the display and angular power collection efficiency of the VR system, the power received by the eye pupil from different pixels can be defined as: and the optical system efficiency (OSE) of the VR system which is the ratio of total power received by the eye pupil to the total power emission from the display can be further defined as: In Eq. ( 1) and Eq. ( 2), I(θ) is the angular distribution of the emission pixel, Ω(θ) is the unit solid angle, and C (θ, y) is the angular power collection efficiency function of the VR system.In addition, the illuminance (E) of each pixel in the eyebox is the ratio of the received power calculated by Eq. ( 1) to the pupil area (4π mm 2 in our case).Then, the field brightness (L) can be further calculated from E = L×Ω×cosθ, where Ω is the solid angle and θ is the angel from normal to the target pixel.
From Eq. ( 1), an effective method to improve the optical efficiency is to match the radiation pattern of each pixel with the angular power collection efficiency function of the VR system.Since the angular power collection efficiency function is concentrated in a small polar angle range, directional displays are widely used to improve the optical efficiency of the VR system.To generally explain the total light efficiency improvement in displays with various radiation patterns, we first use a point source with Lambertian radiation pattern and its exponentiation I(θ) = cos n (θ) in the simulation model.In Sec. 3, when we optimize the OLED display and LCD, a realistic radiation pattern of the display will be applied.Here, the OSE of the VR system with various radiation patterns is normalized to the Lambertian emission panel.As Fig. 4 shows, the directional display can indeed improve the OSE of the VR system.However, in a real display panel, modulating the radiation pattern usually brings up some side effects.Taking an OLED display as an example, applying a strong microcavity to achieve a narrower radiation pattern may cause severe angular color shift and lower out-coupling efficiency.In addition, concentrating the backlight emission toward the normal angle may also lose the overall power efficiency of the LCD.Another side effect of applying directional display in an VR system is the uneven OSE in each pixel.As shown in Eq. ( 1), the OSE varies at different pixel positions.Therefore, a directional emission display may cause a large OSE difference between the center and the edge pixels, resulting in a vignetting effect.Thus, it is critical to systematically optimize the optical structure of the employed LCD and OLED display panels for an VR headset.
In Sec.3.1, the analysis and optimization process of the RGB OLED display in an VR system are presented.The tradeoff between TLE, field color non-uniformity, and field color gamut coverage are considered.Afterwards, in Sec.3.2, a 2D patterned prism with nine local zones will be implemented in the LCD backlight unit to locally modulate the radiation pattern to match the power collection efficiency functions of the VR system in each local region.It is verified that the proposed 2D patterned prism can effectively relieve the vignetting effect and improve the total light efficiency of the VR system, simultaneously.

RGB OLED display
The rigorous dipole model for the planar OLED device is implemented to obtain the out-coupling efficiency and angular emission spectrum of OLED devices.From our previous results [4], two factors are important in designing the microcavity of OLED devices.One is the strength of the microcavity, i.e., the reflectivity of the cavity reflector.As shown in Fig. 5, to conveniently adjust the microcavity strength, we used a different number of SiO 2 and TiO 2 layers to form a Distributed Bragg Reflector (DBR) on top of the OLED device in the simulation model [22].
Here, the DBR is designed to match the emission spectrum of each RGB OLED to achieve the quarter-wave condition for maximum reflectivity.Another factor is the cavity thickness to modulate the resonance wavelength of the OLED display.Here, we vary the HTL (hole transport layer) thickness to achieve a preferred cavity design.The DBR based OLED device in our model is mainly used to demonstrate the optimization process of the RGB OLED display in the VR system.There is no doubt that other types of OLED display, which uses semitransparent metals to form microcavities, can also benefit from the same analysis procedure to optimize their device structures for VR applications.We first use a single green OLED device to illustrate the optimization process.Three metrics (TLE, field color gamut, and field color non-uniformity) are proposed to evaluate the display performance in a VR system.The total light efficiency (TLE) is the product of following two parameters: the output coupling efficiency (OCE) of the OLED device and the OSE of the VR imaging system.The OCE and corresponding radiation patterns of various OLED devices are calculated by the abovementioned home-made MATLAB code.According to different microcavity designs, the radiation pattern changes from a directional emission to a batwing emission.In Fig. 6(a-d), the OCEs of some OLED devices with different cavity designs are shown by the blue dotted lines.Besides, according to Eq. ( 2), the corresponding OSE in the VR system is also drawn with orange dashed lines.The TLE, which is the product of OCE and OSE, is also plotted as the yellow solid line.From Fig. 6(d), the maximum OSE of the VR system is found in the OLED device with 3 pairs of DBR.The stronger cavity effect results in a greater intensity modulation, and therefore more display emission light is concentrated at the normal direction.However, such a strong cavity design also sacrifices the OCE of the OLED device.Considering the tradeoff between the OCE of the OLED device and the OSE of the VR system, the optimized TLE of the green OLED device is with 2 pairs of DBR, and its HTL thickness is 65 nm, as Fig. 6(c) depicts.The second and third metrics are related to color performance: one is field color gamut coverage and the other is field color non-uniformity.As illustrated in Fig. 2(b), the angular power collection efficiency of different pixel positions will also affect the received emission spectrum, which can be further defined as: where S(θ) is the angular emission spectrum of the OLED display, and the definitions of other parameters have been given in Eq. ( 2).
Based on Eq. ( 3), we can define the received emission spectrum of each emission pixel.Here, we use the pixel in the center of the display to define the field color gamut coverage of the VR system.The strong resonant effect produced by a large number of DBRs narrows the full width half maximum of the OLED emission spectrum and increases the color purity.Figure 7(a) shows the field color coordinate (CIE1931) of the green OLED in the VR system.With a strong cavity and a preferred resonant cavity design, a high-color-gamut OLED display is achieved.In addition, the field color non-uniformity is further defined by the color difference between adjacent pixels and the center pixel of the display.As shown in Fig. 7(b-e), on one hand, the stronger the microcavity, the greater the angular color shift, which leads to a larger field color unevenness in the VR system.On the other hand, as Fig. 3(b) shows, as the pixel position moves from the center (y=0 mm) to the edge of the display (y=28 mm), the difference in power collection efficiency function gradually increases and then decreases.The biggest difference occurs at the pixel with y = 23 mm.The trend of power collection efficiency function also causes the largest field color unevenness in the VR system to appear at the pixel y=23 mm.
As discussed above, the microcavity structure of an OLED device greatly affects the three objects (TLE, field color gamut, and field color non-uniformity) of a VR system.Therefore, how to design an RGB OLED display to achieve high TLE, wide field color gamut, and indistinguishable field color unevenness in a VR system becomes crucial.When considering the RGB OLED panel, because each color has its own TLE, we define the object (TLE panel ) as the average value of the RGB colors, i.e., TLE panel = mean {TLE R , TLE G , TLE B }.In addition, based on the field color coordinate of RGB OLED, the field color gamut of VR system can be defined.Regarding the field color unevenness, considering the color mixing of RGB subpixels, the field color unevenness of the VR system is defined by the average field color unevenness of the first 18 colors in Macbeth Color Checker [23], rather than a single color.Nowadays, for high-end mobile devices, the display requires a color gamut coverage more than 95% in DCI-P3 standard, and the color shift less than ∆u'v' = 0.02.These two criteria are also applicable to the requirement of VR system for the field color gamut coverage and field color unevenness.Based on the above discussion, next we would like to find an RGB OLED panel, which can maintain field color unevenness ∆u'v'≤0.02 and field color gamut coverage ≥ 95% in DCI-P3 standard, and then maximize its TLE panel .
As shown by the red dots in Fig. 8, under a given DBR we can have 216,000 RGB OLED combinations (each color has 60 cavity designs) in the optimization process.According to the above requirements, in Fig. 8(a-d), the axis of the field color unevenness is truncated at 0.02.On the other hand, to demonstrate the result of OLED devices without DBR, the axis of the field color gamut coverage (Fig. 8(a-d)) is truncated at 80% instead of 95% DCI-P3.However, during the optimization process, we still use 95% DCI-P3 coverage as the standard.In the optimization process, we first connect all extreme cases of the data with black lines to form the outer surface.Afterward, the TLE panel value of points on the outer surface is further represented by different colors.The colormap is displayed on the right side of the graph.Then, we apply two hard criteria [field color non-uniformity ≤ 0.02 and field color gamut ≥ 95% DCI-P3] to filter out the extreme cases meeting our requirements, and then find the combination of RGB OLED display with a maximum TLE panel .Finally, the optimization points are marked in magenta color for the OLED displays with different pairs of DBR.According to Fig. 8(c), the RGB OLED display with 2 pairs of DBR can provide the maximum TLE panel in the VR system, and the corresponding [HTL thickness; OCE; OSE for RGB OLEDs] is [125 nm; 12.32%; 0.68%], [75 nm; 12.51%; 0.6%], and [50 nm; 11.07%; 0.58%], respectively.The average TLE panel of RGB OLEDs is about 0.0744% (without a circular polarizer).The field color gamut maintains at 97.74% coverage of the DIC-P3 standard, and the field color non-uniformity is kept at 0.0064.Further discussion on the differences between optimizing OLED devices for direct view displays and VR headsets can be found in the Supplement 1.

LCD display system
A transmissive LCD consists of a backlight unit and an LC panel.The backlight provides more freedom to modulate the radiation pattern of the display than its OLED counterpart.It is different from OLED displays that having trade-offs between color gamut, angular color shift, and OCE.The radiation pattern and color gamut of an LCD is mainly determined by the backlight unit, while the angular color shift and dark state light leakage are mainly caused by the LC panel.In our simulation, we use the parameters of a low viscosity LC material reported in [24] in the short-range lurch control in-plane switching (SLC-IPS) mode [25] in order to achieve a fast response time for VR display applications.As shown in Fig. 9(a), the electrode area is shown by green color, the dead zone gap marked by the red dashed lines with space W = 2.3 µm, and the subpixel pitch L = 8 µm for a high-resolution-density [>1000 PPI (pixel per inch)] display panel.The on-state voltage is set at 7 V to drive in the fast response area [26].The obtained average gray-to-gray response time is 2.74 ms.The optical properties of such an LC panel compensated by the + A and -A plates as described in [27] are simulated by a commercial LCD simulator (Techwiz LCD 3D) and results are plotted in Fig. 9(a-d), including 2D transmittance distribution, VT curve, angular color shift, and isocontrast contours.
In an LCD, the emitted backlight is further modulated by the LC panel to control the brightness pixel by pixel.Therefore, its radiation pattern can be defined as: where I display and I backlight is the radiation pattern of the LCD and backlight unit, T LC is the transmittance of LC panel with different incident angle (θ, and φ), wavelength (λ), and gray level (g), and T CF is the transmittance of color filters (CFs).For simplicity, we set T CF = 0.33 for the employed white light source.In addition, in practical applications, the thin film transistors (TFTs), data lines, and gate lines also occupy a portion of the pixel area and block the backlight.Therefore, the aperture ratio should be considered when the LCD panel layout has been defined.
Here, a backlight unit with a Lambertian radiation pattern is implemented in the LCD, and the corresponding display radiation pattern is obtained from Eq. ( 4).As analyzed in the OLED based VR system, the angular light leakage and color shift of the displays cause uneven field brightness and field color in the VR system.However, as shown in Fig. 9(c), due to the wide viewing angle of IPS LCD, the angular color shift of the LC panel can be ignored within the 20°p olar angle range, corresponding to the upper limit of the angular power collection efficiency function.Figure 9(e) shows that the worst field color unevenness of the three primary colors is about 0.0005, which is much smaller than the just noticeable color difference of 0.02.In addition, as shown in Fig. 9(d), the light leakage of the LC panel is better compensated in the horizontal view than in the vertical direction, so the contrast ratio is widely expanded in the horizontal direction.Considering the influence of the VR imaging lens, the light leakage rate of each emitting pixel is shown in Fig. 9(f).In a VR system, pixels located at the edge of the display in the x-direction will have more serious light leakage.In addition to the LC panel, by using patterned prisms with different base and apex angles, the radiation pattern of the backlight unit can be locally modulated.As shown in Fig. 10(a), the mini-LED backlit LCD consists of a backplane reflector (specular reflector: R=90%), a mini-LED array (Lambertian emission source), a diffuser (Lambertian diffuser), and two crossed brightness enhancement films (BEFs; apex angle=90°; base angle=45°; width=50 µm).Firstly, we set the refractive index of the BEF to vary from 1.4 to 1.7 and plot the corresponding angular intensity distribution after crossing the BEFs in Fig. 10(b).When the refractive index of BEF increases from 1.4, 1.5, 1.6, to 1.7, although the total output power of backlight unit drops from 100%, 87.5%, 72.5%, to 56.43%, the light intensity at normal direction increases from 58%,67%, 83%, to 100%.Considering the angular power collection function of the VR system, the larger intensity in normal view may achieve a higher field intensity in the center area of the display panel.To verify our inference, we calculate the radiation pattern of the on-state LCD by Eq. ( 4) and import the results to the optical simulation model in LightTools to get the TLE distribution of the VR system.Here, the TLE is the ratio of power received by the eye pupil to the power emitted from the LED devices.All the losses in backlight unit, LC panel, color filters, and VR system are considered.Figure 10(c-f) shows the results of LCD backlight unit with a BEF, whose refractive index ranges from 1.4 to 1.7.The TLE of pixels at the center of the display panel (x = 0, y = 0) is raised from 0.011%, 0.014%, 0.016%, to 0.018% as the refractive index of BEF increases from 1.4, 1.5, 1.6 to 1.7, respectively.However, the TLE uniformity between the center and margin pixels (the ratio of TLE margin to TLE center ) also changes from 74%, 74%, 77%, to 43%.This uneven TLE may further cause the vignetting effect in the VR system.
To further improve the TLE and solve the vignetting issue of the VR system, the panel is evenly divided into 9 uniform zones, that means the zone width is 1/3 of the panel width.In each zone, the unit cell which is an inverted prism (n=1.4) with an apex angle β=30°, a base angle α=60°, and a width w=50 µm is implemented on the top of the crossed BEF to locally modulate the radiation pattern of the LCD backlight as shown in Fig. 11(a).The angle between the prism and the y axis is different in each zone, ranging from 0°to 315°, with an interval of 45°f rom zone 1 to zone 8. Regarding to the symmetric structure from zone 1 to zone 8, here we only illustrate the radiation pattern of zone 1 to zone 3 in Fig. 11(b) to Fig. 11(d).Finally, the field intensity distribution of the VR system based on the LCD backlight with 2D inverted prisms is shown in Fig. 11(f).Compared to the result in Fig. 10(f), the inserted 2D inverted prism further increases the TLE of the VR system from 0.012% to 0.017% and improves the TLE uniformity from 43% to 77%.

Discussion
After optimizing the LCD and OLED display for the corresponding VR system, let us discuss the pros and cons of these two competing technologies.Firstly, the TLE for both displays when used in the VR systems is relatively low (LCD: 0.017%, OLED: 0.074%), which would increase the power consumption of the VR headsets.It is worth noting that to prevent the stray light reflection from degrading the contrast ratio of the OLED panel, a circular polarizer (CP) is required.The transmittance of a CP is about 42%, which reduces the TLE of an OLED based VR system to 0.031%.That means, the RGB OLED panel exhibits a 1.8x higher TLE than LCD.However, to compare the total power consumption of a VR display at 200 nits, we should also include the electronic driving efficiency.It is known that thin-film transistor (TFT) is not an efficient current source.Therefore, the backplane driving efficiency of an OLED panel is generally lower than that of the corresponding LCD, especially when displaying high brightness and high average pixel level (APL) images.As demonstrated in [1,28], the driving efficiency is related to the electrical characteristics of the employed TFTs, circuit design, and resistance of the metal lines.
Here, we only focus on the optical efficiency of the display panels, and the electrical driving part is left for future work.
In terms of optical efficiency, the major differentiating factor is color filters: the RGB OLED panel does not require color filters, but LCD does.For the LCD based VR system, a high PPI and fast-response LC panel is required to suppress the screen door effect and motion blurs.However, a relatively large ratio of black matrix coverage in a high PPI LC panel and the dead-zone area designed to achieve fast response time jointly reduces the aperture ratio of the SLC-IPS LC panel.Therefore, the average transmittance of the LC panel itself in the voltage-on state is only 6%.For reference, the transmittance of two parallel polarizers used in our simulations is 35%.Such a low transmittance is confirmed experimentally in [25].If we further include the 67% absorption loss from color filters, the average light transmittance of a high PPI, full-color SLC-IPS LCD panel is only about 2%.In a VR system, the required display brightness is about 200 nits.That means, the backlight should provide over 10,000 nits.This is not difficult for the recently developed mini-LED backlight.For a 2.02-inch mini-LED backlit LCD panel, the backlight brightness can achieve 12,000 nits under 10% duty ratio [29].Low duty ratio (10%-20%) is commonly used for both LCDs and OLED displays to reduce motion blurs in VR displays.According to [30], the motion picture response time (MPRT), which describes the severity of motion blurs is proportional to the display duty ratio and frame time.Considering the challenges for the driving circuits to support a high frame rate in a high PPI display, low duty ratio plays a key role to achieve an impulse-type, CRT (cathode ray tube)-like MPRT (1.5 ms).For an LCD panel, the maximum allowable LC response time is equal to the frame time minus data scan time and backlight emission time.Let us take a 90-Hz LCD with 10% duty ratio as an example.If the data scan time is 7 ms, the maximum allowable LC response time is only 3 ms.Therefore, a fast-response LCD is required.On the other hand, if an OLED display is operated at 10% duty ratio to achieve MPRT=1.5 ms, its brightness should be increased by 10× to compensate for the decreased brightness.Under such a high driving current, the burn-in and lifetime degradation issues should be taken into consideration.
In the following, we use one example to illustrate the factors causing the low efficiency of a high PPI LCD panel and analyze the tradeoffs between fast response time and high transmittance.
Here, let us take an LC panel with subpixel size of 8µm×24µm (1058 PPI) as an example.The mesh black matrix has a line width of 3µm [31], covering on top of the metal lines and thin-film-transistors in the driving circuits.The black matrix occupies about 50% area (3µm × (8µm +24µm) / 8µm×24µm) of each sub-pixel.In other words, the backlight can only pass through the panel in the remaining 50% area.The transmitted light is further modulated by the polarizers, CFs, and dead zone areas.A polarizer with a thicker dichroic absorption layer and higher absorption coefficient can provide a higher extinction ratio but lower light transmittance [32].Therefore, under a reasonable balance the light transmittance of two parallel polarizers is normally about 35%.For CFs, a good match between the emission spectrum of the backlight and the transmission spectra of CFs helps minimize the loss [33].In addition, as mentioned above, the dead zones (also called standing layers) are intentionally generated to achieve fast response time.Enlarging the gap between dead zones would improve the transmittance but sacrifice the response time [34].Therefore, the tradeoff between LC response time and panel's transmittance should be considered on a case-by-case basis.In the SLC-IPS LCD, the dead-zone coverage is about 50%.A novel LC panel design that can achieve fast response time with less dead zone coverage is critical to further improve the optical efficiency.So far, considering the optical loss mentioned above, the total light transmittance of a typical LCD panel is about 3% [100% (backlight emission) × 0.5 (aperture ratio) × 0.35 (two polarizers) × 0.33 (color filters) × 0.5 (dead zone coverage) = 3%].This should be a reasonable estimate for a high-PPI LCD panel.In addition, for convenience, the peak transmittance of the above LC panel is assumed to be 100%, but in reality, the peak transmittance could be slightly lower (80-95%), depending on the LC operation mode employed [35].To improve the LCD panel's efficiency, a dual brightness enhancement film is commonly used because it can boost the optical efficiency by ∼1.6x through polarization recycling.To find an LC panel with high resolution density while keeping high transmittance and fast response time is urgently needed.Besides improving the efficiency of LC panel, the backlight unit offers another degree of freedom to locally modulate the radiation pattern of the display, thus overcoming the trade-off between OSE and larger field non-uniformity (vignetting effect), as shown in Fig. 11(f).Therefore, a quasi-collimated backlight unit, together with a 2D patterned prism having more local zones, can further improve the OSE of a VR system without sacrificing the image quality.
On the other hand, for a high-resolution density OLED display with over 1000 PPI, the compensation circuit usually consists of 5-7 TFTs, which is not compatible in such a small pixel area.Thus, an external compensation circuit is commonly used in an OLED-on-glass to compensate the mura.These sensing circuits, external compensation algorithms, and additional control integrated circuits increase the complexity and power consumption of the backplane.In addition, the OLED-on-silicon driven by CMOS (complementary metal-oxide semiconductor) backplane is also proposed for high PPI OLED displays.Its small driving circuit is suitable for high PPI OLED displays, but its high cost limits its widespread applications for VR headsets.
Finally, micro-LED display has attracted much attention because it offers an unparalleled peak brightness and small aperture ratio to enable sunlight readability [1].However, its high cost due to premature mass transfer technologies and reduced efficiency as the chip size shrinks to 5 µm or less could limit its widespread applications in VR headsets.In an immersive VR system, the ability to provide a high peak brightness is not too critical.Thus, micro-LED display should be a strong contender for AR displays, especially when the ambient light is strong.

Conclusion
We have proposed three metrics including total light efficiency, field color gamut coverage, and field color non-uniformity to evaluate the performance of VR displays.Based on these metrics, a approach is proposed to optimize the displays (OLED displays and LCDs) for the VR system.For the RGB OLED displays, we consider the trade-offs between TLE, field color gamut coverage, and field color non-uniformity, simultaneously, and provide guidelines for improving the RGB OLED display in an VR system.Regarding the LCD based VR system, the LCD consists of a SLC-IPS mode, directional backlight unit, and a 2D patterned prism film is proposed for minimizing the field color non-uniformity, increasing the total light efficiency, and reducing the vignetting effect, respectively.Our optimization method is proven to be effective for designing next-generation display devices for the VR system.

Fig. 1 .
Fig. 1.Schematic of (a) direct-view display, and (b) VR display system consisting of a display panel, eyepiece, and eye pupil.

Fig. 2 .
Fig. 2. Schematic of (a) field brightness non-uniformity and (b) field color non-uniformity in VR.According to the pixel position (top, center, and down), three circles on the right of the figure represent the image received by the eye, respectively.

Fig. 3 .
Fig. 3. (a) Schematic of Fresnel VR system in LightTools.(b) Simulated angular power collection efficiency of the 28-point source in an VR system.

Fig. 4 .
Fig. 4. Normalized OSE of a display with various radiation pattern in a VR system.

Fig. 7 .
Fig. 7. (a) Simulated field color coordinate (CIE1931) of the green OLED in the VR system.Simulated field color non-uniformity of the green OLED devices: (b) without DBR, (c) with 1 pair DBR, (d) with 2 pairs of DBR, and (e) with 3 pairs of DBR.

Fig. 8 .
Fig. 8.The optimization results of RGB OLED displays: (a) without DBR (b) with 1 pair of DBR, (c) with 2 pairs of DBR, and (d) with 3 pairs of DBR in an VR system.Three objectives are considered simultaneously.(Magenta point: optimal solution)

Fig. 9 .
Fig. 9. (a) The 2D transmittance distribution of SLC-IPS under on-state driving voltage; W = 2.3 µm, and L = 8 µm (b) Simulated voltage-transmittance curves of the SLC-IPS at λ = 550 nm; The 100% transmittance is normalized to the transmittance of two parallel polarizers and 100% aperture area of fast response region.The simulated (c) angular color shift and (d) isocontrast contour of the SLC-IPS with compensation films.Under Lambertian backlight source, (e) the field color non-uniformity and (f) the dark state light leakage ratio of the LCD based VR system.

Fig. 10 .
Fig. 10.(a) Schematic of the LCD based VR system.(b) The angular intensity distribution of backlight unit with BEF (n = 1.4 to n = 1.7).The TLE distribution of each pixel in the LCD based VR system with BEF having various refractive index: (c) n = 1.4,(d) n = 1.5, (e) n = 1.6, and (f) n = 1.7.

Fig. 11 .
Fig. 11.(a) Schematic of the 2D patterned prism.The backlight radiation pattern in local (b) zone 1, (c) zone 2, and (d) zone 3. (e) The TLE distribution of each pixel in an LCD-based VR system with 2D patterned prism.BEF: n = 1.7.