Pupil steering holographic display for pre-operative vision screening of cataracts

: Cataract is the most common cause of preventable blindness and vision loss where the only treatment is surgical replacement of the natural lens with an intraocular lens. Computer-generated holography (CGH) enables to control phase, size, and shape of the light beam entering through the eye-pupil. We developed a holographic vision simulator to assess visual acuity for patients to experience the postoperative corrected vision before going through surgery. A holographically shaped light beam is directed onto the retina using small non-cataractous regions of the lens with the help of a pupil tracker. A Snellen chart hologram is shown to subjects at desired depth with myopia and hyperopia correction. Tests with 13 patients demonstrated substantial improvements in visual acuity and the simulator results are consistent with the post-operative vision tests. Holographic simulator overperforms the existing vision simulators, which are limited to static pinhole exit pupils and incapable of correcting aberrations.


Introduction
Cataract is the world's most common cause of reversible blindness and could lead to the complete loss of vision if untreated [1]. As people age, the proteins in the lenses, that generally form a transparent lattice start to unfold and agglomerate in clusters, developing a cloudy area that blocks, scatters, and distorts light while passing through the lens. Although aging is the natural cause of cataracts, chronic exposure to UV light can also trigger oxidative stress in the proteins that compose the crystalline lens. Currently, more than 52 million people suffer from vision loss or blindness caused by cataracts [2]. The earlier symptoms are quite unnoticeable, especially if both eyes are affected. Continuous degradation of vision evolves gradually until the symptoms start to manifest, including blurry vision, glare, halos, fading color, and night blindness, depending on the type of the disease. There are several types of cataracts, including nuclear, cortical, posterior subcapsular, and radiation [3]. Nuclear cataract forms in the center of the lens, often darkening as it spreads, leading to hazy, blurry, or yellowed vision. Cortical cataract, on the other hand, begins from the outer edges of the lens and spreads as wedge-like opacities growing centripetally, forming an appearance of 'spokes on a wheel. Posterior subcapsular cataracts form faster and are more likely to occur in younger people when proteins settle at the back of the lens and begin to form a hazy area. This involves a thin layer of clouding and protein clumps affecting the back surface of the lens. Radiation cataract forms after exposure to radiation, either in a radioactive area or after long-term exposure to UV radiation. Neither can be corrected with prescription glasses and eventually lead to blindness if untreated.
The degree of cataract is quantified by the Lens Opacities Classification System III (LOCS III), according to which the amount of cataract is defined and graded upon its nuclear color (NC), nuclear opalescence (NO) cortical (C) and posterior subcapsular (P) components [4]. The VA of a healthy person is considered 20/20, where the first value indicates the distance (in feet) for which a standard person can see, while the second value is the distance for which the subject can see the same image. The World Health Organization (WHO) defines the stages of visual impairments as follows: Mild (<20/40), moderate (<20/60), severe (<20/200), blind (<20/400) [5].
Surgery is the only definitive treatment of cataracts. During surgery, the patient's cataractous lens is replaced with an artificial intraocular lens (IOL) [6]. It is crucial to match the patient to the most suitable IOL since patients lose their ability to accommodate after the surgery due to the rigid structure of the IOL [7]. A multiple choice of IOL types is available for patient satisfaction, including monofocal IOLs for spherical refractive correction, bifocal or trifocal IOLs for presbyopia correction, both with or without toric component to correct astigmatism. Trifocal IOLs are the most demanding as they offer good vision at three different focal distances simultaneously and eliminate the use of eyeglasses. Postoperative clinical tests such as visual acuity and contrast sensitivity are used to assess surgical success; this evaluation is further extended with additional tests that include distant corrected intermediate and near visual acuity, and through-focus visual acuity to assess the performance of an implanted bifocal or trifocal IOL. However, preoperative prediction of potential visual acuity following surgery remains a challenge yet to overcome in clinical practice. Although numerous clinical methods and devices such as pinhole testing [8] and Potential Acuity Meter [9] had been previously developed, none of them had yet been proved to provide satisfactory results in predicting postoperative visual acuity [10][11][12]. Therefore, a system that provides objective measurements for patients' post-operative condition after IOL replacement will be appealing and lead to better understanding of the problem hence the treatment.
Augmented reality (AR) is a rapidly developing technology that enhances the real-world environment by overlaying digital information on the objects or locations [13]. The advances in AR combined with head-mounted displays (HMDs) and computer-generated holography (CGH) offer unique and disruptive solutions that can be used for diagnosis, treatment, and follow-up of several eye diseases, especially presbyopia, glaucoma, and age-related macular degeneration [14]. On the other hand, cataract has not yet been sufficiently benefited by AR technology. There are studies that simulate how patients with cataract perceive their surroundings via virtual reality devices [15]; however, diagnosis or treatment is limited to adaptive optics only [16][17][18]. While off-the-shelf hardware is preferred widely among researchers, including AR-HMDs, some researchers prefer to develop their optical structure for better customization and adaptation. These displays can render the augmented image on top of real objects without covering the eyes with an opaque screen, keeping the user's natural field of view intact and their eyes unblocked [14]. These displays integrate 3D virtual objects into the 3D physical environment, which would allow for new visual enhancement possibilities for diagnosis, treatment, and visual aid for several ophthalmic diseases. Especially, near-eye displays offer portable replacements for bulky, table-top designs to provide ease of use and increase the accessibility of medical instruments throughout a wider region [19].
Herein, we introduce a custom-made holographic vision simulator device for patients with mild or severe cataracts to estimate postoperative visual acuity before surgery. The proposed instrument is a wearable near-eye display integrated with a real-time pupil tracker and transmits a holographic image (Snellen chart) through the patient's pupil, as illustrated in Fig. 1. Our novel eye box steering method allows us to scan over the pupil and find the non-cataractous regions, if any, on the crystalline lens. The adjustable depth of holographic displays allows us to accommodate the virtual image to the desired depth to correct the refractive error of the patient, i.e., myopia or hyperopia. We first describe the concept of eye-box steering and refractive error correction and how they apply to the ophthalmology setting. We then report patient examination results and how the results are verified with conventional imaging methods.

Concept of eye-box steering
The ability to control the exit pupil location is an essential feature of holographic displays [20]. Exit pupil or eye-box steering is a technique that is commonly used by holographic near-eye display architectures to overcome the fundamental trade between the field-of-view and eye-box size. Therefore, there are techniques to enlarge the eye box by changing the focus using a dynamic mirror [21] or by creating multiple focal spots using a holographic optical element [22]. All the focus steering methods require either mechanical motion or fixed positions of the multiple focal spots. On the other hand, CGH algorithms allow us to produce the effect of various optical components on the display system by computationally embedding their wave shaping properties in the objective wave calculation. Therefore, CGH remains as the only technology that provides complete computational control over the exit-pupil (eye-box) size, location, and shape.
The full complex hologram can be calculated using the Fresnel Space Propagation [23] and a phase-only hologram can be obtained using iterative Fourier transform algorithms [24]. Our optical configuration is illustrated in Fig. 2(a), where the exit pupil corresponding to the generated hologram can be placed between the 0 th and the 1 st diffraction orders. The horizontal and vertical distances of the exit pupil in 2D from the 0 th order location of each color can be determined by solving the grating equation: where d x = d y = 5.25mm, a is the pixel pitch, m x and m y are the diffraction order number in the horizontal and vertical axis, λ is the wavelength, and f is the focal length of the lens in Fig. 2(a). For placing the center of the hologram beam between the 0th and 1st diffraction orders, m x and m y can be any number between 0 and 1. For our display system, f is approximately 50mm, and the pixel pitch of our phase-only spatial light modulator (SLM) is 4.5µm. for m x = m y = 1, the maximum value of d x = d y = 5.25mm. While using the Iterative Fourier Transform Algorithm for phase hologram computation, we allocated about 5% of the area to phase noise; therefore, the available region for the eye-box placement is ∼5-by-5mm.
In order to steer the eye-box anywhere in the square area between the 0th and 1st diffraction orders in the 1st quadrant, we added a linear grating phase term in x and y-axis with periods proportional to m x and m y . A different value of m has to be calculated for the horizontal and vertical axis and for each color. For blue wavelength (473nm), the exit pupil location corresponding to m x = m y = 0.5 is shown in Fig. 2(b). We divided the area into 5-by-5 subregions to form 25 different exit pupils as shown in Fig. 2(c), where each exit pupil carries exactly the same scene information. Individual eye-boxes separated with 0.2 mm. While multiple exit pupil locations can be activated simultaneously, we activated only one of the exit pupils at a time and For different wavelengths, the required m value for pupil steering differs. We have calibrated our experimental setup with respect to blue wavelength since it is the lowest wavelength and defines our maximum steerable area. The desired diffracted order is in between m=0 and m=1 diffraction order for blue wavelength. The corresponding order for green and red are adjusted accordingly. I.e., If desired order is at m=0.5 diffraction order for blue, it is m=∼0.4 for green and m=∼0.3 for red. The grating pattern period is adjusted accordingly for each color hologram to keep the wavelength to grating-period ratio constant.
Note that, when multiple exit pupils are activated simultaneously, one can also embed different holograms containing different parallax of the scene into each exit pupil. While such an approach can improve depth perception, it is not trivial as one has to also control the coherent interferences between different exit pupil beams at the retina creating image artifacts.

Refractive error correction
The prevalence of refractive problems among patients raises difficulties to profile the clear spots on the cataractous lens. It is imperative to eliminate the effect of the first-order aberrations caused by the crystalline lens shape since these problems prevent our instrument from providing a sharp virtual image for the patient [25]. The most common aberrations among refractive errors are hyperopia (farsightedness), myopia (nearsightedness) and astigmatism.
Hyperopia and myopia are conditions that cause an image of an object to become unfocused on the retina. Myopia is a condition in which, opposite of hyperopia, an image of a distant object becomes focused in front of the retina [26]. These refractive errors can be corrected with various prescription glasses or contact lenses specifically designed to counteract their effects. Nearsightedness (myopia) is corrected using a concave lens which is placed in front of a myopic eye, moving the image back to the retina and making it clearer. On the other hand, long-sightedness (hyperopia) is corrected using a convex lens, which is placed in front of a hypermetropic eye, moving the image forward and focusing it correctly on the retina. In other words, the focal planes of the optical mediums are adjusted according to patients' measured diopter values of the refractive problems.
Our elimination procedure for hyperopia and myopia can be described as a graphical replication of the effect of the prescribed eyeglasses. The near point of a human eye is the shortest object distance that a healthy eye can accommodate or to image onto the retina. Depending on the given age, the near point has a range from 20 cm to 67 cm for 40 and 55 years of age, which could be estimated using the Hofstetter formula [27]. As described in the virtual scene creation section under methods, the nearest depth plane is described as 25 cm for an ideal case, whereas the digital version of the Snellen chart is located at the farthest depth plane, which is located at 4 m. Depending on the refractive problem, these virtual depth planes are adjusted during our plane discretization procedure of CGH calculation, as illustrated in Fig. 3. Depth values that are retrieved from a rendering software are adjusted to replicate the patient's prescribed eyeglasses effect on these planes. Since we use a pinhole-based display principle, the effect of refractive errors become smaller as the aperture of the imaging system increases. However, the HVA improved when the refractive correction was added to the hologram computation. For those patients with dense cataracts, their refractive error cannot be measured. Therefore, we iterated the refractive correction until the best HVA was obtained. We can subjectively assert that, those patients who required larger than 2D correction, holographic refractive error correction helped improve the VA.

CGH calculation
CGHs are diffractive-optical elements that offer the possibility of creating wave-optical display systems that are complete computer control [24]. Our CGH calculation involves four major steps: content generation, focal plane discretization, object wave computation, and 3D image reconstruction. The desired virtual content is formed, rendered perspective frames are discretized into multiple focal planes with respect to their depth map values. Once the optical properties of the system are defined, the next step computes the object wave of the scene planes with respect to Fresnel Space Propagation. The complex-valued objects wave that are calculated to represent the 3D scene. The CGH system generally uses three methods for encoding: the amplitude holograms where the amplitude of the reference wave that is modulated, phase holograms which modulate its phase and complex holograms where both amplitude and phase are modulated. In order to display the computed holograms on our phase-only SLM, complex valued hologram frames are phase mapped using Iterative Fourier Transform Algorithm (IFTA) [14]. As a final step, once the encoded CGH has been acquired to reproduce the 3D image of the scene, it can be displayed on a beam shaping device that is explained in the optical setup section.

Optical setup
The head-worn vision simulator, as it is shown in Fig. 4 and Fig. 5, has a compact form due to its simple optical architecture. The architecture for one eye module of the head-worn device consists of a point light source, a phase-only SLM, optical components, exit pupil plane and pupil tracking cameras and a computation unit. The simulator utilizes a point light source that illuminates the SLM. The operating wavelengths of the point light source are 473 nm for blue, 532 nm for green and 632 nm for red color. Spatially coherent diverging beam that is generated by a point source is collimated with a lens before illuminating the SLM. The light gets modulated via SLM. The resolution of the SLM is FHD (1920 × 1080 pixels) and the pixel pitch is 4.5 microns. SLM provides phase-modulation with a frame rate of 180fps. SLM does not have a polarizer attached to it. The input light is polarized but an analyzer is not needed after the SLM. The propagated light rays reach the eye (pupil plane) once they are reflected from the beam splitter. This optical architecture provides the correct ray angles from virtual objects that are encoded in the CGH frames. The modulated waves that are reflected via beam splitter propagate and enter the eye pupil to form the retinal image of the virtual object.

Pupil tracking
The pupil tracking cameras are placed in front of the beam splitter to align the patient pupil with the exit pupil of the holographic head-worn unit. The pupil tracking system contains two individual camera units: one visible range, one infrared (IR) range with IR LEDs. IR LEDs that operate at 850 nm illuminate the pupil while the pupil tracking algorithm determines the location and the center of the pupil from the frames captured via IR camera. The visible ranged camera tracks diffraction orders that are formed on the pupil plane. Exit pupil (Eye-box) location is estimated with respect to the 0 th and 1 st order diffraction locations.

Virtual scene
We have constructed a 3D scene via Game Engine, Blender 2.79. The virtual scene is formed by two depth planes which provide the perspective and depth map data as raw rendered frames to the CGH algorithm. To comply with the real eye examination scenario, a graphically constructed Snellen chart is used for far-plane content whereas the near-plane only contains the surrounding box of the Snellen chart. In this way, the patient experiences an illusion of the real eye examination scenario. Far plane and near plane are formed at 400cm and 25cm respectively, as illustrated in Fig. 3.

Cataract screening
The proposed vision simulator instrument uses computer-generated holography to display the virtual content, and it is binocular and head-worn, as illustrated in Fig. 3. The wearable feature of the instrument eliminates the discomfort for the patients during the examination, and its pinhole imaging feature and eye-box steering method provide novel capabilities that are not possible with existing devices. The instrument also incorporates a pupil tracker and automatic algorithmic adjustments using patient's diagnostics data, reducing the measurement times, and making the technology more accessible for patients. Once the device is aligned and the position is calibrated with the aid of the pupil tracker cameras and inter-pupillary distance adjustments, multiple small regions on the pupil were dynamically addressed, through which virtual images are sent to the retina, granting a unique ability to make use of non-cataractous parts of the lens efficiently. Our holographic vision system forms the exit pupil plane on the patient's pupil. The overlapping region of the exit pupil plane and the patient's pupil is divided into 25 small regions (5-by-5 matrix). Each of these small regions is iteratively addressed to form the corresponding eye-box locations. For each grid, a visual acuity test is performed by asking patients to read the Snellen charts that are digitally provided to them.
The visual acuity tests that are performed through the clear regions that the vision simulator acquired, is an indication of the maximum visual acuity level that can be achieved after the cataract surgery.

Verification of results with Scheimpflug Cataract Densitometer
The results of the virtual eye examination yield a density map of the crystalline lens with a 5-by-5 resolution. Besides the fact that the patient is able to see the content clearly, the consistency of this density map is verified by comparing the results with that of Scheimpflug cataract densitometer (Sirius, CSO, Italy). Scheimpflug densitometry is one of the most common and reliable instrument to examine the density profile of the lens [28]. As the density of the crystalline lens varies on the pupil plane, spots with higher density indicate denser cloudy regions caused by the cataractous lens. Figure 6 depicts the densitometry results of healthy and cataractous eyes. The cloudy regions caused by cataracts can be differentiated when compared to the healthy eye in Fig. 6(a). The black circular spot at the center of densitometer images is generated by the device in all images and does not provide any information on lens features. However, the other hypodense (dark regions) areas correspond to transparent zones of the lens whereas more hyperdense (gray/white) regions correspond to zones with lens opacification. In Fig. 6(b)-(c) the areas marked with red arrow are the transparent areas of the lens that coincide with the zones through which holographic visual acuity results of Table 1 were obtained. We select these spots as pinholes, through which we send the holographic stimulus focused on the retina.

Participant recruitment
13 subjects (ages 42-81), all recruited from the patients of Ophthalmology Clinic, Koç University Hospital, and 10 healthy subjects (ages 18-60), all recruited from personnel at Koç University, signed informed consent and participated in the current study. All healthy subjects had selfreported normal vision, and their visual acuity was verified with both a Landolt C test and ETDRS chart prior to the experiments. Our control group consisted of healthy individuals and had excellent vision (Average BCVA was logMAR equivalent of "0" each having logMAR equivalent of "0"). Their BCVA and HVA were consistent. As this group did not have surgery no change from baseline was observed. The study was approved by the Koç University Ethics Committee on human subject research and conducted according to the institutional guidelines, following the tenets of the Declaration of Helsinki.
Patients who had been scheduled for unilateral or bilateral cataract surgery were recruited and compared with a group of healthy subjects for the study. The enrolled patients underwent full ophthalmologic examination including Best Corrected Visual Acuity (BCVA) from 4 meters, slit lamp bio microscopy, applanation tonometry and dilated fundus examination. Patients who had other ocular conditions including corneal opacities, glaucoma, ocular inflammation, diabetic retinopathy, optic nerve abnormalities, retinal detachment and history of previous ocular surgery were excluded. Intraocular lens (IOL) power prior cataract surgery was calculated with optical low coherence reflectometry (Lenstar LS 900, Haag-Streit, Germany). A holographic vision system based visual acuity testing (HVA) was performed, as described below. All surgeries were performed by two experienced ophthalmic surgeons (M.H. and A.S.). Main corneal incision was made with a 2.2 mm blade through the steep meridian. A 5.5 mm anterior capsular opening was created with continuous curvilinear capsulorhexis. Following phacoemulsification procedure (Centurion Vision System, Alcon Inc., Fort Worth, TX, USA), the Tecnis ICB00 monofocal intraocular lens (Johnson & Johnson Surgical Vision, Inc.) was implanted into the capsular bag in all patients. Moxifloxacin 0.5% and dexamethasone 0.1% eye drops were used 5 times per day Table 1. OD is the right eye, and OS is the left eye. BCVA is the best-corrected visual acuity, HVA is holographic visual acuity with refractive error correction and PVA is the post operative visual acuity. *: refractive error immeasurable due to severity of cataract. **: refractive notation includes spherical power (D) -negative cylindrical power (D) × axis of cylinder (degrees)

Discussion
The results of the trials on patients suggest that the proposed vision simulator can be used as a potential acuity meter so that the patients can experience corrected vision. Besides a persuasive and assistive decision factor for the patients and the ophthalmologists towards cataract surgery, the simulator can be used to detect those who will not benefit from IOL replacement and provide a more guarded indication for surgery for cases who would likely present postoperative complications due to other possible ocular and systemic conditions. This study reveals that low visual acuity performances of the patients with cataracts may not be the effect of the cloudiness of crystalline lenses only. Most of the patients that are suffering from cataracts tend to experience additional eye-related problems, whether on the cornea or retina since cataracts often develop in older age [29]. Since the cataractous lens does not allow to visualize the posterior segment of the eye, it is unlikely to detect these diseases accurately. Having lens-related aberrations results in a challenging situation where the ophthalmologist has to predict these major diseases. These patients perform poorly on the HVA test; therefore, the low visual acuity in the virtual eye examination indicates the existence of a retinal impairment. The proposed simulator enables clinicians to measure the best possible visual acuity that can be obtained after the crystalline lens is replaced. However, artifacts due to multifocal IOLs or any remaining refractive errors after the surgery cannot be predicted with the simulator.
The clinical results demonstrate that all patients with cataracts have performed better in the holographic visual acuity test compared to the conventional test, as shown in Fig. 7. However, it can be seen that not every patient can experience perfect vision with the simulator. The reason for this is the difference in the type and severity of cataracts of each patient. Depending on the level of the disease, the less-affected regions in the lens may still be opaque, or there can be no clarity in the lens at all. Even in these cases, the pinhole imaging method provides the patient a better vision.  Table 1.
Some of the patients in the test group (patients 12 and 13) are scheduled for IOL replacement to fix their refractive errors instead of cataracts. In these cases, the vision simulator cannot overperform their current visual acuity. The reason for this is that the advantage of the holographic simulator is the pinhole imaging technique which is beneficial when most of the crystalline lens is clouded. When the patient suffers from refractive errors with a clear lens, the simulator does not make a major difference. For patients with cataracts, although some of the subjects' BCVA results have been dramatically increased in the HVA test. Patient 1 and Patient 2 had dense cataracts and essentially had no vision on their cataractous eyes (BCVA > 1.00 logMAR). Our simulator still predicted VA quite well despite the dense cataracts. On the other hand, some of them do not experience the same level of improvement due to the nature of their cataract. For instance, our test favors the patients with cortical cataractous lenses due to the inhomogeneous density distribution of the crystalline lens. Whereas the nuclear cataract remains as a more homogenous solution provides additional scattering issues while the imaging of the hologram plane to the retina.
As previously mentioned, other retinal and corneal disorders such as AMD (patient 4), glaucoma, or keratoconus introduce additional challenges during the imaging process which cannot be overcome with pinhole imaging or aberration corrections. A major setback in HVA tests is that patients with almost no vision have difficulties fixating their gaze to the hologram, even with the help of the pupil trackers. This is usually the case with AMD patients, for they are accustomed to seeing with their peripheral vision so that the center of their pupil is directed downwards.
Previous efforts for preoperative prediction of visual performance are mainly focused on two examination methods: Potential Acuity Meter (PAM), first introduced by Minkowski et al. in 1983, is an instrument which projects a visual acuity chart through a narrow beam of light using a small section of the pupil [9]. The alignment of the light beam is adjusted by the doctor during the examination, requires strong patient cooperation, performed when the pupil is dilated, and the examination could take long for patients.
Potential Acuity Pinhole (PAP), introduced by Melki et al. in 1999, is a modification of the traditional single point 1.2 mm pinhole examination with enhanced illumination [8]. PAP pinhole is placed several mm in front of the pupil. Therefore, it has a small field-of-view. Patients hold the pinhole in their hand and need to align and stabilize it relative to their pupils with mm precision, which is a difficult task and limits the utility of the measurement. PAM also requires strong user cooperation, and the test duration can also be long while the patient finds the less dense regions of the cataracts and aligns it with the exit pupil of the device [10,12]. Although being among the few reliable methods to assess potential visual acuity, both PAM and PAP had shown limited capacity to predict clinically significant outcomes in patients with moderate to advanced cataract, thus had very few benefits in common clinical practice [11].
Our system operation bears similarity to PAM in its operation principle. On the other hand, our system employs a pupil tracker and can steer the exit pupil to the desired section of the pupil in real-time using CGH algorithms. Furthermore, our system does not require pupil dilation, utilizes coherent light, and is able to show dynamic images and change the refractive correction dynamically based on the pupil position using algorithms.
Furthermore, both PAM and PAP methods rely on presenting a two-dimensional image with increased depth of focus. On the other hand, holographic display technique has potential to project simultaneous binocular views as well as three dimensional holograms that could introduce a simulation of augmented reality involving both real-world environment and the 3D holographic scene.
In summary, we have developed a near-eye display that uses CGH and forms a small exit pupil where the size and the position of the exit pupil can be digitally controlled with algorithms. It allows us to predict the PVA and the improvement compared to pre-op BCVA with good accuracy. Our novel eye-box steering method enables us to scan the pupil of the patient to find a non-cataractous region and then send the hologram through this region onto the retina. In this way, not only the patients with cataracts will have the glimpse of the vision after the surgery, but also the retinal or corneal disorders, if any, will be detected. Although we were able to perform holographic tests on a monocular setting, our device has potential to be upgraded as a binocular testing system, which will further allow performing clinical tests that require binocularity, including depth perception. The future work includes matching the patient to the most suitable intraocular lens by adjustable depth feature of holography and eye-box steering method introduced here. Disclosures. The authors declare no conflicts of interest.
Data availability. Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.