Increased crystalline lens coverage in optical coherence tomography with oblique scanning and volume stitching

: A three-dimensional optical coherence tomography (OCT) crystalline lens imaging method based on oblique scanning and image stitching is presented. The method was designed to increase OCT imaging volume of crystalline lens in vivo . A long-range swept-source (SS)-OCT imaging system, which can measure the entire anterior segment of eye in a single acquisition, is used to acquire one central volume and 4 extra volumes with different angles between optical axis of OCT instrument and the pupillary axis. The volumes are then stitched automatically by developed software. To show its effectiveness and verify its validity, we scanned the subjects before and after pupil dilation drops and compared the experimental results. By determining the number of voxels representing the signal from the crystalline lens in 3-D OCT images, our method can provide around 17% additional volumetric lens coverage compared with a regular imaging procedure. The proposed approach could be used clinically in early diagnosis of cortical cataract. Wider field of view offered by this method may facilitate more accurate lens biometry in its peripheral zones, which potentially contributes to understanding of lens shape modifications of the accommodating eye.


Introduction
Crystalline lens imaging has clinically important applications in the study of eye diseases such as cataracts and presbyopia. In vivo visualization of the crystalline lens can be performed with either non-optical techniques, specifically MRI [1][2][3], or optical methods . Although MRI can provide non-distorted images of the entire lens, its relatively long acquisition time makes it prone to motion artifacts. In addition, low image resolution, high cost and complexity also limit wide applications of this technology in in-vivo volumetric (three-dimensional; 3-D) crystalline lens imaging. The morphology of the crystalline lens can be also obtained with optical techniques like Scheimpflug imaging [3,15,[20][21][22][23] or optical coherence tomography (OCT) [5][6][7][8][9][10][11][12][13][14][16][17][18][19][24][25][26]. In addition, quantitative information on the geometry of lens surfaces can be also extracted from Purkinje images [4,15]. Among all these optical approaches, OCT has an advantage of high image resolution and sufficient imaging depth. The performance of OCT in the crystalline lens imaging has also been demonstrated in the studies with isolated lenses [5][6][7][24][25][26].
Several challenges need to be addressed to effectively image the crystalline lens in vivo. Firstly, the crystalline lens is a highly dynamic element of the optical system of the eye. The geometry of the lens depends on accommodation. Therefore, an ideal lens in vivo imaging method should be able to control the eye accommodation (through a fixation image for the patient to focus on) and to acquire the images fast enough to avoid motion artifacts. Secondly, a proper visualization and morphometry of the crystalline lens require the device to image the structures in front of the crystalline lens (the cornea and the anterior chamber) and the lens thickness itself. As a result, this poses the requirement for imaging capabilities in terms of axial depth range (no less than 10 mm in air). Recent advances in wavelength-tunable light sources and wide-bandwidth acquisition boards significantly improved the imaging depth of laboratory SS-OCT set-ups [8,9,27]. Thirdly, human crystalline lens is characterized by the gradient refractive index, which is the most distinguishable optical property from all ocular structures. Consequently, extraction of quantitative data on posterior lens surface requires implementation of a complex algorithm. Several approaches have been proposed to account for gradient refractive index of the lens [6,7,24]. However, they are mostly applicable to in vitro or ex vivo conditions [6,7,24].
Finally, in vivo imaging of human crystalline lens suffers from a significant shadowing artifact due to the non-transparent iris. The light is strongly reflected from the iris, which limits the imaging area to the pupil area. With the light blocking, current in vivo OCT-based crystalline lens imaging use the central part of the crystalline lens under orthographic projection of the pupil, and the shadowing effect appears in the lens images so that peripheral parts are not visible [8][9][10][11][12][13][14][16][17][18][19].
Visualization of the periphery of the lens would be beneficial in early detection of cataract features as development of cataract (in particular cortical cataract) usually starts in the peripheral parts of the crystalline lens. Fundamental ophthalmic research would also take advantage of the imaging of lens periphery. Specifically, accommodative processes in the eye involve changes in the shape of the crystalline lens. Access to structural information from a larger field of view could help in developing more precise models of the eye behavior during accommodation.
The examples given above show that the missing volume disturbs clinical observation and diagnosis although some parameters of crystalline lens can be estimated from current OCT images [10]. Specific method for enlargement of the imaging volume is required to alleviate this disturbance. With the help of the oblique OCT scan [7], more crystalline lens can be imaged in a certain direction but it also results in more missing volume in the other direction. Consequently, a direct solution is to stitch OCT volumes obtained by central and selected oblique scans. Although OCT volume stitching methods have been proposed [28][29][30][31][32], yet they are not tailored for in vivo crystalline lens imaging.
In this study, we present a new approach enabling increased coverage of the crystalline lens in volumetric in vivo OCT imaging. The technique is based on the sequential acquisition of volumetric data sets: one centrally scanned volume and four obliquely scanned volumes using a long-range anterior segment OCT. The volumes are stitched together by the developed software-based method. We assess the effectiveness of the method and validate the proposed approach. In addition, the extra imaging volume is shown by 3-D volume rendering and determined quantitatively.

Methods
The proposed method for extension of scanned volume of the crystalline lens is based on acquisition of 5 OCT volumes, i.e. one central volume and 4 extra volumes with different angles between OCT illumination direction (optical axis of the instrument) and the normal to the pupil (pupillary axis). The scanning system is a long-range SS-OCT instrument enabling imaging of the entire anterior segment of the eye. Each volumetric OCT data set is resized with an aspect ratio corresponding to both lateral and axial voxel dimensions, and an image registration algorithm is implemented. Later on, a wide-angle OCT volumetric image of the crystalline volume is obtained by applying maximum fusion algorithm. Descriptions of the OCT system and the image stitching procedure which includes image registration and fusion are presented in the following sections.

SS-OCT system and data acquisition
A custom long-range SS-OCT imaging system operating at the speed of 50 kA-scans/s was used in this study. The system, shown schematically in Fig. 1(a), utilizes a swept-source laser (Axsun Technologies, Billerica, MA, USA) with a central wavelength of 1050 nm and a wavelength sweep range of 100 nm. The details of the system are described in [14,19]. The sample arm consisted of the collimating lens (diameter D = 8 mm, focal length f = 20 mm) and the objective lens (D = 2 inch, f = 150 mm) that focused the beam on the anterior segment. The numerical aperture of the fiber (ca. 0.14) determined lateral resolution (43 µm), and limited back-scattered light collection efficiency. The axial resolution of the system was 8 µm. The throughput of the single-pass reference arm with the retroreflector was ca. 70-80%, and the light from reference arm was controlled by the iris to avoid saturating the balanced photodetector (PDB). The sensitivity of the system was measured to be 103 dB. Spectral fringes were acquired with an acquisition card, and 8192 samples were obtained for each sweep. Before wavenumber domain linearization and Fourier transformation, 5000 samples from central part of the sweep were cropped to minimize the laser noise. Therefore, each A-scan contained 2500 pixels in depth. For a single acquisition, a transverse area of 16 mm × 16 mm was scanned with 300 × 300 A-lines raster scan pattern, which took 1.9 seconds. As shown in Fig. 1(b), we took 5 OCT volumes including one central volume and 4 extra volumes with different orientation of illumination angle. This was provided by tilting the eye with respect to the OCT system interface due to the shift in the position of the eye fixation target (cross) towards nasal, temporal, superior and inferior pole. In this study, the angle between the optical axis of the instrument and pupillary axis was chosen to be 20 degrees. 1% Tropicamide was administered to dilate the pupil and paralyze accommodation. If the pupil area was covered by the eyelid, the data set was discarded.

Automated image stitching procedure
The automated image stitching procedure was developed in this study. The algorithm shown in Fig. 2 includes four steps: (1) pupil alignment; (2) rough registration with deflection angle and scaling; (3) fine registration using B-spline registration; and (4) fusion of volumes. In the first step, we align the pupil in centrally and obliquely scanned volumes to correct crystalline lens location mismatch that arise from the differences of the incident angles and the differences of corneal ray refraction. During this alignment, the front surfaces of the crystalline lens are rigidly registered. In the second step, we simplify the optical distortions differences to deflection angle and scale for each B-scan along oblique direction, and then use the deflection angle and the scale to optimize image similarity for oblique volume registration. Later on, we use three-dimensional (3-D) B-spline registration to correct residual registration artifacts. In the final step, we fuse all volumes by taking the maximum value for each voxel.

Pupil alignment
The entire algorithm to extract pupil shape from en-face image is shown in Fig. 3. As the first step of pupil alignment, we segment the pupil in the en-face projection OCT image. To generate en face projection image from volumetric data set, the maximum value in axial direction is taken for each A-scan. Then, this en-face projection image is segmented by thresholding. Afterwards, we fill the small holes in the image, which are caused by higher scattering, using morphological image processing. After filling the holes in the image, the pupil area is defined as the largest connected area whose eccentricity is lower than 0.7 which is set empirically.
With the pupil area in en-face plane, 2-D en-face pupil edge (contour) is determined by Canny method, which searches local maxima in the gradient of binary image of pupil area [33]. Then, we can identify the pupil contour in 3-D volume by searching the location of high OCT intensity value in each A-scan corresponding to the pupil contour (Fig. 4). In detail, high OCT intensity values, which are higher than E + 3 × σ (E is the mean intensity value of the A-line and σ is the corresponding standard deviation value), are selected, and the bottom voxel location of these high OCT intensity values are treated as pupil edge to exclude the cornea scattering. In this way, the 3-D pupil edge is obtained. Afterwards, it is possible to determine the pupil plane by fitting the 3-D pupil edge with plane equation. To make the algorithm more robust with some error pupil edge points, which can be caused by the eyelashes etc., the pupil plane is fitted by RANSAC (Random Sample Consensus) method [34]. In RANSAC method, the pupil edge points with fitting error lower than the threshold are considered as supporting data, and the other points are considered as error data. In the RANSAC, we randomly choose 20 points to get plane equation by least-squares method and iterate 1000 times for searching the equation that contains maximum supporting points as the result. Thus, by using this method, the pupil plane is determined, and error data are excluded. The result of fitting pupil plane is shown in Fig. 5. After excluding the error data, we calculate the pupil center in en-face plane by fitting en-face pupil edge using ellipse. With the pupil center in en-face plane and the pupil plane equation, the pupil center is determined. Using the normal of pupil plane and center of pupil, we align all the volumes together and only reserve crystalline lens in the volume.

Volume registration with deflection angle and scale
Due to the refraction properties of the cornea and the crystalline lens, the optical distortions between centrally and obliquely scanned volumes are different, which causes image mismatch in the overlapping parts of volumes. The image misalignment can be minimized by registering obliquely scanned volumes to the centrally scanned volume using deflection angle and scale. Obliquely scanned volumes are registered to central volume one by one independently. As an approximation after applying the previous step of pupil alignment, we assume that the cross-sectional images along oblique direction (from central and oblique volume) show the same cross-section of real crystalline lens structure. Thus, to register one oblique volume, we create B-scans, which have different direction with real OCT B-scans in 3D OCT images, along oblique direction for both centrally and obliquely scanned volume, and then register each B-scan from oblique volume to corresponding B-scan from central volume. To register one B-scans pair, we approximately assume that all A-lines in the registering B-scan (from obliquely scanned volumes) are refracted by same deflection angle and set this angle as one degree of freedom for registration to the central volume (Fig. 6). In addition to the deflection angle, the scale along the beam is also set as one degree of freedom to register one B-scans pair. Thus, for each B-scans pair, we have two degrees of freedom for registration. To determine deflection angle and scale, we maximize Structural Similarity Index (SSIM) [35] between registering B-scan and reference B-scan using traversal method. For saving computation time and maintain enough accuracy of registration, we firstly register the center B-scan (across pupil center) with a large optimization searching range (whose scale is restricted from 0.9 to 1.1, and deflection angle is from 0°to 10°), and then the other B-scans are registered one by one from center to edge with very small optimization range around the optimum value from the one next to them from the center.
To show the effectiveness of this registration, we compare results before and after registration using red-green composite images from centrally and obliquely scanned volumes. As we can see in Fig. 7, there is a misalignment of posterior lens surface in the cross-sectional image before registration (pupil aligned only), and this misalignment is mostly corrected during this registration process.

B-spline volume registration
Although previous registration corrects most of mismatch between centrally and obliquely scanned volumes, some residual mismatch still remains due to complexity of optical distortion. To correct these residual mismatches, we use 3-D B-spline registration. In this 3-D B-spline registration, all obliquely scanned volumes are registered to the centrally scanned volume one by one independently (sequentially). The sum of squared intensity differences between reference volume and registering volume is minimized to optimize the image similarity.

Fusion of volumes
The highest scattering in the OCT images of the crystalline lens occurs in the regions close to the optical axis of OCT instrument. Therefore, each volumetric image has different regions characterized by high signal (intensity). Consequently, we fuse centrally and obliquely scanned volumes by taking maximum intensity value from all the OCT volumes for each voxel location. To confirm this, we color five volumes (center, left, right, up and down) with five colors. Considering color distance, linear separation, and color category [36], the five colors are selected as soft red (center), strong yellow (up), moderate cyan (down), soft blue (left) and soft magenta (right). Cross-sectional images (for both horizontal and vertical directions) in Fig. 8 and (B)-scan fly-through in Visualization 1 illustrate the colored fused volume created from the centrally and obliquely scanned volumes by taking voxels, which are from the volume with high scattering ratio direction approaching to optical axis.

Computational time
We measured the computation time of all steps of our algorithm for the acquired data sets. The pupil alignment typically takes about 52 seconds for a single volume. The deflection and scale correction for one volume typically takes about 9.5 minutes. The B-spline registration typically takes 15.1 minutes for one volume. Total computation for our case (center, left, right, up, down) is about 103 minutes. In the future, we can use parallel computation to process all the volumes at the same time. We anticipate that the whole automated image stitching procedure will be completed within 26 minutes in total. The test was performed on a PC with AMD Ryzen 5 2600 Six-Core CPU processor @ 3.40GHz and 32.0 GB RAM using MATLAB (MathWorks, Natick, MA, USA).

Effectiveness and validity of our method
To evaluate the effectiveness and validity of our method, 3 right eyes from 3 subjects, who are males of 28, 31 and 46 years old, measured by our method with central, left and right volumes before and after mydriasis (1% Tropicamide) using the largest angle we can achieve. Thus, we generated 6 wide-angle imaging volumes in total. During data acquisition, the subjects focused to the far.
Our stitching method was successful all the 6 volumes although some tiny image mismatches exist. To show the effectiveness of our method, we present two typical volumes from the same subject before and after taking drops by fly-through cross-sectional movies (Fig. 9, Visualization 2 and Visualization 3). In these typical volumes, we also determined an additional lens volume coverage as the result of stitching. We calculated the number of voxels representing the lens in OCT images as the percentage relative to central volume at baseline (before mydriasis) ( Table 1). Both pupil dilation and our technique enabled improvement of volumetric lens imaging. Although change in pupil diameter helps in imaging larger portion of the lens, our method contributes also to a more comprehensive visualization of the lens. It was estimated that even 85% larger volumetric lens coverage could be obtained with the proposed method compared to the central scan of the non-dilated eye. We also compared the stitched volume before taking drops and central volume after taking drops, which is regarded as wide-field ground truth volume, by creating green-magenta composite cross-sectional movies ( Fig. 10 and Visualization 4).
As shown in Fig. 10, most areas of composite cross-sectional images are matched well by the evidence of white appearance in overlapping areas. Although peripheral of crystalline lens images rich in green, it is due to high scattering power of structures whose high scattering ratio direction approaching to optical axis. To confirm this, we present gradated in-and-out movie for central cross-sectional image ( Fig. 11 and Visualization 5). As we can see in gradated in-and-out movie, the structures in stitched and central volumes are matched well.

Wide-angle imaging results
To show the performance of our method, we present a wide-angle volume (fused by center, left, right, up and down volume image) by fly-through cross-sectional movies (for both horizontal and vertical directions). As we can see in the two fly-through cross-sectional movies, the edge is smooth although some motion artifacts exist ( Fig. 12 and Visualization 6). The 3-D rendering image shows the center imaging volume area (using gray color map) and the extra imaging volume area (using blue-green color map) (Fig. 13). As a result of manual image segmentation and corresponding voxels counting for crystalline lens in 3D OCT images, the extra imaging volume is ca. 17% of the crystalline lens volume sampled with centrally scanned 3-D image.

Discussion
We presented an image-stitching-based in vivo OCT imaging of the crystalline lens, which is aimed at extending the imaging volume coverage. This method has potential to show more pathological volume of diseased crystalline lens in vivo, which is very important for clinical observation and study of eye disease like cataract. Our results confirm that the in vivo OCT imaging volume of crystalline lens can be extended by stitching central scanned volume and oblique scanned volumes.
To further enlarge the imaging volume by our volume stitching method, we provide two possible solutions. One is taking more OCT volumes towards different orientations. For example, the oblique volumes towards top-left, top-right, bottom-left and bottom-right can be added. The other one is taking a series of OCT volumes towards the same orientation with increments of oblique angle which is to ensure that the maximum oblique angle is large and there are no gaps in stitched volumes. These two solutions need take more measurements, attention should be paid to preventing large eye motion due to eye strain and avoiding the artifacts such as anterior segment shadowing by the eyelid.
Although obliquely scanned volumes are registered to the centrally scanned volume well, our method has two limitations that might be overcome by combining the current algorithm with other correction methods. Firstly, the fan and optical distortions are not corrected perfectly. This issue is important if one wants to extract biometric information on the shape of crystalline lens surfaces, e.g. in study on accommodation. Secondly, the OCT volume can suffer from motion artifacts but implementing motion artifact removal algorithm prior to stitching should allow for addressing this issue. If the two limitations can be solved, the wide-angle imaging OCT volume has potential to estimate crystalline parameters with a better accuracy and provide a wider topography than only central OCT volume is used.
Our current method needs centrally and obliquely scanned volumes to image the center and the periphery of the crystalline lens due to cuboid shape of scanning volume. To scan wide-angle volume with a single OCT volume, one option is changing scanning pattern from a cubic volume to a pyramidal volume by replacing pre-objective scanning with the post-objective scanning [27].
The limitation of the study is also associated with fact that the method is computationally intensive. The anticipated time of current method using parallel computation to process all the volumes at the same time is 26 minutes in total. The time can be significantly reduced by using a more computationally powerful computer and applying proper C programing and graphics processing unit (GPU) acceleration.
The proposed approach of increasing the crystalline lens coverage can pave the way for an earlier detection of cataractous lesions as well as for a better lens biometry. The latter application may potentially contribute to understanding of lens shape modifications of accommodating eye. Additionally, some intraocular lens (IOL) power estimation methods have emerged in the last years based on geometrical parameters that describe the complete shape of the crystalline lens [37,38]. A potential application of our method could be the improvement of these IOL power estimation approaches, that clearly would benefit from an extended view of the crystalline lens. Finally, the concept of imaging the object at different angles (angular compounding) could be used to reduce speckle noise in OCT images, thus facilitating an enhanced image quality [39,40].

Conclusion
We demonstrated a method accounting for enabling 3-D in vivo OCT imaging of the crystalline lens with an extended coverage. The anterior segment of eye is imaged by the acquisition of 5 (center, left, right, up and down) OCT volumes. By confirming validity of our image stitching based method, we showed that this method can provide 3-D OCT images of the crystalline lens in vivo with increased volumetric coverage. As a result of the manual image segmentation, the extra imaging volume are 16.78% of original imaging volume. The developed approach can facilitate more comprehensive analysis of the anatomy of the crystalline lens.
Disclosures. The authors declare that there are no conflicts of interest related to this article.