Retinal choroidal vessel imaging based on multi-wavelength fundus imaging with the guidance of optical coherence tomography

: A multispectral fundus camera (MSFC), as a novel noninvasive technology, uses an extensive range of monochromatic light sources that enable the view of diﬀerent sectional planes of the retinal and choroidal structures. However, MSFC imaging involves complex processes aﬀected by various factors, and the recognized theory based on light absorption above the choroid is not suﬃcient. In an attempt to supplement the relevant explanations, in this study, we used optical coherence tomography (OCT), a three-dimensional tomography modality, to analyze MSFC results at the retina and choroid. The swept-source OCT system at 1060 nm wavelength with a 200 kHz A-scan rate and an MSFC with 11 bands at 470 to 845 nm are employed. A quantitative evaluation procedure is proposed to compare MSFC and OCT en face images. The comparative study shows that 1) the MSFC images with the illumination wavelength of less than 605 nm could mainly provide the retinal structure information; 2) Relative choroidal layer thickness information could be inferred from the MSFC images, especially the image acquiring under the wavelength more than 605 nm. According to the results, further investigation revealed the contribution of the perivascular tissue and the sclera scattering in the diﬀerence of vascular brightness in MSFC images.


Introduction
Fundus vessels, known to be associated with a wide range of ophthalmic diseases [1][2][3][4][5], have great significance for being visualized in clinical. Retinal vessels and choroidal vessels are the two main categories of fundus vessels at different axial locations. As a kind of wavelength-resolved fundus camera, MSFC can acquire multi-band fundus images simultaneously or sequentially to provide the structural information from different depths. In addition, it can provide functional information such as blood oxygen. Typically, such depth-resolved capability of the MSFC comes from that each biological tissue type has different optical absorption and reflection coefficients at different wavelengths [6][7][8][9][10]. The wavelength range for imaging is usually among the visible and near-infrared regions [7,11]. With longer illumination wavelengths, more choroidal structures can be revealed using MSFC [12].
MSFC images have been used clinically, but the analysis of choroid via MSFC imaging is not sufficient. Previous studies suggested that the absorption coefficient of the tissue above the choroid, such as the pigment in the retinal pigment epithelium (RPE) layer [11,12], should be the main factors influencing the choroid imaging results in MSFC. However, this mechanism cannot give a clear explanation for the signals from bright vascular structures in MSFC images. Hence, more critical factors, such as optical properties of choroid itself and high scattering coefficients of the sclera, need to be considered, which is important for better understanding and diagnosis of ocular fundus diseases as well as enabling quantitative measurement.
It's worth mentioning that accurate three-dimension spatial analysis can promote a better understanding of MSFC imaging results so that optical coherence tomography (OCT) is a solid choice to investigate fundus structures. As a noninvasive optical imaging modality, OCT performs 3D volumetric imaging of the internal microstructure in biological tissue by measuring the echoes of backscattered light [13]. Benefiting from its extremely high resolution and 3D imaging mode, OCT has been fully used in ophthalmic diagnostics [14][15][16]. Recently, a few newly designed light sources and systems have been proposed to acquire a larger field of view, making it possible to obtain complete fundus vessel images in a short time [17][18][19][20][21]. The deeper imaging capability of OCT brings the potential to explore the explanation of choroid imaging results via MSFC.
To conduct better investigations of retinal, choroidal vessel imaging, an MSFC with longest illumination wavelength of 840 nm, and a custom-built 200 kHz, 1060 nm resolved swept-source OCT system was employed for deep region imaging reaching sclera in this work. The segmented OCT en face images were generated and compared with the MSFC images from different illumination wavelengths using an introduced quantification method.
The contrastive studies between the MSFC and the OCT revealed that for the wavelengths below 605 nm, the penetration would be limited by RPE, and the MSFC images only show the retinal information. On the other hand, MSFC images with wavelengths longer than 605 nm will demonstrate the information on choroidal and sclera. The choroid thickness plays an essential role in the near-infrared fundus images via MSFC, and thicker choroid would induce a blurred choroidal structure in the MSFC images. We also analyzed the differences between choroidal and retinal perivascular tissue to explain the different brightness performance of blood vessels in different imaging modalities.

Material and method
In this paper, eight sets of fundus data were collected from eight healthy volunteers without eye medical history using OCT and MSFC, respectively. In order to compare the two different data sets, OCT images were segmented, and planar intensity projections of each layer were obtained. After the registration, the two types of images were compared by our proposed evaluation method.
The human study protocol was approved by the institutional review boards of the Peking University First Hospital. After explaining the study, all subjects obtained written informed consent. All procedures adhered to the tenets of the Declaration of Helsinki. Eight healthy volunteers with no known history of retinal disease participated in the study.

OCT system and scanning protocol
Swept-source OCT is a typical swept-source system [18]. Briefly, we used a commercially available swept laser (Axsun 1060, Axsun Technologies, Billerica, MA, USA) with a sweep rate of 200 kHz, a center wavelength of 1044 nm, and a tuning range of 104 nm. Based on the system, a typical raster scanning protocol was used to generate 3D images. In the x (fast-scan) direction, 1024 A-lines were acquired to form a B-frame, determining a frame rate of around 200 frames per second (fps); in the y (slow-scan) direction, 512 scanning steps were sequentially acquired. The total acquisition time and the field of view were ∼ 3.6s and 10 × 10 mm (30°× 30°), respectively. The measured power on the cornea was 1.8 mW, which is consistent with safe ocular exposure limits set by the American National Standard for Safe Use of Lasers [22].

Multispectral fundus camera system
A custom-built MSFC imaging system with an angle of view of 45°was employed in this study. The system adopted a set of electronically controlled fast-switching LEDs for illumination. An optical annular fiber bundle was used to deliver the LED light to the system. The cross-sectional area of the fiber bundle showed a donut shape, which formed an annular illumination pattern on the corner of the imaging subject. Twelve representative wavelengths were selected in the system, i.e., 470 nm, 500 nm, 520 nm, 548 nm, 605 nm, 610 nm, 635 nm, 665 nm, 740 nm, 810 nm, 820 nm, and 845 nm, respectively. The unweighted anterior segment visible and infrared radiation irradiance of the system is listed in Table 1 [23]. The maximum optical lateral resolution of the MSFC imaging system is 16µm in the center 30-degree region and 25 µm beyond the central 30-degree region. The depth of field is around 245µm at 845 nm. The spectrum switching and camera exposure are synchronized by an STM32 controller (STM32-F103C8T6, STMicroelectronics, Geneva, Switzerland). The pupil tracking and split-line focus assist system are employed to ensure the focusing accuracy. All twelve band images can be acquired within 0.2 seconds, which makes the system superior to clinical usage.

Image preprocessing
In order to contrast OCT images with MSFC images, the OCT images were segmented into eight layers using a graph cut based method [24]. For each layer, the mean intensity projection map was calculated to obtain the corresponding en face image. For the choroid layer in OCT images, the attenuation compensation method is employed to enhance the contrast of the result [25]. Then, image registration was performed between the MSFC images and en face images to eliminate the effect of micro-saccades. That is, paired feature points (x, y) and (u, v) from two corresponding images were first manually selected using MATLAB's Point Selection Tool. Based on the feature points, affine transformation matrix was estimated between images, which can be denoted as follows: where a i and b i (i ∈ 0, 1, 2) are the parameters of the reflection transformation determined by the least-square method.

Evaluation process
In order to efficiently conduct the contrastive studies, we have developed an evaluation pipeline, which is illustrated in Fig. 1. As shown in the figure, the vessel structure was first extracted using a hessian filter based algorithm [26]. By parameter modification, the hessian filter is able to distinguish bright and dark tubular structures. In fact, either structure could be contained in OCT en face images of different layers and MSFC images of different wavelengths, respectively. Hence, it is necessary to adapt the parameters of the filter to different scenarios. Next, the area of interest (around 6×6 mm in the center of the registered images) was selected in the respective group of images being evaluated. Considering that non-rigid distortions caused by the ocular motion along the OCT slow scan direction still exist in the registered images, the images were resized (using bicubic interpolation) to a size of 30×30 pixels to reduce the effect of image distortion on evaluation. Finally, a Pearson correlation coefficient (R) between the images was calculated. R is defined as: where δ yz is the covariance between two input images y and z. R represents the degree of correlation between the two images, which is between 0 and 1. A larger R-value reflects a higher relationship.

Fig. 1.
Overview of the evaluation pipeline. The images have been registered in preprocessing. Hessian filters with different parameters are used to separate the dark and light vessel structures, respectively. Afterwards, down-sampling is adopted to reduce non-rigid distortions, and a Pearson correlation coefficient (R) between the images is calculated for the quantitative evaluation.

Fundus imaging results from MSFC and OCT
Generally speaking, longer wavelengths have better penetration capabilities of biological tissue. Within the wavelength range selected in this article (470∼845 nm), MSFC imaging also obeys this rule and shows different tissue in different band images. The MSFC images from a normal eye are shown in Fig. 2. It can be seen that retinal structure is clear with few choroidal information in the MSFC images with short wavelengths less than 605 nm. For the MSFC images with long wavelengths of more than 605 nm, the choroidal structure is much clear, whereas the retinal structure becomes blurred. Figure 2(a)-2(d) (wavelength band centers of 470 nm, 500 nm, 520 nm, and 548 nm, respectively) show significant visual similarity. Figure 2(m) presents the respective R-value calculated between the 470 nm MSFC image and the other band images. It can be seen that R-values tend to decrease as the wavelength increases. Although the R-value is increasing again for wavelengths longer than 740 nm, it still remains a low level. We would recommend paying attention to both the R-value and its standard deviation. It should be noted for the wavelength increase above 740 nm, the R-value is increasing again, but the standard deviation remains large, which means data are unstable. In this study, only the R-value of more than 0.75 with a standard deviation of less than 0.070 could be considered as well correlated results. Note that, the R values are higher than 0.8 for the wavelengths below 605 nm, demonstrating high similarity between the 470 nm MSFC image and these band images. However, Fig. 2(n) poses a high contrast with those below 605 nm, in which the R values calculated with the pictures of longer wavelengths, indicating a broader distribution range that infers various tubular structures being displayed in each image. We speculate that different penetration depths of wavelengths may lead to this result. Different from the cross-sectional B-scan image, en face OCT image provides similar sight to the traditional 2D imaging modality. The en face OCT images from different layers of the same eye in Fig. 2 are shown in Fig. 3. Despite vessel shadowing artifacts exit in some OCT en face images, it can be noticed that the information from the deep layers differs from the images of surficial layers. From Fig. 3(a) and Fig. 3(h), it can be seen that the choroid vessels were shown up as dark signal regions in the images [19]. Recent findings suggest that the fringe washout and the weak scattering of near-infrared light of the blood may affect the weak OCT signals in choroid vessels [27,28]. The sclera layer is also displayed in Fig. 3(i), and the vasculature was marked with arrows. These structures were bright in B-scan [ Fig. 3(a) marked with arrows].

Comparison on retinal vessels
For the comparison of the two different modalities, the images were first registered with the MSFC image as the baseline image. Figure 4(a),4(b) shows the averaged en face OCT images of the retina (above RPE layer) and RPE layer, respectively. Meanwhile, Fig. 4(c) presents the 470 nm MSFC image. The correlation coefficients R between the retina OCT en face images, and the 470 nm MSFC image is over 0.8. However, it can be seen that the vessels in the en face image of the retina show higher intensity than the other regions. On the contrary, the vessels in the 470 nm MSFC image show lower intensity than the other non-vessel regions, and a similar phenomenon can also be observed in the en face image of the RPE layer in Fig. 4(b). Therefore, it can be considered that the reflected signals from the RPE layer contribute a lot to the MSFC imaging results, as vessels show up as dark in MSFC images but display as bright in en face images from layers above the RPE. Since blood has a higher absorption coefficient than the prevascular tissue, the vessels present lower signals in both 470 nm MSFC images and en face images of the RPE layer. In Fig. 4(d), higher R-values can be observed from layers above the RPE. It's because the layers above the RPE show the ideal vessel signals, and the signals of RPE are from the shadow of the vessel. However, only a little difference exits between images of layers above the RPE and RPE layer, which may occur due to the complex composition of the returning signal in MSFC. The signals from the vascular surface, hemoglobin, and other layers (including RPE) can contribute to the intensity of the vascular signal in MSFC imaging [29][30][31][32].

Comparison of choroidal vessels
Contrastively studies between en face OCT images below the RPE layer and long-wavelength MSFC images on the same eye were conducted to evaluate the imaging results on choroid.   Fig. 5(a)] [33]. As shown in the enlarged images of selected regions (dashed boxes) in Fig. 5(a) and Fig. 5(b), most structures of the choroidal blood vessels shown in the MSFC image [ Fig. 5(b)] can match the en face OCT image [ Fig. 5(a)]. Note that vessel structures are blurred on the right side of Fig. 5(b) and Fig. 5(c). One primary reason for such a phenomenon is that the imaging regions have the thicker choroidal layer, which can be verified by the choroidal thickness map presented in Fig. 5(d). It can be found that the choroidal layer is thicker in those regions, resulting in blurred choroidal vessel structure in the images. Figure 6 shows the results from another eye with a thicker choroid layer. As presented in Fig. 6(d)  In order to verify the association of retinal thickness with MSFC imaging, the choroidal thickness maps were overlaid with the OCT en face image of the choroidal layer and the 840 nm MSFC image, respectively. Three regions were respectively divided in Fig. 7(a) and Fig. 7(b) based on the thickness of the choroid, and pixels in the different areas are listed separately. Subsequently, the correlation coefficients R of the different regions were calculated between the choroidal en face image and the MSFC image. In this work, three different thickness levels were 0∼150 µm, 150∼220 µm and >220 µm, respectively. At present, the division of the thickness map is based on the experience value and mainly depends on image observation. Statistical results show that the results from different regions indeed present very different correlation coefficients [ Fig. 7(c)]. The correlation value R is large for thin choroidal layers regions and is small for thick regions. Generally, the correlation value R tends to decrease as the choroidal thickness increases.

Discussion
Based on the proposed contrastive studies, we can conclude that the imaging capability of the 840 nm MSFC on choroidal vessel structures is correlated to choroidal thickness. The vessel structure is clearer in the thin choroidal regions and is blurred in thick regions. We believe that sclera, as a tissue with the highest fundus scattering capacity [ Fig. 8(a)], plays an important role in explaining such a phenomenon. In thin regions, light is more likely to reach sclera, resulting in more scattered light signals [ Fig. 8(b)] of the sclera returning to the detector [marked in Fig. 5(e) by the dashed box]. The intensity of these scattered signals from the sclera will be different, owing to the optical properties of the tissue above the sclera. As shown in Fig. 5(c) and Fig. 5(e), the OCT signal below the choroid vessels is stronger in the thin choroidal areas than in the thick choroidal areas. The choroid contains a large amount of melanin, which has higher absorbing and scattering coefficients than blood vessels [3,34,35]. Because the choroidal vessels themselves do not have a strong scattering signal in OCT B-scan [ Fig. 5(e), Fig. 6(e)] and the weak attenuation of blood vessels, it is convincible that the scattered signals from sclera transmitted through choroidal vessels are much stronger than the reflection signals of the vessels [ Fig. 8(b)]. Additionally, the perivascular tissue of the choroid has strong attenuation. Building on these findings, we speculate that the majority of the choroidal vessel imaging in MSFC is not from the light reflection on the vessels but the scattered light from sclera transmitted through the vessels. Therefore, it is reasonable that the choroidal vessels were shown as bright structures in the long-wavelength image of MSFC.
In order to estimate the image similarity, we resized the image to a small size, although details of images will be lost. The method is inspired by the Perceptual hash algorithm (PHA) [37], the difference is that we use Pearson correlation to quantify the relevance of images in the space domain, obtaining results that are suitable for statistical analysis. There are two reasons that we resize the image to a small size. 1) We are interested in the correlation among the different regions instead of a specific pixel. 2) To reduce the effect of the image distortion. The distortion in both MSFC imaging and OCT imaging is mainly caused by eye motion during the imaging acquisition and related to the optical design. The effectiveness of resizing operation in reducing image distortion is presented in Fig. 9. Two affine transforms are applied to a typical enhanced  Table 2, the R-values of different size images are listed. From the table, we can find that the R-values between the full-scale images has been greatly affected (0.63 for translation transform and 0.53 for affine transform). By resizing the image to a small size, R-values increases. In general, the method in the manuscript is justified. But, resizing the image to a small size can result in loss of detail and false-positive results. So, the accurate distortion analysis, cross-modal non-rigid registration, faster imaging should be applied in the future. To the best of our knowledge, the obtained fundus images are usually blurred when the fundus camera illuminated with relative long-wavelength (NIR region). This could be seen from typical fundus camera preview images, which usually uses ∼800 nm wavelength. Such a phenomenon could also be seen from the fundus camera with even longer wavelength (see Refs. [11,12]). This blurriness may be due to the following factors. 1) The longer wavelength intrinsically has a lower resolution.
2) The longer wavelength can penetrate more in-depth so that the light will experience many more multiple scattering effects. 3) Fundus images are based on the reflection/scattering signal from all the depth that the light could penetrate. They are generally a summation of information of all depth. The MSFC we used is a fundus camera with pupil tracking and a split-line focus assist system to ensure the focusing accuracy. There are many methods, such as local frequency signal and picture sharpness evaluation, which can quantify the "blurry" level. But in this study, our chief concern is the correlation between the OCT en face images, OCT thickness map, and MSFC images. For this purpose, we employed the Pearson correlation. "Blurry" level is important for fundus imaging, and we do believe that certain retina pathologies, such as retina edema and leakage, may affect the "blurry level" of the vessel structure. So its further impact should be included in our future studies.

Conclusion
Multispectral fundus camera and OCT have been widely used in ophthalmic imaging. It can be expected that with the improvement of system speed, the MSFC functional imaging and large 3D-field SS-OCT imaging will play more important and crucial roles in improving early diagnosis, treatment, social and medical rehabilitation. In this work, a quantification method was employed to compare the imaging characteristics of the two modalities on the blood vessel structure, and the signal sources of the vascular features in MSFC images are analyzed. From the comparison results, we can obtain three significant conclusions: 1) For the retinal vessels above RPE layers, the OCT en face images match well with the short-wavelength (< 605 nm) MSFC images and present a correlation coefficient higher than 0.7, which suggests that the MSFC images with the illumination wavelength of less than 605 nm could mainly provide the retinal structure information. On the other hand, for MSFC images with wavelengths longer than 605 nm, a clear difference exists between the retina and choroidal structure, and more choroidal information will be displayed with the increase of wavelengths.
2) Relative choroidal layer thickness information could be inferred from the MSFC images, especially the image acquiring under the wavelength of more than 605 nm. The areas with a thick choroid show a blurred choroidal structure. In contrast, thinner choroidal regions present a clear structure.
3) The difference in scattering properties between choroidal vessels and retinal vessels were also investigated. In OCT en face images, due to differences in perivascular tissue, it can be concluded that the retinal vessels show a higher signal than the non-vessel regions. In comparison, the choroidal vessels show a lower signal than the non-vessel regions. In the MSFC images, the retinal vessels show a lower signal than the non-vessel regions, while the choroidal vessels show a higher signal than the non-vessel regions.
On the other hand, this study also has several limitations. We resized the image to a small size to eliminate distortion. The loss of detail has, to some extent, affected the result. So, the accurate distortion analysis, cross-modal non-rigid registration, faster imaging should be applied in the future. Our analysis suggested that scleral scattering has a considerable effect on imaging, but proving experiments were not included in this work. As a result, additional phantom experiments and simulations should be studied in the future. On the other hand, considering that choroid thickness is related to axial length, position, age, and myopia [38][39][40], more imaging data and imaging methods could be introduced to verify the hypothesis further. Finally, clinical imaging studies of retinal disease patients and age-matched healthy people are required to assess the ultimate clinical value of choroidal vessels.