Introduction

Sickle cell disease is the most common genetically inherited haematologic disorder, with a birth prevalence of over 300,000 new cases per year worldwide [1]. The most common causes of vision loss in patients with sickle cell disease stem from complications of proliferative sickle cell retinopathy (PSR), a condition in which chronic peripheral retinal microvascular occlusion and ischaemia stimulate the proliferation of sea fan neovascularization, which can cause vision-threatening vitreous haemorrhage or tractional retinal detachment [2, 3]. School- and working-age patients are disproportionately susceptible to these vision-threatening changes; by the age of 26 years, up to 43% of patients with Haemoglobin SC disease and 14% of patients with Haemoglobin SS disease have been found to develop PSR [4].

To enable early detection and possible prophylactic treatment with scatter laser photocoagulation prior to development of vision loss, consensus guidelines published in 2014 by an expert panel on multi-organ management of sickle cell disease included a strong recommendation to commence routine screening with dilated fundus examinations at age 10 years [5]. However, the globally low rates of adherence to these guidelines, as revealed by publications from Toronto [6], Jamaica [3], and Saudi Arabia [7], underscore the need for new approaches to increasing access and adherence to screening retinal examinations for sickle cell retinopathy (SCR). Moreover, a recent survey of practicing retina specialists demonstrated variable practice patterns for treating vision-threatening PSR, highlighting a need to build evidence-based treatment guidelines through existing data and clinical trials [8]. There is rapidly growing literature on the use of artificial intelligence to aid automated classification of retinal photographs for a variety of retinal conditions. The recent FDA approval of the artificial-intelligence-enabled IDx-DR [9] and EyeArt systems [10, 11] for screening for referrable diabetic retinopathy from fundus photographs has paved the way for application of similar systems to augment screening for other retinal diseases, such as SCR.

This review aims to summarize published literature on and highlight future research directions for the possible coupling of artificial intelligence and multimodal retinal imaging to enhance the diagnosis and management of SCR. For the purposes of this review, artificial intelligence refers to the broad field that includes the subsets of machine learning, whereby machines learn from data without human supervision, and deep learning, the subset of machine learning that employs multi-layered neural networks that mimic human cognitive processing without human supervision [12].

Search methodology

We evaluated the current literature on artificial intelligence and sickle cell retinopathy by searching PubMed and Google for studies published in English up to 30 December 2020, using keywords such as ‘deep learning’, ‘machine learning’, ‘artificial intelligence’ and ‘sickle cell retinopathy’. We also used reference lists and performed similar searches using specific search terms such as ‘fluorescein angiography’, ‘optical coherence tomography’ and ‘optical coherence tomography angiography’ to find applications of artificial intelligence to these types of retinal imaging in other retinal diseases.

Results

Artificial intelligence for screening for sickle cell retinopathy from ultra-widefield fundus photographs

One of the most important potential applications of artificial intelligence to SCR lies in the use of automated interpretation of ultra-widefield fundus photographs (UWF-FPs) to enhance access and adherence to screening for the disease. The combination of automated artificial-intelligence-based algorithms with point-of-care fundus photography, which could be obtained at primary care or haematology office visits as has been previously demonstrated for diabetic retinopathy screening [9], may help consolidate care for visually asymptomatic patients with sickle cell disease who have difficulty juggling routine ophthalmology visits with school or work obligations and medical appointments or hospitalizations associated with their systemic disease burden [13]. Moreover, the convenience and scalability of automated screening systems have the potential to help overcome disparities in access to retinal screening examinations in Africa and other medically underserved regions of the world, where the prevalence of sickle cell disease far exceeds the availability of retinal specialists [14].

UWF-FPs are particularly amenable to artificial intelligence-based detection of SCR, given their ability to image the retinal periphery in a single frame and established advantages for SCR staging. For example, several retrospective and prospective studies have reported that graders of UWF-FPs more often detect nonproliferative SCR than clinicians performing dilated fundus examinations on the same patients [15,16,17]. Although the current standard of care for SCR screening relies on expert clinician interpretation of dilated fundus examinations and ancillary retinal imaging, well-validated artificial intelligence algorithms may offer unique advantages in terms of increased speed, accessibility and accuracy of image classification compared to human interpretation [18, 19].

Toward the goal of using artificial intelligence to automate screening for vision-threatening SCR from retinal photographs, our group recently developed a deep learning convolutional neural network that achieved 97.4% sensitivity and 97.0% sensitivity compared to retinal specialists in classifying Optos (Optos plc, Dunfermline, UK) UWF-FPs from patients with sickle cell haemoglobinopathy for presence or absence of sea fan neovascularization [20]. We trained, validated and tested this convolutional neural network using 1182 images from 190 adult patients with sickle cell haemoglobinopathy imaged at the Wilmer Eye Institute, of whom 53% were women, 94% were of Black American or African descent, 63% had Haemoglobin SS disease, 24% had Haemoglobin SC disease, and 30% had sea fan neovascularization in one or both eyes captured on Optos UWF-FP. Because some patients had multiple imaging encounters and most patient eyes had more than one image per eye included in this study, the performance of the deep learning algorithm was also assessed after clustering images from the same eyes and imaging encounters, yielding a sensitivity of 100% and specificity of 94.3% in detecting sea fan neovascularization from ‘eye encounters’.

The promising performance of this first application of deep learning to the automated detection of PSR from fundus photographs sets the stage for further work validating and refining our algorithm for real-world screening for SCR. A few limitations merit follow-up study. First, a common limitation to many deep learning algorithms is their “black box” nature, meaning that there is not a direct means for determining how the algorithms generated their final output classifications from the input images [19, 21,22,23]. Without transparency into the features of the input images that determine the final outputs, a concern is that potentially erroneous confounding features could affect the performance of the deep learning algorithm. For example, if a particular type of imaging artefact is often associated with sea fan neovascularization in a certain dataset, it is possible that a deep learning algorithm could learn to recognize the imaging artefact rather than sea fans themselves when classifying fundus photographs for presence or absence of PSR. To address this concern, a variety of saliency map visualization techniques have been developed to generate colorized heatmaps that visually highlight pixels of the input image that may be most important for the convolutional neural network output [24]. In our study, we used two different saliency map visualization techniques, called Guided Grad-CAM [25] and SmoothGrad [26], to assess which parts of the input UWF-FPs, when changed, most affected the final convolutional neural network output classifications of presence or absence of sea fan neovascularization [20]. On average, both saliency map types highlighted the relative importance of the peripheral retina, and especially temporal peripheral retina, for the convolutional neural network output classifications, which is consistent with the clinically observed predilection for sea fan neovascularization to arise in the temporal retinal periphery (Fig. 1) [2]. While this correspondence between the saliency map findings and clinical experience is encouraging, it is important to keep in mind the absence of standardized methods for reporting and comparing visualization methods for explaining how deep learning algorithms translate inputs into outputs.

Fig. 1: Representative Optos ultra-widefield fundus photograph and corresponding saliency maps for deep learning detection of sea fan neovascularization.
figure 1

The Optos image (left) is from a patient with Haemoglobin SC-associated proliferative sickle cell retinopathy. The color-coded Guided Grad-CAM (centre) and SmoothGrad (right) saliency maps highlight in red the regions of the input fundus photograph (including the temporal retina and in particular the two sea fan neovascularization complexes) most relevant to our convolutional neural network [20] for correctly classifying this fundus photograph as having sea fan neovascularization.

Full size image

A second challenge for interpreting the performance of a deep learning model for classifying UWF-FPs for presence or absence of referable SCR lies in the need to assess whether UWF-FPs can reliably image with high resolution the areas of peripheral retina where sea fan neovascularization is most likely to occur. Peripheral retinal visualization can be limited on Optos UWF-FPs by eyelid artefact, peripheral image blurring and peripheral distortion [27]. In our study, 10% of included patients had at least one Optos colour UWF-FP graded as not having sea fan neovascularization but also a corresponding UWF-FA revealing an area of leakage not well captured on the UWF-FP due to image artefact, poor image resolution and/or image decentration [20]. Since repeated imaging from a single session, particularly with different centration, can increase the likelihood of adequately capturing peripheral retinal pathology, further research is needed to determine the best protocol for repeated images to optimize the sensitivity of detecting referable SCR. With larger multicentre databases of UWF-FPs, it should be possible to use machine learning algorithms to automate classification of UWF-FPs as being of sufficient or insufficient quality for grading of SCR. To the best of our knowledge, deep learning has previously been applied to the problems of automated retinal image quality classification for posterior pole fundus [28, 29] and Retcam images [30], but an analogous automated system for ultra-widefield images awaits development.

A third area important to assess before applying a deep learning algorithm for detection of PSR to real-world settings is its generalizability to different ethnic populations, patient ages and imaging systems. The patient population included in our study [20] was of predominantly Black American or African descent, consistent with the population with the highest rate of sickle cell disease worldwide [1]. Further research is needed to assess whether different fundus pigmentation patterns in patient populations from different ethnic backgrounds may differentially affect the sensitivity and specificity of machine learning models for detecting SCR from fundus photographs. In addition, given the recommendations to begin screening for SCR in childhood, it is important to assess the performance of deep learning models for sickle cell screening in the paediatric population. In particular, paediatric UWF-FPs are especially susceptible to the challenges of eyelid artefacts and image decentration, although a single-institutional prospective study of children aged 3 to 17 years suggested that paediatric UWF-FPs can capture an average of 50% greater retinal area than indirect ophthalmoscopic examination [31]. Finally, although the literature on ultra-widefield imaging in SCR has been dominated to date by Optos imaging, more research is needed to assess the potential value of the more recently developed Clarus (CLARUS 500, Carl Zeiss Meditec AG, Jena, Germany) ultra-widefield imaging system for imaging pathologic changes in SCR [27]. Relative advantages of the Clarus compared to Optos are true-colour imaging and reduced eyelid or eyelash artefact, but relative disadvantages of the Clarus are a brighter light flash and a smaller field of capture of 133° from individual images – although montaging increases the overall field of view [27]. One study reported that Optos images tend to capture retinal vascular detail in the superotemporal quadrant better than montaged Clarus images, which in turn may better capture retinal vascular detail in the inferonasal quadrant [32]. The implications of these findings for imaging of SCR (which preferentially affects the temporal retina) merit further study.

Going forward, machine learning methods may be particularly powerful for analysing longitudinal progression of pathologic vascular findings from UWF-FPs of eyes with SCR. In retinopathy of prematurity, deep learning has been used to develop a quantitative retinopathy of prematurity vascular severity score from Retcam fundus photographs that has been shown to correlate with disease severity, progression and post-treatment regression [33,34,35]. In diabetic retinopathy, deep learning has been used to show that baseline 7-field fundus photographs can predict future 2-step worsening on the Early Treatment Diabetic Retinopathy Severity Scale at 12-month follow-up with 91% sensitivity and 65% specificity [36]. With adequate long-term data correlating UWF-FPs with stage and need for treatment for vision-threatening PSR, it will be valuable to explore what machine learning may be able to add to the clinically established Goldberg [2] and Penman [37] sickle cell staging systems for predicting which patients are most at risk of losing vision from SCR.

Artificial intelligence for analysing ultra-widefield fluorescein angiography to improve sickle cell retinopathy staging

Although fluorescein angiography (FA) is invasive and more time- and resource-intensive than fundus photography, and thus less practical for large-scale SCR screening purposes, FA is the current gold standard for staging and monitoring of SCR [27]. Application of machine learning methods to large numbers of FAs collected from patients with sickle cell disease may offer new insights into retinal vascular features useful for monitoring and predicting SCR progression. A better understanding of features that may predict the rate of progression of SCR could in turn aid the development of future evidence-based consensus guidelines about recommended frequency of monitoring of patients with advanced nonproliferative or early proliferative SCR [8].

Thus far, deep learning has been used to augment the creation of retinal vessel masks from FA images, which can subsequently be used to aid automated computation of quantitative metrics such as vessel length, vessel area, ischaemic index (the percentage of total visualized area with nonperfusion) and geodesic index (the shortest distance from the optic nerve centre to the edge of the peripheral vasculature) in ultra-widefield FAs (UWF-FAs) from patients with sickle cell disease [38]. In a recent study by Sevgi et al. of 74 eyes from 45 patients with sickle cell haemoglobinopathy with two imaging visits spaced at least 3 months (and a mean of 23.0 ± 15.1) months apart, the mean ischaemic index was shown to increase and mean vessel area and geodesic index were seen to decrease over time [38]. A nonsignificant trend toward a more rapid increase in ischaemic index over time was observed among patients with Haemoglobin SC than in the Haemoglobin SS or other Haemoglobin S variant cohorts. The authors noted that deep learning algorithms were superior to alternative image processing techniques in capturing detailed pathologic changes of SCR in the skeletonized retinal vessel masks (Fig. 2). In the future, larger longitudinal studies may be useful for correlating the rates of change of quantitative markers of ischaemia with the risk of development of sea fan neovascularization and vision-threatening complications of PSR.

Fig. 2: Representative ultra-widefield fluorescein angiogram (left) and deep learning-enhanced vessel segmentation mask (right) [38] capturing detailed vascular abnormalities in a patient with Haemoglobin SS-associated sickle cell retinopathy.
figure 2

(Image courtesy of Duriye Damla Sevgi, MD, and Justis P. Ehlers, MD, the Tony and Leona Campane Centre for Excellence in Image-Guided Surgery and Advanced Imaging Research, Cole Eye Institute, Cleveland Clinic, Cleveland, OH, USA).

Full size image

Given the value of FA in identifying leakage as a marker of PSR, future work aided by machine learning could also help quantify patterns of progression as well as identify pathologic vascular precursors to leakage on FA. Using manual grading methods, Barbosa et al. previously used ImageJ to retrospectively assess extent and intensity of leakage in patients with PSR treated with scatter laser photocoagulation [39]. Automated methods could improve the speed and reproducibility with which leakage can be quantified serially on FAs from eyes with SCR, with or without treatment. For the related but distinct condition of diabetic retinopathy, Ehlers et al. developed an automated algorithm for segmenting retinal vessels and then detecting and quantifying leakage from Optos FA images [40]. This algorithm merits formal validation in images from patients with SCR. A deep learning algorithm that Sevgi et al. [41] have recently presented for automated selection of early and late phase FA images with maximum visualized retinal vessel area and similar captured fields of view between the early and late phases may also prove very useful in longitudinal analysis of FAs from eyes with sickle cell disease, aiding discovery of machine-learning-assisted methods for staging and quantifying vascular abnormalities in SCR.

Artificial intelligence for using optical coherence tomography angiography to improve sickle cell retinopathy screening and staging

In contrast to fundus images, which only highlight the major retinal vessels, or FA, which is relatively time-consuming and invasive, OCT angiography (OCTA) allows for relatively rapid, noninvasive imaging of retinal vasculature with capillary-level resolution as well as the ability to separately analyse different retinal layers. The emergence of widefield swept-source OCTA (SS-OCTA) with montages of scans taken with different visual fixation points has shown promise for replacing or complementing UWF-FA in the staging of diabetic retinopathy, including identification of areas of neovascularization [42,43,44,45,46,47]. Compared to spectral-domain OCTA systems, SS-OCTA uses a tuneable light source with a longer wavelength and more advanced sensor, enabling high-speed acquisition of images with a wider field of view and greater depth of tissue penetration [48]. To date, there have been limited reports using SS-OCTA in SCR, and these studies have focused on macular rather than peripheral findings [49, 50]. Because neovascularization in PSR tends to develop more peripherally than neovascularization in proliferative diabetic retinopathy, further work is needed to assess how widefield SS-OCTA compares to UWF-FA in capturing peripheral pathology in patients with sickle cell disease (Fig. 3). If widefield SS-OCTA can be optimized for detection of the peripheral changes of SCR, the combination of artificial intelligence with widefield SS-OCTA and UWF-FP could be a powerful noninvasive screening tool in the future for PSR.

Fig. 3: Widefield montage swept-source optical coherence tomography angiography from patient with Haemoglobin SC-associated proliferative sickle cell retinopathy.
figure 3

A montage of five 12 × 12 mm swept-source optical coherence tomography angiography scans enables a non-invasive, non-contact widefield view (>50 degrees), including details of temporal peripheral vascular pathology beyond the macula. Areas of decreased flow signal inferiorly are due to blockage from overlying vitreous haemorrhage. (Image courtesy of Ian C. Han, MD).

Full size image

Meanwhile, machine learning is already being used to identify and classify eyes with SCR from pathologic changes visible on OCTA imaging of the macula. Alam et al. have previously taken the approach of identifying optimal combinations of quantitative parameters extracted from macular OCTA images that can distinguish between eyes with and without sickle cell-related vascular changes and also distinguish different severities of SCR [51]. Numerous studies have highlighted such changes as decreased superficial and deep capillary plexus vessel density [52], greater deep capillary plexus foveal avascular zone area [50] and greater foveal avascular zone acircularity [53] in eyes with sickle cell disease than controls. Alam et al. found that application of machine learning methods to a combination of 6 OCTA quantitative parameters (blood vessel tortuosity, blood vessel diameter, vessel perimeter index, foveal avascular zone area, foveal avascular zone contour irregularity and parafoveal avascular density) – previously confirmed by the same authors to significantly differ between eyes with SCR and control eyes [54] – was found to yield superior performance than any individual OCTA parameter alone [51]. Their support vector machine classifier achieved 100% sensitivity and 100% specificity for distinguishing between 6×6 mm spectral-domain macular OCTAs from eyes with SCR and healthy controls [51]. Importantly, this classifier also achieved 97% sensitivity and 95% specificity in distinguishing between eyes with Goldberg stage II and Goldberg stage III SCR as staged by retinal specialists from dilated fundus examinations, suggesting a correlation between macular and peripheral vascular changes in SCR. A prospective study by our group demonstrating a significant correlation between macular OCTA vessel density measurements (both in the deep capillary plexus and temporal superficial capillary plexus) and peripheral nonperfusion as quantified by the ischaemic index on UWF-FA further supports the idea that macular OCTA may be a viable proxy for peripheral OCTA in identifying eyes at risk of vision loss from PSR [55].

Whereas Alam et al. used machine learning to classify SCR using preselected quantitative metrics derived from OCTA images, an alternative approach is to use deep learning to classify SCR based on the OCTA images themselves rather than extracted metrics. For example, in age-related macular degeneration, a convolutional neural network has been shown to achieve 100% sensitivity and 95% specificity compared to a retinal specialist grader in automatically classifying OCTAs as having or not having choroidal neovascularization [56]. Similar convolutional neural network algorithms could be used in the future to classify diagnostically or prognostically relevant OCTA findings in eyes with SCR. Given the many potential artefacts that can hinder OCTA interpretation [57], it will be important in future work to build on previously developed deep learning models for automation of vessel segmentation [58,59,60,61] and classification of OCTA image quality [62].

Future opportunities for applications of artificial intelligence to optical coherence tomography of sickle cell retinopathy

In addition to the application of artificial intelligence to the imaging modalities discussed above, there is great potential for automated analysis of optical coherence tomography (OCT) scans of patients with SCR. Multiple structural OCT changes have been observed in SCR, such as foveal splaying, inner and/or outer retinal thinning and choroidal thinning [63,64,65,66,67,68,69]. Not only may these structural changes have visual consequences in terms of decreased retinal sensitivity on microperimetry [70] and impaired contrast and colour vision [71], but a previously reported association between macular retinal thinning on OCT and the ischaemic index on UWF-FA highlights a potential correlation between macular structural and peripheral vascular pathology [72]. The ability to capture high-quality registered volume scans and the availability of longitudinal OCT imaging and normative data compared to more nascent technology such as OCTA make automated analysis of OCT particularly appealing for following microstructural changes in SCR over time. A recent longitudinal prospective study reported higher rates of macular OCT thinning over time in patients with SCR compared to age- and race-matched controls [73]. We anticipate that machine learning can be used to automate identification and tracking of OCT-based biomarkers of SCR progression, drawing on the rapidly growing literature using deep learning to diagnose and localize pathologic findings from OCT such as diabetic macular oedema [74,75,76], age-related macular degeneration [77], serous retinal detachments [78], cavitations in macular telangiectasia [79] and epiretinal membranes [80].

Conclusions

The integration of artificial intelligence with multimodal imaging holds great promise for improvement of multiple facets of the diagnosis and management of SCR – from screening for PSR to automating detection and quantification of pathologic vascular changes that may help predict risk of future vision loss and need for treatment. While considerable literature on artificial intelligence in ophthalmology has been devoted to the classification of conditions using individual imaging methods, some studies have also highlighted an opportunity to improve performance by simultaneously incorporating multiple types of imaging [81, 82]. Also, it is important to keep in mind that patients with sickle cell disease may have comorbid ocular conditions such as diabetic retinopathy and glaucoma that may be detectable from retinal imaging and merit referral to an ophthalmologist [83]. Future work is needed to determine how best to integrate simultaneous screening for multiple ocular diseases from multimodal imaging.

In regions of the world where access to retinal specialists is limited, strategic deployment of artificial intelligence may allow automated interpretation of imaging obtained at non-ophthalmology medical offices or community screening centres to help identify patients with sickle cell disease most in need of referral to retinal specialists for further evaluation and management of their SCR. When access to or availability of retinal specialists is not the limiting factor, the combination of artificial intelligence and point-of-care imaging can still help to reduce the burden of medical visits for patients with sickle cell disease, improving adherence to routine screening for SCR. In children, where ultra-widefield imaging may provide superior peripheral retinal visualization to dilated fundus examinations limited by patient cooperation [31], retinal imaging is an especially attractive screening approach. Given the high cost of currently commercially available imaging devices, cost-effectiveness analyses and technological advancements toward portable and less expensive imaging options will be critical to realizing the global public health potential of artificial intelligence for screening purposes.

Equally important to the potential benefits of artificial intelligence for reducing the dependence on in-person retinal specialist visits for SCR screening are the opportunities for artificial intelligence to help inform clinical practice in the care of patients with SCR. Global migration patterns are changing the population distribution of SCR, such that ophthalmic providers with limited prior exposure to SCR may find automated imaging interpretation algorithms to be a particularly useful aid for enhancing recognition of clinically relevant vascular changes. In addition, the currently used Goldberg and Penman sickle cell classification systems were developed through astute clinical observations and based on fundus findings and non-widefield FA from relatively small subsets of patients decades ago [2, 37]. While expert clinicians have the ability to make broad inferences based on small data, artificial intelligence-based approaches have distinct advantages when learning from large amounts of data [84]. By harnessing the power of modern computational methods and retinal imaging techniques, it may be possible to develop more quantitative and nuanced staging systems for SCR, which can be potentially combined with additional systemic clinical data to assess personalized risks to individual patients of future vision loss from PSR. Due to frequent asymptomatic auto-infarction of sea fan neovascularization [85], criteria for timing and methodology of treatment for PSR are still controversial, guided by a single randomized trial of scatter photocoagulation from three decades ago [86]. Artificial intelligence may improve our ability to study clinical and imaging data to identify best-practice patterns for managing SCR.

One important prerequisite for the development of robust artificial intelligence, and especially deep learning, methods for classification of SCR is to have adequately large datasets that are well-labelled with a reliable gold standard [84]. Current clinical practice patterns are highly variable with respect to the frequency and types of imaging obtained as part of routine screening or follow-up of SCR [8]. Our hope is that greater awareness of the diagnostic and prognostic utility of multimodal imaging in SCR, and also of the opportunities for using deep learning to process large numbers of images collected across the world to make new discoveries, will lead to increased imaging of as well as research into artificial intelligence-assisted analysis of images and clinical data from patients with sickle cell disease.