Smartphone-Based Accurate Analysis of Retinal Vasculature towards Point-of-Care Diagnostics

Retinal vasculature analysis is important for the early diagnostics of various eye and systemic diseases, making it a potentially useful biomarker, especially for resource-limited regions and countries. Here we developed a smartphone-based retinal image analysis system for point-of-care diagnostics that is able to load a fundus image, segment retinal vessels, analyze individual vessel width, and store or uplink results. The proposed system was not only evaluated on widely used public databases and compared with the state-of-the-art methods, but also validated on clinical images directly acquired with a smartphone. An Android app is also developed to facilitate on-site application of the proposed methods. Both visual assessment and quantitative assessment showed that the proposed methods achieved comparable results to the state-of-the-art methods that require high-standard workstations. The proposed system holds great potential for the early diagnostics of various diseases, such as diabetic retinopathy, for resource-limited regions and countries.

methods with high accuracy often target at high quality images captured with DF-cameras. Moreover, these methods often utilize complicated algorithm designs, such as large-scale data training on high quality images, and are implemented in high-standard workstations. Although there are a few attempts in the establishment of a portable POC diagnostic system 12,26 , they were only tested on public databases consisting of high-quality fundus images captured with DF-cameras. Therefore, there is still an unmet need for automatic systems working with low quality fundus images as obtained by the emerging HF-camera. The desired algorithms for a POC diagnostic system should be not only accurate and robust enough to work at resource-limited settings, but also fast and simple enough to be implemented in a portable device (e.g., smartphone).
In this study, we developed a fully automatic retinal image analysis system that can deal with low quality images captured with a HF-camera combined with a smartphone (iExaminer, Welch-Allyn Inc., Skaneateles Fall, NY, USA). This system can read a fundus image, segment retinal vessels, analyze individual vessel width, and store or uplink the results. More specifically, we developed a visual saliency based vessel segmentation method and also a graph-theoretic vessel width measurement method. These two methods were compared with existing methods on high quality fundus images, and also tested on low quality clinical images taken with a smartphone. At last, the proposed system was implemented independently in a smartphone app, which provides a user-friendly interface for image acquisition, test analysis, and result management.

Results and Discussion
An overview of the whole system is given in Fig. 1. The visualized results of proposed method is given in Fig. 2. Taking a typical image in the DRIVE (Digital Retinal Image for Vessel Extraction) dataset as an example, the results are demonstrated in detail. Starting from the original fundus image (Fig. 2a), the blood vessels are detected (Fig. 2b). We can see the vessel segmentation method was able to detect most of the large vessels, but it missed some of the fine vessels. It also showed high false positives around the optic disc. Then the vessel width was measured at locations where the blood vessel was detected (Fig. 2c). We can see that both large and fine vessels were accurately measured. Figure 2d-g show the inset view of the original fundus image, vessel segmentation, vessel centerline, and vessel width measurement. Specifically, Fig. 2f shows the vessel centerline used in the two-dimensional graph construction, in which the vessel segments with a length smaller than certain pixels were excluded from further vessel width measurement.
To quantitatively assess the performance of the vessel segmentation algorithm, we evaluated the proposed method on the DRIVE and STARE (Structured Analysis of the Retina) databases by comparing the proposed method with existing methods in Table 1 (Hoover et al. 15 , Jiang et al. 18 , Staal et al. 19 , Mendonca et al. 20 , etc.). The average accuracy (Acc), sensitivity (SN), and specificity (SP) obtained from our method are 0.933, 0.786, and 0.955 for the DRIVE database, and 0.920, 0.825, and 0.931 for the STARE database, respectively. Even though the proposed method was implemented in a Java Android platform for a smartphone, it showed comparable results with the state-of-the-art methods implemented in standard computer workstations. The running time using the proposed method is ~118 seconds for an image from the DRIVE database (565 × 584 RGB color image) and ~130 seconds for an image from the STARE database (700 × 605 RGB color image). To show the overall performance of the method as the threshold varies, we assessed the receiver operating characteristic (ROC) curve of the proposed method on DRIVE and STARE databases (Fig. 3). For the DRIVE database, the proposed method showed results comparable with human observer (with the red star on the red line), while for the STARE database, the human observer showed a slightly better result than that from the proposed method (with the green star to the left up corner of the green line). The area under the ROC curve (AUC) for DRIVE and STARE were 0.9585 and 0.9590, respectively.
To assess our method for measuring the vessel width, we evaluated the proposed method on the REVIEW database and compared the performance with various existing methods ( Table 2 and Table 3). The proposed two-dimensional graph method showed comparable results with the state-of-the-art methods in mean and standard deviation of average vessel width and mean and standard deviation of individual differences. Specifically, compared with the three-dimensional method, the proposed method showed a great improvement in running time. For an image of size 2160 × 1440, the three-dimensional method took 41 seconds to solve all graphs on a standard computer workstation (3.40GHz Intel ® Core ™ i7-3770 CPU with 8 GB of RAM) and the two-dimensional graph took around 90 seconds to solve all graphs on a smartphone (Qualcomm snapdragon 801, 2.5GHz, RAM 2 GB).
To assess the ability of the proposed system to work as a POC diagnostic tool, we evaluated the performance of the proposed method on low quality clinical images acquired with a smartphone (Fig. 4). For all images, the blood vessels were segmented with the proposed method. To assess the consistency of width measurement, individual vessel segment from the high quality image and its counterpart from the low quality image were selected and measured. For each image, the three widest vessel segments around the optic disc were manually selected by one observer and visually checked by a second observer to guarantee the validity of the vessel, resulting in 30 pairs of vessel segments in total. For each selected vessel, the average vessel width was calculated and adjusted to physical unit (μ m) for comparison. The visualized results are given in Fig. 4a,b, in which the left images are high quality images and the right images are the low quality counterparts from the same participant. As can be observed, the low quality images showed smaller FOV, lower image resolution, and poorer contrast compared with their high quality counterpart. To quantitatively assess the algorithmic consistency between high quality images and low quality images, we performed the paired student's t-test and found no significant difference between the two measurements (with a p-value of 0.07 in the paired t-test). The scatter plot shows good agreement between the two measurements with a Pearson's correlation of 0.922 (Fig. 4c). These results indicate that the proposed method can give consistent results on high quality image and low quality image. Therefore, the developed system holds great potential as a POC diagnostic tool at resource-limited settings. A user-friendly app was developed to facilitate on-site image acquisition, analysis, and management (Fig. 5). Figure 5a shows the image acquisition and Fig. 5b shows screenshots of the app, which are the home screen, a low quality image after vessel segmentation, and a low quality image after width measurement, respectively.
Retinal vasculature analysis is important for the early diagnostics of various microvascular diseases, as the changes in retinal vessel usually precede the advent of other signs, making it a potentially useful biomarker. However, the manual segmentation and measurement is extremely tedious, difficult, and prone-to-error. The proposed system is proved to be a valuable tool for retinal vasculature analysis at resource-limited settings. Validation on various public databases indicated that this system showed comparable results to the state-of-the-art methods that are complex in design and require high standard workstations as platforms. Testing on low quality images captured by smartphone further confirmed that the proposed system was able to handle low quality images taken at resource-limited settings.
POC diagnostics at resource-limited settings puts high requirements on the on-site diagnostic tools. In smartphone-based image analysis, the first challenge is the ability to deal with low quality images taken with HF-camera. In this study, we not only showed that the proposed system was able to deal with low quality images, but also showed that the performance on low quality images was consistent with the performance on their counterpart high quality images. This is important not only because high quality images are regarded as golden standard and almost all clinical studies are based on high quality images, but also because it provides the patients with a wider choice in later clinical visits. Another important challenge in POC diagnostics is the large variety in test samples. Human retina and retina images show large diversity in background color and vascular geometry because of systemic, environmental and genetic factors 27 . In this respect, one advantage of the proposed system is its independence from any training data, making it readily applicable to unknown images obtained at resource-limited settings. A third challenge in POC diagnostics is its requirements on real-time analysis. Taking advantage of the smartphone, we were able to achieve real-time analysis by keeping the whole test inside a smartphone, including image loading, analysis, and result display. Even though it takes ~3 minutes to finish a test, it is still able to provide on-site and real-time analysis, which is important for resource-limited settings where well-trained clinicians and tele-medicine are not always available.
Although the proposed system has proved to be promising for the detection of within-subject follow-up changes of retinal vasculature, which is important for individual monitoring of disease progression, the between-subject differences also need to be addressed. As discussed above, human retina and retina images show large diversity in background color as well as vascular characteristics, meaning the baseline for health and disease may vary greatly between individuals with different ethnicity, age, etc. Future studies will focus on how these between-subject variabilities can be appreciated.

Conclusion
In this study, we developed a smartphone-based retinal image analysis system targeting at POC diagnostics at resource-limited settings. This system shows comparable performance with the state-of-the-art methods with a much lower computational complexity. This system can be further combined with commercialized HF-cameras for POC diagnostics or large population screening at resource-limited settings. Future work will include studies on between-subject vascular variability and implementing this algorithm in a POC diagnostic system for the early diagnostics of DR.

Materials and Methods
Public database. The vessel segmentation method was evaluated on two popular public databases, which are widely used for the evaluation of retinal vessel segmentation methods. The DRIVE database consists of a set of 40 RGB color fundus photographs obtained from a diabetic retinopathy screening program 17 . The images (565 × 584 pixels) were acquired using a Canon CR5 non-mydriatic 3-CCD camera with a FOV of 45 o . DRIVE database is Data  divided into two sets, the training set and the test set, each containing twenty images. The test set was manually segmented by two observers and the first observer is accepted as ground truth. The STARE database contains a set of twenty images, ten of which show signs of pathology 15 . The images (700 × 605 pixels, 8-bit per color channel) were captured using a TopCon TRV-50 fundus camera with a FOV of 35 o . Two experts manually segmented all images and the first observer is regarded as ground truth. The vessel width measurement method was evaluated on the REVIEW database, which is widely used for the evaluation of vessel width measurement methods 28 . The REVIEW database contains four image sets with vessel measurement from three observers. The VDIS dataset contains severe disease cases and the KPIS dataset includes only parts of retinal images. These two datasets were thus excluded in this study. The other two datasets (i.e., HRIS and CLRIS) were included for validation. The HRIS dataset represents different stage of DR and consists of 2368 manual vessel profiles from 90 vessel segments. The CLRIS dataset includes a strong central light   Clinical data. The clinical images were collected at Xi'an No.1 Hospital. Ten normal eyes from ten participants were included in this study. We captured both high quality fundus images using a DF-camera (Topcon TRC-NW8 fundus camera, FOV of 45 o ) and low quality fundus images using a HF-camera (iExaminer plus 'Panoptic' , FOV of 25 o , Welch-Allyn Inc., Skaneateles Fall, NY, USA) 10 ). The low quality images showed a smaller FOV, lower resolution, and poorer contrast due to the hardware limitations, greatly increasing the difficulties of image analysis. Informed consent for research use of data was sought and obtained from each study participant before participation. The study was approved by the Ethics Committee of Xi'an Jiaotong University. The methods were carried out in accordance with the approved guidelines.
Visual saliency based vessel segmentation. The vessel analysis system contains a vessel segmentation algorithm and a vessel width measurement algorithm. Vessel segmentation is the fundamental task for further analyses of retinal vasculature, such as vessel width, vessel tortuosity, branching angle, and arteriovenous ratio. In this study, a fast and accurate vessel segmentation algorithm based on visual saliency is introduced. One advantage of saliency based method is that only global operators and linear local neighborhood operators are used, meaning it is simple in algorithmic design and fast in computational performance. In this method, multi-scale salient features are generated, including spectral residual, orientation, morphological features, and self-information. When all four salient features are extracted, a gray scale vessel image is created by a linear combination of all normalized saliency features. A binary vessel image is created using Triangle thresholding. More details are given in Supplementary Materials.

Graph-theoretic vessel width measurement.
We previously reported a vessel width measurement method based on three-dimensional graph search, which converts the simultaneous two-boundary detection problem into a two-slice three-dimensional minimum closed set problem 29 . This method achieves high accuracy but suffers from high computation complexity because of the three-dimensional graph design. Here, we improved this method by breaking the three-dimensional graph down to two separate two-dimensional graphs. To do this, the inter-slice smoothness constraint, i.e., the connection between the two slices, is removed and the intra-slice smoothness constraint is strengthened. In this way, the computational performance can be greatly improved with limited sacrifice in accuracy. After the two-dimensional graphs are constructed, they are solved as separate minimum closed set problems. Once both vessel boundaries are solved, the vessel width is determined as the distance between the left boundary and the right boundary along the normal direction of the blood vessel. More details are given in Supplementary Materials.
Smartphone-based GUI system. We developed an Android app as a graphical user interface (GUI).
Specifically, this app is designed for multiple tasks. (a) account management;(b) loading fundus images;(c) fully automatic image analyses;(d) result display, storage, and up-linking;(e) user instruction. The details are described below: (a) Account management: this system allows multiple users on a single smartphone, each of which is protected by a user name and a password. (b) Loading fund images: the system allows loading fundus images from the smartphone gallery or capturing new fundus images if the smartphone is equipped with a portable fundus camera. (c) Automatic image analyses: the system allows automatic analyses of a fundus image, including vessel segmentation and width measurement as described above.  FN), reflecting the ability of an algorithm to find vessel pixels, where TP means true positives and FN means false negative. SP is defined as TN/(TN + FP), reflecting the ability of an algorithm to find non-vessel pixels, where TN means true negative and FP means false positive. Acc is defined as the ratio of the total number of correctly classified pixels (TP + TN) to the total number of pixels in the image. The ROC curve shows the tradeoff between sensitivity and specificity as its discrimination threshold varies. The closer an ROC curve approaches the top left corner, the better the performance of the method. The AUC, ranging from 0 to 1, reflects the overall performance of the method, in which a perfect test has an AUC of 1.
In the vessel width measurement, for each measurement from each observer, the vessel centerline is defined as A reference standard (RS) is created by averaging the manual measurements from the three observers. Comparison between different methods was performed by five parameters: success rate, mean and standard deviation of average vessel width, and mean and standard deviation of individual differences. The success rate is defined as the ratio between matched points and total RS centerline points, in which a matched is defined as success if at least one detected centerline pixel is found within certain distance to an RS centerline point. The mean and standard deviation of the average vessel width reflect the overall similarity between different methods while the mean and standard deviation of the individual differences reflect the overall differences between individual measurements.
In the validation on low quality clinical images, the paired student's t-test was used to evaluate the difference between the vessel widths from high quality image and low quality image. A p value of < 0.05 was considered statistically significant. The scatter plot and Pearson's correlation were also reported.