Skip to main content

Advertisement

SpringerLink
Log in

Automated OCT angiography image quality assessment using a deep learning algorithm

  • Retinal Disorders
  • Published:
Graefe's Archive for Clinical and Experimental Ophthalmology Aims and scope Submit manuscript

Abstract

Purpose

To expedite and to standardize the process of image quality assessment in optical coherence tomography angiography (OCTA) using a specialized deep learning algorithm (DLA).

Methods

Two hundred randomly chosen en-face macular OCTA images of the central 3 × 3 mm2 superficial vascular plexus were evaluated retrospectively by an OCTA experienced reader. Images were defined either as sufficient (group 1, n = 100) or insufficient image quality (group 2, n = 100) based on Motion Artifact Score (MAS) and Segmentation Accuracy Score (SAS). Subsequently, a pre-trained multi-layer deep convolutional neural network (DCNN) was trained and validated with 160 of these en-face OCTA scans (group 1: 80; group 2: 80). Training accuracy, validation accuracy, and cross-entropy were computed. The DLA was tested in detecting 40 untrained OCTA images (group 1: 20; group 2: 20). An insufficient image quality probability score (IPS) and a sufficient image quality probability score (SPS) were calculated.

Results

Training accuracy was 97%, validation accuracy 100%, and cross entropy 0.12. A total of 90% (18/20) of the OCTA images with insufficient image quality and 90% (18/20) with sufficient image quality were correctly classified by the DLA. Mean IPS was 0.88 ± 0.21, and mean SPS was 0.84 ± 0.19. Discrimination between both groups was highly significant (p < 0.001). Sensitivity of the DLA was 90.0%, specificity 90.0%, and accuracy 90.0%. Coefficients of variation were 0.96 ± 1.9% (insufficient quality) and 1.14 ± 1.6% (sufficient quality).

Conclusions

Deep learning (DL) appears to be a potential approach to automatically distinguish between sufficient and insufficient OCTA image quality. DL may contribute to establish image quality standards in this recent imaging modality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Schmidt-Erfurth U, Sadeghipour A, Gerendas BS, Waldstein SM, Bogunović H (2018) Artificial intelligence in retina. Prog Retin Eye Res 67:1–29

    Article  PubMed  Google Scholar 

  2. De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24:1342–1350. https://doi.org/10.1038/s41591-018-0107-6

    Article  CAS  PubMed  Google Scholar 

  3. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  Google Scholar 

  4. Burlina P, Pacheco KD, Joshi N, Freund DE, Bressler NM (2017) Comparing humans and deep learning performance for grading AMD: a study in using universal deep features and transfer learning for automated AMD analysis. Comput Biol Med 82:80–866

    Article  PubMed  PubMed Central  Google Scholar 

  5. Rahimy E (2018) Deep learning applications in ophthalmology. Curr Opin Ophthalmol 29(3):254–260

    Article  PubMed  Google Scholar 

  6. Treder M, Eter N (2018) Deep learning and neuronal networks in ophthalmology: applications in the field of optical coherence tomography. Ophthalmologe 115(9):714–721

    Article  CAS  PubMed  Google Scholar 

  7. Treder M, Lauermann JL, Eter N (2018) Deep learning-based detection and classification of geographic atrophy using a deep convolutional neural network classifier. Graefes Arch Clin Exp Ophthalmol 256(11):2053–2060

    Article  CAS  PubMed  Google Scholar 

  8. Prahs P, Radeck V, Mayer C, Cvetkov Y, Cvetkova N, Helbig H et al (2018) OCT-based deep learning algorithm for the evaluation of treatment indication with anti-vascular endothelial growth factor medications. Graefes Arch Clin Exp Ophthalmol 256(1):91–98

    Article  PubMed  Google Scholar 

  9. Bogunovic H, Montuoro A, Baratsits M, Karantonis MG, Waldstein SM, Schlanitz F et al (2017) Machine learning of the progression of intermediate age-related macular degeneration based on OCT imaging. Invest Ophthalmol Vis Sci 58(6):BIO141–BIO150

    Article  PubMed  Google Scholar 

  10. Schlegl T, Waldstein SM, Bogunovic H, Endstraßer F, Sadeghipour A, Philip AM et al (2018) Fully automated detection and quantification of macular fluid in OCT using deep learning. Ophthalmology 125(4):549–558

    Article  PubMed  Google Scholar 

  11. Jia Y, Bailey ST, Hwang TS, McClintic SM, Gao SS, Pennesi ME et al (2015) Quantitative optical coherence tomography angiography of vascular abnormalities in the living human eye. Proc Natl Acad Sci U S A 112:E2395–E2402

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Cole ED, Ferrara D, Novais EA, Louzada RN, Waheed NK (2016) Clinical trial endpoints for optical coherence tomography angiography in neovascular age-related macular degeneration. Retina 36(Suppl 1):S83–S92

    Article  PubMed  Google Scholar 

  13. Lauermann JL, Eter N, Alten F (2018) Optical coherence tomography angiography offers new insights into choriocapillaris perfusion. Ophthalmologica 239(2–3):74–84

    Article  Google Scholar 

  14. Ang M, Tan ACS, Cheung CMG, Keane PA, Dolz-Marco R, Sng CCA et al (2018) Optical coherence tomography angiography: a review of current and future clinical applications. Graefes Arch Clin Exp Ophthalmol 256(2):237–245

    Article  Google Scholar 

  15. Say EA, Ferenczy S, Magrath GN, Samara WA, Khoo CT, Shields CL (2017) Image quality and artifacts on optical coherence tomography angiography: comparison of pathologic and paired fellow eyes in 65 patients with unilateral choroidal melanoma treated with plaque radiotherapy. Retina 37(9):1660–1673

    Article  PubMed  Google Scholar 

  16. Sadda SR, Wu Z, Walsh AC, Richine L, Dougall J, Cortez R et al (2006) Errors in retinal thickness measurements obtained by optical coherence tomography. Ophthalmology 113(2):285–293

    Article  Google Scholar 

  17. Lauermann JL, Treder M, Heiduschka P, Clemens CR, Eter N, Alten F (2017) Impact of eye-tracking technology on OCT-angiography imaging quality in age-related macular degeneration. Graefes Arch Clin Exp Ophthalmol 255:1535–1542

    Article  CAS  Google Scholar 

  18. Al-Sheikh M, Ghasemi Falavarjani K, Akil H, Sadda SR (2017) Impact of image quality on OCT angiography based quantitative measurements. Int J Retina Vitreous 15(3):13

    Article  Google Scholar 

  19. Spaide RF, Fujimoto JG, Waheed NK (2015) Image artifacts in optical coherence tomography angiography. Retina 35:2163–2180

    Article  PubMed  PubMed Central  Google Scholar 

  20. Fenner BJ, Tan GS, Tan AC, Yeo IY, Wong TY, Cheung GC (2018) Identification of imaging features that determine quality and repeatability of retinal capillary plexus density measurements in OCT angiography. Br J Ophthalmol 102(4):509–514

    PubMed  Google Scholar 

  21. Alten F, Lauermann JL, Clemens CR, Heiduschka P, Eter N (2017) Signal reduction in choriocapillaris and segmentation errors in spectral domain OCT angiography caused by soft drusen. Graefes Arch Clin Exp Ophthalmol 255(12):2347–2355

    Article  CAS  PubMed  Google Scholar 

  22. Spaide RF, Curcio CA (2017) Evaluation of segmentation of the superficial and deep vascular layers of the retina by optical coherence tomography angiography instruments in normal eyes. JAMA Ophthalmol 135(3):259–262

    Article  PubMed  Google Scholar 

  23. Lauermann JL, Wötzel AK, Treder M, Alnawaiseh M, Clemens CR, Eter N (2018) Prevalences of segmentation errors and motion artifacts in OCT-angiography differ among retinal diseases. Graefes Arch Clin Exp Ophthalmol 256(10):1807–1816

    Article  CAS  Google Scholar 

  24. Bogunovic H, Waldstein SM, Schlegl T, Langs G, Sadeghipour A, Liu X et al (2017) Prediction of anti-VEGF treatment requirements in neovascular AMD using a machine learning approach. Invest Ophthalmol Vis Sci 58(7):3240–3248

    Article  CAS  PubMed  Google Scholar 

  25. Wang Y, Zhang Y, Yao Z, Zhao R, Zhou F (2017) Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images. Biomed Opt Express 7:4928–4940

    Article  Google Scholar 

  26. Venhuizen F, van Ginneken B, van Asten F, van Grinsven M, Fauser S, Hoyng C et al (2017) Automated staging of age-related macular degeneration using optical coherence tomography. Invest Ophthalmol Vis Sci 58:2318–2328

    Article  PubMed  Google Scholar 

  27. Alsaih K, Lemaitre G, Rastgoo M, Massich J, Sidibe D, Meriaudeau F (2017) Machine learning techniques for diabetic macular edema (DME) classification on SD-OCT images. Biomed Eng Online 7;16(1):68

    Article  Google Scholar 

  28. Waldstein SM, Montuoro A, Podkowinski D, Philip AM, Gerendas BS, Bogunovic H et al (2017) Evaluating the impact of vitreomacular adhesion on anti-VEGF therapy for retinal vein occlusion using machine learning. Sci Rep 7:2928

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Vogl W, Waldstein S, Gerendas B, Schmidt-Erfurth U, Langs G (2017) Predicting macular edema recurrence from Spatio-temporal signatures in optical coherence tomography images. IEEE Trans Med Imaging 36(9):1773–1783

    Article  PubMed  Google Scholar 

  30. Kim S, Cho K, Oh S (2017) Development of machine learning models for diagnosis of glaucoma. PLoS One 12:e0177726

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Murugeswari S, Sukanesh R (2017) Investigations of severity level measurements for diabetic macular oedema using machine learning algorithms. Ir J Med Sci 186(4):929–938

    Article  CAS  PubMed  Google Scholar 

  32. U.S. Food & Drug Administration (FDA) (2018) https://www.fda.gov/newsevents//newsroom/pressannouncements/ucm604357.htm; Accessed 08 Sept 2018

  33. Van der Heijden AA, Abramoff MD, Verbraak F, van Hecke MV, Liem A, Nijpels G (2018) Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System. Acta Ophthalmol 96(1):63–68

    Article  PubMed  Google Scholar 

  34. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine 1 Article number: 39

  35. Huang D, Jia Y, Gao SS, Lumbroso B, Rispoli M (2016) Optical coherence tomography angiography using the Optovue device. Dev Ophthalmol 56:6–12

    Article  PubMed  Google Scholar 

  36. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, et al (2018) TensorFlow: large-scale machine learning on heterogeneous distributed systems. TensorFlow; https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45166.pdf; Accessed 08 Sept 2018

  37. Google Developers (2017) https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0. Google Developers; Accessed 08 Sept 2018

  38. Treder M, Lauermann JL, Eter N (2018) Automated detection of exudative age-related macular degeneration in spectral domain optical coherence tomography using deep learning. Graefes Arch Clin Exp Ophthalmol 256(2):259–265

    Article  CAS  PubMed  Google Scholar 

  39. Angermueller C, Parnamaa T, Parts L, Stegle O (2016) Deep learning for computational biology. Mol Syst Biol 12:878

    Article  PubMed  PubMed Central  Google Scholar 

  40. Zhang M, Hwang TS, Campbell JP, Bailey ST, Wilson DJ, Huang D et al (2016) Projection-resolved optical coherence tomographic angiography. Biomed Opt Express 9 7(3):816–828

    Article  PubMed  PubMed Central  Google Scholar 

  41. Uji A, Balasubramanian S, Lei J, Baghdasaryan E, Al-Sheikh M, Sadda SR (2017) Choriocapillaris imaging using multiple en face optical coherence tomography angiography image averaging. JAMA Ophthalmol 135(11):1197–1204

    Article  PubMed  PubMed Central  Google Scholar 

  42. de Sisternes L, Jonna G, Moss J, Marmor MF, Leng T, Rubin DL (2017) Automated intraretinal segmentation of SD-OCT images in normal and age-related macular degeneration eyes. Biomed Opt Express 8(3):1926–1949

    Article  PubMed  PubMed Central  Google Scholar 

  43. Spaide RF, Fujimoto JG, Waheed NK, Sadda SR, Staurenghi G (2018) Optical coherence tomography angiography. Prog Retin Eye Res 64:1–55

    Article  Google Scholar 

  44. Rommel F, Siegfried F, Kurz M, Brinkmann MP, Rothe M, Rudolf M et al (2018) Impact of correct anatomical slab segmentation on foveal avascular zone measurements by optical coherence tomography angiography in healthy adults. J Curr Ophthalmol 30(2):156–160

    Article  PubMed  PubMed Central  Google Scholar 

  45. Nguyen A, Yosinski J, Clune J (2014) Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. http://arxiv.org/abs/1412.1897v4. Accessed 08 Sept 2018

Download references

Author information

Author notes

  1. J.L. Lauermann and M. Treder contributed equally to the work presented here and should therefore be regarded as equivalent first authors

Authors and Affiliations

  1. Department of Ophthalmology, University of Muenster Medical Center, Domagkstrasse 15, 48149, Muenster, Germany

    J. L. Lauermann, M. Treder, M. Alnawaiseh, C. R. Clemens, N. Eter & F. Alten

Authors
  1. J. L. Lauermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Treder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Alnawaiseh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. R. Clemens
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. N. Eter
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F. Alten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to F. Alten.

Ethics declarations

Conflict of interest

Author Lauermann declares that he has no conflict of interest. Author Treder declares that he has no conflict of interest. Author Alnawaiseh declares that he has no conflict of interest. Author Clemens declares that he has no conflict of interest. Author Eter declares that she has no conflict of interest. Author Alten declares that he has no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lauermann, J.L., Treder, M., Alnawaiseh, M. et al. Automated OCT angiography image quality assessment using a deep learning algorithm. Graefes Arch Clin Exp Ophthalmol 257, 1641–1648 (2019). https://doi.org/10.1007/s00417-019-04338-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00417-019-04338-7

Keywords

Search

Navigation