Abstract
Although machine learning (ML) has made significant improvements in radiology, few algorithms have been integrated into clinical radiology workflow. Complex radiology IT environments and Picture Archiving and Communication System (PACS) pose unique challenges in creating a practical ML schema. However, clinical integration and testing are critical to ensuring the safety and accuracy of ML algorithms. This study aims to propose, develop, and demonstrate a simple, efficient, and understandable hardware and software system for integrating ML models into the standard radiology workflow and PACS that can serve as a framework for testing ML algorithms. A Digital Imaging and Communications in Medicine/Graphics Processing Unit (DICOM/GPU) server and software pipeline was established at a metropolitan county hospital intranet to demonstrate clinical integration of ML algorithms in radiology. A clinical ML integration schema, agnostic to the hospital IT system and specific ML models/frameworks, was implemented and tested with a breast density classification algorithm and prospectively evaluated for time delays using 100 digital 2D mammograms. An open-source clinical ML integration schema was successfully implemented and demonstrated. This schema allows for simple uploading of custom ML models. With the proposed setup, the ML pipeline took an average of 26.52 s per second to process a batch of 100 studies. The most significant processing time delays were noted in model load and study stability times. The code is made available at “http://bit.ly/2Z121hX”. We demonstrated the feasibility to deploy and utilize ML models in radiology without disrupting existing radiology workflow.
Similar content being viewed by others
References
Choy G, Khalilzadeh O, Michalski M, et al: Current Applications and Future Impact of Machine Learning in Radiology. Radiology 288(2):318–328,2018.
Ding Y, Sohn JH, Kawczynski MG, et al: A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18F-FDG PET of the Brain. Radiology 290(2):456–464,2018.
Kallenberg M, Petersen K, Nielsen M, et al: Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring. IEEE Trans Med Imaging 35(5):1322–1331,2016.
Polan DF, Brady SL, Kaufman RA: Tissue segmentation of computed tomography images using a Random Forest algorithm: a feasibility study. Phys Med Biol 61(17):6553–6569,2016.
Pedoia V, Majumdar S, Link TM: Segmentation of joint and musculoskeletal tissue in the study of arthritis. Magn Reson Mater Phys Biol Med 29(2):207–221,2016.
Bickelhaupt S, Paech D, Kickingereder P, et al: Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography. J Magn Reson Imaging 46(2):604–616,2017.
McDonald RJ, Schwartz KM, Eckel LJ, et al: The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad Radiol 22(9):1191–1198,2015.
Jodogne S: The Orthanc Ecosystem for Medical Imaging. J Digit Imaging 31(3):341–352,2018.
Chokshi FH, Flanders AE, Prevedello LM, Langlotz CP: Fostering a Healthy AI Ecosystem for Radiology: Conclusions of the 2018 RSNA Summit on AI in Radiology. Radiol Artif Intell 1(2):190021,2019.
Berkowitz SJ, Wei JL, Halabi S: Migrating to the Modern PACS: Challenges and Opportunities. RadioGraphics 38(6):1761–1772,2018.
Sickles E, Bassett L, et al: ACR BI-RADS® Mammography. American College of Radiology; 2013. https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-Rads/.
Chollet F: Xception: Deep Learning with Depthwise Separable Convolutions. ArXiv 161002357(Cs),2016;http://arxiv.org/abs/1610.02357. Accessed July 1, 2019.
Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC: Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. Npj Digit Med 1(1):39,2018.
Kanagasingam Y, Xiao D, Vignarajan J, Preetham A, Tay-Kearney M-L, Mehrotra A: Evaluation of Artificial Intelligence–Based Grading of Diabetic Retinopathy in Primary Care. JAMA Netw Open 1(5):e182665,2018.
Topol EJ: High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25(1):44,2019.
Gianfrancesco MA, Tamang S, Yazdany J, Schmajuk G: Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern Med 178(11):1544–1547,2018.
Heydari A, Lituiev D, Vu TH, Seo Y, Sohn JH: A DICOM-embedded Annotation System for 3D Cross-sectional Imaging Data. Abstr Radiol Soc N Am,2019.
Acknowledgments
We thank the NVIDIA corporation for the donation of NVIDIA-Titan Xp GPU.
Funding
Jae Ho Sohn was supported by the National Institute of Biomedical Imaging and Bioengineering T32-EB001631 grant.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sohn, J., Chillakuru, Y.R., Lee, S. et al. An Open-Source, Vender Agnostic Hardware and Software Pipeline for Integration of Artificial Intelligence in Radiology Workflow. J Digit Imaging 33, 1041–1046 (2020). https://doi.org/10.1007/s10278-020-00348-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-020-00348-8