Luca Barillaro
Mario Cannataro
ABSTRACT
This paper provides an overview on scalable deep learning platforms and how they are used in medical context. An introduction highlights the key factors, then an overview on medical context is provided. Afterwards, the basic concepts about deep learning and parallel and distributed computing are briefly recalled. Then a specific deep learning library for medical applications is described. The last part of the paper is focused on a real use case application of deep learning on medical data. As a result, the main contribution of this paper is a short survey on main scalable deep learning platforms with a first analysis of their features, and the description of a practical example.
- Anaconda, Inc 2016. Anaconda Software Distribution. Anaconda, Inc. https://anaconda.com/Google Scholar
- 2019. Deep Health - Deep-Learning and HPC to Boost Biomedical Applications for Health. https://deephealth-project.eu/.Google Scholar
- Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viegas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv:1603.04467 [cs] (March 2016). arXiv:1603.04467 [cs]Google Scholar
- Giuseppe Agapito, Mario Cannataro, Pietro Hiram Guzzi, Fabrizio Marozzo, Domenico Talia, and Paolo Trunfio. 2013. Cloud4SNP: Distributed Analysis of SNP Microarray Data on the Cloud. In Proceedings of the International Conference on Bioinformatics, Computational Biology and Biomedical Informatics. ACM, Wshington DC USA, 468--475. Google ScholarDigital Library
- Abien Fred Agarap. 2019. Deep Learning Using Rectified Linear Units (ReLU). arXiv:1803.08375 [cs, stat] (Feb. 2019). arXiv:1803.08375 [cs, stat]Google Scholar
- Syed Muhammad Anwar, Muhammad Majid, Adnan Qayyum, Muhammad Awais, Majdi Alnowami, and Muhammad Khurram Khan. 2018. Medical Image Analysis Using Convolutional Neural Networks: A Review. Journal of Medical Systems 42, 11 (Nov. 2018), 226. Google ScholarDigital Library
- Ekaba Bisong. 2019. Google Colaboratory. Apress, Berkeley, CA, 59--64. Google ScholarCross Ref
- G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000).Google Scholar
- Mario Cannataro, Pietro Hiram Guzzi, and Alessia Sarica. 2013. Data Mining and Life Sciences Applications on the Grid: Data Mining and Life Sciences Applications on the Grid. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 3, 3 (May 2013), 216--238. Google ScholarCross Ref
- Francois Chollet et al. 2015. Keras. https://github.com/fchollet/keras.Google Scholar
- Alex Clark. 2015. Pillow (PIL Fork) Documentation.Google Scholar
- Ronan Collobert, Samy Bengio, and Johnny Mariéthoz. 2002. Torch: A Modular Machine Learning Software Library. Technical Report. Idiap.Google Scholar
- Alexander Craik, Yongtian He, and Jose L Contreras-Vidal. 2019. Deep Learning for Electroencephalogram (EEG) Classification Tasks: A Review. Journal of Neural Engineering 16, 3 (June 2019), 031001. Google ScholarCross Ref
- Rodrigo Crespo-Cepeda, Giuseppe Agapito, Jose Luis Vazquez-Poletti, and Mario Cannataro. 2019. Challenges and Opportunities of Amazon Serverless Lambda Services in Bioinformatics. In Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. ACM, Niagara Falls NY USA, 663--668. Google ScholarDigital Library
- Li Deng. 2012. The Mnist Database of Handwritten Digit Images for Machine Learning Research. IEEE Signal Processing Magazine 29, 6 (2012), 141--142.Google ScholarCross Ref
- Endre Grøvik, Darvin Yi, Michael Iv, Elizabeth Tong, Daniel Rubin, and Greg Zaharchuk. 2020. Deep Learning Enables Automatic Detection and Segmentation of Brain Metastases on Multisequence MRI. Journal of Magnetic Resonance Imaging 51, 1 (Jan. 2020), 175--182. Google ScholarCross Ref
- Vishakh Hegde and Sheema Usmani. 2016. Parallel and Distributed Deep Learning. (2016), 8.Google Scholar
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (Nov. 1997), 1735--1780. Google ScholarDigital Library
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv:1408.5093 [cs] (June 2014). arXiv:1408.5093 [cs]Google Scholar
- Andreas M. Kist. 2021. Deep Learning on Edge TPUs. arXiv:2108.13732 [cs] (Aug. 2021). arXiv:2108.13732 [cs]Google Scholar
- Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, and Carol Willing. 2016. Jupyter Notebooks - a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas, F. Loizides and B. Schmidt (Eds.). IOS Press, 87 -- 90.Google Scholar
- Fatma Murat, Ozal Yildirim, Muhammed Talo, Ulas Baran Baloglu, Yakup Demir, and U. Rajendra Acharya. 2020. Application of Deep Learning Techniques for Heartbeats Detection Using ECG Signals-Analysis and Review. Computers in Biology and Medicine 120 (May 2020), 103726. Google ScholarCross Ref
- NVIDIA. 2022. NVIDIA Tensor Cores: Versatility for HPC & AI. https://www.nvidia.com/en-us/data-center/tensor-cores/.Google Scholar
- NVIDIA, Péter Vingelmann, and Frank H.P. Fitzek. 2020. Nvidia CUDA.Google Scholar
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv:1912.01703 [cs, stat] (Dec. 2019). arXiv:1912.01703 [cs, stat]Google Scholar
- Pierre Raybaut. 2009. Spyder-Documentation. Available online at: pythonhosted.org (2009).Google Scholar
- Alessia Sarica, Antonio Cerasa, Roberta Vasta, Paolo Perrotta, Paola Valentino, Graziella Mangone, Pietro H. Guzzi, Federico Rocca, Matteo Nonnis, Mario Cannataro, and Aldo Quattrone. 2014. Tractography in Amyotrophic Lateral Sclerosis Using a Novel Probabilistic Tool: A Study with Tract-Based Reconstruction Compared to Voxel-Based Approach. Journal of Neuroscience Methods 224 (March 2014), 79--87. Google ScholarCross Ref
- Alex Sherstinsky. 2020. Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. Physica D: Nonlinear Phenomena 404 (March 2020), 132306. Google ScholarCross Ref
- Eduardo Soares, Plamen Angelov, Sarah Biaso, Michele Higa Froes, and Daniel Kanda Abe. 2020. SARS-CoV-2 CT-scan Dataset: A Large Dataset of Real Patients CT Scans for SARS-CoV-2 Identification. Preprint. Health Informatics. Google ScholarCross Ref
- Sudarshan Srinivasan, Pradeep Janedula, Saurabh Dhoble, Sasikanth Avancha, Dipankar Das, Naveen Mellempudi, Bharat Daga, Martin Langhammer, Gregg Baeckler, and Bharat Kaul. 2019. High Performance Scalable FPGA Accelerator for Deep Neural Networks. arXiv:1908.11809 [cs] (Aug. 2019). arXiv:1908.11809 [cs]Google Scholar
- The Theano Development Team, Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, Anatoly Belikov, Alexander Belopolsky, Yoshua Bengio, Arnaud Bergeron, James Bergstra, Valentin Bisson, Josh Bleecher Snyder, Nicolas Bouchard, Nicolas Boulanger-Lewandowski, Xavier Bouthillier, Alexandre de Brébisson, Olivier Breuleux, Pierre-Luc Carrier, Kyunghyun Cho, Jan Chorowski, Paul Christiano, Tim Cooijmans, Marc-Alexandre Côté, Myriam Côté, Aaron Courville, Yann N. Dauphin, Olivier Delalleau, Julien Demouth, Guillaume Desjardins, Sander Dieleman, Laurent Dinh, Mélanie Ducoffe, Vincent Dumoulin, Samira Ebrahimi Kahou, Dumitru Erhan, Ziye Fan, Orhan Firat, Mathieu Germain, Xavier Glorot, Ian Goodfellow, Matt Graham, Caglar Gulcehre, Philippe Hamel, Iban Harlouchet, Jean-Philippe Heng, Balázs Hidasi, Sina Honari, Arjun Jain, Sébastien Jean, Kai Jia, Mikhail Korobov, Vivek Kulkarni, Alex Lamb, Pascal Lamblin, Eric Larsen, César Laurent, Sean Lee, Simon Lefrancois, Simon Lemieux, Nicholas Léonard, Zhouhan Lin, Jesse A. Livezey, Cory Lorenz, Jeremiah Lowin, Qianli Ma, Pierre-Antoine Manzagol, Olivier Mastropietro, Robert T. McGibbon, Roland Memisevic, Bart van Merriënboer, Vincent Michalski, Mehdi Mirza, Alberto Orlandi, Christopher Pal, Razvan Pascanu, Mohammad Pezeshki, Colin Raffel, Daniel Renshaw, Matthew Rocklin, Adriana Romero, Markus Roth, Peter Sadowski, John Salvatier, François Savard, Jan Schlüter, John Schulman, Gabriel Schwartz, Iulian Vlad Serban, Dmitriy Serdyuk, Samira Shabanian, Étienne Simon, Sigurd Spieckermann, S. Ramana Subramanyam, Jakub Sygnowski, Jérémie Tanguay, Gijs van Tulder, Joseph Turian, Sebastian Urban, Pascal Vincent, Francesco Visin, Harm de Vries, David Warde-Farley, Dustin J. Webb, Matthew Willson, Kelvin Xu, Lijun Xue, Li Yao, Saizheng Zhang, and Ying Zhang. 2016. Theano: A Python Framework for Fast Computation of Mathematical Expressions. arXiv:1605.02688 [cs] (May 2016). arXiv:1605.02688 [cs]Google Scholar
- J. Teye Brown and Walid Zgallai. 2020. Deep EEG: Deep Learning in Biomedical Signal Processing with EEG Applications. In Biomedical Signal Processing and Artificial Intelligence in Healthcare. Elsevier, 113--151. Google ScholarCross Ref
- Yuchi Tian, Ziyuan Zhong, Vicente Ordonez, Gail Kaiser, and Baishakhi Ray. 2020. Testing DNN Image Classifiers for Confusion & Bias Errors. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (June 2020), 1122--1134. arXiv:1905.07831 Google ScholarDigital Library
Index Terms
- Scalable deep learning for healthcare: methods and applications
Recommendations
DL4HC: Deep Learning for Healthcare
CODS-COMAD '23: Proceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD)In Over the last few years, Machine Learning (ML), and particularly Deep Learning (DL), has made great strides and has been successfully deployed in many real-world applications such as healthcare, customer care, finance, autonomous driving etc. One of ...
Scalable Deep Learning via I/O Analysis and Optimization
Scalable deep neural network training has been gaining prominence because of the increasing importance of deep learning in a multitude of scientific and commercial domains. Consequently, a number of researchers have investigated techniques to optimize ...
Making use of cloud computing for healthcare provision
Cloud computing has opened many possibilities for organizations that did not exist before.Among the new sectors that are embracing and/or exploring the cloud are healthcare providers.The cloud has the potential to offer huge benefits for healthcare ...
Comments