Skip to main content
SpringerLink
Log in

Improving the Generalization of Deep Learning Classification Models in Medical Imaging Using Transfer Learning and Generative Adversarial Networks

  • Conference paper
  • First Online:
Book cover Agents and Artificial Intelligence (ICAART 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13251))

Included in the following conference series:

Abstract

Data sets for medical images are generally imbalanced and limited in sample size because of high data collection costs, time-consuming annotations, and patient privacy concerns. The training of deep neural network classification models on these data sets to improve the generalization ability does not produce the desired results for classifying the medical condition accurately and often overfit the data on the majority of class samples. To address the issue, we propose a framework for improving the classification performance metrics of deep neural network classification models using transfer learning: pre-trained models, such as Xception, InceptionResNet, DenseNet along with the Generative Adversarial Network (GAN) - based data augmentation. Then, we trained the network by combining traditional data augmentation techniques, such as randomly flipping the image left to right and GAN-based data augmentation, and then fine-tuned the hyper-parameters of the transfer learning models, such as the learning rate, batch size, and the number of epochs. With these configurations, the Xception model outperformed all other pre-trained models achieving a test accuracy of 98.7%, the precision of 99%, recall of 99.3%, f1-score of 99.1%, receiver operating characteristic (ROC) - area under the curve (AUC) of 98.2%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brats 2015 - MICCAI Brats 2017. https://sites.google.com/site/braintumorsegmentation/home/brats2015. Accessed 27 Jan 2021

  2. CVPR 2017. https://cvpr2017.thecvf.com/program/main_conference#cvpr2017_awards. Accessed 31 Oct 2020

  3. Introduction - drive - grand challenge. https://drive.grand-challenge.org/DRIVE/. Accessed 27 Jan 2021

  4. IXI dataset - brain development. https://brain-development.org/ixi-dataset/. Accessed 27 Jan 2021

  5. The stare project. https://cecas.clemson.edu/~ahoover/stare/. Accessed 27 Jan 2021

  6. Antin, B., Kravitz, J., Martayan, E.: Detecting pneumonia in chest X-rays with supervised learning. Semanticscholar.org (2017)

    Google Scholar 

  7. Ayan, E., Ünver, H.M.: Diagnosis of pneumonia from chest X-ray images using deep learning. In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), pp. 1–5. IEEE (2019)

    Google Scholar 

  8. Beers, A., et al.: High-resolution medical image synthesis using progressively grown generative adversarial networks. arXiv preprint arXiv:1805.03144 (2018)

  9. Bowles, C., et al.: Gan augmentation: augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863 (2018)

  10. Bullitt, E., et al.: Vessel tortuosity and brain tumor malignancy: a blinded study1. Acad. Radiol. 12(10), 1232–1240 (2005)

    Article  Google Scholar 

  11. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  Google Scholar 

  12. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

  13. Chouhan, V., et al.: A novel transfer learning based approach for pneumonia detection in chest X-ray images. Appl. Sci. 10(2), 559 (2020)

    Article  Google Scholar 

  14. Chuquicusma, M.J.M., Hussein, S., Burt, J., Bagci, U.: How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis (2018)

    Google Scholar 

  15. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49

    Chapter  Google Scholar 

  16. Dar, S.U., Yurt, M., Karacan, L., Erdem, A., Erdem, E., Çukur, T.: Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans. Med. Imaging 38(10), 2375–2388 (2019)

    Article  Google Scholar 

  17. Glasser, M.F., et al.: The minimal preprocessing pipelines for the human connectome project. Neuroimage 80, 105–124 (2013)

    Article  Google Scholar 

  18. Hashmi, M.F., Katiyar, S., Keskar, A.G., Bokde, N.D., Geem, Z.W.: Efficient pneumonia detection in chest Xray images using deep transfer learning. Diagnostics 10(6), 417 (2020)

    Article  Google Scholar 

  19. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

    Google Scholar 

  20. Iqbal, T., Ali, H.: Generative adversarial network for medical images (MI-GAN). J. Med. Syst. 42(11), 1–11 (2018)

    Article  Google Scholar 

  21. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017)

  22. Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131 (2018)

    Article  Google Scholar 

  23. Kora Venu, S.: An ensemble-based approach by fine-tuning the deep transfer learning models to classify pneumonia from chest X-ray images. In: Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, pp. 390–401. INSTICC, SciTePress (2021). https://doi.org/10.5220/0010377403900401

  24. Kora Venu, S., Ravula, S.: Evaluation of deep convolutional generative adversarial networks for data augmentation of chest X-ray images. Future Internet 13(1) (2021). https://doi.org/10.3390/fi13010008. https://www.mdpi.com/1999-5903/13/1/8

  25. Korkinof, D., Rijken, T., O’Neill, M., Yearsley, J., Harvey, H., Glocker, B.: High-resolution mammogram synthesis using progressive generative adversarial networks. arXiv preprint arXiv:1807.03401 (2018)

  26. Liang, G., Zheng, L.: A transfer learning method with deep residual network for pediatric pneumonia diagnosis. Comput. Methods Programs Biomed. 187, 104964 (2020)

    Google Scholar 

  27. Liu, M.Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. arXiv preprint arXiv:1703.00848 (2017)

  28. Madani, A., Moradi, M., Karargyris, A., Syeda-Mahmood, T.: Chest X-ray generation and data augmentation for cardiovascular abnormality classification. In: Angelini, E.D., Landman, B.A. (eds.) Medical Imaging 2018: Image Processing, vol. 10574, pp. 415–420. International Society for Optics and Photonics, SPIE (2018). https://doi.org/10.1117/12.2293971

  29. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)

  30. Mittal, A., et al.: Detecting pneumonia using convolutions and dynamic capsule routing for chest X-ray images. Sensors 20(4), 1068 (2020)

    Article  Google Scholar 

  31. Nahid, A.A., et al.: A novel method to identify pneumonia through analyzing chest radiographs employing a multichannel convolutional neural network. Sensors 20(12), 3482 (2020)

    Article  Google Scholar 

  32. Pickhardt, P.J., et al.: Population-based opportunistic osteoporosis screening: validation of a fully automated CT tool for assessing longitudinal BMD changes. Br. J. Radiol. 92(1094), 20180726 (2019)

    Google Scholar 

  33. Qin, X., Bui, F.M., Nguyen, H.H.: Learning from an imbalanced and limited dataset and an application to medical imaging. In: 2019 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), pp. 1–6. IEEE (2019)

    Google Scholar 

  34. Rahman, T., et al.: Transfer learning with deep convolutional neural network (CNN) for pneumonia detection using chest X-ray. Appl. Sci. 10(9), 3233 (2020)

    Article  Google Scholar 

  35. Rajaraman, S., Candemir, S., Kim, I., Thoma, G., Antani, S.: Visualization and interpretation of convolutional neural network predictions in detecting pneumonia in pediatric chest radiographs. Appl. Sci. 8(10), 1715 (2018)

    Article  Google Scholar 

  36. Rajpurkar, P., et al.: CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)

  37. Salehinejad, H., Valaee, S., Dowdell, T., Colak, E., Barfett, J.: Generalization of deep neural networks for chest pathology classification in X-rays using generative adversarial networks. CoRR abs/1712.01636 (2017). http://arxiv.org/abs/1712.01636

  38. Sandfort, V., Yan, K., Pickhardt, P.J., Summers, R.M.: Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. Sci. Rep. 9(1), 1–9 (2019)

    Article  Google Scholar 

  39. Simpson, A.L., et al.: A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063 (2019)

  40. Stephen, O., Sain, M., Maduh, U.J., Jeong, D.U.: An efficient deep learning approach to pneumonia classification in healthcare. J. Healthcare Eng. 2019 (2019)

    Google Scholar 

  41. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. arXiv preprint arXiv:1602.07261 (2016)

  42. Van Essen, D.C., et al.: The WU-Minn human connectome project: an overview. Neuroimage 80, 62–79 (2013)

    Article  Google Scholar 

  43. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)

    Google Scholar 

  44. Welander, P., Karlsson, S., Eklund, A.: Generative adversarial networks for image-to-image translation on multi-contrast MR images-a comparison of CycleGAN and unit. arXiv preprint arXiv:1806.07777 (2018)

  45. Yan, K., Wang, X., Lu, L., Summers, R.M.: DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5(3), 036501 (2018)

    Google Scholar 

  46. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Harrisburg University of Science and Technology, Harrisburg, PA, 17101, USA

    Sagar Kora Venu

Authors
  1. Sagar Kora Venu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sagar Kora Venu .

Editor information

Editors and Affiliations

  1. LIACC, University of Porto, Porto, Portugal

    Ana Paula Rocha

  2. ICREA, Institute of Evolutionary Biology, Barcelona, Barcelona, Spain

    Luc Steels

  3. Leiden University, Leiden, The Netherlands

    Jaap van den Herik

Appendix

Appendix

Fig. 9.
figure 9

Inception - A Block.

Full size image
Fig. 10.
figure 10

InceptionResNet - A Block.

Full size image
Fig. 11.
figure 11

InceptionResNet - B Block.

Full size image
Fig. 12.
figure 12

InceptionResNet - C Block.

Full size image
Fig. 13.
figure 13

Reduction - A Block.

Full size image
Fig. 14.
figure 14

Reduction - B Block.

Full size image

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kora Venu, S. (2022). Improving the Generalization of Deep Learning Classification Models in Medical Imaging Using Transfer Learning and Generative Adversarial Networks. In: Rocha, A.P., Steels, L., van den Herik, J. (eds) Agents and Artificial Intelligence. ICAART 2021. Lecture Notes in Computer Science(), vol 13251. Springer, Cham. https://doi.org/10.1007/978-3-031-10161-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10161-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10160-1

  • Online ISBN: 978-3-031-10161-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation