Abstract
The high complexity of deep learning models is associated with the difficulty of explaining what evidence they recognize as correlating with specific disease labels. This information is critical for building trust in models and finding their biases. Until now, automated deep learning visualization solutions have identified regions of images used by classifiers, but these solutions are too coarse, too noisy, or have a limited representation of the way images can change. We propose a novel method for formulating and presenting spatial explanations of disease evidence, called deformation field interpretation with generative adversarial networks (DeFI-GAN). An adversarially trained generator produces deformation fields that modify images of diseased patients to resemble images of healthy patients. We validate the method studying chronic obstructive pulmonary disease (COPD) evidence in chest x-rays (CXRs) and Alzheimer’s disease (AD) evidence in brain MRIs. When extracting disease evidence in longitudinal data, we show compelling results against a baseline producing difference maps. DeFI-GAN also highlights disease biomarkers not found by previous methods and potential biases that may help in investigations of the dataset and of the adopted learning methods.
Keywords
Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The code is available at https://github.com/ricbl/defigan.
- 2.
IRB_00104019, PI: Schroeder MD.
- 3.
- 4.
VA-GAN’s original preprocessing code, list of subjects, and TensorFlow implementation are available in https://github.com/baumgach/vagan-code/.
References
About ADNI (2017). http://adni.loni.usc.edu/about/. Accessed 11 Mar 2020
Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for deep neural networks. In: ICLR (2018)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: ICML (2017)
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1–46 (2015). https://doi.org/10.1371/journal.pone.0130140
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: CVPR (2018)
Baumgartner, C.F., Koch, L.M., Tezcan, K.C., Ang, J.X.: Visual feature attribution using Wasserstein GANs. In: CVPR (2018). https://doi.org/10.1109/CVPR.2018.00867
Bigolin Lanfredi, R., Schroeder, J.D., Vachet, C., Tasdizen, T.: Adversarial regression training for visualizing the progression of chronic obstructive pulmonary disease with chest x-rays. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 685–693. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_76
Ciccarese, F., et al.: Saber-sheath trachea as a marker of severe airflow obstruction in chronic obstructive pulmonary disease. Radiol. Med. (Torino) 119(2), 90–96 (2013). https://doi.org/10.1007/s11547-013-0318-3
Dalca, A., Rakic, M., Guttag, J., Sabuncu, M.: Learning conditional deformable templates with convolutional networks. In: NeurIPS (2019)
Feng, X., Yang, J., Laine, A.F., Angelini, E.D.: Discriminative localization in CNNs for weakly-supervised segmentation of pulmonary nodules. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 568–576. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_65
Fonov, V., Evans, A.C., Botteron, K., Almli, C.R., McKinstry, R.C., Collins, D.L.: Unbiased average age-appropriate atlases for pediatric studies. NeuroImage 54(1), 313–327 (2011). https://doi.org/10.1016/j.neuroimage.2010.07.033
Fonov, V., Evans, A., McKinstry, R., Almli, C., Collins, D.: Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage 47, S102 (2009). https://doi.org/10.1016/S1053-8119(09)70884-5
Foster, W.L., et al.: The emphysemas: radiologic-pathologic correlations. RadioGraphics 13(2), 311–328 (1993). https://doi.org/10.1148/radiographics.13.2.8460222
Goodfellow, I., et al.: Generative adversarial nets. In: NeurIPS (2014)
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of Wasserstein GANs. In: NeurIPS (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016). https://doi.org/10.1109/CVPR.2016.90
Hägele, M., et al.: Resolving challenges in deep learning-based analyses of histopathological images using explanation methods. CoRR abs/1908.06943 (2019)
Iglesias, J.E., Liu, C., Thompson, P.M., Tu, Z.: Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trans. Med. Imag. 30(9), 1617–1634 (2011). https://doi.org/10.1109/TMI.2011.2138152
Jaramillo, J.D., et al.: Reduced bone density and vertebral fractures in smokers. Men and COPD patients at increased risk. Ann. Am. Thorac. Soc. 12(5), 648–656 (2015). https://doi.org/10.1513/AnnalsATS.201412-591OC
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Langlotz, C.P., et al.: A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 NIH/RSNA/ACR/the academy workshop. Radiology 291, 781–791 (2019)
Ledig, C., Schuh, A., Guerrero, R., Heckemann, R., Rueckert, D.: Structural brain imaging in Alzheimer’s disease and mild cognitive impairment: biomarker analysis and shared morphometry database. Sci. Rep. 8 (2018). https://doi.org/10.1038/s41598-018-29295-9
Lee, J., Ryan, J., Andreescu, C., Aizenstein, H., Lim, H.K.: Brainstem morphological changes in Alzheimer’s disease. NeuroReport 26, 411–415 (2015). https://doi.org/10.1097/WNR.0000000000000362
Lowekamp, B., Chen, D., Ibanez, L., Blezek, D.: The design of SimpleITK. Front. Neuroinform. 7, 45 (2013). https://doi.org/10.3389/fninf.2013.00045
Martin-Harris, B.: Optimal patterns of care in patients with chronic obstructive pulmonary disease. Semin. Speech Lang. 21(04), 0311–0322 (2000). https://doi.org/10.1055/s-2000-8384
Morais, N., Cruz, J., Marques, A.: Posture and mobility of the upper body quadrant and pulmonary function in COPD: an exploratory study. Braz. J. Phys. Therapy 20, 345–354 (2016). https://doi.org/10.1590/bjpt-rbf.2014.0162
Rajpurkar, P., Irvin, J., et al.: CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. CoRR abs/1711.05225 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992). https://doi.org/10.1016/0167-2789(92)90242-F
Scahill, R.I., Schott, J.M., Stevens, J.M., Rossor, M.N., Fox, N.C.: Mapping the evolution of regional atrophy in Alzheimer’s disease: unbiased analysis of fluid-registered serial MRI. PNAS 99(7), 4703–4707 (2002). https://doi.org/10.1073/pnas.052587399
Smith, S.M.: Fast robust automated brain extraction. Hum. Brain Mapp. 17(3), 143–155 (2002). https://doi.org/10.1002/hbm.10062
Smith, S.M., et al.: Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage 23, S208–S219 (2004). https://doi.org/10.1016/j.neuroimage.2004.07.051
Tabatabaei Jafari, H., Walsh, E., Shaw, M., Cherbuin, N.: The cerebellum shrinks faster than normal ageing in Alzheimer’s disease but not in mild cognitive impairment. Hum. Brain Mapp. 38 (2017). https://doi.org/10.1002/hbm.23580
Tustison, N.J., et al.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imag. 29(6), 1310–1320 (2010). https://doi.org/10.1109/TMI.2010.2046908
Washko, G.R.: Diagnostic imaging in COPD. Semin. Resp. Crit. Care 31(3), 276–285 (2010). https://doi.org/10.1055/s-0030-1254068
Xiao, C., Zhu, J.Y., Li, B., He, W., Liu, M., Song, D.: Spatially transformed adversarial examples. In: ICLR (2018)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR (2016). https://doi.org/10.1109/CVPR.2016.319
Zhou, Y., et al.: The association between BMI and COPD: the results of two population-based studies in Guangzhou, China. COPD 10(5), 567–572 (2013). https://doi.org/10.3109/15412555.2013.781579
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bigolin Lanfredi, R., Schroeder, J.D., Vachet, C., Tasdizen, T. (2020). Interpretation of Disease Evidence for Medical Images Using Adversarial Deformation Fields. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12262. Springer, Cham. https://doi.org/10.1007/978-3-030-59713-9_71
Download citation
DOI: https://doi.org/10.1007/978-3-030-59713-9_71
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59712-2
Online ISBN: 978-3-030-59713-9
eBook Packages: Computer ScienceComputer Science (R0)