Skip to main content

Advertisement

SpringerLink
Log in

3D CNN-based Deep Learning Model-based Explanatory Prognostication in Patients  with Multiple Myeloma using Whole-body MRI

  • Original Paper
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Although magnetic resonance imaging (MRI) data of patients with multiple myeloma (MM) are used to predict prognosis, few reports have applied artificial intelligence (AI) techniques for this purpose. We aimed to analyze whole-body diffusion-weighted MRI data using three-dimensional (3D) convolutional neural networks (CNNs) and Gradient-weighted Class Activation Mapping (Grad-CAM), an explainable AI, to predict prognosis and explore the factors involved in prediction. We retrospectively analyzed the MRI data of a total of 142 patients with MM obtained from two medical centers. We defined the occurrence of progressive disease after MRI evaluation within 12 months as a poor prognosis and constructed a 3D CNN-based deep learning model to predict prognosis. Images from 111 cases were used as the training and internal validation data; images from 31 cases were used as the external validation data. Internal validation of the AI model with stratified 5-fold cross-validation resulted in a significant difference in progression-free survival (PFS) between good and poor prognostic cases (2-year PFS, 91.2% versus [vs.] 61.1%, P = 0.0002). The AI model clearly stratified good and poor prognostic cases in the external validation cohort (2-year PFS, 92.9% vs. 55.6%, P = 0.004), with an area under the receiver operating characteristic curve of 0.804. According to Grad-CAM, the MRI signals of the spleen and bones of the vertebrae and pelvis contributed to prognosis prediction. This study is the first to show that image analysis of whole-body MRI using a 3D CNN without any other clinical data is effective in predicting the prognosis of patients with MM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The training and validation datasets analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Greipp, P. R., San Miguel, J., Durie, B. G., Crowley, J. J., Barlogie, B., Blade, J., et al. International staging system for multiple myeloma. J. Clin. Oncol. 23:3412–3420, 2005.

    Article  PubMed  Google Scholar 

  2. Palumbo, A., Avet-Loiseau, H., Oliva, S., Lokhorst, H. M., Goldschmidt, H., Rosinol, L., et al. Revised International Staging System for Multiple Myeloma: A Report From International Myeloma Working Group. J. Clin. Oncol. 33:2863–2869, 2015.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. D’Agostino, M., Cairns, D. A., Lahuerta, J. J., Wester, R., Bertsch, U., Waage, A., et al. Second Revision of the International Staging System (R2-ISS) for Overall Survival in Multiple Myeloma: A European Myeloma Network (EMN) Report Within the HARMONY Project. J. Clin. Oncol. 40:3406–3418, 2022.

    Article  PubMed  Google Scholar 

  4. Hillengass, J., Usmani, S., Rajkumar, S. V., Durie, B. G. M., Mateos, M. V., Lonial, S., et al. International myeloma working group consensus recommendations on imaging in monoclonal plasma cell disorders. Lancet Oncol. 20:e302-e12, 2019.

    Article  Google Scholar 

  5. Moreau, P., Attal, M., Caillot, D., Macro, M., Karlin, L., Garderet, L., et al. Prospective Evaluation of Magnetic Resonance Imaging and [(18)F]Fluorodeoxyglucose Positron Emission Tomography-Computed Tomography at Diagnosis and Before Maintenance Therapy in Symptomatic Patients With Multiple Myeloma Included in the IFM/DFCI 2009 Trial: Results of the IMAJEM Study. J. Clin. Oncol. 35:2911–2918, 2017.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Belotti, A., Ribolla, R., Cancelli, V., Villanacci, A., Angelini, V., Chiarini, M., et al. Predictive role of diffusion-weighted whole-body MRI (DW-MRI) imaging response according to MY-RADS criteria after autologous stem cell transplantation in patients with multiple myeloma and combined evaluation with MRD assessment by flow cytometry. Cancer Med. 10:5859–5865, 2021.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Hiremath, A., Shiradkar, R., Fu, P., Mahran, A., Rastinehad, A. R., Tewari, A., et al. An integrated nomogram combining deep learning, Prostate Imaging-Reporting and Data System (PI-RADS) scoring, and clinical variables for identification of clinically significant prostate cancer on biparametric MRI: a retrospective multicentre study. Lancet Digit. Health. 3:e445-e54, 2021.

    Google Scholar 

  8. Jayachandran Preetha, C., Meredig, H., Brugnara, G., Mahmutoglu, M. A., Foltyn, M., Isensee, F., et al. Deep-learning-based synthesis of post-contrast T1-weighted MRI for tumour response assessment in neuro-oncology: a multicentre, retrospective cohort study. Lancet Digit. Health. 3:e784-e94, 2021.

    Google Scholar 

  9. Xiao, W., Huang, X., Wang, J. H., Lin, D. R., Zhu, Y., Chen, C., et al. Screening and identifying hepatobiliary diseases through deep learning using ocular images: a prospective, multicentre study. Lancet Digit. Health. 3:e88-e97, 2021.

    PubMed  Google Scholar 

  10. Singh, S. P., Wang, L., Gupta, S., Goli, H., Padmanabhan, P., Gulyás, B. 3D deep learning on medical images: a review. Sensors. 20:5097, 2020.

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  11. Zhang, L., Wang, Q., Wu, X., Zhao, A., Feng, J., Zhang, H., et al. Baseline bone marrow ADC value of diffusion-weighted MRI: a potential independent predictor for progression and death in patients with newly diagnosed multiple myeloma. Eur. Radiol. 31:1843–1852, 2021.

    Article  PubMed  Google Scholar 

  12. Zhang, B., Bian, B., Zhang, Y., Zhang, L., Zhang, R., Wang, J. The Apparent Diffusion Coefficient of Diffusion-Weighted Whole-Body Magnetic Resonance Imaging Affects the Survival of Multiple Myeloma Independently. Front. Oncol. 12:780078, 2022.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Wennmann, M., Klein, A., Bauer, F., Chmelik, J., Grozinger, M., Uhlenbrock, C., et al. Combining Deep Learning and Radiomics for Automated, Objective, Comprehensive Bone Marrow Characterization From Whole-Body MRI: A Multicentric Feasibility Study. Invest. Radiol. 57:752–763, 2022.

    Article  CAS  PubMed  Google Scholar 

  14. Wennmann, M., Neher, P., Stanczyk, N., Kahl, K. C., Kachele, J., Weru, V., et al. Deep Learning for Automatic Bone Marrow Apparent Diffusion Coefficient Measurements From Whole-Body Magnetic Resonance Imaging in Patients With Multiple Myeloma: A Retrospective Multicenter Study. Invest. Radiol. 58:273–282, 2023.

    Article  CAS  PubMed  Google Scholar 

  15. Wennmann, M., Ming, W., Bauer, F., Chmelik, J., Klein, A., Uhlenbrock, C., et al. Prediction of Bone Marrow Biopsy Results From MRI in Multiple Myeloma Patients Using Deep Learning and Radiomics. Invest. Radiol. 2023.

  16. Li, Y., Liu, Y., Yin, P., Hao, C., Sun, C., Chen, L., et al. MRI-Based Bone Marrow Radiomics Nomogram for Prediction of Overall Survival in Patients With Multiple Myeloma. Front. Oncol. 11:709813, 2021.

    Article  PubMed  PubMed Central  Google Scholar 

  17. International Myeloma Working G. Criteria for the classification of monoclonal gammopathies, multiple myeloma and related disorders: a report of the International Myeloma Working Group. Br. J. Haematol. 121:749–757, 2003.

    Article  Google Scholar 

  18. Rajkumar, S. V., Dimopoulos, M. A., Palumbo, A., Blade, J., Merlini, G., Mateos, M. V., et al. International Myeloma Working Group updated criteria for the diagnosis of multiple myeloma. Lancet Oncol. 15:e538-48, 2014.

    Article  PubMed  Google Scholar 

  19. Hara, K., Kataoka, H., Satoh, Y. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 6546-6555, 2018.

  20. Selvaraju, R. R., M C, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE international conference on computer vision, pp 618–626, 2017.

  21. Van der Walt, S., Schönberge, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., et al. scikit-image: image processing in python. PeerJ. 2:e453, 2014.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Fluss, R., Faraggi, D., Reiser, B. Estimation of the youden index and its associated cutoff point. Biom. J. 47:458–472, 2005.

    Article  MathSciNet  PubMed  Google Scholar 

  23. Durie, B. G., Harousseau, J. L., Miguel, J. S., Blade, J., Barlogie, B., Anderson, K., et al. International uniform response criteria for multiple myeloma. Leukemia. 20:1467–1473, 2006.

    Article  CAS  PubMed  Google Scholar 

  24. Kanda, Y. Investigation of the freely available easy-to-use software ‘EZR’ for medical statistics. Bone Marrow Transplant. 48:452–458, 2013.

    Article  CAS  PubMed  Google Scholar 

  25. Wolleb, J., Sandkühler, R., Bieder, F., Barakovic, M., Hadjikhani, N., Papadopoulou, A., et al. Learn to Ignore: Domain Adaptation for Multi-Site MRI Analysis arXiv:211006803v3 [csCV], 2022.

  26. Martensson, G., Ferreira, D., Granberg, T., Cavallin, L., Oppedal, K., Padovani, A, et al. The reliability of a deep learning model in clinical out-of-distribution MRI data: A multicohort study. Med. Image Anal. 66:101714, 2020.

    Article  PubMed  Google Scholar 

  27. Wennmann, M., Thierjung, H., Bauer, F., Weru, V., Hielscher, T., Grozinger, M., et al. Repeatability and Reproducibility of ADC Measurements and MRI Signal Intensity Measurements of Bone Marrow in Monoclonal Plasma Cell Disorders: A Prospective Bi-institutional Multiscanner, Multiprotocol Study. Invest. Radiol. 57:272–2781, 2022.

    Article  CAS  PubMed  Google Scholar 

  28. Wennmann, M., Bauer, F., Klein, A., Chmelik, J., Grozinger, M., Rotkopf, L. T., et al. In Vivo Repeatability and Multiscanner Reproducibility of MRI Radiomics Features in Patients With Monoclonal Plasma Cell Disorders: A Prospective Bi-institutional Study. Invest. Radiol. 2023;58:253–64.

    Article  CAS  PubMed  Google Scholar 

  29. Nyul, L. G., Udupa, J. K. On standardizing the MR image intensity scale. Magn. Reson. Med. 42:1072–1081, 1999.

    Article  CAS  PubMed  Google Scholar 

  30. Kondrateva, E., Pominova, M., Popova, E., Sharaev, M., Bernstein, A., Burnaev, E. Domain shift in computer vision models for MRI data analysis: an overview, 2021.

  31. Ma, J. Histogram Matching Augmentation for Domain Adaptation with Application to Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Image Segmentation. arXiv:201213871.

  32. Terao, T., Machida, Y., Tateishi, U., Tsushima, T., Narita, K., Ikeda, D., et al. Association of loss of spleen visualization on whole-body diffusion-weighted imaging with prognosis and tumor burden in patients with multiple myeloma. Sci. Rep. 11:23978, 2021.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  33. Zamagni, E., Tacchetti, P., Cavo, M. Imaging in multiple myeloma: How? When? Blood. 133:644–651, 2019.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

This study was supported by the research fundings from the Mitani Foundation for Research and Development and the Japanese Society of Hematology Research Grant. The authors thank all physicians and staff members of the collaborating institutes and also Editage for a Language Editing Service.

Author information

Authors and Affiliations

  1. School of Electrical, Information and Communication Engineering, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan

    Kento Morita

  2. Institute of Liberal Arts and Science, Kanazawa University, Kanazawa, Japan

    Shigehiro Karashima

  3. Department of Hematology/Oncology, Kameda Medical Center, Kamogawa, Japan

    Toshiki Terao & Kosei Matsue

  4. Department of Hematology and Oncology, Okayama University Hospital, Okayama, Japan

    Toshiki Terao

  5. Department of Radiology, Kanazawa University, Kanazawa, Japan

    Kotaro Yoshida

  6. Division of Internal Medicine, Keiju Kanazawa Hospital, Kanazawa, Japan

    Takeshi Yamashita & Ryoichi Murata

  7. Department of Hematology, Ishikawa Central Prefectural Hospital, Kanazawa, Japan

    Takeshi Yoroidaka, Mikoto Tanabe & Takeharu Kotani

  8. Department of Hematology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan

    Takeshi Yoroidaka, Tatsuya Imi, Yoshitaka Zaimoku, Akiyo Yoshida, Hiroyuki Maruyama, Noriko Iwaki, Go Aoki, Toshihiro Miyamoto & Hiroyuki Takamatsu

  9. Department of Radiology, Kameda Medical Center, Kamogawa, Japan

    Youichi Machida

  10. Faculty of Transdisciplinary Sciences for Innovation, Institute of Transdisciplinary Sciences for Innovation, Kanazawa University, Kakumamachi, Kanazawa, Ishikawa, 920-1192, Japan

    Hidetaka Nambo & Hiroyuki Takamatsu

Authors
  1. Kento Morita
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigehiro Karashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Toshiki Terao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kotaro Yoshida
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takeshi Yamashita
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Takeshi Yoroidaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mikoto Tanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tatsuya Imi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yoshitaka Zaimoku
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Akiyo Yoshida
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hiroyuki Maruyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Noriko Iwaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Go Aoki
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Takeharu Kotani
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Ryoichi Murata
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Toshihiro Miyamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Youichi Machida
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Kosei Matsue
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Hidetaka Nambo
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Hiroyuki Takamatsu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Contributions: S.K., H.N. and H.T. designed the study; K.M. S.K., K.Y., H.N. and H.T. analyzed data and wrote the manuscript; T.T., T. Yamashita, T. Yoroidaka., M.T., T.I., Y.Z., A.Y., H.M., N.I., G.A., T.K., R.M., T.M. Y.M. and K.M. provided the patients data; and all authors interpreted the data and reviewed and approved the final manuscript.

Corresponding authors

Correspondence to Hidetaka Nambo or Hiroyuki Takamatsu.

Ethics declarations

Competing interests

H.T. received consulting fee from SRL, honoraria from Janssen Co. Ltd., Ono Pharmaceutical Co., Sanofi Pharmaceutical Co. and Bristol-Myers Squibb and research fund from Bristol-Myers Squibb. T.M. received honoraria from Takeda Pharmaceutical Co., Otsuka Pharmaceutical Co., Astellas Pharmaceutical Co., Janssen Co. Ltd., AbbVie Pharmaceutical Co. Ltd., and Sanofi Pharmaceutical Co. The remaining authors declare no competing interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morita, K., Karashima, S., Terao, T. et al. 3D CNN-based Deep Learning Model-based Explanatory Prognostication in Patients  with Multiple Myeloma using Whole-body MRI. J Med Syst 48, 30 (2024). https://doi.org/10.1007/s10916-024-02040-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-024-02040-8

Keywords

Search

Navigation