Skip to main content
SpringerLink
Log in

Brain tumor detection with mRMR-based multimodal fusion of deep learning from MR images using Grad-CAM

  • Research
  • Published:
Iran Journal of Computer Science Aims and scope Submit manuscript

Abstract

The brain tumor is considered a deadly cancer disease that can be seen among people of almost all ages. Early diagnosis and treatment by radiologists are very important in terms of diagnosing an average of 14,000 people with brain tumors per year. Magnetic resonance imaging (MRI) is the best technical expert used for tumor detection via computer-aided systems. However, due to the complex tissue characteristics of a large number of images, manual examination with traditional methods may cause various errors. In this study, a hybrid convolutional neural network (CNN) method based on minimum-redundancy maximum-relevance (mRMR) is proposed, which aims to classify brain tumors over brain MRI with machine learning (ML) algorithms. According to the classification results, DarkNet53, EfficientNet-B0, and DenseNet201 pre-trained CNN architectures, which are three architectures that give the best results as feature extractors, were used as hybrids among 10 different deep learning (DL) architectures. By means of these CNN architectures, the features trained on the features obtained by Gradient-weighted Class Activation Mapping (Grad-CAM) are concatenated. The mRMR method has been used to optimize all concatenated features. Then, the optimized features have classified with SVM, KNN, and Ensemble algorithms. DarkNet53, EfficientNet-B0, and DenseNet201 were used as feature extractors and three different machine learning classifiers were used as classifiers. As a result, the success rate of classification of brain MR images using derived feature vectors has been revealed as 99.6% accuracy with the SVM classifier. According to the experimental results, the use of a combination of feature selection approaches and CNN models helped to successfully classify brain MR images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The Br35H :: Brain Tumor Detection 2020 dataset used in this paper is available from the following link https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection.

References

  1. Khan, A.H., Abbas, S., Khan, M.A., Farooq, U., Khan, W.A., Siddiqui, S.Y., Ahmad, A.: Intelligent model for brain tumor identification using deep learning. Appl. Comput. Intell. Soft Comput. 2022, 1–10 (2022). https://doi.org/10.1155/2022/8104054

    Article  Google Scholar 

  2. Kibriya, H., Amin, R., Alshehri, A. H., Masood, M., Alshamrani, S. S., lshehri, A.: A novel and effective brain tumor classification model using deep feature fusion and famous machine learning classifiers. Comput. Intell. Neurosci. 2022, 1–15 (2022). https://doi.org/10.1155/2022/7897669

  3. Arif, M., Ajesh, F., Shamsudheen, S., Geman, O., Izdrui, D., Vicoveanu, D.: Brain tumor detection and classification by MRI using biologically inspired orthogonal wavelet transform and deep learning techniques. J. Healthcare Eng. 2022, 1–18 (2022). https://doi.org/10.1155/2022/2693621

  4. Abd El Kader, I., Xu, G., Shuai, Z., Saminu, S., Javaid, I., Salim Ahmad, I.: Differential deep convolutional neural network model for brain tumor classification. Brain Sci. 11(3), 352 (2021)

    Article  Google Scholar 

  5. Gu, X., Shen, Z., Xue, J., Fan, Y., Ni, T.: Brain tumor MR image classification using convolutional dictionary learning with local constraint. Front. Neurosci. 15, 679847 (2021)

    Article  Google Scholar 

  6. Louis, D.N., Perry, A., Reifenberger, G., Von Deimling, A., Figarella-Branger, D., Cavenee, W.K., et al.: The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131(6), 803–820 (2016)

    Article  Google Scholar 

  7. Amin, J., Sharif, M., Yasmin, M., Fernandes, S.L.: A distinctive approach in brain tumor detection and classification using MRI. Pattern Recognit. Lett. 139, 118–127 (2020)

    Article  Google Scholar 

  8. Zeineldin, R. A., Karar, M. E., Elshaer, Z., Coburger, J., Wirtz, C. R., Burgert, O., Mathis-Ullrich, F.: Explainability of deep neural networks for MRI analysis of brain tumors. Int. J. Comput. Assisted Radiol. Surg. 17(9), 1673–1683 (2022)

  9. Patel, J., Doshi, K.: A study of segmentation methods for detection of tumor in brain MRI. Adv. Electron. Electr. Eng. 4(3), 279–284 (2014)

    Google Scholar 

  10. Bunevicius, A., Schregel, K., Sinkus, R., Golby, A., Patz, S.: MR elastography of brain tumors. NeuroImage: Clin. 25, 102109 (2020)

    Article  Google Scholar 

  11. Ari, A., Alpaslan, N., & Hanbay, D.: Computer-aided tumor detection system using brain MR images. In: 2015 Medical Technologies National Conference (TIPTEKNO), pp. 1–4. IEEE (2015)

  12. Mzoughi, H., Njeh, I., Wali, A., Slima, M.B., BenHamida, A., Mhiri, C., Mahfoudhe, K.B.: Deep multi-scale 3D convolutional neural network (CNN) for MRI gliomas brain tumor classification. J. Digit. Imaging 33(4), 903–915 (2020)

    Article  Google Scholar 

  13. Salçin, K.: Detection and classification of brain tumours from MRI images using faster R-CNN. Tehnički glasnik 13(4), 337–342 (2019)

    Article  Google Scholar 

  14. Mohsen, H., El-Dahshan, E.S.A., El-Horbaty, E.S.M., Salem, A.B.M.: Classification using deep learning neural networks for brain tumors. Future Comput. Inform. J. 3(1), 68–71 (2018)

    Article  Google Scholar 

  15. Maqsood, S., Damasevicius, R., Shah, F.M.: An efficient approach for the detection of brain tumor using fuzzy logic and U-NET CNN classification. In: International Conference on Computational Science and Its Applications, pp. 105–118. Springer, Cham (2021)

  16. Sarkar, S., Kumar, A., Chakraborty, S., Aich, S., Sim, J.S., Kim, H.C.: A CNN based approach for the detection of brain tumor using MRI scans. Test Eng. Manag. 83, 16580–16586 (2020)

    Google Scholar 

  17. Afshar, P., Mohammadi, A., Plataniotis, K. N.: Brain tumor type classification via capsule networks. In: 2018 25th IEEE international conference on image processing (ICIP) Athens, Greece, pp. 3129–3133. IEEE (2018)

  18. Khawaldeh, S., Pervaiz, U., Rafiq, A., Alkhawaldeh, R.S.: Noninvasive grading of glioma tumor using magnetic resonance imaging with convolutional neural networks. Appl. Sci. 8(1), 27 (2017)

    Article  Google Scholar 

  19. Kokila, B., Devadharshini, M.S., Anitha, A., Sankar, S.A.: Brain tumor detection and classification using deep learning techniques based on MRI images. In: Journal of Physics: Conference Series, vol. 1916, no. 1, p. 012226. IOP Publishing, Bristol (2021)

  20. Shahzadi, I., Tang, T. B., Meriadeau, F., Quyyum, A.: CNN-LSTM: cascaded framework for brain tumour classification. In: 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES) Sarawak, Malaysia, pp. 633–637. IEEE (2018)

  21. Sajjad, M., Khan, S., Muhammad, K., Wu, W., Ullah, A., Baik, S.W.: Multi-grade brain tumor classification using deep CNN with extensive data augmentation. J. Comput. Sci. 30, 174–182 (2019)

    Article  Google Scholar 

  22. Deepak, S., Ameer, P.M.: Brain tumor classification using deep CNN features via transfer learning. Comput. Biol. Med. 111, 103345 (2019)

    Article  Google Scholar 

  23. Noreen, N., Palaniappan, S., Qayyum, A., Ahmad, I., Alassafi, M.O.: Brain tumor classification based on fine-tuned models and the ensemble method. Comput. Mater. Contin. 67(3), 3967–3982 (2021)

    Google Scholar 

  24. Pashaei, A., Sajedi, H., Jazayeri, N.: Brain tumor classification via convolutional neural network and extreme learning machines. In: 2018 8th International conference on computer and knowledge engineering (ICCKE) Mashhad, Iran, pp. 314–319. IEEE (2018)

  25. Swati, Z.N.K., Zhao, Q., Kabir, M., Ali, F., Ali, Z., Ahmed, S., Lu, J.: Brain tumor classification for MR images using transfer learning and fine-tuning. Comput. Med. Imaging Graph. 75, 34–46 (2019)

    Article  Google Scholar 

  26. Saxena, P., Maheshwari, A., Maheshwari, S.: Predictive modeling of brain tumor: a deep learning approach. In: Sharma, M.K., Dhaka, V.S., Perumal, T., Dey, N., Tavares, J.M.R.S. (eds.) Innovations in computational intelligence and computer vision. Advances in intelligent systems and computing, vol. 1189. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-6067-5_30

    Chapter  Google Scholar 

  27. Anaraki, A.K., Ayati, M., Kazemi, F.: Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms. Biocybern. Biomed. Eng. 39(1), 63–74 (2019)

    Article  Google Scholar 

  28. Hamada, A.: Br35H :: Brain Tumor Detection 2020. Available online: [15 July 2022]. https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection

  29. Chakrabarty, N.: Brain MRI Images for Brain Tumor Detection 2019. Available online: [15 July 2022]. https://www.kaggle.com/datasets/navoneel/brain-mri-images-for-brain-tumor-detection

  30. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp. 618–626. Venice, Italy, October 22-29, 2017 (2017)

  31. Selvaraju, R. R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., Batra, D.: Grad-CAM: Why did you say that?. arXiv preprint. arXiv:1611.07450 (2016)

  32. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  33. Mahendran, A., Vedaldi, A.: Visualizing deep convolutional neural networks using natural pre-images. Int. J. Comput. Vis. 120(3), 233–255 (2016)

    Article  MathSciNet  Google Scholar 

  34. Wang, H., Zhang, F., & Wang, L.: Fruit classification model based on improved Darknet53 convolutional neural network. In: 2020 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) Vientiane, Laos, pp. 881–884. IEEE (2020)

  35. Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International conference on machine learning, California, pp. 6105–6114. PMLR (2019)

  36. Yu, X., Zeng, N., Liu, S., Zhang, Y.D.: Utilization of DenseNet201 for diagnosis of breast abnormality. Mach. Vis. Appl. 30(7), 1135–1144 (2019)

    Article  Google Scholar 

  37. Banerjee, C., Mukherjee, T., Pasiliao Jr, E.: An empirical study on generalizations of the ReLU activation function. In: Proceedings of the 2019 ACM Southeast Conference, Association for Computing Machinery, pp. 164–167. New York, United States (2019). https://doi.org/10.1145/3299815.3314450

  38. Özbay, E., Çinar, A., Özbay, F.A.: 3D human activity classification with 3D zernike moment based convolutional, LSTM-deep neural networks. Traitement du Signal 38(2), 269–280 (2021)

    Article  Google Scholar 

  39. Tantisatirapong, S., Davies, N.P., Rodriguez, D., Abernethy, L., Auer, D.P., Clark, C.A., et al.: Magnetic resonance texture analysis: optimal feature selection in classifying child brain tumors. In: XIII Mediterranean Conference on Medical and Biological Engineering and Computing 2013, pp. 309–312. Springer, Cham (2014)

  40. Gao, W., Hu, L., Zhang, P.: Feature redundancy term variation for mutual information-based feature selection. Appl. Intell. 50(4), 1272–1288 (2020)

    Article  Google Scholar 

  41. Zhao, J., Meng, Z., Wei, L., Sun, C., Zou, Q., Su, R.: Supervised brain tumor segmentation based on gradient and context-sensitive features. Front. Neurosci. 13, 144 (2019)

    Article  Google Scholar 

  42. Khan, A. N., Nazarian, H., Golilarz, N. A., Addeh, A., Li, J. P., Khan, G. A.: Brain tumor classification using efficient deep features of MRI scans and support vector machine. In: 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) Chengdu, China, pp. 314–318. IEEE (2020)

  43. Huda, S., Yearwood, J., Jelinek, H.F., Hassan, M.M., Fortino, G., Buckland, M.: A hybrid feature selection with ensemble classification for imbalanced healthcare data: a case study for brain tumor diagnosis. IEEE Access 4, 9145–9154 (2016)

    Article  Google Scholar 

  44. Özbay, E.: An active deep learning method for diabetic retinopathy detection in segmented fundus images using artificial bee colony algorithm. Artif. Intell. Rev. (2022). https://doi.org/10.1007/s10462-022-10231-3

    Article  Google Scholar 

  45. Özbay, E.: Transformatör-Tabanlı Evrişimli Sinir Ağı Modeli Kullanarak Twitter Verisinde Saldırganlık Tespiti. Konya Mühendislik Bilimleri Dergisi 10(4), 986–1001 (2022)

    Google Scholar 

  46. Özbay, F.A., Özbay, E.: A new approach for gender detection from voice data: feature selection with optimization methods. J. Fac. Eng. Archit. Gazi Univ. 38(2), 1179–1192 (2023)

    Google Scholar 

  47. Nabizadeh, N., Kubat, M.: Brain tumors detection and segmentation in MR images: gabor wavelet vs. statistical features. Comput. Electr. Eng. 45, 286–301 (2015)

    Article  Google Scholar 

  48. Ismael, M. R., Abdel-Qader, I.: Brain tumor classification via statistical features and back-propagation neural network. In: 2018 IEEE international conference on electro/information technology (EIT), Rochester, MI, USA, pp. 0252-0257. IEEE (2018)

  49. Özbay, F.A., Özbay, E.: Feature selection method with atom search optimization for detection of diabetic retinopathy. J 9(16), 88–104 (2022)

    Google Scholar 

  50. Özbay, E., Cinar, A.C.: A comparative study of object classification methods using 3D zernike moment on 3D point clouds. Traitement du Signal 36(6), 549–555 (2019)

    Article  Google Scholar 

Download references

Funding

There is no funding source for this article.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Software Engineering, Firat University, 23119, Elazig, Turkey

    Feyza Altunbey Özbay

  2. Faculty of Engineering, Computer Engineering, Firat University, 23119, Elazig, Turkey

    Erdal Özbay

Authors
  1. Feyza Altunbey Özbay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erdal Özbay
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study’s conception and design. Software coding, methodology, validation, and visualization were performed by EÖ. Writing, editing, and review of the study, and data curation were performed by FAÖ.

Corresponding author

Correspondence to Feyza Altunbey Özbay.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

This article does not contain any data, or other information from studies or experimentation, with the involvement of human or animal subjects.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Özbay, F.A., Özbay, E. Brain tumor detection with mRMR-based multimodal fusion of deep learning from MR images using Grad-CAM. Iran J Comput Sci 6, 245–259 (2023). https://doi.org/10.1007/s42044-023-00137-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42044-023-00137-w

Keywords

Search

Navigation