Abstract
In this paper, we present a multi-scale attention assembler network (MSAA-Net) tailored for multi-scale pathological image analysis. The proposed approach identifies essential features by examining pathological images across different resolutions (scales) and adaptively determines which scales and spatial regions predominantly influence the classification. Specifically, our approach incorporates a two-stage feature integration strategy. Initially, the network allocates the attention scores to relevant local regions of each scale and then refines the attention scores for each scale as a whole. To facilitate the training of the MSAA-Net, we employ the technique of multiple instance learning (MIL), which enables us to train the classification model using the pathologist’s daily diagnoses of whole slide images without requiring detailed annotation (i.e., pixel-level labels), thereby minimizing annotation effort. We evaluate the effectiveness of the proposed method by conducting classification experiments using two distinct sets of pathological image data. We conduct a comparative analysis of the attention maps generated by these methods. The results demonstrate that the proposed method outperforms state-of-the-art multiscale methods, confirming the effectiveness of MSAA-Net in classifying multi-scale pathological images.
T. Yoshida, K. Uehara—These authors contributed equally to this manuscript.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albertina, B., et al.: The cancer genome atlas lung adenocarcinoma collection [TCGA-LUAD]. The Cancer Imaging Archive (2016). https://doi.org/10.7937/K9/TCIA.2016.JGNIHEP5
Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. Adv. Neural Inf. Process. Syst. 15, 577–584. MIT Press (2002)
Araújo, T., et al.: Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6), e0177544 (2017)
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Chen, R.J., et al.: Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 16144–16155. IEEE (2022)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1), 31–71 (1997)
Feng, J., Zhou, Z.H.: Deep MIML network. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 1884–1890. MIT Press (2017)
Hashimoto, N., et al.: Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with unannotated histopathological images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3852–3861. IEEE (2020)
Herrera, F., et al.: Multi-instance Regression. In: Multiple Instance Learning, pp. 127–140. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47759-6_6
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: Proceedings of the 35th International Conference on Machine Learning, vol. 80, pp. 2127–2136. PMLR (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 14318–14328. IEEE (2021)
Li, J., et al.: A multi-resolution model for histopathology image classification and localization with multiple instance learning. Comput. Biol. Med. 131, 104253 (2021)
Liu, Y., et al.: Detecting cancer metastases on gigapixel pathology images. arXiv:1703.02442 (2017)
Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. Adv. Neural Inf. Process. Syst. 10, 570–576. MIT Press (1997)
Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713–1721. IEEE (2015)
Shao, Z., et al.: TransMIL: transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural Inf. Process. Syst. 34, 2136–2147. MIT Press (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representations (2015)
Tellez, D., Litjens, G., van der Laak, J., Ciompi, F.: Neural image compression for gigapixel histopathology image analysis. IEEE Trans. Pattern Anal. Mach. Intell. 43(2), 567–578 (2021)
Yoshida, T., Uehara, K., Sakanashi, H., Nosato, H., Murakawa, M.: Multi-scale feature aggregation based multiple instance learning for pathological image classification. In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - ICPRAM, pp. 619–628. Scitepress (2023)
Zhouhan, L., et al.: A structured self-attentive sentence embedding. In: 5th International Conference on Learning Representations, ICLR 2017, pp. 24–26 (2017)
Acknowledgements
The authors thank Prof. Junya Fukuoka and Dr. Wataru Uegami from Nagasaki University Graduate School of Biomedical Sciences for providing the dataset and medical comments. Computational resource of AI Bridging Cloud Infrastructure (ABCI) provided by the National Institute of Advanced Industrial Science and Technology (AIST) was used. This study is based on results obtained from the project JPNP20006, commissioned by the New Energy and Industrial Technology Development Organization (NEDO). This study was approved by the Ethics Committee (Institutional Review Board) of Nagasaki University Hospital (No. 19081929-2) and the National Institute of Advanced Industrial Science and Technology (No. Hi2019-312) and complied with all the relevant ethical regulations. The results here are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yoshida, T., Uehara, K., Sakanashi, H., Nosato, H., Murakawa, M. (2024). MSAA-Net: Multi-Scale Attention Assembler Network Based on Multiple Instance Learning for Pathological Image Analysis. In: De Marsico, M., Di Baja, G.S., Fred, A. (eds) Pattern Recognition Applications and Methods. ICPRAM 2023. Lecture Notes in Computer Science, vol 14547. Springer, Cham. https://doi.org/10.1007/978-3-031-54726-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-54726-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54725-6
Online ISBN: 978-3-031-54726-3
eBook Packages: Computer ScienceComputer Science (R0)
