Abstract
Breast cancer is the most commonly diagnosed cancer, and accurate molecular subtype prediction plays a crucial role in determining treatment strategies. While immunohistochemistry (IHC) is commonly used for molecular subtype diagnosis, it suffers from cost and labor limitations. The prediction of molecular subtypes using hematoxylin and eosin (H&E) stained slides has gained importance. However, the task is challenged by limited samples, weak annotations, and strong tumor heterogeneity. This paper proposes a scalable framework, BM-SMIL, for molecular subtype prediction of breast cancer based on self-supervised pretraining and multi-instance learning. Firstly, a self-supervised pretraining framework utilizing multi-scale knowledge distillation is introduced to obtain a representative patch encoder. Then, an attention-based instance selection strategy is employed to filter out noise instances. Finally, a Transformer integrated with subtype contrastive loss is proposed for effective aggregation and WSI-level prediction. Experimental results on the dataset from cooperative hospital demonstrate the effectiveness of our proposed framework. The BM-SMIL framework has the potential to enhance molecular subtype prediction performance and can be extended to other pathology image classification tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Pusztai, L., Mazouni, C., Anderson, K., et al.: Molecular classification of breast cancer: limitations and potential. Oncologist 11(8), 868–877 (2006)
Sengal, A.T., Haj-Mukhtar, N.S., Elhaj, A.M., et al.: Immunohistochemistry defined subtypes of breast cancer in 678 Sudanese and Eritrean women; hospitals based case series. BMC Cancer 17(1), 1–9 (2017)
Rawat, R.R., Ortega, I., Roy, P., et al.: Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images. Sci. Rep. 10(1), 1–13 (2020)
Jaber, M.I., Song, B., Taylor, C., et al.: A deep learning image-based intrinsic molecular subtype classifier of breast tumors reveals tumor heterogeneity that may affect survival. Breast Cancer Res. 22(1), 1–10 (2020)
Liu, H., Xu, W.D., Shang, Z.H., et al.: Breast cancer molecular subtype prediction on pathological images with discriminative patch selection and multi-instance learning. Front. Oncol. 12, 858453 (2022)
Han, B., Yao, Q., Yu, X., et al.: Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Breunig, M.M., Kriegel, H.P., Ng, R.T., et al.: LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
He, K., Fan, H., Wu, Y., et al.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Chen, T., Kornblith, S., Norouzi, M., et al.: A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning, vol. 119, pp. 1597–1607. JMLR.org, (2020)
Caron, M., Touvron, H., Misra, I., et al.: Emerging properties in self-supervised vision transformers. In: ICCV 2021 - International Conference on Computer Vision, p. 1 (2021)
Grill, J.B., Strub, F., Altché, F., et al.: Bootstrap your own latent-a new approach to self-supervised learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 21271–21284 (2020)
He, K., Chen, X., Xie, S., et al.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)
Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1422–1430 (2015)
Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14313–14323 (2021)
Li, J., Lin, T., Xu, Y.: SSLP: spatial guided self-supervised learning on pathological images. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 3–12. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_1
Wang, X., Yang, S., Zhang, J., et al.: Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022)
Lerousseau, M., et al.: Weakly supervised multiple instance learning histopathological tumor segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 470–479. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_45
Campanella, G., Hanna, M., Geneslaw, L., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1 (2019)
Tomita, N., Abdollahi, B., Wei, J., et al.: Attention-based deep neural networks for detection of cancerous and precancerous esophagus tissue on histopathological slides. JAMA Netw. OpenNetw. Open 2(11), e1914645 (2019)
Hashimoto, N., Fukushima, D., Koga, R., et al.: Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with unannotated histopathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3852–3861 (2020)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: Dy, J., Krause, A.: Proceedings of the 35th International Conference on Machine Learning, vol. 80, pp. 2127–2136. PMLR (2018)
Shao, Z., Bian, H., Chen, Y., et al.: TransMIL: transformer based correlated multiple instance learning for whole slide image classification. In: Advances in Neural Information Processing Systems (2022)
Lu, M.Y., Williamson, D.F., Chen, T.Y., et al.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5(6), 555–570 (2021)
Yu, S., et al.: Mil-vt: Multiple instance learning enhanced vision transformer for fundus image classification. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 45–54. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_5
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 815–823 (2015)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern.Cybern. 9(1), 62–66 (1979)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2022)
Acknowledgement
This work was supported by the National Natural Science Foundation of China (62276250).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shang, Z., Liu, H., Wang, K., Wang, X. (2023). BM-SMIL: A Breast Cancer Molecular Subtype Prediction Framework from H&E Slides with Self-supervised Pretraining and Multi-instance Learning. In: Qin, W., Zaki, N., Zhang, F., Wu, J., Yang, F., Li, C. (eds) Computational Mathematics Modeling in Cancer Analysis. CMMCA 2023. Lecture Notes in Computer Science, vol 14243. Springer, Cham. https://doi.org/10.1007/978-3-031-45087-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-45087-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45086-0
Online ISBN: 978-3-031-45087-7
eBook Packages: Computer ScienceComputer Science (R0)
