Skip to main content
SpringerLink
Log in

Prostate cancer grade using self-supervised learning and novel feature aggregator based on weakly-labeled gbit-pixel pathology images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Prostate cancer (PCa) is the second most common cancer in men worldwide. The Gleason score, determined by pathologists through microscopic examination of pathological tissue, is the most powerful prognostic indicator for patients diagnosed with prostate cancer. However, there is inter- and intra-observer variability among pathologists, and the scoring process imposes a significant workload on them. This study presents a deep learning model that utilizes gigapixel pathology images and slide-level labels for prostate cancer detection and Gleason grading. WSIs are first cropped into small patches, and a deep learning model trained using self-supervised learning is used to extract features from the patches. An attention-LSTM is then used to aggregate features and perform the final classification. Our pipeline for Gleason grading, which utilizes a 6-level classification system, achieves a quadratic weighted kappa of 0.930 on the internal test dataset (n = 531). Furthermore, it demonstrates good generalization on the external test dataset (n = 155) with a quadratic weighted kappa of 0.8668. The proposed model integrating a self-supervised feature extraction model and an attention LSTM aggregator, is one of the most advanced scoring methods for grading prostate cancer using only slide-level labels. It represents a significant advancement in automating the extraction of prostate cancer grading information in digital diagnostic pathology and will help to address the critical shortage of pathologists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availibility

Data used in the present study are publicly available, and ethical approval and informed consent were obtained in each original study.

Code Availability

On the https://github.com/maliang07/mycodes.git

References

  1. Siegel RL, Miller KD, Fuchs HE, Jemal A (2022) Cancer statistics, 2022. CA: A Cancer Journal for Clinicians 72:7–33. https://doi.org/10.3322/caac.21708

  2. Ryu HS, Jin M-S, Park JH, Lee S, Cho J, Oh S, Kwak T-Y, Woo JI, Mun Y, Kim SW, Hwang S, Shin S-J, Chang H (2019) Automated gleason scoring and tumor quantification in prostate core needle biopsy images using deep neural networks and its comparison with pathologist-based assessment. Cancers 11:1860. https://doi.org/10.3390/cancers11121860

  3. Bulten W, Pinckaers H, Boven H, Vink R, Bel T, Ginneken B, Laak J, Kaa CH-v, Litjens G (2020) Automated deep-learning system for gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol 21:233–241. https://doi.org/10.1016/S1470-2045(19)30739-9

  4. Tellez D, Litjens G, Laak J, Ciompi F (2021) Neural image compression for gigapixel histopathology image analysis. IEEE Transactions on pattern analysis and machine intelligence 43:567–578. https://doi.org/10.1109/TPAMI.2019.2936841

  5. Wang J, Mao X, Wang Y, Tao X, Chu J, Li Q (2023) Automatic generation of pathological benchmark dataset from hyperspectral images of double stained tissues. OpticsOptics & Laser Technology 163:109331. https://doi.org/10.1016/j.optlastec.2023.109331

  6. Li J, Weng Z, Xu H, Zhang Z, Miao H, Chen W, Liu Z, Zhang X, Wang M, Xu X, Ye Q (2018) Support vector machines (svm) classification of prostate cancer gleason score in central gland using multiparametric magnetic resonance images: a cross-validated study. European J Radiol 98:61–67. https://doi.org/10.1016/j.ejrad.2017.11.001

  7. Toth R, Schiffmann H, Hube-Magg C, Büscheck F, Gerhuser C (2019) Random forest-based modelling to detect biomarkers for prostate cancer progression. Clin Epigenetics 11(1):148

  8. Karimi D, Nir G, Fazli L, Black PC, Goldenberg L, Salcudean SE (2020) Deep learning-based gleason grading of prostate cancer from histopathology images-role of multiscale decision aggregation and data augmentation. IEEE J Biomed Health Inform 24(5):1413–1426

    Article  Google Scholar 

  9. Duong LT, Le NH, Tran TB, Ngo VM, Nguyen PT (2021) Detection of tuberculosis from chest x-ray images: boosting the performance with vision transformer and transfer learning. Expert Syst Appl 184:115519. https://doi.org/10.1016/j.eswa.2021.115519

  10. Lucas M, Jansen I, Savci-Heijink CD, Meijer SL, Boer OJ, Leeuwen TG, Bruin DM, Marquering HA (2019) Deep learning for automatic gleason pattern classification for grade group determination of prostate biopsies. Virchows Archiv 475:77–83. https://doi.org/10.1007/s00428-019-02577-x

  11. Otálora S, Marini N, Müller H, Atzori M (2021) Combining weakly and strongly supervised learning improves strong supervision in gleason pattern classification. BMC Med Imaging 21:77. https://doi.org/10.1186/s12880-021-00609-0

  12. Silva-Rodríguez J, Colomer A, Dolz J, Naranjo V (2021) Self-learning for weakly supervised gleason grading of local patterns. IEEE J Biomed Health Inform 25:3094–3104

    Article  Google Scholar 

  13. Nagpal K, Foote D, Liu Y, Po-Hsuan Chen Wulczyn E, Tan F, Olson N, Smith JL, Mohtashamian A (2018) Development and validation of a deep learning algorithm for improving gleason scoring of prostate cancer

  14. Arvaniti E, Fricker KS, Moret M, Rupp N, Fankhauser C, Hermanns T, Wey N, Wild P, Rüschoff J, Claassen M (2018) Automated gleason grading of prostate cancer tissue microarrays via deep learning. European Urology Supplements 17(14):3020–3021

  15. Bulten W, Balkenhol M, Belinga J-JA, Brilhante A, Çakıc A, Egevad L, Eklund M, Farré X, Geronatsiou K, Molinié V, Pereira G, Roy P, Saile G, Salles P, Schaafsma E, Tschui J, Vos A-M, Panel IPIE, Boven H, Vink R, Laak J, Kaa CH-v, Litjens G, Delahunt B, Samaratunga H, Grignon DJ, Evans AJ, M.Berney D, Pan C-C, Kristiansen G, Kench JG, Oxley J, Leite KRM, McKenney JK, Humphrey PA, Fine SW, Tsuzuki T, Varma M, Zhou M, Comperat E, Bostwick DG, Iczkowski KA, Magi-Galluzzi C, Srigley JR, Takahashi H, Kwast T (2021) Artificial intelligence assistance significantly improves gleason grading of prostate biopsies by pathologists. Modern Pathology 34:660–671. https://doi.org/10.1038/s41379-020-0640-y

  16. Silva-Rodríguez J, Colomer A, Sales MA, Molina R, Naranjo V (2020) Going deeper through the gleason scoring scale: an automatic end-to-end system for histology prostate grading and cribriform pattern detection. Comput Methods Programs Biomed 195:105637 https://doi.org/10.1016/j.cmpb.2020.105637

  17. Bulten W, Kartasalo K, Chen P-HC, Ström P, Pinckaers H, Nagpal K, Cai Y, Steiner DF, Boven H, Vink R, Kaa CH-v, Laak J, Amin MB, Evans AJ, Kwast T, Allan R, Humphrey PA, Grönberg H, Samaratunga H, Delahunt B, Tsuzuki T, Häkkinen T, Egevad L, Demkin M, Dane S, Tan F, Valkonen M, Corrado GS, Peng L, Mermel CH, Ruusuvuori P, Litjens G, Eklund M, Brilhante A, Çakır A, Farré X, Geronatsiou K, Molinié V, Pereira G, Roy P, Saile G, Salles PGO, Schaafsma E, Tschui J, Billoch-Lima J, Pereira EM, Zhou M, He S, Song S, Sun Q, Yoshihara H, Yamaguchi T, Ono K, Shen T, Ji J, Roussel A, Zhou K, Chai T, Weng N, Grechka D, Shugaev MV, Kiminya R, Kovalev V, Voynov D, Malyshev V, Lapo E, Campos M, Ota N, Yamaoka S, Fujimoto Y, Yoshioka K, Juvonen J, Tukiainen M, Karlsson A, Guo R, Hsieh C-L, Zubarev I, Bukhar HST, Li W, Li J, Speier W, Arnold C, Kim K, Bae B, Kim YW, Lee H-S, Park J (2022) Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge. Nature Medicine 28:154–163. https://doi.org/10.1038/s41591-021-01620-2

  18. Mun Y, Paik I, Shin S-J, Kwak T-Y, Chang H (2021) Yet another automated gleason grading system (yaaggs) by weakly supervised deep learning. npj Digital Medicine 4:99. https://doi.org/10.1038/s41746-021-00469-6

  19. Hammouda K, Khalifa F, Alghamdi NS, Darwish HE, El-Baz AS (2022) Multi-stage classification-based deep learning for gleason system grading using histopathological images. Cancers 14

  20. Li W, Li J, Sarma KV, Ho KC, Shen S, Knudsen BS, Gertych A, Arnold CW (2019) Path r-cnn for prostate cancer diagnosis and gleason grading of histological images. IEEE Transactions on Medical Imaging 38:945–954 https://doi.org/10.1109/TMI.2018.2875868

  21. Duran-Lopez L, Dominguez-Morales JP, Gutierrez-Galan D, Rios-Navarro A, Jimenez-Fernandez A, Vicente-Diaz S, Linares-Barranco A (2021) Wide & deep neural network model for patch aggregation in cnn-based prostate cancer detection systems. Comput Biol Med 136:104743. https://doi.org/10.1016/j.compbiomed.2021.104743

  22. Xiang J, Wang X, Wang X, Zhang J, Yang S, Yang W, Han X, Liu Y (2023) Automatic diagnosis and grading of prostate cancer with weakly supervised learning on whole slide images. Comput Biol Med 152:106340. https://doi.org/10.1016/j.compbiomed.2022.106340

  23. Pinckaers H, Bulten W, Laak J, Litjens G (2021) Detection of prostate cancer in whole-slide images through end-to-end training with image-level labels. IEEE Transactions on medical imaging 40:1817–1826. https://doi.org/10.1109/TMI.2021.3066295

  24. Campanella G, Hanna MG, Geneslaw L, Miraflor AP, Silva VWK, Busam KJ, Brogi E, Reuter VE, Klimstra DS, Fuchs TJ (2019) Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med pp 1–9

  25. Yao J, Zhu X, Jonnagaddala J, Hawkins N, Huang J (2020) Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med Image Anal 65: 101789. https://doi.org/10.1016/j.media.2020.101789

  26. Lu MY, Williamson DFK, Chen TY, Chen RJ, Mahmood F (2021) Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng pp 1–16

  27. Pasa F, Golkov V, Pfeiffer F, Cremers D, Pfeiffer D (2019) Efficient deep network architectures for fast chest x-ray tuberculosis screening and visualization. Scientific Reports 9:6268. https://doi.org/10.1038/s41598-019-42557-4

  28. Foysal Haque K, Farhan Haque F, Gandy L, Abdelgawad A (2020) Automatic detection of covid-19 from chest x-ray images with convolutional neural networks. In: 2020 International conference on computing, electronics & communications engineering (iCCECE), pp 125–130. https://doi.org/10.1109/iCCECE49321.2020.9231235

  29. Wang X, Du Y, Yang S, Zhang J, Wang M, Zhang J, Yang W, Huang J, Han X (2023) Retccl: clustering-guided contrastive learning for whole-slide image retrieval. Med Image Anal 83:102645. https://doi.org/10.1016/j.media.2022.102645

  30. Wang X, Yang S, Zhang J, Wang M, Zhang J, Yang W, Huang J, Han X (2022) Transformer-based unsupervised contrastive learning for histopathological image classification. Med Image Anal 81: 102559. https://doi.org/10.1016/j.media.2022.102559

  31. Sun L, Zhou M, Li Q, Hu M, Wen Y, Zhang J, Lu Y, Chu J (2022) Diagnosis of cholangiocarcinoma from microscopic hyperspectral pathological dataset by deep convolution neural networks. Methods 202:22–30. https://doi.org/10.1016/j.ymeth.2021.04.005

  32. Wang Q, Sun L, Wang Y, Zhou M, Hu M, Chen J, Wen Y, Li Q (2021) Identification of melanoma from hyperspectral pathology image using 3d convolutional networks. IEEE Transactions on medical imaging 40:218–227. https://doi.org/10.1109/TMI.2020.3024923

  33. Lu MY, Williamson DFK, Chen TY, Chen RJ, Barbieri M, Mahmood F (2021) Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5:555–570. https://doi.org/10.1038/s41551-020-00682-w

  34. Ilse M, Tomczak JM, Welling M (2018) Attention-based deep multiple instance learning. In: International conference on machine learning. https://api.semanticscholar.org/CorpusID:3537028

  35. Bilal M, Jewsbury R, Wang R, AlGhamdi HM, Asif A, Eastwood M, Rajpoot N (2023) An aggregation of aggregation methods in computational pathology. Med Image Anal 88:102885. https://doi.org/10.1016/j.media.2023.102885

  36. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

  37. Li B, Li Y, Eliceiri KW (2021) Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: 2021 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 14313–14323. https://doi.org/10.1109/CVPR46437.2021.01409

  38. Singhal N, Soni S, Bonthu S, Chattopadhyay N, Samanta P, Joshi U, Jojera A, Chharchhodawala T, Agarwal A, Desai M, Ganpule A (2022) A deep learning system for prostate cancer diagnosis and grading in whole slide images of core needle biopsies. Sci Rep 12(1):3383. https://doi.org/10.1038/s41598-022-07217-0

    Article  Google Scholar 

  39. Pati P, Jaume G, Ayadi Z, Thandiackal K, Bozorgtabar B, Gabrani M, Goksel O (2023) Weakly supervised joint whole-slide segmentation and classification in prostate cancer. Med Image Anal pp102915. https://doi.org/10.1016/j.media.2023.102915

  40. Bilal M, Raza SEA, Azam A, Graham S, Ilyas M, Cree IA, Snead D, Minhas F, Rajpoot NM (2021) Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study. The Lancet. Digital health 3(12):763–772

Download references

Acknowledgements

The authors would like to thank the Radboud University Medical Center and Karolinska Institute for making the PANDA dataset publicly available. This work was not supported by any research funding. The authors would also like to express their gratitude to Mr. Lu and Mr. Jiang for their meticulous guidance and outstanding technical support. I also need to thank my colleague, Mrs. Xie, for her guidance and valuable advice on my writing.

Author information

Author notes

  1. Chen Hao and Gong Ming contributed equally to this work.

Authors and Affiliations

  1. Information Center, Affiliated Hospital of Nantong University, Xisi Road, Nantong, 226300, Jiangsu, China

    Ma Liang, Chen Hao & Gong Ming

Authors
  1. Ma Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gong Ming
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Liang Ma and Hao chen performed the data analysis and wrote the manuscript. Liang Ma, Hao chen and Ming Gong conduct the experiment together.

Corresponding author

Correspondence to Ma Liang.

Ethics declarations

Conflicts of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, M., Hao, C. & Ming, G. Prostate cancer grade using self-supervised learning and novel feature aggregator based on weakly-labeled gbit-pixel pathology images. Appl Intell 54, 871–885 (2024). https://doi.org/10.1007/s10489-023-05224-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05224-w

Keywords

Search

Navigation