Skip to main content

Advertisement

SpringerLink
Log in

Expectation-maximization algorithm leads to domain adaptation for a perineural invasion and nerve extraction task in whole slide digital pathology images

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

In addition to lymphatic and vascular channels, tumor cells can also spread via nerves, i.e., perineural invasion (PNI). PNI serves as an independent prognostic indicator in many malignancies. As a result, identifying and determining the extent of PNI is an important yet extremely tedious task in surgical pathology. In this work, we present a computational approach to extract nerves and PNI from whole slide histopathology images. We make manual annotations on selected prostate cancer slides once but then apply the trained model for nerve segmentation to both prostate cancer slides and head and neck cancer slides. For the purpose of multi-domain learning/prediction and investigation on the generalization capability of deep neural network, an expectation-maximization (EM)-based domain adaptation approach is proposed to improve the segmentation performance, in particular for the head and neck cancer slides. Experiments are conducted to demonstrate the segmentation performances. The average Dice coefficient for prostate cancer slides is 0.82 and 0.79 for head and neck cancer slides. Comparisons are then made for segmentations with and without the proposed EM-based domain adaptation on prostate cancer and head and neck cancer whole slide histopathology images from The Cancer Genome Atlas (TCGA) database and significant improvements are observed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL (2018) Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. C: A cancer Journal for Clinicians

  2. Mete MMM (2006) Head and neck cancer detection in histopathological slides. In: IEEE international conference on data mining-workshops

  3. Ang K, Smith B, Haffty B (2009) Prognostic factors in patients with head and neck cancer. Head and Neck Cancer: Multidiscip Approach, 51–75

  4. Sethi S, Lu M, Kapke A, Benninger MS, Worsham MJ (2010) Patient and tumor factors at diagnosis in a multi-ethnic primary head and neck squamous cell carcinoma cohort. J Surg Oncol 99(2):104

    Article  Google Scholar 

  5. Soo K, Carter RL, O’Brien CJ, Barr L, Shaw HJ (2010) Prognostic implications of perineural spread in squamous carcinomas of the head and neck. Laryngoscope 96(10):1145

    Google Scholar 

  6. Zhang LJ, Wu B, Zha ZL, Qu W, Zhao H, Yuan J, Feng YJ (2018) Perineural invasion as an independent predictor of biochemical recurrence in prostate cancer following radical prostatectomy or radiotherapy: a systematic review and meta-analysis. BMC Urol 18(1):5

    Article  Google Scholar 

  7. Ström P, Nordström T, Delahunt B, Samaratunga H, Grönberg H, Egevad L, Eklund M (2020) Prognostic value of perineural invasion in prostate needle biopsies: a population-based study of patients treated by radical prostatectomy. J Clin Pathol 73(10):630–635. https://doi.org/10.1136/jclinpath-2019-206300, https://europepmc.org/articles/PMC7513266

    Article  Google Scholar 

  8. Liebig C, Ayala G, Wilks JA, Berger DH, Albo D (2009) Perineural invasion in cancer. Cancer 115(15):3379. https://doi.org/10.1002/cncr.24396, https://acsjournals.onlinelibrary.wiley.com/doi/abs/10.1002/cncr.24396

    Article  CAS  Google Scholar 

  9. Aurello P, Berardi G, Tierno SM, Vinciguerra GLR, Socciarelli F, Laracca GG, Giulitti D, Pilozzi E, Ramacciato G (2017) Influence of perineural invasion in predicting overall survival and disease-free survival in patients with locally advanced gastric cancer. The American Journal of Surgery

  10. Skancke M, Arnott SM, Amdur RL, Siegel RS, Obias VJ, Umapathi BA (2019) Lymphovascular invasion and perineural invasion negatively impact overall survival for stage II adenocarcinoma of the colon. Dis Colon Rectum 62(2):181

    Article  Google Scholar 

  11. Tanaka A, Watanabe T, Okuno K, Yasutomi M (1994) Perineural invasion as a predictor of recurrence of gastric cancer. Cancer 73(3):550

    Article  CAS  Google Scholar 

  12. Chatterjee D, Katz MH, Rashid A, Wang H, Iuga AC, Varadhachary GR, Wolff RA, Lee JE, Pisters PW, Crane CH (2012) Perineural and intraneural invasion in posttherapy pancreaticoduodenectomy specimens predicts poor prognosis in patients with pancreatic ductal adenocarcinoma. Am J Surg Pathol 36(3):409

    Article  Google Scholar 

  13. Zhao B, Lv W, Mei D, Luo R, Lin J (2020) Perineural invasion as a predictive factor for survival outcome in gastric cancer patients: a systematic review and meta-analysis. Journal of Clinical Pathology pp. jclinpath–2019–206,372

  14. Ayala EG (2004) Growth and survival mechanisms associated with perineural invasion in prostate cancer. Cancer Res 64(17):6082

    Article  CAS  Google Scholar 

  15. Luchini C, Veronese N, Nottegar A, Riva G, Pilati C, Mafficini A, Stubbs B, Simbolo M, Mombello A, Corbo VA (2019) Perineural invasion is a strong prognostic moderator in ampulla of vater carcinoma. Pancreas 48(1):70

    Article  Google Scholar 

  16. Fagan JJ, Collins B, Barnes L, D”Amico F, Myers EN, Johnson JT (1998) Perineural invasion in squamous cell carcinoma of the head and neck. Arch Otolaryngol Head Neck Surg 124(6):637

    Article  CAS  Google Scholar 

  17. Kraus RD, Barsky A, Ji L, Garcia Santos PM, Cheng N, Groshen S, Vapiwala N, Ballas LK (2019) The perineural invasion paradox: Is perineural invasion an independent prognostic indicator of biochemical recurrence risk in patients with pT2N0R0 prostate cancer? A multi-institutional study. Adv Radiat Oncol 4(1):96

    Article  Google Scholar 

  18. Bulten W, Balkenhol M, Belinga JJA, Brilhante A, Litjens G (2020) Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists. Modern Pathology

  19. Egevad L, Swanberg D, Delahunt B, Strm P, Eklund M (2020) Identification of areas of grading difficulties in prostate cancer and comparison with artificial intelligence assisted grading. Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin (6)

  20. Cheng HD, Shan J, Ju W, Guo Y, Zhang L (2010) Automated breast cancer detection and classification using ultrasound images: a survey. Pattern Recognit 43(1):299

    Article  Google Scholar 

  21. Rouhi R, Jafari M, Kasaei S, Keshavarzian P (2015) Benign and malignant breast tumors classification based on region growing and CNN segmentation. Expert Syst Appl 42(3):990

    Article  Google Scholar 

  22. Moon WK, Shen YW, Huang C, Chiang LR, Chang R (2011) Computer-aided diagnosis for the classification of breast masses in automated whole breast ultrasound images. Ultrasound Med Biol 37 (4):539

    Article  Google Scholar 

  23. Wei D, Liangping L, Xiaoyi L, Tianqi F, Dexiang L, Guo D, Hanwei C (2017) Head and neck cancer tumor segmentation using support vector machine in dynamic contrast-enhanced MRI. Contrast Media Mol Imaging 2017:1

    Google Scholar 

  24. Salvi M, Acharya UR, Molinari F, Meiburger KM (2021) The impact of pre- and post-image processing techniques on deep learning frameworks: a comprehensive review for digital pathology image analysis. Comput Biol Med 128:104129

    Article  Google Scholar 

  25. Alom MZ, Aspiras TH, Taha TM, Asari VK, Bowen TJ, Billiter D, Arkell S (2019) Advanced deep convolutional neural network approaches for digital pathology image analysis: a comprehensive evaluation with different use cases. arXiv:1904.09075

  26. Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147

    Article  Google Scholar 

  27. Zhou N, Gao Y (2017) Optimized color decomposition of localized whole slide images and convolutional neural network for intermediate prostate cancer classification. In: Society of photo-optical instrumentation engineers (SPIE) conference series. https://doi.org/10.1117/12.2254216, vol 10140, p 101400W

  28. Naik S, Doyle S, Madabhushi A, Tomaszewski JE, Feldman MD, Automated gland segmentation and Gleason grading of prostate histology by integrating low- high-level and domain specific information (2007). In: Workshop on microscopic image analysis with applications in biology, Piscataway

  29. Sirinukunwattana K, Pluim JP, Chen H, Qi X, Heng PA, Guo YB, Wang LY, Matuszewski BJ, Bruni E, Sanchez U, Böhm A., Ronneberger O, Cheikh BB, Racoceanu D, Kainz P, Pfeiffer M, Urschler M, Snead DR, Rajpoot NM (2017) Gland segmentation in colon histology images: the glas challenge contest. Med Image Anal 35:489. https://doi.org/10.1016/j.media.2016.08.008, https://www.sciencedirect.com/science/article/pii/S1361841516301542

    Article  Google Scholar 

  30. Giusti A, Cireşan DC, Masci J, Gambardella LM, Schmidhuber J (2013) Fast image scanning with deep max-pooling convolutional neural networks. In: 2013 IEEE international conference on image processing, pp 4034–4038. https://doi.org/10.1109/ICIP.2013.6738831

  31. Su H, Liu F, Xie Y, Xing F, Meyyappan S, Yang L (2015) Region segmentation in histopathological breast cancer images using deep convolutional neural network. In: 2015 IEEE 12th international symposium on biomedical imaging (ISBI), pp 55–58. https://doi.org/10.1109/ISBI.2015.7163815

  32. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH (2016) Deep learning for identifying metastatic breast cancer. arXiv:1606.05718

  33. Duong QD, Vu DQ, Lee D, Hewitt SM, Kwak JT (2019) Scale embedding shared neural networks for multiscale histological analysis of prostate cancer. In: Digital pathology

  34. Li W, Li J, Sarma KV, Ho KC, Shen S, Knudsen BS, Gertych A, Arnold CW (2019) Path R-CNN for prostate cancer diagnosis and Gleason grading of histological images. IEEE Trans Med Imaging 38(4):945. https://doi.org/10.1109/TMI.2018.2875868

    Article  CAS  Google Scholar 

  35. Yoo S, Gujrathi I, Haider M, Khalvati F (2019) Prostate cancer detection using deep convolutional neural networks. Sci Rep 9:19518. https://doi.org/10.1038/s41598-019-55972-4

    Article  CAS  Google Scholar 

  36. Cao R, Zhong X, Shakeri S, Bajgiran AM, Sung K (2019) Prostate cancer detection and segmentation in multi-parametric MRI via CNN and conditional random field. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI)

  37. Zhao B, Soraghan J, Caterina GD, Grose D (2019) Segmentation of head and neck tumours using modified U-net. In: 2019 27th European signal processing conference (EUSIPCO), pp 1–4. https://doi.org/10.23919/EUSIPCO.2019.8902637

  38. Wang Y, Zhao L, Song Z, Wang M (2019) Organ at risk segmentation in head and neck CT images by using a two-stage segmentation framework based on 3D U-Net. IEEE Access PP(99):1

    Google Scholar 

  39. Lim JY, Leech M (2016) Use of auto-segmentation in the delineation of target volumes and organs at risk in head and neck. Acta Oncol 55(7):1

    Article  Google Scholar 

  40. Tappeiner E, Pröll S, Hönig M, Raudaschl PF, Zaffino P, Spadea MF, Sharp GC, Schubert R, Fritscher K (2019) Multi-organ segmentation of the head and neck area: an efficient hierarchical neural networks approach. International Journal of Computer Assisted Radiology and Surgery

  41. Yamato N, Matsuya M, Niioka H, Miyake J, Hashimoto M (2020) Nerve segmentation with deep learning from label-free endoscopic images obtained using coherent anti-stokes raman scattering. Biomolecules 10(7):1012

    Article  CAS  Google Scholar 

  42. Wang R, Shen H, Zhou M (2019) Ultrasound nerve segmentation of brachial plexus based on optimized ResU-Net. In: 2019 IEEE international conference on imaging systems and techniques (IST)

  43. Dolati P, Eichberg D, Golby A, Zamani A, Laws E (2016) Multimodal navigation in endoscopic transsphenoidal resection of pituitary tumors using image-based vascular and cranial nerve segmentation: a prospective validation study. World Neurosurg 95:406

    Article  Google Scholar 

  44. Knijn N, Mogk SC, Teerenstra S, Simmer F, Nagtegaal ID (2016) Perineural invasion is a strong prognostic factor in colorectal cancer: A systematic review. Am J Surg Pathol 40(1):103

    Article  Google Scholar 

  45. Park J, Megow A, Swalling A, Hodge J, Foreman A, Boase S, Valentine R, Krishnan S, Ooi EH (2020) Prognosis of oral squamous cell carcinoma with perineural invasion: a comparative study of classification types. Clin Otolaryngol 45

  46. Alkhadar H, Macluskey M, White S, Ellis I (2020) Perineural invasion in oral squamous cell carcinoma: incidence, prognostic impact and molecular insight. J Oral Pathol Med (4)

  47. Zhu J, Zhou R, Wang Y, Yu M (2019) Perineural invasion as a prognostic factor in head and neck squamous cell carcinoma: a systematic review and meta-analysis. Acta Otolaryngol 139(11):1

    Article  Google Scholar 

  48. Bakst RL, Glastonbury CM, Parvathaneni U, Katabi N, Hu KS, Yom SS (2019) Perineural invasion and perineural tumor spread in head and neck cancer. Int J Radiat Oncol Biol Phys 103(5):1109

    Article  Google Scholar 

  49. Cao Y, Cheng R, Chen N, Dahai J (2019) p75 Nerve growth factor receptor as a specific nerve marker in the diagnosis of perineural invasion of squamous cell Carcinoma. Am J Clin Pathol

  50. Li J, Jiang Y, Chen C, Tan W, Yin W (2020) Integrin β4 is an effective and efficient marker in synchronously highlighting lymphatic and blood vascular invasion, and perineural aggression in malignancy. Am J Surg Pathol 44(5)

  51. Fraz MM, Khurram SA, Graham S, Shaban M, Hassan M, Loya A, Rajpoot NM (2020) FABnet: feature attention-based network for simultaneous segmentation of microvessels and nerves in routine histology images of oral cancer. Neural Comput Appl 32(14):9915

    Article  Google Scholar 

  52. Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH (2016) Patch-based convolutional neural network for whole slide tissue image classification. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2424–2433

  53. Ronneberger O, Fischer P, Brox T (2015) Convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241

  54. Tian M, He J, Yu X, Cai C, Gao Y (2021) MCMC guided CNN training and segmentation for pancreas extraction. IEEE Access, 1–1. https://doi.org/10.1109/ACCESS.2021.3070391

  55. Long J, Shelhamer E, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence

  56. Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function. arXiv:1707.04912

  57. Yuan X, Tao X, Han Z, Long LR, Huang X (2018) SegAN: Adversarial network with multi-scale L 1 loss for medical image segmentation. Neuroinformatics 16(3):383

    Google Scholar 

  58. Hu K, Zhang Z, Niu X, Zhang Y, Cao C, Xiao F, Gao X (2018) Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing :S0925231218305423

  59. Gebru T, Hoffman J, Li FF (2017) Fine-grained recognition in the wild: a multi-task domain adaptation approach. In: 2017 IEEE international conference on computer vision (ICCV)

  60. Sun B, Saenko K (2016) Deep CORAL: Correlation alignment for deep domain adaptation. In: Hua G, Jégou H (eds) Computer vision – ECCV 2016, Workshops. Springer International Publishing, Cham, pp 443–450

  61. Botev ZI, Grotowski JF, Kroese DP (2010) Kernel density estimation via diffusion. Ann Stat 38(5):2916

    Article  Google Scholar 

  62. Matthews BW (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta 405(2):442

    Article  CAS  Google Scholar 

  63. Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv:1801.05746

  64. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  65. Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking Atrous convolution for semantic image segmentation. arXiv:1706.05587

Download references

Funding

Xue Li, Jun Huang, Cuiting Wang, Xiaxia Yu, and Yi Gao report that this work is suppored by the Key-Area Research and Development Program of Guangdong Province grant 2021B0101420005, the Key Technology Development Program of Shenzhen grant JSGG20210713091811036, the Department of Education of Guangdong Province grant 2017KZDXM072, the National Natural Science Foundation of China grant 61601302, the Shenzhen Key Laboratory Foundation grant ZDSYS20200811143757022, the Shenzhen Peacock Plan grant KQTD2016053112051497, and the SZU Top Ranking Project grant 86000000210.

Author information

Author notes

    Authors and Affiliations

    1. The School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, Guangdong, People’s Republic of China

      Xue Li, Jun Huang, Cuiting Wang, Xiaxia Yu & Yi Gao

    2. Department of Pathology, The Renaissance School of Medicine at Stony Brook University, Brookhaven, NY, USA

      Tianhao Zhao & Chuan Huang

    3. Department of Radiology and Imaging Sciences, Emory University, Atlanta, Georgia, USA

      Chuan Huang

    4. Shenzhen Key Laboratory of Precision Medicine for Hematological Malignancies, Shenzhen, 518060, China

      Yi Gao

    5. Marshall Laboratory of Biomedical Engineering, Shenzhen, 518060, China

      Yi Gao

    Authors
    1. Xue Li
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Jun Huang
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Cuiting Wang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Xiaxia Yu
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Tianhao Zhao
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Chuan Huang
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Yi Gao
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Yi Gao.

    Ethics declarations

    Conflict of interest

    Chuan Huang and Tianhao Zhao have no conflict of interest to declare.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Xue Li and Jun Huang contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Li, X., Huang, J., Wang, C. et al. Expectation-maximization algorithm leads to domain adaptation for a perineural invasion and nerve extraction task in whole slide digital pathology images. Med Biol Eng Comput 61, 457–473 (2023). https://doi.org/10.1007/s11517-022-02711-z

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11517-022-02711-z

    Keywords

    Search

    Navigation