Skip to main content
SpringerLink
Log in

The segmentation of nuclei from histopathology images with synthetic data

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The segmentation of nuclei from histopathology images provides the necessary information to diagnose cancer. Existing deep learning-based nuclei segmentation approaches demand a large amount of annotated data from expert pathologists where the size of annotated datasets is relatively small. This paper proposes the generation of synthetic annotated data utilising a small number of labelled data. The background of synthetic patches is created using base image patches, where the generative adversarial model is applied to preserve the characteristics of texture regions. By deploying a feature learning-based CNN filtering method, more realistic backgrounds are assigned to the highest probability, and the backgrounds with lower probability scores are discarded. The foreground nuclei shapes are collected from the original images using ground truth masks, and the shape transformations are applied using image processing techniques. These nuclei shapes are placed on the filtered background by preserving the characteristics of shape and distribution of the nuclei. The synthetic image patches with corresponding masks are used to train the modified U-net segmentation network in order to segment the nuclei regions. The segmentation results in terms of DSC and JI are 0.875 and 0.791, respectively. Synthetic images performed better than original images across a range of diverse histopathology image backgrounds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Kurmi, Y., Chaurasia, V., Kapoor, N.: Histopathology image segmentation and classification for cancer revelation. Signal Image Video Process. 15(6), 1341–1349 (2021)

    Article  Google Scholar 

  2. Pandey, S., Singh, P.R., Tian, J.: An image augmentation approach using two-stage generative adversarial network for nuclei image segmentation. Biomed. Signal Process. Control 57, 101782 (2020)

    Article  Google Scholar 

  3. Lagree, A., et al.: A review and comparison of breast tumor cell nuclei segmentation performances using deep convolutional neural networks. Sci. Rep. 11(1), 1–11 (2021)

    Article  MathSciNet  Google Scholar 

  4. Liu, X., Guo, Z., Cao, J., Tang, J.: MDC-net: a new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information. Comput. Biol. Med. 135, 104543 (2021)

    Article  Google Scholar 

  5. Q. Zhu, X. Huang, L. Wang, and J. Li, "U-shaped Feature extractor used on mask R-CNN for cell nuclei image segmentation. In: Journal of Physics: Conference Series, vol. 1646(1), p. 012069. IOP Publishing (2020)

  6. Bancher, B., Mahbod, A., Ellinger, I., Ecker, R., Dorffner, G.: Improving mask R-CNN for nuclei instance segmentation in hematoxylin & eosin-stained histological images. In: MICCAI Workshop on Computational Pathology, PMLR, pp. 20–35 (2021)

  7. Yi, C., Chen, X., Quan, L., Lu, C.: Attentional dilated convolution neural network for nuclei segmentation in histopathology images. In: 2020 Chinese Automation Congress (CAC), pp. 6737–6740. IEEE (2020)

  8. dos Santos D.F., Tosta, T.A., Silva, A.B., de Faria P.R., Travençolo, B.A., do Nascimento, M.Z.: Automated nuclei segmentation on dysplastic oral tissues using cnn. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 45–50. IEEE (2020)

  9. Banik, P.P., Saha, R., Kim, K.-D.: An automatic nucleus segmentation and CNN model based classification method of white blood cell. Expert Syst. Appl. 149, 113211 (2020)

    Article  Google Scholar 

  10. Kowal, M., Żejmo, M., Skobel, M., Korbicz, J., Monczak, R.: Cell nuclei segmentation in cytological images using convolutional neural network and seeded watershed algorithm. J. Digit. Imaging 33(1), 231–242 (2020)

    Article  Google Scholar 

  11. Gehlot, S., Gupta, A., Gupta, R.: Ednfc-net: convolutional neural network with nested feature concatenation for nuclei-instance segmentation. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1389–1393. IEEE (2020)

  12. Qu, A., Cheng, Z., He, X., Li, Y.: An integration convolutional neural network for nuclei instance segmentation. In: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1104–1109. IEEE (2020)

  13. Vahadane, A., Atheeth, B., Majumdar, S.: Dual encoder attention U-net for nuclei segmentation. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 3205–3208. IEEE (2021)

  14. Xu, Q., Duan, W.: An automatic nuclei image segmentation based on multi-scale split-attention U-Net. In: MICCAI Workshop on Computational Pathology, PMLR, pp. 236–245 (2021)

  15. Rashmi, R., Prasad, K., Udupa, C.B.K.: Semantic segmentation of nuclei from breast histopathological images by incorporating attention in U-Net. In: International Conference on Computer Vision and Image Processing, pp. 137–148. Springer (2020)

  16. You, Y., Luo, C., Jin, Q.: An improved U-Net for nuclei semantic segmentation. In: 2021 5th International Conference on Communication and Information Systems (ICCIS), pp. 197–201. IEEE (2021)

  17. Long, F.: Microscopy cell nuclei segmentation with enhanced U-Net. BMC Bioinform. 21(1), 1–12 (2020)

    Article  Google Scholar 

  18. Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: a nested u-net architecture for medical image segmentation. In: Cardosa, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., Nascimento, J.C. (eds.) Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer, Berlin (2018)

    Chapter  Google Scholar 

  19. He, H., et al.: A hybrid-attention nested UNet for nuclear segmentation in histopathological images. Front. Mol. Biosci. 8, 6 (2021)

    Article  Google Scholar 

  20. Zhao, B., et al.: Triple U-net: hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Med. Image Anal. 65, 101786 (2020)

    Article  Google Scholar 

  21. Li, Y., Wang, Y., Leng, T., Zhijie, W.: Wavelet U-Net for medical image segmentation. In: International Conference on Artificial Neural Networks, pp. 800–810. Springer (2020)

  22. Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review. Med. Image Anal. 58, 101552 (2019)

    Article  Google Scholar 

  23. Das, A., Devarampati, V.K., Nair, M.S.: NAS-SGAN: a semi-supervised generative adversarial network model for atypia scoring of breast cancer histopathological images. IEEE J. Biomed. Health Inform. (2021). https://doi.org/10.1109/JBHI.2021.3131103

    Article  Google Scholar 

  24. Kausar, T., et al.: SA-GAN: stain acclimation generative adversarial network for histopathology image analysis. Appl. Sci. 12(1), 288 (2021)

    Article  MathSciNet  Google Scholar 

  25. Muramatsu, C., et al.: Improving breast mass classification by shared data with domain transformation using a generative adversarial network. Comput. Biol. Med. 119, 103698 (2020)

    Article  Google Scholar 

  26. Yao, K., Huang, K., Sun, J., Jude, C.: AD-GAN: end-to-end unsupervised nuclei segmentation with aligned disentangling training (2021). arXiv:2107.11022

  27. Zhang, H., Liu, J., Yu, Z., Wang, P.: MASG-GAN: a multi-view attention superpixel-guided generative adversarial network for efficient and simultaneous histopathology image segmentation and classification. Neurocomputing 463, 275–291 (2021)

    Article  Google Scholar 

  28. Cong, C., Liu, S., Di Ieva, A., Pagnucco, M., Berkovsky, S., Song, Y.: Texture enhanced generative adversarial network for stain normalisation in histopathology images. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1949–1952. IEEE (2021)

  29. Hou, L., Agarwal, A., Samaras, D., Kurc, T.M., Gupta, R.R., Saltz, J.H.: Robust histopathology image analysis: To label or to synthesize?. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8533–8542 (2019)

  30. Ružić, T., Pižurica, A.: Context-aware patch-based image inpainting using Markov random field modeling. IEEE Trans. Image Process. 24(1), 444–456 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  31. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  32. Husain, S.S., Bober, M.: REMAP: Multi-layer entropy-guided pooling of dense CNN features for image retrieval. IEEE Trans. Image Process. 28(10), 5201–5213 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  33. Hossain, M.S., Sakib, N.: Renal cell cancer nuclei segmentation from histopathology image using synthetic data. In: 2020 16th IEEE International Colloquium on Signal Processing & Its Applications (CSPA), pp. 236–241. IEEE (2020)

  34. Fisher, S.P.R., Walker, A., Wolfart, E.: Affine Transformation. https://homepages.inf.ed.ac.uk/rbf/HIPR2/affine.htm. Accessed 27 Mar 2022

  35. Khan, T.M., Bailey, D.G., Khan, M.A., Kong, Y.: Efficient hardware implementation for fingerprint image enhancement using anisotropic Gaussian filter. IEEE Trans. Image Process. 26(5), 2116–2126 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  36. Siddique, N., Paheding, S., Elkin, C.P., Devabhaktuni, V.: U-net and its variants for medical image segmentation: a review of theory and applications. IEEE Access (2021). https://doi.org/10.1109/ACCESS.2021.3086020

    Article  Google Scholar 

  37. Alom, M.Z., Yakopcic, C., Hasan, M., Taha, T.M., Asari, V.K.: Recurrent residual U-Net for medical image segmentation. J. Med. Imaging 6(1), 014006 (2019)

    Article  Google Scholar 

  38. Irshad, H., et al.: Crowdsourcing image annotation for nucleus detection and segmentation in computational pathology: evaluating experts, automated methods, and the crowd. In: Pacific symposium on biocomputing Co-chairs, pp. 294–305. World Scientific (2014)

  39. Zhang, B., Hu, X.: A medical image classification model based on adversarial lesion enhancement. Sci. Program. 2021, 1–9 (2021)

    Google Scholar 

  40. Isaksson J., Arvidsson, I., Åaström, K., Heyden, A.: Semantic segmentation of microscopic images of H&E stained prostatic tissue using CNN. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1252–1256. IEEE (2017)

  41. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

  42. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer (2015)

  43. Tensorflow. TensorFlow Core. Image Segmentation. https://www.tensorflow.org/tutorials/images/segmentation. Accessed 25 Mar 2022

Download references

Author information

Authors and Affiliations

  1. School of Science, Edith Cowan University, Joondalup, 6027, Australia

    Md. Shamim Hossain, Leisa J. Armstrong, Jumana Abu-Khalaf & David M. Cook

Authors
  1. Md. Shamim Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leisa J. Armstrong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jumana Abu-Khalaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David M. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Shamim Hossain.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hossain, M.S., Armstrong, L.J., Abu-Khalaf, J. et al. The segmentation of nuclei from histopathology images with synthetic data. SIViP 17, 3703–3711 (2023). https://doi.org/10.1007/s11760-023-02597-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02597-w

Keywords

Search

Navigation