Abstract
Double staining in histopathology is done to help identify tissue features and cell types differentiated between two tissue samples using two different dyes. In the case of metaplastic breast cancer, H &E and P63 are often used in conjunction for diagnosis. However, P63 tends to damage the tissue and is prohibitively expensive, motivating the development of virtual staining methods, or methods of using artificial intelligence in computer vision for diagnostic strain transformation. In this work, we present results of the new xAI-CycleGAN architecture’s capability to transform from H &E pathology stain to the P63 pathology stain on samples of breast tissue with presence of metaplastic cancer. The architecture is based on Mask CycleGAN and explainability-enhanced training, and further enhanced by structure-preserving features, and the ability to edit the output to further bring generated samples to ground truth images. We demonstrate its ability to preserve structure well and produce superior quality images, and demonstrate the ability to use output editing to approach real images, and opening the doors for further tuning frameworks to perfect the model using the editing approach. Additionally, we present the results of a survey conducted with histopathologists, evaluating the realism of the generated images through a pairwise comparison task, where we demonstrate the approach produced high quality images that sometimes are indistinguishable from ground truth, and overall our model outputs get a high realism rating.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, B., Yang, X., Li, Y., Zhang, Y., Pillar, N., Ozcan, A.: Deep learning-enabled virtual histological staining of biological samples. Light: Sci. Appl. 12(1), 57 (2023)
de Bel, T., Hermsen, M., Kers, J., van der Laak, J., Litjens, G.: Stain-transforming cycle-consistent generative adversarial networks for improved segmentation of renal histopathology (2018)
Chen, H., Yan, S., Xie, M., Huang, J.: Application of cascaded GAN based on CT scan in the diagnosis of aortic dissection. Comput. Methods Programs Biomed. 226, 107130 (2022). https://doi.org/10.1016/j.cmpb.2022.107130, https://www.sciencedirect.com/science/article/pii/S0169260722005119
Cheng, Y., Gan, Z., Li, Y., Liu, J., Gao, J.: Sequential attention GAN for interactive image editing. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4383–4391 (2020)
Collins, E., Bala, R., Price, B., Susstrunk, S.: Editing in style: uncovering the local semantics of GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5771–5780 (2020)
de Haan, K., et al.: Deep learning-based transformation of H &E stained tissues into special stains. Nat. Commun. 12(1), 4884 (2021)
Harding, M.C., Sloan, C.D., Merrill, R.M., Harding, T.M., Thacker, B.J., Thacker, E.L.: Peer reviewed: transitions from heart disease to cancer as the leading cause of death in US States, 1999–2016. Prevent. Chronic Dis. 15(12) (2018). https://doi.org/10.5888/PCD15.180151, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6307835/
Nagisetty, V., Graves, L., Scott, J., Ganesh, V.: xAI-GAN: enhancing generative adversarial networks via explainable AI systems (2020). https://doi.org/10.48550/arxiv.2002.10438, https://arxiv.org/abs/2002.10438v3
Pajouheshgar, E., Zhang, T., Süsstrunk, S.: Optimizing latent space directions for GAN-based local image editing. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1740–1744. IEEE (2022)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, pp. 2234–2242. Curran Associates, Inc. (2016)
Shen, Y., Zhou, B.: Closed-form factorization of latent semantics in GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1532–1540 (2021)
Siegel Mph, R.L., Miller, K.D., Sandeep, N., Mbbs, W., Ahmedin, Dvm, J., Siegel, R.L.: Cancer statistics, 2023. CA: Cancer J. Clinic. 73(1), 17–48 (1 2023). https://doi.org/10.3322/CAAC.21763
Sloboda, T., Hudec, L., Benešová, W.: xai-cyclegan, a cycle-consistent generative assistive network. arXiv preprint: arXiv:2306.15760 (2023)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, M.: Mask CycleGAN: unpaired multi-modal domain translation with interpretable latent variable (2022). https://doi.org/10.48550/arxiv.2205.06969, https://arxiv.org/abs/2205.06969v1
Xu, Z., Huang, X., Moro, C.F., Bozóky, B., Zhang, Q.: GAN-based virtual re-staining: a promising solution for whole slide image analysis (2022)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A., Research, B.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks (2017). https://github.com/junyanz/CycleGAN
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sloboda, T., Hudec, L., Benešová, W. (2024). Editable Stain Transformation of Histological Images Using Unpaired GANs. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing - ICIAP 2023 Workshops. ICIAP 2023. Lecture Notes in Computer Science, vol 14366. Springer, Cham. https://doi.org/10.1007/978-3-031-51026-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-51026-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-51025-0
Online ISBN: 978-3-031-51026-7
eBook Packages: Computer ScienceComputer Science (R0)