Abstract
Automatic segmentation of the prostate in transrectal ultrasound (TRUS) images provides useful information for prostate cancer diagnosis and treatment. However, boundaries between the prostate and other tissues are often absent or ill defined in TRUS images, which means that automatic segmentation of the prostate in TRUS images is highly challenging. In this study, we attempted to overcome these challenges by developing a novel method we termed “automatic prostate segmentation” (Auto-ProSeg) that is capable of effectively segmenting the prostate in TRUS images. Auto-ProSeg comprises two steps: the first step is a preprocessing step that uses attention U-Net to extract approximate prostate contours automatically; then, in the second step, the approximate prostate contours are optimized via a modified principal curve-based method linked to an evolutionary neural network, whereby a mathematical mapping formula based on the parameters of an enhanced evolutionary neural network is used to generate smooth prostate contours. Our results illustrate that Auto-ProSeg exhibits better prostate segmentation performance than other recently developed methods: the average Dice similarity coefficient and Jaccard similarity coefficient (Ω) of Auto-ProSeg-generated prostate contours against ground truths were 94.2% ± 3.2% and 93% ± 3.7%, whereas those for other state-of-the-art fully automatic segmentation methods were approximately 90% ± 5% and 89% ± 6%, respectively.
Similar content being viewed by others
Data availability
Data will be made available on reasonable request.
Notes
Unlike the K-segments polyline segment tracking (KPST) method [27], the closed polyline segment tracking (CPST) method that we have previously devised [28,29,30] has several constraint conditions and is the first method to do so. Moreover, we developed the hybrid prostate segmentation (H-ProSeg) [23] method, which uses vertex cleaning conditions to control the influence of abnormal data.
References
Goldenberg SL, Nir G, Salcudean SE (2019) A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 16:391–403
Xu X, Sanford T, Turkbey B et al (2022) Shadow-consistent semi-supervised learning for prostate ultrasound segmentation. IEEE Trans Med Imaging 41:1331–1345
Jiang J, Guo Y, Bi Z et al (2023) Segmentation of prostate ultrasound images: the state of the art and the future directions of segmentation algorithms. Artif Intell Rev 56:615–651
Chen G, Dai Y, Li R et al (2021) SDFNet: automatic segmentation of kidney ultrasound images using multi-scale low-level structural feature. Expert Syst Appl 185:115619
Peng T, Zhao J, Wang J (2021) Interpretable mathematical model-guided ultrasound prostate contour extraction using data mining techniques. In: IEEE 15th International Conference on Bioinformatics and Biomedicine (BIBM), pp 1037–1044
He K, Lian C, Adeli E et al (2021) MetricUNet: synergistic image- and voxel-level learning for precise prostate segmentation via online sampling. Med Image Anal 71:102039
Chen G, Li L, Dai Y et al (2022) AAU-net: an adaptive attention U-net for breast lesions segmentation in ultrasound images. IEEE Trans Med Imaging:1–1
Liu Z, Yang C, Huang J et al (2021) Deep learning framework based on integration of S-mask R-CNN and inception-v3 for ultrasound image-aided diagnosis of prostate cancer. Futur Gener Comput Syst 114:358–367
Chen G, Dai Y, Zhang J (2023) RRCNet: refinement residual convolutional network for breast ultrasound images segmentation. Eng Appl Artif Intell 117:105601
Peng T, Gu Y, Ye Z et al (2022) A-LugSeg: automatic and explainability-guided multi-site lung detection in chest X-ray images. Expert Syst Appl 198:116873
Peng T, Wang C, Zhang Y, Wang J (2022) H-SegNet: hybrid segmentation network for lung segmentation in chest radiographs using mask region-based convolutional neural network and adaptive closed polyline searching method. Phys Med Biol 67:075006
Shahedi M, Halicek M, Guo R et al (2018) A semiautomatic segmentation method for prostate in CT images using local texture classification and statistical shape modeling. Med Phys 45:2527–2541
Anas EMA, Mousavi P, Abolmaesumi P (2018) A deep learning approach for real time prostate segmentation in freehand ultrasound guided biopsy. Med Image Anal 48:107–116
Kervadec H, Dolz J, Tang M et al (2019) Constrained-CNN losses for weakly supervised segmentation. Med Image Anal 54:88–99
Zhang T-T, Shu H, Lam K-Y et al (2023) Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation. Appl Intell 53:9548–9561
Bi H, Jiang Y, Tang H et al (2020) Fast and accurate segmentation method of active shape model with Rayleigh mixture model clustering for prostate ultrasound images. Comput Methods Prog Biomed 184:105097
Jia X, Ren L, Cai J (2020) Clinical implementation of AI technologies will require interpretable AI models. Med Phys 47:1–4
Xing L, Krupinski EA, Cai J (2018) Artificial intelligence will soon change the landscape of medical physics research and practice. Med Phys 45:1791–1793
Orlando N, Gyacskov I, Gillies DJ et al (2022) Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound. Phys Med Biol 67:074002
Vesal S, Gayo I, Bhattacharya I et al (2022) Domain generalization for prostate segmentation in transrectal ultrasound images: a multi-center study. Med Image Anal 82:102620
Lei Y, Wang T, Roper J et al (2021) Male pelvic multi-organ segmentation on transrectal ultrasound using anchor-free mask CNN. Med Phys 48:3055–3064
Liu Q, Dou Q, Heng P-A (2020) Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains. In: International conference on medical image computing and computer-assisted intervention, pp 475–485
Peng T, Wu Y, Qin J et al (2022) H-ProSeg: hybrid ultrasound prostate segmentation based on explainability-guided mathematical model. Comput Methods Prog Biomed 219:106752
Xu X, Sanford T, Turkbey B et al (2022) Polar transform network for prostate ultrasound segmentation with uncertainty estimation. Med Image Anal 78:102418
Oktay O, Schlemper J, Folgoc LL, et al (2018) Attention U-Net: learning where to look for the pancreas. In: Medical Imaging with Deep Learning (MIDL)
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241
Kégl B, Linder T, Zeger K (2000) Learning and design of principal curves. IEEE Trans Pattern Anal Machine Intell 22:281–297
Peng T, Wang Y, Xu TC et al (2018) Detection of lung contour with closed principal curve and machine learning. J Digit Imaging 31:520–533
Peng T, Xu TC, Wang Y et al (2020) Hybrid automatic lung segmentation on chest CT scans. IEEE Access 8:73293–73306
Peng T, Xu TC, Wang Y, Li F (2020) Deep belief network and closed polygonal line for lung segmentation in chest radiographs. Comput J 65:1107–1128
Zhou H, Zhao H, Zhang Y (2020) Nonlinear system modeling using self-organizing fuzzy neural networks for industrial applications. Appl Intell 50:1657–1672
Ali MZ, Awad NH, Suganthan PN, Reynolds RG (2017) An adaptive multipopulation differential evolution with dynamic population reduction. IEEE Trans Cybern 47:2768–2779
Laredo JLJ, Fernandes C, Merelo JJ, Gagné C (2009) Improving genetic algorithms performance via deterministic population shrinkage. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, Montreal, Canada, p 819
He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer Vision. Venice, Italy, pp 2961–2969
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867
Gao Y, Zhou M, Metaxas D (2021) UTNet: a hybrid transformer architecture for medical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 61–71
Peng T, Tang C, Wang J (2022) Prostate segmentation of ultrasound images based on interpretable-guided mathematical model. In: International conference on multimedia modeling (MMM). Springer, pp 166–177
Nam H, Kim H-E (2018) Batch-instance normalization for adaptively style-invariant neural networks. In: Advances in neural information processing systems
Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: ICML workshop on deep learning for audio, speech and language processing
Hara K, Saito D, Shouno H (2015) Analysis of function of rectified linear unit used in deep learning. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp 1–8
Zhang J, Chen D, Kruger U (2008) Adaptive constraint K-segment principal curves for intelligent transportation systems. IEEE Trans Intell Transport Syst 9:666–677
Kabir W, Ahmad MO, Swamy MNS (2015) A novel normalization technique for multimodal biometric systems. In: 2015 IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE, Fort Collins, pp 1–4
Chen P (2019) Effects of normalization on the entropy-based TOPSIS method. Expert Syst Appl 136:33–41
Storn R (1997) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11:341
Parouha RP, Verma P (2022) A systematic overview of developments in differential evolution and particle swarm optimization with their advanced suggestion. Appl Intell 52:10448–10492
Zeng Y-R, Zeng Y, Choi B, Wang L (2017) Multifactor-influenced energy consumption forecasting using enhanced back-propagation neural network. Energy 127:381–396
Peng T, Zhao J, Gu Y et al (2022) H-ProMed: ultrasound image segmentation based on the evolutionary neural network and an improved principal curve. Pattern Recogn 131:108890
Peng T, Tang C, Wu Y, Cai J (2022) H-SegMed: a hybrid method for prostate segmentation in TRUS images via improved closed principal curve and improved enhanced machine learning. Int J Comput Vis 130:1896–1919
Peng T, Gu Y, Wang J (2021) Lung contour detection in chest X-ray images using mask region-based convolutional neural network and adaptive closed polyline searching method. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), pp 2839–2842
Kegl B, Krzyzak A (2002) Piecewise linear skeletonization using principal curves. IEEE Trans Pattern Anal Machine Intell 24:59–74
Zhu Q, Du B, Yan P (2020) Boundary-weighted domain adaptive neural network for prostate MR image segmentation. IEEE Trans Med Imaging 39:753–763
Lei Y, Tian S, He X et al (2019) Ultrasound prostate segmentation based on multidirectional deeply supervised V-net. Med Phys 46:3194–3206
Peng T, Wang Y, Xu TC, Chen X (2019) Segmentation of lung in chest radiographs using Hull and closed polygonal line method. IEEE Access 7:137794–137810
Gu R, Wang G, Song T et al (2021) CA-net: comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans Med Imaging 40:699–711
Floridi L (2019) Establishing the rules for building trustworthy AI. Nat Mach Intell 1:261–262
Liang W, Tadesse GA, Ho D et al (2022) Advances, challenges and opportunities in creating data for trustworthy AI. Nat Mach Intell 4:669–677
Acknowledgments
Professional English language editing support is provided by AsiaEdit (asiaedit.com). The author would like to thank Dr. Jin Wang for checking and editing the manuscript.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Daqiang Xu, Caiyin Tang, Jing Zhao, and Yuntian Shen. The first draft of the manuscript was written by Tao Peng, and writing checking and review, and supervision were performed by Cong Yang and Jing Cai. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Ethical and informed consent for data used
It is the retrospective study, and the clinicians have obtained patients’ agreement before the ultrasound examination, which is an item covered by the medical insurance program. In summary, there is no need for patient consent in our study.
Conflicts of interest/competing interests
The authors declare that they have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Peng, T., Xu, D., Tang, C. et al. Automatic coarse-to-refinement-based ultrasound prostate segmentation using optimal polyline segment tracking method and deep learning. Appl Intell 53, 21390–21406 (2023). https://doi.org/10.1007/s10489-023-04676-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04676-4