Pancreas Segmentation in CT Images: State of the Art in Clinical Practice

Pancreas adenocarcinoma is a lethal disease with poor outcomes. With increasing incidence worldwide, it is predicted to become the second leading cause of cancer death in many countries. The main factor influencing disease outcome is the tumor stage at the time of diagnosis. The first step to successfully diagnose and treat pancreatic cancer is the efficient recognition and segmentation of the target organ. Several methods based on deep learning and data fusion for pancreas segmentation have been developed and applied over the years. This paper presents a state of the art in the application of the existing methods that have been presented for the pancreas and pancreatic cyst detection and segmentation. The most successful method so far has accuracy equal to 90.18% and AUC equal to 94.55%. Also, this paper looks into software designed for 3D segmentation that is simple and potentially might be used by users from non-medical fields.


Introduction
Pancreatic cancer is one of the leading causes of cancer-related deaths in the world. According to the data of the National Cancer Institute, about 450-500 people in Lithuania get pancreatic cancer every year. It is known that men get this disease more often than women. Pancreatic cancer mortality is quite high, as most patients (>80%) have cancer-specific symptoms only at an advanced stage. The world survival rate for pancreatic cancer patients over five years is just 6%, but it may range from 2% to 9% (McGuigan et al, 2018). Unfortunately, pancreatic cancer often presents late, and only 20% of patients with pancreatic cancer have surgically resectable disease at the time of presentation. For patients who can undergo successful surgical resection, the 5-year survival is 27%.
Due to pancreas anatomical location, inner-patient variability in both shape and volume in the abdominal cavity, automatic pancreas segmentation in radiology images is frequently a challenge. However, computer-aided screening, preoperative diagnosis, and quantitative assessment of the pancreas as an organ are essential for establishing a treatment plan. Having pancreas segmented provides a reliable and quantitative representation than cross-sectional measurements. In addition, computer-automated, reliable, rapid, and accurate segmentation of pancreas processing thousands of image scans can provide new diagnostic information, assist in preoperative surgical planning, and be adopted in clinical practice.
The purpose of this paper is to review different mathematical methods that have been developed and applied in pancreas segmentation problems. There will be pros and cons presented for each analyzed research.
The remaining part of the paper is organized as follows: first, Section 2 presents available previous studies. Then the various issues and suggestions are discussed in Section 3. Finally, Section 4 presents the concluding remarks and future work.

Previous studies
Recently, with the development of technology and the evolution of deep neural networks, the computer-aided diagnosis (CAD) is becoming more accessible and accurate. The high incidence of pancreatic cancer leads to interest in the development of CAD methods useful for diagnosis and treatment, in which it is very important to identify pancreatic segments accurately. Therefore, it is necessary to develop an advanced method of pancreas segmentation.

Pancreas segmentation
Segmentation of the pancreas from computed tomography (CT) images is a big challenge due to its shape, size and location in the abdomen.
Research by (Roth et al., 2015) entitled "Deep convolutional networks for pancreas segmentation in CT imaging" presents a fully-automated bottom-up method for pancreas segmentation using CT abdominal images. The authors extract super-pixels from the abdominal region and classify them by using hierarchical coarse-to-fine classification. The convolutional neural network is trained with a standard architecture for binary image classification and ends with two SoftMax layer for "pancreas" and "nonpancreas" classification. The dropout method is then used to avoid overfitting.
The proposed method was evaluated on CT images of 82 patients. The dataset was split into training, validation, and testing sets with a proportion of 60/2/20. With this data, the authors achieved an average Dice score of 68% ± 10% in testing. This result is a promise that CAD can be used for accurate pancreas segmentation. Fig. 1. shows the segmentation results. Dice score for the third patient is the worst, which could be caused by the lesser amount of visceral fat.
The work showed that the convolutional neural networks (ConvNet) could be easily used in medical image analysis. The proposed approach could also be applied to multiorgan segmentation. Red solid denotes manual ground truth annotations (Roth et al., 2015) Another segmentation method was proposed by (Fu et al, 2018) in the paper "Hierarchical combinatory deep learning architecture for pancreas segmentation of medical computed tomography cancer images". There the authors extend the Richer Feature Convolutional Network (RCF) and put a new pancreas segmentation network. They use this method for edge detection. In this work, the authors improve the RCF network with a multi-layer up-sampling structure and get over 1% better performance in pancreas segmentation compared to the old RCF network.
The dataset consists of 59 patients, including 15 patients with non-pancreas diseases and 44 with pancreas-related diseases. With the designed architecture, authors get a Dice score (DSC) of 76.36%, which is significantly higher than the result from previous research. Some examples of the output are presented in Fig. 2. Moreover, this result is more reliable as the data contained not only healthy pancreas. Five different pancreasrelated diseases were included; therefore, the model could learn different scenarios that leads to reliability and robustness.
The most recent work on pancreas segmentation is presented by (Zhao et al, 2019a) in their paper "Fully automated pancreas segmentation with two-stage 3D convolutional neural networks". The authors propose a two-stage method for automated pancreas segmentation on 3D CT images. The first stage is based on down-sampled 3D volumes segmentation for candidate region generation, while the second stage is to refine the pancreas segmentation on smaller regions of interest.
Dataset consists of 82 abdominal CT scans, where 20 were chosen for validation. The average Dice score of the proposed method reaches 85.99%, which outperforms the previous results. Moreover, authors are using 3D scans, where the location of the pancreas on 3D volumes of different subjects are more consistent than that on 2D slices. Some examples of the segmentation results are presented in Fig. 3.  Fig. 2. Some examples of pancreas segmentation result. The red curve shows the ground truth while the green for the predicted. Row1 is in the best performance, row2 is on the quartile2, and row3 on the quartile1 (Fu et al, 2018).

Pancreatic cyst and tumor segmentation
The CNNs have been developed and applied for pancreas segmentation quite successfully. However, pancreas cyst and tumor segmentation topics are less studied.
The following are the most recent researches that were reviewed. Zhou et al., 2017, in their work "Deep supervision for pancreatic cyst segmentation in abdominal CT scans" focus on segmenting the pancreatic cyst, which is a more challenging but very useful task. The authors introduce an approach that takes into consideration the pancreas and the cyst. First of all, the pancreas is found and localized. Then the cyst is being segmented based on the predicted mask. An experiment was done on 131 pathological samples from CT scans. Since there was not enough training data, the authors cut each 3D volume into a series of 2D pieces. The proposed method achieved Dice score of 63.44%. However, at the first stage, where authors investigated pancreas segmentation, Dice score reached 79,23%. Fig. 4. shows some of the segmentation results, not all of which are successful. The most recent publications related to pancreatic tumor segmentation are (Zhao et al., 2019b) "Markerless pancreatic tumor target localization enabled by deep learning" and (Zhang et al., 2020) "A novel and efficient tumor detection for pancreatic cancer via CT images".
The paper (Zhao et al., 2019b) focuses on developing a network to localize pancreatic tumor target for image-guided radiation therapy. It is important to accurately localize the tumor in order to have successful radiation therapy. Authors propose a strategy that is based on synthetically generated digitally reconstructed radiographs without relying on the use of a vast number of clinical kV x-ray images. The accuracy of the model was evaluated using Lin's concordance correlation coefficient between the predicted and actual positions. It was found that the coefficient is better than 93%, which means that the proposed model has been successfully developed for pancreatic tumor target localization. Furthermore, the authors foresee how to implement the deep learning-based model into the radiation therapy workflow in order to have a benefit in patient care.
In the second research, the authors propose a new pancreatic tumor detection framework, which is designed to fully exploit the context information at multiple scales. The proposed method consists of augmented feature pyramid networks, self-adaptive feature fusion, and a dependencies computation module.
The dataset contains 2890 CT images. Authors have compared their novel method to existing tumor detection methods and calculated the ROC curve value (true positive to false positive rate), and the novel method had the higher 'under the curve" value = 0.9455. This shows that the new approach outperforms other methods, and it can be stated that the new method is currently the most accurate and precise in pancreatic tumor segmentation. The comparison of all methods can be observed in Table 1. Also, some of the examples are shown in Fig. 5.   Fig. 5. Example results of tumor detection. The first row is the ground truth; the second row is the corresponding detection results of the proposed method (Zhang et al., 2020). As was mentioned above, the novel method from (Zhang et al., 2020) has better values in sensitivity, specificity, accuracy, and area under the ROC curve. A close second could be DetNet (based on ResNet but maintains higher resolutions in deeper levels) or cascade R-CNN (uses cascade regression as resampling aid).

Issues and suggestions
The following section presents various issues and suggestions based on the previous work on pancreas segmentation.
One of the possible suggestions may be incorporating data fusion into already known methods, and combining results of several methods to get a more accurate inference. In (Cai et al., 2016) authors use tissue detection and boundary detection algorithms separately and combine them with a conditional random field model. If a certain region contains both tissue and boundary pancreas features, then it is assigned a higher likelihood value (and is in fact the pancreas). By applying this inference fusion method authors managed to boost their DSC by 3%. Although the accuracy was still worse than that of methods mentioned in Table 1, it can still be worth a try to apply inference fusion to other known methods and check, if that would improve the accuracy and AUC. Another study (Chen et al., 2020) used feature fusion for model training so it could detect pancreas with different size and shape.
Furthermore, there are several discussions on whether to use 2D or 3D scans, as both have advantages and disadvantages. Since 3D CT scans have high dimensionality, it can lead to a high cost of computational power and memory, which limits the depth and architecture of the networks. On the other hand, there are usually more training samples for the 2D network, which can also be produced from 3D scans in the case of a smaller dataset.
Also, the problems of data limitation persist. The first analyzed work (Roth et al., 2015) is using only a healthy pancreas; if the method would be applied to the wider data, it is questionable how it would work. Even though the second work (Fu et al, 2018) is using patients with non-pancreas diseases and with pancreas-related diseases, there are only 59 patients. This leads to the same issue as previous. The pancreas can vary depending on the patient, so it is very important to have as many different patients as possible to get a more reliable result.
Since the pancreas is quite a difficult organ to locate for a person with no medical degree (unhealthy tissue can cause swelling and increase in size, which may lead to confusion), some applications have been developed to aid pancreas tumor segmentation. For an inexperienced doctor, manual segmentation of one tumor can last up to an hour or even more in difficult cases with multiple tumors. Therefore, it would be useful to see if the segmentation process can be automated to the point that a non-specialist could perform the job.

Pancreatic tumor segmentation with 3D Slicer
The benefits and drawbacks of the four open-source programs designed to aid the segmentation process can be viewed in Table 2. There were more -Velocity (WEB, a) and Mimics (WEB, b); however, it was not open-source and was not tested. Overall it can be concluded that 3D Slicer (WEB, f) is so far the best software for pancreatic tumor segmentation since it has a semi-automatic tumor segmentation plug-in made specifically for pancreatic cancer.
To examine how well 3D Slicer does the segmentation, an experiment was performed when pancreatic tissue contains cancer. 3D Slicer has a fully automatic segmentation option. However, that is only suited for brain cancer. Therefore, semi-automatic pancreatic cancer detection was performed with bound points added manually that specify the region of interest. The results, unfortunately, were very unprecise and inconsistent. Therefore, at this point, segmentation without a medical professional seems impossible.

Conclusions and future work
While the pancreas remains a challenging abdominal organ due to its individually based anatomical placement in the abdominal cavity, accurate automatic segmentation is helpful not only for surgical planning but also for image-guided radiation therapy.
In conclusion, this short overview provided proof of concept that pancreas segmentation can be applied in the multidisciplinary management of pancreas cancer (such as early cancer detection, preoperative planning, or image-guided radiation therapy). Best results were provided by (Zhang et al., 2020) with accuracy and AUC 90.18% and 94.55%, respectively, with their novel method of combining Augmented Feature Pyramid networks, Self-adaptive Feature Fusion, and a Dependencies Computation Module.
Since segmentation of pancreatic tumor takes much time, it was proposed to use 3D Slicer software so a non-medical person could perform the segmentation; however, the results seem not accurate enough. Also applying fusion to already known methods might give them a possible accuracy boost however this must be tested.
In the nearest future, it was proposed to create and train a neural network that would first perform a binary classification for the heathy or cancerous pancreas and then move on to more difficult tasks. The long-term goal would be to use CT images together with other medical data for a fusion framework that can produce inferences of future actions for the patient: chemo, operation, medical surveillance, etc.