Deep learning and radiomics analysis for prediction of placenta invasion based on T2WI

: The purpose of this study was to explore whether the Nomogram, which was constructed by combining the Deep learning and Radiomic features of T2-weighted MR images with Clinical factors (NDRC), could accurately predict placenta invasion. This retrospective study included 72 pregnant women with pathologically confirmed placenta invasion and 40 pregnant women with normal placenta. After 24 gestational weeks, all participants underwent magnetic resonance imaging. The uterus and placenta regions were segmented in magnetic resonance images on sagittal T2WI. Ninety-three radiomics features were extracted from the placenta region, and 128 deep features were extracted from the uterus region using a deep neural network. The least absolute shrinkage and selection operator (LASSO) algorithm was used to filter these 221 features and to form the combined signature. Then the combined signature (CS) and clinical factors were combined to construct a nomogram. The accuracy, sensitivity, specificity and AUC of the nomogram were compared with four machine learning methods. The model NDRC was trained on the dataset of 78 pregnant women in the training cohort. Finally, the model NDRC was compared with four machine learning methods on the independent validation cohort of 34 pregnant women. The results showed that the prediction accuracy, sensitivity, specificity and AUC of the NDRC model were 0.941, 0.952, 0.923 and 0.985 respectively, which outperforms the traditional machine learning methods which rely on radiomics features and deep learning features alone. sequence, TE 6.8ms, TR 13.9, FOV 375 × 375 mm, layer thickness 8.5 mm, layer spacing 1.5 mm, matrix 375 × 375; (2) T2WI fast self-selected echo (FSE T2WI) sequence of axial, sagittal and coronal planes, TE 1200ms, TR 80ms, FOV 375 × 375, layer thickness 4.5 mm, layer spacing 2 mm, matrix 256 × 256.


Introduction
Placental invasion is defined as the abnormal invasion of trophoblast cells into the myometrium at different depths of infiltration [1]. It occurs mostly in patients with placenta previa or prior cesarean section, with common complications including catastrophic perinatal hemorrhage and bladder, bowel, and urethral injuries [2]. Early diagnosis of placental invasion before delivery is critical for appropriate treatment planning [3]. Ultrasound is the mainstay in the imaging of placental invasion [4]. However, due to the influence of the shape of pregnant women, gas in the intestines and bones, ultrasonic penetration power would decrease and reflected waves increase, which results in the unclear observation of lesions, especially the lesions of the posterior wall of the uterus and placenta. While MRI is less affected by intestinal gas and bones, has high tissue resolution and can be imaged at any angle in multiple directions, so it is especially recommended for cases of posterior placenta and cases whose ultrasound results are equivocal and/or clinical suspicion is high [5,6].
In the clinical practice of detecting placenta invasion, a recent study showed that the performance of experienced radiologists was significantly better than that of junior radiologists [7]. In order to reduce the dependence on doctors' clinical experience and improve the diagnosis of placental invasion, some scoring systems for diagnosing placental invasion have been proposed in recent years, but they have not been extensively tested [8]. Obvious placental heterogeneity and irregular thick dark intraplacental band in the placenta on T2WI are recognized signs of placental infiltration in MR images [9]. However, the visual representation of placental heterogeneity and the identification of abnormal dark intraplacental bands on T2WI may be difficult and highly subjective. Therefore, quantitative image analysis has become an attractive research field that overcomes the subjectivity of visual interpretation and improves diagnostic accuracy [10].
With the rapid development of medical imaging technology, a comprehensive analysis method of medical images called radiomics has received extensive attention in recent years [11]. Radiomics usually refers to extracting high-throughput quantitative features from medical images and realizing auxiliary diagnosis of diseases through machine learning. Researchers can use complex machine learning tools to develop a variety of radiomics models, thereby potentially improving the accuracy of disease diagnosis, prognosis and prediction because radiomics transforms medical images into obtainable data [12,13]. At the same time, deep learning has achieved great success in image analysis in recent years as an important means of realizing artificial intelligence (AI), and has also greatly promoted the development of medical image processing technology [14,15]. Different from the low-level image features extracted by radiomics methods, deep learning can mine various high-level semantic information of images. Therefore, we hope to combine deep learning with traditional radiomics to establish a more effective placenta invasion detection model. At present, the conventional idea of combining radiomics and deep learning is to extract radiomics features and deep features of medical images, and then build better auxiliary diagnostic models through feature-level fusion. This idea is easy to implement, but radiomics features and the deep features are at different semantic levels, which restricts the fusion effect. Therefore, the idea of using the radiomics features and the deep features to independently train two models, and then the decision-making fusion of the model output has also attracted scholars' attention.
Inspired by the above research, we combined the radiomics and deep features, used the LASSO algorithm for feature screening, formed an effective feature combination and combined with clinical factors, and established a placenta invasion detection model by constructing a nomogram. This method not only takes full advantage of the ability to represent different levels of placental heterogeneity of radiomics features, deep features and clinical factors, but also takes into account the respective advantages of feature-level fusion and decision-level fusion, thereby contributing to the development of auxiliary diagnosis of placenta invasion.

Participants
This retrospective study was approved by the Ethics Committee of the Affiliated Hospital of Medical School of Ningbo University, and identity information of all patients had been de-identified to protect patient privacy. One hundred forty-one pregnant women were initially selected. In all, twenty-nine patients were excluded for various reasons including multiple pregnancies (n = 2), MRI performed before 24th gestational weeks (n = 18), presence of significant fetal/maternal anomalies (n = 7), and severe motion or other type of artifact on either imaging sequence (n = 2). The final cohort included 112 patients. Based on these cases, we conducted a multi-center study. We took 78 cases from Ningbo Women & Children's Hospital as the training cohort, and 34 cases from the Affiliated Hospital of Medical School of Ningbo University as the independent validation cohort, as shown in Table 1.
Documents showed that the main signs of placenta invasion in MR images included: uneven placental signal, abnormally prominent placenta or (and) uterine limitations and low signal banding in the placenta, etc. Therefore, we finally chose the supine sagittal image of the conventional T2WI sequence (side-lying imaging is prone to curling artifacts due to the bulge of the abdomen) as the reference imaging sequence for this study. Typical T2WI sagittal images are shown in Figure 1. In Figure 1, (a) The patient was 35 weeks gestational and had a placenta increta. The arrow in the figure showed that the placenta and the anterior inferior wall of the uterus were unclear, and signs of implantation were seen locally. (b) The patient was 35 weeks gestation and had a placenta percreta. The arrow in the figure showed that the placenta penetrated the lower wall of the uterus and involved the bladder.
Some researchers reported that the use of contrast media to observe the structure of the placenta would help the detection of placental invasion [16][17][18]. These researchers believed that for the placental tissue rich in blood vessels, the application of contrast media could show the placenta more clearly. However, since the contrast media might pass through the placental barrier, the side effects caused could not be ignored. Zhu [19] believed that although there are no reports on the adverse effects of gadolinium contrast media on human fetuses, the half-life of the contrast media in the fetus was still unclear, so the MR images used in this study were all obtained by non-enhanced scanning.

Region of interest segmentation
The features we extracted mainly include two types: radiomics features and deep features. The radiomics features mainly included the geometric shape features of the target area, the pixel grayscale distribution features and the texture features. In order to better characterize the feature information contained in the placental tissue, we segmented the placental region to extract the radiomics features. Considering that most patients with placenta invasion have placenta previa, that is to say, the relative position of the placenta in the uterus is also an important factor in predicting placenta invasion. The deep features extracted by deep neural networks can indicate the relationship between the targets. Therefore, we segmented the entire uterine region from the MR image and use it as the input of the deep neural network to extract deep features.
Segmentation of the uterine region: Background information occupies a large proportion of MR images and it is not helpful for extracting radiomic features and deep features. Radiologists usually focus on the uterus area and automatically filter the background information. Therefore, we cut out the black background area as much as possible while preserving the complete uterine area. Finally, the size of each slice after cropping was determined to be 256 × 256. Each sequence of the original data contained 24 slices. The size of the MRI was 256 × 256 × 24 after the above processing. We normalized the images after the above processing by z-score and used them as the input of the neural network to extract deep features.
Segmentation of the placenta area: In the uterus area, the placenta was further segmented by an MRI radiologist with 15 years of work experience and a radiologist with 10 years of work experience using the open source software ITK-SNAP version 3.8.0. Both radiologists were blind to the pathological results. The segmentation results were also evaluated by three independent reviewers to make sure they agree. We normalized the segmented placental region images by z-score and used them to extract radiomics features.

Radiomic feature extraction
In this paper, radiomics features were extracted from the region of placenta. The extracted radiomics features were divided into three categories: (I) geometry, (II) intensity, (III) texture. Geometric features described the three-dimensional shape of the placenta. The intensity features described the first-order statistical distribution of voxel intensity in the placenta. Texture features described the second-and higher-order spatial distribution of patterns or intensity. A total of 93 radiomics features were extracted. The specific features can be viewed in Appendix A. All the features were extracted through the pyradiomics package version 3.0 [20].

Deep feature extraction
Radiomics features are clearly designed or hand-made. Although the number of features can reach tens of thousands, these features are shallow and low-level image features, which may not be able to fully characterize the heterogeneity of the placenta, and therefore may limit the potential of survival prediction models. In this case, it is necessary to extract deeper and higher-order features.
Given a target task (for example, in our case, diagnosing invasive placentation by MRI), the training data for this target task is limited. Transfer learning aimed to use large-scale data from other sources of tasks and artificially provided labels to learn expressive and universal feature representations to help achieve the target task. A commonly used method [21] is to pretrain a deep neural network for feature extraction on a large data set in the source task (such as ImageNet) by inserting human annotation labels in it, and then finetune the pre-trained network in the target task. When applying this strategy, we should pay attention to the following points. First of all, there is a big difference between the image data in the source task and the MR image of placenta invasion. For example, ImageNet images usually belong to general domain categories, such as cats, dogs, chairs, etc., while the images in our task are MR images, which casts doubt that the portability of MRI from other image sources to placenta invasion. In order to solve this problem, we conducted neural network pretraining on 23 medical data sets (including brain MR images and lung CT images, etc.). Compared with ImageNet, the difference between the pretrained image data and the image data of the target task is smaller.
In this research, we chose 3D-ResNet50 as the pretrained CNN model [22]. The hyperparameters of 3D-ResNet50 are as follows: weight attenuation was 0.001, momentum was 0.9 and the initial learning rate was 0.001. Deep feature extraction includes 3 steps: pre-training of 3D-ResNet50, fine-tuning and feature extraction.
1) Pre-training: The 3D-ResNet50 model was pre-trained on 23 magnetic data sets. The model had the ability to recognize the basic contours and details in the magnetic resonance image after pre-training which could accelerate the gradient descent in the subsequent fine-tuning process.
2) Fine-tuning: The placenta training data was used to fine-tune the model. The specific method is as follows, set the output dimension of the softmax layer to 2, that is to predict the invasive placentation (the prediction result is 2 classes: normal or invasion), then calculate the loss between prediction result and the labels and fine-tune the entire network according to the back propagation error. The fine-tuned network has the ability to identify invasive placentation.
3) Feature extraction: First, linear transformation was used to normalize the image gray values to the range of [0, 255]. Then, the entire placenta area was used as the input of 3D-ResNet50. Finally, deep features that can only be calculated by forward propagation are extracted from the fully connected layer before the softmax layer. In total, 128 deep features can be extracted for each patient. This procedure was accomplished by using the deep learning toolkit Pytorch. The process of transfer learning to extract deep features is shown in Figure 2. The details of 3D-ResNet50 can be found in Appendix B.

Selection of clinical factors
One hundred twelve pregnant women who meet inclusion criteria were enrolled in the current study. Among them, seventy-eight are training cohort and thirty-four are independent validation cohort. In the training cohort, fifty-one had histologically confirmed placenta invasion (PI) (16 accreta, 32 increta, and 3 percreta) and 27 were with normal placenta. We collected clinical data of all patients from the hospital's HIS system, including age, weight, history of cesarean section or curettage (0 means no history of cesarean section or curettage, 1 means the patient had a history of cesarean section or curettage) and placenta previa (0 means no placenta previa, 1 means placenta previa), the relationship between each clinical data and placenta invasion was analyzed using paired sample t test. IBM SPSS 25.0 for Windows was used to perform the t-test. The results are shown in Table 2.  Table 2 shows that the mean age for the PI group (35.02 ± 4.94 years, range from 22 to 44) was significantly higher than the normal group (33.26 ± 5.02 years, range from 25 to 44) (p < 0.001, two-sample t test). The mean weight for the PI group was 62.65±5.27 kg (range from 50 to 75), while for the normal group it was 61.15 ± 5.11 kg (range from 50 to 70). No significant difference was observed in weight (p = 0.137, two-sample t test) between the two groups. The number of women who had a history of cesarean section or curettage in the PI group was significantly more than that in the normal group. And the number of women who had PP in the PI group was significantly more than that in the normal group. Therefore, we combined three clinical factors (excluding weight) with combined signatures to construct the nomogram.

Model building and comparison
The flowchart of the study is shown in Figure 3, radiomics and deep features were combined to build the NDRC model. In Figure 3, a variety of machine learning methods were performed to classify placental invasion (invasion or normal) based on the extracted radiomics features, including decision tree, Random Forest, Naive Bayes and Support Vector Machine in order to compare the prediction accuracy of the proposed NDRC model. Random Forest was performed to classify placental invasion based on the deep features. Their implementation details can be viewed in Appendix C. In addition to comparing the prediction accuracy with the NDRC model, we explored whether it was possible to predict placental invasion accurately by using radiomic features or deep features alone, and which features have higher prediction accuracy. Finally, the prediction accuracy of traditional machine learning methods and proposed NDRC was compared by the four indicators of accuracy, sensitivity, specificity and area under the ROC.
For each patient, we combined 93 radiomics features and 128 deep features, used the LASSO [23] algorithm to filter the merged 221 features, then used the result of the linear combination of the 12 features and their corresponding weights as the combined signatures. The paired sample t-test was used to analyze the relationship between various clinical factors and placental invasion, and the factors that were significantly related to placental invasion were retained (p-value < 0.05). In order to make the model more robust and explanatory, clinical factors remained after the t-test analysis and combined signatures were used to construct the nomogram. The method of construction is multi-factor logistic regression.

Combination and selection of features
A total of 221 features were extracted from the MRI images of the training cohort, and these features were filtered by using the LASSO model. As shown in Figure 4, (a) represents the coefficient distribution of each feature, and a coefficient distribution map is generated for log(λ) (penalty intensity). It can be seen that as log(λ) increases, the coefficients corresponding to less and less important features become 0; (b) is to use 10-fold cross-validation to adjust the parameters in the LASSO model to obtain the minimum standard. The mean square error was plotted versus log(λ). The dotted vertical lines were drawn with the best value by using the minimum criterion, that is, the dotted line indicates the number of selected features. As shown in the figure, twelve features with non-zero coefficients were selected, including 7 radiomics features and 5 deep features.  where the model provides its best fits to the data. The optimal λ value of 1.0299 with log (λ) = 0.0295 was chosen via 10-fold cross-validation based on the minimum criteria.

The nomogram and its performance
In the training cohort, we integrated the combined signature with clinical information to generate a nomogram ( Figure 5). For the independent validation cohort, the nomogram was used to predict of placenta invasion with the accuracy, sensitivity, specificity and AUC of 0.941, 0.952, 0.923 and 0.985 respectively.
In the nomogram, CS, CSC and PP stand for combined signature, history of cesarean section or curettage and placenta previa respectively.

Compared with traditional machine learning methods
Various techniques and methods were used for prediction of placenta invasion, each technique has its own respective accuracy level. Here, NDRC was compared with Decision Tree, Random Forest (RF), Naive Bayes and Support Vector Machine methods based on radiomic features in terms of accuracy, sensitivity, specificity and AUC. Then, NDRC was also compared with RF method based on deep features. Table 3 summarizes the results of all different methods on the independent validation cohort with a classification cutoff of 0.5. It can be seen in Table 3 that the prediction accuracy of the machine learning method based on deep features is slightly higher than that of the machine learning method based on radiomics features, and the accuracy of the NDRC method is higher than the above two methods. In addition to the accuracy, sensitivity, and specificity, we have evaluated different methods using receiver operating characteristic (ROC) curves ( Figure 6).

Conclusions
In this study, we developed a nomogram combining radiomics, deep learning and clinical data, which can predict placenta invasion prenatally after 24 weeks of pregnancy. Radiomics features are clearly designed or hand-made and these features reflect the shallow and low-level characteristics of the image which cannot fully characterize the heterogeneity inside the placenta when applied to the detection of placenta invasion. According to previous reports, excessive hypertrophy and fibrin deposition in the placenta can lead to intra-placenta heterogeneity [24]. Therefore, the use of imaging data to find intra-placenta heterogeneity can effectively predict placenta invasion. However, placental heterogeneity the imaging data reflects is difficult to quantify and usually develops with gestational age after the second trimester, which complicate the visual assessment of placenta invasion. Therefore, we further train the deep neural network to extract deep features inside the placenta to better characterize the placental heterogeneity.
On the other hand, previous studies have found that as the number of cesarean delivery increases, the risk of PI also increases [25][26][27]. In order to further improve the accuracy of the prediction model, we take age, history of cesarean section or curettage and placenta previa as clinical factors of placental invasion and combine them with the extracted radiomics and deep features optimizing through feature screening, then construct a nomogram that can be used for prediction of placenta invasion. The results show that the nomogram model constructed in this research achieves higher detection accuracy and AUC than machine learning methods based on radiomics or deep features alone. At the same time, the nomogram model also confirmed that history of cesarean section or curettage and placenta previa are the factors that have the dominant ability to predict placenta invasion.
We need to point out some limitations of current research. Because this study is retrospective, there will inevitably be some biases or may affect the analysis and the sample size (n = 112) is relatively small. Despite these limitations, it has the advantage that not all data is collected in a single location using a single scanner. The training cohort and validation cohort data are from different hospitals, so the results obtained in this study have been externally verified by data from different vendors with different acquisition parameters.
A number of studies have shown that delivery in a specialized center for PI patients can significantly reduce the incidence of maternal morbidity [28-30]. However, due to the uneven distribution of medical resources and the uneven development of urban and rural areas, medical institutions at and below the county level cannot detect and diagnose PI early and lack treatment experience, which lead to a poor prognosis for the parturient and perinatal period. The nomogram developed in this paper can help doctors make clinical decisions for pregnant women with suspected placental invasion. These results also provided support for the initiation of a new prospective study to systematically evaluate the diagnostic capabilities of the combination of radiomics features and clinical parameters and to improve diagnosis and patient care.
where Y is the output variable;   x H is the Y that returns the most votes; k is the number of decision trees;   x h i is the classification model of a single decision tree; I is the indicative function.

C.3. Naive Bayes
Naive Bayes is a method based on the classic Bayes principle, which can be used to build models for classification, prediction and evaluation. The classic Bayes principle can be expressed as Eq (3) is the probability of event B when event A occurs which is the posterior probability of event A . Given there is a data set