Machine learning classification models for fetal skeletal development performance prediction using maternal bone metabolic proteins in goats

Background In developing countries, maternal undernutrition is the major intrauterine environmental factor contributing to fetal development and adverse pregnancy outcomes. Maternal nutrition restriction (MNR) in gestation has proven to impact overall growth, bone development, and proliferation and metabolism of mesenchymal stem cells in offspring. However, the efficient method for elucidation of fetal bone development performance through maternal bone metabolic biochemical markers remains elusive. Methods We adapted goats to elucidate fetal bone development state with maternal serum bone metabolic proteins under malnutrition conditions in mid- and late-gestation stages. We used the experimental data to create 72 datasets by mixing different input features such as one-hot encoding of experimental conditions, metabolic original data, experimental-centered features and experimental condition probabilities. Seven Machine Learning methods have been used to predict six fetal bone parameters (weight, length, and diameter of femur/humerus). Results The results indicated that MNR influences fetal bone development (femur and humerus) and fetal bone metabolic protein levels (C-terminal telopeptides of collagen I, CTx, in middle-gestation and N-terminal telopeptides of collagen I, NTx, in late-gestation), and maternal bone metabolites (low bone alkaline phosphatase, BALP, in middle-gestation and high BALP in late-gestation). The results show the importance of experimental conditions (ECs) encoding by mixing the information with the serum metabolic data. The best classification models obtained for femur weight (Fw) and length (FI), and humerus weight (Hw) are Support Vector Machines classifiers with the leave-one-out cross-validation accuracy of 1. The rest of the accuracies are 0.98, 0.946 and 0.696 for the diameter of femur (Fd), diameter and length of humerus (Hd, Hl), respectively. With the feature importance analysis, the moving averages mixed ECs are generally more important for the majority of the models. The moving average of parathyroid hormone (PTH) within nutritional conditions (MA-PTH-experim) is important for Fd, Hd and Hl prediction models but its removal for enhancing the Fw, Fl and Hw model performance. Further, using one feature models, it is possible to obtain even more accurate models compared with the feature importance analysis models. In conclusion, the machine learning is an efficient method to confirm the important role of PTH and BALP mixed with nutritional conditions for fetal bone growth performance of goats. All the Python scripts including results and comments are available into an open repository at https://gitlab.com/muntisa/goat-bones-machine-learning.


INTRODUCTION
In the early stages of embryonic development of mammals, the fetal skeleton development is composed of fibrous membranes and hyaline cartilage. The bone is usually formed through endochondral ossification and intramembranous ossification regulated by intra-or extra-factors in the middle or late gestation. Chondroblasts play an extremely important role in chondrogenesis by forming chondrocytes and extracellular matrix (EM). Moreover, the mineral metabolism of fetus skeletal development is essentially dependent on parathyroid hormone (PTH), and PTH-related proteins (PTHrP) (Mendes et al., 2019), but not calcitonin, vitamin D/calcitriol, fibroblast growth factor (FGF-23) or sex steroids (Kovacs, 2015). For PTH, it is critical to regulate calcium and skeletal homeostasis, and fetalplacental mineral homeostasis (Simmonds et al., 2010), via calciotropic and phosphotropic hormones (Kovacs, 2014).
For bone formation, the new bone is formed by osteoblasts, and some biomarkers reflect the activity of osteoblasts. The homeostasis of bone formation and bone resorption is achieved and regulated through the local mediators and systemic hormones. In general, the most commonly measured bone formation biomarkers are the bone alkaline phosphatase (BALP) and its isoforms, osteocalcin (OC) (Lee et al., 2007), and the procollagenbreakdown products (procollagen type 1 N-terminal and C-terminal propeptides, PICP and PINP). The BALP is a bone-specific isoform of liver alkaline phosphatase (ALP) on the surface of osteoblasts, reflects the biosynthetic activity of bone-forming cells, secreted by the liver, bone, placenta or intestines (Shipman, Holt & Gama, 2013;Tanaka et al., 1997). Osteocalcin is a bone-derived hormone which affects glucose metabolism by regulating insulin secretion and sensitivity (Ferron & Lacombe, 2014). The quality of bone strength is characterized by the EM, turnover rate, and mineral homeostasis. EM contains hyaluronic acid, proteoglycans, glycoproteins and collagen (type I collagen, the most abundant protein secreted by osteoblasts in the procollagen form of PICP and PINP) (Wallace et al., 2010).
For bone resorption, the most commonly used biomarkers are divided into collagenrelated or non-collagenous markers, including the hydroxylysine-glycosides, pyridinoline, deoxypyridinoline, N-terminal and C-terminal cross-linked telopeptide of type I collagen (NTX-I and CTX-I, respectively) for collagen-related proteins, and cathepsins, tartrateresistant acid phosphatase (TRAcP), and bone sialoprotein for non-collagenous proteins in the serum (Seibel, 2005). Normally, the bone formation and bone resorption are tightly coupled to each other in the fetal development (Naylor & Eastell, 2012), therefore, the biomarkers of bone formation and resorption are useful for reflecting the properties of fetal bone development.
Maternal malnutrition is a major factor contributing to the adverse pregnancy outcomes for human beings and livestock. Some previous researches had reported that maternal malnutritional intake during middle-or late-gestation can also negatively impact fetal growth trajectory, and result in intrauterine growth retardation (IUGR) (Sharma, Shastri & Sharma, 2016), fetal growth restriction (Malhotra et al., 2019), low birth weight (Colón-Ramos et al., 2015), high cardiovascular disease (Zohdi et al., 2014), and fetal kidney development issue (Wood-Bradley et al., 2015). MNR affects fetal development by first to support the brain, heart and liver development by limiting bone development, further results in long-term consequences on postnatal health of offspring (Zhu et al., 2006). For adults, the influences of dietary nutrients on mesenchymal stem cells, including osteoblasts and osteoclasts, are complex, some nutrients, like calcium, magnesium, silicon, vitamin D/K/A/C/B, protein, iodine, docosahexaenoic acid, phosphorus, potassium and boron, can promote bone formation, whereas others (excessive zinc, manganese, copper) may have adverse effects on bone formation (Price, Langford & Liporace, 2012).
Bone is identified as an endocrine organ regulating glucose and energy metabolism (Lee et al., 2007). The lacking of osteoblast-secreted osteocalcin in mice decreases β-cell proliferation, glucose intolerance and insulin resistance, further proving that skeleton has been accompanying with an endocrine regulation in sugar homeostasis (Lee et al., 2007). The maternal nutrition status is the key factor for fetal bone development. For instance, the insufficiency of maternal vitamin D is associated with fetal health issue in later life, childhood rickets, schizophrenia and type 1 diabetes (Mahon et al., 2010). Vitamin D insufficiency can also influence the fetal femoral development (Wheeler et al., 2018).
Meanwhile, it has been demonstrated that a short-term maternal energy restriction in late-gestation of beef calves influences gene expression related to energy metabolism, immunity, stress response and muscle contraction (Sanglard et al., 2018). Impacts of prenatal nutrition on ruminant production and performance have also been investigated: a focus on growth and metabolic and endocrine function in sheep (Khanal & Nielsen, 2017). Therefore, it would be essential to depict the fetal bone development profiles with the maternal bone metabolic markers, to monitor fetal bone development performances with maternal bone metabolic proteins.
Machine learning (ML) is a kind of artificial intelligence with the statistical methods for clinical medicine data classification. Until right now, several ML techniques have been widely applied in clinical bone metabolism disease or bone researches with higher accuracy performance for diagnosis of osteopathy. In here, we briefly summarize a few successful applications of machine learning on osteopathy, such as the osteoporosis risk assessment for postmenopausal women (Yoo et al., 2013), pediatric bone age assessment (Halabi et al., 2019), the occurrence of bisphosphonate-related osteonecrosis (Kim et al., 2018), bone surface modifications (Dominguez-Rodrigo, 2019), trabecular bone mechanics (Sohail et al., 2019), or bone marrow associated with relapsed acute leukemia (Li et al., 2019a;Li et al., 2019b). However, none of them reported the association of fetal bone development with maternal bone metabolites by machine learning techniques.
Herein, we searched for the best classification models that will be able to predict the bone parameters (femur and humerus weight, diameter and length) of the fetus by using Machine Learning methods with goat serum metabolic data and experimental conditions as inputs. Previous works demonstrated the efficiency of mixing original inputs with the experimental conditions into experiment-centered features (Moving Averages, MAs) (Liu et al., 2017a;Liu et al., 2017b;Ran et al., 2016). We compared this methodology with the classical machine learning (ML) where the experimental condition information is included as input features using one-hot representation (Pedregosa et al., 2011). The power of ML algorithms had been successfully demonstrated in many biomedical applications (Le & Ou, 2016;Le, 2019;Le et al., 2019).

Experimental design and animal management
All the protocols for use of animal and experimental procedure in present study were approved by the Animal Care Committee according to the Animal Care and the Use Guidelines of the Institute of Subtropical Agriculture, Chinese Academy of Sciences (ISA-CAS), Changsha, China (No. KYNEAAM-2015-0009).
The experiment was conducted to discover the influence of MNR on the fetal bone growth performance in the middle (from days 45 to 100) and late gestation (from days 96 to 135) periods. Forty healthy female goats (Liuyang black goat, a Chinese local meat-production goat breed), in 2nd parity with the similar initial body weight and genetic background were collected and assigned to four groups (10 maternal goats for each) in a completely equally randomized design. These groups included the control (C) group in middle gestation (C-M), maternal nutrition restriction (R) group in middle gestation (R-M), C group in late gestation (C-L) and R group in late gestation (R-L), respectively.
The goats in control group were fed with 100% maintenance requirements of meatproducing goats of China (MpGC-2004), where, R group with 60% maintenance requirements of MpGC-2004(Zhou et al., 2019. All experimental goats were provided by Liuyang Black Goats Nutritional Metabolism Innovation Breeding Base of ISA-CAS, natural crossing with breeding male goats after synchronization. The mating day was recorded to calculate the gestation day. All maternal goats used were free-range grazing before the experiments of maternal nutritional restriction. In the trial period, each pregnancy goat was kept in a well-ventilated individual cage with the proper temperature and humidity with free access to clean drinking water in an adaption of 3 days. The average daily intake was measured and recorded for each pregnant goat. The ratio of concentrate to roughage (fresh crushed Miscanthus spp.) fed was set as 4:6 for middle gestation, and 6:4 for late gestation. The ingredients of feed concentrate on a dry matter (DM) basis are shown in Table 1. In general, the amount of intake increased gradually with increasing gestational days. The initial daily intake of pregnancy goats was around 0.85-0.95 kg, 1.15-1.25 kg for middle and late gestation periods, respectively. The feed In the gestation days 100 and 135, maternal goats were slaughtered and used to collect bone and blood samples. During the experimental period, some pregnancy goats aborted for some uncontrolled reasons. In total, twenty-four maternal goats (nine goats for C-M group, six goats for R-M group; four goats for C-L and five goats for R-L) were slaughtered to collect bone and blood samples. In addition, 34 fetal goats were collected for sampling in different treatments and gestation periods, including 10 fetuses in C-M, 10 fetuses in R-M, six fetuses in C-L and eight fetuses in R-L, respectively.

Sample collecting and analysis
The vein blood and umbilical cord blood samples were collected with normal vacuum blood collection tubes from jugular vein and umbilical cord before slaughter, respectively. All blood samples were allowed to stand for 2 h, centrifuged at 3,000 g/min for 15 min at 4 • C, and separated into 1.5 ml centrifugation tubes, stored at −80 • C refrigerator until further analysis. After slaughtered, the left femur and left humerus of the young and maternal goats were collected entirely and peeled off the associated soft tissue completely. The bone samples were washed with normal saline, dried with filter paper, and recorded the bone weights. The length and diameter of the middle spine of bone were measured with a Vernier Caliper and recorded all the parameters.

Original experimental data resources
In present work, the fetus and maternal femur and humerus bone weights were collected and measured in the conditions of maintenance nutrient requirements (MNR) and nutrition restriction (60% MNR) for goats in the middle and late gestation periods. We also determined the serum bone metabolic proteins of young and maternal goats, including PTH, BALP, BGLAP, INTP, TRAP, and CTX-I. In total, there were 61 fetus instances for the bone parameters and 58 instances for the goat serum bone metabolic proteins. As bone parameters there are femur weight (Fw, g), femur length (Fl, mm), femur diameter (Fd, mm), humerus weight (Hw, g), humerus length (Hl, mm), and humerus diameter (Hd, mm). The values were measured in three types of experimental conditions (ECs): animal type (Animal), gestation periods (Period), and nutrient restriction (Treat). Animal included 34 foetuses and 24 maternal goats, gestation periods were about middle and late gestations, and nutrient restriction parameters were 100% or 60% nutrient requirements for pregnancy maternal goats.

Dataset construction and predictive modelling
The entire project was structured in two principal parts: (1) dataset pre-processing before ML and (2) building of ML classifiers that are able to predict the bone parameters using serum metabolic data and experimental conditions information. Each part had several sections presented below (see Fig. 1).
The entire project calculations were included into two open repositories, GitHub and GitLab (Munteanu & Liu, 2019a;Munteanu & Liu, 2019b). At first, two scripts (Bones_1_CreateMA.ipynb, Bones_2_CreateMAi.ipynb) were integrating the data, transforming the output variables into classes, and calculated new features such as Moving Averages (MAs, experimental-center transformations = difference between original values and the average of values in specific ECs) of six bone metabolic features using 3 ECs (see Eq. (1)), and added probabilities of the separated and mixed ECs.
Feature ij is the bone metabolic input feature j measured in experimental condition EC i. Avg i (Feature ij ) represents the mean of all metabolic features j in EC i. These are the script transformations of the initial data: • Bone parameters and metabolic data were integrated using the data from serum ID and ECs: SerNum, Animal, Treat, and Period. A dataset with 56 instances was created.
• The output variables were transformed using Robust scaler (manage better the outliers) for each type of Animal. The continuous values of the output variables were converted into classes using a cutoff = 0 as the median of the values because of the Robust transformation. Six output variables would be used for the classifiers: Fw_Class, Fl_Class, Fd_Class, Hw_Class, Hl_Class, Hd_Class. The reason for separated transformation is that the values of the Fetus and Mon are already very different and a general scaling + cutoff will separate the values by animal type. In consequence, the models would predict if an output is from Fetus or Mon, but not if it is low or high values. Our models would be predictable that a specific output variable had low and high values. • One-hot encoding of ECs to be used as inputs in classical ML: one column for each value of the ECs (6 new features = 3 ECs * 2 values): Animal_Fetus, Animal_Mon, Treat_Con, Treat_Res, Period_Late, Period_Mid; • Added probabilities for each ECs in the dataset (P i = 3 new features for each of 3 ECs): Prob_Animal, Prob_Treat, Prob_Period; • Added probability for mixed ECs in the dataset (1 new feature for 1 set of ECs): Prob_Mix (P mix ).
In the next script (Bones_3_CreateDatasets.ipynb), different types of data were used to generate 72 datasets in order to check what data were needed for the best classification model that can predict bone parameters using metabolic data: six output variable classes, six one-hot encoding of ECs, six metabolism, six mixed MAs of metabolic data, 18 MAs of metabolic data for each ECs, one mixed probability, and three EC probabilities.
With these 72 datasets we tested seven ML classifiers with default parameters from sklearn/python (504 prediction models): KNN (KNeighborsClassifier), SVM linear (Support Vector Classifier, SVC, with linear kernels, SVM, Support Vector Machines), SVM (SVC with RBF kernels), LR (LogisticRegression), DT (DecisionTreeClassifier), RF (RandomForestClassifier, RF, Random Forest), and XGB (XGBClassifier, XGB, XGBoost). The used pipelines consisted in (1) scaling input features with RobustScaler and the classifier. Two types of cross-validation were tested: n-fold CV (n = 2, 3, 5, and 10) and LOOCV (Leave-one-out cross-validation). The best CV for a small dataset is always LOOCV. The results for n-fold CV are presented only in the project repository. Balanced accuracy was used as optimization score for ML methods in the case of n-fold CV (Bones_4_OuterCV-Pipelines-ACC.ipynb) and LOOCV (Bones_4_OuterCV-Pipelines-LOOCV.ipynb). The scripts tested both the MA datasets and the classical ML datasets using one-hot encoding of ECs (Bones_4_OuterCV-Pipelines-LOOCV-OneHot.ipynb). Therefore, we tested if MAs inputs were better than the original metabolic inputs with one-hot encoding ECs. In other words, we tested what methodology was better for our task: the encoding of ECs into MAs (mixed information of metabolism and ECs) or the one-hot representation of ECs as input binary variables (1/0 values). In ML, the categorical features cannot be handled by all the ML methods. Therefore, there is a need for the transformation of the categorical values into numerical ones. The one-hot encoding is used when there is no natural order between the categories. If we use the simple integer encoding (an integer for any category), we will allow the model to assume a natural ordering between categories. This could produce poor performance or unexpected results.
In addition, we evaluated the feature importance by feature removal method for each of the best models. Thus, for each linear or nonlinear SVM model, we removed each input variable and calculated the new ACC values (LOOCV). (see Bones_5_FeatureI-mportance-Pipelines-LOOCV.ipynb). The differences between the ACC with pool dataset (all features) and the new ACC for each dataset without one feature were presented. If the removal of a feature will determine the decrease of the ACC, this feature is considered important. If the removal of a feature will not change the ACC, the feature is not important for the model. If the removal of one feature will increase the ACC, this feature represents a noise for the model and it should be eliminated. For the best models for each output, a different feature selection was tested: create models with the same ML method and dataset but using only one feature (Bones_6-OneFeature-Models-LOOCV-ACC.ipynb).

Fetal bone developmental profiles
The effects of MNR on gestation goats for fetal bone growth profiles (including weight, length and diameter of femur and humerus) are presented in Table 2. In the middle gestation, MNR improved the fetal bone weight, but decreased the length and diameter of bone (femur and humerus). In the late gestation, MNR enhanced the weight and diameter  of humerus and femur, and humerus length, but decreased the femur length. However, for the local goat nutritional research, except the nutritional status providing the nutritional requirements of fetal skeletal development, the genetic factors of each individual might also play an important role in fetal skeletal development.

Nutritional restriction on goat bone metabolic indexes
Except the nutritional supplementation, the regulators or biomarkers of bone metabolism were also taken into consideration for fetal bone development. Here, we measured the goat bone metabolic proteins (PTH, BALP, BGLAP, TRAP, INTP and CTX-1) for maternal jugular venous serum and fetal umbilical cord serum (showed in Table 3 and Table 4, respectively). For maternal goats, the nutritional factor, MNR, reduced the levels of PTH, BALP, BGLAP, TRAP, INTP and CTX-1 in peripheral serum. Under the malnutritional condition, low maternal BGLAP level represents low efficient maternal bone turnover rate. However, our results showed that the other biochemical markers of bone turnover, TRAcP, had no difference in malnutrition condition. Compared with significant differences in bone metabolic proteins for maternal individual, fetal bone metabolic proteins showed a minor variation in the control and malnutrition groups. The  NTx and CTx were reduced in response to maternal malnutrition, CTx was obviously lower in the middle-gestation, while NTx significantly lower in late-gestation. The PTH for maternal malnutrition was higher or lower in middle-and late-gestations, respectively. Whereas, the fetal BALP, BGLAP, and TRAP showed no statistical difference between the control and malnutritional groups.

Machine learning predictive models
Seventy-two datasets were used to find the best ML model to predict six bone parameters by using seven ML methods such as KNN, SVM linear (linear kernels), SVM (RBF kernels), LR, DT, RF and XGB. The input features came from 6 one-hot encoding of ECs, six metabolic original data, six mixed MAs of metabolic data, 18 MAs of metabolic data for each ECs, 1 mixed probability, and three EC probabilities. The Python script was used Robust scaler for data scaling, cross-validation (n-fold CV, where n = 2, 3, 5, 10 and LOOCV), and balanced accuracy as performance score. Table 5 presents the best accuracy (ACC) values obtained for all outputs using only LOOCV. For 5-, 10-fold CV results or other details, see the repository in GitLab (Munteanu & Liu, 2019a) or GitHub (Munteanu & Liu, 2019b).
The best CV method for a small dataset was the LOOCV one. The results based on this CV showed models with all ACC >0.70, with classifiers of five from six outputs with ACC between 0.75 and 0.90, and one classifier with ACC >0.90. In this study, we observed the performance of SVM method with bone dataset and LOOCV (all classifiers are SVM/SVM linear). The most preferred inputs were MAmix, and Metab. MAi was used only for one output and probabilities are missing. The best classifier with ACC of 0.911 was obtained for prediction of Fw by using SVM with RBF non-linear kernels and three types of inputs such as MAi, MAmix, and Metab. There were three outputs (Fl, Hw and Hl) where the best models used only six MAmix features (MAs for the set of ECs: MA-PTH-experim, MA-BALP-experim, MA-BGLAP-experim, MA-INTP-experim, MA-TRAP-experim, and MA-CTX-I-experim). The model for Fl prediction could achieve an ACC of 0.875 with only 6 input features (MAmix) and a SVM model with linear kernels (equivalent of a linear model). Fd prediction model increased a little with the ACC of 0.893 by adding to MAmix the Metab features (6 original metabolic features) and by using SVM with RBF non-linear kernels. Hw could be predicted with less ACC of 0.75, with only MAmix inputs and SVM with linear kernels. The model for Hd prediction had a good ACC of 0.857 using SVM with RBF kernels and MAmix + Metab inputs. An interesting pattern was observed for the preference of input information for each type of bone parameter: MAmix was used of bone length (Fl, Hl) prediction, and MAmix + Metab for bone diameter (Fd, Hd) prediction. In the case of bone weight (Fw, Hw), the preference of inputs was different.
The last test was the comparison of the best ACC values obtained by different inclusion of EC information into the model inputs: classical one-hot encoding of ECs as binary inputs for each categorical value versus MA encoding as difference between the original metabolic data and the average of this data in specific ECs. In Fig. 2, it could be observed that the classical use of categorical EC values and original metabolic features was unable to provide good results with the current small dataset for bone parameters prediction. Thus, one-hot encoding of ECs was able to classify only Fd with ACC of 0.714 (versus 0.893 with MAs) and Fw with ACC of 0.82 (versus 0.911 with MAs). One-hot encoding models were not able to predict Hd, Hl and Fl (ACC < 0.60). This test demonstrated the need of MAs to be able to classify low and high values of bone parameters. Additional details about the one-hot encoding results can be found in our repository notebooks (Munteanu & Liu, 2019a).

Feature importance
To evaluate the feature importance for the best model for each output, we presented a feature selection method by individual feature removal (see Bones_5_Feature-Importance-Pipelines-LOOCV.ipynb). The more an attribute is applied for making key decisions with decision trees, the higher its relative importance (Tang, Alelyani & Liu, 2014). This importance is calculated explicitly for each attribute in the dataset, allowing attributes to be ranked and compared to each other. The ranking of features was based on the difference between the ACC calculated without a feature (new-ACC) and original ACC calculated with all features (see Table 6). If the difference is smaller (more negative), it means that, if you remove that specific feature, the ACC of the model will suffer more (feature is more important). If the ACC difference is positive, it means that, if you remove that specific feature, you will obtain a better ACC (feature not important or even noise for the model). All details are available in our GitLub repository (FeatImpbyRemoval-LOOCV_SVMlinear_ACC.csv, FeatImpbyRemoval-LOOCV_ACC.csv).
In the case of Fw best model (SVM RBF, ACC = 0.911), the most important features were MA-PTH-Animal and MA-BALP-experim with a decrease of ACC of 0.054 and 0.036. In addition, MA-PTH-experim represents a noise for the model and its removal increased the ACC of the model with 0.035. Thus, the best model for Fw could be considered the one obtained without MA-PTH-experim, with test ACC of 0.946. Therefore, PTH moving average for each type of animal was important for the model but not the PTH moving average in mixed ECs. The feature importance for Fd best model (SVM RBF, ACC = 0.893) was different. The impact of feature removal of moving averages for INTP, PTH, CTX-I, and BALP in mixed ECs was very important, with differences in ACC between 0.125 and    As a general view for the feature importance, it is noticeable that the moving averages in mixed ECs were generally more important for the majority of the models. MA-PTHexperim was important for Fd, Hd, and Hl but its removal for Fw, Fl and Hw improved the model performance. In general, we observed the important role of PTH and BALP mixed with ECs.
With the best models and ML methods obtained Table 5, one feature models have been tested for all outputs. Therefore, Table 7 presents the ACC values for these models and the differences with the pool feature models. In the case of the ACC improvements, the text has a bold style. First of all, it should be pointed out that the current study is interested into models that include experimental conditions in order to use them to understand the relations with the fetus bones.
Fw models based on only one feature can be more accurate than the best model presented with the pool dataset. Thus, a better model with ACC values of 1.00 (LOOCV) could be obtained with PTH and TRAP or the mixed feature MA-TRAP-experim. A number of 18 features could be use separately to improve the best models with pool features. INTP and the mixed features with INTP are not able to generate any stable model. In consequence, the best model for Fw (ACC = 1.00) become the one that uses SVM (radial kernel) and MA-TRAP-experim.
In the case of Fd classifications, it can be observed that using simple features without experimental conditions such as BGLAP and PTH can be used as unique feature of prediction models with better ACC (

Nutritional restriction on bone metabolic profiles
Maternal limited nutrition intake during gestation impairs fetal growth and development by exacerbating deleterious outcomes (Abu-Saad & Fraser, 2010;Wu et al., 2004;Wu, Imhoff-Kunsch & Girard, 2012), such as intrauterine growth restriction (IUGR) (Sharma, Shastri & Sharma, 2016), birth defects and low birth weight (Abu-Saad & Fraser, 2010), bone development (Kueper et al., 2015, and low fetal immunity (Macpherson, De Aguero & Ganal-Vonarburg, 2017). Generally speaking, maternal instinct is to mobilize the body potential with the priority to meet the nutritional requirements of the fetal or placental growth in malnutrition condition (Wu, Imhoff-Kunsch & Girard, 2012). In present work, the biomarkers of bone resorption (BGLAP, TRAP, and CTX-1) or bone formation (BALP, and INTP) in serum reduced in goat group under nutrient restriction, indicating maternal malnutrition indeed influences the maternal or fetal bone metabolic turnover (Kovacs, 2001;Wu et al., 2004;Wu, Imhoff-Kunsch & Girard, 2012). According to the previous  ACC values corresponds to LOOCV for models the ML methods of the best models (see Table 5). report, fetal growth and development is most vulnerable in the early-gestation (Wu et al., 2006). However, the influence of maternal nutrition intake also plays the vital role for fetal bone development in middle-and late-gestations. Under the malnutritional condition, low maternal osteocalcin (BGLAP) level represents low efficient maternal bone turnover, which may be decreased during pregnancy (Charles et al., 1985;Naylor et al., 2000).
The results also showed that fetal bone metabolites showed a minor discrepancy compared to high difference of mother in malnutritional condition. The BALP, a bonespecific isoform of ALP on the surface of osteoblasts, an important biomarker reflects the biosynthetic activity of bone-forming cells (Shipman, Holt & Gama, 2013;Tanaka et al., 1997). In addition, BGLAP level in the circulation system is also a symbol for evaluating the bone formation (Booth et al., 2012;Mendes et al., 2019). The serum BALP level decreased under the nutrition restriction, reflecting that malnutrition (protein deficiency, zinc deficiency or malnutrition) negatively regulates the secretion of maternal bone alkaline phosphatase (Sun et al., 2011). Maternal malnutrition might decrease the glucose metabolic process by decreasing osteocalcin level in maternal peripheral blood by regulating the insulin secretion and sensitivity (Ferron & Lacombe, 2014;Kanazawa et al., 2009) in middle-gestation.
In adults, the increasing TRAcP expression might be associated with the osteoporosis (Solberg et al., 2014), cortical bone mineral content and density (Gradin et al., 2012). Interestingly, in present work, we found that the biomarkers of bone turnover, the bone metabolic markers, including NTx, CTx, BALP, BGLAP and TRAP except PTH, decreased along with gestation time (from middle-gestation to late-gestation). The BGLAP concentration increased in maternal serum compared to that of fetus, reflecting enhanced osteoblastic activity. The bone turnover metabolic proteins decreased in late-gestation is due to maternal goats giving the priority to meet the nutritional requirements of fetal skeletal development through decreasing the self-bone turnover or resorption. The mother limits self-bone turnover/resorption rate or further to release more mineral contents of maternal bones to sustain the fetal bone development while undernutrition issues.

Machine learning predictive models for bone metabolic profiles
Machine learning (ML) is a kind of artificial intelligence with the statistical methods for clinical medicine data classification. Until right now, several ML techniques have been widely applied in clinical bone metabolism disease or bone researches with higher accuracy performance for diagnosis of osteopathy. In here, we briefly summarize a few successful applications of machine learning on osteopathy, such as the osteoporosis risk assessment for postmenopausal women (Yoo et al., 2013), pediatric bone age assessment (Halabi et al., 2019), the occurrence of bisphosphonate-related osteonecrosis (Kim et al., 2018), bone surface modifications (Dominguez-Rodrigo, 2019), trabecular bone mechanics (Sohail et al., 2019), or bone marrow associated with relapsed acute leukemia (Li et al., 2019a;Li et al., 2019b). For instance, the variability of sclerostin (a physiological inhibitor of bone formation) varied with sex, 25-OH-D and phosphorus levels in haemodialysis (HD) patients by multivariate regression analysis (Pietrzyk et al., 2019), low serum parathyroid hormone associated with malnutrition-inflammation complex in chronic kidney disease by Chis-square test, linear regression and multivariate logistic regression analysis (Dukkipati et al., 2010). In addition, the abnormalities of biochemical markers of bone turnover, ALP and PTH, are associated with mortality of HD patient across age by Cox proportional hazard models (Lertdumrongluk et al., 2013), and a decrease in intact parathyroid hormone (iPTH) is associated with higher mortality (Villa-Bellosta et al., 2017). There are also some previous works that combined with the machine learning algorithms with data processing analysis of moving average and feature selection to biological phenomena (Deng et al., 2017;Liu et al., 2016a;Liu et al., 2016b). However, none of them reported the association of fetal bone development with maternal bone metabolites by machine learning techniques.
In present work, seven ML methods, including KNN, SVM linear (linear kernels), SVM (RBF kernels), LR, DT, RF and XGB, were used to find the best ML model to predict 6 bone parameters. Then, the feature selection method within individual feature removal was used to evaluate the feature importance for each output (predictive parameter). To some extent, we got some good performance predictive models for matching bone profiles with serum bone metabolic biomarkers. Generally, it can be observed that isolated PTH and its mixed features are able to improve the prediction of bone dimensions. The next important features are BGLAP and its mixed features. Therefore, it could be concluded that some original features such as PTH and BGLAP are the most important for fetal bone development performance predictions and experimental-mixed features could be used to predict bone features in specific experimental conditions. With other words, the original features could predict bone properties but the experimental conditions could offer more details.

CONCLUSIONS
The maternal and fetal bone metabolic proteins decrease from middle to late gestation. Maternal nutrition restriction alters the bone development of offspring (bone weight, length and diameters of femur and humerus) and the fetal bone metabolic protein levels, including CTx level in middle-gestation and NTx level in late-gestion. Maternal nutrition restriction also influences the maternal bone metabolic protein compositions, resulting in lower BALP level in middle-gestation and higher BALP level in late-gestation. Furthermore, we constructed a machine learning model to elucidate the fetal bone performances associated with maternal or placental serum bone metabolic proteins. This study built the classifiers with accuracy greater than 0.70 for all bone parameters. In the view of the feature importance, the moving averages in mixed with experimental conditions are generally more important for the majority of the models. Particularly, the moving average of PTH within experimental condition (MA-PTH-experim) is important for Fd, Hd, and Hl but its removal for Fw, Fl and Hw improves the model performance. In general, we observe the important role of PTH and BALP mixed with experimental conditions. In addition, using only one feature with the Machine Learning methods of the best models, five of the six outputs can be improved: Fw with MA-TRAP-experim (ACC = 1.00), Fd with PTH (ACC = 0.982), Fl with MA-PTH-experim (ACC = 1.00), Hw with MA-INTP-experim (ACC = 1.00) and Hd with MA-BALP-experim (ACC = 0.946). Hl best model using mixed features has ACC of only 0.696. Thus, the best classification models with ACC = 1.00 were obtained for the prediction of femur weight and length, and humerus weight using support vector machines.