Backpropagation Neural Network-Based Machine Learning Model for Prediction of Soil Friction Angle

In the design process of foundations, pavements, retaining walls, and other geotechnical matters, estimation of soil strengthrelated parameters is crucial. In particular, the friction angle is a critical shear strength factor in assessing the stability and deformation of geotechnical structures. Practically, laboratory or field tests have been conducted to determine the friction angle of soil. However, these jobs are often time-consuming and quite expensive. erefore, the prediction of geo-mechanical properties of soils using machine learning techniques has been widely applied in recent times. In this study, the Bayesian regularization backpropagation algorithm is built to predict the internal friction angle of the soil based on 145 data collected from experiments. e performance of the model is evaluated by three specific statistical criteria, such as the Pearson correlation coefficient (R), root mean square error (RMSE), and mean absolute error (MAE). e results show that the proposed algorithm performed well for the prediction of the friction angle of soil (R� 0.8885, RMSE� 0.0442, and MAE� 0.0328). erefore, it can be concluded that the backpropagation neural network-based machine learning model is a reasonably accurate and useful prediction tool for engineers in the predesign phase.


Introduction
e internal friction angle is one of the most important parameters in analyzing soil geotechnical properties. It characterizes the soil shear strength and is determined by the Mohr-Coulomb failure standard [1]. e determination of shear strength parameters, including the effective soil friction angle, is vital for assessing the stability and deformation of geotechnical structures such as foundations, slopes, and retaining walls [2][3][4][5][6][7][8]. Generally, the determination of geotechnical parameters is performed in the laboratory and several others are estimated on the field [9]. Each geotechnical parameter depends on different factors, in which the internal friction angle of the soil depends on several factors such as density, particle size distribution, angle, and interlacing of particles. erefore, based on the ground profile properties, various tests such as the direct shear test and the triaxial test are recommended to obtain an internal friction angle parameter [10]. However, these results may not fully represent the correct soil properties due to the potential that soil may be disturbed during sampling [11,12]. While performing sampling, these experiments are often time-consuming and expensive [5]. To overcome the above limitations, Salari's study proposed an equation for determining the internal friction angle of the soil using the standard penetration test for different soil types [10]. Besides, the work of Motaghedi and Eslami [13] has proposed an approach of predicting the unit cohesion and the friction angle from the cone penetration test (CPT) considering the bearing capacity mechanism of failure at the cone tip and direct shear failure along the penetrometer sleeve. In fact, the soil formation is unsimilar in different regions, so the correlations developed for one region might not be applied to another [9]. erefore, accurate prediction of the internal friction angle of the soil is a critical task in geotechnical design, at the same time, to save time and reduce costs in construction projects [13].
In recent years, machine learning (ML) techniques have been widely applied in a variety of fields related to computational mechanics [14,15], structural engineering [16,17], environmental engineering [18], materials science [19,20], and Earth sciences [21][22][23][24][25][26]. Artificial neural network (ANN) is currently one of the popular models due to its structural flexibility, excellent predictive performance, and availability of a significant number of training algorithms [27,28]. e backpropagation algorithm (BP) is widely used to adjust ANN's parameters [29]. Such an algorithm uses a set of input and output values to find the desired weight and bias of the neural network. However, in traditional BP networks, there are some shortcomings, such as the low convergence speed and an easy fall to the local minimum [30]. erefore, in order to minimize the error related to the backpropagation algorithm, some generalization methods such as Bayesian regularization (BR) [31] and Levenberg-Marquardt (LM) [32] are employed, owing to their advantage in obtaining a lower mean squared error. As an example, Kayri's study [33] showed that BR performed better than LM. In addition, BR algorithm has been used successfully in many areas, such as data mining, stock price volatility prediction, and stock market prediction [34][35][36].
To the best of the authors' knowledge, there are currently limited studies proposing the ANN model to estimate the soil internal friction angle [6,9,37]. However, the feasibility of using BR algorithm has not been investigated to predict such an important soil property.
In this study, Bayesian regularization-based algorithm coupling with neural networks is employed to predict the soil internal friction angle. Common evaluation indicators, such as the Pearson correlation coefficient (R), mean absolute error (MAE), and root mean square error (RMSE) are used to evaluate the performance of the proposed model. A database containing 145 experimental results is collected from the Danang-Quang Ngai expressway project, Vietnam, and used to develop the ML model. e construction and statistical analysis of the database are presented in Section 2. Next, a brief introduction of the ANN using Bayesian regularization backpropagation algorithm is presented in Section 2.2. e methodology in the study and the performance indicators of the ML model are given in Section 2.3, followed by the results and discussion in Section 3. Finally, several conclusions and perspectives are given in Section 4.

Materials and Methods
e present study is carried out based on the proposed methodology that comprises three main steps as follows: (1) data preparation, (2) construction of the model, and (3) validation of the proposed model. Data preparation: in this first step, the data taken from the laboratory tests are employed to create two datasets: the testing and training dataset. e training dataset is generated from 70% of total data, whereas the testing dataset is built from 30% of the remaining data. Construction of the models: in this second step, the training dataset is employed for training the ANN model based on the Bayesian regularization backpropagation algorithm. In this step, the effects of the number of iteration (or epochs) and random sampling technique are investigated. Validation of the proposed models: in this final step, the testing dataset is adopted for validating proposed models. Statistical indicators, including RMSE, MAE, and R are employed to validate the model.

Database Collection and Preparation.
In fact, the soil internal friction angle (denoted as φ) is affected by many factors. However, this study will focus on the main factors that significantly govern the soil internal friction angle to reduce the model complexity. In the current research, 145 data of soil samples were collected from Da Nang-Quang Ngai expressway project, as shown in Figure 1. en, the soil samples were tested in the laboratory to determine input parameters, namely, clay content (X 1 ), natural moisture content (X 2 ), liquid limit (X 3 ), plastic limit (X 4 ), specific density (X 5 ), and void ratio (X 6 ), and the output of these parameters in modeling is the soil internal friction angle. Detailed definitions and how to determine the input variables from particle composition analysis in the laboratory can be found in [5,38]. In the collected dataset, the value of the clay content varies in the range of 4.09-47.96%, the natural moisture content is in the range 15.53-115.41%, the liquid limit varies from 20.8-154.12%, the plastic limit ranges between 13.42 and 63.96%, the specific density value varies from 2.59-2.75 g/cm, and the void ratio ranges from 0.58-3.25. Besides, the friction angle values were in the range of 0.04 to 0.37 rad. Table 1 details the symbol, unit, and role, as well as the statistical analysis (minimum, maximum, mean, standard deviation, and skewness) of the six input variables and one output variable. In addition, 145 data used in this work are randomly divided into two subdatasets using a uniform distribution, of which 70% of the data is used for training of the ANN model and 30% of the remaining data is used for validating the model. All data is scaled to the range [0-1] to reduce numerical error during ANN processing, as recommended by [39]. is process ensures that the training phase of the AI models can be performed with functional generalization capabilities. Such proportions are represented by where x max and x min are the maximum and minimum values of the considered variable and x n is the normalized value of the variable x. For illustration purposes, Figure 2 shows a histogram of all parameters in this study.

ANN Bayesian Regularization Backpropagation Algorithm (ANN-BR
). An artificial neural network (ANN) is a powerful machine learning-based data analysis algorithm [40]. is machine learning approach attempts to simulate the process of knowledge acquisition and inference occurring in the human brain [41,42]. ANN has been widely used   to address nonlinear regression analysis problems. It has been demonstrated that an ANN with a hidden layer can simulate very complex nonlinear functions [43]. To create a reliable model, proper training of a neural network is the most important. Backpropagation is an algorithm commonly used to train neural networks [44]. Typical backpropagation networks typically use a gradient descent algorithm such as Widrow-Hoff arithmetic. In this network, weights are changed or moved along the negative value of the gradient of the executing function. e term backpropagation propagation is used because it relates to the way the gradual computation of nonlinear multilayer neural networks performed. However, some backpropagation training algorithms, such as gradient descent, have a slow convergence rate [45]. erefore, one of the algorithms that improve the convergence or learning rate of the neural network is the backpropagation training network, according to the Bayesian regularization algorithm.
Bayesian regularization is the linear combination of Bayesian methods and ANN to determine the optimal regularization parameters automatically. In contrast to conventional network training, in which the optimal weight set is chosen by minimizing the error function, the Bayesian approach involves the probability distribution of network weights. As a result, the network predictions are also a probability distribution [46,47]. In the training process, a common performance function is used for computing the distance between real and predicted data, such as the mean sum of squared network errors: For the purpose of improving the generalization of the model, the gradient-based optimization algorithm is preferred to minimize the target [48,49]. e target function in equation (2) extended with the addition of a term E w which is the sum of the squares of the lattice weights: Here, the α and β are parameters that are to be optimized in the Bayesian framework of MacKay [50]. For the purpose of finding the optimum regularization parameters, a Bayesian regularization method is employed. e optimal regularization parameters can so be obtained in an automated fashion. Bayesian optimization of the regularization parameters requires the computation of the Hessian matrix of the objective function F. However, the optimal  regularization technique requires the costly computation process of the Hessian matrix. To overcome this drawback, Gauss-Newton approximation to the Hessian matrix is used. e approximation with Bayesian regularization backpropagation algorithm for network training [51] is used in this study.

Performance Indicators.
Evaluating the model accuracy is an essential part of the process in creating machine learning models to describe how well the model is performing [52,53]. In this research, the mean absolute error (MAE), root mean square (RMSE), and Pearson correlation coefficient (R) are used to evaluate the predicted error rate and proposed model performance. MAE represents the difference between the original and predicted values, extracted by averaging the absolute difference over the chosen dataset (equation (4)) [54][55][56]. Besides, RMSE is the error rate by the square root of MSE, as shown in equation (5) [5,57,58]. R is an important indicator of regression analysis [59,60].
e R index represents the correlation between the predicted results and the actual output, varying from −1 to 1, as shown in equation (6). e closer the absolute value of R is to 1, the better the model is [61]: where N is the number of samples in the database, r 0 , and r 0 are the actual experimental value and the average real experimental value, and r t and r t are the predicted value and the average predicted value, calculated according to the model forecast.

Analysis of the Number of Iteration.
In this section, the optimization of the weight parameters of ANN is presented using the BR algorithm. e performance of the ANN model depends on the structure of the neural network (NN), that is, the number of hidden layers and the number of neurons in each hidden layer. Depending on the issue of interest, predictive results can show a significant change from architecture usage to architecture [62,63]. When the number of inputs and outputs is fixed, the undefined architecture parameters are the number of the hidden layer (s) and the number of neurons in each hidden layer (s) [64]. erefore, the number of hidden layers is usually determined firstly, based on the complexity of the relationship between input and output. e process of building the network structure is, thus, the process of trial and error test. Some studies have shown that most specific problems using only one hidden layer can be enough to successfully solve the complicated nonlinear relationship between input and output [65,66]. In this study, the number of selected hidden layers is one, and the number of neurons in each layer changes from 1 to 20. e results show that the ANN structure [6-10-1] provides optimal performance. e structure of the ANN model is illustrated in Figure 3.
Training dataset with 102 samples with six input parameters and one output parameter is used to build the ANN tool. In the backpropagation neural network, the training iteration parameter can have a significant effect on the generalization accuracy [67,68]. e generalization accuracy and the neural network architecture training are directly influenced by whether the number of training iterations is small or large [69,70]. In this study, the impact of training iterations on ANN application using Bayesian regularization backpropagation algorithm is analyzed. e number of iterations varies from 100 to 500, with a step of 100. Figure 4 depicts the effect of the number of iteration on the values in function of the statistical error criteria, with 25%-75% interpreted as the value in the first and the third quartiles, and StD is the standard deviation. e results show that, considering the value of RMSE and MAE, 100 iterations provide the lowest error and StD, and the mean value of R is highest. For the remaining number of iterations (i.e., 200 to 500), it is seen that the errors are higher. Overall, selecting 100 iterations is the optimal choice to obtain the best prediction results.

Prediction Capability of ANN-BR.
In this work, the evaluation of the effectiveness of the ANN-BR model with the structure [6-10-1] is performed. e correlation between the friction angle of the experimental data obtained (solid line) and predicted values (dashed line) from the training and testing process, according to the ANN model, is shown in Figure 5. e predicted friction angle of 102 samples in the proposed model's training data is quite close to the results of the experiment. With the testing dataset, 43 experimental results are also predicted with small errors.
e error of the model with respect to the training and testing data is shown in Figure 6.   Mathematical Problems in Engineering     Mathematical Problems in Engineering values of the friction angle. Besides, most of the predicted values of training and testing datasets are close to the 95% confidence bounds. It could be concluded that predicting the internal friction angle of soil is possible using the ANN-BR model.
In comparison, we have observed that the performance of the ANN developed in this study is slightly better than Regression Tree (RT) (R � 0.882) used to predict the shear strength of soil in the road construction site of Vietnam [71] and outperforms Adaptive Network-based Fuzzy Inference System (ANFIS) and its hybrid models (R � 0.49-0.61). However, it is noticed that the ML models' performance might depend on the quality of the data used, and its performance might be different for different case studies. erefore, it is required to perform a separate investigation on each case study.

Conclusion
In this study, an ANN model with BR algorithm is proposed to predict the internal friction angle of the soil. A total of 145 experimental results are collected from the Danang-Quang Ngai expressway project, Vietnam, for the construction of the ANN-BR model. e input data for the network training process is clay content, natural moisture content, liquid limit, plastic limit, specific density, and void ratio. ree statistical criteria, namely, the Pearson correlation coefficient (R), mean absolute error (MAE), and root mean square error (RMSE) are used to evaluate the correlation between the predicted values by the ANN-BR model and actual experimental ones. e results show that the ANN-BR model is a good predictor in predicting the internal friction angle of soil, with R � 0.8885, RMSE � 0.0442 (rad), and MAE � 0.0328 (rad) for the testing dataset. e results can help build a reliable soft computing tool for engineers to predict the internal friction angle of soil. However, in machine learning problems, data is the key factor in creating a reliable predictive tool. erefore, collecting additional data to improve the algorithm is the highest aim of the present study, which helps to avoid costly on-field experiments.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.  Mathematical Problems in Engineering