Modelling the Flow Stress of Alloy 316L using a Multi-Layered Feed Forward Neural Network with Bayesian Regularization

In this paper, a multilayer feedforward neural network with Bayesian regularization constitutive model is developed for alloy 316L during high strain rate and high temperature plastic deformation. The input variables are strain rate, temperature and strain while the output value is the flow stress of the material. The results show that the use of Bayesian regularized technique reduces the potential of overfitting and overtraining. The prediction quality of the model is thereby improved. The model predictions are in good agreement with experimental measurements. The measurement data used for the network training and model comparison were taken from relevant literature. The developed model is robust as it can be generalized to deformation conditions slightly below or above the training dataset.


Introduction
Material modelling is a challenging area in manufacturing processes such as machining. The large variation in strain, strain rates and temperature brings in complications leading to the problem of localized deformation in simulation [1]. In localized deformations, state variables such as strain, strain rate and temperature interactions are highly nonlinear and thus complete physical mechanisms are complex and difficult to understand [2]. Constitutive modelling of metals at high strain rates can be categorized into phenomenological, physical-based and artificial neural network (ANN), based models. Phenomenological models [3] are analytical model expressed in a convenient mathematical function. Physical-based models such as the dislocation density based model [4] combine and use physical and thermal deformation mechanisms such as thermodynamics and kinetics of slip. ANN modelling involves using an arbitrary number of computational processing units that learn from observed material data [5].
ANN is most suitable to capture the highly nonlinear relationships of the deformation behaviour of metals at high strain rate and temperature, as there is no need for a priori assumption of the mathematical model or a priori identifying material parameters that could be many. The material behaviour development is based on the experimental data available in training [6]. Also, material model based on ANN is very flexible and dynamic. The model can be extended within the training session as new experimental data about the behaviour of the material becomes available. This work aims to show the accuracy and robustness of ANN using Bayesian regularization in modelling alloy 316L during high strain rate and high temperature deformation. Instead of using gradient descent optimization methods, multi-layered feedforward neural network (MFNN) with Bayesian regularization [7] enables us to minimise overfitting and thus improve the generalization of the neural network model. Experimental data with both low and high strain rates for varying temperatures are utilized in the MFNN model. The data are taken from [8]

Multi-layered feedforward neural network
An ANN is a mathematical model involving a group of interconnected artificial neurons that tries to simulate the neural structure of the human brain. Artif and output data. This training process also called a learning process, is used to determine the neural network parameters for the model. An AN an output layer. The layers are connected by independent processing units named neurons. The number of neurons in the input and output layers is determined by the number of input and output variables respectively. The number of hidden layer or layers depends on the level of complexity suitable for the modelling problem. The number of neurons in the hidden layer or layers is determined during the training process. An example of a neural network with o  presented. This type of learning process is known as supervised learning in which every input pattern has its associated target output [9]. In ANN modelling, the goal is to find a vector of weights such that are used as inputs of the MFNN, and the flow stress σ is the output. Predicted flow stress results from the MFNN model are compared with the experimental measurements. The results indicate good layered feedforward neural network An ANN is a mathematical model involving a group of interconnected artificial neurons that tries to simulate the neural structure of the human brain. Artificial neural networks are trained against input and output data. This training process also called a learning process, is used to determine the neural An ANN is composed of an input layer, hidden layer or layers and an output layer. The layers are connected by independent processing units named neurons. The number of neurons in the input and output layers is determined by the number of input and output s respectively. The number of hidden layer or layers depends on the level of complexity suitable for the modelling problem. The number of neurons in the hidden layer or layers is determined during the training process. An example of a neural network with one hidden layer is depicted in . Data flow only in one direction through the network, from input to output. The mapping of the inputs to the output can be expressed in a MLP network written as are transfer functions also known as activation functions. wandω weight parameters. The transfer function mathematically defines the non-linear relationship between inputs and output of a neuron in a neural network.
. Schematic illustration of neural network architecture The neural network parameters, w and ω , are unknown and they are determined by training the network. For the training set of data comprising a set of input xn, n =1,..., N and a corresponding set , the objective of ANN model is to minimize the sum of square error function . The error function is given as 2 is the target output for the nth output neuron when the nth input pattern is presented to the is the actual output of the nth output neuron when the input pattern is presented. This type of learning process is known as supervised learning in which every input pattern . In ANN modelling, the goal is to find a vector of weights such that is the output. Predicted flow stress results

nts. The results indicate good
An ANN is a mathematical model involving a group of interconnected artificial neurons that tries to icial neural networks are trained against input and output data. This training process also called a learning process, is used to determine the neural is composed of an input layer, hidden layer or layers and an output layer. The layers are connected by independent processing units named neurons. The number of neurons in the input and output layers is determined by the number of input and output s respectively. The number of hidden layer or layers depends on the level of complexity suitable for the modelling problem. The number of neurons in the hidden layer or layers is determined ne hidden layer is depicted in essed in a MLP network written as (1) ω are network linear relationship between , are unknown and they are determined by training the and a corresponding set , the objective of ANN model is to minimize the sum of square error function (2) is the target output for the nth output neuron when the nth input pattern is presented to the is the actual output of the nth output neuron when the input pattern is presented. This type of learning process is known as supervised learning in which every input pattern . In ANN modelling, the goal is to find a vector of weights such that Equation (2) takes its smallest value.
The artificial neural network makes use of an optimization algorithm to determine how good the error on the training input -output data set is minimised. It also determines the number of iteration steps. Overfitting defined as the effect of generalization error resulting from the overly complicated model in an optimization process may occur. Overfitting can be addressed by using Bayesian regularization methods [10,11]. The method automatically incorporates model uncertainty thereby imposing prior probability distribution on the model parameters.
Using Bayesian regularization, E given by Equation (2) is modified by adding another term. The new error function, which can be called, augmented error function becomes where µ and αij are the regularization parameters known as the weight decay constant and weight connection from the node j to node i respectively. The additional terms lead to small values of the weight parameters thereby decreasing the tendency of the model to overfit the data.

The data set
The chemical composition of alloy 316L considered in this work is presented in Table 1. The data set used in the development of the MFNN was taken from the flow stress measurement curves in Svoboda et al. [8]. The curves were obtained from Split Hopkinson Pressure Bar (SHPB) experimental study of the alloy. Experimental measurement T range from room temperature (RM) to 950 0 C and varies from 1000 s −1 up to 9000 s −1 . The maximum ε obtained is about . In this work, a total of 363 datasets were collected by digitizing the curves. This data set is used to train and test the developed MFNN model.

Model selection, experiment and simulation
The MFNN structure for the flow stress prediction for alloy 316L under high temperature and high strain rate deformation is depicted in Figure 2. Following the Kolmogorov theory [12], we are using a 3-7-1network as shown n [ normalization procedure is used so that larger values of the input data do not overwhelm the smaller input data. This helps to reduce the network error. The o network is trained on all of the data. The NN model was trained six times with different randomly generated initial parameters, and the best results are reported in this paper. After training, the MFNN model parameters are as in the appendix.  Figure 3 shows the mean square error (MSE) curve after the MFNN training with the dataset. The MSE decreases as a function of the number of epochs. At convergence, the error is 0.000301, which is not far to the ideal value of zero. The linear regression analysis, in terms of the target and network's output is shown in Figure 4. The R exhibit good correlation, and they fall almost on the same line with the angle of the passing of the output line 45 0 . This illustrates that the network has been trained properly with no occurrence of overfitting. The predictive ability of the MFNN model for the training dataset is shown in The plot provides excellent matching of the measurement. The predictive accuracy and robustness of the MFNN model are further shown by comparing with the experimental measurement for the alloy at the various condition of in [14]. This comparison is shown in measurement in the figure is obtained by digitizing the appro 0 C .Schematic diagram of the artificial neural network structure.

Results and discussions
A learning rate of 0.01 was used. The number of epochs is 10000. An epoch can be defined as one presentation of all training examples to the network, followed by adjustment of the weights. The neural network was optimized according to the Bayesian regularization procedure in as the performance evaluation function. The Bayesian regularization method helps to determine the actual number of parameters, i.e. weights and bias, that are important to the NN model. All the 363 datasets from the flow stress curves is presented to the MFNN for training the network. The dataset composition is shown in Table 2. However, the dataset is pre-processed so that the input 1 ] . The MATLAB "mapminmax" is used for this purpose. This normalization procedure is used so that larger values of the input data do not overwhelm the smaller input data. This helps to reduce the network error. The output values were de-normalised. The neural network is trained on all of the data. The NN model was trained six times with different randomly generated initial parameters, and the best results are reported in this paper. After training, the MFNN meters are as in the appendix.
. Composition of the dataset used for training the MFNN , Total number in the dataset  2850  8345  2930  2965  2965  8670  1020  1075  8810   38  37  34  48  44  43  25  44  50 the mean square error (MSE) curve after the MFNN training with the dataset. The MSE decreases as a function of the number of epochs. At convergence, the error is 0.000301, which is far to the ideal value of zero. The linear regression analysis, in terms of the target and network's The R-value is 0.999. In the figure, the target and the network output exhibit good correlation, and they fall almost on the same line with the angle of the passing of the . This illustrates that the network has been trained properly with no occurrence of . The predictive ability of the MFNN model for the training dataset is shown in The plot provides excellent matching of the measurement. The predictive accuracy and robustness of the MFNN model are further shown by comparing with the experimental measurement for the alloy at . This comparison is shown in Figure 6. The experimental measurement in the figure is obtained by digitizing the appropriate curves in [14]. The MFNN model s −1 A learning rate of 0.01 was used. The number of epochs is 10000. An epoch can be defined as one followed by adjustment of the weights. The ayesian regularization procedure in [13] using as the performance evaluation function. The Bayesian regularization method helps to determine the actual number of parameters, i.e. weights and bias, that are important to the NN model.
rves is presented to the MFNN for training the network. processed so that the input . The MATLAB "mapminmax" is used for this purpose. This normalization procedure is used so that larger values of the input data do not overwhelm the smaller normalised. The neural network is trained on all of the data. The NN model was trained six times with different randomly generated initial parameters, and the best results are reported in this paper. After training, the MFNN Total number in the dataset the mean square error (MSE) curve after the MFNN training with the dataset. The MSE decreases as a function of the number of epochs. At convergence, the error is 0.000301, which is far to the ideal value of zero. The linear regression analysis, in terms of the target and network's value is 0.999. In the figure, the target and the network output exhibit good correlation, and they fall almost on the same line with the angle of the passing of the . This illustrates that the network has been trained properly with no occurrence of . The predictive ability of the MFNN model for the training dataset is shown in Figure 5. shows good agreement with the measurement over the full range of data and also compared very well with the dislocation density based physical model results in [14]. The MFNN generalise very well for deformation conditions that are slightly below of above the training dataset.

Conclusions
The flow stress of alloy 316L at high temperature and high strain rate deformation has been modelled using an artificial neural network with Bayesian regularization, MFNN. The input level for the neural network consists of strain rate, temperature and strain and the output level consist of the flow stress. High strain rate and high temperature deformation of 316L leads to localized deformation that is highly nonlinear and a challenge for modelling and simulation. From this work, the following conclusion can be drawn: i. Using ANN with Bayesian regularization affords us the opportunity of combining nonlinear functions in parallel for the modelling of the alloy.  ii.
The predicted results from the MFNN model are in good agreement with experimental measurement and also consistent with the physical based results in [14]. iii.
The MFNN is robust as it can generalize to deformation condition slightly below or above the training dataset and therefore able to capture the different physical mechanisms of the alloy during high strain rate and high temperature deformation. iv.
The model can be improved as more experimental measurement becomes available for the alloy.
Thus, the MFNN model developed in this paper is viable for use in the simulation manufacturing processes, particularly machining.