A Deep Learning-Based Computational Algorithm for Identifying Damage Load Condition: An Artificial Intelligence Inverse Problem Solution for Failure Analysis

: In this work, we have developed a novel machine (deep) learning computational framework to determine and identify damage loading parameters (conditions) for structures and materials based on the permanent or residual plastic deformation distribution or damage state of the structure. We have shown that the developed machine learning algorithm can accurately and (practically) uniquely identify both prior static as well as impact loading conditions in an inverse manner, based on the residual plastic strain and plastic deformation as forensic signatures. The paper presents the detailed machine learning algorithm, data acquisition and learning processes, and validation/verification examples. This development may have significant impacts on forensic material analysis and structure failure analysis, and it provides a powerful tool for material and structure forensic diagnosis, determination, and identification of damage loading conditions in accidental failure events, such as car crashes and infrastructure or building structure collapses.

used. We then outline our process for gathering the required data to train the machine learning algorithms. This is followed by examples that demonstrate how we solve the inverse problem, including a cantilever beam of inelastic materials statically loaded at different locations, and the same beam loaded dynamically with impact loading. We seek to demonstrate with these examples that the machine learning algorithms can accurately identify both static loading and impact loading conditions based on observed residual plastic strain or deformation.

Deep neural network model
Recently, machine learning, especially deep neural networks, has become one of the most popular key words in every scientific or engineering field. Deep learning architectures, such as deep neural networks, deep belief networks and recurrent neural networks, have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design and board game programs, where they have produced results comparable to and in some cases superior to human experts, e.g. [Ciresan, Meier and Schmidhuber (2012); Krizhevsky, Sutskever and Hinton (2012)]. Using machine learning methods, one could make relatively accurate predictions on some problems that were difficult to be solved before. In this work, we adopted a fundamental model of deep neural network (DNN), namely an artificial neural network. There are five stages in our model, as shown in Fig. 1 .
(1) Data collection: Collect data from software Abaqus.
(2) Data cleaning and processing with feature selection and feature engineering. During this stage, we applied dimension reduction, removed irrelevant data, and created new features so the model could perform better. The original data of static has 120 variables, which indicate 60 pairs of displace from x and y directions. The 60 pairs of data represent three parts of the cantilever beam which are the top, middle, and bottom parts. Due to the similarities, the top and bottom data was eliminated and we only keep the middle parts. Since the cantilever beam has a relative small displacement on x-direction, we only keep the y displacement to simplify the model. When doing the feature engineering, we tried to use our expertise and mathematical methods to create new relative features and established five new variables, including finding slope from plotting data of displacement x and displacement y, the summation of displacement y, the amplitude of displacement x, and the centroid distance. (First, we find a linear fit for the deformation change, namely, the approximate slope of the displacement x and displacement y from plotting data. Second, we generate the sum of displacement of y as another new feature. Third, to make the feature more obvious, we create a variable by using the top displacement x minus the bottom displacement x. Last but not least, we made centroid distance as a new feature. The equation is where ux is the displacement of x and uy is the displacement of y) For the dynamic data, we did a relative radical method. Like the static data we pick only 20 variables, instead of keep them we use the product of displacement x with displacement y, which lead all variables new. Then we applied summation of the variable and the centroid distance of the new dataset.
(3) Building network: Initialize bias, weight, number of layers and number of neurons in each layer.
(4) Application of deep neural network to obtain a specific mathematical model. It is noted that the basic mathematical model for the first stage of neural network is a simplified projection pursuit regression [Friedman and Stuetzle (1981)], where g(x) is an activation function, w is a distributed weight, and x is an observation. During the second stage, the w is redistributed to optimize the loss or error used through back-propagation.
(5) Using a chosen test data set, which is different than the training sets, to validate the model and analyze errors.

Models and settings
As mentioned earlier, a deep neural network [LeCun, Bengio and Hinton (2015)] is used in this study to solve this inverse problem with procedures considered for the features. One sample fully-connected layer is shown in Fig. 2 [Castrounis (2016)]. In one certain neuron j, the employed mathematical model is described as CMES, vol.117, no.3, pp.287-307, 2018 One can clearly observe from Fig. 2 that, the input data flow(X) multiplied by the distributed weight(β k ) and then added a bias (β 0 ) as a parameter of an activation function (σ(x)), reflects the mathematical model in Eq. (3). Among all the options of activation functions, such as hyperbolic tanh, Sigmoid, or Relu. According to Ramachandran, "currently, the most successful and widely-used activation function is the Rectified Linear Unit (ReLU)." [Ramachandran, Zoph and Le (2018)] In our case, Rectified Linear Unit (ReLU) performs well as an activation function σ(x) for both static and dynamic loading conditions. ReLU provides a nonlinear function f (x) = max(0, x). Over-fitting is usually a concern that a model would perform too "well" on the training data set. In order to prevent over-fitting, a dropout was applied during the training process, "the key idea is to randomly drop units (along with their connections) from the neural network during training." [Srivastava, Hinton, Krizhevsky et al. (2014)].

Copyright © 2018 Tech Science Press
MSE is commonly adopted as a loss function in statistical models. It provides an intuitive measurement of errors. The objective is to minimize MSE and make the model fit both the training data and validation data well. An optimizer will adjust between forward and backward propagation in order to achieve this objective. The optimization tool used herein to minimize MSE is the Adam optimizer [Kingma and Ba (2014)] with exponential descent in learning rate.

Implementation
TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks. In this project, we developed a machine learning computer code by using TensorFlow.
There are four layers in the developed DNN code, with the learning rate of 0.0035, and the dropout of 0.05. We use the ReLu as the activation function, and we ran it in 18000 steps. In general, the number of hidden neurons should be between the size of the input layer and the size of the output layer. The input parameters of neural network is the plastic displacements or the permanent displacements along both the horizontal and vertical directions of all the nodes of the FEM mesh, except the boundary nodes, of the cantilever beam. The size of input layer is 60 or more after the numerical process, depending on the mesh of the model. The size of output layer is 3 or 8, depending on different numerical models. Thus, we choose from eight to sixty hidden layers in our machine learning code. After trying different numbers of hidden layers, we found that four hidden layers neural network provides good computation results for the FEM mesh size used and the numerical model. More hidden layers will lead to much more calculation time, but not much better results. Thus, we choose four hidden layers as the default structure for our machine learning test code. The number of hidden neurons should be less than twice the size of the input layer. Hence, we choose the first hidden layer 32 neurons, the second and third hidden layers have 64 neurons each. The final hidden layer has 8 neurons.
3 Data collection

Geometric and material properties
A 2D cantilever beam was chosen for the study, which has a length of 5 meter and a width of CMES, vol.117, no.3, pp.287-307, 2018 1 meter. The finite element model of the cantilever beam was developed with the ABAQUS software. Plane strain condition was assumed throughout this study, and CPE4R element was used, which is a 4-node bilinear plane strain quadrilateral element [Simulia (2011)]. In order to ensure the accuracy of the numerical calculation, different mesh sizes of the beam were chosen for the convergence analysis, and the final mesh size was chosen as 0.25 m. Accordingly, the number of nodes is 105, and the number of elements is 80.

Copyright © 2018 Tech Science Press
We choose AISI 4340 steel (33 HRc) as the material of the cantilever beam, which is modeled by using the Johnson-Cook plasticity model ( [Guo and Yen (2004); Johnson and Cook (1985)]). It is model as a thermo-elastoplastic solid, as expressed in the following equations, where σ is the flow stress, p is the equivalent plastic strain,˙ p is the strain rate,˙ 0 is the reference strain rate, A, B, C, m, n are material constants, and T * is the homologous temperature which is related to the absolute temperature T , the reference temperature T 0 and the melting temperature T m . The critical failure strain is defined as [Guo and Yen (2004); Johnson and Cook (1985)]: where D i are material constants, and σ * is the dimensionless pressure-stress ratio. Material parameters of AISI 4340 steel are listed in Tab. 1.  of simulation data were generated. In most cases when loads are applied to the three points closer to the support of the cantilever beam (nodes 5, 6 and 7 in Fig. 3), the residual displacement and plastic strain are all zero at all the nodes, which leads to the issue multiple-answer issue to DNN since zero residual displacement corresponds to three different locations. Thus, the loading at these points are excluded and only the loading points 1-4 are considered in the training, as shown in Fig. 3.

Boundary conditions and loads
Four case studies have been conducted to test, validate, and verify the deep learning algorithm and the trained neural network: (1) Prediction of static loads acting on the numbered nodes (also referred as training nodes in this paper). Static loads simultaneously imposed to one up to four numbered nodes shown in Fig. 3. Responses of the beam under static loads of different amplitudes were numerically predicted, and the database for the deep learning was developed. Then, a different deformation of the beam caused by loads applied to the numbered nodes was given by ABAQUS program, and the amplitude of the static load was predicted by the deep learning algorithm and compared with the exact solution.
(2) Prediction of static loads acting between the numbered nodes. In this case study, the database for the deep learning was also developed by applying static forces to the numbered nodes. Then, a deformation of the beam caused by a load acting on a node between the two adjacent numbered nodes (as shown in Fig. 3) was given by ABAQUS program. Amplitude and position of the static load were predicted by the deep learning algorithm.
(3) Prediction of the impact load acting on the numbered nodes. Impact loads imposed Both static and dynamic responses of the cantilever beam under concentrated loading forces are computed by using ABAQUS. It should be noted that the loading history of each concentrated force acting on the beam, which can cause plastic deformation of the beam, is featured with a bi-linear loading and unloading curve of bandwidth t and loading amplitude F max , as shown in Fig. 4. CMES, vol.117, no.3, pp.287-307, 2018
(4) Prediction of the impact load acting between the numbered nodes. This case study is similar to the second case study. Position, amplitude and duration of the impact load were all predicted by the deep learning algorithm. The bi-linear pulse load-time history curve is assumed throughout this study as shown in Fig. 4. For the static analysis, the step time T equals 1, and durations of loading and unloading are 0.5. For the dynamic response, durations of loading and unloading are significantly reduced to orders of 0.01-0.03 s to simulate the impact loads. Meanwhile, in order to minimize the influence of inertial effects and get the final stable deformation of the beam, the step time T for the dynamic analysis is set as 50-100 s. For the static and dynamic analysis, amplitude, duration and step time for the first two case studies are listed in Tab. 2 and Tab. 3, respectively. For the multi-points condition, amplitude of the load is reduced. For static analysis as an example, amplitude of the load was between 2.0 × 10 7 -7.0 × 10 7 N for two nodes cases, 1.0 × 10 7 -5.0 × 10 7 N for three nodes cases, and 1.0 × 10 7 -4.0 × 10 7 N for four nodes cases.

Training data
When applying above magnitude force to the cantilever beam, it will have permanent plastic deformation. One of particular plastic displacement examples is as shown in Fig. 5, while the associated plastic residual strain distribution is shown in Fig. 6. After obtaining the residual plastic displacements from all FEM nodes of the beam, we combine them together as the input of training data of DNN. The process is as shown in Fig. 7. The right table of Fig. 7 is the input of one set of raw training data of DNN. The output of this set of training data is the value and location of the outside force according to this plastic displacement distribution, for static modes, and the location, magnitude and duration according to this plastic displacement distribution, for dynamic models. Generally speaking, in machine learning, the more the data are available, the more accurate a prediction may be. However, there is still no rule about how much data is enough. It CMES, vol.117, no.3, pp.287-307, 2018 (   In this study, the mean square error (MSE) between the outputs and the correct results was chosen as the training loss. There is an updated loss in each epoch which can show the fitting of the model. The record of the loss throughout the process of our case is shown as Fig. 8: CMES, vol.117, no.3, pp.287-307, 2018  In Fig. 8, the loss gradually decreases and reaches the minimum value of 0.2611 after 18k steps. The minimum value and the smooth descend curve of the loss indicates that the model was trained steadily and has fitted the training data very well.

Copyright © 2018 Tech Science Press
To test the performance of the model, four sets of testing data were generated. Each testing data represents one type of loading combination, as shown in Tab. 4. For this problem, we designed our output layers with eight neurons which represent the two direction loads in four valid loading points, as mentioned in Section 3.2. Therefore, inputting the testing data to our DNN model, we get the output as shown in Tab. 4. The errors of the prediction are listed in Tab. 5. In Tab. 4 , due to the eight-neuron output layer, all the eight values of the output were nonzero, but the values on the real-loaded nodes were much larger than the values on the other nodes. For example, in Case 1, the real load is located in node 4 . In the predicted result, the values of 4 x and 4 y is obviously much larger than the values on the other nodes. And the values of 4 x and 4 y is very close to the real values, with the errors of 1.71% and 1.998%. The output of the other three test cases have the same pattern. Predicted errors are all smaller than 5%, as shown in Tab. 5. Therefore, the trained DNN model can correctly predict the loading locations and the magnitude of the static loads which act on the training nodes.   (10 7 N) -12.065 -10.808 -9.9 -9.383 -6.108 -6.707 -6.599 -7.877 In Section 4.1 we demonstrated the capability of the DNN model to predict the statics loads acting on the training nodes. In this section, following the data collection rules outlined in Section 4.1, we trained the DNN model with 133 sets of data that were obtained by static loads acting on the four valid training nodes individually. Then, we tested the model with 8 sets of deformation states caused by 8 different loads acting in the interval between the training nodes, to see if the DNN can make extended prediction. The testing data is shown in Tab. 6. We designed the output layer with three neurons which represent the location of theload and the magnitudes of the loads in x and y directions because the loads are all acting individually in this section. The predicted results are shown in Tab. 7 and Tab. 8. According to the results, all the predicted locations were in the correct intervals. The correct interval is an important information which can indicate the location range of the load. Then one can further locate the load by subdividing or reducing the interval. For the prediction of the specific location and the magnitude in x and y directions, the maximum errors are 6.554%, 7.366% and 4.424% respectively. The average error of the output is 3.098%, which is less than 5%. The errors of the prediction are small and can meet a lots of engineering requirements. Therefore, the results show that the DNN model is able to predict the static loads in the interval between the training nodes.  CMES, vol.117, no.3, pp.287-307, 2018  Compared to the static problems, dynamic problems are more common in real situations but also more complicated. By following the data collection rules stated in Section 4.1, for the dynamic problem discussed here, about 40 sets of data for each load case would produce a good training effect. We finally trained the DNN model with 175 sets of deformation states caused by different single vertical impact load acting on the training points. Then five sets of test data were generated to test the performance of the trained model. All the training and testing data are shown in Fig. 10. The output layer has three neurons which represent the location, magnitude and the duration of the impact loads. The training loss throughout the training process is shown as Fig. 9. The minimum loss (MSE) reached 0.056 after 18k iteration steps which shows a good fitting of the model to Figure 11: Predicted result of the loads on the nodes the training data.The prediction of these impact loads are shown in Fig. 11. The predicted loads and the correct loads are very close to each other in the shown three dimension space. The maximum errors of the predicted location and the magnitude are 3.321% and 2.338%, respectively, as shown in Tab. 9. The maximum error of the duration is 12.321%. During the study, we found that the load duration is more difficult to be predicted than the other two parameters. The overall errors of these five testing cases are less than 5.4%. It shows that the DNN also works very well in the prediction of the impact loads which are located on the training nodes, especially for the location and the magnitude of the impact.  CMES, vol.117, no.3, pp.287-307, 2018 Figure 12: Training and testing data  Fig. 13, and the error of the output is listed in Tab. 10. The maximum error of the location, magnitude and duration are 9.718%, 6.869% and 7.213%, respectively. Compared to the Section 4.3, the overall errors are increased but still less than 10%. And the predicted locations are all in the correct intervals. It is concluded that the DNN can predict the loads in the interval between the training nodes, too.

Prediction of the impact loads between training nodes
The results show that the DNN model is capable to do interpolation between the training data by itself. However, the interpolation accuracy still depends on the density of the training data. The higher the density of the training data, the higher the predicted accuracy will be.
Thus, while training the DNN model, we can choose some typical loading cases as training data depending on the accuracy demand, which would save a lot of time in training phrase. To further reduce the predicted error, another approach is developed as explained in the Section 4.5.

Conclusions
In this study, we applied deep (machine) learning techniques to solve an inverse engineering problem of identification of the location, amplitude, and duration of impact forces on a structure, both static and dynamic loads, based on the permanent residual deformation as well as the permanent strain distributions. For static problems, the developed machine learning algorithm can predict both the location as well as the amplitude of the force with high accuracy. The prediction on the location of the applied load is not necessarily from the training data, after studying the training data, the machine learning algorithm can automatically use interpolation to find the applied load location that is not in the train data. This is to say that we only need to train the neural network with a limited number of loading sets, it can then predict any loading locations on the boundary of the cantilever beam. This is a remarkable success for an inverse problem solution that was impossible to realize because of the non-unique solutions for inverse problems. Based on this study, we have come to a conclusion that by using the forensic signatures such as permanent deformation and residual plastic intervals are 4.25∼4.375 m, 4.375∼4.625 m and 4.625∼5 m, as shown in Fig. 14. Eighteen sets of new training data were generated on each of the new numbered points 5 and 6 , respectively. Then, we retrained the DNN model with the data on Node 1 , 2 , 5 and 6 . Different from the previous model which was trained with all data, the retrained model would only predict the loads in interval 1 ∼ 2 and achieve a much higher accuracy in the prediction. Two testing data were generated, as shown in Tab. 11. The predicted results and the errors before and after the retraining are shown in Tab. 12 and Tab. 13. The predicted errors of all three parameters are reduced greatly by refining the interval, from 9.02%, 10.583% and 5.945% to 0.798%, 2.978% and 4.864%, respectively. Also, the prediction of the smaller intervals reached 100% accuracy. Therefore, by refining the interval and retraining the DNN model step by step, it is possible to finally reach the required accuracy. Based on the convergence tendency, it is also shown that finer mesh will lead to correct finer intervals and high accuracy results. Thus, if we could have large amounts of data with forces applied on a large variety of different locations, which is equivalent to dividing the cantilever beam to large numbers of sections, the predicted results will still be in the correct interval. The accuracy will achieve almost 100% if we have unlimited amount of data.
strain distributions one can uniquely (practically) determine the applied load conditions inversely with high accuracy for most engineering purposes.
For dynamic loading problems, the current version of our machine learning algorithms can also predict both the location and amplitude of applied loads with high accuracy, whereas the accuracy of the prediction on the duration of the loads is not high, even though it still have good accuracy i.e. the error is within 5% to 10%. These results are very much promising for both static and dynamic inverse problems. Results herein demonstrate that the ML or AI based approaches can solve many previously intractable problems. We shall soon report our machine learning algorithm for inverse failure analysis of threedimensional structures with more complex geometries and loading conditions, including crashworthiness analysis and seismic or other extreme hazardous event induced structure failure forensic analysis.