Mechanical response of additively manufactured foam: A machine learning approach

This paper uses ensemble and automated machine learning algorithms to predict the mechanical properties (tensile and flexural strength) of a three-dimensionally printed (3DP) foamed structure. The closed cell foams were made from the most commonly used thermoplastic, High-Density Polyethylene (HDPE). The hollow glass microspheres are infused in HDPE at varying volume %. The available data on these foams ’ mechanical properties are used by the chosen machine learning (ML) algorithms to propose the best suited algorithm for such a three-phased microstructure as these closed cell foams exhibit. Finally, the strength predictions from the models were validated using experimental data. The models were trained with nozzle temperature, bed temperature, and force values as input parameters. The output parameters predicted were the tensile and flexural strength. LightGBM outperforms all other models in terms of performance among ensemble-based models, while H2OAutoML outperforms all other models. All the ML algorithms produced models with greater than 95% accuracy. Finally, memory and time consumption for each model are presented.


Introduction
Additive manufacturing (AM), commonly known as 3D printing, is a process in which the part is manufactured layer by layer, following the path generated with the help of stereolithography (STL) file formats [1].Fused Filament Fabrication (FFF) is the most widely used AM technology.The filament is fed into a heated nozzle that heats the solid filament and deposits it on the bed in a semi-solid state in a layered fashion [2].The ease of making critical parts made AM technologies have various applications in biomedical, aerospace, jewelry, and automotive industries [3,4].Initially, AM was focused only on polymeric materials, advancing/widening its scope to metamaterials, ceramics, metals, and composites [5].Machine learning (ML), a form of artificial intelligence (AI), enables software systems to become better at making predictions [6,7].ML algorithms use historical data as input to predict new output values.Nowadays, the application of ML is widely accepted in almost every field.The role of AI using ML techniques in manufacturing has significantly enhanced in the last five years [8].It helps to automate the additive manufacturing process without the operator's intervention depending on the available data of the printed object.Pazhamannil et al. [4] worked on the Artificial Neural Network (ANN) model to predict the tensile strength of PLA.The author took layer thickness, nozzle temperature, and infill speed as input variables, noting that the prediction error is below 5%.Zhang et al. [9] found that the Long Short-Term Memory (LSTM) prediction model has shown better results than Random Forest (RF) [10] and Support Vector Regression (SVR) models for strength predictions after feeding extruder temperature, printing speed, and layer height as an input process parameter to the models.Deshwal et al. [11] used Genetic Algorithm-Artificial Neural Network (GA-ANN), Genetic Algorithm-Response Surface Methodology (GA-RSM), and Genetic Algorithm-Adaptive Neuro Fuzzy Inference System (GA-ANFIS) models for the improvement of PLA's tensile strength by giving infill density, temperature and speed as model input parameters and observed that their accuracies are greater than 99%.Trivedi et al. [3] worked on fuzzy logic, and reported the prediction error of 1.43%.Yadav et al. [12] worked on the Adaptive Neuro Fuzzy Inference System (ANFIS) model to determine the effect of significant parameters like extrusion temperature, layer height, and material density on the tensile strength of material materials like Polyethylene Terephthalate Glycol (PETG), Acrylonitrile Butadiene Styrene (ABS) and multi-material (60% ABS + 40% PETG), which shows a minimal error percentage of 2.63%.Sood et al. [13] have predicted compressive strength and optimum process parameters of ABS material using ANN and Quantum-behaved Particle Swarm Optimization (QPSO) models after feeding layer thickness, orientation, raster angle, raster width, and air gap as input variables to the model.Ali et al. [14] used the ANN model to predict natural frequency of Polycarbonate (PC) material after taking raster angle, air gap, build orientation, and the number of contours as model input parameters.In short, a maximum number of times tensile strength prediction has been performed [15][16][17].Other properties like flexural strength [16,18], compressive strength [13,18], vibrational frequencies [14], deposition angle [8], drug release from diazepam [19], topographic defects [20], warping detection [21], etc. have been predicted in previous works.Zhang et al. [22] used XGBoost based model for seam tensile strength prediction of Al-Li alloy in laser welding, and observed that XGBoost based model outperforms other conventional models like RF, Decision Tree (DT), Linear Regression (LR), etc. in terms of evaluation criteria of Mean Squared Error (MSE), Mean Absolute Error (MAE), Coefficient of determination (R 2 ), etc, after taking laser power, weld speed and plate thickness as model inputs.Era et al. [23] found that the quality of XGBoost can be improved by using F-score for selecting important features.XGBoost outperforms RF [22,24] and K-Nearest Neighbors (KNN) [25].Liang et al. [26] and Cui et al. [27] compared Extreme Gradient Boosting (XGBoost) with Light Gradient Boosting Machine (LightGBM) model and found that LightGBM gives higher accuracy than XGBoost and RF with least computational time.Mahmood et al. [28] found that LightGBM has a higher processing speed than conventional algorithms and can handle large amounts of data efficiently while consuming lower memory.Further, it also outperforms conventional algorithms in terms of robustness [29].Czako et al. [30] suggested that, in terms of Automated Machine Learning (AutoML), Auto-Sklearn (AutoSKL) works superior to Auto-Weka, Particle Swarm Optimization-Simulated Annealing (PSO-SA) and Hyper-parameter optimization for Sklearn (Hyperopt-Sklearn) as its searching time is less.Van et al. [31] shows that AutoSKL works superior with complex datasets.Other AutoML techniques like MLBox [32,33], Tree-based Pipeline Optimization Tool (TPOT) [34] and H2OAutoML [35,36] worked very well and advantageous over conventional algorithms.
Based on the literature survey, it is observed that AutoML is not yet explored for the prediction of 3D-Printing parameters in foams.The complex material parameters and the different 3D printing parameters make them experimentally investigated, making it an expensive, tedious, and time-consuming proposition.Hence, the present work uses the ML approach to predict the mechanical properties of 3D printed closed cell foams without considering structure-related parameters due to the complexities involved.In particular, AutoML is explored for foam response predictions.It can automate the time-consuming, iterative tasks of other ML model development.Further, it enables ML model   building with high scalability, efficiency, and productivity while maintaining model quality for developers, analysts, and data scientists.Traditional ML model construction is time and resource-intensive, requiring extensive domain expertise where dozens of models must be compared.This can be overcome with the help of AutoML, where one can quickly and effectively shorten the time to create ML models suitable for use in production.In this work, different ML algorithms, namely XGBoost, LightGBM, AutoSKL, MLBox, TPOT, and H2OAutoML, have been explored for 3D printed foams and are subsequently trained in python language through Google Collaboratory.Training of the model is carried out using 3D Printing process parameters and force values as input variables, whereas tensile and flexural strength were the output variables.Furthermore, the comparative prediction accuracy among non-automated and automated algorithms toward mechanical properties using model metrics is explored.Such an approach helps the materials scientists and industrial practitioners to design the specialized class of materials, the foams, to effectively and efficiently cater to the envisaged requirements from the marine, automobile, aerospace, and space sectors.

Materials and processing
The processing details are available in the author's earlier published works [37][38][39].This study developed HDPE-based syntactic foam by reinforcing hollow GGMB as a filler material.The typical HDPE (supplied by Indian Oil Corporation Limited, India) material properties are listed in Table 1 [40,41].Hollow GMBs (glass microballoons),used as filler material, were supplied by 3 M Corporation, Singapore.These fillers had an average diameter of 15.3 ± 1.63 μm with a density of 600 kg/m 3 , and a wall thickness of 1.4 ± 0.079 μm.Different configurations     of HDPE/GMB composite were developed by varying GMB (filler) volume fractions (20-60% in the increment of 20).These developed composite blends were extruded into their respective filaments of 2.58 ± 0.5 mm diameter using a single screw extruder.Such fabricated filaments are used as feedstock material for 3D printers.The different 3D printing parameters, extrusion multiplier, nozzle temperature, layer height, infill percentage, bed temperature, printing speed, chamber temperature significantly affect the part quality.Using Star, AHA 3D Printer, multiple samples of H, H20, H40, and H60 (H: HDPE, 20: GMB volume %) are 3D printed by varying the parameters in the ranges mentioned in Table 2.These 3DP coupons are subjected to tensile and flexural characterization using Zwick-Roell Z020 USA (20 kN load cell).

Data-set analysis
Experimental data of FFF based 3D-printing is used for this study based on the author's earlier Refs [37][38][39].Different process parameters of 3D-printing are used as input parameters from the datasets, such as nozzle temperature, extrusion multiplier, chamber temperature, bed temperature, and printing speed, along with the force values to predict the ultimate tensile, and flexural stress values of the closed cell foams and are considered as the output variables.Multiple rows of datasets were used for this study obtained from pilot studies.The dataset was generated considering the strain rate (constant) mentioned in ASTM standard and is implicitly a part of the experiment.

Multicollinearity and feature selection
The feature selection step in ML needs to be defined before creating the ML algorithm, using which one can finalize the most influencing parameters to be considered in subsequent investigations.The primary benefit of the feature selection is the reduction in the learning time.In the current study, feature selection was carried out using heat-map analysis, multicollinearity concept, and feature importance plot.Heatmaps is a graphical representation in which correlation among every feature is represented, as shown in Fig. 1.From the heat map, it is observed that the extrusion multiplier (flow rate of the melt) exhibited negligible correlation with the output variable, as clearly evident from the lower variational magnitudes.Such a variation in the dataset won't show a significant effect on the output variable; hence, in the subsequent analysis, the extrusion multiplier is not considered.This feature selection process further progressed by applying multicollinearity.Multicollinearity is a concept that is used to estimate the correlation magnitude between any two independent variables [42,].This is crucial as two highly correlated independent variables embark on a similar effect during prediction.In this work, chamber temperature is scratched out, showing a high correlation of 0.98 with bed temperature.This factor is dropped out because it does not contribute much to the strength prediction, whereas bed temperature directly influences the warpage/shrinkage owing to the residual thermal stresses and difference in coefficient thermal expansions of all the three phases in the closed cell foams, as mentioned earlier.Furthermore, based on the domain knowledge, printing speed has comparatively been less significant towards the strength.Nonetheless, the order of significance among different selected parameters study has been verified through the feature importance plot.Fig. 2 shows the representative image of the plot wherein the top three features among all the input parameters are mentioned.Based on all the outcomes of the feature selection, bed and nozzle temperatures are considered for subsequent analysis.

Machine learning algorithms 2.4.1. Ensembles-based algorithms
The present work applies the two ensemble-based algorithms, i.e., XGBoost [43][44][45] and LightGBM [46,47].Both algorithms work on the principle of tree formation and gradient boosting.Gradient boosting is one of the most popular ensemble modeling techniques that try to minimize the error present in the previous models (GBM 2 will be developed such that it has less error than GBM 1 ).Fig. 3 presents the flow chart of the gradient boosting technique.This process is continued until the prediction on the training dataset is correctly formalized or until the number of models reaches its maximum value.The difference between XGBoost and LightGBM is that, in XGBoost, trees grow level-wise, while in LightGBM, trees grow leaf-wise, as shown in Fig. 4. In XGBoost, each level of nodes is created in one go, while on the other hand, a node continues to grow until the splitting of its sub-nodes stops in LightGBM.Briefly, the tree growth in XGBoost is horizontal, while in LightGBM, it is vertical architecture.

Hyperparameter optimization for ensemble-based algorithms
The process of selecting optimum parameters of the ML algorithm through which model capabilities are enhanced is called hyperparameter optimization or hyperparameter tuning.Every ensemblebased ML algorithm has a default value of model input parameters like n_estimators, learning rate, maximum depth, etc.In the absence of the user defined specific values for these parameters, default values get allotted automatically.Further, the default values might not fetch  accurate results.Hence, values are defined for these model parameters in accordance with the default values.The randomizedsearchcv is used in the current work to compute the optimum parameters to verify the parametric values for higher efficiency of the models.The primary reason behind the selection of this technique is its efficiency in giving accurate optimum parameters within a minimum time.Figs. 5 and 6 show the output plots of hyperparameter optimization indicating the variation of model input parameters on MSE.The model input parameters for which the minimum MSE value is observed, was selected as hyperparameter for XGBoost and LightGBM ML algorithms.AutoML eliminates the efforts towards conducting hyperparameter optimization manually.With the help of AutoML, one with no prior knowledge of ML can effectively and efficiently utilize productivity enhancements and advance ML research.It can automate the laborious, iterative efforts involved in developing ML models.It makes it possible for analysts, data scientists, and developers to create ML models with large scale, efficiency, and productivity while maintaining model quality.Traditional ML model development is resource-intensive, requiring significant domain knowledge and time to produce and compare a host lot of models.AutoML can quickly accelerate the time needed to get production-ready ML models with great ease and efficiency.A few of the AutoML examples are AutoSKL [27], MLBox [28,29], TPOT [48][49][50][51][52], and H2O-AutoML [53], which are used in this work.These AutoML algorithms can apply different ML algorithms, verifying the suitability for a given dataset by comparing each model's accuracy or prediction error.The generalized working principle of AutoML is presented in Fig. 7.

Results and discussions
The current work considers six ML algorithms for predicting tensile and flexural strength.The model metrics and resource consumption are evaluated to compare the models' efficiency.Initially, for training the algorithm to predict the variation of output parameters with respect to input parameters, both input and output variables were considered in the training step of the algorithm.Once training is accomplished, the efficacy of the ML models is checked based on input variables.Further, for training the ML models, nozzle temperature, bed temperature, force, tensile and flexural strength are provided to the models.In testing, only nozzle temperature, bed temperature, and force values are provided to predict the tensile and flexural strength.

Model metrics for ML algorithms
In this study, the model metrics criteria are performed by evaluating R 2 -value [8][54], Explained Variance Score (EVS) [55], MSE [22], Root Mean Squared Error (RMSE) [22,24], and MAE [56] for both tensile and flexural datasets [33,34].Model metrics results are represented graphically in Fig. 8.It is observed that these model metrics for the tensile dataset can give model accuracy greater than 99% for H, H20, and H60 foams, whereas H40 exhibited a 96.28% of R 2 value with the H2OAu-toML algorithm.Apart from the R 2 -value, MSE, RMSE, and MAE are almost zero, and EVS is nearly equal to one, which implies the excellent working efficiencies of all the models.Tensile strength prediction for each closed cell foam is represented graphically in Fig. 9a.It is observed that all the ML algorithms predict tensile strength very precisely and accurately, whereas the error predicted using the LightGBM model for H40 is comparatively has lower prediction accuracy.Further, Fig. 10 depicts the different model metrics for the flexural dataset, wherein the model accuracy is noted to be greater than 99% from all the envisaged models.Fig. 11 shows the overlapping results of the predicted flexural strength values with the experimentally deduced results of all the 3D-printed foams.AutoML algorithms work efficiently and effectively for 3D printed closed cell foams as seen from the minimum deviation between the predicted and experimental results compared to the ensemble-based ML algorithms.

Noise addition
Noise is added to the dataset to minimize the overfitting problem.Post noise addition to the independent variables of the dataset, if ML models work as anticipated resulting in minimum deviations, model non overfitted is affirmed.In the current work, Additive White Gaussian Noise (AWGN) is induced in the datasets.The AWGN is a basic noise model similar to the Gaussian Distribution, requiring two parameters (standard deviation σ, mean μ).When μ = 0, noise variations can be achieved by altering σ values in the python language.Fig. 12 presents the effect of AWGN addition on the tensile dataset of HDPE and H60 foam.Similar plots are observed for all the other foams for both the tensile and flexural datasets.Different value of noise is applied for comparison purpose.Based on the strength values, the value of σ for tensile data is taken as 5, 10, and 20, whereas 0.2, 2, and 5 are considered for flexural data.The trend in plots clearly shows that the model accuracy is not affected post noise induction, as very small increments are observed in RMSE values.
Fig. 13 shows RMSE values in training and testing datasets for all composite materials for the tensile datasets, whereas Fig. 14    in the testing set.When the error in the training set is far greater than the error in the testing set, it is a case of overfitting.In contrast, when the error in the training set is far lower than the error in the testing set, it is a case of underfitting.In our case, error values in both the training and testing set are very close, implying that our models are working fine.It is a case of a balanced fit.Among ensemble-based models, LightGBM shows the least error, and among Automated ML algorithms AutoSKL and TPOT show the least error, and the H2OAutoML algorithm is also working well.

Effect of noise addition on RMSE for all ML algorithm
Fig. 15 shows the effect of different noise addition on RMSE values over the tensile dataset; similarly, Fig. 16 is for the flexural dataset.It has been observed that RMSE values continuously increase as AWGN increases in the dataset.This is an obvious fact, as by inducing noise to the dataset, the model will be able to memorize fewer training samples.Nonetheless, the accuracy is maintained to a value greater than 95%, post noise induction, implying that all six ML algorithms are working on a balanced fit.The change in trends is observed in RMSE values from MLBox and H2OAutoML algorithms after initially inducing noise to the datasets.Later, the trend exhibits linearity.On the other hand, other algorithms show almost a linear trend for a tensile dataset.For the flexural dataset, RMSE values change almost linearly in all models on inducing noise to the dataset.Figs. 15 and 16 show that TPOT shows the least error in all the cases for all composite materials.

Resource consumptions
Apart from model accuracy and model metrics, the efficiency of the ML model depends on the computational resources, i.e., memory usage and time consumption during the execution of the ML algorithms as applied to 3D printed closed cell foams mechanical response.Fig. 17 shows the memory usage and time consumption by all the six ML algorithms for a representative HDPE case, and the values for all the 3D printed closed cell foam compositions are listed in Table 3.It can be seen that, among AutoML, the maximum memory consumed during execution is observed in the TPOT.Next to TPOT, AutoSKL has consumed maximum memory, followed by MLBox and H2OAutoML.Among ensemble-based ML algorithms, LightGBM consumed the least memory.With respect to the time consumption perspective, which might be a crucial parameter for advanced closed cell foams like functionally graded foams and sandwiches therein, AutoML algorithms, MLBox consumes the least time to predict the results.In contrast, among ensemble-based algorithms, LightGBM consumes the least time.

Ensemble-based algorithm
LightGBM shows the least training error of 0.110 and a testing error of 0.141, followed by XGBoost exhibiting the training and testing error of 0.168 and 0.269, respectively.The dataset utilized in the present work is voluminous and contains all distinct values post-noise addition.LightGBM works superior on large datasets.In contrast, XGBoost works better on lower-sized datasets and is the reason for LightGBM exhibiting less error than XGBoost, although the working principle of both models is almost the same.XGBoost works on level-wise tree growth and has a balanced tree structure, whereas LightGBM works on leaf-wise tree growth and has a comparatively unbalanced structure.The difference in model parameters will lead to a difference in the model accuracy, processing time, and memory consumption during the execution.In XGBoost learning rate is 0.15, the maximum depth is 8, and n_estimators are 5000, whereas in LightGBM learning rate is 0.25, the maximum depth is 4, and n_estimators are 500.Hence, the LightGBM consumes the least time of 0.293 s and memory of 182.2 MB as it has to work on a ten times lower combination of trees, almost double the learning rate and half tree depth.While the memory consumed by XGBoost is 201.8MB, with an execution time of 20.685 s.

Automated machine learning algorithms
Among automated ML algorithms, the TPOT algorithm shows a significantly lower training error of 0.122 and a testing error of 0.121.This might be due to the TPOT algorithm among four automated ML algorithms being capable of testing multiple pipelines according to the given time of program execution, i.e., 10 min in our case, and 3 to 4 pipelines are created during the execution of the program.The best pipeline obtained by the algorithm is the AdaBoost strategy with a learning rate of 0.5 and n_estimators of 100, and a cross-validation value of 5.This is why it consumes a maximum memory of 549.4 MB and a program execution time of 613 s.After the TPOT algorithm AutoSKL algorithm consumes a peak memory of 387.8 MB with an execution time of 177.86 s.It gives a training error of 0.102 and a testing error of 0.121.It checks 50 different models during the execution of the program with a maximum time limit of 3600 s.Since we have set a manual execution time limit of 180 s, the eight numbers of the model were computed in which ard_regression shows the least error values, which assigns weights of the regression model in a Gaussian distribution manner.Among automated ML algorithms, H2OAutoML consumes minimum memory of 193.9 MB with a model execution time of 92. with the feature importance plot.

Conclusions
This study applied six ML algorithms (XGBoost, LightGBM, AutoSKL, MLBox, TPOT and H2OAutoML) to the earlier published datasets.The model accuracy of each model was compared, and strength values were predicted.Results show that all four AutoML algorithms are working superior to the powerful traditional ensemble-based ML algorithms.The following are the main conclusions obtained from this study - • Predicted values from all the algorithms almost overlap with the actual tensile and flexural strengths implying their suitability for the envisaged work with a model accuracy of greater than 95%.
• The training and testing error differences were significantly less, implying a balanced fit.
Future works will focus on generating more variability in process parameters data that can be obtained by performing several experiments from the FFF 3D-printing process and considering material parameters.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability
Data will be made available on request.
exhibits the RMSE of flexural datasets.In these graphs, the blue bar shows the error values in the training set, whereas the orange bar shows the error values

Fig. 12 .
Fig. 12.Effect of AWGN addition on the tensile dataset of (a) HDPE and (b) H60 foam.

Fig. 15 .
Fig. 15.Effect of noise addition on RMSE for all ML algorithms for the tensile dataset.

Fig. 16 .
Fig. 16.Effect of noise addition on RMSE for all ML algorithms for the flexural dataset.

Fig. 17 .
Fig. 17.(a) Memory consumed in execution of algorithms and (b) Execution time for 3D Printed HDPE.

•
Among the two ensemble-based algorithms, the LightGBM algorithm gives training and testing errors of 0.110 and 0.141, respectively, and consumes memory of 182.2 with a program execution time of 0.293 s.In contrast, XGBoost gives a training error of 0.168 and a testing error of 0.269 and consumes memory of 201.8 MB with a program execution time of 20.685 s.•Among automated algorithms, H2OAutoML is working superior to the other three algorithms (AutoSKL, MLBox, and TPOT), as it consumes the least memory of 193.9 MB during the execution of the program with an execution time of 92.617 s besides significantly less training and testing errors, respectively of 0.308 and 0.613.AutoSKL was best, provided the time constraint was relaxed.

Table 3
Memory and time consumption.