Achieving better connections between deposited lines in additive manufacturing via machine learning

: Additive manufacturing is becoming increasingly popular because of its unique advantages, especially fused deposition modelling (FDM) which has been widely used due to its simplicity and comparatively low price. All the process parameters of FDM can be changed to achieve different goals. For example, lower print speed may lead to higher strength of the fabricated parts. While changing these parameters (e.g. print speed, layer height, filament extrusion speed and path distance in a layer), the connection between paths (lines) in a layer will be changed. To achieve the best connection among paths in a real printing process, how these parameters may result in what kind of connection should be studied. In this paper, a machine learning (deep neural network) model is proposed to predict the connection between paths in different process parameters. Four hundred experiments were conducted on an FDM machine to obtain the corresponding connection status data. Among them, there are 280 groups of data that were used to train the machine learning model, while the rest 120 groups of data were used for testing. The results show that this machine learning model can predict the connection status with the accuracy of around 83%. In the future, this model can be used to select the best process parameters in additive manufacturing processes with corresponding objectives


Introduction
The 3D printing technologies (also known as additive manufacturing (AM)) have been developed for more than twenty years.This technology has become a competitive technique for manufacturing parts with complex structures that are difficult or even impossible to be produced via conventional manufacturing technologies [1][2][3][4][5].Fused deposition modelling (FDM) is one of the most common AM technologies for manufacturing 3D parts.It has been widely used for fabricating not only prototypes, but also real life products [6][7][8].The principle of FDM is based on the manner of layer-by-layer printing and each layer is printed in a line-by-line manner [9,10].Figure 1 shows the process of slicing 3D models for fabrication.Once a 3D model is obtained, it needs to be sliced into layers and each layer will be sliced into lines as shown in Figure 1(a).Then the final sliced 3D model part can be obtained, which will be sent to the printers for fabrication.
Recently, machine learning technique and its applications have been largely developed due to the availability of large amounts of data and computer technology [11].Machine learning has the advantage of dealing with complex relationships between data and can handle many problems without knowing the exact theoretical principles behind.Increasing studies are published in the field of additive manufacturing with machine learning.Aoyagi et al. [12] proposed a machine learning method to predict a process condition that is effective at manufacturing a product with low pore density.Menon et al. [13] used hierarchical machine learning to simultaneously optimize material, processing variables and formulation in a 3D printing process.He et al. [14] used different machine learning techniques for modeling and predicting the proper printing speed in a vat photopolymerization additive manufacturing process.Francis and Bian [15] developed a deep learning method that can accurately predict distortion of parts in laser-based additive manufacturing.Similarly, Khanzadeh et al. [16] proposed an unsupervised machine learning approach (self-organizing map) to quantify the geometric deviations of additively manufactured parts in FDM.Zhu et al. [17] also developed a strategy coupled with machine learning to address the modelling of shape deviations in additive manufacturing.Zhang et al. [18] described a convolutional neural network strategy for monitoring porosity in laser additive manufacturing processes.
Process parameters are important in additive manufacturing.Tamburrino et al. [19] studied the influence of slicing parameters on the multi-material adhesion mechanisms of FDM printed parts.Different slicing parameters will affect the final printed properties.We previously [20][21][22][23][24][25] also studied different parameters' effects on printable overhang and bridge features.We also [26][27][28] proposed some new printing methods via improving path planning process.Luu et al. [29] proposed an efficient slicing method for Catmull-Clark solids with functionally graded material.A new slicing algorithm was proposed by Wang and Li [30] to guarantee non-negative error of parts fabricated in AM processes.A novel toolpath generation method was proposed by Flores et al. [31], with the aim of fabricating metallic components in laser metal deposition technique.Volpato and Zanotto [32] studied the influence of different deposition sequences on final printed properties in FDM.Ezair et al. [33] proposed a volumetric covering print-path slicing for printing 3D models with superior properties (such as mechanical strength and surface finish).Currently, Xiong et al. [34] proposed a process planning method for adaptive contour parallel toolpath in AM with variable bead widths.The reasons of carrying out the above studies are mainly because of that the slicing parameters in AM will influence the final printed properties and accuracy.Slicing parameters change in each fabrication process may lead to unstable results of 3D printed properties and dimension accuracy.As shown in Figure 1(b), will the designed width of the part (Wdesigned) equal to the final fabricated width (5 × w)?In fact, it is hard to achieve Wdesigned = 5 × w in a real fabrication process due to the change of process parameters.Here, w is the width of each printed line, as well as the distance value for slicing the part in each layer.In this paper, we propose a machine learning (deep neural network) model to predict the connection status between paths (lines) in printed layers.The aim is to achieve the best connection between lines in layers in different process parameters (filament extrusion speed, print speed, line distance and layer height), resulting in better dimension accuracy of fabricated parts as shown in Figure 1. Figure 2 illustrates the connection status between printed lines (paths).The connection statuses are divided into five classes with class 0 representing large space remaining between printed lines, class 1 meaning little space remaining, class 2 representing a good connection, class 3 meaning a little material overflow and class 4 being large material overflow between printed lines.If the printed lines are in the situation of 'class 0', then the printed materials in the next layer (upon current layer) will probably 'collapse', leading to lower dimensional accuracy of the printed parts.In the situation of 'class 4', the final printed parts will probably have a larger size then the original designed 3D model.This paper proposes a machine learning model to predict these connection statuses for improving dimensional accuracy of printed parts in the future.Section 2 introduces the machine learning model used in this study.Section 3 illustrates the experimental setup.Results are discussed in Section 4. The last section summarizes this study with some conclusions.

Machine learning model
Machine learning is one of today's most rapidly growing technical fields [35].It has the ability to learn a specific task without being explicitly programmed.The performance of the machine learning algorithms can be improved when they are exposed to more data over time.The most commonly used machine learning techniques include three types: Supervised learning [36], unsupervised learning [37] and reinforcement learning [38].Supervised learning technique (specifically, deep neural network) will be used in this study.The process of using machine learning for prediction is shown in Figure 3. Generally, the training data needs to be given for training the machine learning algorithm.In supervised learning, the training data are a collection of (x, y) form pairs and the objective is to get the predicted result yʌ in response to a query x.Both x and y can be more than one element which will be expressed as vector in machine learning.After obtaining the training data, machine learning algorithm will be trained based on these data and then update its parameters.This new updated machine learning algorithm can then be used to predict the result yʌ once given new input data x.
Deep neural networks will be used in this study.A neural network consists of some nodes (neurons) in a layer and links connecting the neurons to form a network.A typical neuron of a neural network generally has three main components: A weighted sum of the input values (∑ ), a bias (b) and an activation function (g) as shown in Figure 4(a).The input values (xi) in each neuron from previous connected neurons are multiplied with a weight (wi), and then summed.A bias (b) is then added to the weighted sum to control how easily a neuron is activated.This value is then passed to an activation function (e.g.Relu, Sigmoid or softmax), obtaining the final output (y) of the neuron.This process can be expressed as Eq 1.The weights (wi) and biases (b) are neural network parameters that can be tuned and learned automatically in the training process.
∑ (1) A number of neurons can consist a layer and a set of layers can be designed as a full neural network.A typical neural network has at least three layers: An input layer, a hidden layer and an output layer.The number of hidden layers can be one or more.Once a neural network has more than one hidden layer, this neural network is called deep neural network.In this study, a deep neural network model with 6 layers (as shown in Figure 4(b)) will be used to predict the connection status between printed lines in FDM.The corresponding numbers of neurons and activation functions used in each layer are listed at the bottom of each layer in Figure 4(b).ReLU and softmax are the activation functions used in this model.The full name of ReLU is Rectified Linear Unit.It returns an output equal to the input for all positive input values while returning zero for all other input values, as shown in Eq 2. The advantage of ReLU is that it can reach convergence much faster than Tanh/Sigmoid and it has no vanishing gradients problem [39].Therefore, ReLU has now been widely used in deep neural networks.The deep neural network model used in this study also used ReLU activation function in the first 5 layers.In the last layer of this model, softmax was used as the activation function instead as this model needs to categorize the connection statuses into 5 classes as shown in Figure 2. Softmax will normalize all values to the range (0, 1) and to a total sum of 1.For instance, the result of softmax ([0, 1, 2]) is [0.09, 0.24, 0.67].This makes softmax suitable for classifying multiple classes as the output values of softmax can represent the probability of each outcome.Eq 3 shows the softmax function.
The objective function used in our established deep neural network was categorical_crossentropy.The equation of this function is as follows: where y is the actual output, y^ is the predicted output and n is the number of input samples in each training process.The optimizer used for training in this model was Adam [40], and the metrics was set as "accuracy" in this model.The batch size in this model was set as 10 and epochs as 500.The algorithm codes of this machine learning model have been uploaded to Github and should be available through (https://github.com/jjia547/ML-for-AM).

Experimental setup
A Mbot3D FDM printer was used for all experiments as shown in Figure 5(a).The material used for all experiments was PLA (Polylactic Acid).The filament diameter of PLA was 1.75 mm and the nozzle diameter was 0.4 mm.In a fabrication process, the following four main process parameters are considered having influence on the connection status: Filament extrusion speed, print speed, layer height and line distance between paths.Therefore, a full factorial experiment with four factors was designed as shown in Table 1.Factor filament extrusion speed has five levels; print speed has five levels; line distance and layer height both have four levels.Therefore, totally 5 × 5 × 4 × 4 = 400 experiments were carried out.The two-line part was designed for observing the connection status between printed lines as shown in Figure 5(b).The red points are the positions for cross-section observation after printing.The reason of using two-line part is to reduce the fabrication time and cost as this part is simple but able to see the connection status.After fabrication, a HIROX KH-8700 3D digital microscope was used to observe these cross-sections for dividing the results into 5 connection statuses as shown in Figure 2.

Results and discussion
The printed 400 parts were labeled to classes 0, 1, 2, 3 and 4 based on the connection status as illustrated in Figure 2. As the data is too large, Table 2 only lists 12 of the 400 experimental results.We have uploaded the full labeled results on Github that should be available through (https://github.com/jjia547/ML-for-AM). Seventy percent of these data were randomly selected as the training set, while the rest 30% of data were used as the test data for evaluation.The train_test_split algorithm from sklearn in python was used to randomly split these data.Finally, the 280 groups of data were obtained for training, while 120 groups of data were obtained for evaluating the machine learning model's performance.The prediction accuracies in the training data after 100, 200, 300, 400 and 500 iterations are listed in Table 3.As can be seen, the final prediction accuracy on the training data achieved at 82.5%.After updating the parameters of this machine learning model, the test data were fed into this model for evaluation.The prediction accuracy was 83.33% in the test data set.Figure 6(a) shows the corresponding real classes and predicted classes in the training data samples, while Figure 6(b) displays that in the test samples.The blue point means the class of that specific sample and the orange point represents the predicted connection status.Note that for the overlapped points (that means the predicted equals to the real class), only orange points are visible as the blue points are behind them.As can be seen in Figure 6, most points are overlapped, which proves that this machine learning model works well and can be used for future predictions.

Tips on the machine learning model
The current machine learning model with 6 layers was tuned by the authors.We tried to change different algorithm parameters (e.g.neuron number in each layer) to obtain a better prediction accuracy.In a machine learning training process, there are many parameters that may influence the model performance.Generally, the following parameters can be tuned: Layer number, neuron number in each layer, the activation function (ReLU, Sigmoid or others), the optimizer (Adam, etc.), dropout, batch size and normalization.The current machine learning structure shown in Figure 4(b) is the best in the authors' trials.We have uploaded our data and the algorithm codes on Github (https://github.com/jjia547/ML-for-AM).The readers are encouraged to alter the codes and structure to tune this model if interested.
Currently we have 400 groups of data and 70% of them were used for training.The prediction accuracy on the training set is around 82.5% while the accuracy on the test set is about 83.33%.
Theoretically speaking, the more data fed in for training, the better performance of the model can be achieved.Therefore, another way to improve the prediction accuracy of the machine learning model is to obtain more data.
Another tip is based on the focus of the observation.For instance, if we only to distinguish the best connection status (class 2) and do not care about the others.Then we can label these data into only two categories (0, 1).Among that 1 means the connection status is class 2, while the others will be labeled as 0. In this case, it should be easier for machine learning to learn the features of these data, thus resulting in better prediction performance.

Applications
Figure 7 shows the framework of using the proposed machine learning model to select the best process parameters for AM fabrication.With different aims, the process parameters can be tuned correspondingly with a good connection between lines in the fabricated parts.

Future works
In the studies of [41,42], the authors showed that process parameters (e.g.layer thickness) in FDM can influence the mechanical properties of the printed parts.For example, the lower print speed can lead to higher mechanical strength of the printed parts [43].However, the above studies are based on the experimental results on changing one parameter and keeping others the same.They did not consider improving the connection status between lines by finding the best process parameters combination.In fact, the better connection status (class 2) may probably have better mechanical properties than classes 0 and 2. If this better connection status can be achieved by tuning the parameters through the machine learning model, different print speeds may all be able to achieve required mechanical strength.In this paper, though our aim is to achieve better dimensional accuracy of the printed parts, this study can be further used to investigate the mechanical properties of printed parts in the future.

Conclusions
In this paper, a machine learning model is proposed to predict the connection status between printed lines as shown in Figure 2.This model consists of six layers with ten neurons in the first layer, twenty neurons in the second layer, thirty neurons in the third layer, twenty neurons in the fourth layer, ten neurons in the fifth layer and five neurons in the last layer.The first five layers use 'ReLU' as the activation function and the last layer uses 'softmax' as the activation function.There are 400 experiments that were carried out on an FDM machine to obtain the data.Then these data were divided into five classes based on the connection status between lines as shown in Figure 2.There are 280 groups of experimental data that were used as the training set and the rest 120 groups of data were used as the test set for evaluation.The proposed model can predict the connection status with an accuracy of 82.5% on the training set and 83.33% on the test set.In the future, this model can be used to select the best process parameters combination for AM fabrication, with different aims.

Figure 1 .
Figure 1.(a) Process of slicing 3D models into layers and lines for printing.(b) Fabrication process and illustration of slicing width.

Figure 2 .
Figure 2. Illustration of connection status between printed lines (paths).

Figure 3 .
Figure 3. Procedure of using machine learning for prediction.

Figure 4 .
Figure 4. (a) Working principle illustration of a typical neuron.(b) Structure of deep neural networks used in this study.

Figure 6 .
Figure 6.Prediction performance on the training data (a) and test data (b).

Figure 7 .
Figure 7. Framework of machine learning enabled process parameters selection.

Table 2 .
Part of experimental results.