An active visual monitoring method for GMAW weld surface defects based on random forest model

In the automatic manufacturing of robotic welding, real-time monitoring of weld quality is a difficult problem. Meanwhile, due to volatilization of zinc vapor in galvanized steel and complexity of welding process, the existence of welding defects greatly affects industrial production process. And real-time detection of welding defects is a key step in development of intelligent welding. To realize real-time monitoring of weld surface defects, an active visual monitoring method for weld surface defects is proposed. Firstly, after applying Gabor filter to remove interference signals such as arc and noise, obtain weld centerline image; then employ the slope analysis method to extract peak valley coefficient of weld defects, extract five feature points of weld surface quality by Douglas-Puke algorithm, and analyse geometric and spatial distribution features of different types of defects in weld laser stripe images. Finally, using eight feature vectors extracted from weld features, design a weld defect recognition random forest model based on decision tree. After optimizing the decision tree depth and number of model evaluators, compared with the traditional decision tree ID3 and CART algorithm model, this model has better performance than traditional machine learning algorithms on five weld surface defect data sets. The experimental results show that accuracy of weld defect identification in the training set is 99.26%, accuracy of weld defect recognition in the test set is 96%, and processing time of a single image is only 5.3 ms, which overcomes difficulty of real-time weld defect detection in intelligent welding and ensures real-time and accuracy.


Introduction
Galvanized steel is extensively employed in applications such as architecture and automotive industry [1][2][3], providing structural components with excellent corrosion resistance improving product lifespan. Nevertheless, during the gas metal arc welding (GMAW) welding process of galvanized steel [4], the zinc metal on the surface coating will evaporate and melt under the high temperature and high pressure of the weld pool, and reduce its corrosion resistance with the generation of zinc vapor. When these zinc vapors can not escape smoothly, weld defects including pores and dent will be formed, which will affect the quality of welding seam [5][6][7]. The problem of weld formation has always been a difficult project to resolve in large-scale automotive structural components. Therefore, real-time quality control of galvanized steel in the GMAW welding process is one of the essential technologies in the development of intelligent welding.
During the welding process, the welding quality is easily disturbed by various factors. A significant proportion of traditional offline testing methods have been automated or semi-automated, but whether they are destructive or non-destructive, they may increase consumption in time, materials, and productivity. There are mainly x-rays [8], ultrasound [9], penetrating liquids [10], magnetic particles and eddy current tests for detection [11,12]. However, these five methods all have shortcomings. X-ray images have the characteristics of complex image texture, uneven illumination, and poor quality, and the image processing is relatively time- Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
consuming. Ultrasonic methods generally have inferior detection capabilities for shrinkage defects, requiring professionals to operate before testing. The false negative rate of penetrating liquid is as high as 48%. Magnetic particle detection is limited by the material and shape of the test piece, which is difficult to achieve automation. The current test of eddy current can be automated, but the applicable materials are limited and the accuracy is low. Therefore, numerous researchers have proposed several solutions for real-time welding quality monitoring through sensing technology to provide real-time information to control the welding process, containing vision sensors [13], acoustic sensors [14], arc sensors and spectral sensors [15,16]. Among them, acoustic sensors are expensive and difficult to be used in large-scale actual welding production. Arc sensors and spectral sensors generally limited to the application of the continuous and stable GMAW process of the welding process because they are susceptible to arc fluctuations. Compared with them, The passive vision method is suitable for low requirements for lighting and no requirements for scenes. However, welding is a very complex process, which is easy to be disturbed by noise such as arc light, splash and smoke, which makes it very difficult for subsequent image processing and information extraction, thus affecting the monitoring of welding quality. Active visual monitoring has the advantages of low cost, simple equipment, non-contact measurement, and high accuracy. It can obtain abundant visual information in the welding process. Therefore, it has plenty of applications in welding automation, such as weld extraction [17], weld tracking and welding quality monitoring [13,18].
Visual monitoring is proverbially employed in different fields of industry, especially in intelligent welding manufacturing. Han et al [13] proposed a structured light vision sensor to realize real-time weld measurement and quality inspection. Chu et al [19] designed a vision-based automatic inspection system for welding quality of shell and tube, which can accurately identify the position of the undercut. Zeng et al [20] employed a weld inspection platform based on the light shadow features of directional light, which is expected to be applied for precise positioning and automatic guidance of probes in non-destructive testing. Wu et al [21] utilized a threecolor hemispherical array CCD camera to classify and predict the defect types of chip welded joints, with an accuracy rate of 97.7%. Bacioiu et al [22] designed a high-dynamic camera to collect images of the molten pool in the TIG process of aluminum alloy, and combined with an adaptive neural network to accurately identify five different weld defects. Lertrusdachakul et al [23] adopted visual sensors to complete automatic wire guidance and molten pool measurement, and verified to ensure that the welding system has high reliability and robustness in different welding experiments. Du et al [24] devised Convolutional Neural Network (CNN) to reduce the difficulty of parameter adjustment and improve the stability of the seam tracking system. Yang et al [25] realized the latest single-stage target detection algorithm YOLOv5 applied to the field of steel pipe weld defect detection, which can improve the detection accuracy of welding quality. Pan et al [26] devised a new transfer learning model based on MobileNet as a feature extraction tool for welding defects, with fast speed and prediction accuracy of 97.69%. Consequently, machine learning and deep learning algorithms based on vision sensors demonstrate excellent characteristics and are expected to realize real-time monitoring. However, there are not enough training data sets in actual production. Using deep learning algorithms to identify weld defects is timeconsuming and less accurate, which is easy to lead to redundancy in the classifier.
In this paper, an active visual monitoring method for weld defects is proposed. By calculating the feature parameters of the weld laser stripes and the machine learning algorithm, this method can monitor the welding process and weld quality, find the type and location of weld defects in real time, and improve the stability of welding quality and industrial production efficiency. It is of great significance to promote the development of intelligent welding. In addition, the random forest algorithm used can generate new unknown samples by itself, enhance the data set, and improve the generalization ability. The structure of the thesis includes: section 2 introduces the experimental device and the image algorithm of weld profile extraction, and discusses the relationship between feature parameters and weld features, and then builds a decision tree algorithm model. In section 3, we employ experimental results to explain the weld profile information and weld defect algorithms we have established. In the last part, we summarize the application scenarios and performance expectations of the technology, and discuss further research directions.

Materials and experimental platform
In order to realize the real-time monitoring of weld defects of galvanized steel, this paper establishes the GMAW intelligent welding system, as shown in figure 1. The welding system is mainly divided into two parts: welding equipment system and image acquisition system. The welding equipment system mainly includes Huayuan NB-350IGBT welding power supply, wire feeding system, welding torch, 99.9% pure Ar shielding gas and stepping motor control system. The image acquisition system is mainly divided into two parts: hardware and software. The hardware part refers to the red visible light emitted by the FU655AL30-HGD1465 line laser with a wavelength of 660 mm and a power of 50 mW. The reflected laser stripe contains a large amount of weld information, as shown in figure 1. The original image collected by the WAT-902H2 CCD camera is a gray image with a size of 768×576 pixels, and then use dimming film and filter to remove the interference information during the welding process, such as spatter, arc and noise signals. The software part is mainly based on Viscal C+ + platform to detect the weld quality. Table 1 shows the main parameters of GMAW welding system.

Image processing algorithm
Due to interference signals such as arc light and noise, the background environment of laser stripes is quite complex. As shown in figure 2, red data represents laser stripes and blue represents background information.
Common filter functions such as Gaussian filter and median filter can not successfully extract the details of the weld and while removing the arc light, it also desalinates the laser fringe, as shown in figure 3.
Gabor filter is an extraction algorithm based on directional features. The expression of directional features by Gabor filter is very similar to human vision. For the original welding image, the weld profile can be accurately extracted by changing the angle and direction. On the one hand, it can highlight the details of the weld seam, on the other hand, it can effectively suppress the interference of arc and weld pool on weld extraction. Gabor filter function is generally obtained by multiplying Gaussian function and cosine function, and its equations are (1) and (2).
Where (x, y) is the pixel coordinate of the weld image, λ is the filter wavelength, θ is the filtering direction of the image, σ is Gauss standard deviation, j is the phase offset, usually 0, γ is the ellipticity of the filter core function, usually 1. Among these parameters, θ has the greatest influence on the results of laser stripe extraction. After numerous offline image tests, it is found that when θ=5°, the laser stripe extraction is the best, as shown in figure 4. Before threshold segmentation, first region of interest (ROI) process the Gabor filtered image, extract the effective part of the laser stripe and remove the interference of the board, which can improve algorithm speed and reduce the calculation time. Threshold segmentation can remove the influence of background image on laser fringes. Otsu is the most commonly used algorithm in threshold segmentation [27]. It can automatically find the best threshold. The image after threshold segmentation is shown in figure 4. After threshold segmentation, use morphological method to remove the interference of surrounding small blocks.
In order to achieve real-time detection, extracting the centerline of laser stripes is an indispensable step. In this paper, the geometric center method is used to extract the centerline of laser stripes. Compared with the thinning and skeleton methods in morphological processing, the geometric center method uses the average value as the coordinate of feature points, which has faster speed and meets the requirements of accuracy. The other two methods need to calculate the pixel value in the moving window, which is difficult to meet the realtime requirements of weld defects. The principle is that the laser stripes are distributed symmetrically in the center, and the geometric center point of its section is taken as the center point of the stripe. The specific steps are: traverse each column of pixel points of the image from top to bottom and from bottom to top respectively, find two coordinate points x 1 and x 2 , and take their average value as the coordinates of the central point. If it cannot be found, it is regarded as a hollow point here. Figure 4 shows the extraction result of the laser stripe centerline.

Feature extraction of weld defects
With the significant improvement of computing power, machine learning algorithms are extensively employed in the field of intelligent welding to ensure the detection and classification of weld defects during the welding process. Real-time monitoring of weld quality during GMAW welding of galvanized steel includes weld profile extraction, feature vector extraction, prediction model. In the welding experiment of galvanized steel, it is difficult to control the welding quality due to the existence of zinc vapor. The common weld surface defects of galvanized steel are generally divided into five types: dent, undercut, pore, burn through and normal seam.

Slope distribution
After extracting the centerline, the laser stripe image provides accurate coordinate information for welding automation, but it is not enough to study the weld surface defects, so judge whether there are defects at this position according to the slope change of weld profile. The slope change rate formula is shown in equation (3). It determine whether the weld profile is continuous, so as to detect whether there are defects on the weld surface and the specific location of defects.
In the equation (3), x and y respectively represent the abscissa and ordinate on the weld profile, and k j represents the slope change value of the j-th point. Figure 5 shows the variation diagram of weld slope, where peak valley index (PVI) represents the peak valley coefficient of weld profile, and the minimum threshold in this study is 0.1. When the weld is normal seam and pores, the PVI changes ⊕ from to !. When the weld surface defect is dent, the PVI is ⊕ to ! and then to !. When the weld surface defect is undercut, the PVI is ! to ⊕ and then to !. When the weld defect is burn through, the PVI is ⊗. Therefore, the variation diagram of welding slope can generally reflect three PVI features.

Distribution of feature points
Firstly, in order to reduce the interference points on the centerline of weld profile, we adopt Douglas-puke algorithm [27], also known as the iterative adaptive point extraction algorithm. As shown in figure 6(a), the main steps of the algorithm are: connect the two points a and b at the beginning and end of the centerline of the weld profile to form a straight line ab; Calculate the distance L from the point on the weld centerline to the straight line ab; Find the maximum coordinate point c between these distances, divide the centerline of point c into two curves ac and bc, and repeat the previous processing steps until the centerline of the weld is processed. Figures 6(e)−(f) shows the feature point extraction results of various weld defects. According to the centerline and feature points extracted by the weld structural light, we define a defect index to reflect the weld feature state, and its expression is W / N. Where N is the number of 255 pixels in the centerline, and W is the defect width.

Random forest algorithm
Random forest algorithm (RF) is one of the most commonly used and most powerful supervised learning algorithms which combines the ability to solve regression problems and classification problems. RF is an algorithm that integrates multiple decision trees through the idea of ensemble learning. For the classification problem, the output category is determined by the mode of individual tree output. In the regression problem, the output of each decision tree is averaged to get the final regression result. Compared with other machine learning classification algorithms such as Logistic Model Trees (LMT) and CART, RF algorithm is insensitive to multivariate common linearity, the results are more robust to missing data and unbalanced data, and can well predict the results. As shown in figure 7, the basic unit of RF is decision tree, which contains several important features, low computational complexity, high-precision processing, simple model. The steps of RF algrithom are as follows: Step 1. Assuming that the training data set has a total of M object data, N samples are randomly selected from the sample data by the method of Boostrap, because there is replacement extraction, some data may be selected multiple times, and some data may not be selected. The samples taken out each time are not exactly the same, and these samples constitute the training data set of the decision tree.
Step 2. Each sample set is used to construct a decision classification tree. Assuming that each sample data has K features, k features are randomly selected from all the features, and the best segmentation attribute is selected as the node to establish the CART decision tree. The size of k remains unchanged during the growth of the decision tree, allowing each tree to grow fully until the purity of each leaf node is minimized without involving the pruning process.
Step 3. Repeat the previous steps to establish m CART trees, these trees must be fully grown and not pruned, these trees form a forest; According to the constructed classifier, a new unknown sample is predicted, and then the simple majority voting method is applied to the voting results of each tree classifier to determine the classification results of unknown samples.
An important parameter of RF is Gini coefficient. The selection standard of Gini coefficient is that each child node reaches the highest purity, that is, all observations falling in the child nodes belong to the same category. At this time, Gini coefficient is the smallest, purity is the highest and uncertainty is the smallest. For a general decision tree, if there are K categories in total and the probability that the sample belongs to class k is: p k , the Gini index of the probability distribution is: The greater the Gini index, the greater the uncertainty; The smaller the Gini coefficient, the smaller the uncertainty, the more thorough the data segmentation and the cleaner.When we traverse each segmentation point of each feature and using feature A=a, D is divided into two parts, namely D 1 (a sample set satisfying A=a) and D 2 (a sample set not satisfying A=a). Then, under the condition of feature A=a, the Gini index of D is:

Definition of weld defects
The welding experiment collected a total of 1218 welding seam laser fringe images. The number of images of each defect type is shown in table 2, normal seam is represented by label 1; the dent is indicated by label 2; undercut is indicated by label 3; pores are indicated by label 4; burn through is indicated by label 5. The ratio of training set to test set in machine learning algorithm model is usually 7:3, so 853 images are selected for training data set and 365 images for test data set. In order to ensure the accuracy of the result data, all experiments are run on a computer with CPUi7-7700HQ, clocked at 2.80GHz and 8G of memory.

Performance of RF model
At present, Decision Tree (DT) is the basic model of the tree model series. Like RF, Boosting Tree, GBDT, XGBoost, etc have evolved on its basis. DT and its evolution model are extensively used in data mining, data analysis and intelligent marketing, but they are rarely used in the industrial field, especially for defect classification and regression. This paper attempts to construct the basic knowledge of decision tree, such as information entropy, information gain, information gain ratio and Gini coefficient. DT is a classic classification and regression method based on rules. Its model structure presents a tree structure which can be regarded as a set of if-then rules. It generally includes three steps: feature selection, decision tree construction and decision tree pruning. Therefore, the accuracy of ID3, CART and RF algorithms in the decision tree are compared with the  weld defect data. Figure 8(a) shows the test set results. In all experiments, the mean value of the results of the ten cross-validation experiments is the final result, excluding the contingency of the experiment. ID3 decision tree constructs a decision tree based on information gain; for the training set or subset D, calculate the information gain of each feature, compare the size, select the feature with the largest information gain as the feature of the node, and establish child nodes from different values of the node. The above steps are recursively performed on the child nodes to construct a decision tree; until the information gain of all features is less than the preset threshold or there are no feature. The disadvantage is that the information gain tends to take more values. C4.5 decision tree constructs a decision tree based on the information gain ratio; the C4.5 and ID3 algorithm just replace the information gain selection feature in ID3 with the information gain ratio selection feature. The CART decision tree constructs a decision tree according to Gini coefficient. The RF model is based on the CART decision tree and then voting results, and its model performance is slightly higher than the result of the CART decision tree. Figure 8(b) shows the feature importance score map of the RF model, in which the number of weld profile points, peak-valley coefficient and defect index weight are relatively high, and the total proportion of the three is nearly 63%, which is an important classification index for the RF model to improve the classification of weld defects.
In order to further verify the influence of RF model parameters on weld defect classification results, we conducted two groups of experimental tests on the depth of RF model tree and the number of feature evaluators, in which the maximum depth of tree is 20 and the number of model feature evaluators is 60. The experimental results are shown in figure 9. When the depth of tree reaches 4, the accuracy of test sets is 93.03%, and when the depth of tree reaches 5, the accuracy of test sets is 97.13%; when the depth of tree continues to increase, the accuracy of model increases slowly. Figure 10 shows the calculation process of RF model tree with a depth of 5. Figure 9(b) shows the change trend of test sets accuracy rate with the number of model evaluators. As shown in figure 9(b), when the number of evaluators reaches 10, the model accuracy rate increases slowly.
As shown in figure 10 below, when the Gini coefficient is large, the RF model's classification result of weld defects is not obvious. As the depth of the tree increases, the Gini coefficient gradually decreases, and the first identification is burn through. Because the sum of the weld profile points is the smallest and the standard deviation skewness can also distinguish the burn through from other weld features; then normal seam and pore defects are distinguished according to kurtosis and peak valley coefficient. Undercut and dent are distinguished by the height and width of the defect features.

Discussion
According to RF decision tree model, the sample training of weld defect is carried out and model parameters are set: the number of model evaluators is 10, the model depth is 5, and the model evaluation standard is Gini coefficient. 853 groups of data in the training set experimentally verified, and the classification results are shown in figure 12. The accuracy of weld defect classification and identification on the training set is 99.26%; 365 groups of data in the test set are experimentally verified. The classification results are shown in figure 12. The classification accuracy of weld defects on the test set is 96%, and figure 11 shows part of the weld defect data set. As can be seen from figure 12, undercut on the training set is misidentified as a pore, and the recognition accuracy of the other four weld types on the training set is 100%. The reason is that when size of undercut defect is small, laser stripe image of the weld is very similar to that of pore, and for some weld edge pore and larger pore, it is also easy to be misjudged as undercut. As shown in figure 13, multiple groups of dents and undercuts are wrongly identified as pores because the size is small, and the feature points are particularly similar to pores. On the contrary, larger pores on the weld are also easy to be identified as undercuts and other defects. It can be seen from figure 13 that with the increase of tree depth and estimator, the convergence speed of the RF model becomes slower, and accuracy of model's recognition of weld defects also increases more slowly. However, for the calculation time on weld defect recognition model, running time of each weld laser stripe image is only 5.3 ms, which is 25% higher than other traditional detection algorithms. It overcomes difficulty of real-time detection of weld defects, ensures the real-time and accuracy of defect detection, and improves the intelligent level of welding.

Conclusions
In this paper, a method is developed for weld seam recognition. The method realizes the recognition and classification of weld surface defects of galvanized steel in complex GMAW welding environment, and provides guidance for welding robots to solve the problems of poor real-time, poor adaptability and low degree of automation. According to structured light sensor image collected by CCD camera, a weld seam feature extraction algorithm based on active vision sensor is designed, and then a weld defect recognition model is established combined with machine learning algorithm, which calculates the feature parameters of each defect on laser stripe and establish a RF decision tree model. The following conclusions can be drawn from the experimental results: (1) A Gabor linear filtering algorithm is proposed, which can effectively remove the interference factors such as arc and noise during the real-time detection of GMAW weld defects. Combined with traditional image processing algorithms, the extraction of the centerline of laser stripe is completed, which has strong robustness.
(2) According to the features of the surface defects of different weld types in laser stripes, the Douglas-Puke feature point detection algorithm is proposed for the centerline of the laser stripes. Compared with other feature point detection methods, this algorithm has higher accuracy and better stability.
(3) The feature parameters of the weld surface defect are calculated. In traditional machine learning algorithms, the classification results of decision trees are better than logistic regression, SVM and clustering, etc, with low computational complexity, high precision, simple processing process and model. Therefore, a RF weld surface defect recognition and classification model is proposed based on the decision tree. The weld defect recognition rate is as high as 96%, and the single image processing time is only 5.3 ms. This model is more suitable for the practical application of intelligent welding.