Optics and Laser Technology An improved methodology of melt pool monitoring of direct energy deposition processes

(cid:129) A novel image processing technique to more accurately measure the width of a melt pool was developed. (cid:129) Validation of the technique was provided through single frame analysis. (cid:129) Results highlighted fundamental issues with emissivity-based edge detection methods. (cid:129) New technique can create enhanced control systems and optimise deposition quality. Additive manufacturing processes have previously bene ﬁ ted from the introduction of melt pool dimensioning systems. These typically measure melt pool width by performing binary thresholds and highlighting edges using common edge detection algorithms. Melt pool monitoring systems have been successfully used to develop control systems and enhance process understanding. This paper presents an improved machine vision technique to enhance images in melt pool monitoring systems. Enhanced images contain features that indicate true melt pool edges. The research highlights potential ﬂ aws in more established emissivity-based image processing algorithms and a new image processing technique is developed. The new technique produced improved accuracy and performed melt pool measurements independent of emissivity values.


Introduction
Additive manufacturing (AM) is a growing technology used for component fabrication and repair that has gained significant interest from numerous industries in recent years, particularly the aerospace sector. The term AM encompasses a range of manufacturing techniques, all involving the net-shape production of three-dimensional prototypes and near-fully-dense components by consolidating material layer-bylayer with a heat source specific to the derivative. Much research is currently available where the AM process has been monitored to enable a more holistic understanding of the build process and a closer observation of any anomalies, with a large portion of literature focusing on measuring data surrounding the melt pool phenomenon. The melt pool phenomenon is a phase in the AM process where energy meets material to form a molten metal pool. It is the first instance where key variables interact, and monitoring can provide in-situ information about the build. Melt pool monitoring has been used to predict deposition quality, optimise parameters, develop control systems and detect defects.
Much literature details the use of visual, Near Infrared (NIR), Infrared (IR) and/or thermal cameras to monitor the melt pool behaviour. Meriaudeau et al integrated multiple CCD cameras into build chambers to distinguish what the key process variables are in AM process [1][2][3]. The trend of identifying key variables in AM processes to create better process understanding and to improve part quality is documented in the following research publications [1][2][3][4], with Ding et al producing a recent study correlating melt pool geometry, process parameters and microstructure [5].
Kruth et al published multiple papers using a system that combined a CMOS camera with a photodiode [6][7][8][9][10]. Both sensors were set-up in https://doi.org/10.1016/j.optlastec.2020.106194 Received 20 December 2019; Received in revised form 22 February 2020; Accepted 29 February 2020 a coaxial manner to monitor the melt pool throughout PBF processes. The photodiode in the system is used to integrate all the recorded radiation, whilst the camera is used to extract melt pool geometry. Craeghs et al further developed this system by implementing an off-axis camera to monitor powder distribution between different build layers in the PBF process [11].
Klenzczski et al used an off-axis camera set-up to develop automatic image processing techniques that detect dimensional accuracy and powder distribution properties [12,13]. This was also studied by Scime and Beuth who designed and compared multiple machine learning techniques to determine which performed best in detecting powder distribution defects [14,15]. Rodriguez et al. implemented an off-axis IR cameras into an EBM system to monitor parameters and improve build quality, firstly using subjective decision [16], before subsequently automating the process [17]. They were able to automatically detect geometry deviations and porous regions during builds. The same research group used their developed parameter monitoring technique to identify porosity in-situ and perform corrective actions through laser re-melting [18,19].
Colodrón used a coaxial CMOS camera to monitor melt pool behaviour and used an emissivity-based algorithm to calculate melt pool width [20,21]. This was later refined by Araujo et al. who used an elliptical approximation [22]. Hofman et al used a similar technique to successfully reduce the effects of dilution in a DED process [23].
Clijsters et al. used NIR cameras to calculate melt pool dimensions by correlating pixel intensity values with the solid-liquidus interface of the melt pool [9]. Melt pool dimensions were calculated using maximum pixel counts on a binary image. A similar emissivity-based technique was developed by Cheng et al. [24].
Multiple research papers have eradicated melt pool dimensioning and have instead introduced techniques that measure melting event signals, splatters, and/or signatures to predict process condition. Ye et al. investigated plume and splatter signatures and correlated them to balling defects [25]. Zhang et al. used image processing techniques to extract signal features and related them to input parameters and formed defects [26]. A series of other studies have correlated melt pool event signatures to defects or poor melt conditions in built components [27][28][29][30][31], with others using these techniques to predict mechanical properties [32].
Recent research has used machine learning techniques to develop melt event monitoring [28,29,33,34], including Zhang et al. who used them to identify anomalies during Powder Bed Fusion (PBF) processes [34]. Zhang et al. subsequently went on to use machine learning techniques to identify porosity during builds [35]. Khanzadeh et al. used machine learning to develop a defect monitoring system that could alter parameters in-situ [28]. Scime and Beuth implemented machine learning to counteract varying melt pool sizes in PBF processes [29].
Academic advancements are being propelled into industrial applications and commercial products. Concept Laser, a General Electric company, have introduced a range of process monitoring systems with one of them being melt pool focused [36,37]. The commercialisation of these products shows that melt pool monitoring techniques are valid for industrial applications.
Most melt pool monitoring techniques used in AM use highly saturated melt pool images and emissivity values to calculate melt pool dimensions [22,23,[38][39][40]. The most common image processing techniques are based on common edge detection algorithms (such as Canny) to locate feature boundaries in binary images [8,11,22,40]. Common edge detection algorithms often lead to unreliable melt pool measurements, with documented inaccuracies in readings [24]. This paper improves upon the current melt pool monitoring techniques by advancing machine vision techniques to produce higher quality images. Improved images of the melt pool allow for a better understanding of radiation mechanics. Using improved melt pool images, a new algorithm is developed that is not emissivity-based or material dependent. This new melt pool monitoring system is directly compared to an emissivity based technique using a parametric study.

Melt pool image optimisation
The new algorithm calculates melt pool width without the need for emissivity values. To achieve this, a system is derived to identify features that best indicate the true melt pool edge. Improved machine vision techniques have allowed for detailed melt pool images and edge features when depositing with EN25.

Adaptive exposure times
To achieve a series of high quality melt pool images, a range of laser power settings were adopted in this research with various exposure times. Preliminary studies highlighted that low exposure times produced true melt pool edge features, but varying laser power reduced image quality. To overcome this, a series of scans were produced to identify the best exposure time for various laser power settings. It was discovered that for the specific material adopted in this research (EN25), Equation (1) could be used to identify the optimum exposure time for laser power settings between 600 and 1200 W.
where E is the exposure time setting to be used, and P is the laser power setting being used when depositing with EN25 powder. Fig. 1 shows the melt pool images captured using optimum exposure times over multiple laser power settings. None of the images show high level of saturation and details of the melt pool are resolved.

The directional emittance phenomena
The directional emittance phenomenon occurs in melt pool images when using optimised exposure times as a result of grey body radiation not emitting equally in all directions [41]. This results in part of the substrate emitting stronger radiation in the direction of the sensor even though the surface temperature is lower.
Radiation emitted from the melt pool in the direction of the sensor is dependent on the melt pool's surface angle. Measured intensity is reduced at the edges of the melt pool due to higher surface angles. Higher emission levels are measured from the substrate due to low surface angles. This is referred to as the directional emittance phenomenon throughout this paper and results in images with true melt pool edges. Fig. 2 is a schematic representation of the emission theory.

Image processing algorithm
The conventional method of calculating melt pool width uses an emissivity-based binary threshold [7,20,22,23,39]. Binary thresholds are user defined and calculated by correlating a pixel intensity value with the materials melting temperature. Binary images are then commonly subjected to erosion and dilation, before edge detection algorithms are employed to identify the melt pool edge. The Canny edge detection algorithm is a common technique that will be used in this research [42].
The new algorithm uses the directional emittance phenomenon to calculate melt pool width. The contrast between the bright substrate and the dark regions of the melt pool provides features whose locations can be extrapolated using new techniques. The width of the melt pool is calculated by locating two edges of the melt pool independently. These edges are named the north and south locations and are depicted in Fig. 3.

Frame averaging
The image processing algorithm averages frames together to produce a more stable melt pool image for subsequent feature extraction.
Averaging reduces the noise produced by radiation emitting, scattering and reflecting from stray powder particles.
Frame averaging summates pixel intensities (at each given pixel location) across multiple frames. The resulting summations are divided by the total number of frames with each average frame being created by summating frames that have been previously acquired. The mathematical formula for this is given in Equation (2).
where P xy is the average pixel value at a given xy location in the averaged melt pool image, n is the number of frames averaged across, and a xy is the pixel value at a given xy location in melt pool image i.
The value of n used was seven. Values higher than seven resulted in blurred images with a lack of resolution, and values lower resulted in images with noise.
The frame averaging technique is performed using frames that have been previously recorded, meaning that performing frame averaging before frame number n requires a slightly different equation. Equation (3) shows the mathematical formula for this.
where m is the number of frames that have been acquired.

Centre of mass
The centre of mass calculation is used as an anchor point for subsequent melt pool width extraction. It is performed by reducing both rows and columns, before scanning the resultant arrays. Scanned arrays are then subject to conditional statements to determine half intensity locations in both the x and y directions.
The reduction equations for columns and rows are described in Equations (4) and (5) where R x represents the reduction value for a given column ( x ) in image a, y represents the column number, Y represents the maximum number of columns and a xy represents a pixel at the given xy location.
where R y represents the reduction value for a given row ( y ) in image a, x represents the row number and X represents the maximum number of rows.
The resultant arrays from these equations are then subject to an inclusive scan algorithm, as defined in Equation (6).
where S n is the resultant scan array with total length N, and R is the input array subject to the cumulative scan summation. The two scan arrays are then used to calculate the centre of mass in both the x (C x ) and y (C y ) directions using Equations (7) and (8) respectively.
where X and Y are the width and height of the image respectively.

Edge detection
The centre of mass is used in the edge detection algorithm as an anchor point for signal extraction. A signal is extracted, by averaging a set of columns located about the calculated C x , that contains the directional emittance phenomenon. The average signal extracted is defined in Equation (9).
x j x j (9) where A y is the average signal, 2j + 1 is the size of the average window, C x is the centre of mass in the × direction, and a is the averaged image.
The size of the image window used in this research was eleven. Values higher than eleven resulted in signals with reduced directional emittance prominence and values lower resulted in noisy signals.
The signal was then subjected to a moving average equation to eradicate excess noise using Equations (10) and (11).
T y is the smoothed signal, 2 k + 1 is the size of the smoothed window, y is the location of the cell in the array and Y is the length of the array.
The size of the smoothed window used in this research was seven. Values higher than seven reduced the prominence of directional emittance feature and values lower resulted in noisy signals.
Upper and lower thresholds were used to section parts of the signal that contain the directional emittance phenomenon. This reduced the chance of image processing error caused by noise in the melt pool image. The upper and lower thresholds are defined in Equations (12) and (13), with Equation (14) defining I c .
U is the upper threshold for a given laser power (P), U max is the user defined upper threshold value at the lowest laser power setting, U min is the user defined upper threshold value at the highest laser power setting, I c is the central intensity, and P max and P min are the maximum and minimum operating laser powers respectively. The values used for U max and U min were 0.7 and 0.4 respectively.
where L is the lower threshold for a given laser power (P), L max is the user defined lower threshold value at the lowest laser power setting, L min is the user defined lower threshold value at the highest laser power where T n is a cell in the smoothed signal at a given location y. C y is the centre of mass in the Y-direction and 2 m + 1 is the window size selected to calculate the central intensity I c .
The window size value used to calculate I c was 21. Values higher than 21 resulted in lower intensity pixel values being used in the calculations and values lower resulted in calculations being more subjected to noise.
The equations for extracting the north and south locations of the melt pool using the upper threshold, lower threshold, and smoothed signal are described in Equations (15) (16) where y North is the pixel location of the melt pool's north edge and y South is the pixel location of the melt pool's south edge. An example of a still image with its calculated north and south locations is depicted in Fig. 3.

Equipment set-up
In this research a Trumpf Trudisk 8002 5.3 kW disc laser DED system was used with a TruControl 1000 controller. Trumpf BEO D70 processing optics with a motorised collimation laser deposition head were employed with a Reis RV60-40 robot, a Reis RDKVv05 two axis manipulator and a Sulzer Metco 10-C powder feeder with a dual 1.5 kg hopper arrangement.
A NIR CMOS machine vision camera was installed to improve melt pool imaging and was combined with a UV/VIS cut off imaging filter with a 135 nm notch. The filter had an optical density of 3.0 for wavelengths of 200-750 nm and 4.0 for wavelengths of 1000-1200 nm. The notch filter and NIR CMOS camera were coaxially installed into the laser deposition head to allow for a clear birds-eye view of the melt pool.

Material
The deposition material for this study was the steel alloy EN25, supplied by Carpenter with produce code MicroMelt EN25. The composition is presented in Table 1. It should be noted that powder from different manufacturers and even different batches of powder can cause changes in melt pool characteristics.

Method
To determine the accuracy of the new technique's melt pool dimensioning, single line cladding tracks were deposited and recorded for image process analysis. Clad tracks of 150 mm length were deposited with all parameter settings fixed (3.5 L/min carrier gas flow rate, 6 L/ min shielding gas flow rate, 11.25 mm/s velocity, 1.5 mm laser spot size and 4.8 g/min powder mass flow rate), apart from laser power which was varied between 600 W and 1200 W. Forty-two clad tracks were deposited using both fixed and adaptive exposure times with increments of 30 W. Recorded videos of deposition tracks were subjected to both the conventional Canny edge detection and new directional emittance image processing algorithms in a MATLAB environment. Hand calculations and track width measurements were performed to determine algorithm accuracies. Hand calculations were performed using a second MATLAB program where measurements of random sample frames (total of 20) were used to calculate melt pool width averages. Track width measurements were performed using a VHX-700F digital microscope.

Results
Melt pool width averages were calculated for all techniques on multiple laser power settings. The averages were plotted against each other, to establish algorithm accuracies. The results of this study are displayed in Fig. 4.
The relationship between track measurements and melt pool width calculations proved to be complex. The change in polynomial trend between these curves is expected to be a result of both shrinkage and 'slumping'. Slumping is when the forces of gravity act on the melt pool during its liquid state, causing the track to spread laterally. At higher laser power settings, slumping is more prominent as the melt pool is in a liquid state for a longer period. Due to the complex relationship between melt pool and solidified track widths, correlating algorithm calculations with measured track widths was deemed unreliable. Fig. 4 shows that there are large differences between the calculated Canny edge detection values and both the directional emittance and hand calculations for the lower laser power settings. Observing measurements on individual frames highlighted major issues with the emissivity-based (Canny edge) technique. It was found that the melt pool edge cannot be approximated using a single binary threshold. Fig. 5 displays a melt pool image with its corresponding binary threshold. The image shows a correctly and incorrectly calculated edge, proving that the melt pool edge doesn't always occur at the same pixel intensity. The directional emittance measurements showed a very strong relationship with the hand calculations. Both techniques display a similar relationship and all data points of the direction emittance measurements are closer to the hand calculations than the Canny edge measurements. A second order polynomial was fitted to all data sets to display the relationship between melt pool/track width and the laser power settings.
To determine the accuracy of the new algorithm, hand calculations were deemed to be true values. Both the Canny and directional emittance algorithms were directly compared. The new algorithm calculated melt pool width values within 1% of the hand calculations. The Canny edge algorithm calculated width values within 4.8% of the hand calculations.

Discussion
This research analysed the accuracy of existing melt pool dimensioning techniques and created an improved alternative. The new technique calculates melt pool width without material dependant emissivity values. This allow users to develop versatile melt pool monitoring systems that can perform material independent melt pool width measurements. The new algorithm showed improved accuracy. Melt pool monitoring systems with higher accuracy will likely improve process understanding, control system effectiveness and deposition quality.
There have been multiple studies that have attempted to calculate melt pool width in AM processes, but many have used emissivity-based values to correlate the melt pool edge with a pixel value [7,20,22,23,39]. Others have approximated that melt pool edges occur at the same pixel intensity [24]. Research papers have claimed that using emissivity values in this way to correlate a pixel intensity value with temperature is not reliable [16,17,24]. This research has expanded on these claims to provide clear evidence, whilst introducing a new solution that calculates melt pool width based on the newly discovered edge features.  Conventional edge detection techniques have been used on multiple occasions to develop advanced AM control systems [8,11,22,40]. The new algorithm could be used in a similar manner.
The rate at which DED clad tracks shrink during solidification could not be found in literature, but shrinkage rates in PBF processes are said to be influenced by both layer thickness and powder packing density [43]. Performing solidified track width measurements and comparing them to melt pool width measurements highlighted that the solidified track width is influenced by the laser power setting. Although both studies concern different processes, both this research and the referenced literature complement each other by suggesting that shrinkage rates can be influenced by process parameters.
Determining algorithm accuracy had limitations as there is no way of performing a physical measurement of the melt pool itself. This research determined the new algorithm was more accurate than the conventional Canny edge technique, but there was no determination of how accurate the systems were compared to the true melt pool width value. Hand calculations are a good indication of system accuracy, but further work should explore physical melt pool width measurements to improve confidence.
Studies have used melt pool monitoring systems to better understand the effects of parameter changes within AM processes including work from Meriaudeau et al. [1][2][3] and more recently Ding et al. [5]. Future work should utilise the new algorithm, with improved accuracy, to provide a more detailed understanding of process parameter interactions in DED processes. In addition, studies should be carried out to analyse the effects that changing both powder mass flow rate and path velocity has on melt pool dimensions, as improved process understanding often leads to improved deposition quality [20,25,26,31]. The use of the new algorithm on multiple materials should also be explored. The newly developed technique is not reliant on material dependant emissivity values, meaning that it should be able to perform melt pool calculations on new materials.
Conventional emissivity-based edge detection techniques do not accurately measure melt pool width. The newly developed directional emittance algorithm outperformed the Canny edge detection technique and was considered more accurate. This research has produced a melt pool dimensioning technique, with improved accuracy, that does not require material dependant emissivity values to perform calculations.

Conclusions
This research introduced a novel image processing technique that can more accurately measure the width of a melt pool when compared to conventional emissivity-based edge detection techniques. The accuracy of the new technique was validated by comparing measurements to hand calculations and performing single frame analysis. Measurements of solidified tracks were performed using an optical microscope, but it was discovered that the correlations between these and the melt pool could not be made due to the complex changes in width that occur during the solidification stage.
The results highlighted fundamental issues with emissivity-based edge detection techniques and showed that the melt pool edge often occurs at different thresholds. The new directional emittance algorithm detects the edge of the melt pool using changes in intensity and is not subject to the same issues. The advanced melt pool dimensioning techniques developed can be used to improve process understanding, create enhanced control systems and to optimise deposition quality.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.