1 Introduction

Automation in welding production is essential to meet the current needs of the manufacturing industry in both high-wage and low-wage countries. The transfer of manual production to an automated system is motivated by the need for consistent weld quality, increased productivity, reliability etc. [1]. Therefore, the benefits of an automated welding system are accompanied by higher requirements for seam preparation, part alignment and process irregularities. To overcome these challenges and provide a more flexible and reliable system, adaptive process control is used. Process adaptivity can overcome varying process conditions [2]. In recent years, a variety of adaptive approaches has been presented. An industrially widespread application of adaptive welding is seam tracking. Many system solutions are commercially available on the market. The adaptive strategy is based on measuring the joint preparation and adapting the welding path accordingly. Seam tracking can be divided into pre-process and in-process approaches. The pre-process methods involve tactile or optical triangulation scanning of the seam. The in-process measuring principle can be based on the preliminary joint measurement using a triangulation sensor or in the process via an arc sensor [3]. In the past, the arc sensor was dominant for seam tracking applications due to its robustness and ease of handling. However, the limitations with regard to sheet thickness, seam preparation and material are disadvantageous. The current trend is towards optical sensor technology, which also includes melt pool monitoring [4]. Zou et al. present a method for seam tracking via the processing of weld pool images. By extracting the shape of the melt pool, the seam position is determined, and path adjustments are made. A CCD camera is used to capture the weld pool. The images are binarized using image processing algorithms and scanned pixel by pixel. This allows deviations between the actual and target position to be recognised [5]. Gao et al. present a method for seam tracking via the processing of melt pool images using artificial intelligence methods (Elman neural network) [6]. Other areas of application for adaptive welding include the control of seam geometry. Cao et al. realise an adaptive welding production system for regulating the backside weld width. The welding voltage in the TIG pulsed process is used as the measured variable. The pulse current time is controlled by modelling the welding voltage for the resulting backside weld width [7]. Additive manufacturing offers a large field of application for seam geometry control. Wang et al. present an implementation for layer-width control. By measuring the applied geometry layer by layer according to the triangulation principle, the deviation of the target geometry from the actual geometry is determined. For the next layer, the welding speed is determined segment by segment to compensate for the deviation [8]. The prerequisite for such process control is a closed-loop system in which sensors detect process deviations, and process adjustments are determined via feedback [9]. The adjustments can consist of different control parameters. Common parameters are, e.g. start point guidance, seam guidance, penetration control, stop point detection [10, 11]. The motivation for the use of adaptive process control is that variations in weld position due to thermal deformation and workpiece processing/assembly errors cannot be corrected by an automated welding system without the ability to detect with sensors and adapt with appropriate responses [11, 12]. Equally important is the adjustment of process parameters set on the welding power source [13]. In industrial environments, active vision sensors such as triangulation or arc sensors are used. A major disadvantage is the need for a pre-run of the sensor (passive vision) or restrictions on suitable seam preparation/oscillation (arc sensor). Active vision sensors for weld pool monitoring are currently a focus of research [14, 15]. The information density in weld pool images is very high. This allows a variety of sensors to be replaced. Possible actuators are the robot in robotic welding or the welding power source. Path correction is widely used in seam tracking applications. Camera-based process monitoring opens up new possibilities for implementing alternative adaptation strategies [16,17,18]. This requires customisable peripherals and functionalities. Manually applied process adaption strategies based on weld pool observation by a welder can be transferred to an automated welding system. Usually, manual adaption strategies are the torch orientation, torch positioning, welding speed and power input. The aim of the work presented here is to set up a robotic welding system that allows access to the main components of the welding system and allows flexible implementation of adaptation strategies of multiple welding parameters on the basis of weld pool images. The methods used are the adjustment of mechanical welding parameters (welding speed, torch orientation). In order to utilise the process stabilisation potential of current welding power sources, it is advantageous to operate in the synergic mode. Therefore, the adjustable electrical parameters are the synergic parameters of the welding power source. These parameters do not directly correspond to physical values but result in a specific regulation behaviour of the power source. Comparable process regulation intervention possibilities are found in most welding power sources for GMAW which offer a synergic mode.

  • Wire feed speed/welding current

  • Arc length correction/voltage correction

  • Dynamic factor

In previous work, a camera-based weld pool monitoring system was realised. It was shown that the system was able to provide geometrical weld pool information at a frequency of 20 Hz and could be used for adaptive process control [19]. In further work, a robotic welding system was built that allowed torch height correction and path correction based on weld pool images [20, 21]. Based on this work, the following describes the further development of an optimised welding system for adaptive process control This system is essentially characterised by:

  • Runtime optimization

  • Sequentially separating components for robust and flexible deployment

  • Extension of adaptive strategies

  • Real-time capable system

The functionality of the robotic welding system is verified through test welds. Two use cases are considered.

2 Requirements for the welding system

The welding system consists of four components, the welding power source, the robot, the process monitoring unit and the process control unit. Each component performs a specific task in the system. The robot is responsible for the mechanical movement of the welding torch, while the welding power source provides the welding energy. The process monitoring system provides the information needed for adaptive process control. The process control unit is responsible for communication between the individual components. The adaptive welding strategies are implemented here and are retrieved according to the task. The components should work independently in order to achieve a modular architecture. Therefore, the following requirements for the components are defined for the design of the adaptive welding production system.

2.1 Welding power source

The welding power source shall have an interface for external process control. The synergy parameters must be adjustable. Process status information shall also be available for sequence control and synchronisation with the welding motion. This information relates to successful arc ignition and completion of the process end sequence.

2.2 Robot

The robot must perform the welding and positioning movements. An interface for position specification and correction must be provided for adaptive process control. As the welding speed is an essential process parameter, it must be adjustable during the process. It shall also be possible to adaptively control the orientation of the welding torch. The welding movement shall be performed without stalling.

2.3 Process monitoring

A camera is used for process monitoring to observe the weld pool. The images taken must be of consistent quality throughout the process. The recorded images need to be processed by image processing algorithms. The relevant information about the process behaviour must be extracted and made available to the process control system for the generation of adaptation strategies.

2.4 Process control unit

The process control unit is the central control component in the welding production system. It links the individual components and is responsible for sequence control. Communication with the individual components must be parallel and deterministic to avoid latency. The process control must have a modular structure. This should allow a choice of process adaptation strategies and the addition of new functions.

3 Setup of the welding system

In order to achieve adaptive process control, a welding system consisting of four main components was set up.

  • Welding power source

  • Process monitoring

  • Robot control

  • Process control unit

In Fig. 1, the architecture of the components is illustrated.

Fig. 1
figure 1

Concept for adaptive robotic welding system

Full size image

At the centre is the process control unit, which combines all the inputs and outputs of the three components of the welding system. In the model setup, a primary switched welding power source EWM alphaQ 552 was used. To achieve adaptive parameter specification and automated process control, the interface between the process control unit and the welding power source was established through the robot interface module RINTX12. In order to acquire the geometrical information of the weld pool, the industrial camera TIS DMK 33GX273 is used for process monitoring. With a maximum image resolution of 1440 × 1080 pixels, an image recording rate of 30 fps is possible. For welding torch manipulation and adaptation of mechanical welding parameters, the ABB IRB2600 with IRC5 controller is used.

The components and functions implemented are described in detail below.

3.1 Welding power source

The synergic welding parameters can be adjusted via the welding power source. The RintX12 robot interface is used to enable parameter setting via the process control unit. Table 1 gives an overview of the functions used.

Table 1 Functions used for welding power source control
Full size table

Three analog inputs are used to control the synergic parameters for an impulse arc process. The wire feed speed, voltage correction and dynamic factor can be set between 0 and 10 V in the range of the maximum values of the process characteristics. Two relays are used to provide information on the welding start and stop status. As soon as the welding current is above 0 A, the relays close and signal that the start process (gas pre-flow time, arc ignition) is being carried out. At the end of the welding process, the machine changes its status from “not ready” to “ready to weld,” and the welding process is started by a digital input representing an external start.

3.2 Process monitoring

The process monitoring unit is responsible for camera control, image acquisition and image processing. The results of image processing are transmitted to the process control unit via TCP/IP protocol. The process monitoring unit is implemented as a server.

The server waits for the client to connect. When the client connects, the pre-processing mode is initialised, and live images are displayed. It allows the operator to set the camera orientation and focus.

When the server receives the process start message, the camera is initialised to in-process mode. The camera attributes are set to the welding settings. In trigger mode, each captured image is triggered by an external digital signal. The captured image is then processed. Finally, the captured images are stored for documentation purposes.

In order to achieve sufficient image quality, a self-developed optical system is used that can withstand the challenges of GMAW welding (spatter, arc radiation, welding fumes). A bandpass filter with a wavelength of 690 nm ± 10 nm is used to reduce the arc radiation.

3.2.1 Image acquisition

Due to the differences in brightness in a pulsed arc process as a result of different arc intensities, it is necessary to synchronise the camera and the transient welding current. The welding current is recorded and processed online. The trigger signal generated is shown in Fig. 2

Fig. 2
figure 2

Trigger signal in dependency of the welding current

Full size image

The base current phase is detected by a threshold. The threshold value depends on the wire feed speed, as this significantly influences the level of the base current phase. For wire feed speeds in the range between 3 and 10 m/min, the appropriate threshold values were determined with a resolution of 1 m/min. For the current curve shown, the threshold value is 150 A. The trigger signal is generated with a delay of 2 ms. This ensures that the image is always captured under the same exposure conditions and that the image quality remains constant throughout the process.

3.2.2 Image processing

The acquired images are scanned vertically and horizontally through a grid. The processing output is visualised in Fig. 3.

Fig. 3
figure 3

Image processing output and calculation of weld pool characteristics

Full size image

Contrast changes are detected by the grid. The threshold is set to 30 (8-bit grey scale). The subsampling ratio of the grid is set to 2 pixels. In the image shown, the ratio is set to 20 pixels for clarity. The three points describe the left corner of the weld pool (p1), the right corner of the weld pool (p2) and the position of the weld pool front (p3). The three points can be used to calculate the width of the weld pool (w) (w = p2 − p1). The offset from the centre of the weld pool can be calculated by the following equation:

$$offset= \frac{a-b}{2} with$$
$$a=p3-p1\;and$$
$$b=p2-p3$$

The output unit is pixels. Calibration to the distance in mm is done using the known wire diameter (1.2 mm). Calibration is performed each time the camera is modified or repositioned. The image processing itself is robust against changes in camera perspective, as there are no changes in brightness in the recorded image.

The quality of the generated weld pool characteristics is crucial for stable control. Welding spatter or welding fume can cause noise in the measurement. Therefore, the measured weld pool width is post-processed. Figure 4 compares the original and post-processed weld pool width curves.

Fig. 4
figure 4

Influence of the processing algorithms on the measured weld pool width

Full size image

The filtering is done in two steps. The first step is to check that the measurements are within a realistic range. All measurements above the threshold of 500 pixels (13.33 mm) are discarded. In addition, a median filter over four measurements is used. As illustrated by the red curve (weld pool width processed), deviations and peaks in the measured weld pool width can be compensated.

3.3 Robot

The welding robot controller works with a Digi Metrix robotics library for ABB. It allows remote control of the robot over a TCP connection. The functions relevant to process control are shown in Table 2.

Table 2 Overview of used robot functions
Full size table

The set speed command specifies the speed at which the tool centre point moves. A speed of 20 mm/s is used for positioning movements. During the process, the speed is set to the welding speed. The command “move linear” specifies the target position of the tool centre. The Euler coordinate system is used (x, y, z, ax, ay, az). These two functions and the options in welding speed, welding path and torch orientation are enabled. The “wait motion stop” function is used to synchronise the welding power source and the welding path. When the welding robot is in position (start position, end position), a signal is sent to the process control unit. For a smooth movement, the robot controller needs several positions in advance. Therefore, a buffer size of four commands is used. A balance between fast reaction time and smooth movement is achieved. Measurements show that a new command is sent every 25 ms (40 Hz). In order to realise a fast reaction in the mechanical welding parameters. The time between a new command being sent and its execution should be minimised. On the other hand, the robot should move smoothly. The time dimensioning assumes a maximum welding speed of 40 mm/s in applications with torch oscillation. The position increments are therefore calculated accordingly:

$$dx=b*v{s}_{s}$$
$$dx=\mathrm{0,025}\;s*40\frac{mm}{s}=1\;mm$$

Accordingly, the minimum increment for tool centre positioning is 1 mm. The minimum reaction time at a minimum welding speed of 4 mm/s is therefore:

$$t=\frac{dx*n }{v{s}_{s}}$$
$$t=1\frac{1\;mm}{4\;mm/s}*4=1\;s$$

In any case, a smooth movement is realised in the range of welding speeds between 40 and 4 mm/s. The reaction time from the sending of the mechanical adaptations to the execution is 1 s or 4 mm in path resolution.

3.4 Process control unit

The process control unit is the interface to all components of the welding production system and to the operator. The architecture is shown in Fig. 5.

Fig. 5
figure 5

Software architecture of the process control unit

Full size image

The architecture consists of three components. The HMI, the real-time controller (cRIO9049 RT) and the FPGA (cRio 9049).

The HMI allows conventional teach-in operation. The robot can be moved incrementally (x, y, z, ax, ay, az). By reading the robot positions, individual points can be inserted as positioning or welding movements. The welding speed and the synergy parameters of the welding power source are predefined. A check box is available to activate/deactivate the adaptive process control strategies. Depending on the selection, the taught positions and/or welding parameters are transferred to the process control loop at the start of the process.

The real-time controller is responsible for the control and communication between the individual components. A control loop is implemented for each component of the welding production system. Communication takes place exclusively within the individual loops. The connection to the process monitoring unit is established in the camera control loop. The process start/stop signals are sent. Image processing results are received and sent to the process control loop. The robot control loop communicates with the robot controller. The state of the robot controller is constantly monitored. During the path, position information and any changes in welding speed are sent to the robot controller. The loop monitors the state of the buffer and informs the process control loop when more position points are required. The welding power source loop communicates with the welding power source via the FPGA’s analog/digital inputs/outputs. The process control loop handles all incoming and outgoing information. The process control loop implements the process control sequencing and adaptive strategies. The pre-programmed path is broken down into 1 mm increments and made available to the robot control loop piece by piece. This makes it possible to adapt the positions that have not yet been executed.

The FPGA is divided into three tasks. The first task is to acquire the welding parameters (welding current, welding voltage), which are stored for documentation purposes. The second task is to process the measurements. This generates the trigger signal that is sent to the camera. It also controls the analog and digital inputs/outputs.

4 Verification of functionality

Welding tests are performed to verify the functionality of the adaptive welding system. First, the implemented functions are tested using simulated controller inputs. Two use cases are then considered to verify the adaptive process control. In the first use case, the adaptivity of the wire feed speed is considered. In the second use case, a seam tracking task with adaptation of the position data is tested. The test setup for carrying out the welding tests is shown in Fig. 6.

Fig. 6
figure 6

Experimental set-up for welding tests

Full size image

The camera is mounted on the welding torch and directed towards the front of the weld pool. The base material used is mild steel S235JR. The filler metal is EN ISO 14341-A G 3Si1 with a diameter of 1.2 mm. The shielding gas used is EN ISO 14175 M21-ArC-18 (15 l/min).

4.1 Simulated controller inputs

To verify the adaptivity of the individual process parameters, an interface for the simulation of controller inputs is implemented. The parameter changes are entered manually and communicated to the respective actuators. The following process parameters are considered:

  • Robot path (x, y, z coordinates)

  • Welding torch orientation

  • Welding speed

  • Synergy parameters of the welding power source

The welding tests result in a correct execution of the simulated controller inputs. For adaptive welding, the simulated controller inputs can be replaced by a process adaptation strategy, for which the following use cases are considered.

4.2 Use case one

In application one, a build-up weld is performed in the welding position PA. For the adaptive test, changes in weld pool size are provoked by a continuous, pre-programmed adjustment of the welding speed. The weld production system must be able to detect the changes in weld pool width from the weld pool records and adjust the wire feed speed accordingly. The aim is to achieve a constant seam width over the entire length of the weld. Two welds are carried out to visualise the influence of the process control. In the first weld, the controller is deactivated. This means that the wire feed speed is constant. In the second weld, the process control is activated, and the wire feed speed is adjusted according to the controller output of the system.

4.2.1 Adaption strategies

For the adaptive process control, a closed-loop control is used. The control loop is illustrated in Fig. 7.

Fig. 7
figure 7

Closed-loop control for weld pool width control

Full size image

The setpoint of the closed loop is the weld pool width, which is set to 400 pixels (10.7 mm). The difference between the setpoint and the actual measured weld pool width (xr) is the input to the controller (e). A PI controller is used. The proportional gain (Kc) is set to 0.01. The integral time (Ti) is set to 0.01. The controller output is set to a range of 3 to 10. This corresponds to the range of the relevant wire speed, which is the output of the controller (y). The wire feed speed is transmitted as an analog voltage to the welding power source and thus influences the welding process. The welding speed is adjusted stepwise as a disturbance variable (z). The control loop time is 40 ms (25 Hz).

4.2.2 Results

The curves for welding speed and wire feeding speed of the controlled weld are shown in Fig. 8.

Fig. 8
figure 8

Controlled wire feeding speed throughout the weld

Full size image

The welding speed is increased. This increases continuously as the welding speed increases.

Figure 9 shows the measured weld pool width over the weld length.

Fig. 9
figure 9

Comparison of the weld pool width with and without process control

Full size image

The weld pool width without control decreases continuously from 10.8 to 7.5 mm. The weld pool width with active control remains constant in a range of 10.6 mm ± 0.4 mm.

The weld result and snapshots of the weld pool images are shown in Fig. 10.

Fig. 10
figure 10

Welding results with and without control

Full size image

The results described above can also be seen in the weld pool images. The weld geometry of the uncontrolled weld becomes progressively narrower, whereas the weld width of the controlled weld is uniform.

The results of use case one show that wire feed control works under the condition of a constant weld pool width. The control was able to achieve a constant weld pool width despite variations in welding speed.

4.3 Use case two

In Use Case two, a seam tracking system is implemented. The weld production system needs to detect and compensate for deviations in seam position and weld pool position. To achieve this, a path is programmed, and the pattern is then moved. The deviations between the programmed path and the actual weld path are shown in Fig. 11.

Fig. 11
figure 11

Approach for seam tracking application testing

Full size image

The weld length is 150 mm. After moving the weld sample, the difference between the programmed end point and the actual end point is 11 mm.

4.3.1 Adaption strategies

For the adaptive control of the welding path a closed loop control for correcting the positions, which are sent to the robot is used. The control loop is illustrated in Fig. 12.

Fig. 12
figure 12

Closed control loop for welding path control

Full size image

A P element is used as the controller. The gain K is set to 0.33. The input to the controller (e) is the difference between the measured offset (xr) from the weld pool observation and the predetermined offset (w). The output of the controller determines the correction factor of the path (y). This correction factor is interpreted by the process control unit and transmitted to the robot controller. The disturbance is caused by the misalignment of the plates. The control loop time is 160 ms. Due to the latencies between the detection of the position error (see Section 3.3) and the adaptation of the robot path, a higher adaptation frequency is not applicable.

4.3.2 Results

The welding result for path tracking is shown in Fig. 13.

Fig. 13
figure 13

Result of welding test and curve of the compensated offset

Full size image

At the beginning of the process, the position deviation is compensated. From about the seventh second, the position correction decreases until the tenth second. This reduction can also be seen in the seam pattern. From second 10, the offset continuously increases until a total deviation of 11 mm is compensated at the end of the process. Figure 14 shows excerpts from the weld pool images.

Fig. 14
figure 14

Excerpts from the weld pool images throughout the welding process

Full size image

At time 1 (~ 6 s), there is a deviation of the joint position from the torch position. At time 2 (~ 15 s), this deviation has been compensated, and at the end of the weld (~ 25 s), the torch position is also centred on the joint.

The results of use case two show the functionality of the seam tracking system. Deviations of the seam from the torch position could be detected, and appropriate compensation of the pre-programmed paths was performed. There was a brief misalignment at the beginning of the weld. These images may indicate over-control. It can be assumed that this over-regulation can be compensated by further optimisation of the control parameters.

5 Conclusion

A concept for a model adaptive welding system has been presented. In order to show the applicability of such a system during a real welding process, validation experiments for two use cases were made on a laboratory system implementation. The components of the system work autonomously, which means that the system can be flexibly extended with additional components. The welding production system is able to control mechanical (weld path, welding speed, torch orientation) and electrical (synergic parameter settings) process parameters on the basis of weld pool images. The functionality of the system has been verified in two use cases. It was shown that the weld pool width could be controlled based on a setpoint. This was done using a wire feed speed control. A seam tracking system was also implemented to compensate for deviations between the programmed weld path and the actual seam position. The system can be expanded to include other adjustment strategies. The modular design of the system also allows for multi-parameter control. The weld production system has demonstrated the suitability of camera technology for adaptive process control in several applications.