An Algorithm for Painting Large Objects Based on a Nine-Axis UR5 Robotic Manipulator

Featured Application

The proposed algorithm for painting large objects based on a nine-axis UR5 robotic manipulator can be applicable in many automobile repair shops where paint jobs can be performed. With the help of a nine-axis UR5 robotic manipulator with the proposed algorithm, vehicles can be automatically painted with the least amount of human manual labor. Simultaneously, the quality and efficiency of the paint jobs can be drastically improved, since the UR5 robot maintains its consistency, accuracy, and proficiency while conducting paint jobs.

Abstract

An algorithm for automatically planning trajectories designed for painting large objects is proposed in this paper to eliminate the difficulty of painting large objects and ensure their surface quality. The algorithm was divided into three phases, comprising the target point acquisition phase, the trajectory planning phase, and the UR5 robot inverse solution acquisition phase. In the target point acquisition phase, the standard triangle language (STL) file, algorithm of principal component analyses (PCA), and k-dimensional tree (k-d tree) were employed to obtain the point cloud model of the car roof to be painted. Simultaneously, the point cloud data were compressed as per the requirements of the painting process. In the trajectory planning phase, combined with the maximum operating space of the UR5 robot, the painting trajectory of the target points was converted into multiple traveling salesman problem (TSP) models, and each TSP model was created with a genetic algorithm (GA). In the last phase, in conformity with the singularities of the UR5 robot’s motion space, the painting trajectory was divided into a recommended area trajectory and a non-recommended area trajectory and created by the analytical method and sequential quadratic programming (SQP). Finally, the proposed algorithm for painting large objects was deployed in a simulation experiment. Simulation results showed that the accuracy of the algorithm could meet the requirements of painting technology, and it has promising engineering practicability.

1. Introduction

In recent years, UR5 robots have been widely popularized in industrial production fields such as painting, assembly, and micromanipulation [1]. In the above-mentioned fields, the painting process is the integral manufacturing procedure for automatically coating large objects, and it is one of the essential technologies for improving the surface quality of painted objects, which can then offer better performances under different working conditions [2,3,4]. Painting feasibility and trajectory planning are critical in the painting field.

To paint large objects, some engineers tried to mount UR5 robots on a mobile platform to enlarge the workspace of the UR5 robots [5,6], while other engineers tried to widen the workspace by changing the structure of the UR5 robots [7]. However, the method of changing the structure of the UR5 robots entails enormous cost, and the UR5 robot with changed structure offers low adaptability. Thus, this paper proposes a UR5 robot which is mounted on an zyz three-axis motion platform to achieve the painting of large objects.

A well-designed trajectory can improve painting efficiency. Bureerat, S. et al. employed the MRPEIL-DE algorithm to optimize the trajectory of a six degree-of-freedom (DOF) UR5 robot [8]. Yin, S. et al. utilized mechanical learning methods to devise an energy-saving trajectory for industrial UR5 robots [9]. Serralheiro, W. et al. proposed employing a non-based trajectory planning method for time-energy optimization of a completely wheeled mobile UR5 robot [10]. Kazim, I.J. et al. compared improving the artificial potential field (APF) by the traditional particle swarm optimization (PSO) algorithm and the serendipity-based PSO (SBPSO) algorithm to control the path of a universal robot UR5 with collision avoidance [11]. Kazim, I.J. et al. utilized differential evolution (DE) optimization with the MATLAB toolbox, which has an applied robot operating system (ROS), to quantify the contour tracking performance of a collaborative universal manipulator robot (UR5) [12]. Al-Shanoon, A. et al. proposed a novel reliable framework for deep ConvNet combined with visual servoing using a single RGB camera [13]. Balanji, H.M. et al. proposed a novel calibration framework based on a single camera and computer vision techniques using ArUco markers [14]. Vivas, A. et al. designed the implementation of the control of a real UR5 robot from Matlab/Simulink using ROS [15]. Araki, R. et al. proposed a 6D pose estimation method for an object from a single RGB image for a robotic grasping task [16].

Nonetheless, most of the above-mentioned algorithms neither consider the singularities of the kinematics of the UR5 robot’s joints nor meet the requirements of the painting process. Therefore, this paper proposes a painting algorithm which can generate a painting trajectory satisfying the painting of large objects. The painting algorithm employs the standard triangle language (STL) file, algorithm of principal component analyses (PCA), and k-dimensional tree (k-d tree) to create a digital model of the object to be painted. The digital model is converted into multiple traveling salesman problem (TSP) models, and each TSP model is created with a genetic algorithm (GA). The painting trajectory offers high precision and efficiency.

2. Construction of a Point Cloud Model of the Target Object

STL files are popular in computer graphics application systems, and their file formats are simple and widely harnessed in machine vision and 3D reconstruction [17,18].

2.1. STL File Parsing

A STL file consists of multiple triangles and can be divided into the American Standard Code for Information Interchange (ASCII) format and binary format as per the storage format. The STL file obtained in this paper was suitable for ASCII. The analysis of a STL file in the ASCII format is shown in Figure 1.

In Figure 1, η_i, η_j, and η_k represent the x, y, and z components of the triangle patch normal vector, respectively; v_ix, v_iy, and v_iz represent the x, y, and z coordinates of the i-th triangular patch, respectively.

2.2. Filling Triangles

The STL file only contains the coordinates of the vertices of the triangles, and the coordinates of all points in the model cannot be obtained. To obtain the full 3-dimensional information of the model, the triangles must be filled. This paper utilized the depth-first search (DFS) to fill the triangles. The triangle is represented by S. The center of the triangle is represented by o. Lengths of the 3 sides of the triangle is represented by l₁, l₂, and l₃. The triangle S is later separated into 3 smaller triangles S₁, S₂, and S₃. The algorithm model and the main flowchart of the filling process are shown in Figure 2 and Figure 3.

The stopping condition of DFS recursion is that the minimum height corresponding to the three sides of the triangle is less than the specified minimum height. This requirement is shown in Equation (1), where the minimum triangle height h_min is given.

$$S>max\left(l_1,l_2,l_3\right)\cdot h_{\min }/2$$

(1)

The area of the triangle in Equation (1) is difficult to directly calculate. Therefore, Heron’s formula, which is shown in Equation (2), is introduced, and the area is calculated from the lengths of the sides. The recursive stop condition is shown in Equation (3).

$$\begin{array}{l}S=\sqrt{\epsilon \left(\epsilon -l_1\right)\left(\epsilon -l_2\right)\left(\epsilon -l_3\right)}\\ \epsilon =\left(l_1+l_2+l_3\right)/2\end{array}$$

(2)

$$S\ge 2\cdot \max \left(l_1,l_2,l_3\right)\cdot h_{min}$$

(3)

The initial vertices are randomly selected as (10, 10, 0), (520, 100, 0), and (260, 410, 0), and h_min equals 0.05. The filling results are shown in Figure 4a.

However, there are a large number of unfilled lines in Figure 4a, and the filling effect is not satisfactory. After the filling process had been analyzed, it was found that the point filling method was harnessed in the filling, which could result in the points on the straight lines from the center of the triangle to the vertices of the triangle not being filled. If the number of filling points needs to be increased, the coordinates of the recorded point during the filling process can be changed into the coordinates of each point above the line connecting the recorded point and the vertex of the triangle with the same initial value condition and iteration termination condition. The final filling result is shown in Figure 4b. Compared to Figure 4a, Figure 4b is better filled, but the time taken to fill Figure 4b was much longer than to fill Figure 4a. If the accuracy is not high or the time is short, point filling is recommended. For high-precision conditions, linear filling is recommended.

2.3. Determining the Normal Vector of a Target Point on the Painting Trajectory

In order to ensure that the painting area is uniform during the painting process, it is necessary to ensure that the axis of the end of the spray gun coincides with the normal vector of the target point on the painting trajectory. Thus, the normal vectors of the target model must be obtained. As is shown in Figure 3, the target points on the painting trajectory can be divided into two types, which include the trajectory points obtained through the filling algorithm and the vertices of the original triangle.

The points obtained by filling are still coplanar with the original triangles. The normal vectors of the points obtained by filling can be determined with the normal vector of the plane. The normal vectors of the triangles can be directly extracted from the STL file. However, for the normal vectors of the vertices of the original triangle, there is a common vertex of multiple triangles. Ergo, the normal phase of the plane cannot be employed instead of the normal phase of the points. In this paper, local plane fitting and principle component analysis (PCA) estimation methods were harnessed to create the normal vectors of the target points [19,20,21]. In order to acquire the target point field conveniently, the k-d tree algorithm was harnessed to store the coordinates of the target points [22]. The step-by-step algorithm is shown in Figure 5 and Figure 6.

In the actual processing procedure, the normal vector of the target model should take its external normal phase, and the external normal vector of the model can be determined by Equation (4), where $\overline{p}$ is the center of all target points, and $\left|\right|x|{|}_2$ is the second norm of the vector x.

$$n=\{\begin{array}{l}n|{\Vert (p+n)-\overline{p}\Vert }_2\ge {\Vert (p-n)-\overline{p}\Vert }_2\\ -n|{\Vert (p+n)-\overline{p}\Vert }_2<{\Vert (p-n)-\overline{p}\Vert }_2\end{array}$$

(4)

2.4. Determining the Pose of a Target Point on the Trajectory

In the actual painting process, the pose matrix is often deployed to describe the rotation of the target point in space. During the inverse kinematics calculation of the UR5 robot, the pose of the target point needs to be described by the matrix, and Rodrigues’ rotation formula can determine the pose of the target point by the axis direction of the target point.

The diagram of Rodrigues’ rotation formula is shown in Figure 7, where X₁, Y₁, and Z₁ represent the three axes, with O being the origin of the 3-dimensional Cartesian system after rotating.

The rotation matrix R is shown in Equation (5),

$$E\cos \theta +(1-\cos \theta )r^T\times r+\sin \theta \times \left[\begin{array}{ccc}0& -r_z& r_y\\ r_z& 0& -r_x\\ -r_y& r_x& 0\end{array}\right]$$

(5)

where E represents the third-order identity matrix, and θ represents the angle created by vectors n and n₀. In Equation (5), r is the vector product of vectors n and n₀; n₀ = [0 0 1]^T, and n is the normal vector of the point.

2.5. Compression of the Data of Target Points

The design of the painting process parameters ensures that the end of the spray gun will produce a circle with a radius of r at a specified distance. During the painting process, it is necessary to ensure that the trajectory coverage during painting is 0.4. The painting model is shown in Figure 8, and the coverage of Figure 8, denoted by p, is shown in Equation (6).

$$p=\frac{4r^2\cdot \arccos \left(d/2r\right)-d\sqrt{r^2-d^2/4}}{2\mathsf{\pi }{r}^2}$$

(6)

When the painting trajectory is executed, not all target points are to be painted. Only the path of key points should be painted. The distance between key points should meet the requirements of painting uniformity. In Figure 8 and Equation (6) d is employed to reduce the time and space complexity of the trajectory planning. The compression formula for the data points is shown in Equation (7).

$$\begin{array}{l}k=floor\left(d/\left(\sqrt{3}\right)\right)\\ P_x=round\left(P_x/k\right)\cdot k\\ P_y=round\left(P_y/k\right)\cdot k\\ p_z=round\left(p_z/k\right)\cdot k\end{array}$$

(7)

where P_x, P_y, and P_z represent the x-y-z coordinates of the target points in the point cloud, floor represents the floor function, which takes as input a real number q, and gives out as output the greatest integer less than or equal to q, and round represents the round function, which rounds off a numeric value to its nearest integer.

3. Nine-Axis UR5 Robot Forward Kinematics Model

The space that the UR5 robot covers is limited, and large-size objects may not be painted at once. When an area that needs to be painted is outside the working space of the UR5 robot, the D-H parameters of the UR5 robot must be changed, or auxiliary equipment must be added to help with the painting. The former is costly. Changing joints is not conducive to the popularization of UR5 robots. The proposed 6-DOF UR5 robot is installed on a 3-axis motion platform that moves horizontally, laterally, and vertically to assist painting. The 3D drawing of the UR5 robot is shown in Figure 9. The 9-axis UR5 robotic manipulator includes the 3-axis motion platform that has 3 prismatic pairs and the UR5 robot that has 6 revolute pairs. The 6-DOF UR5 robot selected in this paper is a UR5 robot, which is shown in Figure 10.

The pose of the actual UR5 robot end T is shown in Equation (8).

$$\begin{array}{l}T=Trans(t_x,t_y,t_z)Rot(x,\pi )T_{\mathrm{robot}}{T}_{\mathrm{tool}}{T}_{\mathrm{dis}}\\ Trans(t_x,t_y,t_z)=\left[\begin{array}{l}1\hspace{1em}0\hspace{1em}0\hspace{1em}{t}_x\\ 0\hspace{1em}1\hspace{1em}0\hspace{1em}{t}_y\\ 0\hspace{1em}0\hspace{1em}1\hspace{1em}{t}_s\\ 0\hspace{1em}0\hspace{1em}0\hspace{1em}1\end{array}\right]\\ Rot(x,\pi )=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& \cos \pi & \sin \pi & 0\\ 0& -\sin \pi & \cos \pi & 0\\ 0& 0& 0& 1\end{array}\right]\end{array}$$

(8)

in which t_x, t_y, and t_z represent the movement of the motion platform along the x, y, and z axes, respectively, and Trans represents the movement operator. Rot represents the rotation matrix of the x-axis. T_tool, which is shown in Figure 11, represents the pose of the end-effector relative to its installation center. T_dis, which is also shown in Figure 11, represents the end fixture of the UR5 robot’s position. T_tool is determined by the structure of the end-effector, and T_dis is determined by the parameters of the end fixture of the UR5 robot’s position. The UR5 robot system has 9 axes which are described in Figure 11. Axes 1 to 6 are the 6 joints of the UR5 robot. Axes 7 to 9 are the three-dimensional Cartesian coordinate system of the motion platform.

Figure 12a is the structural diagram of the 6-DOF UR5 robot composed of 6 revolute pairs, represented by 1, 2, 3, 4, 5, and 6 and 0 represents the end fixture of the UR5 robot. In order to analyze the pose change of the UR5 robot’s end fixture coordinates relative to the end-effector, the D-H model is harnessed to establish the kinematics forward solution model of the UR5 robot.

The UR5 robot’s pose change matrix in the D-H model is shown in Equation (9).

$$\begin{array}{l}{}_i{}^{i-1}T=Rot(z,{\theta }_i)Trans(0,0,d_i)Trans(a_i,0,0)Rot(x,{\alpha }_i)\\ Rot(z,{\theta }_i)=\left[\begin{array}{cccc}\cos {\theta }_i& \sin {\theta }_i& 0& 0\\ -\sin {\theta }_i& \cos {\theta }_i& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\end{array}$$

(9)

In Equation (9), θ_i is the angle for which x_i−1 spins around z_i to become x_i; d_i is the distance between x_i−1 and x_i along z_i; a_i is the distance between z_i and z_i+1 along x_i, and α_i is the angle for which z_i spins around x_i to become z_i+1. Rot(z, θ_i) represents the rotation matrix of the z-axis; θ_i, d_i, α_i, and a_i are shown in Figure 12b. ${}_i{}^{i-1}T$ is shown in Equation (10), where s represents the sine function and c represents the cosine function.

$${}_i{}^{i-1}T=\left[\begin{array}{cccc}\mathrm{c}{\theta }_i& -\mathrm{s}{\theta }_i\mathrm{c}{\alpha }_i& \mathrm{s}{\theta }_i\mathrm{s}\alpha & a_i\mathrm{c}{\theta }_i\\ \mathrm{s}{\theta }_i& \mathrm{c}{\theta }_i\mathrm{c}{\alpha }_i& -\mathrm{c}{\theta }_i\mathrm{s}\alpha & a_i\mathrm{s}{\theta }_i\\ 0& \mathrm{s}{\alpha }_i& \mathrm{c}{\alpha }_i& d_i\\ 0& 0& 0& 1\end{array}\right]$$

(10)

T_robot, which represents the pose of the UR5 robot, is obtained via Equation (11). The UR5 robot’s pose should satisfy Equation (12), and the D-H parameters of the UR5 robot are shown in

Table 1. The D-H parameters of the UR5 robot.

i	θi	di/mm	αi	αi/mm
1	−2π to 2π	89.2	π/2	0
2	−2π to 2π	0	0	425
3	−2π to 2π	0	0	392.3
4	−2π to 2π	109.2	π/2	0
5	−2π to 2π	94.7	−π/2	0
6	−2π to 2π	82.3	0	0

.

$$T_{\mathrm{robot}}={\displaystyle \prod _{i=1}^6{}_i{}^{i-1}T}$$

(11)

$$\begin{array}{l}Trans{(x,y,z)}^{-1}\cdot Rot(x,\pi {)}^{-1}\cdot T\cdot T_{tool}{}^{-1}={\displaystyle \prod _{i=1}^6{}_i{}^{i-1}{\rm T}}\\ \end{array}$$

(12)

4. The Inverse Solution Model of Kinematics of the 9-Axis UR5 Robot

Nevertheless, since the system that controls the UR5 robot and the system that controls the motion platform are separated, communication between the systems may cause problems, such as delays and errors. In order to improve painting accuracy, the motion platform cannot be moved frequently. During the painting process, each time the motion platform is moved, the UR5 robot paints an area. After the painting is completed, the motion platform is moved again. The recommended operating space of the UR5 robot is shown in Figure 13.

The recommended working area in Figure 13 represents the feasible area recommended by the UR5 robot. The area between the maximum working area, which is represented by Max. working area in Figure 13, and the recommended feasible area represents the non-recommended feasible area that the UR5 robot may not reach. The non-recommended area represents the singularity of the UR5 robot. The non-recommended area takes the largest cylinder contained in the green area in Figure 13 as the largest area for each painting procedure.

However, the recommended area in Figure 13 is not a complete ball, and the cylindrical area with a diameter of 151 mm is not recommended. If this area is removed, the volume of the largest cylinder obtained is 0.25 times that without removal. Therefore, the platform will be moved slightly. In order to move the platform as little as possible, the green area is assumed to be a complete ball when the maximum cylinder is calculated.

In order to prevent the UR5 robot from colliding with the platform, it is necessary to keep the end of the UR5 robot below its installation center during the movement of the UR5 robot. The area under the side wall of the machine is taken as the effective area. The volume of the cylinder included in Figure 13 is shown in Equation (13).

$$V=\pi {\left(rcos\theta \right)}^2\cdot \left(163-rsin\theta \right)$$

(13)

where r, θ, and V represent the maximal length of the UR5 robot, the radian of the intersection of the cylinder, and the maximum machining space of the UR5 robot, respectively. When θ equals $-$0.54 rad, the volume of the cylinder is the largest. At this time, the cylinder has a radius of 727 mm and a height of 602 mm. Due to the existence of the non-recommended area, the UR5 robot inverse kinematics model is divided into a recommended area inverse kinematics model and a non-recommended area inverse kinematics model.

4.1. Inverse Solution Model for Recommended Regions

The three joint axes of the UR5 robot, joint axes 2, 3 and 4, which are shown in Figure 14, are parallel, and their spatial structure satisfies the Pieper criterion. The closed-form solution method can be harnessed to solve the angular sequence of the UR5 robot joint under the target pose constraints.

Equation (14) determines the pose of the end-effector of the UR5 robot.

$${\displaystyle \coprod _{i=1}^6{}_i{}^{i-1}T}={(Trans(t_x,t_y,t_z)Rot(x,\pi ))}^{-1}\cdot T\cdot {(T_{tool}\cdot T_{dis})}^{-1}=\left[\begin{array}{cccc}{n}_x& o_x& a_x& x^{’}\\ n_y& o_y& a_y& y^{’}\\ n_z& o_z& a_z& z^{’}\\ 0& 0& 0& 1\end{array}\right]$$

(14)

The main idea of the closed-form solution is creating the constraint equation established by matrix changes. The constraint matrix of the UR5 robot, which is shown in Equation (15), is established in accordance with Equation (11). The third row of the left and right sides of Equation (15) is expanded to establish Equation (16). Equation (16) can be regarded as the equation set of θ₁, θ₅, and θ₆.

The procedure for finding the values of θ₂, θ₃, and θ₄ is similar.

$$L=({}_1{}^{0}T^{-1})T_{\mathrm{robot}}={\displaystyle \prod _{i=2}^6{}_i{}^{i-1}T}=R$$

(15)

$${\left[\begin{array}{c}\mathrm{c}{\theta }_6\mathrm{s}{\theta }_5\\ \mathrm{s}{\theta }_6\mathrm{s}{\theta }_5\\ \mathrm{c}{\theta }_5\\ d_2+d_3+d_4+d_6\mathrm{c}{\theta }_5+a_5\mathrm{s}{\theta }_5+a_6\mathrm{c}{\theta }_6\mathrm{s}{\theta }_5\end{array}\right]}^T={\left[\begin{array}{c}{n}_y\mathrm{c}{\theta }_1+n_x\mathrm{s}{\theta }_1\\ o_y\mathrm{c}{\theta }_1+o_x\mathrm{s}{\theta }_1\\ a_y\mathrm{c}{\theta }_1+a_x\mathrm{s}{\theta }_1\\ y\mathrm{c}{\theta }_1+x\mathrm{s}{\theta }_1\end{array}\right]}^T$$

(16)

By employing Equations (15) and (17), the result can be obtained in Equation (18),

$$L=({}_1{}^0{\rm T}^{-1}){}_6{}^{0}T({}_5{}^{4}T^{-1})({}_6{}^{5}T^{-1})={\displaystyle \prod _{i=2}^4{}_i{}^{i-1}{\rm T}}=R$$

(17)

$$R=\left[\begin{array}{cccc}\mathrm{c}{\theta }_{234}& 0& -\mathrm{s}{\theta }_{234}& a_3\cdot \mathrm{c}{\theta }_{23}+a_3\cdot \mathrm{c}{\theta }_2+a_4\cdot \mathrm{c}{\theta }_{234}\\ -\mathrm{s}{\theta }_{234}& 0& \mathrm{c}{\theta }_{234}& -a_3\cdot \mathrm{s}{\theta }_{23}-a_3\cdot \mathrm{c}{\theta }_2-a_4\cdot \mathrm{c}{\theta }_{234}\\ 0& -1& 0& d_2+d_3+d_4\\ 0& 0& 0& 1\end{array}\right]$$

(18)

where ${\theta }_{i_1{i}_2\cdots i_n}$ is defined in Equation (19).

$${\theta }_{i_1{i}_2\cdots i_n}={\displaystyle \sum _{j=1}^n{\theta }_{ij}}$$

(19)

In Equation (18), θ₂₃₄, θ₂₃, and θ₂ can be acquired by the fourth column of R in Equation (17); θ₂, θ₃, and θ₄ can be obtained afterwards. In Equations (16) and (18), s represents the sine function and c represents the cosine function. Nonetheless, the solution involves a large number of inverse trigonometric functions, and the range of the angles within which the UR5 robot’s joints move is −2π to 2π, which results in multiple solutions. Therefore, the solution with the smallest Euler distance from the initial joint angle sequence is selected.

4.2. The Inverse Solution Model for Non-Recommended Regions

In a non-recommended area, the UR5 robot may not be able to achieve certain poses. It is necessary to move the motion platform to match the position of the UR5 robot. Then, the inverse solution model of the manipulator can be changed to obtain the minimum joint rotation radian and the number of movements of the manipulator’s motion platform under posture and pose constraints. This problem is a nonlinear optimization problem with constraints. Sequential quadratic programming (SQP) is an iterative method for constrained nonlinear optimization. When the input value is close to the real solution, the algorithm has a second-order convergence speed and can quickly solve the target solution [23,24].

4.2.1. The Objective Function and Constraints

In the process of solving the inverse kinematics, in order to avoid singularities, the platform is allowed to move slightly. The number of movements of the platform and the total arc of the UR5 robot joint give the objective function, which is shown in Equation (20).

$$f=\sqrt{{\displaystyle \sum _{i=1}^{i=6}{\left({\theta }_i{}^r-{\theta }_i\right)}^2}+K_d\left({\left(x^r-t_x\right)}^2+{\left(y^r-t_y\right)}^2+{\left(z^r-t_z\right)}^2\right)}$$

(20)

where θ_i^r and θ_i are the actual joint curvature and the starting point arc of the UR5 robot. The target point K_d represents the number of movements of the platform; x^r, y^r, and z^r represent the actual moving distance.

The error es is allowed in the actual working situation. The constraint condition should be that the Euler distance between T and the target pose T’ is less than es, which is shown in Equation (21).

$$\sqrt{{\displaystyle \sum _{j=1}^{j=3}{\left({}_6{}^{0}T{}_{j4}-{}_6{}^{0}T_{j4}^{’}\right)}^2}}-es\le 0$$

(21)

During the painting process, an error es’ between the end axis of the spray gun and the target point axis is allowed. The pose constraints are shown in Equation (22).

$$\left|\arccos \left(\frac{\mathbf{n}\cdot {\mathbf{n}}_0}{\left|\mathbf{n}\right|\cdot \left|{\mathbf{n}}_0\right|}\right)\right|<e{s}^{’}$$

(22)

where n is the normal direction of T to the z axis, and n₀ is the normal direction of T’ to the z axis.

4.2.2. The Solution Process of the SQP Algorithm

The SQP algorithm decomposes the problem into the quadratic programming (QP) sub-problem, obtaining the descending direction d by solving the current angle sequence θ_k in the QP problem and updating θ_k+1 via d until the Karush–Kuhn–Tucker (KKT) condition is satisfied or the maximum number of iterations is reached. The specific solving process of the algorithm is described below.

(1) The Lagrange function L(θ_k, λ) of the current angular sequence θ_k is constructed, and the expression of L(θ_k, λ) is shown in Equation (23).

$$L({\theta }_k,\lambda )=f\left({\theta }_k\right)+\lambda g\left({\theta }_k\right)$$

(23)

where λ is the Lagrangian multiplier, and λ ≥ 0.

(2) The descending direction d of the current angular sequence θ_k is obtained. Equation (23) is converted into the solution to d, which is shown in Equation (24).

$$\begin{array}{l}{\min }_d\hspace{1em}\nabla f{\left({\theta }_k\right)}^{T}d+\frac{d^T{H}_{k}d}{2}\\ st:\hspace{1em}\hspace{1em}g\left({\theta }_k\right)+\nabla g{\left({\theta }_k\right)}^{T}d\le 0\end{array}$$

(24)

In Equation (24), H_k represents the Hassel matrix of the Lagrangian function L(θ_k, λ).

(3) The angle sequence in which θ_k+1 equals θ_k plus d is updated, and whether the result of the solution satisfies the KKT condition or reaches the maximum number of iterations is determined. If the KKT condition is satisfied, the iteration is stopped. Step 2 is repeated if the KKT condition is not satisfied. The KKT condition is shown in Equation (25) [25].

$$\begin{array}{l}\nabla f\left({\theta }_k\right)+\lambda g\left({\theta }_k\right)=0\\ \lambda \ge 0\\ g\left({\theta }_k\right)=0\\ \lambda g\left({\theta }_k\right)=0\end{array}$$

(25)

In the KKT condition, the third condition means that the result of the solution satisfies the constraint. If g(θ_k) equals 0, the KKT condition becomes ▽f(θ_k), which equals 0. If g(θ_k) is smaller than 0 and λ equals 0, the KKT condition also becomes ▽f(θ_k), which equals 0.

5. The Painting Trajectory

During the painting process, not only should the trajectory of the motion platform be planned, but the trajectory of the UR5 robot’s movement should also be planned. During the painting process, the posture of the UR5 robot end is shown in Equation (26).

$$Trans(x,y,z)\cdot Rot(x,p)\cdot T_{robot}=T\cdot T_{\mathrm{dis}}{}^{-1}\cdot T_{\mathrm{tool}}{}^{-1}$$

(26)

Since Rot(x, π) only affects the directions of the x-axis and z-axis of the 3-dimensional Cartesian system of the UR5 robot system, it does not change the range of the UR5 robot’s operating space. Equation (26) can be transformed into Equation (27).

$$Trans(x,y,z)\cdot T_{\mathrm{robot}}{}^{’}={\rm T}\cdot T_{\mathrm{dis}}{}^{-1}\cdot T_{\mathrm{tool}}{}^{-1}$$

(27)

where T’_{UR5 robot} equals Rot(x,π)∙T_{UR5 robot}.

5.1. The Trajectory of the Motion Platform

When the size of the object to be painted exceeds the maximum painting area of the UR5 robot, the auxiliary movement of the motion platform is required to complete the painting. During the painting process, the motion platform is moved to bring the UR5 robot close to the target area, and the UR5 robot paints the area. After painting, the motion platform is moved again, and the UR5 robot paints another area. Then, the trajectory planning of the motion platform is optimized by taking the least number of movements of the motion platform as the objective function.

Because the UR5 robot’s permitted painting space is a cylinder and the height of the painted object is generally less than the maximum height of the painting space, the model can be projected onto the xy plane. The model can be transformed to cover all target points with the fewest circles.

5.1.1. The Minimum Envelope Rectangle of the Target Points

Since the number of target points is large, in order to simplify the calculation, a rectangle enveloping the target points—instead of a set of target points—is harnessed. The solution model, which is shown in Figure 15, is created by employing the minimum rectangle method of the main direction target points. The dotted section in the figure represents the target points; the blue line represents the main direction of the target points, and the red rectangle represents the minimum envelope rectangle of the target points.

The method for obtaining the principal direction of the plane is shown in Equation (28).

$$\theta =\frac{atan2\left(2M_{11},M_{20}-M_{02}\right)}{2}$$

(28)

in which θ is the minimum angle between the major axis direction and the direction of the positive side of the x-axis, and M_pq is the p + q order center moment of the projection surface.

M_pq is shown in Equation (29).

$$M_{pq}={\displaystyle \sum _x{\displaystyle \sum _y{\left(x-\overline{x}\right)}^p}}{\left(y-\overline{y}\right)}^q$$

(29)

where $\overline{x}$ and $\overline{y}$ represent the center coordinates of the original target point.

5.1.2. The Minimum Number of Movements of the Motion Platform

When a circle fills a rectangle, the effective filling area of each circle is generally rectangular. Then, the model can be simplified again by filling the target rectangle with the smallest number of rectangles inside the circle. The filling model is generally shown in Figure 16. The smaller rectangle in the figure represents the target rectangle that needs to be filled. The larger rectangle represents the largest filled area, and the circular areas outside the largest rectangle represent the areas not recommended for use by the UR5 robot. Each circle represents the maximum processing range of the system each time. P_x and P_y represent the vertical distance and horizontal distance between the target rectangle and the largest filled area; l and h represent the length and width of the effective rectangle inside each circle.

The size of the rectangle inside the circle indicates the effective area size of the circle. The length and width of the maximum inscribed rectangle in the circle are both $\sqrt{2}$r. The length and width of the inscribed rectangle of the circle are generally close to $\sqrt{2}$r when the target rectangle is selected. The constraints of the filled model are shown in Equation (30).

$$\begin{array}{l}n=\left[l_{\max }/\sqrt{2}r\right]+1\\ {\displaystyle \sum _{i=1}^n{l}_n\ge l_{\max }}\\ \sqrt{4r^2-l_i^2}\ge h_{\max }\end{array}$$

(30)

where [l_max/$\sqrt{2}$r] represents the largest integer that does not exceed $\frac{l_{\max }}{\sqrt{2}r}$.

Since the number of circles in Figure 16 is an integer, there are many kinds of P_x, P_y, h₁, and h₂ satisfying Equation (29). However, the UR5 robot’s non-recommended area may cause the movement of the motion platform. The minimum number of target points in the non-recommended area during processing is the optimization goal, and the optimal P_x, P_y, h₁, and h₂ are obtained by employing the exhaustive method.

5.1.3. The Movement Sequence of the Motion Platform

Reasonably planning the movement sequence of the motion platform can reduce the movement of the motion platform. During processing, the initial point of the motion platform is at zero. The movement sequence planning model of the motion platform can be transformed into a TSP model starting from zero and returning to zero after accessing all target circle centers. Because the number of points is low, the problem lies in creating a small TSP model. The problem can be directly solved by employing the example method. Since each axis of the motion platform is controlled by a separate motor, the Halton distance between two points is taken as the weight when creating the TSP model.

5.2. The Kinematic Trajectory of the UR5 Robot

According to Figure 8, some painting areas overlap, and, therefore, the greedy principle is harnessed to include as many target points as possible for each time of painting. The painting process requires uniform motion during the painting process, and, therefore, the shorter the total painting path is and the shorter the painting time is, the higher the painting efficiency is.

When each area is painted, the painted model can be transformed into a TSP model that finds the shortest path which can access all of the target points [26]. When different areas are painted, in order to prevent the UR5 robot from colliding with the processed objects, the UR5 robot’s joints must be moved after returning to the origin of the 3-dimensional Cartesian system. Then, the UR5 robot kinematics trajectory planning model can be transformed into the solution of multiple TSP models.

However, due to the large number of points to be painted, this model is a large TSP model. Genetic algorithms (GA) are harnessed to create each TSP model [27,28].

5.2.1. The TSP Model Based on the Genetic Algorithm

A genetic algorithm-based computational model that simulates the evolutionary process of natural selection and genetic mechanisms of Darwin’s theory of evolution is proposed. The genetic algorithm does not need continuous function limitation and has a better global optimal solution. The core method of GA is to evaluate the fitness of all individuals in each generation of the population and select some individuals for genetic selection, crossover, and mutation to form the next generation population. The main solution steps of the genetic algorithm are shown in Figure 17.

5.2.2. Operations Related to the Genetic Algorithm

(1) Coding method: all target points are encoded into one chromosome G_iG_j…G_wG_k, and G_i in the chromosome represents the i-th number of the target point. In this genetic algorithm, each individual has one and only one chromosome.

(2) Weight: the Euler distance between two points is taken as the weight between the two points.

(3) Chromosome distance and individual fitness: the fitness of the individual f equals $N/{\displaystyle \sum }_{i=2}^n{d}_{i\left(i+1\right)}$, where N represents the gain coefficient. The value of N only affects the value of fitness and does not affect the final TSP solution result.

(4) Selection operation: the fitness of the population is calculated; the best individual is screened, and the coding method of the individual is retained. For the remaining individuals, the individual distribution function is calculated in accordance with their individual fitness. The maximum and minimum normalization algorithms are employed to normalize the distribution function to 0-1. The selection operation process is shown in Figure 18.

(5) Crossover operation: in order to ensure the positive optimization of the genetic algorithm, crossover operations are performed only on non-optimal individuals in the population. The crossover operator uses the Order Crossover operator. The crossover algorithm is shown in Figure 19, where P_c represents the crossover probability.

(6) Mutation operation: in order to prevent the GA from falling into a local optimal solution, an adaptive mutation rate is harnessed. The equation for calculating the mutation probability is shown in Equation (31),

$$P_m=\{\begin{array}{l}\frac{K_1\left(f_{\max }-f\right)}{f_{\max }-f_{avg}},f\ge f_{avg}\\ k2,\hspace{1em}\hspace{1em}\hspace{1em}f<f_{avg}\end{array}$$

(31)

where f_max represents the maximum fitness of the group; f is the fitness of the individuals to be crossed. K₁ and K₂ are the adaptive parameters. The mutation operation is shown in Figure 20.

6. Simulation Experiments

The spray gun of model WA-101 was adopted as the painting equipment, and the offset distance of the spray gun from the UR5 robot end axis was T_tool = Trans (5.5, 94.5, 165.5).

The painting process requirements are described in this paragraph. The distance between the fixture and the target point was T_dis, which equals Trans (0, 0, 113), and the end of the spray gun generated a circle with a radius of 50 mm at this distance. The maximum position error of the target point was 5 mm, allowing a 5° error between the gun’s end axis and the target’s normal vector.

6.1. The Pose Acquisition of the Target Objects

The analysis of the STL file of a car is shown in Figure 21, and the point cloud model of the car is shown in Figure 22. The shapes and sizes of Figure 21 and Figure 22 are the same as the actual three-dimensional model. The target extraction algorithm proposed in this paper has promising practicality.

The part of the car roof where z coordinates are larger than 1000 mm was taken for the target points for painting. The normal vector at the end of the target point was created by employing the PCA algorithm, as is shown in Figure 23 (because the performance of the computer graphics card was not satisfactory, and only the normal phase of some points is shown in the figure). As can be seen from Figure 23, the normal vectors of the target points conform to the shape of the roof, and the algorithm can better establish the normal vectors of the target points. After the variable r in Equation (6) had been given a value of 50, the model of the compression points of the car roof, which is shown in Figure 24, was achieved as per Equation (7). After Figure 23 and Figure 24 are compared, it is clear that the size and shape of the target points remain unchanged after compression.

6.2. Simulation of the Establishment of the Motion Platform Trajectory

After Equation (27) is given values T_tool and T_dis, the distribution of points at the ends of the UR5 robot joints is shown in Figure 25. By utilizing the smallest rectangle model in Figure 15, the smallest rectangle containing the target points was obtained, which is shown in Figure 26.

The rectangle in Figure 26 has a width of 2635 mm and a length of 1959 mm. Data in Figure 26 were extracted and employed in Figure 16 and Equation (30) to obtain the preliminary partition model of the target points, which is shown in Figure 27.

The model in Figure 27 was created by the exhaustive method. The result is shown in Figure 28, which describes the minimum number of target points in the non-recommended area when h₁ and h₂ are determined. The blue areas in the figure represent an invalid combination that does not satisfy Equation (30). In this optimal combination, h₁ equals 980 mm; h₂ equals 980 mm; P_x equals 7 mm; and P_y equals 3 mm.

The model in Figure 27 was given values P_x, P_y, h₁, and h₂ to obtain the partition model of the target points which are shown in Figure 29. The red circles in Figure 29 are the maximum processing area and the non-recommended processing area of the UR5 robot. C₁ represents the coordinates of the i-th circle, and the coordinates of C_i are shown in

Table 2. The center point C_i.

Ci	X/mm	Y/mm
1	469	2685
2	465	1611
3	461	537
4	1451	552
5	1447	1622
6	1443	2691

.

All of the possibilities in Figure 29 were iterated, and the processing order with the smallest total movement of the motion platform was taken. The final painting order of the UR5 robot was C₁ C₂ C₃ C₆ C₅ C₄. The painting intervals of the UR5 robot are shown in Figure 30. The dots in different colors in Figure 30 represent different painting intervals.

6.3. The Establishment of the Motion Platform Trajectory

The target points with the same color in Figure 30 were introduced into and utilized in GAs, and the population number was set to 5000. The maximum number of iterations is 25,000, and the cross probability was 0.9. K₁ equals 0.15, and K₂ equals 0.2 in the selection probability. The GA solution process is shown in Figure 31, and the final painting trajectory is shown in Figure 32.

6.4. The Inverse Kinematics Simulation of the UR5 Robot

When the UR5 robot found its inverse kinematics, it generated multiple solutions. When the UR5 robot’s inverse kinematics were being solved, the angle values of the target points on the previous painting trajectory were harnessed as the initial angle values, and the target points on the final painting trajectory in each colored area with the smallest radian changes from the initial angle values were selected. The solution was the inverse kinematics of the target points. The position errors and axis errors of the statistical UR5 robot simulation are shown in Figure 33 and Figure 34, respectively.

As per Figure 33 and Figure 34, the errors are within the technical requirements of painting large objects, and the algorithm has satisfactory engineering practicability. Since the main solution in this paper was an analytical solution, when the joint radian of the UR5 robotic manipulator’s motion platform in the recommended area needed to be solved, an iterative method was harnessed to solve Equations (14) to (18).

7. Discussion and Conclusions

This paper proposes an algorithm that employs a nine-axis robotic manipulator to automatically paint large objects. With the proposed algorithm, automatic processing of complex objects is achieved. The algorithm is divided into three aspects that consist of extraction of the target model, the establishment of the inverse kinematics model of the manipulator, and the planning of the target trajectory.

In the section of the proposed algorithm for extracting the target model, as per the STL file of the target trajectory, the triangles of the target trajectory are extracted. A full 3D data point cloud model is obtained with a triangle-based filling algorithm. Subsequently, the PCA algorithm is harnessed to identify the normal vectors of the target points inside the point cloud model. With the normal vectors, Rodrigues’ rotation formula is harnessed to extract the pose of each point of the painting trajectory. Finally, the number of target points is compressed to reduce the time and space complexity of the algorithm so that the painting process requirements can be satisfied.

In the section of the proposed algorithm where the inverse kinematics model of the manipulator is established, in conformity with the processing range of the robot, the inverse solution algorithm is divided into the inverse solution of the recommended region and the inverse solution of the non-recommended region. The corresponding inverse kinematics model is established by employing the closed-form solution method and SQP, respectively.

During the planning of the painting trajectory, in line with the point cloud model of the target points, the minimum envelope rectangles of the target points are found. The target points are divided into different painting areas in accordance with the minimum rectangles. For each painting area, a TSP trajectory planning model based on GA is harnessed to plan the robot’s painting trajectory with which the inverse kinematics model of the UR5 robot is created.

Not only does the trajectory created by the proposed algorithm consider the singularities of the kinematic joints of the UR5 robot joints, but it also helps the UR5 robot to paint large objects with precision and efficiency. Multiple reliable simulations of the painting process on the car roof surface were conducted, and the results show that the algorithm can meet the technical requirements of painting and that the algorithm has promising practicability.

The proposed algorithm has several benefits. The trajectory created by the proposed algorithm allows the motion platform of the UR5 robotic manipulator to have the least amount of movement while the UR5 robot paints a large object. Therefore, the proficiency of the painting process can be significantly improved. The standard triangle language (STL) file, algorithm of principal component analyses (PCA), and k-dimensional tree (k-d tree) were employed to obtain the point cloud model of the car roof to be painted. The point cloud model was later converted into multiple traveling salesman problem (TSP) models, each of which was created with genetic algorithms (GAs). In this way, the UR5 robot could identify the object to be painted and generate a trajectory to paint the large object more efficiently.

However, limitations still exist in the current research. The closed-form solution method and SQP were employed to establish the inverse kinematics model of the UR5 robot. SQP is not intelligent enough and still requires a myriad of training solutions. If a more intelligent neural network model had been utilized, the preparation time for the entire painting process would have been reduced. In this research, only the painting trajectory was created, but the dynamics and velocity-planning of the UR5 robot were not involved. Therefore, future studies need to focus on a more intelligent neural network model that helps to establish the inverse kinematics model of the UR5 robot as well as the dynamics and velocity-planning of the UR5 robot.