Robust image registration of printed circuit boards using improved SIFT-PSO algorithm

: Printed circuit board (PCB) layout is becoming high density, high performance, light, and short. In the automatic PCB defect detection system, image registration of PCB plays an important role. However, most of the traditional registration methods are inefficient, and cannot cope with the problems of image distortion, affine, noise, and so on. To address this issue, the authors propose an improved scale invariant feature transform (SIFT) feature extraction algorithm combined with particle swarm optimisation (PSO) to register the images of PCB which placed on a conveyor belt. The advantage of the presented approach is that the registration results are more robust and efficient by optimising the existing PCB image matching framework. The experimental results on the proposed PCB datasets show that the speed of the proposed method (improved SIFT-PSO) is faster than the traditional SIFT feature registration method, and the average computing time of processing single picture can be improved by 10 s, the registration accuracy can be improved by 3–4%. Compared with the experimental results of other algorithms, the root-mean-square error can be reduced to 0.5146 by using the proposed method. Thus, the proposed method (improved SIFT-PSO) is more accurate and robust in real-time inspection system of PCB.


Background
Printed circuit board (PCB) is the electrical connection provider of electronic components. The use of PCB can greatly reduce layout and assembly error. The PCB layout technology is becoming high density, high performance, light, short, and small. The accurate registration of PCB images plays an important role in the real-time inspection system of PCB.
In the process of PCB image acquisition, there may be uneven illumination changes, the CCD camera itself may also have sensor noise, as well as particle noise in the photosensitive process, so that the image registration of PCB is becoming very difficult.
The goal of PCB image registration is to find the best transformation that represents the relative transformation between two comparison images [1]. In the system of real-time inspection of PCB, our goal is to find an acceptable accurate solution in a large geometric transformation search space to provide better registration images for high-quality products of PCB.

Relation to prior work
A number of methods have been proposed for PCB image registration. These methods can be divided into two categories: region-based methods and feature-based methods [2].
(i) The region-based method: The matching information is found by calculating the similarity measure of the original pixel intensity in the overlapped regions of two images. The mainly used algorithms include normalised cross-correlation and mutual information [3]. However, the region-based method is desirable in the case of a few prominent feature points, but this method is particularly affected by the external environment, such as lighting changes and image distortion. It suffers from heavy computational complexities [4]. (ii) The feature-based method: This method extract more robust features between two images, and establishes the registration mapping relationship between two images through the matching relationship of the features. Generally, these features include points, edges, outlines, and centroid of a particular area, such as HOG, scale invariant feature transform (SIFT). In contrast, featurebased methods are more robust, it can register PCB images in completely different environments, and can handle complex image distortions [5].
SIFT algorithm can achieve invariant features such as rotation, scale, affine, and illumination on the extracted image features. It is currently the most effective algorithm in feature detection and matching algorithms. There are also some researchers focus on improving the SIFT feature extraction method. For example, the SIFT feature and mean shift's expectation maximum algorithm [6] have achieved better tracking results, but how to update the target template has not been described. Wu et al. [7] proposed the FSC-SIFT algorithm, which performs well for images with highly distortion. Paul and Pati [8] proposed a M-UR-SIFT algorithm which can significantly improve the distribution quality of the extracted features. Ye et al. [9] proposed fuse SIFT and convolutional neural network features for remote sensing image registration. Saleem and Sablatnig [10] proposed a modified SIFT algorithm which has better performance in the multi-spectral image of the structure scene, and in the texture scene, the method has better performance compared with the SIFT.
In this paper, we combine the improved SIFT algorithm with the particle swarm optimisation (PSO) algorithm, and propose an improved SIFT-PSO algorithm to accelerate the registration efficiency of PCB images and reduce the registration time. In Section 2, we will describe the algorithm we proposed, including the improved SIFT algorithm and improved feature matching using our methods. In Section 3, we will do the experiments on ten different PCB image pairs. Finally, we conclude the work based on the experimental results.

Methodology
In this section, we elaborate on how we perform our method to register PCB image pairs. The flowchart of our method is given in Fig. 1
However, for the PCB image, the scaling difference between the test image and the reference image acquired in real time is not large so that the image scale is irrelevant. Therefore, the SIFT algorithm only needs to care about the registration of extreme points and does not need to search the entire scale space [12]. So we improve the SIFT feature extraction mainly in two aspects: (i) Reduce the scale space complexity: The scale space of a PCB image is defined as shown in the following euation: where * denotes convolution operation, G(x, y, σ) denotes the Gaussian function, I(x, y) denotes original image. The Gaussian function can be computed as where m and n denote dimension of the Gaussian template, x and y denote the pixel position of the image, respectively, and σ denotes the scale space ratio. First, after building the Gaussian pyramid, the image pyramid was divided into G groups. The first group had g layer image, and the rest of the groups had only one layer. Compared with the traditional SIFT algorithm, the number of layer image can be reduced g*(G − 1). Second, the first groups of images use the same Gaussian difference algorithm as the traditional SIFT algorithm to generate the g − 1 layer with the same size of the original image, the other group images are sampled by half of the upper group image. For example, the Gaussian pyramid image of the reference image constructed is shown in Fig. 2.
(ii) Improved gradient magnitude calculation method: First, we calculate the gradient magnitude of a Gaussian-scale spatial image. The formula is as follows: represents the horizontal direction of Gaussian scale spatial, while G v (x, y) represents the vertical direction. Then, we improve the gradient amplitude compute formula as follows: where G h1 represents the horizontal direction of Gaussian scale space of G h , while G v1 represents the horizontal direction. We use the sobel filters to approximate all the derivatives [13].
In the process of calculating the descriptor by the improved SIFT algorithm, we locate the SIFT feature point, and obtain the gradient magnitude and direction information. Then we use a set of vectors to describe the feature point descriptors, which contains not only the feature points, but also the pixels that contribute to the feature points and around them. The following steps are as follows: (i) correct the rotational main direction to ensure rotation invariance; (ii) generate descriptors to form a 128-dimensional feature vector; (iii) normalisation. In order to ensure the rotation invariance of the feature vectors, we rotate the coordinate axis (the main direction of the feature point) in the neighbourhood of the centre with the feature point as the centre, that is, rotate the coordinate axis as the main direction of the feature point. The new coordinates of the pixels in the neighbourhood after rotation are where x and y denote the coordinates of the pixels, respectively, and θ denotes the rotation angle

Improved feature matching using our methods
In this paper, we mainly study the registration of PCB images based on the optimisation problem of PSO algorithm. In addition to fully considering the accuracy and stability of the algorithm, the convergence speed and global optimal effect of the algorithm are also important considerations. Search optimisation is the process of choosing the similarity measure to optimise the measure by using the optimisation algorithm. The PSO optimisation algorithm [14] is a global optimisation algorithm based on group evolution, which can obtain a nearly perfect global solution. The optimisation process does not rely on gradient information, but also has the advantages of high search efficiency and strong robustness.
it is extended to a Ddimensional space. Each particle knows the best position P best and current position X i , the best position g best found for all particles based on the fitness value are determined by the fitness function. Particles use their own experience and best experience in peers to determine the next move position. The evolution of particle i from kth generation to k + 1 th generation is updated by the following formula [15]: where x id k denotes the d-dimensional component of the particle i position vector at kth iteration. P id is the d-dimensional component of the best position P best of the particle i. P gd is the d-dimensional component of the best position g best . ω is the weight of inertia that helps in the search for new areas c 1 , c 2 denote weight of the statistical acceleration that pushes each particle to P best and g best . r 1 k and r 2 k are random numbers that change in the interval [0,1]. The key to applying PSO solving problems lies in the setting of PSOrelated parameters and the selection of fitness function. We use Euclidean distance between matching points as the fitness function of PSO, and find the optimal affine transformation parameters through the powerful search performance of PSO algorithm.
Assume that we have extracted two matching point sets from the reference image and the test image, they are p = p 1 , p 2 , . . . , p N and p = p 1 ′, p 2 ′, …, p N ′ . Then we use Euclidean distance in position scale orientation (PSO-ED) [16] as the fitness function in our method: (see (9)) (x i , y i ) is the position of the key point p i , while s i and θ i are the scale and main orientation, respectively, in the reference image. x′ i , y′ i is the position of the key point, while s i ′ and θ i ′ are the scale and main orientation, respectively, in the test image.
r* and Δθ* denote the ratio between the reference image and the test image and the pattern position of the main azimuth difference, respectively.

Experiments and discussions
In this section, the various experimental results for testing the efficiency and image registration accuracy of the proposed algorithm are presented. The experiment is conducted under the hardware environment of Intel(R) Core(TM) i5-4200U CPU 2.3 GHZ, 12 GB memory. The experiment environment is MATLAB 2014(a) under Windows 7.

PCB datasets and settings
There are 30 pictures, and the test pictures have the characteristics of uneven illumination, affine transformation, and large noise in our PCB datasets. However, due to space limitations, we only select two pairs of registration images for detailed description. We select a standard PCB image as a reference image, as shown in Figs. 3a and c; and then we give a set of parameters to transform the reference image to obtain a set of test images, as shown in Figs. 3b and d. It can be seen from the image that the PCB images have a single texture and there are a large number of similar areas. We use the reference image as a reference map to register the transformed PCB image as a test image. The actual transformation parameters are shown in Table 1.
Δx denotes the parameters of horizontal displacements. Δy denotes the parameters of vertical displacements. Δθ denotes the parameters of angle motion.

Matching accuracy:
We use the root-mean-square error (RMSE) criterion [3] to evaluate the matching accuracy. It can be computed by the formula as follows: where M represents the total number of corresponding point pairs (x i , y i ), (x′ i , y′ i ) from the reference image and the test image, where x′′, y′′ denotes the transformed coordinate of (x′, y′). For each set of reference images and test images, we calculate RMSE for 15 times and then take the average of the results as the final result [16].
PSOED p i , p′ j = 1 + e p p i , p′ j 1 + e s p i , p′ j 1 + e o p i , p′ j ED p i , p′ j e p p i , p′ j = ∥ (x i , y i ) − T x′ j , y′ j , μ ∥ e s p i , p′ j = 1 − r* s′ j s i

Correct matches points number:
We use the number of correct matches to evaluate the robustness of the proposed method.

Computing time:
The algorithm is executed 20 times, and the average of 20 computing time is taken as the final computing time.

Experimental results of image feature point extraction and feature matching
We do experiments on ten image test pairs. The results are shown in Fig. 4. It shows the space ratio of the matching key points, the histogram of main azimuth, horizontal displacement, and vertical displacement of the matching keypoints in Figs. 3a and b after applying the improved SIFT feature extraction algorithm. From the results shown in Fig. 4, it can be seen that the new gradient information calculation method solves the problem of the intensity difference of PCB image pairs. Fig. 5 shows the initial keypoints distribution and keypoints after using improved SIFT.

Experimental results of image registration
We conduct four sets of comparative experiments, including the improved SIFT-PSO, SIFT-PSO, SIFT, and SURF. The experimental results are shown in Table 2, where M represents correct matches points number.
After using our method, the image registration result is shown in Fig. 6.
As we can see from the results of Table 2, we effectively increase the number of the correct matching based keypoints on the improved SIFT-PSO, which has strong robustness and accuracy. According to Fig. 5, we can draw a conclusion that the correct matches points number increased. According to the image registration results shown in Fig. 6, and transformed images are  almost completely restored, laying a solid foundation for the PCB defect detection. Finally, we do experiments on ten different image pairs. The number of correct matches points number are shown in Fig. 7.
The comparison results of the computing time are shown in Fig. 8.
As we can see from Figs. 7 and 8, the improved SIFT-PSO algorithm improves the performance by simplifying operation speed of the feature descriptor and the registration accuracy can be improved by 3-4%. The computing time of processing single image is reduced by 10 s on average.

Conclusions and future works
In this paper, we propose an improved SIFT feature extraction algorithm to extract features and optimise the search space by using the PSO algorithm to enhance the search efficiency in the real-time inspection system of PCB. We named this algorithm improved SIFT-PSO. Compared with traditional SIFT, SURF, and SIFT-PSO, our method efficiently cope with the problems of image distortion, affine, noise between reference images and test images. Experimental results on our PCB datasets show that the newly proposed improved SIFT-PSO algorithm has obvious advantages in registration efficiency and registration accuracy, which is suitable for real-time production inspection of PCB.
Our proposed method shows superiority in most PCB image datasets, but the registration efficiency depends on feature points. When PCB images acquired on a conveyor belt are blurred or the resolution ratio is low, the feature points may not be detected. Our future work focuses on dealing with this issue.