Industrial Printing Image Defect Detection Using Multi-Edge Feature Fusion Algorithm

. Online defect detection system is a necessary technical measure and important means for large-scale industrial printing production. It is eﬀective to reduce artiﬁcial detection fatigue and improve the accuracy and stability of industry printing line. However, the existing defect detection algorithms are mainly developed based on high-quality database and it is diﬃcult to detect the defects on low-quality printing images. In this paper, we propose a new multi-edge feature fusion algorithm which is eﬀective in solving this problem. Firstly, according to the characteristics of sheet-fed printing system, a new printing image database is established; compared with the existing databases, it has larger translation, deformation, and uneven illumination variation. These interferences make defect detection become more challenging. Then, SIFTfeature is employed to register the database. In order to reduce the number of false detections which are caused by the position, deformation, and brightness deviation between the detected image and reference image, multi-edge feature fusion algorithm is proposed to overcome the eﬀects of these disturbances. Lastly, the experimental results of mAP (92.65%) and recall (96.29%) verify the eﬀectiveness of the proposed method which can eﬀectively detect defects in low-quality printing database. The proposed research results can improve the adaptability of visual inspection system on a variety of diﬀerent printing platforms. It is better to control the printing process and further reduce the number of operators.


Introduction
e online defect detection system based on machine vision is widely used in the field of industrial automation, such as welding defect detection [1], glass manufacturing industry [2], machine-parts processing [3], printed circuit board industry [4], textile industry [5], and printing industry [6,7]. In printing industry, manual detection has been far from meeting the quality control requirements of modern largescale printing production. Online defect detection system is an indispensable link to ensure the quality of printed matter [8]. Figure 1 gives an example of industrial printing line, and Figure 2 shows a typical defect detection system using machine vision for roll-to-roll printing line.
is paper is organized as follows. We present related research in Section 2, and Section 3 gives the methodology, which includes the architecture of the proposed defect detection system, image registration method, and feature extraction method. e experimental results and discussion are given in Section 4. Section 5 presents the conclusion and future works.

Related Research
Many defect detection methods have been proposed for roll-to-roll printing process, and these methods can effectively detect a variety of printing defects in real time. Paper [9] detected some common printing defects, such as ink drop, stripe, character loss, and color defect. Compared with the traditional method to extract the gradient edge of gray image, the proposed edge detection algorithm has better detection performance, and it can reduce the information loss of RGB three channels and make the edge extraction more accurate. Paper [10] provided an image fusion method, which used multi-channel image subtraction to segment defects. e method can update the reference image continuously with the printing production. Next, a twice template matching algorithm was established in paper [11], which firstly matched the template and then performed differential operation on matched image to find the location of the defect. In paper [12], the authors designed a bidirectional image difference algorithm to avoid the error detection of contour artifacts. In order to better locate the first row of captured image, a fast-computational algorithm based on image projection was given, which can convert 2D image searching into 1D feature matching. Paper [13] adopted laser scanning technology to detect overfill and underfill defects on upper surface of the deposited parts in the additive manufacturing process by comparing the existing point cloud with the presliced stereolithography (STL) model. In paper [14], Chervyakov et al. utilized two modified adaptive median filtering methods of impulse noise in images. e experiment showed potential application in processing satellite and medical imagery, geophysical data, and in other areas of digital image processing. Paper [15] provided a solution to the problem of distinguishing the defects and their own characteristics in robot 3D printing. e research findings can help to detect the defects online, improve the detection accuracy, and reduce the false detection rate without being affected by its own characteristics. e abovementioned detection methods all applied image difference to extract defective patterns. Because the defective image and standard image collected in reality often have interferences with size, rotation, deformation, and other factors, whether the two images can be well registered will directly affect the accuracy of defect detection [16]. Image registration needs to be considered from the feature space, search space, interpolation method, search strategy, similarity measurement, and other aspects [6]. In addition, these inspection systems also need to meet some other requirements: (1) high-end and expensive line scan CCD camera; (2) special high-precision mechanical installation structure and lighting mode; (3) stable and reliable feeding platform; and (4) high-quality image data acquisition. ese requirements limit the application range of the detection system [17][18][19][20]. For example, it is difficult to apply the online quality inspection system in the sheet-fed platform and the low-end printing production line because the vibration and interference of the platform are too large, and it is impossible to collect qualified images. For the research issue of lowquality printing image defect detection, the research results are few.
Currently, more and more researchers pay attention to the defect detection method based on machine learning [21]. Du used the deep learning method to improve the performance of X-ray image defect detection of automotive aluminum castings [22]. In paper [23], the defect detection of railway track fastener was studied by combining image processing and deep learning. However, labeling defect regions was time consuming, and it was difficult to collect enough defect samples for artificial neural network learning, which limited the application and promotion of deep learning in the field of defect detection. Paper [24] proposed an automatic inspection system with five-plane array chargecoupled device (CCD) cameras and four LED light sources  2 Complexity in a closed environment. A support vector machine algorithm was adopted to classify defects based on the extracted features in candidate defect regions. In paper [25], Abul'khanov and Kazanskiy created visual and numerical tools to analyze a rough surface, through characterizing the rough surface by building its information pattern through imaging micro-roughnesses on the controlled surface and using the parameter value. Paper [26] designed a bridge cracks detection algorithm by using a modified active contour model and greedy search-based support vector machine. In paper [27], a novel cascaded autoencoder (CASAE) architecture was designed for segmenting and localizing defects. e defect regions of segmented results are classified into their specific classes via a compact convolutional neural network (CNN). Paper [28] adopted a single convolutional neural network (CNN) model that can extract effective features for defect classification without using additional feature extraction algorithms, and the proposed method can identify defect classes not seen during training by comparing the CNN features of the unseen classes with those of the trained classes. Paper [29] proposed a vision-based method using a deep architecture of convolutional neural networks (CNNs) for detecting concrete cracks without calculating the defect features. In addition, many other methods were also proposed for defect detection using machine learning, e.g., the generative adversarial networks [30] and reinforcement learning [31].
In summary, traditional defect detection methods based on image difference cannot be applied to low-quality images effectively and it is difficult to get enough defect samples to train a machine learning model for defect identification. In order to improve the adaptability of visual inspection system on a variety of different printing platforms (sheet-fed or rollto-roll), in this paper, a new printed image database is established using a small CCD area array camera to collect images on a sheet-fed machine platform. en, a new multiedge feature fusion algorithm is proposed to adapt to defect detection in low-quality dataset.

Materials and Methods
e architecture of the proposed detection system consists of image registration, image sub-block, feature extraction, feature fusion, and feature matching, as shown in Figure 3. Initially, some basic image preprocessing methods are introduced and then we describe the detection algorithm using multi-edge feature fusion in detail. Because the variation of uneven illumination and deformation between samples can greatly affect the edge feature extraction, which will lead to false detection and missed detection, we adopt the feature fusion method to eliminate the influence of those interferences. Next, the detection evaluation criteria selected in this paper are described elaborately. Lastly, we present and analyze the experiment results.

Image Registration Using SIFT Feature.
e obtained original image contains many interference factors, such as rotation and deformation [32]. e inconsistency of images needs to be corrected in advance. In this paper, SIFT feature is applied to eliminate these variations in original images, and Figure 4 shows the architecture of image registration using SIFT feature, including key points, matching images, and corrected images [33]. We cut off the redundant boundary with size of 50 pixels directly, and the image resolution is reduced from 800 * 550 to 750 * 500. e corrected images are as follows: where C is the corrected image, i and j are the coordinates of each pixel, R stands for the registered image, and k is the image sequence number. en, we divide the whole image into several sub-blocks with resolution of 50 * 50, as shown in image sub-block step in Figure 3.

Feature Extraction Using Canny Edge Detection
Operator. e mainstream edge detection operators include Roberts, Sobel, Prewitt, and so on [34]. Robert operator is sensitive to noise. Prewitt and Sobel operators have better detection performance on the image with gradual gray level and low noise, but for the image with mixed multi-complex noise, the processing effect is not ideal. e detection effect of the Canny operator is better than that of the gradient operator, which can detect the thin edge of the image. ere are four processing steps consisting of noise reduction, gradient calculation, non-maximum suppression, and double threshold filtering. We can change the edge with multiple pixel width into a single pixel wide edge and remove the weak edge to retain the strong edge. erefore, it is necessary to select the appropriate operator to detect the edge feature according to different environmental conditions and requirements.
e output edge images are as follows: where B is the detected edge feature binary image, Nms represent gradient amplitude images after non-maximum suppression, i and j are the coordinates of each pixel, and TL and TH represent high and low thresholds. Figure 5 presents the edge detection effect of the Canny operator, and the defective part is marked with a red circle.

Feature Similarity Matching Using Euclidean Distance.
After obtaining the edge features, similarity with feature matching is used to judge the defect. Firstly, the edge feature image is reduced to one-dimensional feature vector. e formula is as follows: where F is the one-dimensional eigenvector of edge block feature, B represents every pixel on the binary edge feature image, m and n represent the row number and column Complexity 3 number of feature image, and k is the image sequence number. en, Euclidean distance is used to match the similarity between two eigenvectors [17]. e formula of Euclidean distance is

⎧ ⎨ ⎩ (5)
When the matching result is greater than the threshold value, the test eigenvector is recognized as defect and 1 is output.

Fluctuation Analysis of Feature Matching Similarity.
Firstly, we defined feature similarity matching as four types: (a) single qualified pattern and multiple qualified patterns matching; (b) single qualified pattern and multiple defect patterns matching; (c) single defect pattern and multiple qualified patterns matching; and (d) single defect pattern and multiple defect patterns matching. Figure 6 shows the corresponding matching results. According to analysis of matching results, the four matching results show uniform fluctuation and there is no obvious similarity difference between them. It is due to reflection of pattern surface, uneven external light, and mechanical vibration. ese images with inconsistent fluctuation can lead to false detections. In the next section, we will utilize feature fusion method to eliminate matching similarity fluctuation between qualified patterns.

Template Establishment and Defect Segmentation.
In order to eliminate the interferences as much as possible, we propose a multi-template edge feature fusion algorithm to increase the accuracy of defect identification. e architecture of the proposed method model is shown in Figure 7, and the formula of feature fusion is expressed as follows: where F Template represents the template feature vector, F Defect stands for the defect image feature vector, and n is the sequence number of template images. eoretically, the more the features are fused, the better the robustness of the template will be, and subsequent experiments will test the proposed template detection performance with different fusion sizes.
For visualization of defects, we extract the defect part using feature image difference, as shown in Figure 7. e defect feature is obtained using image difference between test feature and fusion feature. According to the block position number, we mark it in the original image to complete semantic segmentation and display the defect parts.

Results
It is difficult to collect enough defect samples in industrial production site. We ended up with 4035 complete qualified images and 135 defective images. Because most of the samples are defect free, we take 135 of them as a genuine class. us, the data used for test experiment contain 135 samples for each type and all the images are divided into 50 * 50 sub-blocks. e dataset and some samples are shown in Table 1 and Figure 8, and the defective parts are marked with a red circle.
To demonstrate the effect of fusion size on the detection accuracy, some experiments under different fusion sizes were carried out, and the results are shown in Table 2. We designed seven different fusion sizes from 1 to 256. e two truth values represent genuine and defective patterns, respectively. Apparently, the detection accuracy increases significantly when the fusion size becomes larger. However, when the fusion size is larger than 160, the detection accuracy decreased. erefore, the optimal fusion size is 160 for this case with accuracy of 95.18%, precision of 94.20%, and recall of 96.29%. When the feature fusion scale is too large or too small, it cannot achieve the ideal detection effect, such as using a single defect pattern for template feature generation.
In addition, we draw the similarity matching results with fusion size of 160 in Figure 9. e horizontal axis is the image number from 1 to 270, and the vertical axis is the Euclidean distance of feature matching from 0 to 1. From the distribution characteristics of red dots, there are only 5 missing detections with a matching result to 0 in the detection results   Figure 10. e results are the same as shown in Table 2; the detection performance first increases to the green curve with fusion size 160 and then decreases. e front four curves are in a state of underfitting with insufficient amount of feature information learning and the last two curves are in a state of overfitting state with too much redundant information learning. erefore, too much or too little feature fusion cannot achieve ideal recognition effect. In addition, we employ average precision (AP) and mean average precision (mAP) to evaluate the performance of the defect detection method. When fusion size is 160, the mAP and recall are 92.65% and 96.29%, respectively, as shown in Table 3.
Lastly, we test detection performance of the proposed method in extended dataset with 4035 patterns and 135 defects. Table 4 shows the experimental results in asymmetric dataset. e precision of small-sample dataset Complete Pattern defects   To demonstrate the effectiveness of the method proposed in this paper, a comparison is made for the other detection methods including non-fusion registration difference and convolutional neural network (CNN). e result is shown in Table 5.
e method proposed in this paper achieves an accuracy of 93.09% which outperforms other methods. e reason for this result is that the low-quality images are not suitable for the traditional differential detection algorithm  and it is different to provide sufficient defect training data for convolutional neural network, which leads to misclassification. In this paper, a new multi-edge feature fusion algorithm is used to recognize printing defects in low-quality datasets, which achieves a higher precision for the industrial printing image defect detection.

Conclusions
In this work, a new defect detection method using multiedge feature fusion is proposed to improve the detection accuracy of low-quality printing images. e specific contributions are as follows: (1) We set up a new and more challenging print image dataset which consists of 4170 images and has more rotation, deformation, and uneven illumination changes, compared with the existing printing database. (2) e proposed multi-edge feature fusion algorithm can effectively distinguish pattern defects and interference changes. (3) Different feature fusion sizes will greatly affect the detection accuracy, and we also found that for all fusion scales, an optimal value exists for the detection accuracy; too large or too small amount of fusion information will reduce the overall detection performance of the system. e current detection system mainly solves the problem of fine edge defect detection of low-quality printing image, while in further work, the detection system should be promoted to identify more types of defects. In addition, most samples of industrial printing products are qualified and authentic, which leads to unbalanced data types. In future work, how to collect enough defective images in industrial production field and how to use machine learning method to detect defective patterns need to be further studied.

Data Availability
e data used to support the findings of this study are included within the article.

Conflicts of Interest
e authors declare that they have no conflicts of interest.