A Machine Vision System for Online Metal Can-End Rivet Inspection

Can end rivet fracture is considered a serious defect that may arise during the manufacturing of metal cans used in food industry. Its occurrence may result in inefficiency of the easy-opening system, leakage or contamination of the food. Thus, an inspection procedure must be performed in order to remove the defective can ends from the production line. However, the location of the rivet and the characteristics of the fracture defect make it difficult to use simple devices for online inspection of 100% of the cans. In this paper, a new method for detecting fracture defects in can end rivets based on machine vision is described. In the proposed technique the rivets are detected employing Otsu segmentation, morphological filtering and connected component analysis. The classification of each rivet as being defective or non defective is performed using a minimum distance classifier based on circularity and area features. The software and hardware architecture of a real time inspection system and the technical aspects for its integration into a production line are presented. The described system can achieve an accuracy as high as 98.13% for rivet defects detection according to experiments and tests performed in real conditions of use.


Introduction
Metal cans have been widely used in industry due to features such as light weight, recyclability, ease of storage and reduced cost [1]. The ability of the can to deliver safe and undamaged products to the consumer is of paramount importance, particularly in the case of canned products in the food industry [2].
Cans commonly used in industry are comprised of two main parts: the body and the can end. A number of defects may arise during the various stages of manufacture of these parts. The can end is a considerably more common source of defects when compared to the can body, due to the number of parts of which it is composed [3]. In the can end, a pull tab that allows rapid opening without the need for tools is attached to the end panel (known as the full panel easy open or FPEO end) [3]. For the placement of this component, a rivet is inserted through notches in the pull tab and the panel, in order to mechanically join these parts. The insertion of this rivet is carried out by a press. Technological equipment is used in this process, but even so problems such as misalignment between the rivet and the press, and a lack of lubrication, can occur, resulting in incorrect insertion or damage to the rivet. A defect often introduced when these parts are joined is rivet fracturing. Fracture, or dent, is considered a serious defect [3]. Its occurrence may result in inefficiency of the easy-opening system. In addition, a fractured rivet can lead to leakage and contamination of the food. Thus, an inspection procedure must be performed in order to remove the defective can ends from the production line.
This type of inspection is usually carried out in the industry by a human operator. However, the volume and speed of production on canning lines prevents the inspection of 100% of the cans, and thus only a small set of samples is inspected. Factors such as operator ability and experience, work environment, inspection time, characteristics and number of defects occurring simultaneously can further reduce the effectiveness of the inspection activity.
The location of the rivet and the characteristics of the fracture defect make it difficult to use simple devices such as proximity sensors or strain gauges, which allow the inspection of 100% of the products for defects associated with thickness or tab absence [4,5]. More complex inspection systems, such as vision systems, have been successfully employed in the inspection of different manufactured products [6,7,8]. However, work addressing the development of systems employing this technology for can end inspection is rarely encountered and does not address the specific inspection of rivet defects [9,10]. There are also general purpose commercial vision systems available, but technical details regarding their implementation and performance have not been reported in the literature.
This paper describes the development of an online inspection system based on machine vision for the detection of defects in rivets used in can ends. In this context, a combination of different image processing techniques allowing the classification of the inspected can ends as defective or non-defective, according to the presence of defects in the rivets, is proposed. The main contributions of this study are the proposal of a new method for detecting fracture defects in rivets in real time, including the technical aspects of the image processing pipeline employed and the installation and use of the inspection system integrated into a production line.
In Section 2, the approaches found in the literature for the inspection of can ends and other similar products in the industry are described. In Section 3, the system architecture and aspects of the proposed inspection method are detailed. In Section 4 the experimental results obtained with the use of the system developed are discussed. Finally, in Section 5 the conclusions and perspectives for future work are presented.

Automatic Inspection of Metal Can Ends
Typical defects that occur during can end production include excessive scoring of the metal, improper rivet formation, improper embossing, among others. Such defects can lead to problems including can end leakage, improper tab attachment and can rupture [5].
Some simple sensing modules, such as metal proximity sensors or strain gauges, can be used to acquire the operation parameters of the tooling system during can-end forming [4,5]. The data obtained can then be used to detect abnormality and malfunction in the operations. However, this approach can only be applied to detect specific types of defects. Dimensional and shape analysis based on more elaborate sensors, such as cameras, has been successfully employed in different industrial scenarios to ensure the quality of parts. However, specifically for the case of can ends, few studies have been reported in the literature [1,9,10].
Recently, Chen et al. [1] described an inspection system for the detection of scratches, bent edges and the presence of contaminants in the end panels. A multiscale ridge detection algorithm was employed to search for defects along the can-end profiles. The inspection results for the selected defects reached an accuracy rate of 99.48%.
Feng et al. [9] and Marino et al. [10] also proposed systems for detecting similar defects, including glue shortage, deformation and surface scratches. These systems can detect defects with up to 98.7% of accuracy, evaluating the gray values and shape characteristics observed on images of the can end.
For the specific case of rivet inspection, no papers could be found in the literature. Similar studies are related to applications and tools associated with machine vision techniques aimed at  [11]. However, inspection methods focused on other parts that can be compared to the present case, do not address the specific challenges encountered in online inspection for can production.

Proposed System Architecture
The functional requirement that guided the development of the proposed system was the automatic detection of fracture or dent defects on metal can end rivets. The non-functional requirement of integrated real-time inspection was also observed. In order to meet these requirements, the architecture shown in Figure 1 was defined.
In this configuration, an acquisition system and a processing system are integrated into a linear conveyor belt through which the can ends to be inspected pass. An image of each can end is captured by the acquisition system as the moment of its passage is detected by an inductive sensor. These images are transmitted to the processing system, where a computer, running specifically developed software, analyzes the images in order to detect and evaluate the characteristics of the pull tab rivet. Based on the evaluation result, a signal is generated indicating the acceptance or rejection of the can end so that a mechatronic discard system can act in the process, removing defective can ends from the line when the defect is detected. Figure 2 shows the processing pipeline, which includes the main steps of the defective rivet detection software. Each of these steps is described in the following section.

Image acquisition
Image acquisition is an important step in any vision system. Here, aspects related to lighting, field of vision and resolution, among other, must be considered, in order to obtain an image where the characteristics to be evaluated are highlighted. In the case of the rivet inspection addressed in this study, a configuration was sought considering both the acquisition system and the part to be evaluated, so that the presence of a fracture could be evidenced.
A front image of a can end (a) as well as details of the pull tab region for can ends without (b) and with (c) fracture defects are shown in Figure 3. It is possible to observe the large amount of details and components present in the image, corresponding to elements, such as the pull tab borders, silk-screen printing, and the rivet of interest. In the images showing the enlarged detail of the defective and non-defective rivets, it can be observe that the defect is visible due to the change in the shape of the rivet. However, the presence of other elements close to the rivet, as well as the variance in the position of the serigraph printing in relation to the rivet, make it difficult to isolate the object to be evaluated.
Thus, a second configuration was considered, where a rear image of the cover is captured, due to the greater ease of detecting the rivet under these conditions and the possibility of using turning mechanisms on the conveyor belt. A diffuse lighting dome was also used to homogenize the illumination along the can end and increase the contrast between the rivet borders and the central panel. In Figure 4 an image of the back of the same can end (a) is also shown, with details of the pull tab region for the rivets without (b) and with (c) defects. It can be observed that few components are evident along the surface of the can end, with the most prominent points corresponding to the contours of the notches and the rivet. Also, in the described setup, the longitudinal and transverse position of each can end over the conveyor belt will be the same at the moment of image acquisition. Thus, the images can be delimited by a region of interest, as illustrated in Figure 5, which is defined as the area between an internal delimiter circle (magenta) and an external delimiter circle (cyan). These circles, defined during a step in the system configuration, determine the region around the can end center where the rivet will be located, thereby reducing the points of the image that need to be processed in subsequent steps.  Inspection processing pipeline.

Image segmentation
In the segmentation step, we seek to isolate the points of the image that correspond to the object of interest to be evaluated. The histogram in Figure 6 (a) shows the distribution of the intensity of the values for the points that make up the image in Figure 4 (c).
It is possible to observe two peaks that represent the intensities around the mean of the phase value associated with the central panel and the mean of the phase value associated with the edges and shadow regions present on the can end.
In this way, we can select a threshold intensity value T that separates the phases. The automatic determination of the T value for each inspected image can be performed using the Otsu method [12]. The resulting segmented image is shown in Figure 6

Rivet Detection and Description
A binary image where the can end components, such as the rivet and folds, are highlighted is obtained as a result of the segmentation step. As a first step in the rivet detection and description, a sequence of morphological opening and closing operations is applied to eliminate isolated points resulting from noise or points of high reflectivity.
In the next step, each of the point groupings present on the image is identified through a connected component algorithm. Each component shape found is then represented by the corresponding area and circularity descriptors.
The area descriptor corresponds to the number of points belonging to the connected component. The circularity descriptor can be calculated by: where: A s is the area of the component shape P s is the perimeter of the component shape i and this quantity can be used to indicate how close the shape is to a perfect circle [13].  These descriptors were selected because they are invariant to translation and rotation, an important feature for the observed scenario, where there is no guarantee that the orientation and position of the rivets remain the same throughout the inspection of the can ends. The circularity descriptor in particular was selected because the presence of fracture or dent defects in the rivet causes a change in its original circular shape. Table 1 shows some images of rivets with and without fracture or pressing defects, i.e., belonging to the non-defective and defective classes. Their description according to the characteristics of area and circularity are also provided.

Rivet classification
As can be observed, the mean values for the descriptors of each class are at a considerable distance from each other, when compared with the dispersion or randomness of each class in relation to its mean. Thus, the minimum distance classifier with parametric descriptions based on area and circularity was used, since there is a well-defined decision boundary between the classes [14].

System Implementation and Evaluation
In this section, the implemented system, the evaluation scenario and a reference method are described. The results obtained in the evaluation of the system when applied to inspection activities are then discussed.

Implemented System and Evaluation Scenario
An automated inspection system based on the architecture described in Section 3 was assembled and integrated into a canning industry production line. Figure 7 shows the image acquisition system, including an industrial camera model Basler aCA1300-30gm equipped with a 4 mm focal  The processing system consists of a computer with a Core i7 processor and 8 GB of RAM. The software that implements the inspection algorithm was developed using the C++ programming language. The designed user interface is shown in Figure 8. This setup was used to create an image dataset with 320 images. In order to create a ground truth, each can end was evaluated by a specialist and 96 were classified as defective and 224 as non-defective. The diameter of the can ends used was 73 mm, the resolution was set at 1280 * 1024 pixels, and the samples were acquired at a frequency of 4 can ends per second.

Reference Method Description
Although no methods were found in the literature addressing the problem of detecting fracture or dent defects in rivets for the inspection of can ends, some of the proposed processing pipeline steps could be replaced by other techniques. For comparison purposes, a variation in the version of the defect detection algorithm was implemented employing different techniques for description and classification.
In this approach, the acquisition and segmentation are the same as those used in the original method. For the description step, the Fourier descriptors are used to characterize the shape of each rivet. These descriptors are able to represent the essential shape of the image components using a few descriptors in the frequency domain [14]. This approach has been used successfully in industrial inspections and the identification of objects in monitored environments, among other applications [15].
The classification of each rivet as either non-defective or defective according to the description based on Fourier descriptors is performed using a feed-forward neural network with a backpropagation algorithm. The number of neurons in the input layer is equal to 18, that is the number of Fourier descriptors used in the description of the rivet shape. A hidden layer with an equal number of input-layer neurons is used. Finally, in the output layer two neurons are used, one representing a defective rivet and the other representing a non-defective rivet.

Defect Detection Assessment
The image database created, composed of defective and non-defective images classified by the specialist, was used as a reference for comparison with the classification of each can end as approved or rejected by the system developed. Table 2 shows the confusion matrix obtained from this comparison, where we can identify the can ends that were rejected and defective or true positives (T P ), non-defective and approved or true negatives (T N ), non-defective and rejected or false positives (F P ), and defective and approved or false negatives (F N ). Non-defective 2 222 The performance evaluation of the system can be performed by calculating the precision (P r ), the recall or sensitivity (R c ) and the accuracy (A cc ) [1]. The performance statistics for the system developed according to these indices are presented in Table 3. The results obtained with the statistical method based on circularity and area and the minimum distance classifier (CAMDC) as those obtained with the reference method based on Fourier Shape Description and Neural Network (FDANN) are shown. It can be observed that most defects are easily detected with the proposed system. However, exceptions may include small shape deformations or distortions that are not detected. Nevertheless, the results prove that the minimum distance classifier combined with simple descriptors can achieve detection rates equivalent or superior to those obtained with more expensive methods for the given inspection scenario.

Computational Cost Evaluation
The average time consumed for the processing of the inspection pipeline, including the detection and classification steps, was 38.09 ms. This is shorter than that consumed by the reference method (80 ms). A comparison with literature studies is not possible due to the lack of an inspection report for the same type of defect. However, the speed of operation at tuna canneries, for example, ranges from 4 to 8 cans per second. Thus, the system can be used online to carry out the inspection of 100% of the products.
It is important to note that the inspection time can be reduced by making use of resources not exploited in this work such as parallelism features available in the current processor architectures or even in specialized processing systems (GPU and FPGA) [16,17].

Conclusions and Future Work
In this paper, a machine vision system for online can-end rivet inspection has been described. The method developed for the automated inspection of fracture and dent defects was evaluated under real conditions and provided 98.13% accuracy. The processing time required to run the inspection pipeline is 38 ms.
The observed accuracy and robustness are sufficient to satisfy the strict industrial requirements and allowed the implementation and online use of this system, integrated into a conveyor belt, in the canning industry. Moreover, the algorithms developed could be applied in many other industrial rivet inspection applications.
In future work, the proposed system will be implemented in a dedicated embedded system that reduces the size of the solution and increases the speed of inspection.