Detection of Surface Defects of Type ‘orange skin’ in Furniture Elements with Conventional Image Processing Methods

Abstract

An attempt was made to differentiate between surfaces of furniture elements having the orange skin defect and those free from it. As the detectors, the directional derivative of the image intensity along the dominating light direction and the modulus of the image intensity gradient were used. The detectors were tested on series of images with the small and large light incident angles. In case of both detectors, there existed sufficiently wide ranges of thresholds for which both sensitivity and specificity were $100\,\%$ for all the 19 images tested. The ranges of thresholds were wider for the light closer to tangential, and for the detector using the gradient modulus, than for the other cases. The optimum scale of the detectors was found different for each light conditions.

1 Introduction

Quality inspection is a vital element in furniture manufacturing. In this type of production the dimensions and shape accuracy [1, 2] are equally important as the aesthetic aspect related to the visual appearance of the elements. To our best knowledge there are very little or virtually no reports on the quality inspection in furniture industry with the use of image processing methods. We have tried to demonstrate the applicability of these methods to some basic measurement tasks in our previous study [3] in which we have analyzed the images taken with a 3D scanner. In that study we have found that one of the common defects is at the border or outside the range of applicability of the measurement technique considered. This was the surface defect called orange skin which can emerge in the painted surfaces. In this paper we shall demonstrate that orange skin can be successfully detected with the conventional 2D image processing methods.

As we have mentioned in [3], the status in the domain of furniture elements quality control is much different from that in the timber industry, where image-based analysis of structural and anatomical defects is a well developed technology with broad literature (see the reviews [4, 5]).

The remainder of this paper is organized as follows. In the next Section the defect to be considered and its images will be presented. In Sect. 3 the method of detection of the defect will be described. In Sect. 4 we shall outline the way in which we shall assess the proposed method. The results of the assessment will be shown and discussed in Sect. 5. Finally, we shall conclude the paper in Sect. 6.

2 Defects and Images

Orange skin is a defect of finishing the surface with lacquer which manifests itself with uneven structure of the hardened surface. The reasons for this defect can be insufficient quantity or bad quality of dilutent, excessive temperature difference between the lacquer and the surface, bad pressure or distance of spraying, excessive air circulation during spraying or drying, and insufficient air humidity. The analyzed surfaces are flat and covered with lacquer, so the defect can be safely treated as the only reason for surface unevenness. Therefore, the considered surface of a furniture element has been divided into only two classes: the orange skin called also simply skin and the good surface called also good.

A number of typical furniture elements containing flat surfaces belonging to the two classes have been imaged with a typical, good quality color camera, at a moderate resolution of 2.5 M pixels (1288 $\times$ 1936 pix). Two light conditions were used, with light falling onto the object surface at a smaller angle (nearly tangentially) – conditions light1, and at a larger angle – conditions light2, to check for the influence of this angle on the contrast of the defect. Examples of images with the defective surface taken in light conditions light1 and light2 are shown in Fig. 1. An example of Good surface for light2 can be seen in Fig. 2g1.

Fig. 1.

Example of images with the orange skin surface defect made in light conditions: (1) light1, (2) light2. (a) Source image, dimensions 1288 $\times$ 1936 pixels; arrow corresponds to 12 mm $\equiv 205$ pixels. (b) Detail marked in the corresponding figure a with a square, dimensions $70\times 70$ pixels, contrast enhanced. Coordinate system is shown displaced.

In Fig. 2 it has been shown how the images were prepared for testing the method. For each image, two more images were prepared by manually marking some parts of their surfaces. In the mask image, the part of the surface to be subject to analysis was marked with white color. In this way, these parts of the images were excluded which did not belong to the furniture element, and which did not belong to the planar surface where the defect could appear. In the color image, the regions belonging to the white mask in which the orange skin did not appear were marked by the green color, and the regions in which the skin appeared – with the red color. The characteristics of the orange skin defect is such that it appears in the large part or whole surface of the element, or it does not appear at all. Therefore, there were no objects with both skin and good surfaces. The numbers of images in the two sets are shown in Table 1.

Fig. 2.

Example subsets of test images for conditions light2 for one of the elements with (1) class good surface (good) and one with (2) class orange skin (skin). (g) Grey image – the source; (c) color image with pixels belonging to the classes marked with colors: green for good and red for skin; (m) mask image with pixels belonging to the considered surface of the objects marked with white. Hand-written marks excluded (Color figure online).

Table 1. Numbers of test images in the two sets used.

3 Method

In tangential light the orange skin manifests itself with inhomogeneity of brightness, while the good surface is homogeneous, so it can be argued that a good method to distinguish between such two surfaces should be chosen from the textural measures (see e.g. [6]). However, in this introductory study we have decided to test the simpler method and to use the derivative operation. We have chosen the numerical approximation of the derivative combined with the Gaussian function filtering as proposed in classical literature [7] and later used extensively (e.g. [8]). This formulation makes it possible to take into account the scale at which we observe the differentiation result. This well known operation resolves itself to the convolution of the image intensity function with the functions shaped like that in Fig. 3. The parameter $\sigma$ of the Gaussian function will be further referred to as the scale parameter or simply scale.

We shall use two versions of the detector: the directional one calculated as the derivative of the image intensity function along the direction of the light, and the nondirectional one found as the modulus of the gradient of the image intensity function. The output from the detector will be thresholded with threshold T. Pixels with the output exceeding the threshold will be treated as defective.

4 Methodology of Verification

The object to be detected is the orange skin defect, and the remaining surface is the good surface. If one pixel is considered, then the true positive (TP) result is to properly classify the skin pixel, and the true negative (TN) result – to properly classify the good pixel. The erroneous classifications, false positive (FP) and false negative (FN) ones, are defined in a classical way. The detector would work well if the threshold were set so that the number of errors is minimal. However, there is no need that every single pixel is classified properly, but it would be still perfect if there are at least some true positive detections in each defective furniture element, and no false positive detections in any of the good elements. In more detail, if at least some defective pixels were detected in every defective element, the method would exhibit no false negative errors and it would be perfectly sensitive. If no defective pixels were detected in good elements, the method would exhibit no false positive errors and would be perfectly specific. If such a single threshold could be set for all the tested objects, the method could be considered good. The method would be acceptable also if there were some errors of the said types, but their numbers were small.

Fig. 3.

Mask for the derivative with respect to x, for $\sigma =1$.

Typically the relation of the sensitivity and specificity of a detector is displayed in ROC curves (cf. [9]). This is possible if the ground truth data are available. In the case of the classification of pixels, this is possible if the defective and good pixels can be univocally marked in the test images. In Fig. 1 one can see that this would be difficult for the skin defect, because its symptoms are sparsely displaced all over the surface of the furniture element. So, we shall use another way of displaying the results. We shall count the numbers of pixels with true positive (TP) detections of skin and false positive (FP) detections of good erroneously classified as skin. For this purpose, the pixels marked with colors, like in Fig. 2c1 and c2, will be used. We shall see if it is possible to find a threshold for which the number of true positive skin detections is over zero in every image containing a defective object, and simultaneously the number of false positive detections in images containing a good object is small or zero.

A limited set of images can by no means be treated as complete. However, due to that all the available images, without selection, were considered, and in each of them as many pixels were marked for tests as reasonable, the tests can be considered as a sufficient demonstration of the viability of the method.

5 Results and Discussion

5.1 Light Conditions 1

We shall start the analysis from the image set for light conditions light1 because it seems that this set will constitute an easier problem to solve for the tested method due tu that the defect is easier to be seen in tangential light.

To use the directional detector it is necessary to set the angle at which the derivative will be calculated. In Fig. 4 it can be seen that the sensitivity of the detector to angle is not large. Therefore, the choice of the angle is not critical to the results. The light direction in the conditions light1 was $0^\circ$.

Fig. 4.

Numbers of detections of orange skin regions with the directional detector at threshold $T=300$, scale $\sigma =2.0$, in the set light2 for angles around the actual light direction $\alpha =0^\circ$.

Fig. 5.

Numbers of pixels above the threshold T in images of objects with orange skin and good objects, for conditions light1, detected with: (d) the directional detector, and (n) the non-directional detector, and for two scales: (1) $\sigma =2.0$, and (2) $\sigma =3.0$. Total numbers of properly detected defects at thresholds for which no false detections in good objects occurred are marked with circles. min Skin TP is the number of skin pixels in the image in which this number was the smallest. Vertical scale is logarithmic so zero is not displayed.

The graphs as described in the previous section for the series of test images are shown in Fig. 5. The graphs were obtained with the two detectors: the directional and the nondirectional one, as described in Sect. 3. The results for 100 thresholds spanning uniformly the whole range of outputs received from the detectors for all the images in the set were calculated. Only the significant parts of these results are shown.

As the threshold goes up, the numbers of detections decreases. The most interesting elements to notice are the threshold at which the number of false positives becomes zero, called the lower threshold, and the threshold at which the number of true positives in the image in which it is the smallest becomes zero, called the upper threshold. If the upper threshold is larger than the lower one, then there is a range of thresholds for which the detector works with perfect sensitivity and specificity, as far as the test images are considered. Such ranges are shown in Fig. 6. Absolute values of thresholds greatly depend on scale, so the indexes $i_T$ of the thresholds in the set of 100 thresholds, $i_T=0,1,\ldots ,99$, for a given series, are used, to put the graphs into a common scale.

Fig. 6.

Aggregated results for conditions light1: (a) bounds for the useful thresholds: the lower and the upper threshold (their indexes $i_T$): (b) total number N of skin true positives at the lower threshold, for scales $\sigma \in \{1.5,2.0,3.0,5.0,8.0\}$. In Fig. a the results for directional and nondirectional detectors are displaced from their respective scales for better visibility. In Fig. b there are no results of the nondirectional filter for $\sigma =8.0$ because the range of acceptable thresholds was empty.

In this Figure also the numbers of pixels found as true positives, for the given scales, is plotted. The width of the range of acceptable thresholds as well as this number of true positives can be treated as the quality measures of the detector.

Fig. 7.

Numbers of pixels above the threshold T in images of objects with orange skin and good objects, for light conditions light2, detected with: (d) the directional detector, and (n) the non-directional detector, and for two scales: (1) $\sigma =1.0$, and (2) $\sigma =1.5$. Vertical scale is logarithmic so zero is not displayed. Total numbers of properly detected defects at thresholds for which no false detections in good objects occurred are marked with circles.

The range of acceptable thresholds is larger for the nondirectional detector, and is the largest for $\sigma =2$ and 3. This is why the graphs in Fig. 5 were plotted for these particular scales. The thresholds can easily be set so that the outputs of the detectors perfectly fits the classification of the furniture elements tested.

The ranges of useful thresholds tend to have a maximum at some scale. For example, in Fig. 5a it is the largest for the nondirectional detector, for the scale $\sigma =3$ and it is $\langle 24,58\rangle$ which corresponds to not less than $41\pm 40\,\%$. It is important to check which scale is the most appropriate for the calculations. In any case, the scale should be matched to the resolution of the image.

The number of true positives grows with scale, but not in all the cases: in Fig. 6 it decreases between $\sigma =3$ and 5. This suggests that the number of detection goes down as the scale of the detector moves away from the optimum.

Fig. 8.

Aggregated results for conditions light2: (a) bounds for the useful thresholds: the lower and the upper threshold (their indexes $i_T$); (b) total number N of skin true positives at the lower threshold, for scales $\sigma \in \{1.5,2.0,3.0,5.0,8.0\}$. In Fig. a the results for directional and nondirectional detectors moved slightly to the left and right from their respective scales, for better visibility.

5.2 Light Conditions 2

For light conditions light2 the respective graphs are shown in Fig. 7. Also in this case it can be seen that there exist wide ranges of useful thresholds for all examples considered. These ranges are slightly wider for the non-directional detector than for the directional one, and wider for the scales $\sigma =1.0$ and 1.5 than for the other scales (Fig. 8). The ranges of acceptable thresholds for light2 are more narrow than in the case of light1. This could have been expected, as the images are now less contrasted. However, it is still possible to set such a threshold so that all the examples can be properly recognized, and that this threshold does not have to be set very precisely. For example, for the nondirectional detector it is the largest for the scale $\sigma =1$ and it is $\langle 35,58\rangle$ which corresponds to not less than $46\pm 23\,\%$. At $\sigma =1.5$ setting the threshold to $T=165$ would give no false positives in any good object, and over 1000 pixels of defects detected in all bad objects (at minimum, 38 bad pixels in an image. The threshold could be safely changed by at least $\pm 10$, which is at least $\pm 6\,\%$.

The results for both light conditions can be considered as very good, since they provide for both sensitivity and specificity equal to one in all the examples.

An example of results for the image with orange skin of Fig. 2g2 is shown in Fig. 9. The three thresholds were chosen from the optimum range, seen in Fig. 7n2, so that the number of false detections of good pixels as skin in any test image was zero (only pixels belonging to white as well as color masks were counted). The number of pixels truly found as defective is safely large, at least for the two lower thresholds.

The simple convolution is a quick operation, but its time grows with $\sigma ^2$. Therefore, it is profitable that good results can be obtained for small $\sigma$.

Fig. 9.

Example results of detection of pixels from class skin (marked in black) in the lower half of image from Fig. 2g1 for scale 1.5, nondirectional detector, at different thresholds T: (b) 140, (c) 165 and (d) 190.

6 Conclusion

Images of furniture elements having the orange skin surface defect were considered. As the detectors of the defective surfaces two simple image processing operations were used. The first one was the directional derivative of the image intensity along the light direction. The second one was the modulus of the image intensity gradient. Both detectors performed well enough to consider the method perfectly sensitive and perfectly specific for the tested 19 images, of which 11 contained the defect and 8 did not. This was observed for both the lighting with a light close to tangential to the surface and the light with a smaller incident angle. For the tangential light the range of acceptable thresholds was wider. The results depend on the scale at which the derivatives were taken, but it is easy to find an optimum scale. The nondirectional detector using the modulus of the gradient occurred to preform better than the directional one, so the method does not have to be trimmed according to the light direction.

The results obtained so far indicate that the defect of type orange skin can be easily detected even with not very advanced image processing methods.