Research on Character Computer Intelligent Recognition of Automobile Front Bar Bracket Based on Machine Vision

Aiming at the problem that the characters in the superimposed character area on the surface of the front bumper bracket of the car are difficult to recognize, a machine vision-based automobile front bumper bracket character recognition method is proposed, use Python, OpenCV and Halcon computer vision library for software system design. For eight images collected from different angles of light source directions, an innovative method of polynomial fitting gray value can reducethe uneven illumination of the images. The photometric stereo algorithm is used to obtain a high-contrast character image, and the separation of the two types of characters is simultaneously achieved. After the image filtering, opening and closing processes canremove background interference, use a morphological improvement algorithm based on scanning algorithm to complete character positioning, and then the improved algorithm of projection dichotomy based on the connected domain size is used to complete the character segmentation, and finally the support vector machine is used for character feature recognition. The experimental results claim that these methods can quickly separate superimposed characters and complete the recognition of non-color difference convex characters and inkjet characters, and the average recognition accuracy rate is over 96%, which meets the expected recognition requirements.


Introduction
The car front bumper bracket is one of the important parts of the car which is becoming an indispensable means of transportation for people's daily travel. There are some micro-convex non-chromatic characters on the surface of the car front bumper bracket, which express the important meaning of the product such as the specification and production batch. At the same time, these characters are also the unique identity code of each part, which has the important meaning of product anti-counterfeiting and product quality traceability. Therefore, the process of character recognition in the manufacturing process of parts is very important. At present, the manual recognition method is still commonly used, which is not only low in efficiency but also labor-intensive for workers and prone to errors in recognition.
At present, character recognition technology is widely used in many fields after a long development, but in industrial manufacturing, this type of character recognition has a greater market prospect due to 2. System overview The character recognition system of the car front bumper bracket includes image acquisition, image processing and character recognition. First of all, image acquisition is an important part of the character recognition, and the selection of suitable hardware can facilitate the subsequent algorithm design, and the image acquisition hardware system built is shown in Figure 1. The industrial camera of the system adopts the MV-EM200M face-format camera with a resolution of 1600X1200, and the industrial camera lens model is M1614-MP2. In order to eliminate the influence of black spray codes on the identification of non-chromatic character on the surface of the part at the overlapping characters, a K-WELL quadrant light source with stroboscopic light controller was selected for the light source hardware. Finally, an experimental inspection platform was built based on the experimental stand, and the industrial camera was co-centered with the quadrant light source, with the part located directly below the light source.  In order to separate the non-chromatic embossing characters in the overlapping area and recognize them, a new image pre-processing method was designed. The specific processing flow is shown in Figure 3.

Lighting method
The acquisition of images of non-chromatic embossing characters is more demanding than that of conventional flat characters, which differ in color from the image background. According to the knowledge of image processing, if light is diffusely reflected on the surface of the object, it indicates the presence of defects (embossing characters) on the surface. After considering the effect, the illumination method uses the dark lighting method, where the embossing characters area is brighter than the flat area after the light is diffusely reflected from the embossing characters area into the camera. As shown in Figure 4, with the lighting method of bar light source, the contrast of the character images captured by the industrial camera is poor, and it is obvious that the embossing characters in the image cannot be identified.  According to the idea of photometric stereo method [4], some images of the same object are obtained under different irradiation directions of the light source, and then the normal vector and reflectance of the surface are calculated, and finally the features with depth information are obtained. In the experiment, bar light source was used to irradiate from four different angles and located in the optimal irradiation direction. In the image, there were still obvious areas of uneven brightness due to the specular reflection of the surface, which would cause large errors in the calculation of the normal vector and reflectivity of the image. Therefore, in order to reduce the calculation error, we propose a lighting method using a quadrant light source and a stroboscopic light source controller, in which each light source in the quadrant light source can be lit individually and the industrial camera can shoot simultaneously at high speed, and a total of 8 images are acquired in two different irradiation directions. The lighting method of image acquisition is shown in Figure 5.
In the actual inspection environment, using the lighting method of quadratic light source, it is found that there are areas of difference in brightness on the surface of the image captured by the industrial camera. The ROI character region is obtained based on a priori knowledge, as shown in Figure 7(a). It is found that the further the measured surface is from the light source, the lower the intensity of the light source irradiation, and the lower the brightness. The uniformity of the image brightness has a large impact on the calculation of the normal vector, which directly affects the image processing results with certain errors. The intensity of light exposure to each pixel on the surface is reflected by the gray value in the gray image. The larger the gray value is, the stronger the light exposure is. In order to solve the above problem, we propose a polynomial fitting method for the surface gray value to make the surface brightness more uniform. Considering the comprehensive aspects of experimental equipment and the complexity of the higher-order polynomial model, the first, second and third order polynomial surfaces were used to fit the discrete gray pixel points. The general expressions are as follows:  In equations (1)- (3), (x, y) are the pixel coordinates of the detected surface area, and Z(x, y) is the gray value of the pixel. The mean square error between the predicted value of the fitted surface and the actual gray value of the surface is calculated when the pixel values of x and y coordinates and z coordinates of the image surface are fitted to the surface. The mean square error equation is as follows: (8) denotes the actual gray value of the image surface, and denotes the predicted gray value of the fitting surface model. After comparing the mean square error values of the first-, second-and thirdorder polynomial fitting surfaces, the third-order polynomial with the smallest error was chosen to fit the discrete points of the grayscale image to the model surface, and the fitted surface image is shown in Figure 6. The brightness of the right side of the image captured by the industrial camera is obviously higher than that of the left side, as shown in Figure 7(a). After using this polynomial fitting gray value method to optimize the difference in surface brightness, the brightness of the image surface becomes more uniform, as shown in Figure 7

Image processing of non-chromatic embossing characters
The industrial camera acquires eight images of characters with different lighting angles. The photometric stereo method is used to reconstruct the surface based on the difference of the luminance of the eight images. The luminance of the eight images are represented by , and the direction vectors of the eight light sources are: A linear system of equations with respect to the normal vector N and the surface reflectance k is established from the Lambert reflector equation as follows: Based on the known parameters substituted into equation (5) to form a system of overdetermined equations, the normal vector N obtained by the least squares method is used to solve for the gradients in the x and y directions of the surface: (11) To verify the feasibility of the above algorithm, eight images with eight different illumination directions of light sources were selected. The images were optimized using the polynomial fitting surface gray value method for the brightness of the character images, as shown in Figure 8. The gradient and reflectance maps of the surface of the car bracket were obtained by calculating these eight images with the above algorithm, as shown in Figure 9.   The complete characters cannot be extracted after the photometric stereo algorithm processing of the character images. Therefore, it is necessary to further scatter the gradient data to obtain the region of interest with depth information [5] and obtain the non-chromatic embossing character image of the car front bumper bracket with high contrast. The optimized result is shown in Figure 10(a) compared with the conventional image preprocessing algorithm, and the processed image is shown in Figure 10(b) using the algorithm. The algorithm has the following advantages: the image has a high contrast; and the separation of inkjet characters and embossing characters is achieved. Moreover, the algorithm can effectively suppress the effect of inkjet characters when processing embossing character images and then it can prepare for the next step of positioning non-chromatic embossing characters.
(a) Image enhancement map of traditional algorithm (b) Surface scatter map after gradient processing

Inkjet character image processing
In order to recognize embossing characters and inkjet characters at the same time, after obtaining images of non-chromatic embossing characters with high contrast, the histogram equalization algorithm was applied to the inkjet characters images for the purpose of image enhancement [6] . The inkjet characters after increasing contrast were obtained, as shown in Figure 11.

Non-chromatic embossing character positioning
After the image is grayscale processed, the pre-processed image pixels are transformed into black or white pixels by selecting the appropriate binarization threshold value. After morphological algorithms such as open operation and closed operation, the background interference points caused by uneven illumination or defects on the surface are removed, and then the character localization method is used to position the characters accurately. Zhang Tingman [7] realized precise character positioning by detecting the edge contour of license plate. Wei Yuke [8] used the difference value of adjacent pixels to locate the character region. The surface of the car front bumper bracket has no obvious features other than the two lines of characters, and the 12 characters in the second line need to be precisely located. None of the above methods can meet the requirements. In view of this, this paper proposes a morphological improvement algorithm based on the scanning algorithm, which can perform the precise positioning of the characters on the surface of the car front bumper bracket. The steps of the algorithm to achieve the precise positioning are as follows: 1. Based on the a priori knowledge, a larger area of characters of interest in the original image is selected as the candidate area to reduce the time for the subsequent algorithm to locate the characters, as shown in Figure 12(a).
2. After the image is binarized, the image is scanned line by line from top to bottom, and the number of alternating black and white pixel changes of characters is calculated as k after scanning one line, and since the target area of positioning is the 12 characters in the second line, whether k meets the condition is judged. 3. If k meets the above condition, it means that the line is scanned through the character area, and the number of lines that meet the condition is automatically added up. Similarly, the image is scanned vertically. Considering that the characters are skewed, a certain amount of margin is left in both vertical and horizontal directions after the scanning is completed to locate two lines of character areas, as shown in Figure 12(b).
4.The target character area of the second line is further located. Based on the dimensional characteristics of the two lines of characters, the morphological algorithm is used to form a connected domain with two smallest external rectangles of the two lines of characters. Then, we calculate the aspect ratio x of the connected rectangles and determine whether x meets the condition . If so, the second line of characters is precisely located, as shown in Figure 12

Inkjet character positioning
After finishing the precise positioning of non-chromatic embossing characters, the inkjet characters also need to be precisely positioned. After the binarization of the character image, the coarse positioning of the inkjet characters should be done according to the a priori knowledge, as shown in Figure 13(a). The inkjet characters are composed of discrete dots, and the morphological algorithms such as expansion, erosion, and closure operations are used to complete the adhesion of the dots of each character, as shown in Figure 13(b). Finally, the exact positioning of the inkjet characters is done by the row-scanning algorithm, as shown in Figure 13

Character segmentation
After locating the exact position of the character, the character is often skewed, so it is still necessary to rotate the character image to horizontal distribution in order to segment the character accurately in the next step. The character image is corrected by Hough transform [9] to meet the expected correction requirements. The experimental results are shown in Figure 14.   There are already mature methods for conventionalcharacter segmentation. Sun Xiaona [10] successfully segmented the characters on the detected surface with the segmentation algorithm based on vertical projection. The characters are segmented from the milk background image using a segmentation algorithm with color features [11] . However, the segmentation of concatenated characters cannot be completed by the above method. To address this problem, an improved projection dichotomy algorithm based on the size of the connected domain is proposed. The process of this method is as follows: first, divide each character into several connected fields; second, find the minimum outer rectangle of each connected field of the character and calculate the width dimension of each outer rectangle; third, define the standard width of a single character and the image width of a whole line of characters and introduce two threshold values as and : If the width of the outer rectangle of the connected domain is less than the threshold , the outer rectangle is determined to contain only one character. If the width of the outer rectangle of the connected domain is within the threshold value , it is determined that the outer rectangle contains two concatenated characters. Finally, the projection dichotomy method is used to segment the concatenated characters, and the flowchart is shown in Figure 15.  Figure 16(a). The best segmentation point for the "0" and "1"concatenatedcharacters is at the minimum value (the white character pixels are close to the projected value of 0). The results of the segmentation experiments for the concatenated characters are shown in Figure 16(b). This algorithm can accurately segment the concatenated characters compared with the vertical projection segmentation algorithm, as shown in Figure 17.

Character recognition and software design
Character recognition is an important part of character recognition system. At present, the commonly used methods for character recognition are template matching, neural networks and support vector machines [12] . Compared with the traditional character recognition methods, the support vector machine recognition method is similar to the forward neural network in some sense, and is more applicable and has better recognition accuracy for dealing with nonlinear and small samples. Therefore, support vector machine (SVM) classifier is selected for character recognition. However, before recognition, it is necessary to carry out feature extraction on the input character image to provide important guarantee for the accurate recognition of subsequent characters. Firstly, each character image is equalized and normalized by histogram, and then the dimension of each character image is obtained by feature extraction method of projection histogram and directional gradient histogram [13] . In practice, different kernel functions are introduced to achieve different recognition effects. The radial basis kernel function (RBF) was selected as the kernel function according to the characteristics of the characters on the surface of the car front bracket, and the parameters σ and penalty factor C of the kernel function were determined from the experiment.
The software interface of the character recognition system was developed and designed in Visual studio and Opencv environment for the convenience of customers and staffs to operate and study, as shown in Figure 19. The first menu bar of the software interface contains industrial camera settings, communication and algorithm threshold settings. The image display area can be used to view the status of the image acquired by the camera. At the top right of the software interface, there are buttons to start and stop detection, which control the on and off of the detection system. The display area below these buttons contains the results of photometric stereo, character positioning, character segmentation, and character recognition algorithms. The bottom right of the software interface contains the data of the inspection results, including the current total number of parts being currently tested, the number of passes and the pass rate, which can monitor the quality of products in real time.

Experimental analysis
In order to verify that the combination of the quadrant light source and photometric stereo algorithm can effectively separate the superimposed characters and obtain preprocessed character images with high contrast, 100 images were randomly selected as training samples in this experiment. Thirty images were randomly selected as test identification samples. The combination of the quadrant light source and photometric stereo algorithm, the combination of low-angle ring light and histogram equalization, and the combination of bar light source and gamma transform were used to complete thenon-chromatic character image acquisition and pre-processing of the car front bumper bracket, and the experimental results are shown in Table 1.  Table 1 shows that for the recognition of non-chromatic embossing character, the combination of the quadrant light source and photometric stereo algorithm has the best contrast of the pre-processed image. The combination of the quadrant light source and the photometric stereo algorithm can effectively remove the influence of black inkjet characters in the overlapping area and provide a strong support for the next step of character positioning.
Then, the accuracy and speed of the character recognition method using support vector machine (SVM) was compared with those of the character recognition method using template matching, and the experimental results are shown in Table 2. From the above Table 2, it can be concluded that the support vector machine is more accurate than the template matching method in character recognition, and the support vector machine method takes less time to recognize characters, which can better meet the conditions of high efficiency required for industrial production inspection, and can also increase the profit of the enterprise.

Conclusion
A method for automatic character recognition of car front bumper bracket based on machine vision is proposed to address the characteristics of superimposed characters. The quadrant light source is used to obtain eight images with different light angles, and then an innovative polynomial fitting gray value method and photometric stereo algorithm are used to obtain high-contrast character images and separate the superimposed characters. The morphological improvement algorithm based on scanning algorithm can achieve fast and accurate positioning of the character area. Then, the characters are corrected by Hoff transform and accurately segmented by an improved dichotomous algorithm based on the connected domain size. Finally, the character feature recognition is done by support vector machine. Experiments show that the proposed algorithm achieves 96% accuracy for such character recognition, and the detection system is stable, which is valuable for character recognition in industrial production. However, the character plane will be inclined after the irregular shape parts are placed in the recognition position, which will affect the lighting effect of the light source. This aspect is an important topic in the following research.