Fast Target Recognition Based on Improved ORB Feature

Abstract

A novel fast target recognition algorithm is proposed under the dynamic scene moving target recognition. Aiming at the poor matching effect of the traditional Oriented Fast and Rotated Brief (ORB) algorithm on underexposed or overexposed images caused by illumination, the idea of combining adaptive histogram equalization with the ORB algorithm is proposed to get better feature point quality and matching efficiency. First, the template image and each frame of the video stream are processed by grayscale. Second, the template image and the image to be input in the video stream are processed by adaptive histogram equalization. Third, the feature point descriptors of the ORB feature are quantized by the Hamming distance. Finally, the K-nearest-neighbor (KNN) matching algorithm is used to match and screen feature points. According to the matching good feature point logarithm, a reasonable threshold is established and the target is classified. The comparison and verification are carried out by experiments. Experimental results show that the algorithm not only maintains the superiority of ORB itself but also significantly improves the performance of ORB under the conditions of underexposure or overexposure. The matching effect of the image is robust to illumination, and the target to be detected can be accurately identified in real time. The target can be accurately classified in the small sample scene, which can meet the actual production requirements.

1. Introduction

As a new subject, machine vision technology has been gradually integrated into people’s lives in recent years. Machine vision has made great progress in both technical and application fields. Target recognition is a typical application of machine vision technology in the industry. In the industrial field, industrial robots with machine vision technology have gradually begun to replace traditional robots. How to identify the workpiece accurately is the focus and difficulty of a vision robot [1,2,3]. It is also the basis of the grasping operation of a vision robot. In 1999, David Lowe put forward the scale invariant feature transform (SIFT) [4] feature matching algorithm based on the existing feature detection methods based on invariant technology SIFT, which has scale invariance. The algorithm is aimed at local feature points of objects. Image size transformation and rotation will not affect the detection results, and it has high robustness to noise and other influences. Bay et al. [5] proposed a fast robust feature extraction registration algorithm Speed-Up Robust Features (SURF) in 2006 based on the SIFT algorithm. Due to the harr feature and the introduction of the integral image, SURF outperforms the SIFT algorithm and can get faster speed.

Rosten et al. [6] proposed the corner in 2006, which can quickly determine the location of feature points. Calonder et al. put forward the brief descriptor to describe the image area around feature points [7]. Rublee et al. improved the fast corner and brief descriptor and proposed an ORB feature that can effectively replace SIFT and SURF [8]. The ORB algorithm appropriately reduces the accuracy and robustness of feature points, improves the calculation speed, and reduces the calculation time, which is a good compromise between the quality and performance of different types of feature points [9].

Due to the good performance of the ORB characteristics [10,11,12,13,14,15], many scholars at home and abroad have made different improvements to the ORB algorithm [16,17,18,19]. Hong et al. [20] matched the ORB feature point matching algorithm and eight parameters and combined the rotation model, improving the detection speed of the feature point; and Bing et al. [21] improved the rotation of the ORB algorithm in the feature point matching algorithm of the ORB, which enhanced the matching accuracy; but for special scenes, the study of the ORB algorithm is rare under the conditions of poor illumination conditions. For the above questions, in this paper, the template image in the video stream and the image to be input are first processed for grayscale. The second step is to perform adaptive histogram equalization processing on the image to improve the image quality of the input image and the template image. The third step is to extract the feature description through the ORB algorithm. Finally, the KNN matching algorithm is used. By comparing the distance between the optimal feature matching and the suboptimal feature matching, a reasonable ratio is set to eliminate the mismatch. Compared with the traditional ORB matching algorithm, the improved ORB matching algorithm can significantly improve the matching performance under poor lighting conditions. Finally, by comparing the logarithm of correct matching between the image features to be input and all template image features, a reasonable threshold is set, which effectively realizes the target classification of small samples.

2. ORB Algorithm Principle (Orient Fast and Rotated Brief)

Oriented Fast and Rotated Brief (ORB) is based on the famous fast feature detection and brief feature descriptor.

2.1. Feature Point Detection

Image feature points can be simply understood as more important points in the image, such as contour points, high spots in dark areas, and dark spots in bright areas. The ORB algorithm uses a fast algorithm [22,23,24,25] to find feature points. The core idea of fast is to find out those outstanding points, that is, to compare a point with the surrounding points, and if it is different from most of the surrounding points, it can be selected as a feature point [26].

$$N={\displaystyle \sum _{x\forall (circle(p))}\left|I(x)-I(p)\right|}>{\epsilon }_d$$

(1)

where $I(x)$ is the grayscale of any point on the circumference, $I(p)$ is the grayscale of the center, and ${\epsilon }_d$ is the threshold value of the gray value difference. If N is greater than the given threshold value, which is generally three-quarters of the surrounding circle points, P is considered as a feature point.

The specific calculation process of fast is as follows: First, select a pixel point P from Figure 1 and judge whether it can be used as a feature point. It is assumed that its gray value is M. Set a suitable threshold value V (for example, 20% of M). When the absolute value of the difference between the gray values of two points is greater than V, then these two points can be considered to be different. Then, taking the pixel point P as the center, select 16 pixels on a circle with a radius of 3. P can be regarded as a corner point if the gray levels of consecutive L points in these 16 pixels are all larger than M + V or smaller than M − V. Here, L is set to 12 and if at least 12 points exceed the threshold, P is considered as a feature point, Otherwise, it is considered that P is not a feature point. To obtain faster results, additional acceleration measures have also been adopted. If four points around the candidate point are tested at 90-degree intervals, at least three points should have enough difference with the gray value of the candidate point; otherwise, the candidate point should not be considered as a feature point without calculating other points. Figure 1 is a schematic diagram of fast feature point extraction.

Because the fast algorithm cannot detect the direction information of feature points, in the ORB algorithm, the method of establishing a three-layer Gaussian image pyramid is used to increase the scale invariance [27] and the gray centroid method [28] is used to endow feature points with rotation invariance. That is, the coordinate system is established with feature points as the origin, the centroid position is calculated in the neighborhood $S$, and the vector is constructed with feature points as the starting point and the centroid as the endpoint. The moment of neighborhood $S$ is

$$M_{pq}={\displaystyle {\sum }_{x,y}{x}^p{y}^{q}I(x,y)}$$

(2)

where $I(x,y)$ is the gray value of the image, $x,y\in [-r,r]$, $r$ is the radius of the feature point neighborhood, and $S$ is the centroid position of the neighborhood:

$$C=(m_{10}/m_{00},m_{01}/m_{10})$$

(3)

The orientation of feature points of fast is

$$\theta =\arctan (m_{01},m_{10})$$

(4)

To improve the rotation invariance of feature points, it is necessary to ensure that $x$ and $y$ are in the circular area of radius $r$, that is, $x,y\in [-r,r]$. $r$ is the neighborhood radius.

2.2. Calculate Feature Point Descriptors

ORB uses the improved brief algorithm to calculate the descriptor of a feature point and solves the primary defect that brief itself has no rotation invariance. Its core idea is to select $N$ point pairs in a specific pattern around feature points $P$ and combine the comparison results of these $N$ point pairs as descriptors.

The brief descriptor is simple and fast, which is based on the idea that the image neighborhood can be expressed by a relatively small amount of intensity contrast.

Define the criterion $\tau$ of the $S\times S$ image neighborhood P:

$$\tau (p;x,y)=\{\begin{array}{l}1,p(x)<p(y)\\ 0,others\end{array}$$

(5)

Here, $p(x)$ is the pixel intensity at the point of the $\mathrm{x}={(u,v)}^T$, which in the image is spot $P$ after the filtering process. Choose ${\mathrm{n}}_d$ and the $(x,y)$ position pair, which uniquely defines the binary criterion. The brief descriptor is a binary bit string of ${\mathrm{n}}_d$ dimensions.

$$f_{nd}(p):={\displaystyle \sum _{1\le i\le n}{2}^{i-1}}\tau (p;x,y)$$

(6)

The pixel value can be 128, 256, 512, etc. Select different values to weigh between speed, storage efficiency, and identification.

The criterion of image neighborhood in the brief considers only a single pixel, so it is sensitive to noise. To solve this defect, each test point in ORB uses a 5 × 5 sub-window in the neighborhood of 31 × 31 pixels, in which the selection of the sub-window obeys Gaussian distribution and then uses the integral image to accelerate the calculation.

The brief itself is undirected and has no rotation invariance. The solution of ORB is to try to add a direction to the brief. At position $(x_i,y_i)$, for any $n$ binary criterion feature set, defining a $2n$ matrix:

$$S=\left(\begin{array}{l}{x}_1,x_2,\cdots ,x_n\\ y_1,y_2,\cdots ,y_n\end{array}\right)$$

(7)

Using the neighborhood direction $\theta$ and the corresponding rotation matrix $R_{\theta }$, build a corrected version of $S$, $S_{\theta }:S_{\theta }=R_{\theta }S$. So the Steed Brief descriptor is

$$g_n(p,\theta ):=f_{nd}(p)|(x_i,y_i)\in S_{\theta }$$

(8)

After getting Steered brief, use greedy search [29], finding 256-pixel block pairs with the lowest correlation from all possible pixel block pairs, that is, obtaining the final brief. Figure 2 is a schematic diagram of the descriptor calculation.

2.3. The Flowchart of Basic ORB Algorithm

According to the principle of the ORB algorithm, we get the flow chart shown in Figure 3 below:

3. Feature Extraction and Matching Based on an Improved ORB Algorithm

The traditional ORB algorithm has a poor matching effect on underexposed or overexposed images caused by illumination and has a certain sensitivity to image noise. Before matching, the template images and the images to be matched in the video stream are grayscale processed to filter out the noise. On this basis, the template image and the input image are processed by adaptive histogram equalization, which not only increases the number of feature points of the input image and the template image but also increases the number of correctly matched feature points.

3.1. Histogram Equalization

When the gray value distribution of the image is too concentrated, histogram equalization [30,31,32,33] can make the gray probability distribution of the image uniform and make the histogram of the image as stable as possible. Transform as follows:

$$s=T(r)$$

(9)

The relationship between the transform function $T(r)$ and the original graph probability density function is:

$$s=T(r)={\displaystyle {\int }_0^r{p}_r(r)dr}$$

(10)

The discrete forms are as follows:

$$\begin{array}{l}{s}_k=T(r_k)={\displaystyle \sum _{i=0}^k\frac{n_j}{N}=}{\displaystyle \sum _{i=0}^k{p}_r(r_j)}\\ (0\le r\le 1,k=0,1,2\cdots ,L-1)\end{array}$$

(11)

After many transformations, the equalized image is obtained.

3.2. Adaptive Histogram Equalization

Due to shooting or environmental problems, there may be uneven image brightness, low contrast, high noise, and so on. The features after matching with the original image are concentrated in the areas with high contrast, while the features extracted from other areas are relatively few, so the obtained feature points cannot describe the whole image. The histogram distribution of the original image is shown in the upper-left picture and the bottom-left picture in Figure 4. Although ORB feature matching has a good matching effect under good illumination, the matching effect will be greatly reduced under insufficient illumination or overexposure.

Adaptive histogram equalization (AHE) is a computer image processing technology used to improve image contrast. Different from the ordinary histogram equalization algorithm, the AHE algorithm changes the image contrast by calculating the local histogram of the image and then redistributing the brightness. Therefore, the algorithm can improve the local contrast of the image and obtain more image details.

After adaptive histogram equalization, the details of the dark part of the original image become clearer and the candidate points with a higher Harris response value change during feature point detection, so high-quality matching point pairs in other areas of the same image can be obtained. After adaptive histogram equalization, the histogram distribution is shown in the upper-right picture and the bottom-right picture in Figure 4.

3.3. KNN Matching Algorithm to Eliminate Mismatching

The core idea of the K-nearest-neighbor (KNN) algorithm [34,35,36,37,38,39,40,41] is to search the top $K$ feature points with the highest similarity in another feature space as candidate features.

In this paper, the $K$ = 2 KNN algorithm is adopted to find the ratio $R$ between the distances of the nearest neighbor and the second-nearest neighbor:

$$R=\frac{D(V_p,V_{mq})}{D(V_p,V_{nq})}$$

(12)

where $V_p$ is the feature vector of the feature point p, $V_{mq}$ is the feature vector of the nearest neighbor $q$ in an image, $V_{nq}$ is the feature vector of the second-nearest-neighbor feature point in an image, and $D$ is the distance between the vectors.

For each feature point, the optimal feature matching and sub-optimal feature matching are obtained and the distance between them is recorded as m and n, respectively. Each feature point matching pair is screened according to the ratio $T$ (0.7 in this paper), and matching with the excessive ratio is considered as mismatching.

Based on the above, this paper puts forward the idea of combining the image enhancement technology with the ORB algorithm and uses adaptive histogram equalization to process the input images in advance. The experimental results show that the ORB algorithm combined with the image enhancement technology improves feature extraction and matching. The specific process is shown in Figure 5 below.

4. Experimental Results and Analysis

Under natural conditions, illumination has a great influence on collected images and different illumination conditions may cause underexposure or overexposure to affect the quality of images, which brings inconvenience to subsequent matching research. This paper re-distributes the brightness of the image by adopting the adaptive histogram equalization, reducing the influence of the light on the input picture, and the advantages and disadvantages are measured by the number of feature points, the number of matching points, and the running time.

To verify the feasibility of the ORB algorithm, this paper compares the traditional ORB algorithm with the improved ORB algorithm, and the experimental environment is Pycharm 2021 and OpenCV 4.5.2. The recognition target map is a common book in the laboratory, and this paper will verify the two groups of images. The first group of pictures is an overexposed image experimental scene with a size of 523 × 481 in PNG format (Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10). The second group is an underexposed picture experiment scene with a size of 526 × 489 in PNG format (Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15). The experimental results of the ORB algorithm and the ORB algorithm based on image enhancement are compared respectively.

A comparison of the data of

Table 1. Matching data of overexposed images.

Method	Feature Points of the Left Graph	Feature Points of the Right Graph	Matching Points	Running Time (ms)
Traditional ORB algorithm	3067	3112	734	304
SIFT algorithm	722	650	297	109
Improved ORB algorithm	5468	5519	1066	372

and

Table 2. Matching data of underexposed images.

Method	Feature Points of the Left Graph	Feature Points of the Right Graph	Matching Points	Running Time (ms)
Traditional ORB algorithm	2475	2315	210	273
SIFT algorithm	376	346	165	87
Improved ORB algorithm	3130	2866	282	295

shows that the improved ORB algorithm not only maintains the superiority of the ORB itself but also improves the matching effect under conditions of insufficient exposure or excessive exposure, where under excessive exposure conditions, the characteristic points increased by 78% and the matching point increased by 45%, and under the conditions of insufficient exposure, the characteristic points increased by 23% and the matching point increased by 34%. The effectiveness of the algorithm was explained by the above experiments and comparison.

5. Conclusions

Due to problems such as shooting or environment, there may be uneven image brightness, low contrast, and noise. The features after matching with the original image are concentrated in the areas with higher contrast, while the features extracted from other areas are relatively few. The resulting feature point cannot describe the entire image. Although the matching effect is good in the case of light, the matching effect is greatly reduced when the light is insufficient or the exposure is excessive, and even a false match may occur. This paper proposes an improved ORB feature extraction algorithm by combining image enhancement techniques and ORB algorithms. Adaptive histogram filtering algorithms, by calculating the local histogram of the input image and then re-distributing the brightness to change the input image contrast, improve the local contrast of the input image and obtain more image details. The results show that the improved ORB algorithm maintains the superiority of ORB itself and significantly improves the matching effect under the conditions of underexposure or overexposure.