Image perception hash and structural similarity fusion model for parking lot status recognition

Abstract In order to improve the recognition accuracy of parking space change, a new fusion model is proposed. The image-aware hash technique and image structure similarity algorithm are combined to construct a new parking space state combination discriminating index due to the accuracy of the recognition. Time is used to define the discriminating threshold of the parking space occupancy condition, so as to construct a vehicle position state discriminating fusion model. Based on the model, the parking lot recognition software was developed, and the parking space recognition of three environmental states, such as uniform illumination, uneven illumination and snowfall, was carried out and the validity of the model was verified.


Introduction
In order to better manage the parking lot, people began to apply advanced computer technology to the parking lot management system. The parking lot status recognition system came into being due to improve the ease of use and management efficiency of the parking lot.
Researchers have tried a variety of non-video status recognition and video detection methods to intelligentize the parking lot (Min, Juan, & Binghao, 2016). Video detection are more commonly Zhi-Fa Yang ABOUT THE AUTHORS Zhifa Yang is an associate professor and master tutor of the School of Transportation, Jilin University, and a PhD holder from Jilin University. His main research areas are vehicle operation safety technology, transportation system integrated energy conservation and emission reduction technology.
Shiwu Li is a professor at the School of Transportation, Jilin University, a doctoral tutor, and a PhD holder from Jilin University. His main research areas are driver behavior and vehicle operation safety technology.
Huan-jing Zeng, Yu-nong Wei and Xian-jun Fan are graduate students in the School of Transportation, Jilin University.Their main research areas are traffic environment and safety warning.

PUBLIC INTEREST STATEMENT
Parking a car has become an important part of urban life. Because of the complicated condition of the urban parking lot, different technology applied for recognize a parking lot is widely studied among researchers in varies field. The most popular method is using a camera to realize the identification of a outdoor parking lot. However, the method may not be accurate enough to be applied in real daily life due to the complicated environmental states, such as uniform illumination, uneven illumination and snowfall. Is there a method can recognize the parking lot as well as adapt to the different environments? This study proposes such an evaluation model. used in a parking lot detection nowadays (Huan, Feng, & Li, 2017). It is economical, convenient, and more universal.

Research object selection
Through a large amount of research and analysis, a parking lot in the Jilin University campus was selected as the research object of the thesis. The basic situation of the parking lot is as follows: there are 80 parking spaces, the parking lot area is about 1960 square meters, the number of parking rows is 5 rows, and the number of channels is 4(see Figure 1).

Detection area pretreatment
In general, the captured video will have a certain tilt angle due to the limitation of the shooting position. Therefore, before the acquired image is detected, it needs to be geometrically corrected to have an angle of 0°. The correction method used in this paper is affine transformation (Hao, Zhang, & Honghai, 2017).
When the angle of the image is being corrected, the overall area of all the parking spaces to be inspected is selected, and the next step is to separate each parking space. In this paper, two methods are used to extract the individual parking spaces, which are automatic division and manual division. The automatic division has a large interference with the setting of the threshold for judging the state of the parking space. Manually dividing the target area maximizes the difference in occupancy of the parking space, further improving the feasibility of the algorithm.
The purpose of image enhancement is to highlight important information contained in the image and suppress non-important information in the image. Commonly used image enhancement methods include histogram equalization, contrast broadening algorithm, dynamic range adjustment, and the like(Chen Ziyan, 2019). Among them, the histogram equalization is the most ideal for the scene processing that needs to be solved in this paper, because it can effectively suppress the influence of the bad environment on the parking space area. Therefore, this paper chooses histogram equalization as a method of image enhancement.
The image is filtered and denoised using a Gabor filter (Rui & Bing, 2012). Among them, the parameters are set to: 4 frequency scales, 4 directions. The image subjected to the enhanced processing is further processed(see Figures 2 and 3).

Image perception hash processing and structural similarity acquisition
The full name of the LBP operator is a local binary mode, and the texture feature is extracted by comparing the magnitude relationship between the gray value of any point in the gray image and the gray value of the neighboring point (Fei-gang, Wei-ming, & Ling, 2015).
The results of the free parking spaces and occupied parking spaces after LBP texture feature extraction are shown below (see Figures 4 and 5).
The similarities between two images can be easily recognized by using Hash algorithm and structural similarity algorithms (Chaowen, Feifei, & Qiu, 2020) (Ruishuang, Zeng, & Duo, 2019). However, if the two algorithms are directly processed on the acquired image, the accuracy of the recognition result cannot be guaranteed. In this paper, it treats the processed image to extract the texture features, eliminates the influence of illumination, and then combines the image perception Hash algorithm with the structural similarity algorithm to use the monotonicity to expand the occupancy status of the parking space, before acquiring the image similarity degree. The difference in the idle state is proposed, and the combined discriminant index Y is proposed to discriminate the state of the parking space.

Image perception hash processing
The obtained LBC texture feature map is hashed, and the detailed processing steps are as follows: (1) The size of the image.
(2) Simplify the color of the image and perform grayscale processing on the image.
(3) Discrete Cosine Transform (DCT): This transform is to further compress the image to be processed.   (4) Obtaining a hash sequence: Comparing the pixels in the region with the mean value of the DCT coefficients to obtain a hash sequence of the image(Wang Yanchao, 2018).

Acquisition of image structure similarity
The obtained LBP texture feature map is processed for structural similarity, structural similarity data is obtained and saved, and basic data is provided for setting the subsequent discriminant threshold (Song & Jianrong, 2011).

Design of the combined judgment model
It is assumed that the Hamming distance between different pictures is h. The smaller the Hamming distance h, the more similar the two pictures are (Christopher, Qijun, Huang Fenix, & Reidys, 2018). Since all pictures are subjected to discrete cosine transform when hash processing is performed, the hash value of each picture is 64 bits. Therefore, the Hamming distance varies from 0 to 64.
It is assumed that the structural similarity of the image is p, and the structural similarity p value is larger, the two pictures are more similar.
It can be seen from the above that the mathematical characteristic of Hamming distance h is monotonously decreasing, and the mathematical characteristic of structural similarity p is monotonously increasing. In order to make the monotonicity between the two consistent, the difference between the pictures is enhanced, after obtaining the image Hamming distance h value, the mathematical transformation is performed to make H = 64-h. When the H value is larger, The more similar the two pictures are. Let Y = H + p, that is, the larger the Y value, the higher the similarity between the two pictures.
When the parking space status is different, the Hamming distance and the structural similarity between the extracted detection area and the background template are different. Therefore, the Y value can reflect the different states of the parking space. By setting a reasonable threshold, the identification of the parking space status can be completed.

Determination of indicators
For a parking lot of Jilin University, three weather conditions were obtained, such as uniform illumination, uneven illumination and snowfall. The situation of the parking space is simplified as follows: the empty parking spaces and the vehicles on the parking spaces are white, black and gray. The number of samples collected in each case is 100, and then image hash processing and structural similarity calculation are performed to obtain basic data. In addition, it is considered that the range of variation of the acquired data may be affected due to the difference in the image reference template. Therefore, a total of three templates are selected(see Figure 6), which are selected as empty parking spaces with uniform illumination, uneven illumination, and snowfall. After processing, it was found that although the different templates would make a slight difference in the processing results, the fluctuation range was basically the same. Therefore, the selection of the template had no significant effect on the test results. Hereinafter, the case where the vehicle color in the parking space is white, black, and gray is simply referred to as a white vehicle, a black vehicle, or a gray vehicle.

The range of hamming distance h
The change of weather conditions has little effect on the Hamming distance of the empty parking space(see Figure 7), and the Hamming distance is basically between 20~30. When there are vehicles parked in the parking space, although the different weather conditions have an impact on the Hamming distance of different color vehicles, the range of Hamming distance varies from 20 to 35. Because of the large overlap of the Hamming distance of the vehicle on the parking space, a single image Perception Hash technique cannot accurately identify the parking space state.

The range of structural similarity p
The change of weather conditions has little effect on the structural similarity of empty parking spaces (see Figure 8), and the similarity range of empty parking spaces is basically between 50 and 80. When vehicles are parked on the parking space, although different weather conditions have an effect on the structural similarity of vehicles of different colors, the range of structural similarity of vehicles with parking spaces generally fluctuates between 30 and 50. Although the structural similarity algorithm distinguishes the presence or absence of vehicles on the parking space compared to the image perception Hash technique, the difference between the two is not obvious enough. The single-use structural similarity algorithm is used, and the detection effect is not very satisfactory.

The range of p/h
According to the correlation of Hamming distance and structural similarity between the similarity of the picture, the larger the p/h value is, the more similar the two pictures are. The range of variation of p/h is shown below(see Figure 9).   Under the condition of uniform illumination and uneven illumination, p/h can distinguish the state of the vehicle with or without the vehicle; however, the overlap of p/h between the two is large, and the distinction is not ideal in snowfall.

Combination judgment indicator y
Through the above test data analysis, a single Hamming distance or structural similarity cannot accurately determine the state of the parking space. In order to obtain a significant change law, let H = 64-h, and use the Hamming distance h and the structural similarity p to establish the discriminant index Y = H + p = (64-h)+p. The larger Y is, the more similar the two pictures are. The results are shown below(see Figure 10).
The range of the empty parking space Y is basically between 90 and 130. When the vehicle is parked on the parking space, although the different weather conditions have an influence on the combined discrimination index Y of different color vehicles, the combination determination index Y has a variation range of 60~80. It clearly distinguishes the presence or absence of the vehicle on the parking space, and can complete the recognition of the parking space status. Therefore, the determination threshold of the vehicle with the parking space can be set to Y ≤ 90.

Programming
The parking space recognition detection algorithm of this paper relies on the Qt platform and is jointly implemented by OpenCV (Xiaowen, Zhifang, & Weiwei, 2011), and a good GUI interface is established to display the detection results (Jintao & Chenghuan, 2009).
The program runs as follows: (1) The program starts running, and the camera obtains a video stream image of the parking space information of the parking lot.
(2) Target area division refers to manual calibration of parking spaces and background templates.
(3) Save the divided areas to be detected and the background template.
(4) Filter the parking space and the template to reduce the noise in the image. (5) Performing texture feature extraction on the filtered image, and processing by using the LBP operator.
(6) Image perception Hash and structural similarity processing on the image texture feature map, obtaining the Hamming distance h and the structural similarity p of the region to be detected and the image template respectively.
(8) Comparing the calculated Y value with the discriminating threshold value, when Y ≤ 90, the parking space state is determined to have a vehicle; otherwise, the parking space state is determined to be no vehicle.
(9) The obtained parking space information is displayed through the system operation interface to play a guiding role.

Test verification
In order to verify the reliability of the program, the test was verified for different weather conditions, that is, the situation of uniform illumination, uneven illumination and snowfall.
The processing results of the program under different weather conditions are as follows.

Uniform illumination
In the case of uniform illumination, the parking lot video is collected. This test tests whether the program can detect the result in time when the vehicle leaves the parking lot. The first thing to do is to build and save the model.
Three parking spaces are selected for this test. The initial state is: the parking space with the parking space number 1 is the empty parking space, and the parking space with the parking space number 2 and 3 is the occupied parking space. After the model is saved, the test starts(see Figure 11).
It shows the first frame of the video. According to the information displayed on the right side, the current total number of parking spaces is three, the number of vehicles parked is two, and the number of empty parking spaces is 1.
In the 180th frame of the video, the parking space with the parking space number 3 has a vacancy(see Figure 12). According to the information displayed on the right side, the total number of parking spaces is currently 3, the number of vehicles parked is 1, and the number of empty parking spaces is 1 and 3.
It can be seen from the detection results that the fusion model can accurately recognize the change of the parking space state when the illumination is uniform. Figure 11. Detection interface when the illumination is uniform (frame 1).

Uneven lighting and snowfall weather conditions
It can be seen from the test results that the fusion model can accurately identify the change of the parking space state under uneven illumination conditions and snowfall weather conditions. Since the algorithm verification process is similar, it will not be described here.

Conclusion
Aiming at the shortcomings of the existing parking lot management system, a comprehensive discriminant model based on the combination of image perception Hash and image structure similarity is constructed. The test proves that the model has good reliability and is robust to severe weather changes such as illumination changes and snowfall.
(1) The image perception Hash algorithm and the image structure similarity algorithm are effectively combined. On the basis of the mathematical transformation of the Hamming distance, the combination of the structural similarity is used to propose the combined discriminant index Y.
(2) By handling the variation range of Y of the parking lot parking space under different weather conditions, the parking space state determination threshold is set, that is, when Y ≤ 90, the parking space state is determined to have a vehicle; otherwise, the parking space state is determined to be no vehicle. And realized the recognition of the parking space.
(3) The reliability of the model and program was verified by processing the incoming and outgoing video of the parking lot under different weather conditions (uniform illumination, uneven illumination and snowfall).