A New Approach for License Plate Detection and Localization: Between Reality and Applicability

License Plate Detection and Localization (LPDL) is known to have become one of the most progressive and growing areas of study in the field of Intelligent Traffic Management System (ITMS). LPDL provides assistance by being able to specifically locate a vehicle’s number plate which is an essential part of ITMS, that is used for automatic road tax collection, traffic signals defilement implementation, borders and payments barriers and to monitor unlike activities. Organizations can deploy the number plate detection and recognition system to track their vehicles and to monitor each of them in their vital business activities like inbound and outbound logistics, find the exact location of their vehicles and organize entrance management. A competent algorithm is proposed in this paper for number plate detection and localization based on segmentation and morphological operators. Thus, the proposed algorithm it works on enhancing the quality of the image by applying morphological operators afterwards to segment out license plate from the captured image. No assumptions about the license plate color, style of font, size of text and type of material the plate is made of. The results reveal that the proposed algorithm works perfectly on all kinds of license plates with 93.43% efficiency rate.


Introduction
Security, one of the most resounding words in today's world, has influenced our lives (as individuals, groups, and organizations) in many different aspects.One such aspect involves surveillance of motor vehicles.For example, security includes detecting a stolen vehicle by identifying its number plate and automatically giving an alarm signal on its detection.Furthermore, number plate detection and recognition can be used in vehicle indexing and monitoring for anti-terrorism.Joarder et al. (2012) claimed that automatic number plate recognition is one of the important applications that can be used by the government, business' and persons to cope with for example the traffic of vehicles, control the traffic violation, parking lots management, organization entrance management, automatic toll collection enforcement, traffic law enforcement, border surveillance and stolen vehicle search.Moreover, it can be used in law enforcement, road toll data collection, weighing bridges, police barriers and blacklisted vehicles; in general, it can be applied in a number of enforcement applications (Munuo et al., 2014).
In the world of business, organizations can deploy the number plate detection and recognition system to track their vehicles and to monitor each of them in their vital business activities like inbound and outbound logistics, find the exact location of their vehicles and organization entrance management, which helps in reducing human resources costs and increasing revenue.
At present, the problem is well tackled for high resolution clean input images in developed countries like England, United State of America, Japan, Germany, France, Canada, and Australia see Figure 1, due to their standards in localization and recognition license plates.Consequently, existing working systems which are country explicit (Yu & Kim, 2000;Sarfraz et al., 2003;Hamey & Priest, 2005) use definite features found on car registration number like, foreground and background color, number plate size, border of plate and other standard features for localizing of the registration plate.However, in developing countries visual recognition of license plates is still the most used approach due to non-standardized and/or semi-standardized license plates see Figure 2.This study proposes an intelligent LPDL system for captured images without considering the license plate color, size, style of font, size of text and the types of plates used.The proposed algorithm acquires the vehicle's image from a digital camera.The captured image is improved by removing noises in a preprocessing stage through applying median filter.Contrast enhancement is applied on the filtered image to reduce the varying day lighting effect.The text and registration number background contrast is also improved through: Firstly, Sobel Edge detector is applied on the tested image (Saha et al., 2009).Secondly, maximum sum and variance of edges are calculated using proposed method that gives the location of the number plate.Finally, as a last stage, the proposed growing window algorithm is applied in order to remove the wrong number plate areas.For user convenience, the labeling of license plate area is made afterward.This paper is organized as follows: Section 2 presents the literature review.A proposed methodology is presented in section 3. Section 4 presents the experimental results.Section 5 presents the conclusions of this study.Theoretical contributions and practical implications are presented in Section 6. Section 7 presents the limitations and future studies.

Literature Review
License Plate Detection and Localization (LPDL) rests at the heart of any vehicle surveillance system.Various methods and techniques have been proposed to solve efficiently the LPDL system based on general feature of the number plates like shape, color, standard font and style, or by searching for a signature (Parker & Federl, 1996;Kim et al., 1999;Vichik et al., 1999;Ponce et al., 2000;Qadri & Asif, 2009).These methods and techniques are generally based on color information and special characters and symbols.Certainly, these approaches are ineffective and limited, especially if the registration number plates are printed with various styles, colors, font styles, signature patterns, aspect ratio, and/or with unspecified location of characters and symbols.A robust and efficient LPDL method is grounded on joint features, thereby distinguishing various types of license plates are applicable.Earlier methods (Laplacian, Sobel, Prewitt, Canny, and Roberts operators) which used such common features from edge detection (Ponce et al., 2000;Chanson & Roberts, 2001;Hsieh et al., 2002;Kim et al., 2002) have a major drawback which is that edge-based techniques cannot be applied to advance and composite applications in isolation, as they are extremely sensitive to undesirable edges.This would present a higher edge magnitude in addition or alternatively its variance.In spite of that, the unwanted edges are removed by combining the above process with some of the morphological techniques.In this combination method, the degree of disk extraction is comparatively greater and faster as compared to other various approaches.
Common image processing techniques such as Hough Transform, Top and bottom hat and Binary Morphology have also been proposed in literature (Gonzalez & Woods, 1992;Kamat & Ganesan, 1995;Martín & Borges, 2003;Cheng & Yahagi, 2004).All these techniques have the capability to find little entities of considerably varying brightness.However, the above defined algorithmic programs have approximately (80%) significant detection rate and therefore it is extremely obsessed with the displacement of the camera from the vehicle.This happens because the morphological procedures relates to the sizes of the binary entities, which are also computationally expensive.
The most popular morphology method is the Otsu algorithm (Cheng & Yahagi, 2004).This algorithm maximizes and exploits the between-class variance for searching the best threshold in a gray scale image.In fact, the objects and their backgrounds are mostly complex in real-world images, therefore it is not always possible to achieve required best threshold.While on the brighter side, the Otsu technique can be effortlessly extended to a version of multi-threshold, on the other side it greatly suffers from computational inefficiency, especially in case where the segmented clusters are quite large in number.
One of the important methods in binary imaging systems is the Connected Component Analysis (CCA) technique (Kwaśnicka & Wawrzyniak, 2002).CCA manages to scan a previously linearized image and then on the basis of pixel connectivity it labels its given pixels into connected components.Pixel connectivity can be one of two kinds e.g.either 4-connected or, 8-connected.After recognizing pixels, all of them are labeled by a value.The value is assigned based on the previously labeled component.The overall process of extraction, categorization and labeling of several disjoint associated components in an image is then extended to the phase of analysis where these are analyzed for possible number plate parts.A range of number plate text size is assumed depending on the prior knowledge but sometimes-random noise and other background objects fall in the range of assumed text size and are not removed.Besides, in other cases, number plate valid text might be eliminated as noise.Correspondingly, in our case the text size range cannot be limited.
RGA also known as Region Growing Algorithm (Van Heerden & Botha, 2010) uses a recursively region-growing procedure; along with some heuristics, dark areas which represent symbols of license plate, these symbols are enclosed by some lighter areas representing the license plate background, region dimensions and position, width and height (aspect ratio) and color and typestyle size.The limitations of RGA algorithm depend on prior knowledge of number plate height and width (aspect ratio), the background of the license plate should be lighter than the license plate text and the characters should be oriented in a straight line (non-skewed and damage number plates).
Projection method (Saha et al., 2009) is another LPDL technique, which has been quite effective.The method is computationally easy and computes the vertical and horizontal projections.Analysis of these projections then reveals the location of the license plate in the given image.This method assumes no prior information on the location and the type of text for the license plate.Though, this approach is limited and ineffective when dealing with the horizontal projection to find left right crop point for registration plates.The reason is that the movement from left to right in the horizontal projection gives strong edge count; however, the movement from the top to the bottom in vertical projection to count edges gives weak edge counts.
Moreover, the gaps between the characters may sometimes result in count equal to zero.This leads to multiple regions and wrong number plates region localizations as show in the Figure 8.
In this study, a method has been proposed to address the limitations of the projection method while preserving its benefits in low computational cost and no prior information.The proposed scheme is meant to be light weighted and inexpensive, hence it can be useful for real time implementation for recognizing number plates in different scenarios.

Proposed Methodology
This section presents in detail the ten steps approach utilized in the current study, which consist of the following steps: A

A. Image Acquisition
The researchers used two cameras one with low resolution and the other with HD resolution in order to achieve the first step "Image Acquisition".For low quality images, a camera with 2 MP and 320 × 240 resolution is used.Besides, for high quality images, a camera with 8 MP and 3264 × 2448 full HD resolution is used.As well, we download some images from license plate databases such as "MediaLab LPR database" (MediaLab, 2015) with mostly the same specifications as mentioned above.Thus, we obtained more than 350 rear and front different vehicle images with a different plate size, color, orientation, aspect ratio and distance for several days in a free unrestricted environment with continuously changing weather and lighting situations from different countries in RGB format.

B. RGB to Gray Scale Conversion
The images (input) are converted into a standard size of 704 × 576.The images taken by a digital camera are changed into gray scale in order to filter out color component that is chrominance UV.Different number plates have dissimilar color so this property is not used for license plate identification.To speed up the processing only luminance is stored and the following formula is stylized to find out the 8-bit gray scale value: C. Median Filtering Operation Zhao (2015) defines the median filtering as a nonlinear operation that swaps the picture element's gray value with the average of its neighbors gray values.Median filters are frequently used in image computation techniques to effectively reduce noise in the images such as salt and pepper noise.As the purpose is to eradicate noise (salt and pepper) and retaining edges.The median filter results are more effective when compared to the convolution process.Figure 3(b) shows the resulted image after RGB to gray scale conversion and median filtering.

D. Contrast Enhancement
The quality of gray-scaled images should be enhanced due to that; a histogram equalization is used (Saha et al., 2009).For contract stretching, a 256 gray level from 0 to 255 is utilized.The minimum gray level is zero (black) and the maximum gray level is 255 (white).This step helps in reducing varying day light effect and enhancing the contrast between the original image Figure 3(a) and the enhanced image Figure 3(d) (Asif et al., 2014).

E. Vertical Edge Detection:
Licence plates are printed horizontally; therefore, the vertical edges have the maximum edges count and row vice variance.As it appears in Figure 4(a), after a particular distance licence plates appear in a regular pattern and share the same height.Therefore, this common property can be used for identifying the number plate (Asif et al., 2014).A 3 × 3 Sobel edge operator mask (Saha et al., 2009) is used in the proposed methodology to find out the gradient edges in X direction of the contrast enhanced images Figure 4(b).(3)

F. Image Binarization
Image binarization is performed using proposed image binarization process utilising threshold, determined by the following formula (Asif et al., 2014): Where MVx = mean value of this  image.MVx is estimated without including non-zero elements (Thepade, 2014): The proposed binarization algorithm treats each vertical edge based on the power of the edge that is assigned depending on its occurrence location and on the closeness from the previous edge:

G. Row ROI Mask
In this step, we want to calculate the ℎℎ to create a mask using formula (6), so the value of the variance graph are scaled to zero (Asif et al., 2014).
The aim of this step is to apply the mask on the image in order to segment out initial regions of interest (ROI) as shown in Figure 5(a).The mask is then improved and the following ROI are revised, as presented in Figure 5(b):  ROI having height less than 15 pixels are removed.
 ROI having height greater than Image Height/4.5 are removed.
 ROI having approximately the same width and the distance between them is less than 15 points are merged.
In the subsequent step, each remaining ROI is treated independently.The canny edges have been detected and all of the horizontal edges have been removed as shown in Figure 5(c).Next, each edge's value is updated by a power depending on the occurrence distance from previous edge and location in the ROI segmented images by applying the following rules:  Edges that are not in the extreme left or right of the image gets higher power.
 Edges having the same height and occurrence distance less than 25 gets the maximum power.
 The Edges probability to be detected for the region of the number plate increases when move from the top to the bottom of the image.Row ROI is cropped out by finding complete zero rows to non-zero rows as TOP and the non-zero row to a complete zero rows is BOTTOM.
Some outputs of Row ROI stage are in Figure 6: Figure 6.Row ROI

H. Column ROI Mask
The next step after successfully getting row bar containing the number plate is to crop the exact location on the number plate and remove the extra left right regions.Indeed, to find a Left Right crop point horizontal projection method is mostly used (Saha et al., 2009).The horizontal projection method is not effective when compared to the vertical projection method which is used for finding Top and Bottom points see Figure 7(a).The reason for that, is when we move from left to right in the vertical projection to count edges, we get strong edges' count varying from 6 to 25 depending on the registration number of the vehicle see Figure 7(b).However, as we move from the top to the bottom horizontally to count edges we get a minimum count of two and a maximum count of six, additionally, the gaps between the characters' count identify as zero which leads to multiple regions and false number plates localizations as shown in the Figure 8. From Figure 8, for non-standardized and/or semi-standardized number plates, horizontal projection does not provide good results, if we connect the small gaps the results might be improved for the first two cases.Conversely, in the last two cases, it can be easily observed that by connecting the gaps (Kim et al., 2000;Enyedi et al., 2004) extra non-number plate regions and noises images are also added.Therefore, the aspect ratio of the image is changed and it does not remain in the valid range of the aspect ratio for the number plate.Therefore, some algorithms might discard a valid region area of the number plate.Another method used by (Kwaśnicka & Wawrzyniak, 2002) to find the number plate signature for a fixed width window box is also not applicable due to the non-standardized and/or semi-standardized number plate sizes.

I. Location of the License Plate
Our previous stage row ROI conforms the presence of the number plate region in the strap, a rectangular region segmented out, now the exact location of the number plate is founded in the segmented region.The row ROI is calculated as before but this time the threshold is changed to: Now each strap row is check out and a signal wave is formed by detecting 0 to 1 transaction of the signal.H/2 th row signal is showed in Figure 7(c).After the initial check windows width is calculated using the formula: The check length size window is moved on each row to find out maximum non-zero count and its location:

J. Growing Window Filter
As number plate region is cropped according to the check length size and since we are not dealing with standardized number plate for the same vehicle number plate; if the distance between the camera and the vehicle is changed the number plate captured size will also change; which results in incomplete number plate region.To fix this problem and to improve the results, we came up with a growing window filter method.Growing window method consists of two-stage process:  In the first stage, a fix window approximate equal to the length of number plate as shown in Figure 7(f).
 In the second stage the same process is repeated for the left side and the result of final cropped number plate localized is shown in Figure 7(g).
Figure 9 presents the results of growing window stages performed on different registration plates.The first column shows the result after initial cropped out number plate region.The second and third columns show the results achieved after window growing in right and in left directions respectively.
Figure 9. Results of growing window stages performed on different registration plates

Experimental Results
As discussed before, more than 350 number of true colour images and different resolutions captured from real scenes having variable distance, colour, text font and style, scaled, rotated, skewed, multi objects environment different illumination and weather conditions were selected for preparation of dataset.
The results of proposed algorithm as shown in Table 2 were obtained by using MATLAB software and a computer with Intel Core i7 and an 8 Giga Byte of RAM.

Conclusion
This study gives indicators for companies in order to adopt, accept and implement License Plate Detection and Localization system in their vital business activities and draws an algorithm to detect and locate the license plate region efficiently.Proposed algorithm contains ten major steps to find out Region of Interest (ROI).The proposed algorithm is tested by using 350 images of light and heavy traffic vehicles from different countries.Out of 350 number plates, the proposed algorithm detects 327 successfully and efficiently with 93.43% efficiency.Due to that, to achieve better performance the number plates should be clear and well captured.

Theoretical Contributions and Practical Implications
Without doubt, the proposed algorithm will be helpful for different countries even for government, business and individuals by detecting their vehicle registration numbers through utilizing the proposed algorithm.The proposed algorithm provides efficient and anticipated results and can be included as a part of the Intelligent Traffic Monitoring System (ITMS) used by motorway and Traffic controllers.
On a final note, the system will be helpful for example for traffic police to monitor any suspicious movements, properly manage traffic systems, border surveillance, stolen vehicles search, blacklisted vehicles, track and monitor vehicles in vital business activities such as inbound and outbound logistics, find the exact location of their vehicles and the organization of entrance management.This investigation can also be obliging in other academic and scientific fields like digital image processing and computer vision.

Limitations and Future Studies
In closing, the current study has its limitations.However, it has provided observations and also lends itself for more studies and research in the future.Firstly, the dataset used in this study consist of 350 images of number plates however, we recommend in future research to broaden the scope of the dataset.Finally, the current study used just only images conversely, more research on this topic "license plate detection and localization" needs to be adding more diversity to the dataset not only images but also videos and sounds.

Figure 3
Figure 3(a).Original image resized to 704 x 576 Figure 3(b).Gray scaled and median filtered image

For
The row wise sum and variance of the binarized image BiImgVx is calculated .The graphs of sum and variance of the edges are shown in Figure4(c) and (d) respectively.

Figure 8 .
Figure 8. Outputs of horizontal projection

For
Figure7(d) presents the waveform of S "sum of array" which is used to find (MaxCountIndex) maximum count and its location with the help of MaxCountIndex, number plate region is cropped out.Figure7(e) shows the initial cropped out number plate region.

Table 1 .
Strap detection row ROI results Note.LTV: Light Transport Vehicle, HTV: Heavy Transport Vehicle.

Table 2 .
Experimental results