Vehicle Licence Number Plate Recognition Using Convolution Neural Network for Traffic Violators in Indonesia

— In the context of rising traffic violations and the need for efficient traffic management, this study explores the application of CNN in the recognition of licence plates to identify traffic violators in Indonesia. Traditional traffic enforcement methods are labour-intensive and prone to human error, necessitating a more automated and reliable approach. This research aims to enhance the accuracy and efficiency of license plate recognition (LPR) systems. The proposed system involves capturing vehicle images from the Roboflow Universe collected in the Malang area for use. We also use a CNN model to recognize and extract the alphanumeric characters from the plates. The CNN architecture is designed and trained on a comprehensive dataset of Indonesian licence plates, taking into account the unique characteristics and variations in plate designs specific to the region. The research we are doing is detecting number plates to reduce traffic violations. The method used for detection is the CNN method. The datasets used are primary and secondary. The precision, recall, and F1 score metrics further validate the system's reliability and robustness in real-world traffic scenarios. The implementation of this CNN-based LPR system promises a substantial improvement in monitoring and penalizing traffic violators, contributing to better traffic law enforcement and road safety in Indonesia. The accuracy for CRR is 82, and the accuracy for LPR is 85,33. The accuracy for CCR is 76.55 for precisions, 78.51 for recall, and 81.72 for F1 score. The accuracy for LPR is 81.20 for precision, 87.37 for recall, and 83.56 for F1 score.


I. INTRODUCTION
Indonesia has the 4th largest population in the world.As the last update based on BPS (Badan Pusat Statistik) Indonesia in January 2024, Indonesia boasted a population exceeding 279.390.258people, making it the fourth most populous country globally.The population has been steadily increasing, driven by factors such as high birth rates, vehicle growth, improved healthcare, and a growing economy.
Indonesia has a large and diverse population, which presents immense opportunities for economic growth and cultural exchange, but it also poses significant challenges [1].These include ensuring equitable access to education, healthcare, and employment opportunities across diverse regions and communities.Image processing methods are also used for several objects, such as research conducted by [2].
The spread of automobiles has profound implications for global economics, infrastructure development, and environmental sustainability.Automobile manufacturing became a cornerstone of industrial economies, driving job creation and economic growth.However, it also posed challenges such as congestion, air pollution, and dependence on fossil fuels.
The future of vehicle growth is marked by rapid technological advancements, evolving consumer preferences, and shifting regulatory landscapes.Electric and autonomous vehicles are expected to play increasingly prominent roles, transforming the way people commute and interact with transportation systems.
Traffic violations are breaches of the rules and regulations governing road use and safety.From minor infractions like parking violations to serious offenses such as reckless driving or driving under the influence, traffic violations encompass a wide range of behaviours that endanger road users and compromise the efficiency of transportation systems.
With the large number of violations committed by Indonesian people, a system is needed that can detect violations automatically.The government has tried many electronic ticketing systems with computer vision technology.However, there are still many shortcomings because many Indonesian people still use non-standard number plates.From this statement, the problem that often occurs in the field is that law enforcers still look for errors manually from recorded CCTV.This gives rise to errors in enforcement.With the existence of an automatic ticketing system, it is hoped that errors can be reduced.
Licence Plate Recognition technology continues to evolve, driven by advancements in image processing, machine learning, and hardware capabilities [3]- [5].Its wide range of applications and potential to improve efficiency and security make it a valuable tool in modern infrastructure.However, addressing challenges related to accuracy, environmental factors, and privacy is crucial for its successful implementation [6]- [8].
CNNs have revolutionized the field of image recognition, making them an integral part of modern LPR systems.Their ability to automatically extract features and classify characters with high accuracy has significantly enhanced the efficiency and reliability of licence plate recognition technologies.The research uses a CNN algorithm to classify the licence plate number by [9]- [12].The other research uses the CNN algorithm for classification by [13][14] using the R-CNN framework.The result of this research is that the R-CNN algorithm can be used to detect the vehicle.
Machine learning is used to detect the vehicle by [15]-[17] using the SIFT algorithm and HOG algorithm to detect the vehicle.However, the improvement for algorithm machine learning is deep learning.YOLO's real-time object detection capabilities make it a powerful tool for vehicle detection in licence plate recognition systems [18]- [21].Its ability to process images quickly and accurately helps in various applications like traffic monitoring, automated toll collection, and security surveillance.Despite some challenges, ongoing improvements and optimizations continue to enhance YOLO's performance and applicability in real-world scenarios [22].
This research uses primary data and secondary datasets.The dataset was taken from number plates around the Malang.For primary datasets, we take images from random vehicles around Malang City, whether cars or motorcycles in the parking lot.The secondary dataset is taken from the roboflow universe: https://universe.roboflow.com/.

II. RESEARCH METHODOLOGY
This research is divided into three main processes.The first is to explain the dataset the author used.Second is the process of how algorithms work for licence plate recognition.The last is the analysis of the result of how this algorithm works.The process is described in Figure 1.We use primary datasets and secondary datasets.For primary datasets, we take the image from the vehicle in Malang City.The author takes the secondary datasets from the universe roboflow universe dataset project.
2) Primary Datasets: For primary datasets, we take 150 images from random vehicles around Malang City.The datasets were taken from parking areas and several roads near Universitas Merdeka Malang.
3) Secondary Datasets: For primary datasets, we take 150 images from random vehicles around Malang City.The datasets were taken from parking areas and several roads near Universitas Merdeka Malang.

4) Algorithm:
This research algorithm is divided into several important stages, which are explained in Figure 2. 2) Pre-processing: The original image for primary datasets is taken from a mobile phone with a 3024 x 4032 pixels resolution with iPhone 11 Pro Max.The number of data sets that have been taken for primary data is 150 images.The secondary dataset was taken with Samsung SM-G990E with 2020 x 1514 pixels.The number of data sets that have been taken for secondary datasets is 820 images.The secondary datasets were downloaded from https://universe.roboflow.com/.The original image resolution is high the image should be resized and compressed to the required size.Resize the image to a standard size suitable for CNN input.The resolution after pre-processing is 640 x 640 pixels.
3) CNN Model: Design and train a CNN model for licence plate recognition based on the illustration in Figure 3.The CNN model typically consists of several convolutional layers followed by pooling layers and fully connected layers.Train the model using a dataset of labelled licence plate images.A Convolutional Neural Network (CNN or ConvNet) is a type of neural network specifically designed to handle data with a grid structure, like images.A digital image is essentially a binary representation of visual information composed of pixels organized in a grid.Each pixel has a value that indicates its brightness and colour.The kernel moves over the height and width of the image, creating a representation of that specific receptive region.This results in a twodimensional image representation called an activation map, which shows the kernel's response at each spatial position in the image.The distance the kernel moves with each step is known as the stride.Given an input of size of W x W x D and Dout kernels with a spatial size of F, stride S, and padding P, the output volume size can be calculated using Equation ( 1). (1)

8) Analysis:
Convolutional Neural Networks are a powerful tool in the field of machine learning and computer vision, excelling in tasks involving image and spatial data through their hierarchical feature learning and efficient parameter sharing.However, they require substantial computational power and data, and their performance is sensitive to hyperparameter choices.

III. RESULT AND DISCUSSION
The results were described in two sections.The first section explained how the algorithm worked, and the second section presented the results regarding the recognition of words from images and the accuracy of the results.

A. Pre-processing result
The pre-processing was done manually.We used Python to resize and crop the original image to the required dimensions.Additionally, we applied grayscale conversion to simplify the image data and reduce computational complexity.Noise reduction techniques, such as Gaussian blurring, were also implemented to enhance the quality of the images.Finally, we normalized the pixel values to ensure consistent input for the subsequent machine-learning model.

1) Sample datasets:
The sample of primary datasets is described in Figure 5.The primary datasets were captured by a mobile phone.The amount of primary data was 150 images.The secondary datasets are described in Figure 6.The amount of secondary data was 820 images.2) Image Resize: The primary and secondary data had various random resolution measures.Therefore, it was necessary to resize the images so that all images had the same resolution.Of the 150 primary data and 820 secondary data, resizing was carried out during pre-processing to a size of 640 x 640 pixels.This was done to ensure the process did not take too long.

B. CNN Algorithm result
These layers applied convolutional filters (also known as kernels) to input images.The filters slid over the input images to extract features such as edges, textures, and patterns.Pooling layers downsampled the feature maps produced by convolutional layers, reducing their dimensionality.Max pooling and average pooling were common techniques used for this purpose.Activation functions introduced non-linearity into the network, allowing it to learn complex patterns.Common choices included ReLU (Rectified Linear Unit), sigmoid, and tanh.Dense layers connected every neuron in one layer to every neuron in the next layer.They served to learn complex patterns and relationships in the data.Finally, dropout was a regularization technique used to prevent overfitting.It randomly dropped a proportion of neurons during training to reduce interdependence among them.

C. Licence plate localization
Plate localization was performed using YOLO v8 to detect licence plates.The results for YOLO v8 licence plate localization are described in Figure 7.

D. Character Recognition
Individual characters were segmented from the detected licence plates.This was done using image processing techniques or a segmentation model.

1) Convert to Greyscale:
The first step was to convert to greyscale.The results are described in Figure 8.

4) Recognize:
We used a CNN model for licence plate recognition.The results were obtained from datasets, and we tested our system with our own dataset to evaluate its performance.The CNN algorithm was used to classify the edges, resulting in text recognition of licence plates.The first sample result was "B 2196 T8R," and the second result was "N 1015 HE."The first image had an error where the number '5' was mistaken for '8', and the letter 'O' was mistaken for '8'.However, the second licence plate was successfully detected, as all characters were correctly recognized.

E. Post-Processing
Accuracy results for this research were described based on several methods.The first measure was the Character Recognition Rate (CRR), Plate Recognition Rate (PRR), Precision, Recall, and F1 Score.Our steps in analyzing the data involved preparing the data.We collected a dataset of images containing licence plates and their corresponding ground truth values (correctly labelled plates and characters).Next, we used the LPR system to recognize licence plates and characters in the dataset.We compared the recognized results with the ground truth values to determine the number of correctly and incorrectly recognized characters and plates.Then, we used the formulas mentioned above to calculate CRR, PRR, precision, recall, and F1 score.

1) Character Recognition Rate (CRR):
The percentage of individual characters correctly recognized by the LPR system served as a crucial metric for evaluating its accuracy.Consistently high recognition rates indicated the robustness of the system in real-world applications.Our result for correctly recognized characters was 2957 out of a total of 3570 characters, resulting in an accuracy rate of 82.82% using Equation ( 2). (2) 2) Plate Recognition Rate (PRR): The percentage of entire licence plates correctly recognized by the system reflected its overall effectiveness and reliability.Achieving high accuracy in this metric was essential for practical deployment, ensuring that the system performed well under various conditions and scenarios.Our result for correctly recognized licence plates was 129 out of a total of 150 characters, resulting in an accuracy of 85.33% using Equation (3). (3) 3) Precision, Recall, and F1 Score.: The Equation for accuracy, Recall and Precision using on Equation ( 4) to (6), respectively.True positive (TP) shows that the prediction result and target are positive, whereas if the prediction results and target are negative, it is well-known as True negative (TN).If the target is negative, but the prediction result produces a positive, it is called a False positive (FP).Otherwise, it is named a False negative (FN) (4) (5) The results of Precision, Recall, and F1 score were described in Table I and Figure 11.IV.CONCLUSION The conclusion of the research is the use of Convolutional Neural Networks (CNNs) for licence plate recognition to identify traffic violators in Indonesia.Traditional traffic enforcement methods are labour-intensive and susceptible to human error, highlighting the need for a more automated and reliable solution.By harnessing the capabilities of CNNs, renowned for their exceptional performance in image recognition, this research aims to improve the accuracy and efficiency of licence plate recognition (LPR) systems.
The proposed system involves capturing vehicle images through traffic cameras, pre-processing these images to enhance the clarity of licence plates, and then employing a CNN model to recognize and extract alphanumeric characters from the plates.The CNN architecture is specifically designed and trained on a comprehensive dataset of Indonesian licence plates, accounting for the unique features and variations in regional plate designs.Experimental results show a high Character Recognition Rate (CRR) of 82.82% and a Plate Recognition Rate (PRR) of 85.33%, significantly surpassing traditional LPR methods.Precision, recall, and F1 score metrics further confirm the system's reliability and robustness in real-world traffic conditions, with CRR achieving 76.55% precision, 78.51% recall, and 81.72% F1 score, and LPR attaining 81.20% precision, 87.37% recall, and 83.56% F1 score.The implementation of this CNN-based LPR system promises substantial improvements in monitoring and penalizing traffic violators, thereby enhancing traffic law enforcement and road safety in Indonesia.

Figure 1 .
Figure 1.Main Process 1) Datasets Acquisition: This research uses two main datasets.We use primary datasets and secondary datasets.For primary datasets, we take the image from the vehicle in Malang City.The author takes the secondary datasets from the universe roboflow universe dataset project.

Figure 2 .
Figure 2. Flowchart Proposed Method 1) Dataset Retrieval: Obtain the input image containing the vehicle with the licence plate.The primary dataset is taken from local vehicles in Malang City.The licence plates were taken randomly from cars and motorcycles.

Figure 3 .
Figure 3. CNN Model 4) Licence Plate Localization: Apply the trained CNN model to the input image to detect the region containing the licence plate.This step might involve techniques like sliding window detection or more sophisticated object detection methods.This process localization is using Yolo v8.Figure 4 explains Licence Plate Recognition.If the licence plate cannot detect, the algorithm can't continue to the next step.
Figure 4 explains Licence Plate Recognition.If the licence plate cannot detect, the algorithm can't continue to the next step.

Figure 4 .
Figure 4. Yolo Detection 5) Character Recognition: If the licence plate is not segmented from the rest of the image in the previous step, segmentation is performed to isolate it.Techniques like connected component analysis or contour detection can be used.Pass each segmented character through the trained CNN model to recognize the characters.The CNN model should be trained to classify each character (0-9 and A-Z) based on the segmented character images.6) Post-processing: Combine the recognized characters to form the complete licence plate number.Apply any additional post-processing steps like spell-checking or verification against a database of valid licence plate numbers.

7 )
Output: Output the recognized licence plate number along with any relevant information extracted from the image.

Figure 8 .
Figure 8. Convert to Grey2) Blurring Uses Gaussian Blur: The next step was to apply Gaussian blur.The results are described in Figure9.

Figure 9 .
Figure 9. Gaussian Blur 3) Canny detection: We used Canny edge detection to detect edges.The results are described in Figure 10.