Skip to main content
SpringerLink
Log in

A Framework for Designing Unsupervised Pothole Detection by Integrating Feature Extraction Using Deep Recurrent Neural Network

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

One of the significant causes of road accidents is the presence of potholes. In order to overcome this difficulty, several techniques have been designed, from manual reporting to authorities taking steps for auto-detection of pothole regions. Traditional techniques related to pothole detection fail in terms of risk during weather variations, high setup cost and no provision for night vision due to the lack of development in effective automatic pothole detection models effectively. The main objective of this work  is to design an automatic pothole detection model for identifying potholes at the earliest time period. To make the processing more manageable and improvise the detection performance, an optimized deep recurrent neural network (ODRNN) is proposed. The pothole detection model consists of three phases, pre-processing, feature extraction and unsupervised classification. Initially, the the set of images are gathered to make a dataset. The first phase is responsible for image enhancement which performs input image resizing, background noise removal utilizing median filter approach and RGB to grey scale conversion. The second phase is responsible for extracting the most relevant characteristics of the pothole region using Shape-based feature extraction approach. In the final phase, the extracted features are classified with ODRNN. To enhance the efficiency of conventional RNNs, the weight values of the classifier are optimized using the Improved Atom search optimization algorithm (IASO). It classifies the pothole and non-pothole region with a lesser error function, implemented and tested experimentally in terms of accuracy, recall, precision and error performance. This work provides better performance with a maximum accuracy of 97.7% to become a better strategy for pothole detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

There is no availability of data or materials available or report for the manuscript.

Code Availability

No code is available for this manuscript.

References

  1. Fan, R., Ozgunalp, U., Hosking, B., Liu, M., & Pitas, I. (2019). Pothole detection based on disparity transformation and road surface modeling. IEEE Transactions on Image Processing, 29, 897–908.

    Article  MathSciNet  MATH  Google Scholar 

  2. Sathya, R., & Saleena, B. (2022). CNN-MAO: Convolutional neural network-based modified aquilla optimization algorithm for pothole identification from thermal images. Signal Image and Video Processing. https://doi.org/10.1007/s11760-022-02189-0

  3. Kang, B. H. & Choi, S. I. (2017, July). Pothole detection system using 2D LiDAR and camera. In 2017 Ninth international conference on ubiquitous and future networks (ICUFN), IEEE, (pp. 744–746).

  4. Tsai, Y. C., & Chatterjee, A. (2018). Pothole detection and classification using 3D technology and watershed method. Journal of Computing in Civil Engineering, 32(2), 04017078.

    Article  Google Scholar 

  5. Li, S., Yuan, C., Liu, D., & Cai, H. (2016). Integrated processing of image and GPR data for automated pothole detection. Journal of computing in civil engineering, 30(6), 04016015.

    Article  Google Scholar 

  6. Bhatt, U., Mani, S., Xi, E. & Kolter, J. Z. (2017) Intelligent pothole detection and road condition assessment. http://arxiv.org/abs/1710.02595.

  7. Hoang, N. D. (2018). An artificial intelligence method for asphalt pavement pothole detection using least square support vector machine and neural network with steerable filter-based feature extraction. Advances in Civil Engineering. https://doi.org/10.1155/2018/7419058

    Article  Google Scholar 

  8. Fox, A., Kumar, B. V., Chen, J., & Bai, F. (2017). Multi-lane pothole detection from crowd sourced under sampled vehicle sensor data. IEEE Transactions on Mobile Computing, 16(12), 3417–3430.

    Article  Google Scholar 

  9. Wu, C., Wang, Z., Hu, S., Lepine, J., Na, X., Ainalis, D., & Stettler, M. (2020). An automated machine-learning approach for road pothole detection using smart phone sensor data. Sensors, 20(19), 5564.

    Article  Google Scholar 

  10. Anaissi, A., Khoa, N. L. D., Rakotoarivelo, T., Alamdari, M. M., & Wang, Y. (2019). Smart pothole detection system using vehicle-mounted sensors and machine learning. Journal of Civil Structural Health Monitoring, 9(1), 91–102.

    Article  Google Scholar 

  11. Naveen, N., Yadav, S. M., & Kumar, A. S. (2018). A study on potholes and its effects on vehicular traffic. International Journal of Creative Research Thoughts, 6(1), 258–263.

    Google Scholar 

  12. Ng, A., & Loosemore, M. (2007). Risk allocation in the private provision of public infrastructure. International Journal of Project Management, 25(1), 66–76.

    Article  Google Scholar 

  13. Eriksson, J., Girod, L., Hull, B., Newton, R., Madden, S. & Balakrishnan, H. (2008, June) The pothole patrol: using a mobile sensor network for road surface monitoring. In Proceedings of the 6th international conference on Mobile systems, applications, and services (pp. 29–39).

  14. Gothane, S., & Sarode, D. M. (2015). Case study: Analysis and study of different approaches for road network maintenance. International Journal of Scientific Engineering and Research, 3, 1–4.

    Google Scholar 

  15. Dirgahayani, P., Rachmat, S. Y., Nasution, M., & Fauzi, M. (2017). Bicycle lane and intersection facility planning methodology based on safety, comfort and attractiveness criteria in motorist dominated traffic condition. Journal of the Eastern Asia Society for Transportation Studies, 12, 1279–1298.

    Google Scholar 

  16. Huidrom, L., Das, L. K., & Sud, S. K. (2013). Method for automated assessment of potholes, cracks and patches from road surface video clips. Procedia-Social and Behavioral Sciences, 104, 312–321.

    Article  Google Scholar 

  17. Adam, B., Mussari, R., & Jones, R. (2011). The diversity of accrual policies in local government financial reporting: An examination of infrastructure, art and heritage assets in Germany, Italy and the UK. Financial Accountability & Management, 27(2), 107–133.

    Article  Google Scholar 

  18. Sathya, R., Rugveda Muralidhar, I., Sai Harsha Vardhan, K., Sri Karan, R., & Arun Reddy, B. (2019). Data Efficient approaches on Deep Action Recognition in Videos. International Journal of Engineering and Advanced Technology (IJEAT), 8(4), 385–391.

  19. Sathya, R., Rawat, D., Mondal, A., Choudhary, S., & Jain, A. (2019). Economically efficient data feature selection using big data analysis. International Journal of Innovative Technology and Exploring Engineering, 8(7), 983–987.

  20. Feghali, J. (2006). An evaluation of selected asphalt pavements in the City of Montreal. In Masters Abstracts International, 45(05), 1–8.

    Google Scholar 

  21. Koch, C., Jog, G. M., & Brilakis, I. (2013). Automated pothole distress assessment using asphalt pavement video data. Journal of Computing in Civil Engineering, 27(4), 370–378.

    Article  Google Scholar 

  22. Koch, C., Georgieva, K., Kasireddy, V., Akinci, B., & Fieguth, P. (2015). A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 29(2), 196–210.

    Article  Google Scholar 

  23. Fan, R., Ozgunalp, U., Wang, Y., Liu, M., & Pitas, I. (2021). Rethinking road surface 3-D reconstruction and pothole detection: from perspective transformation to disparity map segmentation. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2021.3060461

    Article  Google Scholar 

  24. Wang, H. W., Chen, C. H., Cheng, D. Y., Lin, C. H., & Lo, C. C. (2015). A real-time pothole detection approach for intelligent transportation system. Mathematical Problems in Engineering. https://doi.org/10.1155/2015/869627

    Article  Google Scholar 

  25. Shah, S. & Deshmukh, C. (2019, December) Pothole and Bump detection using convolution neural networks. In 2019 IEEE transportation electrification conference (ITEC-India) , IEEE, (pp. 1–4).

  26. Li, S., & Zhao, X. (2019). Image-based concrete crack detection using convolutional neural network and exhaustive search technique. Advances in Civil Engineering. https://doi.org/10.1155/2019/6520620

    Article  Google Scholar 

  27. Du, Y., Pan, N., Xu, Z., Deng, F., Shen, Y., & Kang, H. (2020). Pavement distress detection and classification based on YOLO network. International Journal of Pavement Engineering. https://doi.org/10.1080/10298436.2020.1714047

    Article  Google Scholar 

  28. Hoang, N. D., & Nguyen, Q. L. (2019). A novel method for asphalt pavement crack classification based on image processing and machine learning. Engineering with Computers, 35(2), 487–498.

    Article  Google Scholar 

  29. Chen, H., Yao, M., & Gu, Q. (2020). Pothole detection using location-aware convolutional neural networks. International Journal of Machine Learning and Cybernetics, 11(4), 899–911.

    Article  Google Scholar 

  30. Kyriakou, C., Christodoulou, S. E., & Dimitriou, L. (2019). Smartphone-based pothole detection utilizing artificial neural networks. Journal of Infrastructure Systems, 25(3), 04019019.

    Article  Google Scholar 

  31. Bhavya, P., Sharmila, C., Sai Sadhvi, Y., Prasanna, C. M., & Ganesan, V. (2021). Pothole detection using deep learning. In Smart Technologies in Data Science and Communication (pp. 233–243). Springer, Singapore.

  32. Gupta, S., Sharma, P., Sharma, D., Gupta, V., & Sambyal, N. (2020). Detection and localization of potholes in thermal images using deep neural networks. Multimedia Tools and Applications, 79(35), 26265–26284.

    Article  Google Scholar 

  33. Ning, X., Gong, K., Li, W., Zhang, L., Bai, X., & Tian, S. (2020). Feature refinement and filter network for person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 31(9), 3391–3402.

    Article  Google Scholar 

  34. Ning, X., Duan, P., Li, W., & Zhang, S. (2020). Real-time 3D face alignment using an encoder-decoder network with an efficient deconvolution layer. IEEE Signal Processing Letters, 27, 1944–1948.

    Article  Google Scholar 

  35. Sathya, A., & Saleena, B. (2020). A survey on content based image retrieval using convolutional neural networks. International Journal of Advanced Trends in Computer Science and Engineering, 9(5), 7387–7396.

    Article  Google Scholar 

  36. Zhu, R. L., Shen, J., & Xie, L. (2016). Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Transactions on Knowledge and Data Engineering., 29(2), 472–486.

    Article  Google Scholar 

  37. Ouma, Y. O., & Hahn, M. (2017). Pothole detection on asphalt pavements from 2D-colour pothole images using fuzzy c-means clustering and morphological reconstruction. Automation in Construction, 83, 196–211.

  38. Bhatia, Y., Rai, R., Gupta, V., Aggarwal, N., & Akula, A. (2019). Convolutional neural networks based potholes detection using thermal imaging. Journal of King Saud University-Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2019.02.004

  39. Koch, C., & Brilakis, I. (2011). Pothole detection in asphalt pavement images. Advanced Engineering Informatics, 25(3), 507–515.

    Article  Google Scholar 

  40. https://www.kaggle.com/atulyakumar98/pothole-detection-dataset?select=potholes

  41. Ryu, S. K., Kim, T., & Kim, Y. R. (2015). Image-based pothole detection system for ITS service and road management system. Mathematical Problems in Engineering. https://doi.org/10.1155/2015/968361

    Article  Google Scholar 

  42. https://drive.google.com/drive/folders/1Z8X76CQnuUOl24_53GCHNJDuehjr0c4M

  43. https://drive.google.com/drive/folders/1reY7vVBdRH2jbzeaXRMmNUscOtO-92td

Download references

Funding

There is no funding provided to prepare the manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Vellore Institute of Technology, Chennai Campus, Chennai, India

    R. Sathya & B. Saleena

  2. Department of Computer Science and Engineering, SRM Institute of Science and Technology, Ramapuram Campus, Chennai, India

    R. Sathya

Authors
  1. R. Sathya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B. Saleena
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Sathya.

Ethics declarations

Conflict of interest

There is no conflict of Interest between the authors regarding the manuscript preparation and submission.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informal Consent

Informed consent was obtained from all individual participants included in the study.

Consent to Participate

There is no consent to participate or any concerns in the manuscript.

Consent to Publish

There is no consent or any copyright needed to get concerns in the manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sathya, R., Saleena, B. A Framework for Designing Unsupervised Pothole Detection by Integrating Feature Extraction Using Deep Recurrent Neural Network. Wireless Pers Commun 126, 1241–1271 (2022). https://doi.org/10.1007/s11277-022-09790-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-022-09790-z

Keywords

Search

Navigation