Skip to main content
SpringerLink
Log in

Benchmark Study on a Novel Online Dataset for Standard Evaluation of Deep Learning-based Pavement Cracks Classification Models

  • Hydraulic Engineering
  • Published:
KSCE Journal of Civil Engineering Aims and scope

Abstract

Highway agencies and practitioners expect to have the most efficient method with adequate accuracy when choosing a deep learning-based model for pavement crack classification. However, many works are implemented on their own dataset, making them hard to compare with each other, and also less persuasive and robust. Therefore, a Road Cracks Classification Dataset is proposed to serve as a standard and open-source dataset. Based on this dataset, a benchmark study of fourteen deep learning classification methods is evaluated. Two parameters, the Ratio of F1 and Training Time (RFT) and Ratio of F1 and Prediction Time (RFP), are proposed to quantify the efficiency of networks. The results show that ConvNeXt_base reaches the highest accuracy among all models but requires the longest training time. AlexNet takes the least training time among all models, but gains the lowest accuracy. Of the four crack types, the block crack has the lowest accuracy, which means it is the most difficult to detect. SqueezeNet1_0 has the highest efficiency among all models in converting the computing power to accuracy. Wide ResNet 50_2 consumes the longest prediction time among CNN models, while the ConvNeXt_base has the highest feasibility on real-time tasks. To implement a suitable deep learning-based pavement crack inspection, we recommend a good balance between computational cost and accuracy. Based on this, we provide practical recommendations according to different user groups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Chen C, Chandra S, Han Y, Seo H (2021) Deep learning-based thermal image analysis for pavement defect detection and classification considering complex pavement conditions. Remote Sensing 14:106, DOI: https://doi.org/10.3390/rs14010106

    Article  ADS  Google Scholar 

  • Cui B, Wang H, Gu X, Hu D (2022) Study of the inter-diffusion characteristics and cracking resistance of virgin-aged asphalt binders using molecular dynamics simulation. Construction and Building Materials 351:128968, DOI: https://doi.org/10.1016/j.conbuildmat.2022.128968

    Article  CAS  Google Scholar 

  • Deng H, Gu X, Wang X, Ww C, Zhu, C (2019) Evaluation of high-temperature deformation of porous asphalt mixtures based on microstructure using X-ray computed tomography. Construction and Building Materials 227:116623, DOI: https://doi.org/10.1016/j.conbuildmat.2019.08.004

    Article  CAS  Google Scholar 

  • Deng J, Dong W, Socher R, Li LJ, Li K, Feifei L (2009) Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, Ieee, 248–255

  • Fei Y, Wang KC, Zhang A, Chen C, Li JQ, Liu Y, Yang G, Li B (2019) Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V. IEEE Transactions on Intelligent Transportation Systems 21:273–284, DOI: https://doi.org/10.1109/TITS.2019.2891167

    Article  Google Scholar 

  • Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A (2017) Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Construction and Building Materials 157:322–330, DOI: https://doi.org/10.1016/j.conbuildmat.2017.09.110

    Article  Google Scholar 

  • Han H, Deng H, Dong Q, Gu X, Zhang T, Wang Y (2021) An advanced Otsu method integrated with edge detection and decision tree for crack detection in highway transportation infrastructure. Advances in Materials Science and Engineering, 2021, DOI: https://doi.org/10.1155/2021/9205509

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778

  • Hou Y, Li Q, Han Q, Peng B, Wang L, Gu X, Wang D (2021) MobileCrack: Object classification in asphalt pavements using an adaptive lightweight deep learning. Journal of Transportation Engineering, Part B: Pavements 147:04020092, DOI: https://doi.org/10.1061/JPEODX.0000245

    Google Scholar 

  • Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M, Wang W, Zhu Y, Pang R, Vasudevan V (2019) Searching for mobilenetv3. Proceedings of the IEEE/CVF International Conference on Computer Vision, 1314–1324

  • Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4700–4708

  • Huyan J, Li W, Tighe S, Xu Z, Zhai J (2020) CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack detection. Structural Control and Health Monitoring 27:e2551, DOI: https://doi.org/10.1002/stc.2551

    Article  Google Scholar 

  • Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv preprint arXiv:1602.07360

  • Kaddah W, Elbouz M, Ouerhani Y, Baltazart V, Desthieux M, Alfalou A (2019) Optimized minimal path selection (OMPS) method for automatic and unsupervised crack segmentation within two-dimensional pavement images. The Visual Computer 35:1293–1309, DOI: https://doi.org/10.1007/s00371-018-1515-9

    Article  Google Scholar 

  • Krizhevsky A (2014) One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv:1404.5997

  • Liu F, Liu J, Wang L (2022a) Deep learning and infrared thermography for asphalt pavement crack severity classification. Automation in Construction 140:104383, DOI: https://doi.org/10.1016/j.autcon.2022.104383

    Article  Google Scholar 

  • Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S (2022b) A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11976–11986

  • Liu J, Yang X, Lau S, Wang X, Luo S, Lee VCS, Ding L (2020) Automated pavement crack detection and segmentation based on two-step convolutional neural network. Computer-Aided Civil and Infrastructure Engineering 35:1291–1305, DOI: https://doi.org/10.1111/mice.12622

    Article  Google Scholar 

  • Liu Y, Yao J, Lu X, Xie R, Li L (2019) DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 338:139–153, DOI: https://doi.org/10.1016/j.neucom.2019.01.036

    Article  Google Scholar 

  • Ma N, Zhang X, Zheng HT, Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. Proceedings of the European Conference on Computer Vision (ECCV), 116–131.

  • Qu Z, Mei J, Liu L, Zhou DY (2020) Crack detection of concrete pavement with cross-entropy loss function and improved VGG16 network model. Ieee Access 8:54564–54573, DOI: https://doi.org/10.1109/ACCESS.2020.2981561

    Article  Google Scholar 

  • Que Y, Dai Y, Ji X, Leung AK, Chen Z, Tang Y, Jiang Z (2023) Automatic classification of asphalt pavement cracks using a novel integrated generative adversarial networks and improved VGG model. Engineering Structures 277:115406, DOI: https://doi.org/10.1016/j.engstruct.2022.115406

    Article  Google Scholar 

  • Radosavovic I, Kosaraju RP, Girshick R, He K, Dollar P (2020) Designing network design spaces. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10428–10436

  • Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4510–4520

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9

  • Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard A, Le QV (2019) Mnasnet: Platform-aware neural architecture search for mobile. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2820–2828

  • Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning, PMLR:6105–6114

  • Wang W, Su C (2021) Deep learning-based real-time crack segmentation for pavement images. KSCE Journal of Civil Engineering 25:4495–4506, DOI: https://doi.org/10.1007/s12205-021-0474-2

    Article  Google Scholar 

  • Wen T, Lang H, Ding S, Lu JJ, Xing Y (2022) PCDNet: Seed operation–based deep learning model for pavement crack detection on 3d asphalt surface. Journal of Transportation Engineering, Part B: Pavements 148:04022023, DOI: https://doi.org/10.1061/JPEODX.0000367

    Google Scholar 

  • Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1492–1500

  • Xu B, Liu C (2022) Pavement crack detection algorithm based on generative adversarial network and convolutional neural network under small samples. Measurement 196:111219, DOI: https://doi.org/10.1016/j.measurement.2022.111219

    Article  Google Scholar 

  • Yang F, Zhang L, Yu S, Prokhorov D, Mei X, Ling H (2019) Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Transactions on Intelligent Transportation Systems 21:1525–1535, DOI: https://doi.org/10.1109/TITS.2019.2910595

    Article  Google Scholar 

  • Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv preprint arXiv:1605.07146

  • Zhang T, Rahman MA, Peterson A, Lu Y (2022) Novel damage index-based rapid evaluation of civil infrastructure subsurface defects using thermography analytics. Infrastructures 7:55, DOI: https://doi.org/10.3390/infrastructures7040055

    Article  Google Scholar 

  • Zhang T, Wang D, Lu Y (2023a) ECSNet: An accelerated real-time image segmentation CNN architecture for pavement crack detection. IEEE Transactions on Intelligent Transportation Systems, DOI: https://doi.org/10.1109/TITS.2023.3300312

  • Zhang T, Wang D, Lu Y (2023b) Machine learning-enabled regional multi-hazards risk assessment considering social vulnerability. Scientific Reports 13:13405, DOI: https://doi.org/10.1038/s41598-023-40159-9

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  • Zhang T, Wang D, Muffins A, Lu Y (2023c) Integrated APC-GAN and AttuNet framework for automated pavement crack pixel-level segmentation: A new solution to small training datasets. IEEE Transactions on Intelligent Transportation Systems, DOI: https://doi.org/10.1109/TITS.2023.3236247

Download references

Acknowledgments

We would like to appreciate the Federal Highway Administration (FHWA) for providing the pavement images (https://github.com/UM-Titan/DSPS).

Author information

Authors and Affiliations

  1. Dept. of Computer Science, Boise State University, Boise, Idaho, 83725, USA

    Tianjie Zhang

  2. Dept. of Civil Engineering, Boise State University, Boise, Idaho, 83725, USA

    Donglei Wang & Yang Lu

Authors
  1. Tianjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donglei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Lu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, T., Wang, D. & Lu, Y. Benchmark Study on a Novel Online Dataset for Standard Evaluation of Deep Learning-based Pavement Cracks Classification Models. KSCE J Civ Eng 28, 1267–1279 (2024). https://doi.org/10.1007/s12205-024-1066-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12205-024-1066-8

Keywords

Search

Navigation