Skip to main content
SpringerLink
Log in

Identification of winter road friction coefficient based on multi-task distillation attention network

  • Short paper
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Road friction coefficient estimation is an important task in the perception system of autonomous driving vehicles. It enables the vehicle to perceive upcoming road friction conditions and helps the decision-making system to adjust the driving styles accordingly in case of potential traffic accidents caused by tire slip. However, to our knowledge, there is currently no recognized image benchmark dataset in this field with enough weather diversity for this task. And many existing methods are measured under different standards. In this work, we present a road friction coefficient estimation dataset that includes all-weather traffic conditions, which is called the winter road friction (WRF) dataset. Then, a novel friction coefficient estimation model based on multi-task distillation attention network (MDAN) is proposed to solve this task in an end-to-end way. The proposed model surpasses existing methods in this field and reaches 86.53% Acc on the WRF dataset. The WRF dataset will be made publicly available at https://github.com/blackholeLFL/The-WRF-dataset soon.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Virgilio G VR, Sossa H, Zamora E (2020) Vision-based blind spot warning system by deep neural networks. In: Mexican conference on pattern recognition, Springer, pp 185–194

  2. Qian Y, Almazan EJ, Elder JH (2016) Evaluating features and classifiers for road weather condition analysis. In: 2016 IEEE international conference on image processing (ICIP), IEEE, pp 4403–4407

  3. Shibata Y, Sakuraba A, Arai Y, Saito Y, Hakura J (2021) Decision analysis of winter road conditions by crowd sensing platform. In: International conference on emerging internetworking. Springer, Data & Web Technologies, pp 276–287

  4. Qi Z, Taheri S, Wang B, Yu H (2015) Estimation of the tyre-road maximum friction coefficient and slip slope based on a novel tyre model. Vehicle Syst Dyn 53(4):506–525

    Article  Google Scholar 

  5. Wu Y, Liu F, Guan L, Yang X (2020) A survey of vision-based road parameter estimating methods. In: International conference on intelligent computing, Springer, pp 314–325

  6. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O (2016) Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530

  7. Kim J, Kim D, Lee J, Lee J, Joo H, Kweon IS (2009) Non-contact terrain classification for autonomous mobile robot. In: 2009 IEEE international conference on robotics and biomimetics (ROBIO), IEEE, pp 824–829

  8. Rateke T, Justen KA, von Wangenheim A (2019) Road surface classification with images captured from low-cost camera-road traversing knowledge (rtk) dataset. Revista de Informática Teórica e Aplicada 26(3):50–64

    Article  Google Scholar 

  9. Kim MH, Park J, Choi S (2021) Road type identification ahead of the tire using d-cnn and reflected ultrasonic signals. Int J Automot Technol 22(1):47–54

    Article  Google Scholar 

  10. Almazan EJ, Qian Y, Elder JH (2016) Road segmentation for classification of road weather conditions. In: European conference on computer vision, Springer, pp 96–108

  11. Šabanovič E, Žuraulis V, Prentkovskis O, Skrickij V (2020) Identification of road-surface type using deep neural networks for friction coefficient estimation. Sensors 20(3):612

    Article  Google Scholar 

  12. Roychowdhury S, Zhao M, Wallin A, Ohlsson N, Jonasson M (2018) Machine learning models for road surface and friction estimation using front-camera images. In: 2018 international joint conference on neural networks (IJCNN), IEEE, pp 1–8

  13. Jonnarth A (2018) Camera-based friction estimation with deep convolutional neural networks

  14. Du Y, Liu C, Song Y, Li Y, Shen Y (2019) Rapid estimation of road friction for anti-skid autonomous driving. IEEE Trans Intel Transp Syst 21(6):2461–2470

    Article  Google Scholar 

  15. Lu X, Da J, Bo L, xin Y, Lihua W (2019) Road friction estimation method for distributed driving electric vehicle based on machine vision assistance (in Chinese). J Tongji Univ 047(0z1):99–103

  16. Karthikeyan M, Sathiamoorthy S (2020) Fully connected deep network based road surface recognition system for an autonomous vehicle. Int J Adv Res Eng Technol 11(12):2396–2405

    Google Scholar 

  17. Yu F, Chen H, Wang X, Xian W, Chen Y, Liu F, Madhavan V, Darrell T (2020) Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2636–2645

  18. Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A (2017) Scene parsing through ade20k dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 633–641

  19. Neuhold G, Ollmann T, Rota Bulo S, Kontschieder P (2017) The mapillary vistas dataset for semantic understanding of street scenes. In: Proceedings of the IEEE international conference on computer vision, pp 4990–4999

  20. Jin D, Leng B, Yang X, Xiong L, Yu Z (2020) Road friction estimation method based on fusion of machine vision and vehicle dynamics** resrach supported by ‘national key research and development program of china’(Grant No. 2018yfb0104805),‘national natural science foundation of china’(Grant No. u15642073) and ‘national natural science foundation of china’(Grant No. 51975414). In: 2020 IEEE intelligent vehicles symposium (IV), IEEE, pp 1771–1776

  21. Yu F, Lin Y (2017) Automotive system dynamics. Mechanical Industry Press, Beijing

    Google Scholar 

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  23. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  24. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  25. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size. arXiv preprint arXiv:1602.07360

  26. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  27. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  28. Minaee S, Boykov YY, Porikli F, Plaza AJ, Kehtarnavaz N, Terzopoulos D (2021) Image segmentation using deep learning: a survey. IEEE Trans Pattern Anal Mach Intel

  29. Singh P, Chadha RS (2013) A novel approach to image segmentation. Int J Adv Res Comput Sci Software Eng 3(4)

  30. Singh P, Shree R, Pandey RP, Shukla V, Chadha RS (2018) A new box segmentation based digital image watermark positioning method in spatial domain. Adv Sci Eng Med 10(7–8):700–704

    Article  Google Scholar 

  31. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2):203–211

    Article  Google Scholar 

  32. Ruder S (2017) An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098

  33. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intel 40(4):834–848

    Article  Google Scholar 

  34. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  35. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154

  36. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

  37. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626

  38. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255

  39. Nolte M, Kister N, Maurer M (2018) Assessment of deep convolutional neural networks for road surface classification. In: 2018 21st international conference on intelligent transportation systems (ITSC), IEEE, pp 381–386

  40. Busch A, Fink D, Laves MH, Ziaukas Z, Wielitzka M, Ortmaier T (2019) Classification of road surface and weather-related condition using deep convolutional neural networks. In: The IAVSD international symposium on dynamics of vehicles on roads and tracks, Springer, pp 1042–1051

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No.U19A2069).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Tongji University, Shanghai, 201804, China

    Feilin Liu, Yan Wu, Xinneng Yang, Yujian Mo & Yujun Liao

Authors
  1. Feilin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinneng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yujian Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yujun Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, F., Wu, Y., Yang, X. et al. Identification of winter road friction coefficient based on multi-task distillation attention network. Pattern Anal Applic 25, 441–449 (2022). https://doi.org/10.1007/s10044-022-01059-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-022-01059-2

Keywords

Search

Navigation