Skip to main content
SpringerLink
Log in

Real-time traffic sign detection model based on multi-branch convolutional reparameterization

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Intelligent detection of traffic signs has great potential in autonomous driving. Certain elements can make the detection difficult. In the road images captured by vehicle cameras, there could be various types of traffic sign objects some of which are quite similar. On top of that the complex background is adding additional noise which makes it challenging to balance detection speed and accuracy. In this paper, we propose a real-time traffic sign detection network based on the Anchor-free mechanism to solve this problem. First, we present the idea of reparameterization. In the training stage, the Conv\(3 \times 3\) convolution in the feature extraction network CSPDarknet53 is reconstructed to enhance the extraction ability of traffic sign features. In the inference stage, the multi-branch structure is equivalently converted into a single-way structure to improve the inference speed of the model. Second, to improve the accuracy of locating small objects, we introduce the SIoU position regression loss function, which addresses the challenge of sensitive position regression for small objects. Lastly, test results on TT-100K, GTSDB and CCTSDB datasets show that the model can achieve a performance trade-off between speed and accuracy. This paper proposes a model that achieves an inference speed of 153.8 FPS on the CCTSDB dataset when tested on the GTX 2080ti, and a speed of 23.3 FPS on the Nvidia Jetson Xavier NX, demonstrating its ability to perform fast and efficient traffic sign detection on different hardware platforms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Yurtsever, E., Lambert, J., Carballo, A., Takeda, K.: A survey of autonomous driving: common practices and emerging technologies. IEEE Access 8, 58443–58469 (2020). https://doi.org/10.1109/ACCESS.2020.2983149

    Article  Google Scholar 

  2. Ren, K., Huang, L., Fan, C., Han, H., Deng, H.: Real-time traffic sign detection network using ds-detnet and lite fusion fpn. J. Real-Time Image Proc. 18(6), 2181–2191 (2021)

    Article  Google Scholar 

  3. Tong, K., Wu, Y., Zhou, F.: Recent advances in small object detection based on deep learning: a review. Image Vis. Comput. 97, 103910 (2020). https://doi.org/10.1016/j.imavis.2020.103910

    Article  Google Scholar 

  4. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  5. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  6. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)

  7. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37 (2016). Springer

  8. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  9. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  10. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

  11. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018). https://doi.org/10.48550/arXiv.1804.02767

  12. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020). https://doi.org/10.48550/arXiv.2004.10934

  13. jocher., G.: YOLOv5. GitHub (2021)

  14. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021). https://doi.org/10.48550/arXiv.2107.08430

  15. Xu, X., Jin, J., Zhang, S., Zhang, L., Pu, S., Chen, Z.: Smart data driven traffic sign detection method based on adaptive color threshold and shape symmetry. Futur. Gener. Comput. Syst. 94, 381–391 (2019)

    Article  Google Scholar 

  16. Hu, Z., Bai, D., Li, N.: Generalized sign recognition based on the gaussian statistical color model for intelligent road sign inventory. Transp. Res. Rec. 2596(1), 28–35 (2016)

    Article  Google Scholar 

  17. Liu, C., Chang, F., Chen, Z.: Rapid multiclass traffic sign detection in high-resolution images. IEEE Trans. Intell. Transp. Syst. 15(6), 2394–2403 (2014)

    Article  Google Scholar 

  18. Houjie, L., Tianshuang, Q., Haiyu, S., Pengchang, W., Pengjie, W.: Separation algorithm of traffic signs based on curvature scale space corner detection. Acta Opt. Sin. 35(01), 247–255 (2015)

    Article  Google Scholar 

  19. Lu, Y., Lu, J., Zhang, S., Hall, P.: Traffic signal detection and classification in street views using an attention model. Comput. Vis. Media 4(3), 253–266 (2018)

    Article  Google Scholar 

  20. Dongtao, Z., Jie, C., Xing, Y., Hui, S., Liangliang, S.: Traffic sign detection method of improved ssd based on deep learning. In: 2018 IEEE 4th International Conference on Computer and Communications (ICCC), pp. 1516–1520 (2018). IEEE

  21. Zhang, H., Qin, L., Li, J., Guo, Y., Zhou, Y., Zhang, J., Xu, Z.: Real-time detection method for small traffic signs based on yolov3. IEEE Access 8, 64145–64156 (2020)

    Article  Google Scholar 

  22. Liu, Z., Li, D., Ge, S.S., Tian, F.: Small traffic sign detection from large image. Appl. Intell. 50(1), 1–13 (2020)

    Article  Google Scholar 

  23. Yang, T., Tong, C.: Real-time detection network for tiny traffic sign using multi-scale attention module. Sci. Chin. Technol. Sci. 65(2), 396–406 (2022)

    Article  Google Scholar 

  24. Zhang, J., Ye, Z., Jin, X., Wang, J., Zhang, J.: Real-time traffic sign detection based on multiscale attention and spatial information aggregator. Journal of Real-Time Image Processing, 1–13 (2022)

  25. Wei, H., Zhang, Q., Qian, Y., Xu, Z., Han, J.: Mtsdet: multi-scale traffic sign detection with attention and path aggregation. Applied Intelligence, 1–13 (2022)

  26. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  27. Wang, C.-Y., Liao, H.-Y.M., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., Yeh, I.-H.: Cspnet: A new backbone that can enhance learning capability of cnn. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391 (2020)

  28. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015). https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  29. Tian, Z., Shen, C., Chen, H., He, T.: Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627–9636 (2019)

  30. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., Sun, J.: Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13733–13742 (2021)

  31. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8759–8768 (2018)

  32. Gevorgyan, Z.: Siou loss: More powerful learning for bounding box regression. arXiv preprint arXiv:2205.12740 (2022)

  33. Zhu, Z., Liang, D., Zhang, S., Huang, X., Li, B., Hu, S.: Traffic-sign detection and classification in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2110–2118 (2016)

  34. Hmida, R., Ben Abdelali, A., Mtibaa, A.: Hardware implementation and validation of a traffic road sign detection and identification system. J. Real-Time Image Proc. 15, 13–30 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an, 710054, China

    Mengtao Huang, Yiyi Wan, Zhenwei Gao & Jiaxuan Wang

  2. Xi’an Key Laboratory of Electrical Condition Monitoring and Power Supply Security, Xi’an, 710054, China

    Mengtao Huang, Yiyi Wan, Zhenwei Gao & Jiaxuan Wang

Authors
  1. Mengtao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiyi Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenwei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaxuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yiyi Wan.

Ethics declarations

Conflict of interest

The datasets generated during or analysed during the current study are available from the corresponding author on reasonable request. The authors have no relevant financial or non-financial interests to disclose. The authors declare that they have no conflict of interest. This research was supported, in part, by grant for the Key Research and Development Program of Shaanxi Province of China (2019GY-0972021GY-131).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, M., Wan, Y., Gao, Z. et al. Real-time traffic sign detection model based on multi-branch convolutional reparameterization. J Real-Time Image Proc 20, 57 (2023). https://doi.org/10.1007/s11554-023-01307-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01307-6

Keywords

Search

Navigation