Skip to main content
SpringerLink
Log in

Polylanenet++: enhancing the polynomial regression lane detection based on spatio-temporal fusion

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Deep learning has made significant progress in lane detection across various public datasets, with models, such as PolyLaneNet, being computationally efficient. However, these models have limited spatial generalization capabilities, which ultimately lead to decreased accuracy. To address this issue, we propose a polynomial regression-based deep learning model that enhances spatial generalization and incorporates temporal information to improve the accuracy. Our model has been tested on public datasets, such as TuSimple and VIL100, and the results show that it outperforms PolyLaneNet and achieves state-of-the-art results. Incorporation of temporal information is also advantageous. Overall, our proposed framework offers improved accuracy and practicality in real-time applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

Dataset TuSimple is publicly available at https://github.com/TuSimple/tusimple-benchmark and Dataset VIL100 is publicly available at https://github.com/yujun0-0/MMA-Net.

References

  1. Xie, Q., Liu, R., Sun, Z., Pei, S., Cui, F.: A flexible free-space detection system based on stereo vision. Neurocomputing 485, 252–262 (2022). https://doi.org/10.1016/j.neucom.2021.05.115

    Article  Google Scholar 

  2. Xie, Q., Hu, X., Ren, L., Qi, L., Sun, Z.: A binocular vision application in iot: Realtime trustworthy road condition detection system in passable area. IEEE Trans. Ind. Inf. 19(1), 973–983 (2023). https://doi.org/10.1109/TII.2022.3145858

    Article  Google Scholar 

  3. Lin, X., Hu, Z., Liu, H.: Accurate object tracking by aligning and refining multiple predictions in siamese networks. Int. J. Wavelets Multiresol. Inf. Proc. (2023)

  4. Torres, L., Berriel, R., Paixao, T., C. Badue, A.S., Oliveira-Santos, T.: Polylanenet: Lane estimation via deep polynomial regression. In: International Conference on Pattern Recognition (ICPR), pp. 6150–6156 (2021)

  5. Li, W., Vijay, J., Seiichi, M.: Enhancing depth quality of stereo vision using deep learning-based prior information of the driving environment. In: Proceedings of the 25st International Conference on Pattern Recognition (ICPR’20), pp. 7281–7286 (2020). IEEE

  6. Zhang, Y., Zhu, L., Feng, W., Fu, H., Wang, M., Li, Q., Li, C., Wang, S.: Vil-100: A new dataset and a baseline model for video instance lane detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 15681–15690 (2021)

  7. Yu, S., Zhang, J.X., Lim, E.: Democracy does matter: Comprehensive feature mining for co-salient object detection. In: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 979–988 (2022)

  8. Pan, X., Shi, J., Luo, P., Wang, X., Tang, X.: Spatial as deep: Spatial cnn for traffic scene understanding. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 32 (2018)

  9. Hou, Y., Ma, Z., Liu, C., Loy, C.: Learning lightweight lane detection cnns by self attention distillation. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1013–1021 (2019)

  10. Qin, Z., Wang, H., Li, X.: Ultra fast structure-aware deep lane detection. In: European Conference on Computer Vision (ECCV), vol. 12369 (2020)

  11. Tusimple: Tusimple benchmark. https://github.com/TuSimple/tusimple-benchmark (2017)

  12. Horowicz, J., Udwin, D., Flaxman, S., Filippi, S., Crawford, L.: Interpreting deep neural networks through variable importance. ArXiv (2019) arXiv:abs/1901. 09839

  13. Li, W., Zhang, Z., Li, S., Tao, D.: Road detection by using a generalized hough transform. Remote Sens. 9(6), 590 (2017)

    Article  Google Scholar 

  14. Wei, Y., You, X., Li, H.: Multiscale patch-based contrast measure for small infrared target detection. Pattern Recognit. 58, 216–226 (2016)

    Article  Google Scholar 

  15. Neven, D., Brabandere, B., Georgoulis, S., Proesmans, M., Gool, L.: Towards end-to-end lane detection: an instance segmentation approach, pp. 286–291 (2018)

  16. Liu, L., Chen, X., Zhu, S., Tan, P.: Condlanenet: a top-to-down lane detection framework based on conditional convolution. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3753–3762 (2021). https://doi.org/10.1109/ICCV48922.2021.00375

  17. Tabelini, L., Berriel, R., Paixao, T., C. Badue, A.S., Oliveira-Santos, T.: Keep your eyes on the lane: Real-time attention-guided lane detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 294–302 (2021)

  18. Sun, M., Xiao, J., Zhang, E.L., Zhao, Y.: Fast template matching and update for video object tracking and segmentation. In: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10791–10799 (2020)

  19. Keuper, M., Andres, B., Brox, T.: Motion trajectory segmentation via minimum cost multicuts. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3271–3279 (2015)

  20. Li, Y., Chen, L., Huang, H., Li, X., Xu, W., Zheng, L., Huang, J.: Key segments for video object segmentation. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1995–2020 (2011)

  21. Wang, W., Shen, J., Yang, R., Porikli, F.: Saliency-aware video object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 20–33 (2017)

    Article  Google Scholar 

  22. Cheng, J., Tsai, Y., Wang, S., Yang, M.: Segflow: Joint learning for video object segmentation and optical flow. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 686–695 (2017)

  23. Chen, X., Li, Z., Yuan, Y., Yu, G., Shen, J., Qi, D.: State-aware tracker for real-time video object segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9384–9393 (2020)

  24. Zhang, Y., Wu, Z., Peng, H., Lin, S.: A transductive approach for video object segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6947–6956 (2020)

  25. Seoung, W., Lee, J., Xu, N., Kim, S.: Video object segmentation using space-time memory networks. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9225–9234 (2019)

  26. Johnander, J., Danelljan, M., Brissman, E., Khan, F., Felsberg, M.: A generative appearance model for end-to-end video object segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  27. Robinson, A., Lawin, F., Danelljan, M., Khan, F., Felsberg, M.: Learning fast and robust target models for video object segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7406–7415 (2020)

  28. Hu, J., Shen, L., Samuel, A., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020)

    Article  Google Scholar 

  29. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Li, F.: Imagenet: A large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009)

  30. Liu, R., Yuan, Z., Liu, T., Xiong, Z.: End-to-end lane shape prediction with transformers. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 3693–3701 (2021)

  31. Neven, D., Brabandere, B., Georgoulis, S., Proesmans, M., Gool, L.: Towards end-to-end lane detection: an instance segmentation approach. In: IEEE Intelligent Vehicles Symposium (IV), pp. 286–291 (2018)

  32. Han, J., Deng, X., Cai, X., Yang, Z., Xu, H., Xu, C., Liang, X.: Laneformer: Object-aware row-column transformers for lane detection. In: AAAI Conference on Artificial Intelligence (2022)

  33. Ventura, C., Bellver, M., Girbau, A., Salvador, A., Marques, F., GiroiNieto, X.: Rvos: End-to-end recurrent network for video object segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5272–5281 (2019)

  34. Liang, Y., Li, X., Jafari, N., Chen, J.: Video object segmentation with adaptive feature bank and uncertain-region refinement. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

Download references

Funding

This work was supported by the Fundamental Research Funds for the Central Universities of China under 2662020LXQD002.

Author information

Authors and Affiliations

  1. School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, China

    Chuanwu Yang & Xinge You

  2. College of Informatics, Huazhong Agricultural University, Wuhan, 430070, China

    Zhihui Tian & Zhibin Pan

  3. Beijing Jiyun Intelligent Technology Co., Beijing, 100102, China

    Kang Jia & Tong Liu

  4. Guardian Robot Project, 2–2-2 Hikaridai, RIKEN, Seika-cho, 619–0288, Japan

    Vijay John

Authors
  1. Chuanwu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhihui Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinge You
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kang Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhibin Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vijay John
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors made substantial contributions to the concept, design, and revision of the paper. We know of no conflicts of interest associated with this publication, and we have contributed to this work.

Corresponding authors

Correspondence to Zhibin Pan or Vijay John.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, C., Tian, Z., You, X. et al. Polylanenet++: enhancing the polynomial regression lane detection based on spatio-temporal fusion. SIViP 18, 3021–3030 (2024). https://doi.org/10.1007/s11760-023-02967-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02967-4

Keywords

Search

Navigation