Skip to main content
SpringerLink
Log in

Simultaneous detection and tracking using deep learning and integrated channel feature for ambint traffic light recognition

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Perceiving the information about ambient traffic lights is an inevitable task for autonomous vehicles. To deal with the issue, this work develops an accurate and fast traffic light recognition strategy for autonomous vehicles by an onboard camera. In this paper, deep learning based detection and object tracking is synthesized to determine the position and color of traffic lights. First, the mechanism of simultaneous detection and tracking is founded, wherein the video reading module, convolutional neural network (CNN) module, integrated channel feature tracking (ICFT) module are run simultaneously. Then, the respective modules of detection and tracking are introduced. CNN model is designed and trained to obtain the position of traffic lights utilized as initial information for tracking. ICFT is applied to continually track the traffic light targets and determine the light color. Finally, the effectiveness of the presented method is validated via comparing with the state of art. Experiments results indicate that the proposed technique can improve the accuracy and speed of recognition. Our contributions are: (1) Establish a mechanism for simultaneous detection and tracking of traffic lights; (2) Carefully design the CNN architecture and ICFT features; (3)The precision and recall rates on traffic lights recognition reached 0.962 and 0.909, respectively, and the recognition speed reached 21.4FPS (GPU: Nvidia Titan Xp).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Bach M, Stumper D, Dietmayer K (2018) Deep convolutional traffic light recognition for automated driving. In: 2018 21st international conference on intelligent transportation systems (ITSC), IEEE, pp 851–858

  • Cai Z, Li Y, Gu M (2012) Real-time recognition system of traffic light in urban environment. In: 2012 IEEE symposium on computational intelligence for security and defence applications, IEEE, pp 1–6

  • Chen Z, Huang X (2016) Accurate and reliable detection of traffic lights using multiclass learning and multiobject tracking. IEEE Intell Transp Syst Mag 8(4):28–42

    Article  Google Scholar 

  • Chen J, Wang K, Bao H, Chen T (2019) A design of cooperative overtaking based on complex lane detection and collision risk estimation. IEEE Access 7:87951–87959

    Article  Google Scholar 

  • Chiang CC, Ho MC, Liao HS, Pratama A, Syu WC (2011) Detecting and recognizing traffic lights by genetic approximate ellipse detection and spatial texture layouts. Int J Innov Comput Inf Control 7(12):6919–6934

    Google Scholar 

  • Ci Y, Wu L, Zhao J, Sun Y, Zhang G (2019) V2i-based car-following modeling and simulation of signalized intersection. Phys A Stat Mech Appl 525:672–679

    Article  Google Scholar 

  • De Charette R, Nashashibi F (2009a) Real time visual traffic lights recognition based on spot light detection and adaptive traffic lights templates. In: 2009 IEEE intelligent vehicles symposium, IEEE, pp 358–363

  • De Charette R, Nashashibi F (2009b) Traffic light recognition using image processing compared to learning processes. In: 2009 IEEE/RSJ international conference on intelligent robots and systems, IEEE, pp 333–338

  • Diaz-Cabrera M, Cerri P, Medici P (2015) Robust real-time traffic light detection and distance estimation using a single camera. Exp Syst Appl 42(8):3911–3923

    Article  Google Scholar 

  • Fairfield N, Urmson C (2011) Traffic light mapping and detection. In: 2011 IEEE international conference on robotics and automation, IEEE, pp 5421–5426

  • Greenhalgh J, Mirmehdi M (2012) Real-time detection and recognition of road traffic signs. IEEE Trans Intell Transp Syst 13(4):1498–1506

    Article  Google Scholar 

  • Hirabayashi M, Sujiwo A, Monrroy A, Kato S, Edahiro M (2019) Traffic light recognition using high-definition map features. Robot Auton Syst 111:62–72

    Article  Google Scholar 

  • Hosseinyalamdary S, Yilmaz A (2017) A Bayesian approach to traffic light detection and mapping. ISPRS J Photogr Remote Sensing 125:184–192

    Article  Google Scholar 

  • Jensen MB, Philipsen MP, Møgelmose A, Moeslund TB, Trivedi MM (2016) Vision for looking at traffic lights: Issues, survey, and perspectives. IEEE Trans Intell Transp Syst 17(7):1800–1815

    Article  Google Scholar 

  • John V, Yoneda K, Qi B, Liu Z, Mita S (2014) Traffic light recognition in varying illumination using deep learning and saliency map. In: 17th international IEEE conference on intelligent transportation systems (ITSC), IEEE, pp 2286–2291

  • John V, Yoneda K, Liu Z, Mita S (2015) Saliency map generation by the convolutional neural network for real-time traffic light detection using template matching. IEEE Trans Comput Imaging 1(3):159–173

    Article  MathSciNet  Google Scholar 

  • Kim HK, Shin YN, Sg Kuk, Park JH, Jung HY (2013) Night-time traffic light detection based on svm with geometric moment features. Int J Comput Inf Eng 7(4):472–475

    Google Scholar 

  • Lee E, Kim D (2019) Accurate traffic light detection using deep neural network with focal regression loss. Image Vis Comput 87:24–36

    Article  Google Scholar 

  • Lee SH, Kim JH, Lim YJ, Lim J (2018) Traffic light detection and recognition based on haar-like features. In: 2018 international conference on electronics, information, and communication (ICEIC), IEEE, pp 1–4

  • Li X, Ma H, Wang X, Zhang X (2017) Traffic light recognition for complex scene with fusion detections. IEEE Trans Intell Transp Syst 19(1):199–208

    Article  Google Scholar 

  • Liu W, Li S, Lv J, Yu B, Zhou T, Yuan H, Zhao H (2016) Real-time traffic light recognition based on smartphone platforms. IEEE Trans Circuits Syst Video Technol 27(5):1118–1131

    Article  Google Scholar 

  • Possatti LC, Guidolini R, Cardoso VB, Berriel RF, Paixão TM, Badue C, De Souza AF, Oliveira-Santos T (2019) Traffic light recognition using deep learning and prior maps for autonomous cars. In: 2019 international joint conference on neural networks (IJCNN), IEEE, pp 1–8

  • Saini S, Nikhil S, Konda KR, Bharadwaj HS, Ganeshan N (2017) An efficient vision-based traffic light detection and state recognition for autonomous vehicles. In: 2017 IEEE intelligent vehicles symposium (IV), IEEE, pp 606–611

  • Shi Z, Zou Z, Zhang C (2015) Real-time traffic light detection with adaptive background suppression filter. IEEE Trans Intell Transp Syst 17(3):690–700

    Article  Google Scholar 

  • Wang K, Xiong Z (2016) Visual enhancement method for intelligent vehicle’s safety based on brightness guide filtering algorithm thinking of the high tribological and attenuation effects. J Balk Tribol Assoc 22(2A):2021–2031

    Google Scholar 

  • Wang JG, Zhou LB (2018) Traffic light recognition with high dynamic range imaging and deep learning. IEEE Trans Intell Transp Syst 20(4):1341–1352

    Article  Google Scholar 

  • Wang K, Huang Z, Zhong Z (2014) Simultaneous multi-vehicle detection and tracking framework with pavement constraints based on machine learning and particle filter algorithm. Chin J Mech Eng 27(6):1169–1177

    Article  Google Scholar 

  • Wang K, Huang X, Chen J, Cao C, Xiong Z, Chen L (2019) Forward and backward visual fusion approach to motion estimation with high robustness and low cost. Remote Sensing 11(18):2139

    Article  Google Scholar 

  • Wang K, Li G, Chen J, Long Y, Chen T, Chen L, Xia Q (2020a) The adaptability and challenges of autonomous vehicles to pedestrians in urban China. Accid Anal Prev 145:105692. https://doi.org/10.1016/j.aap.2020.105692

    Article  Google Scholar 

  • Wang K, Zhang S, Chen J, Ren F, Xiao L (2020b) A feature-supervised generative adversarial network for environmental monitoring during hazy days. Sci Total Environ 748:141445. https://doi.org/10.1016/j.scitotenv.2020.141445

    Article  Google Scholar 

  • Wang k, Ma S, Chen J, Lu J (2021) Approaches challenges and applications for deep visual odometry toward to complicated and emerging areas. IEEE Trans Cogn Dev Syst. https://doi.org/10.1109/TCDS.2020.3038898

    Article  Google Scholar 

Download references

Funding

This research was funded by National Natural Science Foundation of China (Grant No. 51605054), National Key Research and Development Program of China (SQ2020YFF0410766), Natural Science Foundation of Chongqing (cstc2020jcyj-msxmX0575), Chongqing Technology Innovation and application development project (cstc2020jscx-msxmX0109 and cstc2019jscx-fxydX0063), Fundamental Research Funds for the Central Universities (2020CDJ-LHZZ-042).

Author information

Authors and Affiliations

  1. School of Automobile Engineering, State Key Lab of Mechanical Transmission, Chongqing University, Chongqing, 400044, China

    Ke Wang & Xinwei Tang

  2. School of Vehicle and Mobility, Tsinghua University, Beijing, 401122, China

    Shulian Zhao

  3. School of Automobile Engineering, Chongqing University, Chongqing, 400044, China

    Yuchen Zhou

Authors
  1. Ke Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinwei Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shulian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuchen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ke Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, K., Tang, X., Zhao, S. et al. Simultaneous detection and tracking using deep learning and integrated channel feature for ambint traffic light recognition. J Ambient Intell Human Comput 13, 271–281 (2022). https://doi.org/10.1007/s12652-021-02900-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-02900-y

Keywords

Search

Navigation