Skip to main content
SpringerLink
Log in

Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic Flow

  • Published:
Optoelectronics, Instrumentation and Data Processing Aims and scope

Abstract

We propose a two-stage method for adaptive traffic signal control based on an estimate of the predicted weighted flow of vehicles passing through an intersection. At the first stage, the time for each vehicle to pass the intersection is estimated using an artificial neural network model; then, the predicted flow of vehicles through the intersection for a given phase of the traffic light cycle is estimated. At the second stage, the weighted flow is estimated and the vehicle waiting time is considered. The proposed method for choosing the phase of a traffic light is based on maximizing the weighted traffic flow. The results of experimental studies allow one to conclude that the proposed approach is superior to classical approaches and state-of-the-art methods of traffic signal control based on reinforcement learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

REFERENCES

  1. Decree of the President of the Russian Federation on December 1, 2016 no. 642. http://kremlin.ru/acts/bank/41449. Cited September 19, 2022.

  2. D. Schrank, L. Albert, B. Eisele, and T. Lomax, 2021 Urban Mobility Report. https://trid.trb.org/view/1862637. Cited May 20, 2022.

  3. V. V. Myasnikov, A. A. Agafonov, and A. S. Yumaganov, ‘‘A deterministic predictive traffic signal control model in intelligent transportation and geoinformation systems,’’ Kom’yut. Opt. 45, 917–925 (2021). https://doi.org/10.18287/2412-6179-CO-1031

    Article  Google Scholar 

  4. H. Wei, G. Zheng, V. Gayah, and Z. Li, ‘‘A survey on traffic signal control methods,’’ 2020. arXiv:1904.08117 [cs.LG]

  5. M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. Kotsialos, and Yi. Wang, ‘‘Review of road traffic control strategies,’’ Proc. IEEE 91, 2043–2065 (2003). https://doi.org/10.1109/JPROC.2003.819610

    Article  Google Scholar 

  6. K.-L. A. Yau, J. Qadir, H. L. Khoo, and M. H. Ling, ‘‘A survey on reinforcement learning models and algorithms for traffic signal control,’’ ACM Comput. Surv. 50, 34 (2017). https://doi.org/10.1145/3068287

    Article  Google Scholar 

  7. M. Gregurić, M. Vujić, C. Alexopoulos, and M. Miletić, ‘‘Application of deep reinforcement learning in traffic signal control: An overview and impact of open traffic data,’’ Appl. Sci. 10, 4011 (2020). https://doi.org/10.3390/app10114011

    Article  Google Scholar 

  8. A. Haydari and Y. Yılmaz, ‘‘Deep reinforcement learning for intelligent transportation systems: A survey,’’ IEEE Trans. Intell. Transp. Syst. 23 (1), 11–32 (2022). https://doi.org/10.1109/TITS.2020.3008612

    Article  Google Scholar 

  9. B. Abdulhai, R. Pringle, and G. Karakoulas, ‘‘Reinforcement learning for true adaptive traffic signal control,’’ J. Transp. Eng. 129, 278–285 (2003).

    Article  Google Scholar 

  10. M. Eom and B.-In Kim, ‘‘The traffic signal control problem for intersections: A review,’’ Eur. Transp. Res. Rev. 12, 50 (2022). https://doi.org/10.1186/s12544-020-00440-8

    Article  Google Scholar 

  11. R. M. Savithramma and R. Sumathi, ‘‘Road traffic signal control and management system: A survey,’’ in 3rd Int. Conf. on Intelligent Sustainable Systems (ICISS), Palladam, India, 2020 (IEEE, 2020), pp. 104–110. https://doi.org/10.1109/ICISS49785.2020.9315970

  12. F. V. Webster, Traffic Signal Settings (H.M. Stationery Office, London, 1958).

    Google Scholar 

  13. J. Little, M. Kelson, and N. Gartner, ‘‘MAXBAND: A program for setting signals on arteries and triangular networks,’’ Transp. Res. Record J. Transp. Res. Board 795, 40–46 (1981).

  14. M.-T. Li and A. Gan, ‘‘Signal timing optimization for oversaturated networks using TRANSYT-7F,’’ Transp. Res. Record. 1683, 118–126 (1999).

  15. S. El-Tantawy and B. Abdulhai, ‘‘An agent-based learning towards decentralized and coordinated traffic signal control,’’ in 13th Int. IEEE Conf. on Intelligent Transportation Systems, Funchal, Portugal, 2010 (IEEE, 2010), pp. 665–670. https://doi.org/10.1109/ITSC.2010.5625066

  16. S.-B. Cools, C. Gershenson, and B. D’Hooghe, ‘‘Self-organizing traffic lights: A realistic simulation,’’ in Advances in Applied Self-Organizing Systems, Ed. by M. Prokopenko, Advanced Information and Knowledge Processing (Springer, London, 2013), pp. 45–55. https://doi.org/10.1007/978-1-4471-5113-5_3

  17. P. Varaiya, ‘‘The max-pressure controller for arbitrary networks of signalized intersections,’’ in Advances in Dynamic Network Modeling in Complex Transportation Systems, Ed. by S. Ukkusuri and K. Ozbay, Complex Networks and Dynamic Systems, Vol. 2 (Springer, New York, 2013), pp. 27–66. https://doi.org/10.1007/978-1-4614-6243-9_2

  18. W. Genders and S. Razavi, ‘‘An open-source framework for adaptive traffic signal control,’’ 2019. arXiv:1909.00395 [eess.SY]

  19. A. Agafonov and V. Myasnikov, ‘‘Traffic signal control: A double q-learning approach,’’ in 16th Conf. on Computer Science and Intelligence Systems (FedCSIS), Sofia, 2021 (IEEE, 2021), pp. 365–369. doi 10.15439/2021F109

  20. H. Wei, G. Zheng, H. Yao, and Z. Li, ‘‘IntelliLight: A reinforcement learning approach for intelligent traffic light control,’’ in KDD’18: Proc. 24th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining, London, 2018, pp. 2496–2505. doi 10.1145/3219819.3220096

  21. Z. Zhang, J. Yang, and H. Zha, ‘‘Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization,’’ in AAMAS’20: Proc. 19th Int. Conf. on Autonomous Agents and MultiAgent Systems, Auckland, New Zealand, 2020 (Int. Foundation for Autonomous Agents and Multiagent Systems, Richland, S.C., 2020), pp. 2083–2085. arXiv:1909.10651 [cs.LG]

  22. G. Zheng, Yu. Xiong, X. Zang, J. Feng, H. Wei, H. Zhang, Yo. Li, K. Xu, and Zh. Li, ‘‘Learning phase competition for traffic signal control,’’ in CIKM’19: Proc. 28th ACM Int. Conf. on Information and Knowledge Management, Bejing, 2019 (Association for Computing Machinery, New York, 2019), pp. 1963–1972. https://doi.org/10.1145/3357384.3357900

  23. H. Wei, C. Chen, G. Zheng, K. Wu, V. Gayah, K. Xu, and Zh. Li, ‘‘PressLight: Learning max pressure control to coordinate traffic signals in arterial network,’’ in KDD ’19: Proc. 25th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Anchorage, Alaska, 2019 (Association for Computing Machinery, New York, 2019), pp. 1290–1298. doi 10.1145/3292500.3330949

  24. Y. K. Chin, L. K. Lee, N. Bolong, S. S. Yang, and K. T. K. Teo, ‘‘Exploring Q-learning optimization in traffic signal timing plan management,’’ in Proc. 3rd Int. Conf. on Computational Intelligence, Communication Systems and Networks, Bali, Indonesia, 2011 (IEEE, 2011), pp. 269–274. https://doi.org/10.1109/CICSyN.2011.64

  25. J. Gu, Y. Fang, Zh. Sheng, and P. Wen, ‘‘Double deep Q-network with a dual-agent for traffic signal control,’’ Appl. Sci. 10, 1622 (2020). https://doi.org/10.3390/app10051622

    Article  Google Scholar 

  26. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, ‘‘Proximal policy optimization algorithms,’’ 2017. arXiv:1707.06347 [cs]

  27. J. Ault, J. P. Hanna, and G. Sharon, ‘‘Learning an interpretable traffic signal control policy,’’ in AAAMAS ’20: Proc. 19th Int. Conf. on Autonomous Agents and MultiAgent Systems, Auckland, New Zealand, 2020 (Int. Foundation for Autonomous Agents and Multiagent Systems, Richland, S.C., 2020), pp. 88–96. arXiv:1912.11023 [cs.LG]

  28. Y. Li, J. He, and Ya. Gao, ‘‘Intelligent traffic signal control with deep reinforcement learning at single intersection,’’ in ICCAI 2021: 7th Int. Conf. on Computing and Artificial Intelligence, Tianjin, China, 2021 (Association for Computing Machinery, New York, 2021), pp. 399–406. https://doi.org/10.1145/3467707.3467767

  29. M. Aslani, M. S. Mesgari, and M. Wiering, ‘‘Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events,’’ Transp. Res. Part C: Emerging Technol. 85, 732–752 (2017). https://doi.org/10.1016/j.trc.2017.09.020

    Article  Google Scholar 

  30. Sh. Yang, Bo Yang, Zh. Kang, and L. Deng, ‘‘IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control,’’ Neural Networks 139, 265–277 (2021). https://doi.org/10.1016/j.neunet.2021.03.015

    Article  Google Scholar 

  31. Y. Wu, E. Mansimov, S. Liao, R. Grosse, and J. Ba, ‘‘Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation,’’ in NIPS’17: Proc. 31st Int. Conf. on Neural Information Processing Systems, Ed. by U. von Luxburg, I. Guyon, S. Bengio, H. Wallach, and R. Fergus (Curran Associates, Red Hook, N.Y., 2017), pp. 5285–5294. arXiv:1708.05144 [cs]

  32. T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine, ‘‘Soft actor-critic algorithms and applications,’’ 2019. arXiv:1812.05905 [cs.LG]

  33. Traffic Lights — SUMO Documentation. https://sumo.dlr.de/docs/Simulation/Traffic_Lights.html. Cited June 8, 2022.

  34. TAPAS Cologne — SUMO Documentation. https://sumo.dlr.de/docs/Data/Scenarios/TAPASCologne. html.Cited June 8, 2022.

  35. RESCO: Reinforcement learning benchmarks for traffic signal control, 2021. https://github.com/Pi-Star-Lab/RESCO. Cited June 8, 2022.

  36. PFRL: A PyTorch-based deep reinforcement learning library. https://github.com/pfnet/pfrl. Cited June 8, 2022.

Download references

Funding

This work was supported by the Russian Science Foundation, project no. 21-11-00321 (https://rscf.ru/en/project/21-11-00321/).

Author information

Authors and Affiliations

  1. Samara National Research University, 443086, Samara, Russia

    A. A. Agafonov, A. S. Yumaganov & V. V. Myasnikov

Authors
  1. A. A. Agafonov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. S. Yumaganov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. V. Myasnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. A. Agafonov.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Translated by I. Obrezanova

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agafonov, A.A., Yumaganov, A.S. & Myasnikov, V.V. Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic Flow. Optoelectron.Instrument.Proc. 58, 503–513 (2022). https://doi.org/10.3103/S8756699022050016

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S8756699022050016

Keywords:

Search

Navigation