Smart fog based workflow for traffic control networks

https://doi.org/10.1016/j.future.2019.02.058Get rights and content

Highlights

  • A smart fog based workflow architecture is proposed.

  • The architecture relies on the fog computing paradigm and a distributed reinforcement learning algorithm.

  • Workflows designed to relieve traffic congestion, which are connecting traffic lights, vehicles, Fog nodes and traffic cloud.

  • The framework outperforms traditional systems and provides high practicability in future research for building the intelligent transportation system.

Abstract

In this paper, we propose a novel traffic control architecture which is based on fog computing paradigm and reinforcement leaning technologies. We firstly provide an overview of this framework and detail the components and workflows designed to relieve traffic congestion. These workflows, which are connecting traffic lights, vehicles, Fog nodes and traffic cloud, aim to generate traffic light control flow and communication flow for each intersection to avoid a traffic jam. In order to make the whole city’s traffic highly efficient, the fog computing paradigm and a distributed reinforcement learning algorithm is designed to overcome communication bandwidth limitation and local optimal traffic control flow, respectively. We also demonstrate that our framework outperforms traditional systems and provides high practicability in future research for building the intelligent transportation system.

Introduction

With the development of urbanization, ever-growing vehicles bring huge convenience to people’s mobility, on the other hand, lead to traffic jams, causing several serious social problems: longer driving time, more fuel consumption and heavier air pollution [1]. For example, the loss of extra driving time and gasoline due to traffic congestion in the US was already up to 121 billion US dollars in 2011, and carbon dioxide produced during congestion was 25,396 tons, while there were 24 billion US dollars loss and 4535 tons carbon dioxide in 1982, respectively [2].

Growing vehicles, shortage of traffic infrastructures, inefficient traffic signal control and insufficient online traffic condition information (for example, whether traffic jams or accidents have happened on planned routes for vehicles), are primary factors for traffic congestion. It is unrealistic to ban increasing vehicles or invest more traffic infrastructures, especially for developing countries like China. To address this issue, the relatively practical methods is to focus on the last two factors. Thus, our research motivation is to (1) make real-time traffic condition information available for every vehicle on the intersection and (2) improve the efficiency of traffic light signal control.

In recent years, as the rapid development of information and communication technologies (ICT) and the advances of the Internet of Things (IoT) [3], equipping vehicles with wireless communication capabilities, has been a new standard for car makers, especially in electric vehicles. The vehicle is no longer a relatively closed-system, instead nowadays they can connect to the Internet and even other vehicles. Connected vehicles network could get traffic situation around and ahead for vehicles, which could help vehicles to alter their route dynamically to detour around the traffic jam [4]. Meanwhile, with the significant progress of evolution of AI technologies, autonomous driving cars have appeared on the road in several cities in the US, it is easier and more precise for a vehicle to tune its route at any time [5]. Moreover, traffic lights signal controls have significantly been improved owing to deep reinforcement learning [6]. For example, reinforcement algorithms can be applied to control the green light timing and red light timing adaptively.

In spite of a great diversity of technologies having improved the current traffic system, there are still a few unresolved problems. Such as: (1) high latency communication for vehicles with the number of vehicle growing; (2) some reinforcement learning algorithms merely making one intersection traffic flow smoother rather than for the local region or even the whole city [7]; (3) lastly, multi-agent reinforcement learning algorithms are designed to address problem (2) above are limited by communication bandwidth to apply the real traffic infrastructures [8].

An integrated solution for traffic congestion is designed to address smart traffic on a crossroad in the real world. As for contribution this paper, we would focus on optimizing connected vehicles network and reinforcement learning methods, which are the key players to the evolution to the next generation of intelligent transportation systems. In this paper, we propose a novel traffic control architecture, which integrated workflow based fog computing paradigm and a distributed reinforcement learning algorithm. We will give an overall solution to traffic congestion, which is more suitable for driver-less vehicles to some extent. The framework is composed of three components, including connected vehicles network as terminals at the bottom, intelligent fog computing nodes in the middle and traffic cloud center on the top. Connected vehicle network component, is designed to send the internal information of a vehicle such as its current speed, destination. And it receives the outside information flow from intelligent Fog Nodes, which are applied to help the vehicle inner system or driver to make better decision in order to avoid traffic jam. Intelligent fog computing node component generates dynamic traffic light control flow and delivers traffic condition information flow to control traffic lights, and also inform vehicles traffic condition information, respectively. Traffic cloud center analyzes the data flowed from local Fog Nodes and produces generalized control flow back to Fog Nodes to help them to jump out of local optimal, which means optimizing one or a few traffic lights on crossroad not all traffic lights in the city. It also delivers the traffic information to the Fog Node that requires it for the specific vehicle, so that every vehicle has its own information from the cloud.

The paper’s contribution is (1) designed to make real-time traffic condition information available to vehicles and (2)improve the efficiency of traffic signal control with low latency communication delay.

The remainder of this paper is organized as follows: Section 2 introduces preliminary and related works, such as fog computing, connected vehicle network, traffic lights signal control. Section 3 describes our smart traffic network architecture components. Section 4 details the intelligent workflows based on fog computing paradigm. Section 5 shows the evaluation of our architecture in comparison with traditional frameworks. Section 6 concludes the paper and addresses the future work.

Section snippets

Preliminary and related works

This study is primarily related to two broad categories of research, one on the IoT, another on AI technologies. In this section, we briefly introduce cloud computing and fog computing paradigms, connected vehicles network, vehicular automation and reinforcement learning methods for traffic lights control and the related works to address the traffic congestion.

Smart traffic network architecture

Smart traffic network architecture is designed specifically to address the three challenges raised above. In this section, we detail the novel framework based on fog computing, including connected vehicle network component, intelligent fog computing node component and cloud computing component as shown in Fig. 4. We will describe these individual components respectively.

Smart traffic network workflow

Smart traffic network workflows, which are mainly generated by components we detailed above, are composed of four parts, including: (1) vehicle information flow; (2) traffic condition information flow; (3) Fog Node control flow; (4) traffic cloud control flow. Fig. 5 shows these workflows altogether making the whole architecture work.

Simulation and experiments

In this section we have utilized Simulation of Urban Mobility (SUMO) [25], which is an open source simulator for traffic environment. Meanwhile, a Fig. 6 shows, we have assumed that there are six intersection nodes in one city. We have conducted a set of experiments implementing the SUMO TraCI (Traffic Control Interface) extension, which allows for dynamic control of the traffic lights at runtime, and we have compared it to another traffic control frameworks. We will demonstrate that our

Conclusions

In this paper, we have proposed a novel smart traffic control architecture based on fog computing paradigm and a distributed reinforcement learning algorithm to lower the probability of traffic congestion in the city. It can overcome communication bandwidth limitation among vehicles by producing smart traffic control signal locally and delivering traffic condition signal intelligently. Workflows in the framework are designed to make the architecture work efficiently. Although the framework

Acknowledgments

This work was supported by National Natural Science Foundation of China under Grant No. 61402210 and 60973137, State Grid Corporation Science and Technology Project under Grant No. SGGSKY00FJJS1700302, Program for New Century Excellent Talents in University, China under Grant No. NCET-12-0250, Major National Project of High Resolution Earth Observation System under Grant No. 30-Y20A34-9010-15/17, Strategic Priority Research Program of the Chinese Academy of Sciences with Grant No. XDA03030100,

Qiang Wu is a Ph.D. candidate in Lanzhou University. He received BS degree in Computer Science and Technology from Beijing University of Posts and Telecommunications in 2009, and received MS degree from Lanzhou Jiaotong University in 2015. His research interests include deep learning, fog computing, and reinforcement learning for Intelligent Transport System.

References (27)

  • AtzoriL. et al.

    The internet of things: A survey

    Comput. Netw.

    (2010)
  • YuanD. et al.

    A data placement strategy in scientific cloud workflows

    Future Gener. Comput. Syst.

    (2010)
  • AlsabaanM. et al.

    Vehicular networks for a greener environment: A survey

    IEEE Commun. Surv. Tutor.

    (2013)
  • T.J. Schrank, The 2012 urban mobility report, Traffic...
  • AndaJ. et al.

    Vgrid: vehicular adhoc networking and computing grid for intelligent traffic control

  • ThrunS.

    Toward robotic cars

    Commun. ACM

    (2010)
  • MozerS. et al.

    Reinforcement learning: An introduction

    IEEE Trans. Neural Netw.

    (2005)
  • ZhaoD. et al.

    Computational intelligence in urban traffic signal control: A survey

    IEEE Trans. Syst. Man Cybern. C

    (2012)
  • BazzanA.L.C.

    Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

    Autonom. Agents Multi-Agent Syst.

    (2009)
  • LiS. et al.

    The Internet of Things: A Survey

    (2015)
  • LuN. et al.

    Connected vehicles: Solutions and challenges

    Internet Things J. IEEE

    (2014)
  • DresnerK. et al.

    A multiagent approach to autonomous intersection management

    J. Artif. Intell. Res.

    (2008)
  • Rios-TorresJ. et al.

    Online optimal control of connected vehicles for efficient traffic flow at merging roads

  • Cited by (29)

    • Reinforcement learning in urban network traffic signal control: A systematic literature review

      2022, Expert Systems with Applications
      Citation Excerpt :

      They concluded that ERL in distributed edge servers has much better scalability and faster Deep NN training than the cloud service. Wu et al. (2019) proposed a traffic control architecture based on fog computing paradigm and a distributed RL algorithm (that connects traffic signals, vehicles, fog nodes and traffic cloud) to overcome communication bandwidth limitation and reduce communication delay, make real-time traffic condition information available to vehicles, and lower the probability of traffic congestion in the city through generating traffic signal control flow and communication flow for each intersection. This is not only suitable for current vehicles but also more useful for driverless vehicles anticipated in the future, as it will be able to plan its route much more intelligently with information from the fog node.

    View all citing articles on Scopus

    Qiang Wu is a Ph.D. candidate in Lanzhou University. He received BS degree in Computer Science and Technology from Beijing University of Posts and Telecommunications in 2009, and received MS degree from Lanzhou Jiaotong University in 2015. His research interests include deep learning, fog computing, and reinforcement learning for Intelligent Transport System.

    Dr. Jun Shen was awarded Ph.D. in 2001 at Southeast University, China. He held positions at Swinburne University of Technology in Melbourne and University of South Australia in Adelaide before 2006. He is an Associate Professor in School of Computing and Information Technology at University of Wollongong in Wollongong, NSW of Australia, where he had been Head of Postgraduate Studies, and Chair of School Research Committee since 2014. He is a senior member of three institutions: IEEE, ACM and ACS. He has published more than 120 papers in journals and conferences in CS/IT areas. His expertise includes computational intelligence, Web services, Cloud computing and learning technologies including MOOC. He has been Editor, PC Chair, Guest Editor, PC Member for numerous journals and conferences published by IEEE, ACM, Elsevier and Springer. A/Prof Shen is also a current member of ACM/AIS Task Force on Curriculum MSIS 2016.

    Binbin Yong received his master’s degree in Computer Science and Technology from Lanzhou University in 2012, and received Ph.D. in the School of Information Science and Engineering, Lanzhou University in 2017. He is researching in parallel computing of GPU, machine learning, deep learning and general vector machine.

    Jianqing Wu is a Ph.D. candidate at University of Wollongong, Australia. He received the B.Sc. in computer science from Manchester Metropolitan University, UK, in 2014 and Master of Research in computer science from the University of Liverpool in 2016 respectively. His research interests include machine learning and data analytics for Intelligent Transport System.

    Fucun Li graduated from University of Wollongong with master degree of Advanced Information Technology studies in 2014. He is currently a Ph.D. candidate in University of Wollongong. His main research interests are how to use information system to improve competitiveness of iron companies and how to use machine learning to control quality and cost of iron products.

    Jinqiang Wang is a master candidate in Lanzhou University. His research interests include machine learning, deep learning and deep reinforcement learning.

    Qingguo Zhou received the BS and MS degrees in Physics from Lanzhou University in 1996 and 2001, respectively, and received Ph.D. in Theoretical Physics from Lanzhou University in 2005. Now he is a professor of Lanzhou University and working in the School of Information Science and Engineering. He is also a Fellow of IET. He was a recipient of IBM Real-Time Innovation Award in 2007, a recipient of Google Faculty Award in 2011, and a recipient of Google Faculty Research Award in 2012. His research interests include safety-critical systems, embedded systems, and real-time systems.

    View full text