计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (11): 88-96.doi: 10.11896/jsjkx.221000201

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于时间感知Transformer的交通流预测方法

刘起东1,2,3, 刘超越1, 邱紫鑫1, 高志敏1,2,3, 郭帅1,2,3, 刘冀钊4, 符明晟5   

  1. 1 郑州大学计算机与人工智能学院 郑州 450001
    2 智能集群系统教育部工程研究中心 郑州 450001
    3 国家超级计算郑州中心 郑州 450001
    4 兰州大学信息科学与工程学院 兰州 730000
    5 电子科技大学计算机科学与工程学院 成都 611731
  • 收稿日期:2022-10-24 修回日期:2023-05-11 出版日期:2023-11-15 发布日期:2023-11-06
  • 通讯作者: 刘起东(ieqdliu@zzu.edu.cn)
  • 基金资助:
    国家自然科学基金(62276238,61906174,62036010);中国博士后科学基金资助项目(2022T150590,2020M672275);河南省自然科学基金(232300421095);河南省重点研发与推广专项(222102210248)

Time-aware Transformer for Traffic Flow Forecasting

LIU Qidong1,2,3, LIU Chaoyue1, QIU Zixin1, GAO Zhimin1,2,3, GUO Shuai1,2,3, LIU Jizhao4, FU Mingsheng5   

  1. 1 School of Computer and Artificial Intelligence,Zhengzhou University,Zhengzhou 450001,China
    2 Engineering Research Center of Intelligent Swarm Systems,Ministry of Education,Zhengzhou 450001,China
    3 National Supercomputing Center in Zhengzhou,Zhengzhou 450001,China
    4 School of Information Science and Engineering,Lanzhou University,Lanzhou 730000,China
    5 School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China
  • Received:2022-10-24 Revised:2023-05-11 Online:2023-11-15 Published:2023-11-06
  • About author:LIU Qidong,born in 1990,Ph.D,asso-ciate professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include sward intelligence,motion planning and spatio-temporal data analysis.
  • Supported by:
    National Natural Science Foundation of China(62276238,61906174,62036010),China Postdoctoral Science Foundation(2022T150590,2020M672275),Natural Science Foundation of Henan Province,China(232300421095) and Henan Provincial Key Science and Technology Research Projects(222102210248).

PDF (PC)

2544

摘要： 作为智能交通系统的关键一环,交通流预测面临着长时预测不准的难题,其主要挑战在于交通流数据本身具有复杂的时空关联。近年来,Transformer的提出使得时序数据预测的研究取得了巨大进展,但将Transformer应用于交通流预测仍然存在以下两个问题:1)静态的注意力机制难以捕获交通流随时间动态变化的时空依赖关系;2)采用自回归的预测方式会引发严重的误差累积现象。针对以上问题,提出了一种基于时间感知Transformer的交通流预测模型。首先,设计了一种新的时间感知注意力机制,可以根据时间特征定制注意力计算方案,从而更精准地反映时空依赖关系;其次,在Transformer的训练阶段舍弃了Teacher Forcing机制,并采用非自回归的预测方式来避免误差累积问题;最后,在两个真实交通数据集上进行实验,实验结果表明,所提方法可以有效捕获交通流的时空依赖,相比最优的基线方法,长时预测性能提升了2.09%～ 4.01%。

关键词: 交通流预测, 时空建模, 时间感知注意力机制, 非自回归, Transformer

Abstract: As a key part of intelligent transportation systems,traffic flow forecasting faces the challenge of long-term prediction inaccuracy.The key factor is that the traffic flow has complicated spatial and temporal correlations.Recently,the emerging success of Transformer has shown promising results in time series analysis.However,there are two obstacles when applying Transformer to traffic flow forecasting:1)it's difficult for the static attention mechanisms to capture the dynamic changes of traffic flow along the space and time dimensions;2)the autoregressive decoder in transformer could cause error accumulation problem.To address the above problems,this paper proposes a time-aware Transformer(TAformer) for traffic flow forecasting.Firstly,it proposes a time-aware attention mechanism that can customize attention calculation solution according to the time features,so as to estimate the spatial and temporal dependencies more accurately.Secondly,it discards the teacher forcing mechanism during the training phase and proposes a non-autoregressive inference method to avoid the problem of error accumulation.Finally,extensive experiments on two real traffic datasets show that the proposed method can effectively capture the spatial-temporal dependence of traffic flow.Compared with the state-of-the-art baseline method,the proposed method improves the performance of long-term prediction by 2.09%～4.01%.

Key words: Traffic flow Forecasting, Spatial-Temporal modeling, Time-aware attention, Non-autoregressive, Transformer

中图分类号: 

  • TP391
[1]LI M,ZHU Z.Spatial-temporal fusion graph neural networks for traffic flow forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press.2021:4189-4196.
[2]SONG C,LIN Y,GUO S,et al.Spatial-temporal synchronous graph convolutional networks:A new framework for spatial-temporal network data forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:914-921.
[3]DEFFERRARD M,BRESSON X,VANDERGHEYNST P.Convolutional neural networks on graphs with fast localized spectral filtering[C]//Advances in Neural Information Processing Systems.Barcelona,Spain,2016:3837-3845.
[4]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[5]GRAVES A.Long short-term memory[J].Supervised Sequence Labelling with Recurrent Neural Networks,2012,385:37-45.
[6]CHO K,VAN MERRIËNBOER B,BAHDANAU D,et al.Onthe properties of neural machine translation:Encoder-decoder approaches[J].arXiv:1409.1259,2014.
[7]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.Long Beach,CA,2017:5998-6008.
[8]CAI L,JANOWICZ K,MAI G,et al.Traffic transformer:Capturing the continuity and periodicity of time series for traffic forecasting[J].Transactions in GIS,2020,24(3):736-755.
[9]PARK C,LEE C,BAHNG H,et al.ST-GRAT:A novel spatio-temporal graph attention networks for accurately forecasting dynamically changing road speed[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management.Ireland,2020:1215-1224.
[10]CHEN C,LIU Y,CHEN L,et al.Bidirectional Spatial-Temporal Adaptive Transformer for Urban Traffic Flow Forecasting[J].IEEE Transactions on Neural Networks and Learning Systems,2023,34(10):6913-6925.
[11]YAN H,MA X,PU Z.Learning dynamic and hierarchical traffic spatiotemporal features with transformer[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(11):22386-22399.
[12]WILLIAMS R J,ZIPSER D.A learning algorithm for continually running fully recurrent neural networks[J].Neural Computation,1989,1(2):270-280.
[13]SCHMIDT F.Generalization in generation:A closer look at exposure bias[J].arXiv:1910.00292,2019.
[14]BENGIO S,VINYALS O,JAITLY N,et al.Scheduled sampling for sequence prediction with recurrent neural networks[C]//Advances in Neural Information Processing Systems.Montreal,Quebec,Canada,2015:1171-1179.
[15]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.Long Beach,CA,2017:1024-1034.
[16]VELICKOVIC P,CUCURULL G,CASANOVA A,et al.Graph attention networks[C]//6th International Conference on Lear-ning Representations.Vancouver,2018.
[17]ARORA S.A survey on graph neural networks for knowledge graph completion[J].arXiv:2007.12374,2020.
[18]WU S,SUN F,ZHANG W,et al.Graph neural networks in re-commender systems:a survey[J].ACM Computing Surveys(CSUR),2022,55(5):1-37.
[19]JIANG W,LUO J.Graph neural network for traffic forecasting:A survey[J].Expert Systems with Applications,2022,207:117921.
[20]YU B,YIN H,ZHU Z.Spatio-temporal graph convolutional networks:A deep learning framework for traffic forecasting[J].arXiv:1709.04875,2017.
[21]ZHAO L,SONG Y,ZHANG C,et al.T-gcn:A temporal graphconvolutional network for traffic prediction[J].IEEE Transactions on Intelligent Transportation Systems,2019,21(9):3848-3858.
[22]LI Y,YU R,SHAHABI C,et al.Diffusion convolutional recur-rent neural network:Data-driven traffic forecasting[J].arXiv:1707.01926,2017.
[23]WU Z,PAN S,LONG G,et al.Graph wavenet for deep spatial-temporal graph modeling[J].arXiv:1906.00121,2019.
[24]ZHANG Q,CHANG J,MENG G,et al.Spatio-temporal graph structure learning for traffic forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:1177-1185.
[25]ZHANG J,SHI X,XIE J,et al.GAAN:Gated attention networks for learning on large and spatiotemporal graphs[J].ar-Xiv:1803.07294,2018.
[26]GUO S,LIN Y,FENG N,et al.Attention based spatial-temporal graph convolutional networks for traffic flow forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Honolulu,Hawaii,USA:AAAI Press,2019:922-929.
[27]ZHENG C,FAN X,WANG C,et al.Gman:A graph multi-at-tention network for traffic prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:1234-1241.
[28]XU M,DAI W,LIU C,et al.Spatial-temporal transformer networks for traffic flow forecasting[J].arXiv:2001.02908,2020.
[29]REN Y,ZHANG J.Fake news detection on news-oriented he-terogeneous information networks through hierarchical graph attention[C]//2021 International Joint Conference on Neural Networks(IJCNN).Shenzhen:IEEE,2021:1-8.
[30]HU Z,DONG Y,WANG K,et al.Heterogeneous graph transformer[C]//Proceedings of The Web Conference 2020.Taipei:2020:2704-2710.
[31]LIU Q,LONG C,ZHANG J,et al.Aspect-Aware Graph Attention Network for Heterogeneous Information Networks[J/OL].IEEE Transactions on Neural Networks and Learning Systems,2022.https://ieeexplore.ieee.org/abstract/document/9930625.
[32]GROVER A,LESKOVEC J.node2vec:Scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mi-ning.San Francisco:ACM,2016:855-864.
[33]ORHAN A E,PITKOW X.Skip connections eliminate singularities[J].arXiv:1701.09175,2017.
[34]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Computer Society,2016:770-778.
[35]LU B,GAN X,JIN H,et al.Spatiotemporal adaptive gatedgraph convolution network for urban traffic flow forecasting[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management.Ireland:ACM,2020:1025-1034.
[36]LIU J,GUAN W.A summary of traffic flow forecasting me-thods[J].Journal of Highway and Transportation Research and Development,2004,21(3):82-85.
[37]XU D,WANG Y,JIA L,et al.Real-time road traffic state prediction based on ARIMA and Kalman filter[J].Frontiers of Information Technology & Electronic Engineering,2017,18(2):287-302.
[38]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequencelearning with neural networks[C]//Advances in Neural Information Processing Systems.2014:3104-3112.
[39]CHEN W,CHEN L,XIE Y,et al.Multi-range attentive bicomponent graph convolutional network for traffic forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:3529-3536.
[40]MICIKEVICIUS P,NARANG S,ALBEN J,et al.Mixed precision training[J].arXiv:1710.03740,2017.
[41]CHEN T,XU B,ZHANG C,et al.Training deep nets with sublinear memory cost[J].arXiv:1604.06174,2016.
[42]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发