Jiyuan Ren
ABSTRACT
In disaster scenarios, unmanned aerial vehicles (UAVs) can serve as mobile base stations because of their maneuverability and synergy. However, due to constrained UAV communication capabilities and limited battery life, UAV base stations resource allocation for mobile sensors in a data-heterogeneous environment is a significant challenge when optimizing communication quality. To address this, we propose AGUZero, an attention-based graph reinforcement learning (RL) framework. Inspired by MuZero [27], AGUZero is designed to handle dynamic and uncontrollable environments based on Monte Carlo Tree Search (MCTS). Additionally, to tackle data heterogeneity, AGUZero represents the states using heterogeneous sub-graphs and employs an attention-based model to capture relationships among UAVs and sensors. The experimental results show that AGUZero outperforms other baseline models consistently when either the number of UAVs or the number of sensors is varying. AGUZero improves the data transmission ratio by 11.03% and 10.35% in the two cases respectively.
- Shakil Ahmed and Boulat A. Bash. 2019. Average Worst-Case Secrecy Rate Maximization via UAV and Base Station Resource Allocation. 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) (2019), 1176–1181.Google Scholar
Digital Library
- Akram Al-Hourani, Sithamparanathan Kandeepan, and Simon Lardner. 2014. Optimal LAP Altitude for Maximum Coverage. IEEE Wireless Communications Letters 3, 6 (2014), 569–572. https://doi.org/10.1109/LWC.2014.2342736Google ScholarCross Ref
- Mohamed Alzenad, Amr El-Keyi, Faraj Lagum, and Halim Yanikomeroglu. 2017. 3-D Placement of an Unmanned Aerial Vehicle Base Station (UAV-BS) for Energy-Efficient Maximal Coverage. IEEE Wireless Communications Letters 6 (2017), 434–437.Google ScholarCross Ref
- Xinlei Chen, Aveek Purohit, Carlos Ruiz Dominguez, Stefano Carpin, and Pei Zhang. 2015. Drunkwalk: Collaborative and adaptive planning for navigation of micro-aerial sensor swarms. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. 295–308.Google ScholarDigital Library
- Xinlei Chen, Aveek Purohit, Shijia Pan, Carlos Ruiz, Jun Han, Zheng Sun, Frank Mokaya, Patric Tague, and Pei Zhang. 2017. Design experiences in minimalistic flying sensor node platform through sensorfly. ACM Transactions on Sensor Networks (TOSN) 13, 4 (2017), 1–37.Google ScholarDigital Library
- Xinlei Chen, Aveek Purohit, Shijia Pan, Carlos Ruiz, Jun Han, Zheng Sun, Frank Mokaya, Patric Tague, and Pei Zhang. 2017. Design Experiences in Minimalistic Flying Sensor Node Platform through SensorFly. ACM Trans. Sen. Netw. 13, 4, Article 33 (nov 2017), 37 pages. https://doi.org/10.1145/3131779Google ScholarDigital Library
- Xinlei Chen, Carlos Ruiz, Sihan Zeng, Liyao Gao, Aveek Purohit, Stefano Carpin, and Pei Zhang. 2020. H-DrunkWalk: Collaborative and adaptive navigation for heterogeneous MAV swarm. ACM Transactions on Sensor Networks (TOSN) 16, 2 (2020), 1–27.Google ScholarDigital Library
- Xuecheng Chen, Haoyang Wang, Zuxin Li, Wenbo Ding, Fan Dang, Chengye Wu, and Xinlei Chen. 2023. DeliverSense: Efficient Delivery Drone Scheduling for Crowdsensing with Deep Reinforcement Learning. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers (Cambridge, United Kingdom) (UbiComp/ISWC ’22 Adjunct). Association for Computing Machinery, New York, NY, USA, 403–408. https://doi.org/10.1145/3544793.3560412Google ScholarDigital Library
- Xinlei Chen, Susu Xu, Haohao Fu, Carlee Joe-Wong, Lin Zhang, Hae Young Noh, and Pei Zhang. 2019. ASC: Actuation System for City-Wide Crowdsensing with Ride-Sharing Vehicular Platform. In Proceedings of the Fourth Workshop on International Science of Smart City Operations and Platforms Engineering (Montreal, Quebec, Canada) (SCOPE ’19). Association for Computing Machinery, New York, NY, USA, 19–24. https://doi.org/10.1145/3313237.3313299Google ScholarDigital Library
- Xinlei Chen, Susu Xu, Jun Han, Haohao Fu, Xidong Pi, Carlee Joe-Wong, Yong Li, Lin Zhang, Hae Young Noh, and Pei Zhang. 2020. PAS: Prediction-Based Actuation System for City-Scale Ridesharing Vehicular Mobile Crowdsensing. IEEE Internet of Things Journal 7, 5 (2020), 3719–3734. https://doi.org/10.1109/JIOT.2020.2968375Google ScholarCross Ref
- Xinlei Chen, Susu Xu, Xinyu Liu, Xiangxiang Xu, Hae Young Noh, Lin Zhang, and Pei Zhang. 2020. Adaptive hybrid model-enabled sensing system (HMSS) for mobile fine-grained air pollution estimation. IEEE Transactions on Mobile Computing 21, 6 (2020), 1927–1944.Google ScholarCross Ref
- Xinlei Chen, Xiangxiang Xu, Xinyu Liu, Hae Young Noh, Lin Zhang, and Pei Zhang. 2016. Hap: Fine-grained dynamic air pollution map reconstruction by hybrid adaptive particle filter. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM. 336–337.Google ScholarDigital Library
- Xinlei Chen, Yulei Zhao, Yong Li, Xu Chen, Ning Ge, and Sheng Chen. 2018. Social trust aided D2D communications: Performance bound and implementation mechanism. IEEE Journal on Selected Areas in Communications 36, 7 (2018), 1593–1608.Google ScholarDigital Library
- LightZero Contributors. 2023. LightZero: OpenDILab A lightweight and efficient toolkit designed for the MCTS, AlphaZero, and MuZero family of algorithms.https://github.com/opendilab/LightZero.Google Scholar
- Zipeng Dai, Chi Harold Liu, Yuxiao Ye, Rui Han, Ye Yuan, Guoren Wang, and Jian Tang. 2022. AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement Learning. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications. 1029–1038. https://doi.org/10.1109/INFOCOM48880.2022.9796732Google ScholarDigital Library
- C. DeepakG., Alexandros Ladas, Yusuf A. Sambo, Haris Bin Pervaiz, Christos Politis, and Muhammad Ali Imran. 2019. An Overview of Post-Disaster Emergency Communication Systems in the Future Networks. IEEE Wireless Communications 26 (2019), 132–139.Google ScholarDigital Library
- Zuxin Li, Fanhang Man, Xuecheng Chen, Baining Zhao, Chenye Wu, and Xinlei Chen. 2023. TRACT: Towards Large-Scale Crowdsensing With High-Efficiency Swarm Path Planning. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers (Cambridge, United Kingdom) (UbiComp/ISWC ’22 Adjunct). Association for Computing Machinery, New York, NY, USA, 409–414. https://doi.org/10.1145/3544793.3560401Google ScholarDigital Library
- Wei Liu, Yi Ding, Shuai Wang, Yu Yang, and Desheng Zhang. 2022. Para-Pred: Addressing Heterogeneity for City-Wide Indoor Status Estimation in On-Demand Delivery. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3407–3417.Google ScholarDigital Library
- Fangxu Lu, Zhichao Mi, Ning Zhao, Hai Wang, and Yulu Tian. 2022. 3D Deployment of Dynamic UAV Base Station based on Mobile Users. In 2021 International Conference on Advanced Computing and Endogenous Security. 1–5. https://doi.org/10.1109/IEEECONF52377.2022.10013331Google ScholarCross Ref
- Jianqiang Lu, Xiangpeng Xie, Xia Zhou, and Chengjie Bu. [n. d.]. Research on power-communication coordination recovery strategy based on grid dividing after extreme disasters. 675, 1 ([n. d.]), 012038. https://doi.org/10.1088/1755-1315/675/1/012038Google ScholarCross Ref
- Junhai Luo, Zhiyan Wang, Ming Xia, Linyong Wu, Yuxin Tian, and Yu Chen. [n. d.]. Path Planning for UAV Communication Networks: Related Technologies, Solutions, and Opportunities. ([n. d.]), 3560261. https://doi.org/10.1145/3560261Google ScholarDigital Library
- Maurilio Matracia, Nasir Saeed, Mustafa A. Kishk, and Mohamed-Slim Alouini. 2022. Post-Disaster Communications: Enabling Technologies, Architectures, and Open Challenges. IEEE Open Journal of the Communications Society 3 (2022), 1177–1205.Google ScholarCross Ref
- Luoyu Mei, Zhimeng Yin, Xiaolei Zhou, Shuai Wang, and Kai Sun. 2021. ECCR: Edge-Cloud Collaborative Recovery for Low-Power Wide-Area Networks Interference Mitigation. In Wireless Algorithms, Systems, and Applications: 16th International Conference, WASA 2021, Nanjing, China, June 25–27, 2021, Proceedings, Part I 16. Springer, 494–507.Google Scholar
- Mohammad Mozaffari, Walid Saad, Mehdi Bennis, and Merouane Debbah. 2015. Drone Small Cells in the Clouds: Design, Deployment and Performance Analysis. In 2015 IEEE Global Communications Conference (GLOBECOM). 1–6. https://doi.org/10.1109/GLOCOM.2015.7417609Google ScholarCross Ref
- Vidit Saxena, Joakim Jaldén, and Henrik Klessig. 2019. Optimal UAV base station trajectories using flow-level models for reinforcement learning. IEEE Transactions on Cognitive Communications and Networking 5, 4 (2019), 1101–1112.Google ScholarCross Ref
- Jurgen Scherer and Bernhard Rinner. 2017. Short and full horizon motion planning for persistent multi-UAV surveillance with energy and communication constraints. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017), 230–235.Google ScholarDigital Library
- Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, and David Silver. [n. d.]. Mastering Atari, Go, chess and shogi by planning with a learned model. 588, 7839 ([n. d.]), 604–609. https://doi.org/10.1038/s41586-020-03051-4Google ScholarCross Ref
- Angelo Trotta, Leonardo Montecchiari, Marco Di Felice, and Luciano Bononi. [n. d.]. A GPS-Free Flocking Model for Aerial Mesh Deployments in Disaster-Recovery Scenarios. 8 ([n. d.]), 91558–91573. https://doi.org/10.1109/ACCESS.2020.2994466Google ScholarCross Ref
- Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. arxiv:1710.10903 [stat.ML]Google Scholar
- Dong Wang and Yanping Yang. 2020. Joint Obstacle Avoidance and 3D Deployment for Securing UAV-Enabled Cellular Communications. IEEE Access 8 (2020), 67813–67821.Google ScholarCross Ref
- Shuai Wang, Shijie Hu, Baoshen Guo, and Guang Wang. 2023. Cross-Region Courier Displacement for On-Demand Delivery With Multi-Agent Reinforcement Learning. IEEE Transactions on Big Data (2023), 1–14. https://doi.org/10.1109/TBDATA.2023.3262408Google ScholarCross Ref
- Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heterogeneous Graph Attention Network. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19). Association for Computing Machinery, New York, NY, USA, 2022–2032. https://doi.org/10.1145/3308558.3313562Google ScholarDigital Library
- Yuhuai Wu, Elman Mansimov, Shun Liao, Roger B. Grosse, and Jimmy Ba. 2017. Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation. CoRR abs/1708.05144 (2017). arXiv:1708.05144Google Scholar
- Susu Xu, Xinlei Chen, Xidong Pi, Carlee Joe-Wong, Pei Zhang, and Hae Young Noh. 2020. iLOCuS: Incentivizing Vehicle Mobility to Optimize Sensing Distribution in Crowd Sensing. IEEE Transactions on Mobile Computing 19, 8 (2020), 1831–1847. https://doi.org/10.1109/TMC.2019.2915838Google ScholarDigital Library
- Yong Zeng and Rui Zhang. 2017. Energy-Efficient UAV Communication With Trajectory Optimization. IEEE Transactions on Wireless Communications 16, 6 (2017), 3747–3760. https://doi.org/10.1109/TWC.2017.2688328Google ScholarDigital Library
- Heng Zhang, Michael A Roth, Rajesh K. Panta, He Wang, and Saurabh Bagchi. 2020. CrowdBind: Fairness Enhanced Late Binding Task Scheduling in Mobile Crowdsensing. In Proceedings of the 2020 International Conference on Embedded Wireless Systems and Networks (Lyon, France) (EWSN ’20). Junction Publishing, USA, 61–72.Google ScholarDigital Library
- Zeyu Zhang, Weiwei Chen, Junwen Wang, Shuai Wang, and Tian He. 2022. CONST: Exploiting Spatial-Temporal Correlation for Multi-Gateway based Reliable LoRa Reception. In 2022 IEEE 30th International Conference on Network Protocols (ICNP). IEEE, 1–11.Google ScholarCross Ref
Index Terms
- Scheduling UAV Swarm with Attention-based Graph Reinforcement Learning for Ground-to-air Heterogeneous Data Communication
Recommendations
Real-time path planning of controllable UAV by subgoals using goal-conditioned reinforcement learning
AbstractThe conventional path planning problem for an unmanned aerial vehicle (UAV) typically involves a pre-defined environment and mission, with the objective of reaching a single target point. However, in order to perform different missions,...
Highlights- We propose a real-time autonomous path planning of UAVs in unknown environments.
UAV swarm path planning with reinforcement learning for field prospecting
AbstractThere has been steady growth in the adoption of Unmanned Aerial Vehicle (UAV) swarms by operators due to their time and cost benefits. However, this kind of system faces an important problem, which is the calculation of many optimal paths for each ...
A Brief Overview of Waveforms for UAV Air-to-Ground Communication Systems
ICVISP 2019: Proceedings of the 3rd International Conference on Vision, Image and Signal ProcessingUnmanned aerial vehicles (UAVs) have tremendous potential in both military and civilian applications. Due to the increasing demand for cost-efficient yet reliable small UAVs in a plethora of civilian applications, the technical advances in cooperative ...
Comments