Yongjie Yang
ABSTRACT
With the rapid development of IoT and fog computing, mobile devices can offload computing tasks into the fog layer to reduce energy consumption and transmission delay. However, the communication between fog nodes and end users is vulnerable to malicious jamming attacks. In this paper, we propose a reinforcement learning-based computation offloading strategy to choose fog nodes, offloading rate, and transmit power to address jamming attacks and interference problems. We use dueling deep Q-network to accelerate the learning speed in the system, which can effectively determine the offloading strategy. The simulation results show that this scheme based on DDQN is better than DQN and Q-learning in terms of computational latency and energy consumption.
- Hoa Tran-Dang, Sanjay Bhardwaj, Tariq Rahim, Arslan Musaddiq and Dong-Seong Kim. 2022. Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues. Journal of Communications and Networks. 24, 1 (February 2022), 83-98. https://doi.org/10.23919/JCN.2021.000041Google Scholar
Cross Ref
- Shanshan Tu, Muhammad Waqas, Sadaqat U. Rehman, Talha Mir, Zahid Halim and Iftekhar Ahmad. 2022. Social Phenomena and Fog Computing Networks: A Novel Perspective for Future Networks. IEEE Transactions on Computational Social Systems. 9, 1 (February 2022), 32-44. https://doi.org/10.1109/TCSS.2021.3082022Google ScholarCross Ref
- Wenle Bai and Cheng Qian. 2021. Deep Reinforcement Learning for Joint Offloading and Resource Allocation in Fog Computing. 2021 IEEE 12th International Conference on Software Engineering and Service Science (ICSESS). (August 2021), 131-134. https://doi.org/10.1109/ICSESS52187.2021.9522334Google ScholarCross Ref
- Mohammad Aazam, Sherali Zeadally, Khaled A. Harras. 2018. Offloading in fog computing for IoT: Review, enabling technologies, and research opportunities. Future Generation Computer Systems. 87, (October 2018), 278-289. https://doi.org/10.1016/j.future.2018.04.057Google ScholarDigital Library
- Congfeng Jiang, Xiaolan Cheng, Honghao Gao, Xin Zhou and Jian Wan. 2019. Toward Computation Offloading in Edge Computing: A Survey. IEEE Access. 7, (August 2019), 131543-131558. https://doi.org/10.1109/ACCESS.2019.2938660Google ScholarCross Ref
- Jialin Wan, Muhammad Waqas, Shanshan Tu, Syed M. Hussain, Ahsan Shah, Sadaqat U. Rehman and Muhammad Hanif. 2021. An Efficient Impersonation Attack Detection Method in Fog Computing. Computers, Materials & Continua. 68, 1 (March 2021), 267–281. https://doi.org/10.32604/cmc.2021.016260Google ScholarCross Ref
- Shanshan Tu, Muhammad Waqas, Sadaqat Ur Rehman, Talha Mir, Ghulam Abbas, Ziaul H. Abbas, Zahid Halim and Iftekhar Ahmad. 2021. Reinforcement Learning Assisted Impersonation Attack Detection in Device-to-Device Communications. IEEE Transactions on Vehicular Technology. 70, 2 (February 2021), 1474-1479. https://doi.org/10.1109/TVT.2021.3053015Google ScholarCross Ref
- Helin Yang, Zehui Xiong, Jun Zhao, Dusit Niyato, Qingqing Wu, H. Vincent Poor and Massimo Tornatore. 2021. Intelligent Reflecting Surface Assisted Anti-Jamming Communications: A Fast Reinforcement Learning Approach. IEEE Transactions on Wireless Communications. 20, 3 (March 2021), 1963-1974. https://doi.org/10.1109/TWC.2020.3037767Google ScholarDigital Library
- Liang Xiao, Xiaozhen Lu, Tangwei Xu, Xiaoyue Wan, Wen Ji and Yanyong Zhang. 2020. Reinforcement Learning-Based Mobile Offloading for Edge Computing Against Jamming and Interference. IEEE Transactions on Communications. 68, 10 (October 2020), 6114-6126. https://doi.org/10.1109/TCOMM.2020.3007742Google ScholarCross Ref
- Liping Liang, Wenchi Cheng, Wei Zhang, Hailin Zhang. 2018. Mode Hopping for Anti-Jamming in Radio Vortex Wireless Communications. IEEE Transactions on Vehicular Technology. 67, 8 (August 2018), 7018-7032. https://doi.org/10.1109/TVT.2018.2825539Google ScholarCross Ref
- Yulan Gao, Yue Xiao, Mingming Wu, Ming Xiao and Jinliang Shao. 2018. Game Theory-Based Anti-Jamming Strategies for Frequency Hopping Wireless Communications. IEEE Transactions on Wireless Communications. 17, 8 (August 2018), 5314-5326. https://doi.org/10.1109/TWC.2018.2841921Google ScholarDigital Library
- Haojin Zhu, Chenliaohui Fang, Yao Liu, Cailian Chen, Mengyuan Li and Xuemin S. Shen. 2016. You Can Jam But You Cannot Hide: Defending Against Jamming Attacks for Geo-Location Database Driven Spectrum Sharing. IEEE Journal on Selected Areas in Communications. 34, 10 (October 2016), 2723-2737. https://doi.org/10.1109/JSAC.2016.2605799Google ScholarDigital Library
- Andrey Garnaev, Athina Petropulu, Wade Trappe and H. Vincent Poor. 2019. A Power Control Game with Uncertainty On the Type of the Jammer. 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP). (November 2019), 1-5. https://doi.org/10.1109/GlobalSIP45357.2019.8969379Google ScholarCross Ref
- Liang Xiao, Yan Li, Jinliang Liu and Yifeng Zhao. 2015. Power control with reinforcement learning in cooperative cognitive radio networks against jamming. The Journal of Supercomputing. 71, 9 (April 2015), 3237-3257. https://doi.org/10.1007/s11227-015-1420-1Google ScholarDigital Library
- Pengwenlong Gu, Cunqing Hua, Rida Khatoun, Yue Wu and Ahmed Serhrouchni. 2017. Cooperative Anti-Jamming Relaying for Control Channel Jamming in Vehicular Networks. GLOBECOM 2017 - 2017 IEEE Global Communications Conference. (December 2017), 1-6. https://doi.org/10.1109/GLOCOM.2017.8253978Google ScholarDigital Library
- Luying Huang, Tao Xu, Xueqiang Chen, Yitao Xu, Xiao Zhang, and Gui Fang. 2021. Joint relay and channel selection in relay‐aided anti‐jamming system: A reinforcement learning approach. Trans. Emerg. Telecommun. Technol. 32, 9 (September 2021). https://doi.org/10.1002/ett.4243Google ScholarDigital Library
- Junhong Nie and S. Haykin. 1999. A Q-learning-based dynamic channel assignment technique for mobile communication systems. IEEE Transactions on Vehicular Technology. 48, 5 (September 1999), 1676-1687. https://doi.org/10.1109/25.790549Google ScholarCross Ref
- Liang Xiao, Yanda Li, Canhuang Dai, Huaiyu Dai and H. Vincent Poor. 2018. Reinforcement Learning-Based NOMA Power Allocation in the Presence of Smart Jamming. IEEE Transactions on Vehicular Technology. 67, 4 (April 2018), 3377-3389. https://doi.org/10.1109/TVT.2017.2782726Google ScholarCross Ref
- Slimeni Feten, Chtourou Zied, Scheers Bart, L. N. Vincent and Attia Rabah. 2019. Cooperative Q-learning based channel selection for cognitive radio networks. Wireless Networks. 25, 7 (October 2019), 4161–4171. https://doi.org/10.1007/s11276-018-1737-9Google ScholarDigital Library
- Fuqiang Yao, and Luliang Jia. 2019. A Collaborative Multi-Agent Reinforcement Learning Anti-Jamming Algorithm in Wireless Networks. IEEE Wireless Communications Letters. 8, 4 (August 2019), 1024-1027. https://doi.org/10.1109/LWC.2019.2904486Google ScholarCross Ref
- Ibrahim Elleuch, Ali Pourranjbar and Georges Kaddoum. 2021. A Novel Distributed Multi-Agent Reinforcement Learning Algorithm Against Jamming Attacks. IEEE Communications Letters. 25, 10 (October 2021), 3204-3208. https://doi.org/10.1109/LCOMM.2021.3097290Google ScholarCross Ref
- G. M. Shafiqur Rahman, Tian Dang and Manzoor Ahmed. 2020. Deep reinforcement learning based computation offloading and resource allocation for low-latency fog radio access networks. Intelligent and Converged Networks. 1, 3 (December 2020), 243-257. https://doi.org/10.23919/ICN.2020.0020Google ScholarCross Ref
- Minghui Min, Liang Xiao, Ye Chen, Peng Cheng, Di Wu and Weihua Zhuang. 2019. Learning-Based Computation Offloading for IoT Devices With Energy Harvesting. IEEE Transactions on Vehicular Technology. 68, 2 (February 2019), 1930-1941. https://doi.org/10.1109/TVT.2018.2890685Google ScholarCross Ref
- Yifei Wei, F. Richard Yu, Mei Song and Zhu Han. 2019. Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning. IEEE Internet of Things Journal. 6, 2 (April 2019), 2061-2073. https://doi.org/10.1109/JIOT.2018.2878435Google ScholarCross Ref
- Yi Liu, Huimin Yu, Shengli Xie and Yan Zhang. 2019. Deep Reinforcement Learning for Offloading and Resource Allocation in Vehicle Edge Computing and Networks. IEEE Transactions on Vehicular Technology. 68, 11 (November 2019), 11158-11168. https://doi.org/10.1109/TVT.2019.2935450Google ScholarCross Ref
- Liang Huang, Suzhi Bi and Ying-Jun A. Zhang. 2020. Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks. IEEE Transactions on Mobile Computing. 19, 11 (November 2020), 2581-2593. https://doi.org/10.1109/TMC.2019.2928811Google ScholarCross Ref
- Pei-Gen Ye, Yuan-Gen Wang, Jin Li, Liang Xiao and Guopu Zhu. 2020. (T, ∊)-Greedy Reinforcement Learning for Anti-Jamming Wireless Communications. GLOBECOM 2020 - 2020 IEEE Global Communications Conference. (December 2020), 1-6. https://doi.org/10.1109/GLOBECOM42002.2020.9322486Google ScholarDigital Library
- Weihang Wang, Zefang Lv, Xiaozhen Lu, Yi Zhang and Liang Xiao. 2021. Distributed reinforcement learning based framework for energy-efficient UAV relay against jamming. Intelligent and Converged Networks. 2, 2 (June 2021), 150-162. https://doi.org/10.23919/ICN.2021.0010Google ScholarCross Ref
- Weiqing Liu, Jiannong Cao, Lei Yang, Lin Xu, Xuanjia Qiu and Jing Li. 2017. AppBooster: Boosting the Performance of Interactive Mobile Applications with Computation Offloading and Parameter Tuning. IEEE Transactions on Parallel and Distributed Systems. 28, 6 (June 2017), 1593-1606. https://doi.org/10.1109/TPDS.2016.2624733Google ScholarDigital Library
- Angela S. Cacciapuoti, Marcello Caleffi, Mingyue Ji, Jaime Llorca and Antonia M. Tulino. 2016. Speeding Up Future Video Distribution via Channel-Aware Caching-Aided Coded Multicast. IEEE Journal on Selected Areas in Communications. 34, 8 (August 2016), 2207-2218. https://doi.org/10.1109/JSAC.2016.2577198Google ScholarDigital Library
- Negin Golrezaei, Andreas F. Molisch, Alexandros G. Dimakis and Giuseppe Caire. 2013. Femtocaching and device-to-device collaboration: A new architecture for wireless video distribution. IEEE Communications Magazine. 51, 4 (April 2013), 142-149. https://doi.org/10.1109/MCOM.2013.6495773Google ScholarCross Ref
- Dajun Zhang, F. Richard Yu and Ruizhe Yang. 2019. Blockchain-Based Distributed Software-Defined Vehicular Networks: A Dueling Deep Q-Learning Approach. 5, 4 (December 2019), 1086-1100. https://doi.org/10.1109/TCCN.2019.2944399Google ScholarCross Ref
- Boren Guo, Xin Zhang, Qiwei Sheng and Hongwen Yang. 2020. Dueling Deep-Q-Network Based Delay-Aware Cache Update Policy for Mobile Users in Fog Radio Access Networks. IEEE Access. 8, (January 2020), 7131-7141. https://doi.org/10.1109/ACCESS.2020.2964258Google ScholarCross Ref
- Ning Zhao, Hao Wu, F. Richard Yu, Lifu Wang, Weiting Zhang and Victor C. M. Leung. 2022. Deep-Reinforcement-Learning-Based Latency Minimization in Edge Intelligence Over Vehicular Networks. IEEE Internet of Things Journal. 9, 2 (January 2022), 1300-1312. https://doi.org/10.1109/JIOT.2021.3078480Google ScholarCross Ref
- Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas. 2016. Dueling network architectures for deep reinforcement learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (ICML'16). JMLR.org, 1995–2003. https://doi.org/10.48550/arXiv.1511.06581Google ScholarCross Ref
Index Terms
- Deep Reinforcement Learning-Based Computation Offloading for Anti-jamming in Fog Computing Networks
Recommendations
Reinforcement Learning Methods for Computation Offloading: A Systematic Review
Today, cloud computation offloading may not be an appropriate solution for delay-sensitive applications due to the long distance between end-devices and remote datacenters. In addition, offloading to a remote cloud can consume bandwidth and dramatically ...
A survey on computation offloading and service placement in fog computing-based IoT
AbstractIn recent years, fog computing has emerged as a computing paradigm to support the computationally intensive and latency-critical applications for resource limited Internet of Things (IoT) devices. The main feature of fog computing is to push ...
A Deep Reinforcement Learning Approach Towards Computation Offloading for Mobile Edge Computing
Human Centered ComputingAbstractIn order to improve the quality of service for users and reduce the energy consumption of the cloud computing environment, Mobile Edge Computing (MEC) is a promising paradigm by providing computing resources which is close to the end device in ...
Comments