计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (8): 233-242.doi: 10.11896/jsjkx.220900181

所属专题： 智能化边缘计算

• 计算机网络 • 上一篇    下一篇

基于深度强化学习和无线充电技术的D2D-MEC网络边缘卸载框架

张乃心1, 陈霄睿1, 李安1, 杨乐瑶1, 吴华明2   

  1. 1 天津大学数学学院 天津 300192
    2 天津大学应用数学中心 天津 300072
  • 收稿日期:2022-09-19 修回日期:2023-02-06 出版日期:2023-08-15 发布日期:2023-08-02
  • 通讯作者: 吴华明(whming@tju.edu.cn)
  • 作者简介:(15598555618@163.com)
  • 基金资助:
    国家自然科学基金(62071327)

Edge Offloading Framework for D2D-MEC Networks Based on Deep Reinforcement Learningand Wireless Charging Technology

ZHANG Naixin1, CHEN Xiaorui1, LI An1, YANG Leyao1, WU Huaming2   

  1. 1 School of Mathematics,Tianjin University,Tianjin 300192,China
    2 Center for Applied Mathematics,Tianjin University,Tianjin 300072,China
  • Received:2022-09-19 Revised:2023-02-06 Online:2023-08-15 Published:2023-08-02
  • About author:ZHANG Naixin,born in 2000,master candidate.Her main research interests include mobile edge computing and machine learning.
    WU Huaming,born in 1986,Ph.D,associate professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include Internet of things(IoT),mobile edge computing and cloud computing.
  • Supported by:
    National Natural Science Foundation of China(62071327).

PDF (PC)

440

摘要： 物联网设备中大量未被充分利用的计算资源,正是移动边缘计算所需要的。一种基于设备对设备通信技术和无线充电技术的边缘卸载框架,可以最大化利用闲置物联网设备的计算资源,提升用户体验。在此基础上,可以建立物联网设备的D2D-MEC网络模型。在该模型中,主设备根据当前环境信息和估计的设备状态信息,选择向多个边缘设备卸载不同数量的任务,并应用无线充电技术提升传输的成功率和计算的稳定性。运用强化学习方法解决任务分配和资源分配的联合优化问题,也就是最小化计算延迟、能量消耗和任务丢弃损失,最大化边缘设备利用率和任务卸载比例的优化问题。除此之外,为了适应状态空间更大的情况,提高学习速度,提出了一种基于深度强化学习的卸载方案。基于以上理论和模型,使用数学推导计算出了D2D-MEC系统的最优解及性能上限。仿真实验证明了D2D-MEC卸载模型及其卸载策略的综合性能更好,更能充分利用物联网设备的计算资源。

关键词: 移动边缘计算, D2D, 强化学习, 物联网, 计算卸载, 无线能量传输

Abstract: A large amount of underutilized computing resources in IoT devices is what mobile edge computing requires.An edge offloading framework based on device-to-device communication technology and wireless charging technology can maximize the utilization of computing resources of idle IoT devices and improve user experience.The D2D-MEC network model of IoT devices can be established on this basis.In this model,the device chooses to offload multiple tasks to multiple edge devices according to the current environment information and the estimated device state.It applies wireless charging technology to increase the success rate of transmission and computation stability.The reinforcement learning method is used to solve the joint optimization allocation problem,which aims to minimize the computation delay,energy consumption,and task dropping loss as well as maximize the utilization of edge devices and the proportion of task offloading.In addition,to adapt to larger state space and improve learning speed,an offloading scheme based on deep reinforcement learning is proposed.Based on the above theory and model,the optimal solution and upper limit of performance of the D2D-MEC system are calculated by mathematical derivation.Simulation results show that the D2D-MEC offloading model and its offloading strategy have better all-around performance and can make full use of the computing resources of IoT devices.

Key words: Mobile edge computing, Device-to-device(D2D), Reinforcement learning, Internet of things(IoT), Computation offloading, Wireless energy transmission

中图分类号: 

  • TP181
[1]BOYD B,SHILPA T,REZA A,et al.Networks and devices for the 5G era[J].IEEE Communications Magazine,2014,52(2):90-96.
[2]KO H,PACK S.Distributed Device-to-Device Offloading Sys-tem:Design and Performance Optimization[J].IEEE Transactions on Mobile Computing,2021,20(10):2949-2960.
[3]MOLIN L,CHEN T,ZENG J,et al.D2D-Assisted Computation Offloading for Mobile Edge Computing Systems with Energy Harvesting[C]//2019 20th International Conference on Parallel and Distributed Computing,Applications and Technologies (PDCAT).Gold Coast,QLD,Australia:2019:90-95.
[4]MAO Y,ZHANG J,LETAIEF K B.Dynamic computation offloading for mobile-edge computing with energy harvesting devices[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3590-3605.
[5]PU L,CHEN X,XU J,et al.D2D Fogging:An Energy-Efficient and Incentive-Aware Task Offloading Framework via Network-assisted D2D Collaboration[J].IEEE Journal on Selected Areas in Communications,2016,34(12):3887-3901.
[6]FANG T,YANG Y,CHEN J X.Optimization of offloadingstrategy under assisted mobile edge computing[J].Computer Science,2022,49(S1):601-605.
[7]QIAO G H,LENG S P,ZHANG Y.Online Learning and Optimization for Computation Offloading in D2D Edge Computing and Networks[J].Mobile Networks & Applications,2019,27:1111-1122.
[8]GAIBIN L,CHEN M,WEI X,et al.Computation OffloadingWith Reinforcement Learning in D2D-MEC Network [C]//2020 International Wireless Communications and Mobile Computing (IWCMC).Limassol,Cyprus,2020:69-74.
[9]HUANG L,FENG X,QIAN L,et al.Deep ReinforcementLearning-Based Task Offloading and Resource Allocation for Mobile Edge Computing[C]//Machine Learning and Intelligent Communications(MLICOM).Cham:Springer,2018:33-42.
[10]LIANG W,WANG K,PAN C,et al.Deep Reinforcement Lear-ning Based Dynamic Trajectory Control for UAV-assisted Mobile Edge Computing[J].arXiv:1911.03887v2,2019.
[11]MINGHUI M,XIAO L,CHEN Y,et al.Learning-Based Computation Offloading for IoT Devices With Energy Harvesting[J].IEEE Transactions on Vehicular Technology,2019,68(2):1930-1941.
[12]HUANG L,BI S,ZHANG Y J A.Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks[J].IEEE Transactions on Mobile Computing,2020,19(11):2581-2593.
[13]KAN L P G,HAUSWALD J,GAO C,et al.Collaborative Intelligence Between the Cloud and Mobile Edge[J].Acm Sigplan Notices A Monthly Publication of the Special Interest Group on Programming Languages,2017,45(1):615-629.
[14]WANG Z C,HUA J W,ZHU J Q,et al.Vehicle task offloading based on deep reinforcement learning for parking cooperation[J].Small Microcomputer System,2023,44(3):658-664.
[15]LIANG H,ZHANG L,YANG S,et al.Meta-Learning BasedDynamic Computation Task Offloading for Mobile Edge Computing Networks[J].IEEE Communications Letters,2021,25(5):1568-1572.
[16]DENG S Q,YE X G.Multi-objective task offloading algorithm based on deep Q network[J].Computer Application,2022,42(6):1668-1674.
[17]LIU P F,MAO Y C,WANG B L.Task allocation method based on cloud-fog collaboration model[J].Computer Application,2019,39(1):8-14.
[18]CHENG P,ZHANG W Z,XIE S H,et al.Research on multi-objective balanced task offloading method for edge computing of Internet of Vehicles[J].Small Microcomputer System,2022,43(9):1992-1997.
[19]DINH Q,TANG J,LA Q,et al.Offloading in Mobile Edge Computing:Task Allocation and Computational Frequency Scaling[J].IEEE Transactions on Communications,2017,65(8):3571-3584.
[20]ZHAO S,LI Y,WU D,et al.Current-Decomposition-Based Di-gital Phase Synchronization Method for BWPT System[J].IEEE Transactions on Power Electronics,2021,36(11):12183-12188.
[21]WATKINS C.Learning from delayed rewards [D].Cambridge:King's College of University of Cambridge,1989.
[22]CHRISTOPHER J,PETER D.Q-learning[J].Machine Lear-ning,1992,8(3/4):279-292.
[23]XIAO L,LI Y,HUANG X,et al.Cloud-based malware detection game for mobile devices with offloading[J].IEEE Trans Mobile Computing,2017,16(10):2742-2750.
[24]LI J,GAO H,LV T.Deep reinforcement learning based computation offloading and resource allocation for MEC[C]//IEEE Wireless Communication and Networking Conf(WCNC).2018:1-6.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发