A Deep Reinforcement Learning-Based Content Updating Algorithm for High Definition Map Edge Caching

Abstract

Edge caching is a promising technique to alleviate the communication cost during the content update and retrieving. Particularly, it is suitable for the High Definition Map (HDM) caching which needs frequent updates to avoid its contents becoming staleness. In this paper, we aim at minimizing the response latency while satisfying the content freshness of the vehicle’s HDM request under the edge caching scenario. We first depict the change of the content freshness difference, in term of the Age of Information (AoI) difference value, of each request, which are determined by both the vehicular requirements and the content update decision of the Road Aide Unit (RSU). Then, we formulate the HDM content update optimization problem, which jointly considering the AoI difference and the extra responding latency of each request. On this basis, we transform the problem into a Markov Decision Process (MDP), and propose an optimization algorithm based on the deep reinforcement learning-based theory to obtain the optimal update decision by maximizing the long-term discounted reward. Finally, extensive simulations are presented to verify the effectiveness of the proposed algorithm by comparing it with various baseline policies.