SAC-PER: A Navigation Method Based on Deep Reinforcement Learning Under Uncertain Environments

Abstract

In real scenarios, robots usually face dynamically changing environments, and traditional navigation methods require a predefined high-precision map, which limits the achievability of navigation in dynamic and uncertain environments. To solve this problem, this paper uses a Partially Observable Markov Decision Process (POMDP) to model the uncertain navigation planning problem and proposes a soft actor-critic with prioritized experience replay (SAC-PER) method based on multi-sensor perception to achieve efficient navigation. The method uses multi-source information fusion for environment perception and Deep Reinforcement Learning (DRL) for continuous control of navigation. The multi-source SAC-PER method can effectively avoid obstacles and enable robots to perform navigation tasks autonomously in uncertain environments without building high-precision maps. We evaluate the proposed method using Robot Operating System (ROS) and Gazebo simulator. The results demonstrate that the SAC-PER method has high efficiency and robustness in different environments, and shows good generalization ability.