Abstract
The multi agent path planning strategy for unmanned aerial vehicles (UAVs) might play a crucial role in seeking the most feasible path in 3D environment owning to power limitations and other environmental factors. UAVs path planning is a high precision task that is needed for wide range of commercial, military, and rescue operations. However, the path planning for multi-UAVs is a tedious task due to needing an optimal path between source and destination points. There are numerous algorithms available in literature for providing solutions to path planning problems; however, they do not provide an efficient solution, especially in the face of three-dimensional (3-D) aviation workspace. Hence, in this paper, novel reinforcement learning based enhanced variable weight grey wolf optimization algorithm named RLV-GWO is proposed to address this issue. In the proposed algorithm, two prominent variants of GWO named modified grey wolf optimizer (MGWO) and variable weight grey wolf optimizer (VM-GWO) are integrated to advance the performance of baseline model GWO through mitigating the convergence rate and escalating the algorithm speed by assigning variable weights to participating candidates during exploration and hunting process. Furthermore, reinforcement learning is introduced to control the switching operations of individual candidates adaptively based on their accumulated performance. The simulation results demonstrate that the proposed RLV-GWO algorithm can acquire feasible and effective path planning solution for UAVs in three dimensional environments.
Similar content being viewed by others
Data Availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Code Availability
The codes shall be available on reasonable request to the readers.
References
Qu, C., Gai, W., Zhang, J., & Zhong, M. (2020). A novel hybrid grey wolf optimizer algorithm for unmanned aerial vehicle (UAV) path planning. Knowledge-Based systems, 194, 105530. https://doi.org/10.1016/j.knosys.2020.105530
Ming, G., Hong, T., & Peng, Z. (2021). The research status of robot cluster path planning technology. Journal of National University of Defense technology, 43(01), 127–138.
Liu, J., Wei, X., & Huang, H. (2021). An improved grey wolf optimization algorithm and its application in path planning. IEEE Access, 9, 121944–121956. https://doi.org/10.1109/ACCESS.2021.3108973
Dewangan, R. K., & Saxena, P. (2022). Three dimensional route planning for multiple unmanned aerial vehicles using Salp Swarm Algorithm. Journal of Experimental and Theoretical Artificial Intelligence, 49, 1–20. https://doi.org/10.1007/s10489-018-1384-y
Singh, L., & Jaffrey, Z. A. (2018). Computerized detection of breast cancer in digital mammograms. International Journal of Computers and Applications, 40(2), 98–109. https://doi.org/10.1080/1206212X.2017.1395131
Singh, L., Alam, A., & Kumar, K. V. (2021). Design of thermal imaging-based health condition monitoring and early fault detection technique for porcelain insulators using machine learning. Environmental Technology & Innovation, 24, 1–14. https://doi.org/10.1016/j.eti.2021.102000
Kala, R., Shukla, A., & Tiwari, R. (2010). Fusion of probabilistic A* algorithm and fuzzy inference system for robotic path planning. Artificial Intelligence Review, 33, 307–327. https://doi.org/10.1007/s10462-010-9157-y
Valavanis, K., & Vachtsevanos, G. J. (2015). Handbook of unmanned aerial vehicles. Springer.
Radmanesh, M., Kumar, M., & Sarim, M. (2018). Grey wolf optimization-based sense and avoid algorithm in a Bayesian framework for multiple UAV path planning in an uncertain environment. Aerospace science & technology, 77, 168–179.
Orozco-Rosas, U., Montiel, O., & Sepulveda, R. (2019). Mobile robot path planning using membrane evolutionary artificial potential field. Applied soft computing, 77, 236–251.
Morales, N., Toledo, J., & Acosta, L. (2016). Path planning using a multiclass support vector machine. Applied Soft Computing, 43, 498–509. https://doi.org/10.1016/j.asoc.2016.02.037
Wang, W., Deng, H., & Wu, X. (2018). Path planning of loaded pin-jointed bar mechanisms using Rapidly-exploring random tree method. Computers & Structures, 209, 65–73.
Kobilarov, M. (2012). Cross-entropy randomized motion planning. The International Journal of Robotics Research, 31(7), 153–160.
Das, P. K., Behera, H. S., & Panigrahi, B. K. (2016). A hybridization of an improved particle swarm optimization and gravitational search algorithm for multi-robot path planning. Swarm Evolutionary Computation, 28, 14–28. https://doi.org/10.1016/j.swevo.2015.10.011
Bansal, J. C. (2019). Particle swarm optimization, evolutionary and swarm intelligence algorithms. Studies in Computational Intelligence, 8, 11–23. https://doi.org/10.1007/978-3-319-91341-4_2
Deng, W., Xu, J., & Zhao, H. (2019). An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access, 7, 20281–20292. https://doi.org/10.1109/ACCESS.2019.2897580
Yu, X. (2018). A self adaptive artificial bee colony algorithm based on global best for global optimization soft computing–A fusion of foundations. Methodologies and Applications, 22(9), 2935–2952. https://doi.org/10.1007/s00500-017-2547-1
Majumder, A., Majumder, A., & Bhaumik, R. (2021). Teaching–learning-based optimization algorithm for path planning and task allocation in multi-robot plant inspection system. Arabian Journal for Science and Engineering, 46, 8999–9021. https://doi.org/10.1007/s13369-021-05710-8
Heidari, A. A., Mirjalili, S., Faris, H., Aljarah, I., Mafarja, M., & Chen, H. (2019). Harris hawks optimization: Algorithm and applications. Future Generation computer systems, 97, 849–872. https://doi.org/10.1016/j.future.2019.02.028
Mafarja, M., & Mirjalili, S. (2017). Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing, 260, 302–312. https://doi.org/10.1016/j.neucom.2017.04.053
Dewangan, R. K., Shukla, A., & Godfrey, W. (2019). Three dimensional path planning using grey wolf optimizer for UAVs. Applied Intelligence, 49(7), 2201–2217. https://doi.org/10.1007/s10489-018-1384-y
Yan, Y., Liang, Y., Zhang, H., Zhang, W., Feng, H., Wang, B., & Liao, Q. (2019). A two stage optimization method for unmanned aerial vehicle inspection of an oil and gas pipeline network. Petroleum Science, 16, 458–468. https://doi.org/10.1007/s12182-019-0305-y
Qie, H., Shi, D., Shen, T., Xu, X., Li, Y., & Wang, L. (2019). Joint optimization of Multi-UAV Target Assignment and path planning based on Multi-Agent Reinforcement Learning. IEEE Access, 7, 146264–146272. https://doi.org/10.1109/ACCESS.2019.2943253
Sun, S., Song, B., Wang, P., Dong, H., & Chen, X. (2019). Real time mission motion planner for multi-UAVs Cooperative Work using tri-level programming. IEEE transactions on Intelligent Transportation systems, 20, 1303–1315.
Phung, M. D., Quach, C. H., Dinh, T. H., & Ha, Q. (2017). Enhanced discrete particle swarm optimization path planning for UAV vision-based surface inspection. Automation in Construction, 81, 25–33. https://doi.org/10.1016/j.autcon.2017.04.013
Shima, T., Rasmussen, S. J., Sparks, A. G., & Passino, K. M. (2006). Multiple task assignments for cooperating uninhabited aerial vehicles using genetic algorithms. Computers & Operations Research, 33(11), 3252–3269. https://doi.org/10.1016/j.cor.2005.02.039
Shorakaei, H., Vahdani, M., Imani, B., & Gholami, A. (2016). Optimal cooperative path planning of unmanned aerial vehicles by a parallel genetic algorithm. Robotica, 34(4), 823–836.
Ge, F., Li, K., Han, Y., Xu, W., & Wang, Y. (2020). Path planning of UAV for oilfield inspections in a three-dimensional dynamic environment with moving obstacles based on an improved pigeon-inspired optimization algorithm. Applied Intelligence, 50, 2800–2817. https://doi.org/10.1007/s10489-020-01650-2
Khandelwal, A., Bhargava, A., Sharma, A., & Sharma, H. (2018). Modified Grey wolf optimization algorithm for transmission network expansion planning problem. Arabian Journal for Science and Engineering, 43, 2899–2908. https://doi.org/10.1007/s13369-017-2967-3
Kumar, R., Singh, L., & Tiwari, R. (2021). Path planning for the autonomous robots using modified grey wolf optimization approach. Journal of Int. & Fuzzy systems, 40, 9453–9470. https://doi.org/10.3233/JIFS-201926
Gao, Z. M., & Zhao, J. (2019). An improved grey wolf optimization algorithm with variable weights. Computational Intelligence and Neuroscience, 298, 1282. https://doi.org/10.1155/2019/2981282
Gupta, S., & Deep, K. (2019). An efficient grey wolf optimizer with opposition based learning and chaotic local search for integer and mixed integer optimization problems. Arabian Journal for Science and Engineering, 44, 7277–7296. https://doi.org/10.1007/s13369-019-03806-w
Kumar, R., Singh, L., & Tiwari, R. (2020). Comparison of two metaheuristic algorithms for Path Planning in Robotics. IEEE conference. https://doi.org/10.1109/IC3A48958.2020.233289
Pradhan, M., Roy, P. K., & Pal, T. (2016). Grey wolf optimization applied to economic load dispatch problems. International Journal Electron Power Energy System., 83, 325–334. https://doi.org/10.1016/j.ijepes.2016.04.034
Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69(3), 46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
Zhang, X., & Duan, H. (2015). An improved constrained differential evolution algorithm for unmanned aerial vehicle Global route planning. Application of soft computing, 26, 270–284.
Zhang, D. F., & Duan, H. B. (2018). Social class pigeon-inspired optimization and timestamp segmentation for multi-UAV operative path planning. Neurocomputing, 313, 2029–2046. https://doi.org/10.1016/j.neucom.2018.06.032
Wang, Y. B., Bai, P., & Liang, H. L. (2019). Reconnaissance mission conducted by UAV swarm based on distributed PSO path planning algorithms. IEEE Access, 7, 105086–105099. https://doi.org/10.1109/ACCESS.2019.2932008
Wolpert, D., & Macready, W. (1997). No free lunch theorems for optimization. IEEE Transactions Evolutionary Computing, 1(1), 67–82. https://doi.org/10.1109/4235.585893
Qu, C., Gai, W., Zhong, M., & Zhang, J. (2020). A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning. Applied soft computing, 89, 10699. https://doi.org/10.1016/j.asoc.2020.106099
Loquercio, A., Kaufmann, E., Ranftl, R., Dosovitskiy, A., Koltun, V., & Scaramuzza, D. (2019). Deep drone racing: From simulation to reality with domain randomization. IEEE Transactions on Robotics, 36, 1552–3098. https://doi.org/10.1109/TRO.2019.2942989
Funding
This research was not supported by any funding agency.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
Authors have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kumar, R., Singh, L. & Tiwari, R. Novel Reinforcement Learning Guided Enhanced Variable Weight Grey Wolf Optimization (RLV-GWO) Algorithm for Multi-UAV Path Planning. Wireless Pers Commun 131, 2093–2123 (2023). https://doi.org/10.1007/s11277-023-10534-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-023-10534-w