Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach
Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach |
||
 |
 |
|
| © 2023 by IJETT Journal | ||
| Volume-71 Issue-2 |
||
| Year of Publication : 2023 | ||
| Author : Tlotlollo S Hlalele, Yanxia Sun, Zenghui Wang |
||
| DOI : 10.14445/22315381/IJETT-V71I2P222 | ||
How to Cite?
Tlotlollo S Hlalele, Yanxia Sun, Zenghui Wang, "Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach," International Journal of Engineering Trends and Technology, vol. 71, no. 2, pp. 186-196, 2023. Crossref, https://doi.org/10.14445/22315381/IJETT-V71I2P222
Abstract
The voltage control problem due to bidirectional power flows is more apparent when heterogeneous distributed
generation systems (DGS) are integrated into the grid. In this paper, a novel method of voltage control in distributed generation
systems based on a reinforcement learning technique is proposed. DGS incorporating renewable energy resources are highly
complicated nonlinear dynamic systems. There are several challenges in employing the existing control methods. The novel
method presented in this paper entrenches the Q learning algorithm into the voltage control problem of DGS. The Q-learning
algorithm teaches agents responsible for decision taking in controlling the voltage and award the reward if the aim is achieved.
The IEEE 9 bus test system with DG’s integrated is used with various controlling agents connected. The results show significant
improvement in the reliability of agent communication and the efficiency of the proposed method.
Keywords
Distributed generation, Reinforcement learning, Voltage control.
References
[1] Chongjie Zhang, and Victor Lesser, “Coordinating Multi-Agent Reinforcement Learning with Limited Communication,” Proceedings of
the 12th International Conference on Autonomous Agents and Multiagent Systems, pp. 1101–1108, 2013.
[2] Hasan Can Kiliçkiran, Bedri Kekezoglu, and Nikolaos G. Paterakis, “Reinforcement Learning for Optimal Protection Coordination,” 2018
International Conference on Smart Energy Systems and Technologies, pp. 1-6, 2018. Crossref,
http://doi.org/10.1109/SEST.2018.8495830
[3] Praveen Prakash Singh, and Ivo Palu, “State Coordinated Voltage Control in an Active Distribution Network with On-Load Tap Changers
and Photovoltaic Systems,” Global Energy Interconnection, vol. 4, no. 2, pp. 117–125, 2021. Crossref,
http://doi.org/10.1016/j.gloei.2021.05.005
[4] Carlos Guestrin, Michail Lagoudakis, and Ronald Parr, “Coordinated Reinforcement Learning”.
[5] Patrick T. Manditereza, and Ramesh C. Bansal, “Multi-Agent Based Distributed Voltage Control Algorithm for Smart Grid Applications,”
Electric Power Components and Systems, vol. 44, no. 20, pp. 2352–2363, 2016. Crossref, http://doi.org/10.1080/15325008.2016.1219889
[6] Won Nam Koong et al., “Voltage Control of Distribution Networks to Increase their Hosting Capacity in South Korea,” Journal of
Electrical Engineering & Technology, vol. 16, no. 3, pp. 1305–1312, 2021. Crossref, http://doi.org/10.1007/s42835-021-00682-z.
[7] Nicholas S. Coleman, Christian Schegan, and Karen N. Miu, “A Study of Power Distribution System Fault Classification with Machine
Learning Techniques,” 2015 North American Power Symposium, pp. 1–6, 2015. Crossref, http://doi.org/10.1109/NAPS.2015.7335264
[8] M.Reza Tousi et al., “A Multi-Agent-Based Voltage Control in Power Systems Using Distributed Reinforcement Learning,”
SIMULATION, vol. 87, no. 7, pp. 581–599, 2011. Crossref, http://doi.org/10.1177/0037549710367904
[9] S. Anudevi, and Vinayak N. Shet, "Distribution Network with Optimal DG Placement and Protection Impacts: Review Analysis," SSRG
International Journal of Electrical and Electronics Engineering, vol. 4, no. 2, pp. 12-16, 2017. Crossref,
https://doi.org/10.14445/23488379/IJEEE-V4I2P103
[10] Zhen Ni, and Shuva Paul, “A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution,” IEEE Transactions on
Neural Networks and Learning Systems, vol. 30, no. 9, pp. 2684–2695, 2019. Crossref, https://doi.org/10.1109/TNNLS.2018.2885530
[11] Ali Moradi Amani et al., “Voltage Control in Distributed Generation Systems Based on Complex Network Approach,” Energy Procedia,
vol. 110, pp. 334–339, 2017. Crossref, https://doi.org/10.1016/j.egypro.2017.03.149
[12] Kyriaki E. Antoniadou-Plytaria et al., “Distributed and Decentralized Voltage Control of Smart Distribution Networks : Models, Methods,
and Future Research,” IEEE Transactions on Smart Grid, vol. 8, no. 6, pp. 2999–3008, 2017. Crossref,
https://doi.org/10.1109/TSG.2017.2679238
[13] Mulumudi Rajesh, and A. Lakshmi Devi, "Wind, PV Solar, Hydro and Hybrid Energy Storage System-Based Intelligent Adaptive Control
for Standalone Distributed Generation System," SSRG International Journal of Electrical and Electronics Engineering, vol. 9, no. 11, pp.
67-94, 2022. Crossref, https://doi.org/10.14445/23488379/IJEEE-V9I11P108
[14] Ali Bidram et al., “Distributed Cooperative Secondary Control of Microgrids Using Feedback Linearization,” IEEE Transactions on
Power Systems, vol. 28, no. 3, pp. 3462–3470, 2013. Crossref, https://doi.org/10.1109/TPWRS.2013.2247071
[15] Zhibin Liu et al., “Simulation Analysis of Distributed Micro-Energy Clusters Participating in Voltage Regulation and Peak Regulation,”
Energy Reports, vol. 7, pp. 1529–1543, 2021. Crossref, https://doi.org/10.1016/j.egyr.2021.09.089
[16] Tine L. Vandoorn et al., “Voltage Control in Islanded Microgrids by means of a Linear-Quadratic Regulator,” IEEE Benelux Young
Researchers Symposium, 2010.
[17] John W. Simpson-Porco et al., “Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed Averaging,” IEEE
Transactions on Industrial Electronics, vol. 62, no. 11, pp. 7025–7038, 2015. Crossref, https://doi.org/10.1109/TIE.2015.2436879
[18] Xiangyu Wu, Chen Shen, and Reza Iravani, “A Distributed, Cooperative Frequency and Voltage Control for Microgrids,” IEEE
Transactions on Smart Grid, vol. 9, no. 4, pp. 2764–2776, 2018. Crossref, https://doi.org/10.1109/TSG.2016.2619486
[19] Juan S. Gómez et al., “Distributed Predictive Control for Frequency and Voltage Regulation in Microgrids,” IEEE Transactions on Smart
Grid, vol. 11, no. 2, pp. 1319–1329, 2020. Crossref, https://doi.org/10.1109/TSG.2019.2935977
[20] Jonas Engels, Hamada Almasalma, and Geert Deconinck, “A Distributed Gossip-based Voltage Control Algorithm for Peer-to-Peer
Microgrids,” 2016 IEEE International Conference on Smart Grid Communications, pp. 370–375, 2016. Crossref,
https://doi.org/10.1109/SmartGridComm.2016.7778789
[21] Modu Abba Gana, Ishaku Abdul Dalyop, and Musa Mustapha, "Assessment of Reliability of Distribution Network with Embedded
Generation," SSRG International Journal of Electrical and Electronics Engineering, vol. 7, no. 2, pp. 34-37, 2020. Crossref,
https://doi.org/10.14445/23488379/IJEEE-V7I2P107
[22] Nader A. El-Taweel, Hadi Khani, and Hany E.Z. Farag, “Electrical Power and Energy Systems Voltage Regulation in Active Power
Distribution Systems Integrated with Natural Gas Grids Using Distributed Electric and Gas Energy Resources,” International Journal of
Electrical Power & Energy Systems, vol. 106, pp. 561–571, 2019. Crossref, https://doi.org/10.1016/j.ijepes.2018.10.037
[23] Ge Shaoyun et al., “Coordinated Voltage Control for Active Distribution Network on District Distribution Heating and Cooling
Coordinated Voltage Control for Active Network Considering the Impact of Energy Storage,” Energy Procedia, vol. 158, pp. 1122–1127,
2019. Crossref, https://doi.org/10.1016/j.egypro.2019.01.277
[24] P. C. Olival, A. G. Madureira, and M. Matos, “Advanced Voltage Control for Smart Microgrids Using Distributed Energy Resources,”
Electric Power Systems Research, vol. 146, pp. 132–140, 2017. Crossref, https://doi.org/10.1016/j.epsr.2017.01.027
[25] Um-e-Batool et al., "Controller for Voltage Profile Improvement of Double Fed Induction Generator based Wind Generator," SSRG
International Journal of Electrical and Electronics Engineering, vol. 7, no. 12, pp. 21-26, 2020. Crossref,
https://doi.org/10.14445/23488379/IJEEE-V7I12P104
[26] Josep M. Guerrero et al., “Hierarchical Control of Droop-Controlled AC and DC Microgrids - A General Approach Toward
Standardization,” IEEE Transactions on Industrial Electronics, vol. 58, no. 1, pp. 158–172, 2011. Crossref,
https://doi.org/10.1109/TIE.2010.2066534
[27] Xiaorui Liu, and Charalambos Konstantinou, “Reinforcement Learning for Cyber-Physical Security Assessment of Power Systems,” 2019
IEEE Milan PowerTech, PowerTech, pp. 1–6, 2019. Crossref, https://doi.org/10.1109/PTC.2019.8810568
[28] Aleksandar Boričić, Jose Luis Rueda Torres, and Marjan Popov, “Fundamental Study on the Influence of Dynamic Load and Distributed
Energy Resources on Power System Short-Term Voltage Stability,” International Journal of Electrical Power & Energy Systems, vol.
131, 2021. Crossref, https://doi.org/10.1016/j.ijepes.2021.107141
[29] Ana Paula Carboni de Mello, Luciano Lopes Pfitsche, and Daniel Pinheiro Bernardon, “Coordinated Volt / VAr Control for Real-Time
Operation of Smart Distribution Grids,” Electric Power Systems Research, vol. 151, pp. 233–242, 2017. Crossref,
https://doi.org/10.1016/j.epsr.2017.05.040
[30] Madalina M. Drugan, “Reinforcement Learning Versus Evolutionary Computation : A Survey on Hybrid Algorithms,” Swarm and
Evolutionary Computation, vol. 44, pp. 228–246, 2019. Crossref, https://doi.org/10.1016/j.swevo.2018.03.011