Abstract
Demand response (DR) is one of the most cost-effective elements of residential and small industrial building for the purpose of reducing the cost of energy. Today with broadening of the smart grid, electricity market and especially smart home, using DR can reduce cost and even make profits for consumers. On the other hand, utilizing centralized controls and have bidirectional communications Bi-directional communication between DR aggregators and consumers make many problems such as scalability and privacy violation. In this chapter, we propose a multi-agent method based on a Q-learning algorithm Q-learning algorithm for decentralized control of DR. Q-learning is a model-free reinforcement learning Reinforcement learning technique and a simple way for agents to learn how to act optimally in controlled Markovian domains. With this method, each consumer adapts its bidding and buying strategy over time according to the market outcomes. We consider energy supply for consumers such as small-scale renewable energy generators. We compare the result of the proposed method with a centralized aggregator-based approach that shows the effectiveness of the proposed decentralized DR market Decentralized DR market.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Assessment of demand response and advanced metering. Washington, DC, USA, Tech. Rep., Dec. 2012
Y. Li, B.L. Ng, M. Trayer, L. Liu, Automated residential demand response: Algorithmic implications of pricing models. IEEE Trans. Smart Grid 3(4), 1712–1721 (2012)
H. Aalami,M.P. Moghadam, G R. Yousefi, Optimum time of use program proposal for Iranian power systems, in Proceedings International Conference on Electrical Power Energy Conversion Systems, November 2009
S. Ashok, R. Banerjee, Optimal operation of industrial cogeneration for load management. IEEE Transactions Power System 18(2), 931–937 (2003)
J. Joo,S. Ahn, Y. Yoon,J. Choi, Option valuation applied to implementing demand response via critical peak pricing, in Proceedings IEEE Power Energy Society General Meeting, Jun 2007
A.B. Philpott, E. Pettersen, Optimizing demand-side bids in day-ahead electricity market. IEEE Trans. Power Syst. 21(2), 488–498 (2006)
M. Shafie-khah et al., Optimal behavior of responsive residential demand considering hybrid phase change materials. Appl. Energy 163, 81–92 (2016)
F. Wang et al., The values of market-based demand response on improving power system reliability under extreme circumstances. Appl Energy 193 220–231 (2017)
Q. Chen et al., Dynamic Price vector formation model based automatic demand response strategy for PV-assisted EV charging station. IEEE Trans. Smart Grid (2017)
F. Kamyab et al., Demand response program in smart grid using supply function bidding mechanism. IEEE Trans. Smart Grid 7(3), 1277–1284 (2016)
M.G. Vayá, L B. Roselló, G. Andersson, Optimal bidding of plug-in electric vehicles in a market-based control setup, in Power Systems Computation Conference (2014)
J. Mohammadi, G. Hug, S. Kar, Agent-based distributed security constrained optimal power flow, in IEEE Transactions on Smart Grid (2016)
S. Bahrami, M.H. Amini, A decentralized framework for real-time energy trading in distribution networks with load and generation uncertainty (2017), arXiv:1705.02575
M.H. Amini, B. Nabi, M.-R. Haghifam, Load management using multi-agent systems in smart distribution network, in IEEE Power and Energy Society General Meeting (PES) (IEEE, 2013)
M.G. Vayá, Roselló, G. Andersson, Centralized and decentralized approaches to smart charging of plug-in vehicles, in IEEE Power and Energy Society General Meeting (2012)
N. Rotering, M. Ilic, Optimal charge control of plug-in hybrid electric vehicles in deregulated electricity markets. IEEE Trans. Power Syst. 26(3), 1021–1029 (2011)
S. Bashash, S.J. Moura, J.C. Forman, H.K. Fathy, Plug-in hybrid electric vehicle charge pattern optimization for energy cost and battery longevity. J. Power Sources 196(1), 541–549 (2011)
A. Hoke, A. Brissette, D. Maksimovic, A. Pratt, K. Smith, Electric vehicle charge optimization including effects of lithium-ion battery degradation, in IEEE vehicle power and propulsion conference (2011)
J.K. Kok, M.J.J. Scheepers, I.G. Kamphuis, Intelligence in electricity networks for embedding renewables and distributed generations, Intelligent Infrastructures, Intelligent Systems, Control and Automation: Science and Engineering, vol. 42 (Springer, Netherlands, 2010), pp. 179–209
Z. Ma, D.S. Callaway, I.A. Hiskens, Decentralized charging control of large populations of plug-in electric vehicles. IEEE Trans. Control Syst. Technol. 21(1), 67–78 (2013)
L. Gan, U. Topcu, S. Low, Optimal decentralized protocol for electric vehicle charging. IEEE Trans. Power Syst. 28(2), 940–951 (2013)
S. Bahrami, M.H. Amini, M. Shafie-khah, J.P.S. Catalao, A decentralized electricity market scheme enabling demand response deployment. IEEE Trans. Power Syst. (2017). https://doi.org/10.1109/TPWRS.2017.2771279
L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: a survey. J. Artif. Int. Res. 4, 237–285 (1996)
C.J.C.H. Watkins, Learning from delayed rewards. Ph.D. thesis, King’s College, Cambridge, 1989
T. Krause, et al., A comparison of Nash equilibria analysis and agent-based modelling for power markets. Int. J. Electr. Power Energy Syst. 28(9), 599–607 (2006)
H.T. Haider, O.H. See, W. Elmenreich, Residential demand response scheme based on adaptative consumption level pricing. Energy 113, 301–308 (2016)
Acknowledgements
The work of Saber Talari, Miadreza Shafie-khah and João P.S. Catalão was supported by FEDER funds through COMPETE 2020 and by Portuguese funds through FCT, under Projects SAICT-PAC/0004/2015—POCI-01-0145-FEDER-016434, POCI-01-0145-FEDER-006961, UID/EEA/50014/2013, UID/CEC/50021/2013, and UID/EMS/00151/2013. Also, the research leading to these results has received funding from the EU Seventh Framework Programme FP7/2007-2013 under grant agreement no. 309048.
Amin Shokri Gazafroudi and Juan Manuel Corchado acknowledge the support by the European Commission H2020 MSCA-RISE-2014: Marie Sklodowska-Curie project DREAM-GO Enabling Demand Response for short and real-time Efficient And Market Based Smart Grid Operation—An intelligent and real-time simulation approach ref. 641794. Moreover, Amin Shokri Gazafroudi acknowledge the support by the Ministry of Education of the Junta de Castilla y León and the European Social Fund through a grant from predoctoral recruitment of research personnel associated with the research project “Arquitectura multiagente para la gestión eficaz de redes de energía a través del uso de técnicas de intelligencia artificial” of the University of Salamanca.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Najafi, S., Talari, S., Gazafroudi, A.S., Shafie-khah, M., Corchado, J.M., Catalão, J.P.S. (2018). Decentralized Control of DR Using a Multi-agent Method. In: Amini, M., Boroojeni, K., Iyengar, S., Pardalos, P., Blaabjerg, F., Madni, A. (eds) Sustainable Interdependent Networks. Studies in Systems, Decision and Control, vol 145. Springer, Cham. https://doi.org/10.1007/978-3-319-74412-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-74412-4_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74411-7
Online ISBN: 978-3-319-74412-4
eBook Packages: EngineeringEngineering (R0)