Abstract
Next-generation smart environments, an integral part of our modern lives, integrate computing and networking technologies to enrich our experiences. Harnessing cutting-edge technologies like the Internet of Things, Artificial Intelligence, and Edge Computing, they function under the control of critical infrastructures often processing complex computer vision tasks such as object recognition and image segmentation in real-time. These infrastructures manage vast volumes of data with intensive computational demands. In response to these challenges, infrastructures have evolved to distributed that consists of resources of different capabilities and different operators. Within these environments, the security and communication among different domains are fundamental. Each domain potentially has different levels of security requirements and may use various protocols for communication. As data travels across these domains, it is exposed to a variety of threats, including data breaches, cyberattacks, and unauthorized access. In such a environment, where multiple domains co-exist, each with its own unique resources and security specifications, communication constraints across them further complicate the resource allocation process. This complexity is further increased by the diverse computing and networking constraints imposed by applications. In this work, we propose a multi-Agent Deep Reinforcement Learning mechanism that operates based on multi-Agent Rollout and deep Q-learning in order to serve the different applications’ requirements. The proposed optimization mechanism considers multiple objectives during the resource allocation process and tries to fulfill the specific constraints set by the demands and the broader objectives set by the infrastructure operator. Through rigorous evaluations, we showcase the effectiveness and efficiency of our proposed mechanisms in accommodating the heterogeneous and stringent workload requirements, whilst optimizing the use of infrastructure resources. Our simulation experiments confirm that the proposed mechanism can substantially enhance the efficiency of resource allocation in critical infrastructures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, T.M.: Smart grids, smart cities need better networks. IEEE Netw. 24(2), 2–3 (2010)
Milanović, J.V., Zhu, W.: Modeling of interconnected critical infrastructure systems using complex network theory. IEEE Trans. Smart Grid 9(5), 4637–4648 (2018)
Varga, P., et al.: Making system of systems interoperable – the core components of the arrowhead framework. J. Netw. Comput. Appl. 81, 85–95 (2017). ISSN 1084-8045
Bertsekas, D.P.: Multiagent rollout algorithms and reinforcement learning. CoRR abs/1910.00120 (2019)
Li, X., Lian, Z., Qin, X., Jie, W.: Topology-aware resource allocation for IoT services in clouds. IEEE Access 6, 77880–77889 (2018)
Santoro, D., Zozin, D., Pizzolli, D., De Pellegrini, F., Cretti, S.: Foggy: a platform for workload orchestration in a fog computing environment. In: 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom) (2017)
Zubair, B.A.: Multi-agent systems for protecting critical infrastructures: a survey. J. Netw. Comput. Appl. 35(3), 1151–1161 (2012)
Pipattanasomporn, M., Feroze, H., Rahman, S.: Multi-agent systems in a distributed smart grid: design and implementation. In: 2009 IEEE/PES Power Systems Conference and Exposition, Seattle, WA, USA, pp. 1–8 (2009)
Panfili, M., Giuseppi, A., Fiaschetti, A., Al-Jibreen, H.B., Pietrabissa, A., Priscoli, F.D.: A game-theoretical approach to cyber-security of critical infrastructures based on multi-agent reinforcement learning. In: 2018 26th Mediterranean Conference on Control and Automation (MED), Zadar, Croatia, pp. 460–465 (2018)
Mutlag, A.A., et al.: Multi-agent systems in fog-cloud computing for critical healthcare task management model (CHTM) used for ECG monitoring. Sensors 21, 6923 (2021)
Alfakih, T., Hassan, M.M., Gumaei, A., Savaglio, C., Fortino, G.: Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. IEEE Access 8, 54074–54084 (2020)
Wang, Y., Zhang, W., Deng, H., Li, X.: Efficient resource allocation for security-aware task offloading in MEC system using DVS. Electronics 11(19), 3032 (2022)
Chen, L., et al.: IoT microservice deployment in edge-cloud hybrid environment using reinforcement learning. In: IEEE Internet of Things Journal, vol. 8, no. 16, pp. 12610–12622 (2021)
Quang, P.T.A., Hadjadj-Aoul, Y., Outtagarts, A.: A deep reinforcement learning approach for VNF forwarding graph embedding. IEEE Trans. Netw. Serv. Manage. 16(4), 1318–1331 (2019)
Bunyakitanon, M., Vasilakos, X., Nejabati, R., Simeonidou, D.: End-to-end performance-based autonomous VNF placement with adopted reinforcement learning. IEEE Trans. Cogn. Commun. Netw. 6(2), 534–547 (2020)
Pei, J., Hong, P., Pan, M., Liu, J., Zhou, J.: Optimal VNF placement via deep reinforcement learning in SDN/NFV-enabled networks. IEEE J. Sel. Areas Commun. 38(2), 263–278 (2020)
Sun, P., Lan, J., Li, J., Guo, Z., Hu, Y.: Combining deep reinforcement learning with graph neural networks for optimal VNF placement. IEEE Commun. Lett. 25(1), 176–180 (2021)
Jalodia, N., Henna, S., Davy, A.: Deep reinforcement learning for topology-aware VNF resource prediction in NFV environments. In: 2019 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Dallas, TX, USA, pp. 1–5 (2019)
Hester, T., et al.: Deep Q-learning from demonstrations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
Pallewatta, S., Kostakos, V., Buyya, R.: Microservices-based IoT application placement within heterogeneous and resource constrained fog computing environments. In: 12th IEEE/ACM International Conference on Utility and Cloud Computing, pp. 71–81 (2019)
Acknowledgement
The work presented is supported by the EU Horizon 2020 research and innovation program under grant agreement No. 101017171 in the context of the MARSAL project and by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “2nd Call for H.F.R.I. Research Projects to support Faculty Members & Researchers” (Project Number: 04596).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Soumplis, P., Kokkinos, P., Varvarigos, E. (2023). Efficient Resource Provisioning in Critical Infrastructures Based on Multi-Agent Rollout Enabled by Deep Q-Learning. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14361. Springer, Cham. https://doi.org/10.1007/978-3-031-47969-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-47969-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47968-7
Online ISBN: 978-3-031-47969-4
eBook Packages: Computer ScienceComputer Science (R0)