Combining Data-Driven and Model-Driven Approaches for Optimal Distributed Control of Standalone Microgrid

Abstract

This paper focuses on the comprehensive restoration of both voltage and frequency in a standalone microgrid (SAMG). In a SAMG, the power balance is achieved through traditional methods such as droop control for power sharing among distributed generators (DGs). However, when such microgrids (MGs) are subjected to perturbations coming from stochastic renewables, the frequency and voltage parameters deviate from their specified values. In this paper, a novel hybrid-type consensus-based distributed controller is proposed for voltage and frequency restoration. Data-based communication is ensured among the DGs for controlling voltage and frequency parameters. Different parameters such as voltage, frequency, and active and reactive power converge successfully to their nominal values using the proposed algorithms, thereby ensuring smooth operation of inverter-dominated DGs. Additionally, the machine-learning-based long short-term memory (LSTM) algorithm is implemented for renewable power forecasting using historical data from the proposed location for visualising the insolation profile. The effectiveness of our approach is demonstrated through a SAMG, which consists of four inverters, showing that the proposed approach can improve system stability, increase efficiency and reliability, and reduce costs compared to traditional methods. The complete study is performed in Python and MATLAB environments. Our results highlight the potential of data-driven approaches to revolutionise power system operation and control.

1. Introduction

1.1. Motivation and Bibliographic Review

Power systems are being upgraded across the globe as part of the transition towards modernised grids. The trend is to shift from conventional, centralised, carbon-rich energy systems to green, renewable, and reliable frameworks. Conventional sources of electricity such as thermal power plants use fossil fuels as an input, proving costlier, emitting carbon, polluting our environment, and incurring huge losses [1]. In spite of remarkable advancements in delivering electricity, conventional sources of electricity hardly fulfil the power demand of consumers [2]. Modern society demands reliable, cost-effective electricity and the modernization of power systems. Currently, such stressed power grids are being replaced by autonomous, self-sufficient, bi-directional active networks of low-voltage, renewable-rich distributed energy resources (DERs) that lead to the development of microgrids (MGs). These DERs encompass diesel generators and inverter-based resources (IBRs) such as photovoltaic (PV) generation, controllable loads, energy storage systems (ESS), and electric vehicles (EVs) with the capability of exchanging power with the grid and also providing support to the grid in terms of voltage and frequency regulation, thereby keeping system stability intact [3,4]. These MGs encompass multiple resources interfaced with power electronic devices, as a significant contribution of energy comes from renewables.

The world is interested in utilising renewable energy sources (RES), particularly solar energy, owing to their economic and environmental attributes. In the past few years, in our country, India, the Ministry of New and Renewable Energy (MNRE) has also recommended that RES be used rapidly because of environmental concerns, global warming, government programmes, and significant price drops in products used in RES. Among RES, electricity generated from PV systems is perennial and one of the fastest-growing sources of electricity [5]. PV systems generate electricity by utilising absorbed solar irradiance. RES, such as solar and wind, not only contribute to green energy but also reduce operational costs. With the increasing use of RES and the integration of DERs, the optimal operation and management of MGs has become more challenging owing to their stochastic and heterogeneous nature and weak inertia embedded in IBRs, which develop stability issues at various levels in a power system network. Therefore, the incorporation of RES will significantly change the pattern of power system operation and make it highly variable and diversified. Hence, accurate forecasting, planning, and control are becoming essential for power grid planners and operators to avoid power system contingencies.

The power generated from PV systems depends on several factors such as solar irradiance, cloud cover, and other environmental conditions and hence requires proper forecasting and planning strategies for their sustainable development. To ensure reliable operation of MGs, it is important to forecast the power output from PV systems at regular intervals. Furthermore, unlike data-driven methods, conventional methods of power system design rely on both physical mechanisms and static mathematical modelling, referred to as model-driven methods. Data-driven approaches for developing dynamic system models have become highly relevant today for power grids with highly penetrated inverter-based resources (IBRs) [6]. One approach to address this challenge is through data-driven methods, e.g., artificial intelligence (AI) and machine learning (ML) techniques, which help in optimal planning and control of power systems owing to the massive availability of data from PMUs, smart metering, intelligent monitoring, self-recovering capability schemes, and advanced metering infrastructure (AMI) [7]. Data-driven-based forecasting helps remove the intermittent nature of RES by providing information about future generations originating from such renewables.

Among data-driven techniques, ML is a novel approach for power system operation and control. ML algorithms adapt to their environment and compute results quickly by enabling computers to learn from the data, thereby improving their performance on a particular task. It has been observed that these techniques outperform conventional approaches, such as predictive maintenance of transmission lines and transformers, forecasting, and planning [8]. Due to the minimal risk associated with power system functioning, their applications are gradually being adopted in power systems. In spite of the early promising results reported in the research literature about ML, power system operators find it difficult to implement in real-time applications. Researchers are currently working on addressing this scepticism and trying to remove barriers, allowing ML to enter power systems. ML algorithms are extremely fast at forecasting and estimating operating points [9]. ML algorithms can access thousands of scenarios at the same time, which conventional systems cannot. This makes system operators aware of the difficulty of determining critical points amidst thousands of scenarios in a very short period of time [10]. ML algorithms can better capture the non-linear dynamics of RES, provided careful selection of input features is ensured. ML-based forecasting helps in the reliable and economic operation of the electricity grid.

In this paper, the long short-term memory (LSTM) algorithm, which is a kind of recurrent neural network with the property of handling sequential data and the capability of retaining past information, is used to directly predict the solar power of a proposed location from the historical data available at the National Renewable Energy Lab (NREL) [11] based on the geographical latitude and longitude of the selected location. LSTM is applicable to different processes, such as processing, predicting, and classifying time-series data. However, owing to its property of holding data for the maximum time, it is more suitable for prediction compared to other algorithms. Based on the forecast model, planning and operation for MGs are proposed for a local area, including our institute campus, situated in Srinagar, the capital of Jammu and Kashmir union territory. The pictorial representation of the proposed MG is shown in Figure 1.

ML algorithms can better capture the non-linear dynamics of RES, provided careful selection of input features is ensured. ML-based forecasting helps in the reliable and economic operation of the electricity grid. While reviewing the application of novel data-driven methods for MGs, numerous publications reveal that ML-based models will help us forecast generation from RES and thereby assist in the development of real-time models of the system. Past publications have stressed static model-based approaches employing probabilistic distributions to generate renewable power scenarios. The work reported in [12] discusses the heterogeneity of storage devices. Moreover, the authors of [13] have implemented particle swarm optimisation (PSO) for the best sizing of DGs in a stochastic environment. Previous research has mostly covered probabilistic models without considering the randomness of solar and load [14,15]. In [2], the author proposed a load model considering the influence of peak demand, time, price, and temperature. The proposed load model is then formulated by a neural network using realistic data. In [16], neural network models are used to give a time-series power scenario.

Data-driven approaches to active distribution networks have been proposed, addressing methods for optimal power flow (OPF) against uncertainty from renewable sources. Similarly, the authors of [17] suggest an online optimal operation technique for MGs through ADP. Moreover, stochastic mixed-integer non-linear programming (MINLP) has been developed considering the entire modelling of ESSs. The developed MINLP problem is partitioned into several single-programming-type sub-problems and then solved using ADP. During the training stage, an optimal policy can be obtained that can assist in making optimal decisions, e.g., power outputs of dispatchable generators and charging and discharging of ESS pertaining to different system states. In [15], the authors suggest an online energy management strategy for reducing overall operation costs associated with MGs. The subsequent decision-making problem is developed as a finite multi-horizon problem and solved by approximate dynamic programming, which is a type of reinforcement learning (RL). Regarding optimal load demand response, the EMS is then integrated with a trained neural network to predict real power. In [18], approximate forecast error behaviour is modelled using a first-order time-series regressive model with increasing noise, while in [19], an autoregressive moving average concept is implemented to generate spatiotemporal scenarios with given power generation profiles at each renewable generation site. In [20], a radial basis function neural network is combined with a particle swarm optimisation algorithm to generate scenarios with input from weather predictions. The paper has not covered scenario generation to mitigate undesirable uncertainty from such RES. Moreover, the author has not addressed the precise assessment of MG planning with optimal sizing of DERs amid growing load demand and generation uncertainty.

ML algorithms can better capture the non-linear dynamics of RES, provided careful selection of input features is ensured. ML-based forecasting helps in the reliable and economic operation of the electricity grid.

While reviewing the use of new data-driven methods for MGs, many publications show that ML-based models will help us predict generation from RES and, in turn, help us make models of the system that work in real time. Past publications have stressed static model-based approaches employing probabilistic distributions to generate renewable power scenarios. The work reported in [12] discusses the heterogeneity of storage devices. Moreover, the authors of [13] have implemented particle swarm optimisation (PSO) for the best sizing of DGs. The proposed approach enables real-time monitoring, analysis, and control of the MG system, leading to optimal operation and improved energy management. The literature review also reveals that MG planning, operation, and control have been carried out based on traditional optimisation-based methods, but data-driven, model-free scenario-based control, operation, and planning have not been reported. In [21], the author has proposed a short-term wind forecasting model while taking into account the spatio-temporal correlation among different farms. The author reports two methods, namely, regression and feature extraction. The former method is performed by LSTM and then forwarded to the CNN for wind power forecasting.

Most approaches implement a model through historical findings and sample probabilistic models to generate new scenarios. Preprocessing of data is required for most of the proposed methods to eradicate garbage data. Dynamic weather and composite spatial and temporal relations make model-based approaches challenging when more than one renewable power plant is taken into account. These complicated interactions cannot be shown by a single set of model parameters. Based on the above literature review, it has been shown that there is a strong need to propose a combined data and model-based technique that will provide early warnings to power system experts about spatial variations in renewable generation and load and help in proper planning and operational strategies for minimising unscheduled power curtailments and technical challenges, which otherwise create stability issues [22]. Fusing data-driven and model-based techniques assists in the smooth functioning of inverter-dominated MGs. Robust controllers are required for controlling voltage, frequency, and active and reactive powers in a SAMG when subjected to load and generation perturbations. The non-linear properties concealed in the data may be extracted by ML-based methods. Such methods describe the correlation between inputs and outputs and aid in the analysis of power systems. Since deep learning (DL) is capable of feature extraction, it helps implement data-driven strategies in many areas related to power grids. Data-driven optimal planning and control provide a powerful approach to optimising the operation of smart MG systems. The approach uses historical and real-time data to model, forecast, and optimise the system’s operation while ensuring that the energy demand is met and the system operates within the permissible limits [23,24].

Table 1. Comparison between model-based approach and data-driven approach.

Aspect	Data-Driven Approach	Model-Based Approach
Scability	1. It can adapt to changing conditions and unforseen events. 2. Can learn from historical and real-time data. 3. Can predict future events.	1. Static models cannot adapt to changing conditions. 2. Works on static models. 3. Cannot capture future events.
Learning capability	1. Reliable for large systems. 2. Unfolds hidden information embedded in large datasets. 3. Suitable for online applications.	1. Suitable for small-sized microgrids. 2. Suitable for offline applications. 3. Developing accurate models is challenging.
No need for exact system knowledge	1. Data-driven models do not require precise understanding of all components. 2. Less Reliable for large systems.	1. Require significant computational resources. 2. Accuracy of model can be time consuming.
Nature of model	1. In data-driven approach, a dynamic model represents behaviour of a system over time. 2. Execution of method is easy, as time per iterations are least. 3. Intelligently controlled 4. Future power system will be data-driven.	1. A static model gives time-invariant input–output relationship of a system. 2. Other approaches are needed to obtain the desired accuracy. 3. The application of a model-based approach will be less accurate. 4. Works on past experience.

is a comparative table depicting the advantages and disadvantages of the existing approach and the proposed one.

1.2. Aim and Contribution

Renewable energy sources, such as solar and wind, exhibit inherent variability and intermittency, making accurate forecasting and control essential for efficient operation and integration into the power grid. ML algorithms can analyse historical and real-time data, incorporating various factors such as weather patterns, historical generation data, load profile and other relevant variables to generate accurate forecasts. Such information can be clubbed with model-based approaches to ensure better operation and control for renewable rich MGs. Therefore, combining model-driven and data-driven methods is an emerging and preferred approach for smart MG operation and control amid growing challenges due to uncertainty.

The main contributions of this manuscript are:

• A new distributed secondary control scheme for voltage and frequency control and active and reactive power exchange of MGs is proposed without relying on the traditional hierarchical droop control strategy, removing the need to rely on the output impedance of inverter and communication links between primary and secondary control. One of the best and most promising solutions to ensuring grid stability is through new secondary control with the attribute of controlling fluctuations in electrical demand and renewable generation, unlike the conventional centralized secondary control, which is prone to a single point of failure.
• The stability of MGs is analysed through the proposed secondary control, which refers to a decentralised approach for smooth power output from DERs. Distributed secondary control works on ML technology to make self-control decisions by gathering information from neighbouring DGs. The suggested approach offers multiple advantages, such as increased reliability and robustness to dynamic grid conditions and interactions. It should also be emphasised that the proposed methodology developed in this paper can give better results by ensuring the stability of the whole system despite the communication failure.
• A ML-based LSTM approach is proposed for forecasting hourly PV power output using historical as well as real-time data. The area selected is our institute campus, where currently no solar rooftop system has been erected or is operational. Figure 2 illustrates the hourly load demand profile for one complete year, encompassing both days and nights, totaling 8760 h. However, for simplicity and ease of understanding, the 8760 data points have been down-sampled to only 365 samples. This down-sampled representation is based on historical data taken from local substation and serves to visualise the inherent randomness in the load profile for the proposed location.
• The steady-state data of the novel secondary control is utilised for estimating the future disturbance size in the proposed system.

The remainder of this paper is ordered as follows: Section 1 discusses a literature review pertaining to the combined data-driven and model-based approaches in smart grids. Section 2 reports methodology framework of the proposed study. Section 3 discusses applications of a data-driven approach. Similarly, Section 4 presents the novel control strategy of MGs, and Section 5 reports concluding remarks and future research directions.

2. Methodology Framework

Data-driven optimal planning and control uses historical and real-time data to model, forecast, and optimise the operation of the MG system. The approach involves the following steps:

• Data collection. System data, including irradiance, ambient temperature, wind speed, and weather data, are collected.
• Data preprocessing. The collected data is cleaned, normalised, and transformed into a form suitable for training and testing.
• Model development. Mathematical models are developed to predict renewable generation, load demand, and eventually the system’s behaviour.
• Optimisation. An optimisation algorithm is applied to the developed model to find the optimal operation strategy for the proposed MG system. The objective is to minimise the operation cost while ensuring that the energy demand is met and the system operates within the permissible limits.
• Control implementation. The optimized operation strategy is implemented by controlling the DERs, using novel distributed secondary control.

The actual monthly irradiance (kw/m${}^2$) of the proposed location is shown in Figure 3. For ease of understanding, 8760 h of one complete year have been down-sampled to 1200 h only. Furthermore, the data-based actual and forecasted solar irradiance of the proposed location is shown in Figure 4. Owing to the computational burden and for ease of understanding, forecasting of only one complete year has been shown here. One year of hourly solar data has been taken from NREL using proper latitude and longitude (34.083${}^{\circ }$ N and 74.797${}^{\circ }$ E). As illustrated in Figure 5, the data are segregated for training and testing stages using the 80:20 criterion, i.e., 80% for training and 20% for testing. The model parameter details are shown in

Table 2. Model parameter details.

S. No.	Models	Training Time	RMSE	MAPE	Convergence Time
1	LSTM	66.219 s	1.392%	5.5%	66 s
2	Regression learning	132.838 s	42.94%	11.92%	115 s

. Moreover, the performance curve is shown in Figure 6 for depicting the error versus training epochs curve. The training data are what we feed to the model; it is usually much larger in size than the testing data. Similarly, testing data is what we use to evaluate the model. Additionally, in data-driven power system operation, we identify a power system problem and gather pertinent information from measuring units (PMUs) about such a problem. Historical load data and real-time weather data are merged together to generate a load forecast, as shown in Figure 7. While training an agent, it is required to generate a control signal so that unknown inputs fall very close to the distribution of training data, relying on a specific algorithm. During training, the agent acts as a black box, obtaining an input signal and giving a control command. An agent is appreciated for choosing an environment and finding a better solution. Data-driven ML-based models can be implemented in the power system to replace or assist conventional models, thereby removing approximations, and deviations in parameters can be mitigated by extracting information from available data. A desirable ML-based model-free approach is best to counteract an incomplete, stochastic environment promptly and assure system stability. A desirable ML-based model of the power system is then built once information from the raw data has been extracted using appropriate classification and regression-based methods. Various power system characteristics are anticipated using a suitable prediction techniques.

3. Applications of Data-Driven Approach

A smart MG is a modern grid that integrates RES, ESS, intelligent monitoring devices, and advanced ML control technologies to enable efficient and reliable power distribution. Such data-driven methods help in the optimal control and planning of DERs by leveraging historical data. The availability of different ML algorithms assists in the development of dynamic models, which can give better prior warnings to power system experts. Some potential applications of the proposed approach are discussed below:

• Energy management. The actual monthly irradiance (kw/m${}^2$) of the proposed location is shown in Figure 3. For ease of understanding, 720 h of every month have been down-sampled to 100 h only. Furthermore, the data-based actual and forecasted solar irradiance of the proposed location is shown in Figure 4. Data-driven approaches can be used to optimise energy management in smart MG systems. Real-time data from smart metres, weather forecasts, and other intelligent monitoring devices can be used to predict generation, energy demand, and the requirement for energy storage devices to regulate energy supply and demand.
• Fault detection and diagnosis. Data-driven techniques can be used to detect and diagnose faults in smart MG systems. Machine learning algorithms can be trained to identify anomalies at power generation, distribution, and consumption stages, which can help in identifying potential faults and help power system operators to take corrective measures.
• Renewable energy integration. Smart MGs can integrate RES such as solar and wind power on a large scale by using advanced techniques. Data-driven approaches can be used to optimise the use of these sources, taking into account weather conditions and energy demand.
• Load forecasting. Data-driven approaches can be used to forecast load and renewable generation in smart MG systems. ML-based algorithms can be trained to predict future energy consumption patterns based on historical data, which can help in planning power generation and distribution.
• Predictive maintenance. Data-driven approaches can be used to predict maintenance requirements for smart MG systems. ML algorithms can be trained to predict maintenance of different components such as batteries or inverters, which can help reduce downtime and extend component lifetimes.
• Energy trading. Data-driven approaches can be used to enable peer-to-peer energy trading in smart MG systems. Block-chain-based platforms can be used to securely trade energy between prosumers, which can help reduce energy costs and increase energy efficiency.

3.1. Forecasting and Anomaly Detection

Accurate forecasting and planning of RES such as solar and wind energy helps lower generation uncertainty and undesirable power changes, leading to secure grid management. Data-driven-based ML algorithms aid in the accurate forecasting of RES by employing historical data. There are several ML algorithms such as recurrent neural networks (RNN), decision trees, random forests, deep neural networks (DNN), etc. Among ML algorithms, LSTM performs better forecasting and has a past memory-retaining property. The schematic model of LSTM is shown in Figure 8. Some potential applications of the proposed data-driven approaches in smart MGs are depicted in Figure 7.

LSTM unit consists of three stages:

• Forget gate. The forget gate determines what percentage of long-term memory to remember/forget.
• Input gate. The input gate creates a potential for new long-term memory based on input and determines what percentage of that long-term memory is to be retained.
• Output gate. It uses a percentage of long-term memory (determined by input) to give an output.

When it comes to anomaly detection, the protection system must be capable of recovering the system after being exposed to any anomaly. When such protection is used with DERs that are interfaced to inverters, conventional fault detection techniques do not operate well since DERs do not produce enough fault currents to activate protective systems. The data-driven, model-free approach is helpful under such circumstances. The use of measured data aids in the detection of anomalies.

3.2. Energy Management

Data-driven MG energy management involves using real-time data to optimise the performance of a MG, which is a localised grid that can operate independently or in parallel with the main power grid. The goal of data-driven MG energy management is to minimise energy costs, maximise energy efficiency, and ensure reliable power supply to the MGs. Energy management becomes increasingly indispensable with the extensive penetration of new units in the distribution network. The primary goal of power system operators and their users is to get a reliable, affordable, and sustainable energy supply. The management of unpredictable factors, their coordination with various resources, and the development of intelligent methods and potent ML algorithms are all necessary to achieve this goal and provide improved smart grid management. Power system operators are trying to figure out how to use ML approaches to deal with the growing uncertainty caused by RES and to make sure that energy management is carried out in an efficient way.

The objective of an energy management system (EMS) is to coordinate various DERs in order to achieve operational goals. High levels of unpredictability related to DERs make accomplishing this objective difficult. Energy storage (ES) can play a very important role in grid integration and smoothing the intermittent nature of RES by increasing the system’s overall reliability. It can improve power quality, reduce peak demand, enhance the capacity of distribution and transmission grids, reduce deviation penalties, etc. This would also make it possible to reduce the amount of diesel generator usage that acts as a backup power unit. ESS is the main component of not only EVs in terms of cost and performance determination, but use of ESS devices by residential, commercial, or industrial consumers in conjunction with RES has the potential to improve power quality and reliability for such consumers. The push for electric mobility would considerably lessen the nation’s reliance on imported fossil fuels by deploying indigenous, cutting-edge, and reliable energy storage. Data-driven operated ES devices are seen as excellent solutions for addressing load and generation mismatches and enabling maximum incorporation of RES.

3.3. Microgrid Operation and Control

Data-driven power system control is a field of study that uses large volumes of data to make informed decisions about controlling power systems. This approach relies on advanced data analytics techniques and ML algorithms to make sense of the data and provide actionable insights. The main goal of data-driven power system control is to improve the efficiency, reliability, and stability of power systems. By analysing large amounts of data from various sources, such as sensors, smart metres, and control systems, power system operators can gain a deeper understanding of how the system behaves under different conditions. Overall, data-driven power system control has the potential to revolutionise the way power systems are operated and maintained, leading to more efficient and reliable energy infrastructure. Normally, control problems pertaining to power grids are dealt with through regulation and tracking problems. The objective of the former is to chase a reference signal, and that of the latter is to meet a specific control function. ML works well for solving control-based problems, especially in cases where power system information is unknown but relevant data is available. Integration of available data, model-based approaches, and historical experience can assist operators in knowing the future state of a system when properly exploited. Our proposed approach is capable of restoring voltage and frequency parameters to nominal values despite load fluctuations. The traditional droop-based control is not able to bring voltage and frequency to nominal values (Figure 9).

4. Novel Control Strategy of Microgrid

As per the swing equation, the inertial response of synchronous machine can be described as:

$$\frac{J{d}^2\delta }{d{t}^2}=P_m-P_e$$

(1)

where J is moment of inertia in KG-$M^2$, $\delta$ is angular, shift of rotor, and $P_m$ and $P_e$ are the mechanical and generated electrical power, respectively. Furthermore, we know the inertia constant is the ratio of kinetic energy stored in the rotating mass of machine to the rating of the machine (MVA). Mathematically, it can be represented as:

$$H=\frac{1/2J{\omega }_{o^2}}{S}$$

(2)

$$J=\frac{2SH}{{\omega }_{o^2}}$$

(3)

Substituting the value of J in Equation (1), we get:

$$\frac{2HS}{{\omega }_o}\frac{d\omega }{dt}=P_a=P_m-P_e$$

(4)

Choosing $\omega$ = 2$\pi$f, the RoCoF at the beginning stage after power disturbance can be calculated from (2) and can be approximated to:

$$\frac{df}{dt}=\frac{P_m-P_e}{S}\frac{f_o}{2H}$$

(5)

From (5), it can be observed that after a sudden generation loss or load disconnection, the change in frequency at the initial level will be dominated substantially by the large disparity between $P_m\&P_e.$ The frequency deviation will be large if a disturbance hits the system with a relatively weak inertia constant. For geographically interconnected synchronous generators, we can group them into an equivalent rotating mass, and their combined inertia constant can be computed as:

$$H=\frac{{\sum }_i^G{H}_i.S_i}{S}$$

(6)

where G is total number of generators and $H_i$ and $S_i$ are the inertia constant and rated power (MVA) of the generator. Figure 10 illustrates a standalone MG configuration that consists of four inverters represented as DGs in the figures. The presented setup is utilized for the purpose of validating a novel secondary control method.

MGs encompass sophisticated control strategies to manage power flow and coordinate DERs. The hierarchical control strategy is entirely implemented in MGs. Primary, secondary, and tertiary are the three conventional hierarchical control strategies implemented in MGs. The primary and secondary control is not capable of stabilizing the frequency and voltage in a standalone MG when subjected to a perturbation, as shown in Figure 11, Figure 12, Figure 13 and Figure 14, respectively.

A novel secondary approach effectively maintains the power balance between MG units by adjusting the active power to frequency (P-F) and reactive power to voltage (Q-V) when the primary control fails. The conventional secondary, global, and emergency controls involve communication, unlike the proposed integrated approach.

The MG is coupled to the primary distribution grid in conventional operation mode, either partly supplied or injecting some power into it. In a grid-connected manner, the primary grid and local DGs can supply power to loads. However, if any abnormality in the traditional grid erupts, we do not have to make a manual inspection; rather, an ML-based algorithm for abnormality detection can be used using real-time monitoring data, thereby ensuring the stability of the MG. Under such conditions, an autonomous operation mode can be turned on as the whole network is operated in an organised form and local control is employed at the primary stage. Furthermore, DER plug and play capability refers to the ability to add or remove DERs seamlessly without significant disruptions to the overall interconnected system. When a new DER is connected or disconnected (unplugged) from the system, the data-driven ML can detect and adjust changes without manual intervention. The layered control structure of a MG can be operated by integrating both a data-driven and a model-based approach. The layered control structure of a MG typically includes three layers:

• Management layer. This layer is responsible for monitoring the overall status of the MG and making high-level decisions. It includes functions such as load forecasting, energy management, and coordination with the main power grid.
• Control layer. This layer is responsible for maintaining the stability of the MG by regulating the power output of all DERs. It performs functions such as voltage regulation, frequency control, and power balancing.
• Device layer. This layer deals with the individual DERs and their controllers, which adjust their power output based on the signals received from the control layer. It includes functions such as inverter control, battery management, and generation control.

The management layer provides the overall strategy for the MG, while the control layer ensures that the DERs operate in a coordinated manner. The device layer is responsible for the detailed control of individual DERs, such as regulating the output of a solar panel or charging a battery. Overall, the proposed integrated control structure of a MG enables it to operate efficiently, adapt to changing conditions, and maintain reliable power delivery to its customers despite load excursions.

4.1. Data-Driven-Based MG Operation and Control

Access to more computational power and communication networks has improved recently, which has really facilitated the ability to use highly developed data-driven methods for design, optimisation, control, and assessment in all areas of power engineering. The future power grids will be run by AI and ML due to significant development in AI and ML-based powerful algorithms. Traditional model-based control will be entirely replaced by data-driven control. The traditional approach, as shown in Figure 15, requires human rules and modelling, unlike the data-driven ML approach, wherein it is possible to get forecast models using training and testing approaches, which will improve efficiency and remove ongoing challenges.

A static model represents the time-invariant input–output relationship of a system, whereas a data-driven dynamic model represents the behaviour of a system in real time. MG stability refers to the ability of a MG to maintain a balanced and reliable operation under various operating conditions and disturbances. Stability analysis is vital because an unstable MG can lead to frequency excursions, voltage instability, and eventually blackouts. When a MG is disturbed by changes in load or uncoupling of units, traditional model-based primary control cannot keep the voltage and frequency of the system at nominal values because it relies on a good communication system. This is clear from Figure 11. The concept of data-driven intelligent optimal hybrid energy storage with the objective of making it available for a longer duration is emerging for mitigating the heterogeneity of RES. Data-based, novel secondary control helps to mitigate frequency and voltage deviations, thereby allowing the system to run in a stable state despite load fluctuations. The same is reported in Figure 16 and Figure 17, respectively.

This disconnection and reconnection of the MG with the primary grid is performed through a conventional static bypass switch. During this connection and disconnection of MG, several technical issues arise, such as voltage, frequency, and active and reactive power deviations, which must be maintained constant. Such technical issues for the stable operation of MGs have been addressed in many research areas by different research experts without addressing the stochastic behaviour of load and generation through a data-driven approach. Moreover, nobody has addressed the versatile application of data-driven operation and control in MG. Utilising ML for such applications yields better results for forecasting and demand response in a power system, so using this approach can effectively solve the power system’s problems and challenges in a better way.

4.2. ML-Based Optimization

Machine learning (ML)-based optimisation has become a popular approach in power systems due to its ability to handle large and complex datasets and identify non-linear relationships between variables. One key advantage of data-driven methods is their ability to adapt to changing conditions and acquire knowledge from new data. This adaptability allows them to continuously enhance their optimisation performance as additional data becomes accessible. The use of ML in power system optimisation has led to improved efficiency, reduced costs, and enhanced reliability. One of the most common applications of ML in power system optimisation is in renewable generation and load forecasting. Load forecasting involves predicting the amount of power that will be required at a given time, which is critical for ensuring that the power system is able to meet demand. ML techniques such as artificial neural networks (ANNs), support vector machines (SVMs), and decision trees have been used to develop accurate load forecasting models. Another area where ML has been applied in power system optimisation is in the control of DERs. DERs such as solar panels and wind turbines generate power that is fed back into the grid, and their output can be difficult to predict and control. ML techniques can be used to predict the output of DERs and optimise their operation to minimise costs and improve reliability.

Overall, the use of ML in power system optimisation has the potential to revolutionise power systems.

Nowadays, the complexity of power systems has increased due to the increased penetration of RES and EVs, the number of variables in optimisation problems also increases, and classical optimisation techniques fail to solve them. It becomes a challenging task to optimise such problems using traditional approaches. Owing to widespread advancements in data science, the development of efficient ML-based algorithms is continuously progressing to solve conventional static model-based problems. It has been observed that by incorporating the concept of ML with different optimisation algorithms, including those that are inspired by nature, the performance of computational power is increasing while solving complex problems. RES present a large search space and result in a large scenario set for such a scenario generation. The typical approach for scenario reduction is to merge similar scenarios into one specific scenario so that the optimisation time required is reduced.

4.3. Other Aspects

Energy storage systems (ESS) play a critical role in grid integration and stabilising the heterogeneous nature of renewables. ESS assists MGs by increasing the overall flexibility of the system, improving power quality, reducing peak demand, and enhancing the capacity of distribution grids. Moreover, it helps in reducing the use of diesel generators, which incur more expenditure and are environmentally unfriendly. Owing to the weak inertia problem in MGs, ESS helps stabilise voltage and frequency through virtual inertia mode. Optimal operation of ESS involves a multi-period decision-making mechanism. Because optimal ESS performance is dependent on the state of charge (SOC) and state of health. When RES are in excess, they charge storage devices up to their rated capacity. In the absence of RES, battery storage is used to meet load demand.

5. Concluding Remarks and Future Research Directions

This work proposes a novel distributed optimal operation of a standalone MG along with a continuous time-enabled adaptive data-driven secondary controller to improve voltage, frequency regulation, and active and reactive power sharing in SAMG. The suggested controller decouples active and reactive power, frequency, and voltage problems, enhancing transient performance. The proposed approach updates NN weights online without system parameters. The suggested controller can automatically adapt to unexpected uncertainty induced by adding or removing DGs from the MG. Even with partial communication loss, the voltage, frequency, and active and reactive power exchange are good. The proposed controller’s minimum parameters facilitate real-system implementation and adaptation.

Based on our comprehensive observations towards significant proliferation of the Internet of Things and wide intelligent monitoring devices, we suggest several future research directions towards data-driven power system operation and control:

• Data-driven transactive energy for optimal operation of DERs;
• Data-driven demand response management;
• Data-based planning and control of modern grids;
• Data-based cyber security analysis in modern grids.