A Bioinspired Emergent Control for Smart Grids

Satisfying energy demand has become a global problem that is on the rise due to population growth, infrastructure deterioration, a decline in fossil fuel sources, and high costs for investment, among others. Smart grids, in addition to those challenges that they have at the level of energy generation, have other management challenges derived from the great diversity of components that make them up, such as energy storage systems (batteries, capacitors, etc.), the different types of consumers (controllable, non-controllable loads) and prosumers (electric vehicles, self-sustaining buildings, micro-grid, etc.), among others. Consequently, a distributed control problem is presented, mainly oriented to the coordination of its components. A possible solution is to achieve the participation of each component when conditions are more favorable, such as prioritizing production with renewable energy sources, or taking advantage of prosumers so that they can meet local demand, among other things. Therefore, new strategies with a distributed approach such as bio-inspired emergent controls are necessary. The objective of this work is the specification of an emergent control approach to coordinate a smart grid. This approach allows the coordination of the energy supply in various operating scenarios. The results obtained demonstrate a perfect synchronization between the different smart grid components (agents), prioritizing renewable energy sources, regardless of the operational context (for example, in cases of failures, unsuitable environmental conditions, etc.).


I. INTRODUCTION
In recent years, profound changes have occurred in society in relation to energy systems, highlighting the awareness in energy consumption due to the preservation of the environment [1] and the high costs. This has led users to establish load control mechanisms [2] to manage and prioritize its use (management on the demand side) [3], [4] to save energy. But also, a new role in the energy ecosystems has emerged, that of the prosumer, thanks to the incorporation of renewable energy sources in homes, offices, and industries [5]. Likewise, the energy ecosystem has been extended to other The associate editor coordinating the review of this manuscript and approving it for publication was Moussa Ayyash . actors such as electric vehicles (EV), which are capable of producing their own energy from solar panels, or exchanging their surplus when they are parked. All of the above has made energy networks spaces of great intermittence, randomness, and dynamism.
Renewable generation sources such as solar and geothermal energies, among others, take advantage of the potential of natural resources. They become an alternative to avoid or reduce energy from large generation centers (main grid) [6]. The disadvantages of these centers are [7]: high energy losses, high emissions (if based on fossil fuels), excessive use of water (in thermal power plants), hazardous waste generation (if based on nuclear power), expensive infrastructure, fast depletion of fossil fuel, air pollution, and many others. However, the variability of primary sources must be addressed for the renewable generation systems, such as changes in wind speed, daytime cloud cover, or tidal intensity that affect energy conversion [8]. Then, for this, additional components such as energy storage systems (ESS) must be used, which mitigate these fluctuations. But that is not enough, the interconnection of micro-grids (MGs) is necessary to make the system robust, which requires effective coordination mechanisms between the components. The MG control problem in a smart grid is a distributed control problem, mainly oriented to the coordination of its components without a central control that governs them.
The motivation of this work is to adequately integrate and coordinate the various actors in an energy ecosystem, both at the generation and demand level, to meet energy needs, maximizing the use of renewable energy sources, trading with neighboring networks, managing controllable loads according to prioritization levels, charging storage devices only with renewable energy sources, and reducing the energy coming from the main grid. To achieve the above, new distributed strategies are required, such as the one based on emergent control. For example, some bioinspired techniques/models could be used, such as the response threshold model, which is based on the behaviour of ants to respond to environmental stimuli, achieving self-control actions through the use of local variables in a distributed context. The objective of this work is to take advantage of the self-control mechanism of the response threshold model, so that each component of a MG participates in the energy exchange when the conditions are more favorable [38]. For this, it is necessary to identify the local variables and rules for the decision-making, as well as the feedback mechanisms between them, to formulate an emerging control proposal that allows efficient energy management. This work differs from previous ones in the use of a bioinspired emergent control strategy that allows coordination at the micro level. The main contributions of our work are: -The definition of an approach for the coordination and distributed control problems. The original RTM was used to solve the task assignment problem. -The definition of a procedure to deploy various distributed energy resources (DER) in a MG through the use of the response threshold model. -The definition of a distributed mechanism to prioritize the use of renewable sources through the formulation of local decision rules. This article is organized as follows: Section I presents the related works; Section II describes the theoretical framework around the proposed emerging technique and the process to be controlled. Section III presents our bioinspired emergent control strategy based on the response threshold model. Section IV describes the experiments carried out, the discussion of the results, and the comparison with other works. Finally, the last section presents the conclusions and future works.

II. RELATED WORK
From the first works in the multi-MGs (MMGs) concept, Nikmehr and Najafi Ravadanegh [9] discussed the distribution network operation. The economic operation of MMGs is formulated as an optimization problem, with a stochastic modelled of both small-scale energy resources (SSERs) and load demand at each MG. Bandeiras et al. [10] addressed energy trading among multiple MGs for different energy markets to facilitate energy trading among prosumers, producers and customers. Finally, a case study is presented to evaluate the cooperation among five industrial MGs. Yassine et al. [11] proposed some distributed control schemes for MG control.
Abhishek et al. [12] presented a review of hierarchical control strategies that provide control for a MG. Also, the advantages and limitations of these control strategies are discussed in this study. In [13], Ahmethodzic et al. presented a literature review of MG control architectures. Likewise, they describe their main features of them and their positive and negative sides. Sen and Kumar [14] provided a comprehensive survey of different control aspects for MGs, with a classification based on four control strategies: centralized, decentralized, distributed and hierarchical. Also, the principles behind, their applicability and performances are presented.
Palensky and Dietrich [16] proposed a load planning approach dependent on gauge power costs and pre-booked burdens. Aguilar et al. [15] set a scheduling system that automatically generated hours of use of the controllable load appliances in a home, in such a way that the use of renewable energy is maximized. To achieve this, they define an autonomous cycle of data analysis tasks composed of three tasks, two tasks for estimating the renewable energy produced (supply) and the required load (demand), coupled with a scheduling task to generate the plans of use of appliances.
Touma et al. [17] reviewed demand-side management systems, particularly pricing techniques, as a part of a control system of MGs. They identified shortcomings in current researches concerning demand-side management and highlighted future research horizons. In the work of Abou El-Ela et al. [18], a fuzzy logic controller (FLC) system is proposed to decide the charging/discharging priority level of each EV based on the state-of-charge (SOC) level of its battery. Mathur et al. [19] presented different technologies available for EV integration in the MG community. This work also discusses the necessity of vehicle charging stations to meet the technical requirements of the EV.
In [38], Alagoz et al. introduce the concept of an emergent controller applied to smart grids, as an intelligent controlcommunication unit (SCCUs), whose function is to switch the end-user mode between consumer or producer. Thus, in their case study, the flow of energy can be controlled in both directions to export or import the energy. Particularly, the authors focused on smart grids, and the presented methodology on energy balance and energy balance-oriented flow mechanisms could be useful in the analysis of other complex networks involving uncertainty and variability, such as biological or social networks. However, the work does not present the possible intelligent algorithms that can be applied to an SCCU.
Finally, Aguilar et al. [20] presented a systematic review of the literature on recent research about energy management systems for smart buildings using artificial intelligence techniques. They focused on autonomous management systems and identified that many types of research are in the domain of decision-making (a large majority on optimization and control tasks), and defined potential projects related to the development of autonomous cycles of data analysis tasks [21], feature engineering [22], or multi-agent systems [23], among others. As can be seen from recent previous works, there are no proposals linked to using distributed control approaches based on bio-inspired techniques, which allow the emergence of coordination processes between the components of a smart grid. In that sense, this proposal focuses on this field of study.

III. THEORETICAL FRAMEWORK
This section presents the bioinspired algorithm used in this work, the response threshold model, as well as the components of the smart grid that will be modeled.

A. RESPONSE THRESHOLD MODEL (RTM)
This model emulates the reaction of ants to the pheromone intensity (external/internal stimuli) according to the behavior of the division of labor in ant colonies [24]. In this way, the colonies can adapt to changing environments. For instance, ants can have different behaviors depending on their distance from the nest and their abilities for a given task [25]. The reaction to external/internal stimuli can be modeled using a response threshold (θ). Thus, an ant with a low response threshold can be a worker with high probability; and with a high response threshold, it is not a worker, even if external stimuli are high [24]. In a classical RTM, the probability q for an agent works is defined by: where θ ij denotes the response threshold of the ant i to perform task j at time t, and s j is the external/internal stimulus. On the other hand, the ants modify the intensity of the stimulus (s j ) according to equation (2), as a way of exerting control over the system through an individual or collective learning process, by linking rewards to stimuli [15].
In a classical RTM, the intensity of the stimulus depends on the execution of the task. N act is the number of active individuals, N is the number of individuals in the colony, α is a scale efficiency factor to perform the task, and δ is to increase the intensity of the stimulus per unit of time [17]. Also, the response threshold increases when the corresponding task is not performed and decreases when it is performed [24] according to the next equation: where, y ij is the fraction of individuals of type i doing task j, β is the learning rate, and γ is the forgetting rate. Equation (3) specifies that in the next t time units, y ij individuals of type i will do task j, and the rest (1 -y ij ) something else or nothing [27], [28]. In this way, the RTM models the possibility of reacting to respond to the problem of division of labor in a colony. That combined reinforcement process generates specialized workers in specific tasks.

B. MICROGRIDS (MGs)
A Smart Grid is an energy network that integrates the behaviour and actions of all users connected to it (consumers, providers and prosumers) to ensure an efficient sustainable energy system. A smart grid requires innovative services and products for smart self-monitoring, self-control, and selfhealing processes. A MG is a local energy grid with specific elements, acting as a single and controllable entity. It operates in a smart grid but can also operate in island mode. This is the basic element of a smart grid to model. MGs can sell excess power (prosumers) to other grids. In a MG with a non-renewable generation, some of the possible components are diesel engines (DEs), microturbines (MTs), fuel cells (FCs), and combined heat and power (CHP) plants, and a MG with renewable generation includes solar photovoltaic panels (PV) and wind turbines (WTs), among other elements [9]. In this work, we are interested in PV, WT, ESS, EV, external sources like utility grids, synchronous generators (SG) as hydroelectric sources, and demand side control (controllable and uncontrollable loads).

1) PHOTOVOLTAIC SYSTEM (PV)
It is defined as a set of cells connected to provide voltage/ current using a photovoltaic panel. The power output of the PV (P PV ) depends on solar radiation (see Eq. 4). Therefore, a control strategy is necessary to obtain maximum power production [7]. The maximum energy output of a PV is: where: S is the area of the PV array (m 2 ), φ is the Solar radiation (kW/m 2 ), T a is the ambient temperature (e.g., in • C) and η is the conversion efficiency of the PV system.

2) WIND TURBINES (WT)
The production of energy of a wind turbine generator depends on wind speed V W . The wind speed is considered to be the algebraic sum of base wind speed (V WB ), gust wind speed (V WG ), ramp wind speed (V WR ) and noise wind speed (V WN ), given by [29]: The mechanical power output of the wind turbine is formulated as [30]: where ρ is the air density (kg/ m 3 ), A r is the swept area of the blade (m 2 ) and C p is the power coefficient which is a function of tip speed ratio (λ) and blade pitch angle (β).

3) ENERGY STORAGE SYSTEM (ESS)
ESS like batteries is one of the most common devices used for saving electrical energy in various applications. Batteries are a backup for WT or PV systems. They are used to store excess energy captured from wind or solar power during windy or sunny days, and also to release stored energy during times of generation shortages. In the field of battery modeling, various models have been proposed [31]. For this work, we use the next modified simple battery model, considering the power [W] instead of voltage [V]: where Vt is the terminal voltage of the battery, I is the discharge current, Rint is the battery terminal resistance, K is a polarization constant, and Voc is the open circuit voltage.

4) GEOTHERMAL ENERGY PLANTS (GEO)
Geothermal energy (GE) is a non-carbon source of renewable energy based on heat flux from the earth's core; a reliable and abundant energy source [32]. In [33], Zarrouk and Moon defined its power generation based on the operating temperature and the produced power. The active power output in a generating unit is given by: where, T in is the inlet temperature of the primary (geothermal) fluid ( • C), and ATP is the available thermal power (kW).

5) PROSUMERS
It is an energy user who generates renewable energy in his/her domestic environment, and either stores the surplus energy for future use or trades it to interested energy customers in a smart grid [34]. In general, the objective of the prosumers is to produce and consume energy, as well as share and redistribute excess energy to other users in the grid.

6) UTILITY GRID (UG)
Main grid or utility grid are equivalent terms, and means a national integrated power delivery system that transmits and delivers energy to consumers at any voltage level. It consists of three main components [35], i) power plants that produce energy from fossil fuels (coal, natural gas, biomass) or non-fossil fuels (wind, solar, nuclear, hydro); ii) transmission lines that carry energy from generating plants to the demand centers; and iii) the transformers that reduce the voltage so that the distribution lines can deliver energy to the final consumer.

IV. DESIGN OF A BIOINSPIRED EMERGENT CONTROL SYSTEM
In this section, we present the components that we will consider in a MG, and its emergent control model. Specifically, the local emergent controllers of each of the agents that make up the MG will be presented. Initially, we describe the process followed for the design of the system.

A. SYSTEM DESIGN PROCESS
In this section, the design of the emergent controller based on the RTM is explained using the methodology MASINA (MultiAgent Systems for INtegrated Automation) [39], [40]. The steps followed in the design are the following: • Step 1: It defines the agents in the distributed system, with their interactions, states, environment, variables, parameters and behaviors. The complexity of the model will depend on the number of agents and their variables.
• Step 2: This step analyzes and preprocesses the variables of each agent to describe the behavior of the agents, and maybe, their relationships. This may involve converting them to the same dimensions or dimensionless, among other things.
• Step 3: In this step is defined the self-control mechanism based on RTM for each agent. The main objective is to determine the stimulus and threshold of each agent, in order to introduce the reinforcement learning process implicit in the RTM. Also, it defines the feedback mechanisms between the agents.
• Step 4: It defines the activation probability of each agent based on the RTM, for which it uses the stimulus and threshold previously defined. It is the rule for the decision-making of each agent on whether to activate or not.
In summary, the first step is to identify the local variables, and then, the feedback mechanisms between them, as well as the rules for the decision-making of each agent. Specifically, its main variables, the stimulus and the threshold, induce each agent to act or not. The stimulus considers the context information and the threshold is related to the achievement of the objectives. Both use a reinforcement learning process as a self-control mechanism to regulate the agent's response, allowing autonomous coordination actions.

B. ARCHITECTURE OF A MG
This section presents a MG composed of WT, EV, buildings, PV, ESS and external generation sources such as utility grids. They are the agents of our distributed system that describes a MG. Their interactions are defined in Figure 1. All of the above describe the aspects considered in step 1 of our design process. In general, MG requires a mechanism to guarantee coordinated control actions in order to satisfy the demand. The general system variables required for self-control by the agents are the following.
The total power (P T ) in the MG is represented by the sum of the renewable and non-renewable generation sources and the various types of prosumers. In addition, the loads (P L ) are of two types: controllable and non-controllable (priority) loads; which allows establishing the power differential to determine the excess or deficit of energy in the MG, expressed through the following equation: where, P T is the generated power (see Eq. 10) by the renewable systems (P H ) like WT, PV, among others (see Eq. 11), energy storage systems (P ESS ), synchronous generators represented by the geothermal plant (P GEO ), non-renewable or large-scale systems such as UG (P UG ), and the various types of prosumers such as EV (P EV ), self-sustaining buildings, exchange components with other MGs (P LINES ).
Also, the loads will be divided into controllable (P LC ) and non-controllable (P LNC ) loads for the purposes of this work (see Eq. 12): Finally, the exchange prosumer with other nodes P LINE is represented by the power differential P of Eq. 9 since it will allow the trading (purchase/sale) of surplus energy with neighbouring MGs. In this work, only one will be considered, but several will be considered in future works such that different negotiation strategies will be necessary, which are beyond the scope of this work. Thus, Eq. 13 represents the surplus energy available in the MG.

C. EMERGENT CONTROL SYSTEM FOR THE MG
In this section, the control model of each agent will be designed, which will be based on RTM, identifying the context variables to update the stimulus and the threshold. Fig. 1 defines the next agents: A PV agent represents the control mechanism to activate or deactivate the energy coming from solar radiation, and the WT agent represents the control mechanism to activate or deactivate the energy from the wind speed. The synchronous generation agent represents a geothermal power plant. The agent of the public network is the one that allows the supply of energy from non-renewable sources such as gas, and oil, among others. They are reserve producers that compensate for the energy demand when storage systems and/or renewable generation fail. The agent of the ESS represents the charge and discharge control processes of the batteries. This agent stores energy during renewable surpluses and releases it during peak load demand. On the other hand, EV agents impose conditions of randomness in demand and production possibilities, sustainable building agents have their own renewable sources and ESSs, and finally, the exchange agent can buy or sell the surpluses in the MG. Below, we present the design of the agents according to the steps described in section IV-A.

1) PHOTOVOLTAIC AGENT
In steps 1 and 2 of the design process, we identify and analyse the variables, context, and dynamics of this agent: • Behaviour: This agent will be activated when its stimulus tends to increase, which occurs when there is solar radiation (P PVmax ), demand (P L ) and there is no energy stored in the batteries (1 − Soc) Q CAP . In addition, it is necessary that θ decreases because the power delivered to the electrical network is not covered by renewable energies (β PV 1 − P H P L > 0 and γ PV P H P L ∼ = 0). Otherwise, the threshold will increase (γ PV P H P L > 0), causing the controller to turn off. Thus, the RTM, in the context of the PV agent,.
• Environment: Solar radiation and climatic conditions. • External variables: Demand and status/capacity of the ESS (battery).
• Internal variables: Conversion efficiency and solar panel area.
In steps 3 and 4, we define the self-control mechanism and the activation probability of this agent based on the RTM. Thus, the RTM, in the context of the PV agent, for an emergent control scheme, is as follows: where, w PV is an attenuation factor to make the variations of the perceived signals less sensitive, P PVmax is the maximum power output of the PV according to solar radiation (see Eq. 4), (1 − Soc) . (state of charge) is the percent of available capacity of a battery (0 < Soc < 1), Q CAP is the maximum energy capacity of the battery [kW], and P PV is the power supplied by the solar panel. Specifically, Eq. 16 is the probability that the PV agent generates energy, Eq.14 is the intensity of the stimulus for the same agent, and its response threshold is updated according to Eq. 15.

2) WIND GENERATION AGENT
Steps 1 and 2 of the design process identify and specify the variables, context, and dynamics of this agent: • Behaviour: This agent will be activated when the stimulus tends to increase, which occurs when there is wind speed (P Wmax ), demand (P L ) and there is no energy stored in the batteries (1 − Soc) Q CAP . In addition, it is necessary that θ decreases because the power delivered to the electrical network is no covered by renewable energies (β WT 1 − P H P L > 0 and γ WT P H P L ∼ = 0). Otherwise, the threshold will increase (γ WT P H P L > 0), causing the controller to turn off.
• Environment: wind speed and climatic conditions.
• External variables: Demand and status/capacity of the ESS (battery).
• Internal variables or parameters: Conversion efficiency, air density and swept area of the blade. Steps 3 and 4 define the self-control mechanism and the activation probability of this agent based on the RTM. Thus, the RTM in the context of the WT agent, for an emergent control scheme, is as follows: where, w WT is an attenuation factor to make the variations of the perceived signals less sensitive, P WTmax is the maximum power output of the wind turbine that varies according to wind speed (see Eq. 6), and P WT is the power supplied by the wind turbine to the electrical grid according to demand. Specifically, Eq. 19 is the probability that the WT agent generates energy, Eq.17 is the intensity of the stimulus for the same agent, and its response threshold is updated according to Eq. 18.

3) ENERGY STORAGE AGENT
The RTM applied to the energy storage agent is presented in this section. This emergent control system must manage three states: as consumer, producer or passive. The equations of this emergent control system for the 3 previous states are as follows: Producer: Next, we present steps 1 and 2 of the design process in this agent as a producer: • Behaviour: It is a producer when the solar or wind potential is not enough to satisfy the demand and it has stored energy.
• Environment: Does not apply • External variables: Demand and renewable sources. • Internal variables: SOC. Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent as a producer. Eq. 20 describes the stimulus for production when the demand (P L ) is not covered by renewable energies (P H ) and there is stored energy (Soc (P L − P H ) >0). On the other hand, the threshold (Eq. 21) is modified according to the energy stored (SOC) such that the less energy stored (SOC ∼ = 0) its threshold increases to prevent it from being activated. Finally, Eq. 22 is the probability that the energy storage agent gives energy (step 4).
Consumer: Next, we present steps 1 and 2 of the design process in this agent as a consumer: • Behaviour: It is a consumer when there is a surplus of renewable energy and it does not have enough stored energy.
• Environment: Does not apply • External variables: Demand and renewable sources.
• Internal variables: SOC. Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent as a consumer. Eq. 23 describes that the consumption state occurs when the batteries are discharged (Soc ∼ = 0) and there is surplus energy from the renewable system. Also, the threshold (Eq. 24) is adjusted when the batteries are discharged (SOC ∼ = 0) such that the less energy stored its threshold decreases to activate it. Finally, Eq. 25 is the probability that the energy storage agent consumes energy (step 4).
Passive: q BAT ,P is the probability of the passive state based on the previous probabilities.

4) PROSUMER AGENT
This section presents the RTM applied to the exchange agent or the MG as a prosumer, which is responsible for establishing trade with adjacent MGs. This emergent control system must handle three states, such as buyer (consumer), seller (producer), or passive. The agent monitors the surplus or deficit in the MG to control the exchange of energy, taking into account as surplus the generation of renewable energy sources such as solar (Eq. 4), wind (Eq. 6), geothermal (Eq. 8), etc., and the total consumption of the MG (see Eq. 12). Specifically, P can be redefined as: The equations of this emergent control system are: Producer: Next, we present steps 1 and 2 of the design process in this agent as a producer: • Behaviour: This agent will be activated when there is a surplus the generation of renewable energy sources that can sell.
• Environment: Does not apply • External variables: Local and adjacent MG. • Internal variables: Does not apply. Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent as a producer. Eq. 28 represents the stimulus that will increase when the surplus of the local MG is greater than the neighboring MG ( P local − P neighbor >0). On the other hand, the threshold decreases when the locally generated power can satisfy the demand of the neighboring MG ( P local P L,neighbor > 0) (see Eq. 29). Finally, Eq. 30 is the probability that the MG gives (buys) energy to its neighbors (step 4). s LINE,P (t + 1) = s LINE,P (t)+w LINE,P P local − P neighbor (28) θ LINE,P (t + 1) = θ LINE,P (t)−β LINE,P P local P L,neighbor t Consumer: Next, we present steps 1 and 2 of the design process in this agent as a consumer: • Behaviour: This agent will be activated when the MG needs to buy energy (when the solar or wind potential is not enough to satisfy the demand and it has not stored energy).
• Environment: Does not apply • External variables: Local and adjacent MG.
• Internal variables or parameters: Does not apply. Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent as a consumer. Eq. 31 represents the stimulus that will increase when the surplus of the neighboring MG is greater than the local MG ( P neighbor − P local > 0). On the other hand, the threshold is decreased when the power generated by the neighboring MG can satisfy the demand of the local MG ( P neighbor P L,local > 0) (see Eq. 32). Finally, Eq. 33 is the probability that the MG consumes (sells) energy from its neighbors (step 4). s LINE,P (t + 1) = s LINE,P (t)+w LINE,P P neighbor − P local (31) θ LINE,P (t + 1) = θ LINE,P (t)−β LINE,P P neighbor P L,local t + γ LINE,P 1 − P neighbor P L,local t (32)

VOLUME 11, 2023
Passive: Eq. 34 represents the probability of the passive state.

5) UTILITY GRID AGENT
These energy sources, generally from fossil fuels, are used when there are no other options and they must be paid at the market price. In steps 1 and 2 of the design process, we identify and analyse the variables, context, and dynamics of this agent.
• Behaviour: It is activated when renewable sources do not meet the demand.
• Environment: Does not apply • External variables: Demand and renewable sources.
• Internal variables or parameters: Does not apply. Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent. The RTM applied for the utility grid agent is described in Eq. (35). The agent is stimulated when the demand P L is greater than the power of renewable sources (P H ), energy storage system (P ESS ) and synchronous generators (P SG ) such as geothermal plants ([P L − (P H + P ESS + P SG )] >0). Also, the threshold is decreased when the ratio between power from demand and sources is not sufficient P L P H +P ESS +P SG > 0 (see Eq. (36)). Finally, q UG (t) is the probability of activating or not activating the switch (See Eq. 37) (step 4).

6) GEOTHERMAL GENERATION AGENT
Steps 1 and 2 of the design process for this agent define the next information: • Behaviour: This agent will be activated when its stimulus tends to increase, which occurs when there is inlet temperature of the primary (geothermal) fluid (T in ), demand (P L ), and there is no energy stored in the batteries (1 − Soc) .
• Environment: Inlet temperature of the primary (geothermal) fluid (Tin) • External variables: Demand and status/capacity of the ESS (battery).
• Internal variables or parameters: Available thermal power (kW). Steps 3 and 4 define the self-control mechanism and the activation probability based on the RTM of this agent. The RTM applied to the GEO agent is described in Eqs. (38-40).
The agent is stimulated according to the available thermal potential (ATP) (Eq. 38). On the other hand, the threshold is the ratio between the real power delivered by the synchronous generator of the geothermal plant and the total load.

V. SIMULATION STUDY
In these sections are presented the case studies to be analyzed, the metrics used to evaluate the quality of our proposal, a detailed discussion of the results obtained in each case study, and finally, a general analysis of the results.

A. EXPERIMENTAL PROTOCOL
To validate our approach, we address some case studies, each with different interesting properties. The experiments recreate different operational scenarios, such as variations in the dynamics of the renewable generation systems due to weather conditions, battery charging/discharging processes, energy trade, the effect of EV uncontrolled load variations, and compensation of the main grid in the event of deficiencies in the other components of the MG. The power units will be expressed in watts [W]. The quality of the results obtained is evaluated using the next performance criteria [36]: • Integral square error (ISE): the errors are penalized with large values. A control error normally occurs after a disturbance and can be observed as an overshoot. Thus, this index indicates overshoot and aggressive control.
• Integral Absolute Error (IAE): it does not distinguish between positive and negative contributions to the error. It is often used for on-line controller tuning. This index is appropriate for non-monotonic step responses and all kinds of normal operations (see Eq. (42)).
• Mean relative absolute error (MrAE): This index of error is useful in order to show the correlation between the bias of a model and a parameter that can be an input or another variable (see Eq. (43)) [37].
On the other hand, an objective function J is used to determine the optimal hyperparameters of our model (see Eq. (44)), in order to achieve an error close to zero. Particularly, it was used the search grid algorithm to adjust the parameters of our model.
Thus, the metrics presented above are used to determine the quality of our approach in the case studies, due to the absence of similar works. Also, they are used during the tuning process of our model (the optimization of its hyperparameters) in each case study. The calibration procedure allows the estimation of the values for β and γ (the adjustment is made), in such a way that the objective function of Eq. 44 is minimized.

B. TESTS AND DISCUSSIONS
In this section, some case studies are presented, in order to show the versatility of our approach. Table 1 summarizes the parameters obtained from the hyperparameters optimization process.

1) CASE STUDY 1: HIGH ATP (AVAILABLE GEOTHERMAL POTENTIAL)
In this case study, presented in Fig. 2, the scenario of a MG whose main characteristic is the capacity for interconnection to the public grid and synchronous generators (GEO agent) will be analyzed, with components as an ESS, and a connection line with other MGs. Each of these components has a discrete control agent, which allows them to be coordinated to meet the demand of the MG and the load of the ESS. In addition, the purchase/sale of energy to its neighboring MGs is prioritized before the main energy network due to cost differences. In this case, the control of normal loads will not be considered, it will only be assumed that the total demand of the MG must be satisfied through the available energy sources. For this, the geothermal potential is high, with sufficient capacity to supply the local load and sell to its neighboring's, and there is no contribution from wind or solar sources. The tests will be carried out in a period of 20 continuous days.
In Fig. 3 can initially be observed (time 0) that the geothermal generation is high (see Fig. 3.A) and with enough capacity to satisfy the demand (see Fig. 3.E, is 600 MW), so the ESS agent does not come into operation (see Fig. 3.B). When the power is negative, it means that it is sending or transferring the energy of the current MG to another, motivated by the fact that the exchange agent detected excess in the current MG and deficit in the neighboring MG. On the other hand, the UG agent does not come into operation (see Fig. 3.D) because the generation of the GEO agent is enough. At the time 1, a fault occurs in the GEO agent (geothermal power plant, see Fig. 3.A), which forces the supply to be compensated with the activation of the ESS, which is supposed to be loaded (see Fig. 3.B) and can satisfy the local MG. It should be noted that the behavior of interest is the dotted line represented by SOC in Fig. 3.B, which means the percentage of battery charge, while the continuous line shows positive values when it produces and negative values when it consumes.
At time 2, the fault in the GEO agent persists (see Fig. 3.A), the ESS is discharged (see Fig. 3.B) and there is no demand for energy with the neighboring MG (see Fig. 3.C), so the UG agent comes into operation (see Fig. 3.D), satisfying the local demand (see Fig. 3.E) and guaranteeing the balance of energy in the MG (see Fig. 3.F). Thus, when the EES agent load decreases (time 2), then the demand is covered by the UG agent. At time 3, the GEO agent is restored (see Fig. 3.A), so the supply of the UG agent is interrupted (see Fig. 3.D), and the local demand is satisfied (see Fig. 3.E). Thus, when the GEO agent is reactivated (time 3), then the MG load is covered. An important aspect to highlight is that the demand of the neighboring MG cannot be satisfied (see the interval between times 0 and 1, and time 3 in Fig 3.C).
Finally, Fig. 4.F shows that the balance of supply and consumption is around 0, reflected in the IAE of 62.13, which is a very low value for the period of study. On the other hand, the ISE of 2.53 × 10 4 is high due to the peaks that occur in the transitions to achieve the coordination actions of the MG agents, in order to work together and in a distributed way to satisfy the MG demand no matter if failures occur or not (failure/restoration of the GEO agent), or discharge of the ESS agent.

2) CASE STUDY 2: SELF-SUSTAINING BUILDING AS A PROSUMER
In this case, a self-sustaining building is analyzed, which has an array of solar panels on the roof, capable of supplying the entire building when conditions are favorable (see Fig. 4). Also, the MG has an ESS, an interconnection equipment to the neighboring MG but without connection to the utility grid, such that it has to buy or sell when necessary. On the other hand, a study period of 20 days is established with favorable weather conditions with clear skies.
Time 0 is mainly characterized by the deficit in the neighbouring MG, expressed by the negative magnitude of the brown dotted line (see Fig. 5.D). Also, the producers are the solar panels during the day (see Fig. 5.A) and the ESS during the night (see Fig. 5.B), in alternation to the solar cycle. In addition, the ESS is fully charged (see Fig. 5.C) thanks to the surplus of solar energy. On the other hand, energy is transferred to the neighboring MG, visualized in the negative values of the continuous blue line, which is maintained during excess solar energy (see Fig. 5.D), guaranteeing a stable and safe supply during the first 10 days, reflected in the demand, which satisfies both the local and remote loads (see Fig. 5.E and 5.D).
From time 1, which is characterized by a decrease in demand (see Fig. 5.E) and the excess in the neighbouring MG, is evident a compensation behaviour in the transitions reflected in the positive peaks (see Fig. 5.D) to purchase energy from the neighboring MG to satisfy the local's deficit during the nights to try to load the EES agent.
Finally, it is observed the differences between production and consumption (see Fig. 5.F) with a greater number and intensity of peaks reflected in an IAE of 4094 and ISE of 1.03 × 10 6 , motivated by the increase in exchanges or transitions between the components such as PV, ESS and exchange agents. However, the coordinated work allows maintaining the satisfaction of the demand.

3) CASE STUDY 3: ELECTRIC VEHICLE (EV) LIKE PROSUMER
In the case study of Fig. 6, the main components of an EV ecosystem are analyzed, such as the panels installed in the vehicle body, an ESS, the loads represented by the engine, and connection ports to the charging stations (exchange agent). For this case, randomness in the load is considered in order to emulate the connection or disconnection of the EV to the charging station. In addition, the days are clear, so the maximum use of solar energy is guaranteed in a period of 12 days, and the role of the charging station is changed in the last days of the study.
At time 0, the demand is covered by the generation systems such as the solar panel and the ESS, to continuously supply, depending on the time of day. However, the ESS agent tends to be completely discharged due to random changes in the load in the vehicle (see Fig. 7.E). On the other hand, when the discharge battery is completed at time 1, power must be purchased or consumed from charging stations to meet demand (see Fig. 7.D).
Finally, the differences between production and consumption are observed (see Fig. 7.F) with a higher frequency and intensity of peaks reflected through an IAE of 1.05 × 10 4 and an ISE of 7.12 × 10 6 , motivated by the behavior observed from the time 0 to 1, when the PV, ESS and prosumer agents are coordinated (possible energy sale because the solar generation potential is sufficient); while, after time 1, it decreases in intensity since the batteries are discharged.

4) CASE STUDY 4: WIND POWER IN A MG
In this case study, the incorporation of a renewable energy component based on the WT agent is analysed (see Fig, 8).
The MG is made up of ESS, wind generators where randomness is introduced in the speed, loads without the possibility of control and energy exchange with a neighbouring MG (see Fig. 9). Thus, it is possible to sell or buy the excess energy, when necessary, based on the premise that it is cheaper to buy the available neighbouring energy, instead of the energy coming from the UG agent. The study is carried out over a time period of 12 days.
Wind speed is a highly random weather variable, so energy production is conditioned by this factor. In Fig. 9.A, the dotted lines represent the wind generation potential and the continuous lines the actual potential delivered. In time 0, the energy only comes from the wind and the participation of the ESS is not necessary (see Fig. 9.B) nor the purchase of energy from neighboring MGs (see Fig. 9.D). In time 1, the wind speed decreases significantly, so the ESS comes into operation, discharging the battery to approximately 75%, due to the increase in load. In time 2, the wind speed remains constant but the load is reduced, which favors a surplus of energy that is used to charge the batteries. However, it is not enough to cover the demand and the charge of the battery, so the purchase of energy from the neighbouring MG is activated, gradually disabling due to the increase in wind generation, reducing again to reactivate the purchase of energy.
In times 3 and 6, the load is increased and the wind energy is reduced, which results in the activation of the ESS while the wind speed improves. When that happens, the ESS agent is deactivated, changing its role to a consumer, loading again. In times 4 and 7, an emergent behaviour occurs, when the purchase of energy is activated instead of activating the batteries that are almost loaded, which is presumed to occur due to the high demand and the low wind generation. In time 5, there is evidence of an increase in wind generation sources relative to the load, which deactivates the batteries and satisfies the demand using only renewable sources. VOLUME 11, 2023  The brown dotted lines represent the deficit or excess in the neighboring bus, that is, when it is negative then it is a deficit, and it is positive when it is excess, and the continuous blue line is the power exchanged between the MGs E) Bus demand. F) Difference between total production and consumption.
Finally, it is observed in the difference between production and consumption that there is a big number and intensity of peaks reflected in an IAE of 3747 and ISE of 5.69 × 10 5 , motivated by the increase in exchanges or transitions between the various components. However, the coordinated work manages the satisfaction of the demand.

5) CASE STUDY 5: SELF-SUSTAINING BUILDING WITH REAL DATASETS
This case study is based on the work of Alam et al. [41], for which uses its datasets of profiles of solar power, and consumption, among others, for a year, divided into average values per week grouped into 4 climatic periods according to the Hindu calendar seasons. Thus, summer is a hot climate from week 1 to 13, monsoon is characterized by abundant rains from week 14 to 26, autumn is a warm and humid period from week 27 to 35, and winter is a period where low temperatures predominate from week 36 to 52. However, for the purposes of our work, we are not considering the entire year but 2 adjacent weeks for each climatic period. Specifically, weeks 6 and 7 for summer, weeks 20 and 21 for monsoon, weeks 32 and 33 for autumn, and weeks 44 and 45 for winter. In this way, the generation and consumption profiles were used in our simulations for these weeks, which will allow a fairly approximate scenario according to the climatic season.
In this case study, an isolated self-sustaining building is analyzed, composed of an array of solar panels on the roof, capable of supplying the entire building when conditions are favorable, and an ESS in the building as a backup but without connection to the public network (see Figure 10).
In summer, it is where solar radiation is best used due to its intensity and duration, observing in figure 11.A that the solar panels satisfy the demand and recharge the batteries (see figure 11.B / 11.C), evidencing a peak due to the transitions of the PV/ESS ( Figure. 11.E). In monsoon and autumn, the capacity of the solar panels is significantly reduced (Figure 11.A), forcing the ESS to meet the demand,  which causes it to discharge more quickly (Figure 11.C). This shows that this isolated topology is neither viable nor recommended for more than half the year, as it limits the recharging capacity of the batteries from renewable energy sources. Finally, in winter, solar radiation increases but not its duration, and the batteries have little time to charge again.
Finally, the differences between production and consumption are observed (see Fig. 11.F) with a greater number and intensity of peaks in spring and winter, reflected in an IAE of 3840 and ISE of 3.63 × 10 6 , motivated by the increase in exchanges or transitions between components such as PV and ESS when radiation allows demand to be met.

6) CASE STUDY 6: WIND POWER IN A MG WITH REAL DATASETS
In this case study are used the same datasets from work [41], used in the previous case study. This case study considers an isolated self-sustaining building made up of a group of turbines, capable of supplying the entire building when conditions are favorable, and an ESS as a backup but without connection to the public network (see Fig. 12).
It is observed that in summer and winter, the wind speed is significant to satisfy the demand, but the same does not happen in monsoon and autumn (Figure 13.A), where the ESS plays a fundamental role to cover the energy needs of customers in the absence of wind energy (Figure 13.B and 13.C), causing them to discharge considerably. Thus, it is not a recommended topology for half the year. The peaks observed in figure 13.E are the result of the transitions between the generation sources. Finally, it is observed in the difference between production and consumption that there is a big number and intensity of peaks reflected in an IAE of 1388 and ISE of 5.7 × 105, motivated by the increase in exchanges or transitions between the various components.  These last scenarios highlight the importance of interconnected MGs communities to give greater robustness and resilience to electrical networks. Table 2 summarizes the results obtained from the previous case studies. IAE measures the error over time and IAE the peaks in magnitude and frequency, which expresses the transitions in the different components or with other MGs. Each case study explores situations that can occur in an energy network such as variability of climatic conditions (e.g., the wind speed), component failures (e.g., in the GEO agent), changes in consumer demand, exchanges of energy with a neighboring MG, randomness in the consumption of EVs, among others.

C. DISCUSSION OF THE RESULTS
When observing the MrAE metric, it is evident that most of the time, the 6 cases were close to the real value, which in this case refers to the demand, where the peaks caused the differences in most cases.
When analyzing the values obtained, we can conclude that in all the case studies transitions occur according to the environmental conditions, and demand change, among other  reasons, which generates changes in the magnitude and frequency of peaks. The minimum occurs when the generation source is the GEO agent, and the maximum is due to EV agent, which indicates that the main source of production influences the stability of the network. Sources such as wind and solar energy have relatively similar values, showing a lower degree in the WT agent due to its availability at any time (day and night), unlike solar energy, which is only available during the day. In general, our bioinspired emergent control system allows perfect coordination in the different case studies, always covering the demand using different strategies depending on the context.

VI. CONCLUSION
This study proposes a bioinspired emergent control architecture based on RTM. In our work, a MG composed of a set of heterogeneous agents is perfectly coordinated by our strategy, in order to meet consumer demand. It is highlighted the diversity and quantity of components that exist both on the supply side and on the demand side, increasing the robustness of the system.
The most important characteristic of our approach is its robustness, given that, in the event of failure of any of the components, or changes in the climatic conditions, or changes in demand, the agents adapt to respond to unforeseen conditions, and thus guarantee the achievement of global objectives (the demand). Another contribution of the work was the incorporation of exchange agents to control the flow of energy between neighbouring MGs (sell/purchase of energy). The results obtained show its potential to solve some of the challenges around the energy ecosystems that currently exist: i) Local marketing of energy surpluses; ii) managing of the unpredictability of EVs; iii) distributed coordination based only on local information of the environment; iv) very fast transitions, v) prioritization of renewable supply sources.
From the results obtained, it can be concluded that the bioinspired emergent control approach can be adapted to various contexts, to achieve coordinated and distributed control actions between components. In addition, it is evident that the main source of production influences the stability of the network. The work developed is limited solely to studying the power contributions between the various agents of the energy ecosystem, which were modeled in a simple way, without taking into account the electrical complexities of each one such as voltage control, and converters, among others.
In future works, it is proposed to deepen the control on the demand side, through the scheduling of the controllable loads, detailing the equipment contained in a house or apartment by the specification and prioritization of appliances, considering the energy costs in the market, the life patterns of the users, etc. It is also proposed to explore other emerging control models such as deep dynamic reinforcement learning, among others. Although the focus of future work revolves around the problem of distributed control, it could delve into other more complex models of the components, as well as apply MG design methodologies to size the components according to various needs or operational requirements. Finally, future works should study the different business relationships between MG communities, as some may be collaborative, others require negotiation mechanisms, etc.

DISCLAIMER
The content of this publication does not reflect the official opinion of the European Union. Responsibility for the information and views expressed herein lies entirely with the author(s).