A systematic literature review on the use of artificial intelligence in energy self-management in smart buildings

Buildings are one of the main consumers of energy in cities, which is why a lot of research has been generated around this problem. Especially, the buildings energy management systems must improve in the next years. Artificial intelligence techniques are playing and will play a fundamental role in these improvements. This work presents a systematic review of the literature on researches that have been done in recent years to improve energy management systems for smart building using artificial intelligence techniques. An originality of the work is that they are grouped according to the concept of “ Autonomous Cycles of Data Analysis Tasks ”


Introduction
The building sector causes almost 40% of the total non-renewable energy consumption, 40% of greenhouse-gas emissions, and 70% of the electricity use in industrialized countries [1]. Buildings consume more than transportation or industry sectors, and it is imputable mainly to heating/cooling, lighting, and electrical appliances. Additionally, the legislation in these countries stipulates that much of the consumed energy by 2025 must come from renewable and CO2-free energy sources. This transition of the energy systems is leading to a strong investment in smart grid technologies to minimize the overall energy cost. Smart grid technologies can reduce energy consumption, increase the efficiency of the electricity network, and manage electricity generation from renewable technologies. However, the ways to make use of smart grid technologies to improve building energy efficiency are still an open issue.
On the other hand, smart buildings are thought of as a dynamic "living" organism, where technology is used to bring the most of light dimmers and thermostats. In a smart building (academic, commercial, residential, etc.), hundreds of elements must be considered, including the Heating, Ventilation, and Air Conditioning (HVAC) system, plug loads from appliances and Information Technology (IT) devices, etc. One of the main elements of smart buildings is the Building Energy Management Systems (BEMSs), which must seek energy efficiency and its integration with smart grid technologies. BEMS combines strategies to improve energy efficiency and conservation in a building. BEMS must implement key energy management tasks, such as monitoring of energy supply information, automated demand response, detecting of energy use anomalies, supervision of energy costs or automatic control. There are numerous studies about BEMSs and their subsystems. Then again, a set of new concepts are coming up today around smart grid technologies, such as microgrids, demand-side management (DSM), load scheduling strategies, peer-to-peer (P2P) electricity trading, energy storage services, energy hub, energy prosumers, renewable energy resources (RES), etc. that make the functionality of the BEMS more complex. In this new context, the energy is intermittent, distributed, mobile and can be stored. For example, the growing trend of RES, characterized by its variability and intermittency, reduces the prediction capability of the generated energy. These attributes make more challenging a BEMS because more flexibility and stability are needed to secure the normal operation in a building.
BEMSs today do not implement data monitoring, processing, analyzing and controlling capabilities for this highly complex and changing scenario. Some aspects missed include, but are not limited to, adaptability, predictive modeling, multisensor fusion, dynamic optimization or context-awareness. Thus, the Artificial Intelligence (AI) field brings new approaches for the development of smart BEMSs to build useful knowledge like occupancy behavior, fault or weather prediction, energy usage patterns, among others, to address occupant comfort while maximizing energy efficiency.
AI techniques can be used for different tasks in a smart BEMS. For example, they can be used for modeling (e.g., with multi-agent systems), learning (e.g., with machine learning (ML) approaches), reasoning (e.g., with fuzzy systems), among others, which can be embedded in these environments. The motivation of this work is to know what has been done around this topic in the literature, organizing it from a holistic point of view from autonomic computing, to achieve efficient energy management. The main motivation of this paper is to provide a novel framework about the utilization of AI techniques for energy management in smart buildings, considering a recent AI-based concept like autonomous computation, to organize the information. The next two subsections of the introduction describe works similar to ours on systematic literature review and present the contributions of this work.

Previous review
In the literature, some reviews are close to this work, linked to building energy optimization, smart grid, among other topics, but neither are recent, nor smart buildings specific, nor organized in the way proposed in this work, which is based on the concept of "Autonomous Cycles of Data Analysis Task", also called ACODAT [2,3] (see section 2). Next, we comment on some of those works.
In the context of Monitoring problems, Plageras et al. [4] published a survey on the Internet of Things (IoT), Big Data, Cloud Computing, and other topics in the field of sensor data collection & management in smart buildings. Zhou et al. [5] present a study of Big Data-driven smart energy management. They first discuss the sources and characteristics of big data from energy, and then, take the smart grid as their research background, providing a systematic review of Big Data analytics for smart energy management. Atnonopoulus et al. [6] provide an overview of AI-based EMS works on methods utilized for energy demand-side response (DR) applications. They classify the papers with regards to both AI/ML algorithms used and the energy application area. Kumari et al. [7] present a survey on blockchain (BC) and AI-based energy management systems. They review several existing AI algorithms in P2P energy trading, integrating BC and AI in the EMS. Particularly, they analyze works where AI-based techniques support various services, such as energy load prediction, classification of the consumer, where the BC provides data immutability and a trust mechanism for secure energy management. Molina-Solana et al. [8] review how data science is used in the most difficult problems in the field of energy management for smart buildings.
In Fault Detection and Diagnosis studies, Verma et al. [9] discuss the state-of-the-art in intelligent features and IoT infrastructure required for smart building. They focus on papers about virtual sensing IoT infrastructures, which enable the clients to use in detection tasks. Lazarova et al. [10] review the methods that can be utilized for the discovery and diagnosis of faults in buildings, in order to identify the existing gaps. In addition, they review and analyze the types of faults that could occur. With respect to Intelligent Control systems, Hameed et al. [11] present a state-of-the-art in intelligent control systems for energy and comfort management in smart buildings. They consider different aspects, like control systems, intelligent computational methods, and comfort parameters, among other aspects. The survey presented by Schmidt et al. [12] presents several recent works about predictive control strategies for the daily operation of buildings. Chinchero et al. [13] present a review about control methodologies for BEMS, specifically studying the impact of LED Lighting Systems in smart buildings.
In the Scheduling problems, Sadeghi et al. [14] present a state-of-the-art of existing research works focusing on the planning problems of energy systems in energy hubs. Finally, about EMSs, the review presented by Silva, Khan & Han [15] analyses recent literature reports on peak load shaving and demand response for EMS, while Hernández et al. [16] present a review of management strategies for BEMSs for improving energy efficiency. They review the existing studies for building types, building subsystems, and used techniques. Finally, the work of Himeur et al. [17] presents a survey about energy efficiency recommendation systems in buildings, and a taxonomy of these systems based on the nature of the recommender engine, its goal, the computing platforms, the evaluation metrics and incentive measures. Fig. 1 summarizes the areas covered in the articles that present reviews of the literature in the field of EMSs for smart building. Most of the works make reviews of problems related to decision-making tasks (e.g., scheduling, control, optimization), and in particular, of control problems [11][12][13]. Also, in the analysis tasks, several reviews have been made on the use of AI techniques in them [5,7,8], as well as general reviews on what has been done technologically on EMSs [15][16][17]. On the other hand, in the case of decision-making tasks, there are reviews for specific problems (e.g., control problems), but also, there are reviews related to the approach used to solve these problems (e.g., multi-agent systems [13]). Finally, some reviews review specific problems but from a particular resolution approach, as is the case of the work of Chinchero and Alonso [13].
In conclusion, Fig. 1 shows that some works are very oriented to specific problems (e.g., control, diagnosis), while others consider specific components of an EMS (e.g., LED lighting systems). Additionally, those that refer to the use of AI in EMSs, besides being for specific problems, do not propose a methodology to analyze previous works in terms of their contributions in monitoring, analysis, or decision-making processes. These areas are covered in this review. This is the main contribution of this article, which allows identifying the potentiality of the autonomous cycle concept in EMS. It is also important to highlight, there are no works that review previous works in reinforcement learning (RL) for EMSs or autonomous EMSs, also covered by this revision. Thus, the review of the literature carried out by this article is made from a holistic vision considered in the designs of autonomous cycles integrating AI techniques.

Scope and contributions
The present survey paper aims to put together different approaches, strategies and procedures about how and when to use AI techniques for energy management in smart buildings. Particularly, this paper is based on the ACODAT concept to organize the works derived from the Systematic Literature Review. It is defined as a set of data analysis tasks that improve the process under study. These tasks interact with each other, and have different roles: (1) observe the process, (2) analyze and interpret what it is happening, and (3) make the decisions to improve it. The integration of tasks in a cycle, seen as a closed-loop, allows the autonomous resolution of system problems. The models stipulated for these tasks are based on AI techniques. Thus, the aim of this study is to provide a systematic review of the various AI approaches for BEMSs applications. The goal of our review is triple: -First, to provide a comprehensive overview of the AI techniques used in BEMS problems, as well as the main specific applications to which these techniques have been applied. -Second, to propose a useful guide about how AI techniques have been used for monitoring, analyzing, and making decisions in an energybuilding ecosystem. The work is organized from the standpoint of autonomic computation. -Finally, to present a systematic discussion about perspectives and future research paths from the point of view of the ACODAT concept.
Thus, in addition to the definition of potential tasks for some of the phases required by autonomic computation (monitoring, analysis, etc.), preliminary autonomous cycles are also proposed.
The remainder of this work is organized as follows. A preliminary background about BEMSs, smart grid technologies and the ACODAT concepts are described in Section 2. The Systematic Literature Review methodology is formulated in Section 3. The analysis of the literature found, organized according to the ACODAT concept, is reported in Section 4. Finally, a discussion about future directions in this domain is pointed out in Section 5.

Background
In this section, we provide the background for the Literature Review, in particular, we describe ACODAT, and smart buildings and energy terminology.

Autonomic cycles of data analysis tasks
The main objective of a data analysis task is to extract useful knowledge from data to allow decision-making based on it. In general, an ACODAT architecture can generate different models like descriptive, identification, and predictive models, among others, in order to guide the decision-making processes in a system. Thus, an ACODAT is a set of data analysis tasks that acts together, in order to achieve an objective in the process that they supervise [1,2]. The tasks have different roles in the cycle and interact with each other. The roles are observing the process, analyzing and interpreting what happens in it, for making decisions to reach the objective for which the autonomic cycle was designed. The integration of data analysis tasks in a closed-loop allows solving complex problems. The roles of the data analysis tasks are: • Monitoring: These tasks allow observing the supervised system. They must capture data and information about the behavior of the system from different sources, and prepare the data for the next steps: preprocessing, feature engineering, etc. • Analysis: These tasks must enable understand and interpret what is happening in the supervised system. Thus, during these tasks are defined knowledge models to understand the system. • Decision-making: These tasks define and implement the necessary actions based on the previous analysis, oriented to improve the supervised system. Once these tasks are completed, the process restarts with the monitoring and analysis tasks.
The concept of ACODAT has been used in different domains. For instance, it has been used in Smart Classrooms [1,2], Smart cities [18], and industry 4.0 [19], among others. On the other hand, MIDANO (Methodology for the Development of Data Mining Application) is a methodology that guides the design and development of ACODAT architectures, and consists of three phases [20]: • Phase 1: This phase has the main goal of knowing the organization (e. g., mission, its strategic objectives, processes, members, etc.). Additionally, this phase specifies the ACODAT for the problem to solve. • Phase 2: This phase prepares the data for the data analytics tasks.
Thus, it defines the extraction and transformation operations of the data. It includes the definition of the Minable View (MV) with the useful variables for the autonomic cycles, and feature-engineering tasks for feature extraction, feature reduction, and feature selection. • Phase 3: This phase consists of the implementation of all the data analytics tasks of the autonomic cycle. Each task creates a knowledge model, for example, predictive or descriptive models. This phase also includes the implementation of the autonomic cycle.

Smart buildings and energy aspects
In general, energy consumption today is a global problem for society and the environment, as most of the energy is produced with combustion, requiring nonrecyclable materials and producing non desired greenhouse gas (GHG) emissions to the atmosphere. The problem brings attention from many initiatives globally, such as "Go green" [21], recommendations like Low Carbon Transition Program in China [22] and Nearly Zero Building Strategy 2020 in Europe [23], and among others. Energy efficiency and conservation is required to sustain this trend and minimize the harmful effects. Another challenge is the expansion of the alternative sources of energy, like photovoltaic (PV) cells, wind turbine farms, geothermal installations, or tidal turbines, which require consolidating the coexistence with the traditional sources, like oil, coal, nuclear or hydraulic. Economic reasons also push agents to find ways for reducing the costs associated with consumption. For many years, energy strategies applied to the consumer side have been based on the automation and optimization of control, such as the incorporation of Building Automation Systems (BAS) or Home Automation. Technology has provided a means to build a consistent concept to research and develop new lines to improve energy efficiency and reduce operating costs.
Smart Energy is an interesting approach since addresses the problems caused by power generation: (1) Green Energy deals with the protection of the environment; (2) Sustainable Energy is concerned with exhausting nonrecoverable materials; and (3) Renewable Energy seeks the interoperation of the new sources of energy with the existing suppliers. The Smart Energy research requires a delve knowledge in the fields of energy and Information and Communications Technologies (ICTs) to seize the wide opportunities brought by the digital transformation. In ICTs, the advances on the Internet, Ubiquitous Computing (UC), Big Data, Wireless Sensor Network (WSN), Service-Oriented Architecture (SOA) and microservices, allows the integration of new functionality in the EMS (BEMS, Home EMS (HEMS), etc.). The AI plays an important role in smart energy, approaching the different types of tasks: Monitoring, Analysis and Decision-making. The smart energy goals require to be counterbalanced with the improvement of the quality of life (QoL) and the quality of the services (QoS), thus being approached with a "smart environment". Commonly studied "smart scenarios" are the "smart building", "smart home", "smart health", and "smart city", which could be considered either as one of these scenarios or as a cohesive set for them.
Building systems consume 32% of the produced electricity [1], leading to strategies for optimization and adaptation to changes. The HVAC consumes above 33% of the electricity, lighting, 17,1% and IT equipment, 13.6% [1]. The Smart Building improves the quality of living, like users' comfort, their security or the owner's cost-effectiveness, by appropriate building and system designs and operations. The context information, such as weather forecasts, users' behavior, or venue scheduling, enriches the accuracy and allows the implementation of automatic negotiation of energy tariffs public auctions. Comfort, energy and performance are opposite objectives that require optimization. Works on mid and long-term changes perception are required. In the seventies, Home Automation was conceived for remotely monitoring and controlling lighting, air conditioning, heating or appliances at home [1]. The smart home sector consumes 22% of the produced energy and due to inefficiencies 47% out of the total is lost. Today, both small and large residential faces new issues: (1) the heavy loads required for Plug-in Electric Vehicles (PEV); (2) the efficient selection of different traditional sources or buyers, Multi-Energy Systems (MES), and the onsite RES; and (3) the exploitation of huge amount of information that could make the models more precise.
Distributed Energy Resources (DER) are rapidly expanding as users become prosumers, i.e. producers and consumers, capable of generating their own energy with PV cells and others, requiring HEMS or BEMS capable to switch over the optimal provider or the optimal buyer. DER is the proactive use of microgrids, rather than simply emergency systems. Several approaches are under investigation, as the interconnection with P2P Electricity Trading among users or the novel concept of the Energy Hubs, for provisioning turnkey solutions to the users. The emergence of Energy Storage Systems (ESS) improves efficiency but requires additional functionality on the EMSs. One of the most interesting plans in the study on the consumer side is the exploitation of RES for loading EV, avoiding oversizing the grid to supply such loads in the peak hours, causing a significant cost reduction as well.
From the viewpoint of the smart grid, the main drivers come from: (1) the population growth; (2) the new load demands; and (3) energy inefficiencies. The new load demands are characterized by the coexistence of traditional supply with RES and new applications, such as the PEV. The inefficiencies can be seen in today's low Peak to Average Power Ratios (PAPR), which may cause demand-supply imbalance, blackouts and undesirable price variations. The smart grid is thus complex and difficult to design. New elements such as ESS, microgrids, EVs require the addition of capacity, but the onsite generation sources release part of it. Smart meters make the information more accurate, complicating the management but improving the optimization.
Research approaches focus on capacity addition and shifting the load demand out of the peak hours, known as Peak Load Shaving. DSM approaches these problems (1) improving the energy efficiency or its conservation, like the application of the game theory for scheduling; (2) the Demand Response Programs (DRPs) either responsive, by shifting, cutting or curtailing loads or nonresponsive; and (3) with onsite generation and storage backup [15]. DRPs can be achieved via incentives like Direct Local Control (DLC) or demand bidding, or using price-based approaches like Real-Time Pricing (RTP), Time of Use (ToU), Inclined Block Rate (IBR), Critical Peak Pricing (CPP) or Day Ahead Pricing (DAP).

Methodology
In this section, the activities carried out for the revision of the literature are introduced. First, the methodology is defined, then, the research questions and search strategies are presented. Next, inclusion and exclusion criteria, and document selection, are explained. Finally, a summary of the selection process is detailed.
The methodology used in this work is the systematic literature review (SLR), which divides the process into four phases [24]: (i) identification of the need for revision, (ii) definition of a review protocol, (iii) conducting the review, and (iv) analysis of the review. In addition, the organization of the revision of the articles follows the workflow proposed for the ACODAT paradigm, which proposes three types of data analysis tasks (see sections 4 and 5): monitoring tasks, system analysis tasks, and decision-making tasks. Finally, the organization of the reviewed works inside these tasks is divided according to the feature engineering process (in the case of the monitoring tasks), or domain of application of the analysis or decision-making.
The aim of the search process is to identify relevant studies according to research questions. The research questions are based on the objective of the article and are: Q1. What monitoring, analyzing and making-decision machine learning models for Energy Self-Management in Smart Buildings have been developed?

Q2. What Energy Self-Management applications/tools/datasets for Smart Buildings exist?
The rest of this section presents the details of the review protocol.

Search strategy
The following digital libraries have been explored: ScienceDirect, IEEE Xplorer, Google Scholar, Scopus, Elsevier and Springer databases. The inclusion criteria for the selection of publications are listed below: -Articles and book chapters from 2016 in English language, related to the research questions; -Articles and book chapters available in electronic form.
On the other hand, there are two exclusion criteria to discard publications that are not interesting for this work are: -Articles representing the personal opinions of individual experts, -Articles in the form of conference posters, abstracts, short articles and unpublished works, -Articles whose approaches do not apply machine learning methods or techniques.
Also, for the search process, the next group of terms in Table 1 has been defined using the PICOC method [24]. This method is used to describe the five elements in a search question. It is an acronym of Population (Who?), Intervention (What or How?), Comparison (Compared to what?), Outcome (What are you trying to accomplish/improve?) and Context (In what kind of organization/circumstances?). In our case, we have used only Population, Intervention, Outcome and Context (see Table 2).
Even when this work has been iterative, only the final results are presented. The search strings (Boolean research equations) to answer each research question are defined with the previous groups of keywords, as shown in Table 2.

Selection process
The search process follows the four stages proposed by Kitchenham [24]: -Selection by title and keywords: Selection of articles by evaluating their title and keywords, using the search strings.
-Snowballing: inclusion of extra documents, based on checking the references of the previously selected documents. This process can be repeated as many times as new documents are found; however, only the first iteration was applied in this work. -Selection by abstract: Examination of the abstracts of candidate articles from previous stages, using the inclusion and exclusion criteria, to define if they are selected to the next stage. -Selection by full text: Examination of the full text of the candidate papers from the previous stage, using the inclusion and exclusion criteria, for the final selection.

Preliminary analysis
At the end of the search process, 62 articles were selected matching the proposed research questions, as shown in Table 3. Fig. 2 shows the instantiation of the selection process to find the relevant works for this research. Initially, a total of 335 papers were recovered from the scientific libraries. After reviewing the title and keywords, and then removing duplicate elements, 169 papers were selected. In the snowballing stage, 32 articles were added. The review of the abstracts filtered the selection reducing the number to 141 papers.
Finally, after reviewing the full text, 62 articles met all the eligibility criteria.
The obtained articles are then organized according to the ACODAT concept, as depicted in Fig. 3. Fig. 3 shows the Monitoring tasks organized in the next subgroups: data collection, feature engineering (extraction, reduction and selection), and detection and identification.
On the other hand, the Analysis tasks are organized in the next subgroups: prediction, classification, supervision, clustering and diagnostic models, while Decision-Making tasks are organized in control, optimization and scheduling. Finally, there are some articles about   autonomous energy systems reported. According to Fig. 3, the first observation is that most of the articles are classified in optimization (16), control (8) and planning (8). In the field of analysis, the majority is in prediction, but there is not any study using online clustering strategies, for example, in diagnostic tasks. In features engineering, almost everything done is for feature extraction. Therefore, this classification reveals a number of research lines related to applications of specific AI techniques for solving problems in BEMS, such as online clustering and multilabel classification techniques, as well as their integration in autonomous cycles for autonomic energy management.

Analysis of reviewed papers
In this section, the articles are commented and classified according to the ACODAT approach.

Monitoring
Some studies on system monitoring and context data gathering tasks translate the technology achievements from other fields. Articles about Monitoring are divided into some activities required for the preprocessing of the data. The first subgroup links to the problem of data collection in smart buildings, like the smart sensors with smart meters at different levels (appliances, houses, buildings …), or sensor fusion to distinguish occupants' activities. The advances in networked distributed elements and device consumption have led to the Internet of Energy (IoE) [9], which makes use of the Internet to share the information among devices to provide a distributed smart energy infrastructure. The next subgroup is about feature engineering, especially feature estimation, feature reduction or selection and feature extraction. The third subgroup is around the problem of detection or identification of variables/behaviors.
In data collection, Farmani et al. [25] propose a Smart EMS architecture with three modules. The first module, called the data acquisition module, senses different types of data (e.g. weather conditions), and receives the status of units of energy generating/consuming and the control signals. The data fuser module prepares the data, determining the outliers and miss-observed data, in order to replace them using the centroid of each class defined by a K-means clustering method. Finally, it combines several variables to compute new attributes based on their correlation to remove correlated features.
In feature estimation, there are several works applied to the occupation estimation in buildings. Zou et al. [26] propose a mechanism for occupancy detection and crowd-counting using Internet of Things (IoT) devices. Firstly, they design an IoT platform to obtain the channel state information (CSI), and use a wavelet-based denoising scheme to remove the inherent noise of the raw CSI data. Then, they propose a mechanism of occupancy detection based on the signal tendency index (STI) concept, and use a transfer kernel learning (TKL) approach to count the occupant number. For determining the location of the people inside the building in the context of energy management, Borhani et al. [27] design an indoor positioning system composed of the indoor information collection elements, and a radio map with online positioning, using the Wi-Fi fingerprint embedded on smartphones. The indoor positioning system is composed of an offline section for collecting the radio map information. Then, the noise covariance of the received signals is determined by an adaptive Kalman filter. Also, it is composed of an online-offline section where positioning is conducted on a limited number of reference points with the highest clustering.
At the level of feature reduction and selection, Rodriguez-Mier et al. [28] define a knowledge model based on the big data paradigm for the definition of a predictive model for smart buildings of energy consumption. Then they propose a multi-step prediction approach based on a hybrid genetic-fuzzy system, coupled with a feature subset selection method to automatically select the most relevant features in different time steps. Gonzalez-Vidal et al. [29] study multivariate time-dependent series from smart buildings for energy forecasting. Their methodology transforms the time series in a way that standard ML algorithms can process, and applies feature selection methods, such as multivariate and wrapper methods. They use Random Forest (RF), Instance-based Learning and Linear Regression algorithms.
Other works have considered the definition of detection or identification models in the context of smart buildings. For example, Li et al. [30] propose a data mining-based method to identify and interpret the power consumption patterns and their associations. They perform data partitioning and interpretation with two descriptive data mining algorithms: clustering analysis that identifies three distinct patterns in energy consumption: (1) undercharged fault, (2) low and (3) high part-load ratios, and then discover the rules for energy consumption with rule association learning from each pattern. Fahim et al. [31] present a model to find abnormal energy consumption patterns by analyzing the temporal data streams gathered from smart meters by finding the gap between the current usage and the expected consumption, using a support vector regressor (SVR) to learn the energy consumption patterns.
In the work of Förderer et al. [32] the authors describe DERs like surrogate models based on artificial neural networks (ANNs), which can learn a specific DER model with all the relevant constraints. The ANN can be used for different analyses, for example, to discover load profiles. Capozzoli et al. [33] use data from logs in EMSs to describe the energy consumption and anomalous energy patterns in buildings. In this paper, the authors use the Classification and Regression Tree (CART) technique for the detection task. Peña et al. [34] study the energy efficiency anomalies in smart buildings and propose a rule-based system based on the knowledge from energy efficiency experts to detect them. The rule-based system is used as a decision support system to detect anomalies. Fig. 4 shows AI techniques used in the reviewed articles for monitoring tasks. It is observable that there is not any predominant technique over the other.

Analysis
This phase of ACODAT involves tasks that interpret, and understand what is happening in the supervised process. They are required to diagnose, classify, predict or describe what is happening, or discover the pattern that characterizes events, among others. In the analysis tasks, the predominant approach is ML with classical models like Support Vector Machine (SVM), ANN, or their ensembles via Boosting or Bagging approaches, such as RF. Today, the attention of researchers goes to Deep Learning (DL) modeling, suggesting Convolutional Networks (CNN), recurrent Long-Short Term Memory networks (LSTM), or even Generative Adversarial Networks (GAN) to mimic real systems.
In the context of prediction works in smart buildings, Le et al. [35] use the Transfer Learning concept to develop a framework for multiple electric energy consumption forecasting of a smart building. In this framework, they first employ K-means for clustering many profiles of the daily load demand. Then, they train an LSTM model with the cluster-based strategy for MEC forecasting on smart buildings. Hadri et al. [36] develop several energy consumption-forecasting approaches by integrating the occupancy prediction and the context-driven control of building's appliances. They test ARIMA, SARIMA, Extreme Gradient Boosting (XGB), RF, and LSTM. Moreno et al. [37] define predictive models of energy consumption and save energy for buildings based on the Radial Basis Function (RBF) technique. Another work of Gonzalez-Vidal et al. [38] propose ML and grey-box approaches to predict the energy consumption to test if the prior information on the physics of the building heat transfer is currently redundant because of the completeness of the system data. The grey-box uses the physics of the building heat transfer to estimate the energy consumption in a normal operation state, while the ML method combines statistics with SVR, RF and XGB. Aliberti et al. [39] discuss a methodology for short-and medium-term predictions of an indoor air temperature of a building using a Non-linear Autoregressive Neural Network (NAR) technique. The proposed predictive model can estimate the indoor air temperature in individual rooms with a prediction window of up to 3 h, and for the whole building with a prediction window of 4 h. Lawadi et al. [40] compare 36 ML algorithms to estimate the indoor temperature in a building, such as the Extreme Learning Machine (ELM), SVR, and Generalized Regression Neural Network (GRNN) algorithms. The algorithms were evaluated using different metrics like accuracy and robustness to weather changes. Zou et al. [41] define a deep learning-based human activity recognition scheme for smart buildings to identify human activities using only WiFi-enabled IoT devices. They collect CSI measurements from commercial IoT devices, and develop an Autoencoder Long-term Recurrent Convolutional Network (AE-LRCN) to eliminate the noise in the raw CSI data, extract the main features, and find out the temporal dependencies among data, for the human activity recognition problem.
For classification problems in smart buildings, Siddiqui et al. [42] introduce a personalized appliance recommendation system based on Non-Intrusive Load Monitoring (NILM) that uses a DL approach to recommend consumption patterns for the appliances. The NILM   Fig. 4. Summary of AI techniques in monitoring tasks.
algorithm is used to clean the noise of the data. Then, with a classification based on Term Frequency-Inverse Document Frequency (TF-IDF) they quantify and analyze the energy tags by assigning them weights. Once the data is classified, the recommended consumption patterns are generated for the appliances through the recommendation system. Fig. 5 shows the AI techniques used in the reviewed works for analysis tasks. The most used technique is LSTM. It is important to highlight that there are not diagnosis, supervision, recognition or clustering techniques, so common in other contexts, such as in Industry 4.0, and smart cities.

Decision making
ACODAT's decision-making tasks may work in controlling a smart building, defining an optimization scheme or systems planning in smart buildings, among other things. In general, the decision-making tasks can be studied from different views: (1) the automation of management systems, known as Building Automation Systems (BAS); and (2) the energy efficiency in buildings and homes with BEMSs and HEMSs. These tasks can actuate on physically networked elements and solve problems with multiple optimization objectives. In this section, the reviewed works are grouped in these applications.

Control schemes
We will start with the works that propose control schemes for smart buildings. Ghadi et al. [43] investigate the use of fuzzy logic controllers in HVAC systems and light controllers for smart buildings in Australia. The paper highlights the development of intelligent control systems to improve the efficiency of control systems in buildings. Hao et al. [44] present a methodology to optimize the control of HVAC systems and minimize energy cost using a multi-agent RL algorithm. Each agent controls one central HVAC system in one of the buildings, and tends to maximize its profits within the power system constraints. Shaikh et al. [45] developed a multi-agent control system in combination with stochastic intelligent optimization for achieving a balance between energy consumption and wellbeing indoor environmental conditions. In addition, the control system has also been embedded with an evolutionary multiobjective genetic algorithm (MOGA) for optimizing the energy management of the buildings. Anvari-Moghaddam et al. [46] defined an energy management system (EMS) for integrated homes/buildings in a microgrid system with various RESs and controllable loads. The EMS is based on a fault-tolerant ontology-driven multi-agent system, where the agents can be from simple-reflex to complex learning agents. They cooperate to define optimal energetic strategies, which consider the management of the distributed generation (DG) and demand response (DR).
Gao et al. [47] propose a deep reinforcement learning (DRL) based framework for smart buildings based on thermal comfort control and energy optimization. They design a deep neural network (DNN) method with Bayesian regularization for predicting the occupants' thermal comfort by considering different influencing factors. Then, they adopt a DRL approach for thermal control to minimize the overall cost by jointly considering the energy consumption of the HVAC system and the thermal comfort of the occupants. Ashabani et al. [48] design a real-time continuous and adaptive demand control strategy for buildings based on a three-phase multiobjective autonomous/automated load control approach, with regulation commands and autonomous grid ancillary services.
Finally, AI also is present in adaptive controllers that actuate on the systems to compensate for unforeseen load changes, uncertain inertias, and any other disturbances for robustness. The adaptive controllers adjust their control parameters either 1) forcing the error between a reference model of the desired behavior and the current output to zero; or 2) forcing the error between the predicting output and the actual output. A good example of this advanced strategy is the LAMDA-based controller [49], which achieves the most appropriate operational state for the controller with fuzzy logic, providing a fast reaction to achieve the optimal state. Morales et al. [49] propose the application of LAMDA (Learning Algorithm for Multivariable Data Analysis) for advanced control in HVAC systems for buildings. LAMDA defines the control problem as a fuzzy classification approach. Thus, it determines the degree of adequacy for every class of a system and subsequently uses its similarity degree to identify the current functional state of the system. Additionally, an inference method has been added to LAMDA to compute a control action that brings the system to a zero-error state. Finally, Homod [50] considers a control algorithm that could handle the next properties of an HVAC system: large-scale nonlinear characteristics, large thermal inertia, time variability, nonlinear constraints, uncertain disturbance factors, and multivariate systems for both temperature and humidity by using hybridization layers between the physical parameters' memory and the ANN' weight, which is well-structured by the Takagi-Sugeno-Kang Fuzzy inference strategy. Fig. 6 shows the AI techniques used in the reviewed works for control tasks. Two of them combine these techniques with RL.

Optimization models
Different authors have proposed energy management algorithms for smart buildings that either integrate or not renewable energy. All these researches follow the same objective: the minimization of the daily energy cost with the optimization of the comfort of occupants. Wahid et al. [51] propose a multi-objective optimization problem (MOOP) of maximizing user comfort and minimizing energy consumption for residential buildings. The energy consumption of the temperature, illumination and air quality inside the building, is minimized, and the user comfort inside the building is maximized. They use three fuzzy controllers, one for each of the variables. The MOOP approach has been resolved with different multi-objective optimization techniques (artificial bee colony (ABC), ant colony optimization (ACO) algorithms, and Firefly algorithm (FA)). Salehi et al. [52] propose an EMS of interconnected multi-energy hubs (MEH) that minimizes carbon emission and procurement costs. They use ε-constraint and max-min fuzzy decision-making techniques to reach a commitment among these contrary objectives. Wang et al. [53] propose a multi-objective optimization model for a BEMS in a building with a photovoltaic system integrated with other generation sources to optimize the overall cost of the building system and occupants' indoor environmental comforts at the same time, under ToU and price-based DR. Si et al. [54] evaluate algorithms used for building energy-efficient design optimization. They use a set of performance indices to evaluate the performance of the algorithms, which are stability, robustness, validity, speed, coverage, and locality. Hookee-Jeeves algorithm, Multi-Objective Genetic Algorithm II, and Multi-Objective Particle Swarm Optimization algorithm (MOPSO) are evaluated.
The aim of the work of Delgarm et al. [55] is to define a simulation-based multi-objective optimization for building energy efficiency and indoor thermal comfort to find the optimal solutions of the comfort-energy efficient configurations of the building. They propose an optimization method that combines a multi-objective artificial bee colony (MOABC) optimization algorithm with an EnergyPlus building energy simulation tool. Ullah et al. [56] propose EMSs for homes/buildings based on the Moth-Flame Optimization (MFO) and Genetic Algorithms (GA). The EMSs must minimize the energy cost and Peak to Average Power Ratio (PAPR), and maximize end-user comfort. Braun et al. [57] present the optimization of appliances as well as heating and air-conditioning devices in two distinct settings of smart buildings, a residential and a commercial building. In both scenarios, the operation times and operation modes of household appliances as well as HVAC devices are optimized with respect to the minimization of energy costs, CO 2 emissions, and technical wearout as well as to the maximization of comfort, i.e., minimization of discomfort. They compare four state-of-the-art algorithms in realistic simulations: NSGA-II, NSGA-III, and SPEA2. Du and Li [58] define an EMS for a smart multi-microgrid (MMG) that combines two techniques, the DL and the RL methods. The set of microgrids are connected to the main distribution system to purchase energy to maintain local consumption. The goal is to decrease the demand-side PAPR, and to maximize the profit from selling energy.
Many studies use Life Cycle Assessment (LCA) methodology in order to assess the environmental impacts of buildings. Because of the significant resources to analyze all possible scenarios in an LCA study, computational optimization techniques are utilized. Harmathy et al. [59] propose an approach for the overall energy performance improvement of office buildings. They formulate an optimized building envelope model using a multi-criterion optimization approach, which determines efficient window to wall ratio (WWR) and window geometry considering indoor illumination quality, and then, assess the glazing parameters influence on the annual energy demand. The objective of the research of Bre et al. [60] is to optimize the energy and thermal performance of residential buildings, based on an objective function defined as the weighted sum of both performances in the home. Also, they performed a sensitivity analysis to determine the effect of the design variables on the objective function. Finally, they solved this optimization problem using GA. Azari et al. [61] use a multi-objective optimization algorithm to find an optimum building envelope design considering the energy use and the life cycle contribution to the impacts on the environment in an office building. They consider several aspects for the design, such as window type, window frame material, glazing type, wall thermal resistance, insulation material, and south and north WWR. The results of LCA are used in a hybrid ANN and GA-based approach as the optimization technique to identify the optimum design combination.
The paper of Liu et al. [62] is focused on energy trading in the 'distribution network market', where there are a number of participants known as aggregators. The aggregators have contracts with distributed resource owners who choose their aggregators. They use a dynamic pricing methodology for decentralized energy trading to optimize the financial benefits of the distributed energy resource owners. In this paper, a Java Agent Development Framework (JADE)-based multi-agent system is applied to model the participants. Chen et al. [63] propose a prediction-integration strategy optimization (PISO) model to enable interactions among prosumers in distribution grids. Their market prediction model is based on the ELM technique, and learns the relationships between prosumer bidding actions and market responses. This model can be used in a continuous double auction (CDA) market to facilitate prosumers' participation. Ma et al. [64] propose the concept of Smart Building Cluster (SBC), and define a multi-party EMS for SBC based on the non-cooperative game theory. In this context, all participating SBCs are viewed as players in the game. The EMS considers building-integrated PV systems and automatic demand response (ADR). Finally, the Nash equilibrium in the model is determined by a MOOP.
In recent years, there is interest in using RES, but due to the high uncertain of these resources in power systems, continuity and stability have become a big problem. Roukerd et al. [65] describe an index for availability assessment considering the maximum available capacity and the reaction time of generating units. Particularly, in a smart grid environment, virtual demand response resources (DRRs) are used for solving the uncertainty problems of renewable energy resources. They model the uncertainty of DRRs using the Z number approach, which is a possibilistic-probabilistic method. The uncertainty of supply-side resources is also considered in their approach to determine the optimum index of availability, with a minimal total operation cost. Finally, Kim and Lim [66] consider an EMS for a smart energy building connected to an external grid as well as distributed energy resources including a RES, ESS, and vehicle-to-grid station. First, the EMS is modeled using a Markov decision process. Subsequently, an RL-based energy management algorithm is proposed to reduce the operational energy costs of the target smart energy building under unknown future information. Fig. 7 shows the AI techniques used in the reviewed works for optimization tasks. The main used techniques are the multi-objective approaches, some of them based on EA.

Scheduling models
Ullah et al. [67] describe a mechanism to schedule the load units to achieve three objectives: minimization of the consumed energy cost, peak-to-average power ratio, and consumer waiting time due to schedule load. To achieve the previous objectives in the industry, they analyze two bio-inspired heuristic techniques: Grasshopper-Optimization Algorithm and Cuckoo Search Optimization Algorithm (CSOA). Rasheed et al. [68] propose a control system to minimize energy cost, user discomfort, and peak load demand, for residential load management under a real-time pricing environment. The proposal is based on a multi-agent-based multi-layered hierarchical and the idea that these incompatible goals can be achieved with a load scheduling using a real-time information exchange among users. Chouikhi et al. [69] introduce two EMSs in smart buildings using multi-agent systems and game theory. They optimize the energy demand cost using a distributed energy demand scheduling approach and model the interactions between the providers and consumers based on the multi-leader-follower game theory.
The work of Liu et al. [70] present an energy-scheduling scheme for an IoT-based EMS. They design two types of scheduling methods to schedule the devices based on a DRL algorithm: an edge and a cooperative method. In the first method, the edge server executes the energy-scheduling task, which uses the DRL algorithm. In the second method, the DL training is carried out in the cloud server, and the edge server adopts the solution calculated by the cloud server. In another research, Mocanu et al. [71] explore in the smart grid context the benefits of using DRL to perform online optimization of schedules for BEMs. Particularly, they explore two DRL methods, Deep Q-learning and Deep Policy Gradient. Ye et al. [72] propose a real-time autonomous energy management strategy for a residential Multi-energy system using a model-free DRL-based approach to optimally schedule the usage of different devices with the aim of minimizing end-users' energy costs. Baniasadi et al. [73] describe an integrated HEMS for smart residential buildings to manage different resources based on ToU pricing tariff, in order to minimize the operation cost and reduce the mismatch between generation and load demand. They use a colonial competitive algorithm (CCA) to minimize the operation cost, which is an evolutionary optimization algorithm that is inspired by imperialistic competition.
Aslam et al. [74] propose a DSM approach for residential consumers to tackle the home appliances' scheduling problem, which is based on meta-heuristic techniques. Hence, they propose an efficient HEM scheme in a BEMS using GA, the CSOA and a Crow Search Algorithm, which are used for electricity cost and peak load improvements with a minimum user waiting time. Jabarullah et al. [75] describe a scheduling approach for energy hubs using the artificial bee colony algorithm, to determine the optimum short-term scheduling for small-and medium-scale demands. The energy hubs must manage the controllable loads, and provide electrical, cooling and thermal demands. Fig. 8 shows the AI techniques used in the reviewed works for optimization tasks. The main used techniques are the bio-inspired approaches, and the other two techniques used are DRL and multi-agent approaches.

General autonomic energy management architecture
In this section, several works are described that present a general architecture of an EMS in a smart grid, which eventually consider models based on agents, or concepts of the autonomous computing area or middleware, among others. In general, AI allows the conception of Intelligent Agents that implement autonomous tasks, as shown in the architecture of ACODAT [2,3] that becomes autonomous following the four dimensions of IBM's MAPE-K [76]: 1) self-configuration, 2) self-healing, 3) self-optimization and 4) self-protection. AI addresses the integration of the management systems, either horizontally extending the supervision to smart building clusters, smart districts or smart cities, or vertically following any hierarchy implementation, like ISA-95 Standard. One of the works about energy devices is the research [77], which defines an 'adaptable smart thermostat' for residential energy management, utilizing fuzzy logic and wireless sensors capabilities, in order to avoid that residential customers must manually re-program their thermostats in response to dynamic electricity prices or environmental conditions that vary over time. The thermostats must autonomously learn and adapt to users' schedule and preference changes in order to save energy. Also, the results of Poorvaezi et al. [78] indicate that the uncertain supply-side resources cause to decrease in the flexibility level under the smart grid environment. In this paper, they define a new formulation of flexibility-based unit commitment associated with demand response resources, where the uncertainty of DRRs is modeled using the Z number, which is a possibilistic method, with a Monte Carlo procedure.
Linked to multi-agent systems, some papers close to this domain are described next. In the work of El-Baz and Tzscheutschler [79], a double-sided auction mechanism is presented for the smart microgrids, where in-house energy supply and demand devices are participating to maximize economic benefit, and microgrid autonomy. The proposed solution is a micro-market based on a double-sided decentralized bidding auction where prosumers can participate either as producers or consumers in a microgrid depending on their energy need within a discrete-time interval. Monacchi et al. [80] study the use of forward contracts in the context of smart grid. These contracts are based on service-level agreements, such that the prices are defined according to the future supply and demand curves. They develop a learning broker based on ANN to determine the future supply and demand curves, and an energy broker that formulates prices based on these future supply and demand curves. Huang et al. [81] present an EMS for smart buildings with hybrid power grids based on real-time and distributed energy data collection and analysis. The EMS uses an uncertainty-aware minority game approach to extract and classify energy signatures of rooms. Then, they define a demand-response management system to minimize peak demand and to distribute solar energy based on a multi-agent approach.
Smart grid modeling as an autonomous process, eventually with emergent and self-organized characteristics, has also been the object of study [82]. Muralidharan et al. [83] propose a model of a self-governing system for smart objects that autonomously share power without a central controller. The decentralized power distribution (DPD) method distributes the available power among the appliances of the system according to their priorities. Aguilar et al. [84] propose a self-managing architecture for multi-HVAC systems in buildings, based on the ACO-DAT concept. This approach is used for improving energy consumption, as well as to maintain indoor comfort, and maximize equipment performance, by means of identifying and selecting a possible multi-HVAC system operational mode. This architecture is based on a set of data analysis tasks that use the data gathered from the system and the environment to define an autonomic management approach for multi-HVAC systems. Aguilar et al. [85] propose an ACODAT for the supervision of the building's HVAC systems. The supervisory approach of the HVAC system can detect deviations, such as faults or gradual increment of energy consumption.
The work of Schachinger et al. [86] aims at developing an ontology-based abstraction layer that integrates all relevant concepts for Smart grid interaction in order to provide homogeneous knowledge representation for BEMSs. Based on the ontology, a BEMS is able to gain knowledge of both dynamic and static characteristics of the ambient smart grid. Sayah et al. [87] aim at proposing a semantic framework for saving energy in smart cities. They propose an autonomous architecture based on ontologies, Big Data, and Multi-Agent Systems to reduce energy consumption in smart cities.
The energy-aware management of smart environments is a challenger. Following this direction, De Paola et al. [88] define a hybrid intelligent system for smart buildings based on a fog-based architecture, in order to achieve energy efficiency. This proposal combines reactive intelligence, in the edge devices, for a quick adaptation; and deliberative intelligence, for more complex tasks (e.g., optimization, learning) on the cloud. Reactive intelligence is implemented by a fuzzy controller that acts in response to current conditions by automatically choosing the actions the actuators must perform. The deliberative component optimizes the behavior of the reactive intelligence finding the best trade-off between meeting the user's preferences and minimizing energy consumption. Fotopoulou et al. [89] propose an energy-aware IT ecosystem for energy efficiency in a building based on the energy consumption behavioral changes of the occupants. This ecosystem personalizes energy management to lead to occupants' behavioral change towards actions with a positive impact on energy consumption. The energy-aware IT ecosystem is based on the IoT, semantic web, rule-based recommendations, and data mining technologies. They define a recommendation system to suggest personalized actions that lead to behavioral changes to improve energy efficiency. Fig. 9 shows the AI techniques used in the reviewed works for autonomous EMS. The main used techniques are the multi-agent and autonomic approaches.

Challenges
From the ACODAT perspective, research challenges on BEMSs found in recent literature respond to three categories: Monitoring and feature engineering of the system and contextual information, including the occupants' behaviors; Data analysis for knowledge discovery; and advanced control and supervision for decision-making. In the rest of this section, some specific challenges will be grouped in different ways.

Monitoring
The heterogeneous data coming from sensors is the first problem faced in monitoring the building. Also, it is necessary to find ways to improve the data quality, which will be delivered to the rest of the services and applications. The IoT and smart meter concepts extend the opportunities for getting better and more accurate knowledge, but add new necessities like Big Data architectures to manage the massive amount of data, or the feature engineering to extract, select and fuse the data. Some specific challenges are: • Develop a real-time semantic feature engineering process for diagnostic or predictive models for smart buildings. • Propose an optimal sensor location method for smart building, which operates in IoT environments, for efficient energy management. It must guarantee the diagnosticability of the Smart building. • Develop a decentralized smart monitoring system for EMSs considering feature-engineering processes. • Define a non-intrusive occupant detection system for smart buildings considering contextual information like the occupant consumption profile or the concentration level of indoor CO 2 or the audio, or with load forecasting approaches like occupancy prediction, for a robust estimation of building occupancy.

Analysis
The second category of challenges is related to the analysis of the data obtained from the monitoring stage. AI techniques can be used for modeling or understanding situations [90]. For instance, they can define models for predicting consumption, for diagnosing a situation, or for determining occupants' use, among other things. In this section, the challenges are grouped into classic ML (supervised, unsupervised and RL) techniques for specific problems, for smart-based BEMSs.
A. Supervised and Unsupervised Machine Learning Techniques. Advances in the area of ML can be used to attack specific problems in BEMSs. These latest advances refer to multi-labeling strategies, coclustering approaches, semi-supervised learning strategies, among others. Let's look at some of those possible applications. There are several services required in a BEMS, which can be solved with ML techniques: • Energy load prediction, • Classification of the consumers (for example, according to socialdemographic information), • Load profiling using clustering based on consumer energy consumption behavior, • Anomaly/theft detection, • Estimation of the energy consumption for a Home/Apartment, devices, etc.
For some of the services proposed above, develop semi-supervised approaches (e.g. based on LAMDA) or automatic feature engineering processes are required.
Other more complex problems are: -Find abnormal energy consumption patterns by analyzing temporal data streams. -Predict the situations of conflict in a smart grid or a microgrid of a smart building, which can impact the energy efficiency -Develop forecast approaches using boosting or bagging or stacking schemes for smart building contexts. -Define opportunistic dynamic Predictive Models for residential and commercial smart buildings, which consider noise or missing data scenarios. -Formulate surrogate models to represent the device behaviors, which consider all relevant constraints. The same idea can be used for the tariff generation patterns, which infer a price signal from a given load profile -Define temporal energy patterns (e.g., energy usage patterns of lights and air conditions) based on time series and temporal logic. -Develop a multivariate time series feature selection methodology for predicting energy behavior (consumption, etc.) in smart buildings to minimize several metrics like Root-Mean-Square Error (RMSE) and Mean Absolute Error (MAE) and the number of attributes. -Formulate forecasting models (energy consumption, indoor temperature) in smart buildings using incremental learning, interactive learning (with real-time user feedback about comfort levels) and TKL approaches. -Develop approaches to build patterns considering the uncertainty and imprecision using approaches like fuzzy logic and Z-numbers. -Formulate hybrid prediction models of energy consumption, mixing information about the physics of the buildings (e.g., the heat transfer process), with the available current data in real-time.
-Formulate approaches to determine thermal comfort using novel metrics like metabolism rate, considering three aspects of the indoor environment: visual, thermal, and indoor air quality comfort. -Develop diagnostic models using multilabel/multi-clusters approaches. The approaches must consider the feature-engineering problem in an autonomous way.
B. Reinforcement Learning for Intelligence-based Management Systems.
A currently widely used learning technique, because a system can learn from experience, is RL. This particular technique has many potential uses in the field of BEMS, for example: -Define a multi-agent RL based on a DRL approach for the building energy consumption-scheduling problem in a smart building. -Develop a real-time autonomous EMS using DRL or RL approaches to determine the real-time autonomous control strategies. -Formulate multi-agent RL algorithms to control the components of a microgrid (e.g., a multi-HVAC). -Define a smart household appliance usage strategy that allows realtime scheduling based on the reduction or shift of the energy demand during peak periods. -Formulate a BEMS that considers the exploration of the continuous space of the possible actions-states map of an RL using DL or LAMDA techniques, and uncertainty about the behavior of certain variables using Z numbers.

Management and decision making
Research on optimization is key for the decision-making category due to the multiple and opposite objectives, or the number of objectives in different areas of design and operations. The automation of the management system itself is also a research trend that includes autonomous scheduling, smart control, among other things. Also, flexible architectures are required based on distributed AI approaches like multiagent systems. These enable a comprehensive modeling for smart buildings, smart building districts or their integration in smart cities. The abstraction provided by this approach allows the implementation of robust, energy-efficient and cost-effective distributed decision-making processes. This section can be broken down in AI-based EMSs, intelligent control systems, optimization problems, scheduling problems, and multi-agent systems and autonomous processes for EMSs.
A. Artificial Intelligence-based EMSs. With regards to the EMSs for smart buildings, a set of systems/concepts are currently being developed in order to allow their maximum efficiency with the minimum cost and greater comfort of their occupants. In this sense, concepts such as smart grid, microgrid, DSM, energy hub, IoE, among others, have been appearing. Next, we name some challenges regarding the use of AI in their conceptions. -Develop a distributed smart EMS based on the cooperative/federate ML principle for the energy management of smart buildings in smart grids. This scheme can be enriched with a multi-agent system model and a distributed feature engineering process. -Develop a Smart DSM to balance the supply and demand of electricity by shifting energy consumption from unfavorable to favorable time slots. -Formulate a Prosumer Agent that can optimally manage producers and storage devices, and in addition, determine when must generate and consume energy. -Define an adaptive demand-side approach that allows shifting electrical loads generated by different sources (e.g. heat pumps, photovoltaic panels), but also, schedules multiple appliances. -Propose a Smart EMS architecture for the scheduling process based on a real-time feature engineering process. C. Optimization Problems.
In the optimization context, the new energetic systems pose a significant number of optimization problems of different natures. Thus, multi-objective, many-objective, or dynamic optimization strategies, among others, are very useful in this context. Here are some of those challenges.
-Define a dynamic optimization approach of cross-linked microgrids considering different objectives. -Formulate an energy management framework for achieving optimal operations of smart building clusters (SBC) using a multi-objective optimization approach by cluster, and intra-clusters one based on the Nash equilibrium strategy. -Develop an EMS of interconnected multi-energy hubs (MEH) aimed at minimizing the procurement costs as well as reducing carbon emission using multi-objective approaches.
-Define an optimal energy management of interconnected smart residential buildings (multi-smart apartment buildings) considering energy flow among them. -Develop a multi-objective multi-agent framework for the management of the bidirectional energy trading capabilities of an EV fleet arriving at a region of academic/office building. -Define a re-configuration approach of EMS like a MOOP that allows the inclusion of new devices, advanced maintenance, among other things. -Formulate an optimized building envelope model using a multicriterion optimization approach to determine the overall energy performance of buildings, in order to help architects and engineers in the early-design stages of new projects. -Develop a joint dynamic optimization of the load scheduling, energy storage control and indoor comfort management in a smart building. The objectives to be considered are: electrical and thermal load scheduling delay minimization; energy procurement cost minimization from controllable generators and external grid; electrical and thermal energy storage degradation minimization; and indoor user comfort maximization. -Define a distributed interdependent many-optimization strategy in multiple buildings in a microgrid, which considers the uncertainty of the data and forecasting of the data required. -Develop a real-time optimization approach for an EMS that considers many-objectives, and the uncertainty about the behavior of certain variables using Z numbers and ML models. -Develop a real-time optimization approach for an EMS based on Z numbers and ML models that considers many-objectives and the uncertainty of variables. -Optimize the consumption of the components of the system (sensors and actuators), by considering the maximization of the system lifetime of the different components of the system. D. Scheduling Problems. Another very relevant area of decision-making is about the scheduling problems in the context of BEMSs, for example, to schedule the utilization of energy resources, of the appliances in a home/apartment. There is a diversity of scheduling problems in BEMSs where intelligent scheduling strategies using AI techniques can be used.
-Define a dynamic multi-objective scheduling scheme for smart buildings in the incertitude context of EMSs. -Define a distribute device scheduling for buildings considering the as-scheduling, availability of resources, etc. -Define a metalearning approach to select the suitable smart HEMS optimization technique to schedule the home appliances consumption so that the overall energy consumption be minimized. -Develop a smart scheduling approach that considers economicenvironmental constraints, prosumers, user living patterns, energy storage (like a battery), waiting time of home appliances. -Define a distributed BEMS to optimize energy consumption.
-Define a hierarchical DRL method for the scheduling of energy consumptions of smart buildings. -Carry out planning/scheduling schemes that incorporate maintenance tasks in the energy infrastructure.
E. Multi-Agent Systems.
Multi-Agent Systems have been used to model distributed systems to describe complex behaviors such as distributed control tasks, conflict resolution, among others. Also, it is a base model to allow emergent behaviors in the system. In the context of BEMSs, some possible challenges are: -Formulate a decentralized approach for negotiation and conflict resolution in the context of smart grids.
-Develop smart devices (e.g. an 'adaptable smart thermostat') like agents, which can carry out its energy management. -Develop a continuous double auction-based and bilateral contractbased P2P electricity trading mechanism, for properly managing electricity trading among prosumers. -Develop trust models for P2P electricity trading.
-Define a distributed auction mechanism for multi-energy scheduling in an energy hub that serves several building energy users. -Formulate a smart management of a microgrid community composed by local energy management system (LEMS) for optimal planning, -Develop a demand bidding and emergent DRM program mechanism emergent theory. -Formulate a multi-agent building control system based on bioinspired optimization approaches (e.g., particle swarm, ACO, etc.) to achieve the smart building control goal. -Define an EMS based on a distributed uncertainty-aware approach for energy resource allocation. -Develop a multilevel decision system for EDM for Smart Buildings based on multi-agent concepts. -Develop a multi-agent RL algorithm that schedules the energy consumption of multiple smart homes/apartments with distributed energy resources, energy storage systems and controllable home appliances (like air conditioners or washing machines). -Define a distributed load scheduling mechanism using multi-agent systems and uncertain theory to manage dynamically the uncertainty in the price. -Define a self-governing system of smart objects that autonomously share power without a central controller based on a decentralized power distribution (DPD) method based on multi-agent systems that distribute the available power among the appliances according to their priorities.
F. Autonomous Processes. The concept of ACODAT seeks to integrate various AI techniques for the autonomous management of processes. In this sense, there are many possible applications in the context of BEMS. Below, some of them: -Develop an ACODAT like a metalearning approach to build forecasting/diagnostic models for a BEMS, -Define a diagnostic approach of energy consumption in devices (e.g., HVAC systems) using an ACODAT of two steps: a clustering phase to define the power consumption pattern and an association rules phase to interpret the power consumption pattern. -Propose an autonomous DL framework for an IoT platform of BEMSs like an ACODAT to reveal the temporal dependencies among the time series data. -Formulate a recommendation system about the appliance utilization in a BEMS using ACODAT with a DL approach that determines the appliance behavior, and a classification approach to analyze the energy tags. -Develop an IoT-based Energy Management Semantic Model based on the concepts like Linked Data Analytics Ontology and Emergent Ontologies. -Define an EMS based on ACODAT for a smart grid composed of micro-grids to optimize energy dispatch. -Propose an energy-aware management for multi-building scenarios based on ACODAT that exploits a fog-based architecture -Formulate an ACODAT system for Electric Energy Consumption forecasting in a smart building: A first task groups the daily load demand of many profiles (a time series clustering module). The next task is a training algorithm that utilizes the cluster to learn the forecasting models (e.g., LSTM models). -Define an occupancy management system for smart buildings based on the ACODAT concept: The first task carries out the Occupancy Detection; the next task carries out an Occupancy Counting. Then, the third task carried out an Occupancy Tracking (e.g., use the well-known user localization algorithms), and finally, the last task carried out an Occupancy Event/Behavior Recognition: -Develop a hybrid occupant detection system for the detection of anomaly electricity consumption profiles in a building using the ACODAT concept. The first task is a prediction scheme of the loadbuildings (e.g., at the plug-level like desktop computers, laptops, printers and copiers), and the second is a real-time (online) clustering approach to detect the anomaly. -Formulate an autonomous feature engineering process for energy data. It can be coupled to automatically select the more relevant features for different contexts like Fuzzy Rule Learning approaches. -Propose a flexible autonomous energy trading pool for the P2P using interaction protocol of multi-agent systems. -Proposes a generic framework to manage the energy in a community of flexible Smart-Buildings where participants collectively optimize any generic objective, such as grid services or promoting local RES energy consumption. For that, collective consensus (social set points) is required. -Model the auction-based P2P electricity trading mechanism among prosumers using multi-agent systems.
A resume of these challenges is shown in Table 4.

Disclaimer
The content of this publication does not reflect the official opinion of the European Union. Responsibility for the information and views expressed herein lies entirely with the author(s).

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.