Energy and AI

Electricity load forecasting is an important part of power system dispatching. Accurately forecasting electricity load have great impact on a number of departments in power systems. Compared to electricity load simulation (white-box model), electricity load forecasting (black-box model) does not require expertise in building construction. The development cycle of the electricity load forecasting model is much shorter than the design cycle of the electricity load simulation. Recent developments in machine learning have lead to the creation of models with strong fitting and accuracy to deal with nonlinear characteristics. Based on the real load dataset, this paper evaluates and compares the two mainstream short-term load forecasting techniques. Before the experiment, this paper first enumerates the common methods of short-term load forecasting and explains the principles of Long Short-term Memory Networks (LSTMs) and Support Vector Machines (SVM) used in this paper. Secondly, based on the characteristics of the electricity load dataset, data pre-processing and feature selection takes place. This paper describes the results of a controlled experiment to study the importance of feature selection. The LSTMs model and SVM model are applied to one-hour ahead load forecasting and one-day ahead peak and valley load forecasting. The predictive accuracy of these models are calculated based on the error between the actual and predicted loads, and the runtime of the model is recorded. The results show that the LSTMs model have a higher prediction accuracy when the load data is sufficient. However, the overall performance of the SVM model is better when the load data used to train the model is insufficient and the time cost is prioritized.


Introduction
There are many driving factors that make accurate electricity load forecasting models a pertinent issue, the most obvious and pressing of which is climate change.Carbon emissions are one of the most significant driving forces of climate change and with data being published [1] that shows carbon emissions rates are increasing, a solution is urgently needed.Emissions from electricity generation account for the 25% of the whole worldwide emissions [2].Because of physical limitations in storing electricity, the production, transmission, and consumption of electricity must be carried out simultaneously with its demand [3], therefore the power supply and power consumption need to maintain a dynamic balance.
Considering buildings accounted for an estimated 41.1% of primary energy and 74% of the electricity, the necessity for accurate energy prediction models for buildings is obvious.According to [4], more than 25% of the 713 GW of the U.S electricity demand in 2010 could be dispatchable, meaning created and used on demand if buildings change, coupled with the granular benefits of load prediction, it is easy to see why the pursuit of an accurate, scalable, and complete energy prediction solution is desirable.Accurate electricity load forecasting contributes to the supply demand stability of the power grid and it can provide a reference for the planning and operation of power systems [6], often through demand response programs.Demand Response (DR) is one of the Demand Side Management (DSM) measures that has been promoted as a mechanism to increase the percentage of renewable energies in the system [7].It is defined as ''changes in electricity use by demand-side resources from their normal consumption patterns in response to changes in the price of electricity or to incentive payments designed to induce lower electricity use at times of high wholesale market prices or when system reliability is jeopardised'' [8].A DR signals by a demand response aggregator or Trasmission System Operator (TSO), triggers the intentional reshape of the electricity demand profile.The variation can be measured as level of instantaneous demand or total electricity consumption deferred.DR assets can dynamically change the electricity demand curve, providing peak shaving, frequency control, load shifting and load forcing measures [9].
Therefore, predicting peak demand and short term forecasts could support energy management systems devices to actuate optimal strategies during demand response events [10].Depending on the time period, electricity load forecasting can be divided into four types, which are very short-term, short-term, medium-term and long-term load forecasting [6,11].
• Very short-term load forecasting (VSTLF) focuses on electricity load within 1 h in the future.It is mainly used for real-time safety analysis of power systems and monitoring the operation of power equipment [12].• Short-term load forecasting (STLF) refers to one-day ahead and one-week ahead load forecasting.In the four types of electricity load forecasting, STLF has important practical significance, which can help with electricity dispatching and management in power systems [13,14].• Medium-term load forecasting (MTLF) focuses on predicting electricity load in the coming weeks or months.It is used for the operation and maintenance of power systems [15].• Long-term power load forecasting (LTLF) is used for long-term planning of power systems.It is mainly used to predict the electrical load for the next year or even years.[16].
The work described in this paper focuses on STLF, which is an integral part of the smart grid in order to estimate the future electricity loads accurately and minimize errors between actual and predicted loads, which can help to improve the utilization of power generation equipment and the effectiveness of economic dispatch [14].Predictive energy demand techniques for buildings can be broadly divided into white-box model and black-box model [17].The former is physicsbased model, which requires expertise in the field of building.The white-box model uses building energy simulation software with a set of detailed physical rules and building information to generate electrical load data [18].The latter is used to find the correlation between electrical load and historical load data [3].It is also known as a data-driven model, which does not require physical information about the building, but requires sufficient historical data [19].Recent research is focused on exploring predictive techniques for electricity demand, targeting improving the accuracy of forecast results, whether using white-box model or black-box model.The white-box model is an essential tool for calculating and analyzing the building energy load and has been widely used [20].The core of the white-box model is to transform the basic physical characteristics of the building into corresponding simulations [21].If most of the detailed building information and energy transfer processes are considered in the BES model, the prediction accuracy of these models is high.However, some detailed data may not be available to the user during the simulation process, resulting in poor prediction performance [22].The white-box model is computationally expensive and has a long development cycle.Many researchers have tried to simplify the white box model, however, the simplifications cannot solve the above problems and are prone to errors [21].The black-box model predicts the electricity load by learning from historical load data and some external factors.It can avoid the inadequacy of the white-box model given that detailed building information is not required [22].In addition, the high stability and accurate prediction of the black-box model is also the reason why the increasing number of research is taking place [19].In the early stages of black-box model development, traditional forecasting methods for modeling electricity load through stochastic processes are widely used.They are the easy-touse predictive methods that associate electricity load data with impact variables.Kalman filter [23], Time series analysis [24] and statistical regression [25] are typical representatives of such methods.These algorithms have the advantages of fewer parameters, lower computational complexity, and greater interpretability.They are able to achieve good forecast results when dealing with highly stable, periodic electricity load datasets.With the rise of machine learning, many network structures and training algorithms have emerged.These algorithms have powerful learning capabilities and the ability to handle complex nonlinear functions to adapt to the complex influencing factors [26].Currently, among the most common technologies for electricity forecast there are Artificial Neural Networks (ANNs) [22] and Support Vector Machines (SVM) [27].In [28], Support Vector Machines (SVM) are highlighted as the most accurate option for forecasting electricity load, with similar levels of complexity and accuracy of deep neural networks.Support Vector Machines (SVM) do suffer from some disadvantages not seen in ANNs such as low running speed.This is an issue, most notably in projects with large scope.Unfortunately this is a widespread issue with models which are accurate to a fair degree requiring a large computer memory and processing or computation time [29].The upside of using Support Vector Machines (SVM)s is that they require few inputs, so this means the feature selection process is easier.They are also notable for having a relatively simple training process due to requiring a few inputs as mentioned earlier [21].Statistical regression models are a prominent option for forecasting electricity load.These models are beneficial for evaluating the importance of potential inputs for models but struggle with short term predictions, with a relatively large amount of inaccuracy in this field.
A hybrid model is proposed [30] based on improved empirical mode decomposition (IEMD), autoregressive moving average (ARIMA) and wavelet neural network (WNN).The model is said to perform better in comparison to SVMs based upon case study data from America and Australia and can provide a robust, stable and accurate prediction result.A method is introduced [31] based on Gaussian Process Regression (GPR) which also incorporates physical insights about load data characteristics.It achieved an accuracy of up to 94.38% and 99.26% for long-and short-term forecasting, respectively, although interestingly as training data and forecast length increased, so did prediction error.
A new framework based on Long Short-Term Memory (LSTM) Network moving window-based technique is described by [32].LSTMs are a form of Recurrent Neural Network, which excel at time-series forecasting due to ''maintaining a memory cell to determine which unimportant features should be forgotten and which important features should be remembered during the learning process''.This approach is said to outperform regression models, Support Vector Machines and Artificial Neural Networks.Recurrent Neural Networks can be used to predict electricity demand efficiently.There are numerous advantages such as handling non-linear complexities, minimum prediction errors and ease of generalization.Research into Long Short Term Memory (LSTM) shows promise, with many studies being conducted to show the efficacy of the model in handling time-series [32][33][34].One such study cites the use of Convolutional Neural Network (CNN) layers in conjunction with LSTM layers as a method of improving accuracy [34].
The ability of Recurrent Neural Networks, and more specifically LSTMs alongside CNNs to make proven accurate predictions using time-series data makes them a model for consideration.
The aforementioned papers reproduce and improve the forecast models' accuracy using LSTM methods and compare with the performance of ANN and SVM on historical energy demand dataset.In the literature, the applicability of the methods on existing demand response programs is limited.The novelty of the current work is the use of the forecast models to predict both the electricity demand, daily peak load and valley load to dynamically optimize the local generator, the thermal storage and the demand for a new highly efficient commercial building equipped with advanced control systems, which is also a demand response unit.Additionally, the work explores the model's adaptability through multiple cases -on one-hour ahead load forecasting and one-day ahead peak and valley load forecasting that could be used to schedule demand response measures in response to grid signals.Such an objective has been validated comparing the performance of forecasting techniques and adapt the test cases for specific demand response programs such as day-ahead scheduling, and secondary reserve time resolution.The effects of outliers' processing and feature selection on prediction accuracy are also discussed.The paper is organized as follows: Section 2 provides the principles of these two common forecasting methods Artificial Neural Networks (ANNs) and Support Vector Machines (SVM) for STLF.An overview of the experiment is provided in Section 3. Section 4.1 describes the commercial building used for the experiment.Section 4 denotes the experimental setting, dataset details, and evaluation metrics.The experimental results are discussed in Sections 5 and 6 summarizes the work described in this paper.

Artificial Neural Networks
The design of ANNs is inspired by the structure of the human brain [21].Typically, ANNs consists of the input layer, the hidden layer, and the output layer.Each layer contains multiple neurons and their corresponding activation functions.Multiple hidden layers can improve the ability to handle non linearities, making them more accurate for electricity load forecasting [22].The Recurrent Neural Networks-based model has become an important technique for dealing with nonlinear and short-term dependence in sequence data in recent years [35].However, during model training, the multiple uses of matrix multiplication lead to gradient disappearance or explosion.
Long Short-term Memory Networks (LSTMs) technique is a special type of RNNs.It retains the recursiveness of RNNs while having selective memory.The LSTMs avoids the problem of gradient disappearance and gradient explosion through the forgetting mode [36].Like other neural network structures, the LSTMs consists of the input layer, the hidden layer, and the output layer.However, the internal structural unit of the LSTMs is a cell, also known as a memory unit.This memory unit contains the forget gate (blue circle), the input gate (orange circle) and the output gate (red circle), as shown in Fig. 1.
The LSTMs decides which information to discard through the forgot Gate.The function of the input gate is to determine the value of the updated memory state.The output gate is used to determine the output value of the current memory unit.
Bouktif et al. use the genetic algorithm to select the optimal time lag and the number of LSTMs layers to forecast short-and medium-term electrical loads [35].The remaining parameters, such as the number of neurons in the hidden layers, activation functions, and optimizers, are determined experimentally.The main purpose of Bouktif et al.'s study is to compare the performance of LSTMs model and other machine learning models.This study initially evaluates the performance of various machine learning models, such as random forest, ridge and extra trees regressor.Secondly, the model with the best prediction result is selected as the benchmark model.Feature engineering and parameter tuning are used to improve the performance of the benchmark model further.Finally, Bouktif et al. compare the performance of this benchmark model with the LSTMs model.Experimental results show that the performance of the extra tree regression model is best in other machine learning models besides ANNs.Therefore, the extra tree regression model is used as the benchmark model for comparison with the LSTMs model.Through further comparative analysis, the prediction error of the LSTMs model is lower than that of the reference model.The authors conclude that the designed LSTMs model is more accurate and stable [35].

Support Vector Machines
SVM is a kernel-based machine learning algorithm with the ability to solve nonlinear classification and regression problems [22].The process of SVM to solve nonlinear problems can be divided into two steps.SVM firstly determines the appropriate function for projecting nonlinear problems into high dimensional space.Secondly, the kernel function is used to make the complex nonlinear mapping a linear problem.It is worth mentioning that the SVM is outstanding in accuracy and can maintain good performance with only a small amount of training data [28].
Khan et al. use support vector machines and artificial neural networks as short-term electricity load forecasting models to compare their performance [27].In the model design phase, the authors firstly select the Feed-Forward Neutral Network (FFNN) as the representative of ANNs.Secondly, Support Vector Regression (SVR) is used, which indicates SVM for solving regression problems.Then, three SVM models with different kernel functions are designed, which are Linear, Quadratic and Cubic SVM.It ensures that the best performing SVM model can be found because different kernel functions have an impact on their performance.Expressions of these three different kernel functions are as following: where x r , x q stand for two input vectors.The transformation function, which are x ′ r x q , ( 1 + x ′ r x q ) 2 , and ( 1 + x ′ r x q ) 3 , map the input vector to a higher dimension space.
According to Khan et al. the experimental results show that the performance of SVM-based models is better than that of ANNs model.However, the authors only select the FFNN model as a benchmark to compare with the SVM model, which does not prove that the SVM model performs better than all neural networks-based models.In addition, the prediction of the Cubic SVM is the most accurate among the above three SVM-based models.

Methodology
This paper studies the existing electricity load forecasting models in order to improve the prediction accuracy, and applies the effective input features and prediction models for the actual electricity load of the commercial building described in Section 4.1.The workflow of the experimental methodology is shown in Fig. 2.
The pre-processing phase includes missing data processing, outlier processing, and normalization.Firstly, a suitable filling method is used to complement the missing data in order to ensure the integrity of the power load sequence.Secondly, the purpose of outlier processing is to identify and modify the random errors present in the electricity load by the characteristics of the actual load data.Finally, normalization is used to eliminate the effects of the order of magnitude of the data.Many factors affect the performance of electricity load forecasting, and these factors can be divided into internal factors and external factors.Specifically, historical load data is an internal factor, and weather, date type, and economic are external factors.Correlation coefficients evaluate these factors in the feature selection phase because not all factors positively affect predictive accuracy.Following the correlation analysis, the appropriate influencing factors are selected as the input features of the load forecasting model.In terms of experiment setup, this paper uses the selected factors to create multiple cases to study the impact of different input features on the performance of the electrical load forecasting model.
As for model building and evaluation, this paper builds LSTM-based model and SVM-based model with appropriate model parameters and evaluates the predictions in the different cases.

Normalization
Machine learning models are sensitive to the scale of input data, therefore normalization is used to avoid the impact of data magnitude on them [35].Therefore, the raw data is linearly transformed by minmax normalization, so that the data size is constrained between [0,1].The min-max normalization formula is as follows: where x is the data to be processed, x n stands for the normalized data, x max and x min are the maximum and minimum values in the load data, respectively.
After the data has been trained and predicted, a denormalisation operation is required.The formula is as follows: x = ( max− min )  +  min (5) where x  represents denormalized data.
In addition, in the normalization phase, the training set and test set use a uniform normalization standard.Therefore, the maximum and minimum electricity loads of the training set and the test set are the same.

Evaluation metrics
The difference between the predicted load and the actual load is called the prediction error.This section lists several commonly used methods for measuring the prediction error and their values can reflect the performance of the forecast model.These measurement standards are defined as follows: ) Mean Absolute Error (MAE) [14,35] is one of the most commonly used average error metrics, calculated by the average of the sum of absolute errors.Root Mean Squared Error (RMSE) [14,27,35] is used to describe one of the most common metrics of uncertainty.It is worth mentioning that RMSE amplifies the value of larger error terms in the calculation process.Mean Absolute Percentage Error (MAPE) [14,27] represents the average absolute error percentage.

Experiment setup and dataset exploration
This section describes the case study and provide an analysis of the dataset used for training the models.Next section will describe the test case building.

Building description
The test bed building [20] is a building with a strong commercial profile, variability of HVAC systems, space usage and occupancy patterns and is located on the UCD campus in Dublin, Ireland.The building is used as a sports/entertainment center, consists of three floors with a total floor area of 11,000  2 and includes a 50 m  25 m swimming pool, with related ancillary areas such as a wellness suite, fitness center, aerobics and dance studios, drama theater, multimedia and seminar rooms, offices, shops and cafe space.A front view of the building is depicted in Fig. 3.
The building electrical and space-conditioning requirements are provided by two identical CHP units (506 kW thermal and 400 kW electrical output each), two gas boilers (1146 kW each) and an air-cooled water chiller (865 kW).Additionally, heat is provided as well, when necessary, by the campus district heating installation (500 kW).The space conditioning delivery equipment consists of eight air handling   units, fan coil units, underfloor heating and baseboard heaters, while the ventilation throughout the building is mechanical.
The building energy rate is B2, which is a benchmark for excellent performance in a commercial building with such equipment and intended use [37].An Energy Management System controls and monitors all the primary and ancillary HVAC equipment in the building.Operational EMS data has been recorded at 15 min intervals from September 2012 onwards.Total electricity and gas consumption are monitored and there are sub-meters on individual HVAC components (i.e., boilers, CHP units, and the chiller).Pressure, humidity, air temperature and  2 levels are measured at different points of the HVAC systems.Moreover, air temperature, relative humidity and  2 concentration are measured at zone level.The experimental data in this paper includes historical electrical load data and weather factor data from January 1, 2013, to December 31, 2018.

Data pre-processing
The data collection period of the electricity load is 15 min with 96 points per day, and a total of 220,336 data points in 6 years.The hourly outdoor temperature, wind speed and relative humidity are used as weather data, which is available on the Irish Meteorological Agency website [38].A data pre-processing procedure has been applied to the time series to reduce abnormal prediction errors.The process has selected outliers and missing data.Both missing data and outliers can be caused by communication system failure or equipment maintenance.Some errors can also be caused by errors in the data collection hardware infrastructure.

Missing data processing
The dataset used in this paper contains 644 missing data, which account 0.3% of the entire dataset.As illustrated in Fig. 4, the electricity load curve can be divided into weekdays and non-working days based on the day type.
In particular, electricity loads at the same time point between different working days are similar, and that of non-working day also have the same characteristics.Special days such as national and local holidays have been analyzed to assess if they could have been clustered and used for data interpolation to fix missing data.However, demand profiles of special days are similar to non-working days.As per best practice, missing data are filled with the average of the electricity load at the same time point of the same day type.

Outliers processing
Outliers are those in which the data at some time point deviates from the range of most other data.A standard procedure described in [39] has been used to identify outliers in the dataset.The methodology uses the five statistical components, which are minimum, the first quartile (Q 1 ), the median, the third quartile (Q 3 ), and the maximum to describe the distribution of the data.Data should be allocated between  The outliers have been flagged, by comparing the distribution of the box plot and the probability density function of the standard normal distribution, as shown in Fig. 5.It can be seen that the outliers account for approximately 0.7% of the entire dataset.Through the box plot test, 2.9% of the data in the dataset have been flagged as outliers.The source of error can be correlated t sensor calibration for the electricity load data.These outliers are corrected using the same method as filling the missing data.

Feature selection
This paper evaluates the applicability of the factors as input features by correlation.According to Kapetanakis et al. the Pearson and the Spearman correlation coefficient can be used to measuring the linear correlation and monotonic relationship between these factors and the output [40].While the Pearson correlation coefficient identifies a linear correlation between the variables, the Spearman correlation assesses if two variables are monotonically related.Both Pearson and Spearman correlation coefficients range between −1 and +1.The absolute value of the correlation coefficient is equal to 1, which represents a positive or negative correlation between the influencing factor and the output variable.There is no correlation between the influencing factor and the output variable when the correlation coefficient is equal to 0. The threshold for determining the influence factor suitable as an input feature is 0.5 [41].As illustrated in [42], there are three main types of external factors that affect the commercial electricity load forecasting, which are weather, day type and economic factors.Among them, economic factors include population growth, power system regulations and economic development trends, which mainly impact on medium-term, long-term load forecasting [42].Therefore, for VSTLF and STLF, these economic factors can be ignored.In terms of weather factors, some commonly used weather variables, including the outdoor temperature, wind speed and relative humidity, are evaluated for their applicability as input features.As for date types, this paper mainly considers the difference between workdays and non-workdays.The date type has a value of 0 and 1, which is marked as 1 from Monday to Friday and 0 on Saturday and Sunday.In addition, the correlation coefficient between the historical load and the output variable is calculated.Fig. 6 shows Pearson and Spearman correlation coefficients, respectively.In these figures, red indicates that the correlation coefficient exceeds the threshold, and the correlation coefficients that do not exceed the threshold are filled with green.
It can be seen that Pearson and Spearman correlation coefficients between the electrical load and external factors are very low, whether it is hourly load or daily peak and valley load.The absolute value of Pearson and Spearman correlation coefficient for most external factors is less than 0.2.However, the historical load is highly correlated with the output.Therefore, this paper only uses historical load data as input features for electricity load forecasting.

Daily peak and valley load analysis
For day-ahead scheduling the prediction of daily demand peaks and valleys could facilitate demand response programs behind the meters.In the current work, the number of peak or valley load occurrences per day for 2018 has been assessed.The result is shown in Fig. 7.
As illustrated in Fig. 7, although the peak and valley values of daily electricity load are not fixed at a certain moment, the distribution of their occurrence times is concentrated.At 1200 h and 1600 h daily, the peak load occurred the most, accounting for 33.9% of the total, while the daily valley load appeared at 1500 h and 2300 h, accounting for 36.2% of the total.In addition, daily peak and valley loads hardly appear at the same time, except between 1800 h and 1900 h.These two moments of the weekdays and the weekends have different meanings.In other words, the two moments are working hours during the weekdays and non-working hours at the weekends.Therefore, both peak loads and valley loads can occur at these two moments.

Experiment setup
For the purpose of the research described in this paper, various cases are created to compare the performance of the LSTM-based model and SVM-based model.These cases are divided into two parts, one for onehour ahead load forecasting and the other for one-day ahead peak and valley load forecasting.The former is conducted using the historical hourly load data from 2013 to 2018, with a total of 52,584 data.The latter is based on the maximum and minimum daily load from 2013 to 2018, with 2191 data.All data is re-sampled from the original dataset.
According to Khan et al. this paper selects load sequences of different lengths as input features based on the auto-correlation function [27].
In terms of one-hour ahead load forecasting, Fig. 8 shows the autocorrelation between the load of 200 h in the hourly load dataset.It can be seen that the load of hour h-1 has the highest auto-correlation, and the hourly load sequence contains multiple periodicities.The load of h-24 and the load of h-24 multiple have peak auto-correlation.
As for one-day ahead peak and valley load forecasting, as shown in Fig. 9.It shows the auto-correlation between the 14-day loads.The auto-correlation coefficients between the daily peak load data sequences are periodic, while the auto-correlation between the daily valley load data sequences is not significant.Overall, the auto-correlation between the daily peak and valley load decreases as the time interval increases.
Based on the above conclusions, this paper creates 4 cases with different input features, case details are shown in Table 1.
• The input of Case 1 is a 8-dimensional vector that uses the load of hour h-1, h-24, h-48, h-72, h-96, h-120, h-144, h-168 to forecast the load of hour h.• The input of Case 2 is a 168-dimensional vector that uses the load of hour h-1 to h-168, the hourly load of the previous week, to forecast the load of hour h.The input features of Cases 1 and 3 are selected based on the autocorrelation of hourly load data and that of daily peak and valley load data.The time point with a high auto-correlation coefficient and that with peak auto-correlation coefficient in the previous week are used as input features.In addition, it can be seen from Table 1 that the input features of Case 2 and Case 4 contain all of the input features of Case 1 and Case 3, respectively.In this paper, Case 2 and Case 4 are used as the control experiments to verify the accuracy and importance of feature selection.

Table 1
The details of each case.

Forecast horizon
Input feature Output (kW) • LSTM-based model: This model contains two LSTM layers and one fully connected layer.The first LSTM contains 100 units and the second LSTM layer contains 50 units.A dropout layer with 0.2 dropout rate is added between the first LSTM layer and the second LSTM layer to prevent overfitting.The activate function is rule.Mean square error is used as a loss function and Amda acts as an optimizer.
• SVM-based model: The linear function is used as the kernel of support vector regression model.The Tolerance for stopping criterion is set to 0.001.The penalty parameter C of the error term is 1.

Cross validation
Time series cross-validation [43] has been used to avoid over-fitting.Thus, the whole historical load dataset from 2013 to 2017 is used as a training set to ensure that there is a sufficient data training model.Then, the energy demand data for 2018 is treated as a verification set.Unlike the K-Fold cross-validation method, time series cross-validation method is trained using data prior to the test set sequence.

Results
The current section describes the results of the prediction models for each case study developed.The results presented in this section have been validated with data extracted by the building management system.Therefore, both the accuracy metrics and the validation refer to the physical building actual data.In particular one-hour ahead load (Section 5.1), and one-day ahead peak and valley load (Section 5.2).

Performance of one-hour ahead load forecasting models
This sub-section mainly evaluates the performance of LSTM-based model and SVM model for one-hour load forecasting.In terms of Case 1, the 2018 monthly RMSE is shown in Fig. 10.
The actual load and the predicted load for hourly load forecasting in June, 2018 are plotted in Fig. 11.The blue line is the actual power data, the green line represents the predicted electricity load by the LSTM-based model, and the red line stands for the predicted data by the SVM-based model.The absolute values of the predicted and actual load differences are displayed below the corresponding time step.Among them, the blue bar represents the LSTM-based model, and the orange bar stands for the SVM-based model.
It can be seen from Fig. 10 that the monthly RMSE trends of the two models are similar, and the RMSE changes are balanced.This shows that the performance of the two models is relatively stable and there is no over-fitting phenomenon.The monthly RMSE of the LSTM-based model is smaller compared to the monthly RMSE of the SVM-based model.It can also be seen from the predicted load curve and the actual load curve shown in Fig. 11 that the LSTM-based model performs better on the fitted hourly power load curve than the SVM-based model.
As for case 2, due to the influence of input feature selection, the 2018 monthly RMSE in Case 2 presents the opposite result of Case 1, as shown in Fig. 12. Fig. 13 shows the actual load and predicted load in Case 2 in June 2018.
It can be seen that the monthly RMSE of the SVM-based model is generally lower than that of the LSTM-based model.The predicted load of the SVM-based model fits better with the actual load.
In summary, considering both feature selection and hourly load forecasting, it can be concluded that the performance of the LSTMbased model is better than that of the SVM-based model.In addition, as shown in Table 2, the runtime of the LSTM-based model is much longer than that of the SVM-based model.

Performance of daily peak and valley load prediction models
This section assesses the performance of the LSTM-based model and the SVM-based model in one-day peak and valley load forecasting.As in the previous sub-section, RMSE is used to describe the performance of the model, and each month of 2018 is used as a test set.Figs. 14 and 15 plots the monthly RMSE of the model for peak load and valley load forecasting in Case 3 and in Case 4, respectively.
It can be seen that the trend of monthly RMSE in Case 3 and Case 4 is very similar whether it is one-day ahead peak forecasting or one-day ahead valley forecasting.
In order to more intuitively compare the performance of the two models, the actual load and predicted load for one-day ahead peak and valley load forecasting in June are plotted, as shown in Figs.16 and 17.In these figures, the blue line is the actual power data, the red line represents the predicted electricity load by the LSTM-based model, and the yellow line stands for the predicted data by the SVM-based model.The absolute values of the difference between the predicted load and the actual load are displayed below the corresponding time step.Among them, the blue bar represents the LSTM-based model and the orange bar stands for the SVM-based model.It can be concluded from the above figures that the LSTM-based model and the SVM-based model have almost the same prediction accuracy in one-day ahead peak and valley load forecasting.In fact, the SVM-based model is able to perform better with small datasets compared to deep neural network.
The predictions have been tested using cross-validation and data analysis techniques to find outliers in the performance of the models.The validation analysis was also applied on special days such as national or local holidays, but it did not reveal any great variance from the average.

Discussion
The current work aims to forecast the electricity consumption of a large and high efficient commercial building using different machine learning techniques.The results of the four main case studies with the MAE, RMSE, MAPE, and runtime are summarized in Table 2.The current section discuss the performance of LSTM-based model and SVM-based model assessing the advantages and disadvantages of the models.In terms of one-hour ahead load forecasting, it can be seen that based on the LSTM-based model, the models produce more accurate results in Case 1 than Case 2 As mentioned in the experimental setup section, Case 1 contains only 8 features for the input of the predictive model, which is much smaller than the number of features included in Case 2. Therefore, it shows that feature selection based on auto-correlation plays a fundamental role in LSTM-based models.The input features with high auto-correlation are more important than the number of input features.
As for one-day ahead peak and valley load forecasting, the prediction accuracy of Case 3 using the LSTM-based model is not significantly different from that of Case 4. The reason for this result is that the autocorrelation of daily peak and valley load shows a downward trend as a whole.At this point, it is not effective to select the input features on the peak auto-correlation in each cycle.
For the SVM-based model, it can be seen from the experimental results that the more input features provided to the model, the higher the prediction accuracy of the model.In summary, the feature selection focusing on high auto-correlation features can improve the accuracy of the LSTM-based model.Therefore, another contribution of this paper is the method of feature selection.The input of the model can directly affect the predictive performance of the LSTM-based model.Thus, in the feature selection stage, the auto-correlation of historical load data is used to select the time point with high correlation as the input feature.This method is better than directly selecting the historical load of a certain cycle in the past as an input feature.Additionally, this paper evaluates and compares the performance of LSTM-based models and SVM-based models through four cases.Using time series crossvalidation, each month of 2018 is used for validation of the model, and the monthly RMSE of the two models is recorded.
The comparison between a deep neural network prediction with a SVM model revealed that the trained LSTM model is more accurate than the SVM-based model in one-hour ahead load forecasting.In terms of one-day ahead peak and valley load forecasting, the performance of the SVM-based model is better.However, it is worth mentioning that the runtime of the SVM-based model is much shorter than the LSTM-based model, for both hourly load forecasting and daily peak and valley load forecasting.In fact, as illustrated in Table 2, the average runtime of the SVM-based model can be negligible, at 0.05 s, while the average runtime of the LSTM-based model is approximately 70 s.
In summary, the LSTM-based model is better at handling complex, unstable data based on sufficient training data.SVM-based model is suitable for load forecasting of small-scale datasets.In the case of sufficient data and the pursuit of high-precision load forecasting, the preferred choice is the LSTM-based model, while SVM-based model is a better choice when there is not enough training data or time cost is one of the main considerations.
Furthermore, when assessing technology for electricity demand forecast in buildings for day-ahead scheduling or short term prediction to implement demand response measures, training a deep neural network could result in a small if none advantage.In the case of commercial high efficient buildings, the prediction errors are relatively small and the accuracy is enough for an accurate overall assessment for a demand response aggregator to control the electricity generation or reduce the building electricity demand.
The forecast technique can be used to estimate valleys and peak consumption a day-ahead, allowing demand response aggregators to bid on the day ahead electricity market.Additionally, it could also support building managers to schedule a day ahead local CHP generators.The one hour ahead forecast will allow to have an accurate baseline to estimate the impact on DR measures on the buildings and for the optimal scheduling of local generators.
Although the model applied in this paper has good prediction results, it can be further improved to reduce the predicted and actual load difference.This paper only considers history load data for shortterm load forecasting.The article uses the Pearson and Spearman correlation coefficients to rule out the effects of outdoor temperature and date types on load forecasting, however, other factors can be considered, such as the amount of renewable energy produced or occupancy profiles.

Conclusions
The provision of energy system services plays a critical role for the decarbonization of the power system and the integration of renewable energies at local and system levels.However, the growing penetration of renewable and controllable loads require accurate load forecasting techniques.In this paper, a commercial building has been used as a test-bed for a set of forecasting algorithms using machine learning techniques.Besides the hourly energy demand forecast, a day-ahead peak and valley prediction has been trained on the historical data.The current work developed state of the art forecast models to predict the electricity demand and compare it with the daily valley and peak to dynamically optimize the CHP generator, the thermal storage and electricity demand implementing demand response measures.The novelty of the work is the development of three different prediction models that can be combined for the evaluation of flexibility.In future work, hybrid models, for example, combining multiple forecast techniques, may be tested to improve prediction accuracy.Additionally, a more accurate model will be employed to identify anomalies such as power outages and unscheduled maintenance and the prediction models will be used to compute a metric to assess the flexibility of the building and to forecast the impact of the demand response measures on the potential flexibility.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

F
.Pallonetto et al.

Fig. 1 .Fig. 2 .
Fig. 1.LSTM internal structure [36].(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

F
.Pallonetto et al.

Fig. 5 .
Fig. 5.The comparison the distribution of the box plot and the probability density function of the standard normal distribution [39].

Q 1 − 1 . 5 *
and Q 3 + 1.5 * , and data outside this range can be considered an outlier.

Fig. 7 .
Fig. 7.The number of occurrences of peak or valley load per day for 2018.

Fig. 9 .
Fig. 9.The auto-correlation coefficient of daily peak and valley load data.
This paper creates standard LSTM-based model and SVM-based model based on background work.The SVM-based model used in the experiment is created with the scikit learning package, and the LSTMbased model is developed using Keras.The detailed parameters of these two models parameter settings are as follows:

F
.Pallonetto et al.

Fig. 11 .Fig. 12 .
Fig. 11.The actual load and the predicted load in Case 1. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

F
.Pallonetto et al.

Fig. 13 .
Fig. 13.The actual load and the predicted load in Case 2.

Fig. 14 .
Fig. 14.The 2018 monthly RMSE for daily peak and valley load forecasting in Case 3.

Fig. 15 .
Fig. 15.The 2018 monthly RMSE for one-day ahead peak and valley load forecasting in Case 4.

Fig. 16 .Fig. 17 .
Fig. 16.The actual load and predicted load for daily peak and valley load forecasting in Case 3. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

ANNs
Artificial Neural Networks BES building energy simulation CHP combined heat and power DR Demand Response DSM Demand Side Management EMS energy management system FFNN Feed-Forward Neutral Network HVAC Heating, Ventilation and Air Conditioning LSTMs Long Short-term Memory Networks LTLF long-term power load forecasting MAE Mean Absolute Error MAPE Mean Absolute Percentage Error MTLF medium-term load forecasting RMSE Root Mean Squared Error RNNs Recurrent Neural Networks STLF short-term load forecasting SVM Support Vector Machines SVR Support Vector Regression TSO Transmission System Operator VSTLF very short-term load forecasting

Table 2
Comparison of models performances.