Machine learning-based algorithms to estimate thermal dynamics of residential buildings with energy flexibility

In the residential sector, the building heating system is an energy-intensive consumer. Heat pumps are energy-efficient devices to integrate renewable power into buildings and provide flexibility for energy systems. Heat pump controllers assist in the release of flexibility potentials of thermal inertia and storage while meeting residents ’ comfort. The heat controllers optimize the operation of building thermal dynamics which are stated by differential equations mathematically. The differential equations include dynamic thermal characteristics, i.e., thermal resistance and capacity, which are specified by estimation methods. The precision of the estimation methods affects the operation of heat controllers significantly. In this paper, the dynamic thermal characteristics of residential buildings are estimated using two grey-box models, i.e., the Continuous-Time Stochastic Model (CTSM) and Bayesian Optimization (BO), in R and Python software, respectively. Then, the estimated thermal characteristics are exported to UPPAAL-STRATEGO software to unlock the heat-to-power flexibility of heat pumps. The heat flexibility is generated using the probabilistic FlexOffer concept considering uncertain weather variables. Finally, the suggested approaches are examined on a 150 m 2 family house with four temperature zones. Based on the simulation results, the BO exhibits an average of 31% higher accuracy in the estimation of dynamic thermal characteristics than the CTSM. Also, the FlexOffer concept generates 39.03 kWh and 36.93 kWh energy flexibility for the residential building using the BO and the CTSM with a gap of 5.38%. Nomenclature*

In the residential sector, the building heating system is an energy-intensive consumer.Heat pumps are energy-efficient devices to integrate renewable power into buildings and provide flexibility for energy systems.Heat pump controllers assist in the release of flexibility potentials of thermal inertia and storage while meeting residents' comfort.The heat controllers optimize the operation of building thermal dynamics which are stated by differential equations mathematically.The differential equations include dynamic thermal characteristics, i.e., thermal resistance and capacity, which are specified by estimation methods.The precision of the estimation methods affects the operation of heat controllers significantly.In this paper, the dynamic thermal characteristics of residential buildings are estimated using two grey-box models, i.e., the Continuous-Time Stochastic Model (CTSM) and Bayesian Optimization (BO), in R and Python software, respectively.Then, the estimated thermal characteristics are exported to UPPAAL-STRATEGO software to unlock the heat-to-power flexibility of heat pumps.The heat flexibility is generated using the probabilistic FlexOffer concept considering uncertain weather variables.Finally, the suggested approaches are examined on a 150 m 2 family house with four temperature zones.Based on the simulation results, the BO exhibits an average of 31% higher accuracy in the estimation of dynamic thermal characteristics than the CTSM.Also, the FlexOffer concept generates 39.03 kWh and 36.93 kWh energy flexibility for the residential building using the BO and the CTSM with a gap of 5.38%.
Nomenclature** (In this table, the main notations of the mathematical models are stated.In some cases, to clarify the mathematical models, the notations are introduced right after the formulations in the body text)

Problem description and motivation
By increasing the environmental concerns about climate change, the EU Commission is committed to emitting 55% fewer greenhouse gases by 2030 and becoming climate-neutral by 2050.To fulfill the aims, different energy sectors, including residential, industrial, agricultural, and commercial, should be decarbonized gradually.Generally, decarbonization is addressed for the economy and energy systems.The framework to decarbonize the economy aims to design and monitor policies to achieve climate change targets on one hand and boost social growth and cohesion on the other hand [1].In energy systems, decarbonization aims to increase the penetration of low-carbon energy generation and a consequent reduction in the use of fossil fuels [2].This increases the dominance of renewable energies, i.e., wind, solar power, and biomass.In 2020, residential buildings consumed 36% of global energy demand and contributed to 37% of energy-related CO 2 emissions [3].Therefore, to achieve the Paris Agreement, net-zero buildings [4] and building decarbonization are critical issues.
In buildings, heating systems are energy-intensive consumers which contributed to 62.8% and 14.5% of EU household energy consumption for space heating and water heating in 2020, respectively [5].Energy flexibility refers to the capability of energy consumers to modify their energy consumption, including shift, curtailment, and adjustment, in response to external signals based on their comfort preferences, activities, and socioeconomic factors.Based on IEA-EBC Annex 67 [6] and research study [7], "the energy flexibility of buildings must be harnessed across a cluster of buildings or at a district scale to provide an aggregated amount that is sufficiently impactful for the operation of energy grids".
To unlock the energy flexibility of heating systems, heat controllers can optimize heat consumption in response to renewable power availability on the supply side.Controllers satisfy the residents' comfort while counterbalancing renewable power fluctuations.Mathematically, the heat controllers optimize the thermal dynamic of buildings by solving Ordinary Differential Equations (ODEs).The dynamic thermal characteristics, i.e., heat resistance and capacity, depend on the building component properties like insulation quality, envelope/wall material, and window dimension.The thermal parameters are specified for target buildings to design heat controllers.In this way, sensor data are used to train machine learning algorithms.The crux of the matter is that the estimated thermal dynamics affect the heat controller's performance.To estimate dynamic thermal characteristics, two major issues are pointed out as follows: N. Cibin et al.
(1) Building sensor data: a high volume of building sensor data is required to estimate the dynamic thermal characteristics.The data includes indoor air temperature, outdoor air temperature, solar irradiation, and heat energy consumption of the buildings.In Scandinavian countries, especially in Denmark, most buildings are supplied by district heating.Therefore, the buildings are equipped with radiators or floor heating.To calculate the energy consumption of radiators/floor pipes, more sensors should be installed to measure mass flow, inflow temperature, and outflow temperature.Generally, flow meters are expensive to install.Moreover, many residents are unwilling to install such sensors and reveal their occupancy patterns.As a result, it is quite complex to achieve such complete sensor data.(2) Estimation algorithms: apart from the sensor data, different methods are stated in the literature to estimate the dynamic thermal characteristics, including steady-state methods, dynamic methods, and active methods.Regarding the dynamic methods, whitebox, black-box, and grey-box models are surveyed in the literature.Some estimation algorithms show higher accuracy and robustness against noisy sensor data.
This paper takes advantage of (1) using a case study with complete sensor data (2) using two grey-box models to estimate the dynamic thermal characteristics.So, both the case study and the grey-box models are the positive points of the current study.These are the main motivations behind this study to develop machine learning-based algorithms for thermal dynamic estimation.Finally, the interactions between thermal dynamic estimation and energy flexibility of heating systems are investigated.

Literature review
In the literature, many studies have been conducted to discuss thermal dynamic estimation for residential buildings.Recently, heat controllers are more heeded to facilitate renewable power integration into heating systems.Many heat controllers optimize the ODEs of target buildings.The interactions of thermal dynamics and heat controllers are the cruxes of the matter.Regarding the thermal dynamic estimation, the building models are classified into three main categories as follows: 1. Steady-State Methods (S-SM) 2. Dynamic Methods (DM) 3. Active Methods (AM) The S-SMs are applied to studies where simplicity is a key factor and a significant amount of input data is available.These are common approaches in standard protocols, e.g., ISO 8990 [8] and ISO 9869-1 [9], in experimental measurements of buildings parameters.The average method and infrared thermography are two widely used approaches in this class [10].The former uses averaged measurement data as an approximation under steady-state conditions [11].The latter is addressed to estimate the thermal transmittance of building envelopes under stationary conditions [12].
In contrast to S-SMs, the DMs are introduced with more complexity and dynamics.In this method, the measurement data include more dynamic states and fluctuations in heat and temperature.Generally, the DMs are divided into three main categories as follows [13]: 1. White-Box Models (W-BM) 2. Black-Box Models (B-BM) 3. Grey-Box Models (G-BM) The W-BMs are analytical approaches that normally include physical models and mathematical formulations of buildings.The constant parameters of mathematical formulations must be estimated; therefore, the complexity of the model increases with increasing the number of temperature zones [13].Also, identification of the exact thermal dynamic is difficult due to data unavailability and noisy data [14].An iterative reduction methodology is developed to reduce the model complexity of residential buildings using W-BM [15].Meanwhile, due to model complexity, the computational time needed to solve the problem remains normally high.The B-BMs are known as mathematical model-free and data-driven approaches.These models are based on available data and are much easier to implement in comparison to W-BMs.The main limitation of B-BMs is that they need a large amount of building data to train machine learning algorithms.In practice, such sensors data may not be available due to limitations in sensors installation or residents' consent.As a result, the application is mainly dependent on data availability.Artificial Neural Networks [16], Genetic Algorithms [17], and Support Vector Machines [18] are well-known approaches that are proposed in recent studies.
The G-BMs are hybrid approaches that are a combination of W-BMs and B-BMs.The G-BMs are comprised of mathematical models and data-driven approaches to overcome the complexity of W-BM and the applicability limitation of B-BMs.The resultant building models from the G-BM approach are appropriate for heat controllers, especially Model Predictive Control (MPC) [19].Moreover, the G-BMs provide the most accurate and robust building models in comparison to W-BM and B-BM [20].
In contrast to S-SM and DM, the AMs are using artificial thermal loads to achieve (i) less computation time and (ii) less sensitivity to weather conditions [10].Harmonic thermal loads are addressed in a research study to estimate the thermal dynamics of buildings [21].A triangular thermal load is used to estimate thermal resistance, capacity, and conductivity parameters [22].Based on study results, the dependency on long input data and outdoor conditions decreases.To provide a general insight into the fundamentals of thermal dynamic estimation, Table 1 surveys the main characteristics of the aforementioned methods.
The thermal dynamics of buildings are normally used in heat controllers to provide joint heat-power flexibility for upstream energy networks.In a residential building, the heat demand includes space heating [38] and hot water consumption [39].To unlock heat flexibility, heat controllers encounter two different residents' preferences including fixed temperature setpoints and flexible temperature intervals [40].In the former, the controller aims to provide a fixed indoor temperature.As a result, low flexibility is expected from the residents' behavior.Adversely, in the latter, the heat flexibility stems from the lower and upper thresholds of indoor temperature.The thermal inertia of buildings provides flexibility to maintain the indoor temperature within comfort bound [41].In this state, when the heat network faces an energy shortage (excess) and/or when the energy price is high (low), the heat controller adjusts the indoor temperature close to the lower (upper) comfort bound [42].In addition, thermal storage plays a key role in providing heat flexibility.Thermal storage devices, e.g., water tanks, can store heat energy when the energy price is low and supply the heat demands when the energy price is high [43].
Recently, an advanced controller is designed by a Matlab-TRNSYS co-simulator to apply predictive strategy planning models on the HVAC systems of residential buildings [44].The MPC strategies are developed to harness the price and carbon-based energy flexibility of residential heating systems [45].The simulation results showed a 16% reduction in the operation cost and a 10% reduction in the emission production.The rule-based control approach is addressed to improve the energy flexibility performance of an Italian residential building [46].Based on the results, the operation cost and CO 2 emissions are reduced by up to 10% and 79%, respectively.The energy flexibility of residential buildings is evaluated in terms of comfort, capacity, efficiency, and shifting using short-term heat storage [47].The simulation results revealed that the different scenarios of modulation cause cost saving from 3% up to 10%.
Fig. 1 sketches the flexibility potentials of residential buildings in terms of space heating and domestic hot water consumption.

Paper contributions and organization
This study focuses on the interactions of thermal dynamic accuracy and energy flexibility of building heating systems.Also, the probability distribution of building energy flexibility is evaluated under uncertain weather conditions.To achieve the aims, this paper compares the competence of two grey-box models in estimating the thermal dynamic of residential buildings.The Continuous-Time Stochastic Model (CTSM) and Bayesian Optimization (BO) are applied to estimate the thermal parameters of the residential dwelling, including thermal resistance and capacity.Thermal dynamics are presented in the form of ODEs and developed mathematically to address the heat flux between internal envelopes of buildings with multi-temperature zones.The CTSM and BO are coded in R and Python software, respectively.The estimation results of the thermal dynamics are exported to the UPPAAL-STRATEGO software to generate FlexOffers for individual heat pumps.Also, it evaluates the impacts of the thermal dynamics estimation accuracy on the energy flexibility of the building heating system.The FlexOffers unlock the heat-to-power flexibility of the heat pumps by considering optimistic and pessimistic energy consumption patterns, i.e., lower and upper residents' comfort thresholds.All in all, the main contributions of the proposed study can be stated as follows: 1. Developing machine learning-based algorithms for thermal dynamic estimations for buildings with different temperature zones.
The heat flux between internal envelopes is addressed in the thermal dynamic model.2. Comparing the accuracy of grey-box methods, i.e., the CTSM and BO, in estimating the thermal dynamic of residential buildings.
The constant coefficients of ODEs are specified using the two methods and the results are compared.3. Unlocking joint heat-power flexibility of residential heat pumps through generating probability distribution of energy flexibility, called probabilistic FlexOffers, in the UPPAAL-STRATEGO software considering weather conditions uncertainty.The FlexOffer generates the probability distribution function of energy flexibility under uncertain weather conditions regarding minimum and maximum household energy consumption patterns.4. Investigating the impacts of thermal dynamics estimation accuracy on energy flexibility extraction from building heating systems.
The energy flexibility of a Danish test house is quantified using the estimated thermal dynamics and the results are compared for the two grey-box models.
The rest of the paper is organized as follows.In section 2, the problem methodologies are explained for the CTSM, BO, and FlexOffer generation approaches.Section 3 presents the case studies, discussion, and simulation results.Finally, Section 4 concludes the study and states the main limitations as well as suggestions for future works.

Problem methodology
In this section, the fundamentals of the suggested approaches are formulated mathematically.First, the ODEs of thermal dynamics are stated in section 2.1.This section elaborates on the mathematical formulations of thermal dynamics in buildings.The CTSM and BO are explained in sections 2.2 and 2.3, respectively.The two estimation methods are formulated mathematically.Finally, section 2.4 illustrates the probabilistic FlexOffers.It describes how probabilistic energy flexibility is generated under uncertain weather forecasts.

Thermal dynamics of buildings
In this study, the dynamic thermal characteristics are referred to the thermal behavior of building components when it is subject to variable conditions, e.g., variable heat flow and boundary temperatures.The thermal dynamic model describes the mathematical formulations of the thermal behavior of the building which is stated in terms of ODEs.The ODEs are comprised of a set of dynamic thermal characteristics.The characteristics are classified into constant coefficients, i.e., thermal resistance and capacities, and thermodynamic variables, i.e., the internal temperature of rooms and heat energy.The constant coefficients are estimated for buildings using two grey-box models.Then, they are used to calculate the thermodynamic variables of the building under different weather conditions.The thermal dynamic model of the buildings is presented in the form of the RC network based on frequent sensor data of heat consumption, indoor temperature, and weather conditions [48].In the RC model, different parts of the buildings are described by specific elements, e.g., heat resistance and heat capacity.Considering a building with N rooms, in which n is the index of rooms, n ∈ 1, 2, …,N, it is assumed that the target room n is surrounded by N − 1 internal envelopes and K external envelopes.The internal and external envelopes indicate the walls surrounded by other rooms and outdoors, respectively.Therefore, the building structure can be depicted in Fig. 2.
Mathematically, the thermal dynamics of room n can be stated as the following ODEs [49]: Schematic structure of the buildings with internal and external envelopes: room n at time t.
N. Cibin et al. C The ODEs (1)-( 3) describe the θ i θ e θ h approach of the RC network in which θ i , θ e, and θ h stand for the indoor, envelope, and heater temperature, respectively.C i , C e , and C h are the heat capacities of indoor air, envelope, and heater, respectively.R i,h is the heat resistance between the indoor air and heater, and R i,e is the heat resistance between the indoor air and envelopes.π S and π h are solar power and heat consumption, respectively.ρ n states the fraction of solar power absorption and θ a is the ambient temperature.
In Eq. ( 1), the absorption of solar power is presented by the third right term.In Eq. ( 2), the second and third terms explain the heat exchange of room n with internal and external envelopes, respectively.In Eq. ( 3), the heat consumption of the heater affects the indoor temperature.Note that the heat consumption of the heater (radiators) is extracted from the following equation: where θ in , θ out describe inflow and outflow temperatures, respectively; c m is the specific heat capacity of thermal mass and m n is mass flow.
Eq. ( 4) describes the heat consumption of the heater based on mass flow and inflow/outflow temperature.

Continuous-Time Stochastic Model
In the ODEs, dynamic thermal characteristics, including heat resistance, heat capacity, and the fraction of solar power absorption, are dependent on the physical characteristics of the building, e.g., room dimension, quality of insulation, and envelope material.Therefore, the set of constant coefficients which are subject to estimation is stated as follows: To estimate the set Φ, the sensor data of the building is measured as follows: The set of sensor data Ψ is used to estimate the set of thermal coefficients Φ.
The CTSM is a grey box that combines the physical model of buildings with a statistical approach.The physical model includes the three ODEs Eqs. ( 1)-(3).The statistical approach, the so-called data-driven method, uses the information embedded in the sensor data.The data-driven approach addresses the discrete-time measurement as follows [48]: where k is the point in the measurement time, ψ k is the measurement (sensor) data which is the indoor temperature, and ε k is the measurement error.
Afterward, the CTSM uses the maximum likelihood function to estimate the thermal parameters.Let us assume N measurements as follows: Then, the likelihood function is stated as a joint probability density function: where p(ψ k |ψ k− 1 ,λ) is a conditional density stating the probability of measurement ψ k given the previous observations and the parameters λ and p(ψ 0 |λ) denote the initial conditions.Consequently, the maximum likelihood estimates of the thermal parameters are obtained as follows: Due to the linear model, the density function Eq. ( 10) is considered a Gaussian.Kalman filter can be used to calculate the likelihood function.The abovementioned structure is discussed theoretically in the research paper [48].Also, the CTSM software is developed by the Technical University of Denmark and is publicly available [50].

Bayesian Optimization approach
Sequential Model-Based Global Optimization (SMBO) algorithms have been widely used in many applications [51,52] where the evaluation of the fitness function is expensive.In SMBO, the fitness function f: R n → R is approximated by a surrogate probability distribution p(y|x) cheaper to evaluate.Fig. 3 shows the pseudo-code for a generic SMBO algorithm.In the algorithm, x represents the set of coefficients of the ODEs that is subject to estimation, equivalent to set Φ stated in Eq. ( 5), while f (x) represents the Root Mean Square Error (RMSE) between the estimated temperatures computed through the ODEs with coefficients x and the observed data.Parameter z is the number of iterations which is set to 10,000 in this study.The term GetBestY is a generic function that depends on the SMBO variant and it returns a scalar value among the f (x) collected in Δ.The core of the SMBO algorithm is the expected improvement which is stated in Line 5 of the pseudo-code.
Classical Bayesian optimization algorithms [53] use Gaussian processes to model p(y|x) (see Line 5, Fig. 3), while in more recent methods such as Tree-structured Parzen Estimator (TPE) [54], the model is built by applying the Bayesian rule on p(y|x) = p(x|y)p (y)/p(x).Here, TPE is used and the conditional probability p(x|y) is defined as follows: In TPE, the GetBestY in line 4 is replaced with some quantile γ of the observed y values in Δ, so that p(y < y*) = γ.In our case, parameter γ is set γ = 0.25.Furthermore, the expected improvement in Line 5 can be simplified as follows [54]: By maximizing Eq. ( 12), it is possible to retrieve another set x* of ODEs' coefficients, i.e., set Φ stated in Eq. ( 5).Note that ℓ(x) and g (x) are arbitrarily distributions depending on the observed x values in Δ.For ℓ(x) and g(x), an independent Parzen estimator is used for each coefficient.Since y* provides a partition of Δ, i.e., Δ ℓ and Δ g , the Parzen estimators are constructed by summing and normalizing Gaussian distributions centered in each x in Δ ℓ and Δ g , respectively.The standard deviations for the Gaussian distributions are set to the greater distances to the left and right neighbors but are clipped to remain in a reasonable range.

Probabilistic FlexOffer generation
"FlexOffer is the concept that has been developed in the EU FP7 project MIRABEL [55,56].It allows exposing demand and supply loads with associated flexibilities in time and amount for energy trading, load balancing, and other use cases.FlexOffers are generic entities and can accommodate various types of consumers (e.g., electric vehicles, heat pumps, household equipment, industry) and producers (discharging electric vehicles, solar panels)".A FlexOffer is characterized by a maximum and a minimum amount of energy that can be consumed (or provided) by a prosumer, and this flexibility is what is traded with other peers, including prosumers and energy service providers.Probabilistic FlexOffers are an interesting extension to this concept, where stochastic variables, e.g., weather conditions, are also kept into consideration during the FlexOffers generation process.Therefore, in this study, the energy bounds are no more deterministic but are described by a Probability Distribution Function (PDF).Regarding the probabilistic bounds, a success function is defined as follows: where the Min CDF and Max CDF describe minimum and maximum consumption distributions for energy input x associated with Cumulative Distribution Function (CDF).The success function states the probability that the consumption schedule associated with the given FlexOffers can be respected by the offering party.Fig. 4 gives a schematic insight into the probabilistic FlexOffer approach.
The FlexOffer concept is coded in UPPAAL-STRATEGO."UPPAAL is an integrated tool environment for modeling, validation, and verification of real-time systems modeled as networks of timed automata and is developed in collaboration between the Uppsala Fig. 3. Pseudo-code for Sequential Model-Based Global Optimization (SMBO) algorithms.

N. Cibin et al.
University, Sweden, and the Aalborg University, Denmark" [57]."UPPAAL-STRATEGO facilitates the generation, optimization, comparison as well as consequence and performance exploration of strategies for stochastic priced timed games in a user-friendly manner" [58].
In this study, the building thermal dynamic model is mapped into timed automata and imported into UPPAAL-STRATEGO where the built-in query system allows the computing of the two consumption schedules.The consumption schedules maximize and minimize the electric energy consumption of the heating system, keeping into consideration the comfort constraints set by residents.The consumption strategy optimizations are constrained by the upper and lower thresholds of the indoor temperatures of the building rooms.Therefore, the queries used to generate these schedules are described as the following: In lines 1 and 2 of Fig. 5, two strategies are computed that respectively minimize and maximize the expected energy consumption, π h (kWh), for a time horizon equal to H (t = 1,2, …, H).Function E denotes the expectation operator.Then, in lines 3 and 4, the minimum and maximum values of the expected energy consumption are extracted by simulating the system evolution following the two strategies computed in the previous steps.The two simulations are run N times to evaluate the PDF of the FlexOffer energy bounds.
To address the weather uncertainty, the baseline forecasted data, including the ambient temperature and solar irradiation, is blended with stochastic noise with limited error.Hereby, the Gaussian Distribution is addressed to generate the weather data stochasticity.
Considering the aforementioned facts, the suggested approach takes the following steps to generate FlexOffers: Step 1. Receive the weather data forecast, including θ a and π S , for the time horizon H, e.g., the next 24 h, from the meteorological office.
Step 2. Add weather data stochasticity by a PDF, e.g., Normal Distribution with mean μ and variance σ 2 , to the forecasted data.
Step 3. Set the indoor temperature θ n i to the lower threshold of residents' comfort and calculate the minimum energy consumption of the heating system as min.E (π h ) (optimistic energy consumption pattern).
Step 4. Set the indoor temperature θ n i to the upper threshold of residents' comfort and calculate the maximum energy consumption of the heating system as max.E (π h ) (pessimistic energy consumption pattern).
Step 5. Fit the PDF of the energy consumption for the minimum and maximum energy consumption patterns, i.e., Steps 3 and 4, to generate FlexOffers.
Step 6. Calculate the energy flexibility potential of the heating system as the existing gap between the minimum and maximum energy consumption patterns: Energy Flexibility= (max.E (π h ) -min.E (π h )) corresponding to Eq. (13).
Finally, the whole procedure of the suggested approaches, from sensor data collection to generate FlexOffers, is described in Fig. 6.

Numerical study
In this section, the case studies and simulation results are presented.First, the test building, physical characteristics, and input data

Case study and input data
The test residential building is a detached Danish family house with four rooms, with an overall size of 150 m 2 .The building is comprised of a kitchen, living room, bedroom, and bathroom.Based on [59], building materials and parameter values are chosen following the Danish building regulations from 2010 [60].Rooms 1,2, and 4 are wood flooring and room 3 is light concrete flooring.The height of the rooms is 2.5 m.Room 1 has two windows with dimensions of 6.5 × 2 m 2 and 1 × 2 m 2 ; room 2 has two windows with dimensions of 4.5 × 2 m 2 and 1 × 1 m 2 ; room 3 has one window with dimensions of 1 × 2 m 2 and room 4 has two windows with dimensions of 1 × 2 m 2 and 1 × 1 m 2 .The windows are double-layered with 80% transparency of the provided dimensions.The material of the walls is lightweight concrete plus insulation.The ceilings are made of gypsum and insulation.The model is extracted from the non-proprietary, object-oriented, equation-based modeling language Modelica [59].Fig. 7 describes the floor plan of the test building with 4 rooms.In this study, in addition to the original 4-room plan, the internal envelopes between rooms are removed and a one-room building model is created.
The input data include indoor temperature, mass flow, and inflow/outflow temperature as well as the weather data, including ambient temperature and solar irradiation.The data is recorded for one month (30 days) on a minute basis.Fig. 8 depicts a part of the input data for one week on an hourly basis.The input data include indoor air temperature, solar irradiation, and outdoor temperature.
To investigate the competence of the estimation approaches, the test building is evaluated in two case studies: 1.The building with four temperature zones, i.e., the original floor plan.
2. The equivalent single-room model of the building.
The former is normally used to design heat controllers.The temperature setpoints of rooms are defined separately based on occupancy patterns and residents' comfort.The temperature zones should be identified individually to unlock heat flexibility based on  occupancy patterns.In the latter, the test house is considered a single-room building with an average temperature for the whole internal air.This model is addressed in heat network studies where the whole building is observed as a single heat consumer despite the different internal temperature zones.The assumptions of the case studies are stated as follows: 1.The heat pump with 2.5 kW nominal power and a Coefficient of Performance (COP) of 3.7.

Simulation results
In this section, two types of simulation results are discussed.First, the competence of the CTSM and BO approaches are compared to estimate the constant coefficients for the 4-rooms and single-room buildings.Afterward, the estimated coefficients are used to generate FlexOffers for the heating system.Both the CTSM and BO are trained with 15 days of input data and tested over the next 15 days.The results of thermal coefficient estimation for the 4-room model are stated in Table 2. Also, the graphical results of the thermal coefficient estimation are described in Fig. 9.As the figures reveal, the estimation accuracy of the two models varies for different rooms.To elaborate on the estimation accuracy, five conventional error criteria are calculated in Table 3.The error criteria include Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Mean Bias Error (MBE), Normalized Mean Bias Error (NMBE), and Coefficient of Variation of The Root Mean-Square Error (CVRMSE).
In Table 3, the winner cases are pointed out with bold fonts.Based on the table, in some cases, the BO outperforms the CTSM and vice versa.However, the total error (the sum of the absolute values for each column) is typically lower for the BO than the CTSM.The BO exhibits 11.43%, 9.39%, 65%, 63.52%, and 4.74% better accuracy in MAE, MAPE, MBE, NMBE, and CVRMSE, respectively.This could be explained by the fact that the BO allows to jointly learn (and therefore exploits potential correlation among the rooms) the coefficients for all the rooms and the different temperatures simultaneously.Moreover, as a further advantage, the BO does not require to specify an initial starting point which is a requirement for the CTSM.Fig. 10 explains the box plots of the estimation error for the 4-room model.As can be seen, the BO shows lower error variances than the CTSM.
The estimated coefficients for the one-room model are described in Table 4.The graphical comparison between the forecasted and actual temperatures is presented in Fig. 11.Note that the average temperature of the single-room model is calculated based on the simple average of four rooms.As the graphs reveal, both the CTSM and BO exhibit lower accuracy than the 4-room model.The reason is that the average temperature does not describe the precise distribution of indoor temperature in different rooms.Although the two estimation models track the actual temperature reasonably well, the estimation error increases at some points.To elaborate on this issue, Table 5 explains the error values.Based on the results, the BO shows 6.17%, 8.17%, and 7.38% better accuracy than the CTSM in MAE, MAPE, and CVRMSE, respectively.In contrast, the CTSM exhibits 7.61% and 3.33% better estimation than the BO in MBE and NMBE, respectively.The interesting point is that the CTSM improves the estimation performance in the one-room model in comparison     to the 4-room model.In this section, the impacts of the estimated thermal dynamic on the building's heat flexibility are evaluated.The two building models, i.e., the 4-room and one-room, are imported into UPPAAL software to generate FlexOffers.Fig. 12 compares the temperature evolution and heat consumption of the 4-room model in the minimum energy consumption.Therefore, the indoor temperature is scheduled near the lower threshold of 18 • C. The optimal values of energy consumption and indoor temperature are depicted for each room individually.The strategies are optimized for the next 24-h horizon.
Fig. 13 describes the temperature evolution and the heat consumption for the maximum energy consumption.By comparing the graphical performances of the CTSM and BO models, two interesting points can be demonstrated.First, in the minimum energy consumption (Fig. 12), the temperature graph of the BO is more fluctuating than the CTSM.Second, in the maximum energy consumption (Fig. 13), the heat consumption of the CTSM is more fluctuating than the BO.The two abovementioned issues show that the thermal coefficients extracted by the BO have more correlation with the weather conditions.As a result, in Fig. 12, the indoor temperature of the BO model is much more affected by ambient temperature and solar irradiation.A similar pattern is seen in the heat consumption of the BO model in Fig. 13.The heat consumption has a downward trend from morning to midday; consequently, the minimum heat consumption coincides with the high solar irradiation and the highest ambient temperature at hour 12.The UPPAAL optimizer minimizes heat consumption to prevent violating the upper threshold of the indoor temperature.Although the heat consumption of CTSM follows a similar downward trend, it exhibits more fluctuations than the BO.
These discrepancies are even more evident if the weather uncertainties are taken into consideration through probabilistic Flex-Offers.To elaborate more on this issue, Fig. 14 explains the PDF and success functions of the FlexOffers considering the weather condition stochasticity [61].Table 6 presents the mean and variance of the fitted normal distributions.Regarding the 4-room model, the mean value of the CTSM is 2.26 kWh (2.83%) and 4.37 kWh (3.68%) lower than the BO for minimum and maximum energy consumption patterns, respectively.In contrast, the BO exhibits more variance than the CTSM in both energy consumption patterns.It confirms that the thermal dynamics of the BO model are more affected by the weather conditions than the CTSM.
In the one-room model, two interesting points are seen.First, the variance of the PDFs decreases noticeably in comparison to the 4room model.The main reason is that the correlation between indoor temperature and weather conditions is lost due to the oversimplification of the one-room model.Second, the existing gaps between the mean values of the CTSM and BO models increase by 7.64 kWh (8.90%) and 33.12 kWh (18.81%) for the minimum and maximum energy consumption, respectively.
Finally, the flexibility potentials of the building models are shown in Fig. 15.The energy flexibility of the building models is quantified based on the mean values of the PDFs generated by FlexOffer.Based on the graph, the BO and CTSM provide 39.03 kWh and 36.93 kWh energy flexibility in the 4-room model, respectively.For the one-room model, the energy flexibility increases to 81.28 kWh and 55.8 kWh, respectively.While there is just a 5.38% difference between the flexibility of the CTSM and BO in the 4-room model, the flexibility gap increases to 31.34% for the one-room model.The results show that both the CTSM and BO provide consistent flexibility potentials in the 4-room model; adversely, there is a wide gap between the flexibility potentials in the one-room due to the building model simplification.In this state, the BO shows a higher error in comparison with the CTSM.

Discussions, limitations, and future works
This study compared the competence of the CTSM and BO in estimating the dynamic thermal characteristics of buildings.The probabilistic FlexOffer concept was developed to generate energy flexibility under stochastic weather conditions.Through this, the key results can be surveyed as follows: (1) The BO exhibited higher estimation accuracy for thermal dynamic coefficients.In the 4-room model, the BO showed 11.43%, 9.39%, 65%, 63.52%, and 4.74% better accuracy for MAE, MAPE, MBE, NMBE, and CVRMSE, respectively.In the one-room model, the CTSM obtained 7.61% and 3.33% more precise estimations than the BO for MBE and NMBE, respectively.The estimation results confirmed that both estimation algorithms provide accurate and competitive results for building thermal characteristics.
(2) The dynamic thermal characteristics extracted from the BO model had more correlation with weather conditions in comparison with the CTSM.The temperature evolution of the CTSM model presents more robust behavior against weather condition fluctuations.(3) The computation time of both BO and CTSM is quite competitive.The suggested CTSM calculated thermal dynamics for each room individually.Hereby, the thermal coefficients are calculated in approximately 10-12 min for each room using an Intel CPU Core i7 at 2 GHz and 16 GB of RAM (less than 48 min for four rooms in total).In the BO, the estimation approach was converged for four rooms simultaneously in less than 1 h.(4) The FlexOffer concept could quantify the energy flexibility of the building heating system.The FlexOffer generated heat flexibility to meet occupants' comfort bound, including the lower and upper thresholds of indoor temperature.In the 4-room model, the BO and CTSM provided 39.03 kWh and 36.93 kWh energy flexibility with a gap of 5.38%.In the one-room model, the flexibility gap between the two estimation algorithms increased to 31.34%.
The main limitation of the current study stems from the building input data.In buildings with water-sources heat pumps, the required data include mass flow, inflow/outflow temperature, and indoor temperature.Generally, mass flow sensors and control devices are costly installations.Also, many occupants are reluctant to record the indoor temperature due to revealing the occupancy patterns.For these reasons, although, the current study used fined-grained building data to extract thermal dynamics, one may work on new algorithms to extract the thermal dynamics by using coarse-grained data.This is a key issue for future research.The suggested probabilistic FlexOffer concept in UPPAAL gives a software tool to quantify energy flexibility in heating systems with uncertain variables.This approach is not limited to building heating systems and can be extended to other flexible energy systems, e.g., commercial refrigerators with uncertain electricity prices.
Note that this study compared the results of the CTSM and BO approaches in one test building: therefore, the obtained results are specific to the target case and may not be interpreted as general outcomes.

Conclusion
In this paper, a comparative analysis is conducted to estimate the dynamic thermal characteristics of residential buildings using the CTSM and BO algorithms in R and Python software, respectively.The thermal dynamic model is presented using three-state ordinary differential equations, including indoor air, envelope, and heater temperatures.The estimation algorithms are examined on a highfidelity four-temperature-zone building and its single-room equivalent.Afterward, the estimated thermal dynamics are exported to UPPAAL-STRATEGO software to provide heat-to-power flexibility.The software develops the probabilistic FlexOffer concept to generate probability distributions of energy flexibility under stochastic weather conditions.
The simulation results show that both the CTSM and BO algorithms are competent to estimate the thermal dynamics.Regarding the estimation accuracy, although the CTSM obtains better accuracy in some points, the BO shows around 31% lower absolute error than the CTSM.The CTSM and BO converge to the optimal solutions in less than 60 min.
To generate FlexOffers, both the CTSM and BO algorithms unlock the energy flexibility of the buildings while meeting the lower and upper thresholds of residents' comfort.The BO shows more correlation between indoor temperature and weather fluctuations.In contrast, the CTSM presents more robustness against outdoor conditions.It means that the estimated indoor temperatures using the CTSM show fewer fluctuations in response to outdoor weather variations.To quantify the energy flexibility of the heating system, the probability distributions of energy flexibility are generated using the FlexOffer concept under weather conditions stochasticity.The results reveal that the BO and CTSM can provide up to 39.03 kWh and 36.93 kWh energy flexibility with a gap of 5.38%.
The main limitation of the current study emanates from building data scarcity.In real applications and living lab studies, often limited indoor temperature and/or mass flow data are available regardless of the number of temperature zones.Therefore, future works will focus on developing coarse-grained thermal dynamics than fine-grained ones.Although the FlexOffer approach is examined on the residential heating system, other energy systems, e.g., commercial refrigerators and ice banks, are potential candidates to unleash energy flexibility with uncertain electricity prices.

eh
Heat capacity of indoor air (kWh/ o C) C n Heat capacity of walls/envelopes (kWh/ o C) C n Heat capacity of heater (kWh/ o C) R n i,e Heat resistance between indoor air and walls/envelops ( o C/kW) R n i,h Heat resistance between indoor air and heater ( o C/kW) R na e Heat resistance between envelope and outdoor ( o C/kW) R nz e Heat resistance between common envelopes between rooms n and z ( o C/kW) θ n h Temperature of heater ( o C) θ n e Temperature of walls ( o C)

Fig. 1 .
Fig. 1.Heat flexibility of residential buildings stems from upper and lower thresholds of residents' comfort.

Figure: 6 .
Figure:6.The integration of suggested approaches to generate FlexOffers for residential buildings.

Fig. 7 .
Fig. 7.The original floor plan of the building with 4 temperature zones.

2 .
The initial indoor temperatures of the 4 rooms are [20.2,21.0, 19.8, 20.0] • C. 3. The upper and lower thresholds of the indoor temperature are considered 18 • C and 22 • C, respectively.

Fig. 10 .
Fig. 10.Box plots of the estimation error for the 4-room model (a) CTSM (b) BO (the red points are the outliers).(For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Fig. 11 .
Fig. 11.Comparison of measured and predicted room temperatures for the one-room model between days 15-30 (a) CTSM (b) Bayesian Optimization.

Fig. 12 .
Fig. 12. Temperatures evolution and heat consumption of the 4-room building model in the minimum energy consumption pattern (a) CTSM [61] and (b) BO.

Fig. 13 .
Fig. 13.Temperatures evolution and heat consumption of the 4-room building model in the maximum energy consumption pattern (a) CTSM [61] and (b) BO.

Fig. 14 . 6 Fig. 15 .
Fig. 14.Computed probabilistic FlexOffer for different estimated thermal dynamics* (*Note: The red and blue lines describe the probability density on the left axis and the green line shows the probability values of the success function on the right axis).(For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Table 1
Key characteristics of the estimation methods for building thermal dynamics.

Table 2
The results of thermal coefficients estimation for the 4-room model conducted by the CTSM and BO.

Table 3
Error criteria of thermal dynamic estimation by BO and CSTM for the 4-room model.

Table 4
The results of thermal coefficients estimation for the one-room model conducted by the CTSM and BO.

Table 5
Error criteria of thermal dynamic estimation by BO and CSTM for the one-room model.(winners are pointed out with grey background).