Economic heat control of mixing loop for residential buildings supplied by low-temperature district heating

Increasing the penetration of Renewable Energy Sources (RES), the heat pumps are an alternative solution to facilitate the integration of RES into district heating. To hedge against the intermittency of RES, the flexibility potentials of the thermal inertia of buildings are unlocked and integrated into power systems. This paper suggests a novel structure to control the mixing loop of residential heating systems supplied by electrically operated district heating. The Economic Model Predictive Control (EMPC) is proposed to optimize the heat demand of the buildings in response to electricity market price. The EMPC controls the temperature and flow rate of forward/ return water in the mixing loop to minimize the energy consumption cost of households. The thermal dynamics of the buildings are modeled by 3-state differential equations to integrate the flexibility potentials of thermal inertia into the district heating. The approach optimizes the heat consumption of the building with multi- temperature zones. The Continuous-Time Stochastic Model (CTSM) is addressed to determine the thermal dynamics of the building using sensor data. Finally, a 150 m 2 Danish test house with four temperature zones is simulated. The simulation results show that the suggested approach not only minimizes the energy consumption cost but also provides flexibility for the Danish Electricity Market.


Problem description and motivation
In recent years, the penetration of Renewable Energy Sources (RES) increased from 44% (2015) to 55% (2020) in the Danish Electricity Market [1] and is scheduled to be carbon-free by 2050. In Denmark, around 65% (2020) of residential buildings are supplied by district heating. Therefore, the Danish district heating shows considerable potentials to facilitate the integration of RES into the heating systems. To overcome the intermittency and volatility of RES, the power-to-heat flexibility of district heating consumers should be unlocked. As a result, smart heat controllers are required to fulfill the flexibility potentials of the heating system. In electricity markets with high RES penetration, there is a strong correlation between RES availability and electricity price. In this way, the economic controllers not only reduce the energy consumption cost but also provides demand flexibility for the power systems. In modern district heating, the heat demand is optimized by a mixing loop to increase/decrease the energy efficiency/heat loss in the heat distribution network. In recent years, prominent research studies have been conducted to unlock the power-to-heat flexibility of residential heat pumps. In Ref. [2], a cooperative mathematical model is presented to optimize the flexibility potentials for a group of buildings connected to a shared heat pump. The research study [3] proposes a flexible control approach for heat pumps in response to spot market and time-of-use electricity prices. The study [4] characterizes the flexibility potentials of single-family heat pumps not only to provide demand response but also to meet energy demand and thermal comfort constraints. The common feature of the studies in the literature is that the proposed approaches mainly concentrated on the general flexibility potentials of heat pumps in response to electricity price and flexibility requirements. In this way, barely any heat controller is suggested to elaborate on the optimized mixing of forward and return water in residential buildings connected to district heating. Therefore, this is the main challenge of this study to design a heat controller for the mixing loop of residential buildings in response to variable electricity prices.

Literature review
In the literature, the district heating and heat pumps have attracted much attention in north European countries, e.g. Denmark [5], Sweden The flexibility of thermal inertia is reflected in the thermal dynamics of buildings [12]. The heat transfer between indoor air, envelopes, and the outdoor environment play a key role in the heat demand of buildings. The indoor temperature is adjusted by heat controllers to meet the residents' comfort bound [13]. Considering lower and upper thresholds for the indoor temperature, the heat controllers unlock the power flexibility of the thermal inertia. As the difference between the upper and lower thresholds of the indoor temperature reduces, the flexibility potentials of the thermal inertia decrease. The flexibility of the heat carrier is unlocked by adjusting the key parameters of the heat distribution network [14]. In this way, the supply temperature, flow rate, and pressure are the key variables [15]. In the literature, the flexibility of heat carrier is discussed for the half network [16], i.e. forward water pipes, and the whole network [17], i.e. both the forward and return water pipes. Although the flexibility potential of the heat network is greater than the single and/or segregated buildings, the aggregated buildings can provide as great flexibility as the distribution network. The flexibility of thermal storage stems from the thermal storage devices, e.g. water tanks [18] and Phase Change Materials [19]. Thermal storage is divided into three main classes as Sensible Heat Storage [20], Latent Heat Storage [21], and Thermochemical Storage [22]. Sensible Heat Storage, e.g. water and reinforced concrete, is a simple and inexpensive method of heat storage in which the heat energy is stored in solid or liquid states of matter [23]. In this way, the energy is stored/released by increasing/decreasing the temperature of the medium. Latent Heat Storage, e.g. Paraffins, salt hydrates, is the most efficient method of thermal storage in which the energy-storing/-releasing changes phases between solid-solid, solid-liquid, or liquid-gas [24]. In Thermochemical Storage, e.g. Manganese oxide and ammonia, the energy is absorbed/released by endothermic/exothermic chemical reactions [25]. The flexibility potential of this technology is strongly dependent on the extent of conversion and heat of the reaction.
The heat controllers are classified into two groups, including classic and advanced controllers, from viewpoint of demand flexibility [26]. The classic controllers aim to make a balance between heat demand and supply [27]. In this method, the flexibility requirements of the supply side are not satisfied. In contrast, the advanced controller is devised to incorporate the technical requirements of the energy systems, e.g. flexibility requirements, into the demand-supply balance [28]. The advanced controllers can optimize the key variables of the thermal network, including supply temperature, the flow of carrier, and pressure, in response to the flexibility requirements. The flexibility requirements may refer to the availability of RES, energy price, and technical problems in the heat generation/distribution systems. Recently, the Model Predictive Control (MPC) is widely accepted as one of the most efficient controllers in district heating studies [29]. The MPC makes it possible to use the updated data of weather, e.g. ambient temperature, and energy price, to optimize the heat consumption [30]. From the viewpoint of the control level, the heat controllers include central [31] and distributed units [32]. The central controllers are located on the supply side, e.g. the heat network, where the control variables are mainly supply temperature [33] and differential pressure [15]. Adversely, the distributed controllers are applied on the demand side, e.g. the buildings, to adjust the heat consumption and mass flow [34].
Generally, the district heating generations are classified into four categories. The first generation (1GDH) is dating back between 1880 and 1930 when steam was considered as the heat carrier [35]. Later between 1930 and 1980, in the second generation (2GDH), the heat carrier with more than 100 • C temperature was transformed by concrete ducts [36]. In the third generation of district heating (3GDH), the temperature of the heat carrier is lowered below 100 • C [37]. This is the prevailed generation of district heating between 1980 and 2020. Recently, some European countries are pioneers to establish the fourth generation of district heating (4GDH). This generation aims to lower the temperature of the heat carrier by around 50-60 • C [38]. Energy efficiency [39] and low heat loss in the distribution network [40] are the main advantages of the fourth generation. It leads to a transition from high-temperature district heating with the supply temperature around 100 • C to low-temperature district heating with the supply temperature between 50 and 60 • C [41]. To facilitate the development of the fourth generation, the Mixing Loop Control (MLC) plays a key role on the demand side. The MLC is a distributed-advanced control approach to make a hydraulic balance in the district heating. The MLC adjusts the mass flow and temperature of mixing hot water, including the forward water from the district heating and the return water from the consumers, to minimize the energy consumption cost. The MLC facilitates the use of lowtemperature district heating. Besides, it decreases the energy loss in the heating pipes of the distribution networks. In Ref. [42], the MLC is designed using Reinforcement Learning for the heating system of residential buildings. The result shows that if the parameters of the Q-learning are well-tuned, the Reinforcement Learning controller performs better than the industrial controllers. The economic analysis shows that the piping and fitting costs of the low-temperature district heating are relatively lower than the high-temperature district heating [43]. In Ref. [44], low-temperature district heating is suggested as a practical solution to facilitate the integration of renewable heat sources. Reference [45] investigates the power-to-heat flexibility potentials of low-temperature district heating. Finally, the research study investigates the impact of electronically controlled valves for mixing forward and return water in residential buildings [46]. The study concludes that the electronically controlled valves reduce energy consumption up to 40% in comparison with the mechanical valves.

Contributions and organization
Based on the extensive review, what is missing in the literature is to optimize the heat consumption of the buildings supplied by the lowtemperature district heating in response to RES availability. To narrow the gap, this paper proposes an Economic Model Predictive Control (EMPC) to optimize the heat consumption of residential buildings with multi-temperature zones. The EMPC minimizes the energy consumption cost of buildings and unlocks the power-to-heat flexibility of the thermal inertia of rooms. The suggested approach adjusts the temperature and flow rate of the mixing loop by valves of Mass Flow Control (MFC). Besides, it controls the heating demand of multi-temperature zones, i.e. different rooms, by using on-off valves of radiators. All in all, the main contributions of the study can be stated as follows: • Suggesting EMPC to minimize the energy consumption cost of residential buildings with multi-temperature zones. • Providing power-to-heat flexibility for low-temperature district heating with high RES penetration. • Adjusting the temperature and flow rate of heat carrier in mixing loop of residential buildings.
The rest of the paper is organized as follows. In section 2, the fundamentals of the suggested approach are explained qualitatively. Section 3 explains the methodology of the current study. In section 4, the problem is modeled mathematically. In this way, the thermal dynamics, objective function, parameter estimation, and control approach are described in individual subsections. Section 5 presents the simulation results and discussions. Finally, Section 6 concludes the proposed approach.

Problem description
The study aims to design an economic heat controller for the heating system of residential buildings which are supplied by low-temperature district heating. The objective function of the economic controller is to minimize the energy consumption cost of households while satisfying the residents' comfort bound. The residential building is a multi-room apartment block with different temperature needs. Therefore, the thermal dynamics of the buildings are developed to define different temperature zones based on occupancy patterns and room application. To facilitate the development of low-temperature district heating, the residential heating system is controlled by a mixing loop. The mixing loop combines the return water from radiators with the supply water from the district heating to decrease the temperature of return water to the heat network. Fig. 1 describes the schematic structure of the suggested mixing loop for the heating system of residential buildings. Based on the figure, the rooms are subject to external and internal heat exchange between indoor air and ambient, and between the indoor air of adjacent temperature zones. To define different temperature zones, the heat transfer between internal envelopes must be addressed. Besides, the external heat exchange between the envelopes and the outdoor environment is modeled. The rooms have individual temperature zones and heating systems, in the form of radiators. The radiators are controlled by an on/off valve control. The radiators are connected to meshed water pipes including forward (hot) water and return (cold) water. On the supply side, the mixing loop combines the supply water of the district heating with the return water of radiators. The two valves control the mass flow to the building and are called MFC. It is supposed that district heating supplies the building with a fixed temperature. The valve of return water restores the heat energy of return water to minimize (1) the supply heat from district heating and (2) the temperature of return water to district heating.
On the supply side, district heating is supplied by electrically operated heat pumps. The heat pump procures electrical energy from the wholesale electricity market at variable prices. The RES penetration in the electricity market is high; as a result, there is a strong correlation between RES availability and electricity price. The economic controller minimizes the energy consumption cost of the building through the MLC. In this way, the EMPC is designed in MATLAB software.

Methodology
To introduce the methodology of the suggested approach, the scientific steps are taken as follows: First of all, the data collection includes sensor data of a residential building connected to district heating. The sensor data is comprised of an indoor air temperature of rooms, ambient temperature, solar irradiation, water temperature, and mass flow of forward and return pipes connected to district heating. They are collected for 5 days' worth of data on a minute basis.
In the second step, the sensor data is used to estimate the constant coefficients of thermal dynamics of the residential building. The threestate differential equations, including indoor air temperature, envelope temperature, and heater temperature, are trained by the collected sensor data for 7200 min. Continuous-Time Stochastic Model (CTSM) is used as a stochastic gray box to extract the thermal coefficients of the building. As a result, the thermal coefficients, including heat resistance and heat capacity values, are determined.
In the third step, the fundamental structure of the heat controller is designed. The control structure includes a control approach, objective function, and constraints. Regarding the control concept, the model predictive control is adopted. The MPC optimizes the current timeslot while keeping future timeslots in the account. Therefore, the energy consumption of current time slots is optimized while the future estimation of ambient temperature, solar irradiation, and energy price is monitored. The objective function of the heat controller is to minimize the energy consumption cost. As a result, the controller is an Economic MPC to optimize the energy consumption in response to variable energy prices. The key constraints include residents' comfort bounds and technical constraints of the heating system.
In the fourth step, the heat controller is examined on the case study. The case study is comprised of supply and demand sides. The supply-side refers to the Danish sector of the Nordic Electricity Market which supplies the electrically operated district heating. The demand side includes a single-family house with 4 rooms connected to the low-temperature district heating. The building has four temperature zones. The whole building is connected to the district heating through a mixing loop.
Finally, in the fifth step, the mathematical models of the approach are coded in software tools, including R and MATLAB. The second step, i.e. parameter estimation, is coded in R software through the CTSM. The third step, i.e. heat controller, is coded in MATLAB. Besides, the case study, i.e. the fourth step, is analyzed in MATLAB. Fig. 2 provides a general overview of the stages of the methodology. Note that the fifth step is indicated with the software icon.

Mathematical formulations
In this section, the fundamentals of the suggested approach are modeled mathematically. First of all, the thermal dynamics of buildings with multi-temperature zones are described. Afterward, the objective function of the controller with associated constraints are illustrated. The parameter estimation approach is explained in software language R. Finally, the control scheme of the MLC is presented.

Thermal dynamics of buildings
The flexibility potentials of the thermal inertia of buildings are reflected in thermal dynamics. The thermal dynamics follow differential equations. To describe the thermal dynamics, three states are introduced through the following equations [47]: The three differential equations (1)-(3) describe the states of indoor air temperature, envelope temperature, and heater temperature for room r∈ {1, …, R}, respectively. The heater indicates the radiators or floor pipes. In equation (1), the first and second right terms show the heat transfer between heater and indoor air, and between the envelope and indoor air, all for room r. The third term shows the power of solar irradiation. In equation (2), the first term denotes the heat transfer between indoor air and the envelope of room r. The second term explains the summation of heat transfer between the envelope of room r with all the surrounding envelopes of the room r ′ ∈ {1, …, R} subject to r ′ ∕ = r. In this term, the indoor air temperatures of rooms r and r ′ are described by θ r i and θ r ′ i , respectively. This term makes it possible to define different temperature zones in a building. Due to the temperature difference between adjacent rooms, this term addresses the internal heat exchange between the rooms. The third term indicates the summation of heat transfer between envelopes of room r with all envelopes surrounded by outdoor a ∈ {1, …, A}. In equation (3), the first term is the heat transfer between indoor air and the heater. The second term shows the heat extraction from the heater. In this term, the binary variable χ r takes the value of 1 when the valve of the radiator r is open and takes values of 0 for the closed valve. Note that valves of radiators are controlled by onoff states. Therefore, the mass flow of radiators is a constant value during on states.

Objective function and constraints
In this study, an economic controller is designed for the MLC. On the supply side, it is supposed that district heating is supplied by electrically operated heat pumps. The RES penetration in the wholesale electricity market is relatively high. Therefore, there is a strong correlation between the electricity price and RES availability. In this way, the economic controller reduces the energy consumption cost and provides flexibility to the upstream power network. In addition to the economic concept, the controller aims to satisfy the comfort temperature of residents. Considering the abovementioned facts, the objective function of the controller is formulated as follows [47]: The objective function is comprised of two terms. The first term minimizes the Euclidean distance between the indoor temperature and reference temperature. This term addresses the residents' comfort. The second term minimizes the energy consumption cost of the building. To make a compromise between the two competing objectives, the weighting factors (μ 1 ,μ 2 ) are embedded into the objective function. The weighting factors suppress/enhance the role of associated terms.
Increasing the value of μ 1 , the residents' comfort is more emphasized in the objective function of the controller. In contrast, when higher values of μ 2 are adopted, the controller eases the comfort bound to achieve more economic heating solutions. The objective function is subject to the following constraints: Inequality (5) describes the lower and upper bounds of indoor temperature. The temperature bound is interpreted as the residents' comfort bound. Equations (6) and (7) denote the binary control of radiators' valves. Based on this equation, the mass flow of radiators is equal to nominal mass flow (zero) when the valve is open (closed). Equation (8) shows the temperature of return water as a function of temperatures of forward water, radiator, and indoor air. The constant coefficients (α r ,β r ,κ r ,σ r ) are estimated based on linear regression. Equations (9) and (10) explain the heat balances in the meshes of forward and return water. Finally, equation (11) illustrates the heat consumption of the radiators. Fig. 3 clarifies the notations of the heat balance equations.

Parameters estimation
The constant parameters in the thermal dynamics are dependent on the physical characteristics of the buildings, e.g. insulation level, the material of envelopes, the dimension of windows, etc. In order to design the heat controller for a specific building, the constant parameters must be determined. In this approach, the constant parameters are classified into two main categories as follows: • The constant parameters of the thermal dynamics, equations (1)-(3).
These parameters are as follows: • The constant factors of thermal equation (8). These factors are described as follows: First of all, to estimate the thermal constants (12), CTSM is adopted. The CTSM is developed by the Technical University of Denmark, using language R. The software is publicly available [48]. The CTSM addresses a stochastic gray box to estimate the constant parameters (12) by using the measurement sensor data. The sensor data is introduced as follows: Then, the sensor data (14) is stated as the following time-discrete model: where k is the point in time t k of a data measurement; Y k explains the sensor data, and e k shows the measurement error in the form of Gaussian white noise. Given the sensor data, the maximum likelihood estimation of parameters (12) is run. The maximum likelihood is investigated using the probability density function as follows [49]: where p(.) is a conditional density to describe the probability of observing the sensor data Y k given the previous observations and the target parameters θ; and p 0 (.) denotes the initial conditions. Finally, the maximum likelihood for the target parameter θ s is found by (17). Besides, the linear regression model is addressed to estimate the constant factors (13). Let us consider the set of input sensor data .,n in n units. Then, the linear regression model investigates a linear relationship between the main variable y i and the vector of x ip , called regressors. In this study, the controller requires to determine the temperature of the return water from radiators, y i ≈ θ r Re , as a function of regressors (x 1 , x 2 , x 3 , x 4 ) ≈ (θ r Fo , θ r h , θ r i ). Therefore, the linear regression is modeled as follows: (18) where ε i denotes the error variable. The linear regression is also coded in software R to estimate the constant factors (13). The linear regression in the heating system is presented in equation (8).

Control approach
This study aims to supply the space heating of a residential building with multi-temperature zones. To fulfill the aim, the controller has two recourses: • On/off control of radiators' valves.
• Mass flow control of mixing loop for forward and return water.
It is worth mentioning that the mass flow of radiators in each temperature zone is a constant value when the radiators' valve is open. It means that the radiators' valves cannot change the mass flow of the radiators. Although it is feasible to apply MFC to the radiators, the complexity and investment cost of the heating system increases considerably. In contrast, the MFC is applied to the mixing loop where the two valves control the mass flow from the district heating and return water pipes. The MFC valves mix the return water from the radiators with the hot water from the district heating. The main duty of the controller is to determine the operation of two MFC valves in the mixing loop and R numbers of on-off valves in the rooms.
To achieve the aim, an EMPC is designed in MATLAB software. The EMPC considers the thermal dynamics of the building, equations (1)-(3), in the form of state space. Running the CTSM, the constant factors of the thermal dynamics are exported to the controller. Besides, the linear regression determines the relation between the return water temperature and the forward water temperature by equation (8). Incorporating the required constant factors into the controller, the residents set the comfort bound as the preferred reference temperature and lower/upper thresholds of the indoor temperature. The EMPC is an economic controller not only to satisfy the residents' comfort bound but also to minimize the household energy consumption cost. To make a trade-off between the competing objectives, the controller uses the thermal inertia of the building to increase/decrease the energy consumption when the energy price is low/high. In this way, the indoor temperature may deviate from the reference temperature, but it is maintained within the residents' comfort bound, i.e. inequality (5). In fact, the deviation from the reference temperature is the price the controller pays to unlock heat flexibility in response to energy price. The weighting factors (μ 1 ,μ 2 ) are embedded into the objective function (4) to enhance/suppress the role of comfort term, i.e. the first term, and the economic term, i.e. the second term.
In order to determine the optimum operation of radiators, the controller finds the intersection of two linear functions (8) and (11). The former describes the return temperature as a function of forward temperature, indoor temperature, and radiator temperature. This function is extracted from the sensor data by linear regression. Therefore, all the thermal characteristics of the rooms, including radiators, envelopes, and indoor air are captured accordingly. The latter describes the optimized heat demand from viewpoint of the economic term of the objective function (4). Consequently, the intersection of the two functions determines the economic operation of the radiators considering the associated thermal characteristics of rooms. Fig. 4 explains the intersection points of the two functions.
The prediction horizon of the controller is 24 h on an hourly basis. In the EMPC, the energy price and weather variables, including the ambient temperature and solar irradiation, are considered as the measured disturbances. The energy price of the district heating is supposed as a deterministic variable. In contrast, white noise is added to the weather variable to model the stochasticity for the next 24 h. To take the control actions, the controller determines the optimal operation of radiators' valves and MFC valves. The abovementioned workflow of the controller is depicted in three separate areas in Fig. 5. The figure is comprised of three sections, including parameter estimation, the EMPC, and control recourses. In each section, the decision variables with associated explanations are described.

Numerical studies
In this section, the simulation results of the suggested controller are presented and discussed. The controller determines the optimized operation of a residential heating system for the next 24 h on an hourly basis. First of all, the specifications of the case study are described. Then, the main simulation results are shown. Finally, the key results are discussed and compared.

Case study
The test building is a 150 m 2 Danish house with four rooms, including a kitchen, two bedrooms, and a bathroom with a height of 2.5 m. Therefore, four temperature zones are addressed in the building. Fig. 6 shows the floor plan of the test house. The weather data, including ambient temperature and solar irradiation, are the key disturbances that affect the operation of the building heating system. Fig. 7 shows the weather data, i.e. ambient temperature and solar power for the 7200 h (5 days) of the training period.
The controller aims to optimize the heating consumption of the building for the next 24 h. The weather data and energy prices for the 24 h testing period are described in Fig. 8.

Simulation results
To estimate the constant parameters, a five-day worth of sensor data (equivalent to 7200 min) is used on a minute basis. Fig. 9 describes the results of CTSM in R Software. This figure makes a comparison between the actual (sensor) temperature and the predicted temperature by the CTSM. Moreover, the estimated constant parameters are enclosed in the subfigures. As the graphs reveal, the accuracy of the estimation approach is relatively high for bathroom and bedrooms. In contrast, the kitchen follows a different pattern. In the kitchen, the precision of the estimation approach is lower than the other rooms. The reason is that the kitchen faces the waste heat from some kitchen appliances, e.g. oven and refrigerator. Therefore, the waste heat, which is not captured in the thermal dynamics, increases the residual temperature. Table 1 explains the mean bias error of the estimation approach. The data confirm that the accuracy of the approach for rooms with low/zero heat waste, i.e. rooms 2,3, and 4, is relatively high. As a result, the waste heat of household appliances should be addressed in the thermal dynamics of the buildings to increase the accuracy of the estimation approach.
The constant factors of the linear regression for the four temperature zones are described in Table 2. In order to examine the accuracy of the estimated parameters, Fig. 10 compares the actual and estimated values of return water temperature. Subfigure (a) depicts four on-off cycles of the radiator valve for room 2. As the graph reveals, the residual temperature in the first time slots of switching is relatively higher than the other time slots. Besides, the estimation accuracy is reasonably high for the remaining time slots. To justify the switching undershoots, subfigure (b) describes the hysteresis effect of control valves of radiators. The hysteresis effect states that if the spindle movement of an actuator unit is measured at increasing and then decreasing temperatures, there will be a difference between the point where the valve just closes and the point where it opens again [50]. This difference is called the hysteresis effect of the valves.
Estimating the thermal dynamics of the building in Fig. 9, the heat controller optimizes the operation of the heating system in four temperature zones for the next 24 h. The operational process is simulated in MATLAB software using the command line of the MPC approach. The optimized strategies are explained in Figs. 11-14. First of all, the optimized operation of the heating system in room 1 is stated in Fig. 11. The figure includes the heating consumption, temperature of return water, mass flow, and the profile of indoor temperature. Based on the graph, room 1 extracts the heating energy in three time duration, including  Fig. 6. The Danish test house with four temperature zones [47].

H. Golmohamadi and K.G. Larsen
Journal of Building Engineering 46 (2022) 103286 8 midnight, afternoon, and late night. Regarding the profile of energy price in Fig. 8, two peak slots are detected in hours 9-11 and 18-19. The controller normally consumes heat energy in off-peak hours not only to minimize the energy consumption cost but also to provide power flexibility for the upstream network. The nominal mass flow of the radiator is 0.1579 kg/s. As can be seen, the radiator valve is switched on/off to maintain the indoor temperature within the residents' comfort bound. The lower and upper thresholds of the comfort bound are defined as 18 and 24 • C, respectively. Based on the graph, the indoor temperature has upward and downward trends in off-peak and peak price hours, respectively. The return water temperature reaches 29 • C in some hours. As the temperature of return water decreases, the heat loss of the district   The optimized operation of room 2 is described in Fig. 12. As the graphs reveal, the radiator is switched on/off periodically during the daily operation. The heating consumption is normally scheduled out of peak hours. The controller minimizes the heating consumption in peak hours 9-11 and 18-19. Moreover, room 2 maintains the same level of energy consumption from hours 19 to 24. The nominal mass flow of the radiator is 0.1050 kg/s. The temperature of return water is between 30 and 42 • C. The comfort bound is defined between 19 and 24 • C. The controller maintains the indoor temperature within the comfort bound during the daily operation reasonably well. Fig. 13 describes the optimized operation of the radiator in room 3. First of all, barely any heat consumption is scheduled in peak hours 9-11 and 18-19 when the market electricity price increases considerably. In the bathroom, the lower and upper thresholds of indoor temperature are confined to 21 and 23 • C, respectively. As a result, the radiator extracts heat energy in more time slots in comparison with other rooms. It is worth mentioning that the heat capacity of the bathroom radiator is relatively lower than rooms 1 and 2. Based on the graph, the temperature of return water is between 30 and 37 • C. The controller maintains the indoor temperature of the bathroom within the residents' comfort bound by extracting heat energy in off-peak hours. Although the comfort bound is narrowed in comparison to the other rooms, the controller still prevents consuming energy in peak hours.
The optimized heating strategies of room 4 are stated in Fig. 14. The heating consumption of the room follows a similar pattern and is normally operated in off-peak hours. The lower and upper thresholds of the indoor temperature are considered 18 and 24 • C, respectively. The nominal mass flow of the radiator is 0.0420 kg/s. Increasing the mass flow, the average temperature of return water has increased. In this room, the temperature of return water is between 31 and 41 • C which is relatively higher than room 3 with a nominal mass flow of 0.0223 kg/s.
Regarding the optimized heating strategies of four temperature zones in Figs. 11-14, the following key points should be stated: • The controller schedules the heating consumption of rooms in offpeak hours when the electricity price is relatively low. The heating system is barely operated in peak hours, i.e. 9-11 and 18-19, when , where x f ,i and x o,i are ith forecast and observation; N is the total number of samples.    the electricity price is relatively high. It unlocks the flexibility potentials of the heating system to decrease/increase heat consumption when the supply side encounters a deficit/excess of power generation. • The temperature of return water is between 29 and 41 • C. As the temperature of return water decreases, the efficiency of the heating system increases. Therefore, it facilitates the use of low-temperature district heating with lower heat loss and investment costs. • The controller meets the residents' comfort bound for all temperature zones without needing to operate the heating system in peak hours.
One of the key points of the heat controller is to optimize the mass flow in the mixing loop. The optimum control strategies of the mixing loop are described in Fig. 15 and Fig. 16. These figures show how the mixing loop is operated in response to the variation of electricity prices. First of all, Fig. 15 shows the profile of heating consumption and water temperature in response to electricity prices. On the supply side, it is supposed that district heating is supplied by electrically operated heat pumps. Therefore, the electricity price of the Danish Electricity Market is addressed. The price data is extracted from the Danish sector of the Nord Pool [51] which is publicly available. Subfigure (a) describes the profile of energy prices for 24 h. Based on the price graph, peak and off-peak hours are pointed out. Let us consider a correlation between RES availability and electricity price. Therefore, in the peak/off-peak hours, the electricity market may face renewable power shortage/excess. Subfigure (b) expresses the total heat consumption of the building. As the graph reveals, the controller increases the heat consumption during off-peak hours, i.e. 1-6, 14-17, and 21-23. In contrast, the controller minimizes the energy consumption during peak hours 9-11 and 18-19. As a result, the controller unlocks the flexibility potentials of the thermal inertia of the building in response to energy price. Besides, by increasing/decreasing the energy consumption in off-peak/peak hours, the controller reduces the household energy consumption cost. Subfigures (c) and (d) illustrate the temperature of forward and return water, respectively. The supply temperature of the district heating is 55 • C. The forward water temperature indicates the temperature of hot water from the district heating mixed with the return water from the radiators. As can be seen, the forward temperature takes values between 40 and 55 • C. The controller minimizes the forward temperature during peak hours, e.g. 11-12 and 18-19, to provide down-regulation for the power system. Adversely, the controller increases the forward temperature in off-peak hours, e.g. 1-4, to provide up-regulation for the power system. As the graph reveals, the average temperature of return water is around 30 • C. The low temperature of return water facilitates the development of low-temperature district heating in Danish municipalities.
Finally, the mass flow of valves in the mixing loop is explained in Fig. 16. First of all, subfigure (a) explains the mass flow from the district heating. Based on the graph, the controller increases mass flow during off-peak hours, e.g. 3-4, to increase the indoor temperature at low prices. Also, it zeros the mass flow in peak hours, i.e. 9-10 and 18-19, not only to reduce energy consumption cost but also to provide power-toheat flexibility for the Danish Electricity Market. Subfigure (b) describes the mass flow of return water mixed with the mass flow of district heating. The mass flow of mixed water is depicted in subfigure (c). As the graph reveals, the mass flow of forward water increases/decreases during off-peak/peak hours. All in all, the controller optimizes the mass flow of MFC valves and on/off states of radiator valves to (1) reduce the household energy consumption cost and (2) unlock the power-to-heat flexibility of the building.

Discussions
The simulation results show that the proposed control approach optimizes the operation of the mixing loop in residential buildings in response to variable energy prices. The key feature of the approach is that the controller mixes the return water from radiators with the forward water from the district heating. By decreasing the temperature of return water to the heat distribution system, the efficiency of the district heating increases. Besides, it facilitates the integration of lowtemperature district heating to urban areas with lower investment costs. To make a comparison between control methods, the heat controllers can be divided into classic and advanced controllers. In the former, the classic controllers aim to maintain a balance between heat demand and supply. In residential buildings with classic control methods, the controllers track the reference temperature without considering the energy price and flexibility requirements of energy systems [27]. Adversely, the advanced controllers are devised to optimize the operation of the heating systems considering the energy system requirements. In contrast to classic controllers, the advanced controllers not only optimize the energy consumption cost of the households but also integrate flexibility potentials of the heating systems into upstream energy networks [52]. Recently, advanced heat controllers are proposed for residential heat pumps in response to variable electricity prices [11,53]. In these control approaches, the operation of the mixing loop is not addressed. In contrast, the current study elaborated on the optimization of the mixing loop connected to low-temperature district heating. This is the key advantage of the proposed control approach over the previous methods.
Currently, the penetration of renewable energies is more than 50% in the Danish Electricity Market. Considering a close correlation between electricity price and renewable power generation, the electricity price conveys a strong signal about renewable power availability. In this way, the high/low electricity prices indicate a deficit/excess of renewable energy. In such an electricity market, the economic heat controller unlocks the flexibility of heating systems in response to renewable power availability. Although the controller is examined on the power market with high renewable power penetration, the approach is applicable in electricity markets with low/zero renewable power generation. In such markets, the electricity prices follow time-of-use tariffs to provide peakshaving and/or valley filling for the power system. Therefore, the proposed economic heat controller is still a workable solution to increase/ decrease energy consumption in off-peak/peak hours. It is worth mentioning that the penetration of district heating is relatively high in Scandinavian countries, e.g. Denmark and Norway, with the cold winter season. In this way, the mixing loop control is a practical solution to decrease heat loss in the heat distribution system in countries with high penetration of district heating like Poland, Sweden, and Germany. Although the suggested approach is examined on the Danish case study, it can be developed for other regions with different climate conditions and building structures.
To sum up, the key benefits of the current control approach can be summarized as follows: • Minimizing energy consumption cost of households.
• Integrating flexibility potentials of residential heating systems into upstream energy systems. • Decreasing the average temperature of return water to district heating. • Decreasing the heat loss in district heating.
• Facilitating the use of low-temperature district heating with low investment cost.
Although this study proposed a heat controller for the mixing loop, some issues may be subject to further researches as follows: • The Danish Electricity Market uses three trading floors in the daily operation, including day-ahead, intraday, and balancing markets.
The objective function of controllers may be developed to provide hierarchical flexibility for the market floors. • The study can be developed into a cooperative optimization for a group of building supplied by a local district heating. • The impacts of uncertain data, including weather variables and energy price, on the operation of the mixing loop can be investigated.

Conclusion
This paper proposed a control scheme to optimize the space-heating strategies of residential buildings with multi-temperature zones. The building was supplied by low-temperature district heating connected to a mixing loop. The controller had two control recourses including (1) on/off control of radiator valves (2) mass flow control of valves in the mixing loop. To estimate the thermal dynamics of the building, a continuous-time stochastic model and linear regression were addressed in R software. The control approach was coded as a model predictive control in command lines of MATLAB software. The controller aimed to unlock the flexibility potentials of the building in response to dynamic energy prices. Finally, a 150 m 2 Danish Test House with four temperature zones was simulated. Regarding the simulation results, the key points are stated as follows: • The continuous-time stochastic model estimated the thermal dynamic reasonably with high accuracy. Considering the waste heat of household appliances, the thermal dynamics should be developed to capture the waste heat as an input disturbance. • The linear regression model estimated the return water temperature with high accuracy. Switching the radiators from closed valve to opened valve, the hysteresis effect made a very transient undershoot in the return water temperature. • The on/off control of radiator valves maintained the indoor temperature within the residents' comfort bound. If the difference between the upper and lower thresholds of the indoor air increases, the flexibility potentials of the temperature zone increases and vice versa. The simulation results showed that the number of valves switching for radiators with low heat capacity is relatively high. Therefore, this type of radiator showed lower flexibility potentials. Besides, by increasing the mass flow, the average temperature of return water increases. • The mixing loop optimized the mass flow from the district heating and return water pipes in response to dynamic energy prices. In high/low price hours, the controller decreased/increased the mass flow from the district heating to provide down-/up-regulation for the Danish Electricity Market.
All in all, the suggested controller facilitates the integration of renewable power into district heating. Besides, it helps to develop lowtemperature district heating by decreasing the return water temperature. The controller not only unlocks power-to-heat flexibility in response to energy price but also reduces the household energy consumption cost.

Funding
This work was supported by the project of Flexible Energy Denmark (FED) and ERC Advanced Grant LASSO of Professor Kim G Larsen.