Towards standardising market-independent indicators for quantifying energy flexibility in buildings

Buildings are increasingly being seen as a potential source of energy flexibility to the smart grid as a form of demand side management. Indicators are required to quantify the energy flexibility available from buildings, enabling a basis for a contractual framework between the relevant stakeholders such as end users, aggregators and grid operators. In the literature, there is a lack of consensus and standardisation in terms of approaches and indicators for quantifying energy flexibility. In the present paper, current approaches are reviewed and the most recent and relevant market independent indicators are compared through analysis of four different case studies comprising varying building types, climates and control schemes to assess their robustness and applicability. Of the indicators compared, certain indicators are found to be more suitable for use by the end user when considering energy and carbon dioxide emission reductions. Other indicators are more useful for the grid operator. The recommended indicators are found to be robust to different demand response contexts, such as type of energy flexibility, control scheme, climate and building types. They capture the provided flexibility quantity, its shifting efficiency and rebound effect. A final cost index is also recommended given specific market conditions to capture the cost of a building providing energy flexibility. © 2020 Elsevier B.V. All rights reserved.


Introduction
Increasing penetration of renewable energy sources is one of the most widely used and planned measures aimed at decarbonising the electricity sector. However, additional renewable energy sources such as wind and solar in the electricity grid requires increased amounts of reserve in the power system to maintain supply and demand in balance [1] . This is especially true for peaks in demand when, traditionally, fossil-fuel based generators have been used to meet the additional demand. Demand Side Management (DSM) is a broad set of strategies with the aim of optimising the energy system at the end-use (demand) side and can be more cost-effective and environmentally friendly than supply side management [2] . Demand Response (DR) is one facet of demand side management where consumers shift or curtail their electricity usage in return for financial or other incentives [3] .
Buildings represent a significant percentage of final end-use energy consumption by sector, in fact, this number is around 40% in Europe [4] . It is expected that buildings will play an increasingly important role in the balancing of the grid through DSM measures. As part of the Clean Energy Package presented by the European Commission in 2016 [5] , one of the first documents to be approved was the new Directive 2018/844 on the energy performance of buildings. A key part of this directive is the introduction of a framework to calculate the smart readiness of buildings and capture the capabilities of a building to adapt its operation to the needs of the grid [6] . Additionally, the International Energy Agency (IEA) Energy in Buildings and Community Program (EBC) Annex 67 is dedicated to energy-flexible buildings [7] . The energy flexibility of a building is defined by the Annex as the ability to manage its demand and generation according to local climate conditions, user needs, and energy network requirements [7] . Energy flexibility of buildings will potentially allow for DSM/load control based on the https://doi.org/10.1016/j.enbuild.2020.110027 0378-7788/© 2020 Elsevier B.V. All rights reserved.

ADR
Active Demand Response AEEF Available Electrical Energy Flexibility l DR,start start of DR period l DR [7] . Buildings have inherent thermal mass which allows Heating, Ventilation and Air Conditioning (HVAC) loads to be adjusted within the confines of the thermal comfort limits of the occupants. Other sources of flexibility include the use of active thermal energy storage, active electric storage (such as batteries), the ability to switch generation sources, and flexibility in equipment use from occupants. In dual fuel systems, gas or district heating, may provide additional flexibility [8] . Nevertheless, this option is not commonly encountered. Moreover, it is rarely an option for flexibility cooling applications. The stakeholders interested in the amount of building energy flexibility from a given building are building owners, end-users (occupants and building managers) and aggregators. Building owners are interested in maximising revenues from their assets on top of reducing their overall energy cost. The grid operators (either Distribution System Operators (DSOs) or Transmission System Operators (TSOs) or both) are generally more interested in the aggregate flexibility available through an intermediary such as an aggregator. Aggregators are centralised agents who contract with many smaller customers. Typically, such customers are not able to meet the minimum energy reduction for demand response programmes [7] . Generally, the aggregator is responsible for acquiring the flexibility from these customers (who may be prosumers), aggregating it into a portfolio and offering the flexibility services to the grid operator. The aggregator shares the value it creates with the prosumer as an incentive to shift its load. Such aggregation is already operational in many energy markets and countries, with the reader referred to [9] for some examples. For the purpose of DR, aggregators and the grid operators are interested in the energy or power consumption that can be decreased (downward flexibility) or increased (upward flexibility), and both how quickly the building can respond and how long a response can be maintained [10] . Hence considering that energy flexibility is a traded commodity, there is a need for indicators to quantify the energy flexibility available (whether in real-time or in the future) considering the aforementioned context. Given that the provision of energy flexibility generally comes at a cost (either thermal losses or increased spend on energy), this deviation from the business as usual case needs to be quantified [11] . This is of interest to building owners, as it allows them to select the optimal strategy to meet a given demand response request as well as to determine the appropriate compensation for their action.
The current research is concerned with the problem of quantifying building energy flexibility. The focus is on energy flexibility (kWh) as opposed to demand flexibility (kW) which is an area that has been given attention in fields such as Demand Response Measurement & Verification (M&V) [12] . Given numerous approaches to quantification and many different indicators proposed in literature, there is a lack of clarity and consensus in the state of the art. Moreover, there is a gap in the literature of a review and comparison of the various flexibility indicators and assessment of their suitability for the future smart grid and their use by the relevant stakeholders. This research aims to test the robustness of these indicators to different control schemes, DR strategies, building types, climates and model types in a bid to consolidate the state of the art in this field and make a first attempt at standardisation. Given that energy markets may not operate in the same way in different countries, only indicators that are independent of market conditions are considered. The ease of comprehension of the indicators from both the demand and supply side is considered and both "upward" and "downward" energy flexibility events are simulated. This work will form the basis for a standard for quantifying the building energy flexibility available and contractual framework between the stakeholders (end-users, aggregators and grid operators). These are considered to be the main contributions of this work.
In Section 2 , a literature review of existing research on energy flexibility in buildings is presented focusing on the strategies employed and the indicators used for quantifying energy flexibility. Section 3 summarises the main research aim and objectives. Section 4 summarises the methods and indicators used in the current study, which are applied to four separate case studies. Section 5 presents the results with a discussion and finally a conclusion is given in Section 6 .

Demand response programmes and strategies
Generally, there are two types of DR programmes: price (or time) based and incentive based [13,14] . Time-based programmes offer customers specific time-varying prices in advance, e.g., based on the cost of generation. Incentive-based programmes are designed such that customers are incentivised to reduce their electricity usage at times of grid stress whilst also being under specific constraints with penalties for non-conformance of these. In such cases, these DR events usually range from half an hour to a few hours. For instance, in Ireland, whilst volume is measured in MW (or kW) for contracted services such as Primary Operating Reserve (POR), Secondary Operating Reserve (SOR) and Tertiary Operating Reserve (TOR), payment is based on the average volume over the trading period multiplied by the time period (plus some scaling factors) resulting in a per MWh payment rate [15] .
Lund et al. [16] presents a comprehensive review of demand side technologies that can be utilised to provide energy system flexibility. Lund considers the different demand side measures available according to sector (residential, industry and service). The residential loads with DSM potential include night storage heaters, domestic hot water heaters, ventilation systems, refrigerators, freezers, hot water circulation pumps, washing machines, clothes dryers and heat pumps with storage. Thermal energy storage in residential heating systems is considered to have major DSM potential, especially when households are pooled together. In the service sector, DSM potential includes ventilation systems, air conditioning, waste-water treatment, refrigeration, and electric hot water generation [16] .
In general, the sources of energy flexibility in buildings can be characterised as being one of: flexible distributed energy generation (such as on-site solar), shiftable and curtailable electrical loads, electrical storage systems and shifting of heating/cooling/ventilation loads coupled with thermal storage [17] . Setting aside industrial DSM where the potential is process specific, in the residential and commercial sectors, the most significant untapped DSM potential is from thermal mass, either active (thermal energy storage (tanks)) or passive (building thermal mass) [18] . Consequently, this is the area that has also seen the most research in the area of DSM and more particularly DR. DR strategies studied have included chiller water set-point adjustments [19][20][21] , thermostat temperature adjustment [2,19,22,23] , air fan modulation [24] , and heat pump modulation [25][26][27] . Approaches and indicators used to quantify flexibility need to be inclusive of and compatible with the different strategies mentioned above.

Approaches to quantifying building energy flexibility
To most effectively utilise the thermal mass for demand shifting, a model capturing the thermal dynamics of the building and heating or cooling system is beneficial in conjunction with a control framework. The majority of work that has addressed the question of characterisation and quantification of building energy flexibility has utilised building simulation models. The use of simulation models has the added benefit of allowing many scenarios to be analysed, for example with different boundary conditions, which otherwise cannot be easily done using a real building. For a detailed review of the different building modelling and control approaches, the reader is referred to the work of Maasoumy, Mehdi; Sangiovanni-Vincentelli [28] . Building simulation models fit in a spectrum ranging from white-box models to black-box models. White-box models are mostly physics based (first principles based). These models can range from high-fidelity, produced from simulation software, such as EnergyPlus or TRNSYS, to name a few building simulation packages, to reduced order dynamic models, e.g., developed using programming and/or modelling languages such as MATLAB and Modelica. On the other hand, black-box models ignore the physics of the system and are completely data-driven. Grey-box models embody aspects of both white-box and black-box models. They rely on some physical knowledge of the system and where there are unknown parameters, statistical methods are used to estimate them.
When considering building energy management and the problem of quantifying energy flexibility, the control scheme used has to be considered along side the building simulation models. The optimal control of an HVAC system is a complex multi-variable problem. The standard control seen in most buildings today is simple rule-based control (RBC) which are essentially if-then-else based rules [29] . This control is often outperformed by either Optimal Control or Model Predictive Control (MPC) which is a closedloop technique. These strategies are predictive in nature and hence able to take future disturbances into account as well as harness the thermal storage available [30] . MPC is based on the solution of an optimal control problem on an iterative basis for a finite horizon. For every time step, the optimal control sequence is found for a finite time horizon whilst meeting constraints. This optimal control requires a linear model to guarantee a convex optimisation problem and unique solution. Generating linear models for highly nonlinear building thermal dynamics represents one of the most significant challenges of MPC and one of the reasons that it has mostly been constrained to academia to date [31,32] . There exists a significant amount of research using either optimal control or model predictive control in harnessing and assessing energy flexibility of a building. Approaches and indicators used to quantify flexibility need to be integratable not only with the standard RBC control used commonly today but they need to be future-proofed against control paradigms such as MPC.
A review was conducted of different quantification methods in the literature by Lopes et al. [33] . It was found that the energy flexibility was primarily assessed on the basis of the deviation of electricity consumption, although under different and specific scenarios including market considerations and thermal comfort schemes. However, this study did not focus on or critically analyse the indicators specifically used by the various studies.
The work of Reynders et al. [34] evaluated flexibility definitions and quantification methodologies specific to thermal mass. Two quantification approaches were identified: the first exploits historical data in the context of a specific energy system and market, the second predicts available energy flexibility using a bottom-up manner.
Oldewurtel et al. [35] presents a unified framework allowing for the comparison of different types of DR and energy storage resources. Their framework compared batteries, plug-in electric vehicles (EV), commercial building thermal mass and thermostatically controlled loads (TCL). One aspect of the framework is resource characterisation, both of physical parameters (such as power capacity, energy capacity, ramp rate and so on) and scenario-dependent parameters (such as control options, response times and implementation requirements). More recent work from O'Connell et al. [36] presents a standardised 4-step methodology for assessing energy flexibility for demand response, albeit the work focuses on an early-stage assessment and is suitable for contract negotiation between demand side management stakeholders. This framework does not require any models aiding ease of implementation at the cost of accuracy and operational suitability.
In summary, it is evident that although studies attempting to quantify building energy flexibility are numerous and diverse, there is a lack of a common standard quantification methodologies given the numerous variables at play including control schemes, building technologies in question, market structure and respective boundary conditions.

Energy flexibility indicators
Considering the bottom up approach to quantifying energy flexibility, indicators have typically revolved around three metrics: (i) the quantity of energy that can be shifted, (ii) the temporal flexibility, i.e., how long the consumption can be shifted, and (iii) the cost of utilising this flexibility [34] . In all these cases, the determination is based on a comparison with a reference case where no energy flexibility is exploited.
Reynders et al. [37] work was specific to structural thermal storage in residential buildings and the methodology for quantification was based on indicators concerning the size, time and induced losses or costs of energy flexibility. This work was specific to and based on upward flexibility (i.e., an increase in consumption during the DR event) only. The available capacity for active demand, C ADR (kWh), is formally defined as the amount of energy that can be added to the storage system, without jeopardising comfort, in the time-frame of a DR event and subject to dynamic boundary conditions. This is defined mathematically as: where Q ADR is the thermal power emitted to the building during the DR event, while Q ref is the emitted thermal power in the reference case, the 0 limit of the integral refers to the start of the DR event and l ADR is the duration of DR event. Actively storing heat in a building increases the temperature within the building and associated transmission losses. There are losses associated with the storage and only a fraction of the stored heat is available post the DR event. To account for this, a storage efficiency is also defined as the fraction of the heat stored during the DR event that can be used subsequently to reduce heating or cooling power. This is defined mathematically as: The infinite in the integral is used to signify a sufficiently long period following the DR action to capture any rebound effects arising from the DR actions. This definition is intended for use only with upward flexibility and is not applicable for downward flexibility. Further, these indicators reference heating/cooling thermal power and not electrical power, and so are not directly useful for aggregators or the grid. Reynders also defined the power shifting capability as a measure for the instantaneous energy flexibility, which describes the shift in power that can be obtained at a given moment in time and the duration that this shift can be maintained [37] . Six et al. [38] investigated the flexibility potential of a heat pump combined with a thermal energy storage. The energy flexibility was quantified as the maximum time that the heat pump can be deferred or forced to operate. Similarly, Nuytten et al. [39] defined the flexibility of a specific system as the number of hours the electricity consumption can be delayed or anticipated with maximum and minimum curves defining the bounds of operation where the difference between these curves is quantified. The work of De Coninck and Helsen [11] coupled the amount of flexibility (in kWh) with the economic cost of provision through the use of cost curves. To do this, an optimal control framework was required which necessitated the solution of three control objectives -one minimising cost, one minimising energy consumption for downward flexibility and one maximising energy consumption for upward flexibility. A time dependent energy flexibility profile can be obtained by calculating the cost curves at every time step. Furthermore, the energy flexibility provided by different systems can be aggregated using this method. Similarly, D'Ettorre et al. [40] mapped three flexibility indicators: cost-deviation, modulation capacity and efficiency through the use of an optimal control framework.
Whilst not the focus of the current research, reference is also made to the approaches based on specific control and energy systems. Le Dreau [41] proposed a flexibility factor which quantifies the ability to shift the energy use from high price periods to low price periods. Such an indicator is highly sensitive to grid signals (or pricing) and climatic conditions. Maasoumy [24] defined lower power consumption and higher power consumption limits for a model predictive control scheme which still respect constraints and hence derived the flexibility values as the difference between these limits.
Clauß et al. [42] provided an overview of specific energy flexibility indicators. One of their findings was that most energy flexibility indicators focusing on the building were unable to cover all possible DSM services (e.g., grid integration or grid ancillary services). They recommended that future work should examine the limitations and robustness of the reviewed energy flexibility indicators and consider their implementation in optimal or model predictive control. A review of the literature also reveals that studies in quantifying energy flexibility from buildings have typically only considered flexibility from one source. The ability of flexibility indicators to consider flexibility potential from multiple flexibility sources is not known clearly. The need to stress-test building energy flexibility indicators available in literature leads us to our research aim which is described in the next section.

Research aim & objectives
Given numerous approaches to quantify building energy flexibility, in an attempt towards standardisation and consolidation of some of the various indicators that have been proposed, this research aims to stress-test these indicators. As justified by Reynders et al. [34] , to decouple the analysis of various demand response strategies and market operation, and to allow the direct quantification of the energy flexibility that a building can offer, the focus is on indicators that consider a bottom-up approach to quantification and those that are independent of the market conditions. This work determines whether these indicators are robust for a range of demand response strategies, for both RBC and MPC control, for different building types and climates. To limit the scope of this study, sources of flexibility such as the use of active thermal energy storage, active electric storage, switching generation sources, and flexibility in equipment use from occupants, are not part of this study. Note that buildings are also capable of providing additional ancillary services such as frequency regulation to the grid which are required to maintain power system reliability [1] . The scope of the paper excludes provision of ancillary services such as frequency reserves as the nature of this problem is very different to the provision of energy flexibility. This work will allow a standardised set of energy flexibility indicators to be defined that are useful to all stakeholders in DSM and allow the research community to shift the focus to reducing the barriers in exploiting energy flexibility of buildings and aid the transition to the smart grid. To achieve these objectives, the following steps are followed: 1. Select the most recent and relevant building energy flexibility indicators (market independent) from literature for analysis. 2. Test the selected indicators on a range of case studies of differing demand response strategies, control schemes, building types and climates. 3. Compare and analyse the suitability of the selected indicators in terms of ease of comprehension, and applicability to end-use and grid side stakeholders. 4. Recommend a set of consolidated building energy flexibility indicators from the results of the comparison and test of robustness.

Methods and case studies
Three energy flexibility indicators, that are the most recent, unique, market-independent and used for bottom-up quantification, were chosen and developed based on the available literature (detailed in Section 4.1 ). These indicators are applied to four diverse case studies, based on analysis of three case study buildings, to test their robustness and applicability. The case study buildings are virtual DR testbeds using high fidelity white-box models rather than real-world buildings. The case studies provide diversity in terms of the demand response strategies considered, the building type, climate, control strategy employed and the computational tool used for the modelling (although the interpretation of the indicators is independent of this). The overall methodology is summarised below:

Energy flexibility quantification
The three sets of energy flexibility indicators that are tested in this study (Available Electrical Energy Flexibility (AEEF) [20] , Adapted ADR (DR,P) [25] and Flexibility Performance (f) [21] ) are described in this section. These are indicators that focus on a bottom-up approach to quantifying energy flexibility and are independent of energy market considerations. The indicators are expressed using the nomenclature and expressions for a typical DR event profile of the modified power consumption curve, as shown on the top of Fig. 1 . Note that the area in red represents the rebound energy and this is typically expected following a change in consumption during a DR event and/or prior to the DR event (a "prebound" -preconditioning that takes place before the DR event and not after, when predictive control strategies are used) in the case of MPC control as illustrated in the bottom of Fig. 1 .
All three sets of energy flexibility indicators are identical in terms of their representation of capacity, i.e., the quantity of energy flexibility available. They define the capacity as follows: where l DR,start is the time at which the demand response event starts and l DR,end is when the event ends, P DR refers to the building electrical power consumption during the demand response scenario and P Ref refers to the consumption profile that the simulation outputs for normal operation (reference case) without DR. Note that the above expression assumes the existence of continuous functions describing the power consumption. Infinitesimal definitions are used in the definitions as they are conceptual and in practice, the indicators would be calculated with discrete sums based on the measured and/or simulated data available.
This set of indicators mostly originates from Reynders et al. [37] . The main difference resides in the fact that the electrical power P is considered in Eq. (3) , rather than the thermal power Q. In this way, the dependency of the indicators to the chosen build-ing thermal emitter system (radiators, FCU, radiant floors, etc.) is eliminated. Furthermore, the indicator directly considers electrical energy, which is more relevant when considering buildings as active players in grid management. In this regard, the indicator is mapped directly to the needs of the grid, regarding the building mass as a storage means, while the original indicators from Reynders et al. [37] are motivated by a building thermal perspective only. The value of this indicator corresponds to the additional amount of electrical energy that can be indirectly harnessed or stored in the building thermal mass during the DR event, and therefore it is expressed in kWh. This additional amount is calculated with respect to the reference case, which means the outcome of the indicator highly depends on the choice of that reference (standard thermostat, MPC minimizing energy, etc.). The formulation is equal for either upward or downward flexibility cases, however E AEEF / DR,P / f is positive in upward flexibility and negative in downward flexibility.

Available electrical energy flexibility (AEEF) indicators
The indicators proposed by Kathirgamanathan et al. [20] are adapted from the indicators used by Reynders et al. [37] . The "Available Electrical Energy Flexibility" (AEEF) is used as the index to demarcate this set of indicators. Following on from Reynders work, the storage efficiency is defined separately based on whether upward or downward flexibility is provided. For downward flexibility, the efficiency is a measure of the magnitude of the rebound effect (expected following a DR event) over the amount of energy shifted. The definition is given as: The positive sign superscript refers to only the positive area in the P DR − P re f curve and the negative superscript refers to the negative area in this curve. For upward flexibility, the definition is given as: These definitions are based on the thermodynamics based thermal energy efficiency definition, e.g., for up flexibility, the fraction represents the "useful" energy saving in consumption outside the DR duration arising as a result from the extra energy stored in the building thermal mass during the DR period.

Adapted ADR (DR,P) indicators
In this section, the indicators proposed by Pean et al. [25] are described. The storage efficiency is defined as follows: This quantity represents the efficiency of the storage-like operation of the building thermal mass. The denominator refers to the capacity of energy flexibility available. The indicator of Reynders was originally developed only for upward flexibility cases, therefore it did not contemplate cases with negative C ADR [37] . For this reason, an absolute value is added to the denominator, so that η DR,P remains positive in normal flexibility scenarios. Whilst this definition is very similar to η AEEF above for downward flexibility, the ratio is defined differently such that rebound effect corresponding to the DR capacity results in a value of 1. This is illustrated in more detail in Section 4.1.4 .

Flexibility performance (f) indicators
This set of energy flexibility indicators consists of two additional indices as described below and taken from [21] . They are named Flexibility Performance (f) indicators because they aim to measure the performance of a DR strategy from the perspective of the utility.
The rebound energy ( E rb ) captures the change in energy consumption following or before a DR event compared to the reference case and this concept is not expressed directly in the previous indicator definitions ( Sections 4.1.1 and 4.1.2 ). It is defined as: The first part of Eq. (7) indicates the energy consumed during the preconditioning period (prebound); the second part indicates the possible rebound after the DR event. The possible prebound is also included for cases of predictive control where anticipation of a demand response event can occur. The −∞ as well as the + ∞ denotes the prebound or rebound horizon. When calculating E rb , the horizon can be several hours or longer depending on the system response of the control strategy. This information is important for the grid operator to ensure the stability and balance of the grid outside the DR period is not adversely affected by DR measures.
The flexible energy efficiency ( η f ) is a measure of how much energy was shifted relative to the rebound effect. This indicator considers the flexibility from the utility perspective: the rebound energy after the DR event is always considered as "disadvantageous" for the grid operator, unlike the first two sets of indicators which are building-centric and where extra energy consumption by the building is considered less than ideal. This indicator is defined as:

Interpretation of indicators
The interpretation of these indicators is presented for all the possible combinations in Tables 1 and 2 . In Table 1 , a downward flexibility case is considered: the energy consumption is decreased by 10 kWh as an example during the demand response event. Different amplitudes and signs of the subsequent rebound effect are then tested, and the flexibility indicators calculated for these theoretical cases. Similarly in Table 2 , an upward flexibility (increase of 10 kWh) event is considered, and the indicators are calculated for different combinations of the rebound effects.
It should be noted that both in upward and downward flexibility cases, when the rebound effect is equal in amplitude to the activated energy flexibility, but opposite in sign, the efficiency η AEEF / DR,P / f = 1 ( Table 1 row 1c and Table 2 row 2d -except for the AEEF indicator in downward flex). The further analysis then differs, because the rebound effect is interpreted differently for upward flexibility (desired reduction of the energy use after the forced DR activation) compared to downward flexibility (unwanted increase of the energy use following the interruption caused by the DR event). Therefore when the rebound effect is smaller in amplitude than the energy flexibility, it generally corresponds to an efficiency below 1 for upward flexibility (the negative rebound does not compensate enough for the increased energy use of the DR event), with the exception of the η f indicator ( Table 2 rows 2a-2c). In the case of downward flexibility, when the rebound effect is smaller than the energy flexibility, this generally corresponds to an efficiency higher than 1 (the positive rebound effect is limited, and thus the overall energy use is still lower than in the reference case), with the exception of the η AEEF indicator ( Table 1 row 1d).

Case studies
The individual case studies are detailed below. A summary of the cases studies is presented in Table 3 and the buildings are illustrated in Fig. 2 .
Rebound is between one and two times as large as the energy flexibility.
-10 kWh The positive rebound effect has the same amplitude as the energy flexibility, the DR activation had no effect on the overall consumption of energy.
No rebound effect, the total decrease in energy use exactly corresponds to the energy flexibility.
Continued to use less energy than the reference case even after DR event.

Case study A
To represent a typical commercial building, a US-DOE (United States Department of Energy) commercial building archetype model was selected as the virtual DR testbed building [43] . The DOE provides these reference models as EnergyPlus input files, which are intended to be used as starting points in energy efficiency research. The version with "new construction", which comply with the minimum requirements of ASHRAE Standard 90. , was selected for climate zone 4C. This is a suitable model for the weather of Dublin, Ireland, which is used in this case study. The large office has a floor area of 46,320 m 2 over 12 floors. The building operates from 6.00 am to midnight on weekdays and 6.00 am to 5.00 pm on Saturdays (with no occupancy on Sundays). The building has a "Mass Wall" wall type based on ASHRAE Stan-dard 90.1-2004 with a U-Value of 0.857 W / m 2 . K [43] . A simulation time-step of 15 min was selected for this case study.
The large office reference building has a gas boiler for heating (1,766 kW) and two water-cooled chillers in parallel for cooling. The primary chiller is rated at 1,343 kW and the secondary chiller is rated at 141 kW and feeds a cold water thermal energy storage (TES) tank. The TES has a storage capacity of 100 m 3 . The building features a multi-zone variable air volume (MZ VAV) system for air distribution. The reader is referred to [20] for more details on these specifications.
With this case study, demand response strategies (of hourly duration) are considered as outlined in Table 3 , where the strategies are based on adjusting global zonal temperature set-points (GSA) to shift the cooling load. Downward flexibility involves increasing Table 2 Interpretation of the energy flexibility indicators for upward flexibility cases The DR case continues to use more energy than the reference even after the end of the DR event.
10 kWh 0 No rebound effect, the total increase in energy use exactly corresponds to energy flexibility, and the system goes back to reference after the event.
10 kWh 0 0 ∞ 2c A negative rebound effect, lower than the energy flexibility, is observed at the end of the event and the energy used is still higher than the reference case.
The negative rebound effect has the same amplitude as the flex, the DR activation had no effect on the overall consumption of energy.    the cooling set-point to reduce the cooling load, whereas upward flexibility involves reducing the cooling set-point to increase the cooling load. This takes advantage of the building passive thermal mass. Operative drift values as per ASHRAE Standard 55 [44] are used for the set-point adjustment. The simulations are carried out for the cooling design day.

Case study Bb
This case study presents a different building typology and a different climate zone: a residential flat situated in the Mediterranean area of Spain. The apartment comprises four bedrooms in addition to the living room, kitchen and bathroom, summing up to a floor area of 109 m 2 . A refurbished version is considered here, with 12 cm of insulation added into the external walls, which brings their U-value down to 0.203 W / m 2 . K . The flat is modelled in TRN-SYS, with the model using a time step of 3 min and with the weather files from Terrassa, Spain.
The flat is designed for a family of four; the occupancy and the resulting internal gains are modelled deterministically. The normal set-point is 20.5 • C when the dayzone is occupied (from 6.00 to 9.00 am and from 7.00 to 9.00 pm), with a setback to 19.5 • C otherwise. The space is conditioned via a radiator circuit supplied by an air-to-water heat pump of nominal power 4.3 kW, controlled by a standard thermostat. The heat pump also supplies a DHW storage tank, but it is not included in the demand response strategies, only space heating is considered for that purpose in the present work. The reader is referred to [25] for more details on the building specifications.
The demand response strategies implemented in this case consist of modulations of the indoor temperature thermostat set-point.
From the reference set-point previously described (20.5 • C and setback of 19.5 • C), a change of T SP = ±1 • C is maintained during a period of two hours. The upward flexibility case corresponds to the set-point increment ( T SP = +1 • C) while the downward flexibility case corresponds to the set-point decrement ( T SP = −1 • C) (see Table 3 ).

Case study C
The third case considered in the paper is a detached house in Montreal, Canada. The house was built according to the Canadian R-20 0 0 building standard [45] . It represents a common three-story single-family Canadian home with a basement, a living floor and a sleeping floor, with a living area of 210 m 2 . The construction is a typical North-American timber frame structure with brick veneer as exterior finish. The U-value of the exterior wall is 0.285 W / m 2 . K and the windows are double low-e coated. Only space heating is considered in this case study and the heating system is electric resistance heating, with each floor independently controlled by a thermostat. The baseline set-point for the two floors is a constant 21 • C and 17 • C for the basement. A validated building model was built in TRNSYS [46] with the CWEC weather file for Montreal, Canada [47] . The heating system was modeled using the idealized heating in TRNBuild; therefore, the set-point control was also idealized in the simulation. The simulation timestep is also 15 min. More details about the house and the model can be found in [21] .
The set-point modulation is only applied on the occupied zones (i.e., living and sleeping floors). The rule-based control scenario (see Table 3 ) during the DR event is:

Case study D
A Model Predictive Control (MPC) scenario is also applied to investigate the flexibility potential with an economic (cost minimisation objective) controller used. The same model is assumed from Case Study C with just the control scheme modified. This is realised using a co-simulation with MATLAB and TRNSYS. The thermal comfort constraint is assumed to be within 2 • C of the reference case for the optimal controller, to be consistent with the rule-based control strategy. Each DR event also lasts 2 h and it is assumed that the price during the DR period is 2 times higher than non-DR times. The controller aims at finding the optimal set-point profile with a control horizon of 24 h.

Capacity and rebound indicator results
Figs. 3 , 6 and 9 illustrate the application of the energy flexibility indicator for capacity ( E AEEF / DR,P / f ) for the different case studies. Note that the plots in this section represent one design day with 24 simulations (simulation of DR event for each hour, i.e., the value at each hour corresponds to the energy flexibility available during that hour given a DR event) combined to produce each graph and provide a profile for the day in question. The schedule of the building and the reference profile of electricity consumption significantly determines the quantity of energy flexibil-ity available. Downward flexibility is not available in hours where the HVAC system is not operating (e.g., Fig. 3 and 6 ). In the case of upward flexibility for case A, upward flexibility is not able to be provided at 6.00 am as this is when the chiller is turned on during reference operation and is running at full load. This impact of the schedule of the building and hence the reference profile is seen clearly in Case B ( Fig. 6 ). Here, a temperature setback exists in the reference case, with a higher set-point when the occupants are active at home (during the morning and the evening). The heat pump in this case tends to use more energy during these periods, and therefore there is potential for decreasing this consumption (downward flexibility) but not for increasing it (upward flexibility) as the heat pump is already operating at its maximum capacity. On the other hand, the periods of lower set-point such as midday present a lower energy consumption, with potential to increase it by forcing the heat pump to operate. In this regard, the existing reference profile shapes the capacity profile to a certain extent.
The rebound energy ( E rb ) is illustrated for each case study ( Figs. 4 , 7 and 10 ). The rebound energy generally appears to show the opposite profile compared to the energy flexibility capacity for both upward and downward flexibility. Case A provides an exception (considering upward flexibility) with the rebound effect being positive following an upward flexibility event, corresponding to an increased consumption rebound effect even following increased consumption during the demand response period. This occurs because energy consumption after the DR event is still higher than the reference case with there being some lag in the immediate time steps following the event. The consumption is not lowered post DR event with the RBC control ignorant of the stored thermal energy. The rebound energy indicator is of interest to the aggrega-  tor or grid operator as opposed to the end-user. The grid operator needs to ensure that the rebound effect does not adversely affect grid stability or balance outside the demand response implementation period.

Efficiency indicator results
Next, the various energy flexibility efficiency indicators ( η AEEF / η DR,P / η f ) are compared. These are illustrated for each case study in Figs. 5 , 8 and 11 . The interpretation of these indicators are discussed on a case by case basis.
Considering Case A ( Fig. 5 ), η AEEF notably shows lower relative values than the other two efficiency indicators in the downward flexibility analysis. Both η AEEF and η DR,P show similar trends with η AEEF being constrained to limits of [0,1], unlike η DR,P . However, the negative value of η DR,P is able to provide more information about the further increases in energy consumption due to the upward flexibility provided by this building during the evening hours. The η f indicator is not constrained and leads to large values when the rebound energy E rb is small or close to 0. 6.00 am is an anomaly as the building is not able to provide upward flexibility (as explained in the previous section) and instead sees a decrease in power consumption. Both η AEEF and η DR,P show a high efficiency in this case as opposed to η f which shows a low efficiency. In this case, η f better captures this anomaly but it can be argued that the efficiency indicator is not relevant given that the building is not able to provide the required energy flexibility capacity. The indicator η AEEF has separate definitions for upward and downward flexibility, unlike the other two sets of indicators.
Case study B is considered in Fig. 8 . The "downward flex" case has generally lower efficiency because the rebound effect is high. In this case, greater rebound energy means that the heat pump consumes more energy in order to recover from the demand response event, which could be considered to be a negative effect, depending on context. On the other hand, the "upward flex" case has high efficiency which is also due to the same reason, i.e., high rebound energy. However, this "negative rebound" actually indicates that the heating or cooling loads are reduced following the DR event given the extra energy stored during the DR event; therefore a greater rebound quantity is beneficial in this case. The η AEEF value is slightly greater than 1.0 at 1.00 am, which corresponds to a larger negative rebound than the positive upward flexibility of the DR event. This only occurs because the reference control is a rulebased version and one that is not energy-optimal. This would not be expected with MPC control with an optimal energy consumption objective. η AEEF also can take negative values here, because the rebound effect is greater than the obtained energy flexibility. This effect would be expected from an MPC scenario ( Fig. 13 ), but it is shown here that it can also happen in a rule-based control scenario. The two latter efficiency indicators show similar profiles for the "downward flex" but show inconsistencies in the "upward flex" case. They present quite different trends, even though the values are close. The η DR,P shows a minimum efficiency at 8.00 am and a maximum efficiency at 1.00 am whereas η f shows the inverse.
Case C shows a similar trend for the three efficiency indicators in the "downward flex" case but inconsistent trends with the "upward flex" case ( Fig. 11 ). η AEEF and η DR,P show the lowest efficiency of the day around 11.00 am in the "upward flex" case whereas η f shows the highest efficiency. Investigating the capacity and re-     bound values at this particular hour ( Fig. 9 and 10 ), we see that the capacity decreases slightly but the rebound energy decreases compared to the reference case, which causes the peak values of efficiency. The opposite phenomenon of efficiency trend in the "upward flex" case is due to different perspectives regarding whether rebound energy is beneficial or not. From the grid perspective, any deviation from the reference case is unplanned and undesirable. In this sense, the η f indicator captures the relative magnitude of the rebound effect. Fig. 12 shows the capacity and rebound energy profiles for the MPC case (Case D). First, the difference in the profiles for Case C and D are compared, noting that both cases are identical, except MPC is deployed in Case D. For Case D, the capacity magnitude is greater than Case C. This is because MPC can anticipate the occurrence of DR events, therefore storing energy in the thermal mass before the event; while RBC is only reactive to the events. The MPC strategy does not deliver a significant increase for flexibility capacity compared with the RBC strategy because a 2 • C set-point adjustment is quite effective for this particular case. In addition, the MPC strategy is also subject to maximum 2 • C adjustment. The rebound energy plot appears to be the mirror of the flexible energy profile for most hours of the day. Note that the rebound energy in this case includes also the "prebound" energy before the DR events. Considering the efficiency values ( Fig. 13 ), the efficiency is rather close to 0 or 1 for the duration of the day with the exception being around midday. This shows that the energy saved during the DR event is close to the increased rebound energy. However, during the midday hours, the MPC strategy is not an "energyefficient" approach. Note that the objective of the economic controller used for MPC is to minimize the overall energy cost (with there being a higher price during the DR event) instead of total energy consumption. Here, as in Case C for downward flexibility, all indicators show similar trends with a decreasing efficiency at midday. The only difference between the indicators is the value of the efficiency indicator.

Discussion
The capacity indicator is identical between the three sets of indicators considered and hence is not given any further discussion. The 'Flexibility Performance (f)' indicators feature the rebound energy ( E rb ) which is not expressed directly by the other two sets but presents useful information especially to the grid operator. This indicator captures the size of the deviation in consumption prior to and following the DR event and the grid operator needs to ensure that this rebound does not compromise the stability and balance of the system.
Based on the analysis in the previous subsections, the three sets of flexibility efficiency indicators are seen to exhibit similar trends for the downward flexibility event but present quite different results for the upward flexibility. The first indicator η AEEF is consistent with the conventional energy efficiency definition with constraints from 0 to 1, with a few special cases violating this. This presents an advantage given that it is closer to the common agreement of the efficiency definition. The second indicator η DR,P can be interpreted as the "storage efficiency" of the building mass. It provides results most aligned with a purely energy balance efficiency (i.e., no energy losses corresponds to efficiency of 1). The first two indicators are more relevant for the end-user or building owner/operator. They are also helpful if the goal is to reduce CO 2 emissions. The third indicator ( η f ) definition is closer to the coefficient of performance rather than efficiency and hence presents quite different efficiency results compared with the first two indicators. It also prioritises the grid operators perspective in that any kind of rebound behaviour is seen as less than ideal.
Whilst the indicators presented illustrate the energy flexibility available, they do not capture the cost of providing the energy flexibility as this requires a deviation from the optimal control in terms of thermal comfort or economic measures. An economic indicator is required to enable a financial contract to be settled between the building owner and the aggregator or grid operator. It is somewhat difficult to capture the deviation from optimal thermal comfort due to the varying standards and thermal comfort requirements from different occupants but an indicator could be based on the amount of time that absolute thermal comfort limits are breached.

Conclusions & future work
The method by which building energy flexibility metrics are defined and quantified is an open issue with no real standardisation in place. Such metrics should allow easy quantification and interpretation of demand side management measures by both the demand side (i.e., building owners or operators) and the grid side (i.e., grid operator or aggregators). They should also take into account existing demand response frameworks and be compatible with them. A review is conducted analysing the robustness of market-independent indicators used in literature to a range of boundary conditions such as control type, climate, building type and demand response strategies.
The capacity indicator was found to be almost identical in the literature reviewed and a consolidated version is presented here. An indicator ( E rb ) is recommended to measure the size of the prebound or rebound energy and this is of most use to the grid side. In terms of efficiency of harnessing energy flexibility, if considering energy and carbon-dioxide emission reduction, the 'AEEF' and 'DR,P' indicators are recommended for use by the end user; however, the grid operator may be more interested in the 'Flexibility Performance (f)' indicator which considers the rebound effect to be disadvantageous and is a measure of this. Overall, the proposed indicators are shown to be relatively robust to different building types, climates and control schemes (RBC and MPC).
A type of indicator that has not received much attention in this work is an economic indicator. The primary reason is that the focus of the indicators considered was to be market independent. Given a DR request, it is vital to know the economic cost of implementing a DR action (given cost-optimal MPC, any DR action will always be a deviation from the optimal). A building owner will be interested in financial compensation from complying with a DR event and the aggregator or the grid operator requires this information to work out incentives. In the application of these indicators and quantifying flexibility for a given case, based on the specific market context, the economic cost can be calculated for the provision of flexibility.
In this study, it was decided to focus on demand response as applied to the electricity grid as the particular challenges faced by the electricity grid has led to the creation of demand response and ancillary services markets to date, unlike in the natural/LNG gas or district heating energy markets. The concept of these indicators could also be extended to these other energy vectors but this is left for future work.
This research did not consider the case of multiple and consecutive DR events occurring within the analysis horizon. Given the complex dynamics of some buildings with longer time constants, it is quite unlikely that the effects of consecutive DR events will be independent for these buildings. This will be the subject of future research. Another limitation is that the current work is limited to consideration of a single building. Although identified as stakeholders earlier, both the grid operator and aggregators are interested in quantifying energy flexibility and harnessing energy flexibility for a portfolio of buildings, which could range to many thousands of buildings. The question arises whether the indicators considered in this paper can be applied to a population of buildings to understand the energy flexibility potential of a portfolio of buildings as well as to potentially rank these buildings in their suitability or value given a certain demand response need. Future work will investigate the application of such indicators to a population of buildings and a framework for assessing such a portfolio.
The proposed set of indicators are recommended as a standard set of terminology and nomenclature in quantifying energy flexibility of buildings and allows the basis for a standard to be formed that can be used by the stakeholders including end-users, aggregators and grid operators. This would help enable a smart grid where building energy flexibility plays an integral role.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.