Renewable and Sustainable Energy Reviews Do domestic heating controls save energy? A review of the evidence

Claims about the bene ﬁ ts of heating controls are often biased, unsubstantiated, misleading, or incorrect. This paper presents a systematic and critical international review of the evidence for the energy saving, cost e ﬀ ectiveness and usability of heating controls. The focus is domestic, low-pressure hot water heating systems in temperate climates. Eleven di ﬀ erent types of standard, advanced and smart controls are assessed plus ﬁ ve components and features that add smart functionality. The review retrieved over 2400 documents from on-line databases and other sources. Screening criteria and quality assurance scoring identi ﬁ ed just 67 items, mainly from the UK and USA, which appeared to contain relevant evidence. This evidence was derived from computer modelling, ﬁ eld trials and full-scale experiments, and for usability, from expert evaluations and controlled assessments. The evidence was synthesised and its quality classi ﬁ ed as very low, low, moderate or high using the GRADE system which is more commonly applied in evidence-based medicine. The energy savings of most heating controls depends strongly on whether the heating system is operated with a continuous or periodic heating pattern, as well as on the energy e ﬃ ciency of the dwelling and the severity of the climate. For most control types, the quality of the evidence for energy savings was low, very low or non-existent. However, there was moderate quality evidence that, when appropriately commissioned, zonal controllers, which heat individual spaces to di ﬀ erent temperatures at di ﬀ erent times, could save energy compared to whole-house controllers, and that low-costsystemsofthistypecouldbecost-e ﬀ ective.Therewasmoderatequalityevidencethatsmartthermostatsdonot save energy compared to standard thermostats and programmers and may, in fact, increase energy demand. The usability studies focussed on general heating controls and programmable thermostats and provided high quality evidence that heating controls are di ﬃ cult to use, especially by older people. However, no studies were uncovered that quanti ﬁ ed the consequent energy penalty. There was no high quality evidence about the impact on energy demand of any of the heating controls studied, mainly because there have been no well-founded, large-scale, multi-disciplinary, multi-year ﬁ eld trials.

are also provided to enable spaces to be heated to the occupants' desired temperature. Originally operated entirely manually, such controls have become progressively more sophisticated and increasingly automated, and, very recently, remotely operable through digital, wireless communication protocols. At the same time, there has been increased recognition of the need to save energy and reduce greenhouse gas emissions due to the burning of fossil fuels. Consequently, the energy saving possibilities of heating controls have become a focus of interest.
Credible, unbiased, documented evidence about the energy savings of heating controls is essential if they are to be promulgated by governments, local authorities or others concerned with the domestic energy efficiency. Too often, claims about the benefits of controls are biased, unsubstantiated, misleading, or incorrect. Superficially compelling evidence often evaporates when studied in detail: test conditions are unrealistic, trials use atypical households, there is no monitoring before controls are introduced to provide a benchmark for calculating 'savings', etc. Trials can have poor characterisation of the dwelling, heating system and occupants and they often have few participants, which makes it impossible to extrapolate findings to the wider population or to identify which homes, with which occupants would benefit most. Robust evidence-informed policymaking is therefore very difficult.
This research utilises a systematic review to grade the quality of the global evidence about domestic heating controls, their potential to make energy savings, ease of use and cost-effectiveness. The work was undertaken as part of the UK government's Smarter heating controls research programme, which has run since 2012, and aims to develop the heating controls evidence base to inform policy development in this area [1].
With the exception of the rapid evidence assessment of Munton et al. [2], previous relevant reviews lack critical synthesis, being merely summaries of the literature with heating controls considered in the broader scope of heating systems (e.g. Consumer Focus [3]; Meier et al. [4]; NHBC Foundation [5] and Peffer et al. [6]). Such reviews simply map out the current state of knowledge, whereas systematic, critical reviews, such as this one, provide new analysis, synthesis and a grading of the evidence [7]. This paper integrates and expands research presented in two government publications [8] and [9]. These were commissioned partly in response to the review of Munton et al., conducted for the Department of Energy and Climate Change (DECC), 1 which concluded there was no rigorous evaluation of the effect of improved heating controls on household energy demand. This paper reanalyses the evidence, provides an in depth critical assessment and, most importantly, provides a grading of the quality of the evidence. To the authors' knowledge this is the first time that the grading system has been used in this field of research.
The approach used here is fully described such that others might mirror the process in future examinations of this, or related, topics. The systematic review, synthesis and grading of the evidence is fully documented, a classification of heating controls is presented, and the quality of the evidence for seventeen standard, advanced and smart control types is tabulated. The details of the literature search strategy can be found in the supplementary material (available at [10]) and the documents that the screening process identified as containing relevant information are listed in the Appendix to this paper.

Domestic heating systems and controls
In this paper, heating controls are defined as 'Controls that allow the central or local regulation of temperature through the heating system'. The focus is predominantly on controls that are applicable to domestic hydronic, low-pressure hot water systems such as the modern system illustrated in Fig. 1. The system shown has a conventional boiler and a hot water storage tank, but systems may have combi-boilers that heat hot water at the time of use and so do not need a water tank. 2 Control of space temperatures is the raison d'être of a heating system and so boiler or room thermostats are intrinsic features, even in older systems. Eleven types of heating control have been identified based on their functionality, which can be divided into two broad categories: standard controls and advanced controls (Table 1).
Standard controls are installed primarily to ensure that thermally comfortable conditions are provided and that the system operates in a  [11]). safe, reliable, efficient and maintainable manner. They enable the 4. Reducing the temperature to which spaces are heated.
Gradual improvements to the design of boilers means that modern domestic heating systems (e.g. Fig. 1) can have in-use efficiencies of 80-90%. Standard heating controls (Table 1) ensure the system's overall efficiency, safety and maintainability without the need for direct occupant intervention. It is therefore difficult to find further efficiency gains, so the energy saved by installing advanced, non-smart controls (Table 1), is likely to be small.
Standard heating controls offer opportunities to reduce energy use and cost by setting lower house and room temperatures, shortening the heating periods or reducing the number of rooms that are heated. Simple lock-shield valves 3 (Fig. 1) enable the spatial extent of heating to be varied. Manual TRVs control both the extent and degree of heating; and zonal control, using programmable TRVs, also permit individual rooms to be heated at different times of the day. Modern space temperatures controls (Table 1), such as programmable thermostats or zonal controllers, seek to make such control easier, enabling temperature set-back (during the night for example) or permitting different spaces to be heated to different temperatures. 4 Given the control capability and energy saving potential of standard controls it is difficult for new control systems to provide tangible and worthwhile benefits.
Because people are an integral part of the control/system feedback loop, the usability, as defined in ISO/DIS 9241-11 [17], is crucial to encouraging and supporting energy efficient behaviour and to sustaining such behaviour over time. However, individuals' use of the controls depends on many factors, including the controls' design: are the dials and switches accessible? are they readily manipulated, for example by elderly people who may be less dexterous? are the controls' labels and the readout/feedback readable, even by the poorly sighted? and is it clear what must be done to achieve a desired effect?
The daily periods of winter heating are usually dictated by a central timer or programmable thermostat. However, these controllers enable occupants to manually override timer settings to turn the heating on or off. Sometimes people use the room thermostat or manual thermostatic radiator valves (TRVs) to switch heating on and off. The behaviour of a dwelling's occupants, either individually or collectively, has therefore, a significant impact on energy use. This means that the energy used by similar households living in the same house can be very different, see for example, Urban and Gomez [18], and so too can any saving from new controls. In fact, a new control could save energy in some households but actually lead to higher energy demand in others. Consequently, any quantification of energy savings requires consideration of both the social and technical contexts.
Whilst the designers of controls speak of optimizing, people, as Leaman and Bordass [19] put it, "are 'satisficers' not optimizers". They seek space temperatures that are comfortable enough and will tolerate thermal discomfort if they know it is likely to be short term and optional. In fact, people might enjoy conditions that are, by a classical definition, uncomfortable. The freshness of a cool house in the morning, after a night cocooned in a warm bed, can be pleasurable 5and save energy; but this wouldn't be provided by an optimizing smart-thermostat, or captured by a learning algorithm. Smart controls, which wrest control away from the people, and so diminish their freedom to control their heating as they wish, can therefore lead to increased energy demand.
The variability in human behaviour, set against the magnitude of the energy savings possible, means that measuring the energy savings when a new system, controller or device is installed is difficult. The fuel used in dwellings with the new controller must be compared with that used by other, matched, homes without the controller or with an estimate of what the energy demand would have been had the new controller not been installed. If changes to controls are made at the same time as other interventions it may be impossible to disentangle, and so quantify, the effect of each change, especially as the energy savings from the controls change could be an order of magnitude smaller than those achieved by other energy efficiency measures, such as insulation. Furthermore, changing controls can have a consequential impact on the heating systems' operation. In particular, reducing the load on the boiler and introducing intermittent operation, is likely to reduce the overall system efficiency slightly. Methods of evaluating the impact of controls should account for such effects.
Whether new controls will save energy or not, crucially depends on the heating system and the controls that they replace, because 'saving' implicitly requires a comparison. Very sophisticated controls may save energy when installed in a poorly controlled system, but so might simpler and cheaper controls. Both the new and benchmark system must therefore be clearly defined. The thermal comfort provided by the two systems also matters, and this can be indicated by the space temperatures measured when people are present. 6 Improved controls might not save energy but they could deliver comfort to previously cold occupants. Conversely, reduced space temperatures could increase indoor humidity levels, thus risking damp and mould growth. Unintended consequences such as these become manifest when interventions are made to complex human-technical systems.
The cost-effectiveness of heating controls depends on the fuel use before and after the intervention, the fuel cost, the price of the new controls and their installation cost, and the required payback time and assumed discount rate. These are all factors that vary over time, and sometimes quite dramatically so. Translating cost-effectiveness from one context to another can be difficult. Most of the literature examined in this research focussed on determining the energy savings of controls, with very few documents commenting on cost effectiveness. In fact, many methods used to evaluate controls could not provide and annual energy savings estimate and thus a realistic cost effectiveness figure.
Evidence about the effect of controls needs to be based on data from countries, and areas of countries, that are climatically similar; and in this research, similar to the UK now or in the near future. Extrapolating the results of studies undertaken in one economic, climatic and cultural context to another must be undertaken with care. For example, in the UK, people have a cultural tendency to periodically heat their homes and many people partially heat their home, or prefer cool bedrooms. They are increasingly conscious of winter heating costs and intervene to turn the central heating on or off (but they have fast-responding, often oversized, hydronic heating systems). In fact, many UK homes have secondary heating in the main living room, such as a gas or electric fire or, increasingly, wood burners, which recreates the focal point that open fires once provided, and which may reduce the time for which the central heating system is used. 7 Elsewhere, notably in the USA, continuous wholehouse, air-based heating, perhaps with night set-back, within a colder winter climate, is more common. Many systems in the USA use a heat pump rather than a boiler and operate year-round to provide cooling in the summer. Clearly, if the benchmark for calculating savings is a heating system that is on continuously, reducing the thermostat setting and/or turning heating off when people are absent is likely to save more energy than if the benchmark is a system that is operated periodically. 8 3 Lock shield valves permit the flow of hot water to an emitter to be turned off, thus isolating the emitter for maintenance purposes. They are not very effective for controlling the temperature of the emitter. 4 Sometimes, measurements of temperature are used to estimate fuel use. However, estimates of the energy savings using this approach are likely to be very inaccurate, not least because the relationship between temperature and energy demand is weak and temperatures can change both spatially and temporally. 5 For more on thermal alliesthesia, see Parkinson and de Dear [20]. 6 The presence of otherwise of people is important. Some controls save energy my lowering the temperatures when people are not present. 7 Thus, in the UK, studies of domestic heating energy demand need to consider the energy consumed by secondary heating devices as well as the central heating system. 8 In this research, energy saving claims for systems installed in the USA were inspected very carefully, in fact some air-based systems also used energy in summer to provide cooling.

Methodology for assessment of the evidence
To identify relevant documents, a systematic evidence review was conducted in two parts, both of which followed the Government Social Research Service guidelines [21]; one focusing on the UK evidence base [8], and the second updating this and extending it to the international literature [9].
The two reviews used the same transparent, systematic and

Table 2
Screening criteria applied to abstracts and entire documents (adapted from [8] and [9]).
reproducible search strategy, which identified and excluded a different number of documents at each stage of the process (Fig. 2). Then, clear and objective criteria were used to appraise the quality of the information and to synthesis this in order to produce credible and defensible conclusions about each control's energy saving potential. Ten databases and compendia of publications were identified, which, when taken together, cover the main sources of relevant documents that are in digital form [10]. Search criteria tailored to each database were created in order to identify and extract the relevant documents. The final search strings were devised by firstly conducting preliminary searches using draft search queries. The databases accessed, and a description of them, together with the final search queries, the strings, words and Boolean operators, are given in the Supplementary Material to this paper [10] in a form that allows others to use them in order to repeat or refine the work reported here.
A two-stage process was used to select the documents that would be studied in detail (Table 2), firstly based only on the information in the abstract only and then based on the content of the entire document (see Fig. 2). The criteria used were substantially the same for both the UK and international review. However, the international review sought information about experiences with heating controls only for climatic zones similar to that which the UK experiences now, or will experience in the near future [22]. This was taken to be Köppen-Geiger category classifications [23]: Cfb (temperate oceanic climate), which includes much of Europe, SE coastal Australia, Central Chile and Eastern USA; Csb (warm summer Mediterranean), which includes the West coast of the USA, Spain and Portugal; and Csa (hot summer Mediterranean), which includes Southern Spain.
Although the UK review focused on a narrower range of control types (see Table 1), the international review subsequently searched for UK evidence about the remaining types. The key point about this screening process is that it is able to identify, from within the many documents uncovered by the database searches, those that may contain useful information and do this in a relatively straightforward, repeatable and documentable way. It does not however, provide any indication of the quality of the evidence.
The quality of the reporting and the research within each of the documents that passed the screening criteria was assessed using a quality assessment scale developed for this research (Table 3). Each document could score from 0 to 9 points and those which scored 6 or more passed through to be read thoroughly and the evidence synthesised. Some of the elements in the scoring table entailed subjective judgement, therefore, to ensure the reliability and replicability of the process, for both the international and UK reviews, a sample of documents were scored by at least two researchers. Any small differences in the score did not alter the judgement on whether the document should be included in the synthesis of evidence.
The 67 documents that passed the quality assurance threshold are listed in the Appendix which also indicates the control types that were assessed and the method used for making the assessment. Of these documents, 45 were deemed to provide evidence on the energy saving potential of controls, but only five provided evidence about cost-effectiveness and only three about usability. 9 A further 24 documents discussed usability in isolation of energy savings. Some documents about usability did not provide useful information 10 and so were not considered further. These findings suggest a lack of integrated, multi-disciplinary collaboration between researchers from different disciplines; collaboration that can be invaluable to understand building energy use [34].
Key features of the BEIS quality assessment scoring is that it is based on the clarity of the written material, the rigour of the peer review process, and the credibility of the document's authors, which was invariably the team that conducted the research. Also, not all the points are needed for the document to pass. Thus, even documents that score 6 or more may contain very weak evidence of energy savings, cost effectiveness or usability.
The critique and synthesis of the documents sought to uncover and classify the strength of any evidence. Evidence-based assessment is most highly developed in the medical field and the well-known GRADE system (Grading of Recommendations Assessment, Development and Evaluation) but with the modification of the Evidence-Based Medicine guidelines editorial team [35] is adopted here without modification (Table 4). In translating this to the evaluation of heating controls, the strengths and weaknesses of different assessment methods needs to be considered. For some controls there may be no evidence at all.

Methods used to evaluate heating controls
The documents examined provided evidence about energy savings, cost effectiveness or usability based on six different research methods. The methods reported in each of the 67 documents that passed the screening and QA process, are given in the Appendix. Most documents report research using a single assessment method, but many used that method to evaluate more than one control type. The different methods are described in this section, and the inherent advantages and disadvantages of each, which strongly affect the strength of the evidence they can provide, is summarised in Table 5.
Computer modelling and full-scale experiments do not enable the real-world interaction between people and controls to be assessed, although both can test the effect of different prescribed modes of Table 3 The quality assessment scale (adapted from [8] and [9]).

Points Score
Quality assessment question Reporting Quality 0 or 1 Does the author or publishing organisation have a credible track record in the area? 0, 1 or 2 Are the rationale and research questions clear and justified? 0, 1 or 2 Does the document acknowledge funding sources, project contributors and advisors, and list possible conflicts of interest? 0 or 1 Are the methods used suitable for the aims of the study? Research Quality 0, 1 or 2 Has the document been peer reviewed or independently verified by one or more reputable experts? 0 or 1 Do the conclusions match the data presented? Table 4 Quality of Evidence classification (source: [35]).

Quality of Evidence Definition
A High Further research is very unlikely to change our confidence in the estimate of effect.
• Several high-quality studies with consistent results • In special cases: one large, high-quality multi-centre trial B Moderate Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
• One high-quality study • Several studies with some limitations C Low Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
• One or more studies with severe limitations D Very Low Any estimate of effect is very uncertain.
• Expert opinion • No direct research evidence • One or more studies with very severe limitations 9 Although it would be possible to estimate the likely cost effectiveness from the data in the documents, this hasn't been attempted here. 10 Four documents were reviews [3][4][5] and [6] and so did not provide additional primary evidence; four, all from the USA [24][25][26] and [27] described how controls might save energy but provide no evidence for savings; and seven [26,[28][29][30][31] and [32] and [33] used very small samples and focused on matters other than heating controls so did not contribute useful evidence. controller operation. They also fail to capture the interactions between control installers and dwelling occupants, a process which can influence occupants' understanding of the controls and how best to use them and the default settings that are provided at installation. Full-scale experiments do, though, produce a clear-cut measure of energy savings for the particular experimental conditions chosen, something other approaches struggle to achieve. Experiments also reveal any unanticipated interactions between different control components. Field trials, using occupied homes, expose the controls to the full socio-technological complexity of home heating. They can therefore Size of cohort means that many locations, types of dwelling, configurations of heating system, and occupancy types cannot be studies. Effects of controls that are not represented in the cohort may not be captured. Thus the variability in energy savings when controls are deployed widely cannot be fully quantified. The pre-existing heating system and controls defines the benchmark against which the new controls are compared. The pre-existing system is not tested under the same weather as the new controls so savings quantification requires modelling. The fabric heat loss of the dwellings cannot be fully characterised.
Practical retrofit installation problems that occur in existing homes revealed. Large-scale field trials in occupied homes Direct monitoring in occupied homes possibly with associated physical survey and/or occupant survey.
Provides an insight into full, real world, sociotechnical complexity of control use. Range of house types, weather conditions and pre-existing heating systems can be included. Longitudinal studies allow for comparison over time and for pre-and post-intervention studies (e.g. changing control type). A large control group, that has no heating system intervention can be established.
Expensive, time consuming and complex to set up and maintain. The pre-existing heating system and controls defines the benchmark against which the new controls are compared. The pre-existing system may not be tested under the same weather as the new controls so savings quantification requires modelling. Large cohort captures wide range of home, heating system and occupant types.
Interaction between controls installers and occupants can be studied.
The fabric heat loss characteristics of the houses cannot be fully quantified. Practical retrofit installation problems that occur in existing homes revealed.
A substantial data management system with robust confidentiality may be required. Statistical analysis needed to extract energy savings and the impact of exogenous factors. Cohort maintenance over prolonged time period can be difficult.

Usability only Expert evaluation
Heating control features assessed against established usability criteria by an expert Can be quick and relatively cheap to perform.
Relies on the quality of the expert and their understanding of the heating control in the wider system. May not identify difficulties faced by nonexperts.
Requires suitable usability criteria and interpretation of these.

Controlled assessment
Heating control assessed against established usability criteria in a controlled environment by selected participants Provide end user perspectives on interaction with controls. Controlled experiments possible, with specific metrics such as time to perform a task, or number of errors made, allowing comparison across control types. Usability criteria are usually high level and generic and so may not provide targeted or detailed assessment of heating controls. Do not provide evidence of actual use or the consequences of poor usability in real homes. produce high quality evidence about the effects of controls and are an essential step in the development of controls with which people interact. Field trials, especially small-scale trials, may enable the contemporaneous evaluation of usability and energy savings, although it can be difficult to separate out the usability of a specific control from the usability of the whole heating system. Field trials can also track how occupants' interaction with controls changes over time and between households of different composition. In large-scale field trials, the effect of different dwelling characteristics on energy savings, especially different insulation standards, can be isolated, but identifying the many socio-technical effects and accurately quantifying the influence of each one is difficult. Expert evaluations and controlled assessment, in contrast to the other methods, can provide insights into the usability of specific control features but not into energy savings and cost effectiveness, except anecdotally. Large-scale field trials using a multi-disciplinary, mixedmethods approach may effectively assess energy savings and cost effectiveness as well as usability.

Computer modelling
The computer modelling studies in this review were of two main types. Firstly, the use of simplified, standard, building energy performance models, such as normative, national, policy-driving domestic energy rating tools, which in the UK is the Standard Assessment Procedure (SAP) [36]. For example, Firth et al. [37] undertook wholehouse energy efficiency assessments using BREDEM8 [38] which uses a calculation method similar to the SAP. Simple models represent reality by making gross approximations to create simple algorithms to describe controls' effects. The model predictions reflect the encoded, presumed behaviour of the heating system, controls and occupants rather than providing new evidence of their actual effects.
Secondly, the use of dynamic thermal simulation models, which aim to capture the thermal physics of the interaction between the geometrical form and construction of the building, the weather and the heating system, controls and occupants. Such models include EnergyPlus [39], TRNSYS [40] and ESP-r [41] that have a long pedigree, thousands of users across the world, and have undergone traceable, and reasonably extensive, validity testing. The models' predictions are, though, very dependent on the assumptions made by the model user, most notably about the characteristics of controls and how they are used. In this review, particular weight was given to studies that included a comparison with, and/or calibration against, measured data relevant to the particular situation being studied. The work of Rogers et. al. [42] reports a comparison of modelled performance with predictions of another model rather than with real monitored data, so this work is not considered further herein.
More generally, it should not be presumed that the algorithms encoded into models are based on the actual effects of real controls, for example as measured in occupied dwellings. This is so even for such models that simply try to capture the best available evidence, which may well be very weak. Furthermore, some models, such as the SAP, ignore user effects, for example by assuming that all homes will be heated to the same pattern. 11 Many documents, often by academics from fields allied to computer science, report the use of computational techniques to create learning algorithms that could be embedded in controllers, in particular to learn when occupants will be at home. These studies are concerned with the development of control algorithms rather than independent testing of the final product's energy saving capability. Where algorithms are developed, then trained and tested using monitored data, usually from occupied buildings, they are included below in the small or large-scale trials sections.

Full-scale experiments
The full-scale experiments were also of two main types: trials in unoccupied single houses, or a house-like 'building'; or, trials in unoccupied, matched-pair houses. For both types of experiment, occupancy can be synthesised by opening and closing windows and doors, changing the heating set-point and/or schedule, and turning on and off appliances and heat sources to represent the movement and actions of people.
Experiments in a single building are often undertaken sequentially so the weather changes from one test to the next. Some form of 'model' is therefore needed to normalise the results to a common weather basis and this makes it difficult, or impossible to detect small energy-savings. An alternative is to alternate from one controller to another on alternate days, which may improve the experimental resolution.
Matched pair testing enables two nominally identical houses to be exposed to the same external weather conditions and the same prescribed, synthetic occupancy behaviour. Different controls can be installed in each house, and because both experience the same boundary conditions, quite small differences in the energy demands can be detected.
It is possible to synthesise 'weather', as well as occupants' behaviour, though at some cost, by building a house inside a larger, climatecontrolled building (e.g. Fitton et al. [43]). Of course, actual homes, built as they are in practice, and which have been exposed to the elements (perhaps for decades), cannot (easily) be used. The approach does enable sequential testing at any time of the year, but the synthetic weather may be a gross simplification of that which is found in reality.
With any of these experimental methods, if the houses are wellcharacterised, predictions by either empirically-based models or by first-principles building physics models, can be made to extrapolate the measured energy savings to other weather and occupancy situations.
The key here is that it is the savings that are extrapolated and not the absolute energy demands (from which savings must then be calculated by subtracting one large number from another large number to find the potentially small difference).

Small-scale field trials
Small-scale, in the context of this paper, is loosely defined as around 20 or so homes. Cohorts of this size enable a level of detailed data collection that is usually not possible in large-scale trials. This might include: physical house surveys and heating system characterisation; detailed energy demand monitoring; weather monitoring; ethnographic and/or interview-based recording of occupants' behaviours and attitudes; and, potentially, monitoring of parameters that can help explain any changes in energy use, such as, internal temperatures, electricity demand (for internal heat gain estimation), the operation of controls, and disaggregation of heat and hot water energy use. Very small field trials, of one to five homes, have been used for the development of model predictive controllers, notably those that try to learn when occupants will be at home so that comfortable temperatures are provided when, and only when, needed.
In intervention studies, the pre-existing heating system, controls, and occupant behaviours, and the consequential energy demand, provide the basis for appraising any changes to the heating controls. The measured energy savings and usability assessments may be unique to the specific socio-technical situation (including the pre-conditioned occupancy behaviours) into which a new controller is deployed. Because the cohort is small, the full variability in the energy savings achieved by a controller may not be accurately quantified. Such variability may of course include increases in energy demand as well as the hoped-for decreases.
Field trial interventions that are conducted sequentially produce even more analysis difficulties than sequential experiments (see 5.2 above) because the behaviour and number of occupants can change as well as other external factors, including the weather. Thus, the uncertainty associated with the calculated energy savings for each house can be quite large.
Small-scale field trials are frequently reported in connection with the development of model predictive controllers, notably those that try to learn when occupants will be at home so that comfortable temperatures are provided when, and only when, needed. Such trials may be very small in scale, for example, this review revealed five different trials with between one and five homes. Apart from the small scale, there are other limitations such as short duration, the use of researchers' homes and/or the inference of energy savings from the reduction in heating duration rather than actual measurements.

Large-scale field trials
Large-scale field trials can embrace the full diversity of dwellings, heating systems, controls and occupant behaviour encountered in society as a whole. This enables, for example, the measured energy savings to be extrapolated to the national level as an aid to policy development. However, because any energy savings will be small compared to the naturally occurring inter-dwelling variability, the cohort sizes may need to be large, see e.g. Heap [44].
Robust field trials, of sufficient size, are very expensive and hence rare. They require a period of monitoring, probably a whole winter, to determine the baseline energy use and the internal temperatures. A period of monitoring of similar duration is also needed after any intervention to change the heating controls. A thorough survey of the homes and occupants before the intervention helps in understanding the reasons for any measured differences in energy demand.
Multi-year field trials require diligent cohort management: to track changes in occupancy or radical changes in occupant behaviour; to maintain the monitoring system and so ensure an unbroken flow of data; and to manage the risks associated with interventions to peoples' heating system. Such trials therefore require a dedicated and cohesive team of investigators that can work together for a long period of time. Multidisciplinary teams, which can cover the technical, social and analytical aspects of the project, are ideal. Because of the costs and time involved, well-conceived trials of sufficient scale are rare.
It is important to distinguish between research trials that involve many discrete, spatially-distributed, homes, each with a separate heating system, and those which reported evidence from apartment buildings in which multiple apartments are served from a single plant room with a number of boilers that serve a whole-building pipe network. In the latter case, some controls may work at the apartment level (such as TRVs) whilst others (e.g. weather compensation) might affect the central plant. The cohort may only represent a narrow sector of a whole society.
Evaluations of the use and usability of heating controls have been undertaken at scale, through: the gathering of opinions by surveys or interviews; self-reported use diaries; heating control audits; and as part of wider investigations about energy consumption. In some cases, the numbers of participants are over 1000, but in many, there are less than 50. Within this paper, these smaller cohort studies have still been classified as large-scale, to keep a consistent definition, but it is recognised that a survey of 30 people will not be considered large in some disciplines.

Expert evaluation
Evaluations of the usability of controls can be undertaken by experts, often against usability criteria, without the direct involvement of users. Usability assessments often form part of a suite of methods and provide an initial review of an interface before more interactive trials are undertaken. Usability criteria, or heuristics, can be from established sources (e.g. Nielsen [45]) using generic principles of good interface design or bespoke. Experts have used an Exclusion Calculator in conjunction with a hierarchical task analysis to estimate the number of people excluded by the design of a domestic heating controller [46]. Others developed a functional usability assessment matrix to evaluate controls found throughout the home, including heating controls [32].
In an expert evaluation, it is typical for the features of the heating controls to be systematically assessed against the usability criteria, resulting in pass/fail or, more typically, a score for each control, whereby the higher the score the more usable the control. As expert evaluations do not require the involvement of users, they can be relatively quick and   cheap to undertake, but rely on judgement and do not enable evaluation of real-world context. Crucially, the level of usability of heating controls does not necessarily translate to their energy saving potential.

Controlled assessment
To involve users in assessments, controlled assessments can be undertaken. These provide an end-user perspective on the interaction with heating controls, employing usability metrics such as time to perform a task, number of errors made or route taken to navigate the system, allowing for quantitative and qualitative comparison across control types. These trials are usually undertaken in a controlled environment, asking participants to complete specific tasks; to date this has most often been undertaken with programmable thermostats which are often linked to simulations of the heating system to provide artificial feedback to the user. As the set tasks are defined by the research team, it is possible that evaluations are undertaken of aspects of the system that may never be used in practice, in particular relating to system set up, which may be completed by an installing engineer or another family member who is not responsible for day-to-day operation of the controls. As controlled assessments are, by nature, somewhat artificial, they do not provide evidence of actual usability or the consequences of poor usability, and the impact on energy use, in real homes.

Critique and synthesis of evidence
The critique and synthesis for this review required careful reading of the 67 documents that met the quality assurance threshold. The results of this are outlined below for each control type, noting in particular the applicability of the evidence for hydronic systems with gas boilers that are run using a periodic heating schedule. There was no evidence at all for some control types.
The evidence for each control type was combined and the overall quality graded using the GRADE system (Table 4). Where the quality of the evidence, as summarised in Table 6, is moderate or high, the energy savings and cost effectiveness are stated.

Standard controls
Evidence about standard controls was limited; in fact, there were no documents about on/off switches, boiler thermostats or central timers. This is, perhaps, not surprising as these controls provide the functionality necessary to ensure systems are safe and maintainable and provide basic thermal comfort. Some components have been integral to UK heating systems for decades and demanded by the UK Building Regulations [12,47] so there is, therefore, little incentive for manufacturers and others to evaluate them.
Awareness of a household's energy use, and an interest in reducing it, are precursors to reducing energy demand. However, in a survey of over 1700 Dutch households, Brounen et al. [48] found that only half the households knew their energy consumption and understood how energy efficient their houses were but many of these did not use the available controls to save energy. The study suggests that knowing and acting are only loosely linked. Perhaps by making households aware of their energy demand and then even standard controls could be used more effectively.
Critchley et al. [49] studied 888 UK low-income households, in which the temperature was measured in two rooms, twice a day for 1-2 weeks; energy demand was not measured. Telephone interviews with a subsample of 79 people, indicated that controlling the heating system was a problem for many, with around 33% of those over sixty saying,"they were too complicated". Likewise, following a controlled assessment with 75 UK participants, Wall and Healy [50] found that older people had difficulties using their controls, in part due to poor eye-sight and/or lack of dexterity. They also found the high cost of the controls and the complexity of installing and setting them up acted as a barrier to their adoption and eroded any net financial benefit from energy savings.
A UK study of over 1500 social housing properties in Newcastle during one winter (October 2013-May 2014) [51] found that neither an information leaflet nor in-home advice from a controls engineer significantly reduced gas consumption compared with a control group (where no advice was given). The authors speculated that there may have been a different result, had a group with higher energy use been studied. Qualitative interviews with 61 participants suggested that the advice did actually help people to use their heating controls better. However, some households may have used the knowledge to improve their thermal comfort rather than reduce energy consumption. An unintended, but for households in under-heated homes, valuable effect of installing new controls may be an improvement in house temperatures. Studies of the effect of heating controls thus need to measure indoor temperatures and energy demand before and after installation.
Wade et al. [52] examined the role of the heating engineer more closely in a UK-based ethnographic study. The installers tended to select particular devices for particular users, e.g. standard, simple controls for elderly households, smart controls for those alert to new technology and programmable thermostats for families, and provided guidance on controls' operation and the initial settings. This suggests that there is tacit knowledge in the industry that different controllers suit different household types; although this knowledge may or may not be correct.
Some researchers have tried to make inferences about the energy saving of heating controls by using data from large-scale trials that were undertaken for other purposes. For example, Kelly et al. [53], Shipworth et al. [54] and Shipworth [55] use temperature data from c. 427 homes collected as part of the CARB project [34] to understand thermostat use, but no direct evidence of energy savings was provided. Similarly, the systematic evidence review identified the 2011 EFUS, 12 which included 823 UK homes, as a potential source of evidence [56] but this too contained no conclusions about the effect of heating controls.
Ahern and Norton [57] compare the energy demands of 45 dwellings, with the demand in 19 dwellings and 11 multi-family apartments that had undergone renovation of the whole heating system. The work reported energy savings but could not indicate the savings attributable to each type of control.
Taken together the studies of Critchley et al. [49] and Wall and Healy [50] supported by Brouen et al. [48], provide high quality evidence ( Table 4) that standard heating controls are difficult for people to use, especially those who are elderly. However, these studies provided very low quality evidence about how usability affects energy demands and the monitoring studies provide no evidence at all.

Room thermostats
Fitton et al. [43] conducted full-scale experiments in a UK house, which represented a typical, solid wall, end of terrace house of the late 1800s, heated by a gas-fired condensing boiler. The house was built in an experimental laboratory and exposed on the outside to a fixed temperature of 5°C. It was heated for nine hours, from 07:00-09:00 and 16:00-23:00 for two days, with the second day being the test period. In the first trial, there was no temperature control other than the boiler thermostat but in a second trial a 'room thermostat' 13 was installed. (A third trial, with TRVs added, is reported below). Reported energy savings were up to 12% but in the house with no controls, the internal temperatures were between 20 and 31°C and even with the thermostat, between 20 and 29°C. Such high temperatures are unlikely to be acceptable to most households and the steady-state external temperature is unrepresentative of the real world. Therefore, as noted in [43], the results lack realism, and the energy savings measured are unlikely to reflect the savings from real controls in real homes. The shortness of the 12 The Energy Follow up Survey (EFUS) is undertaken to add further information to that acquired via the bi-annual rolling English Housing Survey, notably about energy demands and internal temperatures. The latest EFUS is collecting data for 2017/18. 13 Actually, a calibrated air temperature sensor was used because the thermostat was not sufficiently accurate for experimental purposes! trial precludes any comment on cost effectiveness, and the work offers no insight into the controls' usability.
Overall, therefore, the quality of the evidence for the energy saving potential of room thermostats is very low (Table 4) with no evidence about cost-effectiveness and usability.

Thermostatic radiator valves
Evidence on energy savings and/or cost effectiveness of TRVs is derived from full-scale experiments and computer modelling, sometimes in studies that combined both methods.
About 40 years ago, Rayment et al. [58] fitted a pair of semi-detached homes with equipment to synthesise the effects of occupants. In one house, TRVs were tested 14 whilst the other house was controlled with only a thermostat. The houses were operated in numerous different synthetic occupancy modes and the energy demands of the two homes compared. There was no discernible difference in energy use between the house with the TRVs and the other house.
Working in Turin, Italy, Monetti et al. [59] tried to calculate the effect of installing TRVs in an early C19th historical building that had been converted into large apartments, three on each of the first to fourth floor. Commercial space occupied the ground floor and 11 studio apartments comprised the attic-like roof area. The building was connected to a district energy system via the basement heat exchanger. Using TRNSYS, several different TRV control scenarios were modelled, including control whereby the apartment temperatures differed by height up the building and differed spatially by room type. Although the predictions suggested heating energy savings of 2-10%, with a payback time of sevenyears, there were no data on occupant behaviour, limited information about the heat loss characteristics of the buildings and the model was calibrated only against the limited measurements of the heat input to the whole building.
Focussing on documents concerned with the energy savings from TRVs in the USA, Dentz and Ansanelli [60] report that savings can be up to 15% [61]. However, TRVs are not generally used in the USA as a residential retrofit measure. Dentz and Ansanelli report their own small-scale trial of TRVs that were installed in an apartment block in Flushing, New York. The building was heated from a single, wholebuilding boiler using a one-pipe steam heating system 15 with the heat emitters located inside protective cabinets; a configuration quite different to the low-temperature, hydronic systems found in the UK. Although TRVs were fitted throughout the whole apartment building, monitoring was conducted in just two, very similar, first floor apartments, where room air temperatures and emitter temperatures were monitored. Analysis of heating bills showed no energy savings, either for the building as a whole or for individual apartments. Several explanations for this are offered: the existing heating system was not functioning properly; the TRV sensors were not positioned appropriately; failure to optimise boiler control set-points; and occupants' tendency to open windows. The fact that the occupants did not pay directly for the heat they used may also have contributed.
In addition to the two trials in the indoor experimental house (Section 6.1.1), Fitton et al. [43] also conducted a third trial with TRVs installed on all the radiators except for the living room which had the room thermostat; five TRVs in all. The trial protocol, involving 24 h commissioning followed by 24 h of test, was retained. With the TRVs added, the heating energy demand was 42% lower than the benchmark value (boiler thermostat only) and 33% lower than with the boiler thermostat and room thermostat. With the TRVs, room temperatures were between 18 and 23°C; much lower than the benchmark temperatures, i.e. without the TRVs. As noted above however, the unrealistic external temperature conditions (steady 5°C) and the uncomfortably high temperatures in the benchmark trial(s), undermine the credibility of the energy saving estimates. During the TRV trial, the doors between rooms were shut, which is atypical of occupied homes, and would, as the researchers note, lead to improved TRV energy savings.
Overall, the quality of the evidence on the energy savings from TRVs is therefore low (Table 4): the studies provide either no conclusive evidence, or evidence based on work with significant methodological limitations; which leads to very different estimates of the energy savings. The quality of the evidence on cost effectiveness [59] is very low.
6.2. Advanced heating controls: System efficiency controls

Time Proportional Integral (TPI) control
Cockroft et al. [62] report a new component to the ESP-r dynamic thermal model, which was developed to enable the effect of advanced controls on energy demand to be evaluated prior to their inclusion in the normative UK domestic energy model, the Standard Assessment Procedure (SAP). The new ADEPT 16 interface enables different combinations of dwelling type, heating system, control schedule and control type to be evaluated. A companion paper [63], which focuses on the methods underpinning ADEPT, does report a comparison between two thermostats, one with standard on/off control and the other with TPI control. The calibration of the model using temperatures and boiler cycling data collected from test rooms within a larger temperature controlled building is reported, the predictions from the calibrated ESP-r model matched the measurements well for a single period heating schedule.
A comparison is reported between the annual heating energy demands predicted for a house where the heating system is controlled by a standard on/off thermostat and the same house with TPI control. The house was modelled to meet the insulation standards prescribed by the prevailing UK Building Regulations, fitted with a condensing combi-boiler, and heated using a single on period (07:00-23:00). The TPI controller was able to maintain the room temperature using a lower average water temperature, and run the boiler at a lower temperature, and hence more efficient firing rate. It was predicted to reduce the space heating energy demand by 6.2%; the authors state that this "would very likely be deemed cost effective", though calculations are not provided.
Kershaw et al. [64] measured the energy savings achieved in 52 UK homes by installing TPI controllers, in place of the existing room thermostats, in 47 of the homes during the winter of 2008/09. The homes illustrate the diversity that is typical of the UK stock, being a mix of types (detached, terraced and semi-detached) up to 150 years old. However, almost all the homes had modern controls and efficient, condensing gas-fired boilers. 17 Monitoring included space temperatures, heating energy use and the efficiency of the boiler, with a year of data, both before and after the TPI installation, being provided by 28 dwellings. The results showed that, with intermittent heating patterns adopted by the households in the trial, which are typical for UK households, the boilers did not operate at, or near, the thermostat setpoint temperature for very long (Kershaw et al. estimate just 9% of the winter) and so switching, in response to the TPI control signals, was rare. Consequently, TPI control made no difference to the heating systems' efficiencies or to the overall energy efficiency of the homes.
The difference between the savings predicted by the modelling study of Cockroft et al. [63], in which single period heating was assumed, and the field measurements of Kershaw et al. [64], which captured the real-world heating patterns of UK households, starkly illustrates the risk in relying on model prediction. Further, it seems 14 The TRVs has a temperature sensor that could be positioned away from the heat emitter, i.e. unlike modern domestic TRVs, the sensor was not embedded in the actuator that controls hot water flow to the emitter. 15 In such systems the steam condenses in the emitter as heat is lost and the water flows back to the boiler down the same pipe that was used to deliver the steam. 16 Advanced Domestic Energy Prediction Tool. 17 Two boilers had a seasonal efficiency (SEDBUK rating) of B and the rest were Arated seasonal efficiency over 90%. They were a mix of combi-boilers (38) and standard boilers (14) which incorporated a hot water tank in the system. All the boilers were modulating except one. A room thermostat and programmer was incorporated in all but three homes, and TRVs were used in all but nine.

K.J. Lomas et al.
Renewable and Sustainable Energy Reviews 93 (2018)  apparent that, amongst other things, the heating schedule is very important for determining whether or not a TPI controller will save energy. If homes are heated for longer periods, for example all day, or for a prolonged single period each day, the closer control offered by the TPI controller could save energy but in homes that are periodically heated, they may save no energy at all. Overall, whilst the quality of the evidence on energy savings from the modelling work is low, that from the field study is moderate (Table 4). It shows that in the periodically heated homes, TPI controllers did not save energy. The quality of the evidence on cost effectiveness [63] is very low. However, since a TPI controller is very unlikely to result in higher energy demands compared to a standard thermostat, if the cost is low, there may be no harm in installing TPI controllers in association with modulating gas boilers.

Weather compensation
Dentz et al. [65] report the replacement and upgrade of heating controls in three, 3-storey multi-family buildings in Cambridge, MA, USA. All had basement plant rooms with gas boilers supplying hot water to manually controlled, underfloor heating systems in 18 apartments (Building 1), 15 apartments (Building 2) or 8 apartments (Building 3). The upgrade of the boiler controls was motivated by the desire to avoid wintertime overheating and the attendant waste of energy. The installed controls sought to reinstate effective weather temperature compensation (called outdoor temperature reset control) and to improve the night-time temperature set back control.
Comparison of monthly gas consumption in the winter of 2010/11 (pre-intervention) and 2011/12 (post intervention) revealed weatheradjusted space heating energy savings of 18.1%, 10.1% and 15.5% in buildings 1-3 respectively, 18 with payback times of 7.1, 2.9 and 1.6 years. Caution is, however, needed in interpreting this study, especially in a UK context. Firstly, because the new controls replaced malfunctioning, rather than effective, pre-existing controls; secondly, because the controls acted on large (sometime multiple) boilers that served many apartments; and thirdly, because the local ordinances required both minimum day and night time temperatures to be provided.
Using the ADEPT interface to ESP-r (see above), Cockroft et al. [63] modelled the effect of a weather temperature compensator by comparing a house with a weather compensation and a gas underfloor heating system, to a house without weather compensation and wallmounted radiators. Interestingly though, the work does suggest that the type of emitters (i.e. radiators vs underfloor systems) could affect whether or not a weather compensator could save energy.
Lindelöf et al. [66] present a small-scale field trial, of a patented add-on model predictive controller (MPC) for existing weather temperature compensators. The MPC algorithm utilises measurements of ambient temperatures and solar radiation, internal space temperatures, and boiler flow and return temperatures, to build a model of the dynamics of the building. It then replaces the signals sent by the preexisting weather temperature compensator to generate different the water flow temperatures. The trial was undertaken in eight single-family houses in Switzerland and two apartments in Germany, which had a wide mix of fuel types, hydronic heating systems, heat emitters and controls, some homes had a room thermostat and some not. The system was tested in all ten homes, using an alternating test protocol, at least two weeks with the MPC and then two or more without. Energy savings varied from 5% to, in the apartment which had the lowest energy demand, a saving of 60%! The document provides no explanation of how the device could generate such large savings. In particular, the control capability and functionality of the pre-existing weather temperature compensator, controls and heating systems are not described.
Overall, the quality of the evidence supporting the energy saving effect of weather compensation is therefore very low (Table 4); there is no evidence about cost effectiveness. However, as for TPI control, it seems unlikely that boilers with weather compensation would have higher energy demand than those without and so provided costs are low, weather compensating boilers could be installed without risking increased energy demands or diminished indoor comfort.

Advanced heating controls: Space temperature controls
The evidence for the energy saving of advanced space temperature controls is presented here for zonal control, programmable thermostats and smart thermostats, which are sometimes called model predictive controllers (MPCs). Occupancy sensors and learning algorithms provide input to MPCs and so contribute to their overall performance. Whilst no specific evidence was uncovered about geolocation and geofencing, MPCs could make use of such features. This could be enabled by an app installed on a mobile device that might also enable human intervention, for example to override automated heating patterns.

Zonal controls
Scott et al. [67] describe a prototype zonal control system, with learning capability, installed in two gas-heated homes in Cambridge, UK. The study, called PreHeat, examined the potential of "occupancy sensing and historical occupancy data to estimate the probability of future occupancy, allowing the home to be heated only when necessary." The homes, occupied by the authors' colleagues and project workers, all had two adults and at least one child. One home had underfloor heating and wall-mounted radiators, whilst the other only radiators. Six of the eight rooms in one house and eight of the ten in the other had occupant motion detection and the opportunity to specify the set-point temperature in each room. The occupancy sensors signalled to a computer in each house which determined, based on historical occupancy data for the house and room, if the room should be heated or not. The computer then activated the relevant thermostatic radiator valve. The learning algorithm turned the room emitter on three hours ahead of the anticipated occupancy time so that rooms were pre-warmed. The occupancy sensors also meant that presence was detected even when it was unexpected, which reduced the number of times occupants intervened to manually override the TRV settings. Studies lasted 48 and 61 days during the winter of early 2011, alternating a day with the zonal-learning with a day of normal (programmer) control. This approach, compared to trials conducted in sequence, enables better estimation of differences in fuel use.
The zonal controller reduced gas use by 8% in one house and 18% in the other compared to operation using the whole house heating schedule set on the programmer. Savings compared to having the system on permanently were 27% and 35% respectively. The savings were primarily because the zonal control enabled each room to be independently heated (or not) and heated to a different chosen temperature. The predictive capability reduced the incidence of rooms being cold when occupied, but the specific effect of this feature on energy use is not reported.
Beizaee et al. [68] and Beizaee [69] used a very similar experimental method to that of Rayment et al. (Section 6.1.2). They used adjacent, semi-detached, UK houses with synthetic occupancy that had been built in the 1930s and had not been refurbished or insulated. Each house did, though, have a Building Regulationscompliant 19 central heating system [47], with a gas combi-boiler and programmable room thermostat. In one house, six radiators had standard TRVs whilst in the other they were fitted with programmable thermostatic radiator valves (PTRVs) which could be independently programmed from a wirelessly-connected interface unit. In both homes, the heating system was scheduled to come on for two periods on weekdays (06:00-09:00 and 15:00-22:30) and for a single period on weekends (06:00-23:00). However, the PTRVs restricted the delivery of heat within these periods to the time when each room was 'occupied', plus 30 min to ensure warm up from the set-back temperature of 16°C. Typical room occupancy, as defined by the UK time use survey [70] was assumed. During the heating periods the living spaces were heated to 21°C and the bedrooms to 19°C. Empirical modelling enabled extrapolations of the results to whole years and to UK locations with different weather conditions.
Compared to the house with standard TRVs, the one with the PTRVs used 11.8% less gas over the 49-day wintertime experimental period, the boiler output reduced by 14.1% but the boiler efficiency dropped by 2.4% points due to the smaller and more intermittent heat load. Empirical modelling suggested similar percentage energy saving across geographical regions in the UK (11.8-12.5%). However, primarily because of differences in the cost of zonal control systems, but also because of differences in absolute energy demand between regions, the cost effectiveness was variable. At 2015 fuel prices, a luxury system (cost £1200) might yield a net saving of 3% on fuel bills over a 15 year period. A basic system (cost £120) may offer net savings of 11% over 15 years.
Beizaee [69] extended the work using the dynamic thermal model EnergyPlus, which had been calibrated against the experimental results. He examined how improvements to the energy efficiency of the house, through insulation and draught proofing, would affect energy saving. For the refurbishment measures studied, zonal control savings reduced by 0.2-2.2% points depending on location.
Marshall et al. [71] used computer modelling with the TRNSYS programme to predict the likely energy savings from zonal control, as well as other energy efficiency measures and heating system changes. They modelled a typical, poorly insulated, UK semi-detached house in London and a Building Regulations-compliant heating system and twoperiod heating. The modelled situation was therefore similar to the Beizaee et al. experimental conditions. In the simulations, the room temperatures and duration of heating was reduced to mimic the effect of a zonal control system, which constrained heating to each room's occupied period plus 30 min. The predicted winter season energy savings, for three different occupancy schedules, were between 10% and 15%, similar to the savings measured by Beizaee et al. There was no calibration of the model or comparison with actual monitored data and the interaction between heating systems components was not modelled in detail. Thus, the modelling could not detect real control effects, such as the decrease in the time-averaged boiler efficiency that may result installing zonal control; this was accounted for in the experiments of Beizaee et al. and the field trials of Scott et al.
Overall, the quality of the evidence for the energy saving from zonal control is considered as moderate ( Table 4). The three independent studies, each using a different assessment method, have all shown heating energy savings for UK homes. The measured or predicted savings varied from 10% to 18% compared to scheduled whole house heating via a programmer and thermostat or programmable thermostat. Modelling showed that savings of this magnitude could be obtained in UK locations with differing weather conditions. The quality of the evidence for cost effectiveness is considered to be low, because the figures are based on a single study, in one house, with one system and synthetic occupants. But, it is worth noting that a low cost zonal control system could be cost effective in intermittently occupied dwellings. A large-scale field study is underway to quantify the energy saving potential of zonal controls in diverse, occupied UK homes [72]. This will enable the effect of occupants' interactions with the controls, including overriding any initial settings, to be quantified as well as providing an insight into the reliability of the system.

Programmable thermostats
Urban and Gomez [18] reported a large-scale field trial to understand occupants' use of programmable thermostats [73] and the effect that use has on wintertime heating energy demand. Standard thermostats were replaced with programmable thermostat in 82 apartments in a single block in Revere, MA, USA, each of which had its own gas-fired heating system. The occupants, all of whom rented their apartment, had low income and paid their own energy bills. Two types of thermostat were installed, one considered to have much higher usability than the other. They were able to use their new programmable thermostats however they wanted but they were installed with defaults of 21°C (06:00-08:00 and 18:00-22:00) and 15.5°C at other times. The gas use and internal air temperatures were recorded over three months during the winter of 2011-12; 60 apartments provided adequate data. Overall, the occupants preferred warmer temperatures than expected [74] 20 and the heating energy demand varied by a factor of ten between apartments.
To try and disaggregate the effects of thermostat behaviour from other factors that could influence energy demand, EnergyPlus simulations were undertaken by feeding the model with the observed temperature histories and set-point schedules for each apartment. These simulations attributed a factor of three to differences in the way the programmable thermostat was set.
Urban and Gomez report that "less than five" of their 82 households successfully re-programmed their thermostat although many manually intervened to ensure permanent heating to a chosen set-point. The usability of the thermostats had no discernable impact on energy saving behaviour or the apartments' temperatures [73]. Assigning the apartments into four groups, based on the way the thermostat was used, identified that the 25% of households that used set-back schedules, with infrequent manual override, used, on average, 65% less energy for heating than the other household groups. Unfortunately, no pre-intervention data was reported and so there is no indication of whether the introduction of the programmable thermostats led to an overall reduction of the energy demand.
Suter & Shammin [75] reported on a small-scale field trial of 24, 100-year-old, gas-heated homes in NE Ohio, USA, that were let to college students. Each home accommodated three to five students who did not directly pay the energy bills. Three years of monthly gas consumption provided a baseline against which to compare gas consumption over the two years of the trial. Six homes provided a control group whilst groups of six homes each had a different intervention. In the first year, there was no significant difference in the average gas consumption of the six homes with programmable thermostats compared to the six control homes. However, homes that had roof insulation installed, or where students were offered financial incentives to reduce demand, produced significant reductions. In Year 2, three of the homes with programmable thermostats were also given financial stimuli and this produced the largest fall in gas use of any of the interventions and combinations thereof.
The usability of programmable thermostats has been evaluated, primarily in the USA, using both large-scale field trials and usability assessments. Meier et al. [76,77] explored the use that is made of the programmable functions of thermostats by collecting responses from 81 people in 57 US cities via an online survey. They found that nine out of ten respondents rarely if ever used the thermostat to programme a heating schedule. Meier et al. [76,77] also undertook a controlled usability assessment of five programmable thermostats using 29 of their survey respondents. The test revealed a range of problems, including extended times, or even failure, to complete tasks, confusion over labelling and difficulty with the physical design of the devices. Interestingly, they also proposed a new usability metric combining task completion time and success rate to effectively evaluate thermostat interfaces and to better distinguish one device from another.
Pritoni et al. [78] reported on a survey using a crowdsourcing tool that had 192 responses from people living in 38 different US states. There were inconsistencies in the data as a result of self-reporting but, 20    of the 42% that said they had a programmable thermostat, 40% said they did not use the programmable features, with over 30% disabling or overriding the programmable features. The user interfaces were often mentioned as being confusing. A survey of 7000 Dutch households by Guerra-Santin and Itard [79] focused on occupants' influence on heating energy demand by comparing programmable and manual thermostats. Interestingly, although the hours of heating with the two types of thermostat were not significantly different, households intervened more often to adjust the heating schedule when they had a manual thermostat. The authors suggest that smart thermostats that detect the presence of people, rather than pre-programmed thermostats, might be effective at saving energy (see below).
Horn et al. [80] tried to understand children's engagement with programmable thermostats and heating controls more generally. The study, which included 17 adults and 39 children from different types of US households, showed that children do not interact with thermostats because they are disinterested in them and because their guardians will dissuade them from touching, which might be considered a good thing! Should thermostats, as the authors suggest, be made easier for younger generations to use?
Controlled assessment of usability has been used to assess various heating controllers, some were prototypes designed as part of the study (e.g. Combe and Harrison [81]) and some compared existing controllers, primarily programmable thermostats (e.g. Combe et al. [82] and Peffer et al. [83] and Meier et al. [84]).
Peffer et al. [83] and Meier et al. [84] conducted controlled laboratory assessment of five types of programmable thermostats and assessed the results using their new usability metric (see above [76,77]). From their cohort of 31US participants aged between 18 and 65, the highest task success rates were found for thermostats that provided the clearest indication of the available actions, offered feedback once actions had been undertaken, and were operationally consistent. The tests also demonstrated the ability of their methodology to distinguish between the usability of different thermostat with different interfaces.
Combe et al. [82] undertook a controlled assessment involving 24 people in the UK exploring the usability of three digital programmable thermostats. A range of usability problems were identified, most notably that excessive cognitive demand was placed on users, with the difficulties exacerbated in older people such that none of them completed the task of programming for any of the thermostats. In a separate trial, Combe and Harrison [99] reported that 23 of their 31 UK participants (aged between 23 and 78 years) could not successfully set a prototype programmable thermostat. Of those that failed, 21 were over 60 years of age.
Overall, the two monitoring studies, [18] and [75], provided no evidence about the energy savings and cost effectiveness of programmable thermostats; they do suggest, though, that the way they are used can have a substantial impact on heating energy demand. This was born out by the findings of both large-scale field trials and controlled assessments. These produced high quality evidence ( Table 4) that people find it difficult to use the programmable functions of thermostats. This is especially so for older people. Consequently, households, and the individuals within them, may not use the full functionality of their heating controls, preferring instead to manually override any pre-programmed settings. Like the monitoring studies, the usability studies did not quantify the energy consequences of this lack of usability.

Smart thermostats
Smart thermostats try to automate the control of the whole heating systems in order to avoid heating at times when occupants are absent, overheating when occupants are asleep or under heating when occupants are present and active. These aims are not necessarily compatible with each other. There are two main criteria by which most studies gauge the success of the thermostat (or the model predictive controller (MPC) within it), energy saved and miss-times (i.e. the time for which the house is occupied but under-heated). The challenge is to identify the three states, active, asleep and absent, and predict when to turn on the heat prior to returning home or waking up. Typically, a learning algorithm is used to achieve this by taking signals from one or more occupant sensors, and sometimes also from other sources such as the global positioning system in mobile devices. The smart thermostat then sends signals to the boiler (or heat pump) either to turn the heating on and off, or to switch to a set back temperature. Alternative uses of learning algorithms are described by Lindelöf et al. [66] and Scott et al. [67]. The former describes the use of an MPC to improve the performance of weather temperature compensators (see 6.2.2) whilst the latter introduces an MPC within a zonal control system (see 6.3.2).
Kleiminger et al. [85] provide an excellent review of the smart thermostat field and note that, due to their novel nature, performance data for available smart thermostat products are sparse. Most research has focussed on MPCs and the learning algorithms therein, and Kleiminger et al. present a useful systematic review of these observing that,"notations and terminology are often inconsistent across different contributions, making it hard to compare existing approaches in a qualitative way"; this review also seeks to bring further clarity.
Three documents passed the acceptance criteria which were in fact concerned with the use of MPCs for electrical load shifting to avoid high, peak load, prices. All were studies in US homes with typical electrical heating ventilating and air-conditioning (HVAC) systems. Perez and Burger [86] studied heating demand shifting whilst Ivanov et al. [87] and Harding and Lamarche [88] focussed on summer cooling. These three documents are not considered further herein. Three other documents presented MPCs that sought to save energy by optimizing the set-point or schedule set on pre-existing, standard, on-off thermostats, timers or programmable thermostats. Iyengar et al. [89] presented an approach that used smart meter data to provide US households with improved schedules for their electrical air-based HVAC systems, whilst Gupta et al. [90] and Drgona et al. [91] offered early prototype propositions for MPCs that adjusted the set-point to maintain comfort with reduced switching and on time. Likewise, Rogers et al. [92] reported work to develop a prototype MPC using a building-like structure (a road truck body). Whilst the controller worked well in the truck, it was much less successful when exposed to the dynamics of a real house. The relevance for any quoted energy savings in these four documents to available systems is not clear and it is not possible to tell if they have potential in hydronic heating systems; these four documents are not considered further here.
In the UK, Boait and Rylatt [93] report a small-scale trial of a prototype, whole-house MPC, tested in one house. The controller tried to learn when people were in the house based on electricity use and hot water runoff. The controller used the occupancy status (active, asleep or absent) for the same day of the previous weeks (i.e. 7, 14, 21, etc. days before) as the basis for predicting the probable occupancy profile for the current day. The controller adjusted the whole-house set-point depending on the occupants' assumed status, with further adjustment when occupants were 'active' depending on external temperature. Comparing a 2-week period with the MPC with the previous two weeks under programmer and thermostat control, the heating energy saving was 14%. Around 9% of the saving was due to the combination of reduced heating time and the lower room set-point temperatures; improved boiler operation accounted for the other 5%.
Three documents reported the development of learning algorithms by US researchers. Lu et al. [94] used dynamic thermal modelling to evaluate the energy savings from a prototype MPC which used PIR occupancy sensors and contact sensors on doors to quickly estimate the probability of the occupants being active, asleep or absent. The MPC also used historical occupant arrival patterns to decide whether to preheat the home prior to reoccupation or to simply heat on arrival. It also allowed the set-point to drift well away from comfortable if it was confident the home was unoccupied, so-called deep set-back.
Occupancy data from eight homes (the type of occupants and the location of the homes is not stated), were fed into the learning algorithm, which produced the heating and cooling schedule. This was simulated in the EnergyPlus model to predict the energy demand for a winter heating and summer cooling using a heat pump and air-based system in Charlotte, Carolina, USA. The average of the heating and cooling energy was compared against a baseline algorithm derived from a survey of household heating patterns. The average energy saving was 28% but the savings varied from 38% for homes with regular occupancy patterns to 17% for homes occupied most of the day. The heating and cooling energy savings were not disaggregated. These results are unlikely to be applicable to temperate climates, where summer cooling is rare, 21 or to hydronic heating systems with boilers which are operated using a periodic heating schedule.
Ellis et al. [95] undertook a largely calculation-based 22 to study if, and how much, energy might be saved if the time that people left their house was predicted. Two approaches were trialled, one that switched the heating off when people left the house (Big Drop) and another that switched heating off just before they left (Early Off) -rather than later when it was programmed to go off. Using the actual gas consumption and the known occupant departure times for two homes in Cambridge, UK, and three in Seattle, WA, USA, algorithms were developed for the Big Drop and Early Off strategies. The calculated energy saving was compared with that saved by a 'perfect' controller (the Oracle), which switched the heating system on or off reliably at every arrival and departure. The Early Off control was calculated to save between 4% and 5% heating energy in the UK homes and 4-12% in the US homes. The Big Drop control, which could be implemented in a real controller, saved just 1% in the UK homes but 2-8% in the US homes. In one UK house, the Early Off algorithm turned the system off when the house was still occupied on 60% of occasions. The authors caution that the calculated savings may not be realised in practice.
Scott et al. [67] tested their PreHeat strategy (see 6.3.2) between January and April 2011, in three homes in Seattle, WA, USA. The homes had gas-fired, whole house, air-based heating systems, and were occupied by two adults who were researchers or their colleagues and a least one child. The adults were provided with RFID tags which signalled their presence to the MPC, which gradually learned when the home was, or was not, likely to be occupied. The heating system was controlled by the MPC and by the existing programmable thermostat on alternate days in trials lasting 58, 64 and 72 days. Compared to the schedule which the occupants had programmed into the thermostat, the MPC did not save any heating energy (savings −5, −1 and +2%) but the miss-time (when the house wasn't heated but was occupied) was significantly reduced (by 84-92%); this was despite frequent manual interventions by the occupants to restore comfort by overriding the schedule set on the thermostat. As is often the case with energy efficiency measures and heating controls, the benefits were, it seems, revealed as improved comfort rather than reduced energy demand.
Hong and Whitehouse [96] reported the development of a learning algorithm which, based on GPS data and historical location and arrival data, sought to determine when people will return home. They calculated the potential energy saving of the learning algorithm using a data set from another study [97] comprising four US citizens movements for periods of three to six months. The algorithm aimed to avoid heating during absence whilst avoiding miss-times. Less energy was saved if the HVAC system was switched on too early or too late. 23 Compared to the assumed heating schedule, the predictive algorithm resulted in between 8.3% and 27.9% less electricity for heating with approximately 15-60% lower miss-time. However, the way this was estimated is unclear and so, therefore, is the relevance to a real system, especially to hydronic heating systems in occupied homes.
Kleiminger et al. [85] conducted a thorough study which provides the most credible evidence of the energy saving potential of MPCs. They provided a review and model-based assessment of the energy saving potential of five pre-existing and documented learning algorithms. The authors used the simplified, dynamic thermal model 5R1C, which is described in ISO13790 [98], to conduct a parametric study to compare the different algorithms for numerous combinations of dwelling, occupancy schedule and weather conditions. There was however, no calibration of the model or validation of the predictions. The model predicts the hourly energy demands and internal temperatures in a single-zone for a single day in response to an imposed heating schedule. Their modelling presumed some form of presence detection and represented the operating characteristics of a heating system. They did not, therefore, model a specific, gas, electric, air or water-based system. 24 The predictions were made for a flat and a house each with high or low fabric heat loss and for four different Lausanne, Switzerland, weather days. Each of these 16 variants was modelled for 45 occupancy schedules, which covered at least 100 days. The occupancy schedules were for actual households and were derived from information captured through the Nokia Lausanne Data Collection Campaign [99]. The dwellings were occupied on average for between 10 and 24 h per day (mean 17.6 h). The annual heating energy savings for each dwelling/occupancy combination was estimated by summing the predicted savings in proportion to the occurrence of each of the weather days in a typical Lausanne year.
The energy demand and discomfort 25 for each schedule and dwelling/weather combination was predicted for each of seven different algorithms: a 'naive' controller; five different learning algorithms -presence probabilities (PP) and a simplified version thereof (PPS) [100], the PreHeat algorithm [67] and two heuristic prediction strategies based on Mean Arrival Time and Minimum Distance Mean Arrival Time which emulate Lu et al.'s [94] algorithm; a perfect predictive controller (Oracle); and a non-learning reactive 'algorithm' (REA) which just switches the heat on and off when people arrive and departure. Each algorithm strove to ensure that the indoor temperature was at the set-point (20°C) during occupancy, but allowing the temperature to fall when not.
The predicted energy saving was taken to be the difference between the energy used to heat for 24 h a day at 20°C and that predicted by the control algorithm. Savings ranged between 6% and 17% depending on the control algorithm, and for all algorithms, including REA, there was minimal loss of thermal comfort due to miss-times (under-heating during occupied times). Of the predictive algorithms, PP and PPS performed the best but only marginally so. The 25% of households with the lowest occupancy had a 4-5 times higher potential for energy savings than the quarter of homes with the highest occupancy. 26 The savings for the poorly insulated flat and house were almost double those of the well-insulated buildings. At lower ambient temperatures and under cloudy conditions less energy is saved and discomfort is increased due to under-heating.
Importantly, especially when considering typical UK home heating practices, all the predictive algorithms resulted in heating energy demand that was 2-4% greater than with the reactive algorithm (REA), which simply switches the heating on and off as people come and go.
Also, the REA strategy did not unduly diminish thermal comfort during occupancy. This result is important in the context of UK heating practices, where some households are rather diligent about switching off their heating when it is not needed, or even when it 'ought' to be on but habit, and concern for cost, means it is turned off. In any case, the most common UK practice is two-period on/off heating, rather than alwayson, which was the benchmark assumption of Kleiminger et al.
The modelling results of Kleiminger therefore concur with those found in the small-scale trial in Seattle by Scott et al. [67] namely that MPCs may not save energy in periodically heated homes and may, in fact, use up to 4% more energy than a manual programmer or programmable thermostat with fixed heating schedules. But, both studies show that under-heating during occupied times might be reduced. The studies also show that smart thermostats with MPCs may save energy in homes that are heated for prolonged periods, e.g. always on, as may be the case with heat-pump systems.
Only two studies relevant to the usability of smart thermostats were uncovered. The study by Rubens and Knowles [101] required 43 households to record the use of their heating system. This was followed up with interviews in participants' homes, and finally, a list of requirements for heating controls was prioritised in four workshops and three concepts for smart heating controls were evaluated. There was no attempt to quantify the effects of controls or estimate the consequences on energy use of poor usability and the participants were not statistically representative of any larger population group.
Dimitrokali et al. [102] conducted a large-scale UK field trial to gather evidence to aid the development of future technology. They explored the perceptions of behaviour change in 71 UK households following the installation of a new controller that could be controlled remotely via an app. An online questionnaire was delivered over a 6 month winter period with follow up interviews with 12 participants. Control using the app was preferred by almost 60% of the participants and, whilst 71% of participants thought that the controller had influenced the way they heated their home, no evidence of actual behaviour change was reported. The heating behaviours prior to the installation of the new control were not recorded and, the figures for the use of different control features, which were self-reported, 27 were uncorroborated. The authors recognised that the use of the app could not be linked to a change in energy demand and so cost effectiveness could not be assessed.
Overall, the availability of evidence about the energy savings potential of smart thermostats is thanks largely to the work of Kleiminger et al., supported by Scott et al. The quality of their evidence is graded as moderate ( Table 4). The other studies report small-scale trials using prototype MPCs and learning algorithms, often with serious methodological limitations. All report on prototype controllers rather than commercial products and the Kleiminger work suffers from the inherent limitations of modelling studies (Section 5.1, Table 5). The study of Dimitrokali et al. [102] provides very low quality evidence of the consequence of usability on energy use.

Methodology
This paper presents the first systematic international review of the evidence for the energy saving of heating controls, the influence on energy demand of controls' usability and their cost effectiveness. The review focusses on domestic hydronic, low-pressure hot water heating systems for temperate climates, with particular focus on gas-fired boiler systems.
The evidence review classified eleven heating control types into two broad groups: standard controls, which ensure the safe and efficient operation of heating systems, and advanced controls, intended either to improve the overall efficiency of the system, or to improve the control of the space temperatures. In addition, five components and features were documented that add smart functionality to controllers. A systematic key-word search of eight databases and search engines and organisations' repositories, uncovered over 2460 documents that were concerned with the eleven types of heating controls and their components. Screening criteria isolated just 122 documents that reported the energy saving, cost effectiveness or usability of the controls within the UK, temperate or Mediterranean-like climates. Evidence was provided from small-and large-scale field trials, full-scale experiments and computer modelling. Usability was also assessed by expert evaluation and controlled assessment.
Quality assurance scoring identified just 67 documents, mainly from the UK and the USA, that were synthesised by in-depth reading. A further 24 documents discussed usability in isolation of energy saving considerations. Only five studies combined energy savings evaluation with an estimate of cost effectiveness and just three integrated a study of usability.
The quality of the evidence about each control or component was classified as high, moderate, low or very low quality using the GRADE system. The strongest evidence emerged when a combination of approaches produced similar results either within a single study or through related studies by different researchers. Within the GRADE system, evidence classed as high quality is such that further research is unlikely to change estimates of the effects of a control. Whilst there was high quality evidence about the lack of usability of heating system controls, there was no high quality evidence about the energy saving and cost effectiveness.

Energy savings, cost effectiveness and usability
Energy use is heavily dependent on the energy efficiency of the dwelling, the climatic conditions and the characteristics of the heating system. The energy saved by heating controls depends on the system against which they are compared and, crucially, the way that the system is operated. Some controls may save energy when a system is always on but not when heating is periodic. Some will be effective when used by some groups of people, others not. Consequently, any quantification of energy savings requires both the social and technical contexts to be defined.
The interaction of people with their heating controls has a significant impact on energy use, and the energy saved if new controls are installed. The majority of usability studies focused on general heating controls and programmable thermostats and provided high quality evidence that heating controls are difficult to use, especially by older people. However, there are no studies that have quantified the consequential energy penalty of poor usability.
The cost effectiveness of controls strongly depends on the reduction in the use of fuel and its cost, the price of the control system and its installation, and the required payback time. With only five exceptions, none of the documents reported any attempt to quantify the cost effectiveness of the controls studied.
Standard controls provide the basic functionality needed to ensure systems are safe and maintainable and provide basic thermal comfort. There was no evidence for the energy saving potential of on/off switches, boiler thermostats or central timers. The quality of the evidence for room thermostats and thermostatic radiator valves was either low or very low.
Advanced controls provide additional functionality. Time proportional integral (TPI) controllers, weather compensation and load compensation seek to improve the efficiency of the heating system, other controllers, programmable thermostats, zonal control and smart thermostats can save energy by reducing the duration, level or spatial extent of heating. 27 E.g. the temperature increment that was used in boost mode, how often the app or the on-line interface was used, and how often the heating schedule was changed.

K.J. Lomas et al.
Renewable and Sustainable Energy Reviews 93 (2018)  There was no evidence about the energy savings produced by programmable thermostats, and load compensation, and very low quality evidence about weather compensation.
A single large-scale field study, on a range of periodically heated UK houses with condensing, modulating boilers produced moderate quality evidence that TPI controllers provide no improvement in the efficiency of a heating system compared to conventional on-off thermostatic controls. This was because, in the periodically heated homes, the boiler spent very little of the on-time operating close to the set-point.
There is moderate quality evidence that zonal control can save energy and be cost effective in homes where rooms are intermittently occupied. A small-scale UK field trial, a full-scale UK experiment, and computer modelling each conducted by different research teams, all showed savings in the region of 10-18% compared to whole-house scheduled periodic heating. There was low quality evidence that lowcost zonal control systems can be cost effective. The percentage energy saving is similar across different UK regions, although the absolute energy saving diminishes in warmer areas and in more energy efficient (better-insulated) homes.
There is moderate quality evidence that smart thermostats may not save energy compared to non-smart thermostats. A small-scale, short duration, field trial of a prototype controller in the USA, and computer modelling for five different learning algorithms using the same real occupancy schedules, indicated that the learning algorithms increased energy demand by 2-4% compared to simple on-off control by a programmer or in response to occupancy. They did, of course, save energy compared to a system that was always on. There is though, a need for evaluation through a large-scale field trial to fully account for the effect of occupant interaction and other socio-technical effects.

Observations on the evidence base
The pre-existing technical, behavioural and social contexts have a strong impact on whether new heating controls will save energy or not; this is true for all the controls studied, even those that do not demand occupant interaction. It is hard to save energy with new controls in modern building regulation-compliant homes that are well insulated with an efficient, well-controlled, heating system. Conversely, even simple new controls can save energy in older homes with inefficient and poorly controlled systems. It is much harder to save energy where there is a culture of switching on and off a heating system either wholly or in part to save energy or cut fuel bills, but much easier when the norm is to leave the system on permanently. Given these observations, it was disappointing that most of the documents reviewed did not provide an adequate description of the baseline condition prior to the installation of a particular control, or incorporate a control group, that didn't receive the intervention. The reported work could not therefore provide high quality evidence of energy savings, or define the socio-technical settings most likely to deliver any savings.
A number of studies reported changes of controls as part of a package of energy efficiency measures. In such studies, it is impossible to separate out the effect of the controls, which may well yield savings that are an order of magnitude less than the energy efficiency measures applied to the building fabric.
All the field studies reported have only been conducted for short periods of time, so it is not possible to understand the long-term effects of the interventions. Such effects could include long-term drifts in energy demand, for example as people lose interest in the controls they had been provided, additional wear and tear on boilers, for example by more frequent switching, failure of the controllers, lack of reliability and increased risk of system malfunctions. Long duration field trials would also help reveal the unintended consequences of making interventions in complex socio-technical systems.
None of the studies considered, with a sufficient sample sizes, the cost and disruption of installing new controls and there was limited information about the influence that the installers of controls and heating systems can have on their effectiveness of controls. These factors could strongly influence the propensity to take up new controls, the initial set-up of the controller, and occupants' understanding of how to use them; and thus the energy savings.
Full-scale field trials, perhaps in association with modelling, can produce compelling, high quality evidence about the real-world effects of controls accounting for the full socio-technical complexity and the potential for unintended outcomes. Unfortunately, the systematic review did not uncover results from any large-scale field trials for any type of heating control that was sustained over a sufficiently long time period. There is therefore a clear need for large-scale, multi-disciplinary heating controls study to provide a robust assessment of energy savings, the provision of thermal comfort and usability.

Table A1
Focus and assessment methods reported in the documents that passed the screening and quality assessment criteria. •■ Document contains factual evidence about energy saving, cost effectiveness or usability. ○□Document does not contain factual evidence about energy saving, cost effectiveness or usability.