Non-domestic building stock energy and carbon modelling for policy advice – a user requirements survey

In recent years, many building stock models have been developed to advise and guide policymakers. In most models, however, user needs were not formally established. Therefore, the aim of this paper is to formally undertake a user needs‘ assessment for building stock energy and carbon models. To achieve this aim, a bilingual exploratory online user requirements’ survey was developed. The survey was designed to gather information in the following areas: the general interests of the potential model users; their experiences with related studies and models; the desired properties of non-domestic building stock energy models; and any technical limitations, such as computational resources. A total of 19 responses were obtained. Users favour tools which are: usable, transparent; flexible; compatible with other tools; and provide clear, understandable results.


Introduction
In recent years, many building stock models have been developed to advise and guide policymakers on the pathway to climate neutrality. However, in most models, user needs were not formally established, but have been assumed by the model developers [1,2,3]. Nonetheless, some information on user needs and requirements for building stock models for policymaking are available in the research literature. For example, Piette et al. [4], surveyed several city governments in the U.S. on the use of different models and tools. They found a preference for easily implementable, transparent, low cost, open-source software which considered uncertainty; these attributes were found to be unavailable in existing tools. They state that the most useful tool capabilities to end users included an ability to estimate building stock energy savings resulting from policies and new technologies, as well as the identification of the most priceefficient energy reduction options. Another study [5], focusing on the success factors of models for advising policymakers, highlights the attributes that influence the deployment and use of evidence-based policymaking models in general. Using a grounded theory exploratory research approach, eight models were assessed by interviewing both the model developers and users. While the study focuses does not explicitly focus building stock models, it does offer valuable insights on criteria for model acceptance. Eight criteria marked with * were identified to be influenced by the model developer. However, for two of these (Reputation and Consistency), the model developer generally has some, but limited influence; user knowledge and alternative model quality influences the perception of consistency, while user reputation depends somewhat on the actions of others [5].
Four of the criteria (Quality, Tractability, Efficiency and Flexibility) listed were identified 'based on the perception of the intended user' [5], which highlights the importance of considering the user's views.

Aim and Methodology
The overall aim of this paper is to address the gap in the available literature addressing nondomestic building stock model user needs. More specifically, the aim is to formally undertake a user needs' assessment for building stock energy and carbon models.

General approach
To achieve this aim, an online user requirements' survey was developed under the umbrella of the ENOB:dataNWG -non domestic building stock research data base project 1 . This project involves the development of the first research database of Germany's non-domestic building stock, leading to the development of a non-domestic building stock model for exploring the effects of low-carbon interventions and policies. There is no established market for building stock modelling software applications. For this reason, the nature and extent of potential users was difficult to establish and so a statistically representative survey could not be conducted. Instead, an exploratory research approach [6] was chosen to identify a hypothetical user's interests, preferences and problems with regard to non-domestic building stock models by surveying and understanding the experiences of a sample of potential model users. Such an approach is typically taken in cases where there is "little or no scientific knowledge about the group, process, activity, or situation" under examination but nevertheless there are reasons "to believe it contains elements worth discovering" [6]. In total 60 potential users were contacted by email and asked to answer the survey comprising 21 questions. All survey participants were in some way known to the project team of ENOB:dataNWG, either through former projects or by targeting specific contacts based on their occupations. The survey was bilingual (German and English) due to its international nature.

Survey
The survey was designed to gather information in the following areas: the general interests of the potential model users; their experiences with related studies and models, particularly regarding non-domestic building stocks; the desired properties of non-domestic building stock energy models; and any technical limitations, such as computational resources or software platform compatibility. Scope, content and comprehensibility of the questionnaire was reviewed by three experts in related fields of research.
Due to its size, the survey is not further repeated in this paper, but can be accessed online via Google Forms 2 . The presented findings in the 'Results' section refer back to the survey questions via the question reference numbers in brackets. The survey is split in three parts that are subdivided into seven survey-sections. The sections where chosen primarily for organisational purposes such as guiding the 'jump' instructions, allowing respondents to skip irrelevant questions. The first part, including the first, second and third survey-sections, addressed the general interests of respondents and their experiences with relevant studies and models. The second part of the survey included the fourth and fifth survey sections, and focused on the desired properties of non-domestic building stock energy models, while the third part (sixth and seventh survey-section) covers the technical implementation of the model. The following section summarises the results of the survey.

User groups
A total of 19 responses were obtained, giving a response rate of 32%. Based on these responses 11 user groups were identified from question 1 (Q1). These were further clustered, based on most likely use of the model and similarities between the respondents' economic activities, into four primary user groups: 'Research'; 'Consultancy, Planning, Certification'; 'Government Department / Ministry / Advisory Body, Politics, Civil Service'; and 'Associations, Church, Industry'.
Members of the primary user group 'Research' are involved in the generation and publication of knowledge, while those in 'Consultancies, Planning, Certification' tend to be professionals providing consulting and compliance services. Three quarters of this group belonged to consultancies and planning. It should be noted that this group has influence on the policy development process, as it has been observed that they regularly provide advisory services to the group of 'Government Department / Ministry / Advisory Body, Politics, Civil Service'. The 'Government Department / Ministry / Advisory Body, Politics, Civil Service group members were all in the public sector involved in implementing policies and advising policy makers. The fourth primary user group includes all other occupation; this group's members share interests in the stock asset ownership (church, industry and associations) and in the supply of energy, materials and services to building stocks (industry).
Because respondents could choose more than one economic activity, a small number could be allocated to multiple end-user groups. Where this occurred, they were allocated to the group containing the majority of their stated economic activities.

General user interest and experience
Almost 80% of the sample had experience in the use of modelling software simulation tools for non-domestic building stocks or were using the results of such tools (Q5). This shows that the survey-participants-targeting correctly identified potential users.
Almost all respondents (n=18) believed it important to have such tools for the analysis of the non-domestic building stock (Q8). Almost all users (n=17) across all user groups wanted clear information on the non-domestic building stock's structural composition and physical attributes (Q2). Furthermore, respondents would like to us the model to understand the current dynamic development of the stock (n=14) and its possible future development under different scenarios (n=9). The most important model output requirements were stock energy use (n=10) and greenhouse gas emissions (n=5), followed by economic analysis (n=2) (Q3). The reason for the strong preference for energy analysis is not entirely clear. It might reflect a historic regulatory focus on energy rather than CO 2 . The answers indicate that costs are not the first priority for most users.
Regarding the analysis time period (Q4), a majority were interested in modelling the current performance of the stock (n=14), while the ability to simulate possible future characteristics was ranked as secondary (n=4). This correlates with the responses to Q2 which indicated a main interest in the performance of the current building stock.
To provide more insight into the probable use of a new non-domestic building stock model, participants were asked which questions they would use it to answer (Q11). Responses indicate that most users would like to quantify building stock operational energy demand (n=9), embodied energy (n=4), GHG-emission (n=5) and costs (n=5). These outputs would be used by the respondents to quantify 'saving potentials' (n=6) and impacts of 'refurbishments' (n=6) on the NDBS. Two respondents wanted to explore how 'climate protection targets' could be achieved and how a 'refurbishment master plan' would perform. Some participants indicated how they would go about this: six proposed using life cycle assessment and three suggested conducting scenario analysis.

Desired model properties
The respondents identified several helpful model requirements (Q9): usability (n=6); flexibility(n=5); a high-quality user interface (especially for easy visualisation of results) (n=5); compatibility (technical) with other software-programs (n=4); and transparency (n=4). The attributes of flexibility and usability (user to software compatibility) where also identified to be criteria for model acceptance for policy making by [5]. Beside these helpful model requirements, users were asked to identify important shortcomings of existing models and studies (Q10). While the responses to this question were varied, two shortcomings were notable because they were reported by members of different user groups; these were the 'high complexity of models' and 'poor data availability'. Proposed solutions to these shortcomings (Q10), included to 'simplify' the models and include 'uncertainty quantification'. The first solution correlates clearly with the high model complexity shortcoming. The lack of good quality data can only be solved by providing better data (the aim of the ENOB:dataNWG project). However, the proposal to quantify model output uncertainties would help to communicate the limitations of poor data, thus mitigating the impact of poor data. In summary, no major common shortcoming was identified. However, the users indicated a preference for simple and understandable model methodologies rather than complex ones (Q9).
The need for a model which considers the whole life-cycle is supported by all respondents (Q13) and the majority (n=13) rate the life-cycle phases A1-C4 according to the EN 15978:2011 as most relevant (Q14). The relevant life-cycle indicators identified were GHG emission (n=16), energy (renewable (n=14), non-renewable (n=10) and primary (n=9)) and cost (n=13), while the material resources (n=6) were secondary (Q15). Regarding the most useful reference value for normalisation (Q16), most users identified building gross floor area (n=5), lettable area (n=5) and conditioned (heated or cooled) area (n=8) of a building. The answers relating to life-cycle indicators seems to contradict those for Q3 on preferred model outputs. The latter did not appear to prioritise cost, whereas the former (Q15) gives it almost equal weighting with emissions and energy. The reason may lie in the way the different answers to both questions were structured. Q3 (main model outputs) allowed only one answer per person, while Q15 allowed multiple answers. Therefore, while cost is of secondary importance to energy and GHG emissions, it may be a close second. Beside the consideration of the whole life-cycle, the majority (n=16) of respondents also recognised the need for uncertainty quantification of model results (Q12).

Technical model framework
Regarding the technical model requirements, the maximum acceptable model computational time (Q18) was identified as less than one hour, independently of the stock's population (n=10). Unsurprisingly, the preferred software solution (Q19) was one based on familiar software, with a slight tendency toward standalone (n=6) rather than browser-based platforms (n=4). This preference for familiar software was also identified by [5].

Requirements Elicitation
The findings of this exploratory survey are summarised below as a general requirements elicitation list which can be used in the development of holistic energy and carbon non-domestic building stock models suitable for informing policymaking.
• Model outputs of energy, GHG-emission and cost (Q3 and Q15); • Possibility to simulate the current and future building stock (Q4); • Usability, flexibility, high-quality user interface (especially for easy visualisation of results), compatibility with other software programs and transparency (Q9); • Appropriate, clear methodology that considers the whole system and can forecast the future development of the non-domestic building stock (Q9); • Simple, understandable model methodology (Q10); • Uncertainty quantification (Q10 and Q12); • Life-cycle perspective (Q13); • Consideration of the whole life-cycle (A1 -C4) (Q14); • Provide outputs normalised by building area (Q16); • The model's computational time has to be below 1 hour (Q18); • The model should be based software infrastructure that allows easy access to inputs and outputs also to non-expert users -that are non-programmers-and at the same time allow expert users with programming experiences to further develop the model to their needs (Q19).

Discussion and Conclusion
This paper undertakes a formal user needs' assessment for building stock energy and carbon models using an exploratory research approach. The survey analysis of a sample of potential users provides an understanding of their requirements, thus allowing the design of modelling tools which help meet their expectations. Four primary user groups were identified: 'Research'; 'Consultancy, Planning, Certification'; 'Government Department / Ministry / Advisory Body, Politics, Civil Service'; 'Associations, Church, Industry'. Almost 80% of respondents had experience in the use of modelling software simulation tools for non-domestic building stocks, or were using the results of such tools, thus indicating that a relevant sample of potential end-users was obtained. Almost all respondents believed it important to have such simulation-tools for the analysis of the non-domestic building stock.
In summary, the results indicate that users favour tools which are: familiar and intuitive to use (are usable); are well documented (usable and transparent); offer several features (flexible); and are open source (flexible and transparent); are interoperable with other tools (compatible); and provide clear, understandable results. The model should provide representative results and be based on a clear methodology that considers the whole system and can forecast its future development.
The preference for the implementation of the resulting computer application was for one which allows easy access to inputs and outputs, and can be used by expert and non-experts (i.e. non-programmers) alike.