Thermal simulation software outputs: a framework to produce meaningful information for design decision-making

This paper describes a process used to develop and test a framework to produce thermal simulation post-processed information meaningful to building design decision-making. The framework adopts a user-centred approach in which the building designer is considered the ultimate simulation tool user either directly or indirectly when supported by consultants. The framework supports the building designer's ‘modus operandi’ and is developed through a set of interdisciplinary research methods. Participatory Action Research, Thematic Analysis and Grounded Theory are used, together with principles from Information Visualization, dynamic thermal modelling and Building Design, following a design approach to problem-solving taken from the discipline of Interaction Design. The various elements of the framework and their connections are derived from analysis of sequences of design actions made by novice designers undertaking complex design activities. Tests of the framework are undertaken through an online questionnaire and five semi-structured interviews with UK architectural design practices.


Introduction
The aim of this paper is to describe how and why a new approach has been taken towards developing thermal simulation outputs that are intended to be of more use in design decision-making than are those currently offered. Taking this approach has resulted in the construction of a framework within which thermal simulation post-processed information useful to building designers can be identified, developed and ultimately be made available. Building designers typically do not make use of thermal simulation, despite the many efforts of the building performance simulation (BPS) community to integrate simulation into design processes. A user-centred approach has been taken to ensure that simulation outputs are designed specifically for the needs of the building designer. Following the practice of Interaction Design, simulation outputs are seen as a 'product' which must by definition be useful to this specific 'user'.
It is questionable, and outside the scope of this study, to discuss which type of users simulation tools are aimed at when developed. Some tools seem to be much more oriented towards use by computer software engineers rather than by heating, ventilation and air conditioning (HVAC) engineers and building physicists. However, simulation tools do tend to be aligned with the paradigms of knowledge and praxis of building physicists and HVAC engineers (Bleil de Souza * Corresponding author. Email: bleildesouzac@cardiff.ac.uk 2012), straightforwardly outputting information for these professionals to make sense of and act upon.
The same cannot be said in relation to simulation tools and building designers. Pushed towards using these tools, either directly or indirectly, by increasing demands of legislation and more informed clients, these professionals are forced to interpret performance results in disconnection with the building elements that are causing them. Simulation 'outputs rarely provide direct useful information about how and where to act in the building itself so that envelope and spatial arrangements can be manipulated to improve thermal behaviour' . This problem might be mitigated in collaborative environments as designers indirectly interact with the tools via consultants. However, problems still arise in relation to how the work of consultants get integrated into the design process as systematic and scientifically based approach provided by these consultants can easily clash with the constructivist and experimental ways of working of building designers (Bleil de Souza 2012). The problem of integrating simulation tools to the design process gets even worse if designers cannot afford the help of experts because they are forced to use simulation software unfit for purpose, developed in disconnection with their 'modus operandi'.
Expanding the work of Bleil de , this paper describes in more detail the process used to develop and test a framework to produce thermal simulation post-processed information meaningful to building design decision-making. It starts by providing a critical review of the literature on frameworks for integration of simulation tools throughout the building design process, highlighting assumptions that these studies make about how designers think and work. This is followed by an overview of the 'modus operandi' of the building designers extensively discussed in the literature from Building Design Research, from which key points of the design process are highlighted providing a different basis for production of meaningful simulation outputs for design decision-making as proposed in this paper. An Interaction Design approach towards developing and testing the framework is proposed. In this approach, Participatory Action Research is initially used for data collection; data analysis is undertaken through Thematic Analysis and Grounded Theory supported by principles from the fields of Information Visualisation, dynamic thermal modelling and Building Design and the testing of the proposal is undertaken through semi-structured interviews and an on-line questionnaire.

Frameworks and initiatives from software developers
There are not many studies which specifically discuss and propose frameworks to produce meaningful simulation outputs for design decision-making. Comprehensive initiatives which explore the problem at a more conceptual level can be found in the COMBINE projects (Clarke and MacRandal 1993;Augenbroe 1994;Clarke et al. 1995); design analysis integration (DAI) initiatives (Augenbroe 2002;de Wilde, Augenbroe, and van der Voorden 2002;Augenbroe et al. 2003Augenbroe et al. , 2004van der Voorden 2003, 2004), in the SERI (1985) guide and in the work of Mahdavi (Mahdavi and Suter 1998;Mahdavi 1999;Mahdavi 2004;Mahdavi, Bachinger, and Suter 2005 and especially in the work of Mahdavi and Gurtekin 2001). Franconi (2011) explores different simulation tool uses, highlighting that mismatches between customers' expectations and simulation outputs occur because different users have different modelling objectives.
The COMBINE project explored operational frameworks focusing on managing transactions between users and tools through the creation of a 'discussion room': a communication centre for all collaborators involved in a project (Figure 1(a)). The framework is described in terms of object-oriented and process models. To fit these models, the design process is broken into fragments of work-steps and event-handlings to start and stop the use of simulation tools. A particular feature of this project is the automatic identification of performance patterns (such as overheating, for example) displayed to the user in relation to where they happen in the building. Indications of performance causes seem to be provided through identification of main building and usage-related variables contributing to performance (e.g. internal gains identified as main causes of overheating).
The DAI initiative also explored an operational framework for collaboration adopting a process centric approach in which a workbench (Figure 1(b)) organizes moves from design information to simulation tool through two intermediate layers involving expert advice. 'Those intermediate layers provide context to a specific interaction by capturing the relevant process step (the associated task) and model aspects' (Augenbroe et al. 2003). In this initiative, design teams generate a series of analysis requests to consultants who set up analysis scenarios and use expert knowledge to retrieve the appropriate analysis functions to provide performance feedback. This type of interaction implies a restructuring in the building design process which according to researchers should follow systems engineering design methods: definition of an option space, identification of functions and selection criteria, specification of performance indicators, performance prediction of all options and assessment of these options to make decisions van der Voorden 2003, 2004). An interesting aspect of this initiative is the idea of using analysis functions as independent entities in the framework. This, in theory, Figure 1. (a) The 'discussion room' proposed in the COMBINE project (Clarke et al. 1995). (b) The four-layered DAI workbench (Augenbroe et al. 2003). enables flexibility in establishing cause-effect relationships and exploring different design alternatives.
Many of the features developed in these frameworks can be seen in current simulation tools expressed as workflow guides or data manager interfaces to modelling (ESRU, ESP-r 2013), Open Studio (NREL 2013), Design Builder (Tyndale 2013) to cite a few). In all cases, interesting concepts are present but these proposals still fail to comprehensively address building designers' needs. They do not have design aims at their centre and imply heavy structuring and interferences in the design process. When not fostering the adoption of engineering design methods, these approaches explicitly say design should literally follow prescribed design stages from accredited bodies such as the Royal Institute of British Architects, American Institute of Architects, etc. (Clarke et al. 1995;Hand 1998;Morbitzer 2003;Hobbs et al. 2004 to cite a few), which are mainly for management purposes and/or to set up stages for deliverables to clients (see examples from design literature such as Rowe 1987;Schon 1988Schon , 1991Lawson 1997 to cite a few).
Performance results are seen as ultimate aims rather than identification of the causes behind them. Designers are assumed to do mainly building performance query, i.e. to go 'from design to performance' (Mahdavi 2004) for which examples are commonly found in the building simulation literature. Performance queries are frequently found in output interface systems mainly with a focus on: Meeting targets from building regulations or performance standards (Prazeres and Clarke 2005;BRE 2013); Exploring design changes using Multi-Criteria Evaluation strategies (Papamichael 1999a(Papamichael , 1999b(Papamichael , 1999cPapamichael et al. 1999;Soebarto and Williamson 2001;Prazeres and Clarke 2005); Querying databases with several design alternatives (Mahdavi, Bachinger, and Suter 2005;Stravoravdis and Marsh 2005;Dondeti and Reinhart 2011); Simplified tools or interfaces to enable designers to freely explore design alternatives (Marsh 1996a(Marsh , 1996bGratia and De Herde 2002a, 2002b, 2003. They imply designers always have to come up with a design proposal first to then check how this proposal performs. However, designers are frequently after design advice, i.e. they want to go 'from performance to design' (Mahdavi 2004) to be able to effectively act upon design parameters that significantly contribute to performance results.
On one hand, a lack of understanding in design aims and in the way designers work impinge a huge burden on consultants, who often have no means to properly interpret design queries and end up providing information about what they think designers want. On the other hand, this lack of understanding results in software developers providing solutions to what they think are building designers' needs. In both cases, information is not conveyed in the best way it could be.
More explicit user-centred frameworks can be seen in the pioneering guide from SERI (1985) and in the work of Mahdavi and partners. In the Solar Energy Research Institute framework, performance queries are complemented by simple design advice derived from 10 elimination parametric tests, each of them reporting the impact of one standard pre-set variable on heating, cooling and lighting energy demands. Results from these tests are supposed to be comprehensive enough to characterize the causes of building performance, informing designers where they can potentially act to improve building behaviour. Guidelines are then structured around non-incremental design changes 1 and incremental design changes 2 , a way of approaching different types of design actions in the different contexts of the building design process. Elimination parametric studies suggest parametric investigations in building design variables can be automatically structured and executed independently from user preferences. They are in fact 'simplified' sensitivity tests which do not require any extra user information input.
A framework which examines sensitivity tests specially tailored for building designers can be seen in the work of Mahdavi and partners. One example is the workflow of GSN, a system developed to: generate, simulate and navigate through a solution space of a combination of user defined and automatically generated design alternatives. Outputs are a series of structured 3D graphs which display how different combinations of parameters affect performance results. Although being a starting point in responding to user requests via sensitivity tests, this attempt is restricted to a small number of design parameters to reduce simulation time and facilitate integrated information display.
The idea of 'going from performance to design' implies strong connections between simulation outputs, analysis processes and design inputs. In collaborative environments, these connections tend to be interrupted as they are the locus for expert knowledge and/or the points to link designers with consultants. This heavily affects the fluidity of the design process in which problem and solution co-evolve depending on the interpretation of the designer.
Aware of the problems involved in interrupting the fluidity of the design process and on the needs for simulation design advice, many researchers from the simulation community started developing output interfaces to facilitate the use of parametric studies 3 and optimization processes 4 in building design. These are empirical propositions which focus on making these two different types of analysis processes operational and accessible to building designers via a mixture of GUI/simplified interfaces with automatic postprocessing features of performance pattern recognition. The key aspect of them is the acknowledgement that designers want to know where to act to improve building behaviour, something that can only be provided if simulation outputs are connected with analysis processes, preferably without interruption. However, these proposals are far from comprehensive in terms of addressing the constructivist way of thinking of building designers and still assume a structured and procedural approach to building design problem-solving 5 .

The 'modus operandi' of the building designer
Building designers approach 'a practice problem as a unique case' (Schon 1991) attending to the peculiarities of the situation at hand without having any cues about standard solutions. Particular features of a problematic situation are discovered and an intervention is designed from this gradual discovery (Schon 1991). The problem is not given, 'the situation is complex and uncertain' (Schon 1991), the brief may be highly specified but the 'design situation is always partly indeterminate' (Schon 1988). 'There is a problem in finding the problem' (Schon 1991).
'In order to formulate a design problem to be solved, the designer must frame a problematic design situation: set its boundaries, select particular things and relations for attention and impose on the situation a coherence that guides subsequent moves' (Schon 1988). 'To frame a problem you have to begin with a "what if" situation to be evaluated' (Schon 1991). However, in this 'what if' situation, the practitioner needs to set a problem (s)he can solve, frame the problem for which (s) he feels (s)he can find a solution. This frame imposed on the situation lends the practitioner to a method of enquiry in which (s)he has confidence as 'when trying to solve the problem (s)he has set, the practitioner seeks both to understand the situation and to change it' (Schon 1991). The 'situation is understood through the attempts to change it and changed through the attempts to be understood' (Schon 1991) and problem framing triggers a process of reflection in action.
'Reflection in action necessarily involves experiment' (Schon 1991). However, local and global experiments that happen in practice are not controlled, i.e. they do not allow phenomena to be isolated and variables to be separated. In practice, several kinds of experiments, exploratory experiments, move-testing experiments as well as hypothesistesting experiments are all mixed together. Exploratory experiments are those in which action is undertaken only to see what follows, without accompanying predictions or expectations. In exploratory experiments designers discover new things in the situation 'back-talk'. Move-testing experiments are used to affirm or negate moves depending on the type of changes they produce. Moves that get intended consequences are affirmed, whereas moves that do not get intended consequences are negated. At the same time, the practitioner appreciates the value of the situation, judging if (s)he likes what (s)he gets from the action undertaken in terms of local and global consequences. Hypothesis-testing experiments are used to discriminate among competing alternatives. The best alternative is defined based on confirmations of the consequences of a given hypothesis together with predictions derived from alternative hypothesis that conflicted with observations. In hypothesis-testing experiments designers are constantly reframing the problem through a new hypothesis to be tested.
Experiments in practice have a very specific aim: The 'practitioner has an interest in transforming the situation from what it is to something (s)he likes better' (Schon 1991). That means the practitioner needs to solve the problem at hand and at the same time (s)he has to like what (s)he can make out of what (s)he gets. Moves are evaluated in terms of how desirable their consequences are in relation to intentions, how desirable the moves are in terms of their conformity to or violation of implications set up by earlier moves and how desirable the moves are in terms the designer's appreciations of the new problem or potentials they have created.
In any case, practitioners 'seek to exert influence in such a way as to confirm not refute their hypothesis' (Schon 1991). They try to make the situation conform to their hypothesis but they remain open to the possibility that it will not. The 'practitioner shapes the situation in conversation with it, so that his own models and appreciations are also shaped by the situation' (Schon 1991). The practitioner tends to be always inside the situation (s)he seeks to understand and '(s)he understands the situation by trying to change it and considers the resulting changes … the essence of its success' (Schon 1991).
The practitioner has to learn by reflection on the situation's resistance if his/her hypothesis and framing are inadequate and in what way. Whether (s)he ought to reflect in action and how (s)he ought to experiment will depend on the changes produced by his earlier moves. (Schon 1991) In general, the criteria of fit are set in a way that 'slightly' is enough. The process is stopped when changes in the whole are satisfactory or when new features which give the situation new meanings and affect the nature of questions to be explored are discovered. Overall hypothesis are only worth being tested if they can be immediately translated into design.

Adopting a user-centred approach
Understanding that the 'modus operandi' of the building designer has no structure other than a sequence of moves that go from conception to refinement in which problem-setting and problem-solving co-evolve continuously starting with problem framing and finishing with a coherent idea for the materialization of the artefact through a series of complex experiments, provides a different basis to produce meaningful simulation outputs for design decision-making: (i) Meaningful simulation outputs for design decisionmaking need to be part of a designer's conversation with the materials of the situation, i.e. they need to be embedded within a sequence of moves directed by reflection in action. (ii) This means simulation tools need to be 'in tune' with the complex types of experiments designers undertake while designing, providing answers to the different 'what if' situations designers generate throughout the design process.
(iii) Research needs to unfold what type of simulation output information can be useful in these experiments and how it should be represented to be effectively used. (iv) Proposals should guarantee freedom to designers in stopping the process whenever they judge changes in the whole are satisfactory and should also guarantee flexibility to explore new questions discovered from new meanings unfolded from sequences of design experiments.
Achieving these four aforementioned points requires invention and multidisciplinary explorations of research methods and principles, including the participation of building designers on the research.
As designer's conversations with the materials of the situation are idiosyncratic to the problem being solved as well as to the person solving it, records of them, especially those involving the use of building thermal physics in design decision-making, provide valuable material to work upon. Even if a sample is limited in terms of the type of problem being solved, records provide insights into how building thermal physics information is in tune with the complex experiments designers undertake. Specific types of building thermal physics information and the way they can be effectively represented to inform design decisions in each record can be mapped, categorised and organised into themes and subthemes. A framework can emerge from several iterations with data collection, categorisation and identifications of themes and subthemes. Such types of data collection and data analysis, respectively named Participatory Action Research, Thematic Analysis and Grounded Theory, are commonly used in Social Sciences and Interaction Design research, but uncommonly reported in the field of BPS. The authors believe once combined with principles from Information Visualization, dynamic thermal modelling and Building Design, these research methods could provide a different basis to develop a framework to produce thermal simulation post-processed information meaningful to design decision-making.
This paper explores the development of a framework using the aforementioned methods and principles. The elements of the framework are tested with building designers, consultants and researchers through an on-line questionnaire. The framework itself is tested in five semi-structured interviews with building designers in practice in the UK. Results from the questionnaire and interviews are not supposed to be statistically significant 6 but to provide insights into and criticism on the effectiveness in adopting a usercentred approach and exploring the use of interdisciplinary research methods and principles.

Methodology
The constructivist nature of the design process and the particularities involved in dealing with hypotheses based on the uniqueness of every different situation called for a Participatory Action Research approach (Whyte 1990;Kindon, Pain, and Kesby 2007) to this problem: Let building designers propose what types of building thermal physics information can be useful in the complex types of experiments they undertake while designing. Also, let building designers propose how this information should be represented for effective use in supporting design decision-making.
A data sample of 140 design journals in which building designers narrate all steps used to solve a design problem which included thermal comfort, energy efficiency and the testing of passive design strategies was used to provide insights into meaningful thermal simulation post-processed information for design decision-making (Appendix 1). The participatory phase was followed by a Thematic Analysis on the data sample 'to extract information and assist in the development of a theory about how designers make sense of building thermal physics information throughout their design process' (Bleil de Souza and Tucker 2013). Thematic Analysis is perhaps the most commonly employed qualitative research method from the Social Sciences, consisting basically on identifying and recording patterns or themes, recurrent motifs in the data set, to describe a phenomenon (Boyatzis 1998;Bryman 2008;Guest, MacQueen, and Namey 2012). Thematic Analysis was initially used to characterise design aims, working processes and ways of interpreting and manipulating information meaningful to decision-making.
Complemented with principles from Information Visualization and dynamic thermal modelling, the data set was revisited and the Thematic Analysis was expanded and refined several times through use of Grounded Theory, a Social Science research method commonly used to develop theoretical frameworks. Grounded Theory is a research method which aims to formulate theory from the data, 'systematically gathered and analysed through the research process' (Strauss and Corbin 1998 in Bryman 2008), contrarily to classic research methods in which data analysis is used to confirm a hypothesis. 'Grounded theory is not a theory -it is an approach to the generation of theory out of data' (Bryman 2008). Themes are used in Grounded Theory 'to identify concepts, categories and properties to develop hypothesis from which "a theoretical framework that explains some relevant social … or other phenomenon" (Strauss and Corbin 1998 in Bryman 2008) can emerge' .
Under the umbrella of Grounded Theory, Information Visualization principles 7 were used to organize and explore interactions with the data as well as to extract, catalogue and propose relevant types of displays to designers. Information Visualization is a field of study concerned with the visual representation, presentation and interaction with data in order to aid human cognition (Spence 2007). According to Information Visualization principles, users should be allowed explorative analysis of the data as well as confirmative analysis on hypothesis on the data (Mazza 2009). They should also be provided with a structured way to interact with the data. The phenomena generally needs to be described initially at an overview level, users should be able to zoom into a specific area of interest, retrieve detailed information on demand, retrace previous steps, compare and relate information (Schneiderman 1996).
Dynamic thermal modelling principles were used to organize and explore analysis processes aimed at providing design advice as well as to extract and propose relevant metrics for design decision-making. The work produced by designers was mainly based on simplified methods meaning design advice possibilities were limited by not taking into account the dynamic, systemic nonlinear and stochastic nature of building thermal physics phenomena.
From this combination of research methods and theories, a framework to produce meaningful simulation outputs to design decision-making emerged. The elements of this framework were queried in an online questionnaire with regards to their usefulness and frequency of use amongst building designers. Queries were not restricted to the direct opinion of building designers, but also included consultants' and researchers', professionals involved in working with designers. Each part of the framework was explored using a multiple choice question, clearly presenting all the elements relevant to it, followed by an open end question to collect further opinions on it. Although designed to enable full quantitative data analysis, results were not intended to provide a statistical panorama but to gather further insights into each topic of the framework. The framework was also presented to five architecture practices in the UK, each followed by a discussion and a semi-structured interview to obtain critical feedback, and further insights into the framework.
The different interdisciplinary research methods and principles used in this work were combined and structured as summarized in Figure 2. The underpinning rationale behind this structure was inspired by the design approach to problem-solving from the discipline of Interaction Design (shown in light grey): a discipline focused on developing interfaces for people to interact with machines (Cooper, Reimann, and Cronin 2007). In this discipline, the user is central and, therefore, basic activities consist of the establishment of user requirements and the design of alternatives to meet them, followed by prototyping these alternatives for user assessment. There is a comprehensive evaluation of user experiences in all stages (Rogers, Sharp, and Preece 2011).
Interaction Design activities were strongly interrelated especially in the beginning of this work. User requirements were explored in conjunction with alternatives to meet them through design journals which enabled problem definition and solution to co-evolve. The framework emerged progressively through an iterative process of Thematic Analysis of the data, with reference to the principles of Information Visualisation and dynamic thermal modelling. This iterative process of analysis, drawing on appropriate methods and relevant theory is characteristic of Grounded Theory. Participatory Action Research provided the data for this process, and as the framework emerged this data was continually revisited. The questionnaire and semi-structured interviews were undertaken at the end of the process, once a framework was ready to be tested. The three main parts of this work are highlighted in capital bold underneath the column called 'structure' in Figure 2. Insights from designers are described in conjunction with the emergence of the framework in Section 4. Framework concepts and connections are summarized in Section 5 and Section 6 describes further insights and criticisms.

The emergence of the framework
The participatory phase comprised a set of experiments in which building designers were provided with fundamentals of simplified heat balance methods and requested to solve a design problem tailored to facilitate the extraction of meaningful performance information to decision-making unconstrained from any kind of software interface 8 .
By experimenting, one constantly updates his/her knowledge on the subject, creates a repertoire of tested solutions for a set of different problems and, more importantly, manipulates and tests different ways of applying knowledge to solve a problem, i.e. develop different strategies. (Bleil de Souza 2013) Besides that, 'experimenting sets one free to manipulate the ways of solving a problem and therefore opens the possibilities for one's imagination to interfere in the process' (Bleil de Souza 2013). The idea behind the experiments was to observe how different designers would propose integrating the fundamentals of physics throughout their design processes and how they would interpret and manipulate physics information to undertake design decisions The task was to design a façade for an office building, for which designers had already proposed a structural skeleton and a customized internal layout. Results were presented as a series of design journals with narratives of all steps involved in the design process specifically focusing on how and in which parts of it performance assessment informed design. Design journals are generally a collage of the 'conversation with the materials of the situation' designers undertake while designing, i.e. they are generally an organized series of annotated drawings, pictures, calculations, etc. with a bit of formal text, rarely a structure, to guide their reading by someone external to the project. They are a useful instrument to help designers to organize their thoughts either throughout or after finishing a design project. This type of data collection causes little interruption or interference in the fluidity of the design process, especially if the bits of it are assembled at the end of the project. It provides a good media to report the 'modus operandi' of the building designer and showing the idea of structuring problem-solving disappears in sequences of design moves directed by reflection in action underpinned by complex experiments.
Design journals provided an opportunity to unfold meaningful building thermal physics information for design decision-making embedded within sequences of design moves directed by reflection in action. The unfolding of this information was undertaken through a series of Thematic Analyses in the data sample with the help of the qualitative research tool NVivo 2012 (QRS International 2013) 9 . The following three main themes proved to be recurrent and provided the necessary breadth for subthemes to be constantly extracted and refined until a suitable set of relevant concepts to enable the generation of a framework could emerge: • Theme 1: Strategies to approach the design of low energy buildings (mainly passive design strategies). • Theme 2: Sequences of design decisions and analysis in which either design moves were followed by performance queries (e.g. proposing a façade construction and analysing its compliance with regulations) or analysis processes were used to seek design advice prior to undertake a design move (e.g. doing a sensitivity test in solar heat gain coefficient (SHGC) prior to choose the type of glass). • Theme 3: Examples of visualization of meaningful building physics data to design decision-making.
From themes 1 and 2, it was possible to infer building designers had very specific aims when using performance data to inform design decisions. A synthesis of these aims (Table 1) provides insights into understanding why designers potentially want to use simulation tools. These aims are not only to obtain performance results, but describe the specific uses of physics to inform design decisions. They were not always explicit in the data but implicitly embedded in sequences of design actions and strategies, meaning they could only be identified from concrete examples of building design problem-solving.
Aims or uses for physicis to inform design decisions were readily and often changed by the designers within the design process. Reasons behind changes were many: Changes could be related directly to the results of an assessment (e.g. a piece of design is not meeting a target and the designer wants to understand why). They could also happen because of the different implications of a move (a designer might interrupt the process of optimizing a shading device to understand the impact of it in internal lighting levels). They could be related to the fact that other design criteria (cost, costruction, aesthetics, etc.) were not satisfactorily achieved (a cladding compliant with building regulations could be changed if not satisfactory meeting cost or construction criteria). They could be related to unintended effects of an action leading to a new idea or direction for the design development, to cite only a few.
Design journals showed each designer gradually formulates a design problem to be solved by making specific Undertake a specific design action and assess the consequences of this action in the overall performance Meeting a target Quantify how far a specific type of performance result is from a prescribed benchmark and inform the user which building design variables are the responsible for this mismatch Assessing a specific product Assess the performance result of integrating a specific system or product in the design of a building Optimizing Find the optimum quantities for a specific set of parameters to achieve a best performance target design moves followed by performance queries or by seeking design advice prior to undertaking specific moves. Each designer decided when and what aspect of building thermal physics was more appropriate to be used depending on the specific design action to be undertaken or evaluated. Sequences of design moves were clearly different from designer to designer but types of actions and the means by which building thermal physics was used to inform design decisions were easily identifiable and recurrent. For example, when designer 1 was exploring different façade construction systems to comply with building regulations, designer 2 was optimizing the design of a shading device. Later, designer 1 examined the performance of a set of proposed shading devices while designer 2 tested a proposed construction system with regards to compliance with building regulations.
Common types of design actions in this data set are reported in Table 2. These actions were recorded and categorized under the headings of parametric and non-parametric changes, depending on the type of design parameter they involve. Parametric changes comprise design parameters that can be controlled within a numerical scales (e.g. window area), whereas non-parametric changes comprise design parameters that have to be discretely controlled (e.g. lists of window materials etc.). Some parameters can be listed as both parametric and non-parametric, depending on how designers decide to manipulate them. Indexes can be used to shortcut certain actions or to explore connections between parameters (e.g. glazing ratio).
Further iterations with the data set enabled the researchers to see that aims or uses of physics in design decision-making could be achieved through different means, i.e. using different types of analysis processes, depending on the type of design action(s) undertaken or sought to be undertaken. For instance, a designer trying to understand a specific performance result could use elimination parametrics or a series of sensitivity tests on specific design parameters of interest. Elimination parametrics would provide an overview of the most important variables that affect the building performance, whereas sensitivity tests would rank the importance of a specific set of parameters. Types of analysis commonly found in the data set are presented in Table 3.
The majority of analysis processes used in the data set were either descriptive or comparative, potentially due to the limitations imposed by the use of simplified methods. Descriptions were mainly used as a base case or starting point for comparisons. Comparisons were generally used to understand contributions of the different heat balance components into overall building behaviour, i.e. to inform the main causes of building performance, potentially highlighting to designers where to act in order to improve it. Optimization techniques were used to inform the design of shading devices. Sensitivity tests were used to guide decisions related to window areas and materials. Elimination parametric was used to understand the role of internal gains in overall building behaviour. Decisions about when and which analysis processes were appropriate to be used throughout the design process were always undertaken by designers themselves.
In parallel with identifying and mapping design aims, design actions and analysis processes, the researchers also structured and categorized meaningful building physics representations to design decision-making (Theme 3). Designers tend to produce information rich displays to connect thermal building physics information with building design parameters. Figure 3(a) and 3(b) shows an image gallery of some representation systems found in the data set together with a description of the metrics and types of displays used by designers to attribute meaning to the information they are manipulating. Figure 3(a) shows a collection of common design displays with building physics information represented through colours, text or line coding. Figure 3(b) shows a collection of more abstract types of representations (graphs and tables) complemented by information related to building location and building parameters. In both cases, it is not uncommon to find redundant information about building thermal physics displayed using different media (e.g. colour coding and texts displaying the same metric) potentially to reinforce their meaning in relation to where changes can be made.
All non-duplicated examples of visual information contained in the data set were put into a database (Microsoft Access 2012) and constantly queried and filtered until a suitable set of relationships among its subthemes could be  To explain causes of a specific building behaviour or performance results Sensitivity analysis To inform on the sensitivity of the model to changing a single parameter To inform on the sensitivity of the model to changing n parameters Optimization To inform on the best performance for the optimum combination of a group of pre-defined parameters   established. Recurrent subthemes referring to meaningful representation systems to building designers could then be organized into a structure containing: • Metrics used to display relevant information • Type of display used to provide meaningful information • Types of interaction afforded by each representation • Analysis processes (as in Table 3) afforded by each representation The meaning of metrics relevant to display useful information to design decision-making was extensively explored in relation to the design aims and analysis processes listed in Tables 1 and 3. As the primary aim was to gather insights into dynamic thermal simulation outputs meaningful to design decision-making, some metrics needed to be adjusted to coherently illustrate the dynamic, systemic, nonlinear and stochastic nature of results provided by these tools. This means that heat balance component metrics, used by designers to understand causes behind performance results, needed to be replaced by overall performance metrics (e.g. temperatures or loads) post-processed from elimination parametric or sensitivity tests. In addition, temperature metrics were adjusted to provide better indications of comfort (e.g. air temperatures were replaced by environmental/operative temperatures) and new metrics were added to expand the scope and enable more comprehensive analyses (e.g. time exceeding glare index setpoint etc.). The list of relevant metrics to design decision-making is summarized in Table 4. Types of displays used by designers were subdivided into location-based representation systems and abstract representation systems. 'Location based representation systems mean representation systems used to inform where a specific parameter or performance result would happen or which specific building design element is the main one responsible for causing specific resultant behaviour' . In these representation systems, building thermal physics metrics are connected to familiar building design displays (e.g. plans, sections, elevations, construction details and different types of perspectives). Abstract representation systems comprise displays that complement location-based representation systems and include many types of graphs and tables. They illustrate data not easily represented in space such as information related to time of events, or information related to statistical analysis, etc. An indicative notation system in the form of a pseudo-code was developed to describe each type of display together with an indicative of how information in each different type of display could be highlighted to facilitate interpretation.
Different types of interaction with data afforded by each representation were explored according to the categories proposed by Schneiderman (1996), combining insights from the data set with information from Table 3. Information from the data set showed that whilst designers would keep an eye on overview information to assess and foresee the consequences of their moves, they were extensively making use of 'zooms' in specific areas of interest in search for information to fulfil design aims. Insights from the data suggested designers should be enabled to: • Zoom into different time frames (seasonal, monthly, typical days, etc.) -to understand when performance needs to be improved • Zoom into different building locations/orientations (different spaces, sectors, facades) -to understand where performance needs to be improved • Zoom into design parameters (if they were varied across different models) -to understand their effect in performance when changed Combining information from the data set with analysis processes listed in Table 3 suggested designers would also benefit from: • Zooming into building or user-related variables used in elimination parametric studies -to understand what is causing a specific building performance • Zooming into design parameters varied in sensitivity tests -to understand how important each of these pre-selected parameter is on the overall building performance • Zooming into optimization results -to retrieve optimum combinations of parameters for best performance results The aforementioned metrics, types of display, types of interaction with data and analysis processes proved to be useful categories to structure a conceptual data model (discussed in a different paper) to produce meaningful output thermal simulation data to building design decisionmaking. This conceptual data model is intended to be the embryo of a database which connects an open-ended list of the most appropriate types of displays for each different type of metric, interaction with data and analysis process meaningful to design decision-making. Lists of appropriate types of display imply that users are free to choose the representation they are more comfortable with within boundaries that prevent inappropriate associations (e.g. displaying monthly peak loads using line graphs). These lists are not seen as exhaustive, enabling contributions from users and further studies to be constantly added.

Framework concepts and connections
The aforementioned section discussed in detail how the different elements of the framework emerged from several iterations with the data set. The tables are far from exhaustive and refer to the empirical data set. Formal definitions for these elements as well as how they relate to each other within the complexity of different design experiments are provided in this section which finishes with patterns for design decision-making, a 'device' by which the framework can produce simulation outputs that are appropriate for design decision-making.
These research findings suggest the elements of a framework to produce meaningful dynamic thermal simulation outputs to building design decision-making should be: • Design aims (Table 1): objective descriptions of the different uses dynamic thermal simulation tools have in design decision-making, i.e. objective explanations about why building designers want to use dynamic thermal simulation tools to inform or assess design decisions. • Design actions (Table 2): common types of actions designers undertake while designing potentially summarized in terms of changes associated to main design parameters. • Analysis processes (Table 3): describe the means to use dynamic thermal simulation tools to inform design decisions, i.e. describe how building designers would use dynamic thermal simulation tools to inform or assess design decisions. • Conceptual data model, embryo of a database of meaningful dynamic thermal simulation outputs to design decision-making: connects an open-ended list of the most appropriate types of displays for each different type of metric, interaction with data and analysis process meaningful to design decisionmaking.
Framework elements are designed to be embedded in sequences of moves directed by reflection in action through a set of questions, with answers to be retrieved from a database of simulation outputs (Figure 4). Each question has a standard part, composed of a design aim plus the analysis process to achieve this aim, followed by a customized part which contains the design actions to be assessed or to seek advice on. The components of a question are used to limit the search for information in a database of outputs structured according to that proposed in Figure 4. Design aim and actions provide information on the type of metric and interaction with data to be made available, whereas analysis process restricts the types of analysis and interaction with the data to be displayed. Designers are then left with a narrow list of output data displays to manipulate and query.
A full list of design aims and analysis processes which comprise the standard part of the questions are displayed in Table 5. Questions are either performance queries or intended to provide design advice. In any case, they are constructed such that aims connect with analysis processes unambiguously. Table 5 shows that questions are independent of design actions, i.e. they enable any type of action to be undertaken prior to or as a result of asking a question leaving the designer free to choose and decide what to do. Table 5 also shows that questions are in tune with the complex types of experiments designers undertake because they are not prescriptive in terms of the way they can be used to assist the design decision-making process. For instance, questions type E1 and E2 can be directly used in exploratory experiments simply to observe what follows, without accompanying predictions or expectations, and they can also be used in move-testing experiments to aid affirming or negating moves.
Just as design actions (i.e. variations of building parameter(s)) can be placed into any type of question, questions can also be placed in any sequence depending on the complexity involved in the 'what if' experiment a designer is undertaking. This means it is up to the designer to stop the process whenever (s)he judges changes in the whole are satisfactory, and guarantees flexibility to explore new questions discovered from new meanings unfolded from sequences of design experiments.
The framework elements and simple structure also enables abstract descriptions of recurrent ways of connecting standard questions with design actions, and recurrent ways of assembling different standard questions already connected to specific types of design actions, both to be easily recorded. These records, patterns of decision-making 10 , can be used to produce catalogues of examples, to populate databases of decision-making processes, to construct design management structures, to exchange information with partners, etc. 'They are the building blocks for designers to create their own sequences of informed design decisions' . Examples of patterns were extracted from the data set showing how different designers proceeded from aims to decisions using specific sets of metrics and representation systems and were used to illustrate the concept of the framework when discussing this research project with UK design practices. The patterns and examples are described more fully in another paper.

Further insights and critical feedback from designers
The elements of the framework were tested in an online questionnaire, from which 65 responses were collected over eight weeks. The elements and structure of the framework were also presented and discussed in five semi-structured interviews with UK architectural design practices. Results of these two tests are reported together in this section.
Design aims from Table 1 were initially queried with regard to their usefulness and frequency of use by building designers. These questions attempted to confirm the importance of these aims and identify potentially hidden user's needs from mismatches between aims not frequently used despite being considered very useful. There was also room in this section for respondents to ascertain whether any aim had been missed.
Results show there are no significant differences between responses from designers, consultants and researchers for these questions. The vast majority of questionnaire respondents and interviewees quoted aims from Table 1 as either meaningful or somewhat meaningful, and always or frequently used. Interviewees in particular considered the list accurate and comprehensive. Reported missing aims relevant to building performance were mainly related to climate and site analysis.
Design actions were queried in detail on the questionnaire with regard to stages of the design process when they tend to occur. Table 6 shows a list of design actions undertaken by designers at the different design stages: conceptual, development and refinement. The colour coding highlights the percentage of responses in each cell using a continuous gradient that goes from black (0%) to white (100%) with shades of grey (50% equivalent to mid-grey) 11 . Even though some actions tend to be predominantly undertaken at a specific design stage (shades from light grey to white), many of them tend to be undertaken by at least 40% of respondents either at the conceptual or design development stages. This means, that boundaries between conceptual stages and design development are particularly fuzzy for most actions, especially if they are related to transparent elements, opaque elements and shading devices. This reinforces and supports the idea that the framework should enable any design action to be undertaken at any moment in the design process.
Interviews reinforced this point and emphasized that building information modelling (BIM) environments also loosen up boundaries between design stages. By connecting geometry information to a database of attributes, they enable designers to explore different design fronts simultaneously, facilitating iterations between strategic (upper level) and detailed decisions.
Analysis processes, similarly to design aims, were queried with regard to their usefulness to building designers to confirm their need to facilitate and/or enable different types of design advice and performance queries. Again, no significant differences were found between responses from designers, consultants and researchers for this question. A large majority of questionnaire respondents and interviewees considered the analysis processes listed as either very useful or somewhat useful. These processes were considered comprehensive enough to cover all types of design requirements and aims and nothing relevant to this topic was reported as missing. Interviewees emphasized analysis processes as instrumental to control design decision-making, claiming this would enable them to set up more comprehensive performance queries and design strategies, while designing to better organise and decide on which design parameters to investigate further and/or gather advice about, work with evidence prior to approaching consultants and clients, be able to challenge consultants proposals thought to be biased by design views from other domains (e.g. HVAC design) and confirm intuitive hypotheses. Interviews unanimously emphasized control (or partial control) of analysis processes as the one of the most important aspects of integrating thermal simulation tools throughout the building design process.
Meaningful information for design decision-making was queried in a very generic way (Table 7). Results show that much of the useful information for design decisionmaking comes from identifying the contribution of building elements and aspects of building usage to building performance. This finding was also confirmed in the last part of the questionnaire in which performance results associated with each specific type of design action were, in the majority of cases, stated as very useful or somewhat useful pieces of information to design decision-making. Interviewees emphasized that results should be provided in a simple, effective and efficient format. Customizable reports and real-time performance feedback were mentioned and overviews similar to car or boat dashboards were considered potentially powerful displays to control parametric design changes. Overviews could/should also include integrated performance metrics to enable comprehensive performance assessment to be undertaken. As, for instance, improvements on thermal performance might impact on lighting performance, metrics for lighting performance should either be provided or integrated with metrics from thermal performance to enable it to be taken into account in the decision-making process. Performance metrics related to cost, in relation to performance outcomes and/or design parameters, were also highlighted as important, especially the rate of return on investments, which was considered important information for discussion of decisions with clients.
Questionnaire respondents with a research background as well as some interviewees pointed out that designers should somehow be aware of the importance and influence of decisions and input information related to HVAC and HVAC controls in building performance. Consultants and interviewees also warned about occupancy-related issues affecting performance, but discussion about how to overcome the fact that assumptions related to user behaviour do not always correspond to reality were inconclusive.
The framework structure was seen by interviewees as consistent and coherent with their 'modus operandi' which would enable them to organize design decisions without being prescriptive. They were enthusiastic about having the tools to be able to construct their own patterns (or procedures) for decision-making, which could accommodate the uniqueness of each different design problem, but at the same time act as a repository of knowledge to provide insights into solving new problems, address many problems prior to getting consultants involved with the project and to act as a platform for sharing information and transferring expertise with team members and partners. Having a framework which affords the creation of patterns for decision-making was seen by all interviewees as potentially a powerful instrument of management and control to organize designer's knowledge and enrich the design process as patterns could also be used as an instrument for quality assurance and benchmarking, and act as an educational resource for newcomers to the office. However, designers pointed out that different types of contracts as well as differences between what is designed and what is actually built compromise the implementation of any framework. The importance of connecting performance results with BIM environments to enable actions to be assessed with regard to their implications in cost and construction processes in parallel with their implications in performance was highlighted by all interviewees as vital for the framework to be fully operational.

Conclusions and future work
The aim of this paper was to describe the process used to develop and test a framework to produce thermal simulation post-processed information meaningful to design decisionmaking. This framework considered building designers the ultimate simulation tool users (either directly or indirectly) and was developed using an Interaction Design approach.
The Participatory Action Research approach was initially used to gather insights about meaningful building thermal physics information for design decision-making, respecting the 'modus operandi' of the building designer. Successive iterations with this data, using a combination of Thematic Analysis and Grounded Theory together with principles from Information Visualization, dynamic thermal modelling and Building Design, enabled the elements of the framework to be identified and described in terms of how they relate to each other. The questionnaire and interviews confirmed the basis of the framework and enabled it to be enriched.
This comprehensive combination of multidisciplinary methods and principles focusing on building designers as ultimate users has not been explored before. The approach enabled a framework to produce thermal simulation post-processed information meaningful to design decision-making to emerge from different premises, embedded within a sequence of moves directed by reflection in action underpinned by complex 'what if' situations designers generate throughout the design process. Key aspects of this framework are the following: • It provides a limited but not limiting number of objective descriptions for the different uses dynamic thermal simulation tools have in design decision-making connected to the different means to achieve these uses, empowering designers to control the design process comprehensively and more independently from consultants. • It provides unlimited design possibilities by enabling unlimited assemblages between design aims and design actions. • It proposes a conceptual data model to construct an open-ended database with the most appropriate types of displays for each different type of metric, interaction with data and analysis process meaningful to design decision-making. This database should be structured to facilitate interaction with data, to guide designers in navigating through performance results and can be expanded and enriched at any time and/or potentially connected to real-time performance feedback systems. • It provides a structure to enable patterns of decisionmaking to be recorded, organizing the plurality of the design process, facilitating information exchange and knowledge transfer. It is opened to constant update and enrichment. • It provides freedom to designers in stopping the process whenever they judge changes in the whole are satisfactory and flexibility to explore new questions discovered from new meanings unfolded from sequences of design experiments.
Future work should be focusing on making this framework operational through: (i) collecting and testing patterns of decision-making in architectural design practices, (ii) developing the conceptual data model into a database of outputs linked to a coding system to produce displays so that it can be tested in a work environment within specific patterns of decision-making, (iii) addressing problems related to different types of contract and differences between designed and built performance, (iv) addressing problems related to the amount of resources needed to produce building physics information to design decision-making (time, skills, etc.) and considering the impact of these in the information summarized in Table 7.