Keywords

1 Introduction

In the environment we inhabit, we spend most of the time inside buildings. The way we experience these environments as users can be very different, depending on the nature and purpose of the buildings, on the social context, on the specific environmental conditions they provide, on our emotional state, etc. These multiple influences make buildings a complex and dynamic construct which architecture has to deal with. Since many interaction contexts happen within built space, many HCI research examples are linked to the built environment [24]. This includes research on home (e.g. [22, 29]) and work environments (e.g. [34, 39]). In these examples, the focus is on describing and analysing interactive artifacts in their context of use, while the built environment which hosts such context appears more as a backdrop than as interactive element in itself. Usually considered as an invariant, the affordances of the built substance may prove to actually have a non-negligible role in these contexts.

Meanwhile, the field of building automation, as understood by building engineers, has begun to install more and more building sensing and automation systems, motivated by the quest for energy efficiency. These systems are designed through a development centered on automatic control and optimisation of technical installations. Energy consumption standards play a central role, requiring quantitative proof of performance and bypassing the uncertainty and diversity of user behaviour to support automatic solutions [36]. As a result, a series of negative user experiences have become stereotypes of the work environment: automated blinds with a behaviour that seems erratic to the user, extremely controlled environments leading to a feeling of being controlled by the building oneself, sick building syndrome [26], etc. With this automation paradigm now reaching the housing context through energy efficiency requirements, such negative experiences are being broadened to a wider set of users and situations. Recent progress in different sensing and actuation techniques for the built environment (think IoT [29]) and their shrinking prices can be seen as signs of even more profound changes to come.

Indeed, the digitalizing of buildings and their existence as interactive objects does not appear to be a question of the near future, but a fact of the present whose implications need to be studied with methodologies able to fully take into account users and usage. This context represents an opportunity for HCI research to play a role in studying the use of existing or newly proposed digital artifacts in relation to the built structure, and also in fostering interactions with non-digital elements of the building through digital means. As a goal to seek with as little energy consumption as possible, the notion of comfort is central to the built environment [26]. It relates to a very specific characteristic of built environment interactive contexts: users are physically immersed within the interactive object, bearing the consequences of their interactions in a multi-sensory way, possibly going as far as creating their own lack of comfort.

The present article builds on research contributions from engineering [48] and HCI [4, 5] to present and consolidate the concept of Human-Building Interaction (HBI). It proposes a common and operational definition through the systematic mapping of the dimensions that are relevant to HBI and the modalities with which these dimensions can be acted upon. Pursuing the aim to establish HBI as a relevant component of HCI research, the article first provides a cartography of the fields that comprise HBI (Sect. 2). Drawing from a large span of cross-disciplinary contributions, the focus is then put on characterizing HBI in terms of dimensions (Sect. 3) and interaction modalities (Sect. 4). The methodological challenges raised are then discussed (Sect. 5) and some examples of research are presented (Sect. 6). Finally, new directions for future research are proposed (Sect. 7).

2 A Definition of HBI

To define HBI in general is a difficult task, and its interdisciplinary nature has resulted in the lack of a single clear definition. As a contribution to the consolidation of the field, we propose to construct a definition in several steps. First, we will develop the immersive specificity of HBI mentioned in the introduction. Second, we will draw an outline of HBI through a cartography of the fields that compose it. In a third step we will enumerate the different dimensions that can be involved when considering a user in interaction with a building. In the final step, the modalities that can be used to affect these dimensions will be described. Together, these steps represent an operational definition of HBI: by evaluating whether a particular dimension is impacted by the use of a specific modality allows the assessment of whether a particular interaction context is part of our definition of HBI.

Fig. 1.
figure 1

Standard HCI (left) compared to Human-Building Interaction specificity (right). Interaction feedback uses potentially multiple sensory channels: tactile, acoustic, visual, radiative, convective or olfactory

Full size image

2.1 Specificity of HBI

With regard to a more standard conception of HCI in which a user interacts with a machine through well-defined and circumscribed modalities, HBI considers users as completely immersed in an interactive object. The building contains its users and influences their experience through multiple channels (e.g. heat radiation, light or sound reflections, air movement, etc. see Fig. 1). An important consequence is that the user cannot terminate the interactive session without leaving the space. Conversely, users’ actions may have a more important impact given this immersion, possibly leading to physiological changes in the user, sometimes to the point of making the interactive space inhospitable.

In addition to this immersive aspect, due to the building’s physical behaviour, users’ actions may have repercussions on very different time scales. Opening a window in the middle of a summer afternoon may have the immediate effect of inducing fresh air convection, but also lets heat enter, which can have consequences on comfort in the days to come. Space is also present at different scales: the comfort sensation can be influenced by minute air movement around the body, as well as by the radiative heat produced by a far away sun.

In the semantic field of HCI, such an interactive context could be seen as an ambient interface pushed to the extreme [59], or more precisely to the conjunction of the extremes of an ambient and a tangible interface. The field of ambient intelligence (AmI) presents an overlap with the HBI approach by linking sensing, reasoning, action, HCI and privacy to achieve intelligent environments and thus shares this immersive component [8, 18]. Aarts and de Ruyter [1] present a particularly interesting discussion sharing many aspects developed in the present article, which however reveals that AmI emphasis lies on digital means to solve tasks and mainly considers interaction with digital artifacts, considering the building structure as a backdrop. For example, Thomas and Cook [64] consider energy efficiency of smart buildings as a task of intelligent turning off of unused appliances, without tackling the comfort question. In contrast, HBI’s primary concern is to consider the built structure as an interactive element whose inclusion is essential to grasp the full complexity of interactions. As a result, the discussion here reflects this change of perspective.

Fig. 2.
figure 2

Cartography of the HBI-relevant research fields

Full size image

2.2 HBI Field Cartography

The first step of our definition is to propose a mapping of the fields composing HBI, that is able to provide a means to attribute to or exclude existing research work from HBI (see Fig. 2). In this process, four fields are identified as components. The component that appears most relevant is the one of architecture and urban design. This component can be understood as an existing or future physical object, but also as a design process. It encompasses complex multi-disciplinary issues related to the definition of the built environment and entails by nature strong cultural differences. In the present paper, we will not consider the urban scale and concentrate only on the scale of buildings.

The building industry and the economy within which it develops constitute the second component. It includes building material availability, and the different processes inherent to the production of buildings, such as norms and laws. This component is strongly influenced by regional differences. By being the context that ultimately adopts a candidate innovative solution, in our opinion this component plays a central role in defining relevant research axes.

The third component represents the technological availability—at an affordable price—of sensors, screens, actuators, algorithms, etc., as well as the recent proliferation of connected personal artifacts. This dimension is encompassed in the concept of ubiquitous computing, and the current developments of IoT and AmI are part of it. This component is less impacted by regional differences, except for differences to privacy laws and technology/Internet access or adoption.

The final component represents the HCI contribution. It encompasses interface design, development and evaluation, but also the study of users’ behaviour with existing interactive architectural elements.

Once these components are defined, it becomes possible to propose a more precise outline of what is meant here by HBI: combining one of the first two dimensions (architecture/urban design or building industry/economy) with one of the last two components (ubiquitous computing or interaction design) would result in a large definition of HBI. The conjunction of all components would result in a more stringent definition. Though probably too restrictive, this last definition enables the identification of directions for project development to broaden research impact.

3 HBI Dimensions

Contributing to the definition we set out to construct, we propose a mapping of the dimensions that impact HBI, drawing a clear distinction of whether these affect users or the infrastructure of the building itself. On the users’ side, these dimensions encompass comfort, emotions, behaviours and awareness; and quality, usability, efficiency and privacy on the infrastructure/building side. Interrelations between these dimensions reflect the complexity of the problem considered (see Fig. 3, right).

In a research article close to the present argument, Rodden and Benford [59] use as basis the “Site, Structure, Skin, Services, Space plan, Stuff” decomposition by Brand [12] to develop an HCI approach to buildings. Supporting our argument on the immersive aspect of HBI, they acknowledge that “Site”, “Structure” and “Skin” receive less attention in HCI research than the remaining dimensions. However, we believe that in following Brand [12], their approach has too strong an architectural edge. Instead, we propose to keep the same emphasis on the built structure while considering a user-oriented approach to grasp the whole depth of human-building interactions.

Fig. 3.
figure 3

Proposed HBI dimensions and some of their interrelations (left), and HBI modalities acting on these dimensions (right)

Full size image

Comfort. Taking its origin in the 19th century hygienist movement for building salubrity, the notion of comfort is ubiquitous in the building context and is related to its immersive specificity: as main purpose of a dwelling, comfort is a space quality to be guaranteed. We draw here a distinction between comfort measured at the building environment scale and at the individual scale. Typically, the time scales differ and are larger when measuring at the building scale. Many studies have looked at factors influencing user comfort in buildings; see [30, 56] for reviews on the subject. Perceived comfort, which is the translation in the users’ mind of a body’s response into a perceived comfort sensation, can be culturally and psychologically influenced [26, 42]. For this reason, we categorize this part as a users’ emotions.

At the building scale, the Fanger model [28], widely used in the building industry, allows to compute the Predicted Percentage of Discontent (PPD) as a function of objective variables such as temperature, humidity, etc. but also user clothing and activity. Aimed more at fully controlled environments, this model has been completed by the adaptive model that takes into account the fact that buildings may be naturally ventilated [17]. For comfort at an individual scale, measurements consist of skin temperature, conductivity, or eye movement to capture the actual response of the users’ body to the environmental conditions [38]. Such responses present large individual differences and have been shown to be gender-dependant [42]. Ranjan and Scott [58] propose to use thermal imaging to infer thermal comfort while Knecht et al. [44] investigate the use of wearables for thermal comfort.

While it must be evident to the HCI community that perceived comfort is the one that matters, it is comfort measured at the environment scale that is used during the architectural design process. This measure is efficient for bringing the project forward without having to care about the users’ variability, unknown at the design time, and which is typically prone to change during the building lifetime. Consequently, this model is applied during all design phases, in very diverse cultural and climatic contexts, which contributes to a global standardization of interior environments and technology requirements. As a result, many HCI research examples rely on the Fanger model, including [17, 39, 66].

User Emotions. By users’ emotions we mean emotions and feelings generated by or in interaction with the built environment. Beyond comfort perception as already stated, there are numerous examples, starting from the happiness or calm felt by a view towards a specific landscape (thus representing interaction with “Site” [12]), to the annoyance created by the arbitrary and alienating behaviour of automatically controlled blinds. Other examples are more subtle, or even subject to discussion, such as boredom or disenchantment generated by low quality architecture. This emotional aspect plays an important role in the users’ perception of their environment [7]. A building presenting a special architecture can become an iconic reference that contributes to identification and pride to be one of its users (see the work of the studio Lacaton-Vassal as an example [62]).

User Behaviour. By behaviour, we mean here the practices consisting in interacting with the different building elements. These behaviours are often part of a culture and function symbiotically with the architecture; for example, closing the windows during the hot part of the day in conjunction with high ceilings and the practice of a siesta in Mediterranean countries. This dimension is fundamental to the building’s function and durability, as improper practices may rapidly degrade it, waste energy, raise security issues or lower comfort. From an engineering perspective, Haldi and Robinson [32] and Langevin et al. [48] present studies of occupants’ interaction with windows or building elements, while Crabtree and Tolmie [23], following an ethnographic approach, thoroughly document interactions with “things” within a day in a domestic environment.

The recent evolution of comfort conventions in the occidental world (uniformly heated space, mechanical air renewal, electric lighting, etc.) in parallel with energy and economic efficiency needs have led to increased automation in the building context, ranging from thermostatic valves to sensor-activated lighting. However, it has been shown that the impossibility to act upon one’s environment in order to adapt one’s comfort induces an increased intolerance to non-standard comfort conditions [33, 41, 56]. The very fact of giving the possibility to interact thus has important implications in terms of perceived comfort. Social dynamics may significantly influence individual perception [66], while the building itself may have an important influence on the relations between users (e.g. through a bad acoustic in a meeting). Following Rodden and Benford [59], HBI should also consider as users the different stakeholders of the building context: users, designers, landlords, engineers, building facility managers, etc.

Building Behaviour Awareness. Through the conjunction of multiple time scales, the influence of external conditions, user presence or not, as well as the physical behaviour of its elements, building behaviour is dynamic and complex. Most interactions actually happen to counter-act or influence building fluctuations, for instance due to changing weather conditions (e.g. lowering blinds, opening windows, turning on lights). To have adequate practices requires a given awareness of such behaviour [25].

This awareness can be acquired by cultural transmission, by experience, or through direct experimentation with the building. In this sense, building automation can induce different biases: the automation concept may not be adapted to the cultural context (for instance the habit of leaving a window open during winter nights in a high performance building equipped with heat recovery HVAC), may induce the user to acquire experience with the automation itself which tends to rapidly become obsolete, or presents irregular (irrational) behaviour in the eyes of the experimenting user.

Architectural Quality. If architectural quality is especially difficult to define, and strongly depends on its function and cultural aspect, an argument is nevertheless commonly accepted: a building presents a certain quality if stakeholders in society invest funds to preserve it, be it for socio-economical or cultural reasons. Even if the reasons to renovate or demolish a building tend to change, this corresponds to a sustainability objective [62]. Indeed, buildings that last are those that are able to adapt to evolutions in functions and use [12]. Optimization of building functions, such as energy optimization, is actually going against such adaptability by tightening the range of possible building behaviour. HBI should thus support the production of quality architecture, for instance by providing measures of space use in different conditions or by augmenting existing buildings with digital artifacts to make them more adaptable.

Building Usability. The design of the building and of the elements modifying its dynamics may by their affordances induce specific behaviours, both positive or problematic. Through their non-definitive and manual aspect, they can induce an interactive behaviour, possibly even fostering experimentation. For example, in the housing buildings by the studio Lacaton-Vassal, thermal comfort in some part of the apartments requires the manipulation of insulating curtains [62]. Inhabitants must therefore actively interact in order to benefit from the space, and typically need an adaptation period after moving in to understand its functioning. This exemplifies an approach where users are asked to take responsibility in a mixed-mode approach (see Sect. 4).

Energy Efficiency. To preserve comfort in the building context, energy is used to heat, cool, lighten or ventilate, in order to preserve interior space from climatic fluctuations or evacuate pollutants emitted by interior activities. The building stock has been identified as presenting the most important energy saving potential through retrofitted or highly efficient new buildings [50]. In order to activate this potential, more and more stringent efficiency constraints are being enforced, while maintaining high comfort levels [68]. In order to achieve quantifiable results, this process often encourages building automation: if the SmartHome concept has yet to demonstrate wide adoption [15], energy efficient buildings have integrated a level of automation that makes the dimension of energy particularly relevant in terms of interaction [52].

In parallel, the same energy resource context has generated a large body of work on “behaviour change”: inducing users to change their behaviour with the aim of lowering their energy consumption. Abrahamse et al. [2] presents a review of interventions aimed at energy conservation, while Pierce and Paulos [57] review and discuss how such intervention acquire meaning for HCI by involving digital artifacts. More recent examples include [3, 10, 58, 70], among others. Intertwined with comfort and related to the immersive context, the dimension of energy efficiency is of primary importance for HBI.

Private Sphere. One of the functions of a building and its interior design is to articulate a physical boundary between public and private. New technologies with their multiple sensing devices and wireless approaches tend to blur this physicality, both from the point of view of information leakage and of limit perception from the user: a wall is more tangible as a limit than a firewall software [31]. It is important to address these questions of perception and the corresponding mental models in order to understand the articulation between physical and digital limits as they are felt by the user [16, 40]; even more since data mining techniques are becoming able to infer users’ behaviours [6, 19].

4 HBI Modalities

Once the dimension set spanning the HBI space is defined, the different levers or techniques able to influence these dimensions can be discussed. We set out this task by presenting a schematic representation (see Fig. 3, right) and developing each element. Here, instead of using the architecturally-centered framework of Brand [12], as used in [59], which for instance differentiates “Skin” and “Structure”, we propose a decomposition centered on change from the user perspective.

External Conditions. Outdoor conditions are important to users inside through the relationship between the outdoor (diurnal/seasonal) and indoor timescales. Comfort perception is influenced by outside conditions [26], while transitions between the outdoor and indoor environment can lead to temporary perceptions of discomfort [38]. Daylight access, but also view access, has an influence on perceived comfort and user health [56]. With lowered heating energy consumption, lowering lighting energy through daylighting strategies has become more important. Gaver et al. [31] present a research study of the implications of digital mediation of outside conditions towards inside users.

Built Structure. Volume, access, openings, circulations, etc. are essential elements channelling user experience, inducing a reaction or its absence [24]. This modality represents all permanent elements typically not open to short term changes from the user and thus represents the immersive constraint. If, for this reason, the elements may appear as static, the building’s dynamic physical behaviour (mechanical, visual, thermal and acoustic) has an important impact on user perception and comfort. Users’ actions may influence and exploit this behaviour, depending on their awareness of it.

This cooperation between user and structure has a long tradition in the built environment, going back to vernacular architecture, and is strongly related to social and technological settings. For instance, occidental comfort requirements, but also organisation of work, are translated into the structure in the form of insulation, ventilation systems and window sizes, for instance. To start considering the user in relation to this permanent structure and how the digital component relates to this structure is a new area of research for HCI. In studying energy aware behaviour, Chetty et al. [16] elicited the important role of the built infrastructure or site location in user behaviour.

Interior Elements. By interior elements is meant here all elements that are in reach of interior user manipulation to influence the user’s immersive context. On top of contributing to space perception, each of these elements presents its own affordances, which translate into use patterns [23], which in turn influence user experience on several scales and dimensions. For instance, a window with several aperture modes will influence air renewal modes and strategies, the relation with the outside, or the building’s thermal behaviour. HCI may be involved in studying the existing digital component of these interior elements or adding one to it, but also in influencing interactions with non-digital elements through digital means.

Fig. 4.
figure 4

Illustration of the underlying user model, in the actual normative framework (right), in the mart home concept (middle) and in the proposed mixed mode automation (left).

Full size image

Building Automation, Sensing/Actuation. To avoid potential discomfort within the immersive context, elements of building control are often essential to contemporary buildings to maintain comfort conditions, for instance in terms of heating/cooling, lighting or air renewal. Control is achieved through the help of sensors and actuators able to take into account user behaviour, possibly only indirectly. However, due to the difficulty to take users’ behaviour variability into account in building system automation [55], building system concepts and user models steering their reactivity are mainly based on a passive user [36], often confining them to the role of preference definition, as some examples related to the SmartHome concept [52]. If HCI involvement in the context of interfaces for automation systems seems evident and necessary [10, 39, 66], progresses in terms of sensors, automation and data analysis, in conjunction with a user-centered approach, could contribute to define a larger interaction space [11].

User Participation, Mixed-Mode Automation. The notion of active adaptive users, responsible both for their comfort and for their energy-related impact is more and more taken into account by building industry professionals [41, 56], held back though by current conservative norms. Research from the building industry has studied building users’ behaviour extensively [36, 48, 67]. However, the main driver in this research has been to reduce uncertainties in models and system control. The impact on perceived comfort, as well as the necessity to improve system robustness, are arguments for a mixed mode approach [37] combining an automatized part while leaving large autonomy to the user—ideally playful and engaging—thereby calling for HCI involvement (see Fig. 4), which corresponds to findings in [46]. In the mixed-mode context, the use of machine learning techniques to infer user preferences has shown limited success [69], while user suggestions may achieve better performance as well as acceptance than automatic techniques [3, 20, 70].

In the context of a building project, thermodynamical simulation is used to predict energy consumption. The complexity of the real system forces drastic simplifications to allow yearly whole building simulation. One such simplifications is user behaviour, often taking the form of occupancy density, which together with space programming, define heat and \(CO_2\) production or electricity consumption. It is in this case essentially a passive user who defines building performance forecasts. This simplification towards passiveness at the design stage induces a bias towards automatic solutions. The behaviour difference between real users and their simplified simulated counterparts ironically translates then into differences between real and predited performance [27].

Feedback and Visualization. With the generalization of sensing it becomes possible for users to get feedback from their interaction with building elements. Here different time scales are relevant, from direct feedback needed to confirm an action to presenting data recorded over time [66]: by analysing building data it becomes possible to offer users a mirror of their behaviour, through ambient or collective visualizations, to trigger awareness or even behaviour change. There is already work from HCI along this line related to buildings, examples include [21, 60]. Schwartz et al. [63] present a critical investigation into user perception of energy feedback. Such visualizations, together with data analysis methods, may also prove useful for other stakeholders such as expert engineers or building facility managers [13, 49, 53].

Telepresence. New communication and working modes induce a reconsideration of space perception, whose interaction questions have been addressed by the field of HCI [54]. However, implications in terms of space redefinition, for instance for teleworking, raise questions related to the modalities mentioned earlier, e.g. the mediation of external conditions or interior elements/use [31], but possibly reaching as far as modifying the built structure. Through its ability to transcend the limits of distance and material boundaries drawn by structure, telepresence naturally extends to implications in terms of private sphere.

Standards and Laws. Standards and laws, harmonized at the European level, are extremely present in the built environment context. Some represent what users expect in terms of comfort and energy performance, while others formulate user presence and behaviour models that serve as a base to forecast energy consumption, as well as dimensioning and control of building systems. In a study about user perception of energy consumption, Chetty et al. [16] emphasize: “infrastructure not just as a set of technical arrangements that provision the smart green home, but also as a set of commercial, legal, and governmental arrangements”. Since demonstrating standard compliance is easier through automated systems, the involvement of the HCI community could allow the evolution of such standard user models towards more realistic—and more active—personas [56].

5 HBI Methodologies

Through its immersive and multimodal aspects involving processes over large spatial and time scales, the HBI context is a typical one in which isolating variables is highly difficult. Fundamental methodological questions thus arise to address this context from the HCI point of view, such as drawing the limits of the system under study or repeating conditions with multiple users to achieve significance. However, we are convinced that HCI methods are particularly adapted to help design and evaluate user-centered buildings. In light of the proposed HBI modalities, some remarks are in our opinion relevant in the evaluation of interactive built environments.

New Comfort, Energy and Usability Metrics. The building automation modality with its general aim to optimize comfort with energy consumption relies on very specific metrics. For comfort, standards define several metrics such as the maximal number of overheating hours to the Percentage of Predicted Discontent (PPD) [28]. These metrics originate from very specific conditions, the climatic chambers, implying mostly user passiveness, and thus inducing a bias towards controlled environments [26]. Moreover, the PPD metrics (and their corresponding Predicted Mean Vote PMV) presuppose the precise determination of several variables such as clothing levels or metabolic rate which are often difficult to evaluate [38]. Additionally, it is now accepted that this set of metrics does not always faithfully reflect the comfort as perceived by the users themselves, as examples have shown that this perception may be modified by user action possibilities [26]. Such metrics would not be suitable for instance to prove the usefullness of a mixed-mode approach. The PMV/PPD metrics consequently need to be decomposed into measurable quantities and user appreciation to discriminate the contribution of each component and their interrelation.

Although energy metrics are less subjective, there are many different strategies to allow comparison of different energy sources, for instance using \(CO_2\) production equivalence or primary energy use, depending on the focus of the quantification. The different energetic consumption metrics are important to test interfaces aimed at fostering behaviour change. However, being aggregates, these metrics are difficult to convey to lay users [14, 43]. Moreover, Chetty et al. [16] pointed out that users have difficulties understanding energy units, such as the difference between energy (in kWh) and power (in kW), and would prefer units of cost; whereas this preferred unit may become less meaningful in the future in light of the electricity market liberalisation [3].

The above methodological difficulties are related to the prevalence of standards and laws in the built environment context. In order to make sense of building users’ behaviour, HBI methods need to take this modality into consideration, with the aim to untangle the influence of what had to be constructed/installed for compliance. This in particular calls for interdisciplinary approaches.

Human-Building Interaction Metrics. Research effort should strive towards developing a measure of the usability of a given space, whether as a consequence of its built structure, or at the level of interior elements. The ability to capture and quantify the different ways of interacting with the elements of space, whether digital or not, has the potential to inform building design. In this respect, the difference in methodologies between HCI [23] and building physics [32] is substantial.

In the scope of evaluating building digital interfaces, standard HCI acquisition methods such as logging interactions or occulometry are directly applicable. On the other hand, if building physical dynamics influence the interactive process, measurement tools need to acquire the relevant physical dimensions, such as temperature, humidity, etc. The HBI immersive specificity implies more important setups and possibly larger time scales. As an example, if occulometric measurements normally happen in the constrained environment of a display screen, physiological visual comfort acquisition in space should be able to measure eye movement in the larger context of a room, and measure a longer time sequence to capture modifications linked to sunlight exposure changes [7].

The acquisition of user interaction with the building may benefit from sensors available through the building infrastructure [32]. However, given that sensors are primarily installed for system control reasons, it is often difficult to infer user behaviour from this data [65]. A method to acquire interactions spanning from users’ movement in space to the manipulation of building elements needs to be formalized and tested; the same being true for a method to acquire the different components of comfort [38]. A longitudinal survey bridging both topics can be found in [48].

Evaluation Methods. Similarly to HCI, interactive HBI elements can be evaluated in controlled environments or directly in “in the wild” situations. In the case of controlled environments, techniques consisting in comparing within or between groups while treating independent variables are applicable. However, to address the HBI immersive specificity, the physical dimension should be taken into account, requiring an important infrastructure that allows to repeat conditions (e.g. thermal conditions), even possibly conduct experiments in parallel in the case where external conditions play a role (e.g. sunlight). For “in the wild” experiments, ethnographic or sociological qualitative techniques are used, consisting in observing and annotating videos, or conducting, transcribing and coding interviews, such as in [3, 16, 63], among others. These methods suffer from the limited time scale that can be covered. The analysis of sensor data through data mining techniques represents a new evaluation source with longer time scales and potentially high relevance [19, 65], but whose true potential remains to be discovered.

6 HBI Examples

We present here a subset of our research projects to illustrate how the modalities proposed for HBI could translate in actual research effort. These projects are developed within the smartlivinglab structure, which represents the collaboration between three Swiss academic institutions [47]. The smartlivinglab is a research program dedicated to the future of the built environment from the technical and societal perspectives. Its aim is to imagine living spaces of the future, focusing on inhabitants’ well-being and environmental concerns. In this context, the research center Human-IST (Human Centered Interaction Science and Technology) is responsible for developing and evaluating technologies able to enhance human-building interactions.

Measured and Perceived Comfort. The gap between perceived comfort and environmental comfort is a fundamental incentive for mixed-mode automation. One research project uses sensors together with interactive techniques to tackle this subject. By proposing a personal emphatic object reflecting comfort conditions in the near environment—a sort of comfort companion—this project aims first at acquiring and understanding user comfort data, and second at directly reflecting comfort conditions to the user, thus also addressing the feedback/visualization modality. Usability studies in this case allow to characterize the dynamics induced by feeding back such information on user behaviour and his felt comfort. Implications in terms of architectural design remain speculative, but it is foreseeable that results from the project may inform the design process of sustainable buildings. Reaching this state would situate the project in the strong definition of HBI as proposed in Sect. 2.2.

In a parallel project using the experience sampling method [35], a mobile application is used to acquire comfort as felt by the user (thermal, acoustic and visual comfort) through simple questionnaires (see Fig. 5), in order to compare this information with measured data from sensors available on the mobile platform [45]. Compared to the previous project, this approach has the advantage of easily scaling up, for instance to gather user perception in a public building, and may use building automation data for feedback. Similar research examples also use the experience sampling approach [38], some with the aim to close the loop by adjusting HVAC services to comfort results [39, 66].

Fig. 5.
figure 5

Through simple user questionnaires following the experience sampling approach, users’ comfort perception is compared with smartphone sensor data.

Full size image
Fig. 6.
figure 6

Live visualizations of tracking to support networking during social events.

Full size image

User Behaviour Analytics. As initial steps addressing the methodological challenge of understanding user interactions with and within space, two different contexts are considered. First, a networking event is digitally augmented through participants’ location tracking in order to induce more targeted interactions. Through fixed RFID readers detecting users’ RFID tags, passive indoor location within zones is possible [61], recording only participants’ presence within detection range. Real time visualizations inform participants of others’ interests, while allowing to locate them (see Fig. 6). The system has been able to track up to a hundred participants in real events.

Second, the smartlivinglab premises are studied as a day-to-day research office context. Bluetooth beacons on users’ wrists track occupants in the building to know how they behave in space [65]. The granularity of the tracking is macroscopic since the technology only allows to detect in which room a specific user is at a given time. The aim is deliberately to extract patterns from data traces to understand who are the occupants and their needs, and to assess the space in terms of occupancy, in order to inform the design process and build better work places, which corresponds to the strong definition of HBI. Further research is needed to refine the granularity of the tracking towards capturing more minute interactions and enlarging the context towards true “in the wild” studies.

Building Data Visualizations. For automation purposes, buildings produce great amounts of data which represent their dynamic non-linear behaviour. In addition to the obvious building automation modality, the exploitation of such data has the potential to nurture research on modalities as different as the external conditions, the built structure, or user implication. Visualizations representing such data can be targeted towards user feedback, but may also represent value for experts trying to guarantee comfort levels and/or optimize energy consumption, who are in need of efficient tools to explore such datasets [49].

As an example, a user study involved experts using interactive visualizations to explore real building data that they did not previously know. Within a limited time span (up to 20 min), they were able to spot various behaviours specific to the building which were not possible to infer from raw data, such as spotting malfunction and identifying non-trivial relations between measured variables [9] (see Fig. 7). A similar example addresses energy portfolio analysis [13]. In these cases, the conjunction of interactive technologies in a building design or industry context qualifies for the strong definition of HBI.

Fig. 7.
figure 7

Interactive visualization of building data

Full size image

7 Discussion

From the exposition of the dimensions and modalities of the field of HBI, together with the presentation of numerous research works and example projects, some considerations emerge that in our view should contribute to establishing stronger foundations for this approach:

 

Unification. :

The survey presented here encompasses several research fields from HCI, ubiquitous computing, applied psychology, HVAC engineering, building physics, etc. What has appeared is that such different fields do not share the same methodologies, and despite the great quality of the work referenced here, some research projects that are sound in one field may present obvious flaws in another, and vice-versa. So it is clear that some exchange, confrontation and unification in the methodologies is needed to bootstrap a process of cross-fertilization among disciplines. To a certain extent, the formulation of the field itself and the possibility for research projects to claim to be part of HBI is an initial step [5].

Consolidation. :

As a corollary of the previous topic, research in the field of HBI needs to be consolidated. New metrics and methodologies need to be defined and tested, that specifically apply to the challenges found in building settings. Interdisciplinary projects need to be formulated and conducted with success. Funding agencies need to be convinced to appreciate the different quality of this field, also with regard to high infrastructure costs.

Recognition. :

In the field of building engineering, the mindset “You can come with your interface when the technical job is done” is still very entrenched and the sector is, for liability reasons, very conservative. The need for HCI often appears either superfluous or straightforward to many stakeholders. Establishing the field of HBI requires an effort to convince these stakeholders that it is able to contribute significantly to current problems involving users, as faced in the building industry. The two previous considerations will contribute to increase recognition, but because of conservatism, researchers should take an active role in disseminating their results to practice.

Further Work. :

From the mapping of dimensions and modalities, some research topics have appeared very popular, such as energy-related behaviour change, while others suffer from an apparent lack of interest. For instance, the relationship between private sphere and buildings is not very articulated, even in the advent of IoT. While user comfort and trust in the presence of dynamic intelligent automation, as well as setting the right limit for user involvement in contrast to automation are topics that see burgeoning interest. Related to this question, the notion of designing for user involvement in the dynamic behaviour of buildings is still largely uncovered.

 

8 Conclusion

The present articles presents a series of arguments to consider Human-Building Interaction as a promising research topic in need of appropriation and consolidation by the HCI community. This domain aims at studying human interactions with and within buildings, as well as the development and evaluation of interactive technologies to encourage user-building interactions. The fundamental characteristics of this context, namely user immersion in the “machine” and extensive space and time scales, are believed to be more than anecdotal and represent a challenge for HCI which justifies an appropriate and specific methodology.

Through the proposition of a taxonomy of HBI dimensions, an enumeration of the different modalities to influence users in this context, blended with a comprehensive survey of cross-disciplinary research addressing the topic, the paper raises research themes relevant for the HCI community. These include energy efficiency, comfort and user awareness of building dynamics improvements.

Although striving for an exhaustive and systematic approach, it is probable that our contribution overlooked some dimensions or research efforts. We thus rely on the HCI community to contribute to this initial proposal. We believe that exposing the concept of HBI not only allows to raise the interest of HCI community members in the specifics of such a context, but also allows to influence building engineering fields with a new approach and its corresponding methods.

As a conclusion, a fundamental transition in the way we interact with buildings is probable; and probably towards styles of interactions that are central research themes of the HCI community. To let users and society as a whole profit from it, it is important from this research community to assert its expertise for proposing human-centered—instead of technology-centered—solutions.