Introduction

The design of physical workspace in the offices and laboratories where knowledge workers work is one of the major factors that affect their health, performance, and job satisfaction [1, 2]. For this reason, such design has changed to suit our work styles since the first modern office was built more than a century ago, taking into account a variety of perspectives, including architecture, ergonomics, physiology, and psychology. One of the most significant changes in our work styles has been the introduction of computers. Computers have significantly improved the productivity of knowledge work and are now an indispensable part of work. However, computers are often considered objects independent of the workspace, and the issue of human-computer interaction (HCI) has been discussed separately from that of workspace design.

By contrast, researchers have recently begun to apply the concept of Ubicomp [3], in which computers are integrated into the environment and the environment behaves interactively with people. According to this idea, computers and their surrounding workspaces are inseparable, breaking the conventional assumption that the physical aspect of workspaces is rigid and difficult to change. More specifically, there have been various efforts not only to develop ubiquitous sensing technologies but also to computerize the components of the workspace themselves (e.g., walls, floor, tables, chairs, etc.)—sometimes by changing their shapes or forms—so that they can influence workers. The purposes of these components are also diverse, covering a variety of perspectives such as productivity, health, communication support, privacy, etc. However, we still do not have a systematic understanding of the objectives of these components and their impact on workers. In order for researchers and practitioners of HCI and space design to co-design successful interactions between people and workspaces, it is crucial to comprehend the multifaceted roles played by each component.

In addition, with the development of technologies related to extended reality (XR,Footnote 1 including virtual reality: VR, mixed reality: MR, and augmented reality: AR), workspaces are expanding into virtual spaces. There are active attempts to enrich our physical workspace by adding digital space, or to replace it entirely with virtual space. In particular, the recent COVID-19 pandemic has further accelerated these research activities. Today, some workers still work remotely, and this trend may be irreversible [4]. Therefore, at this key turning point, it would be beneficial to establish a framework that organizes previous efforts by considering both the physical and virtual approaches as well as how they relate to each other. Such a framework should serve as a useful compass for envisioning the future of workspaces.

In this paper, we propose a framework, named Human-Workspace Interaction (HWI), as a large body of research efforts on workspaces where each component physically/virtually interacts with workers to assist them, and attempt to comprehensively categorize prior works within this framework. Specifically, we provide a taxonomy focusing on types of components, the research approaches, interaction targets and objectives of HWI and elucidate the advantages and limitations of HWI based on these criteria (note that we do not provide a systematic literature review, but provide a comprehensive overview of the taxonomy by presenting representative prior research on each category). Then, we summarize the challenges that must be faced to further advance the development of HWI as well as prospects, taking into account the societal changes caused by the COVID-19 pandemic. We hope this paper will lead to a wider recognition and better understanding of this field and promote further developments for researchers and practitioners in spatial and furniture design in combination with HCI.

Human-Workspace Interaction

Definition and scope

HWI involves the research field that considers various ways to support knowledge workers and their activities by making the workspace interactive. The term workspace here refers to the physical/virtual space where knowledge workers work, such as offices and laboratories (note: the term workspace is sometimes referred to as the on-screen working area of the software where tasks are performed, e.g., workspace awareness [5, 6], but our definition here is different from this). The entity that interacts with the worker involves not only the entire space itself, but also every component of the space that surrounds the workers and their activities (i.e., interactions between the workers or with computer/non-computer objects). These components, called workspace components in this paper, include chairs, tables, walls, floors, lighting, and the environment’s air.

Fig. 1
figure 1

Conceptual difference between conventional HCI and HWI

Full size image

Figure 1 shows the conceptual difference of HWI from conventional HCI. Conventional HCI work has literally considered interactions with so-called computers (e.g., using mouse and keyboard as input, monitors and speakers as output) separately from the working environment, but HWI, similar to the UbiComp concept [3], addresses interactions with computers that are integrated with the working environment. As Fig. 1 shows, HWI can be considered a superset of HCI, but the overlapping part is excluded from the scope of this paper because the issues have been already well discussed.

The specific roles played through HWI also overlap with those of the existing (non-interactive) workspaces and in HCI field, such as to improve productivity, comfort, health, privacy. We thus assume that this framework would be useful not only for HCI researchers but also for practitioners such as space designers and product designers of furniture.

HWI essentially involves interactions with workers, which can be divided into the factors of input (the workspace’s sensing of the user) and output (the workspace’s responding to the user). Both are critical research topics of HWI, but the main interest of this paper lies in output, since ubiquitous input technologies are frequently discussed and reviewed (e.g., [7,8,9,10]), while output is rarely overviewed. Consequently, this paper does not cover approaches using only sensing technologies.

Another notable development is that workspace components have become useful in virtual spaces as well. Since we assume that we will eventually move toward a hybrid workspace in which physical and virtual aspects coexist in the near future, this paper introduces approaches to both physical and virtual workspaces involving HWI and discusses the relationship between them.

Related frameworks

In HCI, the idea of a space that interacts with people has been considered for a long time. As a representative example, Weiser proposed Ubicomp [3], which is the idea that in the future computers will be integrated invisibly into the environment (in this context, the term interactive workspace has sometimes been used in some literature e.g., [11, 12], but it contained a more limited meaning focusing on the use of large displays together with mobile devices). Ambient Intelligence [13] is a similar concept that was introduced later. Proxemics Interactions [14] is a concept that extends Ubicomp by focusing on the proximity between entities (i.e., people and computer/non-computer objects). Based on these concepts, the sensor, display, communication, and actuation technologies required for intelligent spaces have been widely studied. Advances in research on shape-changing interfaces [15] would also be related to this. The literature reviewed in this paper is solidly based on these concepts and technologies, although the works addressed do not explicitly target workspaces.

From the aspect of architecture, the introduction of interactive spaces has also been considered. Schnadelbach et al. broadly defined Adaptive Architecture as buildings designed to adapt to their environment and occupants (automatically or through human intervention) and categorized the elements and methods of the adaptation [16]. Takeuchi reviewed research efforts that aimed to digitize architectural spaces in the context of HCI, and they argued for the adoption of Habitable User Interface technology [17]. More recently, the term Human-Building Interaction (HBI) has been used frequently, covering a wide span of research on the future of human experiences with, and within, built environments [18]. These concepts overlap the scope of HWI, but our interest is more specific to indoor spaces where people work, rather than entire buildings. In addition, HBI is concerned only with physical space, whereas HWI also cover virtual space.

The idea of forming a virtual workspace and sharing it with others has been considered since the early period of virtual reality research [19]. Recently, there has been growing interest in seated XR workspaces for users wearing HMDs, especially with the increasing demand for remote work (work from home) resulting from the pandemic [4, 20]. However, most studies have focused on the interactions with contents in virtual spaces, and there has been little work on a framework to comprehensively encompass the user’s workspace (i.e., both the virtual working environment and the physical seating environment).

Taxonomy of human-workspace interaction

To provide a structured understanding of the extensive research on HWI, we first collected the papers by keyword searching (in ACM digital library, IEEE Xplore, and Google Scholar) in HCI- or VR-related conferences (e.g., CHI, UIST, ISS, TEI, DIS, IEEEVR) and journals within the last 20 years. The search included keywords: “interactive”, “adaptive”, “robotic”, “virtual”, “workspace”, “workplace”, “office”, and “furniture”. We then manually excluded those that were considered out of scope (e.g., those in which “workspace” is used with a different connotation, or those that could be considered traditional HCI issues). Note that we do not provide a systematic literature review, because it was quite difficult to systematically extract the literature of interest due to the lack of a solid existing framework and the considerable variety of vocabulary used in the literature.

Based on discussions in our authors after reviewing the relevant literature while referring to previous taxonomies in different domains (i.e., shape-changing interface [15] and augmented reality and robotics [21]), we introduce the taxonomy from four perspectives: type of workspace component, research approach, interaction target, and objective. Table 1 lists the extracted HWI literature and their features described from the four perspectives. The following describes each of the perspectives more in detail.

Table 1 HWI literature overview
Full size table

Type of workspace component

HWI includes a wide variety of workspace components such as desks, chairs, partitions, walls, and entire workspaces. The conventional personal computer itself and its peripherals are also included in the workspace components, and this paper addresses issues related to their spatial arrangement, but not their own design. These types of workspace component were included in our taxonomy because observing them will help researchers and practitioners of spatial design and furniture product design to understand the applicability of each element.

Research approach

We also considered organizing the technical/conceptual approaches of HWI research to organize how each work represents its research contributions in the interaction between the workspace component(s) and the worker. Accordingly, we have identified six key characteristics of the research approach based on our review of the relevant literature:

  1. 1.

    Projection (PRJ): Workspace component that employs projected images onto its surface [22,23,24, 41, 42, 57, 66].

  2. 2.

    Employing optical displays (OD): Workspace component that comprises or incorporates optical displays (e.g., flat monitors [38, 39, 56] and LED clusters [25] to present information, or LCD films [27, 28, 61] to change transparency).

  3. 3.

    Shape change (SC): Workspace component that changes its own shape (e.g., by means of joint manipulation [41, 42, 56], pneumatic control [40], or shape memory alloys [26]).

  4. 4.

    Position/Posture change (PPC): Workspace component that changes its position or posture physically (e.g., using wheel robots [38, 39, 43, 47, 57, 58, 62], robotic arms [46, 48], pneumatic control [55]) or virtually [29, 30, 32, 36, 37].

  5. 5.

    Virtualization (VTL): Workspace component that has conventionally been physical and is achieved virtually through XR technologies (e.g., by wearing an AR/VR headset [29,30,31,32,33,34,35,36,37, 49, 51, 60] or a headphone [64]).

  6. 6.

    Democratization of design (DD): Workspace or its components that allow workers themselves to design and/or prototype (e.g., by providing design tools [23, 51, 67, 68, 70, 72] or modularizing the components [69, 71]).

This identification may be useful for designers to understand the possibilities of how to implement interactive components, and for researchers to learn about unexplored approaches. However, we acknowledge that this is a formative and exploratory categorization and it could be extended in the future to address additional components previously not covered.

Target

To understand how each workspace component supports workers, we considered categorizing interaction target (i.e., what workers interact with). Based on the rationale of Sundstrom’s focus on the three different perspectives as analysis units of the working environment (i.e., individual, interpersonal relationships, and organization) in environmental psychology [2], we have classified them into three categories, person, inter-person, and environment as shown in Fig. 2.

Fig. 2
figure 2

Three types of interaction targets categorized by unit of interest. A cube

figure a
represents a workspace component

Full size image

Specifically, HWI targeting person involve components that directly interact with each worker Fig. 2a). For example, research efforts to improve the personal work environment correspond to this category. Next, HWI targeting inter-person include components that affect the relationships among multiple workers (Fig. 2b). Examples include approaches that enable workspaces to facilitate smooth communication and regulate interactivity between people. Finally, in the case of HWI targeting environment, the components influence the entire working environment (Fig. 2c), such as improvements in thermal or acoustic comfort. We have classified the relevant literature into these three categories by focusing on the most primitive targets of each work (some works e.g., [58, 64] may affect the entire environment, but are categorized as inter-person by focusing on the primitive interactions they achieve).

Objective

In conjunction with the interaction target, it is essential to comprehend the interaction’s objective, i.e., what each workspace component interacts with the worker for. However, each workspace component may have multiple objectives at the same time (e.g., a table may have the objective of securing a physical working surface, its size and shape may contribute to the interpersonal formation, and displaying content on the tabletop may have the objective of interacting with information).

To organize this, we have carefully examined the interactions between people and each workspace component in prior work. We then classified the objectives into seven major categories, \({\textcircled {1}}\)Visual, \({\textcircled {2}}\)Physical, \({\textcircled {3}}\) Postural, \({\textcircled {4}}\) Social, \({\textcircled {5}}\) Interactivity, \({\textcircled {6}}\) Atmosphere, and \({\textcircled {7}}\) Design, by relating them to fundamental roles that a typical workspace has, as shown in Fig. 3.

Fig. 3
figure 3

Illustration of each HWI objective with a typical workspace; \({\textcircled {1}}\) The wall (as a workspace component) provides visual information to the workers; \({\textcircled {2}}\) The table surface provides workers with a physical support for their work; \({\textcircled {3}}\) The chair supports the worker to sit appropriately; \({\textcircled {4}}\) The table size functions to maintain an appropriate social distance between the workers; \({\textcircled {5}}\) The partitions serve to regulate interactivity between the workers; \({\textcircled {6}}\) The indoor environment is maintained by several factors (e.g., lighting, air conditioning); \({\textcircled {7}}\) The look and layout of the workspace is affected by the designer’s design

Full size image

Table 2 shows the detailed definition of the seven objectives, which are also divided into the three interaction targets (described in Sect. 2.3.3) by focusing on the primitive interaction each objective achieves. We believe this framework based on a typical workspace offers a reasonable overview of HWI, but it is still formative and may be further modified with future societal changes and/or increased diversity of work styles. In the following section, we review the prior research efforts according to these objectives in detail.

Table 2 Categorization of targets and objectives of HWI
Full size table

Prior research efforts

In this section, we review these prior research efforts based on the seven objectives of HWI described in the previous section.

Visual: representing visual contents

Some workspace components behave as screen surfaces or ambient displays to visually present the information content according to the worker’s states. In addition, XR workspaces can place content display surfaces at arbitrary mid-air locations. Since most of these approaches can be assumed to overlap with conventional HCI issues, we introduce only an overview of these approaches below.

Room-scale displays: One advantage of utilizing workspace components such as walls, floors, and tabletops as content display surfaces is that they can constitute large, room-scale display environments. Early attempts to apply the concept of Ubicomp to the workspace often involved utilizing available surfaces in the space as resources to replace desktop monitors. Among them, wall-sized displays and tabletop displays have been major research topics in HCI, and many devices and corresponding interaction techniques have been explored (e.g., [12, 76, 77]).

As a related approach, many works have adopted the idea called spatially augmented reality (SAR [78]), which superimposes information directly within the physical workspace by projecting it onto surfaces such as walls and floors [79,80,81]. For example, Rekimoto et al. proposed Augmented Surfaces [79], which allow users to use projection-enabled tables and walls as spatially continuous extensions of their laptop computers. As approaches specific to meeting spaces, some studies have examined systems that form an omnidirectional display with multiple walls so that each participant can share and edit content equally [22], while others have examined systems that can project contents according to the physical layout in the room [23]. In addition, early research has also considered the idea of seamlessly connecting remote spaces using SAR approaches [19]. Life-sized projection of a remote conversation partner has been shown to improve subjective presence and conversation efficiency [24]. The major advantage of the SAR approach is that the system can add information directly to any location in the workspace without requiring the user to wear any device. Nevertheless, there are some limitations, such as the large setup required for using projectors, occlusion problems, and room brightness.

Ambient displays: Another advantage of applying workspace components as display surfaces for visual information is that they become ambient displays [82] that can present information in a subtle manner without spoiling the design of the space itself. Some studies have attempted to control the visibility of windows and louvers locally, allowing them to become displays by themselves, or to use the shadows created by sunlight to present information [26,27,28]. Other approaches have considered wall-sized ambient displays that implicitly indicate the atmosphere of a particular community or remote location [25]. These attempts may not contribute directly to the efficiency of the worker’s performance, but they can allow the worker to obtain information peripherally without interfering with his or her work.

XR workspaces using HMDs: VR/AR HMDs have been attracting a great amount of attention in recent years as an alternative to physical monitors for knowledge work [4, 20]. Such a function of HMDs has been proposed since their early stages (e.g., [83, 84]), but with the recent development of lightweight, high-resolution, and wide-viewing angle HMDs, it is moving into the practical phase, and many commercial virtual knowledge-working systems are being released (e.g., Spatial,Footnote 2 Mozilla Hubs,Footnote 3 Microsoft Mesh,Footnote 4 etc.).

In XR workspaces, unlike in physical space, content can be placed at arbitrary locations in the air. For this reason, many studies have examined how to place planar virtual contents in 3D space [34,35,36,37]. Ens et al. schematized possible ways of arranging planar content in MR workspaces based on the type of content, interaction method, and so on [34]. More recently, one study has developed a design toolkit that allows creators to arrange UI elements based on ergonomic factors in an XR environment [72]. Similarly for AR/MR workspaces using optical see-through HMDs, several studies have explored the spatial arrangement of content in accordance with the physical space [29, 32, 85]. Focusing more on the advantage of the HMD’s mobility, an increasing number of research attempts have examined the spatial arrangement of content and its interaction techniques specifically during walking [29, 30, 86] or transport seating [87,88,89,90].

Compared to SAR approaches that display information in physical space, HMD-based approaches are superior in terms of mobility and privacy control [31]. In fact, the use of VR HMD has been reported to be superior to a physical office in terms of concentration [33]. Another report has also shown that an AR virtual screen is even feasible for performing serious productivity work [31]. However, there are still many limitations such as limited resolution and field of view [31], as well as physical and visual fatigue and simulator sickness due to continuous use of HMDs [91, 92].

Physical: providing physical working surfaces

Many types of knowledge work require physical surfaces, such as tabletops and whiteboards. These surfaces mainly serve as supporting surfaces for hand-related tasks, mounting bases for devices, and touch interaction surfaces. Recently, there has been a lot of research on adaptively helping workers by changing the presence/absence and shape of the physical surfaces according to their position, orientation, or tasks. In the following, we discuss these efforts in both real and virtual workspaces.

Moving/Transforming tabletops: One approach to adaptively providing physical surfaces to workers is to control the presence/absence and horizontal/vertical position of the physical surfaces. MIT’s Changing Spaces Group has demonstrated in a video the concept of tables approaching the worker or descending from the ceiling depending on the workers’ needs [74]. As a more specific consideration, Takashima et al. have explored the approaching and leaving motions of a table, and they have shown that during these motions, displaying a predictive animation on a tabletop display is effective [38]. In addition, research has also shown the usability of adaptively changing the horizontal position of the table surface by considering the user’s kinematics to reach the content [39].

Several attempts have also explored tables with tabletop configurations other than the conventional horizontal plane. The project TRANSFORM [73, 93] provides a shape-changing tabletop surface by controlling a grid of mechanical actuators, enabling interactions such as supporting, lifting, and carrying objects on the table. Similar to this, LiftTiles introduces a room-scale concept [40] using a grid of linear actuating modules with pneumatic control to produce physical surfaces such as tables and chairs at any location (whereas this does not specifically focus on workspaces). Grønbæk et al. proposed shape-changing furniture that transitions between a digital table and a wall in steps depending on the socio-spatial situation [41] or between multiple configurations of the tabletop inspired by the kirigami mechanism [42].

Those ambitious efforts described above will make our workspace more flexible and attractive. However, they have not yet established the methodologies for how to adaptively support workers or how to evaluate the work performance. For the introduction of the system in the real world, there are also challenges in terms of production cost, as well as safety and building user trust in the system’s operation. Further exploration is needed for these challenges.

Providing haptic feedback in XR: In XR workspaces using HMDs, the lack of haptic (kinesthetic) feedback along the working surface is one of the major limitations, since mid-air hand interaction is problematic due to inaccuracy in delicate tasks as well as fatigue [94, 95]. To address these issues, many efforts have targeted the ability to provide haptic feedback to the user corresponding to the features of the virtual space. For example, some studies proposed providing haptic feedback in VR by appropriating nearby physical objects in reality [90, 96]. In addition, recent commercial HMDs (e.g., Meta Quest 2Footnote 5) have the ability to display tracked real-world physical surfaces and the user’s hands in a virtual environment (VE), allowing users to work with passive haptic feedback on the physical surfaces. However, these implementations depend on the location of the physical objects or surfaces in reality, which undermines the advantage of VR workspaces that can be arbitrarily designed. In response, there is a wide range of approaches to overcome this limitation, such as wearing an actuated device to replicate the haptic sensation around the body (e.g., [97, 98]), adaptively placing physical props around the user (e.g., [99,100,101]), and exploiting visuo-haptic interaction to manipulate the perceived position of physical surfaces (e.g., [102, 103]). A detailed introduction to these technologies is beyond the scope of this paper, so the reader is referred to the relevant review papers (e.g., [104,105,106,107]).

Postural: supporting physical postures

Knowledge workers spend most of their time at work sitting. Since working posture can affect not only short-term task performance [108] and meeting time [109] but also long-term well-being and health [110,111,112], it has been widely studied from both ergonomic and physiological perspectives. Recently, there has been an increasing body of research on HWI that interactively support the workers’ posture during work, and these works are described below according to three different approaches.

Providing physical seats: One basic approach is to provide people with a physical place to sit as needed. Some concept prototypes have demonstrated the usefulness of this by showing that the robotic chair is automatically moved to the worker as needed for their tasks [74] and then returned to its original place after use [75]. With LiftTiles [40] described earlier, it will also be possible to make chairs appear anywhere on the floor. Related to this, Sirkin et al. have explored Mechanical Ottoman as a device that interacts with the user by approaching them to provide a footrest or to let them lower their feet [43]. These systems’ active interventions can be truly beneficial when combined with high-level context-aware techniques; here, considering the balance between obtrusiveness and subtlety is an essential issue [45].

Correcting/Guiding posture: A more advanced posture-related interaction is to correct a worker’s inappropriate posture. It has been reported that inappropriate posture causes musculoskeletal disorders and various associated adverse health effects [113, 114], and thus various designs of office chairs have been examined in the fields of physiology and ergonomics [115].

From the HCI perspective, there have also been various attempts to facilitate workers in achieving correct posture. Among them, one typical method is to notify the user when a bad posture is detected. There have been several notification methods, including on-screen notification [45, 116], vibrotactile feedback from the seat [44, 45], and implicit representation by a shape-changing agent [45, 117, 118]. The study by Haller et al. comparing these three approaches [45] revealed a dilemma: The more effective methods for posture correction are also more likely to interfere with the user’s task.

Another approach to correcting posture is to let the system guide the worker’s posture. Breazeal et al. introduced a method that dynamically changes the height and orientation of the desktop monitor that the user looks at, and they found that this method not only manipulates the worker’s posture but also increases persistence in cognitive tasks and changes subjective comfort [46]. Bailly et al. also mentioned the movement of monitors with consideration of ergonomics in Living Desktop [47], a concept in which devices on a desktop move by themselves. As an extension of these attempts, Shin et al. proposed a method to change a worker’s posture unobtrusively by moving the monitor at a speed unnoticed by the user, showing that this leads to an increase in non-disruptive quick posture correction and a decrease in the duration of unbalanced sitting [48]. This idea has also been applied by them to VR workspaces; they have explored a posture correction technique that slowly changes the position and orientation of a planar virtual screen in front of the user [49]. In a related approach, McGill et al. proposed a method for implicitly manipulating a virtual screen based on the worker’s head orientation in an egocentrically oriented virtual screen workspace [36]. They found that this technique minimizes neck fatigue and discomfort while providing access to a wider screen space.

Some studies have attempted to help workers customize the ergonomic details of their workspace. ActiveErgo [50] is a workstation system that automatically adjusts the height and angle of chairs, desks, and monitors based on the skeletal information of the seated person captured by the depth sensor. Body2Desk [51] is a VR application that allows workers to interactively design their own ergonomically appropriate desk configurations to support the fabrication of customized desks for each worker. These efforts will become increasingly essential in the future as the workforce becomes more diverse.

Reducing/Breaking up prolonged sitting: Recent reports indicate that prolonged sitting is associated with a number of diseases and even all-cause mortality [110,111,112]. To mitigate this problem, workers are recommended to reduce both the consecutive sedentary time per session and the total sedentary time [112, 119]. Thus, many attempts have recently focused on reducing or breaking up sitting time. In ergonomics and physiology, height-adjustable (sit-stand) desks and other exercise-integrated workstations have long been considered [120,121,122]. These have been reported to significantly reduce sitting time [123,124,125], but the limitation is that they require conscious use by the user.

There are several HWI-related approaches to this problem (review papers about the digital intervention tools are available [126, 127]). Probst et al. proposed a method to facilitate physical activity by preparing both sitting and standing workstations and moving between them according to the task [52]. Their background study provided guidelines for software design to enable seamless switching between different postures when working in such distributed environments. Similarly, Damen et al. introduced unusual shaped furniture to stimulate workers to avoid static postures in group meetings [128]. As a more proactive approach, Lee et al. proposed a method that automatically changes the height of sit-stand desks, and their experiments found that the best timing for changing the height is when switching tasks [53]. Several works have examined techniques featuring actuation of the chair, such as horse-riding motion [129], pressure force and height change [54], and slow tilting motion [55] given by the seat surface. In particular, Fujita et al. revealed that a slow inclination of the seat can promote standing without losing the worker’s objective task performance [55]. All of these approaches seem promising for solving prolonged sitting, but long-term follow-up studies of workers are still needed to confirm their effectiveness in changing habitual behavior.

Social: supporting socio-spatial interactions

Mainly in social psychology, researchers have long sought to model socio-spatial relationships of people in everyday social interactions (e.g., [130,131,132,133]). Recent advances in ethnographic analysis have also revealed not only interpersonal formations but also more extensive formation patterns associated with surrounding furniture and display devices [134, 135]. Accordingly, many studies have explored how to change the shape and form of workspace components according to the formation or how to induce formation changes by changing the workspace. In the following, we describe such studies in terms of real and virtual environments.

Supporting in-person interaction: A representative approach to affecting in-person interactions is to employ the shape-changing of tables. For example, TransformTable [56] attempts to transform the tabletop shape between round, square, and rectangular, based on the psychological findings that tabletop shape affects the spatial arrangements of the people around it [136, 137]. Their later work further introduced table-approaching and connecting/separating movements, which confirmed that these movements affect the user’s spatial behavior and workspace awareness [38]. Grønbæk et al. [41] extended proxemics theory [130] to shape-changing furniture; they developed an interactive surface that can transition between tabletop and wall display to allow people to adjust their proxemic arrangements. Their follow-up study developed a digital table with a foldable mechanism to support more diverse formations with four people [42]. These shape-changing tables have great potential to support a variety of socio-spatial formations. However, none of them has conducted enough empirical studies to validate their usefulness in practical situations.

Regarding such work on walls, Takashima et al. have derived possible formations of wall displays in interaction with people, and they implemented shape-shifting wall displays that can shift between these forms [57]. Recently, there have also been attempts to facilitate the social distancing of conversants through robotic partitions [138]. Social distancing will be an essential element in considering interpersonal formations in after-corona society.

Although not intended for workspaces, some studies have also examined the use of chairs to guide people’s formations or socio-spatial relationships, mainly in public spaces. Examples include a method for guiding the direction of sitting by rotating the seat surface of a sloped chair [139] and a method for triggering accidental communication among seated people by changing the undulations of a bench [140, 141]. These methods might also be applicable to workspaces.

Supporting remote interaction: Although there have been extensive studies on interpersonal interactions with remote users to enhance social telepresence, few consider the impact of the workspace. Nakanishi et al., as one of the few studies on the physical workspace, attempted to install a partition that partially blocks the local user’s view of the physical space to maintain physical consistency in mirror-type videoconferencing. They also implemented a robotic table and bench to provide visual and haptic feedback from the remote partner, and they showed these settings can improve the subjective feeling of togetherness [59].

The nature of socio-spatial interaction in immersive VR environments is not yet fully understood. However, many studies have reported that the social behaviors observed in virtual environments are somehow consistent with those in physical space (e.g., [142,143,144]). Williamson et al. analyzed data collected by conducting an academic workshop in VR and found that the size of the space affected group formation, shared attention, and personal space in the same way as in physical space, while non-physically constrained interactions such as flying could be a new dimension of personal space [60]. Based on this finding, virtual workspaces may benefit from being non-physical, and it may be worthwhile to further explore the design of virtual workspaces for more efficient and smoother remote interactions.

Interactivity: regulating interactivity with surroundings

Controlling interactivity with the surrounding environment is a critical issue for efficient knowledge work. It is known that interruptions from the outside (e.g., being talked to by someone) during a certain task can significantly reduce task performance [145], while it is also necessary in our social activities to maintain a situational awareness of the surrounding environment. In addition, workers may want to ensure visual and auditory privacy regarding task content and activities. Existing office workspaces sometimes fail to meet these requirements, and this has been particularly problematic in recent years in work-from-home environments [4]. To deal with this, many attempts have been made to adaptively adjust interactivity with the surroundings. Here, we review these attempts for both physical and virtual workspaces.

Physically/Digitally controlled interactivity: One straightforward approach is to change the visibility of walls and windows. There are several examples of considering visibility control at different scales, such as the partitions between desks [61], windows and indoor partitions [27], and the exterior walls of buildings [28] using glass panels with controllable transparency. These are promising technologies that enable transitions between walls and windows, although they require a certain cost for installation.

In addition, several methods have explored controlling physical openness. For example, Coelho et al. have introduced curtains with louvers that can be locally opened and closed [26], which can change the passage of sound and ventilation in addition to visibility. Another example is the use of wheeled robotic partitions [58, 62, 138], which are capable of creating walls at arbitrary locations, thus enabling a system to control the visibility and accessibility of people. In particular, a user study by Lee et al. [62] showed that the motion of the robotic partition affects people’s approachability and that this motion should be designed while taking into account whether the person is inside or outside the partition.

Virtually controlled interactivity: By contrast, some attempts have explored techniques to virtually reconstruct interactivity through individual workers wearing AR/VR headsets, without using physical props. A study reported that superimposing a virtual partition around the worker’s desk using an AR headset can reduce visual distraction and improve the experience in shared workspaces [63]. Similarly, there is an effort to enhance concentration by blurring the background of the physical workspace using a video-see-through HMD [146]. These techniques are superior in that they do not require any changes to the physical space and are easily user-customizable. However, they can only adjust interactivity unilaterally from the HMD user’s side, not from the side of the people around them. In addition, only simple designs for these virtual visibility controls have been explored so far, and there is room for further investigation along with the arrangement of virtual content and screens (described in Sect. 3.1). Another effort proposed an MR working environment that virtually enables asynchronous physical interactions, by capturing co-located or remote physical events and their causality [147]. Such a technique may allow the regulation of interactivity across time and space.

Auditory interactivity with the surroundings is also an important issue, yet its control is quite difficult. Ordinary floor partitions do not provide much sound insulation [148], and thus sound masking [149] or active noise control systems [150] have been investigated in the field of acoustics. At the same time, Takeuchi proposed a method that allows users to hear only the sound inside a specific “weightless wall” using noise-canceling headphones [64]. This method enables strict interactivity control, while requiring all users in the space to wear headphones. However, there remains a lack of effective and feasible solutions, which needs to be explored in the future.

Atmosphere: improving environmental quality

Physiology and environmental psychology have long considered the importance of maintaining the indoor environment quality (including thermal, air, acoustic, and visual quality, referred to as IEQ) within the workspace, and IEQ has been shown to influence worker performance, well-being, health, and productivity [151,152,153]. Designing spaces for better IEQ has mainly been considered in the field of architecture, which is not reviewed in detail here (AlHorr et al. provide a comprehensive review for this [153]). In the following, we describe approaches to improving environmental quality through the use of interactive workspace components.

Improving visual comfort: Visual comfort (including lighting and views) in the workspace is recognized as important in architecture [1, 2], and it influences worker productivity and satisfaction [154]. Regarding lighting, it is recommended that every room in a workspace have a window with an outside view from the perspective of space design [1]. Windows are preferred by workers and are beneficial in terms of reducing discomfort [155, 156]. For this reason, for example, commercial products such as LED lighting that artificially reproduce sunlight are also available.Footnote 6

Other methods attempted to foster visual comfort by reproducing visual elements of the entire space. Naz et al. [65] introduced a system that simulates design attributes of brightness, color, and texture in space by projection onto a CAVE (a six-sided projected immersive display), and they showed that it could replicate a real environment. Mediated Atmospheres [66] is a system that adaptively creates an atmosphere by presenting multimodal stimuli such as lighting, wall projections, and sound based on the occupant’s biometric information, and it was shown to affect the occupant’s perception as well as physiological responses.

Meanwhile, XR workspaces might make it easier to foster workers’ visual comfort than in reality, as they essentially block out the outside world’s vision and allow arbitrary VEs to be presented. In fact, the experiment conducted by Ruivimova et al. [33] presented participants with office and beach-like environments as VR workspaces, showing that these environments performed better in terms of flow [157] than a non-VR open office environment. Nevertheless, there has been little exploration of what aspects of a VE contribute to better visual comfort and performance, and thus this may be worth pursuing in the future.

Localizing environmental properties: Another approach is to make environmental properties locally modifiable in an unpartitioned space such as an open-plan office, which has conventionally been difficult. For ventilation and thermal environments, several studies have examined systems that allow individual workers to adjust the environmental properties, with the results showing that such systems increase occupant satisfaction (e.g., [158]). There have also been several studies on the lighting environment, including the use of projectors to brighten specific areas in space [159] and the use of LCD shutter glasses by occupants to time-multiplex their lighting environment [160]. As for the sound environment, the weightless wall [64] mentioned above would be a relevant example.

Design: supporting workspace design

While workspace designs are typically updated over a long time span by space designers, several research approaches are enabling workers themselves to design and fabricate their customized workspaces more flexibly and in shorter cycles. While this approach slightly differs from the other categories of HWI described above, we consider it to be an aspect of HWI in that it interactively creates the workspace needed by the relevant worker.

One promising approach is to support workers in designing their own furniture. Examples include tools that allow users to design and fabricate printable chairs from 2D sketches [67] and methods to convert 3D models into fabricatable parts [68]. With the recent increase in 3D printable furniture (e.g., chairs [161]), it will become increasingly easier for users to fabricate their own furniture.

In the design phase of the entire workspace, various approaches to supporting designers are being considered. For example, although spatial layouts are conventionally designed manually, some studies have introduced design support systems that can generate or optimize design variations for 3D spatial layouts (e.g., [162]). Additionally, another study considered a spatial design system that employs projection mapping onto building blocks that can be tangibly arranged to create a layout [70]. Another unique approach is to enable 3D physical sketching at an actual scale by introducing a device that creates tubes with connectors from adhesive tape [69]. At the same time, with the increasing use of digitized information on 3D building models (BIM), it has become easier to visualize a space; moreover, VR simulator applications are often used during the designing phase (e.g., [51, 163]).

Another perspective is to support design by allowing workers to directly change the layout and state of the space by themselves. Pertender et al. [71] proposed a methodology to modularize furniture with cube-shaped blocks called Foxels, and they developed blocks with 24 kinds of functions. LiftTiles [40] can be regarded as a similar attempt, since it enables users to place props of desired height at desired locations. For such dynamically changing workspaces, SpaceState [23] can be used as an authoring tool to define the content and position of the presented information according to the space layout. Thus, workers are becoming more active in intervening in the workspace design, and this trend will continue to accelerate in the future, toward faster and freer workspace design.

Major challenges and future directions

Major challenges

In this paper, we introduced previous research efforts on HWI with diverse approaches and demonstrated their potential for several categories of objectives. However, there still remain challenges across these categories for the further development and practical deployment of HWI research. The following discussion can serve as a basis for the next research goals in the relevant research areas, including Quality of Experience (QoE)- and User Experience (UX)-related research.

Empirical knowledge: One major challenge is the lack of empirical knowledge. The majority of studies have only implemented prototypes or evaluated them through small-scale laboratory experiments, thus it is unclear how beneficial each component of HWI is expected to be when deployed. We believe that two difficulties underlie this. One is the difficulty of quantitatively measuring performance such as productivity and income due to the complexity of people’s interactions in actual workspaces. To deal with this, many research efforts have been made to understand the dynamics within workspaces [61, 134, 135], and the key to the advancement of HWI will be to improve this understanding and thus to create a relevant metrics framework. The other is the difficulty of measuring the impact of long time spans, such as users’ learning or building trust [164] in the system, or changes in users’ behavior or habits. Since long-term monitoring of a certain number of users is essential to overcome this, it will be necessary to establish automatic (i.e., unaided) methods of measuring user status and performance.

Conflicts between multiple components: A related challenge is the potentially conflicting behavior of HWI approaches. Since workers with different individual requirements coexist in a realistic workspace, the system will need to respond to these requirements simultaneously; however, the responses to them may conflict with each other. For example, an interactive tabletop has multiple objectives, including the provision of a physical surface and the creation of a social formation, but no configuration variation may simultaneously satisfy each requirement. Similarly, HWI with different interaction modalities may conflict with each other. For example, a user wearing a VR headset cannot interact with most of physical components. As diverse components will coexist in a single workspace in the future, establishing a standardized authoring system that can consider compatible combinations of HWI approaches will be needed.

Balance between subtlety and obtrusiveness: Yet another challenge with HWI interaction strategies is achieving a balance between subtlety and obtrusiveness. While fully automatic behavior can greatly reduce worker effort, it may also contain some erroneous behavior, which in turn can increase the effort. Similarly, the more the system forces the worker to perform a certain behavior (e.g., changing posture to a standing position), the more it might disrupt the worker’s workflow, thereby discouraging users from continuing to use the system that would otherwise be beneficial. Several studies have also pointed out the importance of such a balance [45, 48, 49, 55]; for example, Fujita et al. show that the inclination angle of the chair seat and its motion speed can affect the obtrusiveness perceived by the user during posture change, and suggest adjusting the angle and speed according to the working context (e.g., the importance of posture change). Ideally, it will be crucial to consider how to protect the worker’s present tasks as much as possible while simultaneously providing the targeted changes in the workspace, but achieving this will require further improvement in the accuracy of estimating the user state and/or working context.

Deployment issues: In addition, one challenge common to most of the physical components is reaching a large-scale implementation of the system. While shape change and autonomous movement of workspace components are clearly beneficial, the implementation cost is still fairly high, and it is not yet such a high priority in workplaces. From the standpoint of the facility manager of an office, it is difficult to make the decision of introducing a system without being able to justify its cost advantages. It will be necessary to evaluate each workspace component from this perspective in the future. In addition, there are certain safety concerns in the research of furniture with actuation, but most such research efforts do not mention safety. To address these concerns, it will be essential to develop guidelines for design and implementation that take into account the safety risks and predictability of the actuation.

Future directions

We believe that HWI will be further explored in the future to meet the demands of our society. We discuss possible directions of such explorations below.

More diverse work styles: First of all, it is impossible to neglect the changes in our needs caused by the recent COVID-19 pandemic. The pandemic is drastically changing and diversifying our work styles, and many believe that this is an irreversible change [4]. In particular, XR workspaces have the potential to greatly increase in popularity as work-from-home environments or mobile workspaces become more prevalent, but the design elements of such workspaces have not yet been fully established. For example, as we have already mentioned, the spatial arrangement of contents, the control of interactivity in the peripheral, and the visual comfort of the entire field of view still need to be considered from many perspectives. In addition to these factors, further research should be done on adaptive space generation to support communication in XR workspaces with multiple users. Since XR workspaces can be manipulated with a higher flexibility than physical spaces, they are expected to be extensively explored in the future. In addition, if we gather findings on the design elements of XR workspaces, it would be beneficial to develop tools to support effective workspace design (e.g., optimization and semi-automatic generation of XR workspaces).

Hybrid workspaces: Furthermore, there will be an increasing number of situations where on-site and remote workers work together, and thus the development of a hybrid environment of real and virtual environments is urgently needed. As this review shows, most HWI research is based on the assumption of use in either purely physical or purely virtual spaces. Therefore, investigating whether (or how much) the findings obtained in the physical space can be applied in the virtual space (and vice versa) might be a good starting point for developing a hybrid workspace. In addition, to achieve seamless interaction between physical and virtual workers, designing interfaces with the consideration of virtuality continuum [165] would be essential. For example, AR and VR headsets could be used together to align the visual information that both sides have access to, such as shared materials, nonverbal information about the other side, and the appearance of the workspace. At the same time, haptic cues consistent with visual cues also contribute to task performance in many cases, so this will be worth considering more in depth in the future.

Well-being considerations: Related to the widespread adoption of work from home, we need to more carefully consider well-being. For a physical workspace, there are building guidelines to ensure the occupants’ well-being from several perspectives (e.g., air, light, thermal comfort, materials, etc.Footnote 7), but such guidelines have not yet been considered for work-from-home environments or virtual workspaces. It has been shown that the continuation of remote work blurs the boundaries between work and life and also increases mental fatigue [166]. In addition, a significant decrease in the amount of physical activity and excessive sitting will be a natural problem. To improve the situation, in the future it will be necessary to examine HWI to ensure the users’ well-being from both mental and physical aspects, for example, by promoting rest and physical activity.

SDGs considerations: Looking at a longer term perspective, one key consideration would be the sustainable development goals (SDGs).Footnote 8 Towards the goals, one aspect that HWI can support is giving due consideration to diversity and inclusion. While each study establishes guidelines for worker interaction with workspace components, individual worker differences are not often mentioned (although ergonomic customizability has already been extensively studied). Given that workspace occupants are now becoming more diverse in terms of age, gender, origin, accessibility, etc., it is critical that workspaces remain customizable. Future design theories and guidelines for workspace components need to be formulated toward this ideal.

Conclusion

In this paper, we define a novel framework of Human-Workspace Interaction as a large body of research efforts on workspaces where physical and virtual components interact with knowledge workers to support them, and we provide their categorization based on component type, research approach, interaction target, and objectives to facilitate their structural understanding. The paper’s review highlighted several major challenges, including the paucity of empirical findings, conflicts between HWI approaches, the balance between subtlety and obtrusiveness, and the requirements of structuring a system on a large scale. In addition, to cope with the recent major social changes caused by the pandemic, we outlined future directions that include the development of hybrid real and virtual work environments and giving consideration to users’ well-being. Future work includes developing more specific and practical guidelines for building interactive workspaces that include a variety of workspace components, and for supporting activities within these spaces.