3.1 Introduction

The movement that defines new usages and physical settings for computing technologies through intermixing and broadening the boundaries of theories, concepts and design methods, is often referred to as ‘third wave HCI’ (Bødker 2006). It has been defined as a paradigm that focuses mostly on interaction as a phenomenon, that is going on between humans and machines and that is embodied and phenomenologically situated (Dourish 2004; Harrison et al. 2007). Third wave HCI is evolving through an approach that encourages user participation in conceptual thinking for designing shared interaction of artifacts (Bødker 2015). Further, it has been outlined that there is a need for methodological investigations in third wave HCI, focusing on artifact ecologies and their shared usage (Bødker and Klokmose 2012; Bødker 2015).

Some main concerns that have emerged within third wave HCI – such as the value, the meaning, and the place of interaction – have historically played important roles in formgiving practices in design. For instance, Harrison et al. (2007) highlight an explicit focus on value-based design, whereas the evaluation of a ‘good’ design is based on the user and the particular context of use. This focus shifts the evaluation of design of computing artifacts from universally valid approaches to more participatory and value-sensitive design approaches. Further, in third wave HCI, knowledge is subjective, bringing new perspectives into practice. This is in contrast to first and second wave HCI that focus on objective knowledge and its general applicability. We see great potential in studying formgiving practices in third wave HCI, and in particular on how to give form to smart objects.

The emergence of smart objects – being mundane yet intelligent products – impose new challenges for interaction design (Rozendaal 2016). In many products that are labeled as being smart or intelligent, the material expression of this quality appears to be somewhat unattended. For example, while voice-controlled smart speakers, such as the Amazon Echo (2017), the Google Home (2017), and the Apple Homepod (2017) are capable of performing actions that qualify as intelligent, hardly anything in their material form is expressing this capacity. As a consequence, a product’s intelligence might be experienced to be ‘stuck on’ as an added feature rather than being integrated into a product’s form. Understanding and developing intelligence as expressed in the form of smart objects is important to tackle the challenges related to our interactions with them. This resonates well with what Krippendorff highlights in his definition of product semantics in relation to human interfaces (1989, 2006). It is about our experiences and interactions with objects and environments that renders things understandable, meaningful, transparent, alive and usable (1989: 4–5).

Within the last decade, materiality and material aspects of computational objects have received increased attention within the HCI and design communities. Under the umbrella of the material turn in HCI (Wiberg 2014, 2018; Wiberg et al. 2013; Robles and Wiberg 2010; Vallgårda and Redström 2007; Rosner 2012; Dourish and Mazmanian 2011), this branch of research spans from exploring formgiving of computational objects and materials, to new theoretical understandings and approaches to material interaction, and to practical oriented and craft-based works. Further, ‘intelligence’ itself has most recently been highlighted as a design material in HCI (Holmquist 2017; Dove et al. 2017) stating that in the near future, intelligence will become available as a resource to be used by non-experts. Therefore, the opportunities and limitations of such material must be carefully considered in the design process, similar to any other design material.

The core aim of this paper is to contribute to these streams of developments in HCI by advocating how intelligence can be used as an interaction design material, particularly in formgiving practices. In doing so, we will provide both a theoretical and empirical grounding on how to give form to interactive and smart objects using intelligence as a resource.

We first review related work that centers around understanding Artificial Intelligence (AI) in the context of HCI as it spans from the 60s to contemporary views on smart objects in the Internet of Things (IoT). Then, we will review the history of formgiving practices and its materials in Product Design and HCI. Subsequently, we will report on a design course, where students were asked to give form to smart everyday objects to empower people in specific problematic situations. By reflecting on the design course, we suggest a new methodology that considers intelligence as being part of, and expressed through, an object's character. The paper concludes by discussing lessons learned from our design course as well as discussing implications for formgiving practices in HCI.

3.2 Artificial Intelligence in HCI

Since the 60s, intelligence has started to become a matter of concern in HCI. Due to their increasing capabilities and autonomy, interactions with computers changed, as they transformed from being ‘tools’ to becoming ‘agents’. Generally speaking, agents are entities that can function autonomously without human intervention, and that can interact with other agents (including humans). They are further able to log, sense and collect data, perceive their environment, and react to changes in a timely fashion (Wooldridge and Jennings 1995).

Based on the emerging trend at that time, Licklider (1960) proposed a new perspective on human-computer interaction – that he labelled human-computer symbiosis – to better understand the new opportunities and challenges when interacting with computers as agents. Licklider highlighted the changing role of computers, as not only performing automated tasks predefined by humans, but also taking part in task-formulation and in collaboratively carrying our complex tasks as partners.

Later, in the 90s, when AI was developed in the context of virtual assistants, people continued to think about what it would be like when agents will be able to help us do things and augment our abilities (Norman 1994). The main challenges foreseen by Norman were related to the social ability of such agents concerning the ways they interact with other agents and humans. Some of the specific issues he mentioned were people’s overblown expectations of what agents can do, and feelings of losing control in relation to the automated actions exhibited by agents. In particular, he addressed issues of privacy, which would become more prominent as agents access personal data and act on this information without human intervention.

Contemporary scholars in HCI have revisited the classical work of Licklider (Farooq and Grudin 2016; Jacucci et al. 2014), proposing that new studies need to be conducted to tackle the issue of increased automation and independence of computers in computer use (Jacucci et al. 2014). Further, this development urges the HCI community to develop new approaches to the design and evaluation of interactive systems (Farooq and Grudin 2016), thus touching upon many of the issues brought forth by Licklider and Norman.

Now in 2017, we find ourselves in a situation where intelligence moves from AI and virtual assistants, to intelligence that is embedded in smart objects as part of the IoT. Here the new question becomes how intelligent sensing and actuating capabilities can be meaningfully connected to human activities, allowing the object to have awareness and interactivity, which require new technological architectures (Allmendinger and Lombreglia 2005; Kortuem et al. 2010; Sabou 2010). Further, the sustained presence of smart objects in everyday life also requires sensitivity in designing their smartness in accord to the materiality and sociability of everyday life (Giaccardi 2015; Janlert and Stolterman 2017).

3.3 Materials in Formgiving Practices

Product design has a long tradition of the engagement with and manipulation of materials (Pevsner 1991; Raizman 2004). At a specific moment in a design process, a transition takes place from a design being an abstract functional description to becoming a concrete design manifestation (Ramduny-Ellis et al. 2010). Muller identifies this moment as the beginning of the form-creation phase, “that starts at the moment that any conceptualization about the material form emerges, and ends when a definitive design is established” (Muller 2001). In this phase, the ideas of the designer are externalized, explored interactively and represented tentatively in a visual form using a variety of media (McKim 1980). The process is not solely concentrated on determining the material conditions for the fulfillment of the function of the product, but is also aimed at establishing the product’s desired semantic and aesthetic qualities in relationship to its intended experience, meaning and use (e.g. Krippendorff 2006; Bergström et al. 2010; Vallgårda 2014).

In product design, a design process involves the use and manipulations of physical materials such as paper, wood, clay, foam or plastics for making mock-ups in the early stages of a design project, and later stages of prototyping may involve electronics such as Arduino’stm and a variety of sensors and actuators. However, these design materials can also include digital ones and media (graphics, sounds, images), as well as data in different types and formats (Crilly et al. 2009). Design materials are often made available to designers in toolboxes, kits, or in other ways that allow designers to sketch by easily changing, combining and reconfiguring them (Tholander et al. 2012; Verplank 2009).

Formgiving has been also defined as the ability to identify form attributes that convey sensorial and interactive experiences (Smets et al. 1994). Under the umbrella of aesthetics of interaction, formgiving has further evolved in HCI (Petersen et al. 2004; Djajadiningrat et al. 2002) and has been applied in interaction design and HCI strongly influenced by Gibson’s theory of ecological perception and later by pragmatist philosophy and phenomenology (Ross and Wensveen 2010). Although shaping such materials helps to gain an extensive knowledge and understanding about their behaviors and potentials, current interactive artifacts increasingly include digital materials and information technology, so the form of objects is no longer driven by the technologies within them (Evans and Sommerville 2007) and traditional modes of product’s expression of meaning may no longer apply (e.g. Vihma 1995).

Researchers working on the material turn in HCI seek to understand the material basis of interactive artifacts in order approach digital and physical aspects of interactive artifacts as a whole (Wiberg 2014, 2018). This branch of research spans from exploring formgiving of computational objects and materials, to new theoretical understandings and approaches to material interaction, and to more practical oriented and craft-based works. Robles and Wiberg (2010) propose texture as a concept that unites these physical and digital elements of interactive artifacts from an experience point of view. The term texture denotes how these aspects come together forming a composition of cultural and aesthetic forms of interaction, where tangible and digital aspects are integrated.

Similarly, the concept of computational composites as proposed by Vallgårda and Redström (2007) considers the interdependency of interactive artifacts as experienced wholes by better understanding three constituting elements: physical form, temporal form and the ‘form’ of interaction. While a physical form is perceptible through its three dimensional and tangible shape, a temporal form is a pattern of the state-changes that a computer can produce over time. The ‘form’ of interaction, is a concept that is related to interaction as a dynamically experienced interplay between a human and an artifact, also referred to as the ‘interaction gestalt’ (Lim et al. 2007). All these three forms are interdependent and should therefore be designed in tandem.

3.4 Artificial Intelligence as a Material

The ongoing development and increased possibilities of computational technologies do call on designers to add another design resource and material in their toolkits, which is intelligence (Holmquist 2017; Dove et al. 2017). Designing with materials usually requires particular skills and knowledge that enable designers to work with the material and explore its opportunities and limitations. For example, a study carried out by Dove et al. (2017) presents some of the difficulties that designers face in dealing with Machine Learning (ML) in their practice. These involved having an accurate understanding of the nature and use of ML, how to prototype it, and how to purposefully design with it.

Holmqvist (2017) talks about the challenges involved when integrating AI in the design of User-Interfaces (UI’s) and mentions the need to design UI-components that clearly communicate how control is shared between humans and AI, and how designers could deal with the new learning capabilities of their designs. This recent view on intelligence as a design material discusses some of the required skills and knowledge. For instance, developing an appropriate technical understanding of AI and ML, and ways to program it. While we share this idea, our position in this paper seeks to go beyond understanding and applying AI as technical instrumentation, but rather to understand how intelligence can meaningfully be dealt with in formgiving practices in interaction design and HCI. The question then becomes, what are the critical material and communicative aspects of intelligence when fully integrating it into the product’s form as a whole?

Building on this understanding of intelligence as a material specifically in relation to formgiving practices, we propose an approach that builds on the work of Janlert and Stolterman on the character of things (1997). Janlert and Stolterman introduced the notion of character as an entry point to understand computer artifacts as meaningful wholes. The character of an artifact is defined as the unity of its multiple characteristics, that involves the sustained impressions of aspects of the artifact’s function, appearance and manner of behaving, aggregated over time in a complete and coherent way. The character of an artifact helps users to interpret its behavior. For example, the simple movement of stretching out one’s arm can be interpreted as either offensive or welcoming, depending on whether the person is perceived to have a hostile or friendly character (Janlert and Stolterman 1997). Thus, actions (i.e. behaviors) of a computer artifact are explained in an interpretive frame provided by its character.

Janlert and Stolterman’s work also links to notions of anthropomorphism and animism in understanding the character of computer artifacts by referring to the work of Reeves and Nass (1996) on people’s inherent ability to treat computers as social actors and by referring to Brenda Laurel’s work on the perceived animism of computer technology (1997). We therefore propose that the overall perceived form of an object, (i.e., its character), should be able to convey information about its function and usage, and also about its intelligence.

We further suggest that in order to give form to the character of smart objects we need to consider their intent, materiality, and agency. Pixar’s well-known animated lamp ‘Junior’(2017) is a good example to clarify these considerations. The intent of a smart object is considered to be shaped by its purpose as a product but also by its motive as an agent. We see Junior as an everyday lamp that acts according to its own beliefs and desires. The materiality of a smart object is concerned with its physical and digital structures that enables its expressiveness as an agent and also communicates its functional, aesthetical and symbolic characteristics. For example, Junior’s structure, materials and components provide the lamp its ability to adjust light in a specific way. However, these elements are also carefully animated to express its inner life. Finally, the agency of a smart object defines the possibilities to interact in the world, both as a tool and as an actor. The hybrid character of Junior (i.e., being a lamp and an agent), makes Junior act in a story in a particular way.

3.5 Design Course

To illustrate how smart objects are given form by considering intelligence as part of an object's character, we will describe and reflect on a design course called Interactive Formgiving, a Master elective within the Industrial Design program at Delft University of Technology. As a design brief for the course, students had to develop smart objects as being both everyday products and intelligent agents. Starting from this design brief, the students went through three design cycles (or phases), in which the relations between intent, materiality and agency of the object were explored and defined on increasing levels of detail and integration (Fig. 3.1). The process involved (1) a Conceptual phase, in which the user, situation and object were identified, the character of the object was defined and a first exploration of its behavior was conducted; (2) an Embodiment phase, in which the expressions and behaviors of the everyday object was explored and articulated through its inherent structure and materials; (3) an Enactment phase, in which the collaboration between user and object in the problematic situation was acted out and experienced in context.

Fig. 3.1
figure 1

Process structure of the design process: Conceptual phase (left), Embodiment phase (middle), and Enactment phase (right)

Full size image

To help the reader understand the type of products that resulted from the course, we will first describe three of the designs and highlight their purpose and target group, functionality, and context of use. This is followed by a detailed description of each of the three phases of the design process and an in-depth discussion of how the activities in each phase informed the design of these objects through the articulation of their intent, materiality and agency.

3.5.1 Design Cases

The smart objects that were developed had the purpose to empower people in problematic situations. As both being everyday products and intelligent agents, these designs are described as having their own motives and personality. In their functioning, these objects directly and actively intervene in activities that people consider problematic. Below three of these designs are described in more detail. See Fig. 3.2 for a visual impression.

Fig. 3.2
figure 2

Image showing Pat the social backpack (left), Harry the power drill (middle) and Waggle the shoe (right)

Full size image

3.5.1.1 Pat the Social Backpack

Pat is a smart backpack specially designed for teenagers that tend to be somewhat insecure and thus have some trouble connecting to others at school. Pat, however, is full of confidence and eager to make new friends, which in Pat’s case means connecting to other backpacks. As Pat cannot move around by itself, it needs the support of the teenager. In this way, Pat will empower teenagers in making contact with other people more easily as well: providing them with a ‘free-ride’. Pat encourages the teenager to approach others by gently nudging him or her forward and rewarding the teenager when doing so, by gently squeezing the shoulders. However, when the teenager doesn’t approach, Pat will show disappointment by sliding of the shoulders of its owner, thereby enticing the teenager to regain control by firmly pulling the backpack back up again. This action reinstalls a sense of confidence in the teenager, thus encouraging him or her to successfully make the next approach.

3.5.1.2 Harry the Power Drill

Harry is a smart power drill designed for construction workers that tend to work improperly or even recklessly, which can ultimately lead to broken equipment and accidents. Harry is a true professional who wants to be used properly and doesn’t like the idea of overheating or being broken. It is eager to participate with humans when used properly, but becomes reluctant to drill when the drilling angle is slightly off or the drilling pressure is too high. Harry will express this by providing a slightly weaker grip on its handle, thus installing a feeling of weakness and instability while drilling. If Harry notices that its user becomes really reckless in its drilling, it gets angry and will shut down by retracting its drill-point and control handles, to make its owner aware of his inappropriate behavior and give him some time to reflect. After a calm-down period, Harry then comes back to life again and is ready to be used once more.

3.5.1.3 Waggle the Shoe

Waggle is a pair of smart running shoes designed for people who run but sometimes cannot find the motivation to hit the track. Waggle acts like an enthusiastic puppy that loves to go outside, which it makes clear by flapping with the flexible sides of the shoes. Once Waggle senses that his owner has noticed its need, it will express its enthusiasm for running by increasing the frequency and amplitude of this flapping motion. Waggle will then make it very easy for its owner to put his running shoes on, by opening and fastening itself. Any actions that might stall the actual running are being counteracted with even more enthusiasm as Waggle starts to get jumpy; physically pushing its owner to move his or her legs. Once back after running, Waggle joins the runner in feeling both proud and tired, by loosening itself up so that the running shoes can be easily kicked of its owner’s feet.

3.5.2 Methodology: Interactive Formgiving

3.5.2.1 Conceptual Phase

Starting from the initial design brief, students first conducted a brainstorm to come up with daily activities that people might need help with. This resulted in three types of users who experienced some kind of difficulties in concrete situations: (1) teenagers that struggle to connect with peers at school due to feelings of insecurity, (2) construction workers that needed to be urged to work safely, and (3) runners who want to go for a run regularly but lack the motivation to do so.

These situations were then further analyzed with the aim to identify an everyday object connected to the problematic situation, that – being made interactive and intelligent – would empower a person to tackle this specific problem. Thus, one group decided to choose a backpack to help teenagers feel more confident. The second group focused on a power drill that promotes safe use during the physical act of drilling, rather than providing verbal safety instructions. The third group settled on running shoes to motivate runners to go for a run.

Then, students were challenged to frame their everyday object as a character. This involved a complex and creative leap, as they needed to characterize the object as a purposeful product and as an intelligent agent. Thus, Pat was conceived to be a popular friend who helps the teenager carrying stuff around but also wants to socially connect with other backpacks, while drill Harry was envisioned to be a professional perfectionist, who is keen to help you to drill safely and efficiently, but gets annoyed when it is not being used in the proper way. Finally, Waggle, the pair of running shoes, was framed as an enthusiastic puppy who motivates people to go for a run by eagerly expressing its own desire to go outside.

After having framed the character of the object, its behavior could now be defined, as being meaningful to its pragmatic function and its motive as an agent. In the course, the number of behaviors was limited to three. For example, Harry could act dedicated or reluctant, and could even refuse to act. Harry’s dedicated behavior provides the construction worker support while drilling, by securing firm and stable grip controls and fast drill-rotations. However, when the construction worker applies too much physical pressure to the drill, it starts to behave reluctantly by loosening its grip controls and hereby becoming less stern. As a consequence, the net-amount of physical force that can be applied to the drill is stabilized, preventing the power drill from overheating and being damaged. Then, when the construction worker applies pressure at unacceptable levels that might damage the power drill, the power drill-controls are drawn-in abruptly. The drill stops and refuses to continue.

Pat the backpack was designed with teasing behavior to nudge the teenager to approach other backpacks, engaged in comforting behavior to motivate the teenager to continue with making the approach, and to show disappointment when the teenager turns away. Based on a combination of sensory inputs (heartbeat frequency as an indication of anxiety combined with the proximity of Pat to other backpacks), Pat could deduce whether its owner is approaching or retreating from other backpacks, and if the teenager feels anxious about this or not. According to this data, Pat could then decide to tease the teenager, if the anxiety-level is not that high, comfort the teenager when he or she feels anxious when making an approach, or might show disappointment if the teenager decides to abort. Making a state-transition diagram helped the students to formalize the information these smart objects required to act and provided students with a deeper technical understanding of their intelligence (Fig. 3.3).

Fig. 3.3
figure 3

State-transition diagram made for Pat the backpack

Full size image

3.5.2.2 Embodiment Phase

In the embodiment phase, students created physical mock-ups to explore the form and composition of their objects as informed by their product category (i.e., backpack, power drill, running shoes) and by their character (popular friend, professional perfectionist,enthusiastic puppy). This involved using existing everyday objects and modifying their materials, structures and dynamic properties to enable their expressive capabilities. For example, rigid types of plastics, flexible soft materials such as textiles, and industrial rubber, etc. were applied. Students were discouraged to add elements to the object that felt alien to the product category or that felt ‘out of character’.

Within the boundaries established by these mock-ups, students were then further urged to animate these inherent structures and materials through human motion (e.g., puppeteering). For this purpose, transparent wires or other mechanisms were used, sometimes supplemented by using the hands directly, enabling students to get a grip on the object’s expressive behavior. For instance, how the backpack could slide down the back of a person in such a way that it showed disappointment, or how reluctance could be expressed through the power-drill by cleverly manipulating the positioning and experienced firmness of the grip-handles while drilling. For the shoes, this meant discovering how the distinct dynamic product attributes of shoes, such as their flexible leather sides and soles, could express enthusiasm.

The next step involved recording these manipulations on video and playing them back to the students in order to better understand and pick-up on the nuances of the expressions and to help students focus on their temporal qualities. For example, how fast should the backpack drop of the shoulders and which trajectory should it follow? Are the movements of the running shoes enthusiastic enough, and can subtle differences between expressions be distinguished from one another?

3.5.2.3 Enactment Phase

The goal of this last phase was to understand and develop the agency of the object, as it is situated and experienced in context: what influence does the object have on the situation, how do human and object engage in collaboration, and can we assess if the object is actually empowering its users? In these enactments, the activity and context were staged as informed by the storyboard that was developed previously by the students. One student puppeteered the object, while another student played the role of the end-user, reflecting on the experience of using the object and discussing possibilities for improving the design. Video was used as an observation and communication tool to capture and reflect on the enactment (Fig. 3.4).

Fig. 3.4
figure 4

Image showing students in enacting the interaction with Pat the social backpack

Full size image

During the enactments, the focus was on understanding the agency of the objects. For example, some enactments showed how the expression of disappointment of the backpack had an emotional impact on its wearer: when students noticed Pat sliding down their backs, they felt abandoned by it as well as made them look silly. The power drill’s refusal behavior, as expressed by its sudden retraction of its drill-head and control-grip, felt so sudden that it stirred discussion about how this might cause a dangerous situation in itself, and raised the issue of how the transition from reluctance to refusal should be made more gradual. The enactment and improvisations with the running shoes also revealed the need for a stronger type of stimulation to motivate the person to go for a run than as previously anticipated.

3.6 Discussion

We will discuss how our formgiving approach to smart objects contributes to the work being done on formgiving practices in third wave HCI, and also on how intelligence can be considered as a design material in these practices. Based on our experiences in the design course, we will then reflect on (1) how students conceptualized smart objects as characters that reconciled their nature of being familiar everyday products and intelligent agents, (2) how students embodied the intelligence of smart objects in authentic ways by using a product's inherent structures and materials as a means to express their intelligence, and (3) how students approached the interaction with smart objects pragmatically (linked to their function as an everyday product) and as a form of social interaction.

3.6.1 Intelligence as Material in Formgiving Practice

Inspired by formgiving practices in Product Design and HCI, we have explored intelligence, not as a technology on the level of programming or algorithms, but as something that can be shaped and communicated through form. Wiberg’s methodology (2014) deals with similar issues. Approaching intelligence as a material, allows one to explore intelligence as texture, which is the “formal relation that appeals to the feel, appearance, or consistency of surface or substance” Wiberg and Robles (2010: 68). The concept is useful to describe computational compositions that unite physical, digital and spatial components into an overall experience. This methodology is inspired by design processes that emphasize iteration – working back and forth – between details and wholeness, materials and textures of a computing composition. We have learned from our design course how moving through multiple iterations allowed the object's character to take form, on increasing levels of detail and integration.

Further, a design process using puppeteering and enactments was used to work with intelligence as a material in a so-called third space. A third space is created when reality is suspended and explored to provide a productive and creative space at the boundary between the actual and the possible, the real and the fictional (Halse et al. 2010). These enactments made students aware of the agency of design materials (Schön 1984; Tholander et al. 2012) and provided a reflective conversation with intelligence as a material that is expressive and social. The use of video in particular, allowed students to adopt a more objective view on their work and helped them to investigate behavior in its temporal details (Pasman and Rozendaal 2016).

3.6.2 Intelligence as Character

In designing smart objects as characters that reconcile their nature of being familiar everyday products and intelligent agents, we argue that aspects like familiarity, authenticity and sociability should be taken into account.

3.6.3 Familiarity

Designing the intelligence of smart objects as part of their character required students to understand and define intelligence through familiar everyday products. In this way, these products themselves established the grounding metaphor for their intelligence, and thus, sets the intelligence of smart objects apart from the intelligence of social robots and conversational agents, which take the human as their grounding metaphor. Taylor takes a similar approach as he talks about machine intelligence, as intelligence that is performed in everyday settings and does not require a human equivalent (Taylor 2009). We argue that framing the intelligence of smart objects within familiar product categories will help people to accurately predict and explain their behavior. Thus preventing them to develop overblown expectations about their intelligence (Norman 1994).

Practice-oriented theories (Kuutti and Bannon 2014) such as activity theory (Kaptelinin and Nardi 2006) consider everyday artifacts to mediate our activities in the world. Knowledge about this relationship can help designers to identify which objects would become empowering given a specific target group and use context (Ghajargar 2017). Further, adopting a thing-centered perspective (Cila et al. 2017) can be used to help reframe these familiar everyday artifacts as agents, for example by developing object personas as a design technique (Cila et al. 2015; Giaccardi et al. 2016).

3.6.4 Authenticity

We have learned how the intelligence of smart objects can be expressed in an authentic way by using a product’s inherent structures and materials. For example, students were urged to think about how the flexible sides of shoes, laces and soles, straps, zippers and textiles of backpacks, and grip-controls of power-drills, etc. could be utilized as expressive means to communicate intent and affect. During the course, students were therefore discouraged to use materials that felt alien to their chosen product category.

To design the intelligence of smart objects in authentic ways, designers need to be aware of how movement can express inner life. i.e., motions and behaviors that communicate intentions, emotions and attitudes (Siegman and Feldstein 2014). This allows designers to take advantage of people’s ability to perceive inanimate objects as being expressive, comparable to, yet different from, humans and animals (Heider and Simmel 1944; Hoffman et al. 2008; Hoffman and Ju 2014; Marenko and van Allen 2016). New design sensitivities and vocabularies are required to design for such perceptions.

3.6.5 Sociability

During the course, we have learned how students approached the interaction with smart objects pragmatically (linked to their function as an everyday product) and as a form of social interaction. For example, seeing Harry the power drill act like a professional perfectionist made it apparent that alongside its main drilling function, Harry could invite, guide and stop the construction worker from drilling based on its judgements about how to drill in the ‘right’ way. Interacting with smart objects then changes from use-oriented to collaboration-oriented relations, where smart objects can influence, take control, or even overrule the actions of their users, as governed by their intent.

Designing smart objects requires a critical understanding of how they collaborate with us. When objects complement humans on the level of ability, competence or awareness, designers needs to be aware of how people would experience this dependency. For example, people could feel patronized by the object or feel dominated by it. In case of Pat the social backpack, these issues were brought to the foreground quite early in the design process. Pat's behavior of nudging the teenager forward or sliding of the shoulders, was experienced as dominating. This stirred discussion whether such (social) behavior is acceptable and useful to empower teenagers in this situation. Thus, to evaluate its appropriateness, traditional design criteria would need to be supplemented by social-oriented criteria, such as an object’s ability to communicate and act politely, firmly or cautiously (Niess and Diefenbach 2016).

3.7 Conclusion

In this chapter, we have reviewed related work that centers around understanding Artificial Intelligence (AI) in the context of HCI, spanning from the 60s to contemporary views on smart objects in the Internet of Things. We then have introduced an approach that brings traditional values of formgiving practices into interaction design, in particular to the design of smart objects. This approach considers intelligence as being part of, and expressed through, an object's character, hereby integrating it into a product's overall physical form. In the development of this approach, we have been informed by material notions on interaction in HCI that deal with the immaterial (data, media, algorithms) and we have added intelligence to the discourse.

We suggest that in order to be successful with this proposed approach, designers need to develop smart objects through iterative cycles, in which their intent, materiality and agency are explored and defined on an increasing level of detail and integration. Furthermore, we have elaborated about how aspects like familiarity, authenticity and sociability should be taken into consideration when designing the object’s character. After having discussed the implications of our approach and findings, within the frame of third wave HCI, we express our hope that this work will contribute to the development and use of intelligence as a design material, as well as spark new theoretical understandings and approaches in interaction design. We would like to end with a word of gratitude by thanking the students who have participated in our design course and by thanking our colleagues for their support.