1 Introduction

The vision of ubiquitous computing (pervasive computing) was introduced by Weiser in the early 1990s [41]. From that time on, topics of researches relates to ubiquitous computing expands into wide fields, such as distributed computing [5, 23, 32, 38], mobile computing [7, 19, 33], location computing [8, 24, 26, 43], mobile networking [17], context-aware computing [11, 12, 20, 37], sensor networks  [1, 3, 9, 25, 27, 29], human-computer interaction [35], artificial intelligence  [14, 15, 15], machine learning [10, 40], psychology, sociology, manufacturing and materials.

After a decade of ubiquitous computing hardware development, many critical elements of ubiquitous computing are now feasible products, such as handheld and wearable computers, wireless LANs, and devices to sense and control appliances. Sensors are smaller and less costly, market-available sensors are wireless, and the resulting changes promote a wide range of sensor applications. Nowadays, these sensors are applied in almost every aspect of life, such as agriculture [1], behavior modeling [22], recognizing and interpreting nonspeech body sounds [32] activity detection [28, 39], gerontechnology [18] and adaptive environments [34]. The resulting widespread availability of ubiquitous computing has pushed a boom in both the research and commercialization fields [4]. Many of these applications have reached maturity on small scales, but work on large scale implementations poses new hurdles to overcome.

This work presents a study to access the usability of a smart home system for end-user installation. We introduce usability metrics and test our ideas in the context of the smart home in a box (SHiB) [13], which is developed by the Washington State University Center for Advanced Studies in Adaptive Systems’ (CASAS). SHiB is a multidisciplinary project, with participation from researchers in the fields of computer science, psychology, health care, and other engineering fields. The SHiB project has designed and installed smart homes to obtain large annotated datasets for research projects in machine learning, human factors, sustainability, and health monitoring and intervention.

The long-term goal of the CASAS SHiB project is to design a smart home kit that can be easily self-installed and used to provide valuable activity information. The goal of this particular study is to determine the viability of smart home self-installation, which will greatly enhance the scalability of smart homes. In particular, we investigated the ease and accuracy with which older adult participants could install a smart home kit in their own home (either by themselves or with assistance from a family member) given a CASAS SHiB and written installation instructions. The study presented in this work is carried out in a collection of private residences. In this paper, we introduce the smart home in a box (SHiB) design together with a methodology for evaluating usability of the SHiB. We collect and analyze usability data based on a study with 13 participants who installed the SHiB in their own homes.

2 Evaluation usability of a smart home system

Smart environment installation is a critical component for the system feasibility. One of the challenges of designing smart homes is to balance the complexity of the system against the usability of the system [36]. Complex installation procedures and user interfaces, which have long been associated with smart environments, have prevented the adoption of this type of technology to all but specialists or technophiles [21].

The goal of this project is for laymen (non-engineers) to install the SHiB system straight out of the box without a specialist’s intervention. To support this goal, the CASAS [13] team designed a Smart Home in a Box kit with the aim of providing equipment and instructions that yield successful smart home installation with minimal user effort. The instructions include ways to determine right placement and procedures to let laymen self-evaluate if the equipment works properly or not. Details of the instructions are presented in Sect. 4.1. Because it is impossible to predict the layout of the volunteer’s house, the SHiB kit and instructions need to be flexible and clear to achieve universal installation.

We define success of the SHiB self-installation project in terms of meeting these defined requirements:

  1. 1.

    The SHiB hardware components should be intuitive for individuals with no engineering background.

  2. 2.

    The installation instructions should be easy for non-engineers to understand.

  3. 3.

    The failure rate of the system installation should be low.

  4. 4.

    The installation process should be fast.

  5. 5.

    The installed SHiB system should not cause uncomfortable for people living in the house.

3 Related work

Deployment of a number of sensors and network related devices in the home usually requires researchers or specialists to perform the installation, which will increase time and monetary cost. Therefore, an installation kit that is easy for laymen to install in a house will help to reduce costs, promote understanding of the technology, encourage studies of applications and adapt to the unique features of each particular home layout. However, the ubiquitous system installation studies were mostly held in a lab and having researchers to record the participants’ performance. A few previous studies had sent the system install kit to the participants and let them do the installation, but these studies focus on a wide age range, on the contrary, our study presents in this paper focus on the participants that were older than 50 years old.

In the work of Abdulrazak and Helal [2] introduced the term “A Smart House in a Box” and proposed the concept of the integration of technologies in the environment of aging people so that they would benefit from using adapted systems and accessible environment to compensate problems of daily activities. The study presents the architecture details to build a smart house as well as a real house, namely Gator tech smart house (GTSH) that had been installed the integration system.

Work presented by Beckmann et al. [6] investigated a ubiquitous computing system supporting end-user sensor installation. The study mailed an installation kit to each recruited participant, the kit included printed instructions, handheld scanner, removable adhesives and 10 sensors. Each sensor was labeled with the barcode and the sensor name code. A user could find the sensor’s installation placement and its purpose by scanning the barcode on the instruction. During the installation procedure, two researchers were sent to the participant’s house. After a brief introduction, the researchers asked participants to install each of the 10 sensors, one at a time. The biggest difference between the study reported by Beckmann et al. [6] and this research is the installation procedure. Participants of CASAS SHiB project could determine the sequence to install the sensors by themselves, no researcher attended the installation procedure, and the researchers only visited the participant once after the installation. The number of sensors in each installation kit of the work of Beckmann et al. [6] was fixed at 10 sensors. The number of sensors of the SHiB kit depended on the participants’ house situation and ranged from 8 to 18.

The work reported by Patel et al. [30] is an end-user installable, real-time power use monitor system. The power use sensors applied in the system do not require installation, the power use information can simply be collected by placing a sensor on the outside of each breaker panel in a home. Therefore, the study did not give any literal installation instructions to the participants. A number of power use sensors supplied by the study ranges 2 to 4, which was much less than the number of sensors contained in the SHiB kit. In addition, the sensors in the work of Patel et al. [30] are the same type, therefore, the placement of each sensor was limited to power plugs or a power panel.

In other research reported by Patel et al. [31], called powerline positioning (PLP), researchers investigated the mobility pattern of wheelchair users by collecting real-time navigation data. The PLP system requires the installation of two small, plug-in modules at the extreme ends of the home (e.g., the upstairs northwest corner and the basement southeast corner). The modules inject a mid-frequency signal throughout the home space, and receive positioning tags, listen for the signals radiated off the power line and transmit the position information back to the environment. In this study, the two sensors are installed by the researchers, not the participants. Age of the participants ranged from 33 to 62. Also, in this study, the two modules can track the motion pattern in the whole area, which may cause the participants concerned about privacy. However, participants of the SHiB project can pick the area they would like to be tracked by the sensors.

A study that relates to a do-it-yourself (DIY) smart home is presented by Woo and Lim [42]. In this study, researchers utilized readily available DIY smart home products, namely “Ninjablocks”, and conducted a 3-week in situ observational study. From the results, they identified six different stages within the DIY smart home usage cycle. These are initial installation, motivation, implementation, use-through-routine, routinization, and removal. This study focused on the process as the participants got familiar with the system and the sensors and other devices installed in the kitchen. In contrast to these earlier studies, the purpose of the CASAS SHiB sensors is to recognize the participants life pattern, therefore, the sensors are installed in all the public areas and the rooms that the participants is comfortable with. Moreover, participants are required to figure out the most optimal position for the sensors.

4 Method

4.1 CASAS SHiB install instructions

The instructions play an important role during an end-user installation because during the process no researchers from the CASAS SHiB team are on-site to help the users to install the SHiB system. This section will present a brief information from the SHiB kit installation manual of how participants were instructed to determine the sensor placement and how to install the components.

The component introduction is located at the beginning of the installation guide, the content includes the SHiB component pictures, their names, description, and quantity. An example of the component introduction is shown in Fig. 1. The details of the SHiB infrastructure are reported in Crandall and Cook [16].

Fig. 1
figure 1

An example of SHiB installation guide component introduction

Full size image

Server box should be located in the participant’s home near internet connection and where the server will not be jostled. Plug one end of the black cable (Part A(2) in Fig. 1) into an internet connection source such as a home internet router, and plug the other end in to the ethernet port (A(1)X in Fig. 1) on the back of SHiB server box. Then plug the SHiB server box’s power cord (Part A (3) in Fig. 1) into A (1) and plug the other end into a power outlet or a power strip. The server and components will begin to run as soon as they have power, but there is no visible output.

The relays help the sensors communicate with the SHiB server box. If there is more than one relay, they should be spread around the house in different rooms at a distance from each other. Each relay should be plugged into an available outlet and fixed on a drywall or painted drywall by the adhesive strip (Part G in Fig. 1). An illustration of an installed relay is shown in Fig. 2.

The placements of motion sensors are important for the SHiB project; each sensor should be placed in appropriate rooms following correct labels and directions to ensure accurate detection of locations and activities occurring in the home environment. A motion sensor should be attached to a ceiling, an illustration of the correct way is shown in Fig. 3.

In each bedroom, the participant is asked to install three motion sensors. One is labeled as “Bedroom Bed”, which means the sensor is placed on the ceiling above the bed, and the position of the sensor should be over the participants’ chest when they are sleeping. Another one is labeled as “Bedroom Area”, which means the sensor’s function is to detect any movement in the whole bedroom. The area sensor should be placed on the ceiling in a location that allows monitoring of the entire room but not motion outside of the room. The participant is told to find a place where he/she cannot see outside of the room (i.e., other rooms or hallways). An illustration of correct placements for area sensors is shown in Fig. 4. The red dots in the layout represent area sensors. The third motion sensor is labeled as “Bedroom Door”, which should be placed on the ceiling just inside the bedroom doorway. The sensor should be centered above the door and about 4 in. (10 cm) from the wall over the door. An illustration of the correct placement of the “bedroom door” sensor is shown in Fig. 5.

Fig. 2
figure 2

An illustration of an installed relay on the wall

Full size image
Fig. 3
figure 3

An illustration of correct way to install a motion sensor

Full size image
Fig. 4
figure 4

An illustration of correct placements for area sensors

Full size image
Fig. 5
figure 5

An illustration of correct placement of motion sensor labeled as “door”

Full size image

In a bathroom, the participant should install three motion sensors. One is labeled as “Bathroom Toilet”, the sensor should be placed on the ceiling above the toilet seat. Another sensor is labeled as “Bathroom Sink”, it should be placed on the ceiling above and in front of the sink, where the participant would stand when using the sink. If the participant has two side by side sinks, the sensor should be placed on the ceiling at the center between the two. The third motion sensor is labeled as “Bathroom Area”, it monitors only the bathroom area and in a way that it does not detect motion outside of the bathroom, the placement of the sensor is the same as described in “Bedroom Area” description.

At the home’s two primary exterior doors, a motion sensor should be placed on the ceiling on the inside of the door. The position is similar to the bedroom door; centered it over the door and 4 in. (10 cm) away from the door’s wall.

For the living room, there are two types of labeled sensors. One is labeled as “Living Room Chair”, it should be placed on the ceiling above the most commonly used chair seat. The other is labeled as “Living Room Area”, it should be situated similarly to how the area sensors are placed in the bedrooms. Position the sensor on the ceiling where it will see as much movement inside the living room as possible while not seeing motion outside the living room.

In the kitchen, there are five types of labeled sensors. The placement of each type of sensor depends on its label. The five types of sensor labels are “Kitchen Sink”, “Kitchen Stove”, “Kitchen Refrigerator”, “Kitchen Dining Chair” and “Kitchen Area”. “Kitchen Sink” should be placed on the ceiling in front of the sink, above where the participant will stand when they are using the sink. “Kitchen Stove” should be placed on the ceiling in front of the range top, above where the participant will stand when they are cooking on the stovetop/range. “Kitchen Refrigerator” should be placed inside the refrigerator. “Kitchen Dining Chair” should be placed on the ceiling above the chair where the participant usually sits when they eat. “Kitchen Area” similar to the area sensors were placed in the bedrooms.

In the dining room, there is one type of labeled sensor, “Dining Room Area” sensor. The sensor should be situated similarly to how the area sensors were placed in the bedrooms.

Motion sensors labeled as “Hallway” should be placed on the ceiling in commonly used hallways. If it is possible for the participant, the best position for the sensors is at the intersections of halls. For example, if there is a “T” junction as illustrated in Fig. 6, place the sensor in the middle of the T, and if there is an “L”-shaped intersection, again as illustrated in Fig. 6, place the sensor at the corner of the L.

Fig. 6
figure 6

An illustration of correct placement of “Hallway” sensors: on the left, the hallway has a “T” junction, and on the right, there is an “L”-shaped hallway

Full size image

Temperature sensors should be installed in kitchen and bathroom according to the SHiB project instructions. The participant should place a temperature sensor on the ceiling above the stove or range in the kitchen next to the Kitchen Stove sensor. In the bathroom, a temperature sensor should be placed just outside the most often used bathtub or shower. The sensor should be positioned on the wall near the shower head, about 6 in. (15 cm) away from the edge of the shower and 6 in. (15 cm) below the ceiling. If there were multiple bathrooms in the house, the SHiB project will send extra temperature sensors to the participant.

Front door sensors consist of two parts, one is large and the other is small. Participants are told to place the larger part of the door sensor on the door frame above the front door, near the door handle end and away from the door’s hinges, and as close to the door as possible so that the unit will not hit when the door opens and closes. Open the door slightly when placing the door sensor to ensure you have the sensor placed high enough to avoid being struck by the door. The smaller part (which is also the magnet part) should be placed on the door itself, directly below the larger part of the sensor and as close to the top of the door as possible without hitting the door frame. A red light on the sensor aims to let the participants perform a test to ensure that the sensor detects when the door is opened and closed by opening and closing the door a few times. An illustration of a correct placement to install the front door sensor is presented in Fig. 7.

Fig. 7
figure 7

An illustration of correct placement of the front door sensor: on the left, the two red dots should be aligned, and on the right is the proper way to place the front door sensor

Full size image

4.2 Participants

Older adults were recruited for this study from the greater Palouse region in northwest Washington State. This was done through local advertisements, presentations at senior health fairs, referrals from physician and community agencies, and by contacting participants who had completed prior studies in the Memory and Aging Neuropsychology laboratory at WSU. Older adults in the study are defined as being age 50 or older. In this ongoing study, we currently have 13 participants, 8 are female, 5 are male; 8 are married, 3 are divorced, and 2 are single. The age ranges from 54 to 85, the average age of the participants is 69.23. All the participants are Caucasian, the lowest diploma is high school and the highest is Ph.D. To be eligible to participate in the study, it was required that participants have internet access and be able to give their own informed consent. Some of the participants lived alone while others had significant others or pets. The participants in the study were given compensation for their involvement by receiving a stipend.

4.3 Apparatus

During this study, participants were asked to install the SHiB kit in their own homes. The houses varied in size and structure. 53.85 % of the houses were one story, the rest of the houses were two stories. The number of rooms ranged from 9 to 18 with an average of 12.84 rooms. The participants were delivered a box (smart home in a box, the SHiB kit) that included all of the equipment that would be needed to participate in the study. As described earlier, the box contained several different types of sensors (i.e., temperature, area, motion, relays and front door), a server box and a manual. Motion sensors and area sensors included labels indicating the area of the home where they should be placed. One temperature sensor measures temperature change by the stove and the other one measures temperature change by the shower. Area sensors detect movement throughout most of the entire room or large region of the home. Motion sensors detect movement at a smaller circumference than an area sensor, and the relays help the signal communication between the sensors and between sensors and the server box. A motion sensor is also placed in the refrigerator to detect the use of the refrigerator. Door sensors detect whether a door has been opened or closed. The server box is a simple structure that has ethernet ports and when connected to the internet it allows the sensor information to be transmitted to a lab at Washington State University. The SHiB manual contains all of the instructions for how to install everything into the home with pictures as well. A questionnaire is also given to the participant later in the study.

4.4 Measure

To evaluate the usability of SHiB installation, a post-installation inspection was carried out after the participant received and installed the SHiB kit. The inspection included a check for sensor installation and a questionnaire interview. The inspection involved assessing the location of sensors, relays, and the server box and determining if they worked properly. The sensors should be placed at positions corresponding to their labels and the sensory batteries should be charged. The relays should be placed in locations specified by the installation manual and plugged into outlets. The server box should be connected as specified by the manual. As mentioned in the requirements list, we are striving for a low failure rate. A failure rate less than 10 %, where fewer than 10 % of the components are installed incorrectly, indicates that the installation process is intuitive for the users [6]. A failure rate greater than 35 % indicates that the failure rate is complicated [6]. Incorrect installation was recorded on a house layout sketch drawn by the examiner. Figure 8 shows an example installation inspection sketch. The circles with a dot in the center means motion sensors, the circles filled with black represent area sensors, the squares with label “D” represent door sensors, the squares with label “R” represent relays, the squares with label “S” represent server boxes, the squares with label “T” represent temperature sensors.

The questionnaire consisted of 19 questions. A detail of the questionnaire is shown in the Appendix. Of the questions, 14 requested ratings using a Likert scale. Question 1 through question 6 relates to ease of the SHiB kit installation. Participants provided a rating on a scale from 1 (very difficult) to 5 (very easy). Questions 7–12 asked for participants’ viewpoint on whether they are concerned about privacy with the SHiB kit installed in their home. Participants provided a rating on a scale from 1 (not concerned at all) to 5 (very much concerned). Questions 13 and 14 were about the participants’ general viewpoints on the whole installation process. Participants provided a rating on a scale from 1 (strongly disagree) to 5 (strongly agree).

Fig. 8
figure 8

An example of layout sketch

Full size image

4.5 Procedure

Once a participant had been recruited, some background information was collected through a phone call. The participant was then visited to fill out some paperwork and to give consent. During the visit, participants were given the box with all of the necessary tools needed for them to take part in the study. They were asked to install the SHiB kit either themselves or with the help of a family member or friend. After a few weeks, two to three researchers visited the participant’s home for an installation check. A sketch was drawn of the home layout indicating where every sensor/relay and server box had been placed. If necessary, during the visit, the researchers might change some sensors around depending on the placement. In two cases, where the sensors were not installed at all by the participant, the researchers installed the sensors. After recording the entire placement of the sensors, the server box was checked to make sure all of the collected data is being correctly and securely transmitted to the CASAS lab. A researcher then asked the participant the questions from the questionnaire. The whole visit takes 45–90 min, depending on how well the SHiB kit was installed. After the study is completed, which is around nine or ten weeks, the researchers return to the participant’s home again and remove the entire SHiB for the participant. It is then returned to Washington State University.

5 Results

5.1 Results of SHiB installation evaluation

In this section, we summarize the inspection results for the 13 study participants. In addition, we summarize questionnaire results for the seven participants that completed the questionnaire. Two of the participants waited for the researchers to install the kit, two participants let their children do the installation, and eight participants installed the SHiB kit by themselves. Among the seven participants who completed the interview (which was initiated after the study began), five of them received assistance during the installation.

Table 1 provides a summary of the installation fail rate by device type as well as the reason for the failure, and Table 2 shows the installation fail rate by room type. The two different fail rates relate to each other because a failed sensor installation also creates a failure for the room in which it was installed. The results show that the installation failure rate of motion sensors (6.13 %), area sensors (4.76 %) and temperature sensors (3.23 %) was low; therefore, the rooms that were installed mostly by these sensors resulted in low installation failure rate. These rooms are bedroom (7.58 %), bathroom (5.71 %), living room (2.63 %), dining room (11.11 %), kitchen (7.46 %), office (0.00 %) and hallway (0.00 %). The results generally met our requirements 1, 2, and 3, that the sensors of the SHiB kit should be intuitive and easy to understand for participants with no prior knowledge of sensors or a background in engineering.

Table 1 Results of installation failure rate based on device types
Full size table
Table 2 Results of installation failure rate based on room types
Full size table

As indicated in Table 1, the highest failure rate was for the door sensor (46.15 %) and it corresponds to the entryway, where the failure rate is also the highest in Table  2 (22.58 %). The door sensor does not meet requirement 3 that the installation failure rate should be low. The higher rate of failures for this sensor is linked to the hardware mechanism. The door sensors consist of one reed switch and one magnet. When the two parts of the sensor are perfectly aligned it creates a closed circuit and when one moves the circuit is broken, triggering a sensor event. Every house has a different size and shape of the door, as well as door frames that extend further than the door. This makes it difficult to find a universal door sensor that will equip all entryways and align the sensor parts correctly. As seen in Table 1, the second highest installation failure rate was for the server box (38.64 %). There are two main reasons for server box installation failure. The first failure occurred when the participants waited for the researchers to set up the server box because they thought the server box was confusing, intimidating, and fragile. The other reason was that the participants thought it would use too much electricity so they kept the server box off most of the time. The server box should be connected to the ethernet or no data will be collected as the sensors send signals to the server in the CASAS lab. Therefore, the server box does not meet requirements 1, 2, and 3. The third highest failure rate as seen in Table 1 is for the relays. The most common reason for the failure was that the participant placed the relays at the correct position but forget to plug them into outlets. However, we conclude that the relays meet requirements 1 and 2 because they were all at the correct locations.

5.2 Results of participant questionnaire

Table 3 shows the result of the participants’ ratings to questions 1–6 from the questionnaire. Detail of each question is presented in the Appendix. The rating ranges from 1 (very difficult) to 5 (very easy), “N/A” means the participant did not answer the question. Question 1,2,3,4 and 6 relate to the SHiB kit installation; participants who did not answer these questions also did not perform the installation by themselves. The average rating for each question did not include the “N/A” response. The highest rating is 4.50 for questions 2 and question 4. Question 2 relates to requirement 1, and question 4 relates to requirement 2. The results show that the participants who did the installation agreed the SHiB kit was intuitive for them, and the installation instructions were easy to understand. The “N/A” responses indicate there still exist participants who anticipated that the SHiB kit installation would be difficult and was not eager to do it by themselves. These participants also had no prior knowledge of engineering.

Table 3 Results of participants’ responds to question 1–6
Full size table

Table  4 the result of the participants’ ratings for question 7 through 12 from the questionnaire interview. The actual questions are presented in the Appendix. Participant ratings range from 1 (Not concerned at all) to 5 (Very much concerned). All of these questions relate to requirement 5, whether the participant was comfortable having the SHiB system installed in their home. The average rating scores for question 7 to 11 are under 2.00, which means that participants were generally not concerned. The average rating score for question 12, which was related to concerns about technical issues, was 2.43, in the interval of not really concerned to neutral. The results indicate that the participants who completed the questionnaire felt comfortable with the SHiB system installed in their home. The results support requirement 5 that the SHiB be comfortable for people living in their home.

Table 4 Results of participants’ responds to question 7–12
Full size table

Table 5 shows the result of the participants’ responses to question 13 and 14 (see Appendix). The two questions relate to the SHiB kit installation. Several participants did not provide ratings because they did not complete the installation by themselves. The average rating score for each question does not count the “N/A” response. Question 13 relates to the length of time the participant spent on the SHiB kit installation. The average rating score is 2.67, which indicates the participants thought the process was not long. Question 14 relates to whether participants felt frustrated or anxious during the installation process, the average rating score is 3.00, meaning neutral. Questions 13 and 14 reflect requirement 4.

Table 5 Results of participants’ responds to question 13 and 14
Full size table

Table 6 shows the result of the participants’ responses to questions 15 through 17. Detail of the three questions is presented in the Appendix. The participants could respond “Yes”, “No” or “N/A” for the three questions. “N/A” means the participant skipped the question. Question 15 relates to requirement 5, questions 16 and 17 relate to the requirement 4. 85.71 % of the participants respond positively to question 15, which indicates they are comfortable having the SHiB system in their home. The reason for the “N/A” response for question 15 is that the participant did not complete the installation, therefore, he/she could not explain the process to other people. Positive responses to question 16 were indicated in 85.71 % of the cases, which shows the installation process can be finished in the indicated time. Question 17 revealed that up to 71.43 % of the participants needed assistance during the installation process. One of the participants waited for the inspection team to do the installation, one of the participants let his/her daughter do the installation, and the others received assistance during the installation procedure. The explanation for this assistance is that the home had tall ceilings and sensors needed to be placed on the ceilings.

Table 6 Results of participant’s responds to question 15–17
Full size table

Table 7 shows the results of ranking the installation difficulty of SHiB kit components. The participants provide a rank for five devices (relay, area sensor, temperature sensor, motion sensor and door sensor) ranging from 1 (easy) to 5 (difficult). “N/A” means no ranking for the component, and the reason is, the participant did not do the installation by themselves. The difficulty ranking result showed that participants found the motion sensors and door sensors the most difficult to install, but relays and area sensors were easier. The participants did not give the reason for the ranking. One possible reason that the motion sensor was ranked at the top could be the difficulty in find a proper position for it.

Table 7 Result of installation difficulty ranking of SHiB kit devices
Full size table

Table  8 shows the responses of the participants relate to the information they would like to know from the project.

Table 8 Responses from participants about the collected data
Full size table

6 Conclusion

This paper presents the results of a study in which 13 participants attempted to install smart home in a box (SHiB) kits in their own homes, seven participants also completed a questionnaire interview during the post-install inspection. This study focuses on the assessment of five usability-based smart home design requirements, which are stated in Sect. 2. Requirement 1, 2, 3 and 4 are partially validated based on the SHiB installation failure rate and the questionnaire responses, requirement 5 is assessed using the results of the questionnaire.

Requirement 1 relates to the SHiB kit installation failure rate and question 2 of the questionnaire. Results of the installation failure rate of motion sensors, area sensors, and temperature sensors show these three types of sensors are intuitive for the participants. Relays also validated the requirement, because the relays were all in correct placements. Some of the participants who did the installation answered question 2; from the interview, some of these participants have engineering background, and some were not. The result indicates that for those who have no engineering background, the system is intuitive. The high failure rates of door sensors and the server box indicate that these two types of devices posed challenges for the participants to conduct the correct installation.

Requirement 2 relates to the installation failure rate and question 4 of the questionnaire. Results of the installation failure rate of motion sensors, area sensors, temperature sensors and relays show the instructions for these devices are easy to understand. The reason why relays also validated the requirement is the relays were located at proper places. The participants who provided rating scores for question 4 performed the installation by themselves. Some of the participants have the engineering background, some were not, the results indicates for people who have no engineering background, the installation instruction is easy to understand. High failure rate devices, in particular, the door sensors and server box, indicate that the instructions for the two types of devices need improvement.

Requirement 3 relates to the installation failure rate. Motion sensors, area sensors, and temperature sensors experienced a low failure rate. However, the failure rate of installing relays is a little high, during the post-installation inspection, the researchers found two participants forgot to plug in the relays. This result implies that a reminder should be added to the relay instructions to prevent participants from neglecting the step. The reason for the high failure rate of door sensors is the hardware design limits the types of doors that can function properly. We, therefore, conclude that an alternative door sensor design should be considered. The reason for the high failure rate of the server box is that its appearance makes participants think it is “hard to deal with” or it will need a lot of electricity, therefore, a server box that the appearance looks simple and will only require one or two steps to set up is needed.

Requirement 4 relates to question 1,3,5,6,13,14,16 and 17. The results indicate that for the participants who did the installation by themselves, the process did not take a long time and can be finished in the indicated time, but assistance is needed during the installation procedure. For participants who waited for the researchers to do the installation or let their children do the installation, they could not evaluate ease of installation, thus, requirement 4 is validated partially.

Requirement 5 relates to questions 7 through 12 and 15. The results show the participants were comfortable having the SHiB installed in their homes and most of them would recommend the study to others, which validated requirement 5.

The results of the participants’ responses to the data collected from SHiB system show the things they were interested to know and the expectation of using technology to improve their life qualities. Some of the participants concerned about the use of the data, which reveals that there is a need for the SHiB project to give an explicit explanation about the project purpose to the participants. Some of the participants hope to see the data collected from the SHiB system could be used to develop new technology applications to help them having a better senior life, on both physically and mentally.

The limitation of this study is the sample size of the participants who took the questionnaire interview, which is small. A large sample size will reflect more information about the viewpoints and ease of the installation procedure. The contribution of this study includes utilizing installation failure rate to evaluate the usability of a smart home system. In particular, the results suggest directions the SHiB kit design could develop in the future. The questionnaire collected more details from the participants about their SHiB kit installation experience.

In summary, the findings indicate that portion of the current SHiB system (contents relate to motion sensors, area sensors, temperature sensors and relays) are intuitive for individuals with no engineering background; the installation instructions relate to them are easy to understand therefore their failure rate is low. Some of the participants who had an engineering background considered the process to be simple. The participants who received the interview said that they were comfortable with the system in the house. Future work could focus on improving the installation instruction, such as adding more figures and reducing descriptions, or having pre-recorded installation videos. To help make the installation easier for individuals without an engineering background, a pre-installation online tutorial or an installation tutorial app could be built to help provide individuals with some context before the installation process begins.