Keywords

1 Introduction

The smart watch has limited input and output capabilities due to its form factor. As such they are used more as of a fashion item and a surrogate slave device to the larger smart phones. For example, the touch screen on the smart watch can only display about four to six easily touchable icons (e.g. on a 2 × 2 grid) and touching in letters or digits, selecting an application out of many menu items can be very difficult. Improving the interactivity of the smart watch can make it more independent and self-contained device. With this motivation, we propose in this poster the concept of “Touch Skin (TS),” that enlarges the interaction space of the smart watch using the large hand (or skin) surface area and proprioceptive sense. Proprioception refers to the human’s capability of sensing of the relative position of neighboring parts of the body and strength of effort being employed in movement [1]. In humans, it is provided by proprioceptors in skeletal striated muscles and in joints. With the TS, while the input interface is displayed on the small smart watch screen, the interaction is carried out on the hand surface to which the input interface elements (e.g. graphical buttons and keys) are mapped. We hypothesize that even though the display and interaction surfaces are separated, the humans are projected to nevertheless perform very well based on the proprioceptive sense. For example a human user might be able to pinpoint (with one’s finger) to a particular part on the hand (e.g. palm, back, fingers) up to some accuracy and resolution, e.g. up to par with similar touch screen devices. The concept of TS is well illustrated in Fig. 1.

To realize “Touch Skin”, sensing movement or position on skin surface and installing such a capability on the smart watch will be the major technical challenge. We believe that, for example, a smart watch equipped with a depth or ultrasonic sensor on its side can detect finger position on the palm at sufficient resolution to realize our proposal (see Fig. 1). In this study, however, as a first step, we explore whether the proposed interaction scheme will perform sufficiently or comparable to that of the smart phone (for certain major tasks) through an enactment study (i.e. pretending as if the finger/skin tracking is operational).

Fig. 1.
figure 1

The concept of the proprioception based “Touch Skin” input method for a small screen device.

Full size image

2 Related Work

Proprioception has been applied to HCI in few occasions. Mine et al. introduced the concept of the “scale-down grab” in which a remote object manipulation was carried out by scaling down the entire scene and bringing the target object within reach and rely on the proprioceptive sense to manipulate it well [2]. Gustafson et al. presented the notion of “imaginary” interface in which one might interact on an arbitrary object well even a without display (e.g. on the palm) as a convenient substitute. While similar to our approach, theirs was not based on the notion of proprioception, but rather similarity in the physical mapping and familiarity [3]. Using the skin (e.g. palm of the hand) for interaction surface is an increasingly popular idea. In the context of wearable augmented reality, it is an attractive option where a interface elements can be augmented onto the hand using a projector or through a see-through display [4] and where the fingers/hands are tracked using a vision based approach. Skinput, on the other hand, by Harrison et al., sought to transform the skin into a touch screen like surface by analyzing the mechanical vibrations that propagate throughout the body [5]. Laput et al. presented an augmented input method for smart watches, by projecting icons on the nearby user skin and making them touch sensitive [6]. Our concept is similar except that we claim that the interaction surface can be relatively far from the small screen device (like the smart watch) without any display.

3 Experiment

3.1 Experimental Procedure

The purpose of the “enactment” pilot experiment was just to quickly and partially validate the feasibility of our approach, that is, whether the proposed proprioceptive interaction can produce input performance competitive to that of the usual smart phone. Five paid subjects, 4 men and 1 women with a mean age of 25 (one left handed), participated in the experiment. The experiment tested four input methods: (1) regular smartphone input, (2) TS with keys mapped to the palm, (3) TS with keys mapped to the back of the hand, and (4) TS with keys mapped to the fingers (see Fig. 2). The dependent variables were the input completion time, error, and answers to a usability questionnaire (omitted here for lack of space).

The task was to enter fifteen 10 digit numbers as quickly and accurately as possible using the respective input methods tried out in balanced order. The experiment was an enactment because detecting of the skin touch was not implemented, thus no sensor error nor delay was considered. The subject performance was recorded by a video camera and measured manually by the experimenter. When the TS was used, the input key layout was displayed on a separate monitor screen in a small scale (as small of a smart watch screen). The subject wore a glove with the input key mapping drawn on it just to aid the experimenter manually measure for the input error while reviewing the recorded video footage. The subjects were instructed to enter the numbers as fast and accurately as possible. When testing for the three TS conditions, they were also instructed to only watch the monitor screen rather than their hands.

3.2 Results

The three TS conditions did not differ in the task completion time but TS with the mapping on the back of the hand showed a lower accuracy with a statistical significance. When compared to the smartphone, the TS finger and TS palm were less accurate but actually slightly faster in performance (see Fig. 2). The survey showed that while the smartphone touch screen interface was still superior in most categories, TS still received a very positive response and even preferred over the smartphone. Overall, we can see that given a stable skin event detection method, the TS would exhibit reasonable input performance with satisfactory usability.

4 Discussion and Conclusion

In this poster, we propose “Touch Skin (TS)” that enlarges the interaction space of the smart watch using the hand (or skin) surface and proprioceptive sense. While the input interface is displayed on the small smart watch screen, the interaction is carried out by touching on the larger hand surface to which the input interface elements (e.g. graphical buttons and keys) are mapped. The pilot enactment study showed promising results of input performance and usability competent to that of the smart phone. We plan to implement and mount a finger detection sensor on the smart watch and conduct a more formal and full-fledged usability experiment to further validate our approach.

Fig. 2.
figure 2

Top: performance results between TS palm, TS finger and smartphone: task completion time (left) and accuracy (right). Bottom: usability survey results.

Full size image