1 Introduction

Smart hand-held devices nowadays come in many different sizes: from smart phones (10–15 cm in diagonal length) to larger tablet devices and hand-held computers (18–25 cm). There have been increasing interests in improving the user experience on hand-held devices by providing richer and more dynamic feedback through vibro-tactile actuators [6, 11, 12]. To avoid resorting to vibro-tactile arrays which would require a sizable and continuous contact area with the hand or body parts, researchers have proposed to apply tactile illusion techniques so that similar effects can be realized with only a few number of vibration motors [6, 11]. In addition, Miyazaki has demonstrated that tactile illusory feedback could be elicited at the “out of the body” space [8]. Kim has applied the “out of the body” tactile illusory feedback for a small-sized smartphone (9.4 cm in diagonal length), and demonstrated the illusory feedback was indeed applicable and extendable for interaction in 2D screen space using only four vibrators [6].

In this paper, we investigate whether the “out of body” tactile illusion can be extended or applied also to larger hand-held devices such as tablet devices. The extension is not expected to be straightforward for few reasons. First the original “out of body” tactile illusion was only validated when the body parts (e.g. fingertips) were stimulated directly and set apart by 7–8 cm span [6], similar to when the phenomenon was originally discovered [8]. With tablet devices, the operating conditions are quite different (see Fig. 1(c) and (d)). For one, the vibratory stimulations are indirect, and the hands/fingers may not be in equidistance from the actuators. No relevant study has been reported for when the size of the hand-held object goes larger than the “8 cm” span, not to mention for the 2D extension.

Fig. 1.
figure 1

The concept of “out of body” tactile experience from a hand-held object. (a) When vibratory stimulations are given to the fingertips properly, phantom tactile sensations can be felt as if occurring from the middle of the hand-held object [7]. The 1D tactile illusion is extended to 2D for (b) a small smart phone where fingers are in direct contact with the vibration motors [6], (c) a small tablet (e.g. iPad mini), and (d) a larger tablet (e.g. iPad).

Full size image

Thus, to find out how to apply the “out of body” tactile illusion to tablet devices, we ran a series of pilot exploratory experiments to first observe the effects of the tactile illusion e.g. with regards to various operational conditions such as the size of the device, holding position, minimal vibration amplitudes and effects of accompanying visual feedback for a fair perception of the illusion. The findings and observations would form the basis for designing a proper 2D tactile illusory rendering method for the tablet device. We shortly describe these experiments, report the newly derived guidelines, propose for a 2D tactile illusory rendering method for a tablet, and empirically validate its effectiveness.

2 Related Work

Funneling is one of the major perceptual illusion techniques specifically for vibro-tactile feedback (along with saltation which we do not consider in this work for now). It refers to stimulating the skin at two different locations simultaneously with different amplitudes and eliciting phantom sensations in between the two (i.e. 1D phenomenon) [1, 3]. The intended location of the phantom sensation (ITL) can be changed by modulating the stimulation intensity.

Several researchers have applied this phenomenon to human interfaces, experimenting with different ways of modulating the vibration amplitudes for detailed controlling of the target phantom sensation locations [1, 2, 5, 9, 10]. For example, the SemFeel was a vibro-tactile feedback system for a mobile touch screen device [11] using five vibrators attached to the backside of the device (4 corners and 1 center). It was capable of producing 10 different perceptible vibration patterns for expressing richer semantic information compared to the usual single vibrator scheme.

Recently, researchers have discovered such phantom sensations can be extended to the “out of body” [8] (see Fig. 1(a)), thus making it possible to generate phantom tactile sensations as if coming from an external object. This result is appealing in its application possibility to mobile devices. That is, there now is a potential method to make a user feel (phantom) sensations indirectly emanating from the hand-held device (e.g. middle of the display), but supplying actual vibrations only to the “natural” holding finger/hand locations (Fig. 1). Kim demonstrated that the aforementioned phenomenon could be extended to 2D by employing vibrators both horizontally and vertically on a small sized (3.5 inch screen) smart phone [6]. But no general guidelines have been reported for further applications to a variety of today’s hand-held media devices.

Finally, many human computer interfaces also rely on cross modal illusions to enrich user experience by fusing feedbacks of different modalities. For instance, a mere single vibrator based tactile feedback (as adopted by most current smart phones), when combined with other modalities, can induce phantom directional haptic feedback and rich user experience [12]. We posit that the tactile illusion feedback, when combined with other usual modalities such as the visual and aural, can similarly improve the interaction experience even further.

Thus, to summarize, the distinguishing point of our work is that we seek to newly extend funneling to an enlarged 2D space, and identify important design guidelines. This would open doors for a more flexible design of vibro-tactile feedback techniques for a wide variety of hand-held devices.

3 Pilot Experiments: “Out of Body” Funneling Effects Along Longer 1D Separation Span

Due to lack of space, we only provide a very short description of how the exploratory pilot experiments were run. Funneling stimulations were given through vibrators attached to an iPad mini and iPad, along three horizontal (along the top/middle/bottom row) or vertical directions (along the left/middle/right column) using two vibrators attached at the respective ends (i.e. six vibrators used for each case). Sixteen participants were asked to report the existence of any illusory tactile perception and indicate where they came from, if any (along the respective row or column). Various rendering parameters were changed and results observed.

3.1 Perception Resolution and Extent

Our data showed the smallest perceptible linear interval size to be comparable to that of the smart phone case (~1.6 cm) [6], “regardless of the size of the device (or distance between the vibration motors),” but only so in the middle part (40–60 % of the bands). The perception resolution was notably lower (or discernable grid size “stretched”, e.g. to as long as 7 cm depending on where the device was held) near the horizontal or vertical ends (see Fig. 2). This was more evident in the vertical direction, because the hands were holding the device along the horizontal direction.

It seems the increased size of the devices and the fact that the hands/fingers are not in direct contact with the vibration motors somehow affect the “out of body” illusory perception near the holding positions. Based on this finding, we conjectured that for 2D situation, the perception resolution pattern and extent would look like what are depicted in Fig. 2 (dotted lines).

Fig. 2.
figure 2

Perceptible grid sizes in horizontal and vertical directions (a) smart phone: ~1.6 cm [6] (b) iPad mini: 2–3.7 cm, (c) iPad (held in the middle): 1.5–4.5 cm, (d) iPad (held in the lower part): 1.3–7 cm. The grids and the dotted area on the iPad mini and iPad show the “projected” effective perception area in 2D. Shaded regions represent the “stretched” grids.

Full size image

3.2 Tactile Rendering Function and Interpolated Stimulation Strength

In the pilot experiment described in Sect. 3.1, we initially tested the three most popular and arguably reliable (in terms of eliciting the phantom sensation from the intended location) stimulation rendering methods for funneling, namely, interpolating the two vibration amplitudes using linear, logarithmic, and tangent functions [1]. Among the three, the tangent based interpolation showed the widest perception band (e.g. 6 intervals vertically and 9 horizontally on an iPad as shown in Fig. 2). However, in the later main experiment, this finding did not carry over to the 2D case. The responses were inaccurate and irregular. This was due to the fact that when tangent-based interpolation was applied in 2D, the interpolated amplitudes at the middle region became relatively too low (compare the two interpolating functions in Fig. 3). Thus, the logarithmic interpolation was chosen to render the tactile vibration amplitude with slightly less effective perception resolution (5 × 7) but with higher accuracy and response regularity.

3.3 Hand Location

Finally, the finger/hand grabbing positions were varied between the middle and lower part on the sides of the tablets. When the users held the lower part of the tablets, the middle effective perception region also shifted in the lower direction and toward the user (see Fig. 2). In the main experiment, we asked the participants to hold the middle part of the device.

Fig. 3.
figure 3

The logarithmic (dotted) and tangent based (full) stimulation interpolation functions (left) and the user’s responses (right). The tangent-based interpolation stimulation produces a near-linear user response (black).

Full size image

4 Main Experiment: Extending “Out of Body” Funneling to Tablets

4.1 Experimental Design and Set up

In the main experiment, we used four single vibrators attached in the four rear corners of the iPad (see Fig. 1). We used the same rendering algorithm as in Kim’s study for the smart phone [6] but with the few parameters adjusted according to our pilot experiment findings described in the previous section. The main purpose of the experiment was (1) to assess the illusory tactile perception behavior and effectiveness of the newly adjusted rendering algorithm with the larger tablet device and (2) how combining the illusory tactile feedback with the visual would further improve the accuracy and user experience. Two cases of 2D illusory tactile renderings were compared: without any visual feedback (NV), and with visual feedback (VF).

For NV, users were given 2D illusory tactile stimulations on an iPad held in the middle part (e.g. as shown in Fig. 1(d)) at the resolution of 5 × 7 (conservatively chosen based on the results of the pilot experiment). Participants were asked to pinpoint (using a stylus pen) where they felt the tactile feedback to be coming from.

For VF, when the stimulation was given, 5 visual icons (matching feedback), only one of them corresponding to the true intended target location (ITL), were shown and the user had to choose the correct one.

Vibrators were controlled by a voltage input using a PWM signal with an amplitude between 0 to 5 V, which in turn produced vibrations with frequency between 0 and 250 Hz and associated amplitudes between 0 to 2G (measured in acceleration, or 0 to 18 μm in position) respectively.

4.2 Detailed Procedure

Seven paid participants (4 men and 3 women) participated in the experiment with the mean age of 24.7. After collecting one’s basic background information, the participant was briefed about the purpose of the experiment and instructions for the experimental task. A short training was given for the participant to get familiarized to the experimental process. The participants wore ear muffs to prevent any bias from the sounds of the vibration. For both NV and VF conditions, the funneling stimulation was given by activating 4 vibrators simultaneously with logarithmically interpolated amplitudes according to the intended target location of phantom sensation (Fig. 4).

Fig. 4.
figure 4

Funneling stimulations for the 5 × 7 grid on an iPad. Amplitudes of four vibrators in the corners are logarithmically interpolated to produce an illusory sensation in the 2D screen space.

Full size image

5 Results

5.1 NV Experiment

ANOVA was applied for analyzing the perception resolution illustrated in the horizontal and vertical directions separately for simplicity. Figure 5 shows the cases for the first row and column where 5 and 3 distinct intervals were perceived respectively. Other rows and columns showed similar statistics. This lead us to project that the 2D perception resolution to be about only 3 × 5, lower than the expected (Fig. 2). The graphs in Fig. 5 show a similar pattern of lowered resolution at the periphery of the screen, i.e. elongated/enlarged grid size (e.g. ITLs at 1, 2 and 6, 7 in Fig. 5(a) and 1, 2 and 4, 5 in 5(b)). Also note that the user response was not linear as desired, an artifact of resorting back to using the logarithmic interpolation and a possible interaction among the stimulations in 2D. Thus, we believe that better and linear responses can be obtained by employing vibration motors with larger amplitude capabilities and a “mixed” interpolation technique (e.g. logarithmic + tangent-based). Although the main purpose of NV experiment was observing user behavior for 2D response in 2D tactile feedback, correct responses for each ITL grid were recorded (Fig. 6).

5.2 VF Experiment

As for the case of VF, Fig. 6 shows the percentages of correct response for each ITL grid with an average of 78.6 % (right part of Fig. 6). Note that with the user having to select from five possible responses, the pure chance performance was only 20 %. Thus the average of near 80 % is considered a psychophysically meaningful response [4, 7]. While the performance is relatively lower in the middle upper and middle lower region (denoted darker), we claim that with the accompanying matching visual feedback, the non-regular and distorted perception pattern was much rectified and resolution improved as well (compare the left and right parts of Fig. 6). In both NV and VF treatments, participants were surveyed about the localizability of the feedback, its clarity, task difficulty, and one’s confidences in their responses on a 7-Likert scale. In all cases, the VF condition exhibited much more positive responses.

Fig. 5.
figure 5

Average percentage of correct responses in the horizontal (left) and vertical directions (right). The graphs show the same pattern of lowered resolution at the periphery of the screen (horizontal ITLs at 1, 2 and 6, 7 and vertical ITLs at 1, 2 and 4, 5).

Full size image
Fig. 6.
figure 6

Overall performance of the NV (left) and VF (right) experiments. The percentages of the correct response are indicated in the grid.

Full size image

6 Conclusion and Future Work

In this paper, we carried out pilot experiments to explore how to extend 2D illusory tactile rendering to tablet devices. Tablet devices exceed in length for which “out of body” tactile illusion is known to work, and the stimulations become more indirect and their interactions in 2D more complex. Based on the experimental findings, we have devised a 2D illusory tactile rendering algorithm and have found that for an iPad sized device, approximately 5 × 7 regular resolution (grid size = ~2.5 cm) of illusory perception was achievable, when a matching visual feedback was provided.

For future work, we will apply our findings to real world examples such as interactive videos and mobile games, and assess whether the UX can be improved by the proposed vibro-tactile rendering method. At the same time, we plan to investigate additional ways to improve the 2D sensation localization accuracy by adjusting various rendering parameters, combining with the aural modality and employing other vibro-tactile illusion techniques such as the saltation and apparent motion.