“Wow! I Have Six Fingers!”: Would You Accept Structural Changes of Your Hand in VR?

1. Introduction

With the development of virtual reality applications, the virtual representation of users (i.e., their avatar) has drawn a lot of interests in various communities, especially regarding its influence on user perception in VR. Avatar representation can affect the sense of embodiment experienced by users, i.e., the senses that emerge when the avatar’s properties are processed as if they were the properties of the user’s own biological body (Kilteni et al., 2012a). Following the original Rubber Hand Illusion (RHI) experiment of Botvinick and Cohen (1998), a large number of researchers have been exploring users’ sense of ownership toward fake limbs and other body parts in order to understand under which conditions the brain accepts them as part of the users’ body [e.g., Tsakiris and Haggard (2005), Ehrsson et al. (2008), and Dummer et al. (2009)]. Virtual reality quickly became a powerful tool for such studies, as it allows to systematically alter and control the avatar representation and behavior in different ways. For instance, VR has been used to explore the effect of visuotactile and visuomotor stimulations on the sense of ownership (Kokkinara and Slater, 2014). It has also been used to understand how users’ self perception can be influenced by the representation of the avatar, such as the effects of altering realism (Petkova and Ehrsson, 2008), skin color (Peck et al., 2013), or shape (Piryankova et al., 2014), or how users accept controlling an additional appendage (Steptoe et al., 2013). However, as most of the research is performed on humanoid avatars, little is known about how users would experience embodiment on avatars whose structure differs from our body structure.

In this paper, we therefore explore how users perceive an avatar with structural differences compared to their own body by focusing on the specific case of a six-digit realistic-looking virtual hand (Figure 1, right). While such a rare configuration called polydactyly does exist in humans, the goal here is to explore how users would accept as their own a fully articulated six-digit virtual hand, while using their five-digit hand, in order to somehow evaluate the possibility of creating a Six-Finger Illusion. To explore this question, we conducted a VR experiment where participants controlled a six-digit realistic hand (one thumb and five fingers) from a first-person view while wearing an Oculus Rift Head-Mounted Display (Figure 1, left). In the experiment, twelve participants performed two tasks successively: (1) a self-manipulation task inducing visuomotor feedback, where participants mimicked finger movements presented in the virtual scene, and then (2) a visuotactile task inspired by RHI protocols, where an experimenter stroked the hand of the user with a brush. The real and virtual brushes were synchronously stroking the participants’ real and virtual hand, and in the case when the virtual brush was stroking the additional virtual digit the real ring finger was also synchronously stroked to provide consistent tactile stimulation and elicit a sense of embodiment. By conducting these experiments, we were interested in participants’ senses of ownership (i.e., the impression that the virtual hand was the source of the sensation) and agency (i.e., the impression to be able to control the actions of the virtual hand), which were assessed through indirect measurements (motor response to lifting the different virtual digits) and questionnaires answered by participants after each task. We also conducted the same tasks on a second group of twelve other participants where the additional finger was present but remained rigid and was not animated or stroked. We expected participants to demonstrate lower levels of ownership and agency in the rigid finger condition, and that the difference in agency and ownership ratings with the animated case would demonstrate if participants indeed considered the animated additional finger to be part of their hand. Interestingly, results of the experiment show that participants did experience high levels of ownership and agency of the virtual hand as a whole in both conditions. In addition, we also found higher levels of ownership and agency for the additional finger when the hand was fully animated, compared to the condition where the additional digit was rigid, and that participants displayed voluntary motor responses to control the additional virtual finger. Through the presented experiment, we found that participants responded positively to the possibility of controlling the six-digit hand despite the structural difference, and perceived the virtual hand and individual digits as their own to some extent.

The remainder of the paper is organized as follows. Section 2 provides an overview of related work on embodiment and body ownership illusions in VR. Then, Section 3 presents the methods and protocols of the experiment, and details the experimental results. Finally, Section 4 discusses the main findings, and Section 5 provides the concluding remarks.

2. Related Work

2.1. Embodiment

Avatars, as virtual representations of the user’s body in virtual reality applications, can elicit in users a sense of embodiment (Spanlang et al., 2014). This complex phenomenon is achieved on different levels, as defined by Longo et al. (2008) and Kilteni et al. (2012a): the phenomenology includes the sense of self-location, the sense of agency, and the sense of body ownership. This section provides a broad overview of the phenomenon of embodiment, but we refer readers to the works of Kilteni et al. (2012a) and De Vignemont (2011) for more comprehensive reading.

2.1.1. Self-Location

Self-location can be defined as the space in which we perceive the self to be located. The body space provides a reference frame for our physical body and determines the space in which body sensations are registered (De Vignemont, 2011). Several factors can affect the sense of self-location. For instance, collocation between the virtual and the real body (first-person perspective) has been shown to elicit a stronger sense of self-location than non-collocated perspectives (third-person point of view) (Slater et al., 2010; Petkova et al., 2011). Synchronous visuo-proprioceptive correlations during passive or active movements also increase the sense of self-location. The well-known Rubber Hand Illusion (RHI) experiment (Botvinick and Cohen, 1998) showed that self-location can be altered when synchronous visuo-proprioceptive correlations are applied between the rubber hand and the hidden real hand, which has been further validated by a number of additional experiments [e.g., Dummer et al. (2009)]. Finally, vestibular sensory information has been shown to play an important role in the sense of self-location (Blanke and Metzinger, 2009).

2.1.2. Agency

The sense of agency is elicited when oneself is perceived as the agent of one’s own actions. When interacting with our body, we have accurate control of the motor activity and we are aware of our actions (e.g., proprioception). Agency is described as motor activity control, which encompasses the obedience of the concerned body part to one’s will and the sensation of movement (Blanke and Metzinger, 2009). In other words, agency is closely related to action awareness and action planning (De Vignemont, 2011).

The sense of agency is present in the use of tools (effectors), for which the knowledge of sensorimotor control and the association between effectors lead to an expected outcome. The close relationship between intention and outcome is also considered as a component of agency (David et al., 2008; Caspar et al., 2015). When controlling virtual limbs of full-body avatars, the sense of agency has little impact on the effector. For example, the sense of agency was demonstrated to be kept even when avatars are not realistic [such as point-light avatars (Wellerdiek et al., 2013)] or differ from the user’s morphology (Kilteni et al., 2012b; Linkenauger et al., 2013).

Although, motor recalibration is required when the effector (e.g., tool, virtual body) differs with respect to the real body, a degree of visuomotor adaptation is tolerated in forms of proprioceptive recalibration, motor learning, or virtual space recalibration (Clower and Boussaoud, 2000). Nevertheless, the perceptual-motor fidelity between individuals and their avatars must be ensured.

2.1.3. Ownership

The sense of ownership is described as the feeling of owning one’s body (or an artificial body) (Longo et al., 2008) and experiencing that the body (real or artificial) is the source of sensations (Tsakiris et al., 2006). Since Botvinick and Cohen (1998)’s RHI experiment, it has been proved that a fake limb can elicit the sense of ownership and that the brain believes the fake limb to belong to the body (Pavani et al., 2000; Tsakiris and Haggard, 2005; Kalckert and Ehrsson, 2014). Still, a basic morphological similarity or spatial configuration between the real and artificial body is required (Kilteni et al., 2012a).

Ownership can be observed when the fake body is threatened (De Vignemont, 2011), such as the strong user reaction observed when hitting the rubber hand in the RHI experiment (Botvinick and Cohen, 1998). Although one might argue that the effect could be related to the surprise effect, neuroscience studies proved that the reaction induced by threatening the rubber hand (which is to retract one’s own hand in a vast majority of cases) is not only a pure reflex but also a cortical anxiety response to a danger perceived toward the body (Blanke, 2012).

While it is often assumed that the RHI is constrained by our structural body model (e.g., that it is not possible to implement supernumerary limbs), recent studies reported illusory duplication of the right hand in subjects exposed to two adjacent rubber hands (Ehrsson, 2009; Guterstam et al., 2011), which was found to be only elicited when the two rubber hands were at the same distance from the subjects real hand (Folegatti et al., 2012).

2.2. Ownership Illusion in VR

With the development of VR devices and applications, ownership illusions have been revisited in a number of experiments, demonstrating that users also respond to threats on their virtual hand in immersive VR setups (Yuan and Steed, 2010), or exploring virtual arm (Slater et al., 2008), leg (Kokkinara and Slater, 2014), or full body (Slater et al., 2010) illusions. Similar to the RHI, visuotactile stimulations between the avatar and the real hand should be synchronous. However, studies performed in VR setups (Petkova and Ehrsson, 2009) show that the virtual arm illusion can be achieved without tactile stimuli. More precisely, Sanchez-Vives et al. (2010) showed that synchrony between visual and proprioceptive information along with motor activity is able to induce an illusion of ownership over a virtual arm.

Such experiments pave the way for further studies to explore how changes in the virtual avatar influence the sense of ownership. For instance, Kilteni et al. (2012b) investigated body space and limb plausibility and found that participants tolerated having a virtual arm longer than their real one, while Argelaguet et al. (2016) found that a realistic virtual hand elicited a stronger sense of ownership than more abstract representations. Other studies have also explored one’s body weight perception by altering the complexity of the avatar (Piryankova et al., 2014), the effects of social anxiety responses to standing in front of an audience when having an invisible body (Guterstam et al., 2015), or how racial bias can be reduced by using an avatar of a black person (Peck et al., 2013).

Although there is evidence that the virtual representation of the user has an impact on the sense of ownership (Peck et al., 2013; Piryankova et al., 2014; Guterstam et al., 2015), and that right hand duplication illusions were reported in recent studies (Ehrsson, 2009; Guterstam et al., 2011; Folegatti et al., 2012), evaluating the sense of ownership when controlling humanoid avatars whose structure differs from our own has rarely been explored. A large body of research focuses on structural changes due to a lost limb [e.g., Ramachandran et al. (1995), Ehrsson et al. (2008), and Giummarra et al. (2010)], which is different from the kind of structural changes explored in this paper as the user experiences sensations associated with the lost limb. In comparison, we are interested in structural differences never experienced before. More relevant to this paper, other work related to the concept of Homuncular Flexibility introduced by Lanier (2006) showed a high potential for users to control strange and different bodies, such as controlling a three-arm character (Won et al., 2015) or mapping real-time human motion input onto non-human characters (Seol et al., 2013). However, none of these studies investigated users’ sense of ownership or agency when controlling such avatars. Recent work also demonstrated the capacity to control a virtual movable tail attached to the user’s avatar (Steptoe et al., 2013), with synchronous tail control providing a higher degree of body ownership and agency than asynchronous control despite the extra-human form. However, the additional appendage’ control was limited to simple pelvic sways and matched only five possible configurations. This paper therefore presents original insights by evaluating senses of ownership and agency in the specific case of a six-digit realistic virtual hand.

3. Experiment

In this experiment, our goal is to explore participants’ senses of ownership and agency when using a six-digit virtual hand consisting of one thumb and five fingers, and to somehow evaluate the possibility of creating a Six-Finger Illusion (SFI). As RHI experiments usually focus on visuotactile (e.g., synchronized strokes) or visuomotor (e.g., synchronized motion) stimulations, we designed two tasks in order to evaluate if participants accepted differently structural changes of the hand depending on the type of stimulation. In the first task, participants manipulated the virtual hand while mimicking finger movements presented in the virtual scene (finger abduction and adduction, opening/closing fist or drumming). The second task was inspired by the RHI experiment, where an experimenter stroked the participants’ real fingers with a brush while a synchronous virtual stimulus was presented in the virtual scene. Following each task, we collected both indirect and direct measures of participants’ senses of ownership and agency toward the six-digit virtual hand. We were first interested in the motor response of participants to the instruction of moving the virtual digits, and recorded for each virtual digit if a real finger was actually moved, and which real finger was moved when it was the case. We then assessed participants’ senses directly through questionnaires for each digit individually, as well as through a more general final questionnaire.

3.1. Apparatus and Participants

Participants were immersed in the virtual environment using an Oculus Rift DK2 Head-Mounted Display, in which head tracking was provided by the Oculus Rift and participants’ right hand was tracked using a Leap Motion. The physical setup was conceived to ensure optimal tracking conditions for the Leap Motion, which was positioned upside down. The shelf board was covered with anti-reflective tape to remove infra-red interferences (see Figure 1, left). Participants were asked to use their right hand when interacting with the virtual scene and to keep their left hand away from the field of view of the Leap Motion to avoid detection artifacts. A virtual shelf matching the dimensions of the physical one was displayed in a virtual office environment used for the experiment (see Figure 1, middle), providing both a reference frame and passive haptic feedback when touching the bottom board. For the visuotactile task, a real brush was attached to a Razer Hydra and used to synchronously drive the virtual brush. The VR application was developed using Unity and ran at an average frame rate of 75 fps.

Twenty-four male participants from the university campus took part in the experiment (Age: min = 21, max = 40, and avg = 28 ± 4), recruited both among general students and staff. Participants were recruited asking for minimal previous experience in VR: 12 subjects had none to very limited previous experience with virtual reality, 10 had some previous experience, and only 2 were familiar with VR. None of the participants knew about the experiment being tested, or that they would be presented with a six-digit virtual hand model. All participants were right-handed Caucasian males, to match the visual appearance of the virtual hand as much as possible.

3.2. Six-Digit Virtual Hand

For the purpose of this experiment, a professional artist was commissioned to create a realistic six-digit hand model (Figure 1, right), inspired from the realistic hand model provided with the Leap Motion SDK. Each digit was driven by three bones: respectively two degrees of freedom (Metacarpophalangeal joint) and one DoF (Proximal and Distal Interphalangeal joints) for the bones of the fingers, and three DoF (Carpometacarpal joint) and one DoF (Metacarpophalangeal and Interphalangeal joints) for the thumb. A realistic texture (4096 × 4096 pixels) was also created by the artist, based on the realistic texture of the Leap Motion SDK model. In the experiment, we also measured the length of the real hand of each participant and uniformly scaled the virtual hand model accordingly.

For the sake of clarity, virtual digits will be referred to as D₀ to D₅ while real digits will be referred to using their common names (Table 1), and D₄ is the additional virtual finger. For this experiment, we positioned D₄ between the ring and pinky fingers, as the thumb, index, and middle fingers are often used more independently than other digits for manipulation tasks (the limitations of this choice are further discussed in the discussion). Animations of the six-digit virtual hand were driven in real-time by the Leap Motion using directly the information provided by the device, except in the case of D₄ for which we linearly interpolated the joint angles of D₃ and D₅ using the following equation:

where q(D_i) represents the joint angle configuration of digit D_i, and k ∈ [0, 1] represents how much digits D₃ and D₅, respectively, influence digit D₄. Based on a short pilot study, we used a value of k = 0.6 in our experiment, which provided the most natural looking animations of the six-digit hand as a whole. This setup provided motions where D₄ was slightly more influenced by the tracking of the ring finger than by the pinky finger. These choices in the position and animation of D₄ resulted in animations visually more consistent with real human hand anatomy, where independent movements of the ring and pinky fingers are more difficult to achieve than for other fingers due to the complex anatomical structure of the hand (Lang and Schieber, 2004), which we therefore also observed with the group of digits D₃, D₄, and D₅.

3.3. Experimental Protocol

After signing an informed consent form, participants were briefed about the experiment: they were told that we were studying how people accept avatars as their own in virtual reality, but were not informed that they would be presented with a six-digit virtual hand model. The experiment was divided into two blocks, where we evaluated ownership and agency of each virtual digit independently through subjective questionnaires and motor measurements after each block. In the first block, our goal was to evaluate how visuomotor stimulation (i.e., hand manipulations) influenced the senses of ownership and agency of the six-digit hand. The second block was designed to evaluate the influence of visuotactile stimulation. While our goal in this experiment was to explore the possibility of somehow creating a Six-Finger Illusion, we did not intend to evaluate the specific contribution of each stimulation to ownership and agency and did not counterbalance the order of the two blocks. Instead, we chose the following progression in the experiment: participants start with standardized hand motions (visuomotor stimulation) to get used to the experimental setup, then we perform the visuotactile stimulation as participants would already be familiar with the virtual hand. Studying the contribution of each effect is a complex research question on its own which could be further studied in future work, and which would require specific calibrations and protocols. In each block, we collected both indirect (motor responses of participants to the instruction of moving the virtual digits) and direct (subjective questionnaires) measures of participants’ senses of ownership and agency toward the virtual hand and its six digits. The whole experiment including the subjective questionnaires was conducted in the virtual scene to avoid breaking immersion, except for the final questionnaire. The experiment lasted around 30 min in total (15 min per block).

A total of twenty-four participants took part in the experiment. Twelve participants performed the experiment with the additional virtual finger animated with the method previously mentioned (Animated condition). To validate the hypothesis that the additional virtual finger did elicit a sense of embodiment in participants, a second group of twelve other participants performed the same experiment while the additional virtual digit D₄ was not animated (Rigid condition). In this condition, the digit D₄ was non-responsive and did not follow the users’ control, as if it was a dead and rigid finger fixed in relation to the hand (Figure 2, right). In this second condition, we expected participants to display lower embodiment ratings. Differences with the animated case would possibly validate that participants considered the additional finger to be more part of their hand when fully animated.

3.3.1. Visuomotor Stimulation

In this block, participants first performed a self-manipulation task. In the virtual environment, they were presented with video examples of hand manipulations displayed on a non-realistic hand model to avoid presenting a realistic five-digit hand. Hand manipulations were performing either (i) finger abduction and adduction, (ii) opening/closing fist (i.e., flexion/extension), or (iii) drumming fingers (see Video S1 in Supplementary Material). Each video example was presented for 10 s, after which participants were asked to mimic for 15 s the movement they just saw. These examples were presented in the virtual reality scene on a virtual TV screen facing the participants, and were performed a total of three times each.

After the task, we measured the motor response of participants to the instruction of moving each of the virtual digits. Each virtual digit was selected in random order and highlighted using a virtual arrow. We then blocked the view of participants to prevent visual feedback, and asked them to lift the virtual digit while recording their finger movements. Each digit was evaluated three times. We recorded if a real finger was actually moved, and which real finger was moved when it was the case.

In addition, participants were asked to assess their subjective impressions for each digit separately by answering questions related to their sense of ownership and agency for each digit. As previously, each digit was selected in a random order and pointed to with a virtual arrow. Participants were then presented with the two following questions on a virtual LCD screen in the virtual scene: (Q₁) I feel I can move the pointed virtual finger if I want to and (Q₂) I feel that the pointed virtual finger belongs to my hand. Participants gave verbally their answer on a 7-point Likert Scale (1: fully disagree, 7: fully agree), which the experimenter recorded using the keyboard. These two questions were selected to evaluate respectively the senses of agency (Q₁) and of ownership (Q₂) toward each finger individually.

3.3.2. Visuotactile Stimulation

In the second block, we conducted a version of the RHI experiment in VR, adapted to our six-digit virtual hand setup. Participants were asked to position their hand flat on the shelf board and instructed that the experimenter would stroke their thumb and fingers with a soft brush (Figure 3). Each virtual digit was stroked from the middle of the top of the hand to the corresponding fingertip. When the additional digit D₄ was stroked, we matched the physical real stroke on the ring finger to provide consistent tactile stimulation and elicit senses of embodiment. Digits to stroke were selected in random order by the experimental application, while ensuring that digits D₃ and D₄ were never stroked consecutively. Each digit was stroked a total of five times. In the Rigid condition, the additional digit D₄ was never stroked in order to simulate the impression that the additional digit did not provide any tactile feedback (as if the digit was dead).

After the visuotactile task, we measured again the motor response of participants to the instruction of moving each of the virtual digits the same way that in the first block. As previously, a virtual digit was randomly selected and highlighted, and participants were asked to lift the digit. Each digit was evaluated three times. We recorded if a real finger was actually moved, and which real finger was moved when it was the case. The same questionnaire as in the previous block was then presented for each digit, in a random order, to assess participants’ sense of ownership and agency for each digit.

3.3.3. Final Questionnaire

At the end of the experiment, and after the experimenter removed the head-mounted display, participants were asked to fill a subjective paper questionnaire to evaluate their overall sense of embodiment of the six-digit hand, in relation to ownership, agency, and self-location (Table 3). Items of the questionnaire were adapted to our specific experiment based on questionnaires used in previous work [e.g., Botvinick and Cohen (1998), Longo et al. (2008), and Kalckert and Ehrsson (2014)].

3.4. Results

While analyzing results, we test for statistically significant differences in participant responses. We are interested in both main effects (i.e., when a variable has an overall effect) and interaction effects (i.e., when the effect of a variable differs depending on the level of one or more of the other variables), and therefore conducted mixed-effect Repeated Measures Analysis of Variance (ANOVA). When dealing with non-parametric data, we first applied Aligned Rank Transform on the data, which allows the usage of ANOVA and to analyze interaction effects (Wobbrock et al., 2011). When main or interaction effects are found, we report the cause of these effects using Neuman–Keuls post hoc tests for pair-wise comparisons of means. For the results presented in this paper, we only consider effects to be significant at the 95% level (i.e., p < 0.05).

3.4.1. Controlling Virtual Digits

To understand participants’ strategies to control the virtual digits, we first looked at how participants behaved when asked to lift a virtual digit in the experiment, and which real digit they lifted when they lifted one. Results are summarized in Table 2. As expected, results show that the thumb and the index, middle, ring, and pinky fingers were consistently lifted by participants when the corresponding virtual digit was pointed to, in both tasks and for both groups of participants (all on average >86%). The main difference in the results between Animated and Rigid groups appears when looking at participants’ behavior when the additional D₄ virtual digit was pointed to. In the Animated group, results show that participants consistently lifted a real finger (ring or pinky) in 100% of cases in the visuomotor task, and in 94% of the cases in the visuotactile task. In comparison, participants in the Rigid group did not lift any finger in 53% of the cases in the visuomotor task, and in 69% of the cases in the visuotactile task.

To further analyze participants’ voluntary responses regarding controlling the additional digit D₄, we extracted the percentage of times when each participant moved or did not move a real finger when asked to lift D₄. Our goal is to understand if they consistently decided to move a real finger, independently of which finger they decided to move, which would demonstrate that they felt that the additional virtual digit was part of their virtual hand. We conducted a two-way mixed-effect Repeated Measures ANOVA on this data, with within-subject factors Task (visuomotor vs. visuotactile) and Motion (percentage of times a real finger was moved vs. no real finger was moved), and between-subject factor Group (Animated vs. Rigid). We found an interaction effect between Motion and Group (F_1,22 = 13.922, p < 0.005), displayed in Figure 4, where post hoc analysis confirmed that participants moved a real finger significantly more often in the Animated group than in the Rigid group.

These results demonstrate that participants of the Animated group created a voluntary motor response using a real finger to the demand of moving the additional virtual digit, and therefore that they considered the additional finger of the animated six-digit hand to be controlled by their own hand’s movements.

3.4.2. Questionnaires

As previously mentioned, participants were asked after each task to assess their subjective impressions for each digit separately by answering questions related to their sense of agency (Q₁) and ownership (Q₂). Participants were presented with the following questions: (Q₁) I feel I can move the pointed virtual finger if I want to and (Q₂) I feel that the pointed virtual finger belongs to my hand, and gave their answer on a 7-point Likert Scale (1: fully disagree, 7: fully agree). To analyze participants’ senses of agency and ownership, we therefore transformed this non-parametric data using ART (Wobbrock et al., 2011), then performed two two-way mixed-effect Repeated Measures ANOVA with within-subject factor Task (visuomotor vs. visuotactile) and Digit (D₀–D₅), and between-subject factor Group, respectively, on the results of questions Q₁ and Q₂.

Regarding ratings related to the sense of agency, we found a main effect of Digit (F_5,110 = 25.84, p < 0.001). Post hoc analysis showed that on average agency ratings for D₃ were significantly lower than D₀, D₁, D₂, and D₅ (all p < 0.05). In addition, D₄ ratings were significantly lower than all others (all p < 0.05), with average agency ratings of 3.6 for D₄. More interestingly, we also found an interaction effect between Group and Digit (F_5,110 = 5.47, p < 0.001), depicted in Figure 5 (left). Post hoc analysis showed that differences were mainly due to the fact that digits D₃ and D₄ were not significantly different from each other in the Animated group (but rated significantly lower than all other digits, with all p_s < 0.05), while they were significantly different from each other in the Rigid group (p < 0.05) with D₄ showing the lowest agency ratings on average 2.6, and D₃ not being rated significantly lower than the other digits. These results suggest that both D₃ and D₄ indeed elicit a sense of agency in the Animated group, even if on average lower than for other digits. This result also seems to be supported by the fact that agency ratings for D₄ were higher when animated than in the Rigid group (p < 0.05). We did not find any main or interaction effect with Task, which seems to indicate that participants’ sense of agency was consistent through the visuomotor or visuotactile conditions, although it is important to remind that we did not counterbalance the order of the task, as our intent was not to evaluate the specific contribution of the two types of stimulation.

Interestingly, we found the same main and interaction effects of Digit (F_5,110 = 18.42, p < 0.001) and Group × Digit (F_5,110 = 3.92, p < 0.01) for ratings of the sense of ownership (Figure 5, right). Similarly, than for agency ratings, these results suggest that D₃ and D₄ elicited a sense of ownership in the Animated group, even if on average to a lesser extent than for other digits, and that the additional digit D₄ did elicit a sense of ownership significantly stronger in the Animated group than in the Rigid group (p < 0.05).

3.4.3. Final Questionnaires

Post-experiment questionnaires were analyzed using the Friedman rank test and Wilcoxon pair-wise tests. We also conducted single t-tests against the mean value of 4 to evaluate if participants gave answers significantly different from the mean scale value. For the purpose of analyzing the results, we divided the set of questions between agency and ownership of the virtual hand as a whole (O₁, O₂, O₃, A₁, A₂, A₃), and ownership and location of the virtual fingers specifically (O₄, O₅, L₁, L₂, L₃, L₄). Before performing the analysis, data were tested for ordering effects, and none were significant. Details of the questionnaire and results are summarized in Table 3 and Figure 6.

By analyzing ratings of questions related to the sense of ownership and agency of the virtual hand as a whole, we observe that participants felt that the virtual hand was part of their body. Results show high levels of ownership (high ratings for O₁ and O₂ and corresponding low ratings for control question O₃) as well as of agency (high ratings for A₁, A₂, and A₃). In terms of sense of ownership of the virtual fingers specifically, ratings of O₄ show that participants displayed a neutral response across both groups to the question of whether they felt that all of the virtual fingers belonged to their hand, trending more toward a negative response for the participants in the Rigid condition (single t-test against mean value of 4 not significant). Results also show that participants identified one of the virtual finger as not being part of their hand (O₅) in both conditions, but to a significantly higher extent in the Rigid condition. However, an interesting result is that participants in both conditions indicated an overall high sense of ownership and agency with respect to the hand as a whole, even when they identified one of fingers as not being part of the hand. In the case when that extra digit was rigid and could not be moved by the participant, participants still retained a strong sense of agency and ownership over the hand as a whole, which we did not expect.

Finally, ratings related to the sense of location demonstrate high average ratings, where single t-tests were all significant (all ps < 0.01). These results therefore demonstrate that participants did perceive the strokes synchronously on the real and virtual fingers (L₁) and had the feelings that the virtual strokes were created by the observed virtual brush (L₂, L₃, and L₄).

4. Discussion

While numerous works are exploring embodiment, either through the use of fake limbs such as in RHI experiments or through virtual reality, little is actually known about how users perceive replicas with structural differences from the human body. In the case of virtual reality, such differences raise the question of how users accept these avatars as being their virtual replica (sense of body ownership), and also of how they perceive controlling these avatars (sense of agency).

4.1. Ownership and Agency

This paper provides preliminary positive insights to these questions through an experiment focusing on the specific case of controlling a six-digit realistic-looking virtual hand. From the results of our experiment, we demonstrated that using a virtual hand with structural differences can elicit in users the sense of both ownership and agency. First, qualitative measures demonstrated that participants voluntarily moved a real finger when asked to lift the additional virtual digit. As expected, this was, however, much less the case when the additional finger was not animated. These results suggest that participants accepted the additional virtual digit as being part of their hand. It also suggests that they responded positively to the possibility of fully controlling the six-digit hand despite the structural difference, and that we somehow managed to create a Six-Finger Illusion. Furthermore, we collected subjective ratings of ownership and agency at the level of both individual fingers or the whole virtual hand through two sets of questionnaires. Results demonstrated that participants experienced senses of ownership and agency for all the digits in the animated condition. In particular, these senses were significantly more potent in the case of the additional digit when it was animated than when it was not. What we did not expect was that participants would experience an overall high sense of ownership and agency with respect to the hand as a whole when the additional digit was not animated, even when they identified that one of the fingers was not part of the hand. This suggests a capacity to accept the whole hand despite the fact that a part of it was identified as not being part of their own hand. In comparison, other related studies where the control case is defined by asynchronous movement (rather than a lack of movement) usually present larger differences in terms of ownership [e.g., Sanchez-Vives et al. (2010), Kalckert and Ehrsson (2012), and Steptoe et al. (2013)].

We also tested the effects of visuomotor and visuotactile stimulations on the elicitation of ownership and agency, and did not find any differences in the qualitative and quantitative results from the experiments. This preliminary result suggests that structural differences might be accepted by participants whether using passive or active stimulation, although we did not counterbalance the tasks to account for ordering effects. Again, we were more interested in this experiment in exploring if a Six-Finger Illusion could be created rather than in evaluating the specific contribution of different types of stimulation to ownership and agency. Further in-depth studies would be required to explore the contribution of different types of stimulation to the acceptance of such structural differences.

4.2. Participants’ Feedback and Interpretation

In addition to questionnaires and finger motion tracking, we also recorded videos of most participants during the experiment. After going through audio recordings for ten participants in the animated group (two were not recorded for technical reasons), we identified some interesting feedback from their experience. We noted that participants did not necessarily notice (or reported) that the virtual hand was made of six fingers at the beginning of the experiment: only three participants mentioned it orally early in the experiment (within 15 s), while five participants did not notice it until after the visuomotor task was complete, usually at some point when they were asked to lift the virtual fingers (from 6 to 8 min after the experiment started). Two participants did not mention when they noticed it during the experiment. We also noted various types of reaction from participants, not necessary exclusive, such as surprise, laughs, or reported strangeness of the effect. We report here some relevant comments when noticing the additional virtual finger, to illustrate their reactions. P₄: There is a problem, I have six fingers! (then laughs). P₇: Is it normal that I see six fingers? P₁₀: I just noticed something… 1, 2, 3, 4, 5, 6. Why do I have six fingers? (laughs). P₁₁: Is it normal that I have more fingers? Is it? (1 min latter:) Wow it’s going to be something crazy with six fingers. Something cool. P₁₂: Wow, I did not notice before (laughs), I like the hand with six fingers. It took me a while to notice. Seriously, I’m surprised I did not notice before. It’s crazy.

Based on the results of the experiment and on participants’ comments from the audio recordings, it seems that participants liked the six-digit virtual hand and did accept the additional virtual finger as being part of their hand. Participants reported that as soon as (he) noticed the finger, (he) felt that it was right here (between the ring and pinky real fingers). In addition, four participants reported that for them the real ring finger was representing both the virtual D₃ and D₄ fingers, suggesting a sensation of duplication of this finger. Participants also appreciated the visuotactile stimulation, and reported that the brush was really impressive in terms of assimilation and incorporation, giving the impression that the real brush was stroking all the fingers (including the additional finger). All these comments demonstrate that participants were definitely interested, surprised, and reported positive feedback about the sensation perceived, even if we cannot support that the experimented perception is a real illusion. However, the fact that several participants reported comments suggesting sensations of “finger duplication” is highly positive for future research in the area.

4.3. Limitations and Future Work

In terms of hardware, we decided to use the Leap Motion for our experiment as it provides finger tracking without the need of wearing gloves or markers. While this device can be prone to errors in terms of animation, we believe that this non-invasive tracking system is still less likely to affect immersion, and therefore ownership, than other wearable systems. While designing our experiment, we therefore carefully took into consideration these possible device-dependent errors. This resulted in a selection of error-free movements for the visuomotor task (finger abduction and adduction, opening/closing fist, or drumming), with the palm facing down to avoid the visual artifact that the fist does not close as fully as a real hand would. For the visuotactile task, the participants’ hand was put flat on the shelf board and clearly recognized by the device. We believe that the positive results from our experiments demonstrate that users did accept the virtual hand and individual digits as their own using this device. Users’ sense of ownership should be even more potent with higher accuracy non-invasive devices, but those are still to come and more related to other fields of research.

To conduct this experiment, we commissioned a professional artist to create a realistic six-digit hand model. While the specific appearance and structure of the virtual hand might have influenced the results, our primary goal was to explore the possibility of somehow creating a Six-Finger Illusion. For this reason, we designed a plausible hand structure with six digits for users to control, and chose a specific position for the additional digit among several possibilities. We believe that placing the additional digit between the ring and pinky finger was one of the most plausible option for a first study, motivated by the fact that the thumb, index, and middle fingers are often used more independently than the other two digits. We therefore made the decision of having three fingers (D₃, D₄, and D₅) coupled together in terms of motion, while keeping the independence of the other digits. In the future, it would interesting to evaluate the effect of adding parts at other locations of the hand or body, or in contrary how users accept a virtual hand model with fewer digits (e.g., four fingers, as it is often the case for cartoon characters). Consistently with the selected location of the additional digit, we also chose to stroke the ring finger when the brush visually stroked the additional virtual digit. We considered that the pinky finger might have been easier to differentiate (because of its position on the end side the palm), and this choice was also more consistent with the weight of the ring finger compared to the pinky on the animation of D₄. Of course, these choices were specific to the protocol evaluated in this experiment, and other conditions could be tested in the future to explore the best approach for participants to experience the strongest visuotactile sensations.

We also chose to compare the results of the fully animated virtual hand with a virtual hand where the additional finger was rigid, but previously considered other conditions to compare to. For instance, we considered using a realistic virtual five finger hand but believed that this condition would have brought little information about how the additional digit was perceived in comparison in the six-digit hand condition, while simultaneously introducing a possible confounding effect because of the difference in shape. Similar to Steptoe et al. (2013), we also considered using unrealistic motions for the additional finger, but preferred a rigid condition to simulate the fact that the additional finger was physically part of the hand but not controllable. Of course, other factors than the animation of the additional finger could also be investigated in the future. For instance, factors related to the visual aspect of the hand (e.g., skin color, hairiness, shape, etc.) can also influence how users perceive a virtual hand to be their own [e.g., Haans et al. (2008)]. A tangible version of the six-digit hand could also be manufactured to conduct a Six-Finger version of the Rubber Hand Illusion and compare results with a setup closer to the original RHI experiment. To facilitate further research in this area, we would be ready to make the 3D model used in this experiment available on request for research purposes. Recent studies reported illusory duplication of the right hand in subjects exposed to two adjacent rubber hands (Ehrsson, 2009; Guterstam et al., 2011), which seem to suggest that virtual characters with supernumerary limbs might possibly be implemented. However, further studies would be required to validate that results obtained in the case of the six-digit virtual hand would generalize to larger structural differences.

Other interesting questions to investigate in the future are related to interacting with and controlling such supplementary parts. In the case of the six-digit virtual hand, tactile interactions were provided by the experimenter through the virtual brush, but different durations of stimulation may affect the sense of embodiment. Exploring how users accept these supplementary parts when touching them, or when interacting with them, would also bring interesting insights, especially regarding how they would perceive such extended virtual replicas of themselves. Concerning interactions with the virtual hand, RHI experiments often also evaluate participants’ reactions to a sudden action toward the rubber hand (i.e., reactions to a threat). Evaluating how participants react to such sudden actions on the virtual extra finger would be interesting, in order to measure an objective reflex response that should provide a measure of participants’ sense of embodiment. In order to measure ownership specifically toward the additional virtual finger, a challenge would, however, be to ensure that the sudden action would only target a response toward the additional virtual finger and not be perceived as a threat toward the whole hand. As such threats usually target larger body parts, it opens interesting future research questions for a more objective measure of embodiment than subjective questionnaires. Another challenge when animating virtual characters with such structural differences might be to manage to elicit the sensation of embodiment, while at the same time providing natural ways to control the additional body parts. In this paper, we implemented a simple strategy to control the virtual hand, which demonstrated its value in terms of ownership and agency ratings. Now that we know that users can experience senses of ownership and agency for additional body parts, we plan to design and compare various ways of controlling them, for instance, in order to identify the best control paradigms to elicit high sensations of embodiment. Manipulating control laws would also be relevant for avatars without structural differences, for instance, to study users’ adaptation capabilities while performing motor tasks (e.g., to learn to write with your left hand by using left-right inverted avatars), as well as for controlling robotic or prosthetic body parts (Hussain et al., 2016).

Overall, findings of this experiment are very promising for future applications, especially when the goal is to control virtual characters with structural differences. This is, for instance, relevant when users are meant to impersonate specific characters, such as for computer graphics, special effects or cartoon characters who often exhibit structural differences. On a larger scale, it also raises many psychological questions about the perception of our body identity, and how we perceive faithful or stylized virtual replicas of ourselves in virtual reality.

5. Conclusion

In this paper, we explored how users perceive controlling an avatar with structural differences compared to their own body by focusing on the specific case of a six-digit virtual hand. Through the presented experiment, we found that participants responded positively to the possibility of controlling the six-digit hand despite the structural difference, and accepted to some extent the virtual hand and individual digits as their own. These results bring some preliminary insights about how avatar with structural differences can produce senses of ownership and agency experienced by users.

Author Contributions

LH, FA, CN, and AL participated in the design of the experimental protocols and research questions. CN was in charge of implementation and of conducting the experiment. LH, FA, and AL were in charge of analyzing the results and writing the paper.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We wish to thank all the people who helped us in this project: Julian Joseph for his help in designing the experimental setup, Christophe Lacaux for the modeling and Polymorph for the rigging of the six-digit virtual hand, as well as Nathalie Lacaux and of course all our participants. We also wish to thank all the reviewers for their helpful comments.

Supplementary Material

The Supplementary Material for this article can be found online at https://www.frontiersin.org/article/10.3389/frobt.2016.00027

References