1 Introduction

Multi-sensory immersion in virtual reality (VR) is an effective method to induce body ownership illusion. It has been shown that illusory ownership of a virtual body could be induced by synchronized visual and motor stimulation of a virtual body and the unseen real body (Banakou et al. 2013; S. Shibuya, S. Unenaka, & Y. Ohki, 2018; Shokur et al. 2013; Slater, Spanlang, Sanchez-Vives, & Blanke, 2010; Wong et al. 2014). The neural correlates of ownership illusion are associated mainly with increased activity in the sensorimotor and premotor cortex. The sensorimotor cortex has a strong plasticity ability that enables dynamic adaption to different stimulation (Dykes 1997; Schaefer et al. 2007), so that the sensory information of a virtual body can be embedded into its representation of the body (Di Pino et al. 2014). The premotor neurons, on the other hand, are capable of responding to multimodal signals and integrating them into a multi-sensory representation of a virtual body (Blanke et al. 2015; Dykes 1997; Ehrsson et al. 2005, 2004; Graziano et al. 1994; Limanowski & Blankenburg 2016).

Previous neuroimaging research on body ownership illusion focused on examining task-dependent activities during the ongoing illusion process (Bekrater-Bodmann et al. 2014; Limanowski & Blankenburg 2016; Shokur et al. 2013). Nevertheless, few studies have explored resting-state functional connectivity patterns related to long-term body ownership illusion. Evidence from skill learning studies indicated that long-term training could induce widely distributed changes in brain activity associated with task performance, highlighting the reallocation of brain resources on the neural system level (Raichle 1998). Moreover, studies on resting-state brain function showed that skill learning changed the pattern of spontaneous cortical activity between different brain networks (Lewis et al. 2009). Since repetitive training on the rubber hand illusion could consistently elicit the improvement of illusion experience and maintenance of illusory ownership (Honma, Yoshiike, Ikeda, Kim, & Kuriyama, 2014; Lev-Ari, Hirschmann, Dyskin, Goldman, & Hirschmann, 2015), the long-time experience of illusory ownership should be regarded as a multi-sensory training and learning process, which would lead to training-induced changes in resting-state functional connection between brain regions.

The present study aimed to characterize changed patterns of resting-state functional connectivity after long-term experience of body ownership illusion. Specifically, we focused on evaluating the functional connectivity changes of the hippocampus, given that the hippocampus is a key structure of learning and new memory formation (Gould et al. 1999; Wirth et al. 2003). To achieve this, we first developed an apparatus based on real-time image matting and VR technology to induce the illusion of virtual hand ownership. As the appearance of the virtual hand affected the degree of ownership illusion, in the current study, the image of real hand captured by the RealSense camera was used as the virtual hand to maximize the construction of sense of virtual hand ownership, given that a recent study has demonstrated that the 3D image of real hand could induce the strongest rating of ownership illusion (Pyasik et al. 2020). Moreover, the movement of the virtual hand was synchronized with the real hand movement, thus both ownership and agency of the virtual hand were induced (Satoshi Shibuya, Satoshi Unenaka, & Yukari Ohki, 2018). Then, the functional imaging data were acquired using resting-state functional magnetic resonance imaging technology (rs-fMRI), immediately after participants underwent an 8-h experience of information input task in either virtual or real office environment. We developed a hypothesis-driven functional imaging data analysis aiming to investigate the changes in resting-state connectivity of the hippocampus after a long-term virtual hand ownership illusion.

2 Materials and methods

2.1 Participants

Twenty-five participants (12 females, 20 – 27 years old) were recruited in this study. All of the participants were right-handed, had normal or corrected to normal vision, and had no history of mental or neurological disorders or sensory disorders. Written informed consent was obtained from each participant. This study was approved by the ethics committee of the Beijing Institute of Technology.

2.2 Virtual office system

The virtual office system was developed using Unity 3D software (see (Guo et al. 2019) for a detailed description of the system) (Fig. 1). The hardware of the system mainly consisted of a desktop computer (Intel Core i7-6700 K, dual-core chips of NVIDIA GeForce GTX 1080Ti, 16 GB RAM) and a head-mounted display (HMD) (using HTC VIVE Pro, field angle, 110 degrees, resolution, 1440*1600 per eye, refresh rate, 90 fps). The virtual office system was a high-presence system that was designed carefully to make the environment as real as possible. It could support light office and home tasks such as e-mail, wording processing, browsing the web, and so on. During the breaks, users could walk around and enjoy the natural scenery out of the window to relax themselves. A monitor placed upon the window reminded users their spatial positions in the real environment (Fig. 2 A).

Fig. 1
figure 1

The virtual office system. The whole system consisted of a head-mounted display (HMD) (HTC Vive), two cameras, a keyboard, a mouse, and four 3-degree-of-freedom orientation trackers. A typing system implemented in the virtual office environment was used to induce virtual hand illusion. A participant wore an HMD while sitting in front of a physical table (third-person perspective)

Full size image
Fig. 2
figure 2

An illustration of virtual hand illusion. A The virtual office environment. B Participant saw a pair of virtual hands operating a virtual keyboard and mouse (first-person perspective). The movement, pose, and position of the virtual hands was synchronous with her own unseen hands. C The real environment consisting of a liquid crystal display and a physical keyboard and mouse. D Visualization of the experimental design in the study

Full size image

2.3 Virtual hand illusion induction

The virtual hand illusion was conducted via a typing system implemented in the virtual office environment (see (Jiang et al. 2018) for details). The typing system consisted of an HMD, two depth cameras, a keyboard, a mouse, and four trackers with 3-degree-of-freedom orientation (Fig. 2 B). The cameras were used to extract real hands and project them into the virtual environment in real time, so the generated virtual hands that participants saw were actually the real-time images of the real hands, and the movement of the virtual hands was synchronized with the real hands. Moreover, in our virtual reality input system, a tracker was tied to a real keyboard and mouse to acquire the real-time pose of the real keyboard/mouse. Then, a virtual keyboard and mouse were put in the same position as the real ones. So, when participants entered information in the virtual environment, they could not only see their virtual hand and the virtual keyboard/mouse, but also feel the haptic feedback from the real one. The real-time sensory feedback, together with the synchronous and congruent movement between the virtual hands and real hands, provided an effective approach to induce virtual hand ownership.

3 Experimental procedure

The entire study was conducted in the seventh medical center, PLA general hospital. All participants came to the hospital twice (one visit per day, each lasting 8 h). On each visit, each subject was required to come at 8:00 A.M. and participate in four information input tasks from 9:00 to 17:00 in either a virtual environment (virtual hand condition) or real environment (real hand condition) (Fig. 2 B, C). The virtual and real hand conditions were counterbalanced across all the participants, with an interval of at least one week. Participants were required to complete four simple information input tasks, including finding wrong written words, searching specific content, entering text, and classifying images. All participants underwent MRI scan immediately after finishing the tasks (Fig. 2 D).

3.1 Imaging data acquisition

Structural and resting-state functional MRI data were acquired using a 3 T Signa GE scanner (GE Healthcare, Waukesha, Wisconsin, USA) with a standard transmit/receive head coil. High-resolution structural images were acquired with echo time (TE), 8.2 ms; repetition time (TR), 3.2 ms; flip angle, 7o, field of view, 256 mm; matrix size, 256 × 256; voxel size, 1 × 1 × 1mm3. Resting-state functional images were acquired with TE, 30 s; TR, 2S; slice thickness, 3.5 mm; and total 240 volumes were obtained during 8 min. During data acquisition, all participants were required to keep their eyes open, and head movements were limited using customized cushions.

3.2 Imaging data preprocessing

Resting-state functional imaging data were performed using Analysis of Functional NeuroImages (AFNI), Statistical Parametric Mapping (SPM), and MATLAB software (MathWorks, Natick, MA, USA). The preprocessing steps included the removal of the first ten volumes, slice timing correction (3dTshift in AFNI), despiking (3dDespike in AFNI), and motion correction (3dvolreg in AFNI). Physiological noise was estimated using the average BOLD signals from regions of white matter and cerebrospinal fluid determined in anatomical images in the original space (Segment in SPM). Then, the voxelwise BOLD signals were analyzed with a general linear regression model (3dDeconvolve in AFNI) using eight regressors, including six motion parameters (three translational and three rotational parameters), white matter, and cerebrospinal fluid. After nuisance signal regression, the denoised functional images were transformed from the original space into the Montreal Neuroimaging Institute (MNI) space with a 3 mm cubic voxel size, and a frequency filter was applied to preserve only the amplitude of low-frequency fluctuations within 0.01–0.1 Hz for the subsequent analysis.

3.3 Seed-based functional connectivity analysis

To explore the functional connectivity map of the hippocampus in either the virtual or real hand condition, a Pearson correlation coefficient r was calculated between the time series of voxels, x and y, with n time points, in the hippocampus and the time series of all other voxels within the brain,

$$ r = \frac{{\Sigma_{{{\text{i}} = 1}}^{n} \left( {\chi_{i} - \overline{x}} \right)\left( {y_{i} - \overline{y}} \right)}}{{\sqrt {\Sigma_{{{\text{i}} = 1}}^{n} \left( {\chi_{i} - \overline{\chi }} \right)^{2} } \sqrt {\Sigma_{{{\text{i}} = 1}}^{n} \left( {y_{i} - \overline{y}} \right)^{2} } }} $$

where \(\overline{x }\) is the mean of x and \(\overline{y }\) is the mean of y. The correlation coefficients were normalized using Fisher transformation. Spatial smoothing was then performed using 8 mm full-width half-maximum Gaussian kernel filter to compensate for inter-subject variability. One-sample and paired-sample t tests (p < 0.05; cluster size > 120) were conducted to obtain functional connectivity maps. Multiple comparisons were corrected using a probability and cluster thresholding program (Alphasim in AFNI; a minimal cluster size > 3960mm3).

To quantitatively evaluate the connectivity strength between the hippocampus and sensorimotor cortex, a standard brain atlas was used first to divide the brain into 116 anatomical regions (Tzourio-Mazoyer et al. 2002). The sum of the correlation coefficient between the hippocampus and precentral and postcentral gyrus was calculated and compared separately between the two experimental conditions.

4 Results

In both virtual and real hand conditions, the hippocampus showed significant functional connections with brain regions in the default mode network, including the posterior cingulate cortex/precuneus, medial prefrontal cortex, lateral parietal cortex, inferior temporal cortex, and several subcortical regions including the thalamus, hypothalamus, parahippocampal gyrus, amygdala, and brainstem (Fig. 3 A, B). Participants specifically exhibited significant positive functional connections with the primary motor (M1) and somatosensory cortex (S1) in virtual hand condition (Fig. 3 A).

Fig. 3
figure 3

Group average of functional connectivity of the hippocampus in virtual hand condition A and real hand condition B

Full size image

One-sample paired t test revealed significant connection differences between the two experimental conditions. Participants showed stronger functional connectivity from the hippocampus to S1 and M1 in the virtual hand condition compared with the real hand condition (Fig. 4).

Fig. 4
figure 4

One-sample paired t test of functional connectivity of the hippocampus between virtual hand and real hand condition A, and region-specific changes in the sum of correlation coefficient (CC) in virtual hand condition (V) and real hand condition (R) B-C

Full size image

5 Discussion

In the present work, the functional connectivity patterns of the hippocampus related to the long-time experience of virtual hand illusion were investigated, using resting-state functional connectivity analysis. The results showed that significant positive functional connectivity of the hippocampus with S1 and M1 was observed only in the virtual hand condition, while no significant connection between the hippocampus and the S1 and M1 was observed in the real hand condition (Fig. 4 B, C). Accordingly, we proposed that the functional connection between the hippocampus and S1 and M1 should contribute to the consistent experience of ownership over the virtual hand.

The effect of the long-term virtual hand illusion on resting-state functional connectivity has not been reported. As the pattern of resting-state functional connection may indicate a history of task-related brain activity (Duan et al., 2012), the long-term virtual hand illusion should result in co-activation of specific brain regions associated with the illusion process. In our study, the resting-state functional connection network of the hippocampus was investigated after the long-term virtual hand illusion, and a significant functional connection between the hippocampus and sensorimotor cortex was identified. However, participants in the real hand condition exhibited different resting-state functional connectivity of the hippocampus, demonstrating illusion-induced changes in resting-state functional connectivity. Our study provided the first neuroimaging evidence that the virtual hand illusion, over time, was associated with neuroplasticity in the resting-state functional networks.

Our result revealed reorganized functional connectivity of the hippocampus after the long-term virtual hand illusion. The result that significant positive hippocampal connectivity with the sensorimotor cortex was only observed in the virtual hand condition might reflect altered functional organization related to long-time immersion into the sense of ownership of the virtual hand. Functional reorganization of S1 responding to the feeling of ownership of a virtual body has been demonstrated in humans and monkeys (Schaefer et al. 2007, 2009). In a recent study, neural responses in both S1 and M1 have been observed in the experience of illusory ownership, suggesting a widely distributed brain network related to the virtual body illusion (Shokur et al. 2013). So, it opened the possibility of functional connection reorganization of the hippocampus with the sensorimotor cortex associated with manipulated body ownership illusion. The hippocampus has been suggested to play an indispensable role in integrating sensory and motor information during motor behavior (Bland & Oddie 2001; Dypvik & Bland 2004); the enhanced communication between the hippocampus and sensorimotor cortex could facilitate the integration of sensorimotor information into memory (Grion, Akrami, Zuo, Stella, & Diamond, 2016). In the current study, the altered functional connection between the hippocampus and sensorimotor cortex could play a role in integrating multi-sensory information into memory and inducing a renewed feeling of owning a pair of virtual hands.

Several previous studies reported the participation of the hippocampus in the sense of self-location during the virtual navigation task. For example, the hippocampal activity pattern predicted the spatial position of the participant in a virtual space, suggesting a role of the hippocampus in the construction of self-location (Guterstam et al. 2015; Hassabis et al. 2009; Rodriguez 2010). However, location-specific coding is only part of the information coding domains of the hippocampus. Animal studies have shown that when the location of the animal was held constant, hippocampal cells fired in a variable pattern which was associated with different memory episode or temporal context (Fortin et al. 2002; Wood et al. 2000), indicating a parallel coding pattern accounting for spatial and non-spatial context, respectively, within the hippocampus (Schiller et al. 2015). Our result was in line with the conclusion and demonstrated the involvement of the hippocampus in the maintenance of somatic memory.

6 Conclusion and future consideration

In our study, based on an almost 8-h virtual hand illusion, we assessed the resting-state functional connection pattern of the hippocampus and identified a significant positive connection between the hippocampus and sensorimotor cortex, compared with the real hand condition. Our study demonstrated the involvement of hippocampal-sensorimotor functional connection in the long-term experience of virtual hand illusion and reflected an altered functional connection organization of brain regions related to a renewed state of sense of owning a virtual hand. This is the first study, to our knowledge, to show that virtual hand illusion, over time, was associated with neuroplasticity in resting-state networks. Moreover, our finding indicated that the functional connection reorganization of the hippocampus might promote multi-sensory information integration into memory and update the sense of body ownership, which provided the underlying neural mechanism of the availability of long-term use of VR technology in healthcare and rehabilitation.

Although our results indicated the reorganized resting-state functional connectivity induced by a long-term experience of virtual hand illusion, we could not conclude that the long-term illusion resulted in lasting and stable changes in the resting-state functional connection of the hippocampus. Future work should focus on longitudinal assessment of virtual hand illusion and resting-state functional connection networks to determine the amount of illusion process that could induce stable and long-lasting changes in functional and structural connectivity within the related networks.

One limitation of this study was that the extent to which participants experienced the virtual hand illusion was not assessed using a questionnaire. Recent studies have provided evidence that virtual hand paradigm is an effective method to induce virtual hand illusion and the experience of illusory hand ownership (Ma, Hommel, & cognition, 2015; Ma & Hommel, 2013; Sanchez-Vives, Spanlang, Frisoli, Bergamasco, & Slater, 2010; Slater, Pérez Marcos, Ehrsson, & Sanchez-Vives, 2008). In the classic virtual hand illusion, the participant was presented 3D images of two hands through a head-mounted display, the illusory ownership of a virtual hand is induced by manipulating the synchronization of the movement between the participant’s own hands and the virtual hand. Once the movement of the virtual hand was synchronized with the real hand movement, both ownership and agency of the virtual hand were induced (Shibuya, Unenaka, & Ohki, 2018). In our study, a classic virtual hand illusion design, in which a pair of virtual hands was projected into an immersive virtual typing system and the movement of the virtual hands was synchronized with the real hands of the participant, was adopted to induce the illusory ownership of the virtual hand. Moreover, a recent study has demonstrated that the similarity of the appearance between the virtual and real hand could affect the degree of illusory feeling of ownership, a strongest rating of the illusion of virtual hand ownership was induced by the 3D images of real hands (Pyasik et al. 2020). In the current study, a RealSense camera was used to extract the images of the real hands of each participant. The images of the real hands were projected into a virtual environment in real time, so that the virtual hands that each participant were presented with were the real-time images of the participant’s real hands. Thus, the experience of illusory ownership of the virtual hand was maximized.