Next Article in Journal
A High Resolution XUV Grating Monochromator for the Spectral Selection of Ultrashort Harmonic Pulses
Next Article in Special Issue
AR Object Manipulation on Depth-Sensing Handheld Devices
Previous Article in Journal
Detection and Treatment Methods for Perfluorinated Compounds in Wastewater Treatment Plants
Previous Article in Special Issue
Model-Based 3D Pose Estimation of a Single RGB Image Using a Deep Viewpoint Classification Neural Network
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 9
Issue 12
10.3390/app9122501
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Psychophysiological Alteration After Virtual Reality Experiences Using Smartphone-Assisted Head Mount Displays: An EEG-Based Source Localization Study

by
Jeong-Youn Kim
1,
Jae-Beom Son
2,
Hyun-Sung Leem
2,* and
Seung-Hwan Lee
1,3,*
1
Clinical Emotion and Cognition Research Laboratory, Inje University, Goyang 10370, Korea
2
Department of Optometry, Eulji University, Seongnam 13135, Korea
3
Department of Psychiatry, Inje University, Ilsan-Paik Hospital, Goyang 10370, Korea
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2019, 9(12), 2501; https://doi.org/10.3390/app9122501
Submission received: 22 May 2019 / Revised: 17 June 2019 / Accepted: 17 June 2019 / Published: 19 June 2019
(This article belongs to the Special Issue Augmented Reality: Current Trends, Challenges and Prospects)
Browse Figures
Versions Notes

Abstract

:
Brain functional changes could be observed in people after an experience of virtual reality (VR). The present study investigated cyber sickness and changes of brain regional activity using electroencephalogram (EEG)-based source localization, before and after a VR experience involving a smartphone-assisted head mount display. Thirty participants (mean age = 25 years old) were recruited. All were physically healthy and had no ophthalmological diseases. Their corrected vision was better than 20/20. Resting state EEG and the simulator sickness questionnaire (SSQ) were measured before and after the VR experience. Source activity of each frequency band was calculated using the sLORETA program. After the VR experience, the SSQ total score and sub scores (nausea, oculomotor symptoms, and disorientation) were significantly increased, and brain source activations were significantly increased: alpha1 activity in the cuneus and alpha2 activity in the cuneus and posterior cingulate gyrus (PCG). The change of SSQ score (after–before) showed significant negative correlation with the change of PCG activation (after–before) in the alpha2 band. The study demonstrated increased cyber sickness and increased alpha band power in the cuneus and PCG after the VR experience. Reduced PCG activation in alpha band may be associated with the symptom severity of cyber sickness.

1. Introduction

The term “virtual reality” (VR) refers to any particular environmental situation or technology that is similar to reality, but is created by a computer. Recently, VR devices that combine smartphones and head mount displays (HMDs) have been attracting the attention of smartphone users because they allow users to experience VR worlds with several merits: low price, accessibility from anywhere and at any time, and unlimited expansion of 3D materials. Smartphone companies are actively investing and developing this technology [1,2].
However, after watching a thrilling VR video, people often feel sick or dizzy to a certain degree [3]. This discomfort could be due to ophthalmological maladjustment by VR [4]. To form three-dimensional (3D) images, stereoscopic vision is necessary, because the brain perceives depth by combining images of objects from the left and the right eyes. Similarly, 3D TVs and VR display images at different angles to the left and right eye to produce stereoscopic images. The process of interlacing two different images creates visual fatigue [5,6]. VR technologies that use HMDs show two different images to both eyes to simulate objects in the same location as real objects. However, since these images are displayed on a very near screen of HMDs, the eye focuses on the screen rather than on the actual distance. Consequently, ophthalmological adjustment through accommodation and convergence does not occur successfully, causing eye strain [7].
One of the main discomforts after experiencing a VR is cyber sickness. The cyber sickness can include symptoms such as dizziness, eyestrain, nausea, and sweating. In one series of experiments, 80% of participants reported an increase in sickness symptoms following immersion in a VR environment [8]. In most participants, this increase was mild and subsided after exiting the VR immersion. However, 5% of the participants could not complete the VR immersion because the effects were so aversive. The aversion to VR could be a major obstacle to growth and progress of this kind of future technology. Understanding the psychophysiology and brain regional activation of the cyber sickness after VR could solve this obstacle and help with the growth and progress of the VR technology.
The changes in brain function after 3D VR experiences have been studied by using electroencephalography (EEG). In EEG experiments, visual fatigue is indicated by increased low frequency band amplitude and decreased high frequency band amplitude after a VR experience. In particular, Wawrzyk et al. [9] observed increased alpha wave activity after participants watched VR simulations with special goggles intended to arouse emotions, such as anger, fear, or excitement. Furthermore, increases in delta and theta activity [10], as well as a decrease in beta activity [11], have been found during visual fatigue. In addition, frontal midline theta power was significantly higher during the encoding phase of VR from 3D television route presentation than during the same phase in 2D presentation [12]. Other studies have revealed increased power in the beta frequency band as an indicator of 3D visual fatigue [13,14].
In a study about brain regional activity, the insula regions of the alpha and theta bands were more highly activated while participants navigated using VR than when they simply watched a video from a desktop monitor or a high resolution power wall screen [15]. Conversely, Baumgartner et al. [16] found that event-related power decreased in the alpha band of parietal brain areas of children and adolescents while they experienced a virtual roller coaster through television. One previous fMRI study in which the participants experienced virtual roller-coaster rides showed that vertical motion engaged the posterior insula, whereas horizontal motion was associated with an enhanced neural response in the hippocampal areas [17]. Furthermore, virtual navigation has been related to increased activation of the hippocampus, parahippocampus, and posterior parietal cortex compared to non-navigational control tasks [18,19,20,21,22,23]. These studies showed inconsistent results in frequency bands and regions of brain activation due to differences in 3D video clips presented by various 3D TVs or screens.
No previous studies have explored psychophysiological alterations after VR experiences involving smartphone-assisted HMDs. In the present explorative study, we used EEG-based source localization to evaluate cyber sickness and changes in brain regional activation after the VR exposure.

2. Methods

2.1. Participants

A total of 30 healthy volunteers (17 men, 13 women) with a mean age of 25 ± 4 (age range: 20–37) years participated in this study. They were recruited using flyers and online social networking sites. The participants were all college students and office workers, who frequently use smart devices. We excluded participants who (1) had systemic or eye diseases; (2) abused drugs, alcohol, or other substances, or had a dependency; (4) were pregnant; (5) had corrected vision worse than 20/20. A trained researcher (J.-B.S.) used a semi-structured interview sheet to screen out participants with any history of drug, alcohol, and other substance abuse or dependency. Refractive error was measured using the Phoropter (HDR-7000; Huvitz, Korea) and the Chart Projector (HDC-9000 N/PF; Huvitz, Korea). Participants with anisometropia, wherein the eyes have unequal refractive power, cannot integrate HMD VR images, which are presented separately to each eye. For this reason, such people were also excluded. All experimental protocols were reviewed and approved by the Institutional Review Board (IRB) at Inje University Ilsan Paik Hospital, Republic of Korea (2017-04-001-002) and were executed in accordance with guidelines and regulations of the board. All participants gave written informed consent in accordance with the Declaration of Helsinki.

2.2. Simulator Sickness Questionnaire (SSQ)

The SSQ was a self-reporting questionnaire used to measure cyber sickness. It consisted of 16 questions about symptoms, which were divided into three main categories: (1) Nausea, which includes general discomfort, increased salivation, sweating, nausea, difficulty concentrating, stomach awareness, and burping. (2) Oculomotor symptoms, including general discomfort, fatigue, headache, eyestrain, difficulty focusing, difficulty concentrating, and blurred vision. (3) Disorientation, which encompasses difficulty focusing, nausea, fullness of the head, blurred vision, dizziness with eyes open, dizziness with eyes closed, and vertigo [24]. Each symptom was rated from 0 (absent) to 3 (extreme). Then, it was summed up with weight (nausea: 9.54, oculomotor: 7.58, disorientation: 13.92) for each category, resulting in a possible score of 0–200.34 for nausea, 0–159.18 for oculomotor, and 0–292.32 for disorientation category [24].
Subjects completed the SSQ before watching the VR video. After watching the video, they completed the remainder of SSQ.

2.3. VR Equipment and VR Video

The VR machine used an HMD that was released by Samsung Electronics in September 2016 (Samsung New Gear VR, SM-R323; Samsung Electronics, Korea). This HMD connects to smartphones to create VR. The smartphone used was the Galaxy Note 5 (SM-N920S; Samsung Electronics, Korea), which was launched in August 2015. The VR videos had a viewing distance of 6–7 cm when the HMD equipment was worn.
The 3D VR video clip was adapted from YouTube [25]. It included a roller coaster ride, scuba diving, driving, airplane control, skydiving, etc. Each of these situations was presented at random to the participants. The VR video clips ran for about 8 minutes and 54 seconds. The researchers have observed the terms and conditions regarding the use of video contents, as the research project has been categorized as a non-commercial report.

2.4. EEG Data Acquisition and Analysis

The subjects sat on a chair in a room with the ambient noise blocked. Their resting state quantitative EEG was recorded with their eyes open for 4 minutes, and then with their eyes closed for 4 minutes. It was recorded both before and after the participants had watched the 3D VR video clip. EEG signals were recorded using a Quick Cap with 62 Ag-AgCl electrodes and NeuroScan SynAmps (Compumedics USA, El Paso, TX, USA). The electrodes were attached in accordance with the extended 10–20 system. The vertical electrooculogram (EOG) was recorded by attaching the electrodes above and below the left eye, while the horizontal EOG was recorded by attaching the electrodes to the outer canthus of each eye. We analyzed the resting EEG data with the participants’ eyes open and closed using CURRY7 (Compumedics USA, Charlotte, NC, USA), which is commonly used for EEG pre-processing. Gross artifacts were removed from the recorded data by visual inspection. Artifacts caused by eye blinks and eye movements were removed using the covariance analysis of CURRY 7 [26]. The pre-processed EEG data were divided into multiple epochs by a length of 2 seconds. Any epochs including significant physiological artifacts (amplitude exceeding ±100 μV) at any site, over all 62 electrodes, were excluded from the analysis. Power spectral analysis was used to compress the rhythmic information of the brain wave signals. In power spectral analysis, periodogram function in MATLAB R2017b (MathWorks, Natick, MA, USA) was used to calculate power spectral density of each epoch. The spectral power was then averaged with respect to randomly selected 30 epochs.
The band powers were classified into seven frequency bands: delta (1–4 Hz), theta (4–8 Hz), alpha1 (8–10 Hz), alpha2 (10–12 Hz), beta1 (12–18 Hz), beta2 (18–30 Hz), and gamma (30–50 Hz). The powers were averaged into three regions: anterior (FP1, AF3, F1, F3, F5, F7, Fpz, Fz, FP2, AF4, F2, F4, F6, and F8), middle (T7, C1, C3, C5, Cz, T8, C2, C4, and C6), posterior (P1, P3, P5, P7, PO3, PO5, PO7, O1, Pz, Poz, Oz, P2, P4, P6, P8, PO2, PO6, PO8, and O2), and global.

2.5. Source Activity Analysis

Since EEG has a high temporal resolution, it has been considered to be the most suitable neuroimaging tool to investigate rapid changes in brain activity. However, EEG has some intrinsic limitations. Firstly, sensor-level EEG has low spatial resolution due to volume conduction. In other words, the brain signals may not reflect brain activity directly below the recording electrodes [27,28]. Secondly, EEG data can be contaminated with various noises and artifacts, so it can have a poor signal-to-noise ratio [29,30]. Using EEG source-imaging methods is one of the cost-effective options to improve the spatial resolution. For example, mapping the potential distribution of the scalp onto the cortical source space below can improve the spatial resolution. Standardized low-resolution brain electromagnetic tomography (sLORETA) is also a well-established source-imaging method [31]; it has calculated brain source signals comparable to those of intracranial recording [32].
Particularly, sLORETA has been used to calculate the cortical distribution of the standardized source current density. This is a representative source-imaging method for solving the EEG inverse problem [31], assuming that the source activation of a voxel is similar to the source activity of the surrounding voxels to compute a particular solution. It also applies an appropriate standardization of the current density. The lead field matrix was computed using a realistic head model that was segmented on the basis of the Montreal Neurological Institute (MNI) 152 standard template, wherein the 3D solution space is restricted only to the cortical gray matter and hippocampus [33]. The solution space was composed of 6239 voxels, with a 5 mm resolution. Anatomical structures, such as the Brodmann areas, were labelled using an appropriate transformation from MNI to Talairach space [34].
This EEG source activity analysis method was similar to that of our previous study [35].

2.6. Statistical Analysis

The paired t-test was performed to compare the SSQ score before the participants watched the VR video with that after. Two-way repeated-measures analysis of variance (rmANOVA) was performed. Time (before and after the participants watched the VR video) and region (anterior, middle, posterior, and global) were used as the within-subject factors of EEG frequency band power. Two-way repeated-measures analysis of variance (ANOVA) was performed. Time (before and after the participants watched the VR video) and region (angular gyrus, anterior cingulate, cingulate gyrus, cuneus, extra nuclear, fusiform gyrus, inferior frontal gyrus, inferior occipital gyrus, inferior parietal lobule, inferior temporal gyrus, insula, lingual gyrus, medial frontal gyrus, middle frontal gyrus, middle occipital gyrus, middle temporal gyrus, orbital gyrus, paracentral lobule, parahippocampal gyrus, postcentral gyrus, posterior cingulate, precentral gyrus, precuneus, rectal gyrus, subcallosal gyrus, sub gyral, superior frontal gyrus, superior occipital gyrus, superior parietal lobule, superior temporal gyrus, supramarginal gyrus, transverse temporal gyrus, and uncus) were used as the within-subject factors of brain source activity. When a significant effect was found, post hoc comparisons were performed using the paired t-test. Since many cortical areas were analyzed simultaneously as dependent variables, Bonferroni correction for cases of multiple comparison was used. Adjusted p-value was calculated by dividing 0.05 by the number of brain regions (i.e., adjusted p-value for frequency band power analysis: 0.05/4, padj < 0.0125; adjusted p-value for source activation analysis: 0.05/33, padj < 0.00151). Pearson correlation analysis was performed to examine the relationship between SSQ score and brain source activation. Statistical analyses were performed using SPSS 21 (SPSS Inc., Chicago, IL, USA).

3. Results

In the SSQ, the following factors were significantly higher after VR than before VR: nausea (16.218 ± 21.133 vs. 47.064 ± 31.684, p < 0.001), oculomotor symptoms (29.562 ± 23.750 vs. 67.715 ± 36.103, p < 0.001), disorientation (25.056 ± 35.517 vs. 101.152 ± 66.801, p < 0.001), and total score (12.332 ± 13.854 vs. 41.044 ± 24.779, p < 0.001). This result is presented in Table 1.
In the eyes open condition, the rmANOVA for frequency band power showed significant main effects of time and brain region in theta (F = 4.879, df = 1.872, p = 0.013), alpha1 (F = 8.065, df = 1.112, p = 0.006), alpha2 (F = 9.317, df = 1.201, p = 0.003), and beta1 (F = 4.988, df = 2.014, p = 0.010) frequency bands. In the post hoc analysis, we did find significant differences in the theta, alpha1, alpha2, and beta bands. Specifically, the alpha1 band power was significantly higher after VR than before VR in the following segments: middle (2.703 ± 2.542 vs. 3.526 ± 3.453, p = 0.002), posterior (5.689 ± 5.595 vs. 9.025 ± 10.547, p = 0.004), and global (4.097 ± 3.960 vs. 6.055 ± 6.942, p = 0.004). The alpha2 band power was significantly higher after VR than before VR in the following segments: anterior (4.533 ± 4.672 vs. 5.627 ± 5.697, p = 0.006), posterior (7.695 ± 8.202 vs. 9.943 ± 9.856, p = 0.003), and global (4.999 ± 5.003 vs. 6.233 ± 5.951, p = 0.003). This result is presented in Table 2.
In addition, in the eyes open condition, the rmANOVA analysis of brain source activities showed significant main effects of time and brain region in alpha1 (F = 3.393, df = 5.355, p = 0.001) and alpha2 (F = 3.446, df = 5.207, p = 0.001) frequency bands. In the post hoc results, the following measurements were significantly increased after VR than before VR: alpha1 (342.635 ± 236.021 vs. 515.076 ± 423.659, p = 0.001) and alpha2 (590.988 ± 462.924 vs. 795.884 ± 683.322, p = 0.001) band activation in the cuneus, as well as alpha2 band activation in the posterior cingulate gyrus (PCG) (370.731 ± 323.503 vs. 493.818 ± 442.367, p = 0.001). This result is presented in Table 3 and Figure 1.
In the eyes closed condition, the rmANOVA analysis of frequency band power showed no significant difference of main effect. However, in post hoc analysis, delta band power of the posterior region (12.301 ± 5.067 vs. 13.434 ± 6.234, p = 0.047) and beta2 band power of the global region (2.988 ± 1.126 vs. 3.186 ± 1.246, p = 0.047) were increased after VR. The rmANOVA analysis of brain source activities showed a significant main effect of time and brain region in beta1 frequency band. However, there was no significant result after post hoc analysis.
Pearson correlation analysis was applied to the change of source activities (after–before) of the cuneus and PCG, as well as to the change of SSQ scores (after–before), to examine whether these factors were related to cyber sickness.
In the alpha2 band, PCG regional source activity was significantly associated with oculomotor symptoms (r = −0.406, p = 0.026) and disorientation (r = −0.380, p = 0.038), as well as with total SSQ (r = −0.386, p = 0.035). Pairs without any significant association between source activation and SSQ scores were not presented. This result is presented in Figure 2.

4. Discussion

In the present study, we found significant changes in psychophysiological and brain activation following the VR exposure using a smartphone-assisted HMD. Firstly, the SSQ total score and its subscale scores (nausea, oculomotor symptoms, and disorientation) were significantly increased after the VR experience. Secondly, several brain regions showed a significant increase in activation. In particular, alpha1 activity showed a robust increase in the cuneus, while alpha2 activity increased in the cuneus and PCG. Thirdly, the change of PCG activation in alpha band showed significant correlation with the changes of SSQ total scores and its two subscale scores (oculomotor and disorientation).
The increase in SSQ total and subscale scores clearly shows that participants experienced a severe level of cyber sickness during and after using VR in a smartphone-assisted HMD. Traditionally, researchers have attempted to explain motion sickness using two main theories: sensory conflict theory and postural instability theory. Sensory conflict theory suggests that the motion sickness occurs when the motion perceived by the visual, vestibular, and somatosensory systems differs from the expected experience [36]. On the other hand, postural instability theory implies that one’s motion sickness occurs due to the user’s inability to maintain a stable posture [37]. The cyber sickness experienced by participants in this study could be explained in the context of both theories. In smartphone-assisted head mount displays, two different images reach both eyes in a separate manner, at a separate time. Since these images are displayed on a close-up HMD screen, ophthalmological adjustment through accommodation and convergence may not occur successfully, causing eye strain and cyber sickness.
Our results show a significant increase in brain regional activation (the cuneus of alpha1 and alpha2, and PCG of alpha2) after the VR experiment. These results suggest that alpha band power could be significantly related to the VR experience. The previous VR based driving simulation experiments have mainly reported changes in alpha band (8–13 Hz) [38,39], when participants felt tired after VR experiences. The increase in alpha band power has been shown to be negatively related to cortical activity [40,41,42]. In addition, low band power (delta, theta, and alpha) increases in the occipital and parietal regions are correlated with the severity of motion sickness symptoms [38,43,44]. In sum, increased alpha power in the cuneus and PCG could represent the reduced cortical activity on those regions after the VR experiment.
The cuneus, which is part of the occipital lobe, has been related to visual processing [45]. It is known to receive visual information from the contralateral superior retina, and the processing of this information is modulated by other effects, such as attention, working memory, or reward expectation [46,47]. In a previous study comparing navigation with video viewing, participants during the navigation showed higher activation in the parietal and occipital brain regions, including the cuneus [48]. Furthermore, the cuneus and post-central parietal lobe have been related to working memory and navigation tasks [46,49].
Finally, considering that the activation changes in PCG were significantly correlated with the changes of SSQ total and subscale scores, the posterior cingulate cortex is a key node of the Default Mode Network (DMN)—a group of neural structures closely related to self-referencing and other self-related processing [50,51,52]. The immersion of 3D VR can induce a user to feel a sense of ownership of an avatar presented in a VR environment. This sense of ownership over one’s body has been related to multisensory integration in the premotor and posterior parietal cortices [53,54,55,56], while the emergence of sense of bodily self-location has been located to the posterior cingulate cortex [57,58]. Indeed, it has been consistently proven that extreme excitation of the PCG is associated with a variety of mental health problems, including attention deficit hyperactivity disorder [59], depression [60], and rumination [61]. In general, activation of the PCG is reduced during meditation, consistent with the concept that self-related thinking is minimized during the activity [62,63]. PCG activity depends on the arousal state, and its interactions with other brain networks may be important for self-awareness. Our results suggest that the changes of PCG activation of alpha2 band after VR experiences using smartphone-assisted HMDs indicates instability of mental awareness and arousal along with the loss of the sense of ownership of body, which eventually causes cyber sickness.
The present study had some limitations. Firstly, it lacked a control condition. A 2D video presentation using a plain monitor to a control group could have supplemented the study design by enabling differentiation of characteristics of the 3D VR presentation. Secondly, the individual head model was not used in the present study, so our results might be subjected to bias regarding the precision of brain regions of interest.
The present study examined psychophysiological alterations in participants after the VR experience using a smartphone-assisted HMD. To this end, we used an EEG-based source localization approach. Our study revealed that cyber sickness symptoms were significantly increased after the VR exposure. The cuneus and PCG were significantly activated in alpha frequency bands after the VR experience, and the activation change of the PCG was significantly correlated with cyber sickness symptoms experienced by participants. It is suggested that more sophisticated VR technology should be developed to avoid these unnecessary psychophysiological alterations related to PCG activation in the future.

Author Contributions

Conceptualization, J.-Y.K.; Methodology, J.-Y.K. and J.-B.S.; Software, J.-Y.K.; Validation, S.-H.L. and J.-Y.K.; Formal Analysis, J.-Y.K.; Investigation, J.-B.S.; Resources, S.-H.L., H.-S.L., and J.-B.S.; Data Curation, J.-B.S. and J.-Y.K.; Writing—Original Draft Preparation, J.-Y.K. and J.-B.S.; Writing—Review & Editing, S.-H.L. and H.-S.L.; Visualization, J.-Y.K.; Supervision, S.-H.L. and H.-S.L.; Project Administration, S.-H.L.; Funding Acquisition, S.-H.L.

Funding

This work was supported by the 2017 creative research program of Inje University, and a grant from the Korea Science and Engineering Foundation (KOSEF), funded by the Korean government (NRF-2018R1A2A2A05018505).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the study design, data collection and analyses, interpretation, or writing of the study.

References

  1. Jerald, J. The VR book: Human-centered design for virtual reality; Association for Computing Machinery and Morgan & Claypool: New York, NY, USA, 2015. [Google Scholar]
  2. Kang, C.-H. Flow of next generation broadcast video, Present and Future of VR contents. Korea Contents Association Review 2016, 14, 14–18. [Google Scholar] [CrossRef]
  3. Roettl, J.; Terlutter, R. The same video game in 2D, 3D or virtual reality—How does technology impact game evaluation and brand placements? PLoS ONE 2018, 13, e0200724. [Google Scholar] [CrossRef] [PubMed]
  4. Kolasinski, E.M. Simulator Sickness in Virtual Environments; Army Research Institute for the Behavioral and Social Sciences: Alexandria, VA, USA, 1995. [Google Scholar]
  5. Howard, I.P.; Rogers, B.J. Binocular vision and stereopsis; Oxford University Press: Oxford, UK, 1995. [Google Scholar]
  6. Lambooij, M.; Fortuin, M.; Heynderickx, I.; IJsselsteijn, W. Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review. J. Imaging Sci. Technol. 2009, 53, 30201–30214. [Google Scholar] [CrossRef]
  7. Hoffman, D.M.; Girshick, A.R.; Akeley, K.; Banks, M.S. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. J. Vision 2008, 8, 33. [Google Scholar] [CrossRef] [PubMed]
  8. Cobb, S.V.; Nichols, S.; Ramsey, A.; Wilson, J.R. Virtual reality-induced symptoms and effects (VRISE). Presence 1999, 8, 169–186. [Google Scholar] [CrossRef]
  9. Wawrzyk, M.; Wesołowska, K.; Plechawska-Wójcik, M.; Szymczyk, T. Analysis of Brain Activity Changes Evoked by Virtual Reality Stimuli Based on EEG Spectral Analysis. A Preliminary Study. In Proceedings of the 39th International Conference on Information Systems Architecture and Technology, Nysa, Poland, 16–18 September 2018; pp. 222–231. [Google Scholar]
  10. Subasi, A. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients. Expert Syst. Appl. 2005, 28, 701–711. [Google Scholar] [CrossRef]
  11. Belyavin, A.; Wright, N.A. Changes in electrical activity of the brain with vigilance. Electroen. Clin. Neuro. 1987, 66, 137–144. [Google Scholar] [CrossRef]
  12. Slobounov, S.M.; Ray, W.; Johnson, B.; Slobounov, E.; Newell, K.M. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int. J. Psychophysiol. 2015, 95, 254–260. [Google Scholar] [CrossRef] [PubMed]
  13. Li, H.-C.O.; Seo, J.; Kham, K.; Lee, S. Measurement of 3D Visual Fatigue Using Event-Related Potential (ERP): 3D Oddball Paradigm. In Proceedings of the 3DTV Conference: The True Vision—Capture, Transmission and Display of 3D Video, Istanbul, Turkey, 28–30 May 2008; pp. 213–216. [Google Scholar]
  14. Kim, Y.-J.; Lee, E.C. EEG Based Comparative Measurement of Visual Fatigue Caused by 2D and 3D Displays. In Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA, 9–14 July 2011; pp. 289–292. [Google Scholar]
  15. Clemente, M.; Rodríguez, A.; Rey, B.; Alcañiz, M. Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG. Expert Syst. Appl. 2014, 41, 1584–1592. [Google Scholar] [CrossRef]
  16. Baumgartner, T.; Valko, L.; Esslen, M.; Jäncke, L. Neural Correlate of Spatial Presence in an Arousing and Noninteractive Virtual Reality: An EEG and Psychophysiology Study. Cyberpsychol. Behav. 2006, 9, 30–45. [Google Scholar] [CrossRef] [Green Version]
  17. Indovina, I.; Maffei, V.; Pauwels, K.; Macaluso, E.; Orban, G.A.; Lacquaniti, F. Simulated self-motion in a visual gravity field: sensitivity to vertical and horizontal heading in the human brain. Neuroimage 2013, 71, 114–124. [Google Scholar] [CrossRef]
  18. Aguirre, G.K.; Detre, J.A.; Alsop, D.C.; D’Esposito, M. The parahippocampus subserves topographical learning in man. Cereb. Cortex 1996, 6, 823–829. [Google Scholar] [CrossRef] [PubMed]
  19. Hartley, T.; Maguire, E.A.; Spiers, H.J.; Burgess, N. The Well-Worn Route and the Path Less Traveled: Distinct Neural Bases of Route Following and Wayfinding in Humans. Neuron 2003, 37, 877–888. [Google Scholar] [CrossRef] [Green Version]
  20. Hassabis, D.; Chu, C.; Rees, G.; Weiskopf, N.; Molyneux, P.D.; Maguire, E.A. Decoding Neuronal Ensembles in the Human Hippocampus. Curr. Biol. 2009, 19, 546–554. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Iaria, G.; Fox, C.J.; Chen, J.-K.; Petrides, M.; Barton, J.J.S. Detection of unexpected events during spatial navigation in humans: bottom-up attentional system and neural mechanisms. Eur. J. Neurosci. 2008, 27, 1017–1025. [Google Scholar] [CrossRef] [PubMed]
  22. Spiers, H.J.; Maguire, E.A. Thoughts, behaviour, and brain dynamics during navigation in the real world. Neuroimage 2006, 31, 1826–1840. [Google Scholar] [CrossRef] [PubMed]
  23. Wolbers, T.; Büchel, C. Dissociable Retrosplenial and Hippocampal Contributions to Successful Formation of Survey Representations. J. Neurosci. 2005, 25, 3333–3340. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Kennedy, R.S.; Lane, N.E.; Berbaum, K.S.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [Google Scholar] [CrossRef]
  25. Amazing People Compilation 360 VR 360. 2016. Video, 0:00–8:54, posted by Videos in 360 Virtual Reality videos, October 15. Available online: https://youtu.be/nJui9sOlb98 (accessed on 19 June 2019).
  26. Semlitsch, H.V.; Anderer, P.; Schuster, P.; Presslich, O. A Solution for Reliable and Valid Reduction of Ocular Artifacts Applied to the P300 ERP. Psychophysiology 1986, 23, 695–703. [Google Scholar] [CrossRef] [PubMed]
  27. Nolte, G.; Bai, O.; Wheaton, L.; Mari, Z.; Vorbach, S.; Hallett, M. Identifying true brain interaction from EEG data using the imaginary part of coherency. Clin. Neurophysiol. 2004, 115, 2292–2307. [Google Scholar] [CrossRef]
  28. van den Broek, S.P.; Reinders, F.; Donderwinkel, M.; Peters, M. Volume conduction effects in EEG and MEG. Electroen. Clin. Neuro. 1998, 106, 522–534. [Google Scholar] [CrossRef]
  29. Lange, D.H.; Inbar, G.F. A robust parametric estimator for single-trial movement related brain potentials. IEEE T. Biomed. Eng. 1996, 43, 341–347. [Google Scholar] [CrossRef]
  30. Lemm, S.; Curio, G.; Hlushchuk, Y.; Muller, K.-R. Enhancing the signal-to-noise ratio of ICA-based extracted ERPs. IEEE T. Biomed. Eng. 2006, 53, 601–607. [Google Scholar] [CrossRef]
  31. Pascual-Marqui, R.D. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find. Exp. Clin. Pharmacol. 2002, 24, 5–12. [Google Scholar] [PubMed]
  32. Lantz, G.; Michel, C.; Pascual-Marqui, R.; Spinelli, L.; Seeck, M.; Seri, S.; Landis, T.; Rosen, I. Extracranial localization of intracranial interictal epileptiform activity using LORETA (low resolution electromagnetic tomography). Electroen. Clin. Neuro. 1997, 102, 414–422. [Google Scholar] [CrossRef]
  33. Fuchs, M.; Kastner, J.; Wagner, M.; Hawes, S.; Ebersole, J.S. A standardized boundary element method volume conductor model. Clin. Neurophysiol. 2002, 113, 702–712. [Google Scholar] [CrossRef] [Green Version]
  34. Brett, M.; Johnsrude, I.S.; Owen, A.M. The problem of functional localization in the human brain. Nat. Rev. Neurosci. 2002, 3, 243–249. [Google Scholar] [CrossRef]
  35. Kim, S.; Kim, J.S.; Jin, M.J.; Im, C.H.; Lee, S.H. Dysfunctional frontal lobe activity during inhibitory tasks in individuals with childhood trauma: An event-related potential study. Neuroimage Clin. 2018, 17, 935–942. [Google Scholar] [CrossRef]
  36. Reason, J.T. Motion sickness adaptation: a neural mismatch model. J. R. Soc. Med. 1978, 71, 819–829. [Google Scholar] [CrossRef]
  37. Riccio, G.E.; Stoffregen, T.A. An ecological theory of motion sickness and postural instability. Ecol. Psychol. 1991, 3, 195–240. [Google Scholar] [CrossRef]
  38. Lin, C.-T.; Ko, L.-W.; Lin, Y.-H.; Jung, T.-P.; Liang, S.-F.; Hsiao, L.-S. EEG Activities of Dynamic Stimulation in VR Driving Motion Simulator. In Proceedings of the International Conference on Engineering Psychology and Cognitive Ergonomics, Beijing, China, 22–27 July 2007; pp. 551–560. [Google Scholar]
  39. Chuang, S.-W.; Chuang, C.-H.; Yu, Y.-H.; King, J.-T.; Lin, C.-T. EEG Alpha and Gamma Modulators Mediate Motion Sickness-Related Spectral Responses. Int. J. Neural Syst. 2016, 25, 1650007. [Google Scholar] [CrossRef] [PubMed]
  40. Laufs, H.; Kleinschmidt, A.; Beyerle, A.; Eger, E.; Salek-Haddadi, A.; Preibisch, C.; Krakow, K. EEG-correlated fMRI of human alpha activity. Neuroimage 2003, 19, 1463–1476. [Google Scholar] [CrossRef]
  41. Laufs, H.; Krakow, K.; Sterzer, P.; Eger, E.; Beyerle, A.; Salek-Haddadi, A.; Kleinschmidt, A. Electroencephalographic signatures of attentional and cognitive default modes in spontaneous brain activity fluctuations at rest. Proc. Natl. Acad. Sci. 2003, 100, 11053–11058. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Gamma, A.; Lehmann, D.; Frei, E.; Iwata, K.; Pascual-Marqui, R.D.; Vollenweider, F.X. Comparison of simultaneously recorded [H215O]-PET and LORETA during cognitive and pharmacological activation. Hum. Brain Mapp. 2004, 22, 83–96. [Google Scholar] [CrossRef] [PubMed]
  43. Chen, Y.-C.; Duann, J.-R.; Chuang, S.-W.; Lin, C.-L.; Ko, L.-W.; Jung, T.-P.; Lin, C.-T. Spatial and temporal EEG dynamics of motion sickness. Neuroimage 2010, 49, 2862–2870. [Google Scholar] [CrossRef]
  44. Chelen, W.E.; Kabrisky, M.; Rogers, S.K. Spectral analysis of the electroencephalographic response to motion sickness. Aviat. Space Environ. Med. 1993, 64, 24–29. [Google Scholar] [PubMed]
  45. Perani, D.; Fazio, F.; Borghese, N.A.; Tettamanti, M.; Ferrari, S.; Decety, J.; Gilardi, M.C. Different Brain Correlates for Watching Real and Virtual Hand Actions. Neuroimage 2001, 14, 749–758. [Google Scholar] [CrossRef] [Green Version]
  46. Haldane, M.; Cunningham, G.; Androutsos, C.; Frangou, S. Structural brain correlates of response inhibition in Bipolar Disorder I. J. Psychopharmacol. 2008, 22, 138–143. [Google Scholar] [CrossRef]
  47. Vanni, S.; Tanskanen, T.; Seppä, M.; Uutela, K.; Hari, R. Coinciding early activation of the human primary visual cortex and anteromedial cuneus. Proc. Natl. Acad. Sci. 2001, 98, 2776–2780. [Google Scholar] [CrossRef] [Green Version]
  48. Clemente, M.; Rey, B.; Rodríguez-Pujadas, A.; Barros-Loscertales, A.; Baños, R.M.; Botella, C.; Alcañiz, M.; Ávila, C. An fMRI Study to Analyze Neural Correlates of Presence during Virtual Reality Experiences. Interact. Comput. 2013, 26, 269–284. [Google Scholar] [CrossRef] [Green Version]
  49. Mishkin, M.; Ungerleider, L.G. Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys. Behav. Brain Res. 1982, 6, 57–77. [Google Scholar] [CrossRef]
  50. Shulman, G.L.; Fiez, J.A.; Corbetta, M.; Buckner, R.L.; Miezin, F.M.; Raichle, M.E.; Petersen, S.E. Common Blood Flow Changes across Visual Tasks: II. Decreases in Cerebral Cortex. J. Cognitive Neurosci. 1997, 9, 648–663. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Raichle, M.E.; MacLeod, A.M.; Snyder, A.Z.; Powers, W.J.; Gusnard, D.A.; Shulman, G.L. A default mode of brain function. Proc. Natl. Acad. Sci. 2001, 98, 676–682. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Davey, C.G.; Pujol, J.; Harrison, B.J. Mapping the self in the brain’s default mode network. Neuroimage 2016, 132, 390–397. [Google Scholar] [CrossRef] [PubMed]
  53. Ehrsson, H.H.; Spence, C.; Passingham, R.E. That’s My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb. Science 2004, 305, 875–877. [Google Scholar] [CrossRef] [PubMed]
  54. Petkova, V.I.; Björnsdotter, M.; Gentile, G.; Jonsson, T.; Li, T.-Q.; Ehrsson, H.H. From Part- to Whole-Body Ownership in the Multisensory Brain. Curr. Biol. 2011, 21, 1118–1122. [Google Scholar] [CrossRef] [Green Version]
  55. Ehrsson, H. The concept of body ownership and its relation to multisensory integration. In The New Handbook of Multisensory Processes; Stein, B.E., Ed.; MIT Press: Cambridge, MA, USA, 2012; pp. 775–792. [Google Scholar]
  56. Blanke, O.; Slater, M.; Serino, A. Behavioral, Neural, and Computational Principles of Bodily Self-Consciousness. Neuron 2015, 88, 145–166. [Google Scholar] [CrossRef] [Green Version]
  57. Guterstam, A.; Björnsdotter, M.; Bergouignan, L.; Gentile, G.; Li, T.-Q.; Ehrsson, H.H. Decoding illusory self-location from activity in the human hippocampus. Front. Hum. Neurosci. 2015, 9, 412. [Google Scholar] [CrossRef]
  58. Guterstam, A.; Björnsdotter, M.; Gentile, G.; Ehrsson, H.H. Posterior Cingulate Cortex Integrates the Senses of Self-Location and Body Ownership. Curr. Biol. 2015, 25, 1416–1425. [Google Scholar] [CrossRef] [Green Version]
  59. Nakao, T.; Radua, J.; Rubia, K.; Mataix-Cols, D. Gray Matter Volume Abnormalities in ADHD: Voxel-Based Meta-Analysis Exploring the Effects of Age and Stimulant Medication. Am. J. Psychiatry 2011, 168, 1154–1163. [Google Scholar] [CrossRef]
  60. Haznedar, M.M.; Buchsbaum, M.S.; Hazlett, E.A.; Shihabuddin, L.; New, A.; Siever, L.J. Cingulate gyrus volume and metabolism in the schizophrenia spectrum. Schizophr. Res. 2004, 71, 249–262. [Google Scholar] [CrossRef] [PubMed]
  61. Berman, M.G.; Peltier, S.; Nee, D.E.; Kross, E.; Deldin, P.J.; Jonides, J. Depression, rumination and the default network. Soc. Cogn. Affect Neur. 2011, 6, 548–555. [Google Scholar] [CrossRef] [PubMed]
  62. Bærentsen, K.B.; Stødkilde-Jørgensen, H.; Sommerlund, B.; Hartmann, T.; Damsgaard-Madsen, J.; Fosnæs, M.; Green, A.C. An investigation of brain processes supporting meditation. Cogn. Process. 2010, 11, 57–84. [Google Scholar] [CrossRef] [PubMed]
  63. Fox, K.C.; Dixon, M.L.; Nijeboer, S.; Girn, M.; Floman, J.L.; Lifshitz, M.; Ellamil, M.; Sedlmeier, P.; Christoff, K. Functional neuroanatomy of meditation: A review and meta-analysis of 78 functional neuroimaging investigations. Neurosci. Biobehav. Rev. 2016, 65, 208–228. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Applsci 09 02501 g001 550
Figure 1. Distribution of differences in brain source activity (after VR vs. before VR) with eyes open. (A) Alpha1 band—cuneus; (B) alpha2 band—cuneus; (C) alpha2 band—posterior cingulate gyrus (PCG).
Figure 1. Distribution of differences in brain source activity (after VR vs. before VR) with eyes open. (A) Alpha1 band—cuneus; (B) alpha2 band—cuneus; (C) alpha2 band—posterior cingulate gyrus (PCG).
Applsci 09 02501 g001
Applsci 09 02501 g002 550
Figure 2. Scatterplots between PCG activation in alpha2 band and SSQ scores. The change in activation of PCG shows significantly negative correlations with the changes of SSQ in the oculomotor, disorientation, and total scores.
Figure 2. Scatterplots between PCG activation in alpha2 band and SSQ scores. The change in activation of PCG shows significantly negative correlations with the changes of SSQ in the oculomotor, disorientation, and total scores.
Applsci 09 02501 g002
Table
Table 1. Differences in simulator sickness questionnaire (SSQ) scores before and after watching a video.
Table 1. Differences in simulator sickness questionnaire (SSQ) scores before and after watching a video.
SSQ ScoreBefore VRAfter VRtp
Mean ± SD
Nausea16.218 ± 21.13347.064 ± 31.684−5.877<0.001
Oculomotor29.562 ± 23.75067.715 ± 36.103−7.771<0.001
Disorientation25.056 ± 35.517101.152 ± 66.801−8.576<0.001
Total12.332 ± 13.85441.044 ± 24.779−8.904<0.001
Table
Table 2. Differences of absolute power of electroencephalogram (EEG) with eyes open before and after watching a video.
Table 2. Differences of absolute power of electroencephalogram (EEG) with eyes open before and after watching a video.
Before VRAfter VRtp
Mean ± SD
Delta
 Anterior17.455 ± 6.01017.261 ± 8.1240.1500.882
 Middle7.202 ± 3.1457.212 ± 2.789−0.0170.986
 Posterior8.844 ± 3.2029.061 ± 3.608−0.6370.529
 Global10.314 ± 3.32010.463 ± 3.934−0.2510.804
Theta
 Anterior6.027 ± 2.6127.017 ± 4.444−2.5440.017
 Middle3.550 ± 1.7624.076 ± 2.884−2.2370.033
 Posterior4.312 ± 2.5475.467 ± 4.779−2.6610.013
 Global4.477 ± 2.2345.380 ± 3.967−2.6490.013
Alpha1
 Anterior3.976 ± 3.9575.764 ± 7.507−2.5590.016
 Middle2.703 ± 2.5423.526 ± 3.453−3.4090.002 *
 Posterior5.689 ± 5.5959.025 ± 10.547−3.1060.004 *
 Global4.097 ± 3.9606.055 ± 6.942−3.1030.004 *
Alpha2
 Anterior4.533 ± 4.6725.627 ± 5.697−2.9910.006 *
 Middle2.991 ± 2.6843.420 ± 2.860−2.3810.024
 Posterior7.695 ± 8.2029.943 ± 9.856−3.2480.003 *
 Global4.999 ± 5.0036.233 ± 5.951−3.2330.003 *
Beta1
 Anterior3.094 ± 1.3583.158 ± 1.174−0.4000.692
 Middle2.819 ± 1.5922.534 ± 1.0891.4890.147
 Posterior3.331 ± 1.7513.676 ± 1.704−2.4410.021
 Global2.942 ± 1.2633.063 ± 1.216−1.1910.243
Beta2
 Anterior4.944 ± 4.0664.708 ± 2.8110.5460.589
 Middle3.394 ± 2.5802.927 ± 1.7211.5580.130
 Posterior2.301 ± 1.4802.562 ± 1.384−1.7320.094
 Global3.220 ± 1.7543.181 ± 1.3890.2400.812
Gamma
 Anterior5.145 ± 4.3564.824 ± 3.4590.5170.609
 Middle3.868 ± 3.5693.214 ± 2.5841.4320.163
 Posterior1.414 ± 1.4391.532 ± 1.308−0.5720.572
 Global3.058 ± 1.8952.950 ± 1.6140.3810.706
* padj < 0.0125 (Bonferroni correction adjusted p-value: 0.05/4 = 0.0125, where 4 refers to four brain sub-regions).
Table
Table 3. Differences of brain source activities with eyes open before and after watching a video.
Table 3. Differences of brain source activities with eyes open before and after watching a video.
Frequency Band and Brain RegionMNI CoordinateBefore VRAfter VRtp
Mean ± SD
Alpha1 and Cuneus(−5, −80, 35)342.635 ± 236.021515.076 ± 423.659−3.631<0.001 *
Alpha2 and Cuneus(−5, −85, 35)590.988 ± 462.924795.884 ± 683.322−3.657<0.001 *
Alpha2 and PCG(5, −70, 15)370.731 ± 323.503493.818 ± 442.367−3.633<0.001 *
* padj < 0.00151 (Bonferroni correction adjusted p-value: 0.05/33 = 0.00151, where 33 refers to 33 brain sub-regions).

Share and Cite

MDPI and ACS Style

Kim, J.-Y.; Son, J.-B.; Leem, H.-S.; Lee, S.-H. Psychophysiological Alteration After Virtual Reality Experiences Using Smartphone-Assisted Head Mount Displays: An EEG-Based Source Localization Study. Appl. Sci. 2019, 9, 2501. https://doi.org/10.3390/app9122501

AMA Style

Kim J-Y, Son J-B, Leem H-S, Lee S-H. Psychophysiological Alteration After Virtual Reality Experiences Using Smartphone-Assisted Head Mount Displays: An EEG-Based Source Localization Study. Applied Sciences. 2019; 9(12):2501. https://doi.org/10.3390/app9122501

Chicago/Turabian Style

Kim, Jeong-Youn, Jae-Beom Son, Hyun-Sung Leem, and Seung-Hwan Lee. 2019. "Psychophysiological Alteration After Virtual Reality Experiences Using Smartphone-Assisted Head Mount Displays: An EEG-Based Source Localization Study" Applied Sciences 9, no. 12: 2501. https://doi.org/10.3390/app9122501

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop