Keywords

1 Introduction

Virtual Reality is quickly becoming a familiar household term. Companies such as Oculus, HTC, and Sony have made virtual reality for entertainment both affordable and available to a much larger consumer base than it has ever been before. The commercialization of Virtual Reality (VR) Head-Mounted Displays (HMD) has driven a rapid increase in the technological capabilities and widespread availability of these types of devices. While the focus of the majority of VR HMD commercial technology is for entertainment, because of their lower cost than legacy HMDs for virtual reality, many other organizations are turning their attention towards these devices as a means of highly immersive training for military, education, or health-related domains.

Despite the fact that the technologies utilized for VR have increased in both efficiency and capability, there are still some factors that warrant further investigation. One of these factors is the occurrence of simulator sickness. Simulator sickness, similar to motion sickness, refers to instances where individuals experience symptoms such as nausea, sweating, disorientation, etc. because of their interaction with some type of virtual environment [1, 2]. In the case of VR, research to date has indicated that simulator sickness is a common occurrence in participants using HMDs. Much of this research has indicated varying contributing factors to the onset and severity of simulator sickness symptoms, ranging from technical capabilities (e.g., refresh latency [3], Field of View [4]), to individual head movements [5], to duration spent interacting with the system [6, 7].

The majority of these findings are associated with how the perception of visual information through the HMD may not necessarily align with the habitual physical movement that is expected when perceiving physical motion visually. Typically, people experience feelings of motion via a combination of visual, vestibular, and proprioceptive sensory input [8], therefore, a mismatch between these sensory inputs is a central supporting theory to the cause of motion and simulator sickness [9, 10].

Despite this, organizations are looking towards VR and HMDs as a solution for training and entertainment without concern for the potential side effects of VR exposure. Part of this issue arises from the fact that newer iterations of these technologies are constantly changing the way with which individuals interact with these technologies (i.e., room-scale motion tracking; controller input; etc.). Therefore, the objectives of this research were twofold:

  1. (1)

    Examine one newer VR HMD product and evaluate the impact of its general usage.

  2. (2)

    Examine how specific technology capabilities of the system, such as the ability to physically move while using the HMD, affects the occurrence of simulator sickness while completing different types of tasks within the virtual world.

While some prior research has addressed the relationship between vection, simulator sickness, and display type [11], current literature search results generally lack examinations or evaluations of the newest HMD technology to date.

2 Background

2.1 Virtual Reality and Head-Mounted Displays

The number of novel immersive HMDs for VR use has grown at a rapid pace recently. These HMDs range in technological complexity, from well-designed cardboard to room-scale headsets with onboard laser and infrared movement tracking systems. The majority of these headsets takes advantage of stereoscopic image or video presentation, which creates a sense of visual depth by presenting two images of the same subject matter, from slightly offset perspectives, to the left and right eyes simultaneously [12]. The visual system then converges the separate images into one, creating a realistic sense of depth in the visual field.

The increases in accessibility and lower relative costs has led to a desire to utilize these technologies for various purposes across different domains. For example, VR-HMDs have been studied as a means of treating posttraumatic stress disorders and rehabilitation [13,14,15]. Additionally, research has tested how well VR may aid in emergency navigational training in spacecraft [16], as well as its efficacy for teaching engineering concepts to students [17]. The military has also shown an increased interest in using VR for various flight and vehicle, medical, and combat training programs [18, 19].

A few possible reasons why these domain areas are interesting in VR stem from the fact that, not only has VR HMD technology become much cheaper, it also provides an incredibly high level of realism and the ability to create virtual environments and interactions that may not be accessible or possible in certain circumstances. However, a significant issue with simulator sickness persists in various forms of VR, particularly those involving HMDs.

Simulator Sickness in VR.

Numerous studies have examined the effects of VR and HMD use on simulator sickness. Simulator sickness is similar to motion sickness in its symptoms; however, the former is elicited from the use and interaction with some form of simulator or virtual environment. A prevailing explanation for the onset and experience of simulator sickness is derived from the sensory conflict theory [10]. Sensory conflict theory posits that a mismatch of sensory information received from the visual and vestibular systems results in symptoms related to simulator sickness. This means that when visual motion observed in an HMD is not accompanied by a physical sensation (i.e., expectation) of movement, a sensory mismatch occurs and results in symptoms such as nausea, fatigue, dizziness, and/or disorientation.

Previous research has pointed to numerous potential causes for the onset of simulator sickness. For example, researchers have reported that factors such as visual motion (e.g., riding a virtual roller coaster) in HMDs can lead to feelings of vection and simulator sickness [11]. In addition, other research has shown that certain types of visual occlusion may lead to increased simulator sickness symptoms [6]. Variations in visual image latency have also been shown to lead to an increased onset of symptoms when using HMDs [3]. Furthermore, the duration in which one uses or interacts with a VR environment has been shown to consistently lead to significant increases in symptom presence [7, 20, 21].

Simulator sickness is an important issue to consider when deciding to use a VR HMD for training, learning, or entertainment. Despite the available information explaining the shortcomings and potential negative effects of using such devices, many industries and organizations are rushing to integrate them into their daily operations. While there are benefits of utilizing VR within these various domains, the negative effects have the potential to lessen the impact and efficacy of these technologies, depending on how they are used.

2.2 The Current Study

While there are many reported causes of simulator sickness while using an HMD in a VR environment, many commercial VR HMD products are still making their way to the market. To date, some of the newest HMDs claim to have minimized the effect of simulator sickness and increased the overall comfort for the user. However, there exists little-to-no published research to substantiate those claims.

Therefore, the purpose of this pilot study was to examine, initially, how different types of head movements while using a VR HMD affect reported feelings of simulator sickness. The HMD of interest in this study was the commercially available HTC Vive. The HTC Vive allows users much more freedom of movement within a given area while using the HMD than what has been available in earlier versions of similar technologies. Therefore, it was predicted that increased physical movement while using and interacting with the HTC Vive to view a VR environment would lead to lower onsets of reported simulator sickness. Lower simulator sickness ratings are expected as a result of the increased alignment of visual and vestibular sensory information due to a user’s physical movement matching the visual input they receive. It was also predicted that participants experiencing lower levels of simulator sickness would obtain higher scores on specific performance tasks in the VR environments.

This experiment was designed as a small piece of a larger project that seeks to develop quick methods for conducting human factors-based evaluations for various commercial and non-commercial HMD and mixed reality technologies.

3 Methods

3.1 Participants

A total of 24 participants (12 female) were recruited from the University of Central Florida and surrounding Central Florida areas for participation in the experiment. The average age of these participants was 19.75 years (SD = 2.21). Participation was restricted to U.S. citizens between the ages of 18–45 who had normal or corrected-to-normal vision. All participants were prescreened to ensure that all eligibility requirements were met. Of the 24 participants, 11 (45.83%) indicated having some type of prior experience with virtual reality applications.

Participants were also screened on whether or not they, at any point in the past, considered themselves susceptible to motion sickness. Of the 24 responses, three participants had indicated being susceptible to motion sickness in the past.

3.2 Experiment Materials and Design

Simulation Equipment.

This experiment utilized the commercially available HTC Vive Virtual Reality HMD. The HTC Vive hardware system includes the core HMD, two handheld controllers, and two optical tracking base stations. The HMD and controllers are tracked through a joint LASER/IR light blast from the base stations, tracking the location, position, and rotational information of the HMD and controllers in real time. The display inside the HMD consisted of two screens with a maximum resolution of 1080P displayed to each eye. Images in the HMD were displayed stereoscopically, creating a visual sense of depth and 3-Dimensional (3D) virtual imagery.

The HMD was connected to a host computer, as required, via HDMI and USB 3.0 connections, and required a dedicated power source. This cable bundle was connected to the HMD along the top of the physical hardware, and was supported by a pulley system that ran along the ceiling above the walkable area and to the host computer, thereby limited potential tripping hazards. The computer was an Alienware desktop PC with an Intel Core i7-4930K processor, 32 GB of RAM, and an NVIDIA GeForce GTX 970 graphics card.

In addition to the HMD and accompanying equipment, the experimental design required some freedom of movement within the lab space. This space was virtually mapped and designated using the accompanying software suite for the HTC Vive and consisted of a square space, or play area, of approximately 8.82 m2 (3.05 m × 2.89 m).

Experimental Design.

This pilot experiment was designed with three conditions to measure how types of head movements, specifically rotation and tilt, affected simulator sickness ratings and usage/performance. All conditions required wearing the HMD during the experimental session. The first condition required participants to tilt their heads to the left or right (e.g., tilting the top of the head towards a shoulder) during the game-based experimental task. The second condition required participants to turn their head from side-to-side (i.e., a “looking” motion, turning the chin towards the shoulders) during the game-based experimental task. The third condition did not require head movements from the participants during the game-based task.

Participants spent approximately the same time using the HMD and completed identical tasks or experiences in all three conditions.

VR Experiences, Experimental Tasks, and Dependent Measures.

Participants experienced and completed tasks in three separate VR environments. The first consisted of an observational VR environment, with low levels of direct user interaction, which was set underwater. The environment was created to simulate a realistic underwater diving experience. Participants controlled all movement, motion, and perspective changes in the environment by moving freely within the previously designated play area. The first VR experience lasted approximately 8–9 min.

The second VR environment was a linear-style game that required participants to fly a virtual plane-like object through a field of obstacles. The objective of the game was to fly as far as possible without crashing. When crashes occurred, the distance score was recorded and participants were told to restart the game. This was the manipulation task. Participants controlled the left-right turning of the plane via one of the head-movement control methods (i.e., tilt or turn) or handheld controller. Participant activity in the game lasted approximately 5–6 min.

The third VR environment consisted of a virtual space, laid out as a square (Fig. 1). Participants were tasked with picking up various colored boxes and placing them into the matching colored bins along the far side of the virtual task space. Movement within the space was required to complete the task. Participants were also required to use of both handheld HTC Vive controllers to pick up and place boxes into the bins. There were five playable levels, with each subsequent level containing an increasing number of boxes for the participant to collect within a two-minute (120 s) time allotment for each level. Time to completion was recorded for each level.

Fig. 1.
figure 1

Virtual environment for the box collection task

Full size image

The Simulator Sickness Questionnaire [22] is a subjective, self-report measure used to evaluate the overall sickness ratings related to simulator usage. Participants provided responses to the 16-items on the measure, indicating the level of severity for each particular symptom associated with simulator sickness along 4-point scale (i.e., None/Slight/Moderate/Severe).

Procedure.

The experiment lasted approximately 1.25 h. Each participant completed all the experiment individually. Participants first reviewed a consent form prior to beginning. After providing consent, participants completed a set of questionnaires consisting of background/demographics items, including age, experience with VR, and prior susceptibility to motion sickness, and an initial SSQ to establish a baseline.

Next, participants were introduced to the HMD and accompanying technology used for the experiment. This introduction included guidance on how to fit the HMD on their head using the adjustable straps. Once the HMD was secured in a comfortable and appropriate manner to the participant’s head, the experimenter launched a VR tutorial application that explained to the participant how to safely move around the play area and use the controllers.

The experimental session began immediately upon completion of the tutorial. The entire session was broken up into three smaller sessions. These smaller sessions corresponded with the three different VR experiences/tasks designed for the experiment. Participants first went through the observational VR experiences, followed by the distance game, and finished with the box collection task. In between each session, they were asked to fill out an additional SSQ pertaining to their current state. After completion of the box collection task, participants filled out a final SSQ, were debriefed on the nature of the research, compensated for their participation, and dismissed.

4 Results

The preliminary data for the pilot test was first analyzed to check for extreme outliers on the first administration of the SSQ. Twenty-six individuals originally completed the study. Two participants reported initial SSQ ratings beyond two standard deviations from the mean. These participants’ data were removed from the analyses because of the potential to alter the subsequent SSQ comparisons due to factors other than the experimental manipulations.

4.1 Individual Session Simulator Sickness Scores

Participants in all conditions completed SSQs prior to wearing and interacting with HMD and VR environments. Additional SSQ scores were collected after each of the three experimental sessions. An analysis of the initial SSQ scores indicated that there were no significant differences between groups prior to exposure to experimental manipulations (F(2,21) = 0.76, p = .48).

SSQ scores for the three experimental sessions were also compared individually between groups. As Table 1 depicts, there were no significantly different scores reported between groups during any of the three experimental sessions.

Table 1. SSQ results between conditions for the experimental sessions
Full size table

4.2 SSQ Scores Over Time

In order to determine if using the HMD led to an increase in SSQ scores throughout the session, SSQ scores were also compared over time. In order to test this, a repeated measures ANOVA was conducted. The assumption for sphericity was violated; therefore, Greenhouse-Geisser corrected statistics were used. Results showed a significant change in overall SSQ ratings over time from all participants (F(1.71, 35.86) = 11.01, p < .001). However, no differences were observed in SSQ scores changes over time between groups, indicating that scores changed at similar rates across conditions. The values for overall mean SSQ scores over time are presented in Table 2.

Table 2. Mean overall scores on all SSQs
Full size table

Post hoc pairwise comparisons revealed that participants reported significantly higher ratings on nearly all of their four subsequent SSQ score, with the exception of SSQ3 and SSQ4 (see Table 3). Results indicated a consistently rising SSQ score throughout the duration of the experiment, except between SSQ3 and SSQ (i.e., over the final session).

Table 3. Pairwise comparisons for SSQ scores over time
Full size table

4.3 Experimental Group Task Performance

Head-Turning Game Task.

The second task of the experiment involved controlling an object through and between obstacles. Participants attempted to fly as far as they could before crashing. In this case, distance was measured in arbitrary units, but these units were standard and consistent throughout the game. All participants completed at least three trials before time expired. The mean of all valid trials was computed and used to compare scores between groups.

Results from a one-way ANOVA revealed differences between groups on the distance task (F(2,21) = 4.076, p = .032). Individual group comparisons results showed that the Head-Tilt group (M = 5166.14, SD = 1530.85) significantly outperformed the Head-Turn group (M = 3383.97, SD = 1184.16; t(16) = 2.79, p = .013), but no other comparisons reached statistical significance with the Controller group (M = 4349.7, SD = 1239.53).

Furthermore, no statistically significant differences were observed on the SSQ collected immediately following the completion of this task between groups(F(2,21) = .034, p = .97), indicating similar scores on the measure.

Box Collection Task.

The box collection task performance was measured on completion time of the task over five levels of increasing difficulty. A repeated-measures ANOVA was used to analyze scores between conditions over time. Results indicated no significant differences between groups over trials, indicating that all groups completed the task’s levels within similar times. Results also indicated a significant difference in level completion time within the entire sample (F(4,84) = 115.13, p < .001). However, this result was expected due to the increasing number of boxes per level (i.e., difficulty level).

Additionally, no differences were observed between groups on SSQ scores immediately following the box collection task (F(2,21) = .646, p = .534).

5 Discussion

The goals of this pilot experiment were two examine a newer VR HMD, and to assess whether or not the newer technological features of the device help to minimize previously consistent findings of simulator sickness during usage. The HTC Vive was considered a much more technologically advanced HMD than similarly marketed VR HMDs (e.g., Oculus Rift DK2, Samsung Gear VR) due to its ability to afford a greater range of motion within the designated play area. Earlier generation VR HMDs typically required users to sit or stand in a generally stationary location. This affordance means that users were able to move around the virtual environment in order to change their perspective or view, potentially minimizing the effects of visual-vestibular mismatch, which could lead to lower levels of reported simulator sickness [1, 2].

It was predicted that the increased affordance of physical movement that matched the visual presentation would lead to lower ratings on simulator sickness. The first analysis revealed no differences in simulator sickness ratings between the three types of head-movement experiment groups on all experimental sessions. Similar scores were expected after the first session because no direct manipulation of head movement occurred; participants were encouraged to walk around and observe the realistic cinematic environment freely. However, the second and third session also failed to reveal any significant differences between group SSQ scores. This could be due to a number of factors. First, the overall time spent in the second session, requiring specific head movement, may have been too short to elicit differences in scores between groups. Second, HTC Vive allowed for much higher levels of physical movement than older versions of similar technologies examined in previous studies. It may be the case that this freedom of movement generally led lower levels of simulator sickness as a whole.

Results revealed an overall increase in SSQ scores throughout the experiment from all participants. While these scores did significantly increase from the original baseline measure, average scores did not reach uncomfortably high sickness ratings. Despite this, these results still support the trend of previous findings that duration of exposure to VR leads to higher simulator sickness ratings. While not specifically tested in this research, longer VR interaction and durations might have shown increased sickness ratings. Future research needs to examine this factor more thoroughly.

Results on the game-based flight-obstacle task only revealed differences between the head-tilt versus the head-turn groups. However, these results could not be explained by simulator sickness ratings, as these groups had similar scores on the SSQ for this session. It may be the case that one type of control was inherently easier to manipulate than the other. Participants in the head-tilt group were able to keep their eyes facing forward the entire time and their head movement matched the visual representation of movement from the plane-like object they were controlling (i.e., a left-side head tilt corresponded with the plane rolling to the left). In contrast, participants in the head-turn group were required to move the position of their eyes as they initiated turns. This prevented a direct, straight-on view of the plane-like object and obstructions in its path. Additionally, the type of movement did not match the visual representation of movement from the plane. These factors may have led to the discrepancies in distance scores for this task.

Finally, box-collection task performance was similar between all experimental groups and no statistically significant differences were observed on simulator sickness scores. All participants were able to complete the task levels in equal times between groups. While not significant, there is a noticeable drop in SSQ score in the controller (i.e., no head movement) group. Unfortunately, the low sample size and high variance in scores from the other groups led to non-significant findings.

6 Conclusions

The findings in this experiment help to shed light on potential testing methods and approaches for verification of manufacture claims for comfort and usability in relation to simulator sickness. Despite the many studies focused on simulator sickness and HMDs, the literature is still lacking a general procedure or taxonomy of testing methods for evaluating various HMDs for their technical abilities and potential effects on users. While small in scale, results from this study will help guide future research focusing on the development of specific procedures for evaluation of new visually-based mixed reality technologies.