High-fidelity vibrokinetic stimulation induces sustained changes in intercortical coherence during a cinematic experience

Objective. High-fidelity vibrokinetic (HFVK) technology is widely used to enhance the immersiveness of audiovisual (AV) entertainment experiences. However, despite evidence that HFVK technology does subjectively enhance AV immersion, the underlying mechanism has not been clarified. Neurophysiological studies could provide important evidence to illuminate this mechanism, thereby benefiting HFVK stimulus design, and facilitating expansion of HFVK technology. Approach. We conducted a between-subjects (VK, N = 11; Control, N = 9) exploratory study to measure the effect of HFVK stimulation through an HFVK seat on electroencephalographic cortical activity during an AV cinematic experience. Subjective appreciation of the experience was assessed and incorporated into statistical models exploring the effects of HFVK stimulation across cortical brain areas. We separately analyzed alpha-band (8–12 Hz) and theta-band (5–7 Hz) activities as indices of engagement and sensory processing, respectively. We also performed theta-band (5–7 Hz) coherence analyses using cortical seed areas identified from the theta activity analysis. Main results. The right fusiform gyrus, inferiotemporal gyrus, and supramarginal gyrus, known for emotion, AV-spatial, and vestibular processing, were identified as seeds from theta analyses. Coherence from these areas was uniformly enhanced in HFVK subjects in right motor areas, albeit predominantly in those who were appreciative. Meanwhile, compared to control subjects, HFVK subjects exhibited uniform interhemispheric decoherence with the left insula, which is important for self-processing. Significance. The results collectively point to sustained decoherence between sensory and self-processing as a possible mechanism for how HFVK increases immersion, and that coordination of emotional, spatial, and vestibular processing hubs with the motor system may be required for appreciation of the HFVK-enhanced experience. Overall, this study offers the first ever demonstration that HFVK stimulation has a real and sustained effect on brain activity during a cinematic experience.


Introduction
The concept of incorporating vibration and kinetic (VK) stimuli to enhance the immersiveness of audiovisual (AV) entertainment has advanced considerably since its conception at the turn of the 20th century with shaky camera tricks in 'phantom rides' and clunky manually operated actuators in Hale's Tours. Over time, VK technology evolved from theme-park ride motion systems to sophisticated high-fidelity 4 Author to whom any correspondence should be addressed. VK (HFVK) technology embedded in specialized seats (hereafter, HFVK seat) where HFVK stimuli are delivered in perfect temporal synchronization with AV effects. Today, HFVK technology is featured in cinemas worldwide to enhance the immersiveness of films of all genres, from action and suspense to musicals and dramas. Despite the advancement of HFVK technology, and evidence indicating that HFVK stimulation does subjectively enhance AV immersion (Oh et al 2011), the mechanism underlying HFVK technology's immersive effects has not been clarified. Consequently, the design of HFVK stimuli largely remains an artistic process (Petersen 2019), thereby limiting the optimization of HFVK design and the expansion of HFVK technology beyond mere entertainment. HFVK stimulation in AV contexts has been reported to increase the perceived loudness and physiologically arousing nature of music, and improve memorability of AV content (Merchel and Altinsoy 2014, Giroux et al 2019, Pauna et al 2019, clearly suggesting effects on cognition and brain activity, and the potential to measure these effects. Studies to identify brain areas and activities affected by HFVK stimulation during AV experiences could be a valuable first step in building a mechanism for how HFVK technology enhances AV immersion. However, this line of research remains unexplored. Immersion is defined by the Oxford dictionary as, 'the state of being completely involved in something' 1 , and can be cognitively linked to concepts of taskdirected attention and engagement. Indeed, studies regarding immersion in AV entertainment such as gaming and virtual reality entertainment have characterized immersion in these terms (Fairclough et al 2013, Burns andFairclough 2015). Neurophysiologically, attention and engagement have been classically indexed by alpha band brain activity (Mulholland 1968), where desynchronized activity has regularly been associated with attention in visual (Klimesch et al 1998, Wright et al 2015 and auditory tasks (Weisz et al 2011, Rapela et al 2012. The phenomenon of alpha desynchronization has also been observed in association with attention to combined aural and visual stimuli in AV tasks. Indeed, Feng et al (2017) observed that the combination of attention to sound and improved visual perception corresponded with desynchronized posterior alpha (8-14 Hz) activity. In the context of sustained attention to AV stimuli, Smith and Gevins (2004) reported that frontal alpha activity was attenuated during engaging compared to non-engaging commercials that had been subjectively rated by a judiciary panel. Particularly relevant to the present study, Dmochowski et al (2012) found that during the viewing of critically acclaimed films, engaging scenes were associated with high betweensubject neural correlations of desynchronized alpha (8-13 Hz) activity. Together, this evidence supports the use of alpha activity as an index of sustained attention and engagement during an extended cinematic AV experience.
Another characteristic of cognitive immersion is thought to be enhanced task-related sensory processing (Ermi andMäyrä 2005, Burns andFairclough 2015). A cinematic AV experience with HFVK stimulation would intrinsically engage sensory processing for auditory, visual, and vestibular stimuli. Theta band activity (5-7 Hz) is thought to reflect bottom-up sensory processing and encoding (Arnal et al 2015). Phase modulation of theta activity in sensory processing areas has been linked to tracking continuous auditory and visual stimuli (Fuentemilla et al 2008, Luo et al 2010. Event related synchronization of theta activity is frequently reported following presentation of visual stimuli, and is thought to reflect encoding of visual information into memory (Klimesch et al 2011). With respect to the vestibular system, much of the work in humans has focused on spatial processing, to which theta activity is also responsive. Processing of spatial cues and landmarks induced electroencephalography (EEG) theta activity (4-8 Hz) in a virtual navigation task (Kober and Neuper 2011). MEG-based hippocampal theta (4-8 Hz) highly correlated with spatial performance in a virtual maze task (Cornwell et al 2008). Additionally, stimulation with theta via transcranial magnetic stimulation improved spatial working memory (Alekseichuk et al 2016). As for sustained attention and engagement, frontal and parietal theta oscillatory activity has been shown to increase to attended versus non-attended repetitive visual and auditory stimuli Demiralp and Başar (1992). Meanwhile, in the aforementioned study by Dmochowski et al (2012), a component of increased theta activity that emerged during engaging movie scenes was also found to significantly correlate across subjects. Collectively, the above supports not only the relevance of theta activity in sensory processing, but also the use of event related changes in theta activity as an index of sensory processing during sustained sensory stimulation.
In addition to its recognized role in sensory processing, in recent years theta oscillatory activity has come to be recognized as important for intercortical communication, which is frequently measured via coherence (Liebe et al 2012, Mellem et al 2013, Muthukrishnan et al 2019. Coherence measures are thought to reflect the level of communication between cortical brain areas, based largely on the concept that postsynaptic neurons are more responsive to coherently firing presynapatic neurons (Fries 2005(Fries , 2015. Coherence has been used in a wide range of tasks in neuroimaging studies. Sauseng et al (2005) have shown that during mental manipulation of memory items, compared to memory retrieval, EEG-based sensor-level theta coherence patterns were indicative of widespread interhemispheric and frontal-parietal connectivity. Chikara and Ko (2019) reported that activation of inhibitory demands using an auditory alarm during a visual task increased theta coherence between auditory processing areas in the temporal cortex and inhibition-associated areas in the right prefrontal cortex. Kitaura et al (2017) observed that during the solving of arithmetic problems, compared to rest, theta coherence increased in left prefrontal areas. Meanwhile, Rajan et al (2018) demonstrated how willed versus passive attention induced increased theta coherence between frontal and parietal brain areas. This evidence, although not specific to AV cinematic experiences per se, illustrates that theta coherence patterns are quite task-specific, and have the potential to provide functional insight beyond that which can be gleaned from measures of cortical band activity alone.
Finally, immersion and engagement in an AV experience cannot be said to be universally guaranteed. These states are arguably dependent upon one's appreciation of the experience. Appreciation, in turn, can also affect brain response. Boksem and Smidts (2015) found that the level of appreciation of viewed movie trailers induced EEG brain activity that was differential enough to reliably predict subjective preference for the trailers. Hayashi et al (2016) reported widespread differences in non-phase-locked sensorlevel MEG theta (4-8 Hz) and alpha (8)(9)(10)(11)(11)(12)(13)(14) activity when comparing brain responses to pleasant versus unpleasant visual stimuli. Meanwhile, in a study looking at brain activities during commercials played intermittently during and extended AV experience, Vecchiato et al (2010) observed that theta (4-7 Hz) and alpha (8-12 Hz) activity were significantly higher in frontal areas for commercials that were liked compared to those that were disliked. This evidence strongly suggests the importance of controlling for appreciation when measuring brain activity during AV experiences, either through homogeneous sample selection or through incorporation of appreciation as a factor into statistical models.
With the present exploratory study, we considered the influence of subjective appreciation as we conducted a between-subjects investigation in healthy middle-aged adults to measure the effect of HFVK stimulation through an HFVK seat on EEG cortical activity during a cinematic musical, one of the film genres targeted by HFVK technology. Subjective appreciation of the experience was measured via post-experiment interviews, and incorporated into statistical models that explored the effects of HFVK stimulation and subjective appreciation, and the interaction of these factors across cortical brain areas. Separate analyses were conducted for alphaband (8-12 Hz) and theta-band (5-7 Hz) activities. In a data driven approach, further functional insight was sought by performing theta-band (5-7 Hz) coherence analyses using cortical seed areas identified as statistically relevant from the theta activity analysis. In line with the aforementioned evidence and the underlying notion that HFVK stimulation can deepen the level of immersion during a cinematic experience, we hypothesized that VK stimulation would lead to decreased cortical alpha activity, increased theta activity, and potentially, increased theta coherence between sensory modalities. We also anticipated that these effects could be moderated according to subjective appreciation of the experience.

Subjects
A total of 22 subjects participated in the present study. EEG data from two of these were lost due to technical problems. Thus, data from 20 subjects (F: 13, M: 8) were used in our analyses, 11 HFVK subjects (F: 6, M: 5; ave. age ± SD: 53.2 ± 14.3 yrs), and nine control subjects (F:7, M:2; ave. age ± SD: 49.7 ± 8.1 yrs). There were no significant differences in sex or age between the HFVK and control subjects as per independent t tests. Written informed consent was obtained from all subjects prior to participation in the present study, which was conducted with approval by the Ethics Review Council of HEC Montréal and in accordance with the principles embodied in the Declaration of Helsinki.

Subject selection criteria and recruitment
To ensure a representative sample appropriate for the AV stimuli used, we targeted regular opera patrons whose age corresponded to that of the predominant opera viewing demographic. Here, we recognized that more than 70% of those that attend opera are aged 35 years and older, with a mean attendance age range of 45-54 years (N. E. of the Arts 2015). Therefore, only subjects aged 35 years and older were considered. Additional screening criteria were: right handedness, two or more years of music instrument experience, and moderate or greater level of opera appreciation (a score of 3 or higher on a scale of 1-5 to the question, 'How would you rate your level of opera appreciation?' , where 1 = none, 2 = low, 3 = moderate, 4 = high, 5 = very high). Exclusion criteria were: aural only music instrument practice method, professional music performance experience, history of neurological or psychiatric illness, history of epilepsy, use of a pacemaker, use of hearing aid device, chronic mobility problems (a precaution due to the use of the HFVK seat).
Subjects were recruited and assessed for our screening criteria via an online questionnaire in French implemented using the software Qualtrics (Provo, UT, USA). An online advertisement including a link to the questionnaire was distributed through the local opera house's newsletter and social media network, as well as through the research participation pool organized by HEC Montréal. Because the questionnaire and corresponding advertisement was in French, we also assume that it indirectly screened for French reading ability, an important point because the spoken language and captions in the AV stimulus were all in French. Subjects that passed screening were scheduled, and alternately assigned to either receive or not receive HFVK stimulation during the experiment (hereafter HFVK and control subjects, respectively).

HFVK stimuli
The HFVK stimuli were designed and programmed by D-BOX Technologies Inc. (Longueuil, Canada), and delivered via a D-BOX HFVK seat during the AV stimulus experience during the time segments shown in table 1. The HFVK stimuli were embedded in a proprietary motion code file that was designed to synchronize with the video track of the AV stimulus (i.e. reprogramming the audio of the AV stimulus does not affect HFVK synchronization) when played by the D-BOX media player.
The design of the HFVK stimuli is an artistic process, carried out using a digital audio workstation (DAW) in high temporal synchrony with the corresponding AV stimulus, which is referenced in its entirety as a channel in the DAW (Pauna et al 2019, Petersen 2019. For details on the engineering of the HFVK seat, the number of actuators and their configuration throughout the seat, please see the D-BOX motion system patents (Roy et al 2003, Boulais et al 2011, Rousseau et al 2018, Ménard and Lemieux 2018. In brief, vibration and motion stimuli are manifested with actuators. These mechanisms are embedded within the seat and provide a vibrokinetic spectrum ranging from 0 to 100 Hz. Moreover, different aspects of their movement capabilities can be programmed independently in separate channels in a DAW, permitting them to be activated simultaneously with different VK characteristics. This is akin to multiple instruments playing simultaneously in an orchestra, hence the high-fidelity aspect of HFVK technology. Furthermore, the DAW permits programing the onset timing of HFVK events with sub-millisecond precision.
Particularly relevant for the present study is that the HFVK stimuli were not synchronized to the dialogue of the opera, nor to the visual stimuli of the theatrical action. Rather, the HFVK stimuli were artistically created and synchronized specifically to target the music of the operatic score-rolling to the tempo accents of the violins, pitching with the melody, vibrating externally with melodic flurries of highpitched instruments, and vibrating core actuators for low pitched rhythmic accents. Hence why all the clips shown in table 1 for which the HFVK stimuli were designed are all musical segments of the opera. Just as each of these segments differed musically, so too did the specific HFVK stimuli designed for them. Thus, there were actually seven different HFVK stimulus designs tested in the present study. However, the analyses of the present study have focused on average brain response across all seven designs. In so doing, our analyses have captured brain response to the overarching characteristic unifying all seven HFVK stimulus designs. That is, their temporal alignment with, and specific targeting of the auditory musical elements of the AV stimulus. This design characteristic is clearly illustrated in figure 1 which shows the synchrony of the left auditory channel during Prélude with the movement of the HFVK seat, the latter which was recorded along with the EEG data via an accelerometer attached to the HFVK seat.

Audiovisual stimuli
The AV stimulus used for the present study was a 720p high definition cinematic recording of Georges Bizet's opera Carmen, produced by Opéra Royal de Wallonie-Liege, directed by S. Scappuci, with staging from H. Brockhaus in 2018. As mentioned, musical genres are existing targets of HFVK technology, and cinematic opera is a relevant emerging musical subgenre in the cinematic marketplace (Steichen 2009). This particular stimulus was chosen as it was a fairly recent overseas production of Carmen that subjects had likely not seen before. Indeed, no subjects reported having previous experience with this particular production. The complete Act I of this production was presented continuously to all subjects.
To prepare the recording for experimental presentation and enable it to be synchronized to the EEG recording, the original left and right channels of the 2.0 audio track of the stimulus were respectively reassigned, without manipulation, to the front left and front right channels of a 5.1 Dolby surround sound audio track using Adobe Premiere Pro (CS6, Adobe Systems Inc. San Jose, California). On the center channel, a 1000 Hz, 5 ms burst square wave was placed two seconds prior to the start, and four seconds after the end of all clips for which HFVK stimuli had been programmed (see table 1). The remaining channels on the 5.1 audio track were left blank. See table 2 for a summary of the 5.1 channel assignments.
The video was then exported at its original resolution with the new 5.1 audio track. This reprogrammed AV stimulus was played on a Windows operated laptop computer using a proprietary media player developed by the manufacturer of the HFVK technology used in the present study. The visual portion of the stimulus was displayed on a 70 × 120 cm high definition Samsung TV at a visual distance of 262 cm. The audio channels were routed via HDMI through a Pioneer VSX-324 AV receiver, with the front left and front right channels played on Mission 761 speakers (Mission, Huntingdon Cambridgeshire, UK) at a mean loudness of approximately 65 dB, with maximum and minimum peaks ranging between approximately 80 and 50 dB (measured using a REED R8080 sonometer). Note that this effectively meant that participants heard the audio in 2.0 stereo, as it was originally formatted. The center audio channel was routed directly from the receiver to the EEG trigger box where the occurrence of the square wave was directly transmitted and registered as an event in the EEG recording. The variables hh, mm, ss, and fr refer to hours, minutes, seconds, and frame, respectively Figure 1. High fidelity vibrokinetic (HFVK) stimuli were designed to correspond with the operatic score. An accelerometer attached to the HFVK seat recorded the HFVK stimuli in the EEG data set in synchrony with audiovisual stimuli. Top shows the entirety of Prélude. The close-up shows the first six and a half measures, with the corresponding musical notation of the main melody for reference. Audio is the left channel and low-pass filtered at 200 Hz for clarity with the slower HFVK stimuli.

Procedures
Subjects were greeted at the entrance of the laboratory and guided to the experiment room. The experiment and its purpose were explained, whereupon written informed consent was obtained. The circumference of the subject's head was then measured and an EEG electrode cap (EASYCAP, BrainProducts GmbH, Munich, Germany) was prepared with 32 electrodes (actiCAP, Brain Products GmbH, Munich, Germany) according to the 32ch Standard Cap layout for actiCAP. While the EEG cap was prepared, the subjects underwent a calibration procedure for eye tracking.
Once the EEG cap was ready, it was placed on the subject's head and conductive gel was applied and the impedance and function of all electrodes was checked and verified. Then, the position of the each electrode was digitized using CapTrak (Brain Products GmbH, Munich, Germany). Finally, subjects were seated in the HFVK seat with their lower back against the seat back and their feet slightly elevated off the floor. Additionally, a travel pillow was fitted around the subject's neck to ensure that the EEG cap did not make direct contact with the seat.
After the subject was properly and comfortably positioned, the lights were dimmed, and the experimenter moved into a neighboring observation room from which the subject could be observed via cameras and one-way glass. Communication to the subject was conducted via microphone and corresponding in-room speakers. After receiving verbal affirmation of readiness from the subject, the experiment was launched remotely using TeamViewer (TeamViewer, Inc. Tampa, FL, USA).
When the video of Act I of the opera completed, the experience was halted, and an interview to assess subjective appreciation was conducted (see next section). Subsequently, an unrelated second experiment was conducted. When both experiments were finished, the experimenter re-entered the experiment room, assisted the subject in the removal of EEG equipment, and then guided the subject to in-house facilities to wash out the electrode gel from the subject's scalp. Once the subject had finished washing, he or she was provided with $35 CAD compensation and guided to the exit of the laboratory building.

Subjective appreciation assessment
After subjects viewed the entire Act I of the opera, the experimenter asked them a verbal question from the observation room (via a microphone): 'Can you describe in words what this experience made you feel (the first words that come to mind)? ' We purposely designed the question to avoid having subjects affix subjective weight to the influence of HFVK in their experience, preferring instead to draw more objective insight from their brain activity. Subjects were free to express their impressions, and no time limit was imposed. The number of words expressed differed among subjects, ranging from less than three words up to 40 words. All answers were then coded and classified according to their valence using a qualitative and deductive approach based on emotion models widely used in sociology and psychology (Hillebrand 2000, Tashakkori and Teddlie 2015, Skjott Linneberg and Korsgaard 2019, Williams and Moser 2019). Borrowing from Watson et al (1988), the central postulate of our approach was that a positive or negative affective state can be subjectively assessed through verbal descriptors. Here, a positive affective state is characterized by a high level of energy, enthusiasm, engagement and pleasure. Conversely, a negative affective state is characterized by aversive emotional states such as anger, contempt, disgust, fear, nervousness, and even indifference. Thus, each interview response in the present study was supported by a positive or negative descriptor or descriptors. The total number of positive and negative descriptors was calculated for each subject and used to categorize them into two groups. Those with positive total values were categorized as appreciative. Those with total values of zero or below were categorized as non-appreciative.

EEG recording and initial processing
EEG signals were recorded raw at a 500 Hz sampling frequency using BrainVision Recorder (Brain Products GmbH). All EEG data processing was performed with Brainstorm 2 running on MATLAB 15 a (MathWorks, Natick, MA, USA). Noisy or dead channels were removed, and components of physiological artifacts and periodic noise were isolated and removed using independent component analysis. A band-pass filter was then applied from 1 to 40 Hz. A 90 s period subsequent to the second second of the opening scene (void of HFVK for both groups), and each ending HFVK event, were marked at fivesecond subintervals. Cleaned and filtered data was then epoched at − 1 to 6 s relative to these subintervals. Each epoch was visually scanned, and those with movement artifacts were removed. Subject head points and fiducials were coregistered to a common template brain. An EEG-appropriate head model was computed based on OpenMEEG BEM forward modeling which compensates for tissue conductance, and minimum-norm estimation was used to calculate cortical currents without dipole orientation constraints. To facilitate our exploratory analyses, the cortical surface was parcellated into 62 areas based on the Mindboggle cortical surface atlas provided with Brainstorm.

Source-level time-frequency decomposition
The time-series of cortical currents in each brain area were decomposed into the theta (5-7 Hz) and alpha (8-12 Hz) frequency bands, and their corresponding envelopes computed using Hilbert transform. Time-frequency envelopes in each frequency band in each brain area were averaged within subjects across all epochs corresponding to post-HFVK stimulation periods, to be used as the signal of interest, and across all epochs corresponding to the opening scene, to be used as baseline activity. The amplitude of the time-frequency envelopes for the signal of interest was standardized across subjects as a percent deviation from baseline using the following equation, Xstd = x−µ µ × 100, where x is the amplitude of the timefrequency envelope at each time point, and µ is the time-averaged signal over the baseline period.
Standardized mean time-frequency envelopes were averaged over the time period of 0-5 s in each of the 62 brain areas for each frequency band for each subject. Resulting values were used in statistical analyses. A diagram summarizing the EEG processing pipeline used to derive the values for these analyses is shown in figure 2.  Figure 2. Diagram of the electroencephalogram processing pipeline for comparisons of alpha (8-12 Hz) and theta (5-7 Hz) band power between high-fidelity vibrokinetic (HFVK) and control groups in accordance with subjective appreciation. Seven 90 s post-stimulus periods of cortical activity were divided into 5 s segments (epoched at − 1 to 6 s) and parcellated into 62 cortical areas based on the Mindboggle cortical atlas included within Brainstorm software. Alpha and theta band envelopes were derived. The mean across epochs in each cortical area was calculated, averaged over time (0-5 s), normalized with the baseline (BL) period, and used in statistical comparisons between groups. * Coherence analyses were conducted at this point in the pipeline, prior to time-frequency decomposition.

Source-level coherence analyses
Magnitude-squared coherence between the mean time-series activity within brain areas was calculated for all 62 x 62 brain area pairs (again based on the Mindboggle cortical surface atlas) at a maximum frequency range of 30 Hz, and a frequency resolution of 3 Hz, across the time window of 0-5 s for ever target epoch (i.e. non-baseline epochs) using Brainstorm's source-level coherence analysis pipeline. 3 The resulting coherence values for each epoch for all brain area pairs were averaged over all epochs to produce a single coherence value for each brain area pair for each subject. Coherence values in the frequency bin overlapping with the theta band used in the present study (5-7 Hz) were extracted for all brain area pairs in each subject. Finally, the coherence values from seed areas (identified as described below) to all other areas were extracted for each subject and used in statistical analyses.

Statistical analyses
A logistic regression was performed to assess the relationship between subjective appreciation and HFVK stimulation, sex, and age. Exploratory analyses to assess between-group differences in normalized source-level brain activity were performed for each frequency band separately using RM ANOVA, with normalized brain activity in each of the 62 brain areas as the repeated measure, and HFVK stimulation and appreciation of the experience as betweensubject factors. In the case of significant two or threeway interactions, parameter estimates are reported for brain areas where significant differences were observed. Additionally for these brain areas, posthoc one sample t-tests (Holm corrected for comparisons in each area) were performed to assess 3 https://neuroimage.usc.edu/brainstorm/Tutorials/Connectivity. whether activity levels in a given group or subgroup differed significantly from baseline. The brain areas where a significant interaction between the effect of HFVK stimulation and appreciation on theta-band activity was observed were used as seeds for subsequent exploratory analyses of between-group differences in coherence using RM ANOVA. Here, RM ANOVA were performed separately for each seed area, with coherence between the seed area and all other areas as the repeated measure, and condition and appreciation of the experience as between-subject factors. Again, in the case of significant two or threeway interactions, parameter estimates are reported for brain areas where significant differences were observed. The significance threshold for all tests was set at p ≤ .05. All statistical analyses were conducted using SPSS software version 22 (IBM, Armonk, NY, USA).

Appreciation
Of the 11 subjects who received HFVK stimulation, total appreciation valence was positive for five, and negative for six subjects. Of the nine control subjects, total appreciation valence was positive for five, and negative for four subjects. Notably, three of the five appreciative HFVK subjects directed their positive comments explicitly at HFVK technology: interesting (2), amusing (1). Moreover, five of the six nonappreciative HFVK subjects directed their negatively charged key descriptors specifically at aspects of the HFVK technology: disturbance (2), bothersome (1), dislike the chair (2). Nevertheless, the rate of positive and negative appreciation was not markedly different between experimental groups. Indeed, according to the logistic regression, subjective appreciation was not significantly impacted by HFVK stimulation, sex, or age (χ 2 (3) = 2.775, p = .428).

Source level alpha (8-12 Hz) activity
RM ANOVA revealed a significant two-way interaction between brain area and appreciation of the experience (F (1, 16) = 1.667, p = .001). However, no significant two-way interaction was observed between brain area and HFVK stimulation (F (61, 976) = 0.858, p = 0.772). Nor was there a three-way interaction between brain area HFVK stimulation and appreciation of the experience (F (61, 976) = .568, p = .997). Thus, alpha band brain activity appeared to have been affected by appreciation, but not by HFVK stimulation. Parameter estimates revealed significant differences in alpha activity in the left paracentral lobule (PCL; β = 39.660, p = .027) and the left and right superior parietal cortex (SPC; β = 39.729, p = .032 and β = 45.676, p = .045, respectively). In all three areas, alpha activity was higher for non-appreciative compared to appreciative subjects. Furthermore, onesample t-tests revealed that alpha activity levels were significantly elevated above baseline bilaterally in the SPC in non-appreciative subjects, and not significantly changed from baseline in any of the three areas in appreciative subjects (Holm-adjusted for comparisons within each area; left PCL: p = .068 & .639; left SPC: p = .002 & .534; right SPC: p = .020 & .298; for non-appreciative and appreciative subjects, respectively). Figure 3 (right) shows cortical maps highlighting these brain areas with significant differences in alpha activity according to appreciation, and (left) representative mean alpha activity in the left SPC according to appreciation of the experience.

Source-level theta band (5-7 Hz) activity
RM ANOVA revealed a significant two-way interaction between brain area and HFVK stimulation (F (61, 976) = 1.585, p = .004). A significant threeway interaction between brain area and HFVK stimulation and appreciation of the experience was also observed (F (61, 976) = 1.599, p = .003). Meanwhile, no significant two-way interaction was observed between brain area and appreciation of the experience (F (61, 976) = 1.042, p = .392). Parameter estimates did not reveal any brain areas where differences theta activity according to HFVK stimulation reached significance. However, brain areas where there was a significant interaction on theta activity between the factors of HFVK stimulation and appreciation of the experience were observed. These brain areas were all in the right temporal-parietal cortex, specifically in the: supramarginal gyrus (SMG; β = − 45.278, p = .036), inferiortemporal cortex (ITG; β = − 45.751, p = .042), and fusiform gyrus (FFG; β = -33.601, p = .045). Figure 4 (right) shows cortical maps highlighting the brain areas where a significant interaction in theta activity between HFVK stimulation and appreciation of the experience was observed. The trend in all three brain areas was for higher theta activity in appreciative HFVK subjects compared to non-appreciative HFVK subjects and appreciative control subjects. A trend towards higher theta activity non-appreciative control subjects was also observed compared to the same subgroups. However, activity in any area in any subgroup was not significantly different from baseline according to one-sample t-tests. Figure 4 (left) shows representative theta activity in the right ITG according to HFVK stimulation and appreciation of the experience.

ITG seed
RM ANOVA of theta-band coherence between the right ITG and other cortical brain areas revealed a significant interaction between HFVK stimulation and brain area (F (61, 976) = 1.828, p ≤ .001), but no significant interaction between brain area and appreciation of the experience (F (61, 976) = 1.064, p = .349), nor three-way interaction between HFVK stimulation, appreciation of the experience, and brain area (F (61, 976) = 1.156, p = .199). Parameter estimates revealed significant differences in coherence according to the factor of HFVK between the right ITG and the following brain areas: left insular cortex (IC; β = − .160, p = .026), right postcentral gyrus (PoCG; β = .110, p = .030), right precentral gyrus (PCG; β = .154, p = .019), right superior temporal gyrus (STG; β = .187, p = .012), and the right transverse temporal cortex (TTC; β = .191, p = .003). For all of these right-hemispheric brain areas, coherence between the right ITG was higher in HFVK compared to control subjects. For the left IC, coherence between the right ITG was conversely lower in HFVK compared to control subjects. A cortical map summarizing these results is shown in figure 5 (left).

SMG seed
RM ANOVA of theta-band coherence between the right SMG and other cortical brain areas similarly revealed a significant interaction between HFVK stimulation and brain area (F (61, 976) = 1.332, p = .049), and no significant interaction between appreciation of the experience and brain area (F (61, 976) = .392, p > .999). However, there was a significant three-way interaction between HFVK stimulation, appreciation of the experience, and brain area (F (61, 976) = 2.256, p ≤ .001). Parameter estimates revealed significant differences in coherence according to the factor of HFVK between the right SMG and the following right hemispheric brain areas: caudalmiddlefrontal cortex (CMF; β = .209, p = .003), entorhinal cortex (ERC; β = .139, p = .031), MTG (β = .148, p = .045), parahippocampal cortex (PHC; β = .126, p = .039), PCG (β = .216, p = .013), STG (β = .187, p = .011), and the TTC (β = .154, p = .016). For all of these areas, coherence was stronger between the right SMG for those that received HFVK compared to those that did not. Of these areas, there was additionally a significant interaction between HFVK stimulation and appreciation of the experience on coherence between the

rFFG seed
RM ANOVA of theta-band coherence between the right FFG and other cortical brain areas also revealed a significant interaction between HFVK stimulation and brain area (F (61, 976) = 1.532, p = .006), and no significant interaction between appreciation of the experience and brain area (F (61, 976) = .718, p = .949). As with the right SMG seed, there was a significant three-way interaction between HFVK stimulation, appreciation of the experience, and brain area (F (61, 976) = 1.576, p = .004). Parameter estimates revealed significant differences in coherence according to the factor of HFVK between the right FFG and the following right hemispheric brain areas: PoCG, (β = .106, p = .048), PCG (β = .151, p = .014), and the STG (β = .192, p = .034). In all cases, coherence was stronger for those that received HFVK compared to those that did not. Moreover, all cases exhibited an additional significant interaction between HFVK and appreciation of the experience (β = − .198, p = .013; β = − .196, p = .023; and β = − .269, p = .037, respectively). Meanwhile, there were additional significant interactions between HFVK stimulation and appreciation of the experience on coherence between the right FFG and the following additional right hemispheric brain areas: ITG (β = − .146, p = .029), IC (β = − .245, p = .032), and the TTC (β = − .248, p = .045). In short, there were no brain areas whose coherence with the right FFG was not influenced by appreciation of the experience. A cortical map summarizing these results is shown in figure 5 (right). Table 3 summarizes the results of the coherence analyses.

Discussion
The present study sought to identify brain areas and corresponding brain activities that are affected by HFVK stimulation during a cinematic experience, with the overarching aim of illuminating how HFVK technology can enhance AV immersion from the perspective of neurophysiology, a topic which has not previously been explored. We considered that engagement and sensory processing were two important aspects of immersion that could be indexed via alpha (8-12 Hz) and theta (5-7 Hz) band activity, respectively, and also potentially be influenced by subjective appreciation. Using an HFVK stimulus design specifically targeting auditory musical elements of the AV stream, our between-group comparisons revealed alpha activity to be significantly related to subjective appreciation, but not to HFVK stimulation. However, there was an effect of HFVK stimulation on theta activity in the right temporal cortex, which was furthermore associated with significant changes in intercortical theta coherence. Behaviorally, HFVK subjects directed positive and negative comments specifically at aspects of the HFVK stimuli, suggesting that HFVK stimulation had some level of specific influence on appreciation. However, the number of subjects with positive and negative appreciation was nearly identical in both groups. Thus, we conclude that whatever influence HFVK had on appreciation, it was not strong enough to alter the rate of positive and negative appreciation between groups as per the methodology of our assessment. Here, it is important to recall that the present study did not intend for subjective appreciation outcomes to discern the influence of HFVK stimulation. Rather, subjective appreciation was assessed under the presumption that it could influence task engagement, and correspondingly brain activity. This notion was supported by our alpha band activity results, which revealed that alpha activity differed significantly according to subjective appreciation, and not according to HFVK stimulation.
Subjects with negative overall appreciation of the AV experience exhibited significantly higher alpha activity bilaterally in the SPC and in the left PCL compared to appreciative subjects, and significantly higher bilaterally in the SPC compared to the baseline period. The SPC is frequently cited for its role in visual-spatial processing, and visual attention (Corbetta et al 1995, Wolpert et al 1998, Behrmann et al 2004, and has been found to be a key neural substrate for visual processing of cinematic content (Han et al 2005, Lankinen et al 2014. Thus, the involvement of the SPC in the present study could be an indication that our subjective appreciation measurement is particularly associated with visual engagement. Meanwhile, the PCL is a key region of the default mode network whose activity is frequently observed to distinguish between conscious states, with increased activation in PET and fMRI related to resting or internally-directed states (Gulyás 2001, Yang et al 2007. Conversely, fMRI-based deactivation of the PCL has been reported in correspondence with increasing demands on task-directed visual attention (Mayer et al 2010). The reason for the left laterality of PCL involvement in our study is not clear. However, the relevance of the left PCL specifically has also been reported in comparisons between states of rest and task engagement (Mazoyer et al 2001).
In line with the conventional view of its modulation according to task engagement, alpha activity in the SPC and PCL or at corresponding electrodes has been reported to increase during internally-directed engagement in musical imagery and meditation tasks (Lagopoulos et al 2009, Boasen et al 2018, and during disengaged states such as drowsiness (Okogbaa et al 1994). Whereas decreased alpha activity in the SPC and PCL has been observed to correspond to task-directed attention and increased task difficulty (Friese et al 2016). Here it is important to acknowledge that, although appreciative subjects did not exhibit desynchronized alpha activity indicative of enhanced engagement, they also did not exhibit significant deviations of alpha activity from baseline levels. Thus, comprehensively, our alpha band results are likely a simple indication that a conscious state of task-directed engagement, particularly in the visual domain, was maintained over the course of the experiment for those that appreciated the experience, and deteriorated for those that did not.
It was a different story for theta band activity, which we considered potentially reflective of sensory processing particularly in modalities outside of the visual domain. Indeed, our analyses revealed that neither factor of appreciation nor HFVK stimulation had an exclusive effect. Rather, the two factors interacted in their effect on theta activity in the right ITG, SMG and FFG. The right hemispheric nature of these results is remarkable for two reasons. One, processing of non-verbal auditory features such as sound intensity, pitch, and timbre has been recognized to occur in the temporal cortex and surrounding areas ipsilateral to the dominant hand (Zatorre et al 1992, Samson and Zatorre 1994, Zatorre and Belin 2001, Kohlmetz et al 2003, Brancucci et al 2005. All subjects in this study were right handed, and the HFVK stimulus was designed to specifically synchronize with these auditory elements. Two, lesion studies have indicated that spatial processing in humans is right lateralized to and around the temporal cortex (Griffiths et al 1997, Karnath et al 2001. Furthermore, the right SMG and ITG specifically have been shown to be particularly important for audiospatial processing, exhibiting greater activation during sound localization than during mere frequency discrimination (Weeks et al 1999). The SMG is also home to the temporal parietal junction, a critical brain area for vestibular processing and bodily self-awareness Zu Eulenburg et al (2012, Pfeiffer et al 2014, Klaus et al 2020, Shaikh et al 2020. Meanwhile, the right FFG, although notable for its role in facial recognition processing, has been shown to be more activated during processing of emotional versus neutral video stimuli, and during visual spatial processing versus visual form recognition (Shen et al 1999, Jessen andKotz 2015). Thus the right ITG, FFG and SMG have combined potential relevance toward audio/visualspatial, emotion, auditory integration and vestibular processing, respectively. We believe these processing modalities to be crucial for a task involving HFVK stimulation during cinematic opera.
In line with our hypothesis, visualization of theta activity in each subgroup revealed a trend towards higher activity in appreciative HFVK subjects compared to appreciative control and non-appreciative HFVK subjects. Nevertheless, a perplexing trend towards elevated theta activity in non-appreciative control subjects also existed. One potential explanation for this is that, sitting in a dark room in the absence of HFVK stimulation while watching a video they did not appreciate, the subjects in this subgroup were particularly drowsy. This notion is supported not only by the subjects' elevated alpha activity, but also by numerous reports that increased theta power accompanies increases in drowsiness (Torsvall 1987, Cajochen et al 1995, 1996, Morikawa et al 1997, including during extended periods viewing low-quality AV stimuli (Arndt et al 2016). The phenomenon is thought to originate from fatiguemediated changes in thalamic activity, and be functionally distinct from elevations in theta power due to attentiveness and task-based arousal (Schacter 1977). In this light, the trend towards elevated theta activity in non-appreciative control subjects was quite likely an artifact of their cognitive state, and functionally distinct from the trend towards elevated theta activity observed in appreciative HFVK subjects. To clarify this however, a deeper functional understanding was needed, and hence why coherence analyses were performed.
Common to all three seed areas, theta coherence was higher in HFVK subjects compared to control subjects with the right PCG and the right STG. The PCG comprises the primary motor cortex whose activation along with the motor system at large is thought to be important for interpreting sensory experiences (Kilner and Lemon 2013). With respect to music listening, a meta-analysis of fMRI-based studies by Gordon et al (2018) stresses the involvement of the motor system during music listening, and highlights the particular involvement of the right PCG. The right PCG has also been particularly implicated in the passive viewing of human movement and facial expressions (Iacoboni et al 1997, Leslie et al 2004. Viewing cinematic opera involves both music listening, and observation of emotional human actions. It thus logical, and perhaps to be expected, that vibration and movement during cinematic opera viewing would enhance coherence between the right motor cortex and areas associated with spatial and vestibular processing. The STG meanwhile has been observed to have preferential involvement in processing musical over non-musical sounds, and was proposed to functionally serve in the interpretation of acoustic features unique to music Angulo-Perkins et al (2014). Enhancement of coherence with the STG could thus be interpreted as reflective of the fact that the HFVK stimuli were designed in synchrony with different features of the musical stimulus, and that the increased recognition and processing of these features was an integral part of the HFVK stimulated experience.
That said, it is important to recognize that observing the subgroups separately revealed the effect of HFVK on increased coherence to be driven largely by appreciation for all seeds, and almost exclusively so for the emotional processing area, the FFG (see representative examples in figure 3 bottom panel).
It is unclear what is causal in this trend. It could either be that HFVK-induced coherence is moderated by appreciation, or that the strength of the HFVK-induced coherence determines appreciation. The latter would imply that coordinated communication across the sensory modalities identified is required for appreciation of a multidimensional HFVK-stimulated experience. The fact that nonappreciative HFVK subjects directed negative comments specifically at HFVK technology is supportive of this view.
The one notable exception to the above mentioned trends in coherence was decoherence between the ITG and the left IC uniformly exhibited in both HFVK subgroups regardless of appreciation. The lateralization of this effect to the left IC is intriguing. Left-right asymmetries in IC function have been acknowledged, though are not yet well understood. Craig (2009) highlighted that the anterior IC is bilaterally crucial for emotion, interoception and selfawareness, but proposed that left-right asymmetries in IC involvement may emerge due relationships with parasympathetic and sympathetic processing, respectively. This notion has since been echoed in a recent review on IC function by Evrard (2019). However, the extent to which parasympathetic or sympathetic processing may have been involved due to HFVK stimulation in the present study is difficult to say. Nevertheless, given the IC's bilateral importance to self-awareness, and the importance of both the right hemisphere and the ITG in spatial processing, perhaps we can offer a more simplified explanation. That is, HFVK stimulation induced a sustained decoherence with self-processing, but only that which was unrelated to task-related stimuli. This could represent a potential mechanism for how HFVK enhances immersion.
There were some limitations to the present study that should be acknowledged. First, only 32 electrodes were used in the whole-head EEG montage employed in this study. While this level of surface coverage would not be sufficient for pinpointing subcortical transient dipole sources, the spatial resolution at the cortical surface is grossly comparable to higher density coverage (Song et al 2015). Given this, we feel our approach to generalize and parcellate source activity into broad areas on the cortical surface was appropriate. However, one should remain cautious to not over-interpret the precise spatial location of these results within any given cortical area. Additionally, the analyses in this study were exploratory, and we acknowledge that the low subject number limits the generalizability of our results. Nevertheless, the significant differences we observed due to our experimental factors were quite reasonable given the nature of our experimental stimuli, and corroborative of related observations in neuroscience. Finally, the HFVK stimuli used in the present study were designed artistically, which precludes them from being duplicated. However, here we remind the reader that there were in fact seven different HFVK stimulus designs tested on each subject, each one embodying the principal of temporally precise alignment with musical auditory events occurring in their corresponding section of the opera. The brain activity measures tested in our statistical analyses were the mean activities across all seven of these HFVK designs. Thus, we contend that our results do not represent brain response corresponding to a single HFVK stimulus design, but rather to the principal of temporally precise musical event alignment which underlies all seven of the designs. This design principal can be duplicated by other research groups. Therefore, the reproducibility of our results should be testable regardless of the specific HFVK stimuli used, or even the specific music targeted.

Conclusion
To our knowledge, the present study is the first to observe that HFVK stimulation has a real and sustained effect on brain activity during a cinematic experience. Collectively, our results point to sustained decoherence between sensory and non-task-related self-processing as a possible mechanism for how VK increases immersion, and that coordination of emotional, spatial, and vestibular processing hubs with the motor system may be required for appreciation of the VK-enhanced AV experience. We hope this study will drive further neurophysiological inquiry seeking to not only corroborate these results, but also test whether similar neurological effects can be observed during AV experiences with HFVK designs targeting non-musicalstimuli.