Modulation of extrinsic and intrinsic processing by naturalistic audiovisual stimulation

Brain activity reflects both the processing of external stimuli (extrinsic processing) and the processing of signals generated internally (intrinsic processing). Intrinsic processing is associated with default mode areas whereas extrinsic processing is associated with sensory areas. However, different stimuli involve different processes such that the level and extent of extrinsic processing varies over time. The relationship between these fluctuations in extrinsic processing and fluctuations in intrinsic processing is largely unknown. Here we present preliminary results (N=2) investigating how extrinsic (stimulus-responses) and intrinsic (default mode network connectivity) processing are modulated by naturalistic audiovisual stimulation. We focus our analysis on brain areas (fMRI voxels) showing a mixture of extrinsic and intrinsic processing, which we identify throughout the cortex. Clustering analysis suggests that in posterior brain areas extrinsic and intrinsic processing compete, whereas in parietal and frontal areas they may cooperate.


Introduction
Neural activity reflects both responses to external stimuli (extrinsic processing) and internally generated activity that is dissociated from the current stimuli (intrinsic processing) (Ringach, 2009;Sadaghiani, 2010). At the macro-scale measured by functional Magnetic Resonance Imaging (fMRI), studies using inter-subject correlations have distinguished between areas that process external stimuli, for example posterior sensory areas, and areas showing stimulusdissociated activity, for example the default mode network (Golland et al., 2007;Golland, Golland, Bentin, & Malach, 2008). More recently, an hierarchical organization from predominantly extrinsic processing in sensory areas to predominantly stimulus-dissociated, intrinsic activity in heteromodal and default mode areas has been reported, demonstrating that many brain areas show a mixture of intrinsic and extrinsic processing (Ren, Nguyen, Guo, & Guo, 2017).
Since different stimuli are processed differently, the nature and extent of extrinsic processing varies over time. How do such fluctuations in extrinsic processing relate to intrinsic processing?

Overview
Our preliminary results are based on data from fMRI scans of two (N=2) healthy human subjects, approved by the ethics and safety committees of the National Institute of Information and Technology. We used multiband gradient echo-EPI sequence and T1weighted MPRAGE sequence for the functional and anatomical scans.
Our analysis focuses on data from passive viewing of four, 150s long, naturalistic audiovisual stimuli that were presented six times each (24 presentations in total). Briefly, these four stimuli depicted (1) scenes from young couples' lives, (2) a dance video, (3) a view from a ship sailing during sunset and (4) drone footage of a train. We refer to these as Couples, Dance, Ship and Drone respectively.
For preprocessing we followed (Nishida & Nishimoto, 2018) with the additional removal of the white matter signal (Friston et al., 1994). We used FreeSurfer to reconstruct the cortical surface and align the functional and anatomical scans  and used Pycortex for visualization (Gao, Huth, Lescroart, Gallant, & Cannon, 2015).

895
This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0
We averaged the correlations across the repeated presentation of each stimulus to obtain an overall measure of the level of intrinsic processing for each voxel and stimulus. We identified significant correlations using randomization tests combined with FDR correction for multiple comparisons across voxels (Benjamini & Hochberg, 1995).

Extrinsic Voxels
Extrinsic processing is reflected in consistent activation by external stimuli. To quantify this, we computed the correlation between the response for each stimulus presentation and the mean response, which we computed using all other presentations of that stimulus. We averaged these correlations for each stimulus to obtain an overall measure of the level of extrinsic processing for each voxel and stimulus. We again used randomization tests and FDR correction to identify significant voxels.

Clustering Analysis
The aim of this analysis was to group together voxels with similar stimuli-modulations of extrinsic and intrinsic processing, thus allowing us to investigate prototypical relationships between extrinsic and intrinsic processing and their distribution across the cortex. To do this we performed K-means clustering (Pedregosa et al., 2011) using all voxels showing both significant extrinsic and intrinsic processing. We treated voxels as samples and the level of extrinsic and intrinsic processing, computed for each stimulus separately, as features. The number of features was thus 2(extrinsic/intrinsic)x4(4 stimuli)=8. Using the "Elbow" criterion (Purnima & Arvind, 2014), we identified the number of clusters for S01 as four and for S01 as three. Here, for simpler comparison between subjects we set the number of clusters to three for both.

Results
We investigated extrinsic and intrinsic processing and how these are modulated by different audiovisual stimuli. For each voxel, we quantified extrinsic processing as the consistency of the response to naturalistic audiovisual stimulation. We quantified the level of intrinsic processing by connectivity to the default mode network. in extrinsic processing. Note that our criterion for extrinsic processing is strictly positive whereas intrinsic processing can be negative. C) Voxels identified as both extrinsic and intrinsic, shown in orange.
Intrinsic processing showed typical default mode network activation, involving the posterior cingulate cortex/precuneus, inferior parietal and prefrontal areas.
However, we also observed sparser yet widespread correlations across more posterior areas ( Figure 1A).
Extrinsic processing was most pronounced in posterior occipital and temporal areas and generally decayed anteriorly. However, some voxels around the precuneus and frontal structures also reflected extrinsic processing ( Figure 1B).
For both subjects, we observed the greatest number of voxels showing both significant extrinsic and intrinsic processing for the Couples stimulus (6046 voxels for S01 and 5102 voxels for S02). The majority of these voxels were located in posterior areas with sparser distribution more anteriorly ( Figure 1C). These results indicate that the neural substrate of extrinsic and intrinsic processing is partially shared, consistent with recent findings (Ren et al., 2017).
Next, we investigated how the different stimuli modulated the level of extrinsic and intrinsic processing. We used K-means clustering to do this in a data driven fashion. We treated voxels as samples and the level of extrinsic and intrinsic processing for each stimulus as features. The aim of this analysis was to group together voxels with similar modulations of extrinsic and intrinsic processing, thereby revealing prototypical relationships between the two.
We identified three clusters for both subjects ( Figure  2A and C). One cluster indicated that an increase in intrinsic processing goes with a decrease in extrinsic processing. We termed this the Competitive cluster (left most panel of Figure 2A and C). In general, this cluster involved more posterior voxels (though we also observed the involvement of some anterior voxels, particularly in the left hemisphere of S02, Figure 2B and D). Another cluster suggested that a decrease in extrinsic processing goes with a decrease in intrinsic processing. We termed this the Cooperative cluster (center panel of 2A and C). This cluster involved parietal areas but otherwise showed limited consistency between the two subjects ( Figure 2B and D). The last cluster reflected a more complex modulation, which we termed the Mixed cluster (right most panel of Figure 2A and C). The distribution of this cluster was not easily comparable across the two subjects. For S02 this cluster clearly involved the precuneus and Angular gyrus ( Figure 2D), whereas S01 showed sparse distribution across posterior and parietal areas ( Figure  2B). Cooperative (blue) cluster shows that a decrease in intrinsic processing goes with a decrease in extrinsic processing. The Mixed cluster (black) shows a more complicated modulation by the stimuli. B) Cluster assignments across the cortex. The anatomical labels have been removed to improve legibility. C and D same as A and B for S02.

Discussion
Here we report our preliminary analysis results investigating the interplay between intrinsic and extrinsic processing during naturalistic audiovisual stimulation. Our analysis suggests that in some brain areas extrinsic and intrinsic processing compete whereas in other areas they cooperate. Our ongoing work examines the relevance of these findings to internal thought processes.