Trial-by-trial voxelwise noise correlations improve population coding of orientation in human V1

Prior empirical and theoretical studies in neurophysiology have suggested that noise correlations between neurons could have a great impact on the fidelity of population codes in macaque visual cortex. However, it remains unclear whether such insights generalize to the large-scale brain activity in human sensory cortex. Here, we use functional magnetic resonance imaging (fMRI) to examine the effect of noise correlations on population coding of orientation in human V1. Trialwise responses of each V1 voxel is estimated for four orientations. We estimate the Fisher information carried by voxel responses for orientation in the empirically observed data (i.e., with noise correlations) and in a simulated regime in which voxelwise noise correlations are absent. Results show that the removal of noise correlations dramatically reduces information by one order of magnitude. This suggests that correlated activity could mediate the accuracy of population codes in the human brain, and that voxelwise noise correlations in human V1 are mostly beneficial, unlike the neuronal noise correlations that are often found to be detrimental.


Noise correlation and population codes
Neuronal activity varies trial-by-trial even towards a repeatedly presented stimulus. In vivo, neurophysiologists have discovered that the magnitude of noise correlations between two neurons tends to be positively correlated with the similarity of their tuning functions. Early work suggests that these noise correlations limit the information that a neuronal population can encode (Zohary et al., 1994). This finding leads to the canonical view that noise correlations are detrimental to population codes. Some recent studies suggest that whether noise correlations are detrimental or beneficial depends on many factors, such as tuning homogeneity (Ecker et al., 2011), task contexts (Bondy et al., 2018), and behavioral relevance (Haefner et al., 2013). Moreover, researchers have found that some cognitive processes (e.g., attention) improve neural processing by primarily altering noise correlation structures (Cohen & Maunsell, 2009). These results substantiate the critical role of noise correlations in mediating the accuracy of population codes.
In the field of fMRI, the accuracy of population codes is typically assessed by multivariate pattern analysis (MVPA). For example, decoding accuracy, such as percent correct in a classification task, is typically interpreted as conveying the amount of information encoded in a voxel population. However, percent correct is a summary statistics that does not provide detailed information on the different properties that underlie decoding performance, such as signal and noise at the single voxel level, and the interactions between voxels (i.e., covariance).
The magnitude and structure of voxelwise noise correlations in the human brain remains underexplored. Some recent studies have demonstrated that the noise correlation between two voxels is also correlated with their tuning similarity (Ryu & Lee, 2018;van Bergen & Jehee, 2018), implying that tuning compatible noise correlations exist in macroscopic brain responses. As such, noise correlations might considerably impact the 721 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0 amount of information carried in human sensory cortex. This hypothesis, however, has yet been thoroughly tested in the human brain.
In this paper, we ask a simple question-whether noise correlations are beneficial or detrimental for population codes in human sensory cortex. In a simple orientation experiment, we compare the accuracy of population codes in human V1 between two scenarios: one in which noise correlations are intact as observed and one in which noise correlations are hypothetically removed. Instead of the conventional MVPA approach, we use an information-theoretic approach that is based on a generative model of voxel responses to directly calculate amount of orientation information. This approach allows us to hypothetically remove noise correlations without changing other aspects of multivariate response distributions (i.e., marginal distributions).

Experiment and data acquisition Stimuli and experiment
We analyze the fMRI data collected by Sengupta et al. (2017). Six subjects participated in the study. Briefly, two flickering sine-wave gratings (Fig.1, 0.8-7.6º eccentricity, 160º angular width on each visual hemifield with a 20º gap on the vertical meridian) were presented on both sides of the fixation point. The orientations of the two gratings varied across trials. All orientations were drawn from 0º, 45º, 90º, and 135º with equal probabilities. A stream of letters was presented at the fixation point throughout each scanning run. Subjects were instructed to perform a reading task to maintain their fixation. At the end of a run, subjects were tested on a question related to the reading text.
On each trial, an orientation stimulus was shown for 3 s and followed by a 5 s blank. Each scanning run contained 30 trials. The 30 trials also included 10 randomized blank trials. The 1st trial could not be a blank trial, and there were no two consecutive blank trials. Blank trials could appear on either side while the orientation stimulus on the other side was intact. Each subject underwent 10 scanning runs.

MRI acquisition
A T2*-weighed echo planar imaging (TR/TE=2000/22ms) sequence was used to acquire fMRI data. In the original experiment, the subjects were scanned at four different resolutions. Here we analyze only the 2-mm isotropic data because decoding results have been shown best at this approximate resolution (Sengupta et al. (2017). 121 functional volumes were acquired in each run (FoV=200mm, matrix size 100 x 100, 37 slices, GRAPPA accel. factor 3). The functional volumes covered the occipital and parietal lobes. A high-resolution T1-weighted image was also acquired for each subject (0.67mm isotropic).
Additionally, four standard retinotopic scans were also conducted and used to define low-level visual areas. In some baseline trials (second trials shown above), only one grating is presented. Subjects are asked to attend and understand the letter stream presented at the fixation point. The pial and white surfaces were reconstructed based on the high-resolution T1-weighted images using the standard FreeSurfer pipeline. All functional imaging data underwent slice-timing correction, motion correction, and registration to T1 images. Retinotopic data were analyzed using the 3dRetinophase tool in AFNI. Polar angle and eccentricity maps were projected onto surfaces to aid delineation of early visual areas.

ROI definition and voxel selection
Bilateral V1 was defined on spherical cortical surfaces based on polar angle and eccentricity maps. V1 vertices on surfaces were then transformed back to EPI space to locate corresponding voxels using the AFNI 3dSurf2Vol utility.
Two general linear models (GLM) were implemented separately for the left and right hemifields. We modeled the effect of each hemifield in the contralateral V1. We coded each trial with a separate predictor in order to estimate trial-by-trial responses (i.e., beta weights) for each voxel. The demeaned head motion parameters, and constant, linear, and quadratic polynomial terms were also included as nuisance predictors. Time series of 10 scanning runs were concatenated and fitted using a single GLM. We selected the 100 voxels with the highest r-square values in the GLM in each hemisphere for further analysis. Therefore, we obtained a 100 (number of voxels) x trials data matrix for each orientation in each hemisphere.

Calculation of Fisher information in voxel populations
For each orientation in each hemisphere, we calculated the covariance matrix Q based on the voxel by trial data matrix.
For a pair of orientations, e.g., 0º and 45º, we define the Fisher information to discriminate them as: , (1) where f is the mean population responses across trials, and Q is the covariance between voxels at each orientation. This format of Fisher information has been shown appropriate if the response distribution conditioned on a stimulus follows the exponential family and linear sufficient statistics (Averbeck & Lee, 2003). The Fisher information of each orientation is calculated as the average across the Fisher information between that orientation and the other three orientations. For example, the Fisher information of 0º (I0 ) is: , ( Fisher information is typically used to assess the encoding of continuous variables. Here we generalize it to the problem of classifying discrete variables (Moreno-Bote et al., 2014). This is also dubbed as "linear discriminability" in some literature (Lin et al., 2015). This metric allows us to further calculate the Fisher information in the population responses without noise correlations. To do so, we artificially created the covariance matrix Q' that shares the same diagonal items with Q but all off-diagonal items (i.e., voxelwise noise covariance) in Q' are set to 0. We then used the Q' to calculate the Fisher information using Eq.1. By this method, all noise correlations between voxels can be removed without altering the signal and noise levels of individual voxels at all. This is similar to the method in which the voxel responses are shuffled across trials Left hemisphere (n=6) with NC no NC such that the noise correlations between voxels are disrupted.

Results
We find that the amount of information is greater to one order of magnitude when noise correlations are present (i.e., realistic voxel responses) compared with the case that noise correlations are hypothetically removed (Fig.  2). This result is robust across all four orientations and two hemispheres. This presents specific evidence that noise correlation can be beneficial to the population coding of orientation in human sensory cortex.

Discussion
Here we show that trial-by-trial voxelwise noise correlations are beneficial for population coding of orientation in human V1. This challenges the longstanding view in neurophysiology that noise correlations are detrimental. This result invites a rethinking of the role of noise correlations in sensory processing. It also suggests that the computational principles underlying microscopic neuronal activity and macroscopic voxel activity might be distinct. However, the source of trial-by-trial voxel response variability still remains unclear. It partially comes from neuronal noise but should also include other non-neuron noise, such as fMRI measurement noise. Understanding the nature of voxel response noise and noise correlations could be an important future direction.