Prediction error and repetition suppression have distinct effects on neural representations of visual information

Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.

where an unexpected stimulus evokes significantly greater negativity than an 90 expected stimulus. To date, however, no study has tested a key premise of 91 predictive coding, namely, that expected stimuli are more efficiently encoded in the 92 brain relative to unexpected stimuli, in terms of their elementary feature 93 representations. Nor has any previous investigation examined whether the 94 mismatch negativity response is associated with a change in neural tuning to 95 stimulus features such as orientation. 96 To test the hypothesis that prediction error can account for repetition 97 suppression effects, Summerfield and colleagues (2008) introduced an experimental 98 paradigm in which the identity of a face stimulus was either repeated in 80% of 99 trials (making the repetition expected) or was changed in 80% of trials (making the 100 repetition unexpected). There was greater attenuation of the BOLD response in the 101 fusiform face area when a face repetition was expected, relative to when it was 102 unexpected, suggesting that repetition suppression is reduced by unexpected 103 stimuli. This attenuation of repetition suppression by failures of expectation has also 104 been replicated using fMRI (Larsson & Smith, 2012)  Here we used multivariate forward encoding methods to test whether 130 repetition suppression and expectation have different effects on the way the brain represents visual information, in this case the orientation of grating stimuli. To 132 anticipate the results, we found that soon after stimulus onset, repetition 133 suppression had no effect on visual orientation selectivity, but violated expectations 134 were associated with a significantly increased orientation-selective response 135 through gain modulation, with no corresponding change in response fidelity. This 136 representation was transiently re-activated at around 200 ms post-stimulus onset, 137 suggesting that feedback influences initial sensory encoding of an unexpected 138 stimulus, which in turn allows for updating of the sensory prior. 139

Results 140
We used a modified version of the paradigm introduced by Summerfield and 141 colleagues (2008), replacing the face stimuli used in that study with oriented 142 Gabors. These low-level stimuli allowed us to quantify the degree of orientation 143 selectivity in EEG activity to determine how the representation of orientation is 144 affected by prediction error and repetition suppression. Each of fifteen observers 145 participated in two EEG sessions. On each trial, two Gabors were presented 146 sequentially (100 ms presentation, 600 ms stimulus onset asynchrony), and these 147 stimulus pairs either repeated or alternated in their orientation ( Figure 1A, Movie 1). 148 The predictability of the repeated and alternating pairs was varied in a block-wise 149 manner to manipulate expectation. In a repeating block, the orientations of the two 150 Gabors in a pair repeated in 80% of trials, and alternated in the remaining 20%. 151 These contingencies were reversed in the alternating block ( Figure 1B). The 152 orientations of successive stimuli across a block were randomized to limit any 153 accumulated effects of adaptation and prediction. As repetition suppression and expectation form orthogonal dimensions of the task, the design allowed us to 155 isolate their respective contributions to neural responses. Participants completed an 156 unrelated task of discriminating (red vs blue) rare coloured Gabors (which occurred 157 on 10% of trials).   Tootell et al., 1998), there was 178 a significant repetition suppression effect (Repeat < Alternating), such that the 179 response to repeated stimuli was significantly reduced compared with the response 180 to alternating stimuli ( Figure 2A). The repetition suppression effect was evident over 181 a large cluster of occipital-parietal electrodes at two time intervals: an early effect 182 from 79 to 230 ms, and a later effect at 250 to 540 ms after the onset of the second 183 stimulus (cluster p < .025; Figure 2B and  greater negativity for unexpected versus expected stimuli, and this effect was most 209 prominent over a cluster of occipital-parietal electrodes around 75-150 ms after 210 stimulus presentation ( Figure 2C). As with the repetition suppression result 211 described above, there was an expectation effect of opposite polarity over 212 occipital-parietal electrodes. This effect was significant at an early time point post-213 stimulus (79-130 ms), but not at later time points (320-390 ms; Figure 2D). Finally, 214 there was no interaction between repetition suppression and expectation (i.e., no 215 significant positive or negative clusters, all p > .05). Taken together, these results 216 reveal both repetition suppression and expectation effects in the neural data, which 217 were indexed separately as shown in Figure 2. orientations ( Figure 3D). These weights were then inverted to reconstruct the model, 252 and multiplied against an independent set of test trials to produce responses in the 253 modelled orientation channels. These sets of responses were then used to evaluate 254 the degree of orientation selectivity in those trials. The procedure was repeated for 255 all time points in the trial, and a cross-validated approach was used until all trials 256 had been used for both training and testing. 257 As shown in Figure 3, the forward encoding revealed a strong, orientation-258 selective response derived from the multivariate pattern of EEG activity. This 259 orientation-tuned response was evident from ~50 to ~470 ms after stimulus onset, 260 and peaked between ~120-250 ms ( Figure 3C). Examination of the regression 261 weights revealed that this response was largely driven by activity centred over 262 occipital-parietal areas ( Figure 3D). 263 To examine our central question of whether repetition suppression and 286 expectation have differential effects on neural representations of orientation, we 287 split and averaged the results of the forward encoding by trial type, and fitted these 288 with Gaussians (see Methods) to quantify orientation selectivity (Figure 4). 289 Repetition suppression did not affect the amount of orientation selectivity contained 290 within the EEG data, with similar selectivity for repeated and alternating trials. This 291 was the case even though the repeated trials had a markedly smaller EEG response 292 over occipital and parietal electrodes (see Figure 2A Examining the effect of expectation revealed a markedly different pattern of 315 results. As shown in Figure 4A, at 79 -185 ms after the onset of the second 316 stimulus in the pair, orientation-selectivity increased significantly (p < .0001) when 317 the stimulus was unexpected relative to when it was expected, and this effect arose 318 at the earliest stages of the brain's response to that stimulus. Moreover, the expectation signal contained enhanced information about the specific features of 320 the stimulus that violated the expectation, in this case the orientation of the second 321 grating. We conducted the same statistical tests on the three other parameters 322 defining the Gaussian function (namely, the width, centre orientation and baseline) 323 to determine how repetition suppression and expectation might affect other 324 properties of the neural representation. There was no reliable influence of repetition 325 suppression on any of these Gaussian parameters (all p > .32). For expectation, 326 there was a significant decrease in baseline activity over the same time period as 327 observed for the increase in amplitude (79-185 ms, p = .001), but there were no 328 significant effects for the other parameters (all ps > .30). 329 We followed up this initial analysis to ensure we did not miss any small 330 effects of repetition suppression or expectation on any aspects of stimulus 331 representation. We increased the signal-to-noise by averaging the stimulus 332 reconstruction over this early time period (79-185 ms after stimulus presentation), 333 and fitted Gaussians to each participant's data individually ( Figure 4B). This again 334 showed that the amplitude of the response was significantly (t(14) = 3.34, p = . representation was not affected by expectation (p = .44), and there was no effect of 358 repetition suppression on orientation selectivity (p = .64). We can thus be confident 359 that the effect of expectation on orientation selectivity that we report here, based on 360 our forward encoding analyses, is not an artefact of the baselining procedure. 361 We also used a number of approaches to determine whether repetition 362 suppression and expectation interacted to affect orientation selectivity. First, we 363 took the difference scores between the combination of factors (e.g., expected 364 repetition minus unexpected repetition, and expected alternation minus unexpected alternation) and compared these using the same cluster-based permutation testing 366 outlined above. This analysis revealed no significant interactions between the 367 factors for any parameter (all ps > .10). Second, we found the largest orientation- As shown in Figure 6, optimal orientation selectivity was on-axis (training 410 time equals test time) between 100 ms and 300 ms after stimulus presentation, 411 suggesting that the stimulus representation changed dynamically over time (King & 412 Dehaene, 2014). There was also significant off-axis orientation-selectivity from 100-413 500 ms after stimulus presentation, suggesting that some aspects of the neural 414 representation of orientation were stable over time. to show clusters (black outlines) of significant orientation selectivity (permutation 420 testing, cluster threshold p < .05, corrected cluster statistic p < .05, 5,000 421 permutations). The difference between the conditions is shown in the right-hand 422 column (permutation testing, cluster threshold p < .05, corrected cluster statistic p < 423 .05). Opacity and outlines indicate significant differences. 424 425 426 There was no effect of repetition suppression on temporal generalization of 427 orientation information (upper panels of Figure 6), suggesting that repetition 428 suppression did not affect the temporal stability of neural representations of 429 orientation. Examining the effect of expectation on cross-temporal generalization 430 confirmed that there was significantly more on-axis orientation selectivity when the 431 stimulus was unexpected than when it was expected (cluster p = .02). This 432 increased on-axis orientation selectivity generalized off-axis at around 300-400 ms 433 after stimulus onset (cluster p = .01), suggesting that the same representation that is 434 activated to process the expectation is reactivated later as the stimulus continues to 435 be processed. Such a signal could constitute the prior of the prediction, as this 436 should be updated on the basis of incoming sensory evidence, which in turn would 437 likely require reactivation of the unexpected stimulus. orientation -our results imply the operation of at least two distinct neural processes 467 at separate times following stimulus onset. Incoming sensory information is first 468 evaluated against the prior (which occurs early after stimulus presentation). When 469 an unexpected stimulus is detected and generates a prediction error, the 470 representation is amplified through gain enhancement. Later, around 300 ms after stimulus presentation, this same representation is reactivated to update the 472 expectation against the initially predicted representation. 473 According to predictive coding theory, expected stimuli should be more 474 efficiently represented than unpredicted stimuli largely because the reduced neural 475 response still encodes stimuli with the same fidelity (Friston, 2005). A more efficient 476 response could be due to sharpening of neuronal tuning to stimulus features, or to a 477 reduction in the gain of evoked neural responses. Our results strongly support the 478 latter interpretation. Specifically, there was no evidence that a fulfilled expectation 479 leads to a sharper representation of orientation information. Our findings might 480 imply that the brain needs to have more information about an unexpected stimulus, 481 so a correct response can be made. Our findings thus provide a novel insight into 482 how predictive coding might change neural representations of sensory information. Trujillo, 1999). The differences between these results may potentially have arisen 508 because the tasks relied upon different types of attention (e.g., spatial versus 509 feature-based). Future studies could determine whether this same divergence 510 occurs for prediction effects. 511 The current work applied multivariate model-based approaches to EEG data 512 to determine how prediction and repetition suppression affect neural 513 representations of perceptual information. We chose to use EEG so we could 514 recover the temporal dynamics of these effects -something that would not be 515 possible with the BOLD signal used in fMRI -and because EEG is the most widely-516 used tool for measuring expectation effects in human participants (see Garrido likely also associated with a significant predictio8n that the next stimulus will be the same as the previous one. Perhaps more relevant to the current results, Patterson 587 et al. (2013) found that the width of orientation tuning in V1 is only marginally 588 sharpened following brief (400 ms) periods of adaptation. Again, however, their 589 study did not control for expectation, so it is impossible to determine the role of 590 predictive coding in their observations. Our finding that repetition suppression did 591 not affect the bandwidth of orientation selectivity measured using EEG is also 592 consistent with models of orientation adaptation based on human psychophysical 593 data, which suggest that adaptation does not affect the tuning width of the adapted 594 neural populations (Clifford, 2002 In summary, we have shown that repetition suppression and expectation 598 differentially affect the neural representation of simple, but fundamental, sensory 599 features. Our results further highlight how the context in which a stimulus occurs, 600 not just its features, affect the way it is represented by the brain. Our findings 601 suggest encoding priority through increased gain might be given to unexpected 602 events, which in turn could potentially speed behavioural responses. This prioritized 603 representation is then re-activated at a later time period, supporting the idea that 604 feedback from higher cortical areas reactivates an initial sensory representation in 605 early cortical areas. 606

Method 608
Participants 609 A group of 15 healthy adult volunteers (9 females, median age = 20.5 yr, 610 range = 18 to 37 yr) participated in exchange for partial course credit or financial 611 reimbursement (AUD$20/hr). We based our sample size on work that investigated 612 the interaction between repetition suppression and prediction error (Summerfield et 613 al., 2008), and that used forward encoding modelling to investigate orientation 614 selectivity using MEG with a comparable number of trials as the current study 615 (Myers et al., 2015). Each person provided written informed consent prior to 616 participation, and had normal or corrected-to-normal vision. The study was 617 approved by The University of Queensland Human Research Ethics Committee and 618 was in accordance with the Declaration of Helsinki. 619

Experimental setup 620
The experiment was conducted inside a dimly illuminated room with the 621 participants seated in a comfortable chair. The stimuli were displayed on a 22-inch 622 LED monitor (resolution 1920 x 1080 pixels, refresh rate 120 Hz) using the 623 PsychToolbox presentation software (Brainard, 1997;Pelli, 1997) for MATLAB 624 (v7.3). Viewing distance was maintained at 45 cm using a chinrest, meaning the 625 screen subtended 61.18º x 36.87º (each pixel 2.4' x 2.4'). 626

Visual task 627
The stimuli were Gabors (diameter: 5º, spatial frequency: 2 c/º, 100% 628 contrast) presented centrally in pairs for 100 ms, separated by 500 ms (600 ms 629 stimulus onset asynchrony) with a variable (650 to 750 ms) inter-stimulus interval between trials. Across the trials, the orientations of the Gabors were evenly spaced 631 between 0º and 160º (in 20º steps) so we could reconstruct orientation selectivity 632 contained within the EEG response using forward encoding modelling. The 633 relationship of the orientations of the pairs Gabors was also used to construct the 634 different repetition suppression and prediction conditions. The orientation presented 635 in the second Gabor in the pair could either repeat or alternate with respect to the 636 orientation of the first Gabor. In the alternation trials, the orientation of the first 637 Gabor was drawn randomly, without replacement, from an even distribution of 638 orientations that was different to the orientation of the second Gabor. To vary the 639 degree of prediction, in half of the blocks 80% of the trials had repeated 640 orientations and 20% of the trials had alternating orientations, whereas in the other 641 half of the blocks these contingencies were reversed. This design allowed us to 642 separately examine the effects of repetition suppression and prediction because of 643 the orthogonal nature of the blocked design. The blocks of 135 trials (~3 mins) 644 switched between the expectation of a repeating or alternating pattern, with the 645 starting condition counterbalanced across participants. 646 The participants' task was to monitor the visual streams for rare, faintly 647 coloured (red or green) Gabors and to discriminate the colour as quickly and 648 accurately as possible. Any trial with a coloured target was excluded from analysis. 649 The orientation match between the pairs was made to be consistent with the 650 dominant contingency (i.e., repeated or alternating) within that block. Pilot testing 651 was used prior to the main experiment to set the task at approximately threshold, to 652 ensure that participants focused exclusively on the colour-discrimination task rather than the orientation contingencies associated with prediction and repetition. Only 654 one participant reported being aware of the changing stimulus contingencies across 655 the blocks when asked at the end of the experiment, and excluding this 656 participant's data had no effect on the key results reported here. Self-paced breaks 657 were provided between each of the 20 blocks within a session, at which time 658 feedback was provided on performance in the preceding block. Subspace Reconstruction), which were then interpolated from the neighbouring 677 electrodes. Data were then re-referenced to the common average before being 678 epoched into segments around each stimulus pair (-0.5 s to 1.25 s from the first 679 stimulus in the pair). Systematic artefacts from eye blinks, movements and muscle 680 activity were identified using semi-automated procedures in the SASICA toolbox 681 (Chaumon, Bishop, & Busch, 2015) and regressed out of the signal. After this stage, 682 any trial with a peak voltage exceeding ±100 uV was excluded from the analysis. 683 The data were then baseline corrected to the mean EEG activity from -100 to 0 ms 684 before the presentation of the second Gabor in the pair. Critically, the orientations 685 of the first and second gratings were precisely balanced across the conditions to 686 avoid any systematic bias in orientation information being carried forward by the 687 first grating within each pair. Specifically, for every unexpected stimulus presented 688 in the second grating there was an equal number of every other orientation that was 689 expected to be presented. As the analysis we employed used a regression-based 690 approach, any carry over of orientation-selective information from the first to the 691 second grating therefore could not systematically bias the results. 692

Experimental Design 693
We used a modified version of a factorial design that has previously been 694 We applied forward encoding modelling to determine how repetition 739 suppression and prediction error affected orientation selectivity. To do this, the 740 second orientation ( Figure 7A) in the Gabor pair in each trial was used to construct a 741 regression matrix, with separate regressors for the 9 orientations used across the 742 experiment. This regression matrix was convolved with a set of basis functions (half 743 cosines raised to the 8 th power ( Figure 7C), which allowed complete and unbiased 744 coverage of orientation space) to allow us to pool similar information patterns across nearby orientations (Brouwer & Heeger, 2009). We used this tuned 746 regression matrix to estimate time-resolved orientation selectivity contained within 747 the EEG activity in a 16 ms sliding window, in 4 ms steps ( Figure 8B; Myers et al., 748 2015). To avoid overfitting, we used a leave-one-out cross-validation procedure 749 where the regression weights were estimated for a training set and applied to an 750 independent test set ( Figure 8D). All trial types (including target trials) were used in 751 training and test sets. This was done by solving the linear equation: 752 B1= WC1 (1) 753 Where B1 (64 sensors x N training trials) is the electrode data for the training set, C1 754 (9 channels x N training trials) is the tuned channel response across the training 755 trials, and W is the weight matrix for the sensors we want to estimate (64 sensors x 756 9 channels). W can be estimated using least square regression to solve equation (2): 757 (2) 758 The channel response in the test set C2 (9 channels x N test trials) was estimated 759 using the weights in (2) and applied to activity in B2 (64 sensors x N test trials). 760 We repeated this process by holding one trial out as test, and training on the 762 remaining trials until all trials had been used in test and training. The procedure was 763 repeated for each trial within the trial epoch. We then shifted all trials to a common 764 orientation, meaning that 0º corresponded to the orientation presented on each trial. 765 The reconstructed channel activations were separated into the four conditions, and 766 averaged over trials. These responses were then smoothed with a Gaussian kernel with a 16 ms window, and fitted with a Gaussian function (4) using non-linear least 768 square regression to quantify the amount of orientation selective activity. 769 Where A is the amplitude representing the amount of orientation selective activity, 771 is the orientation the function is centred on (in degrees), is the width (degrees) and 772 C is a constant used to account for non-orientation selective baseline shifts. orientations (coloured dots and lines, which match the colours of the outlined 780 gratings in A). These coefficients were then used to generate a regression matrix. 781 (D) General linear modelling was used on a subset of training trials to generate 782 weights for each channel. These weights were inverted and simultaneously applied 783 to an independent test set of data to recover orientation selectivity in the EEG 784 activity. As EEG activity has high temporal resolution, we can apply the procedure 785 to many epochs following stimulus presentation to determine the temporal 786 dynamics of orientation processing (see Figure 3). 787 788

Multivariate pattern analysis 789
We conducted a multivariate pattern analysis to build upon the initial forward 790 encoding results which showed that unexpected stimuli elicit greater orientation 791 selectivity than expected stimuli. This analysis used the same data as the forward encoding analysis. We used the classify function from Matlab 2017a with the 793 'diaglinear' option to implement a Naive Bayes classifier. For each time point, we 794 used the same cross-validation procedure as the forward encoding modelling with 795 the same averaging procedure to select train and test sets of data. The classifier 796 was given the orientations of the training data and predicted the orientation of the 797 test data. A trial was labelled correct if the presented orientation was produced. To 798 facilitate comparison of the results with those of (Kok et al., 2012), we found the 799 peak classification accuracy for each participant in the 600 ms following stimulus 800 presentation. The same wide time window was used across conditions to 801 accommodate large inter-individual differences in peak classification without 802 biasing the results toward one particular condition. 803

Statistical testing 804
A non-parametric sign permutation test was used to determine the null 805 distribution for testing (Wolff, Jochim, Akyürek, & Stokes, 2017b). This method 806 makes no assumptions about the underlying shape of the null distribution. This was 807 done by randomly flipping the sign of the data for the participants with equal 808 probability. Fifty thousand (50,000) permutations were used for the time-series data, 809 whereas only 5000 were used for the temporal generalization plots because of the 810 significantly greater computational demands. 811 Cluster-based non-parametric correction (50,000 permutations for timeseries 812 and 5,000 for temporal generalization) was used to account for multiple 813 comparisons, and determined whether there were statistical differences between 814 the contrasting conditions. Paired-samples t-tests were used to follow up the analysis in Figure 4 within a specified time window, and no correction was applied. 816 A two-way repeated measures ANOVA (implemented using GraphPad Prism 7.0c, 817 La Jolla California, USA) was used to analyse the multivariate pattern analysis 818 results shown in Figure 5. shown in Figure 4A. The curves, shown as fitted Gaussians, illustrate how overall 831 stimulus representations are affected by repetition and expectation. While there was 832 no difference in orientation tuning for repeated versus alternate stimuli (left panel), 833 the amplitude of the orientation response increased significantly, and the baseline 834 decreased, for unexpected relative to expected stimuli (right panel). Error bars 835 indicate ±1 standard error. 836 837