Face recognition ability is manifest in early dynamic decoding of face-orientation selectivity d Evidence from multi-variate pattern analysis of the neural response

Although humans are considered to be face experts

Although it is typically taken for granted that humans are all 'face-experts' who can accurately and effortlessly identify a known individual from a brief glance, in fact large and reliable differences exist in adult face identification ability (recognition memory as well as matching). A naturally occurring spectrum of ability ranges from those individuals who report severe problems in face recognition, termed developmental prosopagnosics (Duchaine & Nakayama, 2006) through to socalled super recognisers (Russell, Duchaine, & Nakayama, 2009), with the rest of the population falling between the extremes of this normally distributed trait (Wilmer, 2017). Despite keen research interest in the mechanisms underlying the neurotypical variability, much remains unclear in terms of understanding the functional mechanisms underlying strengths and weaknesses in face processing (Ramon, Bobak, & White, 2019), which could be present in any or indeed all its levels: from our earliest low-level visual responses, through to high-level mental representations.
Given the amount of neuroscientific research that has been conducted into face perception generally, it is perhaps surprising that relatively few studies have explored the neural markers associated with stronger vs weaker abilities in the typical population (see Lander, Bruce, & Bindemann, 2018 for a recent overview). We do, however, know a great deal about the distributed neural system specialised for processing different aspects of faces, including identity (see Gobbini & Haxby, 2007;Haxby, Hoffman, & Gobbini, 2000). This body of work includes insights from neuropsychological investigations of clinically significant identification difficulties (i.e., prosopagnosia) associated with disruption to this system (Barton, 2008;Bruno Rossion, 2008). Thus, it seems appropriate to also consider whether differences in neurological structure or processing might functionally contribute to the variability in face recognition abilities observed in the typical population.
Electrophysiological (EEG) studies are particularly interesting on this point, because of the degree to which they can provide detailed information about neural correlates of perception and cognition as they are occurring. Studying group and individual profiles of EEG activity allows us to pinpoint differences that arise at specific (i.e., informative) points in the processing time course. Such responses are typically explored via event related potential analysis (ERP) of the averaged neural response time locked to presentation of a stimulus/ category. This work has identified clear markers of face processing expertise later in the component time course, often linked to face familiarity and face recognition (e.g., N250, N250R, P300, P600, Belanova, Davis, & Thompson, 2018;Meyer et al., 2021;Parketny, Towler, & Eimer, 2015;Towler, Fisher, & Eimer, 2017). Yet there are only sparse reports of an association between expertise and the early neural responses to faces.
There is some debated evidence that the P100 component, an early positivivity observed in posterior electrode sites, might be sensitive to face information, showing an increased amplitude to faces compared to objects or scrambled faces (Eimer, 1998;Herrmann, Ehlis, Ellgring, & Fallgatter, 2005;Itier & Taylor, 2004b) and to inverted faces compared to upright (Colombatto & McCarthy, 2017;Itier & Taylor, 2004a;2004b;Minami, Nakajima, Changvisommid, & Nakauchi, 2015). Some links between face expertise and the P100 component have been suggested at the group level, with participants with higher face expertise showing an increased P100 for faces but not for cars (Turano, Marzi, & Viggiano, 2016). Furthermore, an association between the P100 amplitude and face memory ability has been found using structural equation modelling (SEM, Kaltwasser, Hildebrandt, Recio, Wilhelm, & Sommer, 2014).
While the association between the P100 and face processing remains debated, the N170, a negativity occurring approximately 170 ms post presentation of a face, is the component most robustly observed to respond selectively to faces (see Eimer, 2011 for a review). Generally accepted to be a marker of the detection and encoding of perceptual face information there have been a few limited reports of a link between the N170 and face expertise. Notably, face identity recognition accuracy has been moderately linked to N170 latency, via SEM Kaltwasser et al., 2014;Nowparast Rostami, Sommer, Zhou, Wilhelm, & Hildebrandt, 2017. Findings regarding the N170 amplitude have been more mixed, but there is some evidence linking it to both face and object memory ability (Nowparast Rostami et al., 2017) and to face memory only in men (Nowparast Rostami, Hildebrandt, & Sommer, 2020). The N170 amplitude was further linked to expertise at the group level (Turano et al., 2016). Further group level studies suggest indirect evidence of a link between the N170 component and expertise via effects such as the other race effect (Weise, Kaufmann, & Schweinberger, 2014) or the impact of early exposure to a larger vs smaller pool of face exemplars, a factor itself tied to a behavioural expertise measure (Balas & Saville, 2015).
The current study seeks to expand on these findings using a less restrictive analysis approach (multi-variate pattern analysis) to explore any association between individual differences in face recognition ability (characterized using behavioural CFMT scores) and participants' early neural (EEG) responses to faces. Importantly, in contrast to the traditional component based framework, multi-variate approaches do not pre-specify the timing and location of particular effects of interest, rather they combine information across electrode sites to detect differences in the pattern of neural response that may not be obvious when only single electrodes (or electrode pairs) are considered in isolation. Further, unlike previous group-level ERP studies, expertise is conceptualized as a continuous variable within our analysis. This approach avoids the need to arbitrarily split the sample based on mean/ median values to impose an artificial good vs. bad face processors dichotomy. Such an approach (i.e., targeting interindividual differences in the relationship between decoding metrics and face processing expertise) promises to extend findings from group-mean studies to more clearly elucidate neuronal as well as cognitive factors that modulate the highly variable outcomes observed in this domain (see Rhodes et al., 2014).
In the present task, participants view upright and inverted faces, alongside a further comparison category of complex visual stimuli: upright and inverted houses. Inclusion of these four stimulus categories in the task permits investigation of expertise-related differences in selective responding to faces as an overall category (contrasting upright faces and houses), and more selectively to the canonical upright configuration of a face (contrasting upright and inverted faces). Brain behaviour links may be revealed in either or both contexts, though the extent to which face orientation effects index specialist processing makes the latter a particularly strong candidate for revealing differences in early neural responses as a function of ability. Moreover, the extant preliminary evidence for links between ability and early face-sensitive ERP components in typical participants (Balas & Saville, 2015) and developmental prosopagnosics (Towler, Gosling, Duchaine, & Eimer, 2012) supports differential neural responses to face orientation, rather than categorization (cf. chairs and houses respectively).
Notably one of the few studies to apply machine learning classification techniques to tackle this question provided intriguing evidence of a difference in classification of face orientation in groups with/without extensive early experience with a large pool of faces (Balas & Grant, 2016). Here we will also employ machine learning techniques to further probe the link between face orientation effects and participants expertise with faces as directly indexed by a cognitive test of face memory ability, the CFMT.

Methods
This paper follows the same methods and general analysis pipeline as two recent studies from our team . No part of the study procedure or analysis was pre-registered prior to the research being conducted. The conditions of our ethics approval do not permit public archiving of anonymised study data. Readers seeking access to the data should contact the senior author MLS. Access will be granted to named individuals in accordance with ethical procedures governing the reuse of sensitive data. Specifically, requestors must complete a formal data sharing agreement and approval must be obtained from the local ethics committee.
Averaged datasets and the experimental task 2 can be found at https://researchdata.bbk.ac.uk/id/eprint/205/. The code underpinning our main analysis is publicly available at https://github.com/fws252/Mares_etal_Cortex_2022. W e report how we determined our sample size, all data exclusions, all inclusion/exclusion criteria, whether inclusion/ exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study.

Participants
A total of 43 participants volunteered to participate in the study (while no previous power analysis was conducted at the time, we elected to recruit a similar number of participants as previous studies, e.g., Balas & Saville, 2015;Balas & Grant, 2016). Participants were included if reporting corrected to normal vision and were aged between 18 and 40 years of age (criteria decided before data collection). One person could not be included as they did not agree to complete the CFMT. Further, to ensure sufficient data quality only those individuals with at least 30 non-artefact trials per experimental condition in the EEG task were included for analysis, leaving a final sample of 39 participants (23 females, M ¼ 26.

Stimuli
This study used greyscale photographs of six male identities presented frontally with neutral facial expressions (see Oliva & Schyns, 1999 for examples of the face stimuli). Faces were digitally manipulated to share the same hairstyle and outline. Similar photographs of houses (N ¼ 6) were used as a control condition. House stimuli were masked with an oval outline sharing the same outline as the one used for the faces (see Luminance and contrast were controlled across stimuli using the Shine toolbox (Willenbockel et al., 2010). Inverted versions of the upright images were created for all stimuli.

Procedure
Participants took part in a multi-experiment session lasting between 90 and 120 min, which included a series of face and object perception and memory tasks associated with a larger project. 3 Participants further completed the CFMTdAustralian 4 at the end of the session which provides an estimate of face identity expertise. In this recognition memory task, participants are introduced to a series of novel target identities shown across three different viewpoints, then asked to discriminate them from 2 other similar-looking foils. As the task progresses the difficulty increases and it culminates in a final stage in which visual noise is added to the test stimuli, making accurate discrimination particularly challenging (further methodological details are available in McKone et al., 2011). The main experimental EEG task required participants to closely observe a series of face and house images, presented in both upright and inverted orientations. To maintain interest and attention, participants were given the explicit task of "spotting butterflies" that occasionally appeared to the left or right of fixation on catch trials (20% of trials), using a keyboard press to indicating its position (left or right). Using Eprime software (Version 2.0), face and house stimuli were presented centrally on a grey background (750 ms) followed by a black fixation cross of randomly varying duration (1650e1850 ms). Participants completed 60 trials with each category of stimuli, for a total of 240 trials, that were fully randomised and randomly intermixed with a further 60 trials of butterflies.
Trials were divided into 10 blocks of 30 trials with short breaks permitted between each block. Responses on these butterfly catch trials were rapid (Mean RT ¼ 436 ms) and highly accurate (M ¼ 97%), which supports high levels of engagement with the explicit behavioural task.
Participants were seated in a comfortable chair in an electrically shielded and sound-proof room at a distance of approximately 70 cm from the screen (such that stimuli subtended a visual angle of approximately 4.09 width by 6.13 height; 176 Â 256 pixels).
Of note, prior to participation in the main EEG experiment, participants completed one face related activity which comprised a short familiarisation task in which they learned by name three of the six presented male identities. They then made a small number of identity categorization decisions on sub-sampled versions of these faces (216 trials, 72 trials per face, see Gosselin & Schyns, 2001 for the experimental paradigm details). 5 Participants were not informed about potential face identity familiarity during the EEG task and this was not highlighted to them, nor was it a focus for the present analysis. Our goal here was specifically to explore the existence of an early neural correlate of face processing expertise and as such our interest was centred around the timing of the most widely acknowledged first reliable and robust selective neural response to faces, the N170 component, which typically peaks 170 ms following presentation of a face stimulus over occipitotemporal brain regions (Rossion & Jacques, 2011). Reliable face familiarity effects are typically observed much later, around 250 ms following face presentation e.g., the N250R (e.g., Schweinberger, Pickering, Jentzsch, Burton, & Kaufmann, 2002), a reduced response to repetition of the same identity, and the N250 (Andrews, Burton, Schweinberger, & Wiese, 2017; Gosling & Eimer, 2011) observed when contrasting highly familiar/famous faces to unfamiliar faces, and are relatively small in comparison to the early critical period around the N170 (see Ramon & Gobbini, 2018 for a review). As such we do not consider the familiarity of the faces further as an independent variable of study in the present work, although see supplementary materials for confirmatory results showing the same pattern of findings as reported below, when results are split by the factor of face familiarity.

EEG recording and analysis
EEG recording was conducted using a fitted cap (EASYCAP) with either 32 (N ¼ 19) or 64 (N ¼ 20) AgeAgCl electrodes placed according to the international 10/10 system. The fitted cap included two electrodes placed laterally to the eyes in order to measure horizontal eye movements. Furthermore, an electrode was placed below one of the eyes to monitor vertical eye movements and blinks. Electrode impedance was lowered below 10 kU. EEG was acquired at a sampling rate of 500 Hz and referenced to FCz, with AFz as the ground. Data was analysed using the Matlab (R2017b) toolbox EEGLAB (Version 13, Delorme & Makeig, 2004). Continuous data was band pass filtered between .1 and 40 Hz, and epoched around stimulus onset from À200 ms to 500 ms. Epochs were baseline corrected 3 Participants also completed a second EEG task in which they viewed images of different individuals varying in gender (male, female) and facial expression of emotion, and took part in a behavioural economics style paradigm in which they controlled their viewing exposure to a range of further different facial identities (varying in facial expressions of emotion, attractiveness and gender). Crucially, these additional tasks were always completed after the present task and therefore could have no bearing on the reported findings.   1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63  64  65   66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95  96  97  98  99  100  101  102  103  104  105  106  107  108  109  110  111  112  113  114  115  116  117  118  119  120  121  122  123  124  125  126  127  128  129  130 CORTEX3597_proof ■ 26 December 2022 ■ 4/14 using the 200 ms previous to stimulus onset. Channels were selected for interpolation due to noise using EEGLAB automated criteria (M ¼ 1.9 for participants with caps with 32 channels and M ¼ 5.8 for participants with caps with 64 channels). Catch trials (i.e., those including the butterfly target images) were removed from all further analysis. The remaining epochs were then visually inspected to remove artefacts, namely eye blinks (large deflections observed across all electrodes, and in the vertical EOG in particular), saccades (identified using the two horizontal EOG electrodes), muscle artefacts (high-frequency activity), and large amplitude noise. A single experienced researcher, naïve to participants CFMT scores completed this pre-processing. Based on visual inspection, 16.93% of trials were rejected (SD ¼ 9.46%). Importantly there was no difference between higher and lower face ability participants (based on a median split of CFMT scores) in number of trials (t (36) ¼ .277, p ¼ .783).

MVPA analysis
Multi-variate pattern analysis (MVPA) was employed to identify potential neural correlates of face processing expertise in a less constrained manner than typical ERP component analysis, which focuses on only a small number of pre-specified electrodes, and tightly constrained time-windows (Grootswagers, Wardle, & Carlson, 2017). Instead MVPA approaches use machine learning algorithms to combine information across a set of electrodes to differentiate the neural response associated with distinct categories. We reasoned that such an approach could provide greater scope to detect expertise driven changes in the neural response to faces complimenting more traditional analysis.
To this end, we first establish, at the individual level, when the neural response to the canonical upright presentation of faces differs significantly to the response to viewing other complex visual objects (here houses), and to the response to the same faces presented in an inverted orientation. The former contrast (faces vs. houses) is commonly used as control category to analyse face selectivity (Eimer, 2000b;Itier, Alain, Sedore, & McIntosh, 2007;Rossion et al., 1999). While the latter contrast is considered a hallmark of face processing, both show some behavioural links with face expertise (Busigny & Rossion, 2010;Rezlescu, Susilo, Wilmer, & Caramazza, 2017;Russell et al., 2009) and it has a wellknown and robust early neural correlate (Itier et al., 2007;Rossion et al., 1999). In a second step, we then explore if this timeline of face sensitive decoding (faces vs. houses, upright vs. inverted faces) is modulated by an independently established measure of face processing expertise. More specifically, we set out to establish if the ability with which a model can predict what stimulus a participant was viewing (e.g., an upright versus an inverted face) differs significantly as a function of participants' face ability. If it does, then we can infer that the electrophysiological data that contains information pertinent to the distinct representation of these two categories (see Grootswagers et al., 2017) differs as a function of expertise.
Finally, in order to confirm any effects of orientation were specific to faces, we also explored the neural response to visual objects presented in two orientations (i.e., upright vs. inverted houses).
MVPA was conducted by training linear support vector machine (SVM) classifiers on single trial ERPs independently at each time sample (downsampled to 250 Hz) using a critical set of occipital electrodes present in all the caps used (O1, O2, P7, P8, P3, P4, Pz, TP9, TP10), for each of the three planned binary comparisons. As in previous work, these electrodes were chosen prior to data analysis, given that they provide coverage of the key occipito-temporal areas critical for the visual processing of face stimuli Mares et al., 2020;. Prior to training with SVM, the activity in each feature in the dataset was normalized to be within a range of À1 to 1 . The test data was similarly normalized with the same parameters in order to optimize the classification performance (Chang & Lin, 2011).
For each of the three comparisons, the classifier was trained (70% of trials) and tested (30% of trials) in independent sets of data. Number of trials was equalized across experimental conditions (to the condition with the minimum number of trials by randomly removing trials from the condition with more trials until matched). The random split of the data in training (70% of trials) and testing (30% of trials) sets was repeated 20 times to form 20 cross-validation iterations. To calculate accuracy, the trained classifier was tested against the average of all trials per condition in the testing set. This was done in order to better the signal to noise ratio Thomas et al., 2010). To increase the robustness of this procedure, we further repeated it 100 times (Cauchoix, Barragan-Jason, Serre, & Barbeau, 2014). A measure of chance level was calculated by repeating the above procedure on trials with permuted labels (i.e., randomly reassigning the category label).
At the group level, significant decoding was calculated independently for each time point. A one-tailed paired t-test (False Discovery Rate [FDR] corrected) was used to compare average decoding performance in each key comparison with our measure of chance level. To limit the number of multiple comparisons, this analysis was only conducted for the time samples between 60 and 500 ms only (111 comparisons, Farran et al., 2020;Mares et al., 2020).
To analyse significant decoding at the individual participant level, an individual chance level distribution was generated by training a further 899 iterations of the classifier using permuted labels (see also Farran et al., 2020;Mares et al., 2020). We included a classifier trained with the true labelling in the chance distribution of each individual, as it is one of the possible outcomes (Pereira, Mitchell, & Botvinick, 2009), making a total of 1000 iterations of the classifier. Significant decoding, established at each timepoint (FDR corrected) was considered when the average decoding performance obtained with correct labels was greater than or equal to 95% of the null distribution (see Pereira et al., 2009;Smith & Muckli, 2010).
As in previous work Mares, et al., 2020) we used four measures to characterise individual level decoding: 1) peak decoding performancedthe maximal positive peak in the key time window between 100 and 300 ms, and 2) latency; 3) decoding onsetddefined as the first time-point where significant decoding exceeds chance and baseline levels and 4) percentage of significant decoding in the analysed window (60e500 ms).
Pearson correlations (IBM SPSS Statistics 26) analysed the relationship between each decoding metric and face processing expertise for the three key categorisations (upright vs inverted faces, upright vs inverted houses, and faces vs houses). To assess the relative strength of the null hypothesis, we also ran equivalent Bayes correlations (JASP, Version .14). The Bayes analysis provides a likelihood ratio, with a value of 1 indicating that both null and experimental hypothesis are equally likely. Values less than 1/3 provide substantial evidence towards the null hypothesis, and values over 3 indicate substantial support of the experimental hypothesis, with values in the middle indicating weak or anecdotal evidence (Dienes, 2014). For completeness and to aid comparison to existing literature, we also report the correlation between the size and latency of the traditional face inversion effect measured on the P100 and N170 ERP components with face processing expertise.

Face decoding: group level
Although the primary goal of the present work was to explore if markers of the early neural response to faces differed as a function of face processing expertise at the individual level, we first visualised the results on the artificially dichotomised group level. We characterise both face orientation decoding i.e., classifying the neural pattern as occurring in response to an upright or an inverted face, and face category decoding i.e., classifying the observed neural pattern as occurring in response to an upright face or a house, as a function of face processing expertise grouping (Fig. 1 A). Both present a visual illustration of the time course of decoding accuracy, with participants grouped as high and low performers on the measure of face processing expertise (the CFMT). We note that significant orientation decoding was observed clearly in both groups, beginning around 80e90 ms after stimulus onset (88 ms for high, 80 ms for low), with markedly different levels of decoding, at least initially, which may be indicative of an enhanced early differentiation in neural response for participants displaying high face expertise. Similarly, significant category decoding was observed in both groups from around 90e100 ms (92 ms for high, 96 ms for low) but with little difference at the group level in decoding magnitude.

Face decoding: individual level
Rather than rely on observations made at the group level we formally considered decoding at the individual level and correlate the 4 markers of decoding profile (peak decoding magnitude, peak decoding latency, onset of decoding, decoding duration) with individual scores on the CFMT. Importantly, we observed significant decoding of both face orientation and face category in all participants in the considered window (60e500 ms, see Supplementary Figures 1  and 2 for results for plots of the decoding time course for each individual participant). There was a clear correlation between peak decoding magnitude of face orientation and face expertise with an increased accuracy of decoding for participants with higher face expertise (r ¼ .51, p ¼ .001, 95% CI [.236 to .714], for completeness also at the group level using a median split, t (29.48) ¼ -2.07, p ¼ .047, Cohen's d ¼ .67, see Fig. 1B). This was supported by the Bayesian correlation analysis with strong evidence (BF 10 ¼ 43.74) for the experimental hypothesis. Importantly this sensitivity to orientation was face specific, with no significant decoding of house orientation (upright vs. inverted buildings) observed (see Fig. 1D), ruling out any simple explanation at the level of pictorial orientation differences. Given that there was no significant decoding at the group level we did not proceed to more in-depth analysis. In marked contrast to face orientation decoding there was no significant relationship between peak decoding magnitude for category decoding and face expertise (r ¼ .037, p ¼ .824 95% CI [À.282 to .348]). Rather, the Bayesian analysis indicated evidence for the null (BF 10 ¼ .204).
The same profile of enhanced decoding of face orientation with greater face expertise was observed when trials were split as a function of potential face identity familiarity (see Supplementary Figure 3), ruling out a possible role for this factor in driving the response. Note that this control analysis is shown for completeness only and no further statistical analysis is presented due to power (insufficient trial numbers).
Furthermore, for both face orientation and face category decoding we observed evidence that decoding began earlier for participants showing high expertise (face orientation:  Fig. 1 e A. Time course of classification accuracy for face orientation (decoding the neural response to upright compared to inverted faces) on the left-hand side, and for face categorisation (decoding the neural response to upright faces compared to upright houses) on the right hand side. For visualisation purposes participants are grouped by CFMT (%) performance into two groups: above median ability (in blue) and below median ability (in red). Dashed lines represent chance performance via group level permutation tests. Asterisks represent points at which group decoding is significantly greater than chance (p < .05, FDR corrected). B. Relation of face orientation classification metrics to behavioural face expertise (%) for decoding onset (left), peak decoding (middle) and decoding sustainability (right). C. Relation of face categorisation classification metrics to behavioural face expertise (%) for decoding onset. D. As A, time course of decoding house orientation.

ERP effects and expertise
In parallel to the multivariate analysis, we explored the relationship between face expertise as measured on the CFMT, with the face inversion effect as measured on the P100 and N170 components (both amplitude and latency). Channels for ERP analysis were selected based on the maximum peak amplitude for the P100 (O1/2, P7/8) and N170 components (O1/2, P7/8 and TP9/10) from the average of all conditions over parieto-occipital channels. The ERP face inversion effect (FIE) was measured at the P100 and N170 components by subtracting the amplitude measured at the selected electrodes (bilaterally) for inverted faces, from the amplitude from the same electrodes for upright faces. JASP (version .16.3) was employed for subsequent statistical analysis.

P100
Mean amplitude was calculated for the P100 ERP component in a 20 ms window centered around the latency of the grand average peak (100 ms). Individual P100 peaks were identified for latency analysis as the maximum positive peaks occurring between 70 ms and 180 ms after stimulus onset in each individual. One participant was removed from the FIE latency analysis due to the lack of a peak in the considered window for inverted faces.

N170
For the N170 component mean amplitude was calculated in a 40 ms window centred around the average peak (160 ms). N170 peaks were identified for latency analysis as the maximum negative peak between 150 ms and 240 ms after stimulus onset. One participant was removed from the FIE latency analysis, and two from the category difference (upright facesdupright houses) latency analysis due to lack of relevant peaks in the considered window.

Discussion
The current study set out to explore if the early neural response to faces differs as a function of typical variability in level of face expertise, as measured by the Cambridge Face Memory Test. Using multi-variate pattern analysis, at the individual level, we observe a clear association between face expertise and the extent to which face orientation information could be decoded from the ongoing neural response (decoding upright compared to inverted faces). Notably this relationship was specific to faces and face orientation, no such decoding was possible for a control object category (upright and inverted houses) indicating that the decoding was not simply that of pictorial orientation cues. Furthermore, no such relationship was observed between the behavioural face expertise measure and decoding of faces as an object category (decoding upright faces compared to upright houses). An increased magnitude of decoding of face orientation in participants with high face expertise indicates a greater difference in the pattern of neural response to upright compared to inverted faces in these participants. It is tempting to interpret this as suggestive of an enhanced neural representation of the upright face configuration in face experts, but unlike traditional analysis approaches where the absolute magnitude of neural response can be visualised and compared across conditions and groups, due to the nature of the decoding approach this is not possible. One could equally conclude that it is in fact a poorer representation of inverted faces in participants with low face expertise that drives this difference.
Moderate links were also observed between the onset of significant decoding both of faces as an object category (upright faces compared to houses) and the canonical orientation of faces (upright faces compared to inverted) with face expertise as measured via the CFMT. These findings are suggestive of an earlier activation of face specific neuronal populations in face experts. Differences in onset were small (~6 ms on average between low and high ability groupings), with both face orientation and face category decoding significantly different from chance from around 100 ms post stimulus onset.
While the timing of peak multivariate decoding is more in line with the N170 component, it is in fact earlier in the decoding timeline that parallel results were also found in the complementary ERP analysis. Here analysis of the P100 component provides further support for an early neural link between the differential response to upright and inverted faces and face expertise, i.e., larger face inversion effects as measured by the amplitude of the P100 component were directly associated with increased face expertise score. This association is in line with extant literature suggesting that the P100 is sensitive to face orientation (Colombatto & McCarthy, 2017;Itier & Taylor, 2004a;2004b;Minami et al., 2015). Further, early significant decoding of the neural response to faces as an object category (cf. houses) is in line past findings indicating face category effects over the P100 (Eimer, 1998;Herrmann et al., 2005;Itier & Taylor, 2004b) in addition to the more typical low-level stimulus properties modulating this early neural response.
Unlike the multivariate analysis, with the standard ERP approach, one can observe the neural response to upright and inverted faces separately. Despite no significant association of the response to either category alone with expertise, the pattern of results provides a tentative indication that it is in the response to upright faces where the variation is greatest with respect to face expertise score, appearing to fall as expertise increases, whereas the response to inverted faces remains more stable.
It is interesting to note the absence of any relationship between decoding or ERP difference measures and expertise grouping for the comparison of upright faces and houses. This suggests an effect that is specific to the expert face mechanisms involved in the face inversion effect. Previous accounts have suggested that the neural correlates of the FIE may be driven by a violation of expectations generated through a lifetime of experience with upright faces (Allen-Davidian et al., 2021;Brodski, Paasch, Helbling, & Wibral, 2015). Developmental findings seem to support this account, with children improving their memory ability with upright faces between 8-9 years and 12e13 years, a timeframe where children start to show a clear shift in their N170 FIE (Itier & Taylor, 2004c). While children between 6 and 10 show an increased N170 amplitude for upright faces compared to inverted, this pattern shifts around 10e11 years of age to the typical N170 FIE with increased amplitudes for inverted faces from 12 years of age onwards (Itier & Taylor, 2004 Q3 ). This pattern would be compatible with an experience based development of face expectations, and the notion that an increase in the N170 for inverted faces reflects an increase in prediction error, associated with the violation of expectations. Stronger support for this account is given by findings in adults where the N170 face inversion effect increases with multiple violations of viewer's expectations (e.g., orientation, illumination from above or gravitational pull; Allen-Davidian et al., 2020). In line with this account, participants with increased face expertise, might be able to construct stronger expectations for upright faces, and thus be more sensitive to violation of expectations in inverted faces as translated in their neural responses.
In the present study, the main association between expert face processing and early neural response was observed in relation to face orientation, both in the decoding of the neural signal and the P100 amplitude. The behavioural face inversion effect has a longstanding history (Yin, 1969) and is often considered a hallmark of expert face processing, alongside e.g., the composite face effect (Farah, Wilson, Drain, & Tanaka, 1998) in highlighting specialised processes for (upright) faces (Valentine, 1988). In interpreting the current findings, it is worth considering the functionality likely to be driving this difference in decoding. Distinct profiles of neural responses to upright vs inverted faces could reflect quantitative and/or qualitative differences in the underlying processes. ERP theorists are divided on whether observed signal enhancement for inverted (cf. upright) faces is best understood as a reflection of the relatively increased difficulty of processing the non-canonical view (i.e., the same mechanisms utilized in both cases), or additional processing resources are recruited for inverted stimuli (e.g., object-general encoding systems), which complement those used for upright faces (see Sadeh & Yovel, 2010 for discussion). Here, although the selective links observed between orientation effects and face recognition ability make it tempting to conclude that our findings highlight the importance of finely-tuned face-selective neurons for expertise, our MVPA and ERP results are actually consistent with either of these possibilities. It is interesting to note that relatively attenuated face inversion effects have also been reported in ERP studies with individuals with developmental prosopagnosia (e.g., Towler et al., 2012). There, researchers remain similarly agnostic regarding whether the reduced differentiation of neural activity associated with upright and inverted faces reflects quantitative or qualitative differences in the encoding of upright vs inverted faces in this clinically impaired group (Towler et al., 2017).
Here we chose to use the Cambridge Face Memory Test as a measure of face expertise in our participants. This behavioural task, with a considerable cognitive and memory demand, is not the most likely candidate to correlate directly with early neural responses that are driven largely by early perceptual processing (e.g., see discussion by Xu, Liu-Shuang, Rossion, & Tanaka, 2017). That there is such a clear association speaks to the importance of considering the earlier, likely more perceptual aspects, of face processing when trying to understand typical variability in face-expertise. Further, as highlighted by Rossion and colleagues (Rossion, Retter, & Liu-Shuang, 2020) it is important to keep in mind that face processing at the behavioural level and the neural response to faces are comprised of a myriad of different factors and neither reflects a "true" baseline marker of face processing per se. In all instances the particulars of the task employed will necessarily drive differences in the extent to which different mechanisms are employed. Here, we choose to use an explicit task unrelated to face processing (detection of butterflies on discarded catch trials) while recording brain response to images of faces and houses. The absence of a specific concurrent face-related behavioral task is important to the extent that it removes any potential biases driven by associated differences in performance/strategy that our differentially skilled participants might have. Future studies could explore how explicit face processing tasks, which tap into different aspects of face processing (e.g., tasks with perceptual vs. a memory component) modify these associations. A potential limitation of the current study is the use of the standard version of the CFMT to index participants' face expertise. This measure is likely to have a more limited range of scores compared to the harder, extended form of the task that is being used increasingly in individual differences research (Russell et al., 2009). Indeed, we observed very high performance on this task in several subjects (Fig. 1), which might have somewhat constrained our ability to identify associations with the targeted neural markers.
A clear strength of the current analysis approach is in the use of machine learning tools, complimenting traditional ERP approaches, to tease apart differences in the measured neural response. In particular, using a multi-variate pattern classification algorithm which inputs data from across a wide region of the scalp (here all occipital and parietal channels) necessarily permits a more diffuse pattern of response to be evaluated and contrasted between conditions. Where traditional ERP analysis tend to be restricted to a small subset of electrodes (in some cases only one or two per hemisphere) and around the peak of the component of interest, MVPA uses information from across the electrodes at each time sample. Importantly, rather than rely on overall group statistics to highlight when classification is possible at greater than c o r t e x x x x ( x x x x ) x x x chance levels as is often the case with such classifier approaches (e.g., Barragan-jason, Cauchoix, & Barbeau, 2015), we extended our approach to classify the response at the individual participant level. It is noteworthy that such classification was possible in each and every participant tested, allowing us to draw out clear metrics of classification (peak decoding accuracy, latency of onset of significant decoding, latency of peak decoding and overall decoding across our epoch of interest) and apply standard inferential statistical approaches alongside Bayesian analysis to directly compare performance in a continuous manner. This represents a clear extension of previous applications of pattern classification to dynamic brain signals and to this topic in particular (e.g., Balas & Grant, 2016) providing a considerably more nuanced account of the topic under investigation. Such an approach where we explore decoding at the individual rather than group level is especially important in studies such as ours, where there is no established principled means to separate participants into dichotomous groups (a CFMT score may be considered to be high in one study and low in another based simply on the specifics of the sample).
In conclusion, we show that differentiation of the early neural response to upright and inverted faces is significantly associated with an independent explicit behavioural measure of face processing ability. Compared to individuals who perform less well on the behavioural task, individuals who perform better exhibit a pattern of neural response to upright faces that is significantly more distinct from their response to inverted faces, as evidenced both by MVPA and by more traditional ERP analyses. Further this classification of upright faces (vs. inverted faces or objects) begins earlier in the processing time course for individuals with better scores on the behavioural task. As results continue to emerge of clear individual differences in the behavioural and neural response to faces within the typical population it is becoming clear that researchers must consider this natural variation in cognitive and neuro-cognitive models of face processing.

Declaration of competing interest
The Authors declare that there is no conflict of interest.