Predictability modulates the early neural coding of spatially unattended fearful faces

In this study, we assessed whether predictability affected the early processing of facial expressions. To achieve this, we measured lateralised early-and mid-latency event-related potentials associated with visual processing. Twenty-two participants were shown pairs of bilaterally presented fearful, happy, angry, or scrambled faces. Participants were required to identify angry faces on a spatially attended side whilst ignoring happy, fearful, and scrambled faces. Each block began with the word HAPPY or FEARFUL which informed participants the probability at which these faces would appear. Attention effects were found for the lateralised P1, suggesting that emotions do not modulate the P1 differentially, nor do predictions relating to emotions. Pairwise comparisons demonstrated that, when spatially unattended, unpredicted fearful faces produced larger lateralised N170 amplitudes compared to predicted fearful faces and unpredicted happy faces. Finally, attention towards faces increased lateralised EPN amplitudes, as did both fearful expressions and low predictability. Thus, we demonstrate that the N170 and EPN are sensitive to top-down predictions relating to facial expressions and that low predictability appears to speciﬁcally affect the early encoding of fearful faces when unattended, possibly to initiate attentional capture. © 2024 The


Introduction
Humans have a unique means of expressing information effectively and rapidly e that is, through our faces.Facial expressions constitute a significant part of communication as they provide prominent non-verbal information about changes in the environment.For example, a happy face may indicate an incoming reward, whilst a fearful face could signal an unforeseen threat.Our visual cortices are highly efficient at detecting faces, thus information derived from facial expressions should be processed rapidly to help regulate behaviour.The threat capture hypothesis posits that threatening signals (such as a fearful face) are prioritised above other visual information, even when they appear outside the focus of attention ( € Ohman & Mineka, 2001).The automatic encoding and rapid orientation towards threatening stimuli would be advantageous for survival as it provides the organism precious milliseconds to prepare and react (e.g., € Ohman, Juth, & Lundqvist, 2010).However, the automatic encoding of threat-related stimuli may be unnecessary when their presence is predicted.This is because the organism is wellprepared to deal with that threat.Accordingly, the question arises as to whether the automatic encoding of threatening stimuli is affected by their predictability, or alternatively, whether predictability affects the automatic encoding of emotional stimuli more generally (i.e., fearful and happy faces alike).
We addressed this research question with event-related potentials (ERPs) due to their high temporal resolution.Three ERPs were selected as they represent distinct stages of face and emotion processing.The P1 e a positive deflection recorded over parieto-occipital electrodes z 100 ms poststimulus e is considered to be an early indicator of visual processing generated within the extrastriate visual cortex (Clark & Hillyard, 1996).P1 modulations for faces compared to other stimuli (Neumann, Mohamed, & Schweinberger, 2011;Schindler, Tirloni, Bruchmann, & Straube, 2021) and for threat-related versus neutral facial expressions are highly diverse (for a review, see Schindler & Bublatzky, 2020) and attributable to low-level information (Schindler, Bruchmann, Gathmann, Moeck, & Straube, 2021).On the other hand, the N170 e a negative deflection recorded over parieto-occipital electrodes z 170 ms post-stimulus e is reliably enhanced for faces compared to objects and thus widely considered to index the early structural encoding of faces (Eimer, 2011).The N170 demonstrates clear emotion effects relative to neutral faces, with particular sensitivity for threat-related facial expressions (Hinojosa, Mercado, & Carreti e, 2015; for a review, see Schindler & Bublatzky, 2020).Both early components have been shown to be sensitive to attention (Luck, 1995;Luck et al., 1990Luck et al., , 2000;;Mangun, 1995;Slagter, Prinssen, Reteig, & Mazaheri, 2016), whilst the N170 has been shown to be sensitive to predictability in a dose-dependent fashion (Robinson, Breakspear, Young, & Johnston, 2020).Following early ERP components is the mid-latency Early Posterior Negativity (EPN) and Late Positive Potential (LPP).Both components reflect a differential between emotionally arousing and neutral images; for the EPN, this occurs as a negative deflection 200e300 ms post-stimulus over temporo-occipital sites, whilst for the LPP it is a positive deflection 300e600 ms poststimulus over central-parietal sites (Maffei et al., 2021;Schindler & Bublatzky, 2020;Wronka & Walentowska, 2011).The EPN is thought to reflect the enhanced processing of emotionally salient stimuli (with a particular sensitivity for threatening faces; for a review, see Schindler & Bublatzky, 2020) and has been related to early attentional selection (Jungh€ ofer, Bradley, Elbert, & Lang, 2001;Maffei et al., 2021;Schupp et al., 2004;Wronka & Walentowska, 2011).Similarly, the LPP is modulated by facial expressions (see Schindler & Bublatzky, 2020) and is viewed as the reflection of high-order evaluation, episodic memory encoding, and affective labelling (for an overview, see Schupp, Flaisch, Stockburger, & Jungh€ ofer, 2006).However, as the LPP reflects a more latestage of visual encoding it will not be explored in this study.
Several studies have investigated how an individual's attentional capacity affect these early-and mid-latency components and whether a threat-bias for faces maintains outside the focus of attention.Manipulations of taskdemands or perceptual load appear to have no effect on P1 nor N170 emotion modulations (Durston & Itier, 2021;Hudson, Durston, McCrackin, & Itier, 2021;Itier & Neath-Tavares, 2017;Maffei et al., 2021;Rellecke, Sommer, & Schacht, 2012;Schindler, Tirloni et al., 2021).In contrast, EPN threat modulations often dissipate or partially reverse when participants engage in a face-unrelated visual tasks (Maffei et al., 2021;Schindler, Bruchmann, Steinweg, Moeck, & Straube, 2020;Schindler, Tirloni, et al., 2021).With respect to manipulations of spatial attention, spatial attention engaged away from the faces appears to mitigate P1, N170 and EPN threat modulations.For example, Burra and Kerzel (2019) employed a paradigm where faces were presented peripherally, and participants engaged in expression-unrelated tasks either at fixation or at peripheral areas.They found that when faces were peripherally attended to, the lateralised N170 to angry faces was enhanced compared to neutral and happy faces; however, when faces were task-irrelevant and outside the focus of spatial attention, the N170 enhancement towards angry faces dissipated.In a similar paradigm, Holmes, Vuilleumier, and Eimer (2003) found that the N170 amplitude was enhanced for peripherally attended fearful faces compared to neutral faces; yet again, when fearful faces were unattended, the N170 enhancement was eliminated.Conversely, Schindler, Richter, Bruchmann, Busch, and Straube (2022) presented faces centrally and required participant to solve a face-unrelated visual task (which varied in task-difficulty) either in peripheral or foveal locations.The authors found that the N170 maintained its enhancement to fearful faces irrespective of task-load or spatial attention; however, the EPN had an increased negativity for fearful faces only when the faces were spatially attended and taskdemands were low, whilst the P1 demonstrated no emotion effects.Again, Holmes, Kiss, and Eimer (2006) employed a similar design but found no effects of emotion on the N170 yet within the EPN window similarly observed spatial attention effects on amplitudes towards fearful faces.
Thus, findings on how attention capacity affects early-and mid-latency components to facial expression appears muddled.Spatial attention but not task-load seems to influence emotion processing at the P1 and N170 timeframes, while the EPN is generally affected by both manipulations.Additionally, a threat-bias rarely maintains when attentional resources are engaged elsewhere spatially (refer to Burra & Kerzel, 2019;Holmes et al., 2003;Holmes et al., 2006) and if a threat-bias does maintain, it is when faces are presented at foveal, not peripherally (Schindler et al., 2022).However, is it appropriate to assume that a threat maintains its relevancy e and thus engenders enhanced neural encoding and automatic detection outside the focus of attention e when its presence is predicted?Indeed, research on the visual mismatch negativity (vMMN) suggests that neural encoding of unattended facial expressions is modulated by a third variable: their predictability.
The vMMN e a negative-going ERP component with a posterior scalp distribution that usually peaks between 100-200 ms and 200e500 ms post-stimulus e is evoked when an unpredicted (deviant) stimulus violates the inherent rule in a series of frequent (standard) stimuli, typically outside the focus of attention (see Stefanics, Astikainen, & Czigler, 2015).With relation to faces, researchers have found that deviant fearful and happy facial expressions elicit a vMMN when presented in a stream of neutral faces e a ERP waveform termed the emotional mismatch negativity ("eMMN"; Astikainen & Hietanen, 2009;Stefanics, Csukly, Komlosi, Czobor, & Czigler, 2012;Kimura, Kondo, Ohira, & Schr€ oger, 2012;Kovarski et al., 2017).Findings suggest that the eMMN exhibits a bias for threat-related facial expressions (Kimura et al., 2012;Kovarski et al., 2017;Stefanics et al., 2012) and that the eMMN is different to a vMMN for deviant neutral faces (Kovarski et al., 2017;Vogel, Shen, & Neuhaus, 2015).Additionally, researchers have posed that the early eMMN could correspond to the face-sensitive N170 e a hypothesis later proven by Astikainen, Cong, Ristaniemi, and Hietanen (2013) e and that the late eMMN could be associated with the EPN (Astikainen & Hietanen, 2009).Thus, as these components have been affiliated with the eMMN, it is plausible that they are similarly affected by predictability.
In summary, prior studies on early-and mid-latency ERP components have found that emotion modulations are dependent on spatial attention and/or task-relevancy, and that a threat-bias rarely maintains outside the focus of attention (Burra & Kerzel, 2019;Holmes et al., 2003Holmes et al., , 2006;;Maffei et al., 2021;Schindler et al., 2020Schindler et al., , 2021Schindler et al., , 2022)).However, the literature on the eMMN suggests that predictability modulates the neural processing of unattended facial expressions (Astikainen & Hietanen, 2009;Kimura et al., 2012;Kovarski et al., 2017;Stefanics et al., 2012;Vogel et al., 2015) with a larger differential negativity to threat-related faces compared to non-threatening faces (Kimura et al., 2012;Kovarski et al., 2017;Stefanics et al., 2012).Furthermore, research has demonstrated that the eMMN at 150e220 ms corresponds to the face-sensitive N170 (Astikainen et al., 2013) and that the eMMN at 250e360 ms could be linked to the EPN (Astikainen & Hietanen, 2009).However, it remains unclear whether the effects found for the eMMN would replicate for early-and mid-latency ERP components in the parieto-occipital region (i.e., the P1, N170 and EPN).
Thus, we aim to assess whether predictability of facial expressions does in fact affect early-and mid-latency ERPs when faces are spatially unattended, and additionally, whether predictability affects facial expressions differently (i.e., happy versus fearful faces).To address this aim, we employed a within-subjects design that manipulated participants' predictions of fearful and happy faces along with their spatial attention towards the faces.Specifically, a cue informed whether happy or fearful faces were more frequent.Participants were then presented with stimuli left and right of fixation: a facial expression on one side and a scrambled neutral face on the other side used as a physically balanced object.Participants were required to attend to one side and respond when an angry face appeared on that side, ignoring all other emotions.Thus, happy and fearful faces were taskirrelevant, yet predictive knowledge was assigned to them, and they could be spatially attended or unattended.Angry faces were used as targets to ensure that the participant would attend to the designated side and track the facial expressions.Additionally, due to the extensive number of trials required to operationalize predictability, we opted against a neutral control to first ascertain whether predictive processes affected happy and fearful faces differentially, as suggested by research on the eMMN (refer to Kimura et al., 2012;Kovarski et al., 2017;Stefanics et al., 2012).Participants' brain electrical activity was measured with scalp-recorded EEG and lateralised ERP components (i.e., the P1, N170, and EPN) were examined.Lateralised ERPs were of concern as this approach has the benefit of removing any asymmetries between the left and right hemisphere.Specifically, lateralised ERPs are triggered by peripheral events where a target stimulus in the left or right visual field competes with a physically balanced nontarget object in the opposite visual field e in this case, scrambled neutral faces were used as physically balanced non-target objects.Instead of analysing ERPs to left and right targets, contralateral and ipsilateral ERPs to the target stimulus are calculated.Consequently, only the processing of the target stimulus is captured, allowing for robust inferences about the timing of corresponding neural events (Luck, 2005).We hypothesize that low predictability would produce larger lateralized N170 amplitudes for both fearful and happy faces irrespective of spatial attention; however, this enhancement would be greater for fearful faces compared to happy faces.Tentatively, we hypothesize a similar pattern of results for the P1 and EPN.

Material and methods
We report how we determined our sample size, all data exclusions, all inclusion/exclusion criteria, whether inclusion/ exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study.

Participants
Our sample size was based on prior research exploring the interactions between attention, prediction, and emotion using the eMMN.Significant results were reported with population sizes of 12e22 participants (see Astikainen et al., 2013;Kimura et al., 2012;Astikainen & Hietanen, 2009;Kovarski et al., 2017;Stefanics et al., 2012;Vogel et al., 2015).We therefore aimed to collect data from a minimum of 20 participants with useable measures.To be included in the study, participants had to be adults over 18 years of age and have no self-reporting neurological or psychiatric disorders.Twenty-eight participants were thus recruited for the study and compensated $40 (AUD) for their time.Participants had normal or corrected-to-normal vision and were all righthanded (as assessed by the Edinburgh Handedness Inventory;Oldfield, 1971).After eye-movements and other artefacts were removed, four participants were excluded from the analysis as they provided insufficient data.Two participants were also excluded due to experimental error.The final sample included 22 participants with an age range from 18 to 54 years (M age ± SD age ¼ 26.5 ± 9.1; 10 males).The study was approved by the University of Queensland's Human Research Ethics Committee (Application ID: 2022/HE000810).All participants provided written informed consent for their participation.

Apparatus and stimuli
Stimuli were presented on a 24-inch ASUS LCD monitor (model VG248QE; refresh rate: 144 Hz; resolution: 1920 Â 1080 pixels) placed 70 cm away from the participants' eyes.A Dell KB522p keyboard was wired to the monitor for participants to record their response.E-prime 3.0 was used to present stimuli and record participants' behavioural data.
The face stimuli were obtained from the Karolinska Directed Emotional Face Database (Goeleven, De Raedt, Leyman, & Verschuere, 2008).Fearful, happy, and angry faces were selected from 40 face identities (20 males, 20 females).A total of 120 face stimuli were used.Face images were cropped into an oval shape of 8 cm Â 6.2 cm (6.5 Â 5.1 in visual angle) to maintain only relevant facial information (i.e., forehead, eyebrows, eyes, mouth, and chin).Images were rendered blackand-white.The Scramble Filter tool (Telegraphics, 2021) was used on neutral faces to generate scrambled faces.This consists of 208 squares (4.4 mm Â 4.4 mm each) that were randomly scrambled to ensure the face was not identifiable but maintaining the same overall luminance.All image editing was performed with Photoshop 2021.
For each stimulus presentation, two face stimuli (one emotional, one scrambled) were presented bilaterally with the centre of the face image position 5 cm (4.1 in visual angle) away from a central fixation cross at the centre of the screen.The word 'HAPPY' or 'FEARFUL' would be presented at the beginning of each block in the centre of the screen, Times New Roman font, 12 font size.

2.3.
Design and procedure The experiment followed a 2 (attention: attended, unattended) x 2 (prediction: predicted, unpredicted) x 2 (facial expression: happy, fearful) within-participants design.Participants engaged in a Hillyard sustained attention paradigm (Luck & Kappenman, 2011), covertly attending to the left or right region of space and responding to angry faces that appeared in the assigned attended region.Before beginning experimentation, participants responded to the Edinburgh Handedness Inventory, received formal instructions on the task, and conducted a practice run consisting of 20 trials.Participants were informed that each block would begin with the word cue 'HAPPY' or 'FEARFUL'.If the block started with the word 'HAPPY', happy faces appeared z 70% of the trials and fearful faces appeared z 30% of the trials.The opposite was true if the word 'FEARFUL' appeared.Participants were required to ignore happy and fearful faces and respond to angry faces that appeared on the attended side by pressing the 'H' key.The word cue at the beginning of a block provided no information about the appearance of angry faces; rather, angry faces would consistently appear 6 times throughout each block (3 on the attended side, 3 on the unattended side).Participants were instructed to spatially attend to one side of the screen (either the left-side or right-side) and respond to angry faces that only appeared on that side.Half-way through the experiment, participants were instructed to attend to the other side that was previously ignored and respond to only angry faces on that side.The ordering as to whether participants attended on the left or right side first was counterbalanced between participants.During the practice trials, it was emphasised that participants must fixate their gaze on the fixation cross and remain still for the duration of each block.Once the participant confirmed they understood the task, the main experimental task began.The experiment began with the word 'HAPPY' or 'FEARFUL' which appeared for 3000 ms.Each trial then began with a fixation cross which lasted for 1000 ms, followed by an arrow e either pointing left or right with the fixation cross superimposed onto the arrow; this was to remind participants to covertly attend to that side e for a variable duration between 500 and 800 ms, and then two faces (one emotional, one scrambled) bilaterally of the arrow for 100 ms.Once the faces disappeared, the arrow would remain on the screen for 800 ms.At this point the participant would respond to whether an angry face appeared in the attended side of space.This processed repeated itself 63 times to make-up one block.At the end of a block, participants received a short break.There was a total of 20 blocks.For the first 10 blocks participants would covertly attend to one-side of the screen, and in the last 10 blocks participants would covertly attend to the opposite side e again, this was counterbalanced between participants.Fig. 1 provides an illustration of a single trial.Each block e which contained 63 trials e took z 2 min to complete.Participants completed the task in approximately 65 min.

EEG recording and pre-processing
Continuous active EEG was recorded at 1024 Hz via a 24-bit, 64 channel BioSemi ActiveTwo system with a resolution of 31.25 nV (±262 mV recording range).The 64 Ag/AgCl scalp electrodes were placed according to the internal 10e20 system location using a Waveguard nylong EEG cap for electrode placement.Recordings were referenced to the CMS/DRL electrodes.Horizontal/vertical eye movements and blinks were recorded using two electro-oculographic (EOG) electrodes placed at the outer canthi and vertically below the right eye.Pre-processing of the EEG data was performed with BrainVision Analyzer 2.0 (BrainVision Analyzer 2.0, Brain products GmbH).Individual electrodes that produced either flatline signals or sustained noise throughout the experiment were interpolated e though, this step was not performed for electrodes in the regions of interest and no more than 4 electrodes were interpolated.Signals were re-sampled to 512 Hz offline, filtered from .1 to 30 Hz, and referenced to the average of all electrodes.Ocular artifacts were corrected using Independent Component Analysis.ERP waveforms were timelocked to the onset of the target stimuli and segmented from 100 ms pre-stimulus to 500 ms post-stimulus.Trials with artefacts of eyeblinks and eye movements were automatically deleted using a threshold of ±80uV.Trials with other artefacts were deleted automatically using a threshold of ±80uV, gradient of ±75uV, and low signal change of .01uV.On average, 93% of trials were kept.Table 1 below demonstrates the average proportion number of trials removed and the average number of trials remaining for each condition.When analysing the proportion of trials removed, a main effect for attention was observed (F ¼ 4.71, p ¼ .041,h p 2 ¼ .06),qualified by an interaction between attention and emotion (F ¼ 6.63, p ¼ .018,h p 2 ¼ .24),such that more trials were removed for c o r t e x 1 7 9 ( 2 0 2 4 ) 2 8 6 e3 0 0 unattended fearful (M ¼ .09,SD ¼ .08)compared to attended fearful conditions (M ¼ .06,SD ¼ .06).The observed discrepancy may stem from the possibility that fearful faces, when not directly attended to, tend to be more disruptive, consequently leading to an increase of EOG artifacts.There were no other differences between conditions (Fs < .26,ps > .616).

Behavioural data
The mean hit, miss, false alarm, and correct rejection rates were calculated across participants.In this example, the happy face which appeared on the screen would be coded as a happy, attended, predicted condition.This is because, at the beginning of the block, the participant was informed that happy faces would be more common, thus the happy face was predicted.In addition, they were instructed to covertly attend to the right side (refer to cue), thus the face happy was attended.In this example, the participant would attend to the righthand side for half the experiment, and then in the second half of the experiment participants would attend to the left-hand side.c o r t e x 1 7 9 ( 2 0 2 4 ) 2 8 6 e3 0 0

ERP processing
Analyses were performed on the mean amplitudes of the P1, N170 and EPN.All components were measured at their maximal occurrence through visual inspection of the topographical distribution of the grand-average ERP.The grandaverage ERP was calculated by adjusting the electrode configuration for each condition such that the right hemisphere's data reflected the average neural response to emotional faces, whilst the left hemisphere's data corresponded to the average neural response to scrambled faces.P1 and EPN components showed maximal activity at the PO7/PO8 and P9/P10 electrode sites, with mean amplitudes calculated within 90e110 ms and 200e300 ms, respectively.For the N170, maximal activity occurred at electrode sites P7/P8 and P9/P10, with mean amplitudes calculated between 158 and 178 ms.Fig. 3 illustrates the electrode sites chosen and averaged topographic activity for each condition.The electrode sites used are consistent with those investigated in prior literature (Burra & Kerzel, 2019;Schindler et al., 2022;Wronka & Walentowska, 2011).
To determine lateralised P1, N170, and EPN amplitudes for attended and unattended conditions, the electrodes contralateral to the presentation of the target stimuli were used for each condition.For example, electrodes P7, PO7, and P9 would be used for right attended and unattended faces, whilst electrodes P8, PO8, and P10 would be selected for left attended and unattended faces.The left and right data was collapsed for each condition creating an overall average for that condition.

Behavioural results
Table 2 demonstrates that, on average, participants correctly responded to 84.7% of targets (angry faces) and incorrectly responded to 3.3% of distractors (happy or fearful faces).Furthermore, the results demonstrated that, on average, participants could reliably discriminate distractors from targets (d' ¼ 2.2, SD ¼ .5),with a slight conservative bias (c ¼ .4,SD ¼ .4).Thus, it was validated that participants understood the task and could reliably discriminate distractors (fearful, happy and jumbled faces) from targets (angry faces).

ERP results
Fig. 2 demonstrates distinct P1, N170, and EPN components in response to attention, prediction, and emotion.These figures suggest that the experimental design and EEG recording, sorting and pre-processing were carried out correctly to uncover lateralised P1, N170, and EPN components.

Lateralised P1
For the P1, there was a main effect of attention To analyse the three-way interaction, paired-sampled t-test were conducted based off the prior hypothesis, as shown in Table 3. Specifically, t-test were conducted to assess whether predictability affects facial expressions differentially across conditions of spatial attention.Table 3 highlights no significant differences in lateralised N170 amplitudes between faces when they were spatially attended.However, when spatially unattended, lateralised N170 amplitudes were significantly more negative for unpredicted fearful faces (M ¼ À6.9, SD ¼ 3.6) compared to predicted fearful faces (M ¼ À6.4,SD ¼ 3.5) and unpredicted happy faces (M ¼ À6.2, SD ¼ 3.4).No other comparisons were significant when faces were spatially unattended.Fig. 5 provides a topographical map of the difference waves for each comparison and Fig. 6 illustrates each condition mean with comparisons highlighted for significance.Specifically, more negative EPN amplitudes were observed for attended (M ¼ À2.6, SD ¼ 1.4) compared to unattended (M ¼ À.5, SD ¼ 1.4), unpredicted (M ¼ À1.7, SD ¼ .8)compared to predicted (M ¼ À1.4,SD ¼ .9),and fearful (M ¼ À1.7, SD ¼ .8)compared to happy (M ¼ À1.4,SD ¼ .9)faces.There were no significant two-way or three-way interactions (Fs < 1.24, ps > .277).Mean amplitude scores for each condition are shown in Fig. 7. Note.The proportions for Hit and Correct Rejection were calculated relative to the number of expected values (60 and 1200, respectively).For example, for Hit, the proportion would be calculated by dividing 50.9 by 60.The proportions of Miss and False Alarm were calculated by subtracting the Hit and Correct Rejection value by 100, respectively.For example, for Miss, the proportion would be calculated by subtracting the Hit value by 100.

Discussion
We investigated whether predictability affects the early-and mid-latency ERP components of facial expressions across conditions of spatial attention.It was hypothesised that low predictability would enhance the lateralised N170 for fearful and happy faces irrespective of spatial attention, and that this enhancement would be greater for fearful relative to happy faces.Furthermore, we tentatively hypothesised a similar pattern of results for the lateralised P1 and EPN.In this study, we observed that spatial attention directed towards the face led to larger lateralised P1 amplitudes.However, there was no difference in lateralised P1 amplitudes according to the type of facial expression nor the predictability of the facial expression.Past studies suggest that P1 modulations to facial expressions are variable, with no systematic demonstration of a threat-bias (Schindler & Bublatzky, 2020).Additionally, research suggests that the P1 is primarily sensitive to lowlevel stimulus properties (i.e., colour, contrast, luminance, and spatial frequencies) and spatial attention (Clark & Hillyard, 1996;Schindler et al., 2022;Schindler et al., 2021).
Our findings align with these conclusions: lateralised P1 amplitudes only differed according to spatial attention.Additionally, we extend on the literature by demonstrating that the predictability of facial expressions does not affect the P1.Our results may suggest that the lateral extrastriate cortex, which is thought to give rise to the P1 component (Clark & Hillyard, 1996), does not receive predictive information relating to facial expressions, irrespective of spatial attention.
Consistent with the literature, spatial attention to faces enhanced lateralised N170 amplitudes (Schindler & Bublatzky, 2022), although contrary to expectations, fearful faces did not further enhance the lateralised N170 compared to happy faces, at least in the spatially attended condition.A threat-bias is often systematically found in the literature for spatially attended faces.For example, when compared to neutral faces, Schindler & Bublatzky (2020) review found N170 emotion effects more consistent for fearful faces relative to happy faces, whilst Hinojosa et al. (2015) meta-analysis demonstrated greater effect sizes for fearful faces followed by happy faces.Additionally, studies have found that angry faces elicit larger N170 amplitudes compared to happy faces (Burra & Kerzel,  c o r t e x 1 7 9 ( 2 0 2 4 ) 2 8 6 e3 0 0 2019; Rellecke et al., 2012).However, our findings are not directly applicable to those mentioned above.For one, our investigation contrasted fearful faces to happy faces e a comparison not commonly performed in the literature.Differences between angry vs. happy are perhaps more pronounced compared to difference between fearful vs. happy faces as the effect sizes for angry faces are larger than those for fearful faces (see Hinojosa et al., 2015).Furthermore, when differences are found between happy and fearful faces, it is when they are compared to neutral faces (Schindler & Bublatzky, 2020).Perhaps if we compared facial expressions to neutral faces, or angry faces to happy faces, a threat-bias may have been uncovered for spatially attended faces.In other words, the type of comparison we performed may explain the lack of threat-bias we found for spatially attended faces.
However, our research is the first to uncover a differentiation in the N170 between happy and fearful expressions when faces are spatially unattended.As expected, this differentiation was dependent on the predictability of facial expression.Specifically, unpredicted fearful expressions enhanced lateralised N170 amplitudes compared to unpredicted happy expressions and predicted fearful expressions, whilst no differences were found between predicted happy and fearful expressions.Previous research has not found an effect of expression on the N170 when faces appear in peripherally unattended and task-irrelevant regions of space, despite an effect when faces are spatially attended and taskrelevant (Burra & Kerzel, 2019;Holmes et al., 2003).As such, some researchers have concluded that without attention, the N170 may be insensitive to facial expressions and, consequently, the enhanced processing of threatening faces is attention dependent (Burra & Kerzel, 2019).However, our research suggests otherwise.Specifically, the N170 is sensitive to threat-related facial expressions outside the focus of spatial attention, so long as the faces are unpredicted.These findings could lead one to conclude that cortical regions associated with the N170 e such as the fusiform gyrus, inferior occipital gyrus, and superior temporal sulcus (Itier & Taylor, 2004;Rossion & Jacques, 2012) e are receiving predictions and encoding information relating to facial expressions independently of attention.Thus, it is plausible that threat-related facial expressions are preferentially and automatically processed, especially when considering that spatially unattended non-threatening (i.e., happy) faces did not similarly enhance the N170 when unpredicted.
Finally, we found main effects of spatial attention, prediction, and facial expressions for the EPN.In the literature, EPN modulations are present for both negative and positive emotions relative to neutral stimuli (Durston & Itier, 2021;Maffei et al., 2021;Schupp et al., 2004;2006), with a slight bias towards negative emotions (Schindler & Bublatzky, 2020).However, the emotion processing of the EPN is dependent on available resources (e.g., see Pessoa, 2009;Pessoa et al., 2013).Specifically, manipulations of spatial attention, taskrelevancy, and perceptual load have been shown to mitigate EPN modulations to facial expressions (Holmes et al., 2006;Maffei et al., 2021;Schindler et al., 2020Schindler et al., , 2021Schindler et al., , 2022)).Our findings align with the literature: we found that the differentiation between happy and fearful faces was enhanced when faces were spatially attended to (Holmes et al., 2006;Schindler et al., 2022), and that this differentiation was biased towards threat-related faces (Schindler & Bublatzky, 2020).Further, we demonstrated that low predictability enhanced lateralised EPN amplitudes e to our understanding, no literature to date has demonstrated this effect.Thus, attention, low predictability, and fearful facial expressions appear to have an additive effect on the lateralised EPN.The converse also appears to be true: the lateralised EPN was not found for spatially unattended happy facial expressions which were predicted.As the EPN is thought to index the awareness of stimuli and further selective attention processes (Schindler & Straube, 2020), our findings suggest that low predictability and fearful expressions engender further selective attention processes, even if faces are spatially unattended and task-irrelevant.Arguably, this effect could be a partial carry over from the N170: an initial alarm is signalled at 170 ms which leads to further selective attention processes from 200 ms onwards.
Taken together, our findings shed new light on the threat capture hypothesis.Specifically, the threat capture hypothesis assumes that threatening stimuli engender enhanced neural encoding compared to non-threatening stimuli, even outside the focus of attention ( € Ohman & Mineka, 2001).However, subsequent research demonstrated that early-and midlatency ERP enhancements to fearful and/or angry faces depend on spatial attention and/or task-demands (Burra & Kerzel, 2019;Holmes et al., 2003Holmes et al., , 2006;;Maffei et al., 2021;Schindler et al., 2020Schindler et al., , 2021Schindler et al., , 2022)).Thus, it was reasonable to conclude that attention is necessary to enhance the early encoding of a threat-related facial expression.However, our research suggest that unattended facial expressions are being predicted as early as 170 ms post-stimulus, and that the enhanced neural encoding of threat-related facial expressions depends on the low predictability of that emotion.Furthermore, the enhancement maintains 200e300 ms later where further selective attention processes can assess the threat.Evolutionarily, it is reasonable to expect that a predicted threat does not engender enhanced neural encoding as its risk to the organism has likely been assessed.Yet an unpredicted threat would facilitate enhanced neural encoding that is resistant to manipulations of spatial attention as it is necessary to attend to and assess that threat.
Subcortical visual pathways could provide a plausible explanation for these findings.Traditionally, visual information is thought to be processed through the cortical retinogeniculo-striate pathway, travelling from the retina to the lateral geniculate nucleus and proceeding to the middle temporal areas and visual areas V1 e V4.However, in conjunction to this pathway, a rapid subcortical pathway has been proposed which carries specific visual information from the retina to the amygdala and subsequently extrastriate visual cortices (Bertini, Grasso, & L adavas, 2016;Morris, € Ohman, & Dolan, 1999).Vuilleumier, Richardson, Armony, Driver, and Dolan (2004) has suggested that threat-related stimuli activate subcortical pathways resulting in increased neuronal activity in the visual areas via retrograde feedback projections from the amygdala.In addition, Framorando, Moses, Legrand, Seeck, and Pegna (2021) has demonstrated that lateralised N170 enhancements to fearful faces relies on the integrity of the right amygdala.Thus, it is plausible that when spatial attention was engaged elsewhere, the enhancement of the lateralised N170 to unpredicted fearful faces was facilitated by subcortical networks and excitatory projections from the amygdala.This reasoning may similarly apply to the EPN: research suggest that neural sources for the EPN are likely driven by lateral occipital extrastriate activity (Jungh€ ofer et al., 2006;Schupp, Stockburger, et al., 2006), thus, excitatory projects from the amygdala to the visual cortex could enable the additive enhancements we observed in the EPN for low predictability and fearful faces.Furthermore, our research may extend on the typical views of the amygdala and suggest that it plays a role in tracking the predictive value of threat-related stimuli; specifically, amygdala activity is further enhanced by threat when it is unpredicted.Although speculative, this interpretation is plausible considering that amygdala activity is reportedly influenced by the predictability of both negative and positive events (Likhtik & Gordon, 2013) and enhanced by fearful faces relative to neutral faces (Bishop, Duncan, & Lawrence, 2004;Framorando et al., 2021;Krolak-Salmon, Henaff, Vighetto, Betrand, & Mauguiere, 2004;Rotshtein et al., 2010;Vuilleumier, Armony, Driver, & Dolan, 2001).However, further research is necessary to confirm these interpretations; in isolation this paper cannot provide evidence that amygdala activity is responsible for our N170 and EPN findings.
Some possible limitations may be put forward.Our experimental task required participants to identify angry faces.This is potentially problematic as angry faces share more common features to fearful faces than happy faces.Thus, fearful faces were perhaps more task-relevant than happy faces due to their configurational similarities to angry faces.However, this issue unlikely provides an alternative explanation to the results: fearful faces did not consistently elicit larger ERP amplitudes across the board, especially not for the N170 e a key marker for the structural encoding of faces e where fearful faces were enhanced only when they were unattended and unpredicted.However, future research should aim to replicate our effects under different task designs to confirm that our effects are not simply due to the unique task design.Furthermore, our findings of predictability can be difficult to interpret.To create a design that operationalises predictability, the predicted stimuli are the repeated stimuli whereas the unpredicted stimuli are seldom repeated.Thus, repetition suppression effects e defined by the reduction in a c o r t e x 1 7 9 ( 2 0 2 4 ) 2 8 6 e3 0 0 measure of neural activity following repeated stimuli presentation e could partially explain differences between predicted and unpredicted stimuli (Feuerriegel, Yook, Quek, Hogendoorn, & Bode, 2021).Although immediate repetition effects could explain our results, we believe that large-scale repetition suppression did not occur as, overall, fearful and happy faces were presented at an equal frequency.Furthermore, we cannot clearly elucidate whether our effects of predictability are due to mechanisms of predictive coding or surprise.For example, our findings may simply demonstrate a surprise response which is present for unpredicted stimuli; not necessarily a suppression affect to predicted stimuli that are "silenced" and an enhancement to prediction errors as described in the theory of predictive coding (Friston, 2005).To combat such inconclusive interpretations, future research should include an equiprobable condition e whereby all stimuli are presented at equal low probabilities e to isolate the effects of predictive coding from surprise.Finally, it may be the case that the observed effects of predictability are attributed to an emotional contrast effect.Or alternatively, more adaption may take place for happy faces compared to fearful faces.In either case, fearful faces are more likely to "pop-out" not due to predictive mechanisms but contrast/adaption mechanisms.Irrespective of how the predictive mechanism are interpreted, it is difficult to deny that fearful faces do not engender a unique form of early encoding that happy faces are not privy to.
In summary, our study investigated how spatial attention and predictability affect early-and mid-latency ERPs to fearful and happy faces.No effect of emotion or prediction was found for the P1, suggesting that the P1 is not a reliable index of facial expression encoding and their predictability.However, spatially unattended fearful faces (compared to happy faces) received preferential encoding at the N170 stage which was facilitated by low predictability.Finally, the EPN demonstrated additive effects to attention, low predictability, and fearful faces.These findings provide novel evidence to the threat capture hypothesis and highlight that predictability of the facial expression is a fundamental mechanism for detecting unattended threat-related facial expressions.
Average d-prime (d')  and criterion (c) values were derived from signal detection theory to examine how participants performed in discriminating targets from distractors.As d 0 is an open-ended scale, values vary from 0 (where participants were guessing) to values of 2 or above (where participants could reliably discriminate targets from distractors).Additionally, c values indicate the bias in participants response.A c value of 0 reflect no bias in response, whilst a negative c value reflects a liberal bias, and a positive c value reflects a conservative bias.Overall, these values were used to validate whether participants understood the task and could discriminate distractors (happy, fearful, and scrambled faces) from targets (angry faces).

Fig. 1 e
Fig.1e Example of a single trial.Note.In this example, the happy face which appeared on the screen would be coded as a happy, attended, predicted condition.This is because, at the beginning of the block, the participant was informed that happy faces would be more common, thus the happy face was predicted.In addition, they were instructed to covertly attend to the right side (refer to cue), thus the face happy was attended.In this example, the participant would attend to the righthand side for half the experiment, and then in the second half of the experiment participants would attend to the left-hand side.

Fig. 2 e
Fig. 2 e Effects of attention, prediction, and emotion on P1, N170, and EPN.Note.ERP waveforms were extracted from the average of two contralateral groups of electrodes (P10/P8 and P7/P9).Left graphs represent absolute waveforms whilst right graphs represent differential waveforms between emotional faces and scrambled faces.

Fig. 3 e
Fig. 3 e Topographic maps for each condition.Note.Topographic maps for P1 and N170 reflect absolute values.Additionally, electrode hemispheres were flipped for when participants attended right.As such, topographic activity represents face processing on the left-hand side in attended conditions and the right-hand side in unattended conditions.EPN topographic maps represent differential negativities by subtracting each condition by neutral scrambled faces.Electrode hemispheres were not flipped for this calculation.Highlighted electrodes represent ROI.

Fig. 4 e
Fig. 4 e Mean Lateralised P1 Amplitudes.Note.Dots represent individual mean scores for each condition.

Fig. 7 e
Fig. 7 e Mean Lateralised EPN Amplitudes.Note.Dots represent individual mean scores for each condition.

Table 1 e
The average proportion of trials remove and average number of trials remaining.Note.Standard deviation appears in the parentheses next to the means.

Table 2 e
Number of hit, miss, false alarm, and correct rejection responses.

Table 3 e
Pairwise comparisons of the interaction between prediction and facial expressions over conditions of attention.