Neural mechanisms of eye gaze processing as a function of emotional expression and working memory load

The aim of the present study was to investigate the role of working memory load on the gaze cueing effect in high and low trait-anxious participants, using electroencephalography. Fearful and neutral faces predicted the location of a target, which was a digit that participants were asked to recall from a series encoded in each trial, in a modified version of the attentional cueing task. Working memory load impacted cueing irrespective of emotion and anxiety in analysis of reaction times. Lateralized EEG components then showed that effects of emotion were only apparent in high anxious individuals, with an initial hypervigilance to target locations cued by fearful faces, followed by a difficulty to disengage from these locations when targets appeared at uncued sites (P1). Enhanced amplitude following fearful faces was observed, when discriminative processes leading to response selection are implemented (N1). Conversely, all the effects of working memory load were independent of emotion in the low anxious group, where the shifting of attention directed by the gaze was only visible when enough resources were available in the working memory span. Working memory loads impacted the processing of gaze differently (P1) in low anxious participants, suggesting that top-down influence may play a role in this case.


Introduction
Trait anxiety is characterised by attentional biases towards threatening stimuli, especially facial expressions [e.g. 1,2]. As such, attentional resources are engaged in the processing of negative information, even when this information is not relevant to the ongoing task [3]. According to the Attentional Control Theory [ACT; 4] this effect can be attributed to decreased attentional control, reducing the gating mechanism for irrelevant threatening information in a top-down manner [3][4][5][6]. Once attended stimuli are subject to enhanced processing, they transfer to working memory. Working memory features such as inhibition, shifting and updating can also be disrupted under some circumstances in high trait anxiety individuals, leading to decreased performance in cognitive tasks in this subpopulation. In this case, the balance between top-down and bottom-up processes is disrupted, leading to enhanced bottom-up processing driven by salient stimuli and decreased top-down cognitive control [4]. In support of this theoretical perspective, psychological research has shown that increased anxiety is associated with increased distractibility and selective processing biases towards threat-related stimuli in a bottom-up manner [e.g. 5,7,8]. Cognitive control processes have also been implicated, with indications that the prefrontal cortex is less activated in attentional tasks in highly anxious participants [9]. This attention towards task-irrelevant negative stimuli in anxious participants invites the hypothesis that over-representation of negatively valenced information in working memory might in turn reduce its capacity to store information relevant to the ongoing task.
Using EEG, Holmes, Mogg, Garcia and Bradley [10] manipulated eye gaze direction and facial expression to identify anterior directing attention negativity (ADAN), a marker of anticipatory spatial attention relative to gaze-congruent trials, that was insensitive to emotional expressions. Another study using a variety of emotional expressions (surprised, happy, fearful and angry) reported similar results [11] in a group of non-anxious participants. Specifically, early directing attention negativity (EDAN) and ADAN components did not differ as a function of emotion. However, in a recent study, Li, Zinbarg and Paller [12] showed larger N2 amplitude for angry faces with direct (versus averted) gaze in high socially anxious participants and in a MEG study, averted fearful faces lead to an enhanced parietal activity in the time range of the C1 component [13]. Further evidence shows specific event-related potential (ERP) responses to fearful faces in patients with anxiety disorders, suggesting an electrophysiological signature for threat in these patients and the early processing of socially relevant cues [14]. All of these studies have used different emotions and different groups of participants (anxious or non-anxious participants), which may explain the lack of homogeneity across their findings.
The electrophysiological correlates of the perception of fearful averted gaze in the context of high trait anxiety therefore remains an open question and may in some cases depend on extraneous task demands such as the engagement of top-down cognitive control processes. Eimer and Kiss [15] found that the N2pc component is activated by task-irrelevant fearful faces in non-anxious individuals. This activation is attenuated when participants are engaged in a concurrent task, revealing a top-down control bypassing the initial capture by fearful faces. Moreover, when participants are engaged in a task which increases working memory load, the attentional interference from distractors (singletons) is more likely to influence reaction times in a search task [16], suggesting that the capture by task-irrelevant singletons is more important when working memory load is high. Pecchinenda and Petrucci [17] showed that high cognitive load (counting backward by 7) enhances the eye-gaze cueing effect following angry faces. This result was mainly interpreted as reduced interference from negative emotions by cognitive control processes. To our knowledge, these effects have never been investigated with emotional faces in anxious participants using EEG. Moreover, all of the studies mentioned above relied on dual tasks where the working memory and gaze cueing tasks were always presented as two distinct processes, making it unclear if working memory really has an impact on gaze perception. To resolve this ambiguity in the current study, the working memory task was embedded in the cueing task in order to directly test the effect of working memory load on gaze processing using different emotions.
The first aim of this study is to investigate the neural mechanisms of the perception of eye gaze as a function of emotion (fearful versus neutral), using neural markers of selective attention. The second aim of this study is to test if high working memory load prevents the attentional orientation by fearful averted gaze in anxiety. If working memory load inhibits the attentional orientation by fearful faces, these neural activations should be diminished in the high working memory load condition. More specifically, this study tested two aspects of ACT [4]: first, we investigated whether anxiety enhances the influence of stimulus driven processes at the expense of goal directed processes. According to this hypothesis, threat-related task irrelevant stimuli would impair processing of a concurrent working memory task in high anxious individuals. Second, this study will also investigate whether high anxious individuals are more influenced by threat related distractors than low anxious individuals.

Participants
Forty participants from the University of Oxford and the local community were preselected based on their scores on the trait scale of the State-Trait Anxiety Inventory [STAI; 18] which was completed online before the experiment. We selected twenty participants scoring high (scores of 45 or above) and twenty participants scoring low (scores of 35 and below) on the questionnaire [19,20]. After the artefact rejection procedure and the suppression of incorrect trials, 1 participant was excluded due to his high percentage of incorrect trials (>50 %) and 3 participants were excluded from the analyses due to a high percentage of trial loss during the artefact rejection procedure (>30 %). The final sample consisted of 36 participants. The high trait anxiety group consisted of 10 females and 9 males with a mean age of 25.1 years (SD = 4.41) and had a mean of 49.42 (SD = 6.04) on the STAI-T. The low trait anxiety group consisted of 8 females and 9 males with a mean age of 23.2 years (SD = 3.19) and had a mean of 30.41 (SD = 4.73) on the STAI-T. The internal consistency of the STAI-T questionnaire is very good (Cronbach's alpha = 0.94). Participants had a normal or a corrected-to-normal vision. They were paid £10 per hour for their participation.

Stimuli and experimental procedure
At the beginning of the task, a fixation cross was presented centrally (500 ms). Then, each trial started with the working memory task consisting of a set of 5 digits (0-4) that participants had to remember. The digits were presented either in order (01234; low load) or in a random order with a sequence of numbers always starting with a 0 (e.g. 03142; high load) [see 16] and stayed on the screen for 1500 ms. The task of the participants was to retain the sequence of numbers. Then, the cueing task started with a face (fearful or neutral) presented centrally with the gaze straight ahead (900 ms) followed by a face with the gaze averted (500 mscorresponding to the cue). A blank screen of 30 ms separated the two faces, which made the model's saccade look more natural. The two faces were followed by the retrieval of the working memory task. A memory probe (the target) appeared either on the right or on the left side of the screen. An Asterix (*) was displayed at the opposite site in each trial. The position of the probe was congruent (50 %) or incongruent (50 %) according to the gaze. The probe remained on the screen until a response was made or until 3 s had elapsed. The memory probe consisted of one digit that was equally likely to be 0, 1, 2, or 3 in the low working memory load condition or 0, 1, 2, 3, or 4 in the high working memory load condition. Participants had to key in the digit that followed the memory probe digit in the memory set of that trial. All the positions in the memory set were equally likely to be probed.
The experiment was divided in eight blocks of 92 trials, for a total of 736 trials.
The face stimuli were photographs of five males and five females expressing fearful (50 %) and neutral (50 %) emotions [21] with gaze directed ahead, to the right or to the left. The borders of the faces were removed, in order to keep only features of the faces and avoid the influence of other features such as the hair or ears. The final stimuli were oval-shaped faces that measured 9.5 cm horizontally and 14 cm vertically. All the stimuli were created using Adobe Photoshop which allowed us to control for isoluminance across all categories.
The study was approved by the Central University Research Ethics Committee at the University of Oxford (CUREC Approval Reference: R57270/RE001) and all participants signed an informed consent form before participating. The study was performed in agreement with the Declaration of Helsinki.

EEG acquisition and processing
EEG was recorded using a 64-channels Neuroscan system (El Paso, TX) with AG/AgCl electrodes positioned according to the extended 10-20 system. Four additional electrodes were placed on the outer canthi of the eyes and above and under the left eye, in order to capture the eye movements and blinks. Each active electrode is represented with an impedance value, which we tried to keep below 10 kΩ for each participant. The EEG was continuously recorded with a sampling rate of 500 Hz. Electrodes were re-referenced offline to the average reference.
Standard processing of EEG data was done offline using the software Brain Vision Analyzer V.2 (Brain Products, Gilching, Germany) and custom Matlab scripts built on the open source EEGLAB toolbox [22; http://sccn.ucsd.edu/eeglab]. Two distinct analyses were performed for the cue-locked (the face with the averted gaze) and the target-locked (the memory probe) ERP's. For both ERP's, epochs were selected 200 ms prior and 500 ms after stimulus onset. A spherical spline interpolation was applied to all the bad electrodes (4.68 % of the electrodes were interpolated). Baseline correction was applied on the 200 ms prestimulus period of cue and target presentation respectively. ERPs were obtained by averaging the trials for each condition, on the data filtered with a low-cutoff at 0.1 Hz and a high-cutoff at 30 Hz. Ocular correction was performed on the EEG using the implemented standard algorithm [23]. Trials with artifacts were automatically removed (amplitude allowed on all the electrodes: ±100 μV). Further, trials with blinks were removed (VEOG ± 60 μV), as well as trials with horizontal eye movements and saccades (HEOG ± 40 μV). On average, 13.5 % of the trials were removed.

Behaviourreaction time analysis
Median reaction times were analysed using a mixed ANOVA with Load (2: low/high) x Emotion (2: fear/neutral) x Congruency (2: congruent/incongruent) as within subjects factors and Group (2: high anxious/low anxious) as between subjects factor. This design allows us to test ACT predictions: we expect high anxious participants to have impaired performance following fearful faces, which should appear in any Group x Emotion (x Other factor) interaction. Moreover, if working memory disrupts the processing of the gaze, this cueing effect should be disrupted in the high working memory load condition. Incorrect trials were excluded from the analyses and trials with RTs shorter than 500 ms and longer than 2000 ms were removed.

Event-related potentials -target-locked ERPs
The time windows of investigation were determined using a collapsed localizer, based on the grand averages across all conditions [24]. The regions of interest were defined a priori based on previous studies [24] and were then aligned to the conspicuous topographic maps of the analysed data set for electrode selection.
Peak amplitudes and latencies were analysed on P1 and N1 components relative to target onset. The P1 is a positive going component with an onset at 60− 90 ms and a peak at 100− 130 ms post-stimulus. It is largest over lateral occipital electrode sites [25]. P1 was extracted at electrode PO7/8 between 90 and 200 ms [26]. The N1 is a negative going wave peaking at 150− 200 ms post-stimlus and is largest over lateral occipital electrode sites [25,26]. N1 was extracted at electrode P5/6 between 140 and 280 ms following target onset. Amplitudes and latencies were analysed using a mixed ANOVA with Hemifield (2: contralateral/ipsilateral) x Load (2: low/high) x Emotion (2: fear/neutral) x Congruency (2: congruent/incongruent) as within subjects factors and Group (2: high anxious/low anxious) as between subjects factor.

Event-related potentialscue-locked ERPs
The EDAN is a negativity occurring at occipito-parietal electrode sites around 200− 400 ms after cue onset [26,27]. Based on grand average topographies over the occipito-parietal region of interest, mean amplitude was computed on electrodes P5/P6 between 190 and 330 ms after the cue onset. The contralateralipsilateral difference was submitted to a mixed ANOVA with Load (2: low/high) x Emotion (2: fear/neutral) as within subjects factors and Group (2: high anxious/low anxious) as between subjects factor. An effect of hemifield would suggest a lateralized shift in the direction of attention. The ADAN is an anterior negativity maximal over anterior sites and occurs around 400 ms after cue onset [28]. Based on grand average topographies over the anterior region of interest mean amplitude was computed on electrodes FC3/FC4 between 340 and 480 ms after the cue onset. The contralateral ipsilateral difference was submitted to a mixed ANOVA with Load (2: low/high) x Emotion (2: fear/neutral) as within subjects factors and Group (2: high anxious/low anxious) as between subjects factor. For significant interactions only, the effects were investigated using post-hoc HSD Tukey tests.

P1
For the amplitude of the P1 component, the hemifield x group interaction was significant (F(1, 34) = 4.85, p = .03, η 2 p = .12); post-hoc HSD Tukey tests showed that in the low anxious group, ipsilateral targets lead to higher amplitude than contralateral targets (p = .02). The emotion x congruency x group interaction was also significant (F(1, 34) = 7.35, p = .01, η 2 p = .17). Post-hoc HSD Tukey tests highlighted two significant differences in the high anxious group: the P1 amplitude of fearful trials was larger for congruent than incongruent trials (p < .01) (see Fig. 1A) and the P1 amplitude of incongruent trials was larger for neutral than for fearful faces (p = .02) (see Fig. 1B). These differences were not significant in the low anxious group (p s >.05).
For the latency of the P1 component, the effect of hemifield was significant (F(1, 34) = 8.4, p < .01, η 2 p = .19) with contralateral targets leading to shorter latencies than ipsilateral targets. The hemifield x load x congruency x group interaction was also significant (F(1, 34) = 5.96, p = .02, η 2 p = .15). Post-hoc HSD Tukey tests highlighted the following significant differences in the low anxious group: in congruent and contralateral trials, the P1 appears earlier for high than low load (p = .03); in high load and contralateral trials, the P1 appears earlier for congruent than incongruent trials (p = .02); in high load and congruent trials, the P1 appears earlier for contralateral than ipsilateral trials (p = .02); in low load and incongruent trials, the P1 appears earlier for contralateral than ipsilateral trials (p = .02).

N1
For the amplitude of the N1 component, the effect of load was significant (F(1, 34) = 8.81, p < .01, η 2 p = .21) with low load leading to higher amplitude than high load. The effect of hemifield was also significant (F(1, 34) = 73.51, p < .01, η 2 p = .68) with contralateral trials leading to higher amplitude than ipsilateral trials. The emotion x congruency x group interaction was also significant (F(1, 34) = 5.29, p = .02, η 2 p = .14). Post-hoc HSD Tukey tests highlighted one significant difference in the high anxious group: the N1 amplitude following incongruent trials was larger for fearful than neutral faces (p = .02) (see Fig. 1C).

EDAN
For the EDAN component, the load x group interaction was significant (F(1, 34) = 4.17, p = .04, η 2 p = .11). Post-hoc HSD Tukey tests showed that in the low anxious group, low load leads to more negative amplitude than high load (p = .03) (see Fig. 1D).

ADAN
For the ADAN component, the load x group interaction was significant (F(1, 34) = 8.23, p < .01, η 2 p = .19). Post-hoc HSD Tukey tests showed that in the low anxious group, the ADAN is more important for low than high load load trials (p = .03).

Discussion
The aim of the present study was to test whether working memory load has an influence on the gaze cueing effect in anxiety using emotional faces. Behaviourally, we found that low load trials lead to overall faster reaction times than high load trials for both groups. This effect shows that the manipulation of working memory load was successful, allowing us to distinguish between high and low load conditions. The valence of the facial stimuli did not affect this result in either of the anxiety groups.
At the electrophysiological level, we found that ERPs differed as a function of group. Effects of emotion (fearful versus neutral facial expressions) were prominent in the high anxious group while effects of working memory load were specific to the low anxious group. In high anxious individuals, we first observed greater amplitude for congruent than incongruent trials in the fearful condition on the P1 component locked to the target. This reflects amplification of the sensory input [31] at the location cued by fearful faces, and is consistent with previous studies showing an attentional bias towards threat in anxious participants [32,33]. In selective attention paradigms, higher P1 is expected at attended locations, therefore suggesting that fearful faces lead to enhanced attention orientation in this case. This might reflect hypervigilance to the locations cued by fearful faces [34] and is consistent with the prediction made by ACT that the balance between bottom up and top down processes is disrupted in anxiety. Indeed, in our paradigm, even though faces were orienting attention to a certain location in space, the emotional expression of the face was not relevant to the task. Effects of facial expression could therefore reflect effects on the stimulus driven attentional system in anxiety, where threat related stimuli override the goal directed processes dictated by the working memory task. Since our paradigm involves emotional faces acting as cues for target location, we are not directly testing attentional biases towards threat. Instead, threatening stimuli with averted gaze might trigger higher vigilance to the cued location in high anxious individuals, and there is evidence for this [19]. Moreover, in incongruent trials, neutral faces lead to enhanced amplitude as compared to fearful faces on this same component. If the P1 component is a signature of sensory gain across cued locations, then this result might sound contradictory. However, it might well reflect difficulty in disengaging from locations cued by fearful stimuli in high anxious participants. For instance, higher amplitude at incongruent locations for neutral faces suggests greater attention re-orientation to that location when neutral cues are compared to fearful cues, therefore indicating a higher difficulty to disengage from fearful as compared to neutral faces. On the other hand, in this group, incongruent trials lead to greater amplitude for fearful than neutral faces on the N1 component. P1 and N1 components have often been shown larger at attended locations and have first been mainly interpreted as reflecting sensory gain amplification at attended locations [31,35]. While the P1 component has consistently been shown to respond to cued locations in selective attention paradigms, the N1 has been more controversial as reversed patterns between P1 and N1 components have been observed [36,37]. The N1 now seems to reflect discriminative processes at attended locations. Here, we observed larger N1 amplitude at unattended locations for fearful faces, which seems to relate to target discrimination in this case. Finding an N1 relative to uncued locations by fearful faces suggests that the task was more demanding following fearful faces in anxious participants, irrespective of working memory load. In other words, once attention is allocated at a certain location in space, discrimination requires more effort when the target follows a fearful face. In summary, electrophysiological results in high anxious participants suggest an initial hypervigilance to locations cued by fearful faces, as well as a difficulty to disengage from these locations when targets appear at uncued sites. At a slightly later stage of processing, uncued locations are characterised by higher processing following fearful faces, when discriminative processes leading to response selection are implemented. Our results suggest that these effects are not influenced by working memory load.
In the low anxious group, effects of load were prominent not only on target locked ERPs, but also on lateralized cue-locked components. Low load resulted in a higher EDAN component than high load and contralateral trials lead to a greater ADAN component when working memory load was low in this group. These results suggest that in the low load condition, markers of preparatory attention are more pronounced, irrespective of emotional expression. EDAN reflects the encoding of taskrelevant aspects of a cue [38] while ADAN is related to the preparation of directing attention through attentional control orchestrated by frontal regions [39,40]. These results on the cue-locked ERPs suggest that the gaze affects the different aspects of attention orientation only when working memory load is low. This is in line with some of the effects observed on the P1 component relative to target onset. In the low load condition and in incongruent trials, contralateral trials lead to shorter P1 amplitude than ipsilateral trials. In incongruent trials, participants must reorient their attention to uncued locations, a process that seems more efficient when working memory load is low, where the shifting of attention seems to be facilitated. This effect does not depend on emotion in the low anxious group.
Interestingly, congruent trials lead to shorter amplitude than incongruent trials in contralateral presentations and high load, showing that when working memory load increases, congruency effects act differently. Effects at uncued target locations appear in low load trials only, while cued target locations are characterised by shorter latencies in high load trials only. We therefore show that the gaze cueing effect is, in this case, under the influence of top-down control, as working memory load impacts the electrophysiological correlates of attention shifting. This interpretation has been suggested by some authors [29,41] questioning the bottom-up generation of gaze processing [42,43]. These effects on the P1 component have usually been observed on its amplitude [31], reflecting sensory gain among congruent trials in attention cueing tasks. The effect we observed in low anxious participants were prominent on latencies, which might reflect earlier processing of incongruent trials when working memory load is low and earlier processing of congruent trials when working memory load is high. This does not reflect sensory gain control per se. In summary, in the low anxious group, all the effects of working memory load were independent of emotion. We showed that cue-locked ERPs were more pronounced when working memory load was low, suggesting that the shifting of attention directed by the gaze is only visible when enough resources are available in the working memory span. Moreover, effects on target-locked ERPs show that working memory load impacts differently the processing of gaze, suggesting that top-down influence may play a role in this case.
It is worth mentioning that we analysed the components related to attention orientation relative to target and cue onsets. We were interested in early brain responses to emotional cues and their modulations by working memory load. However, we did not test the effect of our manipulation on the LDAP (posterior positivity) component, which is a later cue-locked component related to attentional shifting [31]. Other components such as the P3 relative to target onset might also shed light on these effects, but these later components were not analysed due to the overlap with participant's responses on each trial.
In our study, several interactions with working memory load have been restricted to the low anxious group while interactions with emotional expression of the cue were observed in the high anxious participants. It has been shown that gaze in emotional faces might be perceived differently in high relative to low trait-anxious participants [19,44]. Our findings support this interpretation and highlight the importance of including measures of inter-individual differences when investigating the perception of gaze and its different processing stages.