Emotional Facial Processing: Does Cognitive Load Make a Diﬀerence?

The manuscript is titled ‘Emotional facial processing: does cognitive load make a diﬀerence?’ and it describes a research study that measures how emotion and distraction of diﬀerent cognitive loads may impact working memory performance. The ﬁndings show that cognitive load on working memory performance, with poorer working memory performance in the high compared to the low level of distraction. However, no eﬀects of emotional faces were found on task performance. The work therefore has signiﬁcance with regard to cognitive processing and working memory span.


Introduction
Processing human faces spontaneously and accurately is a vital skill for humans (Chen, 2014) because faces are considered as the most socially and emotionally significant visual stimuli in the environment (Gupta, Hur, & Lavie, 2016).They can convey different social information with others to express happiness, sadness, anger, fear, surprise, and disgust (Keltner, Ekman, Gonzaga, & Beer, 2003).Being competent in this skill facilitates individuals to interact appropriately with others and the surrounding environment.However, the environment is often filled with other forms of distractions which are difficult to ignore.Nonetheless, human faces are processed differently and show processing advantage compared to other non-social stimuli such as vehicles and buildings.For example, only a short duration of eye movements (also refer to as saccades) is required for processing facial stimuli (around 100 ms) compared to an average of 140 ms for processing non-face stimuli such as vehicle (Crouzet, Kirchner, & Thorpe, 2010).Besides, individuals only need as little as 360 ms to discriminate between unfamiliar and familiar faces (Barragan-Jason, Besson, Ceccaldi, & Barbeau, 2013).This indicates the uniquenss of facial stimuli and the extent of faces in capturing attention and cognitive demands.
To compare processing speed of animal faces, human faces and non-facial stimuli, Crouzet, Kirchner, and Thorpe (2010) have conducted experiments using the Saccadic Choice Task with the predictions that human faces are quicker and more salient in capturing attention than the other stimuli.The results of the first study showed that rapid saccadic responses were initiated when processing human faces (around 110 ms) following the onset of the stimuli.A followed-up experiment from this showed that even when participants were instructed to saccade toward other non-facial stimuli like vehicles, they still showed faster saccades directed toward faces.Both findings suggest that attentional bias towards facial processing is relatively difficult to suppress which underlie the processing advantage and unique characteristics of human faces from other stimuli.
According to the mood congruency hypothesis (Bower, 1981), a theory that explains how emotional information is more easily retrieved when it has the same emotional content as the current emotional state of the individual (Bower, 1981).Consistent with this hypothesis, studies reported that positive emotional state was repeatedly found to facilitate processing of happy faces relative to neutral faces (D'Argembeau, Van der Linden, Comblain, & Etienne, 2003).Aside from the impact of current emotional state, the intensity of emotional facial expressions could also enhance performance on facial recall (Bate, Parris, Haslam, & Kay, 2010).This is apparent in findings (e.g., Craig, Becker, & Lipp, 2014) supporting happiness superiority effect, wherein positive but not negative emotions of the stimuli are generally processed more quickly than neutral ones.It is explained that attentional bias towards emotional stimuli (especially threatening and aversive ones) can be useful for survival reason, and is regarded as a result from adaptation to environmental danger ( Öhman, 2002).
Although there is general attentional bias towards emotional faces relative to other non-facial stimuli due to adaptation advantage and survival ( Öhman, 2002), there is limited capacity of attentional resources which means that when two or more incoming stimuli need to be processed, priority must be assigned to one or the other at a given time.Working memory can be conceptualized as the interface between internal executive control and external attentional control.It is also used to actively manipulate visual stimuli in the environment and to direct attention to the goal-relevant stimuli (Chun et al., 2011).An example would be to try and do a mathematical calculation while processing emotional facial stimuli.Theories of emotional interference and selective attention (Lavie, 2005;Pessoa, 2009) suggest that the amount of attentional resources allocated to certain stimuli are determined by the trade-off between bottom-up (or stimuli-driven) influences such as emotional salience of the faces and top-down (or goal-directed) influences like doing a mathematical calculation.Research has shown that attentional control can help to select stimuli in the visual environment via top-down or bottom-up mechanism (Hu, Xu, & Hitch, 2011).Working memory plays an important role in this process by actively retaining the information while information is processed (Chun, Golomb, & Turk-Browne, 2011;Vogel, Woodman, & Luck, 2005).
Evidence for the tradeoff between top-down and bottom-up processing were evident by an increased interference for emotional stimuli in cognitive tasks (as reflected by lower accuracy and slower response time).For example, in an emotional Stroop task, the emotional faces were the bottom-up processing whereas the color naming was the top-down processing.Emotional facial processing involves automatic (bottom-up) processing and colour naming involves controlled (top-down) processing.It was found that participants responded slower to colour naming of emotional words compared to neutral ones due to the emotional interference of facial stimuli with the task demand of colour naming (Dresler, Mériau, Heekeren, & Van der Meer, 2009).Similarly, in an n-back task that measures working memory, emotions of the stimuli was found to impede task performance wherein participants responded slower for emotional but not neutral stimuli (Bowling, 2015).It was found that emotional faces yielded lower accuracy and there was longer response time for emotional words when compared with their neutral stimuli.
Although facial expressions appear to be automatically processed and showed processing advantage compared to other non-facial stimuli, emerging evidence by Lavie (2005) suggest that the processing is vastly dependent on the cognitive load of the competing task.He used varying levels of cognitive load (by using short and long number of letter strings in the task) as interfering distractors for a visual attentional task.The topdown goal of the task was match the object or face with the name presented on the screen.Results showed that the higher cognitive load impaired WM ability more than low cognitive load ones.This is possibly due to the difficulty in trying to actively retain prioritized processing for stimulus-driven (bottom-up) goals.If both the goal-relevant tasks require bottom-up processing, participants would direct cognitive resources toward the more salient stimuli with less distraction.That is, bottom-up stimuli are largely dependent on the cognitive load of the distractors, which again reflect the limited capacity of the WM span.Followed from this, O'toole, DeCicco, Hong, and Dennis (2011) examined whether task difficulty might influence attention performance of emotional stimuli.They have used attention task that measures three aspects of attention performance: alerting, orienting, and executive attention.Results showed that emotional faces consistently facilitate to direct attentional resources in the easy task but not the difficult task.These findings suggested that emotions may exert effect on attentional control and working memory (WM) performance but the complexity of the task (i.e., difficult versus easy) must be accounted for in the investigation.
More recently, Allen et al. (2017) examined the interaction of attentional control and WM.In a series of sever experiment using distractions of different shapes, colors, positions timing etc, and examined how WM was influenced by these factors.Their results suggested that the type of distraction is independent to the WM performance.However, Allen et al. (2017) concerned whether the results can be replicated to other forms of cognitive tasks such as complex span paradigm, a dual task that requires memorizing a list of items (e.g., words) while performing other task such as verifying a mathematical equation.This combination of short-term storage and processing requirements implements the basic definition of WM as simultaneous storage and processing (Baddeley, 2012).
Based on the current scope of literature, it is evident that the assessments of WM capacity varies greatly with the difference in contents and measuring methods (Kane et al., 2004).Assessment of WM began with the earliest measures using reading span task (Daneman & Carpenter, 1980) towards the later development of a more complex-span paradigm such as automated version of operation span (Asopan; Unsworth, Heitz, Schrock, & Engle, 2005).The complex-span paradigm measures WM span requires participants to pay attention and retrieve information which is not currently available in the environment.For example,i n the current study, WM is not only used to actively manipulate visual stimuli but also intentionally engage in solving distracting tasks.In this way, there are two types of goal-relevant stimuli: emotional faces and resolving math questions in which the attention should be directed to.
According to the Time-based Resource-sharing model (TBRS; Barrouillet et al., 2004), it is proposed that individual's attention can quickly switch back and forth from the processing to storage WM component to prevent loss of memory trace.There is the assumption that the cognitive load corresponds to the proportion of time required for the processing but predicted that the storage of WM component is independent of the processing component.Although previous research has examined the how emotional face processing or cognitive load may separately impact WM, it is unclear whether emotional faces can possess significant interference on WM task when the complexity of the task is manipulated.
The primary hypothesis of the current study was to examine whether distracting tasks at varying levels of cognitive load may influence the WM performance of emotional facial processing.Manipulation check of distracting tasks was conducted.The second hypothesis of this research was to examine the effect of emotional faces on WM performance.It was predicted that emotional faces would result in better WM performance.Therefore, the current study investigated how emotional faces and distractions may interact to affect WM performance.

Method Participants
A total of 28 participants were recruited by convenience sampling.Students studied overseas degree programmes at the University of Salford (14 male and 14 female; Mean = 23.39,SD = 4.06) were recruited to the experiment.The participants were required to have normal or corrected-to normal vision.The experiment took place at the computer laboratory.All of the participants received no monetary compensatory for their participation.Each participant was individually tested in the laboratory lasting approximately 35 minutes.

Design
The current experiment employed repeated measures design.The experiment design was 3 (Level of distraction: control, low, or high) × 3 (Emotional face: neutral, happy, or sad).The dependent variable was WM performance, partial span score for the serial recall of emotional faces, on the modified Aospan.The partial span score is the WM span for serial recall of emotional faces.For example, if an individual recalled correctly 2 emotional faces in a set size of 3, 6 emotional faces in a set size of 6, the partial span scores would be 2 and 6 respectively.The partial span score has more variance and thus allows for better discrimination between high and low ability participants (Unsworth et al., 2005).

Materials and stimuli
Apparatus.E-prime version 2.0.10.242software (Psychology Software Tools, Pittsburgh, PA) was used.The computer model was Dell Inspiron 22-3264 with Window 10 operation system installed.The display resolution of screen was 1024 × 768.Emotional facial stimuli are presented at a distance of approximate 45 cm with a resolution of 210 × 240 for each emotional face.
Cognitive load.The distraction involved mathematical equations with addition and subtraction as cognitive load.There were 36 trials of mathematical equations in each level of distraction, 108 trials in total.The varying levels of complexity of mathematical equations implicated that the amount of cognitive load was actively maintained in WM.Three levels of distraction were resulted: control, low, and high.Cognitive load was operationalized as a function of distraction in which a given event captures WM resources thus hindering other central activities.The low level of distraction indicated lower cognitive load; the high level of distraction indicated higher cognitive load.
To experimentally varied the three levels of distraction, the operand in each equation increased (e.g., 1+9=8 or 9-7=6 in the control level; 19+1=20 in the low level; 10+2-1=5 in the high level).The processing component was performed progressively, namely the control level of distraction was performed firstly, and then the low level following the high level.
As the relatively increasing complexity of mathematical equations in the subsequent levels, their average time used in the processing component in the control level (plus 3 SD ) was then used for the subsequent processing component for the corresponding individual.
The emotional faces.All emotional facial stimuli were obtained from Yale face database (Georghiades, 1997).Facial stimuli of 15 individuals included positive, negative, and neutral faces.All facial stimuli are grey-scaled images.Originally, participants were required to solve mathematical equations while remembering letters in Aospan.Major modification that changed to-be-remember letters to naturally emotional faces was made to Aospan.A block design was used, with each block contained 36 trials of mathematical equations and 36 trials of emotional faces.Each level of distraction consisted of 3 blocks for each type of emotional faces.Except for the control level, it contained 4 blocks with one of those blocks used for 36 mathematical equations to calculate the time limit for differences in individual ability.The accuracy criterion was 85% which prevented participants from answering these by chance.Three of the remaining blocks in the control level comprised 12 trials of each type of emotional faces.

Procedure.
The participants were individually tested in the computer laboratory and informed consent was obtained from all individual participants included in the study.First, they were instructed to enter their information into the start-up information of E-prime, including their age and sex.Then, they were instructed to perform the computerized experiment -the modified Aospan.They were required to complete the practice trials before each condition of the distracting task, and then complete the real trials.
Each practice session contained 3 trials presented in math-emotional face strings.The presentation of processing component (distraction) and storage component (serial recall of emotional faces) in the control level was serial, the practice session in the control condition required them to solve 3 trials of mathematical equations and then 3 trials of emotional faces with a set size of 3 for recalling in the same presented order.In both the practice and real trial, all the emotional faces were presented on the screen for 800 ms.At the recall, the participant saw a 3 × 2 matrix of emotional faces.
In the control level, they first solved 36 real trials of mathematical equations and then performed 12 real trials for each type of emotional faces at the set size of 3.For each of the subsequent trials of the low and high level of distraction, the mathematical equations were paralleled with the emotional faces.In the low level, the participants performed 3 strings; thus, the set size for recalling the emotional faces was 3. In the high level, they performed 6 strings in which the set size for recalling the emotional faces was doubled to 6 as to increase the difficulty.The sequence of the levels of distraction was in fixed order.The order for types of emotional faces was counterbalance between participants and emotional faces for recalling were pseudorandomized.All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Hosted file
image1.emf available at https://authorea.com/users/720551/articles/705345-emotional-facialprocessing-does-cognitive-load-make-a-differenceAll participants were included in the statistical analysis.The mean (partial span score) and standard deviation for percentage of partial span scores were shown in table 1.
The effect of cognitive load on WM performance.
The data were analyzed by a 3 × 3 repeated measures analysis of variance (ANOVA) to explore the effects of distraction (control, low, high) and emotional faces (neutral, happy, sad) on WM span.
The results indicated that the WM performance was significantly affected by the cognitive load, F (1.64, 44.38) = 58.54,p< .001.Bonferroni post hoc tests revealed that the WM performance was significantly poorer for the high level of distraction (M = 57.9,SD = 4.3) compared to the control level of distraction (M = 85.6,SD = 2.25, p< .001),but not for the control level with the low level of distraction (M = 80.6, SD = 3.13, p = .055,p> .05).

Discussion
The present study examined the effects of cognitive load and emotional faces on WM performance using modified Aospan.The primary hypothesis of the current study was to examine how the distractors with different levels of cognitive load may affect the WM performance of emotional facial processing.The second objective was to examine the effect of emotional faces on WM performance.Overall the results confirmed proper manipulation of different distraction tasks as participants performed better in the low level of distraction than the high level of distraction.This also confirms the design that cognitive load was lower in the low than the high level of distraction and participants consumed less WM resources when performing the mathematical equations with fewer operands than larger operands.Utilizing varying levels of mathematical equations is effective in manipulating different amount of cognitive load, and these different levels of cognitive load affect performance of WM span.
Regarding the impact of cognitive load on WM performance, it was found that when participants were presented with distractor of low cognitive load, WM performance was not affected.However, in exposure to distractor of high cognitive load WM was significantly impeded.The cognitive load of distracting tasks in the modified Aospan are divided into three levels, with the difference in performance of WM span between difficult and control condition being the most prominent.In other words, the difficult distracting task taxes the highest cognitive resources thereby diminishing the WM span performance to the greatest degree.This result is consistent with the previous studies (Allen et al., 2017;Lavie, 2005;O'toole et al., 2011) contradictory that increasing the cognitive load of the distractor has a vast impact in WM performance.When there are two or more incoming stimuli that need to be processed, priority must be assigned to one or the other at a given time.To allocate cognitive resources appropriately between the two competing resources, the WM model (Baddeley & Hitch, 1974) explains that a critical component named the central executive plays a pivotal role in this moment (Baddeley, 2012).The central executive acts as a guide for the delegation of resources and attention allocation of resources to manage and maintain information in mind during the task.Under low cognitive load, this component is working adequately but when cognitive load increases, the WM is affected reflecting the limited capacity of attentional resources and WM span.
The present findings are also consistent with the assumption of the TBRS model which suggests that WM functions to temporarily store items and process information, and these function are fueled by limited attentional resources (Geday, Kupers, & Gjedde, 2007;Vergauwe, Barrouillet, & Camos, 2009).The TBRS model also states that the cognitive load of a given event corresponds to the proportion of time (Geday et al., 2007).In terms of the complex span paradigm, the TBRS model explains that the recall performance of storage component should be a function of the cognitive load of the processing component of the span task.This trade-off is due to the limited-capacity mechanism of WM suggested by Baddeley (2012).The present findings are in line with their suggestion in terms of the limited-capacity nature of WM.It is suggested that both the processing component (mathematical equations solving) and storage component (serial recall of emotional faces) require visuospatial WM, another component proposed by the WM model.This component provides a virtual environment for physical simulation, calculation, visualization and optical memory recall.It is a part of the WM that holds information during the initial processing to produce the recollection of an image such as a place or a face.Therefore, the current results revealed a trade-off between the two processes (emotional facial processing and calculation).This is consistent with the proposal of the limited-capacity mechanism of WM.As a result, the higher the cognitive load is, the lower the levels of performance are.
Regarding the effect of emotional faces on WM performance, the results showed no significant effect of emotional face on WM performance.Although previous studies such as Craig et al. (2014) lend support to the happiness superiority effect, wherein positive facial stimuli were found to be processed more quickly than negative and neutral ones in their visual search task.However, such effect was not apparent in the current study as the type of emotional stimuli being processed did not affect the WM performance.In an explanatory manner, participants were not influenced by the emotions of facial stimuli (bottom-up attention) when preforming the task.In fact, a study has investigated the strategies used in complex span paradigm and found using strategies such as imagery (Bailey, Dunlosky, & Kane, 2011).They found that participants used encoding strategies such as chunking of verbal letters.In this study, although the non-verbal emotional faces can prevent participants from rehearsing or chunking, they plausibly employ alternative methods to facilitate WM performance.For example, they have associated a name for each emotional face based on its features to help with their memory performance.
A key limitation of the current study is that participants were not asked to report the emotionality of the facial stimuli they were exposed to, it remains inconclusive as to whether emotions have no impact on WM or that the participants did not perceive the stimuli as emotionally arousing.Future studies should address this by asking participants to rate the emotionality of the stimuli in the experiment.As the study has only used three different types of emotional faces.The non-significant results might have been attributed to this.In support of this assumption, previous studies (e.g., Craig et al., 2014;O'toole et al., 2011) have used other emotional stimuli such as threat related stimuli.It was found that threatening or angry faces elicited a bigger effect (also known as Anger Superiority Effect) on capturing attention compared to general happy or sad faces.In future studies, more salient emotional facial stimuli are recommended.If the emotionality of faces is more prominent, participants may elicit stronger emotional responses toward them.As the current results are not consistent with the TBRS model (Barrouillet et al., 2004), the modified version of Aospan in the present study with varying cognitive load can be further verified by the TBRS model in the future to address the following questions.First, it has been found that affective state of individuals may either improve or diminish the WM performance, however, few studies (e.g.Yang, Yang, & Isen, 2013) have examined the effects of current mood state on WM using complex span paradigm.Second, the current modified Aospan used emotional facial stimuli as to-be-remembered items, future studies may employ emotional words, aversive or appetitive stimuli to investigate whether these non-verbal but meaningful stimuli have effects on WM performace because these stimuli are of evoluationarily importance.
In conclusion, the current study modified the Aospan in which changed the to-be-remembered items to emotional faces and varied the cognitive load by means of different levels of mathematical equations.In consideration of the non-verbal nature of emotional facial stimuli, the time limit plus 3 SD was applied for differences in ability of individuals.The effect of emotional faces on WM performance was non-significant.Instead, the results reveal a main effect of cognitive load on WM performance.These findings add evidence to the TBRS model.