Abstract
Behavioral and event-related potential (ERP) studies have shown that the expressions of a face displayed in the encoding phase (encoded facial expressions) influences identity recognition of this face in a later recognition phase. As facial expressions displayed in the recognition phase (recognized facial expressions) might also influence facial identity recognition, the current study investigated whether the effect of encoded facial expressions on facial identity recognition changed depending on recognized facial expressions. Therefore, participants were asked to learn facial identities displaying angry, happy or neutral expressions during the encoding phase. In the subsequent recognition phase, participants were presented with the learned identities and several novel identities and asked to judge whether the prompted identities had been learned. Each identity displayed angry, happy and neutral expressions. The results showed that d’ scores were smaller for happy-recognized identities (i.e., learned identities displaying happy expressions during the recognition phase) in the angry-encoded (i.e., learned identities that had displayed angry expressions during the preceding encoding phase) condition than in the neutral-encoded condition and for neutral-recognized identities in the angry-encoded condition than in the happy-encoded condition. ERP results showed that angry-encoded identities elicited less negative responses in the P200 and early posterior negativity (EPN)/N250 components and less positive responses in the late positive potential (LPP) component than happy-encoded identities. For neutral-recognized identities, LPP responses were reduced in the angry-encoded condition compared with those in the neutral-encoded condition. These findings might contribute to understanding the influence of previous and current facial expressions on facial identity recognition.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
All data generated or analyzed during this study are included in this published article.
Notes
Note that the P200 and EPN/N250 components often overlap with each other, and previous studies analyzed the relevant data by combining both components (Calvo & Beltrán, 2013; Lin & Liang, 2020). In addition, EPN and N250 components are often used in studies on emotion and memory, respectively. We refer to this component as “EPN/N250” in our study, as we cannot disentangle the two components in this paradigm.
References
Adolphs, R. (2003). Cognitive neuroscience of human social behaviour. Nature Reviews Neuroscience, 4, 165–178. https://doi.org/10.1038/nrn1056
Batty, M., & Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cognitive Brain Research, 17(3), 613–620. https://doi.org/10.1016/S0926-6410(03)00174-5
Bell, R., & Buchner, A. (2010). Valence modulates source memory for faces. Memory & Cognition, 38(1), 29–41. https://doi.org/10.3758/MC.38.1.29
Blau, V. C., Maurer, U., Tottenham, N., & McCandliss, B. D. (2007). The face-specific N170 component is modulated by emotional facial expression. Behavioral and Brain Functions, 3(1), 7. https://doi.org/10.1186/1744-9081-3-7
Bobes, M. A., Quiñonez, I., Perez, J., Leon, I., & Valdés-Sosa, M. (2007). Brain potentials reflect access to visual and emotional memories for faces. Biological Psychology, 75(2), 146–153. https://doi.org/10.1016/j.biopsycho.2007.01.006
Calder, A. J. (2011). Does facial identity and facial expression recognition involve separate visual routes? In A. J. Calder, G. Rhodes, M. H. Johnson, & J. V. Haxby (Eds.), The Oxford handbook of face perception (pp. 427–448). Oxford University Press.
Calder, A. J., & Young, A. W. (2005). Understanding the recognition of facial identity and facial expression. Nature Reviews Neuroscience, 6(8), 641–651. https://doi.org/10.1038/nrn1724
Calvo, M. G., & Beltrán, D. (2013). Recognition advantage of happy faces: Tracing the neurocognitive processes. Neuropsychologia, 51(11), 2051–2061. https://doi.org/10.1016/j.neuropsychologia.2013.07.010
Curran, T. (1999). The electrophysiology of incidental and intentional retrieval: Erp old/new effects in lexical decision and recognition memory. Neuropsychologia, 37(7), 771–785. https://doi.org/10.1016/S0028-3932(98)00133-X
Curran, T. (2004). Effects of attention and confidence on the hypothesized ERP correlates of recollection and familiarity. Neuropsychologia, 42(8), 1088–1106. https://doi.org/10.1016/j.neuropsychologia.2003.12.011
D’Argembeau, A., & Van der Linden, M. (2007). Facial expressions of emotion influence memory for facial identity in an automatic way. Emotion, 7(3), 507–515. https://doi.org/10.1037/1528-3542.7.3.507
D’Argembeau, A., & Van der Linden, M. (2011). Influence of facial expression on memory for facial identity: Effects of visual features or emotional meaning? Emotion, 11(1), 199–202. https://doi.org/10.1037/a0022592
Dillon, D. G., Cooper, J. J., Grent, T., Woldorff, M. G., & LaBar, K. S. (2006). Dissociation of event-related potentials indexing arousal and semantic cohesion during emotional word encoding. Brain and Cognition, 62(1), 43–57. https://doi.org/10.1016/j.bandc.2006.03.008
Eimer, M. (2000a). Effects of face inversion on the structural encoding and recognition of faces: Evidence from event-related brain potentials. Cognitive Brain Research, 10(1), 145–158. https://doi.org/10.1016/S0926-6410(00)00038-0
Eimer, M. (2000b). The face-specific N170 component reflects late stages in the structural encoding of faces. NeuroReport, 11(10), 2319–2324. https://doi.org/10.1097/00001756-200007140-00050
Eimer, M., & McCarthy, R. A. (1999). Prosopagnosia and structural encoding of faces: Evidence from event-related potentials. NeuroReport, 10(2), 255–259. https://doi.org/10.1097/00001756-199902050-00010
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavioral Research Methods, 39, 175–191. https://doi.org/10.3758/BF03193146
Fisher, K., Towler, J., & Eimer, M. (2016). Facial identity and facial expression are initially integrated at visual perceptual stages of face processing. Neuropsychologia, 80, 115–125. https://doi.org/10.1016/j.neuropsychologia.2015.11.011
Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., & Zhao, D. (2007). The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems, Man, and Cybernetics-Part a: Systems and Humans, 38(1), 149–161.
Gratton, G., Coles, M. G., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography Clinical Neurophysiology, 55(4), 468–484. https://doi.org/10.1016/0013-4694(83)90135-9
Hagemann, J., Straube, T., & Schulz, C. (2016). Too bad: Bias for angry faces in social anxiety interferes with identity processing. Neuropsychologia, 84, 136–149. https://doi.org/10.1016/j.neuropsychologia.2016.02.005
Hajcak, G., & Olvet, D. M. (2008). The persistence of attention to emotion: Brain potentials during and after picture presentation. Emotion, 8(2), 250–255. https://doi.org/10.1037/1528-3542.8.2.250
Herzmann, G., Schweinberger, S. R., Sommer, W., & Jentzsch, I. (2004). What’s special about personally familiar faces? A multimodal approach. Psychophysiology, 41(5), 688–701. https://doi.org/10.1111/j.1469-8986.2004.00196.x
Hinojosa, J. A., Mercado, F., & Carretié, L. (2015). N170 sensitivity to facial expression: A meta-analysis. Neuroscience & Biobehavioral Reviews, 55, 498–509. https://doi.org/10.1016/j.neubiorev.2015.06.002
Hsieh, L. T., Hung, D. L., Tzeng, O. J. L., Lee, J. R., & Cheng, S. K. (2009). An event-related potential investigation of the processing of Remember/Forget cues and item encoding in item-method directed forgetting. Brain Research, 1250, 190–201. https://doi.org/10.1016/j.brainres.2008.11.016
Hugenberg, K., Young, S. G., Bernstein, M. J., & Sacco, D. F. (2010). The categorization-individuation model: An integrative account of the other-race recognition deficit. Psychological Review, 117(4), 1168–1187. https://doi.org/10.1037/a0020463
Jackson, M. C., Linden, D. E., & Raymond, J. E. (2014). Angry expressions strengthen the encoding and maintenance of face identity representations in visual working memory. Cognition & Emotion, 28(2), 278–297. https://doi.org/10.1080/02699931.2013.816655
Johansson, M., Mecklinger, A., & Treese, A. C. (2004). Recognition memory for emotional and neutral faces: An event-related potential study. Journal of Cognitive Neuroscience, 16(10), 1840–1853. https://doi.org/10.1162/0898929042947883
Kaufmann, J. M., Schweinberger, S. R., & Burton, A. M. (2009). N250 ERP correlates of the acquisition of face representations across different images. Journal of Cognitive Neuroscience, 21(4), 625–641. https://doi.org/10.1162/jocn.2009.21080
Kaufmann, J. M., Schulz, C., & Schweinberger, S. R. (2013). High and low performers differ in the use of shape information for face recognition. Neuropsychologia, 51(7), 1310–1319. https://doi.org/10.1016/j.neuropsychologia.2013.03.015
Labuschagne, I., Croft, R. J., Phan, K. L., & Nathan, P. J. (2010). Augmenting serotonin neurotransmission with citalopram modulates emotional expression decoding but not structural encoding of moderate intensity sad facial emotional stimuli: An event-related potential (ERP) investigation. Journal of Psychopharmacology, 24(8), 1153–1164. https://doi.org/10.1177/0269881108097878
Lin, H., & Liang, J. (2019). Contextual effects of angry vocal expressions on the encoding and recognition of emotional faces: An event-related potential (ERP) study. Neuropsychologia, 132, 107147. https://doi.org/10.1016/j.neuropsychologia.2019.107147
Lin, H., & Liang, J. (2020). Negative expectations influence behavioral and ERP responses in the subsequent recognition of expectancy-incongruent neutral events. Psychophysiology, 57, e13492. https://doi.org/10.1111/psyp.13492
Lin, H., Schulz, C., & Straube, T. (2015). Fearful contextual expression impairs the encoding and recognition of target faces: An ERP study. Frontiers in Behavioral Neuroscience, 9, 237. https://doi.org/10.3389/fnbeh.2015.00237
Lin, H., Schulz, C., & Straube, T. (2016). Effects of expectation congruency on event-related potentials (ERPs) to facial expressions depend on cognitive load during the expectation phase. Biological Psychology, 120, 126–136. https://doi.org/10.1016/j.biopsycho.2016.09.006
Lin, H., Schulz, C., & Straube, T. (2017). Contextual effects of surprised expressions on the encoding and recognition of emotional target faces: An event-related potential (ERP) study. Biological Psychology, 129, 273–281. https://doi.org/10.1016/j.biopsycho.2017.09.011
Linden, S. C., Jackson, M. C., Subramanian, L., Healy, D., & Linden, D. E. (2011). Sad benefit in face working memory: An emotional bias of melancholic depression. Journal of Affective Disorders, 135(1–3), 251–257. https://doi.org/10.1016/j.jad.2011.08.002
Liu, C. H., Chen, W., & Ward, J. (2014). Remembering faces with emotional expressions. Frontiers in Psychology, 5, 1439. https://doi.org/10.3389/fpsyg.2014.0143
Luck, S. J. (2014). An Introduction to the Event-Related Potential Technique. MIT Press.
Marzi, T., & Viggiano, M. P. (2010). Deep and shallow encoding effects on face recognition: An ERP study. International Journal of Psychophysiology, 78(3), 239–250. https://doi.org/10.1016/j.ijpsycho.2010.08.005
Moser, J. S., Hajcak, G., Bukay, E., & Simons, R. F. (2006). Intentional modulation of emotional responding to unpleasant pictures: An ERP study. Psychophysiology, 43(3), 292–296. https://doi.org/10.1111/j.1469-8986.2006.00402.x
Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9(1), 97–113. https://doi.org/10.1016/0028-3932(71)90067-4
Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77(3), 247–265. https://doi.org/10.1016/j.biopsycho.2007.11.006
Pell, P. J., & Richards, A. (2013). Overlapping facial expression representations are identity-dependent. Vision Research, 79, 1–7. https://doi.org/10.1016/j.visres.2012.12.009
Pinabiaux, C., Bulteau, C., Fohlen, M., Dorfmüller, G., Chiron, C., Hertz-Pannier, L., & Jambaqué, I. (2013). Impaired emotional memory recognition after early temporal lobe epilepsy surgery: The fearful face exception? Cortex, 49(5), 1386–1393. https://doi.org/10.1016/j.cortex.2012.06.008
Proverbio, A. M., La Mastra, F., & Zani, A. (2016). How negative social bias affects memory for faces: An electrical neuroimaging study. PLoS ONE, 11(9), e0162671. https://doi.org/10.1371/journal.pone.0162671
Reber, P. J., Siwiec, R. M., Gitleman, D. R., Parrish, T. B., Mesulam, M. M., & Paller, K. A. (2002). Neural correlates of successful encoding identified using functional magnetic resonance imaging. Journal of Neuroscience, 22(21), 9541–9548. https://doi.org/10.1523/JNEUROSCI.22-21-09541.2002
Righi, S., Marzi, T., Toscani, M., Baldassi, S., Ottonello, S., & Viggiano, M. P. (2012). Fearful expressions enhance recognition memory: Electrophysiological evidence. Acta Psychologica, 139(1), 7–18. https://doi.org/10.1016/j.actpsy.2011.09.015
Rugg, M. D., & Curran, T. (2007). Event-related potentials and recognition memory. Trends in Cognitive Science, 11(6), 251–257. https://doi.org/10.1016/j.tics.2007.04.004
Rugg, M. D., Mark, R. E., Walla, P., Schloerscheidt, A. M., Birch, C. S., & Allan, K. (1998). Dissociation of the neural correlates of implicit and explicit memory. Nature, 392(6676), 595. https://doi.org/10.1038/33396
Schefter, M., Knorr, S., Kathmann, N., & Werheid, K. (2012). Age differences on ERP old/new effects for emotional and neutral faces. International Journal of Psychophysiology, 85(2), 257–269. https://doi.org/10.1016/j.ijpsycho.2011.11.011
Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2003). Attention and emotion: An ERP analysis of facilitated emotional stimulus processing. NeuroReport, 14(8), 1107–1110. https://doi.org/10.1097/00001756-200306110-00002
Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2004). The selective processing of briefly presented affective pictures: An ERP analysis. Psychophysiology, 41(3), 441–449. https://doi.org/10.1111/j.1469-8986.2004.00174.x
Schupp, H. T., Flaisch, T., Stockburger, J., & Junghöfer, M. (2006). Emotion and attention: Event-related brain potential studies. Progress in Brain Research, 156, 31–51. https://doi.org/10.1016/S0079-6123(06)56002-9
Schupp, H. T., Stockburger, J., Codispoti, M., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2007a). Selective visual attention to emotion. Journal of Neuroscience, 27(5), 1082–1089. https://doi.org/10.1523/JNEUROSCI.3223-06.2007
Schupp, H. T., Stockburger, J., Bublatzky, F., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2007b). Explicit attention interferes with selective emotion processing in human extrastriate cortex. Bmc Neuroscience, 8(1), 16. https://doi.org/10.1186/1471-2202-8-16
Schweinberger, S. R., Pickering, E. C., Jentzsch, I., Burton, A. M., & Kaufmann, J. M. (2002). Event-related brain potential evidence for a response of inferior temporal cortex to familiar face repetitions. Cognitive Brain Research, 14(3), 398–409. https://doi.org/10.1016/S0926-6410(02)00142-8
Sergerie, K., Lepage, M., & Armony, J. L. (2005). A face to remember: Emotional expression modulates prefrontal activity during memory formation. NeuroImage, 24(2), 580–585. https://doi.org/10.1016/j.neuroimage.2004.08.051
Sergerie, K., Lepage, M., & Armony, J. L. (2007). Influence of emotional expression on memory recognition bias: A functional magnetic resonance imaging study. Biological Psychiatry, 62(10), 1126–1133. https://doi.org/10.1016/j.biopsych.2006.12.024
Sessa, P., Luria, R., Gotler, A., Jolicœur, P., & Dell’Acqua, R. (2011). Interhemispheric ERP asymmetries over inferior parietal cortex reveal differential visual working memory maintenance for fearful versus neutral facial identities. Psychophysiology, 48(2), 187–197. https://doi.org/10.1111/j.1469-8986.2010.01046.x
Tanaka, J. W., Curran, T., Porterfield, A. L., & Collins, D. (2006). Activation of preexisting and acquired face representations: The N250 event-related potential as an index of face familiarity. Journal of Cognitive Neuroscience, 18(9), 1488–1497. https://doi.org/10.1162/jocn.2006.18.9.1488
Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J. (2001). Effects of attention and emotion on face processing in the human brain: An event-related fMRI study. Neuron, 30(3), 829–841. https://doi.org/10.1016/S0896-6273(01)00328-2
Wang, B. (2013). Effect of positive emotion on consolidation of memory for faces: The modulation of facial valence and facial gender. Memory, 21(6), 707–721. https://doi.org/10.1080/09658211.2012.753461
Weymar, M., Bradley, M. M., Hamm, A. O., & Lang, P. J. (2013). When fear forms memories: Threat of shock and brain potentials during encoding and recognition. Cortex, 49(3), 819–826. https://doi.org/10.1016/j.cortex.2012.02.012
Wilding, E. L. (2000). In what way does the parietal ERP old/new effect index recollection? International Journal of Psychophysiology, 35(1), 81–87. https://doi.org/10.1016/S0167-8760(99)00095-1
Wild-Wall, N., Dimigen, O., & Sommer, W. (2008). Interaction of facial expressions and familiarity: ERP evidence. Biological Psychology, 77(2), 138–149. https://doi.org/10.1016/j.biopsycho.2007.10.001
Wronka, E., & Walentowska, W. (2011). Attention modulates emotional expression processing. Psychophysiology, 48(8), 1047–1056. https://doi.org/10.1111/j.1469-8986.2011.01180.x
Zhang, D., Lin, Y., Liu, Y., Luo, Y., & Jiang, D. (2019). Memory encoding, retention and retrieval of disgusting and fearful faces. Acta Psychologica Sinica, 51(1), 36–47. https://doi.org/10.3724/SP.J.1041.2019.00036
Funding
This work was supported by the 13th Five-year Planned Project for Education Sciences in Guangdong Province, China (title: The roles of visual–auditory emotional priming on face memory in in young adults; No. 2021GXJK417), the Basic and Applied Basic Research Project of Guangzhou Basic Research Plan (202201011462), the 13th “Five-Year” Plan of Philosophy and Social Science of Guangdong Province (GD19YXL02, GD21YXL04) and Research Program for Humanities and Social Science granted by the Ministry of Education in China (21YJC190006).
Author information
Authors and Affiliations
Contributions
Conceptualization: HL; methodology: HL and JL; formal analysis and investigation: HL and JL; writing—original draft preparation: HL; writing—review and editing: HL and JL; funding acquisition: HL.
Corresponding author
Ethics declarations
Conflict of interest
There are no any conflicts of interest.
Ethics approval
The study was conducted in accordance with standard ethical guidelines as defined in the Declaration of Helsinki. The study was approved by the academic committee of School of Public Administration, Guangdong University of Finance.
Consent to participate
All participants gave their informed consent prior to their inclusion in the study.
Consent for publication
The manuscript or parts of it have not been and will not be submitted elsewhere for publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lin, H., Liang, J. Behavioral and ERP effects of encoded facial expressions on facial identity recognition depend on recognized facial expressions. Psychological Research 87, 1590–1606 (2023). https://doi.org/10.1007/s00426-022-01756-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-022-01756-x