Abstract
The magnitude of the happy categorisation advantage, the faster recognition of happiness than negative expressions, is influenced by facial race and sex cues. Previous studies have investigated these relationships using racial outgroups stereotypically associated with physical threat in predominantly Caucasian samples. To determine whether these influences generalise to stimuli representing other ethnic groups and to participants of different ethnicities, Caucasian Australian (Experiments 1 and 2) and Chinese participants (Experiment 2) categorised happy and angry expressions displayed on own-race male faces presented with emotional other-race male, own-race female, and other-race female faces in separate tasks. The influence of social category cues on the happy categorisation advantage was similar in the Australian and Chinese samples. In both samples, the happy categorisation advantage was present for own-race male faces when they were encountered with other-race male faces but reduced when own-race male faces were categorised along with female faces. The happy categorisation advantage was present for own-race and other-race female faces when they were encountered with own-race male faces in both samples. Results suggest similarity in the influence of social category cues on emotion categorisation.
Similar content being viewed by others
The face carries multiple sources of information that help us to navigate the complexities of the social world. Structural facial cues can indicate social category memberships like a person’s race, sex, or age and moment to moment changes in eye gaze and emotional expression can provide information about a person’s feelings and intentions. Given that these facial cues are simultaneously present on the face, a growing literature has begun to investigate how these cues interact (Freeman & Ambady, 2011; Quinn & Macrae, 2011 for reviews). Of relevance to the current investigation is the finding that facial cues indicating a person’s sex and race can influence perception of emotional expressions (Aguado, García-Gutierrez, & Serrano-Pedraza, 2009, Craig, Koch, & Lipp, 2017; Craig, Lipp, & Mallan, 2014; Craig, Mallan, & Lipp, 2012; Hugenberg, 2005; Hugenberg & Sczesny, 2006; Lipp, Karnadewi, Craig, & Cronin, 2015).
One method that has been used to investigate these interactions is looking at modulation of the happy categorisation advantage, the faster categorisation of happy than negative expressions (Billings, Harrison, & Alden; 1993; Leppänen & Hietanen, 2003). The happy categorisation advantage is observed when emotional expressions are displayed by female posers but is reduced in magnitude or not present when the expressions are displayed by males (Becker, Kenrick, Neuberg, Blackwell, & Smith, 2007; Lipp, Karnadewi, et al., 2015; Hugenberg & Sczesny, 2006). A similar influence of race cues on emotion perception has been found. Caucasian participants recognised happy expressions faster than angry expressions when the expresser was Caucasian but not when the expresser was African/African American (Lipp, Craig, & Dat, 2015; Hugenberg, 2005). This influence of race on emotion categorisation has been observed with Caucasian American participants (Hugenberg, 2005) and replicated with Caucasian Australian participants (Craig et al., 2012; Lipp, Craig, & Dat, 2015) and White Dutch participants categorising expressions on happy and angry White Dutch and Moroccan faces (Bijlstra, Holland, & Wigboldus, 2010).
A number of explanations have been offered to explain the moderating influence of social category cues on the happy categorisation advantage, including differences in facial morphology (Hess, Adams, Grammer, & Kleck, 2009; Sacco & Hugenberg, 2009) or stereotype-based accounts (Hess, Adams, & Kleck, 2004; Bijlstra et al., 2010). However, the evaluative context account (Hugenberg, 2005) seems to offer the most likely explanation for the influence observed. Under this account, faces are quickly categorised and evaluated based on their social group memberships. Previous research suggests that female and own-race faces are evaluated as more positive than male and other-race faces (Eagly, Mladinic, & Otto, 1991; Greenwald, McGhee, & Schwartz, 1998). The congruence between the social evaluation and the valence of the emotional expressions influences the speed of emotion categorisation, enhancing the magnitude of the happy categorisation advantage for (own-race) females relative to males (Hugenberg & Sczesny, 2006) and (male) own-race faces relative to other-race faces when encountered together (Hugenberg, 2005). Recently, it has been demonstrated that the influence of race and sex on emotion categorisation is context specific as a happy categorisation advantage was observed for a set of Caucasian male faces when they were encountered along with African/African American faces but not when categorised along with Caucasian or African/African American female faces even in the same participant sample (Lipp, Craig, & Dat, 2015). However, it is currently unclear whether the influence of race and sex on the happy categorisation advantage shown for Caucasian and African/African American faces in Caucasian participants extends to faces of another racial outgroup not stereotypically associated with aggression (like Chinese faces) or to a non-Caucasian sample.
The broader literature on race perception suggests that the influence of race cues on emotion perception may generalise to Asian and Caucasian faces and be evident in Asian and Caucasian participants. Both the other-race effect, the better recognition of own-race than other-race faces (Meissner & Brigham, 2001), and the other-race advantage, the faster categorisation of other-race than own-race faces by their race (Levin, 1996, 2000), have been observed in Caucasian participants with both Asian and African American faces representing the racial outgroup (Ge et al., 2009; Hugenberg, Miller, & Claypool, 2007) and in Chinese participants with Caucasian faces representing the racial outgroup (Ge et al., 2009). Caucasian participants also tend to implicitly evaluate Caucasian faces as more positive than both African American faces (Degner & Wentura, 2010; Greenwald et al., 1998; Ottaway, Hayden, & Oakes, 2001) and Asian faces (Gawronski, 2002). Chinese participants tend to show an implicit positivity bias for Chinese over Caucasian faces (Wang, Chen, Wang, Hu, Hu, & Fu, 2014). In a similar vein, the intergroup empathy bias, where neural indicators of empathic pain are more strongly activated in response to own- than other-race members experiencing pain, has been observed in both Caucasian and Chinese participants (Xu, Zuo, Wang, & Han, 2009). These findings suggest that the influence of race on emotion recognition may generalise across stimulus materials and participant groups.
To our knowledge, there is currently only one study that investigated the influence of sex cues on emotion perception in a Chinese sample. Using the Garner paradigm, Wu, Zhang, and Sun (2015) asked Chinese participants to categorise happy and angry expressions on Chinese male and female faces. In the Garner paradigm, participants complete two types of categorisation tasks. In control tasks, participants categorise faces or objects along one dimension (e.g., emotional expression) while other dimensions are held constant (e.g., all faces are male or all faces are female). In orthogonal tasks, participants categorise faces along the same dimension (e.g., emotional expression); however, the other dimension is varied (e.g., faces presented are male and female). In three of the four experiments described, participants were slower to categorise anger expressed on female than on male faces, but the sex of the face did not influence categorisation of happy expressions. Differences in the speed of categorising happiness versus anger for male and female faces separately were not reported. However, inspection of the response times in the experiments looking at the influence of sex cues on emotion categorisation suggested a numerically larger happy categorisation advantage for female than for male faces (Wu et al., 2015). This is consistent with the findings of experiments where Caucasian participants categorise male and female Caucasian faces (Lipp, Craig, & Dat, 2015; Hugenberg & Sczesny, 2006). Based on this evidence, it seems likely that the influence of sex on emotion recognition may also generalise to Chinese participants.
Current research
The current study aimed to investigate the separate influence of race and sex cues on emotion categorisation using Chinese faces rather than African American faces as the racial outgroup. Additionally, we aimed to determine whether the influence of sex and race cues on the happy categorisation advantage observed in Caucasian participants living in Australia (Lipp, Craig, & Dat, 2015) would be observed in a sample of Chinese participants living in China. This investigation is important as the current literature cannot speak to the influence of Asian facial cues on emotion categorisation or the nature of the interaction between race or sex cues and emotional expression in a non-western sample.
Across two experiments, Caucasian Australian (Experiments 1 and 2) and Chinese participants (Experiment 2) categorised emotional expressions on faces varying in race (Chinese and Caucasian) and/or sex (male and female) in three separate tasks. Consistent with the procedures of Lipp, Craig, and Dat (2015), the same own-race male faces were presented in three tasks. In one task, they were presented along with own-race female faces to assess the influence of sex on emotion categorisation. In a second task, they were presented along with other-race male faces to assess the influence of race on emotion categorisation. In a third task, they were presented with other-race female faces to determine how concurrently varying sex and race cues may influence emotion perception. Having participants complete three separate tasks allowed us to investigate the separate influences of race and sex cues on emotional expression categorisation as well as to see whether the magnitude of the happy categorisation advantage for own-race male faces varied as a function of the other faces encountered in the task, as has been previously observed (Lipp, Craig, & Dat, 2015). If the influence of sex and race on emotion categorisation is predominantly driven by evaluations and these are consistent across Australian and Chinese samples, the pattern observed for both Caucasian and Chinese participants categorising expressions on Caucasian and Chinese faces should replicate the findings of Lipp, Craig, and Dat (2015). As such, it was predicted that a happy categorisation advantage would be observed for own-race but not other-race male faces when they were encountered together in a task. It was predicted that a happy categorisation advantage would be observed for both own-race and other-race female faces but not own-race male faces when they were encountered together within a task.
Experiment 1
Method
Participants
The target sample size for Experiment 1 was based on previous research (Lipp, Craig, & Dat, 2015), and we continued testing until the end of semester. Based on this recruitment strategy, 26 undergraduate students at Curtin University (6 males, M age = 20.24, SD age = 2.80) took part in exchange for course credit or AU$15. Data from five of these participants (2 males and 3 females) were not included in analyses, because they did not identify as Caucasian; however, results were comparable when these participants were included in analysis.
Stimuli
Pictures of eight individuals from each stimulus category (Caucasian female, Caucasian male, Chinese female, and Chinese male) expressing happiness and anger were presented. The Caucasian faces were sourced from the Nimstim face database (Tottenham et al., 2009) and the Montreal Set of Facial Displays of Emotion (Beaupré & Hess, 2005). The Chinese faces were sourced from the Chinese Facial Affective Picture System (Gong, Huang, Wang, & Luo, 2011). To maintain consistency across sets, the images were resized, converted to grey scale where necessary, and edited so that only the face remained (necks, hair, backgrounds, etc. were removed). Each face was dropped onto a grey background 187 × 240 pixels in size.
Procedure
Participants were tested in groups of up to four. They were seated in front of 24- in LED monitors (resolution: 1920 × 1080 pixels; refresh rate: 120 Hz) that were separated by partitions. Participants were instructed that they would complete only three emotion categorisation tasks for this experiment. Participants were informed that faces would be presented one at a time and were instructed to categorise the expression displayed on the faces as happy or angry as quickly and accurately as possible by pressing the right and left shift keys. The experimental protocol was executed using DMDX (Forster & Forster, 2003). Response mapping was counterbalanced across participants but held constant across the three tasks for each participant.
Participants categorised happy and angry expressions on the same own-race (Caucasian) male faces in all three tasks. These own-race male faces were presented along with own-race (Caucasian) female faces in one task (own-race female task), other-race (Chinese) male faces in a second task (other-race male task), and other-race (Chinese) female faces in the third task (other-race female task). The order of the first two tasks was counterbalanced; however, the task that included other-race female faces was always presented last. Although presenting the other-race female task last for all participants meant that results could be influenced by order or practice effects (Craig & Lipp, 2017), this counterbalancing method was adopted for two reasons. First, it was done to replicate the procedure used by Lipp, Craig, and Dat (2015) in a similar investigation and, second, to ensure that performance on the other-race female task was not differentially influenced by having made only the race or sex dimension salient during completion of only the other-race male task or the own-race female task prior to the other-race female task.
On each trial, a black fixation cross was presented in the centre of a grey screen for 500 ms. This was replaced by a face which remained on the screen until a response was made or for 1,000 ms. If no response was made, a new trial commenced 3,000 ms after stimulus onset. The stimuli were presented in a randomised order. Each task was preceded by eight practice trials. In the main task, each of the 32 stimuli was presented three times resulting in 96 trials per task and a total of 288 trials across the three tasks.
Data processing and analysis
Incorrect responses as well as categorisation times faster than 100 ms or more than 3 standard deviations away from each participant’s mean response time in each task were removed as invalid responses. Within each task, average response times and error rates were calculated for each condition. Response times and error rates were submitted to separate ANOVAs for each task. For the own-race female task, two 2 (Face type: Own-race male, Own-race Female) × 2 (Emotional expression: Happy, Angry) repeated measures ANOVAs were conducted. For the other-race male task, the analyses were two 2 (Face type: Own-race male, Other-race male) × 2 (Emotional expression: Happy, Angry) repeated measures ANOVAS, and for the other-race female task, the analyses were two 2 (Face type: Own-race male, Other-race female) × 2 (Emotional expression: Happy, Angry) repeated measures ANOVAs.
Results
Own-race female task
Response times
As Figure 1 suggests, although the main effects of emotion and sex did not reach significance, Fs < 2.56, ps > 0.125, ηp 2s < 0.11, there was a significant Face type × Emotion interaction, F(1, 20) = 5.89, p = 0.025, ηp 2 = 0.23. Follow-up analyses indicated that participants were significantly faster to categorise happiness than anger when expressed by a female, t(20) = 3.31, p = 0.004, d z = 0.72, but not when expressed by a male, t(20) = 0.67, p = 0.512, d z = 0.14.
Accuracy
The pattern observed in response times was mirrored in error rates (see Table 1). Although there were no significant main effects of emotional expression or sex, Fs < 2.49, ps > 0.130, ηp 2s < 0.11, there was a significant Face type × Emotional expression interaction, F(1, 20) = 16.42, p = 0.001, ηp 2 = 0.45. Follow-up analyses indicated that participants were significantly more accurate to identify happiness than anger when expressed by a female, t(20) = 3.69, p = 0.001, d z = 0.81, but for male faces, there was a trend towards more accurate categorisation of anger than happiness, t(20) = 2.05, p = 0.053, d z = 0.45.
Other-race male task
Response times
As shown in Figure 1, participants were overall faster to categorise happiness than anger (main effect of emotion, F(1, 20) = 5.56, p = 0.029, ηp 2 = 0.22). They also were, overall, faster to categorise expressions displayed on Caucasian faces than on Chinese faces, F(1, 20) = 8.80, p = 0.008, ηp 2 = 0.31. There was no significant Face type × Emotion interaction, F(1, 20) = 0.55, p = 0.549, ηp 2 = 0.02.
Accuracy
As seen in Table 1, accuracy was not influenced by the race or the emotional expression on the faces. All main effects and interactions were not significant, Fs < 0.24, ps > 0.676, ηp 2s < 0.01.
Other-race female task
Response times
Consistent with the other-race male task, participants were overall faster to categorise happiness than anger, F(1, 20) = 4.38, p = 0.049, ηp 2 = 0.18. They were also overall faster to categorise expressions on Caucasian male faces than Chinese female face, F(1, 20) = 12.16, p = 0.002, ηp 2 = 0.38. Although the Face type × Emotion interaction was not significant, F(1, 20) = 1.43, p = 0.245, ηp 2 = 0.07, paired samples t-tests were conducted to determine whether there was a significant happy categorisation advantage for either face type. A marginally significant happy categorisation advantage was revealed when the expressions were displayed on a Chinese female face, t(20) = 2.01, p = 0.058, d z = 0.44, but not when the expressions were displayed by Caucasian male posers, t(20) = 0.52, p = 0.607, d z = 0.11.
Accuracy
Fewer errors were made categorising happiness than anger, F(1, 20) = 6.66, p = 0.018, ηp 2 = 0.25. Fewer errors also were made categorising expressions on own-race male faces than on other-race female face, F(1, 20) = 15.48, p = 0.001, ηp 2 = 0.44. The Face type × Emotional expression interaction was not significant, F(1, 20) = 2.76, p = 0.112, ηp 2 = 0.12. Although the interaction was not significant, paired contrasts indicated that fewer errors were made categorising happiness than anger displayed by other-race females, t(20) = 2.44, p = 0.024, d z = 0.53, but not by own-race males, t(20) < 0.00, p > 0.999, d z < 0.01, (see Table 1).
Discussion
The purpose of Experiment 1 was to determine whether the influence of race and sex on the categorisation of happy and angry expressions observed previously by Lipp, Craig, and Dat (2015) would be replicated if Chinese rather than African/African American emotional faces were used. In line with past findings, a happy categorisation advantage was observed for Caucasian male faces when presented along with Chinese male faces but not when presented along with Caucasian or Chinese female faces. Inconsistent with Lipp, Craig, and Dat (2015), a happy categorisation advantage emerged for other-race male faces (Chinese male faces). This may reflect that, unlike the male African American faces used in prior research, the Chinese male faces are not associated with threat (Lin, Kwan, Cheung, & Fiske, 2005). It also is possible that visual properties of the stimuli or the way that emotions were expressed on the particular faces used in the current experiment facilitated a happy categorisation advantage. These results suggested there may be some differences in the way that Caucasian participants process expressions on Asian and African/African American faces which were further investigated in Experiment 2.
Experiment 2
The purpose of Experiment 2 was to determine whether the influence of race and sex cues observed in Experiment 1 with Caucasian participants was replicable in a different sample and whether a comparable influence of race and sex cues would be observed with Chinese participants for whom Caucasian faces represent the racial outgroup and Chinese faces represent the racial ingroup. Not only can investigating the effect with participants from different ethnic backgrounds who represent both stimulus categories potentially indicate whether the result observed in Experiment 1 are due only to the low level properties of the stimuli, but it also can bring to light potential differences in the way that multiple facial cues interact for participants from different ethnic backgrounds. As in Experiment 1, participants categorised happy and angry expressions on faces varying in race and sex. For the Caucasian participants, the experiment proceeded exactly as described in Experiment 1. For the Chinese participants happy and angry own-race (Chinese) male faces were presented along with either own-race (Chinese) female faces (own-race female task), other-race (Caucasian) male faces (other-race male task), or other-race (Caucasian) female faces (other-race female task).
For Caucasian participants, it was predicted that the results of Experiment 2 would replicate those of Experiment 1. For Chinese participants, as there is some evidence to suggest that gender and race based evaluations are comparable to those of Caucasian participants it was predicted that the results would replicate Lipp, Craig, and Dat (2015). A happy categorisation advantage should be observed only for female faces in tasks where male and female faces are presented together and for own-race male faces only when own-race and other-race males are presented together.
Method
Participants
Sixty-five participants took part in Experiment 2. Participants were 32 volunteers at Renmin University in China (8 males, M = 21.63, SD = 2.31) who identified as Chinese and 33 undergraduate volunteers at Curtin University (7 males, M = 23.33, SD = 7.38) who did not participate in Experiment 1. Of the latter, 1 male and 4 female participants did not identify as Caucasian and were not included in subsequent analysis; however, including these participants produced a comparable pattern of results. Participants at Renmin University of China all received ¥10 as reimbursement for participation. Participants at Curtin University were compensated with partial course credit or AU $15.
Stimuli, procedure, data processing, and analysis
The stimuli were the same as those used in Experiment 1. For the Curtin University participants, the experiment proceeded exactly as described in Experiment 1. At Renmin University, participants were tested individually. The experiment was presented on a 19-in CRT monitor with a screen resolution of 1280 × 1024 pixels and a screen resolution of 60 Hz.
Because one of the aims of the current study was to determine whether sex and race cues influence emotion perception in a comparable manner in participants from Australia and China, the stimuli used in the own-race and the other-race female task differed for the Australian and Chinese participants. In the own-race female task, Chinese male and female faces were presented to the Chinese participants and Caucasian male and female faces were presented to the Caucasian Australian participants. In the other-race female task, Chinese male and Caucasian female faces were presented to the Chinese participants and Caucasian male and Chinese female faces were presented to Caucasian participants. The other-race male task contained the same stimuli for both the Australian participants and the Chinese participants; however, the faces representing the racial in- and outgroup differed for the two groups.
Response times were processed in the same manner described in Experiment 1. Averaged response time and error rates were submitted to separate ANOVAs for each task. For the own-race female task, response times and error rates were submitted to separate 2 (Ethnicity: Chinese, Caucasian Australian) × 2 (Face type: Own-race male, Own-race female) × 2 (Emotional Expression: Happy, Angry) mixed ANOVAs. For the other-race male task, response times and error rates were submitted to separate 2 (Ethnicity: Chinese, Caucasian Australian) × 2 (Face type: Own-race male, Other-race male) × 2 (Emotional Expression: Happy, Angry) mixed ANOVAs. Finally, for the other-race female task, response times and error rates were submitted to separate 2 (Ethnicity: Chinese, Caucasian Australian) × 2 (Face type: Own-race male, Own-race female) × 2 (Emotional Expression: Happy, Angry) mixed ANOVAs. In all of these analyses the between subjects factor was participant ethnicity and the within subjects factors were emotional expression and face type. Data from three of the Caucasian participants could not be included in the analysis of the other-race male task or the other-race female task as they did not follow instruction and either made no valid responses in at least one condition or 50% or more of responses for a particular condition were invalid or incorrect.Footnote 1
Results
Own-race female task
Response time
Inspection of Figure 2 suggests the influence of sex on emotion categorisation depended on the ethnicity of the participants. There was a significant main effect of emotion, F(1, 58) = 16.48, p < 0.001, ηp 2 = 0.22, as well as a significant Face type × Emotion interaction, F(1, 58) = 25.87, p < 0.001, ηp 2 = 0.31, and a significant Face type × Emotion × Ethnicity interaction, F(1, 58) = 15.83, p < 0.001, ηp 2 = 0.21. This three way interaction was followed up by looking at the influence of sex on emotion categorisation separately for Chinese and Caucasian participants.
Caucasian participants were no faster to identify expressions on male or female faces, F(1, 27) = 1.25, p = 0.273, ηp 2 = 0.04, but were overall faster to categorise happiness than anger (main effect of emotion, F(1, 27) = 9.04, p = 0.006, ηp 2 = 0.25). This happy categorisation advantage was moderated by the sex of the face, F(1, 27) = 27.18, p < 0.001, ηp 2 = 0.50. Follow-up analyses indicated that faster categorisation of happiness than anger was only observed when the expression was displayed on female faces, t(27) = 5.61, p < 0.001, d z = 1.06, but not on male faces, t(27) = 1.62, p = 0.117, d z = 0.31.
Chinese participants were marginally faster to categorise expressions on own-race male faces than own-race female faces, F(1, 31) = 3.41, p = 0.074, ηp 2 = 0.10, and they were significantly faster to categorise happy than angry expressions, F(1, 31) = 8.66, p = 0.006, ηp 2 = 0.22. This happy categorisation advantage was not moderated by the sex of the face, F(1, 31) = 1.03, p = 0.317, ηp 2 = 0.03.
Accuracy
As can be seen in Table 1, overall, Chinese participants were significantly more accurate than Caucasian participants, F(1, 58) = 4.43, p = 0.040, ηp 2 = 0.07; however beyond this, ethnicity did not alter the influence of sex on the accuracy of categorising emotional expressions, all Fs < 2.01, ps > 0.162, ηp 2 < 0.04. Overall, fewer errors were made when categorising happy than angry expressions (main effect of emotion, F(1, 58) = 9.02, p = 0.004, ηp 2 = 0.14). This emotion main effect was moderated by the sex of the face, F(1, 58) = 31.73, p < 0.001, ηp 2 = 0.35. Participants were significantly more accurate categorising happiness than anger expressed by females, t(58) = 5.77, p < 0.001, d z = 0.74, but not expressed by males, t(58) = 1.45, p = 0.151, d z = 0.19. Even though the Ethnicity × Face type × Emotion interaction was not significant in error rates (F(1, 58) = 2.01, p = 0.162, ηp 2 = 0.033), to determine whether the pattern of results for errors was generally consistent with that for response times, error rates for Caucasian and Chinese participants were analysed separately.
Consistent with response times, Caucasian participants were more accurate to categorise happiness than anger, F(1, 27) = 4.89, p = 0.036, ηp 2 = 0.15. This emotion effect was moderated by face type (significant Face type × Emotion interaction, F(1, 27) = 20.60, p < 0.001, ηp 2 = 0.43). Follow-up analyses indicated that Caucasian participants were more accurate to categorise happiness than anger displayed on own-race female faces, t(27) = 4.99, p < 0.001, d z = 0.94, but not on own-race male faces, t(27) = 1.30, p = 0.204, d z = 0.25.
Unlike in response times, there was a significant Face type × Emotion interaction for Chinese participants, F(1, 31) = 10.75, p = 0.003, ηp 2 = 0.257. Follow-up analyses indicated that Chinese participants were more accurate to categorise happiness than anger on own-race female faces, t(31) = 3.26, p = 0.003, d z = 0.58, but not on own-race male faces, t(31) = 0.63, p = 0.533, d z = 0.11.
Other-race male task
Response times
Participants were overall faster to categorise happiness than anger, F(1, 55) = 24.81, p < 0.001, ηp 2 = 0.31, and response patterns differed as a function of the ethnicity of the participant. Chinese participants were overall faster to respond than Caucasian participants, F(1, 55) = 6.77, p = 0.012, ηp 2 = 0.11. In addition, there was a Face type × Ethnicity interaction, F(1, 55) = 4.99, p = 0.030, ηp 2 = 0.08, as well as a three way Face type × Emotion × Ethnicity interaction, F(1, 55) = 15.89, p < 0.001, ηp 2 = 0.22. This interaction was followed up, as above, by looking at the influence of race on emotion categorisation for Chinese and Caucasian participants separately.
As in Experiment 1, Caucasian participants were significantly faster to categorise happiness than anger, F(1, 24) = 13.33, p = 0.001, ηp 2 = 0.36, and marginally faster to categorise emotional expressions on own-race than other-race male faces, F(1, 24) = 2.96, p = 0.098, ηp 2 = 0.11. There was, however, no interaction of race and emotion, F(1, 24) = 2.23, p = 0.149, ηp 2 = 0.09.
Chinese participants displayed a pattern of responding more consistent with what has typically been observed when Caucasian participants categorise emotion on Caucasian and African/African American faces. There was, overall, no difference in how quickly Chinese participants recognised emotional expressions on Caucasian and Chinese faces, F(1, 31) = 1.53, p = 0.224, ηp 2 = 0.05; however, they were overall faster to categorise happy than angry expressions, F(1, 31) = 11.33, p = 0.002, ηp 2 = 0.27. This emotion effect was moderated by the race of the face, F(1, 31) = 23.36, p < 0.001, ηp 2 = 0.43. Follow-up analyses indicated that the happy categorisation advantage was only significant for Chinese male faces, t(31) = 5.36, p < 0.001, d z = 0.95, but not for Caucasian male faces, t(31) = 0.85, p = 0.401, d z = 0.15.
Accuracy
Overall, participants were more accurate categorising happiness than anger, F(1, 55) = 21.90, p < 0.001, ηp 2 = 0.29; however, accuracy differed as a function of ethnicity. Chinese participants again performed more accurately than Caucasian participants, F(1, 55) = 5.50, p = 0.023, ηp 2 = 0.09. There also was an Ethnicity × Emotion interaction, F(1, 55) = 5.04, p = 0.029, ηp 2 = 0.08, and a three-way Face type × Emotion × Ethnicity interaction, F(1, 55) = 8.21, p = 0.006, ηp 2 = 0.13 (see Table 1). This was followed up by looking at response patterns for Caucasian and Chinese participants separately.
For Caucasian participants, in line with response times, the was a main effect of emotion, F(1, 24) = 19.08, p < 0.001, ηp 2 = 0.44. There was no main effect of race and no Face type × Emotion interaction, Fs < 0.56, ps > 0.460, ηp 2 < 0.03.
Chinese participants were marginally more accurate categorising happiness than anger, F(1, 31) = 3.72, p = 0.063, ηp 2 = 0.11, a trend that was moderated by the race of the face, F(1, 31) = 14.07, p = 0.001, ηp 2 = 0.31. Chinese participants were significantly more accurate to categorise happiness than anger displayed by a Chinese male, t(31) = 3.37, p = 0.002, d z = 0.60, but not when displayed by a Caucasian male, t(31) = 1.41, p = 0.169, d z = 0.25.
Other-race female task
Response times
Overall participants were faster to categorise emotional expressions on own-race male than other-race female faces, F(1, 55) = 40.08, p < 0.001, ηp 2 = 0.42, and faster to categorise happy than angry expressions, F(1, 55) = 20.99, p < 0.001, ηp 2 = 0.28. A Face type × Emotion interaction also emerged, F(1, 55) = 7.97, p = 0.007, ηp 2 = 0.13. Furthermore, there were differences in response patterns as a function of the ethnicity of the participants. Chinese participants were overall faster to respond, F(1, 55) = 10.96, p = 0.002, ηp 2 = 0.17. There also was a Face type × Ethnicity interaction, F(1, 55) = 24.33, p < 0.001, ηp 2 = 0.31, and a Face type × Emotion × Ethnicity interaction, F(1, 55) = 4.68, p = 0.035, ηp 2 = 0.08. As above, this was followed up by looking at the performance of Chinese and Caucasian participants separately.
Caucasian participants were faster to categorise emotional expressions on own-race male faces than other-race female faces, F(1, 24) = 43.62, p < 0.001, ηp 2 = 0.65, and faster to categorise happiness than anger, F(1, 24) = 6.39, p = 0.018, ηp 2 = 0.21. Moreover, there was a Face type × Emotion interaction, F(1, 24) = 5.82, p = 0.024, ηp 2 = 0.20. Caucasian participants were faster to categorise happiness than anger on other-race female faces, t(24) = 3.90, p = 0.001, d z = 0.78, but not own-race male faces, t(24) = 0.44, p = 0.664, d z = 0.09.
Chinese participants were overall faster to categorise happiness than anger, F(1, 31) = 17.80, p < 0.001, ηp 2 = 0.37. There was no significant main effect of face type or Face type × Emotion interaction, Fs < 1.45, ps > 0.238, ηp 2 < 0.05.
Accuracy
Overall, as suggested in Table 1, participants were more accurate categorising expressions on own-race male faces than other-race female faces, F(1, 55) = 7.82, p = 0.007, ηp 2 = 0.13, and happy than angry expressions, F(1, 55) = 9.97, p = 0.003, ηp 2 = 0.15. There also was a Face type × Emotion interaction, F(1, 55) = 5.61, p = 0.021, ηp 2 = 0.09. Overall, Chinese participants were more accurate, F(1, 55) = 16.45, p < 0.001, ηp 2 = 0.23. There was was a marginally significant Face type × Ethnicity interaction, F(1, 55) = 3.24, p = 0.077, ηp 2 = 0.06. Although the three way Face type × Emotion × Ethnicity interaction, did not reach significance, F(1, 55) = 2.54, p = 0.117, ηp 2 = 0.04, to determine whether the pattern of accuracy results was generally consistent with that for response times, error rates for Caucasian and Chinese participants were analysed separately.
Caucasian participants were more accurate categorising expressions on own-race male faces than other-race female faces, F(1, 24) = 12.42, p = 0.002, ηp 2 = 0.34, and more accurate categorising happy than angry expressions, F(1, 24) = 9.24, p = 0.006, ηp 2 = 0.28. The accuracy of categorising expressions, however, depended on the face type as indicated by a Face type × Emotional expression, F(1, 24) = 5.42, p = 0.029, ηp 2 = 0.18. Participants were more accurate to identify happiness than anger expressed by an other-race female, t(24) = 3.58, p = 0.002, d z = 0.72, but there was no difference in the accuracy of categorising happy and angry expressions posed by an own-race male, t(24) = 0.38, p = 0.709, d z = 0.08.
In this task, the accuracy of Chinese participants was not influenced by the emotional expression or the type of face, all Fs < 1.47, ps > 0.235, ηp 2s < 0.05.
Discussion
The purpose of Experiment 2 was to replicate the findings of Experiment 1 as well as to determine the nature of the influence of race and sex cues on emotion perceptions in Chinese participants. Caucasian participants exhibited a pattern of results comparable to that observed in Experiment 1, a happy categorisation advantage was observed for expressions presented on both Caucasian and Chinese female faces. A happy categorisation advantage for Caucasian male faces was observed only when these faces were presented among Chinese male faces, but in this task, a happy categorisation advantage was also observed for Chinese male faces.
The pattern of results observed for Caucasian participants was only partially replicated in Chinese participants. Similar to the results seen for Caucasian participants, Chinese participants displayed a significant happy categorisation advantage for Chinese male faces in the absence of a happy categorisation advantage for Caucasian male faces in the other-race male task and a happy categorisation advantage for female faces regardless of the race of the face. Unlike the pattern observed in Caucasian participants, the happy categorisation advantage for own-race male faces was also evident in Chinese participants when these faces were categorised along with own-race or other-race female faces.
Chinese participants also tended to be faster and more accurate to respond that Caucasian participants across all tasks. This could have occurred for a number of reasons. For example, as Chinese participants received monetary compensation, whereas the majority of Caucasian Australian participants received course credit, differences in participant motivation could have driven Chinese participants to respond more quickly and accurately. Chinese and Caucasian participants also completed the task under different conditions, so situational differences in ambient lighting and amount of distraction in the room or differences in monitor size, screen resolution, and seating distance could have influenced the overall speed with which participants performed the task.
General Discussion
The current investigation was designed to determine whether the influence of race and sex on emotion categorisation and the modulation of the happy categorisation advantage for own-race male faces previously observed (Lipp, Craig, & Dat, 2015) would be replicated when Chinese rather than African/African American faces represented the racial outgroup. Second, in line with the broader face processing literature, we aimed to investigate whether the influence of race or sex cues on emotion categorisation observed in white Caucasian participants also would emerge in Chinese participants. Additionally, looking at the responses on a comparable task using the same stimuli across two ethnic groups had the potential to clarify whether the happy categorisation advantage observed for male Chinese faces in Caucasian participants was driven by characteristics of the particular stimulus set used (Ge et al., 2009; O'Toole, Peterson, & Deffenbacher, 1996). Consistent with past research (Lipp, Craig, & Dat, 2015), Caucasian participants displayed a happy categorisation advantage for Caucasian male faces when they appeared along with other-race male faces but not when they were presented along with either Caucasian or Chinese female faces. Also consistent with past research, Caucasian participants generally displayed a happy categorisation advantage for female faces regardless of the ethnicity of the faces (although the happy categorisation advantage was marginal for Chinese female faces in Experiment 1).
Inconsistent with previous findings, Caucasian participants produced a happy categorisation advantage for both own-race (Caucasian) and other-race (Chinese) male faces in both experiments. A happy categorisation advantage for outgroup male faces has not previously been observed in studies that used African/African American or Moroccan male racial outgroup faces (Bijlstra et al., 2010; Hugenberg, 2005; Lipp, Craig, & Dat, 2015). This pattern of results may have emerged for a number of reasons. One possibility is that the typical influence of race on emotion categorisation is only observed for physically threatening outgroups and that white participants do not implicitly associate Asians with physical threat (Lin et al., 2005). This pattern also could be accounted for if Asian faces are structurally more similar to female faces (Johnson, Freeman, & Pauker, 2012) as cues of femininity facilitated the categorisation of happiness (Becker et al., 2007; Hess et al., 2009). However, we would argue that this unexpected happy categorisation advantage observed for the Chinese male faces in Caucasian participants was likely to be stimulus driven.
Looking across the patterns of results produced by Australian and Chinese participants, it seems that the unique visual characteristics of the Chinese male emotional faces used in the current experiment may provide the most parsimonious explanation for the differences between in the current study and past research (Bijlstra et al., 2010; Craig et al., 2017; Hugenberg, 2005) as well as the differences between Caucasian Australian and Chinese samples. A happy categorisation advantage was observed for the Chinese male faces in all three tasks completed by Chinese participants and contrary to the results observed for Caucasian participants, the happy categorisation advantage for own-race male faces was not eliminated when these faces were presented along with female faces. Even though a happy categorisation advantage was observed for Chinese male faces across all tasks, this does not mean that the magnitude of the happy categorisation advantage for Chinese male faces did not change as a function of the other faces in the task. Comparing the magnitude of the happy categorisation advantage observed for Chinese male faces across tasks provides some insight into whether the pattern of results observed in Chinese participants is generally consistent with the pattern observed in Caucasian participants. As suggested in the lower panel of Figure 2, the happy categorisation advantage for Chinese participants categorising expressions on Chinese male faces was larger when Chinese male faces were categorised amongst Caucasian males (happy categorisation advantage M = 60.84 ms, SD = 64.21) than when categorised amongst Chinese or Caucasian females (M = 26.09 ms, SD = 75.01 and M = 22.63 ms, SD = 38.71 respectively), ts > 2.56, ps < 0.015, d z s > 0.45. This is consistent with the pattern observed in Caucasian participants across both experiments where the happy categorisation advantage was larger for Caucasian male faces when presented among Chinese male faces (M = 26.76 ms, SD = 74.95) than when categorised amongst Chinese or Caucasian females (M = −3.18 ms, SD = 78.49 and M = −14.83 ms, SD = 58.88 respectively), ts > 2.60, ps < 0.013, d z s > 0.39. This finding of a larger happy categorisation advantage for own-race male faces amongst other-race male faces than among own-race or other-race female faces suggests that on average Chinese participants, like Caucasian participants, evaluate members of their own racial group as more positive than members of other racial groups. These results are consistent with past findings using other methods (Wang et al., 2014; Xu et al., 2009) and inconsistent with previous studies that have found no own-race preference in Chinese participants (Ma-Kellams, Spencer-Rodgers, & Peng, 2011). The finding of a smaller happy categorisation advantage for own-race male faces amongst female faces is consistent with results reported by Wu et al. (2015) and suggests that Chinese participants, like Caucasian participants, evaluate females as more positive than males, regardless of their racial group. We cannot directly address whether the own-race bias that we observed was driven by factors like social identity or differences in experience with the own and other-race faces or both, because we did not take measures of identity or the quality or quantity of contact with people of other races. As such, looking at the influence of identity and contact on how social category information influence emotion recognition may be an interesting avenue for future research.
Finding a similar influence of the task context on the happy face advantage for own-race male faces in the two groups of participants suggests that past findings (Lipp, Craig, & Dat, 2015) are not due to low-level perceptual features of the faces but are likely due to their differential evaluation given the task context they are presented in. Modulation of the happy categorisation advantage for own-race male faces suggests that emotion perception is relative and sensitive to changes in context. As the speed of emotion categorisation is likely to be influenced by implicit social evaluations (Craig et al., 2017; Hugenberg, 2005; Hugenberg & Sczesny, 2006), this pattern of results suggests that own-race male faces are evaluated relative to the other faces they are encountered with. They are evaluated as relatively positive when encountered along with other-race male faces facilitating a happy categorisation advantage, but are evaluative as relatively negative when encountered along with own-race or other-race female faces resulting in an attenuated or absent happy categorisation advantage for the own-race male faces, but a happy categorisation advantage for the female faces.
In summary, the current state of the literature investigating the nature of the interaction between social category cues like race and sex and emotional expression speaks to the complexity of the processes underlying person construal. Findings from the current study along with previous research suggest that the influence of broad evaluations elicited in response to a person’s facial cues indicating race and sex can influence emotion categorisation. Moreover, the current research suggests that these influences are similar across the two ethnic groups considered. Future research may investigate the generalisability of these results to participants from other ethnic backgrounds or determine whether these results hold when the combined influence of multiple social category cues (e.g., race and sex) on emotion perception is considered.
Notes
Previous studies have suggested that the influence of race on emotion categorisation is not influenced by participant sex (Hugenberg, 2005), and that the influence of sex cues on emotion categorisation is directionally similar for both sexes but stronger for females (e.g., Becker et al., 2007; Hugenberg & Sczesny, 2006). As our primary focus in the current investigation was on participant ethnicity and the influence of race and sex on emotion categorisation was found to be comparable for males and females, we targeted recruitment based on ethnicity but we did not recruit with participant sex in mind. This left us with only a small number of male participants meaning that we have insufficient power to detect smaller participant sex effects, and the patterns observed, particularly for male participants, may not be stable. Despite this, we combined data from Experiments 1 and 2 to maximise power and conducted an analysis including participant sex as a factor. Consistent with past literature, there was some evidence that the influence of facial sex cues (but not race cues) on emotion categorisation was stronger for female than for male participants. This analysis is included as a supplement for completeness, but given the relatively small number of males, should be interpreted with caution.
References
Aguado, L., García-Gutierrez, A., & Serrano-Pedraza, I. (2009). Symmetrical interaction of sex and expression in face classification tasks. Perception & Psychophysics, 71(1), 9–25. doi:10.3758/app.71.1.9
Beaupré, M. G., & Hess, U. (2005). Cross-cultural emotion recognition among Canadian ethnic groups. Journal of Cross-Cultural Psychology, 36(3), 355–370. doi:10.1177/0022022104273656
Becker, D. V., Kenrick, D. T., Neuberg, S. L., Blackwell, K. C., & Smith, D. M. (2007). The confounded nature of angry men and happy women. Journal of Personality and Social Psychology, 92(2), 179–190. doi:10.1037/0022-3514.92.2.179
Bijlstra, G., Holland, R. W., & Wigboldus, D. H. J. (2010). The social face of emotion recognition: Evaluations versus stereotypes. Journal of Experimental Social Psychology, 46(4), 657–663. doi:10.1016/j.jesp.2010.03.006
Billings, L. S., Harrison, D. W., & Alden, J. D. (1993). Age differences among women in the functional asymmetry for bias in facial affect perception. Bulletin of the Psychonomic Society, 31(4), 317–320. doi:10.3758/BF03334940
Craig, B. M., Koch, S., & Lipp, O. V. (2017). The influence of social category cues on the happy categorisation advantage depends on expression valence. Cognition and Emotion. doi:10.1080/02699931.2016.1215293. Advance online publication.
Craig, B. M., & Lipp, O. V. (2017). The influence of facial sex cues on emotional expression recognition is not fixed. Emotion, 17(1), 28–39. doi:10.1037/emo0000208
Craig, B. M., Lipp, O. V., & Mallan, K. M. (2014). Emotional expressions preferentially elicit implicit evaluations of faces also varying in race or age. Emotion, 14(5), 865–877. doi:10.1037/a0037270
Craig, B. M., Mallan, K. M., & Lipp, O. V. (2012). The effect of poser race on the happy categorization advantage depends on stimulus type, set size, and presentation duration. Emotion, 12(6), 1303–1014. doi:10.1037/a0028622
Degner, J., & Wentura, D. (2010). Automatic prejudice in childhood and early adolescence. Journal of Personality and Social Psychology, 98(3), 356–374. doi:10.1037/a0017993
Eagly, A. H., Mladinic, A., & Otto, S. (1991). Are women evaluated more favorably than men? An analysis of attitudes, beliefs, and emotions. Psychology of Women Quarterly, 15(2), 203–216. doi:10.1111/j.1471-6402.1991.tb00792.x
Forster, K. I., & Forster, J. C. (2003). DMDX: A windows display program with millisecond accuracy. Behavior Research Methods, Instruments, & Computers, 35(1), 116–124. doi:10.3758/bf03195503
Freeman, J. B., & Ambady, N. (2011). A dynamic interactive theory of person construal. Psychological Review, 118(2), 247–279. doi:10.1037/a0022327
Gawronski, B. (2002). What does the Implicit Association Test measure? A test of the convergent and discriminant validity of prejudice-related IATs. Experimental Psychology, 49(3), 171–180. doi:10.1026//1618-3169.49.3.171
Ge, L., Zhang, H., Wang, Z., Quinn, P. C., Pascalis, O., Kelly, D., … Lee, K. (2009). Two faces of the other-race effect: Recognition and categorisation of Caucasian and Chinese faces. Perception, 38(8), 1199-1210. doi:10.1068/p6136
Gong, X., Huang, Y.-X., Wang, Y., & Luo, Y.-J. (2011). Revision of the Chinese facial affective picture system. Chinese Mental Health Journal, 25(1), 40–46.
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480. doi:10.1037/0022-3514.74.6.1464
Hess, U., Adams, R. B., Jr., Grammer, K., & Kleck, R. E. (2009). Face gender and emotion expression: Are angry women more like men? Journal of Vision, 9(12), 1–8. doi:10.1167/9.12.19
Hess, U., Adams, R. B., Jr., & Kleck, R. E. (2004). Facial appearance, gender, and emotion expression. Emotion, 4(4), 378–388. doi:10.1037/1528-3542.4.4.378
Hugenberg, K. (2005). Social categorization and the perception of facial affect: Target race moderates the response latency advantage for happy faces. Emotion, 5(3), 267–276. doi:10.1037/1528-3542.5.3.267
Hugenberg, K., Miller, J., & Claypool, H. M. (2007). Categorization and individuation in the cross-race recognition deficit: Toward a solution to an insidious problem. Journal of Experimental Social Psychology, 43(2), 334–340. doi:10.1016/j.jesp.2006.02.010
Hugenberg, K., & Sczesny, S. (2006). On wonderful women and seeing smiles: Social categorization moderates the happy face response latency advantage. Social Cognition, 24(5), 516–539. doi:10.1521/soco.2006.24.5.516
Johnson, K. L., Freeman, J. B., & Pauker, K. (2012). Race is gendered: How covarying phenotypes and stereotypes bias sex categorization. Journal of Personality and Social Psychology, 102(1), 116–131. doi:10.1037/a0025335
Leppänen, J. M., & Hietanen, J. K. (2003). Affect and face perception: Odors modulate the recognition advantage of happy faces. Emotion, 3(4), 315–326. doi:10.1037/1528-3542.3.4.315
Levin, D. T. (1996). Classifying faces by race: The structure of face categories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(6), 1364–1382. doi:10.1037/0278-7393.22.6.1364
Levin, D. T. (2000). Race as a visual feature: Using visual search and perceptual discrimination tasks to understand face categories and the cross-race recognition deficit. Journal of Experimental Psychology: General, 129(4), 559–574. doi:10.1037/0096-3445.129.4.559
Lin, M. H., Kwan, V. S. Y., Cheung, A., & Fiske, S. T. (2005). Stereotype content model explains prejudice for an envied outgroup: Scale of anti-Asian American stereotypes. Personality and Social Psychology Bulletin, 31(1), 34–47. doi:10.1177/0146167204271320
Lipp, O. V., Craig, B. M., & Dat, M. C. (2015). A happy face advantage with male caucasian faces: It depends on the company you keep. Social Psychological and Personality Science, 6(1), 109–115. doi:10.1177/1948550614546047
Lipp, O. V., Karnadewi, F., Craig, B. M., & Cronin, S. L. (2015). Stimulus set size modulates the sex–emotion interaction in face categorization. Attention, Perception, & Psychophysics, 77(4), 1285–1294. doi:10.3758/s13414-015-0849-x
Ma-Kellams, C., Spencer-Rodgers, J., & Peng, K. (2011). I am against us? Unpacking cultural differences in ingroup favoritism via dialecticism. Personality and Social Psychology Bulletin, 37(1), 15–27. doi:10.1177/0146167210388193
Meissner, C. A., & Brigham, J. C. (2001). Thirty years of investigating the own-race bias in memory for faces: A meta-analytic review. Psychology, Public Policy, and Law, 7(1), 3–35. doi:10.1037/1076-8971.7.1.3
O'Toole, A. J., Peterson, J., & Deffenbacher, K. A. (1996). An 'other-race effect' for categorizing faces by sex. Perception, 25(6), 669–676. doi:10.1068/p250669
Ottaway, S. A., Hayden, D. C., & Oakes, M. A. (2001). Implicit attitudes and racism: Effects of word familiarity and frequency on the implicit association test. Social Cognition, 19(2), 97–144. doi:10.1521/soco.19.2.97.20706
Quinn, K. A., & Macrae, C. N. (2011). The face and person perception: Insights from social cognition. British Journal of Psychology, 102(4), 849–867. doi:10.1111/j.2044-8295.2011.02030.x
Sacco, D. F., & Hugenberg, K. (2009). The look of fear and anger: Facial maturity modulates recognition of fearful and angry expressions. Emotion, 9(1), 39–49. doi:10.1037/a0014081
Tottenham, N., Tanaka, J. W., Leon, A. C., McCarry, T., Nurse, M., Hare, T. A., … Nelson, C. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168(3), 242-249. doi:10.1016/j.psychres.2008.05.006
Wang, Q., Chen, G., Wang, Z., Hu, C. S., Hu, X., & Fu, G. (2014). Implicit racial attitudes influence perceived emotional intensity on other-race faces. PLoS ONE, 9(8), e105946. doi:10.1371/journal.pone.0105946
Wu, B., Zhang, Z., & Sun, Y. (2015). Facial familiarity modulates the interaction between facial gender and emotional expression. Acta Psychologica Sinica, 47(10), 7–18. doi:10.3724/SP.J.1041.2015.01201
Xu, X., Zuo, X., Wang, X., & Han, S. (2009). Do you feel my pain? Racial group membership modulates empathic neural responses. The Journal of Neuroscience, 29(26), 8525–8529. doi:10.1523/JNEUROSCI.2418-09.2009
Acknowledgments
This research was partly supported by Australian Research Council Discovery Project Grant DP150101540.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 25 kb)
Rights and permissions
About this article
Cite this article
Craig, B.M., Zhang, J. & Lipp, O.V. Facial race and sex cues have a comparable influence on emotion recognition in Chinese and Australian participants. Atten Percept Psychophys 79, 2212–2223 (2017). https://doi.org/10.3758/s13414-017-1364-z
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-017-1364-z