Abstract
Several studies have demonstrated that perceiving an action influences the subsequent processing of action verbs. However, which characteristics of the perceived action are truly determinant to enable this influence is still unknown. The current study investigated the role of the agent executing an action in this action–language relationship. Participants performed a semantic decision task after seeing a video of a human or a robot performing an action. The results of the first study showed that perceiving a human being executing an action as well as perceiving a robot facilitate subsequent language processing, suggesting that the humanness (The term “humanness” is used as meaning “belonging to human race” and not to refer to a personal quality) of the agent is not crucial in the link between action and language. However, this experiment was conducted with Japanese people who are very familiar with robots; thus, an alternative explanation could be that it is the unfamiliarity with the agent that could perturb the action–language relationship. To assess this hypothesis, we carried out two additional experiments with French participants. The results of the second study showed that, unlike the observation of a human agent, the observation of a robot did not influence language processing. Finally, the results of the third study showed that, after a familiarization phase, French participants too were influenced by the observation of a robot. Overall, the outcomes of these studies indicate that, more than the humanness of the agent, it is the familiarity which we have with this agent that is crucial in the action–language relationship.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Mu is a range of electroencephalography oscillations (8–13 Hz). Its suppression is considered to reflect mirror neuron system activity.
2012 report on “Public attitudes towards robots” http://ec.europa.eu/commfrontoffice/publicopinion/archives/ebs/ebs_382_en.pdf.
References
Amoruso, L., & Urgesi, C. (2016). Familiarity modulates motor activation while other species’ actions are observed: A magnetic stimulation study. European Journal of Neuroscience, 43(6), 765–772. https://doi.org/10.1111/ejn.13154.
Anelli, F., Borghi, A. M., & Nicoletti, R. (2014). Grasping the pain: Motor resonance with dangerous affordances. Consciousness and Cognition, 21(4), 1627–1639. https://doi.org/10.1016/j.concog.2012.09.001.
Aravena, P., et al. (2012). Grip force reveals the context sensitivity of language-induced motor activity during “action words” processing: Evidence from sentential negation. PLoS One, 7(12), e50287. https://doi.org/10.1371/journal.pone.0050287.
Arita, A., Hiraki, K., Kanda, T., & Ishiguro, H. (2005). Can we talk to robots? Ten-month-old infants expected interactive humanoid robots to be talked to by persons. Cognition, 95(3), B49B57. https://doi.org/10.1016/j.cognition.2004.08.001.
Aziz-Zadeh, L., & Damasio, A. (2008). Embodied semantics for actions: Findings from functional brain imaging. Journal of Physiology-Paris, 102(1), 3539. https://doi.org/10.1016/j.jphysparis.2008.03.012.
Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., & Iacoboni, M. (2006). Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Current Biology, 16(18), 18181823. https://doi.org/10.1016/j.cub.2006.07.060.
Barsalou, L. W. (1999). Perceptual symbol systems. The Behavioral and Brain Sciences, 22(04), 577–609. https://doi.org/10.1017/S0140525X99002149.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2014). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Beauprez, S.-A., & Bidet-Ildei, C. (2017). Perceiving a biological human movement facilitates action verb processing. Current Psychology. https://doi.org/10.1007/s12144-017-9694-5.
Beauprez, S.-A., & Bidet-Ildei, C. (2018). The kinematics but not the orientation of an action influences language processing. Journal of Experimental Psychology: Human Perception and Performance, 44(11), 1712–1726. https://doi.org/10.1037/xhp0000568.
Beauprez, S.-A., Toussaint, L., & Bidet-Ildei, C. (2018). When context modulates the influence of action observation on language processing. PLoS One, 13(8), 1–12. https://doi.org/10.1371/journal.pone.0201966.
Bidet-Ildei, C., Méary, D., & Orliaguet, J.-P. (2006). Visual perception of elliptic movements in 7- to-11-year-old children: Influence of motor rules. Current Psychology Letters. Behaviour, Brain & Cognition, 19(2), 1–10.
Bidet-Ildei, C., Meugnot, A., Beauprez, S.-A., Gimenes, M., & Toussaint, L. (2017). Short-term upper limb immobilization affects action-word understanding. Journal of Experimental Psychology. Learning, Memory, and Cognition, 43(7), 1129–1139. https://doi.org/10.1037/xlm0000373.
Bidet-Ildei, C., Sparrow, L., & Coello, Y. (2011). Reading action word affects the visual perception of biological motion. Acta Psychologica, 137(3), 330334. https://doi.org/10.1016/j.actpsy.2011.04.001.
Bisio, A., Sciutti, A., Nori, F., Metta, G., Fadiga, L., et al. (2014). Motor contagion during human–human and human–robot interaction. PLoS One, 9(8), e106172. https://doi.org/10.1371/journal.pone.0106172.
Boulenger, V., et al.(2006). Cross-talk between language processes and overt motor behavior in the first 200 msec of processing. Journal of Cognitive Neuroscience, 18(10), 16071615. https://doi.org/10.1162/jocn.2006.18.10.1607.
Calvo-Merino, B., Glaser, D. E., Grèzes, J., Passingham, R. E., & Haggard, P. (2005). Action observation and acquired motor skills: An FMRI study with expert dancers. Cerebral Cortex, 15(8), 12431249. https://doi.org/10.1093/cercor/bhi007.
Chaminade, T., & Cheng, G. (2009). Social cognitive neuroscience and humanoid robotics. Journal of Physiology-Paris, 103(3–5), 286–295. https://doi.org/10.1016/j.jphysparis.2009.08.011.
Chaminade, T., Hodgins, J., & Kawato, M. (2007). Anthropomorphism influences perception of computer-animated characters’ actions. Social Cognitive and Affective Neuroscience, 2(3), 206. https://doi.org/10.1093/scan/nsm017.
Cross, E. S., Kraemer, D. J. M., de Hamilton, A. F. C., Kelley, W. M., & Grafton, S. T. (2009). Sensitivity of the action observation network to physical and observational learning. Cerebral Cortex, 19(2), 315326. https://doi.org/10.1093/cercor/bhn083.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175191.
Gazzola, V., Rizzolatti, G., Wicker, B., & Keysers, C. (2007a). The anthropomorphic brain: The mirror neuron system responds to human and robotic actions. NeuroImage, 35(4), 16741684.
Gazzola, V., et al. (2007b). Aplasics born without hands mirror the goal of hand actions with their feet. Current Biology, 17(14), 12351240. https://doi.org/10.1016/j.cub.2007.06.045.
Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565. https://doi.org/10.3758/BF03196313.
Gutsell, J., & Inzlicht, M. (2010). Empathy constrained: Prejudice predicts reduced mental simulation of actions during observation of outgroups. Journal of Experimental Social Psychology, 46(5), 841845. https://doi.org/10.1016/j.jesp.2010.03.011.
Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004). Somatotopic representation of action words in human motor and premotor cortex. Neuron, 41(2), 301307. https://doi.org/10.1016/S0896-6273(03)00838-9.
Hofer, T., Hauf, P., & Aschersleben, G. (2005). Infant’s perception of goal-directed actions performed by a mechanical device. Infant Behavior and Development, 28(4), 1–11. https://doi.org/10.1016/j.cognition.2011.05.012.
Iacoboni, M., et al. (2005). Grasping the intentions of others with one’s own mirror neuron system. PLoS Biology, 3(3), e79. https://doi.org/10.1371/journal.pbio.0030079.
Kandel, S., Orliaguet, J. P., & Viviani, P. (2000). Perceptual anticipation in handwriting: The role of implicit motor competence. Perception & Psychophysics, 62(4), 706716. https://doi.org/10.3758/BF03206917.
Kilner, J. M., Paulignan, Y., & Blakemore, S. J. (2003). An interference effect of observed biological movement on action. Current Biology, 13(6), 522–525.
Klepp, A., et al. (2014). Neuromagnetic hand and foot motor sources recruited during action verb processing. Brain and Language, 128(1), 4152. https://doi.org/10.1016/j.bandl.2013.12.001.
Kuipers, J.-R., van Koningsbruggen, M., & Thierry, G. (2013). Semantic priming in the motor cortex: Evidence from combined repetitive transcranial magnetic stimulation and event-related potential. Neuroreport, 24(12), 646651. https://doi.org/10.1097/WNR.0b013e3283631467.
Kupferberg, A., Huber, M., Helfer, B., Lenz, C., Knoll, A., Glasauer, S., et al. (2012). Moving just like you: motor interference depends on similar motility of agent and observer. PLoS ONE, 7(6), e39637. https://doi.org/10.1371/journal.pone.0039637.
Liepelt, R., Dolk, T., & Prinz, W. (2012). Bidirectional semantic interference between action and speech. Psychological Research Psychologische Forschung, 76(4), 446455. https://doi.org/10.1007/s00426-011-0390-z.
Liew, S.-L., Han, S., & Aziz-Zadeh, L. (2011). Familiarity modulates mirror neuron and mentalizing regions during intention understanding. Human Brain Mapping, 32(11), 198697. https://doi.org/10.1002/hbm.21164.
Lindemann, O., Stenneken, P., van Schie, H. T., & Bekkering, H. (2006). Semantic activation in action planning. Journal of Experimental Psychology. Human Perception and Performance, 32(3), 633643. https://doi.org/10.1037/0096-1523.32.3.633.
Lyons, I. M., et al. (2010). The role of personal experience in the neural processing of action-related language. Brain and Language, 112(3), 214222. https://doi.org/10.1016/j.bandl.2009.05.006.
MacDorman, K. F., Vasudevan, S. K., & Ho, C.-C. (2009). Does Japan really have robot mania? Comparing attitudes by implicit and explicit measures. AI & Society, 23(4), 485510. https://doi.org/10.1007/s00146-008-0181-2.
Martel, L., Bidet-Ildei, C., & Coello, Y. (2011). Anticipating the terminal position of an observed action: Effect of kinematic, structural, and identity information. Visual Cognition, 19(6), 785798. https://doi.org/10.1080/13506285.2011.587847.
Matsuda, G., Hiraki, K., & Ishiguro, H. (2015). EEG-based Mu rhythm suppression to measure the effects of appearance and motion on perceived human likeness of a robot. Journal of Human–Robot Interaction, 5(1), 68–81. https://doi.org/10.5898/10.5898/JHRI.5.1.Matsuda.
Mollo, G., Pulvermüller, F., & Hauk, O. (2016). Movement priming of EEG/MEG brain responses for action-words characterizes the link between language and action. Cortex, 74, 262276. https://doi.org/10.1016/j.cortex.2015.10.021.
Nomura, T., Kanda, T., & Suzuki, T. (2006). Experimental investigation into influence of negative attitudes toward robots on human–robot interaction. AI & Society, 20(2), 138150. https://doi.org/10.1007/s00146-005-0012-7.
Nomura, T. T., Syrdal, D. S., & Dautenhahn, K. (2015). Differences on social acceptance of humanoid robots between Japan and the UK. In Proceedings of 4th international symposium on new frontiers in human–robot interaction. The Society for the Study of Artificial Intelligence and the Simulation of Behaviour (AISB), Canterbury, United Kingdom (pp. 115–120).
Oberman, L., McCleery, J., Ramachandran, V., & Pineda, J. (2007). EEG evidence for mirror neuron activity during the observation of human and robot actions: Toward an analysis of the human qualities of interactive robots. Neurocomputing, 70(1315), 21942203. https://doi.org/10.1016/j.neucom.2006.02.024.
Pavlova, M. A., Krägeloh-Mann, I., Birbaumer, N., & Sokolov, A. (2002). Biological motion shown backwards: The apparent-facing effect. Perception, 31(4), 435443. https://doi.org/10.1068/p3262.
Press, C., Bird, G., Flach, R., & Heyes, C. (2005). Robotic movement elicits automatic imitation. Cognitive Brain Research, 25(3), 632640. https://doi.org/10.1016/j.cogbrainres.2005.08.020.
Press, C., Gillmeister, H., & Heyes, C. (2006). Bottom-up, not top-down, modulation of imitation by human and robotic models. European Journal of Neuroscience, 24(8), 2415–2419. https://doi.org/10.1111/j.1460-9568.2006.05115.x.
Pulvermüller, F. (2005). Brain mechanisms linking language and action. Nature Reviews. Neuroscience, 6(7), 576582. https://doi.org/10.1038/nrn1706.
Ranzini, M., Borghi, A. M., & Nicoletti, R. (2011). With hands I do not centre! Action- and object-related effects of hand-cueing in the line bisection. Neuropsychologia. https://doi.org/10.1016/j.neuropsychologia.2011.06.019.
Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169192. https://doi.org/10.1146/annurev.neuro.27.070203.144230.
Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews. Neuroscience, 2(9), 661670. https://doi.org/10.1038/35090060.
Stenzel, A., Chinellato, E., Tirado Bou, M. A., del Pobil, A. P., Lappe, M., & Liepelt, R. (2012). When humanoid robots become human-like interaction partners: Corepresentation of robotic actions. Journal of Experimental Psychology: Human Perception and Performance, 38(5), 1073–1077. https://doi.org/10.1037/a0029493.
Tai, Y., Scherfler, C., Brooks, D., Sawamoto, N., & Castiello, U. (2004). The human premotor cortex is “Mirror” only for biological actions. Current Biology, 14(2), 117120. https://doi.org/10.1016/j.cub.2004.01.005.
Wykowska, A., Wiese, E., Prosser, A., & Müller, H. J. (2014). Beliefs about the minds of others influence how we process sensory information. PLoS One, 9(4), e94339. https://doi.org/10.1371/journal.pone.0094339.
Zwaan, R. A., & Taylor, L. J. (2006). Seeing, acting, understanding: Motor resonance in language comprehension. Journal of Experimental Psychology. General, 135(1), 111. https://doi.org/10.1037/0096-3445.135.1.1.
Acknowledgements
This work was supported by the Japan Society for the Promotion of Science. Experiment 1 was conducted during an in-doc by Sophie-Anne Beauprez at the University of Tokyo. We would like to thank Yoshida Fumiaki and Masaoka Shiori for their help in the experiment development, the recruitment of the participants, and data collection.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
The experiments were conducted in accordance with the ethical standards of the institutional and with the 1964 Helsinki declaration.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Prime video and list of verbs
Actions of the videos (English translation/French/Japanese) | Action verbs | Non-action verbs | |
|---|---|---|---|
Congruent | Incongruent | ||
Acquiesce Acquiescer/うなずく | Acquiesce | Read | Want |
Applaud Applaudir/たたく | Applaud | Move back | Dream |
Clean Nettoyer/ふく | Clean | Take | Wish |
Deny Nier/くびをふる | Deny | Throw (a ball) | Recognize |
Kneel S’agenouiller/しゃがむ | Kneel | Scratch | Hope |
Move back Reculer/さがる | Move back | Turn | Envy |
Read Lire/みる | Read | Deny | Progress |
Reverence S’incliner/おじぎする | Reverence | Throw (in a bin) | Believe |
Salute Saluer/てをふる | Salute | Stand up | Cost |
Scratch Gratter/かく | Scratch | Salute | Understand |
Show Montrer/ゆびさす | Show | Acquiesce | Guess |
Stand up Se lever/たつ | Stand up | Show | Choose |
Take Prendre/とる | Take | Kneel | Doubt |
Throw (in the bin) Jeter/すてる | Throw | Clean | Consider |
Throw (a ball) Lancer/なげる | Throw | Reverence | Think |
Turn Tourner/まわる | Turn | Applaud | Have |
Appendix B: Examples of frames, “cleaning” action performed by the robotic and the human agent
Rights and permissions
About this article
Cite this article
Beauprez, SA., Bidet-Ildei, C. & Hiraki, K. Does watching Han Solo or C-3PO similarly influence our language processing?. Psychological Research 84, 1572–1585 (2020). https://doi.org/10.1007/s00426-019-01169-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-019-01169-3