Abstract
The temporal perception of simple auditory and visual stimuli can be modulated by exposure to asynchronous audiovisual speech. For instance, research using the temporal order judgment (TOJ) task has shown that exposure to temporally misaligned audiovisual speech signals can induce temporal adaptation that will influence the TOJs of other (simpler) audiovisual events (Navarra et al. (2005) Cognit Brain Res 25:499–507). Given that TOJ and simultaneity judgment (SJ) tasks appear to reflect different underlying mechanisms, we investigated whether adaptation to asynchronous speech inputs would also influence SJ task performance. Participants judged whether a light flash and a noise burst, presented at varying stimulus onset asynchronies, were simultaneous or not, or else they discriminated which of the two sensory events appeared to have occurred first. While performing these tasks, participants monitored a continuous speech stream for target words that were either presented in synchrony, or with the audio channel lagging 300 ms behind the video channel. We found that the sensitivity of participant’s TOJ and SJ responses was reduced when the background speech stream was desynchronized. A significant modulation of the point of subjective simultaneity (PSS) was also observed in the SJ task but, interestingly, not in the TOJ task, thus supporting previous claims that TOJ and SJ tasks may tap somewhat different aspects of temporal perception.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Note that both the TOJ and the SJ task are subject to response bias; it is just that the nature of the potential bias affecting the performance measures derived from each task differ. For while response biases will most often influence the PSS derived from TOJ data, response biases are more likely to influence the standard deviation (equivalent to the JND derived from a TOJ task; i.e., as when a participant is biased to assume that the stimuli should go together) in SJ tasks. Hence, the most appropriate task use in any given study will depend on the particular perceptual estimate (i.e., temporal discrimination accuracy vs. PSS) that researchers happen to be most interested in theoretically.
References
Alsius A, Navarra J, Campbell R, Soto-Faraco S (2005) Audiovisual integration of speech falters under high attention demands. Curr Biol 15:1–5
Axelrod S, Thompson LW, Cohen LD (1968) Effects of senescence on the temporal resolution of somesthetic stimuli presented to one hand or both. J Gerontol 23:191–195
Baron J (1969) Temporal ROC curves and the psychological moment. Psychon Sci 15:299–300
Coren S, Ward LM, Enns JT (2004) Sensation and perception, 6th edn. Harcourt Brace, Fort Worth
Dixon NF, Spitz L (1980) The detection of auditory visual desynchrony. Perception 9:719–721
Durgin FH, Sternberg S (2002) The time of consciousness and vice versa. Conscious Cogn 11:248–290
Finney DJ (1964) Probit analysis: statistical treatment of the sigmoid response curve. Cambridge University Press, London
Frey RD (1990) Selective attention, event perception and the criterion of acceptability principle: evidence supporting and rejecting the doctrine of prior entry. Hum Mov Sci 9:481–530
Fujisaki W, Shimojo S, Kashino M, Nishida S (2004) Recalibration of audiovisual simultaneity. Nat Neurosci 7:773–778
Grant KW, Seitz PF (1998) The use of visible speech cues (speechreading) for directing auditory attention: reducing temporal and spectral uncertainty in auditory detection of spoken sentences. In: Kuhl PK, Crum LA (Eds) Proceedings of the 16th international congress on acoustics and the 135th meeting of the acoustical society of America, vol 3. ASA, New York, pp 2335–2336
Hirsh IJ (1959) Auditory perception of temporal order. J Acoust Soc Am 31:759–767
Hirsh IJ, Fraisse P (1964) Simultanéité et succession de stimuli hétérogènes [Simultaneity and succession of heterogeneous stimuli]. L’Année Psychologique 64:1–19
Hirsh IJ, Sherrick Jr CE (1961) Perceived order in different sense modalities. J Exp Psychol 62:423–432
Jaskowski P (1991) Two-stage modal for order discrimination. Percept Psychophys 50:76–82
Juola JF, van Eijk RLJ, Kohlrausch A, van der Par S (2007) Synchrony detection and temporal order judgments for bimodal stimuli. Poster presented at the joint meeting of the Experimental Psychology Society and The Psychonomics Society. Edinburgh, 4–7 July
Kohlrausch A, van de Par S (2005) Audio–visual interactions in the context of multi-media applications. In: Blauert J (Ed) Communication acoustics. Springer, Berlin, pp 109–138
Lavie N (2005) Distracted and confused? Selective attention under load. Trends Cognit Sci 9:75–82
Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech perception: a PET study. Neuroimage 21:725–732
Mitrani L, Shekerdjiiski S, Yakimoff N (1986) Mechanisms and asymmetries in visual perception of simultaneity and temporal order. Biol Cybern 54:159–165
Myung IJ (2003) Tutorial on maximum likelihood estimation. J of Math Psychol 47:90–100
Navarra J, Vatakis A, Zampini M, Humphreys W, Soto-Faraco S, Spence C (2005) Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognit Brain Res 25:499–507
Navarra J, Soto-Faraco S, Spence C (2007) Adaptation to audiotactile asynchrony. Neurosci Lett 413:72–76
Schneider KA, Bavelier D (2003) Components of visual prior entry. Cognit Psychol 47:333–366
Shore DI, Spence C, Klein RM (2001) Visual prior entry. Psychol Sci 12:205–212
Shore DI, Spry E, Spence C (2002) Confusing the mind by crossing the hands. Cognit Brain Res 14:153–163
Shore DI, Gray K, Spry E, Spence C (2005) Spatial modulation of tactile temporal-order judgments. Perception 34:1251–1262
Soto-Farco S, Alsius A (2007) Conscious access to the unisensory components of a cross-modal illusion. Neuroreport 18:347–350
Spence C (2007) Audiovisual multisensory integration. J Acoust Soc Japan Acoust Sci Technol 28:61–70
Spence C, Shore DI, Klein RM (2001) Multisensory prior entry. J Exp Psychol Gen 130:799–832
Stelmach LB, Herdman CM (1991) Directed attention and perception of temporal order. J Exp Psychol Human Percept Perform 17:539–550
Tuomainen J, Andersen TS, Tiippana K, Sams M (2005) Audio-visual speech perception is special. Cognition 96:B13–B22
Van de Par S, Kohlrausch A, Juola JF (2002) Some methodological aspects for measuring asynchrony detection in audio–visual stimuli. In: Calvo Manzano A, Perez Lopez A, Santiago JS (eds) Proceedings of the Forum acusticum, Sevilla, Spain, 16–20 September 2002 (on CD-rom). Forum Acusticum, Sevilla, Spain
Van Eijk PLJ, Kohlrausch A, van de Par S, Juola JF (2006) The influence of psychophysical procedure and stimulus type on estimates of human performance in detecting audio–visual asynchrony. Poster presented at the 7th Annual Meeting of the International Multisensory Research Forum, Dublin, Ireland
Van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 102:1181–1186
Vatakis A, Spence C (2006) Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111:134–142
Vatakis A, Spence C (2007) Crossmodal binding: evaluating the ‘unity assumption’ using audiovisual speech stimuli. Percept Psychophys 69:744–756
Vatakis A, Navarra J, Soto-Faraco S, Spence C (2007) Temporal recalibration during asynchronous audiovisual speech perception. Exp Brain Res 181:173–181
Vroomen J, Keetels M, de Gelder B, Bertelson P (2004) Recalibration of temporal order perception by exposure to audio–visual asynchrony. Cognit Brain Res 22:32–35
Welch RB (1999) Meaning, attention, and the “unity assumption” in the intersensory bias of spatial and temporal perceptions. In: Ashersleben G, Bachmann T, Müsseler J (Eds) Cognitive contributions to the perception of spatial and temporal events. Elsevier Science, B.V, Amsterdam, pp 371–387
Welch RB, Warren DH (1980) Immediate perceptual response to intersensory discrepancy. Psychol Bull 3:638–667
Zampini M, Shore DI, Spence C (2003) Audiovisual temporal order judgments. Exp Brain Res 152:198–210
Zampini M, Guest S, Shore DI, Spence C (2005a) Audio–visual simultaneity judgments. Percept Psychophys 67:531–544
Zampini M, Shore DI, Spence C (2005b) Audiovisual prior entry. Neurosci Lett 381:217–222
Acknowledgments
A.V. was supported by a Newton Abraham Studentship from the Medical Sciences Division, University of Oxford. S.S.-F., J.N., and C.S. were supported by grants from the McDowell-Pew Foundation, University of Oxford and the Ministerio de Educación y Ciencia from Spain (TIN2004-04363-C03-02).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vatakis, A., Navarra, J., Soto-Faraco, S. et al. Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments. Exp Brain Res 185, 521–529 (2008). https://doi.org/10.1007/s00221-007-1168-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-007-1168-9