Abstract
In this research, anticipatory perception of an approaching vehicle was examined. By varying the availability of visual and acoustic media, conditions occurred in which the input to the modalities was repetitively interrupted while the presentation of the approach event continued. In these conditions, the audio and the visual signals were presented either in short and concurrent segments at regular intervals or in alternation (interleaved) at regular intervals. It was found that interrupting the signal within a single modality did not affect performance if the approach information was available in the alternate modality. These results are consistent with a modally flexible detection mechanism for the perception of approaching objects. This modal flexibility may provide some evidence that information is detected using a modality-neutral strategy.
Article PDF
Similar content being viewed by others
References
Andersen, G. J., Saidpour, A., &Braunstein, M. L. (1998). Effects of collimation on perceived layout in 3-D scenes.Perception,11, 1305–1315.
Anderson, A. J., &Vingrys, A. J. (2000). Interactions between flicker thresholds and luminance pedestals.Vision Research,40, 2579–2588.
Barnes, R., &Jones, M. R. (2000). Expectancy, attention, and time.Cognitive Psychology,41, 254–311.
Bernstein, L. E., Auer, E. T., Moore, J. K., Ponton, C., Don, M., &Singh, M. (2002). Visual speech perception without primary cortex activation.NeuroReport,13, 311–315.
Blauert, J. (1999). Progress and trends since 1982. In J. Blauert,Spatial hearing: The psychophysics of human sound localization (J. S. Allen, Trans.) (pp. 369–409). Cambridge, MA: MIT Press. (Original work published 1997)
Bülthoff, H. H., &Mallot, H. A. (1988). Integration of depth modules: Stereo and shading.Journal of the Optical Society of America A,5, 1749–1758.
Cabe, P. A., &Pittenger, J. B. (2000). Human sensitivity to acoustic information from vessel filling.Journal of Experimental Psychology: Human Perception & Performance,26, 313–324.
Caelli, T., &Porter, D. (1980). On difficulties in localizing ambulance sirens.Human Factors,22, 719–724.
Calvert, G. A., Bullmore, E., Brammer, M. J., Campbell, R., Iversen, S. D., Woodruff, P., McGuire, P., Williams, S., &David, A. S. (1997). Silent lipreading activates the auditory cortex.Science,276, 593–596.
Cavallo, V., &Laurent, M. (1988). Visual information and skill level in time-to-collision estimation. Perception,17, 623–632.
Chase, W. D. (1970). Evaluation of several TV display system configurations for visual simulation of the landing approach.IEEE Transactions on Man-Machine Systems,11, 140–149.
Chen, Z. (1998). Switching attention within and between objects: The role of subjective organization.Canadian Journal of Experimental Psychology,52, 7–16.
DeLucia, P. R., Kaiser, M. K., Bush, J. M., Meyer, L. E, &Sweet, B. T. (2003). Information integration in judgements of time to contact.Quarterly Journal of Experimental Psychology,56A, 1165–1189.
DeLucia, P. R., &Liddell, G. W. (1998). Cognitive motion extrapolation and cognitive clocking in prediction motion.Journal of Experimental Psychology: Human Perception & Performance,24, 901–914.
DeLucia, P. R., &Novak, J. B. (1997). Judgments of relative time-tocontact of more than two approaching objects: Toward a method.Perception & Psychophysics,59, 913–928.
Eimer, M. (2001). Crossmodal links in spatial attention between vision, audition, and touch: Evidence from event-related brain potentials.Neuropsychologia,39, 1292–1303.
Fowler, C. A., &Dekle, D. J. (1991). Listening with eye and hand: Cross-modal contributions to speech perception.Journal of Experimental Psychology: Human Perception & Performance,17, 816–828.
Gibson, J. J. (1966).The senses considered as perceptual systems. Boston: Houghton Mifflin.
Gibson, J. J. (1979).The ecological approach to visual perception. Hillsdale, NJ: Erlbaum.
Gilkey, R. H., &Anderson, T. R. (1997).Binaural and spatial hearing in real and virtual environments. Mahwah, NJ: Erlbaum.
Gordon, M. S. (2002).Audiovisual time-to-arrival judgments: Testing the primacy of multimodal integration. Unpublished doctoral dissertation, University of California, Riverside.
Gordon, M. S. (2003).Multimodal enhancement of audiovisual timeto-arrival judgments. Paper presented at the meeting of the 4th International Multisensory Research Forum, Hamilton, Ontario.
Gray, R., &Regan, D. (1998). Accuracy of estimating time to collision using binocular and monocular information.Vision Research,38, 499–512.
Gray, R., &Regan, D. (1999). Do monocular time-to-collision estimates necessarily involve perceived distance?Perception,28, 1257–1264.
Green, K. P. (1998). The use of auditory and visual information during phonetic processing: Implications for theories of speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.),Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 3–25). Hove, U.K.: Psychology Press.
Guski, R. (1992). Acoustic tau: An easy analogue to visual tau?Ecological Psychology,4, 189–197.
Hancock, P. A., &Manser, M. P. (1997). Time-to-contact: More than tau alone.Ecological Psychology,9, 265–297.
Hancock, P. A., &Manser, M. P. (1998). Time-to-contact. In A.-M. Feyer & A. Williamson (Eds.),Occupational injury: Risk, prevention, and intervention (pp. 44–58). Bristol, PA: Taylor & Francis.
Hawley, M. L., Litovsky, R. Y., &Colburn, H. S. (1999). Speech intelligibility and localization in a multi-source environment.Journal of the Acoustical Society of America,105, 3436–3448.
Heuer, H. (1993). Estimates of time to contact based on changing size and changing target vergence.Perception,22, 549–563.
Hock, H. S., Kogan, K., &Espinoza, J. K. (1997). Dynamic, statedependent thresholds for the perception of single-element apparent motion: Bistability from local cooperativity.Perception & Psychophysics,59, 1077–1088.
Jacobs, R. A., &Fine, I. (1999). Experience-dependent integration of texture and motion cues to depth.Vision Research,39, 4062–4075.
Jenison, R. L. (1997). On acoustic information for motion.Ecological Psychology,9, 131–151.
Kaiser, M. K., &Hecht, H. (1995). Time-to-passage judgments in nonconstant optical flow fields.Perception & Psychophysics,57, 817–825.
Kaiser, M. K., &Mowafy, L. (1993). Optical specification of time-topassage: Observers’ sensitivity to global tau.Journal of Experimental Psychology: Human Perception & Performance,19, 1028–1040.
Kuhl, P. K., &Meltzoff, A. N. (1984). The intermodal representation of speech in infants.Infant Behavior & Development,7, 361–381.
Ladavas, E., Pavani, F., &Farne, A. (2001). Auditory peripersonal space in humans: A case of auditory-tactile extinction.Neurocase,7, 97–103.
Landy, M. S., Maloney, L. T., Johnston, E. B., &Young, M. (1995). Measurement and modeling of depth cue combination: In defense of weak fusion.Vision Research,35, 389–412.
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y.-F., Field, A. S., &Stein, B. E. (2002). Deactivation of sensory-specific cortex by cross-modal stimuli.Journal of Cognitive Neuroscience,14, 420–429.
Lee, D. N. (1976). A theory of visual control of braking based on information about time-to-collision.Perception,5, 437–459.
Lee, D. N. (1990). Getting around with light or sound. In R. Warren & A. H. Wertheim (Eds.),Perception and control of self-motion (pp. 487–505). Hillsdale, NJ: Erlbaum.
Lee, D. N., van der Weel, F. R., Hitchcock, T., Matejowski, E., &Pettigrew, J. D. (1992). Common principle of guidance by echolocation and vision.Journal of Comparative Physiology A,171, 563–571.
Lewkowicz, D. J., &Turkewitz, G. (1980). Cross-modal equivalence in early infancy: Auditory-visual intensity matching.Developmental Psychology,16, 597–607.
Lutfi, R. A., &Wang, W. (1999). Correlational analysis of acoustic cues for the discrimination of auditory motion.Journal of the Acoustical Society of America,106, 919–928.
Macaluso, E., Frith, C. D., &Driver, J. (2000). Modulation of human visual cortex by crossmodal spatial attention.Science,289, 1206–1208.
MacSweeney, M., Amaro, E., Calvert, G. A., &Campbell, R. (2000). Silent lipreading in the absence of scanner noise: An event-related fMRI study.NeuroReport,11, 1729–1733.
MacSweeney, M., Calvert, G. A., Campbell, R., McGuire, P. K., David, A. S., Williams, S. C., Woll, B., &Brammer, M. J. (2002). Speechreading circuits in people born deaf.Neuropsychologia,40, 801–807.
Marks, L. E. (1978).The unity of the senses. New York: Academic Press.
Massaro, D. W. (1992). Broadening the domain of the fuzzy logical model of perception. In H. L. Pick, Jr., & P.W. van den Broek (Eds.),Cognition: Conceptual and methodological issues (pp. 51–84). Washington, DC: American Psychological Association Press.
Massaro, D. W. (1998).Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
Massaro, D. W., Cohen, M. M., Gesi, A., Heredia, R., &Tsuzaki, M. (1993). Bimodal speech perception: An examination across languages.Journal of Phonetics,21, 445–478.
Maunsell, J. H. R., Nealy, T. A., Sclar, G., &DePriest, D. D. (1989). Representation of extraretinal information in monkey visual cortex. In D. M.-K. Lam & C. D. Gilbert (Eds.),Neural mechanisms of visual perception: Proceedings of the Retina Research Foundation Symposia. The Woodlands, TX: Portfolio Publishing.
Maurer, D. (1993). Neonatal synesthesia: Implication for the processing of speech and faces. In B. de Boysson-Bardies, S. de Schonen, P. Jusczyk, & J. Morton (Eds.),Developmental neurocognition: Speech and face processing in the first year of life (pp. 109–124). Dordrecht: Kluwer.
McLeod, R. W., &Ross, H. E. (1983). Optic-flow and cognitive factors in time-to-collision estimates.Perception,12, 417–423.
Minnaar, P., Olesen, S. K., Christensen, F., &Møller, H. (2001). Localization with binaural recordings from artificial and human heads.Journal of the Audio Engineering Society,49, 323–336.
Morrongiello, B. A., &Fenwick, K. D. (1991). Infants’ coordination of auditory and visual depth information.Journal of Experimental Child Psychology,52, 277–296.
Munhall, K. G., &Vatikiotis-Bateson, E. (1998). The moving face during speech communication. In R. Campbell, B. Dodd, & D. Burnham (Eds.),Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 123–139). Hove, U.K.: Psychology Press.
Neuhoff, J. G. (2001). An adaptive bias in the perception of looming auditory motion.Ecological Psychology,13, 87–110.
Pavani, F., Meneghello, F., &Ladavas, E. (2001). Deficit of auditory space perception in patients with visuospatial neglect.Neuropsychologia,39, 1401–1409.
Petersik, J. T. (1989). The two-process distinction in apparent motion.Psychological Bulletin,106, 107–127.
Pickens, J. (1994). Perception of auditory-visual distance relations by 5-month-old infants.Developmental Psychology,30, 537–544.
Pressing, J. (1998). Error correction processes in temporal pattern production.Journal of Mathematical Psychology,42, 63–101.
Purdy, W. C. (1958). The hypothesis of psychophysical correspondence in space perception (Doctoral dissertation, Cornell University, 1958).Dissertation Abstracts International,19, 1454–1455.
Regan, D., &Beverley, K. I. (1979). Binocular and monocular stimuli for motion in depth: Changing-disparity and changing-size feed the same motion-in-depth stage.Vision Research,19, 1331–1342.
Rosenblum, L. D. (1993). Acoustical information for controlled collisions. In A. Schick (Ed.),Contributions to psychological acoustics (pp. 303–322). Oldenburg, Germany: Bibliotheks.
Rosenblum, L. D. (2004). Perceiving articulatory events: Lessons for an ecological psychoacoustics. In J. G. Neuhoff (Ed.),Ecological psychoacoustics (pp. 219–248). Amsterdam: Elsevier.
Rosenblum, L. D. (2005). The primacy of multimodal speech perception. In D. Pisoni & R. Remez (Eds.),Handbook of speech perception (pp. 51–78). Malden, MA: Blackwell.
Rosenblum, L. D., Carello, C., &Pastore, R. E. (1987). Relative effectiveness of three stimulus variables for locating a moving sound source.Perception,16, 175–186.
Rosenblum, L. D., &Gordon, M. S. (2001). The generality of specificity: Some lessons from audiovisual speech.Behavioral & Brain Sciences,24, 239–240.
Rosenblum, L. D., Gordon, M. S., &Jarquin, L. (2000). Echolocation by moving and stationary listeners.Ecological Psychology,12, 181–206.
Rosenblum, L. D., Gordon, M. S., &Wuestefeld, A. P. (2000). Effects of performance feedback and feedback withdrawal on auditory looming perception.Ecological Psychology,12, 273–291.
Rosenblum, L. D., &Saldaña, H. M. (1998). Time-varying information for visual speech perception. In R. Campbell, B. Dodd, & D. Burnham (Eds.),Hearing by eye II: Advances in the psychology of speechreading and auditory-visual speech (pp. 61–81). Hove, U.K.: Psychology Press.
Rosenblum, L. D., Schmuckler, M. A., &Johnson, J. A. (1997). The McGurk effect in infants.Perception & Psychophysics,59, 347–357.
Rosenblum, L. D., Wuestefeld, A. P., &Saldaña, H. M. (1993). Auditory looming perception: Influences on anticipatory judgments.Perception,22, 1467–1482.
Rosenblum, L. D., Yakel, D. A., Baseer, N., Panchal, A., Nordarse, B. C., &Niehus, R. P. (2002). Visual speech information for face recognition.Perception & Psychophysics,64, 220–229.
Schiff, W., &Detwiler, M. L. (1979). Information used in judging impending collision.Perception,8, 647–658.
Schiff, W., &Oldak, R. (1990). Accuracy of judging time to arrival: Effects of modality, trajectory, and gender.Journal of Experimental Psychology: Human Perception & Performance,16, 303–316.
Scholl, B. J., &Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion: Clues to visual objecthood.Cognitive Psychology,38, 259–290.
Shams, L., Kamitani, Y., Thompson, S., &Shimojo, S. (2001). Sound alters visual evoked potentials in humans.NeuroReport,12, 3849–3852.
Shaw, B. K., McGowan, R. S., &Turvey, M. T. (1991). An acoustic variable specifying time-to-contact.Ecological Psychology,3, 253–261.
Shimojo, S., &Shams, L. (2001). Sensory modalities are not separate modalities: Plasticity and interactions.Current Opinion in Neurobiology,11, 505–509.
Sidaway, B., Fairweather, M., Sekiya, H., &McNitt-Ray, J. (1996). Time-to-collision estimate in a simulated driving task.Human Factors,38, 101–114.
Stein, B. E., &Meredith, M. A. (1993).The merging of the senses. Cambridge, MA: MIT Press.
Stoffregen, T. A., &Pittenger, J. B. (1995). Human echolocation as a basic form of perception and action.Ecological Psychology,7, 181–216.
Summerfield, A. Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.),Hearing by eye: The psychology of lip-reading (pp. 3–51). Hillsdale, NJ: Erlbaum.
Takahashi, M., &Maruyama, K. (1977). Visual masking function to flickering stimulus in “blank-flick” experiment.Tohoku Psychologica Folia,36, 134–137.
Tambiev, A. E., Medvedev, S. D., &Tepina, M. V. (1993). EEGcorrelates of switching of sensory attention.Psikhologicheskiy Zhurnal,14, 65–70.
Tresilian, J. R. (1991). Empirical and theoretical issues in the perception of time to contact.Journal of Experimental Psychology: Human Perception & Performance,17, 865–876.
Turner, J., Braunstein, M. L., &Andersen, G. J. (1997). Relationship between binocular disparity and motion parallax in surface detection.Perception & Psychophysics,59, 370–380.
Wickens, C. D. (2002). Multiple resources and performance prediction.Theoretical Issues in Ergonomics Science,3, 159–177.
Wickens, C. D., Sandry, D., &Vidulich, M. (1983). Compatibility and resource competition between modalities of input, output, and central processing.Human Factors,25, 227–248.
Wiegrebe, L., &Krumbholz, K. (1999). Temporal resolution and temporal masking properties of transient stimuli: Data and an auditory model.Journal of the Acoustical Society of America,105, 2746–2756.
Yost, W. A., Dye, R. H., Jr., &Sheft, S. (1996). A simulated “cocktail party” with up to three sound sources.Perception & Psychophysics,58, 1026–1036.
Young, M. J., Landy, M. S., &Maloney, L. T. (1993). A perturbation analysis of depth perception from combinations of texture and motion cues.Vision Research,33, 2685–2696.
Zangaladze, A., Epstein, C. M., Grafton, S. T., &Sathian, K. (1999). Involvement of visual cortex in tactile discrimination of orientation.Nature,401, 587–590.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was funded by a University of California Graduate Dean’s Dissertation Research Grant, an Intramural Research Grant from the University of California, and National Science Foundation Grant 0111949.
Rights and permissions
About this article
Cite this article
Gordon, M.S., Rosenblum, L.D. Effects of intrastimulus modality change on audiovisual time-to-arrival judgments. Perception & Psychophysics 67, 580–594 (2005). https://doi.org/10.3758/BF03193516
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03193516