Abstract
This paper investigates how detailed a linguistic representation is formed for descriptions of visual events. In two experiments, participants watched captioned videos and decided whether the captions accurately described the videos. In both experiments, videos depicted geometric shapes moving around the screen. In the first experiment, all of the captions were active sentences, and in the second experiment, half of the captions were active and half were passive. Results of these experiments indicate that participants who only encountered active sentences performed less detailed analyses of the sentences than participants who encountered both active and passive sentences, suggesting that the level of linguistic detail encoded reflects the complexity of the task that participants have to perform. These results are consistent with “good enough” models of language processing in which people process sentences heuristically or syntactically depending on the nature of the task they must perform.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altmann, G. T. M. (2004). Language-mediated eye-movements in the absence of a visual world: The ‘blank screen paradigm’. Cognition, 93, B79–B87.
Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73, 247–264.
Bever, T. G. (1970). The cognitive basis for linguistic structures. New York: Wiley.
Carpenter, P. A., & Just, M. A. (1975). Sentence comprehension: A psycholinguistic processing model of verification. Psychological Review, 82, 45–73.
Christianson, K., Hollingworth, A., Halliwell, J., & Ferreira, F. (2001). Thematic roles assigned along the garden path linger. Cognitive Psychology, 42, 368–407.
Clark, H. H., & Chase, W. G. (1972). On the process of comparing sentences against pictures. Cognitive Psychology, 3, 472–517.
Dittrich, W., & Lea, S. (1994). Visual perception of intentional motion. Perception, 23, 253–268.
Ferreira, F. (2003). The misinterpretation of noncanonical sentences. Cognitive Psychology, 47, 164–203.
Ferreira, F., Ferraro, V., & Bailey, K. G. D. (2002). Good-enough representations in language comprehension. Current Directions in Psychological Science, 11, 11–15.
Ferreira, F., & Henderson, N. D. (2007). The ‘good enough’ approach to language comprehension. Language and Linguistics Compass, 1, 71–83.
Fisher, C., Hall, D. G., Rakowitz, S., & Gleitman, L. (1994). When it is better to receive than to give: Syntactic and conceptual constraints on vocabulary growth. Lingua, 92, 333–375.
Frazier, L. (1978). On comprehending sentences: Syntactic parsing strategies. Unpublished doctoral dissertation, University of Connecticut, Storrs.
Gao, T., Newman, G. E., & Scholl, B. J. (2009). The psycholphysics of chasing: A case study in the perception of animacy. Cognitive Psychology, 59, 154–179.
Gelman, R., Durgin, F., & Kaufman, L. (1995). Distinguishing between animates and inanimates: Not by motion alone. Oxford: Clarendon Press.
Knoeferle, P., & Crocker, M. (2005). Incremental effects of mismatch during picture-sentence integration: Evidence from eye-tracking. In Proceedings of the 26th annual conference of the cognitive science society (pp. 1166–1171), Stresa, Italy.
Knoeferle, P., Crocker, M., Scheepers, C., & Pickering, M. (2005). The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events. Cognition, 95, 95–127.
Kucera, N., & Francis, W. N. (1967). Computational analysis of present day American English. Providence, RI: Brown University Press.
Lakusta, L., & Landau, B. (2005). The importance of goals in spatial language. Cognition, 96, 1–33.
MacDonald, M. C., Pearlmutter, N. J., & Seidenberg, M. S. (1994). The lexical nature of syntactic ambiguity resolution. Psychological Review, 101, 676–703.
Masson, M. E. J. (2011). A tutorial on a practical bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690.
Raftery, A. E. (1995). Bayesian model selection in social research (pp. 111–196). Cambridge, MA: Blackwell.
Sedivy, J., Tanenhaus, M., Chambers, C., & Carlson, G. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71, 109–148.
Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (1995). The interaction of visual and linguistic information in spoken language comprehension. Science, 268, 1632–1634.
Townsend, D., & Bever, T. G. (2001). Sentence comprehension: The integration of habits and rules. Cambridge, Ma: MIT Press.
Tremoulet, P. D., & Feldman, J. (2000). Perception of animacy from the motion of a single object. Perception, 29, 943–951.
Tremoulet, P. D., & Feldman, J. (2006). The influence of spatial context and the role of intentionality in the interpretation of animacy from motion. Perception and Psychophysics, 68, 1047–1058.
Trueswell, J., Tanenhaus, M., & Garnsey, S. (1994). Semantic influences on parsing: Use of thematic role information in syntactic disambiguation. Journal of Memory and Language, 33, 285–318.
Underwood, G., Jebbett, L., & Roberts, K. (2004). Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. The Quarterly Journal of Experimental Psychology, 56, 165–182.
Acknowledgments
This research was supported by awards from the National Science Foundation (BCS-0446850 and BCS-0124095) to K.S. We thank the audience at the 11th Annual Meeting of the Vision Sciences Society and the 25th Annual CUNY Conference on Human Sentence Processing for helpful discussions on presentations of this work. We especially thank Jacob Feldman, Eileen Kowler, Nora Isacoff, Choonkyu Lee, Sabrina Angelini, Tina Hou-Imerman, Heather Yaden, and Nikhita Karki for their suggestions and advice in all aspects of this work: from designing the stimuli, to writing this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kharkwal, G., Stromswold, K. Good-Enough Language Processing: Evidence from Sentence-Video Matching. J Psycholinguist Res 43, 27–43 (2014). https://doi.org/10.1007/s10936-013-9239-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10936-013-9239-5