Abstract
Any incremental parser must solve two computational problems: (1) maintaining all interpretations consistent with the words that have been processed so far and (2) excluding all globally-incoherent interpretations. While these problems are well understood, it is not clear how the dynamic, continuous mechanisms that underlie human language processing solve them. We introduce a Gradient Symbolic Computation (GSC) parser, a continuous-state, continuous-time stochastic dynamical-system model of symbolic processing, which builds up a discrete symbolic structure gradually by dynamically strengthening a discreteness constraint. Online, interactive tutorials with open-source software are presented in a companion website. Our results reveal that the GSC parser can solve the two computational problems by moving to a non-discrete blend state that evolves exclusively to discrete states representing contextually-appropriate globally-coherent interpretations. In a simulation study using a simple formal grammar, we show that successful parsing requires appropriate control of the discreteness constraint strength (a quantization policy). With inappropriate quantization policies, the GSC parser makes mistakes that mirror those made in natural language comprehension (garden-path or local-coherence errors). These findings suggest that the GSC model offers a neurally plausible solution to these two core problems.
Acknowledgement
We thank Géraldine Legendre, Akira Omaki, Kyle Rawlins, Ben Van Durme, and Colin Wilson for their contributions to this work, and gratefully acknowledge the support of NSF INSPIRE grant BCS-1344269. We thank Paul Tupper for suggesting the form of the HQ function used in this work.
References
Altmann, Gerry T. M & Yuki Kamide. 1999. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 73(3). 247–264.10.1016/S0010-0277(99)00059-1Search in Google Scholar
beim Graben, Peter & Sabrina Gerth. 2012. Geometric representations for minimalist grammars. Journal of Logic, Language and Information 21. 393–432.10.1007/s10849-012-9164-2Search in Google Scholar
beim Graben, Peter, Sabrina Gerth & Shravan Vasishth. 2008. Towards dynamical system models of language-related brain potentials. Cognitive Neurodynamics 2(3). 229–255.10.1007/s11571-008-9041-5Search in Google Scholar
Bever, Thomas G. 1970. The cognitive basis for linguistic structures. In John R. Hayes (eds.), Cognition and the development of language, 279–362. New York: John Wiley.Search in Google Scholar
Borges, Jorge Luis. 1962. The garden of forking paths (1941). In Anthony Kerrigan (eds.), Ficciones. New York, NY: Grove Press.Search in Google Scholar
Carmantini, Giovanni S., Peter beim Graben, Mathieu Desroches & Serafim Rodrigues. 2017. A modular architecture for transparent computation in recurrent neural networks. Neural Networks 85. 85–105.10.1016/j.neunet.2016.09.001Search in Google Scholar
Cho, Pyeong Whan & Paul Smolensky. 2016. Bifurcation analysis of a Gradient Symbolic Computation model of incremental processing. In A. Papafragou, D. Grodner, D. Mirman & J. C. Trueswell (eds.), Proceedings of the 38th annual conference of the Cognitive Science Society. Cognitive Science Society: Austin, TX.Search in Google Scholar
Elman, Jeffrey L. 1990. Finding structure in time. Cognitive Science 14(2). 179–211.10.4324/9781315784779-11Search in Google Scholar
Frazier, Lyn. 1987. Sentence processing: A tutorial review. In M. Coltheart (eds.), Attention and performance XII: The psychology of reading, 559–586. Lawrence Erlbaum Associates: Mahwah, NJ.Search in Google Scholar
Geman, Stuart & Donald Geman. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6(6). 721–741.10.1016/B978-0-08-051581-6.50057-XSearch in Google Scholar
Gibson, Edward. 2006. The interaction of top–down and bottom–up statistics in the resolution of syntactic category ambiguity. Journal of Memory and Language 54(3). 363–388.10.1016/j.jml.2005.12.005Search in Google Scholar
Hale, John. 2001. A probabilistic Earley parser as a psycholinguistic model. NAACL '01: Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies. 1–8.10.3115/1073336.1073357Search in Google Scholar
Hale, John. 2011. What a rational parser would do. Cognitive Science 35(3). 399–443.10.1111/j.1551-6709.2010.01145.xSearch in Google Scholar
Hale, John & Paul Smolensky. 2006. Harmonic Grammars and harmonic parsers for formal languages. In Paul Smolensky & Géraldine Legendre (eds.), The harmonic mind: From neural computation to optimality-theoretic grammar. Volume I: Cognitive architecture, 393–416. The MIT Press: Cambridge, MA.Search in Google Scholar
Hockett, Charles F. 1960. The origin of speech. Scientific American 203. 89–96.10.1038/scientificamerican0960-88Search in Google Scholar
Jurafsky, Daniel. 1996. A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science 20(2). 137–194.10.1207/s15516709cog2002_1Search in Google Scholar
Konieczny, Lars. 2005. The psychological reality of local coherences in sentence processing. Proceedings of the 27th annual conference of the cognitive science society . 1178–1183.Search in Google Scholar
Levy, Roger. 2008. Expectation-based syntactic comprehension. Cognition 106(3). 1126–1177.10.1016/j.cognition.2007.05.006Search in Google Scholar
Levy, Roger, Florencia Reali & Thomas L. Griffiths. 2008. Modeling the effects of memory on human online sentence processing with particle filters. In D. Koller, D. Schuurmans, Y. Bengio & L. Bottou (eds.), Proceedings of the 21st international conference on Neural Information Processing Systems, 937–944. Red Hook, NY, USA: Curran Associates Inc.Search in Google Scholar
Levy, Roger, Klinton Bicknell, Tim Slattery & Keith Rayner. 2009. Eye movement evidence that readers maintain and act on uncertainty about past linguistic input. Proceedings of the National Academy of Sciences 106(50). 21086–21090.10.1073/pnas.0907664106Search in Google Scholar
Lewis, Richard L., Shravan Vasishth & Julie A. van Dyke. 2006. Computational principles of working memory in sentence comprehension. Trends in Cognitive Sciences 10(10). 447–454.10.1016/j.tics.2006.08.007Search in Google Scholar
Marr, David. 1982. Vision: A computational investigation into the human representation and processing of visual information. W.H. Freeman: San Francisco.Search in Google Scholar
Smolensky, Paul. 1990. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46(1). 159–216.10.1016/0004-3702(90)90007-MSearch in Google Scholar
Smolensky, Paul & Géraldine Legendre (eds.). 2006. The harmonic mind: From neural computation to optimality-theoretic grammar. Volume I: Cognitive architecture. Cambridge, MA: The MIT Press.Search in Google Scholar
Smolensky, Paul, Matthew Goldrick & Donald Mathis. 2014. Optimization and quantization in gradient symbol systems: a framework for integrating the continuous and the discrete in cognition. Cognitive Science 38(6). 1102–1138.10.1111/cogs.12047Search in Google Scholar
Tabor, Whitney & Sean Hutchins. 2004. Evidence for self-organized sentence processing: Digging-in effects. Journal of Experimental Psychology: Learning, Memory, and Cognition 30(2). 431–450.10.1037/0278-7393.30.2.431Search in Google Scholar
Tabor, Whitney, Cornell Juliano & Michael K. Tanenhaus. 1997. Parsing in a dynamical system: An attractor-based account of the interaction of lexical and structural constraints in sentence processing. Language and Cognitive Processes 12(2–3). 211–271.10.1080/016909697386853Search in Google Scholar
Tabor, Whitney, Bruno Galantucci & Daniel Richardson. 2004. Effects of merely local syntactic coherence on sentence processing. Journal of Memory and Language 50(4). 355–370.10.1016/j.jml.2004.01.001Search in Google Scholar
Vosse, Theo & Gerard Kempen. 2000. Syntactic structure assembly in human parsing: A computational model based on competitive inhibition and a lexicalist grammar. Cognition 75(2). 105–143.10.1016/S0010-0277(00)00063-9Search in Google Scholar
© 2017 Walter de Gruyter GmbH, Berlin/Boston
