Working memory in developing and applying mental models from spatial descriptions

Abstract

Four dual-task experiments examined visuospatial, articulatory, and central executive working memory involvement during the development and application of spatial mental models. In Experiments 1 and 2 participants read route and survey spatial descriptions while undertaking one of four secondary tasks targeting working memory components. Converging evidence from map drawing and statement verification tasks indicates that while articulatory mechanisms are involved in processing the language itself, visuospatial and central executive mechanisms are involved in developing spatial mental models, particularly during route description reading. In Experiments 3 and 4 participants undertook the same working memory tasks, but did so during testing; results from memory and secondary task performance converge to demonstrate that using spatial mental models is a visuospatially and centrally demanding process, particularly following route description learning. Taken together, results demonstrate that spatial mental model development and application are contingent upon multiple working memory systems and interact with representational formats.

Introduction

Readers construct cohesive mental models of what a text describes, integrating time, space, causality, intention, and person- and object-related information. That is, readers progress beyond the text itself to represent the described situation (Bransford et al., 1972, Glenberg et al., 1994, Graesser et al., 1997, Johnson-Laird, 1983, van Dijk and Kintsch, 1983, Zwaan et al., 1995, Zwaan and Radvansky, 1998). Most work investigating situation models comes from narrative discourse (e.g., Zwaan et al., 1998, Zwaan et al., 1995). Additional work has examined mental model formation from reading non-spatial and spatial expository texts (Ferguson and Hegarty, 1994, Glenberg and Langston, 1992, Graesser and Bertus, 1998, Lee and Tversky, 2005, Millis et al., 1993, Noordzij and Postma, 2005, Perrig and Kintsch, 1985, Taylor and Tversky, 1992a). Whereas both narrative and expository texts lead to mental model development, it is unclear which working memory mechanisms are responsible for constructing these mental representations and when they may play a role (i.e., de Vega, 1995, Radvansky and Copeland, 2001, Radvansky and Copeland, 2004a, Radvansky and Copeland, 2004b, Radvansky and Copeland, 2006a, Radvansky and Copeland, 2006b, Zwaan and Radvansky, 1998). The present experiments examine this question with spatial descriptions and investigate working memory mechanisms (visuospatial, phonological, central executive; Baddeley, 1992, Baddeley, 2002) involved in mental model formation during reading, and mental model application at test.

The Event Indexing model (Zwaan et al., 1995) proposes that discourse events are indexed along at least five dimensions (temporal, spatial, causal, intentionality, agent- and object-based features) and that each plays a role in comprehension. Situation models (i.e., Glenberg et al., 1994, Graesser et al., 1997, Johnson-Laird, 1983, van Dijk and Kintsch, 1983) are built from interactions amongst these indexes. Recent extensions of the model propose four classes of processes operating on these representations: construction, updating, retrieval, and foregrounding (Zwaan & Radvansky, 1998). Construction involves building a situation model during comprehension, updating is modifying existing models with new information, retrieval is extracting information from models, and foregrounding is maintenance of model information in working memory. The Resonance Model (Myers and O’Brien, 1998, O’Brien and Myers, 1999), in contrast, argues that mental model construction is mediated by the automatic resonance of discourse-level to sentence-level relations, and that updating involves the interplay between these two levels, resulting in differential item interrelatedness in memory. Resonance theory maintains that retrieval varies as a function of this interrelatedness (see also the C-I model, Kintsch, 1988, and the Landscape Model, Linderholm, Virtue, Tzeng, & van den Broek, 2004 ). The present work focuses on the working memory processes involved during construction and retrieval of situation models during spatial description reading and testing. Constructivist situation model theory predicts active working memory involvement in tracking multiple text dimensions (i.e., Kintsch, 1988, Zwaan and Radvansky, 1998), while network-based theory (i.e., Myers & O’Brien, 1998) predicts less differential working memory involvement in the activation and linking of current and existing contextual features of a discourse.

Spatial descriptions convey geographical information through language, and generally adopt a particular perspective. Survey descriptions represent space from an allocentric (“birds-eye”) perspective, use an extrinsic reference frame (i.e., no implied viewer), and convey directions in cardinal terms (i.e., north, south, east, west). In contrast, route descriptions typically represent space from an egocentric (“first-person”) perspective, use an intrinsic reference frame (e.g., in front of you, to your right, to your left), and convey information regarding relevant landmarks, turn sequences, and travel distances (Levelt, 1982, Taylor and Tversky, 1992b). Both perspectives are commonly used, for instance, when giving directions or describing an environment’s layout.

Spatial descriptions are quite effective at conveying spatial information. Taylor and Tversky (1992a) found that mental models derived from both route and survey spatial descriptions were similar to those acquired during map study: after studying each, participants were equally adept at sketching maps and verifying inference statements. It appears that participants were able to develop spatial mental models that are not inextricably tied to the learned perspective (see also Brunyé and Taylor, 2007, Ferguson and Hegarty, 1994, Lee and Tversky, 2005, Noordzij and Postma, 2005). In contrast, some recent work shows that spatial memory may be strongly tied the initial learning format (Shelton & McNamara, 2004). In a series of experiments using route and survey texts and videos, Shelton and McNamara found that participants adhere to a principle reference vector (i.e., first path segment of a route, or north as up with survey) for scene recognition. Thus, spatial memory likely preserves orientation specificity based on a principle reference vector that is defined by certain description elements. Flexible inferencing, in contrast, may be strongly tied to the availability of a spatial mental model. Further insights into the nature of spatial memory can be obtained by detailing the time course by which people develop these models, the processes underlying their development, and the format of the resultant representation.

Recent work has demonstrated the relative difficulty of developing spatial mental models from route versus survey descriptions (Brunyé and Taylor, 2007, Lee and Tversky, 2005, Noordzij and Postma, 2005). With limited exposure (i.e., a single read) to route and survey descriptions, spatial mental models are more likely to develop from the latter; in contrast, with repeated exposure operationalized as three read cycles, spatial mental models develop with either description perspective (Brunyé & Taylor, 2007). However, even with this repeated exposure, reading times suggest greater difficulty processing route descriptions. This difficulty may be due to at least the following: first, route descriptions present spatial information embedded within a sequential framework that requires updating relative to a principle reference vector defined by the initial path segment (i.e., Shelton & McNamara, 2004); second, route descriptions may demand a high degree of complex (i.e., 3D) mental imagery as a reader imagines moving through the environment (i.e., Fincher-Kiefer, 2001); finally, landmark interrelationships must be inferred (rather than directly acquired) from route descriptions (i.e., Tversky, 1993).

A fundamental question for discourse research is whether mental models are constructed during comprehension or later during application. Much recent work suggests that spatial information may only be tracked in the context of relevant cues such as reading goals, temporal shifts, and causal relatedness (e.g, de Vega, 1995, Jahn, 2004, Levine and Klin, 2001, Magliano et al., 2001, Magliano et al., 2005, Morrow et al., 1987, Rapp and Taylor, 2004, Rich and Taylor, 2000, Zwaan et al., 1995, Zwaan et al., 1998). Thus, mental models containing comprehensive spatial information may only develop under very limited circumstances.

Spatial descriptions provide a special case for text comprehension models, in that they differ from narrative discourse with their focus on spatial relation information (e.g., landmark interrelationships) and omission of causality, intentionality, and explicit agent-based information. Recent work has demonstrated the utility of discourse models in explaining spatial and non-spatial expository text comprehension (i.e., Cote et al., 1998, Graesser and Bertus, 1998, Taylor and Tversky, 1992a). When reading spatial descriptions there is a primary implied goal: to understand an environment’s elemental interrelationships. This is supported by work showing that spatial mental models develop without the influence of explicit goals or causal relatedness cues (e.g., Ferguson and Hegarty, 1994, Lee and Tversky, 2005, Taylor and Tversky, 1992a, Taylor and Tversky, 1992b).

An underlying assumption of the present work, therefore, is that spatial inferences may be formed spontaneously during spatial description reading as a consequence of at least two mechanisms. First, by making spatial information exceedingly salient, spatial descriptions implicitly establish the goal of tracking and drawing inferences about spatial relationships (i.e., Levine and Klin, 2001, Taylor et al., 1999, van den Broek et al., 2001). Second, having few dimensions (i.e., spatial, temporal), the spatial descriptions prioritize the allocation of cognitive resources towards tracking spatial information (i.e., Estevez and Calvo, 2000, Linderholm and van den Broek, 2002, Morra, 2001). An accurate spatial mental model results from actively tracking and representing spatial relationships (Morrow, 1994, Rinck et al., 1997). In our view, readers of spatial descriptions actively monitor spatial relation information, make certain inferences about relationships that are not explicit, and develop a spatial mental model. These models are abstractions that go beyond what is read, and they are perspective-flexible in that people use them to think about environments from different perspectives. The present work therefore adopts existing operational definitions of spatial mental models (i.e., Gyselinck et al., 2007, Pazzaglia et al., 2007, Taylor and Tversky, 1992a, Tversky, 1991, Tversky, 1993). Note that these models are quite different from the notions of a cognitive map (i.e., Tolman, 1948) or a cognitive collage (i.e., Tversky, 1993). The first is a perspective-inflexible representation that preserves a map-like allocentric structure in memory, and the second is a multi-perspective but incomplete representation and a potential precursor to a spatial mental model.

We explore the development and use of these models to elucidate when (i.e., reading, application) and how (i.e., which working memory mechanisms) they develop, and what form they take in memory.

The present studies address these goals using a selective interference paradigm during either reading or memory application. We examine three working-memory subsystems: the visuospatial sketchpad, articulatory rehearsal loop, and the central executive (i.e., Baddeley, 1992, Baddeley, 2002; see also the episodic buffer, Baddeley, 2000, which is not examined here). Selective interference paradigms typically involve suppressing one of these working memory subsystems, with any observed memory deficits implicating involvement of that mechanism towards learning.

The visuospatial sketchpad appears to be involved in processing object-based visual features such as found in picture-based procedures, diagrams, and maps (e.g., Brunyé et al., 2006, Garden et al., 2001, Gyselinck et al., 2002, Kruley et al., 1994, Logie, 1995), locations and movements in space (De Beni, Pazzaglia, Gyselinck, & Meneghetti, 2005), and spatial visualization and mental imagery (Farmer et al., 1986, Miyake et al., 2001). Investigating visuospatial involvement during spatial description reading allows us to assess the extent to which readers actively track spatial information during reading (i.e., Zwaan & Radvansky, 1998), and how this tracking may translate into spatial mental model development.

The articulatory component of working memory is generally involved in processing verbal information across the auditory (Baddeley et al., 1984, De Beni et al., 2005, Longoni et al., 1993), visual (Brunyé et al., 2006, Farmer et al., 1986, Goldman and Healy, 1985), and even tactile modalities (Millar, 1990). Using an articulatory secondary task during reading allows us to assess the verbal processes involved in developing spatial mental models, and the extent to which these processes (i.e., developing a propositional base; van Dijk & Kintsch, 1983) lay a foundation for spatial mental model development.

The central executive is relatively less well-understood, but is thought to involve supervisory control of working memory subsystems (Baddeley, 1996, Baddeley et al., 1998, Duff, 2000). The central executive has been implicated in: coordinating performance between two separate tasks or information formats (e.g., Brunyé et al., 2006, Della Sala et al., 1995, Duff, 2000, Duff and Logie, 2001, Gyselinck et al., 2002), generating random sequences (e.g., Baddeley, 1996, Baddeley et al., 1998, Brunyé et al., 2006), inferencing and analogical reasoning (Morrison, 2004), attending to one and inhibiting disruption of another stimulus (e.g., Baddeley, 2002), and temporal tagging (Miyake and Shah, 1999, Miyake et al., 2000). Using central executive tasks during reading allows us to assess their involvement in sequential processing, and the allocation of resources between other subsystems. In turn, it can provide insights into the nature of spatial mental models.

Recent work has demonstrated minimal influence of working memory capacity towards situation model updating (Radvansky & Copeland, 2001), relatively greater involvement in the initial development of situation models (Radvansky and Copeland, 2006a, Radvansky and Copeland, 2006b), and important roles for integrative processes and drawing inferences towards successful narrative discourse comprehension (Radvansky and Copeland, 2004a, Radvansky and Copeland, 2004b). We extend this work by selectively and individually examining working memory subsystems, complementing this group’s findings with general span measures. Specifically, composite span indicators that consider multiple subsystem span test scores may limit conclusions with regard to individual subsystem contributions during reading and retrieval. In line with this work, we propose that working memory will be primarily involved during the initial development of situation models during reading (i.e., Radvansky and Copeland, 2006a, Radvansky and Copeland, 2006b), less involved in the direct retrieval of these models from long term memory (i.e., Pazzaglia et al., 2007, Radvansky and Copeland, 2006a, Radvansky and Copeland, 2006b), and recruited when inference processes are demanded by the task (i.e., Radvansky & Copeland, 2004b). These hypotheses are in contrast to the notion that the automatic resonance of sentence- and discourse-level information can proceed without high working memory demands (e.g., Myers and O’Brien, 1998, O’Brien and Myers, 1999).