Effects of Pointing Gestures on Memory for (In)Congruent Stimuli in Children and Young Adults

We investigated whether finger pointing toward picture locations can be used as an external cognitive control tool to guide attention and compensate for the immature cognitive control functions in children compared with young adults. Item and source memory performance was compared for picture-location pairs that were either semantically congruent (e.g., a cloud presented at the upper half of the screen) or incongruent (e.g., a cloud presented at the lower part of the screen). Contrary to our expectations, pointing had an adverse effect on source memory compared to visual observation only, in both age groups. As expected, superior source memory performance was found for congruent compared to incongruent picture-locations pairs in both age groups. These findings suggest that pointing toward pictures compared to only viewing may hamper memory, and that congruent picture locations are easier to remember than incongruent ones. ABSTRACT— We investigated whether ﬁnger pointing toward picture locations can be used as an external cognitive control tool to guide attention and compensate for the immature cognitive control functions in children compared with young adults. Item and source memory performance was compared for picture-location pairs that were either semantically congruent (e.g., a cloud presented at the upper half of the screen) or incongruent (e.g., a cloud presented at the lower part of the screen). Contrary to our expectations, pointing had an adverse eﬀect on source memory compared to visual observation only, in both age groups. As expected, superior source memory performance was found for congruent compared to incongruent picture-locations pairs in both age groups. These ﬁndings suggest that pointing toward pictures compared to only viewing may hamper memory, and that congruent picture locations are easier to remember than incongruent ones.

features linked to a particular item (Van Petten, Senkfor, & Newberg, 2000). Contextual features congruent to an existing schema are more easily remembered than incongruent features (Brod, Lindenberger, & Shing, 2017). For example, when you always keep your keys in the top drawer in your hallway closet, the information elements "key," "hallway," "closet," "top," and "drawer" will be schematized. When you are in a hurry, and accidentally put the keys somewhere else, information about the location of the keys is incongruent with the existing schema and easily forgotten. Note that in this example, the schema of the location of the keys also involves sensori-motor information.
According to the theory of grounded cognition, perceptual and physical interaction with the world shapes cognition, including memory (for a review, see Barsalou, 2008). These interactions and mental simulations of these interactions are multimodal; apart from the visual modality, they contain relevant motor and mental states that were part of the original experience (Dijkstra & Zwaan, 2015). From this theory, it can be inferred that sensori-motor interaction with stimuli, can enhance (complex) cognitive processes, such as source memory. In support of this claim, a study by Ouwehand, Van Gog, and Paas (2016) showed that pointing at pictures' locations compared with only visually observing them during encoding, improved subsequent source memory for picture-location pairs in young and older adults. Delgado, Gómez, and Sarriá (2011) found that preschool children who had to remember the location of a hidden toy spontaneously pointed at that location more often than those who did not have to remember the location. In a second experiment, children aged between 4 and 6 years were presented with a picture-matching task. In one block, they could point and in the other, pointing was prevented. It was found that restricting children to point had detrimental effects on the performance of children who spontaneously used pointing gestures in the nonrestricted block. The authors explained this finding by suggesting that children who had difficulty with the task tended to rely on pointing gestures, and therefore preventing them to gesture led to worse performance than that of the children who did not spontaneously gesture in the nonrestricted block. In addition, research on the enactment effect (the effect that enacting action phrases compared with only hearing or reading them improve memory) showed that physical interaction with stimuli could enhance episodic memory for action-object associations. Interestingly, Feyereisen (2009) showed superior source memory for congruent (e.g., put money in the wallet) and incongruent action-object associations (e.g., put candy in the wallet) when the action phrases were enacted compared to only verbalized. This led us to our idea that the act of pointing toward picture locations might enhance source memory for semantically congruent and incongruent picture locations.
Neuroscientific research showed that frontal brain areas (especially the dorsolateral prefrontal cortex, DLPFC) involved in source memory processes (Menon, Boyett-Anderson, & Reiss, 2005), and the retrieval of schema-incongruent events (e.g., Brod, Lindenberger, Werkle-Bergner, & Shing, 2015) are maturating until (young) adulthood. It is proposed that the DLPFC, being known for its controlling role in the monitoring of task-relevant information and prepotent encoding responses, can exert top-down control over interference caused by such incongruent events (Ragland et al., 2015). Brod et al. (2017) showed that when retrieving schema-incongruent information, children rely more on hippocampal structures (involved in the integration of features for episodic memory) and young adults more on the DLPFC (and striatum). For the retrieval of congruent events, research showed involvement of the (ventro)medial prefrontal cortex (mPFC) (Brod et al., 2017;Van Kesteren, Rijpkema, Ruiter, & Fernández, 2010;Van Kesteren et al., 2013). Interestingly, Pouw, de Nooijer, van Gog, Zwaan, and Paas (2014) proposed that gestures can support and replace internal cognitive processes and that people especially rely on these gestures when cognitive load is high. In line with this reasoning, we propose that gesturing might be used as an external tool to support the suboptimal (due to immaturity of frontal areas) internal cognitive processes (i.e., source memory for schema-congruent and incongruent information) in children.
From an educational perspective, this knowledge about distinctive developmental trajectories of brain structures and accompanying functions is highly relevant as is the study on gesturing as a learning tool. Earlier evidence has already shown that gesturing can improve children's learning of foreign language (Macedonia & Klimesch, 2014;Macedonia & Knösche, 2011;Toumpaniari, Loyens, Mavilidi, & Paas, 2015), new concepts (e.g., Ping & Goldin-Meadow, 2008;Pouw, Eielts, Van Gog, Zwaan, & Paas, 2016;Valenzeno, Alibali, & Klatzky, 2003) and geography (Mavilidi, Okely, Chandler, & Paas, 2016). These positive effects of gesturing are interesting and might have important practical implications because gestures are easy to implement in the classroom and are free of cost. However, it has not been studied whether pointing gestures can have a beneficial effect on children's source memory and memory for schema-incongruent information (processes that heavily rely on brain areas that are still immature in children). Knowing that memory for schema-incongruent information is underdeveloped compared to that for schema-relevant information leads to the question of whether and how this memory function can be supported. This is educationally relevant because in school it is not only important to learn the main rules about a certain topic (e.g., verb conjugations in learning language grammar), but also the exceptions to these rules (e.g., irregular verb conjugations).

The Present Study
If pointing gestures can function as an external top-down control mechanism, analogous to the internally controlled function of the DLPFC, producing these gestures during the encoding of picture-location pairs might improve the integration between the picture and location and decrease the object-(incongruent)location interference effect. Relating this to the findings of Delgado et al. (2011) that pointing might be especially helpful for children on cognitively demanding tasks, it is also plausible that pointing might decrease such an interference effect, because the more difficult task (encoding the incongruent pairs) would be especially enhanced. If pointing can (partly) compensate for the immature DLFPC function in source memory and memory for schema-incongruent information in children, then this manipulation should lead to higher benefits for children than (young) adults. To test this assumption, item-and source memory performance was compared between children and young adults who either had to point at picture locations during encoding or only visually observe them.
First, it was expected that both children and adults who pointed at the pictures during encoding would perform better on the source memory task for both congruent and incongruent picture-location pairs (higher accuracy and higher speed) than those who only observed the pictures during encoding. However, compared to adults, we expected that pointing as support for immature cognitive control functioning would benefit children more. Second, we expected that item and source memory would be better for object-location associations that are congruent with previous experiences than for those that are incongruent.

METHOD Participants
Participants were 123 children (68 boys, M age = 8.7 years, SD = 0.98 years, range 7-10), who visited a Dutch Science Museum and 65 young adults (seven men, M age = 20.0 years, SD = 2.92, age range 17-34 years), enrolled at a Dutch university. The children were tested in the summer of 2013 and the young adults in winter 2013/2014. Children participated voluntarily and parents had to give written consent for their participation. The experiment took place in a separate room in the museum. The young adults were tested in the University lab rooms and participated for course credit or voluntarily. Participants within each age group were randomly assigned to each of the encoding conditions (e.g., the pointing or observation condition).

Design
A 2 (Age Group: children vs. young adults) × 2 (Encoding Condition: pointing vs. observation) × 2 (Congruency: congruent vs. incongruent picture location) mixed design was used with age group and encoding condition as between-subjects factors and congruency as a within-subjects factor.

Materials
The experimental task was programmed in E-prime 2.0 Psychology Software Tools, Inc., Sharpsburg, PA and presented on a 17-in. ELO touchscreen with a 1,024 × 768 resolution (ELO Touch Solutions, Inc., Milpitas, CA). Stimuli were a set of photos presented at a size of approximately 15 by 10 cm showing natural scenes or natural objects that are associated with looking up (e.g., clouds) or looking down (e.g., grass). In total, 74 pictures were used, 14 for the practice phase and 60 for the experimental phase (see Appendix S1 in the online Supporting Information for all pictures). Half of the pictures depicted objects or sceneries associated with looking up and the other half with looking down. In the encoding condition, 40 pictures (20 congruent and 20 incongruent) were presented. Half of these pictures were presented in the upper part of the screen and the other half in the lower part of the screen. In the test phase, 60 pictures (the 40 pictures from the study phase and 20 new pictures) were shown. Picture presentation was counterbalanced between four versions of the experiment so that all pictures equally often appeared in a congruent and incongruent location and equally often at the upper or the lower half of the screen.

Procedure
Participants were seated behind the touchscreen that was tilted backwards at an angle of 30 ∘ (see Figure 1) and were tested in individual sessions of approximately 10 min in total. The task started with a short practice phase in which participants were familiarized with the task (i.e., the encoding and the test procedure). Then, the experiment started with the encoding phase consisting of 40 trials. In Figure 2, the encoding and retrieval phases of the trial procedure are depicted. In the encoding phase, each trial started with the presentation of a black horizontal midline dividing the screen with a white background color in two equal halves for 1 s. Participants were instructed to fixate on the middle of the line. Next, a picture was presented above or below the midline for 2 s. Half of the participants were instructed to point with their finger at the locations of the pictures and the other half just to look at the pictures. Accuracy and reaction times (RTs) of the pointing responses were recorded by the computer. Immediately after the encoding phase, the retrieval phase started in which all 60 pictures were shown, one in each trial. A trial started with a fixation cross (1 s) followed by a picture in the middle of the screen (2 s). Then, participants had to make an old/new judgment of whether or not they recognized the picture from the encoding phase by pressing on the word "OLD" or "NEW" at the touchscreen. When participants judged the picture to be "NEW," they progressed to a new trial, but when they judged it to be "OLD," they were asked at which location they had seen the picture in the encoding phase, by pressing "TOP" or "BOTTOM" on the touchscreen.

Data Analysis and Design
For performance in the test phase, a mixed analysis of variance (ANOVA) with age group and encoding condition (pointing vs. observation) as between-subjects factors and picture-location congruency (congruent vs. incongruent) as a within-subjects factor was used to test for effects on item-and source memory performance and RTs. Item memory performance was assessed by recognition sensitivity (d-prime) for congruent and incongruent picture-location pairs which was calculated by the following formula: [Z (hit rate) -Z (false alarm rate)]. Because d-prime scores cannot be calculated for hit rate proportions of 1 or false alarm proportions of 0, scores with these values were truncated according to the following formula: scores of 0 were transformed by 1/2N (N = the maximum numbers of false alarms) and scores of 1 by 1-1/2N (Wixted & Lee, 2013). This resulted in d-prime values for the congruent stimuli ranging between −2.73 and + 3.93 and the ones for incongruent stimuli ranging between −2.90 and + 3.93. For source memory, proportion scores were computed by dividing the number of correctly remembered locations by the number of correctly recognized items. Partial eta-squared (η p 2 ) was calculated as a measure of effect size, with values of .01, .06, and .14 characterizing small, medium, and large effect sizes, respectively (Cohen, 1988). Because of skewed data in the young adult group, the effects found were retested by nonparametric tests. For the between-subjects effects an independent-samples Mann-Whitney U test was used and for within-subjects effects a related-samples Wilcoxon signed rank test. Both tests used cutoff scores for significance, alpha of .05 and confidence interval (CI) = 95%.

RESULTS
All participants in the pointing condition pointed at all pictures correctly during encoding. The mean pointing (reaction) times for the congruent and incongruent pairs were, respectively, M = 891 ms, SD = 218, and M = 909 ms, SD = 235 for the children, and M = 853 ms, SD = 206, and M = 855 ms, SD = 212 for the young adults.
Means and standard deviations of d-prime scores for item memory and proportion scores for source memory can be found in Table 1 and a visual representation of the data can be found in Figure 3. The analysis of item memory yielded a main effect of age group, F(1, 184) = 300.86, p < .001, η p 2 = .62, and encoding condition, F(1, 184) = 4.13, p = .044, η p 2 = .02, but not of congruency, F(1, 184) = 1.06, p = .306, η p 2 < .01. No interactions were found between age group and encoding condition, F(1, 184) = 0.48, p = .489, η p 2 < .01, age group and congruency, F(1, 184) = 0.32, p = .570, η p 2 < .01, encoding condition and congruency F(1, 184) = 1.29, p = .258, η p 2 < .01, or age group, encoding condition and congruency, F(1, 184) = 1.91, p = .168, η p 2 = .01. The main effect of age group was also found by the nonparametric test, p < .001, and indicates that young adults outperformed the children. The main effect of encoding condition was also found by the nonparametric test, p = .017, and indicates that participants in the pointing condition, performed worse than those in the observation only condition.

DISCUSSION
Source memory and memory for schema-incongruent information are cognitive functions that heavily rely on brain areas that are still immature in children (i.e., DLPFC; Brod et al., 2015;Menon et al., 2005). The present study investigated whether pointing gestures would support source memory for schema-(in)congruent information. In contrast to our first hypothesis and previous research showing that pointing compared to visual observation can have a positive effect on source memory (Ouwehand et al., 2016), the present study did not show such beneficial effects. In fact, pointing seemed to have a negative effect on picture recognition and no effect on source memory in both children and young adults. This finding does not only suggest that pointing toward picture locations has no effect on source memory, but also that it had a detrimental effect on recognition of the pictures. The present findings are also contradictory to our expectation that children's memory would benefit more from pointing than young adults' memory. Hence, the hypothesized compensatory effect of pointing gestures for children's suboptimal (immature) cognitive control functions could not be confirmed. In line with previous research (e.g., Cycowicz, Friedman, Snodgrass, & Duff, 2001;Sprondel, Kipp, & Mecklinger, 2011), we found that young adults were more accurate and faster on the item and source memory test than the children. Additionally, congruent picture locations were better remembered than incongruent ones. This finding is in accordance with previous research showing superior memory for schema-congruent versus incongruent events (e.g., Brod et al., 2017;Brod et al., 2015). The negative effect of gesturing on memory also contradicts previous studies showing that gesturing can improve children's learning in a more educational setting, such as, for example, learning a foreign language (Macedonia & Klimesch, 2014;Macedonia & Knösche, 2011;Toumpaniari et al., 2015), new concepts (e.g., Ping & Goldin-Meadow, 2008;Pouw et al., 2016;Valenzeno et al., 2003), or geography (Mavilidi et al., 2016), and the finding of Feyereisen (2009) that enacting enhances memory for (in)congruent information. It has to be noted that the abovementioned studies all used representational gestures (i.e., gestures that represent visual aspects of concrete features of the learning material). In the present study, pointing gestures were used, which were less specific for each feature, as the pointing gesture Volume 13-Number 2 remained the same throughout all pointing trials and only the direction of the pointing could differ.

Item Memory
Another important difference between the present study and previous studies that found a beneficial effect of pointing on source memory (Ouwehand et al., 2016) lies in the study design. As the present study used a between-subjects design, the previous studies mentioned used a within-subjects design in which participants had to select a response (point and observe or observe only) depending on the type of stimuli presented to them. This explanation is based on a study of Dodd and Shumborski (2009) who changed the design of a within-subjects paradigm of Chum, Bekkering, Dodd, and Pratt (2007) who found a beneficial effect of pointing on visuospatial working memory. In this study, participants were presented with two subsets of figures (circles and squares) at different locations and were instructed to point to one type of stimulus (e.g., the squares) and only observe the other (e.g., the circles). Interestingly, when Dodd and Shumborski (2009) used this paradigm of Chum et al. with a blocked design (trials in which participants had to point to all stimuli vs. only observe all stimuli), the effect was reversed, in that memory for the pointed figures was worse than for the figures only visually observed. The authors suggest that the positive effect of pointing may be dependent on whether encoding requires a selection process for a subset of the stimuli. They explained this with the selection-for-action hypothesis (Allport, 1987) which states that selection for action creates an attentional bias toward objects that require action, in that a difference in memory performance can only occur when pointing versus only viewing is manipulated within subjects and tasks. The present study did not require such a selection process, because a between-subjects design was used, which makes the "selectivity account" a potential explanation for the contradicting findings between this and Chum et al.'s (2007) study.
Another important difference with previous studies is that whereas we used a fixed presentation time, in the studies of Chum et al. (2007), Shumborski (2009), andOuwehand et al. (2016) the stimuli disappeared as soon as a participant pointed at (and touched) it on the screen. It can be said that in these studies, participants had a sense of agency (SoA), which is the subjective experience of having control over one's actions and their effects (for a review, see Moore, 2016). Evidence suggests that SoA (i.e., agency cues; Cipolotti, Robinson, Blair, & Frith, 1999) is involved in the development of specialized attention and memory processes. For future research, it would be interesting to investigate whether SoA is necessary for positive effects of pointing actions on learning and memory.
A limitation of the present study is that the children were tested in a room in a science museum (with other children, experimenters, and parents close by), whereas the young adults were tested at the university laboratory. However, similar effects of congruency and pointing were found in both age groups, which suggest that the effects of interest might be robust against these different testing environments. Another limitation was that the procedure was restrained by the regulations of the project within the science museum; experiments should have a maximal duration of 10 min. For this reason, we chose to include the maximal number of stimuli per condition (i.e., 20) that could fit within this 10-min time slot, and did not include a distraction task between the encoding and the test phase.
To conclude, the present study did not support our hypothesis that pointing could act as an external cognitive resource to improve memory that could compensate for the immature cognitive system (i.e., the DLPFC) of children. In contrast, the present findings even suggest that pointing has a detrimental effect on recognition. In line with earlier findings that contextual features congruent to an existing schema are more easily remembered than incongruent features (e.g., Brod et al., 2017), source memory for objects in semantically congruent locations was better than for those in incongruent locations. More research is needed to pinpoint conditions under which gestures benefit or hamper cognitive processing before we can make general recommendations for gestures as an educational tool.