In our complex visual environment, visual working memory (VWM) serves as an online workplace where selected visual information is stored, maintained, and retrieved. The efficient use of stored information is important for our daily behaviors, which involve the planning and execution of multiple visual actions within mere seconds. For instance, our visual cognitive capacity appears to be influenced by the information stored in VWM, according to which we selectively perform a series of perceptual and memory operations (Cosman & Vecera, 2011; Downing, 2000; Hollingworth, Richard, & Luck, 2008; Soto, Hodsoll, Rotshtein, & Humphreys, 2008; Soto, Humphreys, & Heinke, 2006).

Studies on VWM have provided convincing evidence that information successfully stored in VWM can resist interference from irrelevant distractors for several seconds, thus enabling suppression of any unwanted information while maintaining essential information in awareness (McNab & Klingberg, 2008; Todd, Fougnie, & Marois, 2005; Vogel, McCollough, & Machizawa, 2005). In this way, VWM plays an important role in maintaining coherent visual experiences across extended periods, during which irrelevant sensory inputs may disrupt the stable perception of one’s external visual environment (Currie, McConkie, Carlson-Radvansky, & Irwin, 2000; Irwin, 1992).

Many studies demonstrating the importance of VWM have assessed memory performance by using a “single-shot” method, wherein each trial contains a single set of memory items that are tested only once. For instance, in a typical short-term recognition task, the memory display comprises an array of items that participants have to remember. After a brief delay (i.e., the memory retention interval), a subsequent test display comprising either a single or multiple probe items is presented in order to examine participants’ recognition accuracy for the in-memory items (e.g., a change detection task; Luck & Vogel, 1997). This paradigm has been widely accepted and is considered useful for demonstrating the essential characteristics of VWM’s storage, maintenance, and retrieval processes. However, despite the popularity of the single-shot method, our daily goal-directed behaviors typically demand a series of discrete processes through which the information stored in VWM is iteratively accessed and repeatedly retrieved. Thus, although it is a powerful testing tool for examining the way in which VWM stores and represents information, the single-shot method yields limited implications for our daily memory behaviors, in that the information in VWM must be “recycled” rather than excluded for a single use.

Being sensitive to the presence of a visual change in the environment is important for adaptive behaviors in our daily life, wherein we must generally cope with a bombardment of sensory inputs that vary constantly, depending on our actions. This sensitivity does not rely exclusively on the sensory capacity of our early visual system (e.g., retinal receptors), but also on our capacity to detect a mismatch between information already in our awareness (e.g., VWM) and incoming sensory information from the external world. This notion has been examined in a study wherein a visual change driven by a mismatch between items stored in memory and those in a test array was found to trigger a rapid shift of attention to the change (Hyun, Woodman, Vogel, Hollingworth, & Luck, 2009). The results of this study demonstrated that a mismatch between information already in VWM and information from new sensory inputs can be salient enough to trigger an automatic shift in attention (i.e., a pop-out), which suggests that our visual system is exceptionally sensitive to the occurrence of a mismatch.

However, this exceptional sensitivity to a mismatch raises some important questions, particularly when we consider that the study in question employed the single-shot method criticized above: (1) What happens to representations in VWM once the mismatch is successfully detected? (2) Can we reinstate the original representations when we need to engage in consecutive retrieval for the same (i.e., the previously accessed) or a different (i.e., not previously accessed) in-memory item?

In the present study, we aimed to answer these questions by examining the influence of retrieval access to a piece of information in VWM on subsequent recognition of the same or of a different piece of information. Specifically, to achieve this aim, we had participants “recycle” the information stored in VWM by manipulating them to retrieve the stored items twice in a row in a single trial. Our analyses focused on the aftermath of the first retrieval attempt in terms of (1) the influence of the presence of a change in the first probe on the subsequent memory test and (2) the spatial correspondence between the first and second probes.

We herein propose that “retrieval-induced forgetting” may occur as a consequence of the recycling attempt under this manipulation. In theory, retrieval access to a memory item takes advantage of the item’s memory trace, which was formed during the retrieval access; thus, retrieval access becomes faster and more accurate when the access is repeated (Carrier & Pashler, 1992). The retrieval-induced forgetting account also posits that retrieval access to the memory item weakens the memory traces of items that have not been accessed (Anderson & Spellman, 1995; Mensink & Raaijmakers, 1988). This retrieval-induced forgetting was originally reported in studies on verbal long-term memory (Anderson, Bjork, & Bjork, 1994), but other studies have proposed its existence for visual long-term memory, as well (Ciranni & Shimamura, 1999; Maxcey, 2016; Maxcey & Woodman, 2014). Additionally, recent studies have noted that the retrieval of an item stored in VWM can be deleterious to other memory representations stored therein (Awh, Dhaliwal, Christensen, & Matsukura, 2001; Woodman & Vecera, 2011).

Accordingly, we can expect that VWM utilizes a retrieval process for a specific in-memory item that in some way influences the remaining memory representations. In other words, every subsequent retrieval access to a memory item after the initial access may improve or at least preserve recognition accuracy for that item, whereas retrieval access to any unselected memory item that has not been previously accessed impairs recognition accuracy for that item. Previous studies reporting retrieval-induced forgetting in VWM used a recognition task in which participants were required to report the features of either the same or different objects sequentially (Awh et al., 2001; Woodman & Vecera, 2011). Note that both these studies and the present one dealt with retrieval-induced forgetting for visual recognition memory.Footnote 1 As such, we employ the term “recognition-induced forgetting,” as justified by Maxcey and Woodman (2014). If such recognition-induced forgetting is a general consequence of VWM retrieval, it should be observed in a widely accepted paradigm such as change detection.

To test the above predictions, we devised a consecutive-change detection paradigm wherein another sequence of change detection is added to a typical single-shot change detection task (Exps. 1 and 2). Specifically, we tested memory items by probing them twice in a row, using either the same memory items or different items, and asked participants whether or not the first and the second probes differed from the corresponding item(s) stored in their memory. Theoretically, the change detection of the first probe item would trigger access and retrieval of the stored representation of the corresponding item in VWM, whereas the second test probe would reinstate retrieval access to the original memory representation. Therefore, change detection performance for the second probe should reflect the aftermath of the first retrieval access for a given memory representation. Then, in Experiment 3, we devised a change detection-plus-continuous recall task wherein participants needed to report the presence of a color change for the first test probe, and then report the original memory color of the second probe on a color wheel constructed according to a circular color space (Zhang & Luck, 2008). The analysis examined the distributions of recall errors with probabilistic mixture models (Bays, Catalao, & Husain, 2009; Zhang & Luck, 2008) to evaluate the specific influences of recognition-induced forgetting and the aftermath of memory–perception mismatch.

Experiment 1

The aim of Experiment 1 was to test how the initial change detection attempt via access to items stored in memory influences the recycling of original VWM representations. To do this, we used a consecutive-change detection task with a whole-probe method wherein the same number of memory items was probed twice in a row for change detection. Based on previous findings of visual pop-out of a memory–perception mismatch and recognition-induced forgetting (Hyun et al., 2009; Maxcey & Woodman, 2014), we hypothesized that the presence of a color change in the first test probe would interfere with change detection in the second test probe. In contrast, when the first test probe exhibited no change, the change detection for the second test probe would be preserved or even improved.

Method

Participants

On the basis of existing VWM literature that used a change detection task (Awh et al., 2001; Hyun et al., 2009), the sample sizes for Experiment 1 and 2 were limited to 15 participants. Fifteen undergraduates from Chung-Ang University, ranging in age from 19 to 26 (seven male), participated in this study for course credits after signing an informed consent document. All participants had normal color vision and normal or corrected-to-normal visual acuity.

Stimuli and procedure

Figure 1a provides a schematic illustration of the stimuli and procedure of the consecutive-change detection task in Experiment 1. The stimuli were presented using MATLAB and the Psychophysics Toolbox (Brainard, 1997) on a 22-in. LCD monitor with a gray background at a viewing distance of 60 cm. Each trial began with a centered fixation point (0.30° × 0.30° of visual angle) for 500 ms, after which a memory array of four colored squares (each with a size of 0.89° × 0.89°) was displayed for 200 ms. After the memory array, there was a blank interval of approximately 1,000 ms, which was followed by the first test array. This array comprised the same number of items in the same locations as the four memory items. The first array was displayed for 200 ms and was followed by another blank interval of 1,000 ms. After the second blank interval, the second test array was displayed for 200 ms; again, the items were presented in the same locations as the items in the preceding arrays. The memory items were displayed at four equidistant locations on an imaginary circle with a diameter of 5.91°, and each item was equally distant from the others by 4.17°, center to center. The colors of the four memory squares were randomly selected without replacement from eight distinct RGB-based colors: red (255, 0, 0), blue (0, 255, 0), green (0, 0, 255), yellow (255, 202, 0), pink (255, 71, 222), cyan (0, 255, 255), purple (102, 0, 102), and white (255, 255, 255).

Fig. 1
figure 1

The stimuli and procedure of the consecutive-change detection task in Experiment 1 (A: whole probe) and Experiment 2 (B: single probe). In Experiment 1, a set of memory items was displayed, after which the first test array was displayed. After another brief blank interval, the second test display appeared. The items in the test arrays were either identical to those in the memory array, or one or both of the test arrays had an item of a different color from the corresponding item in the memory array. When both test arrays showed a color change, the changes occurred either at the same or different locations across the test arrays. The stimuli and procedure for Experiment 2 were identical to those in Experiment 1, except that only a single test item was used as a probe in each test display at either the same location or a different location.

Full size image

Four conditions were created according to the presence or absence of a change in the color of an item between the memory array and each test array. In the trials where neither the first nor the second test array showed a color change (i.e., NC–NC), the colors of the items in the first and second arrays were the same as the colors of the memory items. When both test arrays exhibited a color change (i.e., C–C), the colors of the changed items were selected from the colors not used in the four memory squares nor in the preceding test array, thus ensuring that no color matched across the changed items between the first and second test arrays. Accordingly, there were six different colors for the C–C condition (four for the memory items and two more for the changed items), and five different colors in the NC–C and C–NC conditions (four for the memory items and one more for the changed item). For the first and second test arrays, the color change was manipulated to occur with 50% probability. Thus, the four conditions appeared with equal probabilities (25%). When both the test arrays had a color change (i.e., the C–C condition), the change in the second array occurred either at the same location as or at a different location from the change in the first test array, with 50% probability.

Participants performed a total of 240 trials across the four conditions, with 60 trials in each condition (NC–NC, NC–C, C–NC, and C–C). In each trial, participants had to indicate whether the colors of the items in each test array were identical to those in the memory array, or whether the color of one of the items had changed. To ensure that participants’ behavioral responses and feedback on the first test array did not interfere with their responses on the second test array, participants were asked to make the change detection response for the first test array after being presented with the second test array; in other words, they made two successive responses regarding whether each test array had a color change. During the practice session prior to the main session, an accurate response was signaled by a beeping sound. This feedback was not provided for the main session.

To prevent intrusion errors that might arise from the color change in the first test array, participants were also instructed to pay careful attention while judging whether the change was present or absent. In other words, they were explicitly instructed to make the decision strictly according to the colors of the items in the memory array rather than to anything else (e.g., items in the first test array). Accurate responses were heavily stressed, whereas speed was not.

Results and discussion

In general, the overall accuracy was higher for the first change detection (92.0%) than for the second (85.8%), t(14) = 7.52, p < .001, 95% confidence interval (CI) for the mean difference [4.5%, 8.0%], Cohen’s d = 1.94. This means that the participants were simply better at reporting the presence or absence of a color change in the first probe than in the second probe. Because we were primarily interested in examining the effect of a change occurrence in the first test on change detection in the second test, we sorted the trials according to whether or not the first test array exhibited a color change. The accuracy data from the sorted trials showed that participants exhibited less accurate responses for the second change detection when the first test array exhibited a change (76.4%) than when it did not (95.1%), t(14) = –10.01, p < .001, 95% CI [14.7, 22.7], d = –2.58. Next we examined whether the relative accuracies of the first and second change detections differed according to the presence or absence of a change in the first test array (Fig. 2a). Here, a positive score indicates relative ease in determining the presence or absence of a change in the second as compared with the first test array, whereas a negative score indicates relative difficulty. The relative accuracy score became clearly negative when there was a change in the first test array (–12.3%), t(14) = –7.89, p < .001, 95% CI [–15.7, –9.0], d = –2.03, but remained around zero when the first test array had no change (–0.17%), t(14) = –0.14, p = .887, 95% CI [–2.6, 2.3], d = –0.04. These results suggested that the presence of a change in the first test array made change detection in the second test array somewhat more difficult, whereas the absence of a change in the first test array made the second change detection somewhat easier.

Fig. 2
figure 2

(A) Relative response accuracies for the second test array when compared with the first test array in Experiment 1. (B) Proportions of correct responses for the first (light gray bars) and second (dark gray bars) change detections according to the four consecutive-change detection conditions in Experiment 1. NC herein refers to “no change,” whereas C refers to “change.” Error bars here and in all the subsequent figures represent the standard errors of the means.

Full size image

To further examine how the presence or absence of a change in the first test array influenced the change detection in the second test array, we categorized the trials according to which test array exhibited a color change (C) or no change (NC). This yielded four categories: NC–NC, NC–C, C–NC, and C–C. Figure 2b shows the proportions of correct responses in the C and NC trials for each category. We compared the change detection accuracies for each test array within each condition using a paired t test and obtained a statistically significant result for the C–NC, t(14) = 3.83, p = .002, 95% CI [4.3, 15.1], d = 0.99, and C–C, t(14) = 4.98, p < .001, 95% CI [8.5, 21.5], d = 1.29, conditions—namely, the conditions wherein the first array had a color change. In contrast, the accuracies did not differ significantly for either the NC–NC, t(14) = –0.73, p = .477, 95% CI [–2.2, 1.1], d = –0.19, or the NC–C, t(14) = 0.50, p = .626, 95% CI [–2.9, 4.7], d = 0.13, condition, wherein the first array had no color change. These results again confirmed that a color change in the first test array led to poorer change detection performance in the second test array, regardless of whether the second test array actually showed a change.

How then does the occurrence of a color change in the first test array influence change detection in the second array? Furthermore, does recognition-induced forgetting influence performance in the second change detection? To address these questions, we examined the accuracies for the second test array in the C–C condition after grouping the trials wherein participants successfully detected a color change in the first test array according to the location of the change in the second test array—namely, whether it was the same or a different location as the change in the first test array. Note that for Experiment 1, which used the whole-probe method, we performed this analysis only for the C–C condition because spatial correspondence could be determined only if both test arrays exhibited a color change.

In other words, the “Same” condition comprised trials wherein the color change occurred for the item in the same location between the two arrays. Note that the colors of the changed items between the two test arrays were always different from each other and from the corresponding item in the memory array. The “Diff” condition, by contrast, comprised trials wherein the color change occurred for items in different locations between the test arrays. Specifically, in the second test array, the changed item in the first test array was returned to the color of the corresponding memory item while one of the remaining three test items exhibited a color change.

Figure 3 illustrates the proportion of correct responses for the second test array in each of these two location conditions. Note that this analysis was performed only for trials wherein participants obtained a correct response on the first change detection. In other words, we evaluated change detection performance for the second test array only upon participants’ successful detection of the first change, because this confirmed that participants had accurately encoded and maintained the memory items until at least the first test array. Despite this apparently accurate encoding of the memory items, we found that responses in the Same condition had relatively low accuracies (about 80%). This indicates that participants had difficulty maintaining the original memory items beyond the first test array if they had successfully detected a color change in that first array. Importantly, this difficulty worsened when changes occurred in different locations between the test arrays (i.e., the Diff condition), and a significant difference emerged between the Same and Diff conditions, t(14) = 3.81, p < .002, 95% CI [5.9, 21.3], d = 0.98. Thus, it would appear that detecting a change in the first test array interfered with the maintenance of the untested memory items, which may have been intact around the time of the first test array’s appearance. A plausible explanation for the lower accuracy in this condition is recognition-induced forgetting.

Fig. 3
figure 3

Proportions of correct responses for the second test array for trials wherein participants made a correct response to the first test array in the C–C condition (wherein both test arrays had a color change) in Experiment 1, according to the spatial correspondence between first and second changes (Same vs. Different [Diff] locations).

Full size image

According to this recognition-induced forgetting account, a negative consequence of the automatic selection of an item showing a salient color change in the first test array would be the weakening of the memory traces for the unselected items. Our use of a whole-probe manipulation in this experiment might have demanded retrieval access to these unselected items at least once (i.e., for the first change detection attempt). However, the item selected in the first change detection attempt was likely reexamined in an iterative fashion to confirm that there indeed had been a change in the first test array (see Exps. 4A and 4B in Hyun et al., 2009). Accordingly, the unselected items in memory would be accessed less frequently than would the changed item during the first change detection attempt. The substantial drop in accuracy in the Diff condition (shown in Fig. 3) supports this interpretation.

Nevertheless, our whole-probe manipulation does not specifically allow us to conclude that recognition-induced forgetting is a valid explanation of the results because displaying the whole set of memory and test items reinforces retrieval access to all of the items rather than a specific item of interest. Thus, in Experiment 2, we designed another consecutive-change detection display wherein a single test item was displayed for each test display (i.e., single-probe method) instead of the whole probe array. In this experiment, we sought to test the hypothesis that the spatial correspondence between the initial and subsequent probe items elicits recognition-induced forgetting. Furthermore, we focused specifically on addressing how recognition-induced forgetting is related to the aftermath of a salient visual change as observed in Experiment 1.

Experiment 2

In Experiment 1, our results indicated that response accuracy for the second test array was impaired if a color change in the first test array was successfully detected. Furthermore, the accuracy impairment for the second change detection was more pronounced if the changed item in the second test array was in a different location from that in the first test array. We explained these results using the recognition-induced forgetting account (i.e., recognition of a specific memory item can induce forgetting of the unselected memory items that did not undergo the recognition process).

When considering recognition-induced forgetting, we might predict that the memory accuracy for the Diff condition (i.e., when the changed item in the second array differs in location from the first array) would decrease regardless of whether the first probe exhibited a color change. However, the whole-probe method in Experiment 1 did not allow us to test this prediction because displaying the entire sets of memory and test items made it impossible to manipulate the spatial correspondence between the changed items in the first and second arrays unless the first array exhibited an explicit change in color. Accordingly, Experiment 2 probed memory items twice in a row, just as in Experiment 1, but instead used a single-probe method wherein the memory items were tested by consecutively displaying two single probes. The single-probe method allowed us to present the probes at the same or at different locations across the first and second test displays; in this way, we could manipulate the spatial correspondence between the first and second probes, unlike in Experiment 1.

Considering the results of Experiment 1, we first predicted that the change detection for the second probe would be impaired if the first test probe elicited a color change, regardless of whether the first and second probes occupied the same or different locations. Second, on the basis of the recognition-induced forgetting account, we predicted that change detection performance for the second probe would be impaired when the second probe was displayed at a different location from the first probe, whereas it would not be impaired at all when the second probe was displayed at the same location as the first probe.

Method

Participants

Fifteen new undergraduates from Chung-Ang University, ranging in age from 19 to 25 years (six male), participated for course credits after signing an informed consent document. All participants had normal color vision and normal or corrected-to-normal visual acuity.

Stimuli and procedures

All the stimuli, conditions, and procedures were identical to those in Experiment 1 except that only a single item was presented as a testing probe for each test display. Two consecutive testing probes were displayed in the same location on half of the trials, whereas they were displayed in different locations on the remaining half (Fig. 1b).Footnote 2 As in Experiment 1, the participants performed several practice trials to adapt to the task paradigm; again, a feedback sound was used to signal correct responses only in these practice trials. Accurate responses were stressed over speed.

Results and discussion

In general, accuracy was higher for the first probe (89.7%) than for the second (84.4%), t(14) = 10.39, p < .001, 95% CI [4.2, 6.4], d = 2.68. To determine whether we had replicated the general pattern of change detection impairment for the second test display when there was a change in the first probe, we compared the change detection accuracies for the second probe between trials wherein the first probe did or did not change. The results here indicated that the change detection accuracy was significantly lower in the former trials (80.7%) than in the latter (88.1%), t(14) = –4.30, p = .001, 95% CI [–11.1, –3.7], d = –1.11.

We also examined how the difference in change detection accuracies between the first and second probes differed according to whether the first probe had or had not changed. The comparison revealed that the difference score (representing the drop in accuracy from the first to the second change detection) was evident when the first probe exhibited a change (–12.0%), t(14) = –8.37, p < .001, 95% CI [–15.1, –1.6], d = –2.16, but was not evident when the first probe did not exhibit a change (1.4%), t(14) = 0.98, p = .342, 95% CI [–1.6, 4.4], d = 0.25 (see Fig. 4a). These results confirmed the replication of the findings from Experiment 1—namely, that a salient visual mismatch between a perceived item and the corresponding item in memory can disrupt subsequent change detection.

Fig. 4
figure 4

(A) Relative response accuracies for the second probe compared with the first probe in Experiment 2. (B) Proportions of correct responses for the first (light gray bars) and second (dark gray bars) change detections according to the four conditions in Experiment 2.

Full size image

The proportions of correct responses (both change and no-change) for the first and second probes across the conditions are illustrated in Fig. 4b. Paired t tests for the response accuracies on the first and second test arrays within each condition revealed significant results for both the C–NC, t(14) = 8.47, p < .001, 95% CI [12.7, 21.3], d = 2.19, and C–C, t(14) = 3.28, p = .005, 95% CI [2.4, 11.6], d = 0.85, conditions. This indicates that participants made less accurate responses for the second probe, regardless of whether there was a change in that probe, after being exposed to a change in the first probe. No significant difference was found for the NC–NC condition—participants’ responses for the second probe were as accurate as those for the first probe, t(14) = 1.23, p = .241, 95% CI [–1.8, 6.7], d = 0.32. A significant difference was, however, found for the NC–C condition, t(14) = –2.20, p = .046, 95% CI [–10.3, –0.1], d = –0.57, wherein responses to the presence of a change in the second probe were more accurate than were those to the absence of a change in the first probe. These results indicated that the absence of a change in the first probe made the second change detection somewhat easier than did the presence of a change.

In Experiment 2, our secondary interest was to examine whether recognition-induced forgetting was driven by the spatial correspondence between the first and second probes. To accomplish this, we grouped trials according to whether or not the locations of the first and second probes matched. We then further grouped the trials according to whether or not the first probe exhibited a color change, to examine the interaction between a match in location and the presence or absence of a change in the first probe. Figure 5 illustrates the proportions of correct responses for the second probe according to the spatial location of the probe (Same vs. Diff) and whether or not the first probe had changed (C vs. NC). Note that, in this analysis, we used only the trials wherein the response for the first probe was correct; this was done to ensure that we captured trials wherein the memory items had been encoded and maintained successfully by the time of the first change detection.

Fig. 5
figure 5

Proportions of correct responses to the second probe after a correct response to the first probe, according to the presence of a color change in the first probe (no change [NC] vs. change [C]) and the spatial correspondence between the first and second probes (same [Same] vs. different [Diff] locations) in Experiment 2. Note that the responses to the second probe above were made only on trials wherein the memory items had been accurately maintained by the time of the first probe onset.

Full size image

We performed a repeated measures two-way analysis of variance (ANOVA) on the response accuracies for the second probe, with the factors Change Presence for the First Probe (change vs. no change) and Probe Location (same vs. different). This analysis yielded significant main effects of both change presence for the first probe, F(1, 14) = 24.36, p < .001, η p 2 = .64, and probe location, F(1, 14) = 23.72, p < .001, η p 2 = .63. The interaction of these two factors was also significant, F(1, 14) = 29.13, p < .001, η p 2 = .68. Pairwise comparisons revealed a significant difference between the accuracies of the Same and Diff location trials in the NC condition, t(14) = 6.52, p < .001, 95% CI [10.3, 20.4], d = 1.68, but not in the C condition, t(14) = –0.44, p = .667, 95% CI [–4.8, 3.1], d = –0.11.

Experiment 1 did not allow us to pinpoint whether the observed impairments were due to recognition-induced forgetting, because the whole-probe method established spatial correspondence between the first and second probes only if both probes exhibited a change. Experiment 2 addressed this shortcoming by using a single probe, and thereby designating a single memory item for retrieval access. If recognition-induced forgetting truly had occurred for the unprobed items in memory, then they would show lower accuracy when reporting whether a change had occurred in the second display for an item presented in the location of one of the three unprobed items, regardless of whether the first probe exhibited a change. This was exactly what we observed in the C and NC trials of the Diff condition in Experiment 2 (see Fig. 5). We also found that participants’ reporting of a change absence for both the first and second probes were overall very accurate in trials wherein the first and second probes occupied the same location and the first probe exhibited no color change (i.e., the NC trials of the Same condition, as shown in Fig. 5). The accurate performance in these trials supports the account of recognition-induced forgetting, such that accessing an item in VWM multiple times does not impair memory for that item but does do so for other remaining items in the array.

However, the accuracy dropped substantially in trials wherein the first and second probes occupied the same location and the first probe had exhibited a color change (C trials in the Same condition, as is shown in Fig. 5). This contrasting pattern of accuracy impairment, which was confirmed by the significant interaction between change presence in the first probe and probe location, is not explainable by recognition-induced forgetting. Rather, the accuracy drop appears to be due to a detrimental effect triggered by the successful detection of a salient color change in the first probe. This is perhaps explained by the results of a previous study, wherein a salient change in a memory item shifted attention toward the location of that change (Hyun et al., 2009); thus, the color change for the first probe may have captured participants’ attention to the location of the change. However, it is unclear how this shift in attention for the first color change made it difficult to reinstate the original (i.e., memory display) color when making the second change detection (i.e., C trials of the Same condition).

A possible reason is that the salient color change in the first probe distracted participants sufficiently that they did not maintain the traces of the original memory items, thus leading to an impairment in change detection for the second probe. More specifically, the distraction could have been caused by an attention-driven automatic updating of working memory (Landman, Spekreijse, & Lamme, 2003; Makovski & Jiang, 2007; Makovski, Sussman, & Jiang, 2008; Polich, 2007). Previous studies have demonstrated that items undergoing spatial selection are automatically encoded and consolidated into memory, and can virtually replace or update a corresponding piece of information in memory present within the same location (Landman et al., 2003; Melcher, 2009; Nakamura & Colby, 2002; Polich, 2007; Wittenberg, Bremmer, & Wachtler, 2008; see also Kaganovich, Wray, & Weber-Fox, 2010, for a discussion of the ecological validity of the updating account). Accordingly, it is possible that in Experiments 1 and 2, the color change of the first probe in the Same condition would have directed participants’ attention toward the location of the changed item, which would then lead to an automatic replacement or updating of the memory item’s color with the color of the first probe. This replacement would in turn lead to an inaccurate change detection for the second probe because the color of the second probe would be compared with the color of the first probe rather than with the color of the memory item in its corresponding location.

If this distraction had indeed occurred, then the change detection of the second probe would be vulnerable to intrusion errors from the color of the first probe. These intrusion errors, in turn, could lead to an increased number of “miss” responses for trials wherein the second probe exhibited a change that was the same as the first change (e.g., green, blue, and blue for each memory, first probe, and second probe item, respectively). Experiment 2, however, did not examine the C–C condition in this regard because the first and second probes in that condition were always different colors. Thus, it was difficult to identify the exact source of the distraction that drove the change detection impairment for the second probe when the first probe exhibited a change.

Experiment 3

The results of Experiment 2 suggest that retrieving a specific item stored in memory strengthens the memory trace of a corresponding item but weakens those of the remaining items (i.e., recognition-induced forgetting) in VWM. However, the memory enhancement for previously accessed items was no longer present when the first probe exhibited a salient color change. Instead, the color change led to less accurate change detection performance, especially when the second probe was in the same location as the first—in fact, this led to a complete negation of the expected enhancement of the trace for the original memory item.

A more straightforward interpretation for the detrimental effects of recycling VWM representation on subsequent memory performance can be found in an existing theoretical account regarding the fidelity of VWM representations. Specifically, it is unclear whether the recognition-induced forgetting observed in Experiments 1 and 2 was derived from degraded representational precision or the total loss of a particular representation in VWM, as described in the “sudden death” model proposed by Zhang and Luck (2009). In that model, these alternative accounts were tested using two contrasting parameters, standard deviation (SD) and random guess, estimated from a recall performance. These parameters evaluated the qualitative and quantitative aspects of VWM representations, and provide an excellent theoretical framework for unraveling the reason for the forgetting in VWM. Nevertheless, the consecutive change detection task in Experiments 1 and 2, which utilized a two-alternative forced choice method of response, did not allow us to estimate these two parameters because memory performance was assessed in terms of discrete recognition responses rather than continuous recall.

Thus, in Experiment 3 we attempted to overcome these methodological limitations by conducting a change detection-plus-continuous recall task, wherein the second change detection task was replaced with a color-wheel recall task that aimed to test participants’ memory of the items in the memory array. This recall task enabled us to examine the influence of recognition-induced forgetting and the subsequent state of memory items after a salient color change had been detected in a preceding change detection task.

Method

Participants

Eleven new undergraduates from Chung-Ang University, ranging in age from 19 to 24 years (three male), participated for course credits after signing an informed consent document. All of the participants had normal color vision and normal or corrected-to-normal visual acuity.

Stimuli and procedures

All the stimuli, conditions, and procedures were identical to those in Experiment 2 except for the following. First, we changed the memory set size from four items to three. We expected that decreasing the memory load would increase the likelihood of observing recognition-induced forgetting, and provide more convincing evidence when compared with forgetting observed under a moderate memory load (i.e., four items), as in Experiments 1 and 2.

Second, we replaced the second change detection task with a continuous-recall task (Fig. 6). Furthermore, participants performed a total of 540 trials, of which half had no color change in the change detection display that preceded the recall task. In each trial, the recall task only began if the participant made a correct change detection response (i.e., accurately pointed out the presence or absence of a change) in the change detection display. This ensured that the subsequent recall response for memory items in the recall task was accurate—namely, that participants had retrieved the memory item and made a comparison against the probe in the preceding change detection task. When participants made a correct response in the first change detection, a 1-s blank interval followed, after which an array of black-outlined boxes was displayed (with the boxes corresponding to the locations of the items in the memory array). The boxes were surrounded by a color wheel (diameter 19.1°) at a random rotation. The outline of one of the boxes was twice as thick as those of the rest of the boxes, which designated the memory item to be recalled. Participants were asked to report the color of the designated memory item by clicking with their mouse on the part of the circular color space corresponding to the color of the recalled memory item.

Fig. 6
figure 6

Stimuli and procedure of the change-detection-plus-continuous-recall task in Experiment 3. After the change detection display of 200 ms, participants were asked to detect whether or not the single probe item exhibited a color change (compared to the corresponding item in the memory array). If participants made a correct response for the change detection, the recall task began after a 1-s blank interval. In the continuous-recall task, an array of outlined boxes was displayed (with the boxes appearing in the same locations as the memory items), surrounded by a large color wheel. The to-be-recalled memory item was designated by one of the boxes having a thicker outline. Participants then performed a mouse click on the color wheel to report the color of the cued memory item.

Full size image

Third, we made sure to maintain a consistent color distance between the displayed items, to ensure that the colors were sufficiently distinct. A total of four colors (three for memory items and one for the color change in the change detection display) were selected from an HSV color space with hue values ranging from 1° to 360°. Four evenly spaced colors (90°) were chosen per trial by using a randomly selected seeding hue value, to prevent participants from rehearsing a familiar set of colors (e.g., blue, red, green).

Finally, the to-be-recalled memory item for the recall task was randomly selected with equal probabilities. In Experiment 2, the second probe was presented at the same location as the first probe approximately three times as often as at different locations (i.e., 50% and 16.7%, respectively). Thus, it was difficult to discount the possibility that the relative forgetting of any untested items was due to preferential consolidation and maintenance of the probed item during the first change detection. In Experiment 3, each of the three memory items had an equal probability of being tested for recall (33.3%), to rule out the possible confound resulting from such preferential consolidation and maintenance ahead of the recall task.

Data analysis

Recall errors were measured in terms of the difference, in degrees, between the recalled color of the cued memory item and the actual color of that memory item. First, to identify the source of recognition-induced forgetting, we fitted the error distribution of recall responses with a probabilistic mixture model (Bays et al., 2009; Zhang & Luck, 2008). This model posits two types of recall errors: (1) errors associated with guessing, which are represented by uniformly distributed responses (i.e., the probability of guessing) and indicate that there is no mnemonic evidence of the to-be-recalled item, and (2) errors associated with the precision of a memory representation, represented by the SD of a circular Gaussian (von Mises) distribution of responses. Second, to examine the aftermath of detecting a color change on recall of the corresponding memory item (i.e., the updating or replacement hypothesis), we adopted a third parameter—“swapping” errors (Bays et al., 2009; Golomb, L’Heureux, & Kanwisher, 2014). These errors were expected to be closely associated with intrusion errors (i.e., recalling the color of the probe in the change detection display instead of the actual color of the memory item; i.e., the probability of misreport).

We estimated these three parameters (probability of guessing, probability of misreport, and SD) for each participant and condition by using the MemToolbox (Suchow, Brady, Fougnie, & Alvarez, 2013). The data from one participant were excluded from the analysis because of noticeably poor recall (47.4% probability of guessing).

Results and discussion

Participants were, overall, fairly accurate in the change detection task (90.9%) and showed better performance for the change trials (96.1%) than for the no-change trials (85.7%), t(9) = 6.39, p < .001, 95% CI [6.7, 14.0], d = 2.02. Figure 7A illustrates the probability density functions obtained from the standard mixture-model analysis for each condition in the subsequent recall task.

Fig. 7
figure 7

(A) Probability density functions of recall performance as a function of the distance between the displayed color of the memory item and the reported color of the cued memory item, according to the presence of a change and the spatial correspondence between the change detection probe and the box cue in Experiment 3. (B) Probabilities of guessing and (C) standard deviations estimated according to the standard mixture model (Bays et al., 2009; Zhang & Luck, 2008). (D) Probabilities of guessing contrasted with probabilities of misreport (i.e., swapping errors; Bays et al., 2009; Golomb et al., 2014).

Full size image

First, the probability-of-guessing and SD values were estimated according to the standard mixture model; these are shown in Fig. 7b and c, respectively. For the probability of guessing, which is inversely related to successful memory recall, a two-way repeated measures ANOVA with the factors Presence of Change (no change vs. change) and Probe Location (same vs. different) was conducted. This yielded significant main effects of both presence of change, F(1, 9) = 17.18, p = .003, η p 2 = .66, and probe location, F(1, 9) = 6.82, p = .028, η p 2 = .43. However, the interaction of presence of change and probe location was not significant, F(1, 9) = 0.33, p = .583, η p 2 = .04. Pairwise comparisons in the probabilities of guessing between the same- and different-location trials (Same vs. Diff) in the NC condition revealed a significant difference, t(9) = –3.11, p = .013, 95% CI [–13.2, –2.1], d = –0.98, but the same comparison for the C condition revealed no significant difference, t(9) = –1.28, p = .231, 95% CI [–14.1, 3.9], d = –0.40. This indicates the presence of recognition-induced forgetting when the change detection display did not exhibit a color change. This result replicates the patterns of results observed in Experiment 2, wherein the retrieval access of previously accessed items in VWM was impaired if the previous access had led to a memory–perception mismatch (i.e., if a color change was present in the perceived item), and recognition-induced forgetting was evident for items that were not accessed.

For the SD values, however, only the Presence of Change factor had a significant main effect, F(1, 9) = 7.15, p = .025, η p 2 = .44. The main effect of probe location was not significant, F(1, 9) = 1.87, p = .205, η p 2 = .17, nor was the interaction between the factors, F(1, 9) = 0.29, p = .602, η p 2 = .03. Moreover, neither the difference in SD values between the same- and different-location trials (Same vs. Diff) in the NC condition, t(9) = –1.32, p = .218, 95% CI [–5.8, 1.5], d = –0.42, nor this difference in the C condition, t(9) = –0.24, p = .813, 95% CI [–5.0, 4.0], d = –0.08, was significant, suggesting that recognition-induced forgetting had little influence on the precision of memory items.

Even though we did observe a significant main effect of the presence of change, interestingly, the elevation of SD values was not observed for the same-location trials in the C condition, wherein the memory performance was expected to be impaired for the item that had previously exhibited a color change. This was supported by the absence of a significant difference in SD values between the same-location trials in the NC and C conditions, t(9) = –1.59, p = .146, 95% CI [–8.7, 1.5], d = –0.50. Only the SD values from the same-location trials in the NC condition (i.e., NC–Same) and the different-location trials in the C condition (i.e., C–Diff) were significantly different, t(9) = –2.84, p = .021, 95% CI [–7.4, –0.8], d = –0.89, contributing to a significant main effect of the presence of change. Any other pairs of SDs between the NC and C conditions were not significantly different (e.g., the NC–Diff vs. C–Diff conditions, t(9) = –1.45, p = .181, 95% CI [–5.0, 1.1], d = –0.46; the C–Same vs. NC–Diff conditions, t(9) = 1.06, p = .318, 95% CI [–1.7, 4.6], d = 0.33).

The results of our standard mixture-model analyses suggest that neither the forgetting of a previously probed item with a salient change nor the forgetting of unprobed items was likely due to degraded representational precision of the memory items. Instead, they were presumably derived from a stochastic drop in particular in-memory items (i.e., sudden death; Zhang & Luck, 2009). However, considering the moderate effect sizes derived from the analyses of the SD values, we should note that the present results on competing accounts of a dropout and degradation in precision are not confirmative. Rather, the possibility remains that some extent of degradation in memory precision occurred because of a salient color change, and/or that an improvement in precision for the item in the NC condition occurred because of repetitive exposure to the color matched with the remembered one during the first change detection.

Second, more importantly, we fitted the recall data to the mixture model with swapping errors (Bays et al., 2009). This analysis relied on a third parameter, probability of misreport, which was primarily extracted from the uniform guessing distribution. In the present experiment, the probability of misreport represents recall responses based on the color of the change detection probe rather than the color of the memory item. If there was a clear intrusion error—namely, if participants’ memory of the to-be-recalled memory item was replaced with their memory of the probe in the change detection task—the probability of misreport for items in the same-location trials (i.e., Same–Misreport) would be greater than that in the different-location trials (i.e., Diff–Misreport) in the condition wherein a color change had been exhibited in the change detection display. In contrast, no such differences would be observed between the conditions if the color change in the change detection display impaired memory of the corresponding memory item without updating or replacing it.

To test these predictions, we first examined the recall responses for the same-location trials (C–Same and NC–Same). For both sets of trials, we sampled the fourth color in consideration of a color change in the change detection probe, but only the C trials actually displayed the sampled color during the change presence manipulation. Despite the absence of a color change in the probe in NC trials, if participants reported the selected color for some reason, the response must have been made according to a random guess. Thus, the probability of such recall errors in the NC–Same condition could provide a baseline probability for determining whether the recall errors in the C–Same condition were random guesses or intrusion errors.

Figure 7d illustrates the probabilities of guessing and misreport values for each condition, as estimated by a mixture model in consideration of swapping errors. Although we predicted that there would be many intrusion errors, planned comparisons revealed that the probability of misreport for the same-location trials in the C condition did not differ from the probability of misreport for those in NC condition, t(9) = 1.04, p = .323, 95% CI [–2.3, 6.3], d = 0.33. We similarly found no differences for the comparisons of the remaining trials (i.e., C–Diff and NC–Diff trials), ps > .226, ds < 0.41. The lack of these differences indicates that there was little evidence for the automatic VWM updating of memory items with the change detection probe. Instead, the probability of guessing offers a better explanation for the inaccurate recall performance for the same-location trials in the C condition, supported by a significant difference between these trials and those in the NC condition, t(9) = –3.17, p = .011, CI [–17.0, –2.8], d = –1.00. These results are clearly shown in Fig. 7b and d.

Taken together, the findings from Experiment 3 successfully replicated the results of the previous experiments—namely, that recognition-induced forgetting and salient visual change jointly influence VWM performance. By adopting the continuous-recall paradigm, Experiment 3 assessed both the quantitative and qualitative aspects of the recognition-induced forgetting in VWM, and rejected the automatic-VWM-update account raised from Experiment 2, which had attributed the impaired memory after detecting a salient change to the replacement of the originally remembered item with the changed color, inspired by the concept of automatic memory update with up-to-date schema.

General discussion

Compared with our understanding of the mechanism of perceptual encoding and subsequent VWM consolidation, we know relatively little about what occurs to information in VWM after a subset of that information is accessed and retrieved. In the present study we examined the aftermath of accessing and retrieving a specific item stored in VWM in conjunction with a salient visual change driven by the item as a result of retrieval access. We found that a recognition test of an item in VWM led to impaired memory of the item if a salient change followed at its location, whereas there was no such impairment without a preceding change. Furthermore, memory for the untested items in VWM was also impaired. These findings together demonstrate that recycling of an item in VWM can induce its forgetting if the aftermath of such recycling (i.e., a salient change) distracts individuals’ memory of the item. Furthermore, the findings also demonstrate that recognition-induced forgetting occurs for items in memory if they are not tested during the recycling operation.

When the first test exhibited a salient color change in the consecutive-change detection task, we found a consistent pattern of change detection impairment for the second test array (Exp. 1) or probe (Exp. 2). Furthermore, in Experiment 2, which used a single-probe method, we observed a change detection impairment for the second probe when the locations of the first and second probes differed, regardless of whether the first probe exhibited a change. In contrast, when the location was the same between the first and second probes, the change detection performance for the second probe was primarily determined according to the presence or absence of a color change in the first probe. Specifically, the second change detection was less accurate if the first probe exhibited a change, whereas not if the probe elicited no change during the first change detection. The results of these two experiments demonstrated that subsequent access to an item held in VWM can impair memory for that item if there is a salient color change at the same location as the retrieved item during the first access and retrieval. They further demonstrate that a second access for a memory item can impair memory for the memory items not accessed during the first access, which clearly indicates the presence of recognition-induced forgetting in VWM.

There is considerable evidence to support our findings in the literature of visual attention and memory. For instance, Duncan’s (1984) classic study reported a within-object benefit against the between-object cost, which has been interpreted as being heavily reliant on a spatial switching cost in working memory (Awh et al., 2001; Woodman & Vecera, 2011). Another study on working memory forgetting indicated that not only the simple passage of time but also test interference can contribute to forgetting during a recognition task with rapid serial visual presentations (Potter, Staub, Rado, & O’Connor, 2002). This suggests that running a recognition test can cause a substantial amount of interference for the information held in memory. Other studies reported retrieval-induced forgetting of associative long-term memory (Anderson et al., 1994) and recognition-induced forgetting in visual long-term memory (Maxcey & Bostic, 2015; Maxcey & Woodman, 2014). These studies all suggest that selectively accessing a specific piece of information in memory can lead to memory impairment for the information not previously accessed.

One may find the preferential maintenance of the selected item more suitable for explaining the retrieval cost observed in the present study because, in Experiment 2, the second probe was displayed in the same location as the first probe in half of the trials, and therefore the probability of that item being in the first probe location in the second change detection was three times the probability of the item appearing in each of the three other locations (50% vs. 16.7%). With these uneven probabilities, the change detection attempt for the first probe reinforced participants to selectively focus on and maintain one of the memory items at the corresponding probe location. This prior demand of selective retrieval access for the memory item would not only strengthen the memory trace for the item, but also suppress maintenance of the unselected memory items in VWM. Thus, the representations of items that are not retrieved (i.e., untested) unavoidably degrade, becoming ultimately too vulnerable to resist interference or forgetting during the course of the consecutive change detection task.

In contrast to the other two experiments, in Experiment 3, we corrected the uneven probabilities by probing each of the three item locations with an equal probability (33.3%) for the change detection task, and explicitly examined the representational characteristics of the memory items in the aftermath of retrieval access by running a probabilistic mixture-model analysis on responses from a continuous recall task subsequent to a change detection task. The analysis showed a clear difference in the probability of guessing between the same and different location conditions (i.e., match or nonmatch of locations between the change detection probe and recall cue, respectively), basically replicating the pattern of recognition-induced forgetting observed in Experiments 1 and 2. However, no such clear difference was observed for the SD parameter estimates, suggesting a possibility that there was little decline in memory precision, despite clear differences in the probabilities of guessing. These results suggest that the source of the forgetting of the untested items may not be the general degradation of the representational precision in memory, but rather a complete loss of the item representations.

More importantly, this complete forgetting account can also apply to the memory impairment for items that had previously elicited a salient memory–perception mismatch. Specifically, the probability of guessing, and not the probability of misreport, was found to better characterize the error distribution of recall responses between a previously retrieved in-memory item with a color change and that same item without a color change. These results suggest that the presence of a color change during the first retrieval access to a memory item would lead to forgetting of that memory item, and that this forgetting does not arise from replacing the item with the testing probe (i.e., intrusion) but from a complete representational loss of the item.

It is important to note that we do not claim that the observed loss of the probed memory item in the present study is the same as that observed in Zhang and Luck’s (2009) study, although both their observation of the “sudden death” of memory items and our observation of recognition-induced memory loss in Experiment 3 suggest that VWM forgetting arises from a stochastic loss of memory items in an all-or-none fashion. The sudden death model specifically addresses forgetting in terms of the influence of the passage of time (i.e., retention interval), whereas the present study addressed it in terms of the influence of a memory retrieval attempt.

So how does the VWM-based visual mismatch (i.e., change) impair memory of the original in-memory item? Previous studies have demonstrated that active maintenance of information in working memory is modulated by sustained neural activity over various brain regions such as the majority of visual cortical areas (Awh & Jonides, 2001; Fuster & Jervey, 1982; Miller, Li, & Desimone, 1993) as well as the prefrontal cortices and fronto-parietal network (Cohen et al., 1997; Courtney, Ungerleider, Keil, & Haxby, 1997; Curtis & D’Esposito, 2003; Miller & Cohen, 2001). Additionally, recent studies have associated contralateral delayed activity (CDA), which is sustained negativity in an EEG signal at the posterior scalp electrodes, with items being maintained in VWM (Ikkai, McCollough, & Vogel, 2010; Vogel & Machizawa, 2004).

Given that sustained neural activity appears to aid temporal maintenance of information in working memory, we speculate that the memory impairment following a color change in the present study is presumably due to the onset of a new probe item triggering a transient interruption in the sustaining cortical activity for items already in memory, which is substantial only if the probe does not match the corresponding in-memory item. CDA studies have demonstrated that transient elevation of EEG activity occurring after the presentation of task-irrelevant distractors can be well suppressed during the course of VWM maintenance (Fukuda & Vogel, 2009; Jost, Bryck, Vogel, & Mayr, 2011; Vogel et al., 2005). Nevertheless, unlike ours, these studies specifically aimed to examine individuals’ capacity to filter out distractions presented by the onset of new items rather than their ability to maintain original memory items subsequent to a salient change. We speculate that the probed memory item exhibiting a task-relevant visual change may trigger more substantial and irrecoverable transient elevation in memory-related sustained cortical activity than does the onset of the task-irrelevant items in the previous studies.

This speculation is also behaviorally in line with the reconsolidation hypothesis, which posits that once a memory is retrieved, it must undergo additional consolidation to be retained (Alberini, 2005; Dudai & Eisenberg, 2004; Nader, Schafe, & LeDoux, 2000; Przybyslawski & Sara, 1997; Sara, 2000). Specifically, Baddeley (1998) argued that memories become labile during recall, which in turn has a substantial influence on whether memories must be reconsolidated. Accordingly, in the present study, the first retrieval attempt for a memory item wherein the comparison between the memory item and probe elicited a task-relevant salient change perhaps made the memory representation vulnerable to interference. Resisting the interference brought on by the newly displayed item exhibiting a change from the corresponding item in memory may have been difficult, which could have led to a failure in reconsolidating the original memory item. Therefore, in the present study, the second change detection or recall of a previously probed memory item would have been vulnerable to random-guessing errors because of a lack of mnemonic evidence when a salient color change had been evoked during the course of the preceding change detection.

In summary, we found that a salient visual change occurring during a recognition test of a memory item can impair subsequent recognition or recall of that item. Furthermore, regardless of the presence or absence of a change, memory for items excluded from the recognition test was impaired because of recognition-induced forgetting. Finally, the VWM forgetting observed in the present study occurred due to a probabilistic complete loss of untested memory representations, rather than due to a general degradation of their precision.