The effect of divided attention on global judgment of learning accuracy

This study examined the effect of divided attention (DA) on global judgment of learning (JOL) accuracy in a multitrial list learning paradigm. A word monitoring task was used to divide attention. Participants were assigned to an attention condition (DA at encoding, DA at judgment, DA at retrieval, or focused attention) and completed 4 learning trials, each comprising a study, judgment, and recall phase. Participants showed greater overconfidence in the DA at encoding (Trial 2) and DA at retrieval (Trials 1 and 2) conditions than in the focused attention condition. DA at judgment did not affect JOL accuracy, and there was no effect of DA in Trials 3 and 4 on JOL accuracy across all attention conditions. Results indicate that participants consider conditions of encoding and retrieval but do not engage in recall when forming global JOLs. These findings suggest that people rely on extrinsic cues (Koriat, 1997) when making repeated, global metamemory judgments.

An interesting question in metamemory is how people make and adjust judgments of learning (JOLs).JOLs involve predicting the likelihood that studied items will be recalled at a future time (Mazzoni & Nelson, 1995).Two different kinds of JOLs have been investigated in the literature.The first type, item-by-item JOLs, involves predicting the likelihood that a particular item will be recalled in the future.The second type, global JOLs, involves predicting the number of items that will be recalled out of a set of studied items.For example, participants might study 40 items and then be asked, "Out of the items you studied, how many do you think you will be able to recall in a memory test 5 minutes from now?" JOL formation is thought to correspond with processes that occur as a student studies or learns new material (Mazzoni, Cornoldi, & Marchitelli, 1990), and understanding these processes could lead to a better understanding of how adults allocate study time and monitor learning.Moreover, learning often occurs over multiple study sessions, and understanding how metamemory judgments are adjusted over time would be informative about the dominant processes in everyday learning.Our study was concerned with how the magnitude and accuracy of absolute global JOLs are modulated over multiple study trials and in conditions of focused attention and divided attention (DA).
Two opposing theories on metamemory judgment formation exist.The first, the direct access view, posits that metamemory judgments are based on the same processes that underlie recall (Lovelace, 1984).Accordingly, the direct access view anticipates that variables that affect retrieval should have concomitant effects on metamemory judgments.The second view, the cue utilization framework, argues that JOLs are based on three different cues about the properties of the to-be-learned items and the learning environment (Koriat, 1997).According to this model, people draw on three classes of cues when making a JOL.Extrinsic cues involve characteristics of the learning environment, such as the type of study strategy used and duration of study time.Intrinsic cues concern the intrinsic properties of the to-be-studied items, such as their concreteness and imaginability.Mnemonic cues pertain to characteristics of the resulting memory representations, such as memory strength and retrieval latency.Extrinsic, intrinsic, and mnemonic cues are assumed to influence participants' predictions of future memory retrieval to various extents.
The purpose of our study was to examine the degree to which global JOLs are affected by conditions of DA and multiple study trials.In the context of a JOL task, there are at least three occasions on which DA might influence JOLs: during study, during retrieval, and during judgment formation.Although neither theory of JOL formation was explicitly developed to account for the effects of DA or multiple study trials, the connection between the DA and JOL literatures seems natural.Indeed, one can think of DA during encoding as perturbing the underlying memory representations and DA during retrieval as perturbing the access to the underlying memory representations, both of which might affect mnemonic processing to various extents.On the other hand, DA during judgment might affect the use of cues.Predictions of whether DA during encoding and retrieval might affect mnemonic cues can be derived from the DA literature.
The effects of DA during encoding on recall performance have been well documented (Craik, Govoni, Naveh-Benjamin, & Anderson, 1996;Naveh-Benjamin, Craik, Gavrilescu, & Anderson, 2000;Naveh-Benjamin & Guez, 2000;Fernandes & Moscovitch, 2000, 2002).These studies consistently demonstrate that recall performance is poorer when encoding occurs under DA than under focused attention.The effect of DA at retrieval on recall performance is less clear.On the one hand, Craik and colleagues (Craik et al., 1996; see also Naveh-Benjamin et al., 2000;Naveh-Benjamin & Guez, 2000) showed that DA during retrieval has little effect on recall performance when the DA task involves nonverbal processing.On the other hand, Fernandes andMoscovitch (2000, 2002) showed that DA tasks that entail concurrent verbal or phonological processing systematically reduce recall performance.According to Fernandes and Moscovitch, whether DA at retrieval affects recall depends on whether the DA task competes with the recall task in accessing the underlying representational structures responsible for maintaining the to-be-recalled items (Fernandes & Moscovitch, 2002).Thus, implementing DA at encoding or retrieval can affect recall performance if the DA task requires the participants to process verbal or phonological information.
In the context of metamemory research, the effect of DA at encoding and retrieval seems particularly interesting given that much of everyday learning and retrieval takes place under conditions of DA (e.g., students might study for a test while simultaneously watching television or take a test while construction workers work nearby).However, it remains to be seen whether people can accurately predict the effect DA will have on overall recall performance.
To date, few studies have reviewed the effects of DA on the accuracy of metamemory judgments.Kelley and Sahakyan (2003) examined the effect of a secondary task on confidence judgments and found that DA during encoding significantly lowered confidence judgment accuracy.In addition, DA during encoding led to a trend for lower accuracy at detecting items that had not been studied during encoding.If confidence judgments and JOLs are based on similar information, then we would predict that DA during encoding would lead to a reduction in metamemory accuracy.The effects of DA during the formation of a judgment or during retrieval on metamemory performance were not explored.It is possible that DA may affect the information that contributes to mnemonic cues if attention is divided at either encoding or retrieval by reducing memory strength.This is consistent with the trend for lower accuracy at detecting unstudied items (Kelley & Sahakyan, 2003).However, DA could also alter the information that contributes to external cues by interfering with the learning environment.People may be unable to anticipate the severity with which DA will affect recall performance, resulting in an overestimation of future recall performance on early trials.After observing recall performance, people may be able to accurately predict the cost of DA during encoding and titrate their judgments over multiple trials.
Judgment formation may be particularly vulnerable during DA if the secondary task directly interferes with the processes used to make JOLs.Making a global JOL presumably requires that people assess how much information they will retrieve.The degree to which this assessment requires one to access the contents of memory may leave it vulnerable to the effect of DA.For example, if global JOLs are based on retrieval of target words from long-term memory, then dividing attention with a verbal task that entails semantic monitoring during judgment should affect JOL accuracy.Alternatively, people may account for the secondary task during judgment formation and titrate JOLs in light of the secondary task's potentially detrimental effects on memory.
The underconfidence with practice (UWP) effect is a change in JOL accuracy with repeated judgment formation.Koriat, Sheffer, and Ma'ayan (2002) and Meeter and Nelson (2003) demonstrated the existence of a UWP effect, in which participants' JOLs switch from overconfidence on the initial learning trial to underconfidence on the second learning trial.This effect is counterintuitive in that one might assume that practice in predicting future performance would lead to increased accuracy.Thus, calibration, or absolute accuracy, decreases with repeated practice (Koriat et al., 2002).This pattern is not true for all types of judgments.Resolution, the extent to which trial-by-trial JOLs discriminate between recalled and unrecalled items, improves with practice (Nelson, 1984).
The extent to which global JOLs are calibrated with repeated practice remains unknown.To explore this question, we tested the existence of a UWP across four trials.Although one might expect participants to be poor at accounting for the effect of DA initially, they might learn to adjust their JOLs to account for the negative effect of DA across multiple study (or recall) trials.Note, however, that in multitrial learning participants must also account for the positive impact of the multiple study trials.Thus, we can explore whether participants' JOLs are sensitive to the counteracting effects of enhanced learning from repetition and negative impact of DA.
We hypothesized that participants' JOLs would be sensitive to the presence of a secondary task at encoding and retrieval, especially after the first learning trial.In addition, we hypothesized that dividing attention during the judgment phase would lead to less accurate JOLs than a focused attention control condition.This hypothesis was premised on the idea that JOLs entail an assessment of ease of retrieval.JOLs would be affected when participants perform a concurrent task that interferes with retrieval processes to the extent that the assessment of ease of retrieval is based on actual memory retrieval processes.If participants base their global JOLs on some aspect of the memory retrieval process and the secondary task interferes with the retrieval process, then one would expect participants to make lower JOLs when they make the JOLs under DA conditions.This may reduce the trend for early overestimation of future memory performance (Koriat et al., 2002;Meeter & Nelson, 2003).Alternatively, DA during judgment formation might affect cue use and integration.If cue use were affected by DA, then JOL accuracy would be affected.However, whether this would lead to an increase or decrease in JOL magnitude is unclear.If DA during judgment affects JOL formation, then judgments made under DA should differ from those made under focused attention.In addition, the degree to which participants overestimate or underestimate recall accuracy should also be affected by the presence of DA.

METHOD Participants
Ninety-two University of Maryland introductory psychology students participated in the experiment in return for partial fulfillment of course requirements.

Materials
Study materials consisted of two lists, each with 40 words selected for their medial location on Battig and Montague's (1969) category word norms to avoid highly typical or atypical words.Words were presented in a fixed random order for each study trial, with the order varying between study trials.

Design and procedure
The experimental design was a 4 × 4 × 2 mixed factorial design with attention condition (focused, DA at encoding, DA at judgment, DA at recall) as the betweenparticipant variable and study trial (Trials 1, 2, 3, 4) and word list (list 1 or 2) as the within-participant variables.Participants were randomly assigned to one of the four attention conditions.In addition, the word lists were randomly assigned with the constraint that an equal number of participants saw each list.
The experiment consisted of three phases: study, judgment, and recall.Each participant cycled through all three phases for a total of four times using a single word list.Before the experimental task, participants completed a practice session on the DA task to make sure they understood the task and to prevent surprise about task difficulty.Participants engaged in the secondary task alone until they informed the experimenter that they were confident with the procedure and ready to begin the experiment.Participants in the DA at judgment and DA at recall conditions were informed of the block in which they would practice the secondary task before beginning the practice session.
DA task.Attention was divided using a word monitoring task adapted from Fernandes andMoscovitch (2000, 2002) that required participants to monitor the meaning of a series of words.This task was chosen for its capacity to interfere with the encoding and retrieval of studied words.If mnemonic cues are critical at the judgment phase, then DA during judgment should affect the magnitude or accuracy of JOLs.Words were presented at a rate of one word every 1.5 s via headphones.Participants were instructed to press the "Q" key for "man-made objects."No response was required for "non-man-made object" words.The probability that a man-made object would occur on any given trial was 60%.To ensure that words attention and judgment of learning representing human-made objects occurred equally often for each participant, words were presented in a fixed random order, with the same percentage of "manmade object" words in each quartile of the list.

Study phase.
Words were presented on the computer screen at a rate of one word every 4 s.A total of 40 words were presented to each participant.At the end of the word presentation, participants completed the Digit Symbol subtest of the Wechsler Adult Intelligence Scale-Revised to control for recency effects during the judgment and recall phases.Participants were given 60 s to complete as much of the task as they could but were told they were not expected to finish.
Judgment phase.After completing the Wechsler Adult Intelligence Scale-Revised Digit Symbol task, participants were given 60 s to make five JOLs.The first was a global JOL in which participants estimated the total number of words they expected to recall.The remaining four JOLs were based on subsets of the list (i.e., "How many animals do you think you will remember?").These judgments were not analyzed in this study but were necessary to collect sufficient data for the DA task.
Recall phase.After the judgment task, participants had 4 min to recall as many words as possible.
In the focused attention condition, participants completed the experiment as described earlier.In the DA at encoding condition, the word monitoring task took place from presentation of the first word to the presentation of the last word.
In the DA at judgment condition, the word monitoring task took place from the end of the Digit Symbol test to the end of the JOL period.In the DA at retrieval condition, the DA task took place during the 4 min in which participants were recording the recalled words.
Two computers were used for stimulus presentation and data collection.One computer, in front of which the participant was seated, presented the to-be-recalled words visually.The second computer ran the DA task, with stimuli presented via headphones.Participants recorded JOLs and recalled words manually in a test booklet.A new page in the test booklet was used for each study trial, and participants were not permitted to look back at prior responses.

RESULTS
Three dependent variables were analyzed separately for each of the four study trials: mean recall, mean JOL, and mean bias, defined as the difference between the global JOL and the number of words recalled.Dependent variables were analyzed using analyses of variance (anovas) and Dunnett's t test.All planned comparisons were conducted with alphacontrolled family-wise error at .05, and the focused attention condition served as a control to test for the effects of DA on performance at each phase of each trial.All statistical tests were significant at p < .01unless noted otherwise.In addition, response latencies and accuracy on the word monitoring task were analyzed.

Recall
Recall performance was analyzed to test whether the DA task successfully divided attention.We predicted recall would be impaired in the DA at encoding and the DA at retrieval conditions.DA at judgment would result in poorer recall performance if participants relied on retrieval to make JOLs.In other words, if JOLs monitor memory strength, then practice or DA should affect DA at judgment differently than JOLs made under conditions of focused attention.If recall performance was equal across the focused attention and DA at judgment conditions, different cues may have been used.
Table 1 presents the recall data for the four attention conditions across the four study-recall trials.A 4 × 4 × 2 anova revealed a main effect of attention condition, F(3, 84) = 23.34.Dividing attention during judgment had no significant or systematic effect on recall performance, suggesting that participants are not engaging in retrieval when making JOLs.Planned comparisons revealed that both DA at encoding and DA at retrieval led to significantly poorer recall than focused attention for each of the four study trials.These results are consistent with prior research (Fernandes & Moscovitch, 2000, 2002).The effect of DA at encoding and retrieval was robust across multiple learning trials.To our knowledge, no prior research has examined the effect of DA on memory across multiple study-recall trials.

JOLs
JOLs were analyzed to test whether the DA manipulation yielded differences in global JOLs.We predicted that JOLs would be lower in the DA at encoding and DA at retrieval conditions than the focused attention condition because participants would anticipate the effects of the DA task on subsequent recall.
Table 1 presents the mean total JOLs for the four attention conditions across the four study-recall trials.A 4 × 4 × 2 anova revealed a main effect of attention condition, F(3, 84) = 7.79, MSE = 1,158.05,h p 2 = .22;a main effect of study-recall trial, F(3, 252) = 98.08,MSE = 1,561.80,h p 2 = 54; and a attention and judgment of learning study trial × attention condition interaction, F(9, 252) = 3.59, MSE = 57.17,h p 2 = .11.Word list had no effect, h p 2 = .02,and did not interact with any other variables: attention condition, h p 2 = .01;study trial, h p 2 = .001;attention condition × study trial × word list, h p 2 = .03.A series of one-way anovas confirmed that the effect of attention condition on total JOLs was present for each study-recall trial: Trial 1, F(3, 88) = 11.13,MSE = 371.68,h p 2 = .28;Trial 2, F(3, 88) = 3.39, MSE = 181.64,p < .05,h p 2 = .10;Trial 3, F(3, 88) = 6.70,MSE = 360.33,h p 2 = .18;Trial 4, F(3, 88) = 8.13, MSE = 421.75,h p 2 = .22.Planned comparisons revealed that there was no effect of DA at judgment on JOLs.This indicates that the process used in forming global JOLs was robust to the effects of DA.Moreover, it suggests that participants are not engaging in retrieval to form their JOLs.Arguably, had participants engaged in retrieval in forming their judgments, the magnitude of judgment would have been affected by DA.DA at encoding resulted in significantly lower JOLs than in the focused attention condition for each of the four study-recall trials.This indicates that participants were sensitive to the negative impact of DA at encoding on future recall performance.
Despite training with the word monitoring task in the practice session, participants initially failed to account for its effect on memory when making judgments.As shown in Table 1, total JOLs were statistically different from the focused attention condition for participants in the DA at retrieval condition only on the fourth study-recall trial.Participants in the DA at retrieval condition gave nominally lower JOLs on Trial 2 than on Trial 1, whereas participants' JOLs for the other three conditions increased from Trials 1 to 2. This could indicate that participants learned that DA at retrieval had a negative impact on recall and used this cue in forming subsequent judgments.However, it was not until the fourth study-recall trial that their judgments deviated significantly from the focused condition, and they eventually learned to titrate their judgments to account for the impact of DA at recall.

Bias
We computed a bias score for each participant by subtracting the total number of recalled words from the estimated words recalled.A positive score indicated an overestimation of number of words recalled, and a negative score indicated an underestimation of the number of words recalled. A

DISCUSSION
The purpose of this study was to test the degree to which global JOLs are affected by DA at different periods in multiple learning trials.We revealed four main findings.First, consistent with prior research, we found that when the DA task entailed word monitoring, recall was affected in the DA at encoding and retrieval conditions (Fernandes & Moscovitch, 2000, 2002).Moreover, we found that the effect of DA at encoding and retrieval was robust to multiple study-recall trials.These conclusions are supported by the finding that participants' recall was consistently poorer in the DA at encoding and DA at retrieval conditions than in the focused attention condition.
Second, participants' global JOLs were sensitive to the effect of DA on recall both when the DA task was implemented at encoding and when it was implemented at recall.This was supported by the finding that participants' global JOLs were lower in the DA at encoding and DA at retrieval conditions than in the focused attention condition.However, an interesting caveat to this conclusion is that participants apparently did not appreciate the impact of DA at recall until after the first recall trial, despite having practiced the divided attention task before the experiment.From a theoretical perspective, this finding suggests that global JOLs are sensitive to the external cue of DA.
The third main finding was that the formation of JOLs was unaffected by DA.This was supported by the finding that participants' JOLs in the DA at judgment condition were equivalent to the focused attention condition.This suggests that participants were not engaging in overt recall in forming their global JOLs.Had participants engaged in overt recall, one would expect their judgments to be affected by the DA task in the same way recall was affected by divided attention at retrieval.It is possible that non-mnemonic extrinsic and intrinsic cues may be more robust, or unyielding, under DA conditions.Alternatively, a DA task that interferes with extrinsic (e.g., monitoring noise level in study environment) or intrinsic (e.g., rating items as concrete or abstract) cue formation may shed light on how unyielding these cues actually are.
The fourth main finding was that the UWP effect appears to disappear with sufficient practice.Consistent with the findings of Koriat et al. (2002), attention and judgment of learning participants in the focused attention, DA at judgment, and DA at encoding initially showed the UWP effect.However, after the second learning trail, participants in these groups showed progressively less underestimation.In contrast, participants in the DA at retrieval condition never showed the UWP effect and, in fact, showed slightly more overestimation on Trial 4 than on Trial 3.
Two alternative explanations may account for the differing effects of DA across the study, judgment, and retrieval phases of each trial.First, the formation of category-specific JOLs may have increased subsequent recall.However, all groups made category-specific JOLs, and their effects on recall would be equally present for all trials and for all DA conditions.Second, participants had less time per item in the study phase (4 s) than in the judgment phase (12 s) or recall phase (10 s).However, if time per item influenced task demands across the attention conditions, then we would expect that performance on the DA task would be worse during encoding, when participants had the least amount of time per item.Conversely, we found that performance on the word monitoring task was faster and more accurate in the DA at encoding condition.
In sum, the present research indicates that DA at encoding and retrieval negatively affects recall.However, at the same time, participants seem to be somewhat aware of the negative impact of DA at encoding and retrieval in the sense that they estimate lower recall rates under these conditions.That global JOLs were unaffected by DA suggests that the processes participants use to monitor the acquisition of information can be carried out under DA conditions.Although we failed to find an effect of DA on global JOLs, it would be important to examine whether this effect holds for item-by-item JOLs.

Notes
This research served as partial fulfillment of the requirements of the honors program at the University of Maryland for Kelly Barnes.The authors would like to thank William Hall, Steve Brauth, Dana Plude, Ana Franco-Watkins, and Thomas Nelson for providing valuable insights on an earlier draft of this article.
Correspondence about this article should be addressed to Kelly Barnes, Department of Psychology, 306 White Gravenor, Georgetown University, Washington, DC 20057 (e-mail: kab69@georgetown.edu).

Table 1 .
Mean recall, JOL, and bias scores (SE) for the four divided attention conditions across the four study trials SD = 76.74,t(43)=4.68, p < .01,and DA at encoding condition, M = 79.43%,SD = 8.95, t(43) = 3.68, p < .01.Accuracy did not differ between the DA at encoding and DA at judgment conditions, p = .31.Slower and less accurate performance in the DA at judgment and DA at retrieval conditions probably reflects the costs of switching between the DA task and entering responses in the test booklet.