Abstract
We used a model of hypothesis generation (called HyGene; Thomas, Dougherty, Sprenger, & Harbison, 2008) to make predictions regarding the deployment of attention (as assessed via eye movements) afforded by the cued recall of target characteristics before the onset of a search array. On each trial, while being eyetracked, participants were first presented with a memory prompt that was diagnostic regarding the target’s color in a subsequently presented search array. We assume that the memory prompts led to the generation of hypotheses (i.e., potential target characteristics) from long-term memory into working memory to guide attentional processes and ocular–motor behavior. However, given that multiple hypotheses might be generated in response to a prompt, it has been unclear how the focal hypothesis (i.e., the hypothesis that exerts the most influence on search) affects search behavior. We tested two possibilities using first fixation data, with the assumption that the first item fixated within a search array was the focal hypothesis. We found that a model assuming that the first item generated into working memory guides overt attentional processes was most consistent with the data at both the aggregate and single-participant levels of analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Much of this work has focused on the automaticity of attention to WM content. We are not making this claim, but are merely accepting the notion that there is an attentional bias toward WM’s content, particularly when the goals of the search task are aligned with the WM content (i.e., valid trials; see Carlisle & Woodman, 2013, who found a much stronger n2pc when WM content was associated with the target in the visual search task). In the present work, it was often advantageous for participants to deploy attention to items in the environment that corresponded to the representations in WM that had been retrieved from LTM.
We have found that participants need to be explicitly aware of the possibility of a relationship between the memory prompts and the associated target features in order for the memory prompts to be used (see also Kunar, Flusberg, & Wolfe, 2006). In the present experiment, participants were explicitly told that there could be a relationship between the memory prompts and the color of the target at the beginning of the experiment.
The hypotheses in the present context are the features that the participant believes the target will have, given the memory prompt. This is similar to how a doctor generates a set of hypotheses to explain a pattern of symptoms, in that the hypotheses are the doctor’s best guess of the true disease the patient is suffering from.
For instance, say we have two traces {–1, 0, 1, 1, –1} and {0, 1, –1, 0, 1, 1} that are both above threshold, and that their activation values are .5 and .3, respectively. We would then multiply each element by its activation value and obtain the vectors {–.5, 0, .5, .5, –.5} and {0, .3, –.3, 0, .3, .3} and then sum the corresponding traces for each element in order to obtain one trace {–.5, .3, .2, .8, –.2}. Finally, we would normalize by dividing each element by the absolute value of the largest trace element (.8, in our example) to get the unspecified probe and ensure that each element is between –1 and 1: {–.625, .375, .25, 1, –.25}.
It is possible that the first item generated will not remain in WM, since it may be knocked out if WM capacity has been reached and the item that has been generated is above ActMinH (see Thomas et al., 2008, for more details). However, given the number of hypotheses (five) and Фs (four) for the simulations, this would have been a rare occurrence. Note that Ф is the WM capacity parameter and governs how many hypotheses can be simultaneously maintained.
All pairwise comparisons reported were conducted with Bonferroni correction.
Proportions were calculated on the basis of the raw frequencies of HyGene’s output for the first-generated and highest-activated rules for χ 2 and G 2. These were compared to the frequencies from the experimental data. For RMSD, percentages from the model were used, which were compared to the experimental percentages.
A1 Trials were only consecutive if they occurred within the same block. Additionally, we excluded trials in which an incorrect orientation response was recorded for either the previous time that a particular memory prompt was presented or the following memory prompt trial. For instance, if an incorrect orientation response occurred for Memory Prompt 2, the next Memory Prompt 2 trial was excluded, regardless of whether a correct orientation response was recorded. Finally, we only excluded trials in which the fixation was too far from the center for the current trial under consideration.
A2 We performed dependent-sample t tests comparing repeat and nonrepeat trials for each color (16 tests) and found four significant effects. These significant effects only occurred in the conditions in which the consecutive trials were included. For Experiment 1, both C2 and C3 were fixated more often when C2 and C3, respectively, had been the targets on the previous memory prompt trial. For Experiment 2, C3 was fixated first more often when C3 had been the target on the previous Memory Prompt 2 trial. Also, C5 was fixated first more often in Experiment 2 when C5 had been the target on the previous Memory Prompt 3 trial. All other tests were nonsignificant. The mean of .12 (SD = .13) for all effect sizes (η p 2) indicated a limited effect of recency on the deployment of attention. Additionally, we performed Bayes factor t tests (Rouder, Speckman, Sun, Morey, & Iverson, 2009). A mean of 2.85 (SD = 0.18) for all tests indicated that on the average the null was roughly three times more likely than the alternative (JZS with 1.0 of scale r on effect size). These results provide evidence against recency affecting behavior in the present experiment, on average.
References
Buttaccio, D. R. (2013). Probabilistic visual search (Unpublished doctoral dissertation). Norman, OK: University of Oklahoma.
Buttaccio, D. R., Lange, N. D., Hahn, S., & Thomas, R. P. (2014). Explicit awareness supports conditional visual search in the retrieval guidance paradigm. Acta Psychologica, 145, 44–53. doi:10.1016/j.actpsy.2013.10.014
Carlisle, N. B., & Woodman, G. F. (2013). Reconciling conflicting electrophysiological findings on the guidance of attention by working memory. Attention, Perception, & Psychophysics, 75, 1330–1335.
Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18, 193–222. doi:10.1146/annurev.ne.18.030195.001205
Dougherty, M. R. P., & Hunter, J. E. (2003). Probability judgment and subadditivity: The role of working memory capacity and constraining retrieval. Memory & Cognition, 31, 968–982.
Dougherty, M., Thomas, R., & Lange, N. (2010). Toward an integrative theory of hypothesis generation, probability judgment, and hypothesis testing. In B. H. Ross (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 52, pp. 299–342). San Diego, CA: Elsevier Academic Press.
Downing, P. E. (2000). Interactions between visual working memory and selective attention. Psychological Science, 11, 467–473. doi:10.1111/1467-9280.00290
Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96, 433–458. doi:10.1037/0033-295X.96.3.433
Houtkamp, R. R., & Roelfsema, P. R. (2006). The effect of items in working memory on the deployment of attention and the eyes during visual search. Journal of Experimental Psychology: Human Perception and Performance, 32, 423–442. doi:10.1037/0096-1523.32.2.423
Huang, L., & Pashler, H. (2007a). A Boolean map theory of visual attention. Psychological Review, 114, 599–631. doi:10.1037/0033-295X.114.3.599
Huang, L., & Pashler, H. (2007b). Working memory and the guidance of visual attention: Consonance-driven orienting. Psychonomic Bulletin & Review, 14, 148–153. doi:10.3758/BF03194042
Johnson, J. G., & Raab, M. (2003). Take the first: Option-generation and resulting choices. Organizational Behavior and Human Decision Processes, 91, 215–229.
Kok, P., Jehee, J. F., & de Lange, F. P. (2012). Less is more: Expectation sharpens representations in the primary visual cortex. Neuron, 75, 265–270.
Kramer, A. F., & Hahn, S. (1995). Splitting the beam: Distribution of attention over noncontiguous regions of the visual field. Psychological Science, 6, 381–386.
Kunar, M. A., Flusberg, S. J., & Wolfe, J. M. (2006). Contextual cuing by global features. Perception & Psychophysics, 68, 1204–1216. doi:10.3758/BF03193721
Lange, N. D., Buttaccio, D. R., Davelaar, E. J., & Thomas, R. P. (2014). Using the memory activation capture (MAC) procedure to investigate the temporal dynamics of hypothesis generation. Memory & Cognition, 42, 264–274. doi:10.3758/s13421-013-0354-1
Lange, N. D., Thomas, R. P., Buttaccio, D. R., & Davelaar, E. J. (2012). Catching a glimpse of working memory: Top-down capture as a tool for measuring the content of the mind. Attention, Perception, & Psychophysics, 74, 1562–1567.
Naijmenk, J., & Geisler, W. S. (2005). Optimal eye movement strategies in visual search. Nature, 434(7031), 387–391.
Olivers, C. N. L. (2009). What drives memory-driven attentional capture? The effects of memory type, display type, and search type. Journal of Experimental Psychology: Human Perception and Performance, 35, 1275–1291. doi:10.1037/a0013896
Olivers, C. N. L. (2011). Long-term visual associations affect attentional guidance. Acta Psychologica, 137, 243–247.
Olivers, C. N. L., & Humphreys, G. W. (2003). Attentional guidance by salient feature singletons depends on intertrial contingencies. Journal of Experimental Psychology: Human Perception and Performance, 29, 650–657. doi:10.1037/0096-1523.29.3.650
Olivers, C. N. L., Peters, J., Houtkamp, R., & Roelfsema, P. R. (2011). Different states in visual working memory: When it guides attention and when it does not. Trends in Cognitive Sciences, 15, 327–334. doi:10.1016/j.tics.2011.05.004
Oppenheimer, D. M. (2008). The secret life of fluency. Trends in Cognitive Sciences, 12, 237–241.
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16, 225–237. doi:10.3758/PBR.16.2.225
Soto, D., Heinke, D., Humphreys, G. W., & Blanco, M. J. (2005). Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance, 31, 248–261. doi:10.1037/0096-1523.31.2.248
Soto, D., & Humphreys, G. W. (2007). Automatic guidance of visual attention from verbal working memory. Journal of Experimental Psychology: Human Perception and Performance, 33, 730–737. doi:10.1037/0096-1523.33.3.730
Sprenger, A. M., Dougherty, M. R., Atkins, S. M., Franco-Watkins, A. M., Thomas, R. P., Lange, N., & Abbs, B. (2011). Implications of cognitive load for hypothesis generation and probability judgment. Frontiers in Psychology, 2, 129. doi:10.3389/fpsyg.2011.00129
Stroud, M. J., Menneer, T., Cave, K. R., & Donnelly, N. (2012). Using the dual-target cost to explore the nature of search target representations. Journal of Experimental Psychology: Human Perception and Performance, 38, 113–122.
Thomas, R. P., Dougherty, M. R., Sprenger, A. M., & Harbison, J. I. (2008). Diagnostic hypothesis generation and human judgment. Psychological Review, 115, 155–185. doi:10.1037/0033-295X.115.1.155
Wolfe, J. M. (1994). Guided Search 2.0: A revised model of visual search. Psychonomic Bulletin & Review, 1, 202–238. doi:10.3758/BF03200774
Woodman, G. F., & Luck, S. J. (2007). Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology: Human Perception and Performance, 33, 363–377. doi:10.1037/0096-1523.33.2.363
Zelinsky, G. J. (2008). A theory of eye movements during target acquisition. Psychological Review, 115, 787–835. doi:10.1037/a0013118
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1
One possibility for why the first-generated model fits better than the highest-activated model is that the participants were using the target color information from the previous time that a memory prompt was presented to guide search the next time the prompt was presented. For instance, for Memory Prompt 3 a participant might remember what the color of the target was on the previous Memory Prompt 3 trial (e.g., red) and use that information to guide search the next time that Memory Prompt 3 was presented. This would produce behavior similar to that from the first-generated model, since participants would use C4 to guide search when Memory Prompt 3 was presented roughly 80% of the time, and C5 to guide search 20% of the time, since these were the probabilities for Memory Prompt 3. This explanation differs from the one that HyGene would posit, which is that participants were generating anew each time a memory prompt was presented, regardless of what the target had been on the previous trial. To examine this possibility, we examined their fixation behavior on consecutive memory prompt trials and organized conditions on the basis of what the target color had been the previous time that a particular memory prompt was presented.Footnote 8 Note that consecutive memory prompt trials do not necessarily mean consecutive trials, since there may be intervening trials between when a particular memory prompt is presented and when it is presented again, due to the randomization process of trials.
What is plotted in Fig. 8 is the percentage of times that a relevant color was fixated first for each memory prompt, based on what the target was the previous time that the memory prompt had been presented for Memory Prompts 2 and 3. Because we were interested in whether the target from the previous trial on which a memory prompt was presented influences the present trial, we did not examine Memory Prompt 1, since the target was always C1 and both a recency account and HyGene would make similar predictions. Finally, the nonrepeat data in Fig. 8 are the percentages associated with trials on which a relevant color was fixated that was different from the target color on the previous trial (e.g., C3 was fixated first and C2 had been the target on the last Memory Prompt 2 trial), whereas the repeat data are the percentages associated with when a color was fixated first that had been the target on the previous trial of the same memory prompt. Thus, the height of the repeat bar relative to the nonrepeat bar provides an index of the recency for each relevant color.
First fixation data, based on what the target had been the previous time that a memory prompt was presented. Error bars represent one standard error.
Although there is a hint of recency in the data, the repeat and nonrepeat data are similar to one another, suggesting that participants are generating de novo each time they are presented with a memory prompt, particularly if consecutive memory prompt trials are removed.Footnote 9 If participants were simply remembering the color that the target had been the previous time that the memory prompt was presented (i.e., a recency effect), we would expect a large difference between nonrepeat and repeat trials. This is particularly evident when the consecutive trials are excluded, since the data for repeat and nonrepeat trials are virtually identical.
Although accounting for the slight recency is outside the scope of the present article, in theory it could be implemented in HyGene by assuming that if the memory prompt on a previous trial matches the memory prompt on the current trial, then the model does not regenerate, but rather uses the semantic trace that was updated on the previous trial as the focal hypothesis. At present, however, it is unclear whether participants are engaging in this type of higher-order process (i.e., participants maintaining the target’s feature on the previous trial for the current trial), or whether a combined effect of dimensional weighting and priming effect accounts for the slight recency observed on consecutive trials (Olivers & Humphreys, 2003).
Appendix 2
Below we present the recall, RT, and scan path ratio data across the different blocks of Experiments 1 and 2. The general trend is that visual search behavior and recall improve as participants gain more experience. This is evidenced by the higher recall rates for associated colors given a memory prompt (Table 5), lower scan path ratios (Fig. 9), and faster visual search RTs (Fig. 10). Note that participants went through six blocks in Experiment 1 and five blocks in Experiment 2.
Scan path ratios as a function of block and target type. Error bars represent one standard error.
Visual search reaction times (RTs) as a function of block and target type. Error bars represent one standard error.
Rights and permissions
About this article
Cite this article
Buttaccio, D.R., Lange, N.D., Thomas, R.P. et al. Using a model of hypothesis generation to predict eye movements in a visual search task. Mem Cogn 43, 247–265 (2015). https://doi.org/10.3758/s13421-014-0463-5
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13421-014-0463-5