Storage and binding of object features in visual working memory
Research highlights
► Errors in recall of color and orientation of the same object are strongly independent. ► Inconsistent with discrete working memory “slots” storing integrated objects. ► Misreporting of non-target items is due to errors in maintaining binding information. ► Results support a shared-resource model of visual working memory.
Introduction
What limits the visual information that can be maintained in short-term memory? Historically, this question has been addressed by examining the frequency of recall errors as memory load is manipulated, either in studies of ‘partial report’ (Irwin, 1991, Irwin, 1992, Irwin and Andrews, 1996, Sperling, 1960) or change detection (Luck and Vogel, 1997, Pashler, 1988, Phillips, 1974, Rouder et al., 2008, Todd and Marois, 2004, Vogel et al., 2005, Vogel et al., 2001). The results of these studies have commonly been interpreted as supporting a limit on the number of objects that can be simultaneously represented in working memory. In one influential version of this model, the objects present in a visual scene compete for storage in a small number of independent memory ‘slots’. Each slot maintains a representation of a single integrated object (incorporating all its features, bound together) with high fidelity, and the allocation of visual attention determines which objects gain access to a slot (Cowan, 2001, Hollingworth and Henderson, 2002, Irwin and Andrews, 1996, Luck and Vogel, 1997).
Recently, this conception of working memory has been challenged by studies examining how recall errors are distributed in the space of possible responses, based on discrimination (Bays and Husain, 2008, Bays and Husain, 2009, Palmer, 1990) or reproduction tasks (Bays et al., 2009, Wilken and Ma, 2004, Zhang and Luck, 2008, Zhang and Luck, 2009). These studies have revealed strict limits on the fidelity with which multiple visual objects can be maintained: the precision with which each visual feature is stored declines rapidly as the total number of items in memory increases. This finding is difficult to reconcile with the concept of storage in independent slots, and has led to the development of an alternative, shared-resource account of working memory (Bays and Husain, 2008, Wilken and Ma, 2004). According to this proposal, a single memory resource is flexibly distributed between the elements of a visual scene. As more items are stored, less resource is available per item, with the result that the features of each item are stored with increasing variability (‘noise’). Visual attention provides flexible control over distribution of this resource, such that salient or goal-relevant items are stored with enhanced resolution (Bays & Husain, 2008).
Importantly, in contrast to the slot model, this resource-based account does not predict a fixed upper limit on the number of objects that can be maintained. Indeed a mathematical model based on shared resources (Bays & Husain, 2008) predicts the appearance of such a capacity limit in change detection tasks, previously considered evidence in favor of a fixed slot model. Nonetheless, a number of attempts have been made to find a compromise position between the two models, in which varying resolution of storage co-exists with a fixed limit on the number of objects that can be stored (Alvarez and Cavanagh, 2004, Awh et al., 2007, Zhang and Luck, 2008). In particular, recent studies by Luck and colleagues (Zhang and Luck, 2008, Zhang and Luck, 2009) have presented results from a color reproduction task which appear to provide support for such a ‘hybrid’ model.
In these studies, participants were presented with a memory array of colored squares. After a brief retention interval, one array location was indicated and participants were required to report the color they recalled at that location by clicking on a color wheel. The authors analysed the distribution of responses on the color wheel as a mixture of two components: a gaussian distribution centered on the correct color of the probed item, and a uniform distribution spread equally over all possible responses. The gaussian component indicates variability in the stored representations of the colors in the memory array. Consistent with a resource-model account, the variability with which each item was stored depended on the total number of items in memory, as indicated by an increase in the gaussian width with increasing memory load. In addition, however, Zhang and Luck proposed that the uniform component corresponds to a proportion of trials on which subjects choose a response at random. As in a slot model, this might occur if no information was stored about the probed object as the result of exceeding an upper limit on the number of objects that can simultaneously be maintained in working memory.
Here we put this interpretation to the test, by examining the joint distribution of errors when subjects are required to reproduce from memory two different features (color and orientation) belonging to the same probed object. If only a subset of objects in an array can be stored, the absolute error in reporting color and orientation should be correlated, and the joint distribution of errors in the dual-feature task should consist of two components: one in which the object is stored and both features are recalled (with gaussian variability), and one in which the object is not stored and both responses are random. Neither result was observed: instead our results revealed that both the absolute error and the occurrence of uniform responses were strongly independent across feature dimensions.
This finding is inconsistent with the hypothesis of a fixed upper limit on the number of objects stored in memory. Instead these results support the proposal of Wheeler and Treisman (2002) that visual features in different dimensions are maintained in independent memory stores. These authors’ conclusions were based in part on the observation of ‘binding errors’ in a change detection task: errors caused by incorrectly combining in memory features that belong to different objects (Allen et al., 2006, Robertson, 2003, Treisman, 1998, Treisman and Schmidt, 1982, Wheeler and Treisman, 2002, Wolfe and Cave, 1999).
We have previously proposed (Bays et al., 2009) that the uniformly distributed responses interpreted by Zhang and Luck (2008) as random guesses may instead correspond to mistakenly reporting the features of one of the other items held in memory. Here, by analysing the frequency of these ‘misreporting’ errors within and across feature dimensions, we confirm that they are the result of misbinding features held in independent memory stores, consistent with the storage of visual features in separate sensory representations (Pasternak & Greenlee, 2005). These results have important implications for the nature of visual working memory representations and the locus of binding of separate features belonging to an object.
Section snippets
Experimental protocol
Ten subjects (seven male, three female; age 22–26) participated in the study after giving informed consent. All had normal or corrected-to-normal visual acuity; none reported any difficulty in making color discriminations. Stimuli were displayed on a 21-in. CRT monitor at a viewing distance of 60 cm. Eye position was monitored online at 1000 Hz using a frame-mounted infra-red eye tracker (Eyelink 1000, SR Research Ltd., Canada).
Each trial began with the presentation of a central fixation cross
Bias and precision
The recall task is illustrated in Fig. 1a. On each trial, a subject was presented with an array of colored oriented bars surrounding a central fixation point. After a blank retention interval, one array location was indicated and the subject had to reproduce from memory both the color and the orientation of the item previously displayed at that location (the target item). For each feature dimension, the recall error was defined as the deviation within the space of possible responses (Fig. 1b)
Discussion
The resolution with which visual features are stored in working memory is highly dependent on total memory load, and begins to decline as soon as the number of items in memory exceeds one (Bays and Husain, 2008, Palmer, 1990, Wilken and Ma, 2004, Zhang and Luck, 2008). However, it remains controversial whether this loss of fidelity alone accounts for all errors in recall, or whether it co-exists with a fixed upper limit on the number of objects that can be simultaneously maintained (Alvarez and
Acknowledgements
This research was supported by the Wellcome Trust and the National Institute for Health Research Clinical Biomedical Centre at University College London Hospitals/University College London.
References (51)
Information integration across saccadic eye movements
Cognitive Psychology
(1991)- et al.
A feature-integration theory of attention
Cognitive Psychology
(1980) - et al.
Illusory conjunctions in the perception of objects
Cognitive Psychology
(1982) - et al.
The psychophysical evidence for a binding problem in human vision
Neuron
(1999) - et al.
Is the binding of visual features in working memory resource-demanding?
Journal of Experimental Psychology: General
(2006) - et al.
The capacity of visual short-term memory is set both by visual information load and by number of objects
Psychological Science
(2004) - et al.
Visual working memory represents a fixed number of items regardless of complexity
Psychological Science
(2007) Precision versus capacity of working memory in schizophrenic and healthy individuals
Archives of General Psychiatry
(2010)- et al.
The precision of visual working memory is set by allocation of a shared resource
Journal of Vision
(2009) - et al.
Dynamic shifts of limited working memory resources in human vision
Science
(2008)
Response to comment on “dynamic shifts of limited working memory resources in human vision”
Science
The role of attention in binding visual features in working memory: Evidence from cognitive ageing
The Quarterly Journal of Experimental Psychology
The magical number 4 in short-term memory: A reconsideration of mental storage capacity
Behavioral and Brain Sciences
Comment on “dynamic shifts of limited working memory resources in human vision”
Science
Neural mechanisms of selective visual attention
Annual Review of Neuroscience
Modeling data using directional distributions
Selective attention and the organization of visual information
Journal of Experimental Psychology: General
Statistical analysis of circular data
What are the units of storage in visual working memory?
Journal of Vision
Attentive tracking disrupts feature binding in visual working memory
Visual Cognition
Feature bindings endure without attention: Evidence from an explicit recall task
Psychonomic Bulletin & Review
Accurate visual memory for previously attended objects in natural scenes
Journal of Experimental Psychology. Human Perception and Performance
Memory for position and identity across eye movements
Journal of Experimental Psychology. Learning, Memory, and Cognition
Integration and accumulation of information across saccadic eye movements
Cited by (191)
Disentangling processing and storage accounts of working memory development in childhood
2023, Developmental ReviewWorking memory in action: Transsaccadic working memory deficits in the left visual field and after transcallosal remapping
2023, CortexCitation Excerpt :The first of the two components most prominently loaded with measures of unsystematic error, i.e., standard deviation of errors parallel and orthogonal to the saccade. Equivalent measures of unsystematic error are commonly used in “fixational” working memory paradigms that are similar to the change detection task and that are generally assumed to test working memory capacity (e.g., average deviation or standard deviation in colour space etc.; Unsworth et al., 2014; Salahub et al., 2019; Luria et al., 2016; Bays et al., 2009, 2011; Zhang & Luck, 2008). In addition, a previous study from our lab has shown that unsystematic errors consistently increase as a function of tWM load (Frost et al., 2019).
Swap errors in visual working memory are fully explained by cue-feature variability
2022, Cognitive PsychologyNoisy and hierarchical visual memory across timescales
2024, Nature Reviews Psychology