How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness
Introduction
While reading this text, you might not be aware of other objects on your desk, even if they do project an image on your retinae. Because access to visual awareness is limited, we are not aware of the vast majority of the visual information that is readily available. Our visual system has evolved to prioritize relevant visual information over irrelevant visual information. For example, previous studies have suggested that a fearful face (Jiang, Costello, & He, 2007), a threatening symbol (Gayet, Paffen, Belopolsky, Theeuwes, & Van der Stigchel, 2016), or a salient item (Ding et al., 2019, Stuitet al., 2010) gains preferential access to awareness.
Besides these intrinsically relevant stimuli, our current mental state also influences the priority of stimuli for access to visual awareness. For instance, by requiring observers to memorize a color for later recognition, Gayetet al., 2013, Gayetet al., 2016, Gayetet al., 2019 observed that memory congruent items break interocular suppression faster than memory incongruent items. This finding reveals that an item which is currently stored in visual working memory (hereafter: VWM) is prioritized to enter awareness compared to other items. A follow-up study showed that when two items (i.e., of two distinct colors) are memorized simultaneously, each of them will be prioritized for visual awareness, suggesting that multiple features in VWM can jointly regulate visual awareness (van Moorselaar et al., 2018). Furthermore, the same principle holds for more complex memoranda that are defined by multiple features, such as faces (Liuet al., 2016, Panet al., 2014). Interestingly, Liu et al. (2016) observed that faces with threatening expressions are released from suppression faster when observers are memorizing threatening faces; an effect that was not observed for faces with happy or neutral expressions. As differences between emotional expressions are relatively subtle, this raises the question to what degree complex object representations in VWM interact with the dynamics of visual awareness. More generally, as visual items stored in VWM typically comprise multiple features, it is currently unknown whether and how these features jointly regulate access to awareness.
For the questions introduced above it is important to know how items are represented in VWM. A body of evidence suggests that multiple features of a single item can be maintained in VWM as a bound conjunction. For instance, Luck and Vogel (1997) reported that the number of items stored in memory was not different when only a single feature needed to be remembered (e.g., either their color or their orientation) or when multiple features needed to be remembered (e.g., their color as well as their orientation). Sequential studies replicated this finding thereby supporting this object-based theory of VWM (see Luria, Balaban, Awh, & Vogel, 2016 for a review; Luria and Vogel, 2011, Vogelet al., 2001). These studies suggest that the features of a memorized object in VWM are somehow linked together. Alternative to the object-based theory of VWM is the multiple-resources theory which assumes that there are separate pools of resources for maintaining features from different dimensions (Alvarez and Cavanagh, 2004, Delvenne and Bruyer, 2004, Olson and Jiang, 2002, Parraet al., 2011, Wheeler and Treisman, 2002). For instance, Alvarez and Cavanagh (2004) observed that the object-based theory cannot entirely explain the capacity of VWM. Instead, they argue that VWM load is not only determined by the number of objects, but also by the complexity of objects. Furthermore, Wheeler and Treisman (2002) observed that features from the same dimension (e.g., color) compete for storage capacity while features from different dimensions (e.g., color and orientation) are stored in parallel without competition. Delvenne and Bruyer (2004) replicated this finding, revealing that features from different dimensions can be stored without affecting capacity limits.
The discussion about the nature of VWM representations above, leads us to question whether VWM regulates the priority of an item for visual awareness along a single feature dimension and/or whether multiple features from different dimensions regulate access synergistically (i.e., as a bound entity). In this study, we opted for the feature dimensions ‘color’ and ‘shape’ to facilitate comparison with previous studies investigating the influences of VWM on perception using similar feature dimensions (e.g., Soto and Humphreys, 2009, Oliverset al., 2006, Gayetet al., 2013, Bahle et al., 2018; Soto et al., 2005, Wheeler and Treisman, 2002). To quantify the priority to visual awareness, we used the so-called breaking continuous flash suppression paradigm (i.e., b-CFS; Jiang et al., 2007; Stein, Hebart & Sterzer, 2011; Gayet, Van der Stigchel, & Paffen, 2014; Stein, 2019). In a typical b-CFS task, a target stimulus is initially suppressed from awareness by presenting it to one eye while the other eye is presented with dynamic stimuli (i.e., masks). The duration for a b-CFS target to overcome interocular suppression is an index quantifying the priority of a target to access visual awareness. In the current study, we combined b-CFS and a VWM task. In the first two experiments, we will investigate whether VWM regulates visual awareness at a specific feature dimension. We will ask observers to memorize a single feature (specifically, the color or the shape, respectively) of a memory item that contains multiple features (specifically, both a color and a shape), and vary the congruency between the b-CFS target and the memory item. In the third experiment, we will research whether VWM regulates access to visual awareness of an object along multiple features dimensions (i.e., synergistically) when multiple features of an item are simultaneously maintained in VWM (specifically, as a conjunction of both color and shape).
Section snippets
Observers
A planned number of twenty observers (age: M = 22.7, SD = 3.1; 7 males) participated in Experiment 1 after giving written informed consent. All observers reported normal or corrected-to-normal sight and having no epilepsy. The current study was approved by the local Ethical Committee of the Faculty of Social and Behavioral Sciences of Utrecht University and followed the general guidelines set out in the Declaration of Helsinki.
Design and procedure
The congruency between the memory item and the target during b-CFS
Experiment 3 – Memorize color and shape
In daily life, when memorizing an object, we typically don’t memorize only a single feature dimension (i.e., an orientation or color) but multiple feature dimensions at once (resulting in a face, or a car, etc.). As such, we questioned whether the content of VWM would regulate access to visual awareness of a concurrently presented object along multiple features dimensions (i.e., synergistically) when multiple features of an item are simultaneously maintained in VWM (i.e., as a bound object). To
General discussion
Recent studies have shown that VWM can regulate the access of visual information to visual awareness, by favoring VWM-matching stimuli compared to VWM-mismatching stimuli (Ding et al., 2019, Gayetet al., 2013, Gayetet al., 2019, Gayetet al., 2016, Liuet al., 2016, Panet al., 2014, van Moorselaaret al., 2017). In our current study, we combined a VWM task and a b-CFS task to examine whether items in VWM (comprising multiple feature dimensions) exert this regulation along a single or multi-feature
CRediT authorship contribution statement
Yun Ding: Conceptualization, Methodology, Software, Investigation, Formal analysis. Marnix Naber: Conceptualization, Methodology, Supervision. Chris Paffen: Conceptualization, Methodology, Supervision. Surya Gayet: Conceptualization, Methodology. Stefan Van der Stigchel: Conceptualization, Methodology, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (47)
- et al.
Automatic and strategic effects in the guidance of attention by working memory representations
Acta Psychologica
(2011) - et al.
Semantic and subword priming during binocular suppression
Consciousness and Cognition
(2009) - et al.
Multiple representations in visual working memory simultaneously guide attention: The type of memory-matching representation matters
Acta psychologica
(2019) - et al.
Visual input signaling threat gains preferential access to awareness in a breaking continuous flash suppression paradigm
Cognition
(2016) - et al.
Is a search template an ordinary working memory? Comparing electrophysiological markers of working memory maintenance for visual search and recognition
Neuropsychologia
(2014) - et al.
The contralateral delay activity as a neural measure of visual working memory
Neuroscience & Biobehavioral Reviews
(2016) - et al.
Shape and color conjunction stimuli are represented as bound objects in visual working memory
Neuropsychologia
(2011) - et al.
Working memory can guide pop-out search
Vision Research
(2006) - et al.
Saliency in a suppressed image affects the spatial origin of perceptual alternations during binocular rivalry
Vision Research
(2010) - et al.
The capacity of visual short-term memory is set both by visual information load and by number of objects
Psychological Science
(2004)
The architecture of interaction between visual working memory and visual attention
Journal of Experimental Psychology. Human Perception and Performance
The psychophysics toolbox
Spatial Vision
A neural theory of visual attention: Bridging cognition and neurophysiology
Psychological Review
Does visual short-term memory store bound features?
Visual Cognition
Assessing the generalizability of eye dominance across binocular rivalry, onset rivalry, and continuous flash suppression
Journal of Vision
Visual working memory and saliency independently influence the priority for access to visual awareness
Journal of Vision
Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects
Journal of Experimental Psychology: General
The role of working memory in visual selective attention
Science
Dissociated mechanisms of extracting perceptual information into visual working memory
PloS one
Breaking continuous flash suppression: Competing for consciousness on the pre-semantic battlefield
Frontiers in Psychology
Visual input that matches the content of visual working memory requires less (not faster) evidence sampling to reach conscious access
Journal of Vision
Prospectively reinstated memory drives conscious access of matching visual input
Scientific Reports
Information matching the content of visual working memory is prioritized for conscious access
Psychological Science
Cited by (3)
The linear impact of visual working memory load on visual awareness: Evidence from motion-induced blindness
2023, Consciousness and CognitionInfluence mechanism of icon semantics on visual search performance: Evidence from an eye-tracking study
2023, International Journal of Industrial ErgonomicsCitation Excerpt :Icons play an important role in improving the efficiency of information transmission and in reducing users' cognitive load (ChengPeng and Zeng, 2021; Mu et al., 2022; Pan et al., 2021). Icons convey semantics to users through users' matching of icon features and information, which is affected by their previously stored knowledge and experience (Xie et al., 2022; Ding et al., 2021). Compared with text, icon symbols can transcend language barriers and convey large amounts of information more concisely and efficiently.
VWM-based bias in conscious access can be extended to a new sandwich masking task and real-life stimuli
2023, Attention, Perception, and Psychophysics