How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness

https://doi.org/10.1016/j.concog.2020.103057Get rights and content

Highlights

  • Visual working memory can regulate visual awareness along a single feature dimension.

  • Features from different dimensions regulate visual awareness to a variable degree.

  • A dominant feature (here: color) may suppress the effect of a less dominant feature.

  • And may regulate visual awareness even when the feature dimension is task irrelevant.

Abstract

The content of visual working memory influences the access to visual awareness. Thus far, research has focused on retention of a single feature, whereas memoranda in real life typically contain multiple features. Here, we intermixed a delayed match-to-sample task to manipulate VWM content, and a breaking Continuous Flash Suppression (b-CFS) task to measure prioritization for visual awareness. Observers memorized either the color (Exp. 1), the shape (Exp. 2) or both the features (Exp. 3) of an item and indicated the location of a suppressed target. We observed that color-matching targets broke suppression faster than color-mismatching targets both when color was memory relevant or irrelevant. Shape only impacted priority for visual awareness through an interaction with color. We conclude that: (1) VWM can regulate the priority of visual information to access visual awareness along a single feature dimension; (2) different features of a memorandum vary in their potency to impact access to visual awareness, and the more dominant feature may even suppress the effect of the less dominant feature; (3) even stimuli that match an irrelevant feature dimension of the memorandum can be prioritized for visual awareness.

Introduction

While reading this text, you might not be aware of other objects on your desk, even if they do project an image on your retinae. Because access to visual awareness is limited, we are not aware of the vast majority of the visual information that is readily available. Our visual system has evolved to prioritize relevant visual information over irrelevant visual information. For example, previous studies have suggested that a fearful face (Jiang, Costello, & He, 2007), a threatening symbol (Gayet, Paffen, Belopolsky, Theeuwes, & Van der Stigchel, 2016), or a salient item (Ding et al., 2019, Stuitet al., 2010) gains preferential access to awareness.

Besides these intrinsically relevant stimuli, our current mental state also influences the priority of stimuli for access to visual awareness. For instance, by requiring observers to memorize a color for later recognition, Gayetet al., 2013, Gayetet al., 2016, Gayetet al., 2019 observed that memory congruent items break interocular suppression faster than memory incongruent items. This finding reveals that an item which is currently stored in visual working memory (hereafter: VWM) is prioritized to enter awareness compared to other items. A follow-up study showed that when two items (i.e., of two distinct colors) are memorized simultaneously, each of them will be prioritized for visual awareness, suggesting that multiple features in VWM can jointly regulate visual awareness (van Moorselaar et al., 2018). Furthermore, the same principle holds for more complex memoranda that are defined by multiple features, such as faces (Liuet al., 2016, Panet al., 2014). Interestingly, Liu et al. (2016) observed that faces with threatening expressions are released from suppression faster when observers are memorizing threatening faces; an effect that was not observed for faces with happy or neutral expressions. As differences between emotional expressions are relatively subtle, this raises the question to what degree complex object representations in VWM interact with the dynamics of visual awareness. More generally, as visual items stored in VWM typically comprise multiple features, it is currently unknown whether and how these features jointly regulate access to awareness.

For the questions introduced above it is important to know how items are represented in VWM. A body of evidence suggests that multiple features of a single item can be maintained in VWM as a bound conjunction. For instance, Luck and Vogel (1997) reported that the number of items stored in memory was not different when only a single feature needed to be remembered (e.g., either their color or their orientation) or when multiple features needed to be remembered (e.g., their color as well as their orientation). Sequential studies replicated this finding thereby supporting this object-based theory of VWM (see Luria, Balaban, Awh, & Vogel, 2016 for a review; Luria and Vogel, 2011, Vogelet al., 2001). These studies suggest that the features of a memorized object in VWM are somehow linked together. Alternative to the object-based theory of VWM is the multiple-resources theory which assumes that there are separate pools of resources for maintaining features from different dimensions (Alvarez and Cavanagh, 2004, Delvenne and Bruyer, 2004, Olson and Jiang, 2002, Parraet al., 2011, Wheeler and Treisman, 2002). For instance, Alvarez and Cavanagh (2004) observed that the object-based theory cannot entirely explain the capacity of VWM. Instead, they argue that VWM load is not only determined by the number of objects, but also by the complexity of objects. Furthermore, Wheeler and Treisman (2002) observed that features from the same dimension (e.g., color) compete for storage capacity while features from different dimensions (e.g., color and orientation) are stored in parallel without competition. Delvenne and Bruyer (2004) replicated this finding, revealing that features from different dimensions can be stored without affecting capacity limits.

The discussion about the nature of VWM representations above, leads us to question whether VWM regulates the priority of an item for visual awareness along a single feature dimension and/or whether multiple features from different dimensions regulate access synergistically (i.e., as a bound entity). In this study, we opted for the feature dimensions ‘color’ and ‘shape’ to facilitate comparison with previous studies investigating the influences of VWM on perception using similar feature dimensions (e.g., Soto and Humphreys, 2009, Oliverset al., 2006, Gayetet al., 2013, Bahle et al., 2018; Soto et al., 2005, Wheeler and Treisman, 2002). To quantify the priority to visual awareness, we used the so-called breaking continuous flash suppression paradigm (i.e., b-CFS; Jiang et al., 2007; Stein, Hebart & Sterzer, 2011; Gayet, Van der Stigchel, & Paffen, 2014; Stein, 2019). In a typical b-CFS task, a target stimulus is initially suppressed from awareness by presenting it to one eye while the other eye is presented with dynamic stimuli (i.e., masks). The duration for a b-CFS target to overcome interocular suppression is an index quantifying the priority of a target to access visual awareness. In the current study, we combined b-CFS and a VWM task. In the first two experiments, we will investigate whether VWM regulates visual awareness at a specific feature dimension. We will ask observers to memorize a single feature (specifically, the color or the shape, respectively) of a memory item that contains multiple features (specifically, both a color and a shape), and vary the congruency between the b-CFS target and the memory item. In the third experiment, we will research whether VWM regulates access to visual awareness of an object along multiple features dimensions (i.e., synergistically) when multiple features of an item are simultaneously maintained in VWM (specifically, as a conjunction of both color and shape).

Section snippets

Observers

A planned number of twenty observers (age: M = 22.7, SD = 3.1; 7 males) participated in Experiment 1 after giving written informed consent. All observers reported normal or corrected-to-normal sight and having no epilepsy. The current study was approved by the local Ethical Committee of the Faculty of Social and Behavioral Sciences of Utrecht University and followed the general guidelines set out in the Declaration of Helsinki.

Design and procedure

The congruency between the memory item and the target during b-CFS

Experiment 3 – Memorize color and shape

In daily life, when memorizing an object, we typically don’t memorize only a single feature dimension (i.e., an orientation or color) but multiple feature dimensions at once (resulting in a face, or a car, etc.). As such, we questioned whether the content of VWM would regulate access to visual awareness of a concurrently presented object along multiple features dimensions (i.e., synergistically) when multiple features of an item are simultaneously maintained in VWM (i.e., as a bound object). To

General discussion

Recent studies have shown that VWM can regulate the access of visual information to visual awareness, by favoring VWM-matching stimuli compared to VWM-mismatching stimuli (Ding et al., 2019, Gayetet al., 2013, Gayetet al., 2019, Gayetet al., 2016, Liuet al., 2016, Panet al., 2014, van Moorselaaret al., 2017). In our current study, we combined a VWM task and a b-CFS task to examine whether items in VWM (comprising multiple feature dimensions) exert this regulation along a single or multi-feature

CRediT authorship contribution statement

Yun Ding: Conceptualization, Methodology, Software, Investigation, Formal analysis. Marnix Naber: Conceptualization, Methodology, Supervision. Chris Paffen: Conceptualization, Methodology, Supervision. Surya Gayet: Conceptualization, Methodology. Stefan Van der Stigchel: Conceptualization, Methodology, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (47)

  • B. Bahle et al.

    The architecture of interaction between visual working memory and visual attention

    Journal of Experimental Psychology. Human Perception and Performance

    (2018)
  • D.H. Brainard

    The psychophysics toolbox

    Spatial Vision

    (1997)
  • C. Bundesen et al.

    A neural theory of visual attention: Bridging cognition and neurophysiology

    Psychological Review

    (2005)
  • J. Delvenne et al.

    Does visual short-term memory store bound features?

    Visual Cognition

    (2004)
  • Y. Ding et al.

    Assessing the generalizability of eye dominance across binocular rivalry, onset rivalry, and continuous flash suppression

    Journal of Vision

    (2018)
  • Y. Ding et al.

    Visual working memory and saliency independently influence the priority for access to visual awareness

    Journal of Vision

    (2019)
  • R. Egly et al.

    Shifting visual attention between objects and locations: Evidence from normal and parietal lesion subjects

    Journal of Experimental Psychology: General

    (1994)
  • J.W. de Fockert et al.

    The role of working memory in visual selective attention

    Science

    (2001)
  • Z. Gao et al.

    Dissociated mechanisms of extracting perceptual information into visual working memory

    PloS one

    (2010)
  • S. Gayet et al.

    Breaking continuous flash suppression: Competing for consciousness on the pre-semantic battlefield

    Frontiers in Psychology

    (2014)
  • S. Gayet et al.

    Visual input that matches the content of visual working memory requires less (not faster) evidence sampling to reach conscious access

    Journal of Vision

    (2016)
  • S. Gayet et al.

    Prospectively reinstated memory drives conscious access of matching visual input

    Scientific Reports

    (2019)
  • S. Gayet et al.

    Information matching the content of visual working memory is prioritized for conscious access

    Psychological Science

    (2013)
  • Cited by (3)

    • Influence mechanism of icon semantics on visual search performance: Evidence from an eye-tracking study

      2023, International Journal of Industrial Ergonomics
      Citation Excerpt :

      Icons play an important role in improving the efficiency of information transmission and in reducing users' cognitive load (ChengPeng and Zeng, 2021; Mu et al., 2022; Pan et al., 2021). Icons convey semantics to users through users' matching of icon features and information, which is affected by their previously stored knowledge and experience (Xie et al., 2022; Ding et al., 2021). Compared with text, icon symbols can transcend language barriers and convey large amounts of information more concisely and efficiently.

    View full text