Original ArticlesWhat kind of empirical evidence is needed for probabilistic mental representations? An example from visual perception
Introduction
Within cognitive science the mind is considered to be an information processing system that makes inferences about the external states of the world using information from the senses or memory. However, this information is generally noisy and incomplete. One of the main challenges for cognition is therefore to make reliable inferences about the world in the face of uncertainty. This entails the idea that our brains perform probabilistic calculations involving uncertainty. With advances in computer science and mathematical modeling, probabilistic approaches to cognition involving Bayesian statistics have become a unifying framework for studying human cognition (for reviews see: Chater, Tenenbaum, & Yuille, 2006; Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010).
These probabilistic models and approaches are particularly advanced and successful in the field of visual perception (e.g., Kersten & Yuille, 2003; Mamassian, Landy, & Maloney, 2002). A crucial assumption of these Bayesian theories is that the brain represents information probabilistically. For example, information is considered to be represented as a conditional probability density function of a set of hypotheses about a distal stimulus rather than as a single estimate of that stimulus. While a large amount of experimental results is consistent with probabilistic representations, experiments that directly investigate how information is represented by the brain are scarce (Knill & Pouget, 2004). Carefully designed experiments are needed to elevate this claim to more than simply an assumption in the field.
Recently, empirical support in favour of probabilistic representations in visual perception has been strongly criticized (Rahnev, 2017; Block, 2018; Yeon & Rahnev, 2020, see discussion in Rahnev, Block, Denison, & Jehee, 2021). In Section 2, we provide a short review of these recent criticisms, which involve the claim that proposed empirical evidence for probabilistic representations in the literature can be explained by positing non-probabilistic representations. While we agree with such criticisms, in Section 3 we present a recently developed psychophysical methodology, Feature Distribution Learning (FDL, Chetverikov et al., 2016, Chetverikov et al., 2017a), and argue in Section 4 that the FDL method enables the study of perceptual representations by avoiding the methodological criticisms of experimental studies that attempt to provide empirical evidence for probabilistic representations. We discuss the first criterion for providing evidence for probabilistic representations, which is to demonstrate that representations that involve probability distributions are not imposed on the task by the experimenter, but instead generated by the brain to be utilized later. In Section 5, we propose a second criterion to provide empirical evidence for probabilistic representations. According to our account, experimental results that demonstrate the utilization of correlations between the internal states of the visual system and stimulus uncertainty will not suffice as evidence for probabilistic representations as they are defined in the empirical literature, whereas demonstrating utilization of structural correspondence between the two would. Subsequently, we demonstrate how the results obtained with the FDL method constitute a prime example of such evidence. In Section 6, we present our overall conclusions and discuss unanswered questions that warrant further investigation.
Our main argument here is mainly built on Block (2018) and Rahnev's (2017) criticisms of the notion of probabilistic representations that incorporate probability distributions over possible estimates of a visual feature, which we take to be the notion posited in perceptual sciences. One can argue against Block's criticism by appealing to different notions of representation that incorporate probabilities in different ways (e.g., Gross, 2020; Shea, 2020; Shea & Frith, 2019). However, here, we mostly agree with Block's or Rahnev's criticisms; and we use them as stepping-stones to elaborate on what satisfactory empirical evidence for probabilistic representations should look like, by presenting a particular psychophysical method (and the results obtained from this method) as a prime example of this.
Section snippets
Criticisms of experiments supporting probabilistic representations
Recent criticisms of empirical evidence for probabilistic representations can be grouped into two categories. The first is directed at the design of perceptual studies providing evidence for probabilistic representations, whereas the second category focuses on the interpretation of the results obtained from such studies.
The feature distribution learning (FDL) method
Recently, Chetverikov et al., 2016, Chetverikov et al., 2017a; Chetverikov, Campana, and Kristjánsson (2017c) introduced a new method to assess representations of visual feature distributions. Their results indicate that observers encode not only the summary statistics of visual feature distributions, but also the distributions themselves.
In classical visual search tasks, observers see a display containing several items composed of distractors and a target. Sometimes observers are told what
How does the FDL method differ from methodologies of other experiments supporting probabilistic representations?
The overarching problem with providing convincing empirical evidence for probabilistic representations is that observers' responses vary, but its source is difficult to identify. When faced with a perceptual decision on colour, an observer might respond “red” 70% of the time. This observation does not directly demonstrate that the observer has a representation assigning 0.7 probability to “red”. This response variation may also reflect artefacts from subjective guessing mentioned by Block (2018)
Summary of key results obtained with FDL
In Chetverikov et al. (2016), the distractor distribution used on learning trials was either Gaussian or uniform (Fig. 3). They found remarkable correspondence between the shape of the underlying representation of the distractor distribution (i.e., CT-PD curves) and the shape of the physical distractor distribution used as the stimulus. This correspondence is especially important given that the two different distractor distributions had the same mean and range, which are summary statistical
Conclusions
Probabilistic approaches to perception and cognition have had great success, especially in building computational models of perceptual processes. This has led researchers to propose that the brain represents information probabilistically. Highly influential studies strongly suggest that probabilistic representations are used in visual perception. However, the methodology of such studies prevents them from providing clear evidence in favour of probabilistic representations (Block, 2018).
Acknowledgments
We thank for the valuable feedback from the attendees of the “Representation in Cognitive Science” workshop organized by Tobias Schlicht and Krzysztof Dolega at Ruhr-Universitat Bochum, where an earlier version of this paper was presented. ODT, SHR and AK are supported by grant IRF #173947-052 from the Icelandic Research Fund. AC is supported by Radboud Excellence Fellowship.
References (67)
Representing multiple objects as an ensemble enhances visual cognition
Trends in Cognitive Sciences
(2011)The stage of priming: Are intertrial repetition effects attentional or decisional?
Vision Research
(2008)- et al.
Probabilistic models of cognition: Conceptual foundations
Trends in Cognitive Sciences
(2006) - et al.
Building ensemble representations: How the shape of preceding distractor distributions affects visual search
Cognition
(2016) - et al.
Learning features in a complex and changing environment: A distribution-based framework for visual attention and vision in general
- et al.
Probabilistic rejection templates in visual working memory
Cognition
(2020) What is a visual object?
Trends in Cognitive Sciences
(2003)- et al.
Probabilistic models of cognition: Exploring representations and inductive biases
Trends in Cognitive Sciences
(2010) - et al.
Bayesian models of object perception
Current Opinion in Neurobiology
(2003) - et al.
The Bayesian brain: The role of uncertainty in neural coding and computation
Trends in Neurosciences
(2004)
Bayesian decision theory in sensorimotor control
Trends in Cognitive Sciences
Attentional priming: Recent insights and current controversies
Current Opinion in Psychology
Priming in visual search: Separating the effects of target repetition, distractor repetition and role-reversal
Vision Research
Bayesian brains without probabilities
Trends in Cognitive Sciences
The global workspace needs metacognition
Trends in Cognitive Sciences
Interactions between visual working memory representations
Attention, Perception, & Psychophysics
If perception is probabilistic, why doesn’t it seem probabilistic?
Philosophical Transactions of the Royal Society B
Representing color ensembles
Psychological Science
Rapid learning of visual ensembles
Journal of Vision
Feature distribution learning (FDL): A new method for studying visual ensembles perception with priming of attention shifts
Humans integrate visual and haptic information in a statistically optimal fashion
Nature
Serial dependence in visual perception
Nature Neuroscience
The Organization of Learning
Where meanings arise and how: Building on Shannon’s foundations
Mind & Language
The perception of probability
Psychological Review
Distractor ignoring: Strategies, learning, and passive filtering
Current Directions in Psychological Science
Understanding priming of color-singleton search: Roles of attention at encoding and “retrieval.”
Perception & Psychophysics
Bridging levels of analysis for probabilistic models of cognition
Current Directions in Psychological Science
Perceptual consciousness and cognitive access from the perspective of capacity-unlimited working memory
Philosophical Transactions of the Royal Society, B: Biological Sciences
Probabilistic representations in perception: Are there any, and what would they be?
Mind & Language
Representing color and orientation ensembles: Can observers learn multiple feature distributions?
Journal of Vision
Comparing set summary statistics and outlier pop out in vision
Journal of Vision
Subjective probability as sampling propensity
The Review of Philosophy and Psychology
Cited by (6)
Attentional priming in Go No-Go search tasks
2023, Vision ResearchThe Haptic Intensity Order Illusion Is Caused by Amplitude Changes
2023, ACM Transactions on Applied PerceptionPriming of probabilistic attentional templates
2023, Psychonomic Bulletin and Review