Cognitive penetration of early vision in face perception
Introduction
Cognitive and affective penetration of perception refers to the influence that higher mental states have on perceptual systems. This phenomenon is observed when beliefs, expectations, or feelings, moods, among other states, modulate perceptual processing (Churchland, 1979, Churchland, 1989, Fodor, 1983, Fodor, 1984, Fodor, 1988, Fodor, 2000, Fodor and Pylyshyn, 1981, Raftopoulos, 2001c, Raftopoulos, 2001a, Raftopoulos, 2001b, Raftopoulos, 2009, Raftopoulos, 2011, Raftopoulos, 2017). Specifically, this debate concerns the influence of top-down cognitive or affective signals on early stages of visual processing. Cognitive penetration of the early visual system would be observed if persistent illusions, such as Müller-Lyer or Ponzo illusions, are the result of neural reorganisation elicited by the constant influence of cognitive states, e.g., our knowledge of perspective and geometry, on early stages of visual processing (Churchland, 1988, McCauley and Henrich, 2006). Affective penetration seems to occur when top-down affective states, such as fear or anger, influence early vision so that fearful or threatening objects are perceived more accurately (Gamond et al., 2011, Morel et al., 2009, Morel et al., 2014, Soares and Esteves, 2013, Stolarova et al., 2006, Zhu and Luo, 2012).1
Cognitive and affective penetration of perception appears to influence the system’s behaviour, its structural organisation, and the processing of the stimuli (content of perception). Penetrating states can modify the system’s behaviour by heightening the cortical representation of some stimuli and boosting visual processing (Miskovic and Keil, 2013, Oosterwijk et al., 2016). Emotions like fear seem to modulate the visual processing at very short latencies by sensitising the system to detect potential threat faster (Brendel, Hecht, DeLucia, & Gamer, 2014) and prepare the organism for defensive actions (e.g., flee or fight) (Weymar, Keil, & Hamm, 2014). Thus, compared to neutral or positive stimuli, fearful stimuli like facial expressions (Marchi and Newen, 2015, Morel et al., 2009, Soares and Esteves, 2013, Wang et al., 2012, Zhang et al., 2017, Zhu and Luo, 2012) or threatening stimuli like angry faces (Zhang et al., 2017) or spiders (Brendel et al., 2014, Domínguez-Borràs et al., 2017) can be detected faster and more accurately. Likewise, attentional mechanisms also seem to be modulated by emotions (Aue and Okon-Singer, 2015, Pourtois et al., 2004, Soares and Esteves, 2013, Zhang et al., 2017), intentions (Land, 2006, Land, 2009, Mole, 2015, Wu, 2013, Wu, 2014, Wu, 2017), or previous knowledge (Duclos, 2015).
As a consequence of the plastic condition of the brain, higher states appear to cause structural (architectural) changes (Gilbert et al., 2009, Gilbert and Li, 2012, Pourtois et al., 2008). Top-down modulatory states triggered during a perceptual learning task might induce neural reorganisation (Byers and Serences, 2012, Cecchi, 2014, Gatzia and Brogaard, 2017, Hohwy, 2017, Li et al., 2004, Makino and Komiyama, 2015, Zhang et al., 2015). For instance, the beliefs and intentions involved in an intensive training task might be responsible for subsequent long-lasting neural adaptation in the visual cortex (Byers and Serences, 2012, Cecchi, 2014, Rauss et al., 2009, Rauss et al., 2011).
Cognitive and affective penetrating states can also affect the processing of the stimuli and modulate the content of perception (Arstila, 2018, Borst and Kosslyn, 2010, Gatzia and Brogaard, 2017, Miskovic and Keil, 2013, Morel et al., 2014, Oosterwijk et al., 2016). Desires (Balcetis and Dunning, 2006, Balcetis and Dunning, 2007), motivation or optimism (Witt & Proffitt, 2005), and hunger or thirst (Balcetis & Dunning, 2010) can bias the perception of distances and objects’ sizes.
The cognitive and affective penetration of perception is of great importance for philosophy, psychology, neuroscience, and other fields like psychiatry, marketing, consumer behaviour, and finance.
From a philosophical perspective, cognitive and affective influences on perception represent an epistemic problem. Perceptual experiences are the foundation of our visual knowledge since it is on the basis of what we see that we are justified in believing one way or another (Engel, 2007, Lyons, 2011, Siegel, 2012, Siegel, 2013b, Siegel and Silins, 2015). If the perceptual process is corrupted by our cognitive or affective background, we might not see the world as it is but as we want or expect it to be (Fodor, 1983, Pylyshyn, 1980). Therefore, we might fail to know the world (Clark, 2016, Lyons, 2011, Lyons, 2015, Lyons, 2016, Macpherson, 2012, Macpherson, 2017, Siegel, 2012).
The apparent encapsulation (isolation) of the visual system from cognitive and affective states appears to have behavioural advantages for the speed and the objectivity of perceptual integration (Fodor, 1983, Pylyshyn, 1984, Raftopoulos, 2001c), and the reliability of the visual processing (Fodor, 1983, p. 68). In other words, perceptual modularity seems to make the world safe for knowledge (Lyons, 2011, p. 305). (See also Burnston and Cohen, 2015, Fodor, 1984, Fodor, 1988, Fodor, 2000, Pylyshyn, 1980, Pylyshyn, 1984, Pylyshyn, 1999, Pylyshyn, 2003, Raftopoulos, 2001a, Raftopoulos, 2009, Raftopoulos, 2017.) However, recent research shows that cognitive and affective influences on perception appear to be the norm of the brain. The subject’s cognitive and affective background plays a fundamental role in the way we perceive objects, as it helps to interpret the environment in the most adequate manner (Churchland, 1979, Churchland, 1989, Clark, 2013, Clark, 2014, Clark, 2015, Clark, 2016, Lyons, 2011, Marchi and Newen, 2015, Machery, 2015, Macpherson, 2012, Macpherson, 2015, Macpherson, 2017, Ogilvie and Carruthers, 2016, Teufel and Nanay, 2017).
A similar ongoing debate is observed in psychology. Theoretical approaches (Balcetis, 2016, Churchland et al., 1994, Collins and Olson, 2014, Hohwy, 2013, Hohwy, 2017, Lupyan, 2012, Lupyan, 2015, Lupyan et al., 2010, Newen and Vetter, 2017, Vetter and Newen, 2014) and empirical evidence seem to show that higher influences on visual processing have consequences for human behaviour. Penetrating higher states might harm social interaction if faces look angrier than they really are (Zhang et al., 2017), discourage actions if distances or heights look bigger than expected (Storbeck and Stefanucci, 2014, Stefanucci and Proffitt, 2009), alter the performance of a task if objects look different (den Daas et al., 2013, Witt and Proffitt, 2005), affect business if beverages taste less palatable than they normally do (Harrar et al., 2011, Piqueras-Fiszman and Spence, 2012, Wanab et al., 2015), and the like. Still, some psychologists argue that there is no conclusive evidence for the cognitive and affective penetration of perception (Firestone and Scholl, 2014, Firestone and Scholl, 2015a, Firestone and Scholl, 2015b, Firestone and Scholl, 2016, Pylyshyn, 1999, Pylyshyn, 2003).
Finally, the debate extends to the field of neuroscience. A successful interaction with the surrounding world depends on the organism’s ability to predict future events and plan behaviour. The brain is continuously storing and updating information about associations between objects, events, and their specific contexts. Rather than passively waiting to be activated by external inputs, the brain takes advantage of stored associations to generate predictions about the world it encounters. Predictions facilitate perception by helping to construct a coherent representation of the incoming inputs based on the stored information (Baars and Gage, 2010, Bar, 2004, Bar, 2007, Bar, 2009b, Bar, 2009a, Bar et al., 2006, Friston, 2005, Friston, 2008, Gilbert and Sigman, 2007, Irwin and Thomas, 2008, Kveraga et al., 2007, Panichello et al., 2013, Perlman et al., 2016).
Some predictive processes are achieved by affective and cognitive states that might strengthen visual processing efficiency, enhance synapses, elicit neural reorganisation, and the like (Brendel et al., 2014, Rauss et al., 2011, Weymar et al., 2014). These effects have important theoretical and empirical consequences for brain architecture (e.g., anatomy, plasticity) (Gilbert et al., 2009, Gilbert and Li, 2012, Qin and Yu, 2013), brain functioning (e.g., consciousness, perceptual learning, development) (Bar, 2009b, Bar, 2009a, Byers and Serences, 2012, Cheung and Bar, 2014, O’Callaghan et al., 2017, Panichello et al., 2013, Rauss et al., 2011, Rauss and Pourtois, 2013, Trapp and Bar, 2015), and explanatory models (e.g., predictive coding, ERP analysis) (Clark, 2013, Clark, 2016, Hohwy, 2013, Rauss et al., 2012, Rauss and Pourtois, 2013, Spratling, 2016). For instance, empirical studies seem to show that higher states modulate early visual areas by influencing visual content and brain architecture (Brendel et al., 2014, Morel et al., 2009, Pourtois et al., 2013, Rauss et al., 2009, Rauss et al., 2012, Rauss et al., 2011, Stolarova et al., 2006, Weymar et al., 2014, Zhu and Luo, 2012). Nonetheless, some neuroscientists claim that these results are controversial because they seem to rely on theoretical assumptions regarding how event related potential (ERP) components, such as C1, should be interpreted (Ding et al., 2014, Fu et al., 2010a, Fu et al., 2010b, Fu and Fedota, 2012, Martinez et al., 2001).
The cognitive and affective penetration of early vision has been tackled from different angles in philosophy. Some researchers have focused on understanding the nature of bottom-up and top-down processes in visual perception (Teufel & Nanay, 2017; see Rauss et al., 2011, Gilbert and Sigman, 2007, Gilbert and Li, 2013; for scientific literature). Other philosophers have argued for the necessity of top-down signals to achieve perceptual computation (Marchi and Newen, 2015, Newen and Vetter, 2017; see Bar et al., 2006, Cheung and Bar, 2014, O’Callaghan et al., 2017, Piëch et al., 2013; for non-philosophical sources). Furthermore, theoretical approaches have postulated that top-down effects result from predictive processes in the brain (Clark, 2013; see Fenske et al., 2006, Hohwy, 2013, Hohwy, 2017, Lupyan, 2015; for scientific references). Meanwhile, recent empirical studies seem to provide strong evidence in favour of the cognitive and affective penetration of early vision (Meeren et al., 2008, Stolarova et al., 2006, Zhang et al., 2015, Zhu and Luo, 2012). However, whereas the eager advocates of the impenetrability of perception (e.g., Raftopoulos, 2009, Raftopoulos, 2017) provide rigorous analyses of empirical evidence against the cognitive or affective penetration of early vision, the same analysis is overlooked in the opposite camp. That is, the upholders of the cognitive and affective penetrability of perception barely scrutinise empirical aspects such as the origin and target of brain signals, their loci, and their time course. The literature on the matter remains thus highly speculative.
This article intends to bridge the above-mentioned gap by providing a detailed analysis of empirical evidence supporting the cognitive penetration of early vision. Firstly, I briefly explain visual processing and what is at issue in the cognitive and affective penetrability debate (Section 2). Secondly, I present the problems faced by psychological studies in accounting for the penetration of perception and argue that psychological evidence does not seem to be either sufficient or necessary to support or reject the penetrability of early vision (Section 3). In Section 4, I scrutinise some empirical findings and claim that they can, at best, show penetration of late vision. Thirdly, I introduce a neuroscientific study and scrutinise its findings with regard to the origin, time course, and impact of cognitive signals in face perceptual areas, as well as their epistemic consequences. I argue then that the study provides well-grounded evidence for the cognitive penetration of early visual face perception (Section 5). Later, I refute some alternative explanations to my argument (Section 6) and conclude in Section 7.
Section snippets
Levels of visual processing
The visual system is the human interface with the visual world as it represents the objects, events, and properties of our surroundings. Vision or visual perception is the process that begins with the computation of a physical stimulus on the perceptual organ (i.e., the retina) and finishes, e.g., with the generation of a perceptual experience (i.e., the conscious representation a subject has normally while seeing an object).
This process is frequently divided into early and late vision (
Psychological evidence and cognitive penetration of perception
Arguments for or against cognitive/affective penetrability of perception frequently rely on psychological evidence. The subject’s behaviour or verbal reports are taken as a confirmation of whether a perceptual system has or has not been penetrated. However, this type of evidence is not sufficient (and not necessary, as I argue later) to account for cognitive or affective penetration of perception.
The first problem regarding psychological studies concerns their inadequacy to exclude any
Cognitive penetration of late vision
To show cognitive or affective penetration of late vision, empirical studies need to demonstrate, in addition to behavioural consequences, that top-down higher signals affect the visual system within the time window between 100 to 300 ms post-stimulus onset.
Balcetis and Dunning (2006) show that rewards and penalisations modulate perceptual content. Subjects were rewarded if they identified a figure as a letter or penalised when they identified the same figure as a number. Some of these studies
Cognitive penetration of early vision in face perception
On a first encounter with a person we have never met before, the brain tends to spontaneously infer personality traits and social categories (Bar et al., 2006, Gamond et al., 2011, Kveraga et al., 2007, Willis and Todorov, 2006). Inferring someone’s personality from her face or her photograph is a pervasive and automatic brain behaviour that occurs even when no information about the subject’s character is available. In less than 40 ms of exposure the brain can gather the first impressions about
Alternative explanations to cognitive penetration of early vision
From the mainstream viewpoint, all the physical features of a face are first processed bottom-up and in parallel (at the same time and without preferences for each other). Later, top-down influences achieve the emotional analysis and the face would be perceived as flexible or determined. However, Gamond et al.’s study shows that a short training associating personality traits to facial features can modulate earlier stages of this process.
Brain imagining techniques show, on the one hand, that
Conclusion
The case study analysed in this paper shows that the perceptual content of physical properties (inter-eye distance) was influenced by the conceptual content of personality traits (flexible or determined). Top-down signals neither increased sensitisation to visual features nor modulated attentional mechanisms. In addition, subjects’ responses in the final session did not reflect their judgement but a real perceptual phenomenon resulting from the association between mnemonic (personality traits)
Acknowledgements
This work was supported by the Swiss National Science Foundation (SNSF): 171750 and 139820. I am grateful to two anonymous reviewers for their valuable suggestions on an early version of this manuscript; their insight has been key in improving my argument and situating the present debate within the current framework of cognitive penetration of perception.
References (220)
When is cognitive penetration a plausible explanation?
Consciousness and Cognition
(2018)- et al.
Expectancy biases in fear and anxiety and their link to biases in attention
Clinical Psychology Review
(2015) The proactive brain: Using analogies and associations to generate predictions
Trends in Cognitive Sciences
(2007)Are there unconscious perceptual processes?
Consciousness and Cognition
(2011)Non-visual consciousness and visual images in blindsight
Consciousness and Cognition
(2012)- et al.
Exploring the relationship between perceptual learning and top-down attentional control
Vision Research
(2012) - et al.
Recognition advantage of happy faces: Tracing the neurocognitive processes
Neuropsychologia
(2013) - et al.
Perceptual, categorical, and affective processing of ambiguous smiling facial expressions
Cognition
(2012) - et al.
Time course of discrimination between emotional facial expressions: The role of visual saliency
Vision Research
(2011) Visual cognition
Vision Research
(2011)
Experimental and theoretical approaches to conscious processing
Neuron
The psychology of investment behavior: (De)biasing financial decision-making one graph at a time
Journal of Consumer Psychology
Enhanced visual awareness for morality and pajamas? Perception vs. memory in ‘top-down’ effects
Cognition
How direct is visual perception? some reflections on Gibson’s ’Ecological Approach’
Cognition
Time course of implicit processing and explicit processing of emotional faces and emotional words
Biological Psychology
Attentional load is not a critical factor for eliciting C1 attentional effect – A reply to Rauss, Pourtois, Vuilleumier, and Schwartz
Biological Psychology
Early interaction between perceptual load and involuntary attention: An event-related potential study
Neuroscience Letters
Dissociation of visual C1 and P1 components as a function of attentional load: An event-related potential study
Biological Psychology
Early influence of prior experience on face perception
NeuroImage
Adult visual cortical plasticity
Neuron
Brain states: Top-down influences in sensory processing
Neuron
Priors in perception: Top-down modulation, Bayesian perceptual learning rate, and prediction error minimization
Consciousness and Cognition
Neural events and perceptual awareness
Cognition
Top-down predictions in the cognitive brain
Brain and Cognition
The time-course of activation in the dorsal and ventral visual streams during landmark cueing and perceptual discrimination tasks
Neuropsychologia
Why visual attention and awareness are different
Trends in Cognitive Science
Approach and avoidance as organizing structures for motivated distance perception
Emotion Review
See what you want to see: Motivational influences on visual perception
Journal of Personality and Social Psychology
Cognitive dissonance and the perception of natural environments
Psychological Science
Wishful seeing: Desired objects are seen as closer
Psychological Science
Visual objects in context
Nature Reviews Neuroscience
The proactive brain: Memory for predictions
Philosophical Transactions of the Royal Society B
Top-down facilitation of visual recognition
Proceedings of the National Academy of Sciences of the United States of America
Very first impressions
Emotion
Emotion, decision making and the orbitofrontal cortex
Cerebral Cortex
The value factor in children’s size perception
Child Development
Fear selectively modulates visual mental imagery and visual perception
The Quarterly Journal of Experimental Psychology
Emotional effects on time-to-contact judgments: Arousal, threat, and fear of spiders modulate the effect of pictorial content
Experimental Brain Research
Conscious vision for action versus unconscious vision for action?
Cognitive Science
Vision for action and the contents of perception
The Journal of Philosophy
Is color experience cognitively penetrable?
Topics in Cognitive Science
Value and need as organizing factors in perception
Journal of Abnormal and Social Psychology
Some determinants of apparent size
The Journal of Abnormal and Social Psychology
Lifting without seeing: The role of vision in perceiving and acting upon the size weight illusion
PLoS ONE
Perceptual integration, modularity, and cognitive penetration
How context influences our perception of emotional faces: A behavioral study on the Kuleshov effect
Frontiers in Psychology
Cognitive penetration, perceptual learning, and neural plasticity
Dialectica
Cited by (5)
Recognition of Facial and Vocal Emotions Expressions by SOAR Model
2023, Journal of Information Systems and TelecommunicationPredictive coding and the strong thesis of cognitive penetrability
2021, Theoria (Spain)Thinking and perceiving
2021, Thinking and PerceivingTemporal Structure of Now from a Close-Up View
2021, Understanding Complex SystemsCognitive penetration and taste predicates: Making an exception to the rule
2021, Filosofia Unisinos