Elsevier

Neuropsychologia

Volume 64, November 2014, Pages 145-156
Neuropsychologia

Research report
Cortical and subcortical contributions to state- and strength-based perceptual judgments

https://doi.org/10.1016/j.neuropsychologia.2014.09.025Get rights and content

Highlights

  • Change detection can be based on state-based or strength-based signals.

  • Examined neural underpinnings of state- and strength-based perceptions.

  • Temporo-parietal lesions led to selective deficits in strength-based perception.

  • Subcortical lesions led to both state- and strength-based deficits.

  • State- and strength-based perceptions are neurally dissociable.

Abstract

Perceptual judgments can be made on the basis of different kinds of information: state-based access to specific details that differentiate two similar images, or strength-based assessments of relational match/mismatch. We explored state- and strength-based perception in eleven right-hemisphere stroke patients, and examined lesion overlap images to gain insight into the neural underpinnings of these different kinds of perceptual judgments. Patients and healthy controls were presented with pairs of scenes that were either identical or differed in that one scene was slightly expanded or contracted relative to the other. Same/different confidence judgments were used to plot receiver-operating characteristics and estimate the contributions of state- and strength-based perception. The patient group showed a significant and selective impairment of strength-based, but not state-based, perception. This finding was not an artifact of reduced levels of overall performance, because matching perceptual discriminability levels between controls and patients revealed a double dissociation, with higher state-based, and lower strength-based, perception in patients vs. controls. We then conducted exploratory follow-up analyses on the patient group, based on the observation of substantial individual differences in state-based perception — differences that were masked in analyses based on the group mean. Patients who were relatively spared in state-based perception (but impaired in strength-based perception) had damage that was primarily in temporo-parietal cortical regions. Patients who were relatively impaired in both state- and strength-based perception had overlapping damage in the thalamus, putamen, and adjacent white matter. These patient groups were not different in any other measure, e.g., presence of spatial neglect symptoms, age, education, lesion volume, or time since stroke. These findings shed light on the different roles of right hemisphere regions in high-level perception, suggesting that the thalamus and basal ganglia play a critical role in state- and strength-based perception, whereas temporo-parietal cortical regions are important for intact strength-based perception.

Introduction

How do we detect changes in the environment? Imagine you are shown two photographs of a park and asked whether they are exactly the same or if something about the park was different in the two images. In some cases, you may be able to detect a specific difference—for example, a water fountain that is in one picture but not in the other. Alternatively, you may know that the pictures are different, but are unable to provide details about any specific change.

Thus, there are two kinds of information that can be used for perceptual change detection, which have been referred to as state-based and strength-based perception (Aly and Yonelinas, 2012; for related distinctions, see Fernandez-Duque and Thornton, 2000, Rensink, 2000, Rensink, 2004, Dehaene et al., 2006, Howe and Webb, 2014). State- and strength-based perception have been studied by asking individuals to make same/different confidence judgments on pairs of images (e.g., pairs of scenes, faces, fractals, or objects; Aly and Yonelinas, 2012, Aly et al., 2013, Aly et al., 2014). Receiver-operating characteristic (ROC; Green and Swets, 1966, Macmillan and Creelman, 2005) analyses are then used to estimate the contributions of two kinds of perceptual decisions.

State-based perception is associated with high-confidence responses that are rarely in error; it is a discrete state that either occurs or does not, and when it does occur, it is associated with accurate awareness of specific details that differentiate two images. The probability of state-based perception is reflected in the upper x-intercept of ROCs (Fig. 1). Strength-based perception, on the other hand, is associated with a wider range of confidence responses; it is a continuously-graded signal associated with a feeling that something has changed, with little to no ability to report what that change was. The discriminability afforded by strength-based perception is related to the curvilinearity of ROCs (Fig. 1).

In previous studies, we have found that these two kinds of perception can be doubly dissociated, have different temporal dynamics, and are associated with distinct kinds of conscious experiences (Aly and Yonelinas, 2012, Aly et al., 2013, Aly et al., 2014). For example, state-based perception makes a greater contribution to tasks involving detection of discrete object changes (e.g., a water fountain that is present in one scene but absent in another), is associated with a rapid temporal onset, and subjective experiences are those of consciously perceiving specific, detailed differences. In contrast, strength-based perception makes a greater contribution to tasks involving global or relational change detection (e.g., a subtle manipulation of the distances between component parts of a scene), is associated with a gradual temporal onset, and subjective experiences are those of feeling as if a change has occurred but being unable to pinpoint what that change was (Aly and Yonelinas, 2012; also see Fernandez-Duque and Thornton, 2000; Rensink, 2000, Rensink, 2004; Dehaene et al., 2006; Galpin et al., 2008; Busch et al., 2009, Busch et al., 2010; Howe and Webb, 2014; but see Simons et al., 2005).

Thus, previous behavioral work on state- and strength-based perception has shown that perceptual decisions can be made on the basis of functionally dissociable processes or representations. State- and strength-based perception may reflect differences at early- to mid-level stages of perceptual representation (i.e., what information is represented in visual cortex, depending on the focus of attention) or later stages of decision-making (i.e., what information is used to inform the perceptual decision). While current data do not allow adjudication between these possibilities, it is clear that independent sources of information can be used to guide perceptual judgments.

In a previous neuropsychological study, we investigated the contribution of the hippocampus and surrounding medial temporal lobe (MTL) cortex to state- and strength-based perception (Aly et al., 2013). We tested patients with selective lesions to the hippocampus, bilaterally, and patients with more extensive unilateral MTL lesions that included the hippocampus and surrounding cortex. On each trial, patients and healthy controls were presented with a pair of scenes that were either identical or differed in that the center of one scene was expanded or contracted relative to the other (Fig. 1A). These changes alter the relational or configural information within the scenes without adding or removing any specific objects. Participants made same/different confidence judgments using a 1–6 scale, and these confidence responses were used to plot ROCs (Green and Swets, 1966, Macmillan and Creelman, 2005). The ROCs were in turn used to estimate state- and strength-based perception (see Fig. 1B for hypothetical data). The upper x-intercept of an ROC provides the probability that state-based perception has occurred, while the degree of curvilinearity is proportional to the contribution of strength-based perception (Aly and Yonelinas, 2012; see also Yonelinas, 1994).

Using this approach, we found that the patients were selectively impaired in strength-based perception (graded judgments of the overall configural or relational match/mismatch between images) but showed intact state-based perception (related to the ability to identify specific detailed differences between scenes; Aly and Yonelinas, 2012). This was true for patients with selective hippocampal lesions as well as those with more extensive MTL lesions. These data suggested that the hippocampus is critical for detecting configural or relational match/mismatch between complex scenes, but is not needed for state-based judgments based on identification of specific, item-level differences.

The MTL is just one of several regions that are likely to be critical for perceptual judgments on complex scenes. In a previous fMRI study (Aly et al., 2014), we examined whole-brain data to determine whether activity in different brain regions was differentially correlated with state- or strength-based perception. Individuals performed a task similar to that used in the MTL patient study, in which they viewed pairs of images and made same/different confidence judgments. These judgments were made using a scale that allowed individuals to report when state-based perception occurred, or, if it did not occur, to rate the confidence associated with strength-based perception. Activity in the supramarginal gyrus, posterior cingulate cortex, and precuneus was related to the occurrence of state-based perception, and was not modulated by varying confidence of strength-based perception. Activity in the fusiform gyrus, however, was sensitive to strength-based, but not state-based, perception. The lateral occipital complex showed both effects: that is, this region showed a graded increase in activity as confidence in strength-based perception increased, and showed an additional increase in activity for state-based judgments.

This study provides some insight into how state- and strength-based perception are supported by different brain regions, but, as with any fMRI study, it only indicates which regions are correlated with these different kinds of judgments, and does not indicate whether their activity is necessary for state- or strength-based perception. Thus, in the current study, we took a neuropsychological approach to determine which regions make necessary contributions to state- and strength-based perception.

In addition to this first exploratory aim, we also set out to test competing hypotheses about the role of lateral parietal cortex in state- vs. strength-based perception. The previous fMRI study (Aly et al., 2014) motivated the hypothesis that lateral parietal cortex — specifically, the supramarginal gyrus — might be critical for state- but not strength-based perception. Moreover, the “global neuronal workspace” model (Dehaene et al., 2006) proposes that an extended parietal-frontal network is critically involved in the threshold for conscious access; that is, this network shows a neural “ignition” that is related to conscious awareness of specific visual information. Insofar as state-based perception reflects a discrete signal indicating conscious awareness of detailed visual information, this would suggest a role for parietal regions in state-based perception (also see Lamme, 2003).

There are, however, reasons to predict that parietal cortex might be critical for strength-based perception. Our prior patient study implicated the hippocampus (and more generally, the MTL) in strength-based perception (Aly et al., 2013; also see Elfman et al., 2014). Due to the anatomical and functional connectivity between the hippocampus/MTL and parietal cortex (e.g., Kahn et al., 2008, Kravitz et al., 2011, Libby et al., 2012, Ranganath and Ritchey, 2012), one prediction is that patients with damage that includes parietal regions will show impairments in strength-based perception. Additionally, our findings relating strength-based perception to graded changes in confidence (Aly and Yonelinas, 2012) are reminiscent of the graded signals in monkey LIP neurons, which reflect continuous integration of sensory evidence in the service of perceptual decision-making (e.g., Shadlen and Newsome, 2001, Mazurek et al., 2003, Gold and Shadlen, 2007, Bollimunta et al., 2012). Although at different levels of analysis and different timescales, this parallel suggests that neural signals in parietal cortex may be related to perceptual judgments based on signals that vary in strength (for related fMRI work in humans, see Heekeren et al., 2006, Ploran et al., 2007, Ploran et al., 2011, Kayser et al., 2010, Liu and Pleskac, 2011).

Thus, our aims were twofold: (1) to explore which regions in the brain (outside of the MTL) are necessary for state-based and strength-based perception, and (2) to test competing hypotheses about the role of lateral parietal cortex in state- vs. strength-based perception. In order to examine these issues, we tested perceptual judgments in 11 stroke patients with right hemisphere lesions, which — considered as a group — included parietal, occipital, and temporal cortical regions, insula, thalamus, basal ganglia, and white matter in the vicinity of these cortical and subcortical structures (Fig. 2). Inclusion of patients with damage in heterogeneous regions allowed us to investigate the contributions of distinct brain areas to state- and strength-based perception, in addition to examining the specific hypotheses about the role of parietal cortex. Such an approach offers an important advance over our previous patient study, in which we only tested individuals with damage to the medial temporal lobe (Aly et al., 2013). We focus on right hemisphere structures because previous work has indicated that the right, more than the left, hemisphere plays a necessary role in visuospatial perception and attention (Mesulam, 1981).

We used a perceptual change detection task in which patients and healthy controls viewed pairs of scenes, presented sequentially, and indicated their confidence that the two were the same or different (Fig. 3). Differences consisted of a relational manipulation that slightly contracted or expanded the scenes relative to one another, changing the distances between component parts without adding or removing any particular object. Confidence ratings were used to plot ROCs and estimate the contributions of state- and strength-based perception.

In addition to the main behavioral analyses in which we examined state- and strength-based perception in the entire patient group, we conducted follow-up analyses in order to determine the roles of different right hemisphere regions in state- and strength-based perception. Specifically, we examined lesion overlap images for subgroups of patients depending on their behavioral performance. Such an analysis enabled us to test whether parietal cortical regions played a unique role in state- vs. strength-based perception: if this is indeed the case, patients who do not have damage in the parietal cortex should perform differently from those who do. Thus, we felt that this analysis would be useful in providing further insights into the neural correlates of state- and strength-based perception, and would be important in guiding future studies.

Section snippets

Participants

The study was approved by the University of Liège Psychology ethics review board. All patients and healthy control participants gave their written informed consent prior to their inclusion in this study.

The patient group consisted of 11 patients with right hemisphere damage as a result of stroke. Patients were recruited at Centre Neurologique et de Réadaptation Fonctionnelle Fraiture, Hôpital Sainte-Ode, and University hospitals from Liège and Brussels in Belgium. All patients but one were

Results

Performance was examined by plotting confidence-based ROC curves. The leftmost point on the ROC is the probability of a hit (y-axis) and false alarm (x-axis) for the most confident “same” response, and subsequent points are the cumulative probabilities for hits and false alarms as responses of decreasing confidence are added. Parameter estimates of state- and strength-based perception are obtained using maximum likelihood estimation to find the curve that best fits the observed ROC points (Aly

Discussion

Perceptual judgments can be based on different kinds of information (Fernandez-Duque and Thornton, 2000, Rensink, 2000, Rensink, 2004, Dehaene et al., 2006, Galpin et al., 2008, Busch et al., 2009, Busch et al., 2010, Aly and Yonelinas, 2012, Howe and Webb, 2014). A useful distinction is between state-based judgments in which individuals have conscious access to specific, detailed information, and strength-based judgments, which are based on a graded sense of overall match/mismatch (Aly and

Conclusions

Perceptual change detection can be based on different kinds of information: conscious access to local, detailed information (state-based perception), or graded signals reflecting a sense of relational match/mismatch (strength-based perception). In the current study, we show that right temporo-parietal cortical regions play a critical and selective role in strength-based perception, while the integrity of the right thalamus, putamen, and adjacent white matter is necessary for intact state- and

Funding

National Institute of Mental Health Grant MH059352 to APY.

Inter-University Attraction Pole P7/11, F.R.S.-FNRS (CB is an F.R.S.-FNRS research associate; travel funding 2012/V 3/5/110-IB/JN-758).

Acknowledgments

We would like to thank Joy Geng for valuable feedback on earlier drafts of the manuscript.

References (71)

  • R.W. Guillery et al.

    Paying attention to the thalamic reticular nucleus

    Trends Neurosci.

    (1998)
  • P.W. Halligan et al.

    Spatial cognition: evidence from visual neglect

    Trends Cogn. Sci.

    (2003)
  • K.H. Hower et al.

    Impaired perception of mnemonic oldness, but not mnemonic newness, after parietal lobe damage

    Neuropsychologia

    (2014)
  • H.-O. Karnath et al.

    The anatomy of spatial neglect

    Neuropsychologia

    (2012)
  • V.A.F. Lamme

    Why visual attention and awareness are different

    Trends Cogn. Sci.

    (2003)
  • S.L. Mullally et al.

    Attenuated boundary extension produces a paradoxical memory advantage in amnesic patients

    Curr. Biol.

    (2012)
  • R.A. Rensink

    Seeing, sensing, and scrutinizing

    Vis. Res.

    (2000)
  • C. Rorden et al.

    Age-specific CT and MRI templates for spatial normalization

    NeuroImage

    (2012)
  • Y.B. Saalmann et al.

    Gain control in the visual thalamus during perception and cognition

    Curr. Opin. Neurobiol.

    (2009)
  • M.H. Sharpe

    Distractibility in early Parkinson’s disease

    Cortex

    (1990)
  • A.P. Yonelinas

    The nature of recollection and familiarity: A review of 30 years of research

    J. Mem. Lang.

    (2002)
  • M.L. Albert

    A simple test of visual neglect

    Neurology

    (1973)
  • M. Aly et al.

    Neural correlates of state- and strength-based perceptions

    J. Cogn. Neurosci.

    (2014)
  • M. Aly et al.

    Bridging consciousness and cognition in memory and perception: evidence for both state and strength processes

    PLoS One

    (2012)
  • P. Azouvi et al.

    Sensitivity of clinical and behavioral tests of spatial neglect after right hemisphere stroke

    J. Neurol. Neurosurg. Psychiatry

    (2002)
  • A.D. Baddeley et al.

    Doors and People: A Test of Visual and Verbal Recall and Recognition

    (1994)
  • A. Bollimunta et al.

    Neural dynamics of choice: single-trial analysis of decision-related activity in parietal cortex

    J. Neurosci.

    (2012)
  • N.A. Busch et al.

    ERP effects of change localization, change identification, and change blindness

    NeuroReport

    (2010)
  • N.A. Busch et al.

    Electrophysiological evidence for different types of change detection and change blindness

    J. Cogn. Neurosci.

    (2009)
  • M. Corbetta et al.

    Spatial neglect and attention networks

    Annu. Rev. Neurosci.

    (2011)
  • F. Crick

    Function of the thalamic reticular complex: the searchlight hypothesis

    Proc. Natl. Acad. Sci. USA

    (1984)
  • B. Dubois et al.

    The FAB: a frontal assessment battery at bedside

    Neurology

    (2000)
  • K.W. Elfman et al.

    Neurocomputational account of memory and perception: thresholded and graded signals in the hippocampus

    Hippocampus

    (2014)
  • D. Fernandez-Duque et al.

    Change detection without awareness: do explicit reports underestimate the representation of change in the visual system?

    Vis. Cogn.

    (2000)
  • A. Galpin et al.

    Sensing without seeing in comparative visual search

    Conscious. Cogn.

    (2008)
  • Cited by (4)

    • Visual working memory impairments for single items following medial temporal lobe damage

      2019, Neuropsychologia
      Citation Excerpt :

      In addition to functional independence, neuropsychological, neuroimaging, and neurocomputational studies converge in showing that the dual processes underlying perception, working memory, and long-term memory are also anatomically distinct. For instance, in perception and working memory, sensing relies on the hippocampus but perceiving does not (Aly et al., 2013; Aly et al., 2014; Aly et al., 2014; Goodrich and Yonelinas, 2016). Similarly, in long-term memory, remembering is hippocampally dependent but knowing is not (Düzel et al., 2001; Duzel et al., 1997; Eichenbaum et al., 2007; Elfman et al., 2014).

    View full text