Elsevier

Neuropsychologia

Volume 75, August 2015, Pages 109-118
Neuropsychologia

Neural correlates of active vision: An fMRI comparison of natural reading and scene viewing

https://doi.org/10.1016/j.neuropsychologia.2015.05.027Get rights and content

Highlights

  • We examined the nature of the cortical networks associated with active vision.

  • Subjects read passages and viewed scenes with eyetracking and fMRI.

  • A common eye movement control network was observed in reading and scene viewing.

  • The specific locus of FEF activation varied across reading and scene viewing.

  • Eye movement control draws on domain-general and domain-specific.

Abstract

Theories of eye movement control during active vision tasks such as reading and scene viewing have primarily been developed and tested using data from eye tracking and computational modeling, and little is currently known about the neurocognition of active vision. The current fMRI study was conducted to examine the nature of the cortical networks that are associated with active vision. Subjects were asked to read passages for meaning and view photographs of scenes for a later memory test. The eye movement control network comprising frontal eye field (FEF), supplementary eye fields (SEF), and intraparietal sulcus (IPS), commonly activated during single-saccade eye movement tasks, were also involved in reading and scene viewing, suggesting that a common control network is engaged when eye movements are executed. However, the activated locus of the FEF varied across the two tasks, with medial FEF more activated in scene viewing relative to passage reading and lateral FEF more activated in reading than scene viewing. The results suggest that eye movements during active vision are associated with both domain-general and domain-specific components of the eye movement control network.

Introduction

Visual perception and visual cognition are active processes in which saccadic eye movements play a central role (Dodge, 1903, Buswell, 1935, Findlay and Gilchrist, 2003, Henderson and Ferreira, 2004, Henderson, 2013, Rayner, 2009, Yarbus, 1967). For example, in both natural scene viewing and reading, the eyes move from location to location to enable the acquisition of information as it is needed in real time. Current theoretical approaches and computational models of reading and scene viewing therefore attempt to account for how attention and particularly eye movements are controlled during these activities (e.g., Reichle et al., 1998; Engbert et al., 2005; Nuthmann et al., 2010; Torralba et al., 2006). An important issue in understanding active vision is the degree to which eye movement control systems are domain-specific (i.e., dedicated to a particular active vision domain such as reading or scene perception) versus domain-general (i.e., dedicated to all active vision tasks in which eye movements are controlled in the service of perception and cognition). Evidence from eyetracking shows strong correlations in basic eye movement measures within individuals across tasks like reading, scene viewing, and search, suggesting common neurocognitive eye movement control processes across domains (Andrews and Coppola, 1999, Castelhano and Henderson, 2008, Henderson and Luke, 2014, Rayner et al., 2007). Several computational models of eye movement control that have been developed to account for one domain have been extended to other domains, implicitly endorsing the domain-general hypothesis. For example, the E–Z Reader and SWIFT models of eye movement control in reading have both been extended to non-reading tasks (Reichle et al., 2012, Trukenbrod and Engbert, 2012), and the CRISP model of eye movement control in scene perception has been extended to reading (Nuthmann et al., 2010, Nuthmann and Henderson, 2012). Yet there has been very little direct investigation of the degree to which the eye movement control system is similar or different across domains. In the present study we examined this issue by investigating the similarities and differences in the cortical eye movement control network during natural reading and scene viewing.

Past studies using single-step eye movement paradigms as in the gap and anti-saccade tasks have provided evidence for a frontal-parietal eye movement control network that includes as core components the frontal eye fields (FEF), supplementary eye fields (SEF), and intraparietal sulcus (IPS). Additional cortical areas often observed in single-step eye movement studies include dorsolateral prefrontal cortex (DLPFC), likely associated with maintaining task goals in these tasks. Recently, Hillen et al. (2013) found blood oxygenation level dependent (BOLD) activation in FEF, SEF, and IPS during a target-search task through texts and pseudo-texts that required sequences of saccadic eye movements. These results indicate that the eye-movement network observed for single-saccade tasks is also engaged during sequential scanning. In addition, Choi et al. (2014) compared BOLD activation during natural reading using normal texts and pseudo-reading using nonword control texts. No secondary task was administrated when participants read paragraphs of normal texts. Choi et al. also found common brain areas such as FEF, SEF and IPS associated with the control of eye movements in both reading and pseudo-reading.

Whether this eye movement network is also functional in other active vision tasks such as scene viewing is currently unknown. The present study investigated similarities and differences in the eye movement control network in natural reading and scene viewing by comparing the natural reading condition from Choi et al. (2014) to a scene viewing condition. The data for natural reading in the current study is the same as that reported in Choi et al. (2014). The purpose of the present report is to compare the similarities and differences in the neural correlates of eye movements during natural reading from Choi et al. (2014) with the neural correlates of eye movements in scene viewing to investigate the domain-specificity of eye movement control systems.

A secondary issue investigated here was whether differential activation in cortical regions associated with reading and scene viewing can be observed during active viewing. The majority of neuroimaging studies of both reading and scene viewing have used brief presentation (rapid serial visual presentation or RSVP) methods in which saccadic eye movements are not possible or are discouraged1. In general, reading via RSVP tends to activate a left-lateralized frontal–temporal–parietal network associated with language processing (Friederici, 2011, Martin, 2003, Price, 2012). This activation typically involves an occipital-temporal network including the purported visual word form area (VWFA), middle and superior temporal gyrus/sulcus (MTG, STG/STS), and inferior frontal gyrus (IFG). In comparison, scene viewing tends to activate a right-lateralized frontal-temporal-parietal network associated with visual scene analysis (Epstein, 2005, Epstein, 2008, Grill-Spector, 2003, Maguire, 2001). This network notably includes temporal–occipital junction (TOJ) associated with visual objects, and areas of fusiform gyrus and the parahippocampal cortex associated with scene analysis (Bar et al., 2001, Epstein et al., 2000, Henderson et al., 2008, Henderson et al., 2007, Henderson et al., 2011, Park and Chun, 2009, Walther et al., 2009). Here we investigated whether active vision tasks involving extended viewing with eye movements in reading versus scene viewing lead to the type of differential activation that is typically observed in brief presentations of words and scenes. Similar activation in the hypothesized eye movement network in spite of differential activation in domain-specific processing networks associated with reading and scene viewing would provide additional evidence for the domain-generality of eye movement control.

To investigate these issues, we combined simultaneous high-resolution eyetracking and fMRI data acquisition while subjects read passages and viewed photographs of real-world scenes. Each stimulus was presented for 12 s and was viewed actively. Eyetracking was used to ensure that subjects were engaging naturally with the stimuli. We hypothesized that the eye movement control network identified in single-saccade tasks also supports these active vision tasks, but that reading and scene viewing may also draw on independent eye movement sub-systems given that these tasks use partially independent cognitive operations. We also hypothesized that actively engaging in reading and scene viewing would produce activation similar to that observed previously in brief-presentation RSVP studies of reading and scene perception.

Section snippets

Subjects

Thirty-three subjects (12 male) participated in this study. They were all right-handed, native speakers of English, aged 18–35 years (Mean Age: 21.45). Thirty subjects were students from the University of South Carolina and three were recruited from the Columbia, South Carolina community. All subjects gave informed consent and were screened for MRI safety. They all reported normal or corrected-to-normal vision and were given $10 per hour for participation in the study. Two did not finish the

Eye-movement analysis

The eye-movement data were analyzed to obtain fixations and saccades in the reading and scene viewing conditions (Table 1). All fixations meeting the following criteria were included in this analysis: A fixation could not have a blink immediately before or after, and had to have a duration between 50 and 1500 ms. Data during track losses were also rejected. In total, 13.4% of fixations (11.2% for the reading condition and 16.7% for the scene viewing condition) were excluded. The summary eye

Discussion

An important issue in understanding active vision is the degree to which eye movement control is domain-specific (i.e., dedicated to a particular active vision task such as reading or scene perception) versus domain-general (i.e., dedicated to all active vision tasks in which eye movements are controlled in the service of perception and cognition). The current study was designed to investigate this issue by examining the similarities and differences in brain activity elicited by two active

Conclusions

The present block-design fMRI study demonstrated common and differential brain activation in two active vision tasks that required sequences of saccadic eye movements, reading and scene viewing. As expected, language-related cortical areas including IFG, STG, and MTG were more active in the reading condition relative to the scene viewing condition, whereas ventral occipito-medial areas including PPA and RSC produced more activation for the scene viewing condition. In addition, anterior areas of

Acknowledgments

This research was supported by the National Science Foundation of the USA (BCS-1151358). We thank William Brixius, Jennifer Olejarczyk, and Joseph Schmidt for their help with data collection.

References (80)

  • K. Grill-Spector

    The neural basis of object perception

    Curr. Opin. Neurobiol.

    (2003)
  • P. Hagoort

    On Broca, brain, and binding: a new framework

    Trends Cogn. Sci.

    (2005)
  • J.M. Henderson et al.

    Full scenes produce more activation than close-up scenes and scene-diagnostic objects in parahippocampal and retrosplenial cortex: an fMRI study

    Brain Cogn.

    (2008)
  • S. Malinen et al.

    Towards natural stimulation in fMRI— issues of data analysis

    NeuroImage

    (2007)
  • B.D. McCandliss et al.

    The visual word form area: expertise for reading in the fusiform gyrus

    Trends Cogn. Sci.

    (2003)
  • J.E. McDowell et al.

    Neurophysiology and neuroanatomy of reflexive and volitional saccades: evidence from studies of humans

    Brain Cogn.

    (2008)
  • S. Park et al.

    Different roles of the parahippocampal place area (PPA) and retrosplenial cortex (RSC) in panoramic scene perception

    (2009)
  • M. Perrone-Bertolotti et al.

    What is that little voice inside my head? Inner speech phenomenology, its role in cognitive performance, and its relation to self-monitoring

    Behav. Brain Res.

    (2014)
  • C.J. Price

    A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading

    NeuroImage

    (2012)
  • C.J. Price et al.

    The myth of the visual word form area

    NeuroImage

    (2003)
  • C.J. Price et al.

    The interactive account of ventral occipitotemporal contributions to reading

    Trends Cogn. Sci.

    (2011)
  • K. Rayner et al.

    Eye movements during information processing tasks: Individual differences and cultural effects

    Vis. Res.

    (2007)
  • Z.S. Saad et al.

    A new method for improving functional-to-structural MRI alignment using local Pearson correlation

    NeuroImage

    (2009)
  • F.W. Sabb et al.

    Working memory effects on semantic processing: Priming differences in pars orbitalis

    Neuroimage

    (2007)
  • R. Starrfelt et al.

    The visual what for area: words and pictures in the left fusiform gyrus

    Neuroimage

    (2007)
  • E.J. Tehovnik et al.

    Eye fields in the frontal lobes of primates

    Brain Res. Rev.

    (2000)
  • M.R. Uncapher et al.

    Posterior parietal cortex and episodic encoding: insights from fMRI subsequent memory effects and dual-attention theory

    Neurobiol. Learn. Mem.

    (2009)
  • L. Van Doren et al.

    The amodal system for conscious word and picture identification in the absence of a semantic task

    Neuroimage

    (2010)
  • F. Vinckier et al.

    Hierarchical coding of letter strings in the ventral stream: dissecting the inner organization of the visual word-form system

    Neuron

    (2007)
  • M. Vigneau et al.

    Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing

    Neuroimage

    (2006)
  • D.S. Aichert et al.

    Functional neural correlates of psychometric schizotypy: an fMRI study of antisaccades

    Psychophysiology

    (2012)
  • U. Altmann et al.

    Fact vs fiction-how paratextual information shapes our reading processes

    Soc. Cogn. Affect. Neurosci.

    (2014)
  • A. Bartels et al.

    Functional brain mapping during free viewing of natural scenes

    Hum. Brain Map.

    (2004)
  • M. Ben-Shachar et al.

    Differential sensitivity to words and shapes in ventral occipito-temporal cortex

    Cerebr. Cortex

    (2007)
  • S.Y. Bookheimer

    Functional MRI of language: New approaches to understanding the cortical organization of semantic processing

    Annu. Rev. Neurosci.

    (2002)
  • G.I. Buswell

    How People Look at Pictures: A Study of Psychology and Perception of Art

    (1935)
  • M.S. Castelhano et al.

    Stable individual differences across images in human saccadic eye movements

    Can. J. Exp. Psychol.

    (2008)
  • W. Choi et al.

    The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI

    Front. Hum. Neurosci.

    (2014)
  • L. Cohen et al.

    Language-specific tuning of visual cortex? Functional properties of the visual word form area

    Brain

    (2002)
  • R.W. Cox et al.

    Real-time 3D image registration of functional MRI

    Magn. Reson. Imag.

    (1999)
  • Cited by (19)

    • Individual differences in first-pass fixation duration in reading are related to resting-state functional connectivity

      2021, Brain and Language
      Citation Excerpt :

      Our findings can be linked to a large body of previous research on the neurobiology of reading. In task-state neuroimaging studies, the involvement of the early visual network, the VWFA, and the eye-movement control/dorsal attention network in reading have been well indicated (Bolger, Perfetti, & Schneider, 2005; Choi et al., 2014; Choi & Henderson, 2015; Zhou et al., 2016). In resting-state neuroimaging studies, the RSFCs of these regions/networks have also been found to be related to several behavioural measurements of reading, such as reaction times of single characters (Wang et al., 2012), reading fluency (Zhou et al., 2015; Zhou et al., 2016), and lexical decision scores (Zhou et al., 2016).

    • Linguistic networks associated with lexical, semantic and syntactic predictability in reading: A fixation-related fMRI study

      2019, NeuroImage
      Citation Excerpt :

      This region, like the DLPFC, is linked to cognitive control of actions, and is connected to the DLPFC as well as the frontal eye fields (Grahn et al., 2008). Like the DLPFC, it is commonly activated in studies of connected-text reading (Choi and Henderson, 2015; Henderson et al., 2015) and so may also be involved in allowing predictability in reading to influence eye movements. There was also a significant cluster of activation in the bilateral orbitofrontal cortex.

    • Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading

      2016, NeuroImage
      Citation Excerpt :

      In our study, subjects read natural stories (e.g., The Emperor's New Clothes) presented as paragraphs in an MRI scanner while both eye movements and BOLD signals were recorded. Because effects in an fMRI study may be influenced by inclusion of a secondary task, our general approach is to understand complex cognitive processes in a natural context, an approach that has been used with auditory narratives (Brennan et al., 2010; Wilson et al., 2008), natural reading (Altmann et al., 2014; Choi et al., 2014; Choi and Henderson, 2015), film viewing (Bartels and Zeki, 2004; Hasson et al., 2004; Nishimoto et al., 2011), and active photograph viewing (Choi and Henderson, 2015; Henderson and Choi, 2015; Stansbury et al., 2013). Here we asked participants simply to read naturally for comprehension.

    View all citing articles on Scopus
    View full text