Elsevier

Cognition

Volume 218, January 2022, 104938
Cognition

The impact of visual cues during visual word recognition in deaf readers: An ERP study

https://doi.org/10.1016/j.cognition.2021.104938Get rights and content

Abstract

Although evidence is still scarce, recent research suggests key differences in how deaf and hearing readers use visual information during visual word recognition. Here we compared the time course of lexical access in deaf and hearing readers of similar reading ability. We also investigated whether one visual property of words, the outline-shape, modulates visual word recognition differently in both groups. We recorded the EEG signal of twenty deaf and twenty hearing readers while they performed a lexical decision task. In addition to the effect of lexicality, we assessed the impact of outline-shape by contrasting responses to pseudowords with an outline-shape that was consistent (e.g., mofor) or inconsistent (e.g., mosor) with their baseword (motor). Despite hearing readers having higher phonological abilities, results showed a remarkably similar time course of the lexicality effect in deaf and hearing readers. We also found that only for deaf readers, inconsistent-shape pseudowords (e.g., mosor) elicited larger amplitude ERPs than consistent-shape pseudowords (e.g., mofor) from 150 ms after stimulus onset and extending into the N400 time window. This latter finding supports the view that deaf readers rely more on visual characteristics than typical hearing readers during visual word recognition. Altogether, our results suggest different mechanisms underlying effective word recognition in deaf and hearing readers.

Introduction

Expert reading in alphabetic languages requires general language skills and efficient word recognition through rapid orthographic and phonological decoding (Gough & Tunmer, 1986). After appropriate instruction of grapheme-phoneme correspondences (Castles, Rastle, & Nation, 2018), and with enough practice, most hearing children recognise words rapidly and effortlessly. However, this is not the case for most deaf people, who find reading a challenging task. Indeed, current reading instruction only takes most deaf readers as far as a reading level equivalent to that of a 10-year-old (see, e.g., English: Traxler, 2000; Spanish: Sánchez & García-Rodicio, 2006)—this has a negative impact not only on their academic achievement but also on their social and emotional wellbeing (McArthur & Castles, 2017).

Given that phonological processing plays a key role for skilled reading in hearing people (see, e.g., Frost, 2012), the low reading attainment in many deaf people has often been attributed to their difficulties in phonological processing (see, e.g., Perfetti & Sandak, 2000). However, recent research supports a partially different view. The idea is that deaf readers can achieve a more efficient lexical access during reading using the visual-orthographic route rather than a phonologically based route (for a recent review, see Emmorey, 2020; Emmorey & Lee, 2021). Consistent with this proposal, we recently showed that, for deaf readers of Spanish, more efficient use of the visual-orthographic route correlated with better reading skills; in contrast, increased automatic phonological processing did not (see Gutierrez-Sigut, Vergara-Martinez, & Perea, 2017; Gutierrez-Sigut, Vergara-Martinez, & Perea, 2019). These findings can be accounted for by Bélanger and Rayner's (Bélanger & Rayner, 2015) word-processing efficiency hypothesis. This account proposes that skilled deaf readers “have tighter connections between orthography and semantics” (p. 224) than hearing readers. Bélanger and Rayner also proposed that deaf readers are “extremely attuned to the visual-orthographic makeup of words and quickly detect precise word forms” (p. 224). Indeed, deaf readers can extract more information than hearing readers from a given fixation during sentence reading (i.e., they skip more words, re-read fewer words, and refixate words less often; see also Traxler et al., 2021, for recent converging evidence).

Let's assume that deaf readers are indeed more attuned than hearing readers to the visual-orthographic features of words. In this case, we might expect clear differences between deaf readers and hearing readers of matched reading ability in aspects of lexical access linked to visual-orthographic processing. However, the literature is mixed. While most prior studies of visual word recognition have not found differences in orthographic processing between deaf and hearing participants (e.g Bélanger, Baum, & Mayberry, 2012; Bélanger, Mayberry, & Rayner, 2013; Cripps, McBride, & Forster, 2005; Fariña, Duñabeitia, & Carreiras, 2017; Meade, Grainger, Midgley, Holcomb and Emmorey, 2019, Meade, Grainger, Midgley, Holcomb and Emmorey, 2020), recent neuroimaging research has shown subtle differences in deaf and hearing readers' responses to word stimuli, suggesting a larger prominence of visual-orthographic processing in deaf readers. Specifically, Emmorey, Midgley, Kohen, Sehyr, and Holcomb (2017) found that words elicited stronger left lateralization of the N170 event related potential (ERP) component in hearing than in deaf readers. Similarly, Glezer et al. (2018) found fine-grained orthographic tuning bilaterally for deaf readers but only left lateralized for hearing readers. Furthermore, Emmorey, Holcomb, and Midgley (2021) recently found a reversed priming ERP effect in deaf readers but not in hearing readers in response to case mismatch between prime and target words at short prime durations (see also Perea, Marcet, & Vergara-Martínez, 2016 for similar perceptual-visual effects in deaf readers). These recent findings point to nuanced differences in visual word recognition between deaf and hearing readers that can be assessed using highly sensitive paradigms.

In the present study, we first compared the processes underlying lexical access in deaf and hearing readers of similar reading ability by investigating the effect of lexicality (words vs. pseudowords) in an Event-Related Potentials (ERPs) lexical decision task experiment. The effect of lexicality (measured as the difference between responses to words and pseudowords) has been linked to orthographic knowledge (see e.g., Coch, 2015; Coch & Holcomb, 2003; Cuetos & Suárez-Coalla, 2009; Zoccolotti, De Luca, Di Filippo, Judica, & Martelli, 2008). Thus, the lexicality effect allows us to evaluate whether deaf and hearing readers are equally sensitive to the lexical principles of written language. Secondly, we investigated deaf and hearing readers' sensitivity to the “visual-orthographic makeup of words” by examining the responses to two types of pseudowords that only varied on the outline-shape of their base word. Specifically, we compared pseudowords with an outline-shape congruent with the base word (e.g., the pseudoword mofor is congruent with the outline-shape of its base word motor [engine]) or incongruent (e.g., the pseudoword mosor). Finally, we investigated whether the degree of sensitivity to outline word shape is associated with reading and phonological skills.

In the rest of the Introduction, we first summarize the behavioural and electrophysiological signatures of the lexicality effect. We focus on its relationship with orthographic knowledge, which is the basis of our first research aim. Second, we evaluate the impact of visual-orthographic features (i.e., the word outline-shape) on lexical access, which is the basis for our second research question.

The behavioural and electrophysiological correlates of the lexicality effect in visual word recognition in adult skilled readers are well-known (see, e.g., Grainger & Holcomb, 2009; Stone & Van Orden, 1989; Swaab, Ledoux, Camblin, & Boudewyn, 2012a; Wagenmakers et al., 2004). Behaviourally, the finding of faster RTs obtained for words vs. pseudowords in lexical decision tasks is interpreted as an index of familiarity (words are familiar letter-strings) and accessibility of lexical-semantic information stored in long-term memory (nonwords do not have an entry in the mental lexicon). The best-known electrophysiological correlate of the lexicality effect is a larger N400 amplitude in response to pseudowords than words (also see, e.g., Hauk, Coutout, Holden, & Chen, 2012 for early effects of lexical status), a result thought to reflect increased efforts during lexical access as readers struggle to find a matching lexical entry (see, e.g., Kutas & Federmeier, 2011). Interestingly, the advantage of reading words over pseudowords has been linked to orthographic knowledge: the lexicality effect increases as a function of understanding the conventions of the writing system (Conrad, Harris, & Williams, 2013; Cuetos & Suárez-Coalla, 2009). For example, Zoccolotti et al. (2008) showed that, in children, the advantage in response times for words over pseudowords increased from first to eight grade as words' representations consolidate in the mental lexicon (see also Cuetos & Suárez-Coalla, 2009; Job, Peressotti, & Cusinato, 1998; Orsolini, Fanari, Tosi, De Nigris, & Carrieri, 2006; Seymour, Aro, Erskine, & Network, 2003). That is, as more words are incorporated into the vocabulary, detailed orthographic representations are built and linked to lexical memory (Perfetti, 2007; Zarić, Hasselhorn, & Nagler, 2021).

Similarly, it is generally assumed that the increase in the N400 elicited by pseudowords observed for more experienced readers reflects an increase in the refinement of the word processing system (Coch, 2015; Coch & Holcomb, 2003). In their study with young readers, Coch and Holcomb (2003) found that pseudowords elicited a larger N400 than known words in high- but not in low-ability readers. These findings indicate that being less experienced with word stimuli, low-ability young readers were reading less automatically than their high-ability peers. Therefore, the electrophysiological correlates of the lexicality effect can be used to assess the similarity of the neurocognitive systems underlying word recognition in different groups of readers.

To the best of our knowledge, the present study is the first that directly contrasts the behavioural and electrophysiological correlates of the lexicality effect in deaf and hearing readers of comparable reading ability. To maximise the chances of observing differences related to orthographic knowledge in adult readers, we chose pseudowords that only differed in one letter from their base words (e.g., the pseudoword mosor vs. the Spanish word motor) (see Vergara-Martínez, Perea, Gómez, & Swaab, 2013 for a similar approach and discussion of the prior literature, p. 2). Similar to what we have observed before in a group of deaf readers (Gutierrez-Sigut, Vergara-Martínez, & Perea, 2019) and the previous literature in hearing readers, we expect faster response times and lower amplitude N400s for words than pseudowords for both groups. Given their equivalent reading ability, similar time courses of lexical access between both groups would suggest that the development of orthographic knowledge could support efficient word recognition in deaf readers despite them having a lower ability at a phonological task. Conversely, differences in the timing, size or distribution of the effects would reflect differences in the neurophysiological underpinnings of word recognition.

During visual word recognition, most researchers assume that readers can rapidly access abstract letter/word representations. Indeed, neurally-inspired models of printed word recognition (Dehaene, Cohen, Sigman, & Vinckier, 2005; Grainger, Rey, & Dufau, 2008) assume that perceptual elements (e.g., color, font, size, or letter-case) do not play a role after the initial perceptual stages. In behavioural studies, the visual features' limited role is typically reflected in facilitation from identity primes independently of them sharing or not visual features with the target. For example, Bowers, Vigliocco, and Haan (1998) found a similar degree of facilitation in a behavioural masked priming experiment for English words that were similar and dissimilar in upper- and lower-case (e.g., compare kiss-KISS vs. edge-EDGE), indicating that access to abstract letter identity overrides perceptual similarity (for similar findings in adult and beginner readers in other languages see: Arabic Perea, Mallouh, & Carreiras, 2013; French Jacobs, Grainger, & Ferrand, 1995; and Spanish Perea, Jiménez, & Gomez, 2015). At the electrophysiological level, Vergara-Martínez, Gómez, Jiménez, and Perea (2015) found differences between words preceded by masked identity primes displayed in the same case or a different case in a perceptual component (N/P150; VILLA-VILLA vs. villa-VILLA) that vanished by 200 ms post-stimulus onset. That is, when the initial contact to abstract letter identities is achieved (<200 ms), visual features do not further facilitate word processing.

When looking specifically at outline word shape (e.g., comparing crown [flat word] vs. bishop [non-flat word]), previous research has shown that word recognition times of normo-typical readers, both adult and children, from 4th grade on, are not affected by this visual cue (Lavidor, 2011; Perea & Panadero, 2014). However, the scenario is different for individuals with dyslexia. In a lexical decision task, Lavidor (2011) found that adult dyslexic readers, but not typical readers, responded faster to words with a distinctive physical appearance (i.e., non-flat words such as bishop) than to flat words (e.g., crown). Likewise, Perea & Panadero, 2014 contrasted response times of typical adult and young readers as well as young dyslexic readers to two types of pseudowords that, while differing in just one letter from their base word, had a consistent outline-shape (e.g., viotin vs. word base violin) or an inconsistent outline-shape (e.g., viocin). They found that only young dyslexic readers were sensitive to the outline-shape (worse performance to viotin [base word violin] than to viocin). The larger difficulty in resolving lexical ambiguity for “viotin” pseudowords compared to “viocin” pseudowords can be explained by the increased effort in differentiating “viotin” and “violin” (base word), due to larger perceptual overlap. This suggests poor letter representations in young readers with developmental dyslexia. These results have been interpreted within the framework of interactive models of word recognition, where top-down feedback from the phonological and lexical levels support more precise letter representations (see Carreiras, Armstrong, Perea, & Frost, 2014). In this context, readers with poorer phonological representations are likely to have less precise orthographic representations and hence are likely to be more influenced by visual cues that play a limited role in visual word recognition for expert readers (e.g., font, outline-shape, size, etc.)

Similar to dyslexic readers, deaf readers have underspecified phonological representations. Their poor phonological representations are not likely to fully contribute to improving orthographic precision. This reduced contribution from phonological representations might result in deaf readers also being more sensitive to visual features that, in the context of hearing skilled reader's visual word recognition, are supposed to play a minor role. For instance, in an analysis of spelling errors in deaf readers, Padden (1993) found a high rate of confusions among letters of the same height (t, d, and b) or among letters with descenders (p, q, and g), reflecting attempts to reproduce the overall shape of words. Similarly, Perea et al. (2016) found an advantage of nominally and physically identical priming condition (EDGE-EDGE) over the nominally identical priming condition (edge-EDGE) in deaf but not in hearing readers' behavioural responses, suggesting differences in visual-orthographic processing between the groups (see Gutierrez-Sigut, Vergara-Martínez, & Perea, 2019, for similar behavioural results).

Thus, the second aim of the present experiment is to contrast the behavioural and electrophysiological responses of deaf and hearing readers with similar reading ability but different phonological skills (i.e., lower syllable counting accuracy in the deaf group) to two types of pseudowords. For half of the pseudoword targets, an ascending or descending consonant (e.g., t in motor [engine]) was replaced by another ascending or descending consonant (congruent-shape pseudoword: e.g., mofor). For the other half, the replacement resulted in an incongruent-shaped pseudoword (e.g., mosor). In line with Perea & Panadero, 2014 results, we expect no differences in processing of both pseudoword types in hearing readers. Critically, if deaf readers have developed precise orthographic representations regardless of their poorer phonological representations, we would expect no differences between the two types of pseudowords. Conversely, if deaf readers are more reliant than hearing readers on visual information, we would expect larger interferences in the correct no-responses to congruent-shape (e.g., mofor) compared to incongruent-shape pseudowords (mosor).

Finally, in order to assess the relationship between reading and phonological skill and the effect of outline shape, we performed correlational analyses. First, if less-skilled deaf readers are less finely tuned to the visual-orthographic properties of words, we would expect a negative correlation between the size of the outline-shape effect and reading ability. Second, if better phonological skills help stabilize the orthographic representation, which allows discarding the outline-shape information, we would expect a negative correlation between the size of the outline-shape effect and performance in a phonological task (syllable counting).

In sum, in the present experiment, we aim to track down the time course of lexical access in deaf readers, investigating the similarities and differences in processing between deaf and hearing readers of similar reading ability. We also aim to elucidate whether one visual property of words, outline-shape, modulates visual word recognition in deaf readers. Finally, we explore whether reading and phonological skills are correlated to differences in processing due to outline-shape.

Section snippets

Participants

Twenty-three congenitally deaf participants were recruited for this experiment. Data from 3 participants had to be rejected due to an excessive number of movements and other artefacts (more than 60% of the trials). The remaining 20 participants (8 female) were profoundly deaf and skilled signers. Six participants were native signers of Spanish sign Language (LSE), eight were early signers (learn sign language before the age of 6), and six were late signers. Their ages ranged from 21 to 54 years

Results

The mean lexical decision times and percentage of correct responses per condition are displayed in Table 2. Note that incorrect responses (2.8%) and lexical decision times above and below the 2.5 SDs of the average per participant and condition (2.5%) were excluded from the latency analysis.

Discussion

We designed a lexical decision experiment to compare the time course of the lexicality effect in adult deaf and hearing readers of Spanish with similar reading ability. We also investigated the electrophysiological correlates of processing a visual feature such as the word's outline-shape. Results showed an equivalent lexicality effect in both groups, as well as a larger sensitivity to outline-shape in deaf than in hearing readers. We will discuss these two findings in order.

Conclusion

In summary, the present lexical decision ERP experiment revealed a remarkably similar time course of the lexicality effect in deaf and hearing readers despite deaf readers being less skilled using phonology. At the same time, ERP responses showed that deaf readers were more sensitive to visual features such as the word outline-shape than hearing readers. Taken together, these results suggest that fine-grained visual processing and the development of orthographic knowledge via exposure to words

Author contributions

Conceived and designed the experiment: E.G., M.P., M.V.; Performed the experiments: E.G., M.V..; Analysed the data: E.G.; Interpretation of the findings: E.G., M.P., M.V.; Contributed to writing the manuscript: E.G., M.P., M.V.

Acknowledgements

This work was supported by a Spanish Ministerio de Economía y Competitividad fellowship awarded to EG [Grant Number PSI2014-60611-JIN] and a Grant from the Spanish Ministry of Science and Innovation awarded to MP [Grant Number PSI2017-862120-P]. We would like to thank ASORTE, FASICAN, FESORD and FUNCASOR, Roberto Suarez, Juan Molina, Verónica Rodríguez, Belén Darias for their support for recruiting participants and facilitating data collection, Daniel Díaz for his support programming the

References (64)

  • O. Hauk et al.

    The time-course of single-word reading: Evidence from fast behavioral and brain responses

    Neuroimage

    (2012)
  • G. Meade et al.

    An ERP investigation of orthographic precision in deaf and hearing readers

    Neuropsychologia

    (2020)
  • C.A. Padden

    Lessons to be learned from the young deaf orthographer

    Linguistics and Education

    (1993)
  • M. Perea et al.

    Do young readers have fast access to abstract lexical representations? Evidence from masked priming

    Journal of Experimental Child Psychology

    (2015)
  • M. Vergara-Martínez et al.

    ERP correlates of letter identity and letter position are modulated by lexical frequency

    Brain and Language

    (2013)
  • E.-J. Wagenmakers et al.

    A model for evidence accumulation in the lexical decision task

    Cognitive Psychology

    (2004)
  • M. Bar et al.

    Top-down facilitation of visual recognition

    Proceedings of the National Academy of Sciences of the United States of America

    (2006)
  • N.N. Bélanger et al.

    Reading difficulties in adult deaf readers of French: Phonological codes, not guilty!

    Scientific Studies of Reading

    (2012)
  • N.N. Bélanger et al.

    Orthographic and phonological preview benefits: Parafoveal processing in skilled and less-skilled deaf readers

    The Quarterly Journal of Experimental Psychology

    (2013)
  • N.N. Bélanger et al.

    What eye movements reveal about deaf readers

    Current Directions in Psychological Science

    (2015)
  • N.N. Bélanger et al.

    Skilled deaf readers have an enhanced perceptual span in reading

    Psychological Science

    (2012)
  • Y. Benjamini et al.

    The control of the false discovery rate in multiple testing under dependency

    Annals of Statistics

    (2001)
  • J.S. Bowers et al.

    Orthographic, phonological, and articulatory contributions to masked letter and word priming

    Journal of Experimental Psychology: Human Perception and Performance

    (1998)
  • M.S. Carrillo et al.

    Test de Eficiencia Lectora–TECLE

  • A. Castles et al.

    Ending the Reading wars: Reading acquisition from novice to expert

    Psychological Science in the Public Interest

    (2018)
  • D. Coch

    The N400 and the fourth grade shift

    Developmental Science

    (2015)
  • D. Coch et al.

    The N400 in beginning readers

    Developmental Psychobiology: The Journal of the International Society for Developmental Psychobiology

    (2003)
  • N.J. Conrad et al.

    Individual differences in children’s literacy development: The contribution of orthographic knowledge

    Reading and Writing

    (2013)
  • J.H. Cripps et al.

    Lexical processing with deaf and hearing: Phonology and orthographic masked priming

  • F. Cuetos et al.

    From grapheme to word in reading acquisition in Spanish

    Applied PsychoLinguistics

    (2009)
  • A. Duchon et al.

    EsPal: One-stop shopping for Spanish word properties

    Behavior Research Methods

    (2013)
  • K. Emmorey

    The neurobiology of Reading differs for deaf and hearing adults

  • Cited by (0)

    View full text