Second Language Word Processing in Sentence Contexts: Pre-Lexical Prediction Versus Post-Lexical Integration

. This study investigated the effects of sentence contexts on the second language (L2) word recognition process. It aims at finding whether second language (L2) learners of English perform similar to English native speakers in terms of using the sentence context to predict upcoming word in their L2. A group of L2 participants and a control group of native speakers (L1) participants performed a cross-modal priming task in which they were asked to make a lexical decision on a visually presented word while listening to a semantically related or non-related English sentence. The test was conducted to determine whether both groups of participants were able to predict an upcoming word based on the context of the preceding sentence that is auditorily presented. The study is conducted using PsychoPy software whereas the data was analyzed using linear-mixed effects modeling in RStudio software. The results showed that the L1 speakers were able to predict an upcoming word based on the context of the preceding sentence. That is, a significantly faster recognition of the related word was observed compared to the less related words. On other hand, the English second language participants were not as able to predict an upcoming word as quickly as the English native speaker participants were. However, the L2 participants showed post-access lexical processing or what is called an integrating process of the presented word to the previous sentence context. That is, an effect of the sentence context was observed with L2 participants, yet only after reading the presented word, they decide whether it is appropriate to the preceding sentence context or not.


Introduction
The effects of context on word recognition have been a recent topic of investigation in the field of psycholinguistics, where ample evidence has suggested a facilitative effect of the prior context on upcoming word recognition in either the native language (L1) or the second language (L2) (Batel, 2020;Boston et al., 2008;Demberg & Keller, 2008;Hale, 2001;Levy, 2008;Linzen & Jaeger, 2015). Faster reaction times to plausible word continuation in a given sentence compared with lowcloze probability word continuation have been shown to indicate a facilitative effect of the context of sentences (e.g., Arnon & Snider, 2010;Traxler & Foss, 2000) in different behavioral experiments and event-related potential (ERP) studies (e.g., DeLong et al., 2005;Kutas & Federmeier, 2011), where a reduced plausibility amplitude of N400 was observed in a given sentence. The same effect was observed in eye-tracking studies in which the participants fixated their sight less on plausible word continuation (Rayner et al., 2001;Boston et al., 2008;Demberg & Keller, 2008;Demberg et al., 2013;Smith & Levy, 2013;Staub, 2015). For example, eye-tracking studies (e.g., Altmann & Kamide, 1999) have found that readers spent less time fixating on words with the high cloze probability (HCP) than on words with low cloze probability. Cloze probability is a measurement of a word that is the best fit in the context of a given sentence, expressed as the percentage of subjects who offer that word as a continuation of the context of a given sentence (Block & Baldwin, 2010).
With respect to the effects of context on word processing, three different models have been proposed to offer different explanations of the underlying cognitive process that leads to the facilitative effect of word recognition based on context. First, the serial model assumes that only the word candidate with the HCP is the first word to be predicted by the comprehender. Only if this word candidate is not supported by a given context does the comprehender then turn to the word candidate with the next HCP (MCP) (Van Petten & Luka, 2012, cited in Kuperberg & Jaeger, 2016. Second, in the parallel model, all plausible lexical candidates are computed, considered, and activated in varying degrees of cloze probability. The word candidate with the HCP has a higher threshold compared with the word candidate with the MCP. Therefore, the MCP is also activated, but with a lower threshold compared with the word with the HCP (DeLong et al., 2005;Wlotko & Federmeier, 2012;Staub et al., 2015, as in Kuperberg & Jaeger, 2016. In other words, the serial model assumes that the comprehender predicts only one word candidate at a time, while the parallel model assumes that the comprehender activates all plausible word candidates at the same time in varying degrees. The third is the bottom-up processing model, which assumes that only after lowlevel information is activated by the input is the target word candidate or candidates activated to match the bottom-up input (e.g., Jackendoff & Jackendoff, 2002;Traxler & Foss, 2000). In this model, a large number of plausible word candidates fit a specific sentence context, and predictions can be made of a large number of words prior to encountering them, such that it is a needless waste of cognitive resources (e.g., Forster, 1981;Van Petten & Luka, 2012). Thus, this model suggests the postaccess integration of a word into the prior context. Hence, the fast recognition of a plausible word candidate is the result of the easy integration of this word into the prior context rather than the prior activation of this candidate word (Libben & Titone, 2009). In this case, all plausible word candidates may not vary from each other in terms of recognition times, yet each may be recognized more quickly than low-cloze probability word continuations.

Second Language Studies
With respect to the L2 context-based word recognition process, previous studies have found conflicting results. Some previous studies found a facilitative effect of context on word recognition time (e.g., Ito et al., 2017), which was longer compared with L1 participants. Other studies (e.g., Martin et al., 2013) did not find a facilitative effect of a semantically high-constraint context on L2 word recognition time compared with a semantically low-constraint context. Libben and Titone's (2009) eye-tracking study found that French-English bilingual speaker participants showed shorter gaze times on cognate words (i.e., words of similar spelling and meaning in both languages, such as piano) compared with the control words (i.e., words that had different spellings and meanings in the two languages) in early eye-tracking measurements (i.e., first fixation duration, first pass gaze duration, and proportion of skipped targets when presented in a semantically high-constraint sentence context). In contrast, cross-language homograph words (e.g., coin, [corner in French]) took longer gaze times compared with the control words in the early eye-tracking measurement. However, in the late eye-tracking measurement (i.e., go-past time and total reading time), both homographs and cognates took times that were comparable with the control words in semantically high-constraint sentence contexts. Libben and Titone (2009) interpreted the results of the late effect of sentence context on L2 word recognition as the postaccess integration of a word into a prior context that "reflect[ed] comprehension processes subsequent to lexical access" (p. 387). In other words, the semantically high-constraint sentence context did not affect the process of cognate words, which was expected to take shorter processing times than control words, or the processing of homograph words, which was expected to take longer processing times than control words, until a later stage of processing, when the effects of rich semantic context sentences became evident. These results indicated the presence of a process of word integration into the prior context, which supports the bottom-up processing hypothesis concerning the post-access integration of a given word based on the sentence context.
In line with these results, Martin et al.'s (2013) study showed the effects of semantically rich sentence contexts on L1 words, but not on L2 words during an Event-related Potential (ERP) experiment. Native speakers of English and native speakers of Spanish, who were also advanced L2 English speakers, performed a passive sentence-reading task in English. The sentences were either semantically high-constraint or low-constraint, and critical words were preceded by either the article an (e.g., She has a nice voice and always wanted to be an artist) or the article a (e.g., She has a nice voice and always wanted to be a singer). The N400 amplitude was used in these articles to observe differences in amplitude resulting from the expected upcoming noun (e.g., the more plausible probability "singer" vs. the less plausible "artist") in a given sentence context. The results showed the predicted effect/behavior (modulated by the lower N400 amplitude) in the L1 English speakers on the article preceding the more plausible noun (i.e., a in "a singer") and on the noun itself (i.e., "singer") compared with the article and the noun that were less plausible (i.e., "artist"). However, a non-significant difference was observed between the L2 participants in either the articles or the nouns following them. The study suggested that the lack of an amplitude difference of N400 shown by L2 speakers could have been due to the slower processing of upcoming words in the L2 sentence context. Conversely, this might indicate that the L2 participants adopted an integration process to semantically connect any given word to the proceeding sentence. This post-access integration, which was referred to as L2 "passive integration" by Martin et al. (2013, p. 584), was also observed by Ito et al. (2017), who found that L2 speakers did not show larger N400 amplitudes regarding a more plausible noun continuation compared with a less plausible noun continuation in a given sentence context, whereas L1 participants showed a larger N400 effect on the less plausible word. However, when the time window was extended beyond the 400-msec timeframe, both L1 and L2 speakers showed a lower negativity amplitude in the more plausible word continuation, which indicated the facilitative effect of the sentence context, compared with the less plausible word continuation.
The results of these studies for L2 participants support the bottom-up hypothesis of context-based L2 word processing. That is, the L2 participants did not indicate a pre-access prediction based on the semantic constraint of the preceding sentence context. This result did not eliminate the effect of semantically high-constraint sentences on the L2 word recognition process because easier processing of a more plausible word continuation was observed in an extended timeframe (Ito et al., 2017). These previous studies showed a different interaction between L2 and L1 word recognition regarding the effect of sentence context. However, these studies did not focus on a mechanism for predicting L2 words.
Therefore, the present study explored the effects of sentence context on L2 word recognition to determine whether a prediction process is employed by L2 speakers.
The aim was not to test the effect of sentence context on L2 words per se, but to investigate whether L2 speakers, similar to L1 speakers, could predict upcoming words or whether they would use a bottom-up (post-access integration) process after reading or listening to a given sentence context. In addition, as noted in the preceding review of the results of previous studies on L2 speakers, the effects of semantically high-constraint sentence contexts have been observed in several studies despite the slow recognition time of L2 speakers. However, previous findings regarding slow recognition time neither support nor reject a prediction mechanism that might be employed by L2 speakers. Finally, by integrating visual and auditory modalities, the present study applied a cross-modal priming modality paradigm to elicit results. In this task, the participants were visually presented with a single word while listening to a sentence; they had to make a lexical decision on whether the visually presented string of letters was a word or not. The effect of the sentence context is apparent when the more plausible word (based on its cloze probability ranking in relation to the given context) is recognized more quickly than the low-cloze probability word (Bishop, 2012). The cross-modal priming paradigm has been used since the late 1970s (e.g., Holcomb & Anderson, 1993;Onifer & Swinney, 1981;Swinney et al., 1979;Tabossi, 1988) because it provides a mechanism that allows for capturing the effect of the sentence semantics on a target word during ongoing sentence comprehension (Swinney et al., 1979). It is also one of the few tasks that "can measure moment-to-moment semantic processing while providing only minimal interference with normal ongoing comprehension processes" (Swinney et al., 1979, p. 161).

Study Question
The goal of the present study was to investigate whether L2 speakers showed signs of predicting an upcoming word based on a given sentence context. A group of 36 English L2 speakers and a control group of 36 L1 English speakers participated in this study. Both groups performed a cross-modal priming task in which they were visually presented with a word while listening to an English sentence that was either semantically plausible or low-cloze probability in relation to a given word. They then made a lexical decision on the visually presented word regarding whether it was a word in English or not. This study hypothesized that the stronger the relationship between a word and a given sentence, the faster the recognition time. The main research question (RQ) asked whether a biased sentence context led L1 and L2 participants to predict the upcoming word prior to encountering it.
RQ1: Is there a sentence context-based prediction of an upcoming word in L1 and L2?
It was hypothesized that L1 speakers, in general, would show faster recognition times compared with L2 speakers. It was also expected that the native speaker participants would perform context-based predictions of an upcoming word as an effect of the sentence context. However, in this study, the L2 speakers were not expected to show this predictability, but they were expected to perform a post-access integration process after reading the given L2 word.

Method Participants
The study sample consisted of 72 participants who were divided equally into two language groups: the English L1 speaker group (36 participants) and the English L2 speaker group (36 participants). All participants were compensated for their participation in this study. To reduce variability, the native language of all L2 participants was Arabic. All L2 participants were advanced-level English as a second language (ESL) learners in the fourth year of their academic program, which was the final stage of their studies at Najran University, Saudi Arabia. In addition, the L2 participants completed a self-assessed proficiency rating questionnaire in which they were asked to self-assess their L2 English proficiency in reading, speaking, writing, and listening on a 10-point scale. Self-rating has been widely used as a criterion for selecting L2 participants in the past (e.g., Schwartz & Kroll, 2006). The participants' ages ranged from 23 to 27 years old; 17 were females and 19 were males. The native English speakers were recruited using the MTurk platform (a global online platform used to recruit participants for scientific studies) based on their answers to a pre-study survey related to their native languages, ages, and country of residence. The participants who indicated that they were monolingual native speakers of American English and whose ages ranged from 22 to 28 years old, which matched the age range of the L2 participants, were asked to participate in this study. Eight participants were excluded because they indicated that they spoke a second language in addition to their L1 of English. These participants were excluded from the final sample of 72 participants.

Materials
The materials consisted of 36 experimental sentences and 20 filler sentences, and all created with a software called PsychoPy that records reaction time in millisecond. The experimental sentences were equally divided into three categories, as follows:

1a. Sentences with the high cloze probability (HCP) target words (in brackets)
(1) Emily walked to the bakery in the downtown area to get some (bread) sandwich for dinner.

1b. Sentences with the Medium-cloze probability (MCP)words
(2) Emily walked to the bakery in the downtown area to get some (soup) sandwiches for dinner.

1c. Sentences with the Low-cloze probability (LCP) words
(3) Emily walked to the bakery in the downtown area to get some (cloths)sandwiches for dinner.

Table 2 An Example of the Experiment Sentences and the Target Words for Lexical Decision
Sentence Context High-cloze prob.
Low-cloze prob. Emily walked to the bakery in the downtown area to get some* sandwiches for dinner.
(bread soup cloths) * The time when the target word appears on the screen for lexical decision.
To determine the cloze probability of the experimental words, a plausibility ranking survey was administered by a group of 10 English native speakers who were different from those who participated in this study. These L1 participants worked as judges who were recruited online using Mechanical Turk webpage where each one was asked to write the first and then second word that comes to mind as a continuation when encountering a blank in the given sentence (i.e., the blank represents the position of the critical word). They were also asked to write a third word that is less likely to be a continuation of the given sentence. This third word represents the low-cloze probability.
Based on these judges' word choices, the word that receives the highest percentage was used as the high-cloze probability word followed by the word that received the next highest percentage for each sentence. The same was done with the word with the low-cloze probability where the word that was most frequently written by these judges as a word that is less likely to be a plausible word continuation was used to represent the low-cloze probability word.
The sentences consisted of 10-25 words (average = 17.5 words), and all critical words were nouns. Because the experiment sentences were presented auditorily, a native speaker was recruited to record them in normally paced speech. One-third of the experimental sentences included words in the HCP category (e.g., sentence 1a above), one-third consisted of sentences that were presented with words in the MCP category (e.g., sentence 1b above), and one-third of the sentences included words in the low-cloze probability word category (e.g., sentence 1c above).

Procedure
The participants were required to read the consent form on the screen and sign it by pressing a key on their keyboard before they started the experiment. Each participant performed a cross-modal priming paradigm using the web-based platform Pavlovia (PsychoPy software, https://www.psychopy.org/). Through this platform, the participants were provided with a link through which they could access and perform the experiment online. The results were synchronously sent and recorded in the experimenter's account.
In this task, the participants were required to listen carefully to a recorded sentence. They were told that a string of letters would appear at the center of the screen while they were listening. The participants had to decide whether the presented string of letters was a word or a non-word in English (i.e., a lexical decision) by pressing a designated key on the keyboard as quickly as possible (i.e., the right SHIFT key for a word and the left SHIFT key for a non-word). The string of letters (e.g., the critical word bread in 1a) was presented visually on the offset (i.e., at the end) of the word that preceded the designated position of the critical word in the sentence. For example, in the sentence "Emily walked to the bakery in the downtown area to get some sandwich for dinner," the critical word bread (the HCP in this sentence) was presented at the offset (i.e., at the end) of the word some, which preceded the word sandwich (i.e., the word sandwich was put in the position of the word bread). The participants were required to listen carefully to the sentence until a word was visually presented on the screen. At that moment, they were required to decide whether the visually presented string of letters was a word or a non-word in English by pressing either the right SHIFT key (for a word) or the left SHIFT key (for a non-word) on their keyboard. The same was done in response to the MCP word (i.e., soup in this sentence) and the low-cloze probability word (i.e., clothes). The visual stimuli (i.e., the string of letters) remained on the screen for three seconds before they disappeared. If a participant did not make a lexical decision by pressing on a designated key within this timeframe, their answer was discarded, and it was not included in the data analysis.
The participants were also asked a True & False (T/F) comprehension question immediately after each sentence. This step was included to ensure that the participants were paying attention and carefully listening to the recorded sentences. The participants were informed that the T/F questions were about information that was given in the recorded sentence prior to the visually presented string of letters (i.e., word vs. non-word stimuli) about which they had to make a lexical decision. This made it easier for the participants to focus on the visual string of letters when it was presented on the screen before making a correct lexical decision. Twenty sentences were filler sentences in which the presented strings of letters were nonwords. There were 36 experimental sentences. In addition, the experimental sentences were counterbalanced across the participants so that no single participant worked on the same sentence more than once. Finally, the experiment started with 10 practice trials to ensure that the participants were familiar with the experimental procedure.

Data Analysis
The data were analyzed using a two-way mixed Analysis of Variance (ANOVA), where recognition time (RT) was the dependent factor, and word type (HCP, MCP, and LCP words) and language group (L1 and L2 speakers) were the independent factors. The overall error rate of word/non-word lexical decisions when a participant checked a word as a non-word was less than 4%, which was not included in the data analysis.
The results showed a significant effect of word type (F(2.66) = 68.91, p < .001, η2 = .67) and of language group (F(1.66) = 81.64, p < .001, η2 = .55). In addition, there was a significant interaction between word type and language group (F(2.66) = 4.144, p = .020, η2 = .11), indicating that the two language groups had different results for word type. Based on this result, a post-hoc multiple comparison analysis was conducted by language group (L1 vs. L2) to determine each group's performance on each word type.

Figure 1 Recognition Times for Types of Words by L1 and L2 Participants
The post-hoc results showed that the L1 participants had significantly faster recognition times (p = .0306) for the HCP words compared with the MCP words (high-cloze probability < Medium-cloze probability), and the RT for the words of HCP was significantly faster (p < .0001) than the RT of the low-cloze probability words (high-cloze probability < low-cloze probability). In addition, the RT for MCP words was significantly faster (p = .0008) than the RT for low-cloze probability words (medium-cloze probability < low-cloze probability). In contrast, the post-hoc results showed that the L2 participants had nonsignificant recognition times (p > .05) for HCP words than the RT of MCP words (i.e., high-cloze probability = medium-cloze probability). However, the RT for HCP words was significantly faster (p < .0001) than the RT for the low-cloze probability words (HCP < LCP). Also, the RT for the MCP words was significantly faster (p < .0001) than the RT for the low-cloze probability words (MCP < LCP).

Discussion
The present study investigated whether L1 and L2 speakers predicted upcoming words based on a given sentence context. The effects of prior sentence context on upcoming word recognition were assessed according to three models. The first one was the serial model, which suggests that only a word with the HCP (based on a biased sentence context) is activated. The second hypothesis was the parallel model, which suggests that all plausible lexical candidates become active but to varying degrees, based on their cloze probability level. The third hypothesis was the bottomup processing model, which suggested the ease of integration of a plausible word into the prior context rather than the pre-lexical activation of a target word. Therefore, based on the bottom-up model, a plausible HCP word is not expected to be recognized significantly faster than the MCP word because both words are highly congruent. However, a slower recognition time of a less congruent word was expected because it could not be plausibly integrated into the preceding context.
First, the results showed the facilitative effect of sentence constraint on word recognition in both L1 and L2 participants. low-cloze probability words were recognized significantly more slowly compared with either of the two more plausible words (i.e., the HCP word and the MCP word). In other words, the sentence context led both the L1 and L2 participants to recognize the plausible target word more quickly when it was an appropriate continuation of a given sentence. This result is in line with previous studies that also found a facilitative effect of semantically biased sentence context on word recognition in both L1 (e.g., Altmann & Kamide, 1999;Kuperberg & Jaeger, 2015) and L2 (e.g., Batel, 2020;Kaan, 2014).
The L1 results showed a significantly faster recognition time for the HCP words compared with the MCP and LCP words. In addition, the L1 participants showed a significantly faster recognition time for MCP words compared with LCP words. The L1 participants' hierarchy-based processing of words indicated a prediction pattern that was motivated by the effect of the preceding context. This result suggests that prediction processing occurred prior to encountering the target word, which is compatible with previous studies (Boston et al., 2008;Demberg & Keller, 2008;Demberg et al., 2013;Smith & Levy, 2013;Arnon & Snider, 2010) that found recognition time to be correlated with surprisal as a result of more vs. less predictable words. Hence, words that elicited faster recognition times (i.e., HCP words) reached higher levels of activation compared with words that elicited slower recognition times (i.e., MCP words and LCP words). These results provide evidence that prediction is graded in nature and therefore can be interpreted by the parallel model, in which all possible word candidates are considered to varying degrees based on their cloze probabilities. This finding is based on the significant results for not only HCP words and MCP words but also MCP words and LCP words. That is, if the serial model were used to interpret the results, the difference between the MCP words and LCP words would not be significant because the cognitive process would only activate HCP words and exclude any other word candidates, regardless of their cloze probability (or plausibility) status.
However, the results for the L2 participants differed only slightly from those for the L1 participants. The L2 group showed a non-significant difference in recognition time between HCP words and MCP words. However, the L2 results showed a significant difference between MCP words and low-cloze probability words. In addition, the L2 participants showed a significant difference between HCP words and LCP words (see Figure1).

Table 4 Results for the L2 Group Regarding Types of Words
High-cloze probability words = Medium-cloze probability words High-cloze probability words < Low-cloze probability words Medium-cloze probability words < Low-cloze probability words These results for the L2 participants showed that the effect of the sentence context was not significantly different on HCP words and MCP words. This finding led to the interpretation that the L2 participants, although they showed an effect of the sentence context on slower recognition times for low-cloze probability words, did not conduct a context-based prediction process. In other words, the sentence context did not lead to faster RTs for HCP words than for MCP words, in contrast to the L1 participants. However, both language groups showed significantly slower recognition times for low-cloze probability words, which indicated that they employed the sentence context. Thus, the results for the L2 participants could be interpreted by the bottom-up hypothesis (e.g., Jackendoff & Jackendoff, 2002;Traxler & Foss, 2000), in which a word was integrated into the preceding context after being encountered, but there was no sign of pre-lexical prediction. The large number of plausible word candidates that could fit into the given context would make it difficult for L2 speakers to make pre-lexical predictions. Thus, the bottom-up hypothesis suggests the post-access integration of a word into the prior context. Hence, the fast recognition time for a plausible word candidate resulted from the ease of integration of this word into the prior context rather than its prior activation (Libben & Titone, 2009).

Conclusion
This study examined L2 and L1 speakers' processing of words in English sentence contexts. It was based on three hypotheses regarding whether L2 and L1 speakers could predict an upcoming word based on the preceding context. The first hypothesis was the serial model, the second was the parallel model, and the third was the bottom-up model. The results showed that L1 speakers were able to predict an upcoming word based on the given sentence context, which was compatible with the serial model hypothesis. However, although the L2 speakers showed faster RTs of HCP words and MCP words compared LCP words, they showed no significant differences between HCP words and MCP words, indicating that there was no effect of predictability on these words. The results for the L2 participants could be interpreted according to the bottom-up model, in which ease of integration occurred after the target word was encountered. That is, words in the same category (e.g., food, drinks, transportation, etc.) were semantically supported by the preceding sentence context. Although this process facilitated the integration of these category-related words, it did not lead to the pre-lexical prediction of specific words in this category, which was similar to the processing shown by the L1 participants.