Waiting
Login processing...

Trial ends in Request Full Access Tell Your Colleague About Jove

Behavior

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published: September 5, 2019 doi: 10.3791/59660

Summary

Here, we present a protocol employing dual-modal interference tasks to examine online processing of spoken-Chinese relative clause sentences. Two exemplar experiments involving auditory processing with intra- and extrasentential interferences are described. The paradigm provides a methodology for addressing the nature of working memory and its effects on sentence processing.

Abstract

Working memory (WM) plays a central role in the comprehension of complex sentences. Its function in the processing of spoken complex sentences is especially evident because spoken complex sentence processing is memory-intensive. The dual-modal interference paradigm has been used to examine how the WM system is involved in complex syntactic processing. This article presents two exemplar experiments involving auditory processing with either intra- or extrasentential interferences. In the first experiment, auditory stimuli [spoken-Chinese relative clause (RC) sentences with two syntactic types: subject-gapped (SRC) vs. object-gapped (ORC)] are interfered with via a visually presented lexical decision task within a sentence and manipulated using three different interference timepoints. In the second experiment, the same auditory stimuli, presented via an auditory window moving technique, are interfered with via a visually presented digital recall task beyond the sentence and manipulated using three digital memory loads. By assessing how the primary task of comprehending the RC sentences is affected by the secondary task, we can tackle the controversial issue concerning Chinese RC processing asymmetry. Our results reveal different patterns of RC processing compared to those reported in previous studies. Experiment 1 manifests no clear RC processing advantages in either SRC or ORC; however, a preference for ORC is observed at the ends of sentences, and a preference for SRC is found at the main verb site. Likewise, Experiment 2 presents a dynamic pattern. Under a no-digit load, SRCs show processing advantages in the RC marker region. However, under higher-digit-load interference, ORCs show processing advantages in the same region. These results lead to the conjecture that no obvious or intrinsic processing asymmetry exists in the processing of Chinese RCs. Using the approach of assessing specific interference during syntactic processing, these experiments demonstrate future research applications that explore the processing metrics of spoken sentences involving working memory.

Introduction

The role of working memory (WM) during spoken sentence processing is self-evident: due to the transient nature of speech, listeners must retain the component acoustic forms in their memories until they are processed. This aspect becomes even more important during processing of syntactically complex sentences. Assigning syntactic relations to words in complex sentences involves performing computational operations on items retained in memory for short periods of time, resulting in a higher memory demand. However, how the WM system is involved in spoken sentence processing is controversial.

This controversy involves two major disagreements: some researchers argue that a single WM system exists that is used for all verbal tasks1,2—in other words, syntactic processing relies on the same memory resources used by more general cognitive processes. This is the single-resource model. Others have claimed that determining the meaning of a sentence based on its syntactic structure involves a specialized WM system separate from that used for other verbal tasks3,4. In this vein, syntactic processing is modular. This is the separate-sentence-interpretation resource model.

In psycholinguistic research, the dual-modal interference paradigm has been used to examine the two competing accounts. Based on the assumption that WM storage capacity is limited5,6, the paradigm addresses issues by complicating a primary task with a secondary intervening task. Given that the primary task competes for limited resources with the secondary intervening task, the difficulty increases and the primary task exhibits longer reaction times. Given this situation, the dual-modal interference approach makes it possible to assess the processing load and extent of the WM’s involvement when a participant is given a task that requires fulfilling both tasks simultaneously.

Sentences containing RC components, which cause more comprehension difficulties due to their well-known complex syntactic structures, are widely used to investigate how the WM system is involved in processing complex sentences. However, although processing complex sentences places a higher demand on the WM resources associated with speech processing, it is less clear whether the WM that is thought to contribute to the costs of syntactic movements in languages with head-initial RC constructions (such as English) reflects the syntactic complexity of languages with head-final RCs (such as Chinese). Through the use of a dual-modal interference paradigm, the current study sheds light on this issue.

The difficulties associated with processing two RC structures, subject-gapped and object-gapped relative clauses (SRCs vs. ORCs), have been the subject of extensive debate. These controversies are mainly observed across typologically different languages. In head-initial languages such as English, in which a relative clause follows the head noun it modifies, the general finding is that SRCs such as in example 1(a) below are processed more easily than ORCs in example 1(b).

Equation 1

As shown in example (1), in English, the surface location of the gap differs minimally between SRCs and ORCs. This gap is indexed as e1, the empty position after the head noun ‘actor’ (called filler) left by its removal out of the RC. However, SRCs and ORCs differ substantially in terms of the grammatical structure and function of the gap in the RC domain. The memory cost for integrating and resolving the structural dependency between the filler and the gap is an apt target for experimental study and has been widely used to gain insights into the role of WM in language processing and comprehension.

For example, comprehending and processing these postnominal RCs requires indexing the head noun ‘actor’ as either a functional subject or the object of the verb ‘criticized’ in SRC and ORC and then storing the head noun in WM so it can later be assigned to the grammatical subject of the verb ‘admitted’ in the main clause.

In contrast to the consistent finding with head-initial languages that comprehending SRCs is easier than comprehending ORCs, mixed results have been reported regarding the RC processing asymmetry for Chinese, which is a head-final language in which a relative clause precedes the head noun. Some have observed an SRC processing advantage, while others have reported the opposite pattern (i.e., an ORC processing advantage). The latter lines of research also proposed that RC processing asymmetry can be modulated by WM, as suggested by results obtained from studies of self-paced reading performance7,8,9.

As mentioned above, there are two competing models regarding the role that WM plays in the (complex) syntactic processing. One is that “syntactic processing is modular”, and the other is that “syntactic processing is general”. Complex sentences with well-known differences in comprehension difficulty, i.e., SRCs vs. ORCs in English, are frequently used in dual-modal interference (DMI) tasks to examine these two assertions with respect to the question of modularity because the involvement of WM is claimed to parallel the processing asymmetry. Thus, inducing concurrent memory load through interfering tasks demonstrates WM effects on syntactic processing. The rationale is that whether a single verbal WM system or separate modular syntactic systems exist, engaging the system with an interfering task makes syntactic processing less efficient due to WM resource limitations. The way in which processing syntactically more complex sentences (ORC, in English) suffers in DMI tasks compares to processing syntactically simpler sentences (SRC, in English) provides evidence regarding the specific effect of WM and indicates the extent to which WM is involved.

In contrast to head-initial languages such as English, Chinese RCs manifests a head-final formation and exhibits a gap-filler relationship. The indexed moved-out element, the gap, precedes the head noun associating with it, as illustrated in 2(a), SRC, and 2(b), ORC.

Equation 1

The controversy that stems from processing Chinese RCs is that SRCs are not consistently reported as easier to process than ORCs, and this discrepancy has posed a challenge for theories of language processing and comprehension. Because the prenominal content before the relativizer ‘DE’ must be stored in WM until after the gap—the moved head noun ‘actor’, is linked and retrieved—understanding this process still helps in obtaining insights into the role of WM in language processing.

In the current study, spoken RC sentence processing is examined because listening is highly compressive during processing and is closely related to the functioning of WM. The dual modal interference paradigm is used because interference is a well-established forgetting function in short-term auditory memory. Representations stored in memory can be degraded and subsequently lost when interfering events occur10. Distractors that vary along different aspects (in the current case: intralinguistic and extradigital, see below) to the canonical spoken sentence allow us to measure the cost of integrating the incremental input during different processing phases and under differing interference conditions.

Based on the position that processing more syntactically complex sentences overloads WM more than does processing simpler sentences, one can hypothesize that manipulating the type of interference during the course of comprehension should have effects on sentence processing. By implication, processing syntactically more complex sentences will require either proportionately greater or disproportionately greater listening times online and show worse performance in postonline sentence comprehension assessment than will processing syntactically simpler constructions. The current study examines the hypothesis that interference during sentence processing can index WM involvement and virtually places its value beyond the issue of syntactic modularity: it proposes the idea that the controversy over Chinese RC processing can be elucidated through the investigation of WM due to its fundamental role in language comprehension. Therefore, the significance attached to the use of DMI tasks in Chinese RC processing provides a path to solving the ongoing debate concerning Chinese RC processing asymmetry.

This article presents two exemplar experiments involving auditory processing using both intra- and extrasentential interference. The goal of these two experiments was to explore to what extent WM is engaged in processing Chinese RC under differing types of interference.

In the first experiment, a visually presented lexical decision task was used as the intrasentential interference. As a secondary interfering task, the word/nonword lexical decision task (LDT) was introduced at three points during the auditory presentation of the target relative clause sentence, thus allowing the processing difficulty to be measured at these points. The major concern in this experiment is how the gap in the relative clause (RC) is associated with the filler in the matrix clause (MC) and whether it affects the processing of subsequent MC. Therefore, the three probing sites to be measured were set after the MC region. An example, replicated from (2), of the three probing sites indicated with arrows and aligned with corresponding syntactic concatenation, is illustrated in example 3, where 3(a) shows SRC and 3(b) shows ORC.

Equation 1

Figure 1 shows the procedure of interfering with the continuous auditory RC presentation by the LDT at any of the three probing sites. The timing design follows the conventional protocol of the LDT task in a previous Chinese processing study11. For example, each visual LDT trial begins with a cross sign “+” that indicates a fixation point in the center of the monitor for 500 ms, followed by the visual LDT stimulus, which is displayed on the screen for 3,000 ms and disappears immediately after the subject makes the lexical decision. A typical subject completes Experiment 1, including the practice session, within 30–35 minutes.

Figure 1
Figure 1: Intrasentential interference procedure with a lexical decision task.
Please click here to view a larger version of this figure.

The three probing sites along with the LDT task:

1. Position 1 (P1): Post-SMC region

The first position (P1) to be measured is immediately after the subject of the MC at the region after the RC boundary. Processing load is expected to incur at this site. For one thing, prior to this point (SMC), subject-gap and object-gap construction within the RC domain form contrastive verb-object (VO) and subject-verb (SV) structures, respectively. For another, to integrate the constituents in the RC region with the head noun in MC, listeners must identify the grammatical role of the gap and link it with the upcoming filler head noun

2. Position 2 (P2): Post-VMC region

The second position (P2) to be measured is immediately after the verb in the matrix clause (VMC). This site is also assumed to induce processing load. Integrating the verbal information requires listeners to retrieve the noun arguments in the sentence and to identify the agent of the matrix verb either from the preceding RC domain or from the head noun that the RC modifies.

3. Position 3 (P3): Postsentence region

The third position (P3) to be measured is immediately after the end of the sentence. Previous studies on processing propose that there is an end-of-sentence wrap-up effect—a phenomenon in which nonsyntactic information (e.g., discourse and semantic level) are considered at the end of a sentence to activate and complete comprehension12,13. Therefore, processing load should increase toward the end of the sentence due to the need to integrate this nonsyntactic information14,15. Position 3 is assumed to show a degradation in the processing load because sentence resolution has been attempted around this site.

In the second experiment, an auditory moving window (AMW) task was adopted. The AMW technique is considered to be able to capture patterns of resource allocation during online linguistic processing and has been widely used in attempts to distinguish between the two competing WM approaches16,17. It is presumed that extrasentential interference should cost listeners extra time during the course of processing the transient upcoming spoken sentence. Under the AMW paradigm, the participants heard sentences that were segmented into words, and they pressed a key on the keyboard to initiate playing of the subsequent segment. Thus, the durations of the pauses between keypresses to initiate the subsequent segment and control the flow of incoming information reflect the participants’ responsiveness to the particular linguistic features in question. For example, if the extrasentential interference has certain effects on processing sentences of differing syntactic complexity, participants will exhibit correspondingly longer pause durations before initiating the subsequent segments. The procedures are schematized and presented in Figure 2.

Figure 2
Figure 2: Extrasentential interference procedure with a digit recall task.
Please click here to view a larger version of this figure.

The following protocol shows how researchers use a visually presented lexical decision task as intrasentential interference and the concurrent arithmetic interference load as extrasentential interference to investigate WM involvement and the processing asymmetry of Chinese RCs and elaborate the underlying logic.

Subscription Required. Please recommend JoVE to your librarian.

Protocol

The administration of these experiments followed all research ethics regulations. All subjects provided informed verbal and written consent before the experiments were administered. All the procedures, consent forms, and the experimental protocol were approved by the Research Ethics Committee of the National Cheng Kung University in Taiwan.

1. Experiment 1―dual-modal intrasentential interference task

  1. Recruit 97 students, 54 females and 42 males, from the National Tainan Institute of Nursing and the National Tainan Secondary High School to participate in Experiment 1.
    NOTE: All participants are required to be fluent native Chinese speakers with normal or corrected-to-normal vision and no auditory impairment by self-report.
  2. Material preparation
    1. Select words and nonwords for the LDT. Include a total of 48 bisyllabic (two-character) Chinese words, of which 24 were words and 24 were nonwords.
      NOTE: A Chinese character represents a syllable, which is usually a morpheme (i.e., the smallest meaningful element). The target words here are bisyllabic compound words. Please see the Supplemental File for a list of the visual target words/nonwords used for the LDT task.
      1. Select the 24 words from the Sinica Corpus Technical Report18, while ensure that all of the target words are of medium frequency. Search words of the mean frequency percentage approximately 0.00030 and the ranking order approximately 4,000 in the database.
        NOTE: Choosing words of medium frequency as the target words is intended to reduce the frequency effect, which results in shorter response times (RTs) for high-frequency words and longer RTs from low-frequency words.
      2. Create the 24 nonwords by using two monosyllabic characters that are individually meaningful but whose combination is semantically anomalous. To prevent potential activations, avoid bisyllabic-character words with identical radicals (e.g., haiyang, meaning ‘ocean’, is represented in a Chinese bisyllabic character word as Charcter 1, where the radical component  Charcter 3 related to waters is shared in the characters Charcter 2 and Charcter 4).
      3. Collocate manually the 24 nonwords in filler sentences and the 24 words in target RC sentences.
        NOTE: Collocating words with RCs and nonwords with fillers was necessary because only the RTs of LDT from the 24 words with RCs should be considered and included in the statistical analyses.
    2. Auditory RC and filler sentences
      NOTE: Please see the Supplemental File for examples of the SRC, ORC, and filler sentences.
      1. Compose the auditory stimuli into 72 sentences, involving three types of sentences: 24 SRCs, 24 ORCs, and 24 filler sentences.
      2. Divide the 48 RC sentences evenly into two groups to create an incomplete-counterbalanced design, forming 48 trials (12 SRCs, 12 ORCs, and 24 fillers) in the 2 (SRC, ORC) * 3 (probing site) * 2 (word/nonword) conditions.
  3. Setting up the experimental software
    1. Use a standard experimental software (i.e., E-Prime19) to program the experiment according to the software protocols.
    2. Randomize all the stimuli using the experimental software.
    3. Configure the software system to record the following data: (1) the response time, (2) the accuracy rate of the participant’s responses in the LDT, and (3) the postsentence comprehension based on the participants’ keyboard presses.
    4. Include feedback regarding participants’ incorrect lexical decision or no response. Display feedback on the monitor screen immediately after the participant's incorrect or missing response. No feedback is shown when the participant’s response was correct.
    5. Provide a practice section involving trials with feedback.
  4. After the practice session, start the dual-modal intrasentential LDT interference task. During the experimental sessions, allow the participants to take a break between every 24 trials.
    1. Have each participant perform the task individually. First, provide the participants with instructions both in written form on the computer screen and in verbal form by the experimenter. Seat the participants in front of a computer and equip them with headphones.
    2. Instruct the participants to listen to the sentences played through their headphones, while simultaneously, at some point during the listening process, to perform a lexical decision task.
    3. Ask the participants to decide whether the interfering visual probe displayed on the screen was a word or nonword and instruct them to press the response key ‘Yes’ for a word or ‘No’ for a nonword as quickly and accurately as possible.
    4. Inform the participants that a comprehension question would follow immediately after the sentence. Remind them to listen attentively to the auditory sentence while simultaneously perform the LDT task.

2. Experiment 2―dual-modal extrasentential interference task

  1. Recruit 61 college students, 40 females and 21 males, from the National Taipei University of Technology and the National Tainan Junior College of Nursing as participants in Experiment 2.
    NOTE: All the participants are required to be fluent native Chinese speakers with normal or corrected-to-normal visual acuity and no auditory impairment by self-report.
  2. Material preparation
    1. Auditory RC and filler sentences
      1. Compose the auditory stimuli into three types of sentences: SRCs, ORCs, and filler sentences. Divide the 48 RC sentences evenly into two groups to create an incomplete counterbalanced design with 96 total trials (24 SRCs, 24 ORCs, and 48 fillers) for the 2 (sentence type: SRC, ORC) * 3 (digit load) conditions.
        NOTE: Please see the Supplemental File for target auditory trial examples of the SRC, ORC, and filler sentences.
    2. 0/3/5 digits
      1. Construct a total of 96 digital items, comprised of 0/3/5 digit combinations. Assign each 0, 3, or 5 digit load evenly to all the sentence trials.
  3. Dual modal extrasentential digital interference task with AMW paradigm
    1. Use a standard experimental software (i.e., E-Prime19) to program the experiment according to the software protocols.
    2. Randomly assign the participants to one of the two stimuli sets representing combinations of two within-subject factors of sentence type (SRC vs. ORC) and memory load (no load, 3-digit- load, 5-digit-load).Provide the participants with the 1,500 ms visual presentation of the digits before the AMW task.
  4. Then, start the AMW task.
    NOTE: The AMW task20 is a self-paced listening task.
    1. Instruct the participants to keep the preceding visual presentation (digits or no digits) in memory.
    2. Then, instruct the participants to listen to the sentences segmented into words and played through their headphones. Tell them to pace themselves as quickly as possible by pressing the keyboard to initiate playing of the subsequent segmented word.
    3. Instruct the participants to answer the yes/no comprehension question that appeared on the computer screen after they have listened to each trial sentence. Inform the participants that the question is preceded by a question mark “?” on the computer screen and that the question is related to the information they have heard in the preceding sentence.
    4. A short “beep” sound is played when the participants press the yes/no key to answer the comprehension question. After the beep, following the instruction appearing on the screen, ask the participants to repeat the digit they have seen prior to listening to the sentence.
    5. Have the experimenter to record the participants’ digit-recall responses on a score sheet.

Subscription Required. Please recommend JoVE to your librarian.

Representative Results

The interference effect was observed in both the dual-modal intra-LDT and the extradigit load tasks. Considering the three probe sites in Experiment 1, the RT results of the intra-LDT task manifested a dynamic pattern of RC processing as a function of two RC types. As shown in Figure 3, the ORC type exhibits a processing advantage at the position post the matrix subject (SMC) after the RC (P1) and at the end of the sentence (P3), whereas the SRC type has an advantage at the position post the matrix verb (VMC) after the RC (P2). The simple main effect of sentence type was significant at P2 and P3, indicating that the SRCs had a lower processing load with the LDT interfering task at the matrix verbs (P2), while the ORCs had a lower processing load at the ends of the sentences (P3). These results were different from the consistent SRC preference reported by RC studies in head-initial languages such as English and also varied from the competing SRC and ORC advantages reported in previous Chinese RC studies.

Figure 3
Figure 3: The results of Experiment 1, showing the average RTs (in ms) as a function of sentence types and interfering probe sites.
*The main effect of sentence type was found to be significant under P2 and P3. Please click here to view a larger version of this figure.

The RT results of the extradigit-recall task in light of the three-digit load conditions in Experiment 2 manifested a dynamic pattern of RC processing as a function of two RC types as well. The SRCs and ORCs showed no significant difference across the three extrasentential digit-load interference conditions. Around the post-DE regions (i.e., the same regions detected by Experiment 1), Experiment 2 replicated the SRC advantage at the matrix verb site, T5 (V2=VMC) under 0-digit load interference. However, the RC advantage inclined to favor ORCs under 3- and 5-digit load conditions. This dynamic pattern was observed in the relativizer DE-region (T3), where SRCs showed a processing advantage under a 0-digit load, but a reversed ORC advantage emerged when the load interference increased to 5-digits. All these results provide negative evidence against either the SRC or ORC processing advantages reported in previous Chinese RC studies.

The results of differing digit-load interference on Chinese RC processing in Experiment 2 are shown in Figure 4, Figure 5 and Figure 6, respectively.

Figure 4
Figure 4: The results of Experiment 2, showing the average RTs (in ms) as a function of sentence types under a 0-digit load. Please click here to view a larger version of this figure.

Under the 0-digit load interference, participants showed an advantage for ORCs at the initial processing (T1) region of RC but SRCs had an advantage at the relativizer DE (T3) and the matrix verb (T5) regions.

Figure 5
Figure 5: The results of Experiment 2, showing the average RTs (in ms) as a function of sentence types under a 3-digit load. Please click here to view a larger version of this figure.

Under the 3-digit load interference, no SRC/ORC difference was observed around the pre-DE regions (T1 and T2); however, participants showed an SRC advantage at the relativizer DE (T3) and an ORC advantage at the matrix verb (T5) region.

Figure 6
Figure 6: The results of Experiment 2, showing the average RTs (in ms) as a function of sentence types under a 5-digit load. Please click here to view a larger version of this figure.

Under the 5-digit load interference, participants showed an overall ORC advantage around the pre-DE regions (T1 and T2) and the post-DE regions in the matrix subject (T4) region.

Supplemental Table 1. Please click here to download this table.

Supplemental Table 2. Please click here to download this table.

Supplemental Table 3. Please click here to download this table.

Subscription Required. Please recommend JoVE to your librarian.

Discussion

This study demonstrates that using DMI methods with both intra- and extrasentential interference tasks can help to elucidate the role of WM in spoken sentence processing and shed light on the issue of Chinese RC processing asymmetry. As expected, by measuring the extent to which interference from a secondary task affected listeners’ performances on primary sentence processing, we can infer the patterns of Chinese RC processing and arrive at a feasible solution to the debate on the Chinese SRC/ORC processing advantage.

The interference effect demonstrates that WM plays an essential role in spoken sentence processing—regardless of the source of the interference. In our case, for Experiments 1 and 2, the intrasentential interference is a lexical decision task semantically unrelated to the sentence, while the extrasentential interference is a nonlinguistic arithmetic task involving increasing difficulty levels of digit-load recall. This approach relies on the assumption that the WM involved in the primary and secondary tasks either shares a common cognitive resource1,2,21 or operates through separate systems3,4. Evidence for the generality of memory load under the former model originates from simply finding that participants made more errors or had longer processing/reaction times (RTs) in a sentence comprehension task under interference conditions, while evidence for the specificity of memory load on sentence processing under the latter model stems from the finding that processing syntactically more complex sentences results in disproportionately worse performance compared with processing syntactically simpler sentences under interference conditions. Namely, investigating to what extent the performance of primary sentence processing is affected by the presence of interference would provide evidence for whether a general or specific domain of the WM system is involved in both tasks.

Nevertheless, note that the aim of the current study, rather than addressing the issue of modularity, focuses more on whether we can apply the notion of WM capacity limits to such controversial issues as Chinese RC processing asymmetry, for which mixed and different results have been attained compared to the consistent SRC processing advantage observed in English. In this study, we hypothesized that by measuring the extent to which interference from the WM system affects primary spoken-Chinese RC sentence processing, we can detect the processing profile for both SRC and ORC and determine the RC processing asymmetry. If any RC processing preference exists in Chinese, we should observe either an SRC or ORC advantage from the participants' performances under both intra- or extrasentential DMI tasks.

However, this is not the case in Chinese. Our results showed that Chinese does not have the same RC processing asymmetry as has been observed for English. The results of Experiment 1 manifested in dynamic rather than constant patterns of participants’ RC preferences. An ORC advantage was found around the matrix subject region, while an SRC advantage was found around the matrix verb region. One possible inference is that Chinese RC, given its typological differences from English, does not possess a clear-cut preference for SRC or ORC.

Experiment 1 represents a typical dual-modal procedure with intrasentential interference during sentence processing. Although this is a simple way to measure spoken sentence comprehension processes in real time, precautions must be taken when constructing the test stimuli, when measuring the three probing sites after the RC marker DE, and when aligning the probing results with the LDT task. A critical step when designing the protocol was that the target SRC and ORC sentences had to be paired with words and that the filler sentences had to be collocated with nonwords only. Given this situation, the processing times (RTs) of the RC sentences can be measured and compared solely by their interference with the real-word judgment. Another critical note to consider is the counterbalanced stimuli design. Each of the LDT visual words may appear only once in the task, and each of the RC stimuli can be probed at only one site. In addition, the combination of the word/nonword LDTs and the three probe sites coupled with the two RC sentence types formed 2*3*2=12 conditions, but only 48 trials were constructed in the current study. Although measuring only one site for each sentence during the dual-modal interference task has the advantage of reducing the interference and making the task as similar as possible to normal listening, it also imposes the limitation that a single observation site will require the experimenter to design a large number of stimuli if he or she intends to measure all the regions in the sentence comprehension process and to strike a complete counterbalance. Therefore, using only 48 trials to measure the three probing sites in Experiment 1 achieves merely an incomplete counterbalance, which was the motivation for the subsequent Experiment 2 to overcome the limitation.

Experiment 2 shows the advantage of adopting a dual-modal interference task: it is versatile and can be modified and adapted to address various questions relevant to memory load and the nature of WM. Moreover, dual-modal interference under the AMW paradigm makes it possible to measure all the regions during the flow of sentence processing. The results of Experiment 2 replicated the finding in Experiment 1 that Chinese RC processing is dynamic. Under the no digital interference situation, an ORC advantage was found at the initial region, while an SRC advantage was found around the RC marker DE and the matrix verb. However, as the digital interference increased (the 3- and 5-digit loads), the participants showed an overall ORC advantage around the pre-DE regions and the post-DE regions at the matrix subject, showing that factors that involve WM can alter Chinese RC processing dynamics. These findings provide robust negative evidence for the null RC processing asymmetry model in Chinese. Regarding the claim that there is no clear processing preference for SRC or ORC, supplementary support from an EPR study has recently been reported as well22.

Traditionally, a widely used research method for RC processing is to analyze tasks based on the self-paced reading paradigm, which is considered to be the simplest way to measure language comprehension processes in real time and is considered to consist of “tasks that are as similar as possible to normal reading”23. However, despite providing a window into the largely automatic parsing process, the classical word-by-word self-paced reading task, which does not allow subjects to look back, fails to reflect the back and forth eye movements readers make for integration and comprehension during the natural reading process. That the subjects are limited in going back to process prior materials is in fact much closer to natural listening comprehension task.

In light of the fact that listening is phylogenetically closely related to language processing and language development and that few studies have been conducted to investigate online auditory sentence processing, the current study contributes by offering a feasible application of this DMI approach for spoken-sentence processing.

The significant innovation of this study is that it applies the quest for the nature of WM and the assertion that complex syntactic constructions place high demands on WM resources associated with speech processing to address a long-standing issue of RC processing asymmetry in Chinese. The results of this study demonstrate that Chinese SRCs and ORCs do not show significant differences under either intrasentential or extrasentential interference conditions. These findings differ from the previous mixed results regarding an SRC or ORC advantage in Chinese, thus leading to the conjecture that the RC processing asymmetry consistently found in head-initial languages such as English does not exist in Chinese. This also implies that the complexity metric between SRC and ORC in Chinese might differ from that in English. Due to the characteristics of its syntactic constructions, Chinese RCs should be processed in a language-specific way.

The dual-modal interference paradigm combined with the LDT task and AMW technique is a novel approach that can successfully achieve the goal of measuring the course of processing in real time, and such measurements can help to shed light on the issue of processing complexity. Note that a critical step exists in both Experiments 1 and 2 regarding the instructions to the participants. Because the lexical decision task in Experiment 1 involves comparatively less memory burden but induces a more automatic response, the participants must be instructed to pay close attention when listening to the sentence. In contrast, in Experiment 2, because digit recall induces more memory load, the participants must be instructed to prioritize attempting to remember the digits correctly while listening to the sentence as quickly as possible, but they still need to be reminded to avoid listening too quickly and failing to capture the meaning of the sentence. Using this approach, the tradeoffs between attending to the sentence and the digit recall task can be diminished. In addition, to prevent participants from becoming fatigued when listening to the sentences, we advise giving them brief respites; consequently, the testing time is likely to exceed 30 minutes.

The significant finding in this study underscores the necessity for future studies exploring the processing metrics of RC sentence complexity to consider WM involvement. The dual-modal interference task—in particular the one using the AMW technique—can serve to address this issue when used with either intrasentential or extrasentential interference. Moreover, this procedure should be of broad interest and highly applicable to psycholinguists attempting to further our understanding of the nature of WM and its association with sentence processing.

Subscription Required. Please recommend JoVE to your librarian.

Disclosures

The authors have nothing to disclose.

Acknowledgments

This study was supported by grants from the Ministry of Science and Technology, Taiwan, R.O.C. [NSC-101-2410-H-439-001] to the first author, Tuyuan Cheng. The authors thank lab members, Yang Ya-Hui and Chen Pei-Han, in NTIN, for their assistance in preparing and conducting the experiment.

Materials

Name Company Catalog Number Comments
E-Prime Psychology Software Tools version Professional 2.0
Headphone Logitech
Praat Praat 5.3.43 The online software used to edit the sound files for listening; http://www.fon.hum.uva.nl/praat/
Serial Response Box Psychology Software Tools
Standard PC ASUS K42Jv laptop

DOWNLOAD MATERIALS LIST

References

  1. Just, M. A., Carpenter, P. A. A capacity theory of comprehension: Individual differences in working memory. Psychological Review. 99, 122-149 (1992).
  2. King, J., Just, M. A. Individual differences in syntactic processing: The role of working memory. Journal of Memory and Language. 30, 580-602 (1991).
  3. Caplan, D., Waters, G. S. Verbal working memory and sentence comprehension. Behavioral & Brain Sciences. 22, 77-94 (1999).
  4. Waters, G., Caplan, D., Yampolsky, S. On-line syntactic processing under concurrent memory load. Psychonomic Bulletin & Review. 10 (1), 88-95 (2003).
  5. Cowan, N. Working Memory Capacity. , Psychology Press. New York, NY. (2005).
  6. Miller, G. A. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review. 63, 81-97 (1956).
  7. Chen, B., Ning, A., Bi, H., Dunlap, S. Chinese subject-relative clauses are more difficult to process than the object-relative clauses. Acta Psychologica. 129, 61-65 (2008).
  8. Gibson, E., Wu, H. H. Processing Chinese relative clauses in context. Language and Cognitive Processes. 28, 125-155 (2013).
  9. Hsiao, F., Gibson, E. Processing relative clauses in Chinese. Cognition. 90, 3-27 (2003).
  10. Lewandowsky, S., Oberauer, K., Brown, G. D. A. No temporal decay in verbal short-term memory. Trends in Cognitive Science. 13, 120-126 (2009).
  11. Wu, J. T., Chou, T. L., Liu, I. M. The locus of the character/word frequency effect. Advances in the study of Chinese language processing. Chang, H. W., Huang, J. T., Hue, C. W., Tzeng, O. J. L. , Taipei. Taipei: National Taiwan University (in Chinese) 31-58 (1994).
  12. Fodor, J. D., Ni, W., Crain, S., Shankweiler, D. Tasks and timing in the perception of linguistic anomaly. Journal of Psycholinguistic Research. 25 (1), 25-57 (1996).
  13. Swinney, D., Zurif, E. Syntactic processing in aphasia. Brain and Language. 50 (2), 225-239 (1995).
  14. Balogh, J., Zurif, E., Prather, P., Swinney, D., Finkel, L. Gap-filling and end-of-sentence effects in real-time language processing: implications for modeling sentence comprehension in aphasia. Brain and Language. 61 (2), 169-182 (1998).
  15. Granier, J. P., Robin, D. A., Shapiro, L. P., Peach, R. K., Zimba, L. D. Measuring processing load during sentence comprehension: visuomotor tracking. Aphasiology. 14 (5-6), 501-513 (2000).
  16. Waters, G. S., Caplan, D. Age, working memory, and on-line syntactic processing in sentence comprehension. Psychology and Aging. 16, 128-144 (2001).
  17. Waters, G. S., Caplan, D. Working memory and online syntactic processing in Alzheimer’s disease: Studies with auditory moving window presentation. Journal of Gerontology: Psychological Sciences. 57B, 298-311 (2002).
  18. CKIP. Zhongwen shumianyu pinlü cidian [Dictionary of Chinese written word frequency], CKIP Technical Report, No. 94-01. , Institute of Information Science, Academia Sinica. Taipei. (1994).
  19. MacWhinney, B., James, J., Schunn, C., Li, P., Schneider, W. Step—A system for teaching experimental psychology using E-Prime. Behavior Research Methods, Instruments, and Computers. 33 (2), 287-296 (2001).
  20. Ferreira, F., Henderson, J., Anes, M., Weeks, P., McFarlane, D. Effects of lexical frequency and syntactic complexity in spoken language comprehension: Evidence from the auditory moving-window technique. Journal of Experimental Psychology: Learning, Memory, and Cognition. 22, 324-335 (1996).
  21. Daneman, M., Carpenter, P. A. Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior. 19, 450-466 (1980).
  22. Bulut, T., Cheng, S. K., Xu, K. Y., Hung, D. L., Wu, D. H. Is there a processing preference for object relative clauses in Chinese? Evidence from ERPs. Frontiers in Psychology. 9, 1-18 (2018).
  23. Mitchell, D. C., Green, D. W. The effects of context and content on immediate processing in reading. Quarterly Journal of Experimental Psychology. 30 (4), 609-636 (1978).

Tags

Online Syntactic Processing Spoken Complex Sentences Chinese Dual-modal Interference Tasks Working Memory Processing Asymmetry Chinese Writing Courses DMI Paradigm Co-decision Task Matco Moving Window Technique Cross Of Spoken Sentence Processing Real-time Peihan Chen NTIN Bi-syllabic Chinese Words Lexical Decision Task Sinica Corpus Technical Report Medium Frequency Non-words Semantically Anomalous Identical Radicals Filler Sentences Target Relative Clause Sentences Auditory Stimuli Subject-gapped Relative Clause Sentences Object-gapped Relative Clause Sentences Incomplete Counterbalanced Design
Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
Play Video
PDF DOI DOWNLOAD MATERIALS LIST

Cite this Article

Cheng, T., Wu, J. T. ExaminingMore

Cheng, T., Wu, J. T. Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks. J. Vis. Exp. (151), e59660, doi:10.3791/59660 (2019).

Less
Copy Citation Download Citation Reprints and Permissions
View Video

Get cutting-edge science videos from JoVE sent straight to your inbox every month.

Waiting X
Simple Hit Counter