Optical mapping of brain activation during the English to Chinese and Chinese to English sight translation

: Translating from Chinese into another language or vice versa is becoming a widespread phenomenon. However, current neuroimaging studies are insufficient to reveal the neural mechanism underlying translation asymmetry during Chinese/English sight translation. In this study, functional near infrared spectroscopy (fNIRS) was used to extract the brain activation patterns associated with Chinese/English sight translation. Eleven unbalanced Chinese (L1)/English (L2) bilinguals participated in this study based on an intra-group experimental design, in which two translation and two reading aloud tasks were administered: forward translation (from L1 to L2), backward translation (from L2 to L1), L1 reading, and L2 reading. As predicted, our findings revealed that forward translation elicited more pronounced brain activation in Broca’s area, suggesting that neural correlates of translation vary according to the direction of translation. Additionally, significant brain activation in the left PFC was involved in backward translation, indicating the importance of this brain region during the translation process. The identical activation patterns could not be discovered in forward translation, indicating the cognitive processing of reading logographic languages (i.e. Chinese) might recruit incongruent brain regions. the best our knowledge, this study is the first that uses optical neuroimaging to explore the neural mechanisms underlying Chinese/English sight translation. As expected, our novel findings revealed that cerebral activation patterns vary according to the translation direction, indicating translation asymmetry does exist in the Chinese/English language pair. In particular, forward translation elicited more pronounced brain activation in Broca’s area, suggesting that more cognitive effort is required in this direction. In contrast, the left PFC was involved in backward translation, indicating its importance during the translation process. These findings reveal significant differences between the neural mechanisms of forward and backward translation. The identical activation patterns were not observed between L2-L1 and L2, implying that the cognitive processing of reading ideographic languages (i.e., Chinese) might recruit brain areas that are incongruent to those recruited during translation. In addition, although the fNIRS system has showed the unbeatable advantages in clinical neuroimaging, it also has its limitations. Due to the strong optical absorption of blood, the penetration depth of fNIRS is generally less than 3cm (between 2 and 3cm). Due the strong photon in tissue, imaging resolution of fNIRS is around 6mm, is much lower than that of fMRI.


Introduction
As the world's bilingual population is growing rapidly, bilingual speakers who can translate from one language to another are becoming widespread. Recent work has validated translation asymmetry, which is characterized as the asymmetrical levels of cognitive cost between forward translation (translating from L1 to L2) and backward translation (translating from L2 to L1) [1][2][3]. However, the neural correlates of translation have not been studied in depth. Therefore, investigation into the interaction and difference between forward and backward translation constituting translation asymmetry will pave a new avenue for an improved understanding of the associated neural mechanism.
According to the Activation Threshold Hypothesis [4], when a bilingual speaker selects to speak one language instead of another, the activation threshold of the non-selected language is raised sufficiently to prevent interference during production. In case of translation, it is discovered that both language systems have to be activated at the same time although not necessarily to the same extent. The language that is being decoded may have a higher threshold than the language being concurrently encoded. A word must reach a certain activation threshold in order to become available during the production of an utterance. Since the availability of a word is a function of the frequency and recency of its activation [5], expressions may be more available in one's L1. And the cognitive processing in raising L2, which is required by forward translation, is more effortful. Therefore, it is indicated in previous studies that the forward translation can elicit more pronounced brain activation.
Interestingly, neuroimaging studies have been performed to examine the neural substrates of translation asymmetry using various techniques, such as positron emission tomography (PET), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS), but no congruent results have been obtained [6][7][8][9][10]. Early neuroimaging studies were focused on examining whether the translation asymmetric effect exists, and identifying the brain regions that are correlated with the translation process. For example, a PET study discovered that both forward and backward translation was correlated with significant neural activity in inferior and dorsolateral frontal and prefrontal regions. In particular, forward translation involved greater neural activity in the left putamen than did backward translation [6]. A similar translation asymmetry effect was discovered and they also found that compared to shadowing, both forward and backward translation tasks resulted in a strong increase in activity within the left frontal lobe, including the dorsolateral frontal cortex [8]. More importantly, significantly enhanced brain activation was observed in Broca's area during forward translation. The translation asymmetry effect was also demonstrated in an EEG study [9]. By contrast, they discovered that relative to resting state neural activity, interpreting tasks produced more pronounced brain activation patterns in the left temporal cortex and that the right hemisphere was more involved in forward translation. However, this finding was not replicated in other studies [7,10], in which the translation asymmetry effect was not revealed by neuroimaging techniques. To date, despite growing neuroimaging information about the translation process, there is no systematic neural evidence for specific differences between forward and backward translation. In addition, converging evidence has suggested that the left prefrontal cortex (PFC) and Broca's area play important roles in the translation process [8,10]. In particular, the left PFC is related to the important functions of executive control, such as working memory, cognitive flexibility, planning, inhibition, and abstract reasoning [11][12][13], whereas Broca's area is associated with various language tasks, including verbal working memory [14], morphosyntactic processing [15] and semantic analysis [16]. Based on previous studies and the evidence of the involvement of these two areas in translation-related processing, it seems that both the left PFC and Broca's area are involved in the translation process of right-handed bilinguals.
In addition, although neural mechanism-oriented investigations have been conducted on translation asymmetric effects and associated brain activation patterns, most of the work completed is focused on detecting translation asymmetry between two phonographic languages, such as English/German and English/French language pairs. Although there are studies focusing on two incongruent types of languages (i.e. the phonographic language and ideographic language, especially English and Chinese) [17,18], the tasks in these studies are mainly language switching rather than translation; investigations into the translation between two incongruent types of languages have not been conducted. It remains unclear whether identical brain regions are involved in two categories of the translation process: English vs. Chinese and one phonographic language vs. another phonographic language [19]. More importantly, previous studies did not control the stimuli strictly by ruling out confounding variables at the textual level, such as word frequency, familiarity, readability, difficulty of translation, etc., which will definitely affect the accuracy of the claims.
Based on the Activation Threshold Hypothesis and previous findings, we hypothesize in this study that (1) an asymmetric effect does exist during English/Chinese sight translation, with forward translation eliciting more pronounced brain activation in bilinguals, and that (2) the translation process involves brain regions that include the left PFC and Broca's area. To examine these hypotheses, it is essential to explore the neural substrates underlying asymmetric effects during English/Chinese sight translation, a typical modality for simultaneous interpreting practices. Specifically, fNIRS, as a rapidly growing non-invasive neuroimaging technique, was adopted for the present work. fNIRS is able to offer unsurpassed temporal resolution and provides quantitative hemodynamic information about both oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) [20]. Compared to PET and fMRI, fNIRS exhibits unbeatable advantages in mapping brain activation and networks: 1) it offers a quiet setting for participants; 2) it is proved to be relatively insensitive to movement artifacts; 3) it is user-friendly without causing any nervous tension and can contribute to maximizing the ecological validity for the investigation of brain cognition. In particular, fNIRS is very suitable for the present study, in which continuous overt speech is required. Therefore, fNIRS is the most proper technique for us, which . As a result, because the asymmetric effect and associated brain activation patterns have not been explored for English/Chinese sight translation, this pilot work will definitely pave a new avenue for better understanding the neural mechanisms of such translation.

Participants
Thirty postgraduate students majoring in Translation Studies were recruited from the University of Macau campus. All participants were native Chinese speakers with English as their first foreign language. The participants were first screened using English and Chinese proficiency tests, in which they were required to take both the International English Language Testing System (IELTS) exam to prove that they had reached the required English proficiency and the HANYU NENGLI CESHI (HNC) exam to validate their Chinese proficiency. Only postgraduate students who scored over 80 on both English and Chinese tests were invited to take part in this study to ensure they can complete experiment tasks successfully. Consequently, 13 participants were invited in the experiment; however, data from two participants were excluded from further processing due to incorrect operation during the fNIRS experiment. Therefore, the data reported later was from 11 participants (3 males and 8 females; mean age: 25.73 ± 3.07). Individuals with reported histories of medical illness, neurological disorders or psychiatric disorders were excluded for the present study. All participants were right-handed with normal or corrected-to-normal vision. All of them signed the informed consent forms prior to the experiment, and the protocol was approved by the Clinical Ethics Committee of the University of Macau.

Materials
Although some studies used single word as the stimuli for translation task, we decide to use text with a consideration to maximize the ecological validity of sight translation. Sight translation, a common practice in interpreting market, also requires concurrent processing of input and output. Interpreters usually read and interpret concurrently. However, single word translation task cannot create the same cognitive processing condition in the experiment. Therefore, written texts were used as stimulus in our study. Twelve Chinese texts and 12 English texts were carefully chosen from elementary education textbooks in China and the US, respectively. The content of the texts was very simple and did not involve any technical terms, culturally specific terms, ambiguous expressions or expressions indicating a negative emotion or attitude. In particular, the texts were manipulated according to grade level (educational stage in a system of grades), word count, word frequency, familiarity and translatability.
Grade level: the English texts were selected from Treasures [21], a 2 nd grade textbook used in US elementary schools. The average Flesch-Kincaid Grade Level Test score was 2.61, which can be measured using Microsoft Word 2007 and later versions. The content of the texts were clearly readable by 2 nd grade students in US elementary schools. Therefore, the Chinese texts were selected from the 2 nd grade textbook Chinese Mandarin (KANG HSUAN EDUCATIONAL PUBLISH CORP.), which is utilized in Taiwan's elementary schools. The Chinese stimulus were chosen from Taiwanese textbook because the educational system in Taiwan is modelled on American educational system [19], which makes "grade level" comparable and thus could be manipulated.
Word count: the word count of the English texts was performed using the Microsoft Word's word count function. However, the calculation of Chinese word count is different. The reason is that the idea of a word in a phonographic language such as English is different from that of a word in an ideographic language such as Mandarin [35]. In Mandarin, there are usually two "characters" in a single "word", forming a compound unit which equals to a word unit in English. For example, the Chinese character "沙" means "sand" and "发" means "to deliver; to send out". Juxtaposing the two characters would yield a compound word "沙发" which means "sofa". Therefore, to control word counts for the Chinese texts, the ICTCLAS 3.0 (Chinese Lexical Analysis System) [22] was adopted here to determine the word count of the Chinese texts. The average word counts of the English and Chinese texts were 54.67 and 54.92, respectively, and were not significantly different.
Word frequency: the texts were also examined to exclude most of the infrequent words that ranked above 5000 according to the Corpus of Contemporary American English [23] and the Corpus of Contemporary Chinese [24]. Since these two corpora are different in size, we decided to count on the ranking of frequency of each word in the corpus, instead of counting on the value of frequency. Frequent words that ranked within the 0-5000 interval in the texts of both languages represented 96% of the total word count on average. Familiarity: each individual participant's familiarity with the words in the materials was investigated. The words in the materials used in the experiment were provided to the participants in the form of a word list after the experiment. The participants were asked to tick the words with which they were familiar. Then, the familiarity was calculated as the ratio of the number of familiar words to the total number of words. The average familiarity of the Chinese texts was 99.81%, and the average familiarity of the English texts was 99.36%, which were not significantly different.
Translatability: in this study, translatability is considered the difficulty of translating a text. Ten translation teachers or researchers at UM were asked to read and rate the texts in both languages in terms of translatability, which was scored from 1 to 10 (1 represents "the easiest" whereas 10 indicates "the most difficult"). The average translatability values for the English texts and Chinese texts were 3.37 and 3.40, respectively, which indicated no significant differences in the translation difficulty between texts from the two languages.

Procedures
The experiment consisted of two tasks: the translation task and the reading task. Each task consisted of two conditions. For the translation task, the participants were asked to translate aloud from Chinese to English (C2E, condition 1) or vice versa (E2C, condition 2), whereas for the reading task, the participants were required to read the Chinese and English texts aloud (C, condition 3 and E, condition 4, respectively). To counterbalance the impact of task order, 6 participants were required to complete the tasks in the following order: C, C2E, E, E2C; while the other 5 participants were required to complete the tasks in the following order: E, E2C, C, C2E. Each task was performed 6 times. Therefore, each participant processed 24 texts altogether: 12 Chinese texts and 12 English texts.
All the 12 Chinese texts (or 12 English texts) were randomly allocated in the C2E and C condition (or E2C and E condition): six were for one condition, while the other six were for the other condition. Each text only appeared once for each participant. The stimuli tasks were programmed with E-prime software 2.0 (Psychology Software Tools, Sharpsburg, PA).
As such, there were 6 trials for each condition, which included a pre-stimulus period of 1s with a red fixation cross presented in the center of the monitor, a stimulus period (the stimuli E2C and E, or C2E and E did not disappear until the subjects finished the task and pressed the button), and then a post-stimulus and recovery period of 30s with a white fixation cross displayed in the center of the monitor. The size of the cross matched the size of the letters/characters in the texts. The stimuli tasks were programmed with E-prime software 2.0 (Psychology Software Tools, Sharpsburg, PA). Before fNIRS scanning, all participants were trained to be familiar with the procedure of the test. A post-experiment survey was also conducted for each participant, in which all participants reported successful performance of both the translation and reading tasks, indicating that there were no new words for them.

fNIRS data acquisition
The fNIRS data were recorded with a continuous wave CW6 system (Techen Inc., Milford, MA, Fig. 1(A)), in which four light source emitters, each containing both 690 nm and 830 nm laser lights, and eight detectors were used. Optodes were mounted into a custom-built head cap constructed from plastic and Velcro, which was comfortably worn by the participant (Fig.  1(B)). The configuration of the source and detector pairs is illustrated in Fig. 1(C), in which four emitters and eight detectors were connected by the optical fibers on the scalp to generate 14 channels (Fig. 1(D)) to cover the prefrontal cortex (PFC) and Broca's area of the left hemisphere. The distance between the source and detector for the present study was 3 cm. The fNIRS data were collected at a sample rate of 50 Hz through a custom-built data acquisition interface [25], which can allow fNIRS signals (HbO and HbR) to be visualized in real-time during data acquisition. After the task, the three-dimensional (3D) coordinates of sources and detectors were obtained using a 3D digitizer (PATRIOT, Polhemus, Colchester, Vermont, USA). The average 3D coordinates were then imported to NIRS_SPM [27] for spatial registration to generate the layout of the optodes and MNI coordinates of the channels. The MNI standard coordinates for the 14 channels are provided in Table 1 to show their positions.

Data analysis
For the behavioral data, we collected the mean response time for both the translation and reading tasks. We performed a one-way repeated analysis of variance (ANOVA) to explore the differences between the four conditions (C2E, E2C, C, and E) based on the measures of response time.
The fNIRS data were preprocessed with HOMER2 software [28], and the functions we used and the flowchart of the operations are illustrated in Fig. 2. The measurements of optical density were first converted to the concentration changes in HbO and HbR at different time points. Then, the raw hemoglobin continuous data were processed by a low cut off filter of 0.01 Hz and subsequently a high cut off filter of 0.1 Hz. Data segmentation surrounding the trigger onset time points from the experiment was adopted (e.g., from −5s prior to the trigger onset until 45 s after the trigger onset of this task [29,30]. After data pre-processing, we examined the HbO concentration differences between the translation and reading tasks. We then contrasted the HbO signals from sight translation with that of reading the same input language (sight translation from Chinese to English was compared to reading Chinese (C2E -C), and sight translation from English to Chinese was compared to reading English (E2C -E). This analytic method has been extensively adopted for the study of translation [7,8,31] and aims to eliminate the influence of reading. Accordingly, for both sight translation directions, the comprehension process is languageequivalent for both the sight translation and reference reading tasks, and the translation process is in principle what differentiates the former from the latter [32]. Further, in order to compare the directionality, paired t-tests were used between C2E -C and E2C -E. Since the concentration changes of HbO2 were widely recognized as the most sensitive indicators of the brain's hemodynamic responses, only the analysis of HbO data from all the channels across all the subjects was carried out. The statistical analysis was performed with SPSS software, and the p-value was corrected according to Greenhouse-Geisser when necessary.

fNIRS results
Grand-average concentration changes in oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) were acquired for each channel from the two tasks with four conditions. However, only analysis based on HbO recordings was performed due to its high contrast-tonoise ratio across all the participants. In particular, Fig. 4 shows the average time courses for the concentration change in HbO for the two translation tasks (C2E vs. E2C). Figure 5 depicts the concentration change in HbO for the two reading tasks (C vs. E). It was observed from Figs. 4 and 5 that a task-related increase in the concentration of HbO for most cases occurred five seconds after trigger onset. When the concentration change in HbO reached a peak after 20 or 30 s for the translation task or 15 s for the reading task, it then returned to the baseline again for most of the channels. It should be noted that it took less than 25 s for the participant to finish the reading task, compared to over 35 s for the translation task. In particular, to compare the hemodynamic response between the translation and reading tasks, we used the HbO signals only during the time period from 0 s to 25 s because participants finished the reading task in approximately 25 s. Fig. 4. The mean concentration change in HbO for all channels. The unit on the y-axis is micro-moles. The red line represents the C2E task, and the blue line indicates the E2C task. Fig. 5. The mean concentration change in HbO for all channels. The unit on the y-axis is micro-moles. The black line indicates the C task, and the gray line represents the E task.

C2E vs. E2C (translation asymmetry)
The statistical analysis results showed that the HbO concentration changes between the two tasks were significantly different in channel 5 (t 10 = 2.38, p = 0.04), which was located at Broca's area in this study (Fig. 6(B)).

Discussion
The present study was designed to explore the neural mechanisms underlying translation asymmetry during English/Chinese sight translation. To the best of our knowledge, this study is the first study that uses fNIRS neuroimaging techniques to examine the asymmetric effect for translation between Chinese and English. It was expected that the present work would be able to shed light on revealing the complex brain activation patterns involved in the translation process for other language pairs.
Our behavioral data showed that the L1-L2 (C2E task) translation required a significantly longer time than the L2-L1 (E2C task) translation, suggesting more cognitive effort was involved in the C2E direction. In addition, we discovered that the behavioral results for sight translation showed good agreement with previous findings based on other translation modes between Chinese and English [33][34][35][36]. Interestingly, compared to sight translation, reading required less time. In addition, L1 reading (C task) was less time consuming compared to L2 reading (E task).
Compared to L1 reading (C task), the L1-L2 (C2E) task did not elicit significantly increased HbO concentration changes in either Broca's area or the DLPFC. However, this finding was not the same for the translation process between two alphabetic language pairs [6,8], in which the left frontal area, including the DLPFC, was significantly activated for the translation task. We think the discrepancy is due to the different language pairs involved in the translation. For example, in previous work, the two languages involved for the bilingual participants were both alphabetic language. In contrast, the processing of Chinese, an ancient ideographic language, has been revealed to be associated with different or additional brain areas [37,38]. In particular, for phonological processing of Chinese characters, the left dorsolateral frontal region (Brodmann Area 9) was recognized to be responsible for visuospatial analysis and orthography-to-phonology mapping of Chinese characters. However, for alphabetic languages, the left posterior tempo-parietal regions are known to mediate grapheme-to-phoneme conversion and fine-grained phonemic analysis [39]. Because reading Chinese itself engages the DLPFC, and the DLPFC is also activated during the L1-L2 (C2E) task, it is possible there is no significant difference between the two tasks in brain activation patterns in the DLPFC and Broca's area.
In addition, compared to L2 reading, sight translating into the dominant language (Chinese in this study) yielded increased HbO concentration changes in the left prefrontal cortex (PFC), as shown In Fig. 6(A). This finding was in line with previous observations [6,8], in which increased brain activation was identified in the left frontal lobe, although our investigation was focused on sight translation and the previous studies were focused on word translation and associated simultaneous interpretation. In particular, previous work also indicated that the left PFC was activated during performance of tasks that involved verbal encoding, effortful retrieval, and maintenance or control of semantic information [16,40]. Consequently, the identified brain activation patterns in the left PFC are an essential biomarker for translation asymmetry during Chinese and English sight translation, which involves semantic processing through extracting semantic representation out of the source language and transferring it according to the linguistic structures of the target language. As such, the present findings may not only shed light on revealing the complex neural mechanisms of sight translation but also add more evidence to the function of the left PFC during the translation process.
Interestingly, as shown in Fig. 6(B), sight translating into the non-dominant language (English) elicited significantly enhanced hemodynamic responses in Broca's area compared to translating into the dominant language. The present results also agreed with previous reports that increased neural activity in Broca's area in the left hemisphere was correlated with translation [8]. It is well-known that Broca's area is linked to language processing, which was first identified by Pierre Paul Broca who observed impairments in two patients who had lost the ability to speak due to injuries to the posterior inferior frontal gyrus of the brain [41]. Previous fMRI studies also validated that activation patterns in Broca's area were associated with various language tasks, including verbal working memory [14], morphosyntactic processing [15], and semantic analysis [16], all of which were involved in sight translation and produced different brain activation.
Meanwhile, we also discovered that the brain activation patterns identified indicated differences between the present and previous findings. For example, Klein et al. and Kurz both [6,9] discovered that significant brain activation in the left putamen was associated with forward translation, whereas that in the left temporal and right hemisphere were responsible for backward translation. In addition, Price et al. and Quaresima et al. also highlighted that the brain activation patterns in Broca's area and left DLPFC did not exhibit significant difference between forward and backward translation [7,10]. We suppose that the discrepancies might be due to the inclusion of the Chinese language or the limitations of the fNIRS system, which cannot cover the entire cortex and can only be restricted to the cortical areas.
In this study, fNIRS, as a facilitating tool for exploring and constructing a functional brain network, was used to map the brain activation underlying translation asymmetry during Chinese/English sight translation. To the best our knowledge, this study is the first that uses optical neuroimaging to explore the neural mechanisms underlying Chinese/English sight translation. As expected, our novel findings revealed that cerebral activation patterns vary according to the translation direction, indicating translation asymmetry does exist in the Chinese/English language pair. In particular, forward translation elicited more pronounced brain activation in Broca's area, suggesting that more cognitive effort is required in this direction. In contrast, the left PFC was involved in backward translation, indicating its importance during the translation process. These findings reveal significant differences between the neural mechanisms of forward and backward translation. The identical activation patterns were not observed between L2-L1 and L2, implying that the cognitive processing of reading ideographic languages (i.e., Chinese) might recruit brain areas that are incongruent to those recruited during translation. In addition, although the fNIRS system has showed the unbeatable advantages in clinical neuroimaging, it also has its limitations. Due to the strong optical absorption of blood, the penetration depth of fNIRS is generally less than 3cm (between 2 and 3cm). Due to the strong photon scattering in tissue, the imaging resolution of fNIRS is around 6mm, which is much lower than that of fMRI.