Next Article in Journal
Predictors of Successful Reading Comprehension in Bilingual Adults: The Role of Reading Strategies and Language Proficiency
Previous Article in Journal
Future or Movement? The L2 Acquisition of Aller + V Forms
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Languages
Volume 6
Issue 1
10.3390/languages6010017
languages-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Role of Subtitles in L2 Acquisition and Comprehension: A Pilot Study of Hearing-Impaired Students

by
Mila Vulchanova
1,* and
Ingvild Kjølstad Lervåg
2
1
Language Acquisition and Language Processing Lab, Department of Language and Literature, Norwegian University of Science and Technology, 7491 Trondheim, Norway
2
Fannefjord High School, 6404 Molde, Norway
*
Author to whom correspondence should be addressed.
Languages 2021, 6(1), 17; https://doi.org/10.3390/languages6010017
Submission received: 3 December 2020 / Revised: 7 January 2021 / Accepted: 21 January 2021 / Published: 25 January 2021
Versions Notes

Abstract

:
The purpose of this study was to investigate whether subtitles can facilitate language processing in English as a second language (L2) and, if so, which subtitles would be more beneficial for hard-of-hearing students with Norwegian as their first language. In total, 14 advanced learners of L2 English were recruited and tested on English comprehension and target vocabulary items based on video material provided with subtitles in English or Norwegian in comparison to no subtitles (control condition). Subtitles aided comprehension of the plot, tested immediately after clip presentation, with an advantage for English subtitles over Norwegian subtitles and no subtitles. Furthermore, subtitles were found to enhance the performance of the participants with moderate hearing loss more than they did for mild hearing loss participants. The inclusion of English subtitles only marginally enhanced vocabulary understanding for both mild and moderate hearing loss students. The findings of this study can be transferred to classrooms and may supplement other methods of adjusting the academic environment, in order to meet the need of students with hearing loss.

1. Introduction

Extant research indicates that students who suffer from any degree of hearing loss are at risk for being exposed to language input which is degraded in quality. Sensorineural hearing loss entails lowered hearing thresholds and distortion of sounds which leads to partial and degraded language input (Robier 2001; Delage and Tuller 2007). Furthermore, Moeller and Tomblin (2015) stress that hearing loss could reduce both the amount and regularity of language input for the growing individual and suggest that inconsistent and/or distorted access to input might also be expected to impact on language outcomes. Csizér and Kontra (2020) conducted a qualitative study with deaf and severely hard-of-hearing young adult learners of English as a second language from three European countries (n = 54). They established that what distinguishes this group from their hearing peers is predominantly the lack of adequate native language (L1) skills, and that the lack of a solid L1 foundation is what prevents the successful acquisition of the target language (L2). However, even when a student presents with milder hearing loss, they can find it difficult to successfully learn a second language. Still, few studies have addressed mild-to-moderate hearing loss (MMHL) and second language learning. The goal of this study was to explore factors in the learning environment, such as visual cues in the form of subtitles, and whether they aid hard-of-hearing learners of English in the classroom. This paper is aimed at language teachers and education practitioners, and fills a gap in education research in second language learning for students with a mild-to-moderate hearing loss.
An inclusive learning environment is part of the Norwegian school education policy (Opplæringslova 1998, §1–2). Norwegian schools are thus obligated to make adaptations according to each individual’s abilities. A common technique amongst hard-of-hearing students is lip reading. However, in a classroom setting, it can be difficult for the student to rely on such a tool, as the teacher tends to move around, and the student might not be seated in positions where she can see the teacher at all times. Furthermore, speech intelligibility in classroom settings is also affected by room acoustics, level of background noise and the availability of visual cues for speech (Lewis et al. 2015). Thus, students with any degree of hearing loss might be exposed to non-optimal acoustical environments, elevated hearing thresholds, and the absence or limited use of amplification (Valente et al. 2012; Lewis et al. 2015). Research documents that any degree of hearing loss, even in less severe cases, can interrupt the normal development of communication, social-emotional well-being, and overall academic performance (Dalton 2013).
Most (2004) reported that participants with mild-to-moderate hearing loss displayed lower academic performance than those with a more severe hearing loss. A possible account is that such students often lack classroom support in comparison to more severe cases (Andrews et al. 2011; Reynolds and Fletcher-Janzen 2007; Dalton 2013). As deafness and severe hearing loss have been of interest for a long time, and research has indicated what facilitates language acquisition in such cases, deaf students have, to some extent, benefitted from this in classroom settings. It is also assumed that students with mild-to-moderate hearing loss require no special assistance in an academic setting, and, as a consequence, that there is no need for it (Tharpe and Bess 1991). However, there is little evidence in support of this assumption. In addition, because MMHL students present with mostly intelligible speech, their challenges tend to be overlooked by teachers (Dalton 2013).
The use of films in the classroom is not a new phenomenon, and audiovisual material in general is a frequently used resource in the English as a Second Language (ESL) classroom (Vulchanova et al. 2015). Using audiovisual material is beneficial for several reasons and can provide second language learners with exposure to authentic language material (Dahl and Vulchanova 2014). Specifically, when the film clips are shown with subtitles, the viewing experience has the potential to be extra beneficial for hard-of-hearing students. This is because hard-of-hearing individuals rely more on visual cues in auditory language processing (Cavender et al. 2009). The current study seeks to expand upon these findings and explore the results of utilizing subtitled film clips to supplement education in second language learning for students who experience milder hearing loss. When turning on subtitles the educator is given a choice in deciding which language the subtitles should be shown in, and therefore this paper investigates what language is the best option for the students concerning the subtitling text.

1.1. Mild-to-Moderate Hearing Impairment

It is estimated that up to 15% of students in integrated classrooms in the US suffer from mild-to-moderate hearing loss (MMHL) (Niskar et al. 2001). There are about 700,000 individuals with hearing loss in Norway, and about 0.25 percent of the population under 20 years of age are hearing impaired, although the degree of hearing loss may vary (Hørselshemmedes landsforbund, 2015). Despite the high prevalence of this condition, relatively little is known about the educational outcomes of people with mild-to-moderate hearing loss, and studies in the field of second language learning are almost missing. Mild and moderate hearing disability can be defined as ranging from pure tone air-conduction thresholds of 15–30 decibels (dB) to 30–70 dB (Mehra et al. 2009). However, these terms are audiological threshold categories, and do not necessarily reflect student functionality and academic performance. Dalton (2013) points out that, in comparison to students with profound hearing loss or deafness, such students do not receive intensive educational interventions. The reason is that these students seem to do well in classroom settings, and more importantly, they seem to be able to keep up with their peers. It is therefore easy to overlook their challenges.
People with mild or moderate levels of hearing loss are likely to experience a speech signal that is distorted or degraded (Halliday et al. 2017). This increases the risk for academic, speech and language, and social-emotional difficulties, and often such individuals are not provided with a learning environment which is adjusted to their condition (McKay et al. 2008).

1.2. Consequences of Minimal-to-Moderate Hearing Loss

Goldberg and Richburg (2004) argue that minimal hearing loss causes academic differences between hearing and hard-of-hearing peers. It has been established that hearing problems can cause a delay in language development, and children diagnosed with severe to profound hearing loss are readily identified as needing appropriate intervention at an early stage. At the other end of the continuum are children with mild hearing impairment. Mild hearing impairment is too easily overlooked, mainly due to lack of information and proper awareness (Goldberg and Richburg 2004). Thus, a recent large-scale cohort study designed to establish outcomes for children with hearing loss under-sampled children with mild HL (Tomblin et al. 2015). The authors explain this fact as an indicator that many of these children had not been identified, in addition to the parents of these children being less likely to volunteer to participate, and the referral sources for the study. According to Goldberg and Richburg (2004), the reason why this condition is not taken seriously is because students with a minimal hearing impairment often pass the hearing screening. However, even though students may pass a hearing screening, they remain at risk for not hearing and identifying voiceless and high-frequency consonants in the speech directed towards them (Northern and Downs 2002). Such difficulties often occur in classroom settings, with a lot of background noise and poor acoustics (Goldberg and Richburg 2004; Lewis et al. 2015).
Furthermore, scholars in the field have stressed the importance of appropriate seating for optimal listening conditions due to the classroom acoustics (Bess and McConnell 1981; Crandell and Smaldino 2000). Goldberg and Richburg (2004) argue that seating a student with hearing impairment in front of the classroom will result in sufficient improvement in their hearing ability, however, this will only work if the teacher stands within 3 to 4 meters of the child or faces the student when speaking, as many students with hearing loss rely on lip reading (Crandell and Smaldino 2000). Goldberg and Richburg (2004) further state that this is not realistic, since, in reality, the teacher moves a lot around the classroom. Thus, in theory, preferential seating is a good strategy. Yet, other variables, such as background noise, may interfere for the student to be able to comprehend what is being said. Several studies have demonstrated that young listeners, both with normal and atypical hearing, perform poorly in noisy and busy environments, compared to adults (e.g., Elliott 1979; Soli and Sullivan 1997). These findings show that even though there are different strategies to facilitate a better learning environment, they may not always work due to interfering factors.
Hearing loss has consequences for language development, and specifically for early speech perception. Kishon-Rabin et al. (2015) aimed to evaluate the impact of unilateral hearing loss on early communication skills compared to normal hearing levels. In total, 34 infants with unilateral hearing loss and 331 control infants with normal hearing were divided into two groups, and early auditory skills and preverbal vocalization were assessed using questionnaires. They found that auditory behavior, as measured by the Infant-Toddler Meaningful Auditory Integration Scale, was delayed in 21% of the unilateral hearing loss children compared to 4% in children with normal hearing (Kishon-Rabin et al. 2015).
Fischer and Lieu (2014) compared 20 adolescents with unilateral hearing loss to 13 adolescent siblings with normal hearing on standardized language tests (Oral and Written Language Scales) and the Clinical Evaluation of Language Fundamentals (CELF-4). They found that the group with unilateral hearing loss received poorer overall and expressive language scores than the control group (98 vs. 114; p = 0.001 on total language score; 100 vs. 114, p = 0.006 on expressive language score, and 96 vs. 111, p = 0.019 on CELF score). These findings suggest that unilateral hearing loss in adolescents is associated with a negative effect on standardized language scores (Fischer and Lieu 2014).
A study on language development in MMHL in French adolescents (11–15 years of age) revealed that language skills in that group do not normalize with age (Delage and Tuller 2007). Participants were tested on both oral and written language skills. More than half of the participants displayed language impairment, primarily in the domain of phonology and grammar, and the error patterns were similar to those found in adolescents with language impairment. These results indicate that hearing loss has profound consequences for long-term language outcomes in affected individuals. A follow up study with over 80 participants (6–16 years) establishes that morphosyntactic development in children and adolescents with MMHL is highly likely to be impaired, and may often be severely impaired, and that difficulties continue into adolescence (Tuller and Delage 2014). The authors explain the language impairment and the relationship with hearing loss in terms of (poor) auditory attention, rather than degree of hearing loss.

1.3. Subtitles in SLA

Audio-visual material can be a specifically good source for authentic input in the L2 classroom, when accompanied by subtitles (Baltova 1999; Bianchi and Ciabattoni 2008). The textual information provided in subtitles serves as an extra source of linguistic input either in the L1 or the L2, and, as such, supports the auditory signal. Thus, d’Ydewalle and Van de Poel (1999) and Danan (2004) note that readers automatically read the subtitles, whenever they are available, which indicates that the auditory and the verbal textual information are processed in parallel. There is also evidence that learners spend time attending to both the subtitles and the visual images, thus making use of both channels (Bisson et al. 2014). Whether subtitles in the native language (L1) or in the target language (L2) are more beneficial is still an open question, and empirical results largely depend on what aspect of language is tested and the age and level of proficiency of the participants. Many studies have established that L2 subtitles are more facilitatory (Vanderplank 1988; Markham 1999; Bird and Williams 2002; Danan 2004; Mitterer and McQueen 2009; Vandergrift 2011; Bianchi and Ciabattoni 2008). However, the study by Bianchi and Ciabattoni (2008) documents that target language subtitles can be more advantageous for more advanced learners of English as an L2, while native language subtitles may be more beneficial for lower levels of proficiency. Interestingly, the study by Vulchanova et al. (2015) indicates that for more advanced students, the language of the subtitles is of a lesser importance, with these students performing equally well in the context of L1 and L2 subtitles, whereas for the less advanced students in the study, L2 subtitles were in fact more facilitatory. These results are consistent with the idea that the provision of subtitles may impact differently on comprehension and learning outcomes, depending on the individual characteristics of the learner.

1.4. The Current Study

While research on MMHL has primarily focused on children, and mostly in their first language, few studies have addressed older students, and, to the best of our knowledge, none have looked at second language acquisition in the context of less severe hearing loss (see Howerton-Fox and Falk 2019 for a comprehensive review of language learning in deaf children). Given the evidence of a negative impact of mild-to-moderate hearing loss on first language outcomes, it can be expected that this condition also affects adversely second language acquisition. However, it can also be expected that the provision of adequate adjustments to the learning environment, such as adding visual input in the form of subtitles, can mitigate this negative impact. Moreover, subtitles have been shown to have a positive effect for learners with typical hearing (Vulchanova et al. 2015 and review of findings therein).
Given that hard-of-hearing individuals might rely more on visual cues in auditory language processing, the purpose of this pilot study was to investigate whether subtitles can facilitate language processing and, if so, which subtitles would be more beneficial for hard-of-hearing students. Our research question was, Are subtitles beneficial in second language processing and comprehension for hearing impaired young adult learners of English? We hypothesized that subtitles would enhance participants’ performance both on a comprehension task and on a vocabulary task, and that the participants would appreciate the addition of subtitles. Based on the results in Vulchanova et al. (2015), and consistent with Bianchi and Ciabattoni (2008), we also expected that target language (L2) subtitles would be more beneficial than L1 subtitles, since they provide additional support for the perception of the source language for the students with hearing loss.
To explore this issue, and provide initial evidence, 14 participants with mild-to-moderate hearing impairment participated in an online experimental design. The study included three conditions: film clips with subtitles in the L1 (Norwegian), film clips with L2 (English) subtitles, and a control condition with no subtitles (Vulchanova et al. 2015). After each scenario they responded to questions regarding comprehension and vocabulary relevant to what they had seen. We collected accuracy measures on both comprehension and newly activated words in all three conditions.

2. Method

2.1. Study Design and Stimuli

We followed the methodology in the study by Vulchanova et al. (2015) on effects of subtitles in the context of authentic material on second language comprehension for Norwegian learners of English. The method of data collection and analysis were overall quantitative. The study design was within-subjects, meaning that the same person tested on all conditions and on the exact same stimuli, with the possibility of investigating individual change of behavior across conditions (Charness et al. 2012). Participants were selected based on two criteria: hearing loss and being learners of English as a second language.
Participants viewed a documentary called Expedition Happiness. The reason why a documentary was selected was because documentaries are widely used in ESL classrooms, and because a documentary contains detailed explanations which might facilitate initial stages of word learning and comprehension. There was no accompanying music that could be disturbing for the participants. From the documentary, scenes that were approximately 5–6 min were selected for the experiment. Each scene was presented in one of three conditions, with English subtitles, Norwegian subtitles or no subtitles. Three comprehension and three vocabulary questions were included after each scene (see Supplementary 2 for examples). Thus, within each condition, two variables were of interest: comprehension performance and vocabulary performance. Participants’ level of hearing was used as independent variable. The data analysis was performed in SPSS.

2.2. Participants

14 learners of English in the age range 18–25 years participated in the experiment. All participants were native speakers of Norwegian and had varying degrees of hearing loss and English language proficiency. The only criterion for inclusion in the study was documented hearing loss and being a young adult English learner. At the outset, the study did not require any particular proficiency level of English. However, participants were asked to rate their English writing, reading and communication skills according to Common European Framework of Reference for Languages (CEFR) levels (Milton 2010). This factor was meant to be used in the analysis. All participants rated themselves similarly as conversational to fluent speakers of English, which corresponds to advanced L2 skills, and therefore this factor was not pursued any further in the analysis. Information about the project was sent out to the members of HLFU (Hørselshemmedes landsorganisasjon for ungdom (National Youth Hearing Impaired Organization)) and on social media (Facebook). Since interested participants lived in different parts of the country, it was decided to run an online study rather than a Lab experiment. After signing an online consent form, participants were asked to provide information about their hearing loss. Eight participants reported hearing levels of 26–40 dB: mild hearing loss, and six participants reported moderate hearing loss: 41–55 dB (see Supplementary 1). Data collection and data storage were consistent with GDPR laws and were approved by the Norwegian Personal Data Service (NSD) prior to study start.

3. Analyses and Results

For the analysis, the participants’ level of hearing was used as the independent variable, whereas comprehension and vocabulary accuracy were used as dependent variables. In each set of questions, the participants were able to get between 0 and 3 points. The results focus on performance in each condition and rate of accuracy. The data set was small, and a Shapiro-Wilk test revealed that the data did not follow a normal distribution. Table 1 summarizes the descriptive statistics for the group of participants. A Shapiro-Wilk test for normality was also applied to each variable. The results in Table 1 reveal that the mean score on both comprehension and vocabulary was the highest in the English subtitle condition, and the lowest in the no subtitle condition. This indicates that the group as a whole had higher accuracy scores on the English subtitle condition compared to the two other conditions.
As a next step we ran the Wilcoxon test in SPSS. The Wilcoxon test is a nonparametric test that does not assume normality in the dataset, and is used to compare two sets of scores that come from the same participant. As such, it was most appropriate, given the aim to investigate any change in scores in the same participant when they are exposed to more than one condition (Statistics Solutions 2020). In addition, and in order to control for the impact of level of hearing loss on performance, we split the participants into minimal hearing loss and moderate hearing loss groups. We used Bonferroni correction for multiple comparisons (p ≤ 0.025).
The results in Table 2 show a clearer picture primarily for comprehension, and for the moderate hearing loss group. For this group, significant differences in performance were observed between the English subtitles Condition and the other two conditions, with z-scores ranging between −2.333 (p = 0.020) and −2.232 (p = 0.026) in the comparison, and a trend to significance for the Norwegian subtitles and No subtitles condition (z-score = −2.121; p = 0.034). The comparison failed to reach significance for the minimal hearing loss group.
By providing subtitle support in the target language (English), the participants were able to comprehend more of the plot, and subtitles did enhance comprehension specifically. This is consistent with the idea that the participants struggle to perceive the auditory input in the source language, and therefore need additional evidence of the auditory form of this input via the subtitles. Concerning the results on vocabulary, subtitles in the target language (English) appear to cease providing and advantage over Norwegian subtitles for both groups. In that task, English subtitles are beneficial only in comparison to no subtitles at all for both groups, where a trend to significance was observed (z = −2.121; p = 0.034).

4. Discussion

In this pilot study we set out to replicate results from the benefits of subtitles in L2 learners with normal hearing and to investigate whether subtitles are also beneficial in L2 comprehension and initial vocabulary learning in a special population, namely hard-of-hearing students. We hypothesized that subtitles would enhance participants’ performance both on a comprehension task and on a vocabulary task, and that the participants would appreciate the addition of subtitles. Based on the results in Vulchanova et al. (2015), we also expected that L2 subtitles would be more beneficial than L1 subtitles, since they provide additional orthographic support for the perception of the source language for the hearing-impaired students. The current results indicate that subtitles clearly aid vocabulary and comprehension for learners of English with mild-to-moderate hearing loss.
Video material can be beneficial in second language instruction for a number of reasons. Extant research supports the idea that language learners can use the imagery associated with videos to assist information processing. Research on listening shows that the presence of images has a positive impact on comprehension (Jones and Plass 2002). Furthermore, Pujadas and Muñoz (2019) argue that the media of TV programs complies with Nation’s (2007) five conditions for suitable input: it is processed in large quantities; is familiar to the language learners; provides contextual cues (i.e., through image and dialogue); is comprehensible (Rodgers and Webb 2011; Rodgers and Webb, 2017); and is engaging (Webb 2010).
Previous research examining comprehension of audiovisual material input has consistently shown the positive effects of captioning over non-captioning for viewing comprehension (Gass et al. 2019; Montero-Perez et al. 2013, 2014; Rodgers and Webb 2017). At the same time, it can be expected that the appearance of subtitles might make the viewing session more difficult, due to increased processing and attentional load. Thus, there is more to focus on at the same time, as the student watches the images, while reading at the same time. However, Danan (2004) and d’Ydewalle and Gielen (1992) found that auditory and verbal textual information are processed in parallel, meaning that reading the captions would not hinder the processing of the material that is viewed. Subtitles seem to be more effective for content comprehension (Bianchi and Ciabattoni 2008; Markham et al. 2001), and the current study suggests that the textual information provided in the subtitles indeed aided the overall comprehension and understanding of the plot.
The comprehension data provide clear indication of a specific advantage for the provision of subtitles (in comparison to no subtitles), and specifically, subtitles in the target language (English). This advantage is most evident for the group with moderate hearing loss. This finding confirms the positive impact of subtitling on L2 comprehension and suggests that the provision of subtitles is more beneficial for more severe hearing loss. While Vulchanova et al. (2015) found that both L1 and L2 subtitles were equally useful for comprehension in their advanced L2 group, the current finding reflects an advantage for L2 (target language) subtitling support. A possible account is that, given the hearing loss, the provision of English subtitles supplements in a kind of redundant fashion what might be missing in a degraded auditory input, and, as such, improves the process of speech perception. This result is also consistent with the original findings in Mitterer and McQueen (2009).
There is general consensus that captions are beneficial for language learning and vocabulary acquisition, because they provide more exposure to the target language (Danan 2004; Vanderplank 2010). Interestingly, in the current design, despite a marginal advantage, subtitle provision was less beneficial for vocabulary learning than comprehension, where a trend to significance was observed only for English subtitles over no subtitles for both groups. Indeed, initially, subtitles facilitate primarily comprehension, while word learning needs more time, and more exposure.
Webb and Rodgers (2009) point out that reaching the target vocabulary size may be difficult for many learners and suggest that movies should not be used without providing some learning support. Pre-teaching vocabulary that the learners will encounter in the input seems to provide them with enhanced learning opportunities. Working with words and phrases that are relevant to a movie used in educational settings is something many educators are advised to do as a pre-watching activity (Harmer 2007). If the goal of an L2 learning lesson is to optimize the effectiveness of vocabulary learning through TV-programs, Pujadas and Muñoz (2020) argue that one possibility is to involve intentional or explicit learning. Indeed, research in the area of extensive reading suggests that learning rates can be increased by deliberately focusing attention on vocabulary (Elley 1989; Hulstijn 2013). Pujadas and Muñoz (2019) compared L2 vocabulary learning in two groups of learners, one group which was pre-taught the vocabulary to be encountered in TV-clips, and were watching the series with captions, and another group who had not received vocabulary instruction and watched the series with subtitles. They found that participants did learn L2 vocabulary from extensive exposure to audiovisual input. They also found that the group who were pre-taught words before the viewing session performed better than the group who did not receive such instruction. However, Rodgers (2013) argues that vocabulary acquisition happens incidentally through audiovisual input, and that learning occurs as a by-product of the activity. Indeed, a growing number of studies in this area consistently suggest that incidental vocabulary acquisition does occur through viewing short clips, full movies, and TV-series (Pujadas and Muñoz 2020).
In addition, Neuman and Koskinen (1992) argue that a minimum competency threshold might be necessary in order to benefit from captioning in L2 learning. The participants in this study reported that they all were on the same level, ranging from conversational to fluent. Additionally, they were over 18 years old, which means that they would have been exposed to English at least from when they started mandatory English education in first grade at elementary school. This may explain the minimal variation in the results from the vocabulary questions. Furthermore, the participants were not asked in advance to focus on any particular aspect of the video, they were only told that they would answer questions afterwards. It might have been easier for them to comprehend the general plot, than to focus on specific words per clip. Pujadas and Muñoz (2020) suggest that cognitive overload obtains when the learners’ cognitive processing exceeds the available cognitive capacity, and according to VanPatten (2002) “learners can do only so much in their working memory before attentional resources are depleted and working memory is forced to dump information to make room for more (incoming) information”.
It deserves mention that the vocabulary task yielded a different result in comparison to the comprehension task condition concerning the language of the subtitles. For this task, the comparison between English (the target language) and Norwegian (L1) no longer showed an advantage for the L2 over the L1. This result may find a plausible account in theories of the bilingual lexicon, and lexical access, suggesting a partially overlapping language network with stable connections between the two languages of the bilingual (Costa et al. 2000), whereby word activation works in both directions (Havas and Vulchanova 2018).

5. Conclusions

The results of the current study contribute new knowledge to the under-researched area of second language learning in the context of milder hearing loss and how instruction can be adapted by the use of subtitles. We confirm previous findings regarding the potential of TV-programs for language comprehension as a rich source of comprehensible input. In addition, this study demonstrates that students with a minimal-to-moderate hearing loss can benefit from the provision of captions as a visual cue when watching films in the English as a second language classroom. Although hearing can be improved with technical aids, such as hearing aids, they are not always optimal for L2 learning when it comes to speech perception. Audiovisual material has proven to be an essential tool for language learning (Sherman 2003). Students learn more deeply from the combination of pictures, words, and sound, because these inputs activate both the auditory and the visual channel (Mayer 1997, 2014; Paivio 1986). School instruction plays an important role in how first and second languages are taught, and educators need to implement strategies of inclusion in the classroom that benefit every student, including those with hearing loss (Dalton 2013).
Subtitles aided comprehension of the plot, tested immediately after the screening. Furthermore, subtitles were found to enhance the performance of the participants with moderate hearing loss. The findings of this study can be transferred to classrooms and may supplement or replace other methods. Lip reading, for example, requires intense concentration and is tiring over long periods. It is also a technique that is widely used amongst hard-of-hearing learners. This technique has earlier been regarded as crucial for hard-of-hearing learners (Goldberg and Richburg 2004), but may result in cognitive overload and will potentially lead to learner anxiety (Pujadas and Muñoz 2020; Sweller 2011; VanPatten 2002). Instead, assistive technologies, such as captioned films or documentaries can be easily used in the ESL classroom to assist students with mild-to-moderate hearing loss to lessen the load of extra concentration. In addition, to get the most out of the viewing, it can be beneficial to pre-teach relevant words and phrases that the learners will be exposed to during the viewing, thus enhancing vocabulary learning.

6. Limitations of the Study

The current study is not without its limitations. No control group was included to compare results to typical controls. However, since the results in Vulchanova et al. (2015) come from a representative sample of a similar level of proficiency, we have chosen to use them as a base level for comparison. Secondly, the target group of participants was small, resulting in a small sample size. This was primarily caused by the difficulty in recruiting enough participants with documented hearing loss, as well as the low prevalence of this group in the young population in Norway (0.25%). Future research should aim to confirm those preliminary results based on a bigger sample. In addition, the results did not follow a normal distribution, which necessitated the use of non-parametric tests. However, individual variation is often found in language studies, and even more so in performance by individuals with impairment. In future research, this issue may be resolved by a bigger sample or by the recruitment of bigger and more homogeneous groups of participants to be studied separately.

Supplementary Materials

The following are available online at https://www.mdpi.com/2226-471X/6/1/17/s1, Supplementary 1: Background information, Supplementary 2: Questions.

Author Contributions

Conceptualization, M.V. and I.K.L.; methodology, M.V.; validation, M.V., I.K.L.; formal analysis, I.K.L.; investigation, I.K.L.; data curation, M.V.; writing—original draft preparation, M.V., I.K.L.; writing—review and editing, M.V.; supervision, M.V. Both authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and reviewed and approved by Norsk Senter for forskningsdata [Norwegian Centre for Research Data] [Project identification code: 304406].

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to [participants belonging in a vulnerable group].

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Andrews, Jean, amela Shaw, and Gabriel Lomas. 2011. Deaf and Hard of Hearing Students. In Handbook of Special Education. Edited by James M. Kauffman and Daniel P. Hallahan. New York: Routledge, pp. 233–46. [Google Scholar]
  2. Baltova, Iva. 1999. Multisensory language teaching in a multidimensional curriculum: The use of authentic bimodal video in core French. Canadian Modern Language Review 56: 31–48. [Google Scholar] [CrossRef]
  3. Bess, Fred H., and Freeman McConnell. 1981. Audiology, Education, and the Hearing Impaired Child. St. Louis: Mosby Incorporated. [Google Scholar]
  4. Bianchi, Francesca, and Tiziana Ciabattoni. 2008. Captions and Subtitles in EFL Learning: An Investigative Study in a Comprehensive Computer Environment. Trieste: EUT Edizioni Università di Trieste. [Google Scholar]
  5. Bird, Stephen A., and John N. Williams. 2002. The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics 23: 509–33. [Google Scholar] [CrossRef] [Green Version]
  6. Bisson, Marie-Josée, Walter J. B. Van Heuven, Kathy Conklin, and Richard J. Tunney. 2014. Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholinguistics 35: 399–418. [Google Scholar] [CrossRef] [Green Version]
  7. Cavender, Anna C., Jeffrey P. Bigham, and Richard E. Ladner. 2009. Class In Focus: Enabling improved visual attention strategies for deaf and hard of hearing students. Paper presented at 11th International ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, PA, USA, October 26–28; pp. 67–74. [Google Scholar]
  8. Charness, Gary, Uri Gneezy, and Michael A. Kuhn. 2012. Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization 81: 1–8. [Google Scholar]
  9. Costa, Albert, Alfonso Caramazza, and Nuria Sebastian-Galles. 2000. The cognate facilitation effect: Implications for models of lexical access. Journal of Experimental Psychology: Learning, Memory, and Cognition 26: 1283–96. [Google Scholar] [CrossRef]
  10. Crandell, Carl C., and Joseph J. Smaldino. 2000. Classroom acoustics for children with normal hearing and with hearing impairment. Language, Speech, and Hearing Services in Schools 31: 362–70. [Google Scholar] [CrossRef] [Green Version]
  11. Csizér, Kata, and Edit H. Kontra. 2020. Foreign Language Learning Characteristics of Deaf and Severely Hard-of-Hearing Students. The Modern Language Journal 104: 233–49. [Google Scholar] [CrossRef]
  12. d’Ydewalle, Géry, and Ingrid Gielen. 1992. Attention allocation with overlapping sound, image, and text. In Eye Movements and Visual Cognition. New York: Springer, pp. 415–27. [Google Scholar]
  13. d’Ydewalle, Géry, and Marijke Van de Poel. 1999. Incidental foreign-language acquisition by children watching subtitled television programs. Journal of Psycholinguistic Research 28: 227–44. [Google Scholar] [CrossRef]
  14. Dahl, Anne, and Mila D. Vulchanova. 2014. Naturalistic acquisition in an early language classroom. Frontiers in Psychology 5: 329. [Google Scholar] [CrossRef]
  15. Dalton, C. J. 2013. Lessons for Inclusion: Classroom experiences of students with mild and moderate hearing loss. Canadian Journal of Education 36: 125–52. [Google Scholar]
  16. Danan, Martine. 2004. Captioning and subtitling: Undervalued language learning strategies. Meta: Journal des Traducteurs/Meta: Translators’ Journal 49: 67–77. [Google Scholar] [CrossRef] [Green Version]
  17. Delage, Hélène, and Laurice Tuller. 2007. Language development and mild-to-moderate hearing loss: Does language normalize with age? Journal of Speech and Language and Hearing Research 50: 1–14. [Google Scholar] [CrossRef]
  18. Elley, Warwick B. 1989. Vocabulary acquisition from listening to stories. Reading Research Quarterly 24: 174–87. [Google Scholar] [CrossRef]
  19. Elliott, Lois L. 1979. Performance of children aged 9–17 years on a test of speech intelligibility in noise using sentence material with controlled word predictability. Journal of the Acoustical Society of America 66: 651–53. [Google Scholar] [CrossRef]
  20. Fischer, Caroline, and Judith Lieu. 2014. Unilateral hearing loss is associated with a negative effect on language scores in adolescents. International Journal Of Pediatric Otorhinolaryngology 78: 1611–17. [Google Scholar] [CrossRef] [Green Version]
  21. Gass, Susan, Paula Winke, Daniel R. Isbell, and Jieun Ahn. 2019. How captions help people learn languages: A working-memory, eye-tracking study. Language Learning & Technology 23: 84–104. [Google Scholar]
  22. Goldberg, Lynette R., and Cynthia McCormick Richburg. 2004. Minimal hearing impairment: Major myths with more than minimal implications. Communication Disorders Quarterly 25: 152–60. [Google Scholar] [CrossRef]
  23. Halliday, Lorna F., Outi Tuomainen, and Stuart Rosen. 2017. Language development and impairment in children with mild to moderate sensorineural hearing loss. Journal of Speech, Language, and Hearing Research 60: 1551–67. [Google Scholar] [CrossRef] [Green Version]
  24. Harmer, Jeremy. 2007. The Practice of English Language Teaching. Harlow: Pearson Longman. [Google Scholar]
  25. Havas, Viktoria, and Mila Vulchanova. 2018. The Nature and Effects of Norwegian Diglossia. In Å Skrive Nynorsk og Bokmål. Nye Tverrfaglege Perspektiv. Oslo: Samlaget. [Google Scholar]
  26. Howerton-Fox, Amanda, and Jodi L. Falk. 2019. Deaf Children as ‘English Learners’: The Psycholinguistic Turn in Deaf Education. Education Sciences 9: 133. [Google Scholar] [CrossRef] [Green Version]
  27. Hulstijn, Jan H. 2013. Incidental learning in second language acquisition. In The Encyclopedia of Applied Linguistics. Edited by Carol A. Chapelle. Chichester: Wiley-Blackwell, vol. 5, pp. 2632–40. [Google Scholar]
  28. Jones, Linda C., and Jan L. Plass. 2002. Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. The Modern Language Journal 86: 546–61. [Google Scholar] [CrossRef]
  29. Kishon-Rabin, Liat, Jacob Kuint, Minka Hildesheimer, and Daphne Ari-Even Roth. 2015. Delay in auditory behaviour and preverbal vocalization in infants with unilateral hearing loss. Developmental Medicine & Child Neurology 57: 1129–36. [Google Scholar]
  30. Lewis, Dawna E., Daniel L. Valente, and Jody L. Spalding. 2015. Effect of minimal/mild hearing loss on children’s speech understanding in a simulated classroom. Ear and Hearing 36: 136–44. [Google Scholar] [CrossRef] [Green Version]
  31. Markham, Paul. 1999. Captioned videotapes and second language listening word recognition. Foreign Language Annals 32: 321–28. [Google Scholar] [CrossRef]
  32. Markham, Paul L., Lizette A. Peter, and Teresa J. McCarthy. 2001. The effects of native language vs. target language captions on foreign language students’ DVD video comprehension. Foreign Language Annals 34: 439–45. [Google Scholar] [CrossRef]
  33. Mayer, Richard E. 1997. Multimedia learning: Are we asking the right questions? Educational Psychologist 32: 1–19. [Google Scholar] [CrossRef] [Green Version]
  34. Mayer, Richard E. 2014. Cognitive theory of multimedia learning. In Cambridge Handbooks in Psychology. The Cambridge Handbook of Multimedia Learning. Edited by R. E. Mayer. Cambridge, UK: Cambridge University Press, pp. 43–71. [Google Scholar] [CrossRef] [Green Version]
  35. McKay, Sarah, Judith S. Gravel, and Anne Marie Tharpe. 2008. Amplification considerations for children with minimal or mild bilateral hearing loss and unilateral hearing loss. Trends in Amplification 12: 43–54. [Google Scholar] [CrossRef] [Green Version]
  36. Mehra, Saral, Roland D. Eavey, and Donald G. Keamy Jr. 2009. The epidemiology of hearing impairment in the United States: Newborns, children, and adolescents. Otolaryngology-Head and Neck Surgery 140: 461–72. [Google Scholar] [CrossRef]
  37. Milton, James. 2010. The development of vocabulary breadth across the CEFR levels. In Communicative Proficiency and Linguistic Development: Intersections between SLA and Language Testing Research. EUROSLA Monographs series; Morrisville: Lulu Press, pp. 211–32. [Google Scholar]
  38. Mitterer, Holger, and James M. McQueen. 2009. Foreign subtitles help but native-language subtitles harm foreign speech perception. PLoS ONE 4: e7785. [Google Scholar] [CrossRef]
  39. Moeller, Mary Pat, and J. Bruce Tomblin. 2015. An Introduction to the Outcomes of Children with Hearing Loss Study. Ear and Hearing 36: 4S–13S. [Google Scholar] [CrossRef] [Green Version]
  40. Montero-Perez, Maribel, Elke Peters, Geraldine Clarebout, and Piet Desmet. 2014. Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning & Technology 18: 118–41. [Google Scholar]
  41. Montero-Perez, Maribel, Wim Van Den Noortgate, and Piet Desmet. 2013. Captioned video for L2 listening andvocabulary learning: A meta-analysis. System 41: 720–39. [Google Scholar] [CrossRef]
  42. Most, Tova. 2004. The effects of degree and type of hearing loss on children’s performance in class. Deafness & Education International 6: 154–66. [Google Scholar]
  43. Nation, Paul. 2007. The four strands. International Journal of Innovation in Language Learning and Teaching 1: 2–13. [Google Scholar] [CrossRef]
  44. Neuman, Susan B., and Patricia Koskinen. 1992. Captioned television as comprehensible input: Effects of incidental word learning from context for language minority students. Reading Research Quarterly 27: 95–106. [Google Scholar] [CrossRef]
  45. Niskar, Amanda Sue, Stephanie M. Kieszak, Alice E. Holmes, Emilio Esteban, Carol Rubin, and Debra J. Brody. 2001. Estimated prevalence of noise-induced hearing threshold shifts among children 6 to 19 years of age: The Third National Health and Nutrition Examination Survey, 1988–1994 (US). Pediatrics 108: 40–43. [Google Scholar] [CrossRef] [PubMed]
  46. Northern, Jerry L., and Marion P. Downs. 2002. Hearing in Children. Philadelphia: Lippincott Williams & Wilkins. [Google Scholar]
  47. Opplæringslova. 1998. Lov om grunnskolen og den videregåande opplæringa (LOV-1998-07-17-61). Available online: https://lovdata.no/lov/1998-07-17-61 (accessed on 2 December 2020).
  48. Paivio, Allan. 1986. Mental Representations: A Dual-Coding Approach. New York: Oxford University Press. [Google Scholar]
  49. Pujadas, Geòrgia, and Carmen Muñoz. 2019. Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal 47: 479–96. [Google Scholar] [CrossRef]
  50. Pujadas, Geòrgia, and Carmen Muñoz. 2020. Examining adolescent EFL learners’ TV viewing comprehension through captions and subtitles. Studies in Second Language Acquisition 42: 1–25. [Google Scholar]
  51. Reynolds, Cecil R., and Elaine Fletcher-Janzen, eds. 2007. Encyclopedia of Special Education: A Reference for the Education of Children, Adolescents, and Adults with Disabilities and Other Exceptional Individuals, 3rd ed. Hoboken: Wiley. [Google Scholar]
  52. Robier, Alain. 2001. Les Surdités de Perception [Sensorineural Hearing Loss]. Paris: Masson. [Google Scholar]
  53. Rodgers, Michael P. 2013. English Language Learning through Viewing Television: An Investigation of Comprehension, Incidental Vocabulary Acquisition, Lexical Coverage, Attitudes, and Captions. Ph.D. thesis, Victoria University of Wellington, Wellington, New Zealand. [Google Scholar]
  54. Rodgers, Michael P. H., and Stuart Webb. 2011. Narrow viewing: The vocabulary in related television programs. TESOL Quarterly 45: 689–717. [Google Scholar] [CrossRef]
  55. Rodgers, Michael P. H., and Stuart Webb. 2017. The effects of captions on EFL learners’ comprehension of English-language television programs. CALICO Journal 34: 20–38. [Google Scholar] [CrossRef]
  56. Sherman, Jane. 2003. Using Authentic Video in the Language Classroom. München: Ernst Klett Sprachen. [Google Scholar]
  57. Soli, Sigfrid D., and Jean A. Sullivan. 1997. Factors affecting children’s speech communication in classrooms. Journal of the Acoustical Society of America 101: 3070. [Google Scholar] [CrossRef]
  58. Statistics Solutions. 2020. The Wilcoxon Sign Rank Test in SPSS. [Google Scholar]
  59. Sweller, John. 2011. Cognitive load theory. In Psychology of Learning and Motivation. Cambridge, MA: Academic Press, vol. 55, pp. 37–76. [Google Scholar]
  60. Tharpe, Anne Marie, and Fred H. Bess. 1991. Identification and management of children with minimal hearing loss. International Journal of Pediatric Otorhinolaryngology 21: 41–50. [Google Scholar] [CrossRef]
  61. Tomblin, J. Bruce, Elizabeth A. Walker, Ryan W. McCreery, Richard M. Arenas, Melody Harrison, and Mary Pat Moeller. 2015. Outcomes of Children with Hearing Loss: Data Collection and Methods. Ear and Hearing 36 S1: 14S–23S. [Google Scholar] [CrossRef] [Green Version]
  62. Tuller, Laurice, and Hélène Delage. 2014. Mild-to-moderate hearing loss and language impairment: How are they linked? Lingua 139: 80–101. [Google Scholar] [CrossRef]
  63. Valente, Daniel L., Hallie M. Plevinsky, John M. Franco, Elizabeth C. Heinrichs-Graham, and Dawna E. Lewis. 2012. Experimental investigation of the effects of the acoustical conditions in a simulated classroom on speech recognition and learning in children. The Journal of the Acoustical Society of America 131: 232–46. [Google Scholar] [CrossRef] [Green Version]
  64. Vandergrift, Larry. 2011. Second language learning: Presage, process, product and pedagogy. In Handbook of Research in Second Language Teaching and Learning. Edited by Eli Hinkel. London: Routledge, vol. 2, pp. 455–71. [Google Scholar]
  65. Vanderplank, Robert. 1988. The value of teletext sub-titles in language learning. ELT J 42: 272–81. [Google Scholar] [CrossRef]
  66. Vanderplank, Robert. 2010. Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching 43: 1–37. [Google Scholar] [CrossRef]
  67. VanPatten, Bill. 2002. Processing instruction: An update. Language Learning 52: 755–803. [Google Scholar] [CrossRef]
  68. Vulchanova, Mila, Lisa M. G. Aurstad, Ingrid E. N. Kvitnes, and Hendrik Eshuis. 2015. As naturalistic as it gets: Subtitles in the English classroom in Norway. Frontiers in Psychology 5: 1510. [Google Scholar] [CrossRef] [Green Version]
  69. Webb, Stuart. 2010. Pre-learning low-frequency vocabulary in second language television programmes. Language Teaching Research 14: 501–15. [Google Scholar] [CrossRef]
  70. Webb, Stuart, and Michael P. H. Rodgers. 2009. Vocabulary demands of television programs. Language Learning 59: 335–66. [Google Scholar] [CrossRef]
Table
Table 1. Descriptive statistics from analysis based on score.
Table 1. Descriptive statistics from analysis based on score.
ConditionMeanSDp-Value
Norwegian subtitles comprehension2.070.73p = 0.009
Norwegian subtitles vocabulary 2.420.646p = 0.001
English subtitles comprehension2.850.363p = 0.000
English subtitles vocabulary2.640.497p = 0.000
No subtitles comprehension1.51.019p = 0.052
No subtitles vocabulary 1.70.611p = 0.002
Note. SD = Standard Deviation.
Table
Table 2. Wilcoxon test with Bonferroni correction (p ≤ 0.025).
Table 2. Wilcoxon test with Bonferroni correction (p ≤ 0.025).
Variable TestedLevel of HearingConditions ComparedZp = Value
ComprehensionMild English subtitlesand Norwegian subtitles−2.0000.046
English subtitlesand No subtitles−1.7300.084
Norwegian subtitlesand No subtitles −0.5520.580
ModerateEnglish subtitlesand Norwegian subtitles−2.3330.020
English subtitlesand No subtitles−2.2320.026
Norwegian subtitlesand No subtitles−2.1210.034
VocabularyMildEnglish subtitlesand Norwegian subtitles−1.0000.310
English subtitlesand No subtitles−2.1210.034
Norwegian subtitles and No subtitles−2.0000.046
ModerateEnglish subtitlesand Norwegian subtitles−0.5770.564
English subtitlesand No subtitles−2.1210.034
Norwegian subtitlesand No subtitles−1.8900.059
Note: p values in bold indicate significance after Bonferroni correction.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Vulchanova, M.; Kjølstad Lervåg, I. Role of Subtitles in L2 Acquisition and Comprehension: A Pilot Study of Hearing-Impaired Students. Languages 2021, 6, 17. https://doi.org/10.3390/languages6010017

AMA Style

Vulchanova M, Kjølstad Lervåg I. Role of Subtitles in L2 Acquisition and Comprehension: A Pilot Study of Hearing-Impaired Students. Languages. 2021; 6(1):17. https://doi.org/10.3390/languages6010017

Chicago/Turabian Style

Vulchanova, Mila, and Ingvild Kjølstad Lervåg. 2021. "Role of Subtitles in L2 Acquisition and Comprehension: A Pilot Study of Hearing-Impaired Students" Languages 6, no. 1: 17. https://doi.org/10.3390/languages6010017

Article Metrics

Back to TopTop