Hostname: page-component-8448b6f56d-xtgtn Total loading time: 0 Render date: 2024-04-23T06:09:46.542Z Has data issue: false hasContentIssue false
  • English
  • Français

PROFICIENCY ASSESSMENT STANDARDS IN SECOND LANGUAGE ACQUISITION RESEARCH

“Clozing” the Gap

Published online by Cambridge University Press:  20 July 2011

Annie Tremblay
Annie Tremblay*
Affiliation:
University of Illinois at Urbana-Champaign
*
*Address correspondence to: Annie Tremblay, Department of French, University of Illinois at Urbana-Champaign, 2090 Foreign Language Building, 707 S. Mathews Avenue, Urbana, IL 61801; e-mail: atrembla@illinois.edu.
Get access Rights & Permissions [Opens in a new window]

Abstract

The present study aims to sensitize SLA researchers to the importance of documenting and controlling for their participants’ proficiency in the target language, with the goal of establishing more robust proficiency assessment standards in experimental research. First, this article presents a survey of recent (2000–2008) foreign and second-language (L2) acquisition studies that show that such standards have yet to be met. Second, it demonstrates the validity, reliability, and practicality of a cloze (i.e., fill-in-the-blank) test designed to discriminate among L2 learners of French at different proficiency levels. Subject and item analyses are performed on the cloze test scores of 169 L2 learners of French from various language backgrounds. The relationship between these scores and the learners’ language background is examined. Cutoff points between proficiency levels are identified in the data. The test then is shared with scholars so that those working with a similar population of L2 learners of French can also use it.

Type
Research Articles
Information
Studies in Second Language Acquisition , Volume 33 , Issue 3 , September 2011 , pp. 339 - 372
Copyright
Copyright © Cambridge University Press 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Abraham, R. G., & Chapelle, C. A. (1992). The meaning of cloze test scores: An item difficulty perspective. Modern Language Journal, 76, 468–479.Google Scholar
Alderson, J. C. (1979). The cloze procedure and proficiency in English as a foreign language. TESOL Quarterly, 13, 219–227.Google Scholar
ACTFL (American Council on the Teaching of Foreign Languages). (1999). ACTFL proficiency guidelines C speaking. Retrieved October 15, 2009, fromhttp://www.actfl.org/i4a/pages/index.cfm?pageid=3325.Google Scholar
ACTFL (American Council on the Teaching of Foreign Languages). (2001). Preliminary proficiency guidelines: Writing. Retrieved October 15, 2009 fromhttp://www.actfl.org/i4a/pages/index.cfm?pageid=3326Google Scholar
Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL Quarterly, 16, 61–70.Google Scholar
Bachman, L. F. (1985). Performance on cloze tests with fixed-ratio and rational deletions. TESOL Quarterly, 16, 61–70.Google Scholar
Bachman, L. F. (1990). Constructing measures and measuring constructs. In Harley, B., Allen, P., Cummins, J., & Swain, M. (Eds.), The development of second language proficiency (pp. 26–49). New York: Cambridge University Press.Google Scholar
Birdsong, D. (2005). Interpreting age effects in 2nd language acquisition. In Kroll, J. & de Groot, A. (Eds.), Handbook of bilingualism (pp. 109–127). Oxford: Oxford University Press.Google Scholar
Brown, J. D. (1980). Relative merits of four methods for scoring cloze tests. Modern Language Journal, 64, 311–317.Google Scholar
Brown, J. D. (1983). A closer look at cloze: Validity and reliability. In Oller, J. W. Jr. (Ed.), Issues in language testing research (pp. 237–250). Rowley, MA: Newbury House.Google Scholar
Brown, J. D. (1984). A cloze is a cloze is a cloze? In Handscombe, J., Orem, R. A., & Taylor, B. P. (Eds.), On TESOL ’83 (pp. 109–119). Washington, DC: TESOL.Google Scholar
Brown, J. D. (1988). Tailored cloze: Improved with classical item analysis techniques. Language Testing, 5, 19–31.Google Scholar
Brown, J. D. (1996). Testing in language programs. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Brown, J. D. (1998). Cloze tests and optimum test length. Shiken: JALT Testing and Evaluation SIG Newsletter, 2, 18–22.Google Scholar
Brown, J. D. (2002). Do cloze tests work? Or, is it just an illusion? Second Language Studies, 21, 79–125.Google Scholar
Brown, J. D. (2003). Norm-referenced item analysis (item facilitation and item discrimination). Shiken: JALT Testing and Evaluation SIG Newsletter, 7, 16–19.Google Scholar
Calinski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics, 3, 1–27.Google Scholar
Chapelle, C. A. (1994). Are C-tests valid measures for L2 vocabulary research? Second Language Research, 10, 157–187.Google Scholar
Chapelle, C. A. (1999). Validity in language assessment. Annual Review of Applied Linguistics, 19, 254–272.Google Scholar
Chapelle, C. A., & Abraham, R. G. (1990). Cloze method: What differences does it make? Language Testing, 7, 121–146.Google Scholar
Chaudron, C. (2003). Data collection in SLA research. In Doughty, C. J. & Long, M. H. (Eds.), Handbook of second language acquisition (pp. 762–828). Oxford: Blackwell.Google Scholar
Chihara, T., Sakurai, T., & Oller, J. W. Jr. (1989). Background and culture as factors in EFL reading comprehension. In Oller, J. W. Jr., & Jonz, J. (Eds.), Cloze and coherence (pp. 135–147). London: Associated University Press.Google Scholar
Colantoni, L., & Steele, J. (2007). Acquiring /ʁ/in context. Studies in Second Language Acquisition, 29, 381–406.Google Scholar
Darnell, D. K. (1970). Clozentropy. A procedure for testing English language proficiency of foreign students. Speech Monographs, 37, 36–46.Google Scholar
Delgado, P., Guerrero, G., Goggin, J. P., & Ellis, B. B. (1999). Self-assessment of linguistic skills by bilingual Hispanics. Hispanic Journal of Behavioral Sciences, 21, 31–46.Google Scholar
Douglas, M. O. (1994). Japanese cloze tests: Toward their construction. Japanese Language Education around the Globe, 4, 117–131.Google Scholar
Flege, J. E., MacKay, I. A., & Piske, T. (2002). Assessing bilingual dominance. Applied Psycholinguistics, 23, 567–598.Google Scholar
Flege, J. E., Yeni-Komishian, G. H., & Liu, S. (1999). Age constraints on second language acquisition. Journal of Memory and Language, 41, 78–104.Google Scholar
Fotos, S. (1991). The cloze test as an integrative measure of EFL proficiency: A substitute for essays on college entrance examinations? Language Learning, 41, 313–336.Google Scholar
Freed, B. F., Dewey, D. P., Segalowitz, N., & Halter, R. (2004). The language contact profile. Studies in Second Language Acquisition, 26, 349–356.Google Scholar
Genesee, F. (1985). Second language learning through immersion: A review of U.S. programs. Review of Educational Research, 55, 541–561.Google Scholar
Hanania, E., & Shikhani, M. (1986). Interrelationships among three tests of language proficiency: Standardized ESL, cloze and writing. TESOL Quarterly, 20, 97–110.Google Scholar
Harley, B., Cummins, J., Swain, M., & Allen, P. (1990). The nature of proficiency. In Harley, B., Allen, P., Cummins, J., & Swain, M. (Eds.), The development of second language proficiency (pp. 7–25). New York: Cambridge University Press.Google Scholar
Heilenman, L. (1983). The use of a cloze procedure in foreign language placement. Modern Language Journal, 67, 121–126.Google Scholar
Hinofotis, F. B. (1980). Cloze as an alternative method of ESL placement and proficiency testing. In Oller, J. W. Jr. & Perkins, K. (Eds.), Research in language testing (pp. 121–128). Rowley, MA: Newbury House.Google Scholar
Ionin, T., Zubizarreta, M. L., & Maldonado, S. B. (2008). Sources of linguistic knowledge in the second language acquisition of English articles. Lingua, 118, 554–576.Google Scholar
Jafarpur, A. (1995). Is C-testing superior to cloze? Language Testing, 12, 194–215.Google Scholar
Jia, G., Aaronson, D., & Wu, Y. (2002). Long-term language attainment of bilingual immigrants: Predictive variables and language group differences. Applied Psycholinguistics, 23, 599–621.Google Scholar
Johnson, J. S., & Newport, E. L. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 21, 60–99.Google Scholar
Jonz, J. (1976). Improving on the basic egg: The multiple-choice cloze. Language Learning, 26, 255–265.Google Scholar
Jonz, J. (1990). Another turn in the conversation: What does cloze measure? TESOL Quarterly, 24, 61–83.Google Scholar
Jonz, J. (1994). The effects of textual cohesion and prior knowledge on native and nonnative cloze test scores. In Oller, J. W. Jr. & Jonz, J. (Eds.), Cloze and coherence (pp. 269–285). London: Associated University Press.Google Scholar
Klein-Braley, C. (1983). A cloze is a cloze is a question. In Oller, J. W. Jr. (Ed.), Issues in language testing research (pp. 218–228). Rowley, MA: Newbury House.Google Scholar
Kobayashi, M. (2002). Cloze tests revisited: Exploring item characteristics with special attention to scoring methods. Modern Language Journal, 86, 571–586.Google Scholar
MacKay, I., & Flege, J. E. (2004). Effects of the age of second language learning on the duration of first and second language sentences. Applied Psycholinguistics, 25, 373–396.Google Scholar
Marian, V., Blumenfeld, H. K., & Kaushanskaya, M. (2007). The Language Experience and Proficiency Questionnaire (LEAP-Q): Assessing language profiles in bilinguals and multilinguals. Journal of Speech, Language, and Hearing Research, 50, 940–967.Google Scholar
Markham, P. L. (1985). The rational deletion cloze and global comprehension in German. Language Learning, 35, 423–430.Google Scholar
Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement (3rd ed.) (pp. 13–103). New York: MacMillan.Google Scholar
Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2, 255–273.Google Scholar
Norris, J., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50, 417–528.Google Scholar
Oller, J. W. Jr. (1972). Scoring methods and difficulty levels for cloze tests of proficiency in English as a second language. Modern Language Journal, 56, 151–157.Google Scholar
Oller, J. W. Jr. (1973). Cloze tests of second language proficiency and what they measure. Language Learning, 23, 105–118.Google Scholar
Raatz, U., & Klein-Bradley, C. (1981). The C-test: A modification of the cloze procedure. In Culhane, T., Klein-Bradley, C., & Stevenson, D. K. (Eds.), Practice and problems in language testing, University of Essex Occasional Papers 26 (pp. 113–148). Colchester, UK: University of Essex.Google Scholar
Sasaki, M. (2000). Effects of cultural schemata on students’ test-taking processes for cloze tests: A multiple data source approach. Language Testing, 17, 85–114.Google Scholar
Shanahan, T., Kamil, M. L., & Tobin, A. W. (1982). Cloze as a measure of intersentential comprehension. Reading Research Quarterly, 17, 229–255.Google Scholar
Shohamy, E. (1997). Testing methods, testing consequences: Are they ethical? Are they fair? Language Testing, 14, 340–349.Google Scholar
Taylor, W. J. (1957). “Cloze” readability scores as indices of individual differences in comprehension and aptitude. Journal of Applied Psychology, 41, 19–26.Google Scholar
Tremblay, A. (2007). Bridging the gap between theoretical linguistics and psycholinguistics in L2 phonology: Acquisition and processing of word stress by French Canadian L2 learners of English. Unpublished doctoral dissertation, University of Hawai‘i.Google Scholar
Tremblay, A. (2008). Is second language lexical access prosodically constrained? Processing of word stress by French Canadian second language learners of English. Applied Psycholinguistics, 29, 553–584.Google Scholar
Tremblay, A. (2009). Phonetic variability and the variable perception of L2 word stress by French Canadian listeners. International Journal of Bilingualism, 13, 35–62.Google Scholar
Tremblay, A., & Garrison, M. D. (2010). Cloze tests: A tool for proficiency assessment in research on L2 French. In Prior, M. T., Watanabe, Y., & Lee, S.-K. (Eds.), Selected proceedings of the Second Language Research Forum 2008 (pp. 73–88). Somerville, MA: Cascadilla Press.Google Scholar
Yamashita, S. O. (1994). Is the reading comprehension performance of learners of Japanese as a second language the same as that of Japanese children? An analysis using a cloze test. Japanese Language Education around the Globe, 4, 133–146.Google Scholar