초록

This study aims to set assessment criteria for speaking fluency of Korean learners and also aims to evaluate reliability of the assessments performed in this study. The assessment comprises three parts: holistic assessment, analytic assessment, and individual assessment of raters, in which each rater uses her own criteria. The analytic assessment includes five factors, such as vocabulary, grammar, pronunciation, speech rate, and speech prosody. Sixty-one intermediate-level Korean learners participated the study and they were asked to summarize the 10-minute maim comedy, ’Mr. Bean’. The recorded summaries were evaluated by five Korean lectures who had over three years teaching experience but had no training on speaking assessment. The evaluation data were analyzed to examine reliability using many-faceted Rash mode. Results revealed that each rater obtained high reliability for one’s assessment, but little reliability was shown in assessments among raters. In particular, the analytic assessment showed the lowest reliability. This study suggests that it is essential to establish criteria for fluency assessment in L2 Korean speaking and that training programs for raters are required.

키워드

Speaking assessment, fluency, scoring reliability, holistic assessment, analytic assessment, many-faceted Rash model

참고문헌(77)open

  1. [단행본] Brown, G. / 1983 / Discourse Analysis, Cambridge Textbooks in Linguistics Series / Cambridge University Press

  2. [단행본] Brumfit, C. / 1984 / Communicative Methodology in Language Teaching: The role of Fluency and Accuracy / Cambridge University Press

  3. [학술지] Chambers, F. / 1997 / What do We Mean by Fluency? / System 25 (4) : 535 ~ 544

  4. [학술지] 최은규 / 2006 / 유형별로 본 한국어 능력 평가의 실제와 과제 / 한국어교육 17 (2) : 289 ~ 319

  5. [학술지] Cutler, A. / 1997 / Prosody in the Comprehension of Spoken Language: A literature Review / Language and Speech 40 (2) : 141 ~ 201

  6. [학술지] Nel de Jong / 2011 / Fluency Training in the ESL Classroom: An Experimental Study of Fluency Development and Proceduralization / Language Learning 61 (2) : 533 ~ 568

  7. [단행본] Ellis, R. / 2003 / Task-based Language Learning Fluency Training in the ESL Classroom: An Experimental Study of Fluency Development and Proceduralization and Teaching / Oxford University Press

  8. [학술지] Ellis, R. / 2009 / The Differential Effects of Three Types of Task Planning on the Fluency, Complexity, and Accuracy in L2 Oral Production / Applied Linguistics 30 (4) : 474 ~ 509

  9. [단행본] Ellis, R. / 2005 / Analysing Learner Language / Oxford University Press

  10. [단행본] Faerch, C. / 1984 / Learner Language and Language Learning / Mutilingual Matters

  11. [단행본] Fillmore, C. J. / 1979 / Individual Differences in Language Ability and Language Behavior / American Press

  12. [학술지] Foster, P. / 1999 / The Influence of Source of Planning and Focus of Planning on Task-based Performance / Language Teaching Research 3 : 215 ~ 247

  13. [단행본] Freed, B. F. / 1995 / Second language acquisition in a study abroad context / John Benjamins : 123 ~ 148

  14. [단행본] Hartmann, R. R. K. / 1976 / Dictionary of language and linguistics / Wiley

  15. [학술지] Hedge T. / 1993 / Key concepts in ELT / ELT Journal 47 (3) : 275 ~ 277

  16. [학술지] Hong, K. S. / 2005 / Analysis of Fluency Concept of English Education / Linguistics Research 10 (1) : 129 ~ 140

  17. [단행본] Housen, A. / 2012 / Dimensions of L2 Performance and Proficiency: Complexity, Accuracy and Fluency in SLA / John Benjamins Publishing

  18. [학술지] 장병용 / 2014 / 한국인을 위한 영어 말하기 시험의 컴퓨터 기반 유창성 평가 / 말소리와 음성과학 6 (2) : 9 ~ 20

  19. [단행본] Jang, S. Y. / 2009 / FACETS Program for Language Education Evaluation Research / Globalcontent

  20. [학위논문] Jeong, H. Y. / 2000 / Evaluation Method of Korean Speaking Proficiency - Focusing on FSI Oral Proficiency Test Analysis

  21. [학술지] Jeong, W. K. / 2015 / Korean Language Education, Korean Education: Suggestion of Introduction of Speaking Evaluation of Korean Ability Test -Focusing on Speaking Evaluation of ACTFL-OPI- / Hansung Language and Literature 34 : 465 ~ 497

  22. [단행본] Ji, H. S. / 2017 / Korean Evaluation / Hangul Park

  23. [학술지] 강승혜 / 2005 / 한국어 고급 말하기 평가 도구 개발 기초 연구 - 고급 말하기 토론 활동을 중심으로 - / 외국어로서의 한국어교육 30 : 1 ~ 21

  24. [학술지] Kang, S. H. / 2016 / CURRENT ISSUES /Orientation for Topic Speaking Assessment / Language and Culture 12 (1) : 201 ~ 212

  25. [학술지] Kartik, A. / 2009 / Automatic Evaluation of Fluency in Spoken Language / IETE Technical Review 26 (2) : 108 ~ 114

  26. [보고서] Kim, J. S. / 2006 / Development of Basic Research and Evaluation Model for Korean Speaking Ability Assessment Final Report

  27. [학술대회] Kim, S. J. / 2011 / Current Status and Prospects of Korean Speaking Listening Comprehension - Focusing on the Korean Language Proficiency Test- / Paper presented at The International Association for Korean Language Education Conference 35 : 29 ~ 38

  28. [학술지] Kim, S. S. / 2001 / Analysis Method of Variation of Grading: Generalization Possibility Theory and Application with Reinterpretation of Many-facet Rash model / Journal of Educational Evaluation 14 (1) : 303 ~ 325

  29. [학술지] Kim, S. S. / 2008 / A Study on the Fluency of Korean Speaker / Korean Education 19 (2) : 1 ~ 16

  30. [학위논문] Kim, S. S. / 2009 / Fluency study of Korean Learner Utterance

  31. [학술지] 김태경 / 2015 / 외국어로서의 한국어 발화 유창성 변화 연구 - 중국어 모어 화자를 대상으로 - / 언어과학연구 (75) : 129 ~ 150

  32. [학술지] 김태경 / 2007 / 유창성 요인으로 본 말하기 능력 / 한국언어문화 (34) : 25 ~ 44

  33. [학술지] Kim, Y. A. / 1996 / The Problem of Korean Globalization: A Study on Korean; Evaluation of Korean as a Foreign Language / Bilingualism 13 : 63 ~ 105

  34. [학술지] Kim, Y. J. / 1998 / A Study on Evaluation Method of Korean Language Proficiency: Focusing on Achievement Test / Korean Education 9 (1) : 37 ~ 94

  35. [학술지] Kim, Y. S. / 1995 / Validity of Spoken Ability Evaluation / Duksung Women's University Journal 24 : 17 ~ 35

  36. [단행본] Koponen, M. / 2000 / Perspectives on Fluency / University : 5 ~ 24

  37. [학술지] Kormos, J. / 2004 / Exploring Measures and Perceptions of Fluency in the Speech of Second Language Learners / System 32 : 145 ~ 164

  38. [학술지] 권태현 / 2012 / 쓰기 채점 방식에 따른 국어교사의 채점 신뢰도 비교 연구 -고등학생의 비평문 채점을 중심으로- / 교원교육 28 (3) : 113 ~ 134

  39. [학술지] 이향 / 2012 / 말하기 수행 평가에서 발음 범주 채점의 최적화 방안 연구 -일반화가능도 이론을 활용하여- / 한국어교육 23 (2) : 301 ~ 329

  40. [학술지] 이향 / 2012 / 한국어 말하기 수행평가의 발음 범주 채점 방식에 따른 채점 신뢰도 분석 -다국면 라쉬 모형을 활용하여- / 외국어로서의 한국어교육 37 : 325 ~ 355

  41. [학술지] 이향 / 2013 / 발음 평가에 있어서 정확성, 유창성, 이해명료성, 이해가능성 기준 간의 영향 관계 연구 / 언어와 문화 9 (3) : 221 ~ 243

  42. [학술대회] Lee, H. / 2017 / Consideration of Pronunciation when “You are speaking good at Korean!”-For the study of Korean Pronunciation and the Reconsideration of Education Direction- / Paper presented at the Bilingual conference 2017 (4) : 1 ~ 14

  43. [학위논문] 이향 / 2013 / 한국어 말하기 수행 평가의 발음 범주 채점에 대한 타당성 검증

  44. [학술대회] Lee, H. / 2013 / A Study on the Scoring Trend according to the Characteristics of the Scorer in the Pronunciation Area Scoring of Korean Speech Assessment - Focused on Korean Language Education Experience and Major- / Paper presented at The Korean Language and Culture Education Society Conference 2013 (1) : 127 ~ 141

  45. [학술지] 이효진 / 2013 / 한국어학습 외국인의 읽기, 쓰기, 말하기 유창성 특성 / 언어치료연구 22 (4) : 123 ~ 142

  46. [학술지] 이정희 / 2010 / 인식 조사를 통한 한국어 구어 유창성의 개념 및 요인 연구 / 한국어교육 21 (4) : 183 ~ 204

  47. [학위논문] Lee, J. Y / 2009 / Stepwise Style and Criterion Evaluation of Korean Speaking Achievement Evaluation

  48. [학술지] 이경 / 2015 / 한국어 어휘 유창성과 평가 구인에 대한 인식 연구 - 채점자 논평을 중심으로 - / 이중언어학 (58) : 59 ~ 86

  49. [학위논문] Lee, S. R. / 2002 / The Effects of Scorer Training on English Writing Scale of Secondary School Teachers: Focusing on the Reliability of the Scorer Using Rash model

  50. [학술지] 이선영 / 2013 / 한국어 말하기 성취도 평가에서의 평가범주의 세분화 문제 연구 / 우리어문연구 (45) : 27 ~ 55

  51. [단행본] Lee, Y. K. / 2012 / English Rating Methodology / Munjinmedia

  52. [학술지] 이영식 / 2004 / 한국어 말하기 시험의 유형 및 채점 기준 설정을 위한 기초 연구 / 한국어교육 15 (3) : 207 ~ 228

  53. [학술지] 이영식 / 2014 / 한국어 작문 평가 채점의 타당도 검증: 다국면 Rasch 측정 모형 활용 / Foreign Languages Education 21 (3) : 355 ~ 375

  54. [학술지] Lenon, P. / 1990 / Investigating Fluency in EFL: A Quantitative Approach / Language Learning 40 : 387 ~ 417

  55. [학술지] Lumley, T. / 1995 / Rater Characteristics and Rater Bias: Implications for Training / Language Testing 12 (1) : 54 ~ 71

  56. [단행본] McNamara, T. F. / 1996 / Measuring Second Language Performance / Longman

  57. [단행본] Morrison, D. M. / 1985 / New Directions in Language Testing / Pergamon institute of English : 125 ~ 145

  58. [학술지] 나카가와 마사오미 / 2010 / 한국어교육에서 ACTFL-OPI의 활용 방안 / 국어교육학연구 (37) : 283 ~ 311

  59. [학술지] Nation, I. S. P. / 1989 / Investigating Fluency in EFL: A Quantitative Approach / Language Learning 40 : 377 ~ 384

  60. [학술지] No, D. K. / 1976 / Language Patterns of Firepower Proficiency Test / Yonsei Language and Literature 7 (8) : 91 ~ 102

  61. [학술지] North, B. / 2000 / The Development of a Common Framework Scale of Language Proficiency based on a Theory of Measurement / System 23 (4) : 445 ~ 465

  62. [학술지] North, B. / 1998 / Scaling Descriptors for Language Proficiency Scales / Language Testing 15 (2) : 217 ~ 262

  63. [보고서] National Institute for International Education / 2012 / Development of a Question Type based on CBT / IBT Speaking Evaluation of the Korean Language Proficiency Test

  64. [단행본] Richards, J. C. / 1985 / Longman Dictionary of Applied Linguistics / Longman

  65. [학술지] Marian J. Rossiter / 2009 / Perceptions of L2 Fluency by Native and Non-native Speakers of English / Canadian Modern Language Review 65 (3) : 395 ~ 412

  66. [단행본] Segalowitz, N. / 2000 / Perspectives on fluency / The University of Michigan Press : 200 ~ 219

  67. [단행본] Segalowitz, N. / 2010 / Cognitive Bases of Second Language Fluency / Routledge

  68. [학술지] 신동일 / 2005 / NEW FACETS을 활용한 채점자료 분석방법 / Foreign Languages Education 12 (2) : 191 ~ 212

  69. [학술지] Peter Skehan / 2003 / Task-based instruction / Language Teaching 36 (1) : 1 ~ 14

  70. [학술지] Smemoe, W. B. / 2013 / The Effect of Language Learning Aptitude. Strategy Use and Learning Context on L2 Pronunciation Learning / Applied Linguistics 34 (4) : 435 ~ 456

  71. [단행본] Starkweather, W. / 1987 / Stuttering Prevention: A Clinical Method / Prentice hall

  72. [단행본] Tavakoli, P. / 2005 / Planning and Task Performance / John Benjamins Publishing Company : 239 ~ 273

  73. [학술지] Weigle, S. C. / 1994 / Effects of Training on Raters of ESL Compositions / Language Testing 11 (2) : 197 ~ 223

  74. [학술지] Weigle, S. C. / 1998 / Using FACETS to Model Rater Training Effects / Language Testing 15 (2) : 263 ~ 287

  75. [학술대회] Won, M. J. / 2016 / Procedures for Developing a Test for TOPIK Speaking Evaluation Development / Paper presented at The Korean Language and Culture Education Society Conference 2016 (10) : 164 ~ 170

  76. [단행본] Wright, B. D. / 1982 / Rating Scale Analysis. Rasch Measurement / MESA Press

  77. [학술지] Yingjie I. J. Yang / 2014 / Is Speaking Fluency Strand Necessary for the College Students to Develop in the EFL Class? / Theory and Practice in Language Studies 4 (2) : 225 ~ 231