Skip to main content
SpringerLink
Log in

EFL Writing Assessment: Peer Assessment vs. Automated Essay Scoring

  • Conference paper
  • First Online:
Emerging Technologies for Education (SETE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11984))

Included in the following conference series:

Abstract

This study aimed to explore problems and potentials of new technologies in English as foreign language (EFL) writing education. Forty-six students as a foreign language (EFL) learners in a Chinese university participated in this study. They submitted their draft to Pigai Network and Scholar Network separately and received automated essay scoring (AES) and peer assessment (PA) feedback. Results showed a moderate, positive partial correlation between PA and AES, controlling for performance level. The EFL learners in China preferred AES over PA. These findings raise several relevant issues in how to improve peer assessment feedback effectively, such as writing rubric in peer assessment, specialized peer assessment tool, technology assistant and peer feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Berg, E.C.: The effects of trained peer response on ESL students’ revision types and writing quality. J. Second Lang. Writ. 8(3), 215–241 (1999)

    Article  Google Scholar 

  2. Yang, M., Badger, R., Yu, Z.: A comparative study of peer and teacher feedback in a Chinese EFL writing class. J. Second Lang. Writ. 15(3), 179–200 (2006)

    Article  Google Scholar 

  3. Attali, Y., Burstein, J.: Automated essay scoring with e-rater®; V.2.0. J. Technol. Learn. Assess. 4(2), i–21 (2006)

    Google Scholar 

  4. Dikli, S.: Automated essay scoring. Turk. Online J. Distance Educ. 7(1), 735–738 (2006)

    Google Scholar 

  5. Deane, P.: On the relation between automated-essay scoring and modern views of the writing construct. Assessing Writ. 18(1), 7–24 (2013)

    Article  Google Scholar 

  6. Sommers, N.: Responding to student writing. Coll. Compos. Commun. 33(2), 148–156 (1982)

    Article  Google Scholar 

  7. Chang, C.-C., et al.: Reliability and validity of web-based portfolio peer assessment: a case study for a senior high school’s students taking computer course.”. Comput. Educ. 57(1), 1306–1316 (2011)

    Article  Google Scholar 

  8. Chang, C.-C., Yan, C.-F., Tseng, J.-S.: Perceived convenience in an extended technology acceptance model: mobile technology and English learning for college students. Australas. J. Educ. Technol. 28(5), 809–826 (2012)

    Article  Google Scholar 

  9. Topping, K.J.: Methodological quandaries in studying process and outcomes in peer assessment. Learn. Instr. 20(4), 339–343 (2010)

    Article  Google Scholar 

  10. Falchikov, N.: Product comparisons and process benefits of collaborative peer group and self-assessments. Assess. Eval. High. Educ, 11(2), 146–166 (1986)

    Article  Google Scholar 

  11. Falchikov, N., Goldfinch, J.: Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks. Rev. Educ. Res. 70(3), 287–322 (2000)

    Article  Google Scholar 

  12. Topping, K.: Peer assessment between students in colleges and universities. Rev. Educ. Res. 68(3), 249–276 (1998)

    Article  Google Scholar 

  13. Meek, S.E.M., Blakemore, L., Marks, L.: Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review. Assess. Eval. High. Educ. 42(6), 1000–1013 (2017)

    Article  Google Scholar 

  14. Hovardas, T., Tsivitanidou, O.E., Zacharia, Z.C.: Peer versus expert feedback: an investigation of the quality of peer feedback among secondary school students. Comput. Educ. 71, 133–152 (2014)

    Article  Google Scholar 

  15. Tsai, C.-C., Lin, S.S.J., Yuan, S.-M.: Developing science activities through a networked peer assessment system. Comput. Educ. 38(1–3), 241–252 (2002)

    Article  Google Scholar 

  16. Hanrahan, S.J., Isaacs, G.: Assessing self-and peer-assessment: the students’ views. High. Educ. Res. Dev. 20(1), 53–70 (2001)

    Article  Google Scholar 

  17. Fang, Y.: Perceptions of the computer-assisted writing program among EFL college learners. Educ. Technol. Soc. 13(3), 246–256 (2010)

    Google Scholar 

  18. Enright, M.K., Quinlan, T.: Complementing human judgment of essays written by English language learners with e-rater® scoring. Lang. Test. 27(3), 317–334 (2010)

    Article  Google Scholar 

  19. Chen, C.-F.E., Cheng, W.-Y.E.C.: Beyond the design of automated writing evaluation: pedagogical practices and perceived learning effectiveness in EFL writing classes. Lang. Learn. Technol. 12(2), 94–112 (2008)

    Google Scholar 

  20. Wang, Y.-J., Shang, H.-F., Briody, P.: Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Comput. Assist. Lang. Learn. 26(3), 234–257 (2013)

    Article  Google Scholar 

  21. Roscoe, R.D., et al.: Presentation, expectations, and experience: Sources of student perceptions of automated writing evaluation. Comput. Hum. Behav. 70, 207–221 (2017)

    Article  Google Scholar 

  22. Lai, Y.-h.: Which do students prefer to evaluate their essays: peers or computer program. Br. J. Edu. Technol. 41(3), 432–454 (2010)

    Article  Google Scholar 

  23. Jones, I., Wheadon, C.: Peer assessment using comparative and absolute judgement. Stud. Educ. Eval. 47, 93–101 (2015)

    Article  Google Scholar 

  24. Yu, F.-Y., Wu, C.-P.: Different identity revelation modes in an online peer-assessment learning environment: effects on perceptions toward assessors, classroom climate and learning activities. Comput. Educ. 57(3), 2167–2177 (2011)

    Article  Google Scholar 

  25. Hoon, T.: Online automated essay assessment: potentials for writing development (2010). Accessed 9 August 2006

    Google Scholar 

  26. Yeh, Y.-L., Liou, H.-C., Yu, Y.-T.: The influence of automatic essay evaluation and bilingual concordancing on EFL students. English Teach. Learn. 31(1), 117–160 (2007)

    Google Scholar 

Download references

Acknowledgements

The authors would like to express sincerely acknowledgements:

I am grateful to Professor Ming Ming Chiu for his constructive comments on drafting and revising this paper.

Funding

This work was supported by [Teaching Quality and Teaching Reform Project in Guangdong Province] under Grant [number 236: No. 201, No. 218]; [Guangdong Provincial Philosophy and Social Sciences Project] under Grant [number GD18WXZ18]; and [The Ministry of Education’s Higher Education Department, the second batch of industry-university collaborative education project] under Grant [number 201802083033]; [Guangdong University of Foreign Studies Postgraduate International Talents Training Innovation Project].

Author information

Authors and Affiliations

  1. Eastern Language Processing Center, School of Information Science and Technology, School of Cyber Security, Guangdong University of Foreign Studies, Guangzhou, China

    Meixiu Lu

  2. Guangdong University of Technology, Guangzhou, China

    Qing Deng

  3. School of English and Education, Guangdong University of Foreign Studies, Guangzhou, China

    Manzhen Yang

Authors
  1. Meixiu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manzhen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manzhen Yang .

Editor information

Editors and Affiliations

  1. University of Craiova, Craiova, Romania

    Elvira Popescu

  2. South China Normal University, Guangzhou, China

    Tianyong Hao

  3. National Taiwan Normal University, Taipei, Taiwan

    Ting-Chia Hsu

  4. Lingnan University, Hong Kong, Hong Kong

    Haoran Xie

  5. Sapienza University of Rome, Rome, Italy

    Marco Temperini

  6. Chinese Academy of Agricultural Sciences, Beijing, China

    Wei Chen

Appendix

Appendix

Perceptions Toward Feedback

  1. (A)

    Perceptions towards peer assessment from Pigai:

  2. 1.

    I regard Pigai as real audience.

  3. 2.

    I highly value the comments from Pigai on my writing.

  4. 3.

    I adopt comments from Pigai for revision.

  5. 4.

    I like writing with Pigai

  6. 5.

    I revise my writing more when I use Pigai.

  7. 6.

    Writing with Pigai has increased my confidence in my writing.

  8. 7.

    The essay scores Pigai gives are fair.

  9. 8.

    I feel Pigai won’t avoid giving negative feedback for fear of hurting the writer.

  10. 9.

    I enjoy Pigai activities during this semester.

  11. 10.

    I hope my teacher in writing class will continue Pigai activities next semester.

  12. (B)

    Perceptions towards Feedback from Peer Evaluation:

  13. 1.

    I regard my classmates as real audience.

  14. 2.

    I highly value my classmates’ comments on my writing.

  15. 3.

    I adopt my classmates’ comments for revision.

  16. 4.

    I like writing with my peer.

  17. 5.

    I revise my writing more when I have peer discussion.

  18. 6.

    Writing with my peer has increased my confidence in my writing.

  19. 7.

    The comments and suggestions my peer gives are fair.

  20. 8.

    I feel peer revision won’t avoid giving negative feedback for fear of hurting the writer.

  21. 9.

    I enjoy peer revision activities during this semester.

  22. 10.

    I hope my teacher in writing class will continue peer revision activities next semester.

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lu, M., Deng, Q., Yang, M. (2020). EFL Writing Assessment: Peer Assessment vs. Automated Essay Scoring. In: Popescu, E., Hao, T., Hsu, TC., Xie, H., Temperini, M., Chen, W. (eds) Emerging Technologies for Education. SETE 2019. Lecture Notes in Computer Science(), vol 11984. Springer, Cham. https://doi.org/10.1007/978-3-030-38778-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-38778-5_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-38777-8

  • Online ISBN: 978-3-030-38778-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation