Skip to main content
SpringerLink
Log in

A Study of Scoring English Tests Using an Automatic Scoring Model Incorporating Semantics

  • Published:
Automatic Control and Computer Sciences Aims and scope Submit manuscript

Abstract

The automatic essay scoring (AES) model enables automatic analysis and scoring of texts, which is an essential element in education. This paper designed an AES model to extract text features from lexical, semantic, and thematic aspects. The semantic features were obtained by a convolutional neural network (CNN) and long short-term memory (LSTM) neural network. The thematic features were obtained using term frequency-inverse document frequency (TD-IDF). Then, the neural network-based AES model was used to realize the automatic scoring of English texts. The experiment on the Kaggle ASAP dataset found that the model worked best when using all features for scoring, and its quadratic weighted Kappa and Pearson correlation coefficients were 0.8163 and 0.8535, respectively, which outperformed the baseline model. The experimental results demonstrate that the AES model is more reliable for automatic scoring of English texts than the baseline model. The model can have a better application in practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

REFERENCES

  1. Uysal, I. and Doan, N., Automated essay scoring effect on test equating errors in mixed-format test, Int. J. Assess. Tool. Educ., 2021, vol. 8, no. 2, pp. 222–238. https://doi.org/10.21449/ijate.815961

    Article  Google Scholar 

  2. Hao, S., Xu, Y., Ke, D., Su, K., and Peng, H., SCESS: A WFSA-based automated simplified Chinese essay scoring system with incremental latent semantic analysis, Nat. Lang. Eng., 2016, vol. 22, no. 2, pp. 291–319. https://doi.org/10.1017/S1351324914000138

    Article  Google Scholar 

  3. Stephen, T.C., Gierl, M.C., and King, Sh., Automated essay scoring (AES) of constructed responses in nursing examinations: An evaluation, Nurse Educ. Pract., 2021, vol. 54, no. 1, p. 103085. https://doi.org/10.1016/j.nepr.2021.103085

    Article  Google Scholar 

  4. Shin, J. and Gierl, M.J., More efficient processes for creating automated essay scoring frameworks: A demonstration of two algorithms, Lang. Test., 2021, vol. 38, no. 2, pp. 247–272. https://doi.org/10.1177/0265532220937830

    Article  Google Scholar 

  5. Li, H. and Dai, T., Explore deep learning for Chinese essay automated scoring, J. Phys.: Conf. Ser., 2020, vol. 1631, p. 12036. https://doi.org/10.1088/1742-6596/1631/1/012036

    Article  Google Scholar 

  6. McNamara, D.S., Crossley, S.A., Roscoe, R.D., Allen, L.K., and Dai, J., A hierarchical classification approach to automated essay scoring, Assessing Writing, 2015, vol. 23, pp. 35–59. https://doi.org/10.1016/j.asw.2014.09.002

    Article  Google Scholar 

  7. Xiao, R., Guo, W., Zhang, Y., Ma, X., and Jiang, J., Machine learning-based automated essay scoring system for chinese proficiency test (HSK), NLPIR 2020: Proc. 4th Int. Conf. on Natural Language Processing and Information Retrieval, Seoul, 2020, New York: Association for Computing Machinery, 2020, pp. 18–23. https://doi.org/10.1145/3443279.3443299

  8. Qian, L., Zhao, Y., and Cheng, Y., Evaluating China’s automated essay scoring system iWrite, J. Educ. Comput. Res., 2020, vol. 58, no. 4, pp. 771–790. https://doi.org/10.1177/0735633119881472

    Article  Google Scholar 

  9. Wilson, J. and Rodrigues, J., Classification accuracy and efficiency of writing screening using automated essay scoring, J. School Psychol., 2020, vol. 82, pp. 123–140. https://doi.org/10.1016/j.jsp.2020.08.008

    Article  Google Scholar 

  10. Elalfi, A.E.E., Elgamal, A.F., and Amasha, N.A., Automated essay scoring using Word2vec and support vector machine, Int. J. Comput. Appl., 2019, vol. 177, no. 25, pp. 20–29.

    Google Scholar 

  11. Pennington, J., Socher, R., and Manning, C., GloVe: Global vectors for word representation, Conference on Empirical Methods in Natural Language Processing, Doha, Qatar: Association for Computational Linguistics, 2014, pp. 1532–1543. https://doi.org/10.3115/v1/D14-1162

  12. Beseiso, M. and Alzahrani, S., An empirical analysis of BERT embedding for automated essay scoring, Int. J. Adv. Comput. Sci., 2020, vol. 11, no. 10, pp. 204–210. https://doi.org/10.14569/IJACSA.2020.0111027

    Article  Google Scholar 

  13. Shi, Q., Xu, Q., and Zhang, J., Amended DV-hop scheme based on N-gram model and weighed LM algorithm, Electron. Lett., 2020, vol. 56, no. 5, pp. 247–250. https://doi.org/10.1049/el.2019.2957

    Article  Google Scholar 

  14. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., and Jackel, L.D., Backpropagation applied to handwritten zip code recognition, Neural Comput., 1989, vol. 1, no. 4, pp. 541–551. https://doi.org/10.1162/neco.1989.1.4.541

    Article  Google Scholar 

  15. Ma, Y., Cai, X., and Sun, F., Towards no-reference image quality assessment based on multi-scale convolutional neural network, Comp. Model. Eng. Sci., 2020, vol. 123, no. 1, pp. 201–216. https://doi.org/10.32604/cmes.2020.07867

    Article  Google Scholar 

  16. Shi, P., Zhao, Z., Liu, K., and Li, F., Attention-based spatial-temporal neural network for accurate phase recognition in minimally invasive surgery: Feasibility and efficiency verification, J. Comput. Des. Eng., 2022, vol. 9, no. 2, pp. 406–416. https://doi.org/10.1093/jcde/qwac011

    Article  Google Scholar 

  17. Hochreiter, S. and Schmidhuber, J., Long short-term memory, Neural Comput., 1997, vol. 9, no. 8, pp. 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  18. Liu, M., Liu, R., Ren, Yi., Qi, Yu., Bai, J., and Wang, M., New energy power prediction optimization based on improved TF-IDF single machine information feature extraction, J. Phys.: Conf. Ser, 2020, vol. 1617, p. 012006. https://doi.org/10.1088/1742-6596/1617/1/012006

    Article  Google Scholar 

  19. Kaggle. The Hewlett Foundation: Automated essay scoring. http://www.kaggle.com/c/asap-aes/data. Cited April 1, 2019.

  20. Liu, J., Yang, X., and Zhao, L., Automated essay scoring based on two-stage learning, 2019. https://doi.org/10.48550/arXiv.1901.07744

  21. Cozma, M., Butnaru, A.M., and Ionescu, R.T., Automated essay scoring with string kernels and word embeddings, Proc. 56th Annu. Meeting of the Assoc. for Computational Linguistics, Gurevych, I. and Miyao, Yu., Eds., Melbourne: Association for Computational Linguistics, 2018, vol. 2, pp. 503–509. https://doi.org/10.18653/v1/P18-2080

Download references

Author information

Authors and Affiliations

  1. School of International Education, Zhengzhou Sias University, 451150, Zhengzhou, Henan, China

    Jing Wang

Authors
  1. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Wang.

Ethics declarations

The author declares that he has no conflicts of interest.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jing Wang A Study of Scoring English Tests Using an Automatic Scoring Model Incorporating Semantics. Aut. Control Comp. Sci. 57, 514–522 (2023). https://doi.org/10.3103/S0146411623050115

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S0146411623050115

Keywords:

Search

Navigation