Abstract
Automatic programming exercise assessment aims at determining the correctness of the attempts of programming exercises submitted by students. Automation allows students to receive instant and customized feedback which are important to enhance the learning of novice students. Educators can benefit from saving time and effort in marking students’ attempts, making teaching large, online classes or Massive Open Online Courses (MOOC) possible and effective. Recently, we modelled program outputs using Hierarchical Program Output Structure (HiPOS), which allows instructors to design matching rules to determine correct or partially correct programs depending on the teaching and learning needs. This paper extends our previous work by automating the matching rule construction process through developing a machine learning method for generalizing program outputs from students’ attempts. To achieve this, our approach firstly employs natural language processing techniques to create a HiPOS from a set of students’ program outputs. A greedy algorithm is then applied to generalize the HiPOS and create the associated matching rules. We conducted a case study to illustrate how to apply our proposed method in automated programming exercise assessment and demonstrated the usefulness and effectiveness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Example of judging systems of programming contests include PC2 (https://pc2.ecs.csus.edu/) and Kattis (https://open.kattis.com/).
- 2.
Natural Language Toolkit: https://www.nltk.org/.
References
Crow, T., Luxton-Reilly, A., Wuensche, B.: Intelligent tutoring systems for programming education: a systematic review. In: Proceedings of the 20th Australasian Computing Education Conference, pp. 53–62 (2018)
Dawson, P., et al.: What makes for effective feedback: staff and student perspectives. Assess. Eval. High. Educ. 44(1), 25–36 (2019)
Henderson, M., Ryan, T., Phillips, M.: The challenges of feedback in higher education. Assess. Eval. High. Educ. 44(8), 1237–1252 (2019)
Law, K.M., Lee, V.C., Yu, Y.T.: Learning motivation in e-learning facilitated computer programming courses. Comput. Educ. 55(1), 218–228 (2010)
Lindberg, R.S., Laine, T.H., Haaranen, L.: Gamifying programming education in K-12: a review of programming curricula in seven countries and programming games. Br. J. Edu. Technol. 50(4), 1979–1995 (2019)
Lizzio, A., Wilson, K.: Feedback on assessment: students’ perceptions of quality and effectiveness. Assess. Eval. High. Educ. 33(3), 263–275 (2008)
Lobb, R., Harlow, J.: Coderunner: a tool for assessing computer programming skills. ACM Inroads 7(1), 47–51 (2016)
Mekterović, I., Brkić, L., Milašinović, B., Baranović, M.: Building a comprehensive automated programming assessment system. IEEE Access 8, 81154–81172 (2020)
Pieterse, V.: Automated assessment of programming assignments. In: Proceedings of the Computer Science Education Research Conference (CSERC), pp. 4–5 (2013)
Poon, C.K., Wong, T.L., Yu, Y.T., Lee, V.C., Tang, C.M.: Toward more robust automatic analysis of student program outputs for assessment and learning. In: Proceedings of the 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), vol. 1, pp. 780–785 (2016)
Poon, C.K., Wong, T.L., Tang, C.M., Li, J.K.L., Yu, Y.T., Lee, V.C.S.: Automatic assessment via intelligent analysis of students’ program output patterns. In: Proceedings of the International Conference on Blended Learning, pp. 238–250 (2018)
Queirós, R.A.P., Leal, J.P.: PETCHA: a programming exercises teaching assistant. In: Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education, pp. 192–197 (2012)
Tang, C.M., Yu, Y.T., Poon, C.K.: An approach towards automatic testing of student programs using token patterns. In: Proceedings of the Seventeenth International Conference on Computers in Education (ICCE), pp. 188–190 (2009)
Tang, C.M., Yu, Y.T., Poon, C.K.: An experimental prototype for automatically testing student programs using token patterns. In: Proceedings of the International Conference on Computer Supported Education (CSEDU), pp. 144–149 (2010)
Wasik, S., Antczak, M., Badura, J., Laskowski, A., Sternal, T.: A survey on online judge systems and their applications. ACM Comput. Surv. (CSUR) 51(1), 1–34 (2018)
Acknowledgement
The work described in this paper is fully supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project UGC/FDS11(14)/E02/15).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wong, TL., Poon, C.K., Tang, C.M., Yu, Y.T., Lee, V.C.S. (2020). Automatic Generation of Matching Rules for Programming Exercise Assessment. In: Lee, LK., U, L.H., Wang, F.L., Cheung, S.K.S., Au, O., Li, K.C. (eds) Technology in Education. Innovations for Online Teaching and Learning. ICTE 2020. Communications in Computer and Information Science, vol 1302. Springer, Singapore. https://doi.org/10.1007/978-981-33-4594-2_11
Download citation
DOI: https://doi.org/10.1007/978-981-33-4594-2_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-33-4593-5
Online ISBN: 978-981-33-4594-2
eBook Packages: Computer ScienceComputer Science (R0)