Abstract
This paper is based on a research project aiming at improving learning long arithmetic operations in primary school using pen-based tablets. The goal is to automatically analyze a student’s handwritten answer by comparing it to an expected answer and to provide immediate feedback. This comes down to find any mistake made such as a calculus mistake, missing carry over or symbol misalignment. We use the correspondence obtained by the Graph Edit Distance (GED) computed between both the student and expected answers. In order to reduce graph sizes to overcome the computational complexity of the GED on large graphs, we present a new semantic graph of line segmentation. We propose a backtracking process to correct potential early mis-recognition mistakes for non-corresponding vertices. We evaluate the improvement on the analysis performances for an increasing number of backtracks on an in-house dataset composed of 400 handwritten operations.
With the support from the LabCom ScriptAndLabs founded by the ANR ANR-16-LVC2-0008-01. With the support from the ANRT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xin, Y.P., Tzur, R., Hord, C., Liu, J., Park, J.Y., Si, L.: An intelligent tutor-assisted mathematics intervention program for students with learning difficulties. Learn. Disabil. Q. 40(1), 4–16 (2017)
Huang, X., Craig, S.D., Xie, J., Graesser, A., Hu, X.: Intelligent tutoring systems work as a math gap reducer in 6th grade after-school program. Learn. Individ. Differ. 47, 258–265 (2016)
Mahdavi, M., Zanibbi, R., Mouchere, H., Viard-Gaudin, C., Garain, U.: ICDAR 2019 CROHME + TFD: competition on recognition of handwritten mathematical expressions and typeset formula detection. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1533–1538 (2019)
Blumenthal, D.B., Gamper, J.: On the exact computation of the graph edit distance. Pattern Recognit. Lett. 134, 46–57 (2020). Applications of Graph-based Techniques to Pattern Recognition
Zhang, J., Du, J., Dai, L.: Track, attend, and parse (tap): an end-to-end framework for online handwritten mathematical expression recognition. IEEE Trans. Multimedia 21(1), 221–233 (2018)
Hu, L., Zanibbi, R.: MST-based visual parsing of online handwritten mathematical expressions. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 337–342. IEEE (2016)
Zhang, T., Mouchère, H., Viard-Gaudin, C.: A tree-BLSTM-based recognition system for online handwritten mathematical expressions. Neural Comput. Appl. 32(9), 4689–4708 (2018). https://doi.org/10.1007/s00521-018-3817-2
Sanfeliu, A., Fu, K.-S.: A distance measure between attributed relational graphs for pattern recognition. IEEE Trans. Syst. Man Cybern. 3, 353–362 (1983)
Abu-Aisheh, Z., Raveaux, R., Ramel, J.-Y., Martineau, P.: An exact graph edit distance algorithm for solving pattern recognition problems. In: 4th International Conference on Pattern Recognition Applications and Methods 2015, Lisbon, Portugal, January 2015
Blumenthal, D.B., Gamper, J.: Exact computation of graph edit distance for uniform and non-uniform metric edit costs. In: Foggia, P., Liu, C.-L., Vento, M. (eds.) GbRPR 2017. LNCS, vol. 10310, pp. 211–221. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58961-9_19
Lerouge, J., Abu-Aisheh, Z., Raveaux, R., Héroux, P., Adam, S.: New binary linear programming formulation to compute the graph edit distance. Pattern Recogn. 72, 254–265 (2017)
Fischer, A., Plamondon, R., Savaria, Y., Riesen, K., Bunke, H.: A hausdorff heuristic for efficient computation of graph edit distance. In: Fränti, P., Brown, G., Loog, M., Escolano, F., Pelillo, M. (eds.) S+SSPR 2014. LNCS, vol. 8621, pp. 83–92. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44415-3_9
Bai, Y., Ding, H., Bian, S., Chen, T., Sun, Y., Wang, W.: Simgnn: a neural network approach to fast graph similarity computation. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 384–392 (2019)
Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)
Dutta, A., Riba, P., Lladós, J., Fornés, A.: Hierarchical stochastic graphlet embedding for graph-based pattern recognition. Neural Comput. Appl. 32(15), 11579–11596 (2020)
Hu, L., Zanibbi, R.: Line-of-sight stroke graphs and parzen shape context features for handwritten math formula representation and symbol segmentation. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 180–186. IEEE (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014)
Delaye, A., Anquetil, E.: Fuzzy relative positioning templates for symbol recognition. In: 2011 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 1220–1224. IEEE (2011)
Lods, A., Anquetil, E., Macé, S.: Graph edit distance for the analysis of children’s on-line handwritten arithmetical operations. In: 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 337–342. IEEE (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lods, A., Anquetil, É., Macé, S. (2021). Segmentation and Graph Matching for Online Analysis of Student Arithmetic Operations. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12823. Springer, Cham. https://doi.org/10.1007/978-3-030-86334-0_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-86334-0_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86333-3
Online ISBN: 978-3-030-86334-0
eBook Packages: Computer ScienceComputer Science (R0)
