Abstract
Recently, introducing computer science and educational robots in compulsory education has received increasing attention. However, the use of screens in classrooms is often met with resistance, especially in primary school. To address this issue, this study presents the development of a handwriting-based programming language for educational robots. Aiming to align better with existing classroom practices, it allows students to program a robot by drawing symbols with ordinary pens and paper. Regular smartphones are leveraged to process the hand-drawn instructions using computer vision and machine learning algorithms, and send the commands to the robot for execution. To align with the local computer science curriculum, an appropriate playground and scaffolded learning tasks were designed. The system was evaluated in a preliminary test with eight teachers, developers and educational researchers. While the participants pointed out that some technical aspects could be improved, they also acknowledged the potential of the approach to make computer science education in primary school more accessible.
Supported by the NCCR Robotics, Switzerland.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
With the increasing efforts to introduce computer science (CS) in K-12 worldwide [2], it has become necessary to develop tools and environments that offer an appropriate progression across schools years [13]. Nowadays, aside from Text-based Programming Languages such as Python or C, three main approaches to teach CS exist: CS unplugged activitiesFootnote 1 that do not involve the use of screens and aim at teaching core CS concepts rather than programming per se; Tangible Programming Languages (TPLs) which involve the use of physical manipulatives to program virtual or physical agents [9, 10]; and Visual Programming Languages (VPLs) such as ScratchFootnote 2, which offer block-based solutions to programming, either on tablets or computers [7]. Although VPLs are increasingly present in education settings, they are met with resistance from teachers at the lower levels of primary school mainly due to the presence of screens [2]. Conversely, TPLs, which are still not present in formal settings [8] appear to be better suited for the needs of primary school education. Indeed, the tangibility of TPL approaches aligns nicely with early childhood education practices and principles [3, 15], promotes active and collaborative learning [9, 16], and is less likely to be met with resistance by teachers. Moreover, TPLs seem to appeal more than VPLs to young students [7] and help mitigate gender stereotypes [7, 9]. Lastly, experiments suggest that TPLs have a positive impact on interest, enjoyment and learning [9], all the while reducing the amount of errors produced [17] and the amount of adult support required [8] compared to VPLs.
Educational Robots (ER), which by design embody and enact abstract CS concepts and directives, can be thought of as naturally aligned with the principles of TPL, and thus a potential means to promote the adoption of TPL by teachers. Among the solutions proposed in the literature to combine ER and TPL, those not requiring the purchase of additional dedicated hardware (i.e., beside the robot itself) seem particularly promising for primary school education settings [6]. A recent example is the work of Mehrotra et al. [8], who developed a Paper-based Programming Language (PaPL) for the Thymio II robotFootnote 3, observing that the paper-based interface helps promote the use of tangible interfaces in classrooms by offering a ubiquitous and accessible solution which only requires the robot and a classroom laptop. Similarly, TPL approaches that introduce handwriting as a way to provide instructions/values [14] seem particularly interesting for primary school education settings, enhancing the programming sessions with the possibility to train essential graphomotor skills [18], which have been shown to be linked to improved student learning [12].
In this work, we build on the principles of the first handwriting TPL approaches and the Paper-based Programming Language to introduce a Handwriting-based Programming Language (HPL). By allowing students to program the robot by handwriting instructions on a sheet of paper, HPL relies on two cornerstones of primary school classrooms: the use of paper as material and of handwriting as a means of expression. HPL can thus be seen as an effort to take programming one step closer to primary school practices, all the while capitalising on the benefits handwriting has with respect to student learning [12].
2 Development of HPL
Our goal is to develop a Handwriting-based Programming Language for the Thymio II robot which could serve as a precursor to more advanced TPLs and VPLs for ER. For this reason, we focus on sequential programming and basic robot movements. As movements in TPLs are often represented by arrows, the instructions of the HPL were designed to allow for a “clear mapping between [the] tangible and virtual commands” [13] (see Fig. 1 - A).
A data set of 6888 images of handwritten arrows was acquired from 92 students in the 9 to 12 age range, and split 60%–40% between training and testing. Both during training and testing, each image is first pre-processed using adaptive filtering and binarisation followed by morphological operators, to reduce the noise while preserving edges and remove asperities [5]. The arrows are then isolated by identifying contours. The feature set is created to characterise the shape differences between the arrows by combining a) Fourier descriptors, b) Hellinger distances between the pixel density histograms along the x and y axes of the arrow to be classified and the centroid of the classes computed on the training set, and c) geometric features (circularity, convexity, inertia, rotated bounded rectangle, minimum including circle, moments, etc...). A number of state-of-the-art classifiers were trained and tested, namely: Decision Tree; Support Vector Machine; Multilayer Perceptron (MLP); kNN; Random Forest; AdaBoost; Naive Bayes classifier. The MLP achieved the best performance (reported in Table 1).
While the symbol set here only included motion commands, the paper and pen approach allows to easily change the symbols and provides the opportunity to develop more complex commands. While contingent on the creation of a data set with a selection of symbols which must be both intuitive for participants and sufficiently distinct for classification, we do not perceive any technical limitations to expanding the HPL language to include a larger set of instructions (including conditionals and loops for example).
3 An ER-HPL Learning Activity
Specifically, the activity aims at helping students 1) understand the concept of sequential programming, 2) discover the notions of cost and path planning, inspired from the work done in the JUSThink project [11], and 3) reinforce the understanding of geometry and angles. A set of scaffolded tasks were designed to introduce the students to these concepts using the playground and interface presented in Fig. 2. In the tasks, students are asked to provide a sequence of instructions allowing the robot to reach a target on a map, by first drawing arrows on sheets of paper and then taking images of them using an ad-hoc application running on a tablet. The application analyses and transforms the instructions into robot motion commands, according to the workflow shown in Fig. 1 - B.
The use of paper introduces the possibility to use a physical exercise book, a natural approach which is close to classroom practices and helps monitor student progress over time. Additionally, it gives the teacher the opportunity to scaffold the instructions according to the needs of the individual students. Indeed, to promote collaboration, the teacher can choose to employ a card-based solution (similar to traditional TPLs) requiring students to draw the arrows only once, on separate cards, and then move them around to find the correct sequence (see Fig. 2, Option 1). Conversely, if the objective is to have the students practice their handwriting, the teacher can choose to have them write the arrows on a single sheet of paper (see Fig. 2, Option 2).
A preliminary heuristic evaluation of the HPL platform and corresponding activity framework was conducted based on the methodology developed by Giang [4]. The evaluation included 8 participants with experience in education (2 teachers, 4 engineers, 2 educational researchers) representing the different stakeholders involved in the development of ER content. The participants were interested and evoked the engaging, kinaesthetic, playful character of this low cost approach to teaching programming. Although one researcher mentioned that they were unsure how the HPL would help improve learning CS concepts, other participants appreciated that the HPL would help train graphomotor skills and increase sense of ownership of the system. Participants mentioned that the HPL components were easy to understand (but not necessarily easy to draw), and suggested to continue improving the robustness of the symbol recognition. They believed that drawing the symbols on individual cards would facilitate collaboration, and allow to easily modify the number of symbols to be used. They believed that the activity set designed was intuitive and didactic. They appreciated the various scaffolding possibilities and the presence of a physical exercise book complementing the in-app instructions. They characterised the overall platform as intuitive, but suggested improvements in terms of storytelling and the application’s graphical user interface, specifically recommending additional feedback on the smartphone.
4 Conclusion
In this article we propose the Handwriting-based Programming Language (HPL), aiming at facilitating the introduction of programming in primary school education. HPL relies on paper and handwriting to program Educational Robots (ER) in a tangible way, while leveraging on well-known practices for early childhood education. From the students’ perspective, the use of handwriting as input modality introduces the possibility to train graphomotor skills while programming, and can potentially contribute to an increased sense of controllability and ownership, which, like handwriting in general, are important factors for student learning. From the teachers’ perspective, HPL aims to be close to traditional pedagogical approaches and facilitate the development of learning activities that organically take into account instructions (and instruction modalities), learning artefacts (i.e., the robot, playground and interfaces) and assessment methods [4]. Therefore, we believe that HPL will be more attractive for teachers who are still reticent about educational technologies. Future steps include evaluating these hypotheses in classroom experiments with the target group and expanding to a larger set of instructions. One could even envision a fully tablet-based alternative in conjunction with a digital pen and an automated handwriting diagnostic system [1]. Such a system would not only be able to assess the child’s progress over time but would be able to propose adequate remediation to address common handwriting difficulties [1] through playful means.
Change history
29 October 2022
The original versions of the Chapters 17 and 25 were previously published as non-open access. They have now been changed to open access under the CC BY 4.0 license and the copyright holder updated to ‘The Author(s)’. The book and the chapters have been updated with the changes.
Notes
- 1.
CS Unplugged activities: csunplugged.org/en/.
- 2.
See scratch.mit.edu for Scratch resources and scratchjr.org for Scratch Jr.
- 3.
Thymio II - https://www.thymio.org/.
References
Asselborn, T.L.C.: Analysis and Remediation of Handwriting difficulties, p. 157 (2020). https://doi.org/10.5075/epfl-thesis-8062
El-Hamamsy, L., et al.: A computer science and robotics integration model for primary school: evaluation of a large-scale in-service K-4 teacher-training program. Educ. Inf. Technol. 26(3), 2445–2475 (2020). https://doi.org/10.1007/s10639-020-10355-5
Elkin, M., Sullivan, A., Bers, M.U.: Programming with the KIBO robotics kit in preschool classrooms. Comput. Schools 33(3), 169–186 (2016)
Giang, C.: Towards the alignment of educational robotics learning systems with classroom activities, p. 176 (2020). https://doi.org/10.5075/epfl-thesis-9563
Herrera-Camara, J.I., Hammond, T.: Flow2Code: from hand-drawn flowcharts to code execution. In: SBIM 2017, pp. 1–13. ACM (2017)
Horn, M.S., Jacob, R.J.K.: Designing tangible programming languages for classroom use. In: TEI 2007, p. 159. ACM Press (2007)
Horn, M.S., Solovey, E.T., Crouser, R.J., Jacob, R.J.: Comparing the use of tangible and graphical programming languages for informal science education. In: CHI 2009, pp. 975–984. ACM, April 2009
Mehrotra, A., et al.: Introducing a Paper-Based Programming Language for Computing Education in Classrooms. In: ITiCSE 2020, pp. 180–186. ACM (2020)
Melcer, E.F., Isbister, K.: Bots & (Main)Frames: exploring the Impact of Tangible Blocks and Collaborative Play in an Educational Programming Game. In: CHI 2018, pp. 1–14. ACM (2018)
Mussati, A., Giang, C., Piatti, A., Mondada, F.: A Tangible programming language for the educational robot Thymio. In: 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), pp. 1–4 , July 2019
Nasir, J., Norman, U., Johal, W., Olsen, J.K., Shahmoradi, S., Dillenbourg, P.: Robot analytics: what do human-robot interaction traces tell us about learning? In: 2019 28th IEEE RO-MAN, pp. 1–7, October 2019
Ose Askvik, E., van der Weel, F.R.R., van der Meer, A.L.H.: The importance of cursive handwriting over typewriting for learning in the classroom: a high-density EEG study of 12-year-old children and young adults. Front. Psychol. 11, 1810 (2020)
Papavlasopoulou, S., Giannakos, M.N., Jaccheri, L.: Reviewing the affordances of tangible programming languages: implications for design and practice. In: 2017 IEEE EDUCON, pp. 1811–1816 (2017)
Sabuncuoğlu, A., Sezgin, M.: Kart-ON: affordable early programming education with shared smartphones and easy-to-find materials. In: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion, IUI 2020, pp. 116–117. ACM (2020)
Sapounidis, Theodosios, Demetriadis, Stavros: Educational robots driven by tangible programming languages: a review on the field. In: Alimisis, Dimitris, Moro, Michele, Menegatti, Emanuele (eds.) Edurobotics 2016 2016. AISC, vol. 560, pp. 205–214. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55553-9_16
Sapounidis, T., Demetriadis, S., Papadopoulos, P.M., Stamovlasis, D.: Tangible and graphical programming with experienced children: a mixed methods analysis. Int. J. Child Comput. Interact. 19, 67–78 (2019)
Sapounidis, Theodosios, Demetriadis, Stavros, Stamelos, Ioannis: Evaluating children performance with graphical and tangible robot programming tools. Personal Ubiquit. Comput. 19(1), 225–237 (2014). https://doi.org/10.1007/s00779-014-0774-3
Ziviani, J.: The development of graphomotor skills. In: Hand Function in the Child, pp. 217–236. Elsevier (2006)
Acknowledgements
A big thank you goes out to the teacher A.S. and the students involved in the data collection, as well as our always supportive colleagues.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this paper
Cite this paper
El-Hamamsy, L. et al. (2022). Exploring a Handwriting Programming Language for Educational Robots. In: Merdan, M., Lepuschitz, W., Koppensteiner, G., Balogh, R., Obdržálek, D. (eds) Robotics in Education. RiE 2021. Advances in Intelligent Systems and Computing, vol 1359. Springer, Cham. https://doi.org/10.1007/978-3-030-82544-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-82544-7_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82543-0
Online ISBN: 978-3-030-82544-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)