Numeracy Literacy Course Learning Evaluation for Teacher Education Students – Early Childhood Education

Abstract


INTRODUCTION
Evaluation is one of the most critical factors, especially in the learning process where the purpose of evaluating is to obtain and analyze information, provide an overview of conditions, and make decisions based on the results of the analysis of the information obtained (Hakan & Seval, 2011;Setemen, 2010;Suardipa & Primayana, 2020). The evaluation activity is an inseparable part of a program whose process can be planned or unplanned (Hakan & Seval, 2011;Hanif & Hanif, 2015;Suardipa & Primayana, 2020). By evaluating, we will determine how successful the program is, whether it will be continued or stopped. Therefore, the tools used in assessing must be prepared carefully and consistently, and the role of the evaluator in implementing evaluation procedures can determine the success of the evaluation. (Jatmiko et al., 2020;Setemen, 2010;Suardipa & Primayana, 2020).
Learning evaluation is carried out to assess whether the learning carried out is following the learning objectives set and to determine the achievement and development of students in participating in learning activities (Hakan & Seval, 2011;Jatmiko et al., 2020;Komarudin & Sarkadi, 2017;Nuriyah, 2016;Setemen, 2010;Suardipa & Primayana, 2020). Therefore, the task of an educator will also be an evaluator in the learning process where later, the educator will find out how much the results obtained in educating his students (Febriana, 2019;Jatmiko et al., 2020;Munawar et al., 2023;Setemen, 2010;Suardipa & Primayana, 2020). Apart from this, an evaluation of the learning process in the Literacy and Numeracy Course was also carried out.
Numerical Literacy is a form of learning concepts, knowledge, and skills in (a) processing various kinds of numbers and symbols related to basic mathematics to solve various contexts of problems in everyday life, (b) analyzing information based on various graphic forms, tables, charts, etc. which then the results of the analysis can be interpreted to predict to make a decision and be principled in the implementation of education by developing reading, writing and arithmetic habits for the whole community (Ekowati et al., 2019). Based on this explanation, numeracy literacy, in general, is not only specifically for students, both at the Early Childhood level to the Higher Education level but must also become a concern for educators. Based on data released by UNESCO, Indonesian people are ranked 60th out of 61 countries in the world in terms of Literacy, which means that the literacy level is shallow (Purnomosari et al., 2022;Rahmawati, 2020). Meanwhile, based on the 2009 PISA data, Indonesia is ranked 68 out of 74 countries. In 2012 Indonesia was ranked 64th out of 65 countries with a relatively low level of achievement. While the results of PISA in 2015 showed that Indonesia's ranking experienced a slight increase in order, namely 63 out of 72 countries (Perdana & Suswandari, 2021).  2000  39  41  371  393  367  2003  38  40  392  395  360  2006  50  57  393  393  391  2009  57  57  393  393  391  2012  64  65  396  382  375  2015  64  72  397  386  403  2018  74  79  371  379  396  Sumber : PISA 2000, PISA 2003, PISA 2006, PISA 2009, PISA 2012, PISA 2015, PISA 2018(Dewayani et al., 2021 The research results from Mariamah strengthened the PISA data that only 5% of Indonesian students studied showed the ability to read at a high and advanced level. In comparison, more than 30% had a very low level, almost 40% only had a low level, and even only 25% of students reached the intermediate level (Mariamah et al., 2022). Based on some of the explanations, the government, specifically the Ministry of Education, Culture, Research, and Technology, created a learning design, namely the Independent Campus, where one of the concerns was to increase numerical literacy. The strategies carried out by educators in the process of strengthening numeracy literacy can be carried out on an ongoing basis, both internally and in collaboration with external partners who work together to support the learning process of their students (Henning, 2023;Nuryana et al., 2020). Therefore, educators must be able to understand literacy and numeracy in advance, how strategies are implemented, and also related to their application in everyday life so that students can absorb and apply the lessons taught by educators well in their lives.
Understanding numeracy literacy learning is a concern at all levels for students and educators as a determinant of direction in the learning process. For educators, literacy and numeracy can be used to develop assessment strategies, activities, and tools according to the characteristics of their students; while for students, the perceived benefits are for solving everyday problems (Mahmud & Pratiwi, 2019). However, in reality, not all humans can take advantage of literacy and numeracy skills, and many educators still have difficulty understanding how literacy is and creating a culture of critical literacy (Gustine, 2018). Therefore, it is necessary to doubt the quality of students related to literacy and numeracy if educators are still having difficulties understanding and implementing them. An understanding of literacy and numeracy must be fostered from an early age, especially for prospective educators.
Evaluation in the implementation of learning in literacy and numeracy courses, especially in the Teacher Education -Early Childhood Education (PG-PAUD) study program, has never been carried out, especially at Muhammadiyah University Jakarta. Even though these evaluation activities are very important for improving the quality of prospective teachers in literacy and numeracy learning before entering the field (Fiangga et al., 2019;Nuriyah, 2016;Zuhra et al., 2021). Therefore, this is a separate concern for evaluation. Several studies have been conducted related to the importance of literacy and numeracy, including research conducted by Fiangga which found that 100% of the teachers who participated in the training had never received any training. Only 4% were able to define literacy and numeracy.
Meanwhile, research conducted by Husniati shows that it is necessary to increase teacher competence in developing students' literacy and numeracy skills by improving the quality of teachers in developing, implementing, and evaluating literacy-oriented learning and numeracy (Husniati et al., 2022). In addition, research conducted by Wahyuni revealed that learning styles in early childhood greatly influence literacy and numeracy abilities, which indicated differences in the results of the visual and kinesthetic learning styles. (Wahyuni, 2022).
Based on the results of some of the research above shows that the implementation of literacy and numeracy understanding is very important. From some of these studies, there has also been no evaluation of literacy and numeracy learning, especially for prospective educators who will later work as educators in the field. From the results of an open questionnaire, it can be concluded that most of the teachers in this study may not fully understand the essence of critical literacy, and there are relatively few studies on critical literacy in the context of early childhood in Indonesia (Hidayat et al., 2021). Previous research only described the evaluation process and the Literacy and Numeracy program in school education. Still, no research has been conducted in higher education to see the readiness of prospective teachers, so the writer feels the need to evaluate learning in literacy and numeracy courses. The purpose of this research is to find out how far the implementation of learning has been carried out by evaluating the learning of Literacy and Numeracy courses. This is expected to be material for preparing learning strategies in the future. In addition, through literacy and numeracy courses taught to PG-PAUD study program students, it is hoped that later these students will be able to understand and apply literacy and numeracy concepts in their lives according to the conditions that occur with different students.

METHOD
This research was conducted at Muhammadiyah University Jakarta for one semester, from September 2022 to January 2023. In this study, the correspondence consisted of ten students in three classes who were randomly selected (random sampling). The approach in this study was the descriptive qualitative approach. Crescentini stated that conducting qualitative research involves many choices for the researcher (Crescentini & Mainardi, 2009).
The approach used in this study was program evaluation research. The purpose of program evaluation is to provide recommendations as material for consideration in making decisions about the implemented program (Munthe, 2015). By using this type of evaluation, decision-makers can make alternative decisions regarding training programs that are properly organized (Aryanti, 2016). Regarding the research focus, the program evaluation employed a descriptive qualitative approach. The approach was the evaluation model developed by Stufflebem, namely the CIPP evaluation model (Context, Input, Process, and Product). Ragil explained that this evaluation model provides a comprehensive review of the basic components, namely: (1) Determination of the information needed based on the formulation of the problem; (2) Determination of the necessary information collection techniques; (3) Determination of information collection techniques according to the type and source of information; (4) Determination of instruments to collect information; and (5) Data analysis according to the information collected (Ragil et al., 2020).

RESULT AND DISCUSSION A. Result
The implementation of the Numeracy Literacy course is carried out in the academic year 2022/2023 Odd with a systematic 16 meetings, of which meeting 8 is the implementation of the Mid-Semester Test and meeting 16 is the implementation of the End of Semester Test. The material given in the Numeracy Literacy concept can be seen from the competency map as follows: competency as follows:

Figure 1. Competency Map Chart for Literacy and Numeracy Courses
Concept Number 1, at this stage of the first meeting, an introduction to literacy and numeracy was explained. In the explanation, it is explained that there are the principles of Literacy and Numeracy, how are the Literacy and Numeracy components, the Basic principles of Literacy and Numeracy, and the Urgency of Literacy and Numeracy abilities in the learning process. At this first meeting, it was found that all students still did not understand the concept of literacy and numeracy because these terms were still considered new by some groups.
In concepts 2 and 3, the lecturer explains that literacy skills are derived from language skills, but the two differ. This also applies to the concept of numeracy, which is a derivative of mathematical ability, but the two are not the same thing.
After students understand the basic concepts of literacy and numeracy, they are explained that the concepts of school, family, and community are basic learning units.
In concepts 5, 6, and 7, the students are explained how the role of government is in the learning process of Numeracy and Literacy; this component also explains several regulations that apply, starting from the realm of central government regulations to the most technical regulations issued by the school. Furthermore, an explanation of the cognitive concept as one aspect of development and further discussed is the Graduate Competency Standards which are formulated in an integrated manner in the form of a description consisting of 6 competencies in the Pancasila Student Profile.
In concepts 8, 9, 10, and 11, the students are explained how literacy and numeracy are in the closest environment as life skills at school, in the family, and in society, after previously students were explained about various theories of literacy and numeracy. It was further explained that the success of implementing literacy and numeracy is not only the teacher's responsibility in explaining language and math skills as well as prospective teachers but also the responsibility of all elements in the School, in the Family, and the Community. So there needs to be good cooperation from several existing elements.
In concepts 12 and 13, the concept of evaluation and monitoring and several types of evaluation are used; students will later identify what type of evaluation will be used to measure the achievement of the implementation of Literacy and Numeracy. Furthermore, as a form of evaluating students' understanding of the implementation of the Literacy and Numeracy Course, students are given the task of being able to get down to practice the concepts discussed in several meetings.
In concept 14, lecturers and students carry out a reflection process. Reflection is carried out in the learning process to review the learning process that has been carried out in more detail.

B. Discussion
Evaluation in this study focuses on implementing learning in the Literacy and Numeracy course to carry out the mandate from the government. Based on the results of observations made in the lecture process, which were held several times by the lecturer in the Literacy and Numeracy course, it was presented in several points according to the evaluation model used, namely CIPP, which includes aspects of Context, Input, Process, and Product. Hakan explained that With context component it is aimed at determining the capabilities of subjects, such as the program's convenience for the development of students; With input evaluation component, it is aimed at finding answers to questions; With process evaluation component, it is aimed at finding answers to questions; With product evaluation component, it is aimed to find answers to questions such as meeting the individual needs, characteristics, interests of students completing the program, meeting the available and future needs of students (Hakan & Seval, 2011). Based on the results of the analysis, it was found that the use of the right strategy would affect the learning outcomes and learning motivation of students. It was indicated by various interviews that had been conducted with students who took part in learning activities. Even though there were many obstacles faced, they still participated in these activities until they were finished.

The Aspects of Context
The analysis results on the context aspect provided by the lecturers are still minimal references. This statement follows the results of the interviews conducted where the lecturers took many references from government regulations which were very substantial. This is difficult because technical references are minimal, so the supporting lecturer takes references from learning language, mathematics, and Pancasila, which are linked to literacy and numeracy learning. Then because this course is classified as a new subject, there are no references either from study program leaders or from disbursement on Google about the Semester Learning Plan (RPS) for the Literacy and Numeracy course, so at the beginning of the implementation of learning the lecturer does not yet have a teaching basis or benchmark in carrying out the process learning. The new lecturer has the RPS when learning has entered the fourth meeting. This condition contradicts the statement. In the pre-planning phase, the teacher thinks about how, what, when, and to whom the lesson will be taught. This first phase is a planning process-when the teacher feels about what will be taught (Ndihokubwayo et al., 2022).

The Aspect of Input
The learning process provided was quite conducive, but according to the student, due to their lack of knowledge about the subject, they were required to make a paper and present it, making the student wonder whether the paper and presentation he had given were correct. This condition is a challenge that must be faced and is under the statement. Identifying and understanding students' reasoning is challenging for teachers (Amador et al., 2022). However, in terms of how to teach, according to students, the lecturer is already good in his way of teaching and also in terms of teaching timeliness. The lecturer explained some obstacles to implementing Literacy and Numeracy courses, including the lack of references from the Study Program and Faculty. The lack of references or teaching materials obtained because, by reference, there are still very few that discuss literacy and numeracy. So I took references from learning language, mathematics, and Pancasila, which are linked to learning literacy and numeracy. There is no guideline for lecturers from the study program in the form of a syllabus or lesson plan for Literacy and Numeracy courses.
Students stated that several obstacles were encountered, namely because the learning process was carried out online; signal problems were often the main obstacle from both the lecturer and student sides which caused obstacles to the sound from the lecturer being captured by students when explaining material was not heard properly. "Sometimes a bad signal makes the voice of the lecturer who teaches become inaudible and falls apart" (Student 1). These findings corroborate the results of research conducted by (Fathoni et al., 2021), where the use of the Internet for online learning is increasingly wasteful, and the lack of stability of the Internet network disrupts the learning process that is carried out online.
The learning process provided was quite conducive, but according to the student, due to their lack of knowledge about the subject, they were required to make papers and present them, making these students wonder whether the papers and presentations they had given were correct. However, in terms of how to teach, according to students, the lecturer is already good in his way of teaching and also in terms of teaching timeliness. This is corroborated by several student statements as follows. "Previously, the subjects' knowledge about the material was minimum. However, after the material was delivered, their knowledge was more extensive" (Student 1). " … already good in learning, it's just because we don't know too much about this subject so it's just constrained" (Student 2).
To evaluate students' understanding in implementing the Literacy and Numeracy Course, the Teaching Lecturer tries to design the following questions: (1) what is the meaning of numerical literacy? (2) What is the concept of language as the basis of literacy? (3) What is the concept of Mathematics as the basis of numeration? (4) List and explain several government regulations regarding the legal basis for implementing numerical literacy. (5) How is the implementation of the Pancasila Student Profile to strengthen Numerical Literacy? (6) How is the implementation of numerical literacy in Schools? (7) How is numerical literacy implemented in the Family? (8) How is the implementation of numerical literacy in Society? (9) Some story questions for understanding the concept of reading; (10) Some questions to read tables and graphs to understand the concept of analysis From the description above, it is deemed necessary to have program mapping that can be used as a reference for Faculty leaders in determining the types of requirements needed. This is under the results of research from Rasdi, which explains that universities are required to always improve the quality of management in providing services to stakeholders related to academic and non-academic services (Ekosiswoyo, 2015). So the hope is that if the implementation is designed to the maximum, the learning process will be very good.

The Aspect of Process
Based on the results of observations made by students in their learning, they are required to be active in discussion sessions where the discussion session is initiated by each group presenting the results of their paper. After that, it was followed by a question and answer session and a discussion by the students. During the session, several students asked questions and responded to the answers results based on the students' thoughts. However, in the discussion session of the 19 students who took part in the lecture, only five students actively asked and provided feedback regarding the material presented by the group. Hence, the lecturer then encouraged his students to have continuous discussions.
After students make presentations and discussion sessions, the new lecturer explains the material by inserting questions for students in the material so that an active learning atmosphere is built for both the lecturer and the students. After the lecturer explained the material, the discussion session was reopened, where the lecturer also answered questions and statements submitted by his students several times in the question and answer session and discussion during the presentation of group papers. After the material was given and the question and answer session, and the discussion ended, the lecturer reminded them it was their turn to present group papers at the next meeting. In addition, lecturers also provide advice and words of encouragement to students.

The Aspect of Results
Regarding the changes experienced by students before and after being given the Literacy and Numeracy course, namely what they did not know at all related to the understanding of literacy and numeracy, these students gradually understood the concepts and applications of literacy and numeracy. It is necessary to increase activities loaded with numeration content to strengthen students' numeracy skills (Kemendikbudristek, 2021). Literacy skills, in general, and numeracy literacy, in particular, do not only have an impact on individuals but also society and the nation and state. Literacy skills contribute to social and economic growth and welfare (Kemendikbud, 2017).

CONCLUSION
Based on the evaluation data analysis results, it can be concluded that implementing the learning process in the Literacy and Numeracy course is not optimal. Implementing the learning process in Literacy and Numeracy courses needs to be carefully designed by considering the planning, implementation, and evaluation processes. This follows applicable regulations: Internal Quality Assurance must consist of planned, implemented, and evaluated processes (Dikti-RI, 2016). In addition, the correspondence in this study is still in doubt about the perception of Literacy and Numeracy when juxtaposed with the concepts of reading and arithmetic.
In the first step, the lecturer must make a mature lesson plan by discussing with lecturers with relevant experience with the subject being pardoned. The output of this planning is the formation of a Semester Learning Plan (RPS) which can later be used as a reference for students attending lectures. In the second step, the lecturer must be able to carry out the learning process following the RPS that has been compiled. Furthermore, because Literacy and Numeracy are still lacking in references, lecturers can also provide materials and assignments based on case studies of the concept of Literacy and Numeracy so that the learning process can synergize the theoretical concepts of Literacy and Numeracy with good practices in the field regarding Literacy and Numeracy. The third or final step needs to be a reflection activity on all activities carried out and an evaluation process by the lecturers and from study programs and faculties.
Current research studies on Literacy and Numeracy only explain how Literacy and Numeracy work in the learning process in educational units, both in the Kindergarten and Elementary School domains. So this research hopes to provide a broad view of the concepts of Literacy and Numeracy for prospective teachers before they can actively contribute to becoming kindergarten and elementary school teachers. Based on this, it is necessary to have further research on how the teacher implements to explain concepts and provides Literacy and Numeration practices to students. So that the implementation of Literacy and Numeracy can make a real contribution to students' lives.
The advice given is based on the results of the evaluation, namely in terms of the study program, it is necessary to have special training for lecturers in the subject of the study program, which is seen from the competence of the lecturer, making evaluation instruments to measure the learning effectiveness of the study program. Then from the point of view, the lecturers who support the subject must prepare for the needs to carry out learning, including the SLP as a reference in the learning process and so that the learning can be known and systematically takes place.

ACKNOWLEDGMENT
Researchers would like to thank all students of the Early Childhood Education Teacher Education Study Program at Muhammadiyah University Jakarta who has participated in this research. In addition, thanks are also conveyed to all lecturers for their permission and assistance, both moral and material, which made it possible to complete this article. Finally, the researcher would like to thank the Al-Athfaal Journal for receiving and publishing this article.