Designing PISA-like mathematics problem in covid-19 pandemic (PISAComat)

A new policy demands PISA-like mathematics problems given to improve knowledge of literacy and numeracy as well as life skills in the Covid-19 pandemic. However, this didactic phenomenon has not been utilized as a context in learning. This research was design research with the type of development studies that aimed to produce PISAComat, which was valid, practical, and had potential effects on secondary school students’ mathematical literacy abilities by conducting walk-through, test, document review, and interview as data resources. From the result of expert reviews and one to one phase, the developed PISAComat has been valid. It is viewed from its compatibility with the PISA framework i.e., students are familiar with the current context, the content is indeed studied at the 9th grade, and the difficulty level of 5 in PISA. The language used is precise and does not contain a variety of meanings. The practicality can be seen from students who can understand and solve the problem given easily by involving students’ mathematical literacy. Also, the PISAComat had potential effects on mathematics literacy, such as reasoning and argumentation, to give perception and interpretation and to improve insights related to the education of life skills to face the Covid-19 pandemic.


Introduction
Indonesian Minister of Education and Culture (MOEC) has promoted the new breakthrough with some major changes in the 2013 curriculum through four educational policy programs dubbed "Merdeka Belajar" or freedom of learning. The boldest of the four is the termination of the national examination starting from 2021 will be changed to the assessment of minimum competency (AMC) and survey of character which will be valid for students in the middle level, i.e. grade 4, 8, 11 [1,2]. AMC aims to map the competence of schools and regions based on the minimum competencies which concern on literacy and numeracy based on good educational practices at the international level such as PISA and TIMSS [3]. The term "numeracy" has been quite closely related to the ability of mathematical literacy in PISA as the fundamental mathematical capabilities to formulate, employ and interpret mathematics in variety context to describe, predict and explain phenomena, recognize the role of mathematics play in real life problems [4,5].
Numeracy is included in uncertainty and data content which is one of the four contents evaluated by PISA [6,7]. Learning uncertainty and data is essential for students as this topic is a heart in terms of mathematical analysis for various situations related to probability and statistics which is utilized as technique of describing and representing data [7]. However, Indonesia fared poorly mathematics performance in 2018 PISA with the score of 379, far below OECD average score of 489 [6]. Some evidence were indicated to clarify the factors influencing the instability of students' performance, such as students were lack to solve context-based mathematics problem [8]; teachers could only provide some IOP Publishing doi:10.1088/1742-6596/1657/1/012057 2 material and exercise on routine problems at low level [9,10]; teachers' difficulties on designing PISAlike mathematics task regarding to the authenticity and language structure [11]; and inadequacy of learning resources provided by Indonesian typical mathematics textbook to support the use of contextbased-PISA mathematics task [12].
Indonesian students need to be familiar with mathematics problems like PISA to improve their performance at the international level. One potential effort that can be done by developing problem which has the characteristic of PISA and adapting them to the local context so they are well-known among students [13]. The learning by utilizing context can escalate mathematical concept, the learning process allows students to create abstract mathematical concept which is transformed into the form of easily-understood representation [14][15][16][17]. A learning approach emphasizing the context or situation of problems in daily life is PMRI. The mathematics problems were set to be in the form of real life and well known to students in order to be used as starting point in learning so that students can engage themselves directly in building their own knowledge through their prior knowledge and represent it into the form that is easy to be understood [18][19][20][21].
One of the interesting contexts for the implementation of mathematics learning is the phenomenon of Covid-19 outbreak, the pandemic for the whole world [22]. The Covid-19 pandemic also spread throughout Indonesia and has been declared as a national disaster [23]. In line with it, the Indonesian MOEC instructed the direct learning amid of Covid-19 pandemic should focus on education in life skills to face of Covid-19 [24].
This research is related to the previous studies in developing PISA-like problems with focus on examining local context of Bangka [25], sport branches of Asian Games 2018 [26], concerning on level and one content [27], and attracting students to learn and enhance mathematical literacy: reasoning and representation [28], communication, mathematization and representation [29,30]. However, there have not been any PISA-like mathematics problems using the didactic phenomenon of Covid-19 pandemic. That is, there is a gap between desired demands and conditions in the field (school). Therefore, researchers are interested in developing mathematics problem with the characteristic of PISA using the context of Covid-19 Pandemic (PISAComat) on the content of uncertainty and data which aimed to produce valid and practical PISAComat and had a potential effect on mathematics literacy.

Method
Researchers applied design research method with type of development studies as the main framework [32,33]. The framework phases consisted of preliminary and formative evaluation. Some activities undertaken during the preliminary phase: examining the current curriculum, reviewing PISA problems as a reference to produce the draft of PISAComat and designing initial instruments which consist of lattices, question card, scoring guideline based on PISA 2018 framework.
The formative evaluation was done by emphasizing the prototyping process including selfevaluation, expert review, one to one, small group, and field test. At the self-evaluation phase, researchers examined and evaluated the initial instruments i.e. lattices, question card, and score guideline regard on content, construct, and language. After that, the initial instruments were given to several experts and one to one at once. The validation process was carried out through two ways: mail reviews by sending the prototype to the lecturers of mathematics education at UIN Raden Fatah Palembang along with colleagues who have experience in developing PISA-like problems; panel review was attended by two mathematics teachers. Along with it, one to one performed by involving three students with various capabilities. This phase focuses the clarity and legibility of the PISAComat from the comments given by students and examines students' difficulties in solving the PISAComat in order to revise for improvement.
After revising the prototype, the PISAComat was evaluated by involving six students (2 students of high category; 2 students of medium category; 2 students of low category) at small group phase. Suggestions and comments were taken into consideration to produce prototype III which is valid and practical. Then, the field test phase was conducted by involving 32 students of secondary school number 4 Palembang in online learning through WhatsApp group. A walkthrough, test, document review, and interview were used as data resources which analyzed the result of each step from the phases descriptively.

Result and Discussion
In preliminary, the researchers along with mathematics teachers took several activities: grouping the initial students' ability as research subject, comparing the competencies that exist on uncertainty and data content in PISA with probability and statistics material that the 9 th graders learned in the curriculum, designing PISAComat concerning the existing PISA items which are the 2009 PISA items included in the content of uncertainty and data at level 5 with the context of robberies. The problem tends to ask students to use their reasoning and argument from the data given. The PISAComat which was designed adapted from 2009 PISA item which is students are asked to analyze the data provided by TV Broadcaster about the increasing number of positive cases infected Covid-19 in the United States. Then, students are asked to give their argument and reasoning about the TV broadcaster's statement. To solve this problem, the tendency in viewing data is needed before concluding. PISAComat was categorized difficulty level of 5. Before being piloted to the formative evaluation phase, the PISAcomat included lattices, question card, and score guidelines was examined and produced the developed PISAComat can be seen in Figure 1.  Figure 1 shows the initial prototype which was produced from the preliminary phase. To obtain the validity of the developed PISAComat, expert reviews and one to one phase were carried out at once. The initial prototype was sent to the experts via email to obtain comments and constructive suggestions. The clarity and legibility based on language and figural display were assessed by three students (different abilities) through zoom meeting (online). The reviewed results from both experts and one to one were then simultaneously decided to revise the developed PISAComat. Process of changing the PISAComat generated by the expert reviews and students is displayed in Table 1.

Validations
Comments/Suggestions Revision -The word selection needs to be considered, i.e. "epicenter" not all students understand the meaning of this word -The question is not clear, change it into "Do you agree with the TV Broadcaster's statement? Give an explanation to support your answer" -Add the captured image source The word "epicenter" has been removed and the question has been adjusted to the picture given Source has been added Students -I do not understand what is the meaning of TV Broadcaster's statement especially confused with the word of epicenter The statement was changed and adjusted to the picture given These comments and suggestions from experts, responses, and students' proficiency when solving the problem which made the PISAComat has been categorized valid [18]. Based on the content, the PISAComat is adapted to PISA items and the context used also follows the current situation of Covid-19 pandemic. In the 2013 curriculum, the problem given included probability and statistics lesson which was studied at the 9 th grade. The PISAComat followed the PISA framework characterized by the difficulty level of 5 in PISA and could be mastered by the 9 th graders. There is a problem on the selection of word which is difficult to be understood by students, so the word was changed into the word is easy to understand, does not contains double meanings.
At small group phase, six students were involved to work on it individually to see practicality the developed PISAComat. The result indicated that the developed PISAComat can be easily understood by students and well solved. Students can understand the problem by giving arguments and reasoning about the phenomenon given. There is no change in this phase but a little error of type that needs to be corrected. Furthermore, the revised result produced the prototype III that have been practical. As produced by Zulkardi [34], the small group result reflected the practicality of the developed problem where the given problem could be understood easily and interpreted well by the students.

Figure 2. The developed PISAComat after revision
Then, prototype III was tested at the field test phase which involved 32 students to explore the potential effects in detail on mathematics literacy. From the students' solutions, almost all students could  shows that EPA disagrees with the reporter's statement who said that a major increase in the spread of Covid-19 in the United States, it is because the reporter only display a small amount of data and it cannot be concluded that the United States has a major increase in the spread of Covid-19. According to IRS's opinion who agreed with the reporter's statement which taken only 2 days for boosting the positive patients of Covid-19 twice, in the next 2 days, it reached 100.000 cases. IRS added from the website that he has accessed; the United States was in the top position with more than 1 million cases exceeded China. In contrast with IRS, NDA disagrees with the reporter's statement. This is because IRS calculates the increase the number of a positive patient by looking for the difference every two consecutive days which is showing from on March 24 th until 26 th there was an increasing number of positive cases of covid-19, while on March 26 th until 28 th there was a decreasing number of positive cases of Covid-19.
From the students' solutions, majority students can answer the problem given and have the same opinion as to the reporter's statement because they only see from the data displayed on the plasma TV without considering or finding out in advance the prior and subsequent data of the United States. From  Figure 3, it appears that the students were able to involve reasoning and argumentation abilities in providing the perception and interpretation of the given data. In line with the research result [27][28][29] stated that students can understand, formulate, and solve the problem correctly by providing good reasoning and argumentation.
Also, the interviews showed the PISAComat could make students happy in completing them and could offer the ability of mathematical literacy of students including reasoning and argumentation. This was because of the context given from the current situation and it was interesting to learn so that problem given also allowed them to convey their reasoning and its argumentation. In line with that, the students felt excited and interested in solving PISA-like mathematics problems so that students can involve their mathematical literacy ability [30,31]. Therefore, students need to be trained in solving PISAComat to optimize students' mathematical literacy ability and improve insights related to the education of life skills to face Covid-19 pandemic.

Conclusion
In sum, the development process has produced PISAComat which categorized valid, practical, and had potential effects on mathematics literacy. Process of changing the PISAComat generated by the expert reviews and one to one phase reflects the validity of PISAComat, i.e. the developed PISAComat is adapted to PISA item and the context used also follows the current situation of Covid-19 pandemic, the problem given included probability and statistics lesson which was studied at the 9 th grade; in terms of construct, the developed PISAComat followed the PISA framework characterized by the difficulty level of 5 in PISA and could be mastered by the 9 th graders; the developed PISAComat was precise, using words that are easy to understand, does not contain variety meanings. The practicality can be seen from small group phase which is students can understand and solve the developed PISAComat easily involved their mathematical literacy abilities by giving arguments and reasoning about the phenomenon given. Besides, the developed PISAComat can promote students' mathematics literacy by involving reasoning and argumentation abilities and providing the perception and interpretation of the given phenomena. Also, the developed PISAComat could make students excited and interested in solving the problem given because of the context given from the current situation and allowed to convey answers based on their perception and interpretation as well as improve insights related to the education of life skills to face Covid-19 pandemic.

Acknowledgments
Part of this paper was supported by Directorate General of Higher Education through Master's Education towards a Superior Doctor of Bachelor (PMDSU) grant in 2020/2021. Then, we thank Ms. Luthfiah and her students who participated in this research.