Prospective physics teacher’s ability in developing HOTS questions: CRI analysis

Higher order thinking skills (HOTS) are very important skills to be taught to students in the 21 century. The role of teachers is very important in improving students' high-level skills. This study aims to determine whether or not prospective physics teachers have misconceptions in compiling HOTS cognitive questions. The emergence of misconceptions shows that the ability of prospective physics teachers is still low in composing HOTS cognitive questions. The research subjects were all final semester prospective physics teachers who took the learning evaluation course at the Universitas PGRI Kanjuruhan Malang in the even semester of 2019/2020. The data collection technique is documentation. The data is taken from the final project document in the learning evaluation course, where prospective physics teachers are asked to make cognitive questions from levels C1 to C6 and are asked to state their beliefs about the questions they have made. The data were analyzed in the descriptive qualitative way to determine the misconceptions of prospective physics teachers. The results showed that prospective physics teachers were not yet able to compile HOTS cognitive questions. Recommendations for the preparation of HOTS questions are also given in this paper.


Introduction
Higher order thinking skill (HOTS) is an important 21st century skill taught in schools to adapt to the rapid development of technology (Driana & Ernawati, 2019;Ichsan et al., 2019;Ongardwanich et al., 2015;Schulz & FitzPatrick, 2016; C. K. S. Singh & Marappan, 2020). In general, teachers agree that HOTS learning is important to be taught to students at all levels (Kim et al., 2019;Schulz & FitzPatrick, 2016;Tyas et al., 2019). However, several studies have shown that teachers have difficulty in compiling HOTS questions (Driana & Ernawati, 2019;Ichsan et al., 2019;Jensen et al., 2014;Nurhayani et al., 2018). 21st century skills include critical thinking competencies, problem solving, creative thinking, meta-cognition, communication, digital and literacy technology (Kim et al., 2019). High order thinking thinking (HOTS) is an important component for successfully facing the development of the 21st century. Thus, HOTS must be considered and taught at every level of school.
According to Bloom's taxonomy revised by Anderson et al. (2001), cognitive processes are a continuum starting from the lowest to the highest, namely remembering, understanding, applying, analyzing, evaluating, and creating. Bloom's taxonomy is presented in Figure 1. Remembering, understanding, and applying are categorized as lower-order thinking skills (LOTS), while analyzing, evaluating, and creating are categorized as higher-order thinking skills (HOTS). HOTS consists of critical thinking skills, namely analyzing and evaluating, and creative thinking or creating. The ability of teachers in teaching is one of the most important factors in determining student success (Bal-Taştan et al., 2018;Chinedu et al., 2015). The ability of teachers determines the learning used, whether it is teacher centered or student centered. Teacher centered learning encourages the absorption of information only (memorization) while student centered learning encourages students to construct knowledge (Lak et al., 2017).
High-level skills must be integrated into learning elements (Schulz & FitzPatrick, 2016) by applying problem-solving methods that teach students to think critically and creatively. High-level skills are very important to be taught to students since only people who have the knowledge and skills will be able to successfully adapt to new situations.
The ability or skills of teachers in teaching are very important to become a professional teacher. The teacher's ability to teach is developed since college. Various knowledge and skills are taught to prospective physics teacher students, one of which is in the Learning Evaluation course. One of the learning achievements of the Learning Evaluation course requires the prospective physics teachers to compile cognitive aspect questions from C1 to C6 in the form of multiple choice. Previously, we have been taught how to arrange HOTS questions and provide examples of HOTS questions. The ability of prospective physics teachers in preparing HOTS questions can be recognized through instruments in the form of cognitive questions they made.
This study aims to determine whether or not there is a misconception of prospective physics teachers in the Physics Education Study Program of Universitas Kanjuruhan Malang when composing cognitive questions from C1 to C6, especially emphasizing HOTS questions. The existence of misconceptions shows that the ability of prospective physics teachers in preparing HOTS questions is still low. This study also analyzes the misconceptions experienced by prospective physics teachers at each HOTS cognitive level, then provides recommendations for the preparation of HOTS questions according to their cognitive level. Giving recommendations can be used as a reference to solve the difficulties of prospective physics teachers and teachers in compiling HOTS questions.

Method
This type of research is descriptive qualitative. The research subjects were all prospective physics teachers who took the Learning Evaluation course at the PGRI Kanjuruhan University Malang in the even semester of 2019/2020 with a total of 40 people. The data collection technique is done by using documentation technique. The data are in the form of documents obtained from the final project of the Learning Evaluation course. The final project for the Learning Evaluation course asks subjects to compile cognitive questions at levels C1 to C6 based on Bloom's revised taxonomic criteria, then subjects are asked to write down their level of confidence in the questions they have made using the CRI (Certainty of Response Index) scale.
Data analysis was carried out in a descriptive qualitative way. The data were analyzed based on the subject's answers to the questions he/she had compiled in two stages. The first stage was carried out by analyzing whether the questions that had been prepared by the subject were correct according to their cognitive level, then analyzing the level of confidence of the subjects on the questions that had been prepared using the CRI scale. The CRI scale used refers to the scale compiled by Hasan et al. (1999) and is presented in Table 1. The CRI scale uses six scales from 0 to 5 with 0 indicating a very low level of confidence. The subject answered the question by guessing the answer in total. The number 5 shows the subject's level of confidence is very high, without any guesswork at all. Total guess the answer 1 Almost guess 2 Not sure 3 Sure 4 Almost certain 5 Certain The second stage is to identify the occurrence of misconceptions. Misconceptions are recognized from the average CRI score of each respondent, which is presented in Table 2. Table 2 shows the four possible combinations of answers (right or wrong) and CRI (high or low) for each subject individually in compiling the questions. Correct answers with low CRI indicate not knowing the concept (Tidak Tahu Konsep -TTK), correct answers with high CRI indicate high concept mastery (Penguasaan Konsep -PK), wrong answers with low CRI indicate not knowing the concept (Tidak Tahu Konsep -TTK), and wrong answers with high CRI indicate misconceptions (Miskonsepsi -MK).  (Hasan et al., 1999)

Results
The results of data analysis on the average confidence of prospective physics teachers on the questions they made are presented in Table 3 which informs the CRI scores of the six question categories, namely from C1 to C6. The CRI value for all questions which is greater than 2.5 indicates that the CRI value is high. The high CRI value will reduce two possibilities for the low CRI value. The results of data analysis on the comparison of true and false questions with CRI values at the level of questions from C1 to C6 are presented in Table 4 answer which means understanding the concept (PK) and the wrong answer which means there is a misconception (MK). Understanding the concept shows that prospective physics teachers are already able to compose HOTS questions, while misconceptions indicate that the ability of prospective physics teachers in preparing HOTS questions is still low.  Table 4 shows that there are no misconceptions about the understanding of prospective physics teachers in compiling cognitive questions at C1 and C2 levels. About 5% of prospective physics teachers experiencing misconceptions or not being able to compose cognitive questions at C3 level. This means that all prospective physics teachers are able to arrange the LOTS questions correctly. About 37.5% of prospective physics teachers experienced misconceptions in composing C4 level cognitive questions. 90% of prospective physics teachers experience misconceptions about the preparation of cognitive questions at level C5, and 100% of prospective physics teachers experience misconceptions at level C6. Table 2 shows that prospective physics teachers have been able to arrange cognitive questions at the C1 and C3 levels. For questions with the C4 cognitive level, 67.5% of prospective physics teachers already understood the concept or were able to compile the questions correctly, and 37.5% experienced misconceptions, or were not able to compile C4 level cognitive questions. Examples of the misconceptions in the preparation of C4 level questions are presented in Table 5.  Table 5 shows that the prospective physics teachers are correct in determining that the indicator for the cognitive aspect of level C4 is analyzing. According to Bloom, indicators of cognitive aspects at the C4 level include analyzing, measuring, selecting, and testing. Prospective physics teachers experience misconceptions because they believe that the questions they make are in accordance with the C4 level. The question asks students to name the factors that affect pressure, meanwhile, the indicator of the question is to mention, which is an indicator of the cognitive aspects of the C1 level.

Discussion
Errors in level C4 questions prepared by the prospective physics teachers can be corrected to match the C4 level or analysis level by providing a stimulus in the form of graphs or story problems. Stimulus in the form of a graph can be given by adding a graph of the relationship between pressure and cross-sectional area, then students are asked to analyze based on the graph to show how the relationship between pressure and cross-sectional area is. An example of a stimulus in the form of graphs is presented in Figure 2 and an example of a stimulus in the form of a story problem/ problem solving is presented in Figure 3.  For questions with a C5 cognitive level, 10% of prospective physics teachers understand the concept or are able to arrange C5 questions correctly, and 90% have misconceptions, or have not been able to compose cognitive questions at the C5 level. An example of misconception about level C5 is presented in Table 6. Table 6. An example of Misconception in the preparation of cognitive question at C5 level

Indicator
Question Level Students can conclude between nails and ships which are the implementation of Archimedes' law Ships and nails are objects that are both made of iron, but ships are much larger than nails. Large ships can float in the ocean. This can happen because... a. The density of nails is greater than the density of the ship. b. The mass of the ship is lighter than nails. c. The upward force experienced by the nails is smaller than that of the ship. d. The upward lift received by the ship is greater than the weight of the ship, and vice versa. e. The density of the nails is smaller than the density of the ship. Table 6 states that in compiling cognitive questions at the C5 level, the prospective physics teacher is correct when it is viewed from the choice of operational verbs, namely concluding. According to Bloom, indicators of cognitive aspects at the C5 level include concluding, assessing, criticizing, and deciding. The misconception occurs because the prospective physics teacher believes that the questions made are questions with the C5 level. In fact, in this question, students must be able to analyze the effect of volume and cross-sectional area on Archimedes Law, which is a C4 level cognitive problem.

C5
Developing or evaluating level C5 questions needs to pay attention to the thinking processes of students when solving the problems in these questions. When students are faced with a problem with an evaluating level, of course the question also requires students to remember, understand, apply, and analyze, so that students can evaluate. Examples of questions at the level of evaluating are given in Figure 4. The process of thinking of students when solving these questions is that students must understand Archimedes' Law, what it contains, and how the variables affect it. After Rina was given the task to investigate the effect of cross-sectional area on pressure. Rina has a ruler for an isosceles triangle. How is the position of the ruler against the table so that the effect of pressure on the table is the greatest?
The following is a graph of the relationship between pressure (p) and cross-sectional area (A). Which of the following is true about the effect of cross-sectional area on pressure?
a. The greater the pressure, the greater the cross-sectional area b. The lower the pressure the greater the cross-sectional area c. The wider the cross-sectional area the lower the pressure d. The smaller the cross-sectional area the greater the pressure. that, students can analyze the experiments conducted by Budi and Ahmad so that they can give an assessment of who did the experiment correctly. Cognitive questions at levels C4 and C5 are included in critical thinking. Critical thinking includes the skills of reasoning, investigating, comparing and connecting, finding complexity, and exploring points of view (Changwong et al., 2018;Yusoff & Seman, 2018). Critical thinking can be developed when students are faced with problems and have to think of solutions and evaluate the problem-solving process.
For questions with cognitive level C6, 100% of prospective physics teachers have misconceptions, or there are no prospective physics teachers who are able to compile cognitive questions at level C6. The example of misconception about cognitive question at level C6 is presented in Table 7. In the question, the prospective physics teacher is correct in placing the indicator which is in accordance with operational verbs for level C6, which show creative thinking skills. Nevertheless, the question is an implementation of Ohm's Law and is still classified as level C3 question. The results of this study also indicate that prospective teachers have not been able to compile questions for creative thinking skills. An ampere meter has a resistance within 0.9 ohms and a maximum measuring limit of 100 mA. In order for the ampere meter to be used to measure 1A current, it is necessary to install a resistor of ... a. 0,1 ohm in series b. 0,1 ohm in parallel c. 0,1 ohm in series and parallel d. 1 ohm in series C6 Cognitive questions at the highest level, namely creating, will be more visible if the questions are in the form of essays when compared to multiple choice questions. Based on the answer, it can be seen the students' creative thinking process, whether there is something new that is offered by the students to solve the problem. Creative questions are the ones that generate creative and innovative ideas or creations (Supardi U.S., 2015;Yudha et al., 2018).
The results of the analysis of the questions on cognitive aspects compiled by prospective physics teachers at the Universitas PGRI Kanjuruhan Malang showed that they did not experience any misconceptions about preparing cognitive questions at the C1-C3 level. This means that the ability of prospective physics teachers in compiling LOTS questions is good. The results of this study are also supported by research from Driana & Ernawati (2019); Hartini et al. (2020);and McNeill et al. (2012), which shows that teachers still tend to compile LOTS questions. Questions that only focus on the LOTS will not grow students' skills in critical thinking (Jensen et al., 2014).
Budi and Ahmad conducted an experiment on the effect of the weight of an object immersed in water on buoyancy to apply Archimedes' Law. Budi and Ahmad have logs with masses of 100 gr, 500 gr and 1000 gr. Budi and Achmad were asked to choose one of the wooden blocks that caused the greatest buoyancy. Budi chose wood blocks with a mass of 100 gr, while Achmad chose wood blocks with a mass of 1000 gr. Which block, between Budi and Achmad's choices, is correct and why? a. Budi's, because the smaller the object's mass, the greater the buoyant force. b. Ahmad's, because the greater the object's mass, the greater the buoyant force. c. Both Budi's and Ahmad's are correct because the logs are floating. d. Budi and Ahmad are not correct because the greatest buoyant force is the block with a mass of 500 gr. The results of data analysis also showed that there was a misconception of prospective physics teachers in compiling cognitive questions from C4-C6 or HOTS questions. This means that prospective physics teachers have not been able to compile HOTS questions. Most of the prospective physics teachers have been able to compile questions at the level of analysis but have not been able to compile questions at the level of evaluation and creation. This result is also reinforced by Driana and Ernawati (2019) research, in which teachers are only able to compile analysis questions.
The results of this study indicate that prospective physics teachers experience misconceptions in the preparation of high-level cognitive questions. This illustrates that prospective physics teachers are still not able to compile high-level cognitive questions. Other research results also state that teachers still misunderstand higher order thinking skill (Retnawati et al., 2018;Tyas et al., 2019).
Weak understanding of prospective physics teachers in compiling HOTS questions needs to get serious attention and be solved immediately, because students' high-level thinking is a positive contribution from the teacher's role in learning (Kim et al., 2019). Teaching higher order thinking is very important (Schulz & FitzPatrick, 2016) because today's world is influenced by technology and full of problems (Newbill & Baum, 2012). Higher order thinking skills enable students to be ready to face world competition (Appanna et al., 2018;R. K. A. Singh et al., 2017).
Higher order thinking cannot be obtained by students automatically through learning (Afandi et al., 2019;Schulz & FitzPatrick, 2016), but is obtained through a deliberate and repetitive practice process (Changwong et al., 2018;Yusoff & Seman, 2018). Therefore, it is very important to emphasize that teachers and prospective teachers are able to prepare HOTS questions.
HOTS questions are usually given a stimulus in the form of text, images, graphics, tables, which will be linked to the questions. Stimulus can also be in the form of problems that students encounter daily and story questions. Stimulus directs students to connect knowledge in different situations and can be connected with real situations in everyday life (Widana, 2018). The stimulus in the questions must be interesting and can encourage students to read (Maifajir et al., 2019). Stimulus distinguishes HOTS and LOTS questions.

Conclusion
Based on the results of research and discussion, it can be concluded that the prospective physics teacher in the Physics Education Study Program of Universitas Kanjuruhan Malang has not been able to compile cognitive questions in the higher order thinking skill (HOTS) category. The inability of prospective physics teachers to compile high-level cognitive questions needs serious attention. This can cause students to be less able to solve problems so they have not been able to compete with other countries in the industrial era 4.0. The first recommendation given to lecturers, teachers, and prospective teachers in compiling HOTS questions is to provide basic questions or stimuli to the prepared questions. The stimulus for compiling physics problems can be in the form of graphs, pictures, story problems, cases for problem solving, videos, tables, and experiments. The second recommendation when composing HOTS questions is to pay attention to students' thought processes. If the question asks students to evaluate, it means that students' thinking processes when solving questions must involve the cognitive level below them, namely from remembering to evaluating. The third recommendation is for the preparation of the highest/creative level of questions that is better if they are in the form of essay. The answers of essay questions will show students' creative thinking processes more clearly when compared to multiple-choice questions.