Analysis of Students’ Errors in Solving Higher Order Thinking Skills (HOTS) Problems for the Topic of Fraction

Problem-solving is an activity that can generate Higher Order Thinking Skills (HOTS) among students. However, only some of the students are capable of solving problems and some are having difficulties. The difficulties have caused students to make various kinds of errors. Hence, this study conducted is to identify and analyze students’ errors in solving problems that involve HOTS in the topic of Fraction. The samples consisted of 96 Form One students of a secondary school in Johor. The instrument of the study consisted of a set of test questions that contained four HOTS questions. The errors were analyzed according to Newman’s Error Analysis Model consisting of reading, comprehension, transformation, process skills and encoding. The results showed that students frequently made errors in encoding (27.58%), followed by process skills (27.33%), transformation (24.17%) and comprehension (20.92%). The findings showed that students faced problems to correlate the information and implementation of strategies used in solving mathematical problems involving HOTS.


Introduction
One of the teaching strategies that promote HOTS is problem-solving. Problem-solving is an activity that involves various actions in the mind of thought including accessing and using knowledge and experience (Lester & Kehle, 2003). Thus, teaching strategies that involve the use of non-routine problems in the classroom give students the opportunity to develop higher order thinking skills in the process of understanding, exploration and application of mathematical concepts (Polya, 1973). Since problem-solving involves the activities of processing, the knowledge used in the process of solving the problems is different. As stated by Mayer (1982Mayer ( , 1987, the knowledge covers a) knowledge of language and facts, b) knowledge of schemes, c) knowledge of algorithms, and d) strategic knowledge. In short, students need to equip themselves with various knowledge and high skills in problem-solving, which involves higher order thinking.
In the process of problem-solving, students must go through several phases before getting the final answer. According to Polya (1973), the problem-solving process has four phases, which are: a) understand the problem, b) plan the strategy, c) execute the plans and d) review the answers. Meanwhile, Newman (1977) stated that there are five phases in problem-solving, namely a) reading, b) comprehension, c) transformation, d) process skills and e) encoding. Guided by the phases described by Polya and Newman, students can solve problems more easily and systematically even if they are given problems with various levels of difficulties. However, not all students are able to solve the problems as they are having difficulties in the specific phases. Based on the study by Effandi and Siti Mistima (2010), students have difficulties in the transformation and process skills phases while solving problems of Quadratic Equations. Students' difficulties in problem-solving process have caused them to make various errors. Quoting Widiharto (2008), mathematical concepts and skills that are not fully mastered by students have led to difficulties and errors in solving mathematical problems. A study conducted by Susanti et al. (2014) found out that students are difficult to solve problems that involve the use of HOTS and among the difficulties faced by them are a) reading and interpreting data, b) determining and delegating data and c) making conclusions and arguments. Therefore, teachers need to realize the difficulties and errors encountered by students and take appropriate approaches to improve their teaching practices (Ashlock, 2005). In conclusion, teachers are responsible to identify in advance the difficulties in learning mathematics, which led to errors before teaching other mathematical topics.
In Malaysia, before changing to the School-Based Assessment system, Malaysia adopted the conventional assessment where it relied solely on examination alone to assess the progress and performance of the students. Any error made by the students was not analyzed on "where" and "why" they made such errors. Through a study conducted by Trance (2013), students' errors in problem-solving related to algebra were analyzed according to Newman's Error Analysis (Newman, 1977). Students were given the questions related to algebra and they were asked to complete them orally, before they could show the calculation processes. The students should describe how they understand the issues and processes used to obtain the answers. The findings revealed that 70% of the errors found were comprehension and transformation. In order to solve the problems faced by students, they should be provided with remedial classes so the comprehension and transformation errors can be reduced. Therefore, this study was undertaken to analyze the types of errors that are often done by students in problem-solving involving HOTS in the topic of Fraction.

Fraction and Higher Order Thinking Skills (HOTS)
The education system in Malaysia has highlighted the aspects of thinking skills among students. However, the report of Malaysian Education Development Plan (PPPM) for 2013 to 2025 shows that the implementation of critical and creative thinking skills (CCTS) has failed to be implemented in the classroom. As a result, the performance of Malaysian students in the international assessments that test students' thinking skills, namely Trends in International Mathematics and Science Studies (TIMSS) and Programme for International Student Assessment (PISA) was not at the satisfactory level. Among the factors that caused the failures are the teaching methods that did not apply and focus on HOTS as laid out in the curriculum (Kementerian Pelajaran Malaysia, 2013). This clearly shows the average mathematics scores obtained by the students for TIMSS 1999 of 519, which had dropped sharply to 440 for TIMSS 2011 (Mullis et al., 2012). Meanwhile, the mathematical assessment scores obtained by the students for PISA 2012 were 421, of which the scores were below the OECD average scores of 494 (OECD, 2012).
There are two domains that are being tested in TIMSS assessments, namely content and cognitive domains. For the tested cognitive domain, it covers applying, analyzing, evaluating and creating, which also the components of higher-order are thinking in Bloom's Taxonomy. Meanwhile, the tested content domain covers four areas of mathematics learning namely Numbers, Geometry, Algebra, Data and Probability. Fraction is one of the topics contained in the Number domain. By looking at the average scores obtained by Malaysian students for the Number content domain for TIMSS 2007, the scores were 494 and for TIMSS 2011, the scores were 451, which indicate a sharp decline compared to other countries (Mullis et al., 2012). Meanwhile, the assessment of mathematics for PISA has four content categories, namely Quantity/Number; Data and Uncertainty; Change and Relationship; and Space and Shape. The questions tested in PISA involve the use of HOTS. Mathematics topics such as round numbers, sequences and patterns of numbers, fractions, decimals, percent and integers are components in the content of Quantity/Number. Based on the report of PISA 2012, the average scores obtained by Malaysian students for the content of Quantity/Number were 409, whereas the average scores were below the overall OECD average of 495. If we were to compare with other countries such as Vietnam (509), Thailand (419) and Singapore (569), Malaysia is the last in Southeast Asian countries. The PISA assessment questions which test the students also require HOTS, which is similar to the TIMSS assessment.

Newman's Error Analysis
The model proposed by Newman (1977) has been proven to be a reliable model for mathematics teachers to be used to classify and categorize students' errors (Prakitipong & Nakamura, 2006;Effandi Zakaria & Siti Mistima, 2010) in solving mathematical problems involving HOTS. According to Effandi and Siti Mistima (2010), the Newman's Error Analysis Model has the hierarchy that categorizes types of error based on the levels of problem-solving by the students. The above statement is in line with Ellerton and Clements (1996), who stated that Newman used the "hierarchy" as he gave the reason that students who fail at any level of problem-solving prevented them from getting the required solution. In the process of problem-solving, there are two factors that make students unable to produce the correct answer, namely a) problems in language fluency and conceptual understanding (reading and comprehension) and b) problems in mathematical processing (transformation, process skills and encoding) (Prakitipong & Nakamura, 2006). This clearly shows that students need to understand the meaning of the questions before going through the mathematical processing in order to produce the correct answers.
The Newman's Error Analysis Model is shown in Figure 1. The first type of error is reading, which the ability of students to read the mathematical problems given and to identify the sentences and mathematical symbols used. The second type of error is comprehension, which is the ability of students to understand the mathematical www.ccsenet.org/ass Asian Social Science Vol. 11, No. 21;2015 problems given. The next error is transformation, which sees the ability of students to choose the appropriate mathematical solution methods. The following error is process skills, where students can perform mathematics process correctly or not, and lastly the error of encoding, which is the ability of students to express the final answer. Figure 1. The Newman's error analysis model

Research Methodology
This study is a descriptive study that used a quantitative approach. The sample for this study consisted of 96 Form 1 students (13 years old) in a secondary school of the district of Mersing, Johor. The students were placed in three classes of different levels of achievement based on the results of the Primary School Evaluation Test (UPSR). The instrument used for this study was a set of test questions to identify the types of students' errors. The items contained in the instrument were built by the researcher and it has been adapted with the questions of international assessments of Trends in International Mathematics and Science Study (TIMSS). Four items were built in a subjective form, and the items contained the elements of HOTS which require students to use higher-order thinking as shown in Table 1. 4 P and Q represent two fractions on the numbered line above. P x Q = N. Where is the location of the N on the numbered line above?
The samples of the study were required to answer the questions contained in the instruments that have been prepared under the supervision of individual mathematics teachers who taught the respective class. The time allocated to answer questions was 20 minutes. As this study was a descriptive study, data for the types of error that are often made by students were scheduled based on the percentage. The errors were analyzed using 0 1 2 P Q www.ccsenet.org/ass Asian Social Science Vol. 11, No. 21;2015 Newman's Error Analysis (Newman, 1977) and classified into five types of errors namely, reading, comprehension, transformation, process skills and encoding.

Data Analysis
Background information for the study samples by gender and category of classes is shown in Table 2. Majority of the samples for this study were female students (53.1%) and the rest were males (46.9%). The samples consisted of students in three different classes namely 1 Rancangan Khas (Special Program) (34.4%), 1 Amanah (38.5%) and 1 Murni (27.1%), which achieved good, medium and low grades respectively.

Analysis of the Errors According to Newman's Type of Error
This section discusses the errors made by the students according to the procedure proposed by Newman (1977). Newman has listed five types of errors, namely reading, comprehension, transformation, process skills and encoding. For reading error, no error was recorded. Meanwhile, there were other types of errors made by the students, which include comprehension, transformation, process skills and encoding.

Comprehension Error
Comprehension error occurs when the students are able to read the questions but fail to understand the wants and needs. Examples of comprehension error are shown in Figure 2. The students were not able to interpret the questions and the strategies used to manipulate the question were also less precise, in which they failed to solve the following problem. There were also other errors recorded, namely transformation, process skills and encoding that were caused by comprehension error.

Transformation Error
Transformation error occurs when the students already understand the needs of the question but fail to identify the mathematical operations involved. Examples of transformation error are shown in Figure 3.  The students were able to read and understand the requirements needed for the question. However, they had difficulties in the transformation process as the operations used by them in solving this question were only addition and multiplication, but actually, there was another operation involved, which was subtraction.

Process Skills Error
Process skills error occurs when students fail to perform the procedure correctly. An example of process skills error is shown in Figure 4. Based on Figure 4, although the student was able to read and understand the question correctly and identify mathematical operations that should be used, the student made an error in the calculation procedure, where the final answer should be six kilograms and twenty five grams.

Encoding Error
Encoding error occurs when the students fail to write the desired answer correctly. An example of encoding error is shown in Figure 5. The student has failed to write the final answer correctly, where it was supposed to be 55 students play sepak takraw and badminton, while the student stated that 55 students only play badminton.   Table 3 shows the number of errors according to Newman's items and types of errors from the samples of the study in solving HOTS mathematics problems solving for the topic of Fraction. The highest number of errors made by the students was in item 4, which was 380 (31.68%) in various types of errors compared to other items. Meanwhile, the lowest number of errors made by the students was in item 1, which was 235 (19.58%) in various types of errors. In addition, encoding was the highest type of errors made by the students, which recorded the number of errors of 331 (27.58%). On the other hand, it can be seen that the types of common errors made by students were process skills, transformation and comprehension with the number of errors of 328 (27.33%), 290 (24.17%) and 251 (20.92%) respectively. Overall, the study found that the students almost made all types of Newman's errors although there was only one type of error that was not made by the students, which was reading.

Analysis of Students' Errors Based on Items, Newman's Error Analysis and Level of Achievement
Further analysis was conducted to identify the number of errors in solving HOTS problems in the topic of Fraction according to the level of students' achievements. As shown in Table 4, 288 (24%) errors were made by the students from the group of high achievement level, 537 (45%) errors were made by the students from the group that has medium achievement level and 372 (31%) errors were made by the students from the group with low achievement level. Table 4 also shows the total number of Newman's errors based on the items and levels of achievement made by the samples in solving HOTS mathematics problems for the topic of Fraction. For item 1, the error that was most often made by the students with high achievement was encoding with a total of 22 (1.83%). Next, the most often errors made by the students with medium achievement for item 1 were process skills and encoding, with a total of 29 (2.42%) and the most often errors made by the students with low achievement for item 1 were transformation, process skills and encoding, with a total of 26 (2.17%). For item 2, the errors that were most often made by the students with high achievement were process skills and encoding, with a total of 31 (2.58%). Next, the errors that were most often made by the students with medium achievement for item 2 were transformation, process skills and encoding with a total of 37 (3.08%). Meanwhile, the students www.ccsenet.org/ass Asian Social Science Vol. 11, No. 21;2015 with low achievement made almost all Newman's errors, namely comprehension, transformation, process skills and encoding with a total of 26 (2.17%).
Next in item 3, the error that was most often made by the students with high achievement was encoding with a total of 9 (0.75%). Next, the error that was most often made by the students with medium achievement for item 3 was encoding with a total of 36 (3%) and the errors that were most often made by the students with low achievement for item 3 were transformation, process skills and encoding, with a total of 20 (1.67%). For item 4, majority of the students with high, medium and low achievements had done almost all Newman's errors, with a total of 32 (2.67%), 37 (3.08%) and 26 (2.17%) respectively.

Discussions
Based on the classification criteria proposed by Newman (1977), it was found that a total of 1200 errors have been identified in the problem-solving process involving HOTS. Out of this total number of errors, it was found that a total of 251 (20.92%) errors were caused by the fluency factors (reading and comprehension) and 949 (79.08%) errors were caused by the mathematical processing factors (transformation, process skills and encoding). The results of the study are supported by the research conducted by previous researchers namely, Singh et al. (2010) and Ellerton and Clements (1996), who found that the errors caused by language fluency were the minor errors, and mathematical processing errors were the major errors made by students when answering exam questions.
Students frequently make errors in comprehension, transformation, process skills and encoding. The errors occur because the students fail to understand and explain what is required by the question. Most students did not manage to solve the problems of Fraction involving the use of HOTS. The results are consistent with the study done by Singh et al. (2010) in which the students fail to convert the mathematical problems into mathematical forms. The failures could be due to the teachings by teachers who do not emphasize on the comprehension and mathematical skills needed by a student and eventually affect them for failing to produce the required result. Therefore, teachers should ensure that each student master the mathematical concepts and basic skills before teaching the new topics. As stated by Jones et al. (1999), teaching of mathematics at an early stage should focus on the development of the concept before going to a higher level.
In the study conducted, process skills and encoding were the most common errors. The students used the wrong procedure, did not perform the calculation process carefully and wrongly applied the manipulation. Hence, the students could not state the final answer correctly due to the problems encountered. Newman (1977) and www.ccsenet.org/ass Asian Social Science Vol. 11, No. 21;2015Newman (1983 in Trance (2013) stated that in order to get the correct answer, there are some processes that need to be followed in sequence, namely reading, comprehension, transformation, process skills, and encoding. The statement is in line with Ellerton and Clements (1996), who stated that Newman used the word "hierarchy" as a failure at any level of problem-solving, namely reading, comprehension, transformation, process skills, and encoding, which prevent students to get the required answers. Hence, there are many errors in the process skills and encoding.
The levels of achievement contribute to the frequency of students making errors. The study found out that many errors were made by the students with low and medium levels of achievement. The students with medium level of achievement made the most errors for item 1, 2, and 3. This was due to the number of students with medium level of achievement which took the test were the highest compared to the number of students with low and high levels of achievement. Meanwhile for item 4, most students who made the errors were those in medium and low levels of achievement. This situation may be due to the differences in the level of thinking among students. According to Ismail (2010), students make errors in mathematics that are associated with the following characteristics; a) cognitive activities, b) metacognitive abilities, c) attitudes and d) the knowledge possessed by them. Various levels on the characteristics have led the different errors made by students and different abilities for them to solve the mathematical problems (Hoard et al., 2008). Problem-solving process is one of the cognitive strategies and skills that is carefully planned by individuals to achieve goals. Therefore, for students with low achievement level, they do not have clear plan and strategy to solve the problem. They are facing with difficult and challenging situations while solving problems. The situations are even more complicated when the students do not understand the given problems and cannot identify the mathematical operations involved.
The instrument provided in this study has the structure, manipulation and content that test the levels of HOTS among students. This has caused students fail to complete the given questions. Looking back at the instrument, the structure of the questions is arranged with the easy level to begin with and up to the complex level. This has caused many students to make errors on the final questions rather than the rest of the questions. For questions one to three, the level of difficulty is medium, where the students should have no difficulty in solving the given problems, and the operations involved are basic operations such as addition, subtraction, multiplication and division. As for question four, the level of difficulty is high, where the students need to link previous knowledge with the numbered line in order to obtain the solution. Question four also only involves variables that make students confused in answering the questions. As a result, students cannot convert the problems into mathematical forms. This finding agrees with the study conducted by Norasiah (2002) and Rahim (1997), in which the students have problems to convert mathematical problems into mathematical forms and they also have problems in understanding the specific terminologies that exist in mathematics. This failure may be due to the teachings by teachers, which lack of emphasis in understanding the language of mathematics. Therefore, teachers need to ensure that each student master the basic skills and mathematical terminologies before they learn other topics of mathematics.

Conclusion
Through the study conducted, it is found that students tend to make almost any type of errors found in Newman's Error Analysis, namely comprehension, transformation, process skills and encoding. This indicates that students have problems in interpreting mathematical problems, failed to devise a strategy and develop a strategic plan, which eventually led to errors in choosing the operations involved and failed to state the answers. Students should understand the problems well and have a thorough planning in order to achieve the transformation process. When devising mathematical solutions, students will review and recall all the information and knowledge in their memory in order to understand the mathematical problems. At this stage, students will correlate between the information and formula involved. Therefore, thorough planning is important to help students solve mathematical problems. The questions provided to the students in this study are in the form of HOTS. Based on the observation, majority of the students are not able to solve the questions. This may be caused by the questions that are outside of the context of the common questions that they have seen. The questions are different from the aspect of the organization, concentration and level of complexity. Therefore, teachers should play a role to cultivate the use of questions that demand the use of higher order thinking in the classroom while recognizing the difficulties faced by students in solving problems related to HOTS. This is relevant with the concept of school-based assessment that is being practiced in Malaysia which focuses on the students' learning development.