Analyzing Higher-Order Thinking Skills in Reading Exercises of Indonesian EFL Textbook

To face 21st-century challenges, the elements involved in the pedagogical approach, such as textbooks, must enable students to enhance their higher-order thinking skills (HOTS). This qualitative research aims to determine to what extent HOTS is incorporated in an Indonesian EFL textbook. The comprehension questions of reading texts in the textbook for 7th grade were analyzed through content analysis based on the six cognitive domains in the revised Bloom’s Taxonomy (Anderson & Krathwohl, 2001). The findings of the study revealed: (1) 80,4% of the reading comprehension questions classified as lower-order thinking skills (LOTS), particularly level 1 (remember) and level 2 (understanding), whereas 19,6% of the questions classified as higher-level thinking skills, primarily the level 4 (analyze) and level 5 (evaluate). (2) The reading comprehension check questions were organized systematically from lower-thinking to higher-thinking skills, which develops students’ comprehension and provides instructional organization for teaching texts to English teachers. This study is expected to benefit English teachers, textbook authors, and future researchers to elaborate on implementing HOTS in English language teaching and learning.


Introduction
Implementing higher-order thinking skills (HOTS) in the pedagogical process is getting much attention in education.It is necessary to implement higher-order thinking skills in the teaching and learning process in the classroom, including in the English teaching and learning process.As Lai (2011) mentions, higher-order thinking skills are one of several skills required to prepare students for their secondary education.Teachers, students, curriculum, materials, and learning facilities have been identified as significant factors in the English teaching-learning process (Wati, 2018).Thus, the main elements involved in the classroom, such as the textbook that is being used, should enable the students to increase their higherorder thinking skills.
Reading, as one of the four basic language skills, is a complex and challenging skill to master.Linse (2012) asserts that reading requires higher-order thinking skills.It is far more complicated than simply decoding specific words.Teaching the students to derive meaning from their reading and analyze and synthesize what they have read is essential to the reading process."According to Yu-Hui et al. (2010), reading and higher-order thinking skills are strongly intertwined; students must not only understand the meaning of the words written down when reading.
Previous studies on Indonesian students' reading difficulties revealed that some of the issues which prevent Indonesian students from achieving good results in reading tests are students' passive attitudes and lack of reading strategies (Suryanto, 2017).Some reading difficulties are also found in the students' insufficient vocabulary, lexical inefficiency, structural complexity, language inaccessibility, poor reading skills, lack of schemata, and students' low motivation (Rahman, 2007: 153).These issues relate to higher-order thinking skills, the student's inability to apply their linguistic knowledge to assist them in reading demonstrated a lack of transferring skills.At the same time, their passivity in the classroom may indicate a lack of critical thinking, and their sufficiency in reading strategies demonstrates the problem-solving process.It can be concluded that reading and higher-order thinking skills are strongly intertwined and affect each other.If there is a way to improve students' reading skills, it must be through a qualified textbook that promotes the development of both reading and higher-order thinking skills.
Choosing a suitable textbook for use in the classroom is one of the most challenging tasks teachers often face.Cunningsworth (1995: 5) states, "No coursebook designed for the general market will be ideal for your particular group of learners."One of the most critical aspects that are required to be analyzed is the exercises or tasks of the textbook.Ur (2012) highlights that practice is arguably the essential activity of all the stages of learning because it helps the students practice their ability and acquire good outcomes.
There are some previous studies on the implementation of higher-order thinking skills in reading activities.Keshta & Seif (2013) conducted a study to evaluate the treatment and availability of higher-order thinking skills in English for Palestinian grade 8 students in reading comprehension.The study used content analysis cards to collect data on the extent to which reading exercises include analysis, synthesis, and evaluation skills.The study found that higher-order thinking skills in reading exercises are not well covered, implemented, or distributed.In another research conducted by Atiullah et al. (2019) on evaluating the availability of higher-order thinking skills in reading comprehension questions of the English textbook for Year X of high school, the study's result showed that most reading comprehension questions in the textbook lack higher-order thinking skills.The questions only cover lower-order thinking skills.
Bloom's taxonomy was developed by Benjamin Bloom and his associates in 1956.It was intended to provide for the classification of educational system goals, especially to help teachers, administrators, professional specialists, and research workers discuss curricular and evaluation problems with greater precision.(Bloom as cited in Amer, 2006:215).In its development, the original of Bloom's taxonomy had been revised by Anderson & Krathwohl (2001) to refocus teachers' attention on the value of the taxonomy as well as to incorporate new knowledge into the taxonomy.In the revised version of Bloom's Taxonomy, Anderson & Krathwohl (2001) change the order of evaluation with synthesis.Below is the table of Revised Bloom's Taxonomy.Put elements together to form a coherent whole or make an original product.
Those cognitive dimensions are divided into two categories which are lower order thinking skills, hence, LOTS (remember, understand, and apply) and higher order thinking skills or HOTS (analyze, evaluate, and create).Woodward and Elliot explained that textbooks can be improved by emphasizing problem-solving and higher-order cognitive processes, as cited by Reed and Bergermann (1998).Therefore, the researcher is urged to investigate to what extent the higher-order thinking skills incorporated in the English textbook entitled "English for Nusantara" for 7th grade published by the Indonesian Ministry of Education in 2022, based on the revised Bloom's Taxonomy (Anderson & Krathwohl, 2001).The researcher will be focused on reading comprehension questions since it is a receptive skill requiring a highly complex process.The researcher wants to ensure that questions in the reading comprehension questions enhance the students' higher-order thinking skills.

Method
The content analysis method was applied in this research because the textbook's content was analyzed.The object of this research is the English textbook "English for Nusantara" published by the Indonesian Ministry of Education that consists five chapters.The data in this research are qualitative data, organized as the questions of the constructed responses of reading questions are collected from the textbook, specifically the constructed response of reading tasks.In addition, a simple statistical calculation was employed to determine the distribution of each level of Bloom's taxonomy, particularly the higher-order thinking level, and the data were interpreted qualitatively.
In analyzing the data, the writer adopted eight procedures to analyze this textbook based on Fahriany et.al. (2019)  The writer identified all of the reading comprehension passages featured in the textbook and determined which tasks correspond to research questions (constructed response of reading tasks).Then, the questions are marked and classified according to the cognitive dimension based on Bloom's revised taxonomy.The writer then inputs the data on the checklist table.The checklist table form was used to analyze and compare the distribution of the higher order thinking skills questions levels presented on reading comprehension questions.The checklist table form includes a list of constructed responses of reading questions.The writer only focused to analyze the distribution of higher order thinking levels.

SALEE: Study of Applied Linguistics and English Education
Lastly, the writer interpreted the findings of the study by using a qualitative approach and concluded the findings.

Finding
There are 26 reading tasks in the textbook and 17 tasks are identified as constructed response tasks.This research focused on analyzing the short comprehension response tasks that consist of 11 tasks in the textbook with a total of 51 questions.The writer classified the level of questions using the checklist table based on revised Bloom's taxonomy.The cognitive process dimension is divided into six levels; 1 (Remember), 2 (Understand), 3 (Apply), 4 (Analyze), 5 (Evaluate), and 6 (Create).Out of 51 short comprehension reading questions, 41 questions were classified as lower-order thinking, and another 10 questions were classified as higher-order thinking skills.A simple statistical calculation was employed to find the percentage of the distribution between higher thinking skills and lower thinking skills, the result can be seen in Figure 1.The Figure above shows the dominance of lower order thinking skills that hold 80,4% with a total 41 questions, primarily level 1 (remember) with 37 questions and subsequently followed by level 2 (understand) with 4 questions, and there are no questions that classified as level 3 (apply).On the other hand, a higher thinking level accounts for 19,6% with a total 10 questions.The percentage of the distribution of every level of higher thinking level can be seen in Table 2. Looking at the table above, is apparent that level 4 (analyze) of higher-order thinking is the most distributed in the reading questions with 6 questions or holds the total 11,8%, subsequently, follow by level 5 (evaluate) with 4 questions with a total 7,8%, and for the highest level of higher thinking skills, there are no questions classified as level 6 (create) in the reading questions.Furthermore, the following two tables display the total of the reading questions based on their skills, so the types of questions that are classified as higher-order thinking skills could be seen clearly.Table 3. contains the questions that classified as level 4 (analyze).As shown in Table 3., the highest distribution of questions classified as level 4 (analyze) is from Chapter 4 with five questions, followed by Chapter 1 with only one question.In chapter 4, the students must answer questions related to given texts by inferring implicit assumptions and determining similarities between two assigned texts.In Chapter 1, with only one question, the students are given the task of responding and have to infer implicit assumptions based on the given text.The following Table 4. explains reading questions that are classified as level 5 (evaluate).As can be seen from Table 4., there are four questions out of 51 reading questions from Chapter 1 and Chapter 4 that are classified as level 5 (evaluate).The highest distribution of level 5 (evaluate) is from chapter 4, with three questions, followed by chapter 1, with only one question.At this level, mainly the questions are required students to provide their 566 SALEE: Study of Applied Linguistics and English Education personal opinions about the given texts.Furthermore, no reading questions are classified as level 6 (create), the highest thinking skills among the three higher thinking skills.
To sum up, this textbook's distribution of thinking levels emphasizes lower-level thinking skills, which are level 1 (Remember) and level 2 (Understand).Although the textbook did not a sufficient amount of higher thinking skills, the book still provides several higher-level thinking skills questions, namely level 4 (Analyze) and level 5 (Evaluate).

Discussion
To answer the research question "To what extent are HOTS incorporated in the comprehension questions of reading texts of the textbook?", the result of the data analysis shows that from the total of 51 constructed response reading questions in the textbook.higher thinking skills obtained 19,4% or ten questions, with six questions classified as level 4 (analyze) and four as level 5 (evaluate).Moreover, the level 6 (create) in the reading questions must be concern because there is no question classified as level 6 (create).On the other hand, the lower order thinking questions receive 80,6% or 41 questions, 37 questions classified as level 1 (remember), and 4 questions classified as level 2 (understand).
The distribution of higher thinking skills accounts for almost a quarter of all questions, indicating that the textbook's author did pay attention to the distribution of higher-order thinking skills.For example, level 4 (analyze) accounts for 11.8% of all questions.According to this study, most higher thinking skills questions demand students to provide logical reasoning.Level 4 (analyze) includes numerous purposes, such as linking the conclusion and supporting assertions, understanding how ideas relate to one another, inferring implicit assumptions, and discovering evidence to support the author's purpose (Anderson & Krathwohl, 2001).Questions about logical thinking determine how statements relate to one another, such as why something occurs.For example, the questions in Chapter 1, "Why does Pak Edo always wear his helmet when he goes cycling?" and Chapter 4 "Why does Ibu Ayu teach English online?"In these questions, students must infer the author's unspoken assumption, as there is no single statement that mentions why Pak Edo is wearing a helmet or the reason why Ibu Ayu teaches English online.This is also in line with Kementerian Pendidikan dan Kebudayaan (2022) in Learning Outcomes in Early Childhood Education, Primary Education, and Secondary Education on the Merdeka Curriculum, that at the end of the learning, students are able to identify the purpose of texts and begin to make inferences to comprehend implicit information in the text.
In level 4 (analyze), the textbook also offers questions to determine the similarity between the two texts, such as questions in chapter 4, "Do you find similarities between Monita's tips and Ibu Ayu's online rules?What are they?" and "Do you find similarities between Galang's and Andre's infographics?What are they?".According to Anderson et al. (2001), this question, such as determining similarity, belongs to the subtype of level 4 (analyze), differentiating.This statement is also in line with Brookhart (2010) that assert, comparison and contrast questions do require analyze-level thinking where students are required to identify various elements in it and organize those elements according to whether they are alike or not alike.
Level 5 (evaluate) has 7.8% of the questions or 4 questions out of 51.This level of question appears in chapters 1 and 4. According to the study, most level 5 (evaluate) questions require students to provide their personal opinions.Brookhart (2010) stated that evaluate questions could be modified by requesting literary criticism about the object, asking about the text's quality based on the reader, and asking them the reason.In this study, the majority of the questions seek students' perspectives.As in the questions in Chapter 4, "Do you think 'break time' is important?Why?" and "Do you agree with Andre that you should participate in class discussions?Why?"These questions involve students making a valuable judgment based on criteria and standards (Anderson, 2001), according to Brookhart (2010: 53) the criteria can be standard (e.g literary, historical, scientific) or criteria that the students invent themselves (in which case an element of creativity involved as well).This type of question required students to make valuable judgments based on the criteria that invented by themselves.
Furthermore, there are no questions that are classified as level 6 (create), the highest level of hots.Level 6 (create) involves students putting elements together to form a coherent or functional whole, it has students make a new product by mentally reorganizing some elements or parts into patterns or structures not present before (Anderson, 2001).It could be better if the teachers and the author of the textbook pays more attention to the distribution of the level 6 (create) in the reading questions.In addition, the variation of questions in level 6 (create) could be enriched by asking the students to create a scheme for classifying ideas or propose a plan for research by adding such questions like "How would you explain the reason…?" or "What facts can you gather…?".
These results are in line with the prior studies conducted by Mizbani & Chalak (2017) and Dewayani et al. (2020) that level 1 (remember) has the most frequency in the reading questions then, followed by level 2 (understand).The study conducted by Zainil et al. (2020) also found that reading comprehension questions in the textbook used by the students in all grades of senior high school were mostly lower-thinking questions.They suggested since the role of textbooks is essential in EFL classrooms, the textbook author should pay attention more to the content of textbooks.They also suggested that teachers should not depend on textbooks.They need to create supplementary activities to enhance students' thinking skills.Higher-order thinking skills are essential to be implemented in the EFL classroom as they can increase students' motivation and reading comprehension (Samelian, 2017;Sholihah & Widyantoro, 2021) The book's emphasis on lower-level thinking skills may be due to Indonesian students' lack of proficiency in answering higher-level thinking questions.It stems from the fact that English is a foreign language in Indonesia, and as such, it is not the first or second language of the students.It is in line with Gordani (2010), that the materials in English textbooks only emphasize the first three levels of cognitive domains attributed to students' proficiency levels.Their low English proficiency prevents them from achieving such a high level of cognitive complexity.Daiek and Anter claimed that the create questions had difficulty in answering.It could occur because lower-order thinking questions are familiar and frequently appear in the lesson plan, are the easiest for students to answer, and are much easier for teachers to create.This finding appears frequently in almost all of the studies discussed in the review of related literature.
Posing HOTS in reading comprehension is crucial as it can improve students' critical thinking.The use of HOTS strategies in teaching students reading comprehension texts had a significant positive effect on the students' reading achievement (Nejmeh, 2011).Hove and Hand discovered that students who are asked higher level questions perform significantly better than those who are not.The findings support Blinker's claim in Nesbitt-Hawes (200 5) that the right type of questioning fosters critical thinking in students.This finding also demonstrates that there is a significant positive relationship between critical thinking and reading comprehension (Hosseini et al., 2012).Therefore, the writer thinks that the variation of the higher-order thinking questions must be enriched so that the students could improve their higher-order thinking skills properly and it can improve students' reading comprehension (Allo, 2022).

Conclusion
Based on the results, the textbook emphasizes more on lower-thinking skills.It is proven by the data that from the total of 51 questions, 41 or 80,4% of questions were classified as lower thinking skills.On the other hand, higher thinking skills account for 19,6% or 10 of the questions.Additionally, higher-order thinking skills are incorporated in the textbook as follows: 1. Level 4 (analyze) obtains 6 out of 51 questions with a percentage of 11,8%.It can be seen that the author of the textbook places more emphasis on level 4 (analyze) for higher-order thinking levels.2. Level 5 (evaluate) obtains 4 out of 51 questions with a percentage of 7,8%.3. Level 6 (create) obtains null distribution.
Finally, it concludes that the textbook emphasized more on lower order thinking skills in the constructed response reading questions in the textbook.It also can be seen that not all cognitive levels can be covered in the reading activities.Two major limitations in this study could be addressed in future research.First, this study focused on analyzing one textbook for 7 th -grade junior high school students.Secondly, this research only focused on one aspect of language skills.The other skills should be analyzed as well in order to reach a comprehensive result.
namely; Identifying: finding related information needed to analyze the data; Determining: deciding which parts need to be analyzed; Coding: marking each type of the data; Constructing a checklist: classify the data based on determined characteristics; Tabulating: input the data based on the classification; Analyzing: provide a detailed description on the research problem; Interpreting: create findings of the research then compared to other similar research; Conclusion drawing: conclude findings of the research, make an inference, and answer the research question.

Figure 1 .
Figure 1.Percentage of The Distribution of HOTS & LOTS

Table 2 .
The distribution of HOTS in reading exercises

Table 3 .
The distribution of level 4 (Analyze)

Table 4 .
The distribution of level 5(Evaluate)