Analyze of the Science and Technology Course TEOG Questions based on the Revised Bloom Taxonomy and their Relation between the Learning Outcomes of the Curriculum

Students who are getting education in our country are subject to various exams in order to be placed in a higher education. Since 2013-2014 academic year, Transition from Primary Education to Secondary Education (TEOG) exam has started to be applied for secondary education. TEOG exam questions are prepared in such a way to include teaching program achievements/learning outcomes published by the Ministry of National Education. In this study, the relations between the levels of the TEOG exam science and technology course questions and the 8th class achievements of the Science and Technology course curriculum in the renewed Bloom taxonomy were examined. In the research, data were subjected to descriptive analysis using the document analysis method. A total of 120 Science and Technology questions of the TEOG exams held between 2013 and 2016 were included in the study but 118 of the questions were analyzed since two of the questions were canceled. At the end of the research, it was determined that 58 questions from the Science and Technology course of Semester I included in the TEOG exam were the questions related to the 31 learning outcomes of the total of 34 outcomes included in the examination program. It was observed that 60 questions from the Science and Technology course of Semester II were related to the 42 learning outcomes of the total of 96 outcomes included in the program. In addition, it was also determined that there was no question in TEOG related to some of the course units.


INTRODUCTION
Curricula are required in order to help the learning process to be carried out in a planned, programmed and productive manner. A curriculum is a system of learning experiences that is created in order to provide to an individual, all the activities related to the teaching-learning processes of a course, both in the school and outside the school (Demirel, 2015). With the curricula, it is aimed to allow individuals to learn by experiencing and to lead to a change in behavior of the individuals (Arseven, Şimşek and Güden, 2016;Eryaman, 2010;Uslu and Akgün, 2016). In science curricula, it is aimed to educate individuals who research, question, make effective decisions, solve problems, are active in collaborative processes and able to communicate effectively (Ministry of National Education [MONE], 2006). In line with the general objectives of the applied Science and Technology Curriculum of National Education in Turkey, constructivist approach philosophy was adopted and published in 2006 (MONE, 2006).
In a curriculum, the desired improvements regarding the individuals are expressed with learning outcomes. Achievements defined as the level of achieving the program objectives (Demirel, 2011) are of great importance in reaching the general objectives of MoNE. It is also important to determine the levels of the individuals in terms of these learning outcomes. Therefore, measurement and assessment studies are included in determining the level of the learning outcomes of the individuals.
Measurement and assessment works are carried out for various purposes in education. If it is aimed to maximize the efficiency in learning by determining the deficiencies and difficulties in learning then the evaluation is aimed at shaping and training. If the aim is to reveal the development of the individual in various directions in detail, then the assessment should be carried out according to recognition and placement. The aim of the assessments carried out at the end of the training period at the end of the specific phases of the teaching process is to determine the level of learning (Özçelik, 2010).
In Turkey, secondary education placement process is conducted through a central examination. Between 1998 and 2012, exams were held for the same purpose under different names. These were; The High School Entrance Exam (LGS), Secondary Education Institutions Selection and Placement Examination (OKS), SBS (Level Determining Exam) conducted in the 6th, 7th and 8th grade, and the SBS administered only in the 8th grade. Starting from 2013, the Transition from Primary to Secondary Education (TEOG) exam has started to be applied for secondary school students. Within the scope of TEOG, the exams are held for six core courses in each semester of the academic year. One of these courses is the Science and Technology course. In TEOG, prepared according to the Science and Technology curriculum learning outcomes published before the new Science Course curriculum which is gradually applied in 2013-2014 academic year, there are 20 questions that include this course in each exam period. It is required to have questions in the exam in line with the learning outcomes and in accordance with the plan determined as per the curriculum based on the academic schedule. It is stated that the TEOG exam was conducted in order to observe and evaluate the learning outcomes of the students objectively. At the same time, since the scores received from this examination are used in the transition to secondary education, it is observed that the TEOG exam serves both the purposes of level determination and also recognition and placement, as it was the case in the previous exams.
In order to ensure that the learning outcomes of the students are monitored and assessed in an objective manner, the objectivity and scientific relevance of the exam questions prepared for this purpose must be questioned. One of the suitable scientific tools to examine this is the Bloom Taxonomy, which is commonly known to the educators in our country. Anderson and Krathwohl (2001) found that, regarding the taxonomy developed by Bloom, it was not enough to assess only the cognitive dimensions of the learning outcomes -former name objectives -and that the taxonomy was complicated (Tanik and Saraçoğlu, 2011;Tutkun and Okay, 2012;Zorluoglu, Kızılaslan and Sözbilir, 2016). Therefore, it has been suggested that, in addition to the "cognitive process dimension" steps of learning outcomes, the simultaneous evaluation of the "knowledge dimension" steps could fix the said complexity (Anderson and Krathwohl, 2001). In line with this suggestion, the revised Bloom taxonomy (RBT) is required to be used not only in the recording of the learning outcomes but also in the teaching process and in the assessment of teaching. (Kotluk ve Yayla, 2016;Näsström, 2009;Zorluoglu, Güven and Korkmaz, 2017).
With education, educating individuals who have acquired the knowledge, skills and understanding required by the era, who are creative, who are not memorizing subjects but learning by understanding them, who are critical thinking, questioning, researching, knowing the ways of accessing information, constructing their own knowledge in their mind, synthesizing the information they receive and producing new information, having the power to analyze, are able to use the information in new situations, and who are analytical thinkers, and capable of exploration, has been the target of the education system (MEB, 2006). For this purpose, it is necessary to write down the learning outcomes, to ensure that the students achieve such outcomes and to evaluate the achievements of the students by Anderson and Krathwohl (2001) RBT steps.
The RBT table (Table 1) is drawn up based on the cognitive process dimension which is the horizontal column, and a constructivist approach aimed at meaningful learning. It consists of six steps: remembering, understanding, applying, analyzing, evaluating and creating. The remembering step means the recollection of the information from the long-term memory; the understanding step refers to the restructuring and phrasing of the verbal or written education entries by the students' own sentences; the application step is the problem solving process related to the information that the individual learns, and the practice and exercise process; the analysis step refers to the process of determining how the part-whole and whole-part relation is; the evaluating step is the process of achieving to a judgment based on certain criteria; and the creating step refers to the process of creating a meaningful and functional new product (Anderson and Krathwohl, 2001).
In the knowledge dimension which is the vertical column of the RBT table, practitioners try to find an answer to the question "What should the learners be taught?" The knowledge dimension is comprised of factual, conceptual, procedural, and metacognitive knowledge levels (Krathwohl, 2002). Factual knowledge level refers to the key information that students have to know about on any given topic; the conceptual knowledge level means the knowledge that explains the relations between the concepts within a structure; procedural knowledge level refers to the information on how to do any operation or work; whereas the meta cognitive knowledge level represents the information regarding students' cognition (Anderson and Krathwohl, 2001). Upon examining the related literature, examinations held were found to be inadequate to measure students' high-level cognitive skills when assessed within the learning outcome-unit scope, and assessment questions were found to be usually asked at a low cognitive level (Arı and İnci, 2015;Atila and Özeken, 2015;Ayvacı and Türkdoğan, 2010;Aydın and Güven, Cayhan and Akın, 2016;Çolak and Demircioğlu, 2010;Demir, 2011;Güleryüz and Erdoğan, 2018;Gündüz, 2009;Güven and Aydın, 2017;Gökulu, 2015;Kala and Çakır, 2016;Karadeniz, Eker and Ulusoy, 2015;Kaşıkçı, Bolat, Değirmenci and Karamustafaoğlu, 2015;Koğar and Aygün, 2015;Köğce and Baki, 2009;Özden, Akgün, Çinici, Sezer, Yıldız and Taş, 2014;Şad and Şahiner, 2016). In addition, the studies carried out are mostly aimed at determining the level of the questions asked for an examination according to RBT. No analysis based on RBT of TEOG exam -Science and Technology course exam questions was found in the literature and most importantly, no studies have been found that show how the learning outcomes of Science and Technology course are related based on RBT. For this purpose, we tried to investigate three cases regarding the TEOG exam questions that started in 2013-2014 academic period and applied for three years: 1. Determining the levels of TEOG exam question in RBT, 2. Determining the levels of the 8th grade Science and Technology curriculum learning outcomes in RBT, 3. Determining the compatibility level of TEOG questions and Science and Technology course curriculum learning outcomes based on RBT.
There is no comprehensive study regarding the science and technology courses of the TEOG exam, which was put into practice in the 2013-2014 academic year. The RBT based assessment of the learning outcomes regarding the examination will reveal the current situation and shed light on what levels should be concentrated based on RBT on the exams which will be carried out in the following period. In this context, it is thought that this research will contribute to the literature.

METHOD
In this study, the TEOG questions of the academic years 2013-2014, 2014-2015 and 2015-2016 Semester I and Semester II were analyzed based on RBT. In the study, the learning outcomes of the 8th grade science and technology curriculum (MONE, 2006) and TEOG questions were examined using document analysis method. The analysis of the documents included in the analytical researches is a process of encoding and examining the records and documents related to the designated research field containing the information about the cases or phenomena targeted to be investigated based on a certain system (Çepni, 2010;Yıldırım and Şimşek, 2011). Since RBT was used in the analysis of the data, the study was based on a descriptive analysis.
In the study, the following method was used which is suggested in the literature (Amer, 2006, Anderson, 2005, Bekdemir and Selim, 2008, Krathwohl 2002, Şahin, 2005, Zorluoglu, Kızılaslan and Sözbilir, 2016: (1) Firstly, the selected learning outcome sentence was examined and the step it belongs in the cognitive process dimension was determined based on the verb expression of the sentence.
(2) In order to determine the suitable step of the learning outcome for the knowledge dimension, the noun expression of the outcome sentence is taken into account. (3) If there is more than one verb expression in an outcome sentence, the higher-level verb expression is taken into consideration while determining the cognitive process dimension of the learning outcome. For example, if an outcome involves actions that express both the understanding and the analyzing steps, then the analysis step, which is a higher level, is chosen. (4) In cases where there is more than one noun expression in an outcome sentence, one higher-level noun expression is taken into consideration to determine the knowledge dimension level. For example, if the learning outcome includes both factual knowledge and procedural knowledge, the level of procedural knowledge is determined as the knowledge dimension. (5) Finally, the intersection of the level in the information dimension and the level in the cognitive dimension on the table was marked and the level of the learning outcome based on RBT was determined. Some examples of the learning outcomes and analysis of TEOG questions are presented below.
It is seen that there are two verb phrases which are "collects information" and "recognizes" upon examining the learning outcome sentence stated at the eighth grade level; "collects information about the concept of gene and recognizes dominant and recessive genes." The verb phrase of collecting information is the understanding level of the Cognitive Process dimension whereas the recognizing verb phrase is the analyzing step. In this case, it was decided that the outcome was more suitable for the analyzing step which is a higher level in the dimension. While the knowledge dimension of the learning outcome is determined simultaneously, since knowledge of the methods of gathering information regarding the gene context is required, it falls within procedural knowledge level, and also within conceptual knowledge level since it involves mutual relation knowledge between dominant and recessive genes. In this case, the table position of this outcome is determined as C4 and is shown in Table 2, taking into consideration the higher-level, which is the procedural knowledge level.  The question in Figure 1 is within the Factual Knowledge of the knowledge dimension because it is thought that students should formally know the basic information about the mitotic division. Since the student can remember the information through the forms when he/she sees the question, it is decided that the problem is at A1 level because it is within the remembering level in the cognitive process dimension. However, in some questions, although the question includes a different dimension as a root, since it is necessary to answer the steps involved in the question in order to answer the question, the position of the problem in the RBT was determined considering the dimensions of such steps. For example, examining the root of the question in Figure 2, it is understood that question is asked in the Factual Knowledge -Remembering level. However, upon reviewing the question as a whole, it is determined that its place in RBT is B4 because the question is within conceptual knowledge in terms of knowledge dimension and within analyzing level in terms of cognitive process level, since it is required that the student has knowledge about the reproduction types, the change of the genetics depending on the breeding varieties, and he/she needs to make analysis regarding the question on reproduction types.

Figure 2. Question from 2013-2014 Semester II
The analysis of the data was carried out by one chemistry education expert and two science education experts. Analysis in the study: Three different situations were analyzed. In the first analysis, Since the TEOG questions are required to be related to the learning outcomes of the Science and Technology Curriculum (MONE, 2006), the analysis of the outcomes covering the TEOG Semester I and II was carried out by experts. To do this, specialists came together to analyze the 8th grade learning outcomes of the Science and Technology Curriculum based on the RBT. In the second analysis, TEOG questions were analyzed based on RBT. In order to reach a common judgment in the analysis of the TEOG questions, experts analyzed the questions of TEOG 2013-2014 Semester I and then the analyses of other semesters were analyzed by each expert separately. Then the analyses of the experts were compared with the re-gathering of the experts. Since the experts analyzed the TEOG questions separately based on RBT, the reliability co-efficient in the analysis of the questions was calculated as .79 using the [Agreement / (Agreement + Disagreement)] formula. The analysis was considered to be reliable since the reliability coefficient of the TEOG questions based on the analysis results was greater than .70. In the third analysis, it was tried to answer the questions of "which outcomes do not have any question addressed to them?" and "how is the relationship between the questions and the outcomes according to RBT?".

FINDINGS
When TEOG questions and the related Science course outcomes are examined, it is seen that the questions and outcomes mostly take place in the cognitive process steps of the conceptual knowledge dimension.

Figure 3. Analysis of the Science and Technology Course Curriculum Outcomes within the scope of the TEOG Semester I Questions based on RBT
In Figure 3, it is seen that, when the outcomes of the TEOG Semester I questions belonging to Science and Technology Course are examined, the learning outcomes are overlapped on the conceptual knowledge level of the knowledge dimension according to RBT and distributed heterogeneously to all steps of the cognitive process dimension except for the creating step. It was determined that the distribution of learning outcomes as per the curriculum was C5 as the highest level (n = 2) and A1 as the lowest level (n = 2). It was observed that the maximum number of outcomes was at C3 (n = 5), B4 (n = 5) and B2 (n = 6) levels respectively; and the least number of outcomes were at C4, C2 and B3 levels (n = 1). In addition to these, it seems that there were no outcomes directed at the metacognitive knowledge level of the knowledge dimension and the creating level of the cognitive process dimension. When the TEOG Semester I questions were examined, it is seen as shown in the Figure 4 that the questions were asked mainly at B4 (n = 17), B2 (n = 16) and B5 (n = 5) levels in the conceptual knowledge dimension; and the least amount of questions were asked at the C5, C1, B3 and A4 levels (n = 1) respectively. Based on RBT, it was determined that the questions were asked at the C5 level (n = 1) as the highest level and at the A1 level (n = 5) as the lowest level in the TEOG questions of the 1st Semester. In line with the table shown at Figure 3, it is observed that there were no questions directed at the metacognitive knowledge level of the knowledge dimension and the creating level of the cognitive process dimension. (Figure 4)

Figure 5. Analysis of the Science and Technology Course Curriculum Outcomes within the scope of the TEOG Semester II Questions based on RBT
When the outcomes of the Science and Technology course curriculum including the questions of the second semester of the TEOG exam are reviewed, it was determined that D2 was the highest level (n = 2) and A1 was the lowest level (n = 8) in terms of outcome distribution ( Figure 5). It was observed that the maximum number of outcomes were at B2 (n = 34), B4 (n = 17) and B1 (n = 15) levels respectively; and the least number of outcomes were at D2 (n = 1), C2 (n = 2) and C4 (n = 5) levels. In addition to these, it seems that there were no outcomes directed at the metacognitive knowledge level of the knowledge dimension and the creating and evaluating level of the cognitive process dimension.

Semester II based on RBT
It was observed that Semester II questions of the TEOG exam were mainly towards Conceptual Knowledge dimension B4 (n = 25), B2 (n = 13) and B5 (n = 6) as Semester I, and unlike Semester I, more questions were asked towards the B4 level. The least number of questions were asked at C5 (n = 1), A1 (n = 1) and B3 (n = 3) levels respectively ( Figure 6). It was determined as shown in Figure 6 that the questions were asked at the C5 level (n = 1) as the highest level and at the A1 level (n = 2) as the lowest level in the TEOG Semester II questions, same as the TEOG Semester I questions. It is seen that Figure 6 summarizing the situation regarding the TEOG Semester II questions and the Figure 5 examining the outcomes covering the questions from Semester II are in conformity. Unlike the outcomes, it was determined that questions were asked towards the evaluating step of the cognitive process dimension (B5, n = 6).
In general, reviewing the questions asked in the TEOG exams held between 2013-2016 in details based on RBT: 11 questions were asked at A1 level, 6 in A2 level, 2 in A4 level, 12 in B1 level, 40 in B2 level, 2 in B3 level, 50 in B4 level, 13 in B5 level, 1 in C1 level, 9 in C3 level, 9 in C4 level and 3 in C5 level and there were 98 questions in total. The most amount of questions were asked at B4 (n = 50), B2 (n = 40) and B5 (n = 13) levels and the least amount of questions were asked at C1 (n = 1), A4 (n = 2) and B3 (n = 2) levels. Consistent with the learning outcomes indicating the 8th grade science and technology curriculum, it was determined that there were no questions regarding the metacognitive knowledge level of the knowledge dimension and the creating level of the cognitive knowledge dimension.
In addition, it was determined that 3 questions asked in the Semester I of 2014-2015 academic year were towards the outcomes of Semester II. Besides, it was determined that TEOG exam did not have any questions regarding 57 outcomes included in the curriculum.

CONCLUSION AND DISCUSSION
It has been determined in the study that the TEOG questions are not homogeneously distributed to the learning outcomes and to the cognitive process and knowledge dimension levels of RBT. In this respect, the study has similarities with the views of Atila and Özeken (2015) and Kaşıkçı, Bolat, Değirmenci and Karamustafaoğlu (2015) who were indicating in their studies that TEOG questions were not homogeneously distributed to the curriculum outcomes.
It was determined that in the TEOG exams of 2013-2014, 2014-2015 and 2015-2016 academic years, there were no questions regarding 57 outcomes although they were included in the Science and Technology curriculum. TEOG questions need to address each learning outcome, in order to ensure that students' learning outcomes are assessed in the best way possible. In addition, the results obtained from the analysis of the TEOG questions by considering the cognitive process dimension of RBT, suggests that MONE (2006) is not sufficiently compliant with the goal of educating high-rank thinking students (Güven and Aydın, 2017). It has been determined from the studies in the literature that questions regarding certain outcomes included in the curricula are not being asked for the courses beyond Science and Technology course of the TEOG exam as well. Cayhan and Erhan (2016) examined the relationship between the 2014-2015 -Semester I TEOG questions regarding the Turkish course learning outcomes and it is stated that some of the outcomes were not addressed in the TEOG exam. In addition, Karadeniz, Eker and Ulusoy (2015) determined that 2013-2014 TEOG Semester I and II -Revolution History of Republic of Turkey and Kemalism course exam questions were not in line with the amount of the learning outcomes and also some of the units were not even had a related question in the exam. Arı and İnci (2015) examined the 2013-2014 TEOG Science and Technology -Semester I and II exam question and stated that the 68 out of 137 outcomes in the curriculum were related to the exam questions and that they were concentrated on the sub level cognitive steps and gave more weight to some of the learning outcomes. Koğar and Aygün (2015) investigated the compliance of the 2013-2014 TEOG mathematics questions with the objectives and determined that removing 4 of the questions of the Semester I and 1 of the questions of the Semester II will provide a better validity in terms of scope.
50% of the 2013-2014 TEOG Science and Technology Course exam questions were comprised of remembering and understanding levels (Gökulu, 2015). Similar results were obtained in the study as well. It was determined that there was a question for each level of the cognitive process dimension except for creating level, but the questions were mainly regarding the levels of remembering and understanding. In order for the education to be effective, the outcomes in curricula must be homogeneously distributed to the dimensions of the RBT. In situations where a homogeneous distribution can not be achieved, the teacher should provide the education at the RBT dimensions of the outcomes or higher. In the examinations held for the purpose of measurement and evaluation at the end of the education process, the evaluators should ask questions regarding each level of the RBT. However, when preparing the questions, the questions should be prepared considering the dimension of each outcome, and the questions directed to the higher-level knowledge and cognitive process dimensions should be asked as well (Anderson and Krathwohl, 2001). Upon assessing the TEOG questions in general, it was determined that there were not enough questions regarding the dimensions of upper knowledge and metacognitive process skills, and when the relations between the learning outcomes and the questions were examined it was seen that the questions were prepared towards the sub dimensions of the outcome dimension. In this regard, this study shows similarities to the researches of Özden, Akgün, Çinici, Sezer, Yıldız and Taş (2014) and Kaşıkçı, Bolat, Değirmenci and Karamustafaoğlu (2015). Özden et al. (2014) stated that, in the 2013-2014 TEOG Science and Technology Semester I exam there were fewer questions measuring metacognition skills. Kaşıkçı et al. (2015) revealed that although the Science and Technology questions in the 2013-2014 Semester II TEOG exam was in line with the learning outcomes, the outcome-unit distribution was not homogenous.

SUGGESTIONS
In the TEOG and similar examinations conducted with the aim of providing education in high schools where different cognitive levels are activated, it is required to include questions where each step of the RBT cognitive process skills can be activated and that address each earning outcome within the curriculum. It is thought that the quality of the student selection exams can be increased and the validity of the scope of the exams can be provided by this way. In addition, the preparation of the TEOG questions for each learning outcome in the curriculum will make the curriculum learning outcomes useful.