The Development Research: Assessment Instruments of Science Literacy Based on Minimum Competency Assessment (AKM) Level 4 th for Solar System Concepts

The literacy competence of students is getting lower. The Ministry of National Education issues literacy in schools regulations and develops literacy assessment formats to improve it. It continues to experience improvements because it can still not boost student achievement. AKM is the latest student literacy competency assessment policy. Considering that the program is new and will be carried out next year, the number of question bank on AKM books is still limited. However, a design guide document for the development of the AKM test was launched. Therefore, good quality literacy tests are still needed. This development research aims to produce a valid and reliable AKM-based test package for the Solar System’s Reading Science Information Literacy level IV concept. And then, at the survey stage, respondents worked on the test package in the e-learning format. The test’s scoring will be used to describe the student literacy competency profile per the design guidelines for developing AKM questions. The results indicate that: (1) the AKM tests package that has been compiled is valid and reliable; (2) only 41 out of 181 students accessed the survey; (3) their highest achievements are the sufficient category, while the less category is most dominant; (4) the description is that their competence is only at the level of finding information, sub-competencies: accessing and searching for information in the text; search and select relevant information. The highest competence is understanding, sub-competence: comprehending text literally; explaining the main idea and some supporting ideas in the information text. It is competence at level 2 on the AKM test, the ability of students in grades 3 - 4 elementary school.


INTRODUCTION
In the constitution of the Republic of Indonesia, educating the nation's generation is the responsibility of the State. For these reasons, good governance always tries to perfect the National Education System. It also underlies Indonesia to welcome the vision of literacy for all that UNESCO has promoted since 1945 (https://en.unesco.org/themes/literacy), become a member of the OECD (Organization for Economic Co-operation and Development) and actively participate in an assessment survey of the development of the quality of education through PISA (Programme International Student Assessment) since 2000, also participated in TIMSS (Trends in International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study) organized by IEA -(International Association for the Evaluation of Educational Achievements). The results of the assessment and its recommendations become one of the considerations for improving the quality of national education.
It turns out that the literacy achievement of students in these surveys is still below the average. In particular, the PISA 2012 score was ranked in the bottom ten countries out of a total of 65 participating countries (PISA 2012). This condition is one of the reasons for establishing a regulation that aims to build students' literacy skills, including Government Regulation of the Republic of Indonesia number 32 of 2013 concerning national education standards; Indonesia National Assessment Program (INAP); Regulation of the Ministry of Education of the Republic of Indonesia number 23 of 2015 concerning the Growth of Character (PBP) which is contained in its appendix; revision of the Regulation of the Ministry of Education of the Republic of Indonesia number 20, 21, 22 of 2016 concerning Graduate Competency Standards; Indonesian Student Competency Assessment (AKSI) education achievement mapping program.
In 2018 PISA reported that Indonesian students' Mathematics and Science Literacy competence was at level 1, but the value was still below the average (Schleicher, Andreas 2018). Therefore, the pattern of the learning process and its assessment nationally is improved, one of which is in the form of a Minimum Competency Assessment (AKM) policy. AKM is the latest literacy assessment model that aims to map the success of national education. In addition, the design of the test instrument development is claimed to refer to the TIMMS and PISA assessment frameworks. The implementation is planned to start in 2021 (Pusat Asesmen dan Pembelajaran 2012).
There are several reasons why this research is limited to Literacy Reading Science Information Level IV Solar System Concepts. Science subject is the studies that surround our lives. But there are some facts that the limitations of the human sensory system lead to wrong understanding and reasoning, especially about the Solar System. Among the underlying reasons are the celestial bodies in the solar system are very far from the observer, and these objects move very fast. They are relative to each other so as to form a system that requires an explanation of complex scientific concepts (Comins 1993) and is made worse with limited learning resources and observation instruments (Meneses 2018). One of the impacts is that until now, Astrology is more firmly rooted in most people's minds than Astronomy (Raja 2018). The choice of level IV was because the target of the assessment was 8th grade junior high school students, according to the attachment to the Decree of the Head of the Research and Development and Books Agency No. 018/H/KR/220, in which the concept of the Solar System was introduced to students in 6th grade Elementary School, the emphasis was on Bloom's Taxonomy at C2. Then the Solar System studies will be deepened in the 7th grade of junior high school following Bloom's Taxonomy C4.
Several research reports state that these limitations can be overcome by increasing learning resources (Kovalenko 2019); evaluating and improving the learning process, developing assessments according to their functions, and improving teacher competencies' quality (Bailey & Slater 2003-2004. Through this development research, it is hoped that it will be helpful for students to build their competence. For teachers and stakeholders interested in education, it can be used as a reference in developing AKM-based assessments, as well as a reference in compiling a mapping of students' cognitive abilities and soft skills, especially in the literacy field. Based on the description of the problem, there are two objectives to be achieved in this research, namely to produce: (1) a valid and reliable level 4th package test of science literacy based on minimum competency assessment, (2) a description of the literacy competency profile of the Solar System concept for 8th-grade junior high school students.

METHODS
Based on the objectives and subjects, this research is divided into 2 (two).

Development of Scientific Literacy Assessment Instruments Research
The instrument is based on the Minimum Literacy Competency Assessment for Reading with Information Text content and with a Level 4 Scientific context with the Solar System Material. This study aims to produce a package of Science Literacy AKM questions level IV Solar System Concepts. The development of the instrument has its own rules called the Design for Development of AKM Problems. The design has the same steps as the research design developed by Borg & Gall, which has been simplified into three main stages, namely: (a) preliminary study stage; (b) development study stage; (c) evaluation stage (Borg & Gall 2007in Rusilowati, Kurniawati, Nugroho & Widiyatmoko 2016.
In the preliminary stage, the activities include conducting: a Science Literacy literature study, a study of student learning achievement, especially in the subject of the Solar System, and a study on the correlation of minimum competency targets for learning outcomes of the Solar System learning for 7th-grade students with the reading information literacy competency target at level IV.
The development stage includes: selecting text or reading that functions as a stimulus in the test, namely reading with a scientific context, especially the concept of the Solar System. These reading materials come from the Solar System handbook and news related to knowledge of the Solar System concept. The development of questions follows the following provisions: (1) the number of questions consisting of 40 items in several forms of questions with a specific composition as in TABLE 1; (2) consists of three cognitive levels as shown in TABLE 2. After the questions are arranged, the next activity is to test the validity of the items. There are three aspects: material, construction, and language. Tested by three lecturers of FMIPA UNESA. The validation results become a reference for the revision of the question package.  (Pusat Asesmen dan Pembelajaran 2020) After being revised, the test package must be further tested using a reliability test after being revised. Reliability testing requires test results scores, so there must be a trial stage (Juliandi 2008). In the trial process, 15 respondents in the 8th grade of junior high school in the 2021 -2022 school year took tests without pre-treatment or pre-test. The model follows the experimental design of the "One Shot Case Study" (Campbell and Stanly 1963: 6-7 in Puspitaningrum 2020), namely: Information: X = The treatment is given to the respondent, in this case, the school teacher, who carries out the teaching learning activities of the solar system concept. O1= Observation of the respondent's test scores, in this case, a test package of AKM Scientific Literacy Level 4 Solar System Concepts compiled by the researcher. The scores obtained by the respondents were used as data in items the reliability testing with the statistical formula of Cronbach's Alpha. Then the calculation results are used to analyze whether the research hypothesis is accepted or rejected. The evaluation stage requires the conclusion of the reliability test results as the material of evaluation and revision considerations of the items test.

Reading Literacy Competency Research
This study describes the Reading Literacy Competency of Science Information Level IV Solar System Concepts. Based on these objectives, data collection is needed in this study using the survey method. The subjects of this research were 8th-grade students of SMP (Sekolah Menengah Pertama) and MTs (Madrasah Tsanawiyah). The population research is 320 students. The School locations are in Surabaya and Gresik. The survey question is a package of science literacy level 4 th Solar System concepts test that has been successfully appropriately arranged. The duration of the test is 100 minutes.
Then the score obtained is analyzed. The description of competence is obtained from the narrated score in the form of a description as follows.
The maximum score for each item is 0.25, the minimum score is 0, and the total score is 10. The next step is to make an assessment using the formula: = 400 Furthermore, these values are analyzed to obtain an overview of the competency profile of Science Literacy Level 4 Solar System Concepts. The profile is a student achievement criteria consisting of Very Good, Good, Enough, and Poor with the following conditions: • If the respondent gets a quantity less than 160 (a quantity <160), then the achievement criteria is less. This means that the competence of students is possible below level 3, • The achievement criteria are sufficient if the respondent gets a quantity of 160 to 320 (160≤a quantity≤320). This means that the competence of students below level 4, • If the maximum quantity obtained by the respondent is greater than 320 to 384 (320<a quantity≤384), then the achievement criteria is good. This means that the competence of students is in accordance with level 4, • If the quantity obtained is more than 384 to 400 (384<a quantity≤400), then the achievement criteria is very good. This means that the competence of students allows them to exceed or be above level 4, The profiling is based on the assumption that the composition of the cognitive level on the questions has been converted into test scores. More details are in TABLE 3.

Validation and Reliability Items Test
A total of 40 scientific literacy questions were successfully compiled. The distribution of cognitive level and number of questions on the TABLE 3. The format of the question form consists of multiple choice 1 answers totaling 8 items, complex multiple choice totaling 24 items, matching 4 items, short entries totaling 2 items, and descriptions of 2 items.
Furthermore, the package of questions is tested for validation. Then the average score which includes material aspects, construction aspects, and language aspects is obtained with a score of 3.6 (details of the value are in the appendix). This value is based on the scale compiled by Niza (2018) in Rahayu (2020) which means that the questions have valid criteria. However, there are some improvements to the questions according to the validator's suggestions.
As the following statement that a questionnaire is declared valid if the questions can reveal what is to be measured (Ghozali 2009 in Validity and Reliability Test 2014). Based on this statement, this set of questions can be used in research trials to obtain the score required for reliability testing (test scores e-Journal: http://doi.org/10.21009/1 are on the appendix page). The next step is writing tests in the Moodle eLearning Rakyat, lms application.
Arikunto (2012) in Widyasari (2015) explain that Cronbach's Alpha is used to find the reliability of instruments that score not only 1 and 0. The score obtained is calculated based on the correlation coefficient formula (r) Cronbach's Alpha product moment, and the value is 0.97, while the distribution value of the r product moment table at a significance level of 5% and the number of respondents is 15 is 0.514. The comparison of the r-calculated value, which is greater than the r-table, indicates that the assessment instrument developed is declared reliable. The conclusion is based on the explanation of Juliandi (2008).

Respondent Participation Rate
This stage is also known as the implementation stage. In this study, the respondent were deliberately chosen in various ways. The institution's diversity includes: 1) the location, 2) the status, and 3) the department that oversees it. The diversity of respondents selected is not directly related to the main purpose of this study. Still, we hope to obtain additional information that can be used for further research, considering that Surabaya is the province's capital. At the same time, Gresik is the district, in general, there may be a difference.
When selecting respondents, by applying for research collaboration to several schools, they would send the student's email address. It was used to register to an lsm Moodle eLearning Rakyat account. Due to some reasons, only 191 students got respondent data, and 82 students accessed or took the test, so as many as 109 respondents did not log in to the application.

FIGURE 2. Respondents Competency Comparison
There are several reasons why certain junior high schools are not willing to have their students become respondents, including because the school is responsible for the security and privacy of its students, so they must protect the confidentiality of their emails; the school stated that most of its students were financially limited and could not afford internet-connected gadgets; since students do not go to school and only study at home, so the level of participation in the learning process is low; and so forth.
Based on the number of involved respondents, there is a very sharp difference between 'total respondent' with 'number of respondents who access'. The decrease in participating institutions is generally due to the policy of limiting face-to-face learning directly (due to the PPKM policy due to the Covid-19 pandemic), and also due to limited facilities, in this case, computers or gadgets connected to the internet. Several educational institutions that are willing to cooperate do not guarantee that their students participate or are willing to become respondents, as listed in TABLE 4. The relatively low level of respondent participation occurs due to low student motivation, especially considering that the AKM is not an assessment for grade promotion or graduation requirements.
Students should take advantage of the opportunity to be respondents in this research to hone their skills and gain experience before they have to face the National AKM, which will be held a few months later. But research facts conclude otherwise.

Science Literacy Competence of Middle School Students
Meanwhile, student achievement is based on the scores obtained only in the sufficient and fewer categories as illustrated in FIGURE 2. The respondent's test scores (complete data can be found in the table in the appendix) are then converted into predetermined criteria set on the AKM Question Development Design. The explanation is as follows: (1) scientific literacy competence, categorized as less, shows that their literacy competence is at the 2 nd level. This condition refers to the AKM development design, level 2 nd is an AKM question that is used to assess the literacy competence of grade 3 rd and 4 th students, while at level 3 rd is an AKM question that is used to assess the literacy competence of grade 5 th and 6 th students.
In this study, based on the acquisition of scores, it was determined that the literacy competence of junior high school students revolved around only 2 (two) criteria, namely the category of sufficient and insufficient competence. The level of education and learning progress should be compatible with each other and even allow for a higher level of learning progress, but this time there were no students with this achievement condition. Their information literacy skills are only at the competency stage: Finding information and sub-competencies: Accessing and searching for information in the text; search and select relevant information. Their highest competence is at the Understanding stage, the subcompetence of understanding text literally: explaining the main idea and several supporting ideas in an increasing information text.
Inappropriate policy formulation is one of the causes of the failure to obtain proper literacy achievements. Literacy is not only by getting used to reading books for a certain duration of time before learning begins as stated in the Regulation of the Ministry of Education of the Republic of Indonesia number 23 of 2015 concerning the Development of PBP Characteristics contained in the attachment. But it can be done with various activities: watching movies and news, reading novels, short stories, articles, news, and information after discussing it together and making a conclusion (review). Seeking information by visiting museums, industry, and traditional and modern markets, most importantly, must be in accordance with the objectives that have been formulated within the scope of the literacy curriculum.
Subjects, processes, and learning targets in Indonesia are regulated in the 2013 curriculum as stated in the 2016 Minister of Education and Culture Regulation number 20, 21, and its attachments, also the Decree of the Minister of Education and Culture of the Republic of Indonesia in 2020 number 719 and its attachments. In the regulation, especially the science subject, junior high school students are declared to have passed if they have mastered: 35 basic cognitive competencies and 35 skills in a total of 3 years of study. While the concept of the Solar System is part of the subject of science learning in grade 7 basic competencies 3.11 and 4.11. KD 3.11 is Analyzing the solar system, the rotation and revolution of the earth, the rotation and revolution of the moon, and their impact on life on earth, while KD 4.11. Presenting works on the impact of the rotation and revolution of the earth and moon on life on earth, based on the results of observations or searches of various sources of information. Considering the complexity of the material and the demands for cognitive achievement at the C4 level of Bloom's taxonomy, the time division stated in the regulation is not sufficient, so it is cognitively incomplete. The mastery of cognitive competence and skills in the education unit will affect the mastery of scientific literacy competence.

CONCLUSION
The scientific literacy assessment instrument based on the Solar System concept AKM level 4 consists of 40 questions that the researchers developed that have met the eligibility criteria as an AKM reading literacy assessment instrument for level 4 th Solar System concepts which is a literacy assessment instrument for reading Solar System Science information for class VIII junior high school and the equivalent. The question is valid based on the assessment of 3 (three) validators and reliable based on Cronbach's Alpha analysis.
The description of literacy competencies for students in class VIII SMP based on the AKM level 4 th assessment of information literacy in Solar System Science is generally at level 2 nd or the same as students in grades 3 rd -4 th SD (Elementary School). This explains that participants only complete the competencies: Finding information and sub-competencies: Accessing and searching for information in the text; search and selecting relevant information. His highest achievement is in competence: Understanding, sub-competency: comprehending text literally: explaining the main idea and some supporting ideas in information text which continues to increase.  (who, when, where, why, how) in the information text that continues to increase according to the level. Search and select relevant information: Identify effective keywords to find relevant sources of information in information texts that continue to increase according to level.
Take the text literally: Explain the main idea and some supporting ideas in the information text which continues to increase according to the level. Compile inferences, make connections and predict both singular and plural texts: Concluding changes in events, procedures, ideas or concepts in information texts that continue to increase according to level. Prepare inferences (conclusions) and predictions based on supporting elements (graphs, pictures, tables, etc.) accompanied by supporting evidence in the information text which continues to increase according to the level. Comparing the main things (eg differences in events, procedures, characteristics of objects) in the information text which continues to increase according to the level.
Assessing the quality and credibility of the content in both singular and plural informational texts: Assessing the quality of informational texts based on his personal experience in reading texts which continues to increase according to the level (eg identifying assumptions/opinions from facts). Assessing the accuracy of visual or non-visual information in the information text which continues to increase according to the level. Assess in-text presentation format: Assessing the suitability of choosing colors, layouts, and other visual supports (graphics, tables, etc.) in conveying certain messages/topics in information texts that continue to increase according to level Reflecting discourse content for decision making, making choices, and relating text content to personal experience: Reflecting the new knowledge obtained from the information text on the knowledge it has which continues to increase according to the level.  3  3  3  3  3  3  3  3  3  3  3  3 3.0 V   Information: Score 4: very valid (SV), Score 3: valid (V), Score 2: quite valid (CV), Score 1: less valid (CV), Scores in decimal form are rounded off.