Validity and Practicality of Scientific Literacy Assessment Instruments Based on Tornado Physics Enrichment Book

ABSTRACT


INTRODUCTION
Literacy describes a person's ability to identify, understand, interpret, create, communicate, and use knowledge in various contexts (Igwe et al., 2020;Nurwidodo et al., 2021).Mastery of literacy is an essential indicator for increasing the achievements of the younger generation in achieving success (Aini & Mufadhdhal, 2022;Herawati et al., 2019;Mariati et al., 2022).Literacy must be cultivated as early as possible because it is the main capital for creating an intelligent and cultured nation (Irianto & Febrianti, 2017).There are many types of literacy, including scientific literacy.
The PISA framework from the OECD (2018) defines scientific literacy as the ability to use scientific knowledge in issues that develop in society, explain scientific phenomena, evaluate and design scientific investigations, and draw conclusions based on evidence to understand and make decisions regarding nature and its changes due to human activities so that it becomes a reflective society.Therefore, people with scientific literacy are willing to engage in scientific communication about science and technology (Singh & Singh, 2016), which requires competencies to scientifically explain phenomena, evaluate and design scientific investigations, and interpret data and evidence scientifically (OECD, 2018).Scientific literacy will assist everyone in critically addressing problems as phenomena that frequently occur, particularly those related to science and technology (Abd-El-Khalick et al., 2004;Novitasari, 2018).
PISA identifies scientific literacy as having four interconnected aspects: context, knowledge, competence, and attitude (OECD, 2018).The context aspect, which includes personal, local/national, and global current and historical issues, necessitates some understanding of science and technology (OECD, 2018).Science competence is divided into three categories: scientific phenomena, design and evaluation of scientific research, and interpretation of scientific data and evidence (Wen et al., 2020).The knowledge of science aspect of scientific literacy includes those facts, concepts, principles, laws, hypotheses, theories, and models of science necessary for a scientifically literate individual (Pagan et al., 2023).The attitude aspect is a set of attitudes towards science indicated by an interest in science and technology, valuing of scientific approaches to inquiry, where appropriate, and perception and awareness of environmental issues (Cansiz & Cansiz, 2019) Toharudin et al. ( 2011) argue that individuals with good scientific literacy competence will be able to live in a society currently controlled by the development of science and technology.Someone who does not have scientific literacy skills will face difficulties in making decisions on various phenomena that occur in society, as well as being left behind by increasingly modern technological developments (Asrizal & Dewi, 2018;Permanasari, 2016;Pratiwi et al., 2019).Without scientific literacy, current and future generations of youth are at risk of lacking the ability to compete globally (Mulbar & Bahri, 2021;Sumanik et al., 2021).Ridwan et al. (2015) and Hodson et al. (2003) revealed in their research that scientific literacy prepares citizens to become responsible and sensitive to the problems around them (responsible citizenship).This is in line with the opinion of Syamsiah et al. (2016) that not only are scientists required to think scientifically, but all people also need to have the ability to think scientifically and be willing to be involved in scientific and technological development issues.Thus, scientific literacy is an important thing that needs to be developed for students in every country (Choi et al., 2011).
Scientific literacy skills also cover the field of physics (Mukharomah et al., 2021).Physics is a science that studies natural phenomena, such as natural disasters (Wang et al., 2020;Yani & Wahyono, 2020).One method for integrating natural phenomena into physics learning is integrating tornado phenomena into enrichment books (Firaina & Fauzi, 2021).The integration of natural phenomena into enrichment books makes learning more contextual.
Enrichment books can enrich and increase mastery of science, technology, and art, as well as skills in shaping the personalities of students, educators, education administrators, and other communities.Enrichment books contain certain discussions in the broader and/or deeper curriculum (Subyantoro & Nuryatin, 2015).This book is not entirely structured based on a good curriculum in terms of objectives, subject matter, and method of presentation (Desnita et al., 2016).This book is helpful for students who have difficulty understanding certain subjects in primary subject textbooks (Sitepu, 2012).
One of the enrichment books that the author has developed is the Tornado Physics enrichment book which integrates natural disasters into learning physics.This enrichment book is expected to increase students' scientific literacy.To see how students achieve scientific literacy after using this enrichment book, that is necessary to do an assessment.Enrichment books were chosen because they can be integrated with natural events and provide broader and deeper information and explanations.
Assessment activities are an important and integral component of teaching and learning activities (Abi Hamid, 2016;Ani, 2013;Bariah, 2019).Assessment activities aim to discover improvements or progress, deficiencies, obstacles, and results achieved in a learning process (Indrawati, 2018;Sari, 2018).The results of the assessment in education show the quality of the knowledge and skills that have been achieved (Indrawati, 2018).Assessment is carried out to obtain information about the achievement of the results of students' learning process following the goals set.An instrument is needed to measure all aspects of scientific literacy in the assessment process.
PISA  2016) on all aspects of scientific literacy, which discovered that the profile of students' scientific literacy abilities remained low.The findings of Utama et al.'s (2019) research show that students can still not be scientifically literate, as evidenced by the low scores obtained when working on scientific literacy questions.Given the importance of mastering scientific literacy for students to survive in the face of global competition, the findings of this study are very concerning (Bagasta et al., 2018).
One of the reasons for the low level of scientific literacy of Indonesian students is that the assessment instruments used so far have only been used to measure the cognitive aspect (C2), with very few questions to measure cognitive level (C4) to (C6), even though the scientific literacy questions are at the level dimension cognitive (Astuti et al., 2017).Students must become more accustomed to working on discourse questions, such as the PISA model questions (Akbarudin & Kurniawati, 2023).Sumaryatun et al. (2016) explained the importance of learning-oriented toward mastering students' scientific literacy so that students become accustomed to working on scientific literacy-based questions.Based on the description above, developing a scientific literacy assessment instrument is necessary.
One of the studies that developed scientific literacy instruments was conducted by Chasanah et al. (2022), who developed a valid and reliable science literacy assessment instrument to measure students' scientific literacy abilities.Indrawati's (2018) research led to the development an instrument for assessing students' physics literacy on sound waves.Rusilowati & Linuwih ( 2016) developed an energy-themed scientific literacy assessment instrument as a result of their research.A scientific literacy assessment instrument based on the tornado physics enrichment book had yet to be developed based on several studies that presented scientific literacy assessment instruments.
Developing a scientific literacy assessment instrument is necessary based on the description above.The scientific literacy assessment instrument was developed based on the tornado physics enrichment book.The instrument was created with four components in mind: scientific literacy, context, competence, knowledge, and attitude toward science, and it will be tested for validity and practicality.

METHODS
The research method used was research and development.The product developed is a scientific literacy instrument based on the Tornado Physics Enrichment Book consisting of four aspects: Context, Competencies, Knowledge, and Attitudes.The ADDIE model was used in this research.Analyze, Design, Develop, Implement, and Evaluate are the phases of the ADDIE model.These stages are in order, each depending on successfully completing the previous one.Furthermore, the ADDIE model is an iterative feedback model (Figure 1), which means that the evaluation phase results are returned to feedback, closing the loop and allowing for further refinement of the learning product (Welty, 2008).The first step is to conduct an analysis (see Fig. 1).At this stage, activities include gathering information about the instrument that will be developed.Based on observations and interviews with physics teachers, it was discovered that scientific literacy instruments are not available in schools, so scientific literacy has yet to be measured.The second analysis is curriculum analysis.There are several competencies related to tornadoes.The results of this analysis provide an opportunity to develop instruments related to the topics contained in these competencies.
The second stage is design.The activity at this stage is the development of the instrument's design.This stage's goals are grids of scientific literacy instruments.The grids are made based on the scientific literacy aspect.The aspects used are Context, Competencies, Knowledge, and Attitudes.
Development is the third stage.Instrument development, internal validation, expert validation, and product revision were all completed.This expert validation validated the instrument's content, materials, and language.A questionnaire was used to collect data for expert validation.Aiken's V equation is used to calculate the value of expert validity.

𝑉 = ∑ 𝑠 [𝑛(𝑐 − 1)]
With,  =  −  0 Information:  0 =the lowest number of validity assessments (in this case = 1) c =the highest number of validity assessments (in this case = 4) r =Numbers given by the validator Based on the calculation of the V index, instruments can be classified.The validity is lacking if the index is less than or equal to 0.4; 0.4-0.8indicates moderate validity and greater than 0.8 indicates very valid (Retnawati, 2016).
The fourth stage is implementation.Activities are carried out at this stage to determine the practicality of the developed instrument.The instrument's practicality is demonstrated by teachers' and students' feedback after use.Data from teachers' and students' feedback were collected using a questionnaire.The trial was conducted in public senior high school 2 Padang Panjang with three teachers and 26 students.Using the equation below, calculate the value of the instrument's practicality.
The evaluation stage is the final stage.This stage includes formative and summative evaluations.Formative evaluation is performed at each ADDIE development stage, including the analysis, design, development, and implementation stages.The summative evaluation seeks to determine the overall instrument development's success.In this article, we present data on theoretical validity by experts and practicality by teachers and students.

RESULTS AND DISCUSSION
This research produced a scientific literacy assessment instrument based on the tornado physics enrichment book.The instrument developed contains four aspects of scientific literacy with 21 items: two items of context, five items of competence, eight items of knowledge, and six items of attitude toward science.Aspects of context, competency, and knowledge were developed in essay tests, while attitudes toward science were developed in self-assessment questionnaires.
The first step is analysis.Based on observations and interviews conducted with physics teachers, it was found that scientific literacy instruments are not available at schools, so measurement of scientific literacy has yet to be carried out.The second analysis is curriculum analysis.There are several competencies related to tornadoes.The results of this analysis are shown in Table 1.  1 shows that tornadoes are closely related to physics.In a tornado, there is the concept of energy because every object with mass that moves has kinetic energy.Tornadoes occur due to differences in temperature in a high area and the occurrence of heat transfer by convection.Global warming will increase the potential for tornadoes to occur.
The design stage comes next.The activity at this stage is the development of the instrument's design.The grids are made based on the scientific literacy aspect.The grids are shown in Table 2. Development is the third stage.Instrument development, internal validation, expert validation, and product revision were among the tasks completed.Internal validity is a team effort to improve the assessment instrument.The following is an example of the results of the internal validation revision are shown in Table 3. Internal validation aims to refine the draft instrument by discussing it among team members so that the instrument validated by the expert has been revised in collaboration with the team members.
A good instrument is inseparable from several aspects of its preparation: material, construction, and language (Retnawati, 2016).The expert validation was performed to validate the instrument's content, construct, and language; three instrument experts conducted this test.A questionnaire was used to collect data from expert validation.Aiken's V equation is used to calculate the value of expert validity.Therefore, the scientific literacy assessment instrument based on the tornado physics enrichment book that has been developed is validated by three validators as a theoretical feasibility test.Instrument revision was carried out according to the validator's suggestion.The following is an example of the results of the theoretical validation are shown in Table 4.

No.
Before After 1 Indonesia is located along the equator, so a tornado has great potential.Explain how geographical location and conditions affect the potential for a tornado to occur in Indonesia.
Indonesia is an archipelagic country located between the Asian Continent and the Australian Continent.It is located the Indian Ocean and the Pacific Ocean.The geographical location influences changes in Asian and Australian winds, which alternate towards the Indonesian equator.Explain how geographical location and conditions affect the potential for a tornado to occur in Indonesia.
6 Indonesia has a tropical climate with the rainy and the dry seasons.Between these two seasons, there is a transition season, or what is commonly called the transition season.The following is a table of the occurrence of tornadoes in Indonesia throughout 2021.
Indonesia has a tropical climate with the rainy and the dry seasons.There is a transition season, also known as the transition season, between these two seasons.Based on data from the National Disaster Management Agency (BNPB), the following table shows tornado events in Indonesia throughout 2021.
No. Before After Based on the table of the occurrence of tornadoes in Indonesia throughout 2021, draw a graph of the occurrence of tornadoes, and from the graph that has been made, interpret the occurrence of tornadoes throughout 2021.
Based on the table of the tornado events in Indonesia throughout 2021, draw a graph of the occurrence of tornadoes and provide an interpretation of the occurrence throughout 2021.
The results of the validation analysis using Aiken's V equation are shown in Figure 2. The expert validity data analysis results show that the scientific literacy assessment instrument developed meets the very valid criteria with an average validity value of 0.85.That is, the environmental literacy assessment instrument is used.This is consistent with Kristyasari (2021), which states that a valid instrument can measure what is to be measured.Regarding content, the developed scientific literacy assessment instrument has a validity value of 0.82 with very valid criteria.It shows that the questions developed are per the 2013 curriculum, main competencies, and based competencies, and the question indicators represent each scientific literacy.Valid criteria for scientific literacy assessment instruments also show that the material follows the competencies to be achieved (Helendra & Sari, 2021;Yulianawati, 2020).
The obtained validity value of construction is 0.86, with very valid criteria.The scientific literacy-based questions developed are brief, precise, and firm, and the answer options are homogeneous and logical.The aspects of being measured must be apparent so that different interpretations than the author intended do not arise (Claudhya et al., 2021).According to the language aspect, the validity value obtained is 0.89, with very valid criteria.That means the language used in this assessment instrument follows excellent and correct Indonesian language rules, is communicative, lacks double meanings, and employs common words and correct punctuation (Silalahi, 2020).
The fourth stage is implementation.The implementation stage is carried out after the product developed is valid and revised according to the validator's suggestions.At this stage, everything developed is declared valid to be applied.The instrument's practicality is seen in teachers' and students' feedback after using it.Three teachers and 26 students conducted the practicality test.Table 5 shows the results of the teacher's practicality test.Table 5 shows the practicality of the assessment instrument is rated at 85.3 on average by teachers, indicating that it is very practical.That is, the assessment instrument developed is easy to use, attractive, and efficient use.So, this scientific literacy assessment instrument is practical for teachers.A practicality questionnaire for using the assessment instrument was also given to students.An analysis of the practicality of the use assessment instruments by students can be seen in Table 6.Table 6 shows that the practicality test data analysis on students yielded an 85.3% practicality value using very practical criteria.This criterion shows that the scientific literacy assessment instrument can already be used as an assessment instrument for learning in schools because it has an overall practical value in each aspect.This practicality value is the average of several aspects in the practicality sheet, including ease of use of the assessment instrument, attractiveness, and efficiency of the use of the assessment instrument.The average value of the practicality results shows that the scientific literacy-based assessment instrument meets the high practicality criteria.Learning devices that meet the requirements are very practical, meaning that all learning device components developed in this study are feasible to be implemented in the learning process in the classroom (Badrulaini et al., 2020;Destiana et al., 2020).The assessment instrument was tested and found to be valid and practical.That means the instruments developed are feasible and practicable for assessing students' scientific literacy at school.
Many scientific assessment instruments are made for the learning process, such as science literacy assessment instruments to measure students' scientific literacy abilities (Chasanah et al., 2022), the instrument for assessing students' physics literacy on the subject of sound waves (Indrawati, 2018), and an energy-themed scientific literacy assessment instrument (Rusilowati & Linuwih, 2016).In this research, the scientific literacy assessment instrument was used to measure students' scientific literacy ability after using the tornado physics enrichment book and developed based on the tornado physics enrichment book.The instrument items are presented in scientific discourse, whether in phenomena, images, graphics, or the like.
This research is expected to provide material for research studies relevant to other research, both related to further research that is developing and similar research that is expanding.This research contributes positively to the world of education by advancing education to improve the quality of learning in schools.This research also contributes to varying assessment instruments to measure student achievement.Limitations in this study include the small number of respondents (only 26 students), which still needs to be more to describe the actual situation.

CONCLUSION AND SUGGESTION
Based on the study results, the instrument developed is valid and practical.The average result of the theoretical validity for the scientific literacy assessment instrument is 0.85.With very practical criteria, a practicality value of 83.7% was obtained in the data analysis for the practicality test for teachers.With very practical criteria, the practicality test data analysis on students yielded a practicality value of 85.3%.Teachers are expected to be able to use the test instruments developed to measure students' scientific literacy abilities to train and develop students' scientific literacy abilities.Furthermore, it is necessary to develop scientific literacy assessment instruments on other natural disaster topics.

AUTHOR CONTRIBUTIONS
VM and DD conceptualize ideas, prepare designs, and draft articles.DD designed a scientific literacy instrument.VM researched the development of scientific literacy assessment instruments in high schools.

Figure 1 .
Figure 1.The Research Procedure


Information related to the instrument to be developed  Curriculum analysis to find competencies related to tornadoes. Study of the contents of the book  Develop the grids of scientific literacy and environmental literacy instruments.Draf instrument, Internal validation, Expert validation Design Results of practical analysis Evaluation Formative and summative evaluation

Table 1 .
Tornado Integration Matrix

Table 2 .
The Grids of Scientific Literacy Assessment

Table 3 .
Examples of Internal Validation Results

Table 5 .
Results of the Practicality Analysis of Science Literacy Assessment Instruments by Teachers

Table 6 .
Results of the Practicality Analysis of Science Literacy Assessment Instruments by Students