Analyzing the readability of English for Nusantara textbook for grade seven using the WebFX

: English for Nusantara (EN) books are textbooks the Indonesian government provides and serve as educational materials that teachers can use to teach and learn English in Indonesian junior high schools. Before using them, teachers must evaluate their readability since inappropriate materials can demotivate students and enable them to achieve teaching and learning goals. This study aimed to fill the gap by evaluating the readability of EN. Using a sample EN for grade seven, this study brought a research question: what level of readability of English for Nusantara is. In achieving this goal, a qualitative study was employed using a free online readability tool, the WebFX to measure the text readability level. Among the texts in EN for grade seven, 18 were analyzed using the measurement formula in the above tools. The study found that the readability level was high, and the texts were easy to read. The limitations of the study and suggestions for future research were also discussed.


A. Introduction
Readability is one of the essential aspects that someone should consider when designing and evaluating a textbook since the textbook's content is in the form of written texts.As stated by Dubay (2004), readability defines the level of easiness of a text.A text with a high level of readability will be easy to understand regardless of the language proficiency of the readers (Crossley et al., 2008;Zamanian & Heydari, 2012).The readability of text can also impact someone's motivation in reading the texts (DuBay, 2004).In other words, the readability of a text affects not only the ability to understand a text but also the motivation to read the text.Therefore, the readability of a text plays a pivotal role in an educational context.Vol. 28, No. 1a (Special Issue), 2023, 101-116 In education, especially English language teaching and learning, such as English as a second or foreign language (EFL/ ESL) context, readability brings abundant benefits.The advantage of measuring the readability of the text is beyond text comprehension.In an educational context, having readability skills, in particular, assists educators in choosing proper materials in the form of texts before giving them to students.Authors who produce pieces of text can write works accessible to the target audience when they apply the principles of readability and the readability formula to analyze the texts they produce (Zamanian & Heydari, 2012).The other function of readability is to measure students' work to understand their language proficiency level and beyond (Mahadini et al., 2021).

| INSANIA : Jurnal Pemikiran Alternatif Kependidikan
Further, Dubay (2004) believes that the principles used in readability allow teachers to assess student's reading skills and put them following their reading level.The preceding statements are some benefits of measuring the readability of texts other than just aiming to comprehend the content of a text.Students can also benefit from knowing the tools as they can use them to select readings independently, following their English proficiency level.As stated in the previous sentence, failing to meet the level can hinder students from decoding texts, prohibiting them from comprehending them.In the classroom context, this failure can hinder students from achieving the teaching-learning goals set by teachers as the unacceptable levels of readability of materials given by teachers (Mesmer, 2005).
Research on readability began in 1880, when measuring readability was conducted manually.In the initial work, Prof. Sherman counted the number of sentences from some literary works (DuBay, 2004).Due to the development of science and technology, researchers developed readability formulas to analyze texts.Some of them are the Flesch Reading Ease formula (Flesch, 1948), Coh-Metrix developed by Graesser and McNamara (Graesser et al., 2004), the Miyazaki (Greenfield, 2004), and LXPER Index (Lee & Hyung-Jong, 2020).However, technology has shifted the way in measuring readability where readily available online tools can be accessed.Some of those tools are the WebFX (https://www.webfx.com/tools/read-able/), the Readable (https://app.readable.com/text/),and the readability calculator at Online-Utility.org(https://www.online-utility.org/english/readability_test_and_improve.jsp).Those online tools show instant results when measuring the readability level of a text based on readability formulas such as the Dale-Chall score, Fry Readability Grade Level, SMOG Grade, Flesch-Kincaid Grade Level, Gunning Fog Scale Level, and other formulas.
In an educational context, exploring readability used for educational purposes has become one of the topics in past research.Based on the review of the past research, there were two major themes within the research under that topic.Those themes are about using readability tools for measuring the readability of texts (Morales, 2020;Handayani et al., 2021;Kodom & Pearl, 2019;Mohebbi et al., 2017;Solnyshkina et al., 2017) and developing readability tools (Crossley et al., 2008;Greenfield, 2004;Kotani & Yoshimi, 2011;Lee & Hyung-Jong, 2020).The type of studies were empirical and non-empirical studies.Most of them are quantitative studies.Hence, there is a need to study readability topics other than using a quantitative approach, such as using qualitative or a combined method to balance types of research and explore and get a deeper understanding of an issue (Creswell, 2015).Further, most previous studies involved only one or two readability measurements and did not use multiple measuring approaches, so the reliability of readability results might be affected.In other words, using multiple readability measurements is deemed necessary.
Among the readily available tools, the WebFX is the readability tool used to test the readability of texts.Based on the previous study, another discipline other than the area of ESL/ EFL context used the tool to test the text's readability.A study by Mason et al. (2021) is an excellent example of how the readability test tool benefits medical health.The tool assisted in assessing the readability of some health websites for their readers.One of the study's findings was that the websites assessed needed to be more suitable for those lacking in English (Mason et al., 2021), which means that the websites had low readability.
The study's findings were also in line with the study conducted by Oydanich et al. (2022), who also explored the use of the tool in analyzing websites that provided information on health topics related to dry eye diseases.Using the tool eased the researchers in measuring the website's texts, and they found that the texts on the websites needed to be easier to read (Oydanich et al., 2022).Seeing how useful the tool is, the area of education also takes advantage of its great benefits.However, the use of readability tools, particularly WebFX, to analyze the readability of a textbook still needed to be improved in the previous study.Aiming to provide materials for teaching English in Indonesia, the Indonesian government has published The Nusantara Books, consisting of three different levels of English for Nusantara for grades seven to grade nine.The government provided the book for free to Indonesian students, and teachers served as a textbook teaching English as EFL/ ESL for junior high school students.Junior high schools in Indonesia have used the books.
However, the past literature that focuses on exploring the use of the book has yet to be explored maximally, particularly the readability of the books.This notion aligns with Odo (2018) that more research is needed to explore the readability of textbooks used for teaching and learning purposes.Moreover, a study that assesses the readability of a textbook using an online readability measurement tool still needs to be improved; further study on this topic needs exploration (Hakim et al., 2021).
Some possible reasons explain why the past literature could be more extensive in studying English for Nusantara textbooks.First, the textbook was published in 2022 and was written due to the change of the Merdeka curriculum in Indonesia.The using of this textbook in Indonesia was still on its way.Second, using the textbook for teaching and learning was mandatory since teachers could not use the textbook but develop their own materials or modules instead.Moreover, adopting the Merdeka curriculum allowed teachers to adapt, adopt, or develop teaching materials that suit their school needs (Kemdikbud, 2022).Therefore, seminal works exploring the use of the textbook, mainly English for Nusantara for grade seven, still need to be completed.Apart from the mentioned reason, a study by Johan et al. (2022) explored the textbook in their study.
However, their study focused on studying the speech act in the book (Johan et al., 2022) and did not connect on the readability of the texts in the textbook.Further, the previous study also should have mentioned which textbook grade was analyzed.Forfilling this gap, this study focused on measuring the readability of English for Nusantara textbook grade seven using the WebFX readability tools.To be more specific, this study sought to answer a research question as follows: What is the level of English readability for Nusantara based on the readability formula in the WebFX readability tool?.

B. Method
The study aimed to explore the readability of an English textbook for Nusantara for teaching English as EFL/ ESL at junior high school level, grade seven.As mentioned, the study sought to understand the textbook's readability level and compared the readability measurement using two different readability tools.This study employed qualitative research using document or content analysis since the subject of the study was printed material in the form of a textbook (Ary et al., 2010).The instrument used in this study was a set of readability formulas in a tool, WebFX.Even though the results of the readability assessment using the WebFX are in numerical data, the analysis results were interpreted qualitatively.Below are the details of the methodology section.

Materials
Out of three English for Nusantara books, a textbook for grade seven was selected.
The book was chosen due to being publicly provided online and being the first English for the Nusantara book series.In addition, the English for Nusantara for grade seven was studied purposively due to the nature of the study, which aimed to get an in-depth understanding of the issue (Creswell, 2015), particularly the readability of the texts given in the book.In addition, the textbook served as a material resource for teaching English as EFL/ ESL for grade seven, where English was a mandatory subject at the high school level.
The 18 passages from the textbook were taken as samples and analyzed (see Table 1).
However, the focus of the research was on analyzing longer texts, such as monologue texts.
The texts selected consisted of at least 30 words in a paragraph.The reason was that the goal of the readability measurement in this study was for passages rather than a standalone sentence to get a thorough analysis of the result.

The instruments
This research employed an online readability tool, WebFX (see Figure 1), to answer the research questions.The reasons for using the tool were because 1) the tool was easily accessed as it was provided online, and 2) the tool contained multiple measurement approaches that could ensure the validity and the reliability of the measurement results.
The latter reason also is why this research employed the tool when analyzing the textbook.
Even though there were some debates on the use of the tools within the Indonesian context, using the tools with multiple measurements ensured the accuracy of the analysis rather than relying on only one instrument (DuBay, 2004).WebFX was more of a digital marketing company (The WebFX, n.d.).However, the company also provided a readability test tool to measure the readability of texts.The tool was provided online so that the users could copy and paste texts they wanted to measure or copy the website address they wished to explore its content.Under the tool, there were six readily readability indicators.They were the Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, Gunning Fog Score, Coleman Liau Index, Automated Readability Index (ARI), and SMOG Index.Each formula measured different aspects of a text.For instance, a Flesch Reading Ease, which measured text readability by counting sentences and word numbers, gave a score range from 1 to 100.The higher the score results, the easier the texts are, according to the education system in the United States (Rakt, 2019).
Meanwhile, the Coleman-Liau formula measured the length of letters per word in a text to predict the readability of a text (Coleman & Liau, 1975).In this formula, the lower the score, the easier the text to read.
There were three ways to measure the readability of a text using WebFX.The first was by entering the URL of the text assessed.This way was suitable for measuring text taken from online websites.However, a text not provided online can still be measured as the tool provides a way to assess the readability of a text under this condition.The text assessed should be copied and pasted under the 'Enter Text' menu in the dashboard of the WebFX and is calculated for its readability.In addition to the two above, the tool also provided an option where a link can be embedded in a website that someone has developed.This way, the readers of the text developed can measure the readability score of the text.

Procedure
To answer the research questions, this research followed the following steps.First, one textbook, English for Nusantara (EN), for grade seven, was selected among the three books of EN.The book contained some kinds of texts such as dialogue, short functional texts, and monologue texts.Among them, 18 texts that met the earlier criteria, such as the text length and type of text were selected.After taking some samples, the texts were analyzed for their readability.Each text was analyzed using WebFX.
To analyze each text, the texts were copied and pasted to the website at https://www.webfx.com/tools/read-able/.Each text was analyzed thoroughly using all the measurements provided in each tool.The analysis resulted in grades that were interpreted differently for each formula (see Table 2).In each measurement, the data were transferred into a table.Then, a conclusion emerged after analyzing each text (see Table 5).

Table 2. The interpretation of the readability formula
The Formula The Interpretation Flesch Kincaid Reading Ease The scale is 0-100.The higher the score the easier the text to read and the other way around.Flesch Kincaid Grade Level The Flesch-Kincaid Grade Level is equivalent to the US grade level of education.
Gunning Fog Score The higher the score, the text will be difficult to read.For instance, the Fog index score is 12 for high school students.

SMOG Index
The text analyzed should be 30 sentence long.The ideal score for readability is 7 or 8.The score above 12 is difficult to read.

Coleman Liau Index
The higher the score, the text will be difficult to read.For instance, the scale 10 is appropriate for a 10-11 th grade high school student.

Automated Readability Index
The scale is 1-14.The higher the score the easier the text to read and the other way around.

Data analysis
The 18 texts taken from the textbook were analyzed for their readability using the measurement formula in WebFX.The 18 texts were taken as the sample due to their length, which consisted of at least 30 words, to thoroughly analyze the result.The total measurement formula in the WebFX was six (see Table 2).A label for each measurement formula was given to make it easier to analyze the data.The label was in the form of a number from number 1 to number 6 (see Table 3).Each formula evaluated different aspects of texts, and the results were compared to the data derived from the rest of the measurement formula.The assessment outcomes in one tool resulted in the level of readability of the texts assessed.Then, a translation of the score taken from the readability test was given, which determined the level of easiness of the text.Finally, the theme was derived from the category generated, which reflected the readability of the text overall.

C. Result and Discussion
This study explored the readability of a textbook, English for Nusantara, for grade seven using a readability tool, the WebFX.The research question that the study sought to focus on was understanding the textbook's readability level.Below are the findings and the discussions of the study.

The results of the measurement using the WebFX readability formula
In finding the level of the text readability of the English for Nusantara, particularly for grade seven, the researcher followed the procedure explained in the previous section.
After measuring each selected text through the tool, the researcher created a table containing quantitative data of each measurement formula.Each of the formulas is labeled from 1 to 6. Below are the details of the findings that consist of the raw data measurement in the form of quantitative data and the analysis of the quantitative data to a narrative interpretation.
Table 4 projects the results of analyzing 18 texts using six different readability formulas in WebFX.They are measured using the Flesch Kincaid Grade Level, Gunning Fog Score, SMOG Index, Coleman Liau Index, Automated Readability Index, and Flesch Kincaid Reading Ease.As the table shows, each text measured using a specific formula results in different numbers.Only a few texts, such as Text 7 and Text 13, measured using the Gunning Fog Score, have the same score (score 9).Meanwhile, the rest of the texts have diverse scores, even though they are measured using the same formula.However, WebFX does not rely solely on one specific formula to conclude the text level.The six formulas are combined when measuring a text.Meanwhile, the interpretation of the score is given in Table 5.From the table, it can be concluded that the texts analyzed were taken from English for Nusantara and have various levels.The texts are expected to cater to readers ranging from 9 to 16 years old.
The first level is the texts that 12 to 13-year-old students can read.Those texts are numbered 1, 3, 4, 14, 15, and 18.The next level is the texts that were readable to 11 to 12year-old students, which are texts number 2, 8, 11, and 12.The text numbered 5 is the only text readable to those aged 9 to 10.It is the same with text number 17, the only text in the textbook suitable for students aged 10 to 11. Texts 6 and 16 are suitable for students aged 15 to 16, while texts 7 and 10 are suitable for those aged 13 to 14.Of 18 texts, 9 and 13 are suitable for students aged 14 to 15.The details of the interpretation can be seen in Table 5.
Table 5.The score interpretation

Text
The Interpretation 1 The text should be understood by 12 to 13 years old. 2 The text should be understood by 11 to 12 years old. 3 The text should be understood by 12 to 13 years old. 4 The text should be understood by 12 to 13 years old.5 The text should be understood by 9 to 10 years old.6 The text should be understood by 15 to 16 years old.7 The text should be understood by 13 to 14 years old.8 The text should be understood by 11 to 12 years old.9 The text should be understood by 14 to 15 years old.10 The text should be understood by 13 to 14 years old.11 The text should be understood by 11 to 12 years old.12 The text should be understood by 11 to 12 years old.13 The text should be understood by 14 to 15 years old.14 The text should be understood by 12 to 13 years old.15 The text should be understood by 12 to 13 years old.16 The text should be understood by 15 to 16 years old.17 The text should be understood by 10 to 11 years old.18 The text should be understood by 12 to 13 years old.

The level of readability of English for Nusantara based on the WebFX
To explore the readability of English for Nusantara, one of the English books used for teaching English as a foreign language (EFL) at the secondary level in Indonesia, the WebFX was employed.The tool had six formulas to measure a text readability level.Each of them functions in a different part of a text.For instance, the Flesh Reading Ease measures a text's readability by counting the text's sentences and words.Meanwhile, the Gunning Fog Score gives results on a text based on different measurements, such as the variation of words and the number of simple sentences.Each observes texts differently yet has the same goal: calculating a text's readability.Therefore, it can be concluded that the measurement of the texts in English for Nusantara is valid as there is still a debate on the validity of readability level when using only one readability formula, particularly its validity aspect (Gómez & Sánchez-Lafuente, 2019).The fact that the texts are measured using formulas intended to measure texts' readability ensures the measurement's validity.
Regarding the measurement results, the texts in English for Nusantara for grade seven have various levels.Specifically, five levels range from a text suitable for students aged 9 to 10 to those aged 15 to 16.However, most of the texts are suitable for students in grade seven.When this happens, students will engage with the materials (Kim & Hall, 2002).Meanwhile, the rest of the texts are suitable for the grade above them (four texts).
Those four texts are numbered 6, 9, 13, and 16.However, the texts above the 'students' level were either not the majority texts or contained vocabulary that was not English.For instance, Text number 6 explained Indonesian geographical conditions where the explanations contained words, phrases, or sentence structures that were possibly the tools categorized as strenuous.The examples of those words were Bolon, Sumatera, Joglo, Gadang, and Bale Sakenem, while the phrases were different kinds of 'cultures' and different parts of the country.Apart from that, as stated by Morgan et al. (2000), giving texts above students' proficiency can assist teachers in understanding their students' reading proficiency accurately.
Further, it has been argued that giving reading materials above students' proficiency level helped boost their English proficiency.Morgan et al. (2000) postulated that the difficulty level of materials correlates to students' English proficiency.Giving students materials that might not align with their English proficiency level can assist in attaining English proficiency above their level.This idea also aligns with Kasule (2011), who believed that giving texts that slightly differ above 'level provided students challenges.In other words, the students felt motivated to read the text, and it was not a wrong choice (Kasule, 2011;Niazifar & Shakibaei, 2019).As stated by Namaziandost et al. (2019), the presence of the texts as language inputs that vary their degree of difficulty can improve students' comprehension.Therefore, the presence of texts above the level given in the English for Nusantara book for grade seven is acceptable.

Indonesian 'students' English competence
The measurement of the texts using WebFX concludes that the textbook is suitable for grade seven EFL students in Indonesia.The majority of texts meet 'students' English proficiency level.However, the measurement results reveal that some texts are below or exceed their numbers and do not exceed the majority of texts that suit the targeted students' English proficiency level.This is in line with the goal of the textbook, that the textbook aims to equip students with basic English (Damayanti et al., 2022).The English for Nusantara brings 'teenagers' life as a context.The book allows students to study and familiarize themselves with everyday topics such as school and home-related topics.This idea aligns with the Common European Framework of Reference for Languages (CEFR), one of the language standards widely used for describing language ability.It has been adopted globally as a reference for teaching languages, including English as a Foreign language in Indonesia.Based on the standard, someone needs to understand and be able to use expressions and phrases related to everyday life to attain the A1-A2 level topics such as personal information, family, and shopping (Council of Europe, 2023).In other words, the book's content aligns with the language standards widely accepted worldwide.
A textbook used for teaching EFL/ ESL needs to meet the proficiency level of the users.The data indicated that the readability of English for Nusantara for grade seven was easy to read for seventh graders who were aged 13 years old.The measurement in the tool was designed following the grade level system in the United States schools.The students aged 13 sit in middle school or junior high school, which is the same as in the Indonesia education system (Sawyer, 2018).Even though English served as a foreign language in many parts of Indonesia schools, the findings became the baseline for teachers, especially in Indonesia, before using the textbook.Even if textbooks might be slightly beyond the target ' 'learners' level, the presence of the book can still arguably motivate and challenge students to learn the target language since the progression exists as depicted in the data.

D. Conclusion
This study aims to evaluate the readability of English for Nusantara textbooks for grade seven junior high school students in Indonesia.Using the readability formulas in the WebFX, the study revealed that the level of readability of the 18 texts analyzed varied.
However, the majority of the texts were readable to the target learners.Even though the formulas used in this study were developed for the context where English served as a daily language that students encounter, the formula still applies to the context where English is a second or foreign language.The argument that came along is that the presence of the formulas in the form of readability tools allows teachers, especially, to compare and contrast their judgment upon the readability of the materials they teach English before giving them to their students.The measurement using the tools becomes the baseline to confirm ' judgment on whether to use or put aside texts they want to incorporate into their teachings.Therefore, apart from the judgment towards the validity of the formulas, the lack of them still outweighs the benefits.
From the above, this study provides insight that future researchers might want to develop readability tools that work correctly regardless of the context.Ultimately, the tools are easy to use and access, preferably online, as is the urge for online computer-based technology.The presence of materials for language teaching plays a pivotal role in the success of teaching and learning.In English language teaching, in particular, the materials such as served in the form of textbooks bring advantages to students and teachers.One of them is providing contexts for students since going or immersing themselves in their native country is not possible.However, that goal might only be achieved if the target learners can read the materials in the textbooks due to the low readability level of the materials.

Figure 1 .
Figure 1.The dashboard of the readability test toll in the WebFX

Table 1 .
The sample of texts analyzed from English for Nusantara for grade seven

Table 3 .
The measurement formula of the readability tool and its labelling

Table 4 .
The result of the readability analysis using the WebFX