Does It Really Help? Exploring the Impact of Al-Generated Writing Assistant on the Students’ English Writing

The increasing use of tools that assist English as a Foreign Language (EFL) learners in achieving writing fluency has drawn attention to the rapidly evolving role of AI in education. This study evaluates an AI-generated writing assistant in English language learning, that is the ParagraphAI text generator, focusing on its potential impact and effectiveness for L2 learners’ writing skills. This AI-powered writing software curates writing content according to writers’ preferences. Four seventh-semester EFL students were selected using homogeneous purposive sampling. Data collection involved tests and questionnaires, with subsequent analysis including text comparison to measure Lexical Diversity indices, followed by descriptive analysis. The results indicate that while the AI writing assistant aids in correcting grammatical errors and enhancing text cohesion and coherence, it lacks content density at times. In some instances, the intended message and thoughts of the students were not effectively conveyed, leading to the inclusion of ideas unrelated to the initial topic. This study underlines the importance of considering linguistic and content-related aspects in evaluating AI-generated writing


INTRODUCTION
The rapid research and development of Artificial Intelligence (AI) in Education has undoubtedly reached its peak in the last decade (Hwang et al., 2020).Natural Language Processing (NPL) is a particular area that continuously experiences changes in educational contexts (Chen et al., 2019).NLP technology has various educational applications, including personalized reading plans, feedback on essays, example generation, and question generation (Patel, 2021).Despite the potential of AI in education, more effort is needed to ensure its effective application in the classroom setting.Chen et al. (2019) proposed an AI-based writing application, Smart Compose, which provides real-time suggestions to help users write more efficiently and improve writing quality, particularly for second language (L2) students.
In reality, while both native and L2 students share similar writing patterns, L2 students often struggle with lower-level development skills, such as grammar and vocabulary, resulting in lower-quality texts (Dizon & Gayed, 2021;Silva, 1993).Additionally, L2 students face challenges such as difficulty recalling words and translating ideas from their first language to English (Abrams & Davis, 2016;Wolfersberger, 2003).
Regardless of the approach, second language students must balance cognitive stress and digital mediating devices to complete writing tasks.Cognitive anxiety negatively affects writing performance and learning outcomes (Tsiriotakis et al., 2017), while reliance on translation applications can hinder language learners' writing skill improvement (Purnama, 2022).Over-reliance on AI writing tools may diminish critical thinking skills and creativity (Iskender, 2023;Johinke et al., 2023).Therefore, it is essential to provide students with suitable tools and resources to develop their cognitive skills and writing abilities.
In this digital era, digital technology has become essential in education, serving as a bridge between L1 and L2 writing (Scott & Mouza, 2007).Our study focuses on the ParagraphAI text generator, an AI-powered writing software that curates writing content according to writers' preferences.Unlike automated writing evaluation (AWE), which focuses on providing corrective feedback, ParagraphAI generates complete text based on the writer's instructions.We aim to explore the differences between students' initial works and those produced by the software on the same topic to prevent incoherent and incohesive texts.
Prior studies have examined the impact of AI-based writing assistants on students' writing performance, highlighting improvements in grammatical accuracy, lexical variation, and structured writing (Dizon & Gayed, 2021;Gayed et al., 2022;Mohsen & Alshahrani, 2019;Wang & Wang, 2012).However, little is known about the impact of text generators on students' writing performance.The research question of this present study is: 1.To what extent does the AI-generated writing assistant affect L2 learners' writing?
Therefore, our study compares L2 students' initial work to that generated by artificial intelligence software to understand their influences on writing performance.We aim to examine the differences between the final works and their impact on writing quality, focusing on grammatical correction, spelling, coherence, and cohesion.

AI in Academic Writing
In English, academic writing is known to be a sophisticated and integrative task that is difficult for both native and international students alike (Campbell, 2019).It is a complex process activity (Rahimi & Zhang, 2018) that combines linguistic and educational barriers (Hanauer et al., 2019).Hence, it is unsurprising that even higher education students find this activity hard and demanding, especially English as a Foreign Language (EFL) learners.To overcome this limitation, students use technology to help them produce good-quality text.As technology develops, writing with the assistance of computer-based applications has become an increasingly popular trend to facilitate students' writing.Artificial Intelligence (AI) tools can help nonnative postgraduate students improve their English academic writing skills by providing formative feedback and assessment, promoting learning behavior, and increasing technology acceptance (Nazari et al., 2021).However, there has been a surge in the growth of AI-powered writing tools in a short time.These AI-based writing tools are becoming promising new tools that students can use to aid their learning (Nazari et al., 2021).
AI automated technologies possess the capacity to exhibit human-like behavior and cognitive abilities, including learning, self-correction, and reasoning (Popenici & Kerr, 2017).Therefore, it will not be surprising to witness traditional writing lose its prestige in the near future.Integrating AI-generated writing assistants into the writing curriculum may provide access to automated writing evaluation, corrective feedback, and essay scoring, thus offering flexibility and time-saving in learning (Koltovskaia, 2020).Furthermore, higher education has begun to utilize and introduce teaching and learning platforms combined with AI applications (Pedro et al., 2018), which have an immediate effect on enhancing the learning process, input, and outcomes.These platforms provide immediate feedback, pedagogical flexibility (Cheung, 2015), plagiarism detection, formative assessment, and instructional practice (Zawacki-Richter et al., 2019).

Evolution of AI Writing Tools
The use of digital automated applications to assist students with writing is on the rise.AI, as one of the latest advancements in technology, provides new experiences in teaching and learning for both teachers and students.As the influence of AI grows in the field of education, the technological advancements of AI writing also extend.
Aspects beyond the general AI characteristics of automated written corrective feedback (AWCF), automated essay scoring (AES), and automated writing evaluation (AWE), such as social relation, motivational aspects, comprehensive analytic learning, and affective features, may further facilitate and assist learners' writing skills, noncognitive abilities, and engagement levels (Nazari et al., 2021).
There has been much research on the effect of digital writing tools on various aspects of writing.They can improve learners' writing proficiency, although direct educator guidance is needed to maximize the outcome (Moore et al., 2016).The tools also prove to be efficient for L2 learners when used in a well-structured manner (Perry, 2021), where correct instruction, guidance, and ethical behavior are involved.From the perspective of digital literacy, students who are proficient in utilizing digital tools have a good chance of developing their English academic writing performance (Hamouma & Menezla, 2019).Purcell et al. (2013) added that the positive influence of digital technology on students' writing production has the same effect on users with native-level English proficiency.In terms of the impact of digital writing assistants on students' writing performance, Dizon and Gayed (2021) found in their research on the predictive text of Grammarly that the writing assistant helped correct grammatical errors and improve lexical variation.

METHODS
This study employed a descriptive qualitative method to provide comprehensive information about the case under study (Ary et al., 2010).To investigate the impact of an AI-generated writing assistant on students' writing, we conducted a writing test.

Participants
The study took place during the undergraduate English Education program located in Banda Aceh, the capital city of Aceh Province, Indonesia.The participants comprised four seventh-semester EFL students (2 males and 2 females) selected purposively.The criteria for selecting respondents were based on homogeneous purposive sampling, considering factors such as age, culture, and status as students (Etikan et al., 2016).Additionally, these students had completed all elective courses related to writing, including various levels of writing and grammar courses.It is noteworthy that all students had studied English at intermediate and secondary school levels and had not traveled to any English-speaking countries.

Writing Test
A writing test was utilized to address the research question.The test consisted of two parts.In the first part, the students were instructed to write a descriptive text of 150-300 words or more on the topic of 'The Importance of Technology in Education' within 60 minutes, without the aid of the AI-based writing assistant.They were required to brainstorm their ideas and concepts using the mind-mapping technique beforehand.In the second part, the students were asked to rewrite their essays using the AI-generated writing assistant based on their initial mind-mapping notes.However, they were prohibited from copying their manually written text from part one.The results of both tests were compared and analyzed to assess the differences between students' original works and AI-generated writing texts on the same topic and ideas.The text generator software used by the students was called ParagraphAI.
One way of measuring students' writing ability is through fluency, which can be assessed using Lexical Diversity (LD) to examine lexical richness in terms of word types and count.LD indicates the quality of writing, vocabulary knowledge, and competence (McCarthy & Jarvis, 2010).In this research, the students' writing samples were analyzed using a web-based text analysis tool called Text Inspector (Bax, 2022), which measures LD probability in a text (vocd-D), and the Measure of Textual Lexical Diversity (MTLD).Both students' original work and AI-generated texts were measured, and the means of both types of writing were calculated to draw conclusions.Lexical diversity levels were analyzed based on Durán et al.'s (2004) lexical diversity and language development indices.
Following the writing task, the students completed an eight-question questionnaire to provide insight into their perspectives on the difficulty of writing tasks and their perceptions and attitudes toward using AI-generated writing assistants.All gathered data were summarized descriptively to draw final conclusions.

ParagraphAI
ParagraphAI is a free AI writing assistant designed to boost productivity and writing quality in over 30 languages.Available as a Chrome extension, Android app, and iOS app, it helps users create professional emails, articles, blogs, reports, and essays.The tool ensures impeccable grammar, spelling, and vocabulary, generates plagiarism-free content and helps overcome writer's block.Additionally, it can tailor and correct text to the user's preferred language and tone of voice.Once users have accessed ParagraphAI's primary webpage, they can initiate the writing process by clicking the 'Write' button (as illustrated in Figure 1).Next, they can select one of the five alternatives based on the desired text style.For this study, the 'Paragraph' alternative was chosen because the primary focus is on students' writing (as illustrated in Figure 1).The text generator can be programmed to respond to a given prompt.In contrast to ChatGPT, which tailors dialogue to conform to specific parameters such as length, format, style, level of detail, and language, ParagraphAI operates under different guidelines.It requires the writer to input specific content, after which the tool instantly generates an accurate and grammatically correct draft for any purpose (see Figure 2).Moreover, the tool can construct text that adheres to a clear and consistent structure when provided with the basic idea.The tone of the writing can be customized to the writer's preference using a slider that ranges from an informal and affable style to a pessimistic disposition.Finally, users press the 'Detect Language' button to select their preferred language before proceeding to 'Write'.

Lexical Diversity Indices
The initial segment of the presentation of the results offers evidence and comparisons of Lexical Diversity in students' writing, both in their original works and the AI-generated texts.Subsequently, we calculated the means of both types of texts and compared the overall results with Durán et al.'s (2004) indices for lexical diversity and language development.This index assesses the level of lexical diversity in one's writing and juxtaposes it with the level of language development as learners progress.Figure 3 illustrates the scale proposed by Durán et al. (2004) to measure differences in lexical diversity (D) at each level of learners' progression in their language development stage, specifically in writing.As an indicator of development, D (Figure 3) provides means and sub-ranges for both first and second language learners in their writing across various cohorts.The means measure the time spent studying the subject from 18 months to the Adult ESL and Academic text level of understanding.The subranges address the reliability range of lexical diversity mastery levels.
Figure 3. Lexical Diversity Indices (Durán et al., 2004, p. 238).  1, the mean values for the VOCD and the MTLD under the AI-based writing assistant demonstrate significantly higher average lexical diversity compared to those without the AI writing assistant.This suggests that there were performance improvements with the use of the AI writing assistant.Examining the index graphic, learners who have been studying a language for about 42 months, equivalent to seven semesters at the university level, are expected to have a diversity index above 50 (Durán et al., 2004).However, the VOCD and MTLD results from the students' original texts were all below the average expected index.These results also highlight the irony of the contrasting performance with the assistance of the AI writing assistant.There were statistically significant differences between the VOCD and MTLD analyses in writing under the AI condition.The analysis of lexical diversity shows a positive significance.The means of both indexes were above 75, indicating positive performances in students' writing.This result demonstrates that students who used digital writing aids have a higher chance of scoring higher lexical diversity in their writing.Taken as a whole, we arrived at two main conclusions: first, texts with higher indices reflect the writer's ability to communicate complex ideas fluently (Beers & Nagy, 2009), while texts with lower indices are commonly composed in a simpler manner with lower content density.Second, since high-index texts were written under the AI writing assistant condition, students who write with the help of ParagraphAI could produce better quality writing with greater sentence fluency.

Texts Comparison
The main purpose of this research was to examine the extent to which the AIgenerated writing assistant affects L2 learners' writing.We compared both students' writing texts and analyzed each characteristic difference found in the text.

Linguistic misery/turmoil
Students' cognitive ability plays an important role in producing outstanding writing (Kellogg, 2008).In terms of second language learning and development, we emphasize their linguistic mastery.Of course, many aspects of linguistics are involved, but we underline the priority found in students' writing as a reflection of their current limitations.The main problem was related to grammatical rules.It seems that students were having difficulty following the correct grammatical system of the target language when writing in English.There were multiple occasions on which they failed to write in a 'correct' way but were miraculously able to produce meticulous texts with no mistakes with the help of an AI-generated writing assistant.For example: (1) Initial work : "Technology helping students to be uptodate about news" ParagraphAI : "Technology empowered students in becoming updated about current happenings around the globe." Another factor lies in the choice of vocabulary.We discovered two main setbacks in this criterion.First, students have difficulty selecting and using precise words that accurately represent the meaning they intend to express.Second, they tend to adopt an informal tone by using colloquial words, thus diminishing the formal academic essence of the text.For example: (2) Initial work : "Now we found out many things about far study…."ParagraphAI : "After knowing several things about distance learning,…" However, these obstacles seem to vanish when students write with the help of ParagraphAI.Their writing productions were so different from those they wrote themselves that the final results appeared to be on par with the average academic text written by native speakers of the target language, including the use of advanced vocabulary.

Incoherent and incohesive text
The next domain is related to the basic rule of 'good writing,' in which we will examine the dynamics of text cohesion and coherence.To begin with, the final examined results from both text types were exact opposites of each other.This means that all the positive outcomes we discovered under the AI-based writing assistant text had negative effects on the students' original texts.In other words, the students' original texts yielded results that were the opposite of the excellent quality texts produced under the AI-generated writing assistant.We found that AI-assisted writing helped produce coherent text with clear ideas.Each sentence relates clearly to the topic sentence and the main theme of the text.The ideas were interconnected, creating unity in topic delivery (see Figure 4).The student's thoughts were expressed directly and logically, making every sentence and paragraph of the text easy to read, digest, and understand.AI composed texts or expressions that were difficult for L2 learners to articulate in writing, straightforwardly and easily.It is also interesting how word choices and expressions in the text create a logical bridge that leads to cohesive text.The AI-generated writing assistant also improves the flow of the writing and effectively helps the discourse mature.Most importantly, it constructs texts that are of native-like quality.

Lack of content density
While the other two mostly provided positive feedback on the AI-generated writing product, we discussed some shortcomings of the text assistant.Sometimes, what the students intended was not appropriately expressed; therefore, the message was not delivered.In the worst-case scenario, sometimes the texts were shortened, and the true meanings were lost in translation.We tried to analyze the reason for this 'cutting-down'.We found out that typically when two or more ideas are closely related, though not the same, the AI writing assistant will connect the ideas and simply construct one topic.Ironically, the sentence was ended in two ways.First, the text failed to convey the initial ideas and the student's thoughts.Second, the statement was not on point, but the reader can somehow grasp the meaning behind the sentence.However, regardless of how advanced the AI writing assistant technology is today, it cannot formulate ideas beyond the given keywords.Meaning it cannot foster free thought and creativity.

Survey Data
In this part, we highlight the students' responses regarding their perceptions and attitudes toward using AI-generated writing assistants.We began by attempting to understand their mental perception of the effort required to complete writing tasks.In their opinion, writing is a serious activity that demands time and a significant amount of energy.Aside from skill, brainstorming is required to generate ideas and develop the 'plot'.Not to mention that sometimes they have plenty of ideas but do not know how to start or where each piece of information fits to complement the narrative.In other words, students have difficulty organizing their ideas.Hence, we investigated further into their difficulty in expressing their ideas.In this regard, the main problem was not the idea itself but rather how to express those ideas in the target language.The students found it challenging to write in English because it was not their first language.Translating ideas and thoughts into another language absorbs one's creative mind and cognitive understanding of the language system.
Based on these answers, we asked whether the AI writing assistant helped them write better texts.The answer was yes.The AI writing application "makes my task faster" and easier.Additionally, the software creates high-quality content comprising excellent and advanced vocabulary choices and accurate words that reflect the intended meaning.However, some dissatisfactions arose.Although the AI writing software's final result usually successfully conveys the student's intended meaning, sometimes it fails spectacularly.It "misunderstood the word I used" and interpreted it differently.
Regardless of its advantages and disadvantages, we were curious about their satisfaction view.Some of them favored the AI writing assistant for the direct grammatical corrections provided and the level of language proficiency reflected in the text.However, those who preferred their writing pointed out the confusion of ideas within the text's narrative.So, we sought their opinions on the possibility of using this technology in future education.The general responses were positive.The students believed that AI is rapidly becoming part of human life, including education.The rapid growth of this artificial technology can help students learn to write better texts.The device itself was easy to operate.

DISCUSSION
While human assessment is believed to provide a more holistic understanding of students' performance, it does not mean machine assessment is lacking.Indeed, the correlation between both can further enhance benefits (Shermis et al., 2010) as human assessment can compensate for what machine scoring lacks.For example, machine assessment can only test writing features it was explicitly programmed to assess (Powers et al., 2000).Its algorithms can easily detect definite grammatical errors but fail to consider contextual mistakes (McCurry, 2010).In contrast, human assessors can readily recognize organizational errors, such as coherence and cohesion (Patterson, 2007).
There are recurring themes in the students' writing, and their perceptions of AIbased writing assistants have emerged.The results showed that AI lacked intentionality and taste.It could not fully grasp or recreate the students' intended significance, sense, deliberation, or context of an idea, although it could depict and understand the structural order given.In other words, it lacked the human touch that distinguishes it from the art of human writing.It could write at a basic instructional level based on given prompts but lacked a narrative agenda and was ineffective at producing interesting and tasteful language (Ippolito et al., 2022).AI operates on preconstructed language models based on vast text samples installed as benchmarks of good sentences; hence, the language it produces is always predictable and standard, often failing to follow narrative flow and stylistic consistency.Ippolito et al. (2022) argued that AI is satisfactory if it can complement the writer's diverse perspective and eloquent language choice.However, Sumakul et al. (2021) drew the line by stating that while machines can help construct good writing, humans are still the main actors controlling the flow.Another drawback was related to the occurrence of 'unwanted' parts and sentences within the narrative of the text.AI may respond with inaccurate and unrelated content to the initial topic, and its inability to maintain the narrative (Gallacher et al., 2018).
Despite these shortcomings, AI-powered writing assistants do help students translate their ideas into English and improve their writing quality.AI rewrites the original text and molds it into a text that sounds native to the language.It also offers word options and recommendations to prevent students from getting stuck during the writing process (Zhao, 2022).Additionally, while grammatical and spelling checks are useful for better quality writing, McGee and Ericsson (2002) highlighted the negative influences and implications it may have on pedagogical practice as students may treat and accept this technology as an authority.Therefore, teachers must ensure that students critically understand and are aware of its flaws, as it may jeopardize teachers' authority and the nature of education.Qian et al. (2021) opposed the possibility of AI replacing teachers' roles as assessment raters in EFL writing instruction, noting that the tools have relatively low accuracy in evaluation (Liu & Kunnan, 2016) but with frustratingly high levels of recognition without the benefit of convenience (McCrocklin, 2019).
By examining the impact of digital tools on students' writing, this study aimed to understand their perceptions of this pedagogical practice in the writing environment.Overall, students support the use of digital composing tools in writing.This positive perception fosters confidence in themselves about their development and writing skills (Sumakul et al., 2021;Wingate, 2010;Woo et al., 2011).In addition, students can use digital tools for comprehensive revision instead of making gradual changes (Nobles & Paganucci, 2015).In their opinion, artificial intelligence tools are easy to access, easy to use, and quick to respond (Terblanche et al., 2022).
Although an AI-generated writing assistant can enhance students' writing quality, teacher guidance is crucial for helping students get the most from these tools.The study did not determine whether an AI-generated writing assistant should be used in the English teaching and learning process but rather discussed their advantages and concerns.The results highlighted the strengths and weaknesses of using these tools in EFL/ESL classrooms and encouraged further research.The study may have future implications for educators who want to design curriculum and writing instruction strategies based on an AI-generated writing assistant.Teachers can enhance their writing instruction by exploring digital tools to make it more engaging for students.

CONCLUSION
The key point of this study was to examine the extent of the effect of the AIgenerated writing assistant on L2 learners' writing.The inferential analysis results on lexical diversity indices established a strong preference for writing under software conditions.Texts produced with the help of the AI writing assistant appeared to have a significantly positive impact on learners' performance.This means that texts with higher indexes of lexical diversity are of better quality, reflected in both ideas and sentence fluency.To what extent does the AI-generated writing assistant affect L2 learners' writing?Our objective is to provide a comprehensive overview of the findings.
First, the most common problem faced by L2 learners in writing was linguistic mastery.They have difficulty following the correct grammatical system of the target language when they have to write in English, yet they were able to produce texts with a high level of grammatical accuracy under the AI writing assistant.Second, text written with the help of AI writing software emerged as a good product that offered cohesion and coherence.The ideas in each sentence and paragraph are interconnected, making them easy to read, digest, and understand.Not to mention the level of precision in expression articulation that helped create the flow of the narrative and the discourse and bridged the cohesion between sequences of events.However, this artificial venture had some negative effects, namely, a lack of content density.We talked about of Al-Generated writing assistant on the students' English writing | 1009 messages that were not successfully delivered and meaning lost in translation.Sometimes, the machine also constructed texts that were not within the initial premise of the writer's thoughts or ideas that should not be in the text.
Future studies using ParagraphAI should explore the tool's impact on student writing.It is important to consider how word suggestion and reverse-translation features may affect EFL students' writing process.The primary explanation lies in the ability of word suggestions and reverse translations to introduce new dimensions and sentiments to the written product, thereby influencing the author's perception and attitude towards it.Additionally, follow-up interviews with participants can provide insights into the tool's usefulness and identify any features that may be confusing or counterproductive.

Figure 4 .
Figure 4. Excerpt from students' writing: the student's initial work (left), the initial work after using ParagraphAI (right).

Table 1 .
Lexical diversity level in the students' writings from their original works (left) and the AI-generated writing texts (right).Table1presents the levels of lexical diversity in the students' writings from both types of texts.As shown in Table