EFL students’ perception on the use of Grammarly and teacher feedback

Many studies on the Automated Writing Evaluation (AWE) Program predominantly focused on the outcomes of the students writing and the comparison between AWE programs. However, studies investigating the students' perception on combining an AWE program and teacher feedback are still insufficient. This study examined the students' perception on the use of Grammarly and teacher feedback on their writing. It also sought to know whether the students' English proficiency level influences their perception. The participants included 26 undergraduate students of the Faculty of Law who were taking an English for Specific Purpose (ESP) writing course when the data were collected. The data were the students' responses to the questionnaire and their TOEFL scores. The result of the analysis showed that the students perceived the use of Grammarly and teacher feedback positively. Furthermore, the students' perception on the use of Grammarly and teacher feedback was not influenced by their English proficiency level. Students of high and low English proficiency levels gave positive responses to the use of Grammarly and teacher feedback.


INTRODUCTION
Automated writing evaluation (AWE) program that is generally known as computergenerated feedback has seized increased attention in writing research in recent years. The advances of the program which employed sophisticated language processing technologies provide real-time holistic scoring and corrective feedback. Moreover, it uses an artificial intelligence developed by computational linguistics to rate and to score the writing submitted to the program (Ferster et al., 2012;Wilson, 2016;Wilson & Andrada, 2016) by analyzing the writing on lexical, syntactic, discourse and grammar levels (Chen & Cheng, 2008). Therefore, the users can preview the evaluation result by looking at the feedback and correction given by the system and can start to revise the writing based on the evaluation given by the system (Chen & Cheng, 2008;Ferster et al., 2012) and save their time in checking and evaluating the writings (O'neill & Russel, 2019;Chou et al., 2016;Cotos, 2011;Roscoe et al. (2017). The AWE program is regarded as a great instrument to assist the users to evaluate the writing since it accommodates the diagnostic feature.
Besides its satisfactory features, plenty of AWE programs also lead the users to dissatisfaction. Some research studies investigated some AWE programs by its benefits and drawbacks. Intelligent Academic Discourse Evaluator equips obvious feedback for the users and leads them to focus on revising the writing Cotos (2011).
Criterion which is one of the successful AWE programs provides the users with feedback clarity, but the scoring system can be deceived by omitting the error for longer essay writing (Ebyary & Windeatt, 2010;Wang, 2013). ETIPS solely produces inaccurate score measurement in evaluating the writing which confuses Scharber, Dexter & Riedel (2008). Pigai occasionally presents vague and unhelpful feedback regarding the content of writing (Zhang, 2020). Writing Roadmap TM 2.0 program can improve writing proficiency, and at the same time fails to provide comprehensive feedback and suggestion in evaluating the idea, content, and writing organization Wang & Wang (2012). Therefore, knowing the benefits and the drawbacks of AWE programs is important in deciding the appropriate AWE program to be used for evaluating writing.
From many available AWE programs, Grammarly is one of the most popular AWE programs which can be implemented in the writing class. Grammarly has been claimed as an easy tool that can help students and academies deal with their writing by checking the spelling, grammar, and punctuation errors. It also provides comprehensive and useful feedback including correction and suggestion to make the writing clearer, more precise, more effective, more readable, mistake-free, and impactful with a high rate of accuracy and evaluation speed (Grammarly, 2020). Daniels and Leslie (2013) argue that Grammarly is not only able to identify punctuation (such as missing spaces after periods) and the spelling mistakes, including the proper noun and provided several alternative possibilities for the misspelled words, but also identify fragments and offer advice on verb form, although often no suggested corrections are presented, and explanations were complex. Thus, Grammarly is effective to assist the teacher and students in correcting the writing. In addition, the program could help students reduce the errors of writing Qassemzadeh & Soleimani (2016), improve writing quality, improve writing confidence, and promote independent proofreading O'Neill & Russel (2019).
Considering the benefits and drawbacks of the feedback provided by the AWE program and the teacher, combining these two kinds of feedback providers will produce the feedback that is truly helpful for both the teacher and the students. Moreover, teacher feedback cannot be neglected when the AWE program is used (Grimes & Warschauer, 2010;Qiang, 2014;Ware, 2018). The AWE Programs cannot merely replace teacher feedback since the students still need help from the teacher to enrich the content of their writing (Chen & Cheng, 2008;Zhang, 2020). The program is also limited to the semantic analysis of the language (Zupanc & Bosnic, 2015). Therefore, writing practices will not be effective if the teacher feedback is neglected.
Germane to studies on AWE programs, the previous studies focused mainly on the outcomes or scores of the students' writing (Qassemzadeh & Soleimani, 2016;Karyuatry et al., 2018;Yulianti & Reni, 2018;Wang & Li, 2019), the comparison between AWE feedback and other types of feedback Liu & Kunnan (2016), and the validity of AWE as a scoring system (Wang & Brown, 2007;Chapelle et al., 2015). In addition, the extant studies were also mainly conducted particularly at intermediate-above level Chen & Cheng (2008) or at low-proficiency level Huang & Renandya (2018). In contrast, the present study offers essential issues of difference. Firstly, Grammarly is combined with the teacher feedback in providing feedback for the students' writing, while the previous studies investigated the use of feedback from AWE programs in isolation. Secondly, this study examined the students' perception on the use of Grammarly and teacher feedback to know whether the students take the benefits of the combination or not. The studies which focused on the effectiveness of AWE recomended researchers to find out more about the students' perspective on the implementation of different AWE programs Cotos (2011);Hegelheimer et al. (2015). Lastly, this study, which uses Grammarly and teacher feedback at both high and low language proficiency levels, also investigates whether the two feedback strategies benefit particular students based on the language level.
By reference to the previous research studies and issues of AWE programs, the objective of this study is to examine the students' perception on the use of Grammarly and teacher feedback in the teaching and learning process. Moreover, this study investigates the use of the combined feedback in an ESP writing course. This study enlarges the contribution of the research regarding the use of the AWE program in different subjects and settings. Thus, the research questions are formulated as follows: 1. How do EFL students perceive the use of the combination of Grammarly and teacher feedback to their writing? 2. Is there any difference in the EFL students' perception on the use of the combination of Grammarly and teacher feedback across proficiency levels?

METHODS
This study was a survey on the EFL students' perception of the use of the combination of Grammarly and teacher feedback in the English for a Specific Purpose (ESP) Writing course. It involved 26 undergraduate students who took ESP Writing course offered in the second semester at the Faculty of Law of one of the reputable private universities in Malang City, East Java, Indonesia. In a series of four sessions in the ESP Writing course, the students were taught how to write a cause-effect paragraph. In these four sessions, the combination of Grammarly and teacher feedback was used to help students write cause-effect paragraphs. The steps in the implementation of the strategy are explained in Table 1. The students were introduced to Grammarly and was trained to use it to check errors in a model ofcause-effect paragraph. Meeting 2 (1) The students were asked to work in pair to write a cause-effect paragraph about"flood." (2) The pairs were asked to give a reciprocal feedback (3) Grammarly was then used to check the students' paragraph after a reciprocal feedback activity.
(4) The students revised the draft based on feedback from Grammarly.
(5) The draft of the paragraph was submitted and given teacher feedback.
(6) The students revised the draft based on the teacher feedback.
(7) The students submitted their paragraphs to the teacher. Meeting 3 (1)The students were asked to write a cause-effect paragraph individually about " the positive effect of the Internet." (2) Grammarly was used to check the students' paragraph.
(5) The students revised the draft based on feedback from Grammarly.
(4)The draft of the paragraph was submitted and given teacher feedback.
(5) The students revised the draft based on the teacher feedback.
(6)The students submitted the paragraphs to the teacher. Meeting 4 (1) The students were asked to work individually to write a cause-effect paragraph about "the danger of smoking." (2) The draft of the paragraph was submitted and given teacher feedback.
(3) The students revised the draft based on the teacher feedback.
(4) Grammarly was then used to check the students' paragraph.
(5) The students revised the draft based on feedback from Grammarly.
(6) The students submitted their paragraphs to the teacher.
(7) Students were given a questionnaire and asked to respond to the questions.
Based on the planning of the implementation of the strategy, in the first meeting, the students were taught about the structure and linguistic features of a cause-effect paragraph with a model text. In the second meeting, the students were directed to construct a cause-effect paragraph about "flood" with their classmates in pairs. After the students had finished in making the cause-effect paragraphs, they were asked to give comments on the works of classmates from different pairs. In the third meeting, the students were asked to make a cause-effect paragraph individually on the topic of "The positive effect of the Internet". In the last meeting, the teacher assigned the students to write a cause-effect paragraph about "the danger of smoking." At the end of this meeting, the students were also asked to give their responses to the questionnaire. Thus, in this study, the students were assigned to make a piece of cause-effect paragraph in pairs and a couple of cause-effect paragraphs individually with different topics. It aimed to cover the materials that should be discussed such as floods, internet, and smoking.
Dealing with the use of Grammarly and teacher feedback, the teacher directed it exclusively during the second until the fourth meeting. One reason was that the composing process happened at these last three meetings. Besides, the students were asked to access Grammarly using either their laptops or smartphones, whichever felt easier for the students. It could be seen from Table 1 that the Grammarly was used from the first to the fourth meetings. During the first meeting, the program was used as a practice for the students to learn the correct use of grammar, mechanics, or vocabulary choice in a descriptive text. It was also intended to make sure that the students had enough experience in using the program. In the second meeting until the fourth meeting, the program was used to give feedback and correct the students' errors in grammar, mechanics, or vocabulary. In the third meeting, the first individual cause-effect paragraph was produced with the use of Grammarly before the teacher feedback. Nonetheless, in the fourth meeting, the students were asked to use Grammarly after the teacher feedback and before the final submission only. It was done to reflect the students' progress after using the program several times.
Data of the students' perception were collected by using a questionnaire. The questionnaire consisted of 16 statements constructed in the form of Likert scales with four options: strongly agree, agree, disagree, and strongly disagree. Four options were used to pose a specific tendency so that the students' responses could be identified clearly. The 16 statements were intended to know the students' perception on four aspects: the practicality of Grammarly use (3 statements), the Grammarly feedback (6 statements), the teacher feedback (4 statements), and the combination of Grammarly and teacher feedback (3 statements).
Data were analysis on the basis of the students' responses to each of the statements. The descriptive statistics were used to see the frequency and percentage of the students; responses which are categorized into positive or agree including "strongly agree" and "agree" responses and negative or disagree including "disagre" and "strongly disagree" responses.
Data of the students' English proficiency levels were collected from the students' scores of Test of English as a Foreign Language (TOEFL) that was administered at the end of the first semester by the Language Center.Based on the TOEFL scores, the students were classified into two groups: those who had high English proficiency and those who had low English proficiency. The high English proficiency group consisted of 9 students who got TOEFL scoresof450 or above. On the other hand, the low English proficiency group consisted of 19 students whose TOEFL scores were below 450.
The students' responses to the questionaire were counted andcompared across the two groups to see whether the students' English proficiency level affected the their responses to the use of the combination of Grammarly and teacher feedback.

RESULTS AND DISCUSSION
The results of the study include the students' perception on the use of the combination of Grammarly and teacher feedback and their perception on the use of Grammarly and teacher feedback across English proficiency levels.

The Students' Perception on the Combination of Grammarly and Teacher Feedback
The students' perception on the practicality of Grammarly use is shown in Table 2. As displayed in Table 2, all the students agreed that Grammarly was easy to use. Moreover, most of the students (89%) did not need technical assistance from another person to use Grammarly and only a few students (11%) needed assistance to use Grammarly. More importantly, the majority of the students (93%) thought that the various functions in Grammarly were well-integrated. At average, the majority of the students (93%) considered that Grammarly was a program which is practical to use. The students' perception on the feedback provided by Grammarly is exhibited in Table 3.  Table 3 shows that many of the students (78%) thought that Grammarly's feedback was understandable. Only a few students (14%) thought that the feedback was not understandable. Then, the majority of the students (93%) responded that Grammarly gave detailed feedback. Next, although some students (32%) confirmed that the program gave misleading feedback, many students (68%) responded in the opposite way. Regarding the feedback efficiency, almost all of the students (96%) could save their time in revising their writing by using Grammarly as it provided real-time feedback. Besides, all of the students agreed that Grammarly helped them improve their writing quality and their confidence in submitting their writing to the teacher. All in all, the feedback from Grammarly obtained positive responses from the majority of the students (93%). The students' perceptionon the teacher feedback is shown in Table 4. Table 4 displays that all of the students (100%) considered that the teacher feedback was understandable. All of them also confirmed that the teacher feedback helped them to revise their content and organize their writing better. They also agreed that the teacher feedback improved their writing. In summary, all ofthe students responded positively to the role of teacher feedback in the process of writing.  The students' perception of the combination of Grammarly and teacher feedback is depicted in Table 5. As Table 5 shows, all of the students (100%) stated that the teacher feedback helped them to revise errors which was unnoticed by the program. Moreover, all of the students agreed that the combination of Grammarly and teacher feedback was needed to improve their writing skill and their writing confidence. Briefly stated, the feedback from Grammarly and the teacher could complement each other and enhance the students' writing quality.

Students' Perception of the Combination of Grammarly and Teacher Feedback across English Proficiency Levels
The comparison of the perception of the high and low English proficiency student can be seen in Table 6.
As Table 6 shows, in terms of practicality of Grammarly use, both groups, in general, shared the same positive responses. However, a few students from the low English proficiency group responded negatively. More specifically, they needed help in using Grammarly (16%) and thoughtthatthefeaturesofGrammarly were not wellintegrated (10%).
Dealing with the Grammarly feedback, there were 4 students (21%) from the low English proficiency group who stated that Grammarly did not provide understandable feedback. Besides, 2 students (10%), who were alsofrom the low language proficiency level group, perceived that Grammarly gave undetailed feedback. Furthermore, there were 4 students (44%) and 5 students (26%) from the high and low English proficiency groups, respectively, who agreed that Grammarly gave misleading feedback. However, almost all students from both groups agreed that Grammarly revised their writing efficiently.
In terms of teacher feedback and the combination of two types of feedback, all of the students from both groups perceived positively. In other words, the English proficiency level did not influence the students' perception of both aspects. To sum up, although some aspects of the questionnaire obtained negative responses from the low English proficiency group, both groups perceived positively in almost all aspects. Overall, the data showed that the majority of the students from the two English proficiency levels (97% from high and 94% from low English proficiency groups), responded positively to the two types of feedback provision. Based on the finding, it can be inferred that the English proficiency level did not influence the students' perception on the use of Grammarly and Teacher feedback in the ESP writing course.
The first research question focused on the overall (positive and negative) perceptions of the EFL students. In reference to the data gathered from the questionnaire, the students responded positively to the use of Grammarlyand teacher feedback. The students' positive responses could be seen in many aspects. The majority of the students confirmed that Grammarly was a user-friendly program and it is easy in its use. This is supported by the features of Grammarly which were well integrated. This result is in line with the result of previous study which showedthatcommonly automated writing evaluation (AWE) programswere easy to access be it inside and outside the classroms (Ariyanto at al. 2019).
The Grammarly feedback, in general, led the majority of the students to give their positive responses. The program assisted the students to revise their writing by providing detailed and understandable feedback especially in terms of grammar, vocabulary, and spelling (Warschauer & Grimes, 2008;Wilson & Czik, 2016). In this way, the students could fix the errors and learn from the errors they made through the explanation given by the program as well Arianto at al. (2019). Aligning with the findings of Warschauer & Grimes (2008) and Wilson & Czik (2016), the students could save their time in the revision process because the real-time feedback was provided by the program. Thus, it could also help the teacher to reduce their workload in revising the students' work so that the teacher could focus more on helping students to revise the content and organization of writing (Wilson & Czik, 2016). Meanwhile, the program not only boosted the students' confidence but also improved the students' writing outcomes (Wilson & Czik, 2016). However, the misleading feedback found in the program should be noted by the teacher since nine students stated that Grammarly sometimes gave misleading feedback. This finding is in line with previous studies conducted by Crusan (2015) and Nova and Lukmana (2018) that there were sometimes undetected or misleading errors in AWE programs.
Concerning the teacher feedback and the combination of both feedback strategies, all of the students perceived positively. The students stated that the teacher feedback helped them to revise their content and organization better in which they did not completely get the feedback from Grammarly (Tai et al., 2015). In addition, the undetected errors by Grammarly feedback were also accommodated by the teacher (Zhang, 2020). The Integration of teacher feedback may diminish the faults and inefficiency of the AWE programs (Mohsen & Alshahrani, 2019). Therefore, all the students confirmed that the combination of Grammarly and teacher feedback was the perfect feedback strategy which assisted them to produce better writing and improve their writing confidence. Concerning this finding, the studies investigated by Zhang (2020), Ariyanto et al. (2019), Lu et al. (2015) showed that the AWE advantages can be fully realized when it works in tandem with teacher feedback. In terms of the second research question, the findings revealed that almost all ofthestudents from the high and low English proficiency groups responded positively to all the items of the questionnaire. In other words, the students' English proficiency levels did not influence their responses to the use of Grammarly and teacher Feedback in the ESP writing course. Moreover, Grammarly can be used by students of different language levels but a different approach may need to be taken to maximize its use O'neill & Russel (2019).

CONCLUSION
The study has revealed that the use of Grammarly and teacher feedback in ESP writing course obtained positive responses from the students. Furthermore, the combined types of feedback accommodated the teacher's and students' needs in the writing class. Firstly, the program thoroughly helped the teacher to deal with the time constraints and the detailed explanation of feedback concerning language accuracy. The real-time feedback provided for the students led the teacher to dedicate more time to help the students revise their writing content and organization. Although the program gained positive reactions from the students, it also had some demerits. The program's weaknesses spread in some aspects including misleading feedback and undetected errors. For that reason, the role of teacher feedback cannot be neglected as it can complement the demerits of the AWE program.
Some suggestions were made for English teachers and future researchers. English teachers should consider the appropriate approach when using Grammarly for students with certain English proficiency level because students from different English proficiency levels need a different approach. Additionally, investigation on the same or another kind of AWE program is needed since the literature regarding this combination of types of feedback in different AWE programs is still rarely studied. Furthermore, conducting a similar study in a different setting like more meetings for the implementation, different levels of students, or different AWE programs was suggested to contribute to the area of investigation.