The Effectiveness of Pre-Service English Teachers' Collaborative Genre-Based Writing Feedback

This study investigated the collaborative genre-based effectiveness among the pre-service English teachers (PSETs). Data collection used the genre-based writing feedback observation upon its reflection and instruction and need analysis questionnaire. The data analysis used multivariate statistics method to generalize the writing tests. The findings show that the PSETs' feedback supported the interaction, accountability, and interdependence. These aspects are due to the collaborative participation in groups, in which the PSETs work with the flexibility, entirely performed the quality, and contributed in positive attitude during the meetings and assignments. The feedback emphasiz the learning improvement within the formative reflection through the general linear model (GLM) repeated measures analysis, where F=6,114 and p<0,01. This study concludes that the collaborative genre-based writing feedback has the positive response from the PSETs. The determinant ranges gains in between 85% to 90% after a series of genre-based writing lectures were conducted.


INTRODUCTION
An urgent requirement for learning model needs to bridge and strengthen students-lecturers and studentsstudents relationships, cognitive and non-cognitive skills, mutual responsibilities, maturation, and problem resolutions in order to require creative thinking and problem solving. The instructional strategy requires a collaborative understanding among students to support the learning model. According to Schamber and Mahoney (2006), the collaboration is needed to improve the quality of the critical thinking and decision making among students to complete tasks. The grouplearning or peer-feedback collaboration fosters the reflective thinking and a greater awareness and understanding of the learners toward the writing instruction approach (Storch, 2011), it also integrates learning and promotes collegiality (Souers et al., 2007), including academic achievement, intergroup relationships, acceptance of mainstream students, and self-esteem increase (Osterholt & Barratt (2010).
Collaboration enables students to reduce polarization, to a certain benefit for students who are on the edges of the system and poorly relative performance (Muijs et al., 2011), since the collaborative approach requires learners to work in groups and search for understanding, solutions, and/ or creating a product (Delucchi, 2006). Activities such as discussion, group work and problem-based learning to find solution, mostly enhance conceptual change and individual construction. The objective is to encourage students to convey their ideas and to understand the thinking of others (Nicholls, 2002). An effective instruction is the key to sustain students' academic achievement involved in genrebased writing progress through the lecturer's writing class. Therefore, an effective writing course accommodates students' needs (Schulz, 2009).
In accomplishing students' effective writing within various genre-based topics, an orientation to the process becomes the priority to its step. Ferris and Hedgcock (2005) point out that the orientation to the process focuses particular attention on procedures for solving problems, discovering ideas, expressing themes, and revising emergent texts. Meanwhile, Widodo (2006) proposed two genre-based writing instruction cycles that can be applied, namely: writing within the class and writing independently. Writing within the class cycle associates with (1) building knowledge of the field, (2) exploring genre(s), and (3) joint text construction, whilst writing independently involves: (1) building knowledge of a similar field, (2) drafting, revising, and conferencing, and (3) editing and publishing. Myskow and Gordon (2010) point out that the basic generic constituents of an essay, such as a thesis statement at the end of an introduction and topic sentence(s) at the beginning of body paragraphs have often been prescriptively emphasized as a requirement in the genre-based writing classes that students compose their own essays in accordance with these formulaic patterns. Morgan et al. (2004) exemplify that students' writing works may start from personal letters. The peers provide the constructive feedback on the draft and students repeat to work on the draft as the next tasks. After achieving the constructive feedback, they refine the works further. Thus, students will perform better and use strategic thinking skill (Nayan et al., 2010) by discussing with peers, presenting and defending ideas, exchanging diverse beliefs, questioning other conceptual frameworks, and actively engaging in the writing process (Brown, 2008).
Genre is considered as a simple term and used to identify different types of written text. To support this view, genre-based approach emphasizes the written product as a means of expressing the intended meaning for the specific audience and context with a process approach in order to balance thinking and composing with the final product (Lee, 2012) based on its own features and structures (Sullivan, Zhang, & Zheng, 2012). Genre is a term for grouping texts together, representing how writers typically use language to respond to recurring situations they frequently use and draw on their repeated experiences with such texts to read, understand and write them relatively easily (Hyland, 2008). These necessities generate and organize ideas using an appropriate choice of vocabulary, sentence, and paragraph organization but also to turn such ideas into a readable text (Richards & Renandya, 2002;Widodo, 2006), as well as to transfer ideas from their native language (e.g. Indonesian) into the target language (e.g. English), especially for nonnative students.
The students systemically acquire an awareness of relevant text in writing to manipulate information and accomplish different purposes. Besides, situation also relates to students' confidence to conduct the real and meaningful context in writing (Swami, 2008;Ahn, 2012). They can produce their own examples of the same genre more effectively and provide them with a concrete opportunity to acquire the conceptual and cultural frameworks in particular purposes (Ahn, 2012). Therefore, students are able to develop all phases of writing process and make the sub-processes and skills explicitly through predicting the strategy types (Negari, 2011).
Substantially, genre-based approach can be significant relevance for L2 students learning to write, since the identification and analysis of textual features in genrebased classrooms can support students with knowledge about the linguistic and structural elements of different kinds of writing (Lee, 2012). Writing effectiveness relates to the social and cognitive dynamics in using the specific criteria (Pritchard & Honeycutt (2007;Dix & Cawkwell, 2011). The orientation toward writing instruction, according to Kern (2000) consists of three approaches, namely: product-based, process-based, and genre-based approach. The product-based approach focuses on the inner core of design: (1) the interaction between texts and (2) structural resources needs to create them. The process adds elements from the middle layer of communicative context, whilst genre-based approach addresses the outermost layer of social-cultural context. As the academic clue, Ahn (2012) categorizes seven genres generally instructed to the syllabus to meet a standard qualification, such as recount, procedure, narrative, report, description, explanation, and exposition, which are shown in Table 1. These genres may start from the beginning to the end of the procedure, initiating with texts, so-called how a text is structured and organized at the level of the whole text relating to its purpose, audience and message to make the text effective (Hyland, 2008). Meanwhile, Table 2 shows the connectors used in genre-based writing (Widodo, 2006). These connectors guide the students to compose good sentences or paragraphs and allow for a specific targeting of lexicon-grammatical features on particular structures (Wennerstrom, 2006).
According to Hill (2007), genre-based writing's purposeful feedback focuses on providing how to assist students' cognitive apprenticeship, similarly judges the feedback provision, and advocates learners to improve ability to critique their academic writing. The feedback may be intended on the clinical case study-either real or hypothetical, small discussion groups, collaborative learning, hypothetical-deductive reasoning, and lecturer's style concentrating on the group progress (Gijbels et al., 2005). Meanwhile, the process puts an emphasis on thinking and composing, learner-centeredness, and the facilitative role of instruction in the writing classroom (Lee, 2012) As a part of an evaluation, peer assessment is applied as an interactive assessment method that enhances students' interpretation and reflection, maximizes lecturers' understanding of students-centered performance. Hence, students are capable of learning how to criticize their peer works and accept peer criticisms. Therefore, the critical thinking skills and the self-reinforcement will develop through this method, since peer assessment requires cognitive activities such as reviewing, summarizing, clarifying, providing feedback, diagnosing errors and identifying missing knowledge or deviations (Lai & Lan, 2006). Furthermore, peer assessment also enables students in genre-based writing classes to receive critical feedback, to give critical feedback to peers before the final version of written work (Falchikov, 2001) is submitted and assessed. For example, a better writing can be possible when students' feedback are gained from their peers with the control of writing system, mechanics of writing (punctuation, capitalization, abbreviation, numbering, and spelling), and grammar to solve the problem in writing effectiveness (Nezakatgoo, 2011). In addition, a success in genre-based writing may be related to attitudes upon writing, apprehension about writing, self-efficacy (Erkan & Saban, 2011;Huwari & Aziz, 2011), and anxiety levels increase (Huwari & Aziz, 2011). In brief, writing process provides the necessary information for appropriate feedback in reviewing the rubric, considering the social-cultural and classroom context, being open to diverse expression modesexpressing experiences and purposes upon language use, using feedback information, and assessing students' writing products appropriately (Spence, 2010). The processes and products of students' genre-based writing feedback shall be academically linked to syllabus design, lesson plan, task, and assignment development. Scores, grades, and evaluative feedback shall also consistently contribute to learning processes and to the measurable improvement of their holistic writing skills (Ferris & Hedgcock, 2005). Pappamihiel, Nishimata, and Mihai (2008) emphasize that genre-based writing rubric refers to an objective assessment where it adapts the analytic scoring system due to the genre-based writing components, such as content, organization, vocabulary, grammar, and mechanics as measured to the peer feedback design. The analytic scoring, according to Weigle (2002), also rates on the content, organization, vocabulary, grammar, and mechanics basis to provide more detailed information about their performance in different writing aspects. Ferris and Hedgcock (2005) underlined that the descriptors can be constructed to represent distinct discursive and linguistic course objectives (e.g., argumentative structure, figurative language, logical connectors use, complex as opposed to simple syntax), and weightings for textual components that vary to encourage students to direct their efforts upon improving targeted skills. Cyr et al. (2014) also agreed that the scoring rubric as a set of criteria and standards links to the specific learning objectives and assign a numeric value to coincide with each criteria category. It provides students' feedback outlining the extent to which criteria has been reached and allows for a standardized evaluation. Alternatively, Huang (2009) reported three factors, such as raters' linguistic backgrounds, previous experiences, and prior training in assessment that affect the genre-based writing rating. The impact of these factors corresponds to the accuracy, precision and ultimately the fairness of writing assessment results. However, this study identifies two learning experiences, in terms of pre-service English teachers' collaborative genre-based writing competence and its effectiveness during the collaborative works.

METHODS
Thirty seven pre-service English teachers (PSETs), Sarjanawiyata Tamansiswa University, Yogyakarta participated in writing course III during the running semester. The observation and pre-test and post-test were applied within the first week of September to the fourth week of October 2015. This study began with the Cronbach's Alpha reliability coefficient that fulfilled the requirements with the gained score from 0,795 to 0,869. According to Ghozali (2001), a variable would be reliable if the value of Cronbach's alpha contribution was >0,60. The reliability coefficient calculation adopted the Cronbach's alpha formulation with GENOVA program, as shown in Table 3, and would be valid if the value was r>0,30. The lecturing session was based on the first lesson plan taught  (Santoso, 2001). Out of 37 PSETs, the test of sample distribution data had a normal distribution. Table 4 showed the statistical analysis results.
The normality test of the recount, narrative, and descriptive paragraph were gained through the pre-test and post-test. First, the experimental group pre-test showed that the recount paragraph result was K-S Z=0,836; p=0,487; the narrative paragraph was K-S Z=0,949; p=0,340; and the descriptive paragraph was K-S Z=0,773; p=0,588. Second, the experimental group post-test showed that the recount paragraph was K-S Z=0,838; p=0,483; the narrative paragraph was K-S Z=1,001; p=0,269, and descriptive paragraph was K-S Z=0,806; p=0,534. Third, the control group pre-test showed that the recount paragraph was K-S Z=0,889; p=0,407, the narrative paragraph was K-S Z=0,801; p=0,543), and the descriptive paragraph was K-S Z=0,587; p=0,881). Fourth, the control group post-test showed that the recount paragraph was K-S Z=1, 074; p=0,199), the narrative paragraph was K-S Z=0,962; p=0,312), and the descriptive paragraph was K-S Z=0,199; p=0,400. Based on the pre-tests and post-tests, there was no significant result from the expected distribution difference toward the factual distribution of the normality test. Thus, the normality test had a normal distribution.
Data collection which consisted of (1) instruction process observation, was applied to analyze the genre-based writing feedback; (2) reflection and instruction evaluation, was used to analyze the genre-based writing feedback; and (3) analysis questionnaire, was used to identify PSETs' genre-based writing competence. These three instruments were applied sequentially in the monitoring activities and instruction evaluation steps during the lectures using the Likert scale system. Data analysis used the multivariate statistics method by applying the general linear model (GLM) repeated measures and non-parametric statistics technique. The generalizability design (Webb & Shavelson,  2005) was to generalize collaborative genre-based writing tests. The quasi experimental design was used to conduct the pre-test and post-test (Cohen, Manion, & Morrison, 2007) of recount, narrative, and descriptive genre-based writing.

RESULTS AND DISCUSSIONS
Three aspects were measured to collect the important information during the implementation of pre-service English teachers' (PSETs') feedback. They consisted of interaction, accountability, and interdependence. These aspects verified the participation level in groups, in which PSETs worked with the genre-based writing schedules flexibility, showed the entirely work performance quality, and contributed a positive attitude during the quizzes. The PSETs' collaborative genre-based writing indicated very significant improvement, where p<.01 after the participants attended some genre-based writing lectures, engaged by the collaborative work and collective feedback at the end of the learning activities. The paired t-test showed the gained mean and standard deviation among these genrebased paragraphs. The recount paragraph was M=2,744; SD=1,347, narrative paragraph was M=2,767; SD=1,771, and descriptive paragraph was M=3,488; SD=1,594. The correlation coefficients (P) showed r=0,30 and p<0,05 after the experimental groups joined in three different lesson plans. These PSETs' collaborative genre-based writing competences attempted at experiencing the transfer of learning competence, which reflected to PSETs' creativity, content understanding, social interaction, and problem solving through the learning activities. Further, the rating scale inventory used 5-Likert rating scale, whilst the collaborative genre-based writing reliability used the generalizability analysis. The variance component estimation complied with the generalized analysis of variance system (GENOVA 3,1), a software program for generalizability analysis to determine the percent variance associated with each component (Hunt et al., 2009) towards the assessment criteria. From 37 respondents (n=37), eight groups were assigned to join in genre-based writing lectures, whilst the variance component estimation referred to groups' pre-test results of recount, narrative, and descriptive paragraph as shown in Table 5.
It could be seen from Table 5 that PSETs' pre-test had performed an achievement on the genre-based writing. First, the P value of recount paragraph was 0,235% (as shown by the experimental group, the score was 70,85; whereas the control group's score was 71,18). Second, the narrative paragraph achievement was 0,836% (as shown by the experimental group, the score was 71,73; whereas the control group's score was 71,79). Third, the descriptive paragraph achievement was 0,198% (as shown by the experimental group, the score was 69,86; whereas the control group's score was 70,92).
The pre-test and post-test results, as shown in Table 7 describes PSETs' achievement (after pre-test and post-test) through the mean square of the recount paragraph with the value of 89,485, the mean square of the narrative paragraph with the value of 29,385, and the mean square of descriptive paragraph with the value of 66,989. The generalizability (ρ 2 ) coefficients and collaborative genre-based writing variances upon the feedback procedure were achievable with the value of 0,37919. Thus, the rating scale feedback of the collaborative genre-based writing might generalize the instruction context.
The collaborative genre-based writing accommodated the procedural problem solving, enhanced, and improved the language use. The point concerned with the simple structure, familiar and precise vocabulary used in accommodating PSETs' collective genre-based writing competence. The PSETs performed four collaborative genre-based writing facts. First, based on the class observation held on the odd semester in 2015-2016 academic years, the collaborative genre-based writing groups participated during the lectures. The groups had the creative activities and efforts during the learning experience. The PSETs were proactive and more responsive to think and complete the assessment tasks assigned. They were also well-prepared in influencing and understanding with each other. It was reported that the majority of the group members involved in discussion with others during the lectures, rather than asking the writing lecturer. This learning experience indicated that the collaborative genre-based writing implementation seemed to be showing progress.
Second, the lectures orientation focused on the collective genre-based writing introduction. It provided PSETs opportunities better in understanding the collaborative learning management and procedure, and working with peers, although they needed time more to make adjustment in the collaborative works. Third, the writing lecturing session suggested that the feedback was necessary to drill the PSETs in dealing with problems relating to grammar, content, vocabulary, organization, and mechanics purposes. The existing feedback should support to the procedural problem solving linearity that achieved input-process-output, while the result analysis upon PSETs' writing errors was reflected by the peer feedback's inputprocess-output phase. The sustainable feedback devices would acknowledge the problem solving on the genre-based writing process and its beneficially retrospective design to review the procedural work.
Fourth, the rubric-based feedback device was not apparently sufficient to apply for the genre-based writing selected themes. As the feedback solution, the rubric device was redesigned and supported by applying the checklist system which enabled to control and examine the problem solving. However, the checklist turned out to be more applicable by mostly PSETs to conduct the feedback practices. As an important thing, a number of alternative ideas among PSETs' multiple backgrounds were set forth as a major obstacle to realize that how far they had achieved the genre-based writing competence in a required standard. This condition changed PSETs' learning behavior from individual to collaborative works. The simplification design increased PSETs' problem solving awareness level, so that the transfer of learning actualization could be observed and analyzed through the analytic scoring rubric and checklist system.

CONCLUSIONS
The collaborative genre-based writing was positively influential and essential to the problem solving. The learning management context emphasized on the previous learning experience and competence aspects towards the PSETs' previous competence. The feedback was functionally proven to be applied for the learning outputs improvement through the formative reflection, by which indicated the input variables, the learning difficulty levels, and materials organization designed. The collaborative genre-based writing increased the feedback in each lesson plan, whilst the learning outputs contributed to PSETs' competence. This contribution was indicated through the generalized linear model (GLM) repeated measures analysis for each lesson plan feedback towards the proposed null hypothesis test, where F=6,114 and p<0,01 in terms of gaining the multivariate results. The collaborative genre-based writing effectiveness was gained through the analytic scoring rubrics and/or the checklist system measurement. The analytic scoring system was widely used in the procedural knowledge substance and sequential learning experience to support the learning experience, whereas the checklist system was conditionally addressed to the selected genrebased writing themes. Thus, the collaborative genrebased writing competence was preeminently required to the genres, which are: recount, narrative, and descriptive paragraph. The peer feedback effectiveness had a significant result and its determinant range in between 85% to 90% for collaborative genre-based writing. The effectiveness includes PSETs' creativity factors, collaboration increase, learning difficulty reduction, learning organization strength, and problem solving orientation.