Streamlining verbal feedback: Reflection on a feedback research project in secondary schools

This paper is a reflection on action research I conducted in two classrooms to explore the effectiveness of feedback. As a result of this project, I have changed my practice to streamline verbal feedback. Despite its transience, verbal feedback can be made far more effective if it is reduced to key points only.


Academic rationale
I chose this subject because I use feedback as a key strategy in my classroom. Kluger and DeNisi (1998) mention that research consistently ranks feedback as among the strongest interventions at a teacher's disposal. Hattie (2003) found feedback has an effect size of 1.3. An effect size of 0.5 is equivalent to one grade leap at GCSE, advancing a learner's achievement by one year, or improving the rate of learning by 50 per cent. An effect size of above 0.4 is above average in educational research, constituting the 'hinge point' (Hattie, 2003) at which the impact is greater than just a typical year of academic experience and student growth. In practice, this 1.3 effect size for feedback translates into a leap of over two grades at GCSE level. Students who would, without my intervention, be taking a C grade could reach an A grade through the use of meaningful feedback. I wished to manipulate my feedback effectively enough for this to happen. Also I wanted to assess how effective my current strategies (written feedback in notebooks, whole-class verbal feedback session in a lesson after a major assignment) were, based on student feedback to me.

Research design
For this project, I undertook action research in two boys-only school classrooms, involving 18 target children, 6 of whom were aged 15-16 years (Year 11) and 12 of whom were aged 13-14 years (Year 9). I gathered data by recording classroom activities and, more importantly, by recording student interviews that I later analysed for patterns and relationships. Also I compared records of student work before and after I gave feedback.

Initial reflection
In preparation for the action research, I reflected on how I normally gave feedback to my students. In addition, video recordings were made of feedback classes for my students in .
I noticed that throughout the academic year I gave feedback in several ways. Written feedback was either short comments or symbols annotating the script, or comments at the end of the script. These comments constituted descriptive feedback tabulating medals (what the student had done well) and missions (what the student needed to do to progress to the next level) (Petty, 2004;Black and Wiliam, 1998). I write these comments in two columns next to a drawn symbol of a medal or of a hand pointing to where to go next. The descriptive feedback is written in short bullet points for easy absorption and reference. For instance, one student, Zohrain, saw the following comments in relation to his assignment next to a picture of a medal: • cogently argued • excellent use of argument/rhetorical questions • varied punctuation used.
Next to this was the pointing hand of the mission, advising him to: • perfect your accuracy, especially P/Ag (person agreement) • work on density of ambitious vocabulary to vault directly to the next band (grade). I give students a few minutes to review this feedback when they receive their notebooks. At the time of the next assignment, they are reminded to review the feedback before proceeding, especially the 'missions'. Since errors and achievement vary from student to student, this means that in the next assignment, each student will continue to progress and to build up skills from the point where the last assignment left off. This constitutes ipsative feedback, the idea that a student makes a comparison with the self rather than with norms (achievement of other students) or external criteria alone. This encourages them to act upon developmental feedback to achieve a personal best (Hughes, 2011).
Oral feedback mostly takes place in the form of a whole-class feedback lesson or as individual comments when returning a notebook or upon a student request to discuss his assignment.
I wanted to determine whether verbal or written feedback was more effective. What could I do to leverage my feedback? For this purpose, I decided to conduct interviews with six of my Year 11 students.

Planning
In the English language classroom of the Boys' Branch of the Lahore College of Arts and Sciences, I introduced the students of Year 11 to the idea that research would be taking place to learn what they had to say about the teacher's feedback and to consider how effective feedback was and how to make it better. I emphasized that they were not being judged, that their honest response was required to make feedback better, and that there were no right or wrong answers. Although the students belonged to an older age group, they had not experienced action research before in a classroom setting.
Although details differed, most of the students who were interviewed seemed to feel that the medal and mission comments were more effective than verbal feedback. One student mentioned that he did not remember much of the verbal feedback. This was surprising because he was one of the best students in that class.
In consideration of all this, I planned a lesson in which I would use the medal and mission comments on a convergent assignment. A 'convergent' assignment is the term used by Torrance and Pryor (1998) to denote an assignment in which the answers are expected to be similar, as opposed to an open-ended or divergent assignment where a range of original responses may be expected to the same question (the kind of assignment in which I would normally give medal and mission feedback). Furthermore, I planned changes in the teaching practice by reducing feedback to a smaller number of key concepts during the following whole-class feedback session.

Action
I made changes to my own lesson with Year 11 by introducing medal and mission comments to a convergent assignment and reducing the feedback to fewer core concepts. The six students were then interviewed again, yielding rich data comparing the effect of oral feedback to written feedback and revealing how the change in strategy influenced them.

Observation and reflection
Some interesting responses were offered in the interviews. However, I was uncertain how far the students were affected by the actual feedback given on this one particular assignment as opposed to how much change was due to the way I gave instructions in the first place or to the cumulative effect of feedback given over the academic year.

Reflection in the second cycle
In an attempt to untangle how feedback affects students, I wondered what would happen with fewer variables, for example by removing instructional quality as a factor affecting or reinforcing feedback.

Planning in the second cycle
For the next stage, therefore, I chose students from Year 9 because they had been taught by a different teacher and would be receiving only feedback from me.
The criterion for selection of 12 participants was their teacher's assessment of their attainment of close to the highest, the lowest, and the average marks in this subject, from a single class and section. The target and control groups were chosen by me at random, each group including a range of attainment in English. (Attainment here refers to scores obtained on English tests/examinations marked by their teacher). Two assignments, similar in nature, would be given to the students to see how far feedback had influenced their performance.

Action in the second cycle
After the first assignment (part of their normal coursework), their teacher gave me the 12 notebooks to mark. For the students of the target group, I wrote descriptive feedback in the form of an annotated script, and medal and mission comments, and conducted a short verbal feedback lesson in the library, focusing on only two key points. This lesson was recorded on videotape and the assignments of all 12 participants were photographed. In accordance with normal practice, like the rest of their class fellows the control group received no additional verbal feedback class and received only ticks or a vague evaluative comment (such as 'good work').
The next day, I interviewed all 12 participants in groups of two before their language class, which happened to be at the end of the school day. A second assignment of a similar nature (again a part of the normal planned coursework for the class) was attempted in class. I marked and then photographed the work of these 12 students to see if there was any evidence for feedback alone making a noticeable difference in the performance of the target group. I expected the control group to perform at about the same level in the next assignment and the experimental group receiving feedback to have improved their performance, significantly reducing errors pointed out and making some further progress in areas they had already done well in.

Observation in the second cycle
This time the effects of streamlined feedback were brought into even sharper relief. The second cycle of the action research showed how explicit feedback in small doses affected student performance in the assignment that followed. Despite the range of ability and differing degrees of autonomy possessed by the students involved, the students in the target group showed significant progress due to the implementation of feedback as opposed to those in the control group, most of whom performed at the same level.

Reflections on findings and the change in my understanding of feedback
Students forget. Among the findings that emerged, students mentioned that they did not remember all oral feedback and that many preferred written feedback to oral feedback. For example, Razaak (Year11) and Umer (Year 9) both commented that it was hard to remember oral feedback: Razaak: ... I don't really remember stuff that well and I don't really remember what you say verbally because at the end of the year, the stress and, you know, not much time, so I think the best thing is to just open your notebook and go through your mistakes. That's helpful.
Umer: Actually, I like the written ones because when we get out of the classroom we forget actually 50 per cent of the lesson. When we are doing assignments we can look into that which mistakes we have done earlier and which we should not have done in this.
During the initial interviews, I concluded that the reason why written feedback was effective was because there was a record of the medals and missions that could be referred to for ipsative feedback. However, as a result of the action research, I realized there was another factor contributing to its success. Out of necessity, I always reduced written feedback to a bare minimum (due to constraints of time). When I pared down whole-class verbal feedback to key points in a similar manner, it became equally, if not more, effective.
This ties in with what Shute (2008) has documented. Furthermore, after the reduction of teaching points in the verbal feedback, even Razaak changed his mind: Razaak: Normally the written feedback for the summary writing is much better but this time the oral one was better because you pointed out the major mistakes that students make … Teacher: Was the idea … something new or had it been touched upon earlier in the year? Razaak: It had been touched upon earlier but as I said I don't really remember verbal feedback. By the end of the assignment, I had no idea. I had forgotten.
Teacher: So it struck you as new again?
The interesting thing was that this particular misconception had been addressed several times during the term. However, it seems to have been buried underneath a plethora of feedback offered at the time. Shute has also pointed out how feedback needs to be given in smaller doses to be retained (Shute, 2008).
In relation to the format of feedback, while this highly self-directed student, Razaak, felt that there was no difference in essence between the former casually written mixed comments and the new tabulated medal and mission comments for this kind of assignment, other students of lesser attainment felt the new visual approach made it easier to absorb the feedback. They commented that this feedback was 'new', even though the same learning points had been mentioned in earlier feedback. This seemed to indicate that students who were less self-directed in their learning benefited more from an organized and categorized layout of feedback.
Other interesting patterns emerged from the responses of the students. In separate interviews, students of higher attainment from both Year 9 and Year 11 initially said they preferred written feedback to oral feedback. The reason given was that they felt they would not retain all verbal instructions. Such students had a higher level of self-direction and appeared to be reluctant to depend on verbal feedback because of its transient nature. On the other hand, with struggling students the need for accessible help made verbal feedback far more attractive. For example, Hafi liked being able to ask the teacher questions: Hafi: Both [verbal and written feedback] were effective but I think the verbal one will give a more clear explanation of what to do … Teacher: Why is that?
Hafi: We can ask more questions about what to do and we can get a clear understanding to every point … This seems to be based on more immediate concerns of understanding concepts than secondary issues of retention.
Another pattern that emerges is that across a range of attainment, almost all students preferred customized feedback that related to their own assignments, skills, or errors in particular. Interestingly, when asked if the feedback they received was effective, all students cited examples they found written in their notebooks. Generic comment by the teacher in the verbal feedback class was not referred to even once, unless probed for specifically.
During the interviews with Year 9 students, I was initially quite dismayed to hear two students from the control group (who had received evaluative feedback only) state with great confidence that they felt encouraged by the feedback and express certainty that their next assignment would show progress. At the time, it felt as if the experiment had floundered badly: if there was no difference between the performance of the control group and the intervention group, then how important could feedback be? Progress would be attributed to natural student growth. However, upon marking the second set of assignments, there were distinct differences between the progress of the control group (nominal) and the experimental group (visible). Progress was significantly greater for the most self-directed student in the experimental group than for students in the low-level or mid-level range. The term 'self-directed students' here refers to students who showed initiative, either in approaching the teacher for clarification on related learning or acting upon written feedback without explicit reminders from the teacher. Nevertheless, even the least self-directed learner of the experimental group showed more visible progress than the most independent learners of the control group. For instance, the most selfdirected learner of the experimental group was given feedback on three different areas, of which two types of error disappeared entirely and the third showed improvement. Another student with less self-direction was given equally detailed feedback; however, in the next assignment this student showed partial progress in two areas out of three. Compared to this it was eye-opening to see how so many of the students in the control group were repeating the errors made earlier.
Out of these, the most self-directed student made some progress based on (as I deduced) his close following of ticks and loss or gain of marks. The level of autonomy appeared to correspond to how well the student was doing in his class.

Impact on my practice and on my view of assessment literacy
Initially, I felt my written feedback was of far greater value because it was visible and could be referred to. Now I understand that verbal feedback can hold more value for students who are still struggling with learning. This means that my view of assessment literacy is now perhaps more inclusive. Moreover, while the written medal and mission comments were pared down to essentials out of necessity, due to constraints of time, no such constraint applied to verbal feedback lessons; due to its collective nature I tried to cover all possible errors. Now, however, I have changed my practice by reducing such lessons to between two and four key points of feedback per assignment.
I was dismayed to discover through my reading for this research, that the marks I was giving were depriving the students of the benefits to be gained from my carefully written comments. As a result of this reading, for one topic in the next academic session I planned a short series of assignments that would lead up to a cumulative assignment. The shorter assignments received feedback without marks, merely celebrating what the student could already do (medals) and identifying what more the student could do to attain the next level (missions). This time there was a significant jump in marks (13-23 per cent) for the cumulative task. It is not possible to be sure that this was due exclusively to focus on learning, but it is certainly worth exploring further.
The action research drove home to me the importance of taking into account the less than perfect retention by students, of including only core material in my lessons, of not overcrowding feedback lessons, and of customizing feedback. My classroom practice has changed as a result of these insights. Having viewed myself teaching on the video I made as part of the project, I have also decided to speak more slowly. Moreover, I tend far more frequently to phrase comments in the form of a question (termed 'provocative feedback' by Hargreaves, 2014) instead of an instruction or 'evaluation'. Above all I think it has given me an incentive to create time to hear the students' voice again, something I did regularly as a novice teacher.
The degree to which students expressed appreciation of the use of individualized and customized feedback encouraged me to continue using ipsative feedback. If my students are filtering out much of the generic commentary, can I really afford to rely on it as a primary instrument for improvement?

Notes on the contributor
Rizwana Nadeem is a senior teacher for O Levels at the Boys' Branch, Lahore College of Arts and Sciences, Johar Town Campus in Lahore, Pakistan. Over the last decade, she has had countless enthusiastic, outspoken language students ranging from Year 3 to Year 11, who have taught her that she still has a lot to learn about assessment for learning.