Using Audio for Summative Assessment Feedback: Experiences of Work-Based Postgraduate Sport Coaching Students

Personalised, understandable and balanced feedback is well documented as having a transformational influence upon the academic, professional and personal learning of learners. However, an emerging literature base suggests traditional forms of written feedback may fail to sufficiently engage and empower high numbers of modern-day higher education students. In recent times, academics have been encouraged to consider alternative ways of producing and delivering high-quality feedback which addresses assessment marking criteria, indicates how well students are understanding new material, recommends how future performance may be acted upon and improved, provides constructive criticism when necessary, plus strengthens students’ capacities to self-regulate their own work. Over the past decade, across wide-ranging HE disciplines, university student cohorts have warmly embraced receiving audio feedback as an effective way for providing expert guidance, justifying grades, enhancing motivation to learn confidently and autonomously plus feeding forward. However, current understanding regards its effectiveness in nurturing student achievement is based predominantly upon research employing descriptive quantitative methodologies with full-time undergraduate cohorts. Distinctive aspects of this unique study, focusing on work-based postgraduate sports students, include going beyond simplistically evaluating if audio was considered “superior” or “inferior” than traditional written feedback. Employing semi-structured qualitative interviews, this study explored initial emotional-responsiveness, resulting satisfaction and engagement plus consequent impact upon professional-practice of eight part-time postgraduate sport coaching students who received audio feedback for the first time during the final year of a sport coaching Masters degree at a North of England University. Findings provide evidence to inform future feedback policy that may be more appropriate for meeting the academic capabilities, needs and motivations of future work-based postgraduate cohorts. Recommendations and guidelines for supporting the re-tooling and professional development needs of academic and support staff across the sector are also presented.


Introduction
The merits of timely, honest, respectful, insightful, understandable, personalised, memorable and balanced assessment feedback are well documented as powerful and transformational influencers upon the continued academic, professional and personal development of Higher Education (HE) students (Brown, 2015;Cann 2014;Davis & Ryder 2012;Hattie & Timperley 2007;Lizzio & Wilson 2008;Nichol 2009). Effective feedback directly addresses assessment marking criteria, indicates how well students are understanding and engaging with new material, recommends how future performance may be acted upon and improved , provides constructive criticism when necessary, plus strengthens students capacities to self-regulate their own work (Brown, 2015;Dixon, 2015;Middleton, 2011;Sambell, 2016). Previous studies have reported how HE students value, have high expectations and eagerly await summative grades and supporting comments (Brown, 2015;Higgins, Hartley, & Skelton, 2012). Yet, feedback without action in unproductive (Sambell, 2016), and it has become increasingly apparent how traditional forms of written only strategies may be failing to sufficiently engage, motivate and empower high numbers of modern day century HE learners to improve future work (Dixon 2015;Gibbs & Simpson 2004;Rotheram 2009).
To illustrate, examples from the extant literature demonstrate how they have struggled to apply received written feedback to future work and take the next steps, describing it as being overly complex, challenging to interpret and understand plus generally dispiriting, unhelpful, alienating and unsupportive (Brown, 2015;Cann, 2014;Fawcett & Oldfield, 2016;Glover & Brown, 2006;Ryan & Henderson, 2017;Scott et al., 2013;Värlander, 2008;Walker, 2009). To counter these issues, wide-ranging efforts have recently been made to persuade HE colleagues to consider alternative ways of producing, communicating and delivering increasingly engaging, constructive and valuable summative feedback which justifies grades but also respects feelings, provides personalised and authentic learning experiences, promotes self-esteem and perceived self-efficacy, plus nurtures a positive connection between themselves and their students (Bond 2009;Chalmers, MacCallum, Mowat, & Fulton, 2014;Hepplestone, Holden, Irwin, Parkin, & Thorpe, 2011;Laughton, 2013;Macgregor, Spiers, & Taylor, 2011;Rotheram 2009;Rowe, Fitness, & Wood, 2014;Warner & Miller, 2015).
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Employing digital technology to create and cascade audio feedback whereby helpful praise plus areas for future development are carefully discussed and conveyed using a positive, enthusiastic and caring tone of voice, has been extensively trialed over the past decade (e.g. Chalmers et al., 2014;Fawcet & Oldfield, 2016;Laughton, 2013;Lunt & Curran, 2010;Merry & Orsmond, 2008). Across wide-ranging subject disciplines within HE, student cohorts have warmly embraced receiving audio feedback as a useful strategy for enhancing their motivation to learn, self-awareness, critical-reflection, capability to work autonomously and confidence to approach lecturers (Cann 2014;Dixon 2015;Jackson, 2012;Knauf, 2016;Lunt & Curran, 2010;Merry & Orsmond, 2008;Middleton, 2011;Rowe, 2011). It must also be acknowledged how several negative reactions towards audio feedback populate the extant literature base. For example, Munroe and Hollingworth (2014) revealed recording and editing mp3 files as barriers to creating audio feedback, whilst King, McGugan and Bunyan (2008) struggled to locate quiet locations to actually record the audio. From a student perspective, Rodway-Dyer, Knight and Dunne (2011) found first-year geography undergraduates were more likely to perceive audio feedback negatively than their second and third year peers due to its overly severe perception and Bilbro, Iluzada and Clark (2013) reported there are students who prefer their feedback in written only format.

Study rationale and contribution to literature
The need for this study arose because current understanding of audio feedback effectiveness, as a tool for enhancing student engagement, morale and achievement, is based predominantly upon retrospective recall (e.g. data collected up to 6 months after audio feedback was delivered to students), employing full-time undergraduate student cohorts and descriptive quantitative data collection tools (e.g., Fawcett & Oldfield, 2016;Lunt & Oldfield, 2010;Martini & DiBattista, 2014) which may restrict the potential to fully unearth, explore and explain the meanings of obtained findings. To break new ground and redress this balance within the literature, distinctive aspects of this unique study included utilising semi-structured interviews to go beyond simplistically evaluating if participants considered audio to be 'superior or 'inferior' than traditional written feedback. Therefore, the primary aim of this qualitative study was to capture work-based postgraduate sport students' expectations, attitudes and experiences of receiving audio feedback for all summative assessments within two separate 30 credit work-based learning modules. Specifically, the three objectives of the study were to:

1.
Explore participants' initial emotional response to receiving audio feedback.

2.
Unearth the general impact of using audio feedback upon satisfaction and engagement levels.

3.
Establish the value of audio feedback in supporting participants to feed-forward, plus consequent impact upon professional development and work-based practice.

Background context of work-based programme and modules
Employer engagement is central within all strategic plans at Northumbria University. The institution has pioneered work-based learning (WBL) provision for many years across the HE sector and has a work-based learning framework (WBLF) which enables departments to develop flexible and responsive WBL programmes and modules to meet the needs of learners within the workplace. Nationally, the Department of Sport, Exercise and Rehabilitation had been at the forefront of academic developments within workbased sport-industry provision over the past decade.
Each academic year, students attend three, two-day intensive teaching blocks, comprising seven hours delivery per day; thus, 14 hours contact per month in late September, mid November and mid March respectively. Using a blended learning approach, the part-time programme is delivered using a combination of face-to-face block-delivery including seminars, tutorials and discussion groups, plus additional on-line interaction support, by means of skype and blackboard collaborate.
Part-time students undertake two coaching-specific modules in year one (initial 12 months of programme) and two work-based learning modules in year two (months 13-24 of programme). The dissertation module is completed in year three (months 25-30 of programme). Assessment one promoted personal and professional skills development and was examined through a critically reflective 5,000-word work-based portfolio. A key learning outcome was to raise awareness of the workplace as a ripe learning environment. Assessment two developed techniques required to plan, organise, lead, and critically evaluate a 4,500-word work-based project in a coaching organisation, thus granting opportunity to enhance individual effectiveness, employability and core coaching knowledge

Method Participants
Eight (male = 6 and female = 2) part-time, second-year, work-based postgraduate sports coaching students (aged 24 to 37 years) agreed to participate within the study. Once institutional ethical-clearance was granted, face-to-face debriefs reinforcing study aims, objectives and procedures to be followed were completed. Pre data collection, all were assigned pseudonyms to protect anonymity, informed they could withdraw at any time and provided written informed consent. Individual semi-structured interviews were undertaken with all participants at convenient times and locations throughout May and June 2017.

Procedure and design
When undertaking qualitative research, it is important for the interviewer to build rapport and trust with interviewees (Lincoln & Gubba, 1985). The author previously taught several of the sample as undergraduates, was an ex-postgraduate sport student himself and possessed over 10 years experience of creating and disseminating audio feedback for previous cohorts. This strengthened rapport and bonding, helping to make them feel comfortable and at ease to answer honestly and insightfully. Each interview started with an informal discussion revolving around why they decided to embark upon a postgraduate sport coaching program and their progress to-date. Following this rapport-building exercise, data was collected using semi-structured interviews capturing how the participants viewed the audio feedback provided.
Interviews commenced with open-ended questions which initially probed students emotional responses evoked by audio feedback (e.g. "what was it like to receive for the first time"), moved onto exploring the impact on engagement (e.g. "what was it like receiving audio instead of written feedback") and concluded by focusing on its effectiveness in supporting feeding forward, professional and work-based practice (e.g. "to what extent have you used audio feedback to support your future learning or that of others"). To elicit further richness to responses, supplementary probes were posed ad-hoc including "why did you feel this way", "what was the reason for this" and "how does this compare to previous experiences".

Data analysis
Duration was dependent on how much each participant had to disclose and ranged in length between 23 and 46 minutes. Each interview was transcribed verbatim, scrutinised multiple-times over a seven-day period to detect significant meanings and subjected to similar thematic-analysis guidelines published by Braun and Clarke (2006). Notes reflecting interesting and pertinent participant comments were placed within margins to unearth and capture the essence of the data. Initial associations and connections based on similarities and patterns between emergent themes were made, resulting in the development of three main-categories. Interview extracts representing each theme were selected. The final analysis stage involved developing written-accounts from these themes. Three weeks post-interview, each participant undertook a short member-checking telephone conversation with the author. This reduced ambiguity, enhanced accuracy and validity of responses and enabled participants to add things they may have forgot to initially mention (Lincoln & Gubba, 1985).

Audio feedback construction and dissemination
Each participant received circa eight-minutes of individualised audio feedback in mid-March 2017 which was specific to assignment one (4,500-word portfolio) and again in early May 2017, which was specific to assignment two (5,000-word work-based project). A digital audio-device, with inbuilt universal serial bus port enabling mp3 format recording, was used to create all files. This format is widely accessible and playable on all available modern-day technological devices. All audio files were created, internally moderated, and emailed to participants' university email accounts, ensuring confidentiality and privacy, within 14 days of each assessment submission deadline. As a general rule, it took approximately 30 minutes to produce each audio recording at the desired level of quality for both assessments. This included time taken to read the assignment, identify key points and take home messages to be included and save the file in mp3 format.
All audio feedback, spanning both assessments, was produced and recorded adopting the following six-stage format: 1. Participant greeted in welcoming and pleasant manner with process to follow explained 2.
Clarified which assessment the audio feedback related too 3.
Ensured audio was developmentally focused, supportive and aligned with assessment criteria 4.
Commented logically and insightfully on all assessment sections, emphasising key areas of strength plus future development (even if the work was exceptional) 5.
Reiterated key points to feed-forward, provided grade and offered additional support (e.g. opportunity for informal face-toface follow up meeting) 6.
Provided final summative grade then concluded in a friendly and motivating manner

Initial experience of receiving audio feedback
Each participant had no previous experience of ever before receiving summative assessment based audio feedback throughout their academic studies. All found the mp3 files simple to access with content generally perceived as comprehendible and clearly articulated. They particularly appreciated the understandable terminology and supportive language employed. For both assessments, every participant listened to their individual audio file at least twice within 72 hours of its delivery. This was mainly by phone, but The thought of receiving audio feedback was initially met by all with a general sense of curiousness, vulnerability, apprehension and nervousness, thus provoking several negative emotional responses including "worry", "insecurity" and "uncertainty". They had acclimatised and become accustomed to receiving more traditional written modes of summative feedback. Half wished for their grade to have been provided immediately once the audio commenced, thinking it would help alleviate the heightened discomfort, uneasiness and sense of trepidation they were encountering.
When compared with normal protocol (e.g. waiting for written assessment feedback to be returned), three participants in particular encountered elevated feelings of anxiousness, concern and restlessness in the days leading up to the audio feedback being released. This was particularly prevalent in the preceding 24 hours. In the following quote, Eve explained a discouraging situation of feeling slightly uncomfortable and overwhelmed: For every participant, their early reservations and apprehensiveness speedily eroded as they all started to engage proactively and enthusiastically with the feedback provided. For all, a positive picture began to emerge, with audio becoming perceived as "sincere", "authentic", "poignant", "caring", "helpful", "invigorating" and "heartfelt". The developing "buy-in" was reflected by several participants who discussed a heightened sense of feeling "valued", "encouraged", "appreciated" and "connected" as a result of engaging with the audio feedback.
Broadly speaking, the enthusiasm, expression, nuance and tone of voice employed to create the audio appeared to be a powerful catalyst for nurturing consequent engagement, emotional closeness and self-efficacy. This was clearly the case for Adam who said:

the audio feedback was not condescending or patronising in any way. I am big on relationships and I got a real feel you were going the extra miles to try and help me as a learner. I must say the audio feedback for me, goes beyond, and is empathetic, uncomplicated and critical when needed but all in a nice manner. I felt much better once I had listened through a couple of times after my initial jitters and fears of failure
Orally receiving critical feedback using an enthusiastic but calming delivery style came across to participants as more tactful and considerate. Furthermore, all gauged an accurate initial vibe for quality of work and subsequent summative grade based on the emotive, personable and poignant nature of language provided. For example, Steve stated:

I could tell by the tone of voice if the work was high quality or not which actually really helps and puts me at ease
Participants appreciated the thought, time and effort taken to produce the feedback, plus its potential to positively impact upon their future motivation to learn and individual growth. David emphasises this point in the following passage: hearing the emotion and positive nature of comments gave me so much self-confidence to continue with my studies.

I liked hearing the emotion in the feedback and this helped me digest the key points made in a far better way than any written feedback I have received previously has
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

General engagement
Several participants described their feedback as being concise, informal, balanced and sincere. They liked how the approach encouraged them, more so than written feedback, to take greater responsibility for their learning. For example, John confirmed this by commenting: for me, the audio helped trigger inner-thought and reflection and was far more complimentary than some of the directionless and disjointed written feedback I have received over the years Over half believed audio feedback would lend itself well to other modes of summative assessment, particularly those comprising personalised elements, such as individualised action-plans and reflective-blogs. In comparison to written feedback, participants valued the greater depth, meaning and volume of detail provided. They especially welcomed the insightful nature and non-generic dialogue which supported areas for future development and feeding forwards. The general consensus deemed feedback duration of eight-minutes maximum to be sufficient, with anything longer running the risk of "disengaging" and "losing" participants. To illustrate, Ellie explained:

I would not want any more than seven minutes or so of feedback. I feel if it went on longer, then people may start to get a bit nervous and possibly switch off. I think too much could actually be counter-productive and ultimately lead to people disengaging
For the majority, the experience became a more personable, satisfying, inquisitive and pleasurable way to receive feedback and for consequent learning to emerge. For example, David and Steve discussed similar perceived engagement benefits:

Impact on feeding forward, professional development and work-based practice
Participants valued the bespoke nature of audio feedback, explaining how it felt more like a two-way process than traditional written comments, directly promoting critical reflection, feeding forward and sense of belonging and connection with academic staff. This opposed a sense of passivity, disconnection and alienation which was unearthed as a limitation of previous written-assessment feedback. Half suggested scheduling informal, follow up, face-to-face conversations between themselves and academic staff as being worthwhile, indicating it may allow for further targeted discussion and prompting towards learning independently and implementing effective feeding forward strategies. John said:

I would like to be able to come and meet up with my lecturers having had time to reflect on the information provided and maybe even completed a feed-forward task which encouraged me to think about how best to apply my learning. I think this would take the process to the next level
In their various roles as lecturers, coaches, educators and developers, two participants had already experimented with the approach by mid June 2017, whilst three others had encouraged work-place colleagues to consider doing so in the future. Providing feedback of this nature had not previously crossed the mind of Adam, yet he was in the process of providing grassroots football coaches with audio feedback snippets for short in-situ practical sessions which formed part of their formal national governing body coaching award criteria. As part of her further education lecturer role, Carol discussed having trialed in providing audio feedback to year one foundation degree sports students. She explained: I was interested to go through the process with my own students but I was shocked in that I had to do many repeats and different versions. I now have first-hand experience in that it really does take time and it makes you realise how much effort your lecturer places into creating the feedback for you. I thought it would be quicker providing audio to written feedback.

Discussion and Conclusions
In recent times, audio feedback has grown in popularity across diverse educational contexts and disciplines (Bond, 2009;Carruthers et al., 2014;Dixon, 2015;Lau, 2008;McGarvey & Haxton, 2011;Munroe & Hollingworth, 2014). This project extended the existing research base by carefully considering the voice of eight part-time postgraduate sport coaching students as to their expectations about, opinions towards and experiences of receiving assessment feedback in audio format. The evidence from this study suggests receiving audio feedback has the potential to be a sensitive, unsettling and overwhelming experience for students. Initially, all had limited understanding of the approach and receiving feedback in this manner was met with a general sense of nervousness, apprehension and uncertainty (Lunt & Curran, 2010). Historically, participants had received traditional written forms of summative feedback and were much better acclimatised and familiarised with this process.
Replicating the findings of Gleaves and Walker (2012), Jackson (2012) and Olesova and Richardson (2011), participants initial concerns and worries towards the audio approach soon started to diminish, with all progressively engaging with the feedback This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
proactively and enthusiastically. Consistent with the extant literature, valuable audio feedback was portrayed as being detailed, personable, concise and devoid of repetitive or complex academic language (Dixon, 2015;Ice, Curtis, Phillips, & Wells, 2007;Knauf, 2016;Lunt & Curran 2010;Olesova & Richardson 2011;Rotherham, 2009;Van der Kleij, Adie, & Cimming, 2017;Varlander, 2008;Walsh, Larsen, & Parry, 2009). This clearly emphasises the potential for audio to inspire, motivate and nurture both the learning and day-to-day practice of future work-based postgraduate student cohorts. In line with Hepplestone et al., (2011) and Sipple (2007), ensuring audio is comprehensible, relatable and personable were particularly key findings, whilst potential strategies to promote feeding forward and learning more independently were also unearthed. Whilst the findings of Lunt and Curran (2010) suggest how providing audio feedback has the potential to save staff time, this was not the case for one participant who consequently trialed the approach with their further education sports students, encountering several barriers and operational problems along the way (e.g., having to start recording again after making a mistake). Overall, collective views and findings were generally positively framed and provide valuable contributions to academics, educational-developers, support-staff and senior-management teams, seeking guidance and support to inform future audio feedback design, delivery and practice.
The limited sport-specific audio feedback research is surprising considering the large student cohorts recruited annually to academic programmes within the discipline, and provides avenues for further investigation. For example, mixed-methods studies exploring experiences of both undergraduate sport student recipients plus academic staff who utilise the approach, would prove valuable and timely contributions to the contemporary literature base. Based on the study findings, a collection of 'take-home' recommendations and guidelines are provided in table 1 which may help support greater student engagement with audio feedback in the future. Academic staff, with limited experience or understanding of the approach, should be appropriately supported and provided with appropriate re-tooling and/or professional development opportunities should they wish to create and distribute audio feedback to their students. This applies to all stages of the career-spectrum and not just newly appointed or inexperienced colleagues.
Prior to receiving, it is important that students are suitably informed about the very concept of audio feedback, plus the potential emotional impact it may place upon them. Therefore, encourage them to focus on comments rather than grades. Helping them to understand that everything you say is focused towards supporting them to positively enhance their academic, personal and professional skills forward is crucial.
Be up-front and honest with colleagues with limited experience of producing and disseminating audio feedback. Initially, the process can prove time-demanding, impact heavily on academic workload and challenging to administer, coordinate and manage. This is especially the case with large module cohorts and academic module delivery team. It can also take time to produce for those with limited experience and requires appropriate resource (e.g. equipment to create and edit mp3 files as well as locating quiet places to record).
The early stages of employing audio feedback can be a particularly sensitive time for student buy-in and engagement. Therefore, it is important to offer them additional academic and personal guidance to ensure they have accurate expectations of and feel suitably confident, connected, prepared and supported.
Create communities of practice with colleagues who use audio feedback, or intend on doing so in the future. This may encourage sharing of ideas and best practice, how to improve further and the merits of learning from mistakes.

Biography
Dr Rick Hayman is a Senior Lecturer within the Department of Sport, Exercise and Rehabilitation at Northumbria University, Newcastle and Senior Fellow of the Higher Education Academy. He has over ten years experience in providing summative audio feedback to undergraduate and postgraduate sport student cohorts.