Dialogic assessment in the context of professional recognition: Perspectives from the canoe

This paper draws on the reflective evaluations of four experienced academic practitioners who each have roles in leading or contributing to Fellowship Schemes in a UK context (United Kingdom Professional Standards Framework, 2011) and are all Advance HE accreditors. The focus for the evaluation was the use of a dialogic approach within their respective Fellowship Schemes. The aim of the project was to better understand the conditions in which dialogues can thrive, and to surface the challenges. A collective autoethnographic method was decided upon to frame the evaluative process. Four one-hour online meetings were scheduled over a period of six weeks. The data were transcribed and analysed using a mind-map process and resulted in the emergence of a metaphor – that of a canoe trip in which five phases were identified: designing the vessel, getting aboard, settling in/ settling down, navigating the space and forward-wash. These five phases are overviewed here with the intention of provoking discussion in the academic practice community, drawing into conversation individuals who are concerned with assessment for learning generally, and those who have particular interest in the potential of d ialogic assessments which culminate in a summative judgement. It adds to the literature focused on assessment in higher education by drawing to the fore the conditions in which dialogic approaches can thrive for the individual being assessed, while also fa cilitating reciprocal learning.


Introduction
Assessment in higher education has provided a fertile space for discussion and debate with innumerable papers written to asse rt its purpose and consider how it might be better experienced by students. Yet, despite its potential to support learning and e nable authentic assessment defined by Sambell (2016) as assessment which is meaningful to those undertaking it, with purpose and relevance beyond passing an assignment, the literature suggests that dialogic approaches are under-used, their use confined to feedback processes, rather than being the actual method of summative assessment (Charteris, 2016). In a secondary education context, Brindley and Marshall (2015) believe dialogic assessments are unexploited because of perceived obstacles which include their reliability.
In Higher Education (HE) dialogic assessment has gained traction is in the context of professional learning with adoption into Fellowship Schemes of note (see Smart, Campbell, Asghar and Huxham, 2019 for wider discussion). Yet, there is not one way to design a professional dialogue, possibly because there is variation in their intended purpose. Not all, for example, embrace Appleby and Pilkington's (2014) view that professional dialogues should enable reflection, support ing individuals to explore their tacit understanding and helping them to reconstruct their practice. Smart et al. (2019) reflect on Appleby and Pilkington's (2018) and also Crowley's (2014) description of professional dialogues, pondering the challenge of ensuring the optimum conditions for professional dialogues and how they might thrive, particularly when that which intends to be enabling results in a judgement (an assessment decision) being made. As such that paper provides a valuable grounding for this one. Before exploring its methodology and the metaphor which emerged from the reflection evaluation, further context is required.
The UK Professional Standards Framework (UKPSF) for teaching and supporting learning in HE, first developed in 2006 and revis ed in 2011 (Higher Education Academy (HEA), 2011), provides an international benchmark focused on the practice of learning and teaching (Smart et al., 2019). The popularity of gaining recognition with HEA Fellowship is evident with over 130,000 Fellows now numbered world-wide (Hustler, 2020). The process by which Fellowship is conferred takes the form of direct application to Advance HE, or through accredited institutions offering taught programmes or experiential schemes, the latter being where individuals are recognised on the basis of their experience against one of four Fellowship categories. Irrespective of the route, recognition depends on a successful assessment decision decided by peers. While there are 123 HEA accredited institutions worldwide, written claims are still the most common form of assessment (Hustler, 2020). Some accredited institutions have however created more flexible approaches, enabling the individual's account of practice to be presented in an alternative format, with dialogic approaches the most common of these (Pilkington, 2016). Sector knowledge suggests that few align with Asghar and Pilkington's (2018) description of a professional dialogue (see above). This paper focuses on three accredited schemes which have sought to facilitate meaningful, purposeful professional conversations and as such align with Asghar and Pilkington's (2018) thinking. Each aims to enable individuals to deconstruct and reconstruct experiences to make sense of them, potentially creating new meanings (Bruner, 1991;Clandinin and Connelly, 2000;Rowland, 2001). But they are different one from another (Table 1). For the purposes of this paper, these conversations are referred to as professional dialogues and while it is the case that they take different forms in each of the three schemes, they share a bel ief in the potential power of the professional conversation to promote learning (Danielson, 2009) through the facilitation of 'reflective critique' (Kreber, 2013, p.99). And while recognising the imperative for robust assessment decision making, as required by Advance HE for accredited schemes, the three share a commitment to valuing the person coming forward into the professional dialogue, believing in the need to treat them well in a conversational process which is two -way, value-based and respectful, committed to listening to them speak to their academic practice. As four individuals who know each other, and have worked together in different roles, we decide d to take the opportunity to engage in our own professional dialogue focused on our respective schemes and their dialogic approaches that emerged from a shared concern to better understand the conditions in which they thrive, and to surface the challenges so as to learn and sha re our thinking. We wanted to evaluate our Fellowship schemes, specifically their use of a dialogic approach. We decided upon collective autoethnography, an approach which centres on the personal, drawing on individual experiences and perspectives to build understanding in the context of wider socio-cultural challenges (Wall, 2016, cited by Henderson, 2019. In making our design choice, we were aware of the critique attached to collective autoethnography -that it is solipsistic, an indulgence of little value to others (Henderson, 2019). However, we know that stories have a place, as sources of learning and development (Conlon , Smart and McIntosh, 2020), offering a means to create connections, and challenge realities (Moon and Fowler, 2008). In short, we believed a collective autoethnographic approach would enable us to explore and represent our stories of experience (Adamson and Muller, 2018), and co-create an enhanced narrative to challenge our thinking, and that of our readers. There was no intention to generalise or make claims to validity and reliability in traditional terms (Ellis and Adams, 2014).
To facilitate our collective narrative, we planned three conversations in a virtual space. In keeping with a collective autoethnographic approach, in advance, we each shared an artefact which represented our experience of the dialogic approach in our own schemes. Each virtual meeting lasted one hour and was audio -recorded and transcribed. The transcription was used to create a mind map representing the emergent themes (Kinchin and Winstone, 2017). At our fourth and final meeting we agreed the themes which drew from a metaphor emergent in significant conversation 3 (SC#3). Trust was identified as permeating the process of the professional conversation in each of our schemes .

From Data to a Conceptual Model
In this discussion and exploration of our data, we use the metaphor of a canoe trip to communicate our key findings and have consciously adopted the use of 'we' throughout. It is our own professional dialogue that made sense of the data, identifying emergent themes, and ultimately forming the basis for our learning. Figure 1 presents the five phases of the canoe trip; for clarity we note that what might be termed the summative assessment takes place at the conclusion of the fourth phase. For note, SC refers to significant conversation of which there were four; R1 means Researcher 1 etc.

Building the Vessel
Our three universities share an institutional goal to increase numbers of staff holding Fellowship, a common driver across th e UK sector (Peat 2015, in Shaw, 2018. Each scheme has an established documentary route to Fellowship. While there was no obligation to offer a different approach, the opportunity to do so -in effect to build a different vessel -was identified. For R3, the need for an alternative emerged from prior experience of supporting Fellowship applicants applying directly to Advance HE. Recognition of their difficulties in finding the 'right voice' (SC#1), supported the belief that that a dialogic approach wou ld be: more inclusive […] a better opportunity […] to talk about what they were doing in a way that would do justice to their experience (SC#1).
Focus on a 'right voice' led us to the challenge of using the discourse of learning and teaching, where for many individuals i t is as if they are being asked to articulate their practice in 'another language' (R4:SC#1) there is stuff that people can express in a way that is conversational that they find very difficult to express in academic reflective writing because part of their attention is spent searching for the discourse register (SC#1).
To this challenge, we recognised that the UKPSF can add a further layer of complexity because it requires familiarity with a specific lexis, which may exacerbate the linguistic barrier. For R2, there had been a real experience of being an 'outsider' looking i n on the UKPSF while 'fumbling around in the dark, not being able to actually quite figure out what is required of me' (SC#1).
Listening to each other, we became increasingly sure that a dialogic design is much more than a simple alternative to a documentary approach. Not only must the build allow for the fact that talking about learning and teaching can be difficult and exposing, it needs to focus on strengths rather than requiring 'a defense of practice' akin to a PhD viva (R2:SC#1). Moreover, it should not be 'interrogation-like', but rather offer a medium where the individual can 'express themselves in their own register of conversation', 'genuinely and authentically' (R4:SC#1). Secondly, the design must facilitate a natural conversational 'ebb and flow' which is 'in the moment' and where 'people feel very valued' knowing what they do is 'important enough for others to sit there and listen' (R4:SC#1).
Our respective dialogic designs shared another key feature -an extended scaffolded approach, comprising a range of dialogic interactions and culminating in the final assessed conversation. Thus, 'it is a process, not an event' (R1:SC#4) with support integral to the build. The nature of the support varied in the three schemes, either through the use of a mentorship approach, 1:1 and/ or group. However, while we recognise the potential of other approaches to offer the scaffold -and it is this scaffolded approach which we argue to be essential to the build -all enabled preparatory and practice exchanges with colleagues, where tacit understanding had to be clearly articulated and clarity and confidence of expression could be honed. Put simply, scaffolded support is 'a very critical part of our schemes' and their assessment (R1:SC#2). Closing this section, we acknowledge Shaw's (2018) recognition that introducing a dialogic component to a Fellowship scheme is not without challenge, in terms of resources and robust decision-making, but for us the potential outweighed the risk. We believed in its ability to empower and effect change, with the opportunity for reciprocal learning inbuilt.

Getting on Board
Despite our view that dialogic approaches are attractive propositions (R1:SC#1), similar to Peat (2015) our collective experi ence is that getting people on board with the Fellowship project -documentary or dialogic-can be challenging, not helped by contractual requirements, or institutional imperatives which may engender resistance and result in tokenistic engagement (R2:SC#3). Focu sing specifically on the dialogic approach and the professional conversation it seeks to enable, R3 identified two potential challenges for the individual, fear of the unknown and fear of exposure because 'you can't hide very well in a conversation' (SC#1).

Fear of the Unknown
Unsurprisingly then a dialogic design culminating in assessment may not suit everyone (R4:SC#2). However, in each of our institutions it is a popular choice, selected because individuals perceive they might 'find their voice' (R3:SC#1) more readi ly in the spoken rather than written word. The fact that in our respective schemes we have taken care to explain the dialogic approach and the intention to be valuing, 'not to pick holes' (R2:SC#2), providing a 'safe space' in which participants 'will not be chall enged in a way which makes them feel uncomfortable' (R2:SC#3), is, we would argue, a key principle which we explore further in Navigating the Space in terms of always upholding it.

Fear of exposure
For R3, there is understanding that the individual may not fe el they 'belong within the system' feeding into 'imposter syndrome' (SC#4), where the lack of perceived identity with a professional group does not resonate (Lawler, 2008). When added to R4's thinking -'talking is actually the window on to identity' (SC#1 ) -it is possible that the dialogue might be perceived as just too risky, too revealing, too exposing of a professional identity which is not aligned with learning and teaching. Closing this section, we note that however supportive the dialogic route professes to be, engagement with it may 'be unsettling', taking you out of your 'comfort zone' (R3:SC#4).

Settling in/ Settling Down
Our data highlighted a critical stage in the dialogic process where the facilitators adopt the role of host, enabling individ uals to settle into the conversation. The voice of the host -mentor or assessor-is powerful intending to help the individual se sentir bien dans sa peau -[be comfortable in their skin] (R4:SC#1). The data suggest some differentiation in the tone of the ho st, depending on the role -mentor or assessor-with the voice of the mentor potentially more directive, more probing, more coaching in style (R3:SC#2). For R4, the mentor role is key because it is this relationship: where the practice starts about feeling able to open up about what is an intensely personal thing and something you don' t talk about to everybody all the time (SC#1).
For assessors, a relationship needs to be established quickly and benefits from a 'facilitative approach […] with a level of gentleness and concern for the applicants' knowing it might be 'scary' for them (R3:SC#1). Nevertheless, within the final conversation, where the Fellowship judgement is made, a different voice might sometimes be needed, one which is 'more formal and lacks th at gentleness' (R2:SC#1). Nevertheless, the intention is that the dialogue facilitates reciprocal learning, a feature of the des ign which we suggest should not be understated. Put more strongly, reciprocal and persistent professional learning is a design feature of our dialogic approaches into which all those involved -applicants, mentors and assessors-need to settle. This suggests a need for the right mindset, without which the dialogue can be problematic. The next section highlights what can make navigating the space 'rocky and tricky and difficult' (R4:SC#4).

Navigating the Space
Whilst our dialogic approaches seek to provide a reciprocally beneficial process, ultimately a judgement is made on meeting, or not, Fellowship requirements. In our own conversation, we recognised the challenge of getting this balance right, understanding that some applicants may ultimately not be successful. Therefore, while mentors and assessors seek to engage the applicant in dialogue, there may be 'choppy water' and 'rapids' (R4:SC#4). In the forward movement, there may be 'nervousness' and a need for mutual trust. Primarily, though not exclusively, for the applicant the process of navigating the course, avoiding the 'boulders' (R1 :SC#4) and the risk of capsizing (R4:SC#4), may be fraught with danger. Unpacking our data, enabling smooth navigation of the space is challenging, not least the reality of individuals who just 'didn't want to board. You'll never get them comfortable' (R4:SC#4), those who are there to find the easiest way to 'tick the box' (R2:SC#3) and/ or are unwilling to engage in the spirit of the dialogue (R4:SC#3). But we also identified the significance of assessor skills, with R3 speaking to the importance of not having a script, but of scaffolding the conversation (SC#2), a perspective returned to in SC#4 when drawing on the canoe metaphor: […] the person at the back isn't really giving the forward momentum, they are just gently steering and realigning and nudging […] we facilitate the conversations […] you let the person have the run of it, but […] we've got to nudge them through the UKPSF slalom course, occasionally having to just tip them slightly on the shoulder and kind of, let's go this way R2 (SC#3) develops this further: we've got to give that person the best chance possible to meet the standards that we were assessing against. So , it's about trying to strike balance between a natural conversation […] making sure it is steered in such a direction as the discussion revolves around all of the different dimensions of the practice at the level of the descriptor that we are trying to encourage the candidate to achieve.
Our data acknowledge the challenges of this approach: you're steering, guiding, flexing, waiting, interrupting those are quite signific ant cognitive skills when […] you're playing with the PSF at the same time. It's not just you're steering a conversation; you're actually steering something that's going to make a professional recognition decision (R1:SC#4).

Assessors' Mindset
R3 suggests another potential problem in the navigation of the space which comes from empowering assessors to 'manage the conversation' knowing that pet hobby horses coming galloping through […] where people (i.e. assessors) have got themselves fixed in a belief t hat a certain thing needs to be said or seen or whatever. Maybe a particular pedagogy that they're hung up on, and that I think can be a challenge […] They (i.e. the individual) has their own authentic voice and their own experiences in their own approaches, and we need to listen to that rather than layer on our personal.
R4 supports the reality of this risk speaking about assessors adopting 'a more aggressive or suspicious mindset', an orientation which can be 'very damaging' (SC#3). R2 offered an illust rative example of this speaking of an individual whose experience of the professional conversation did not align with expectations established during the mentoring process, ultimately leaving them disillusioned. As R2 explained, for a positive experience to transpire, 'all parts of the jigsaw […] have to come together and be aligned with the same values to make the end experience a really enjoyable one' (SC#3).
However, as noted previously, our data highlight that the navigation of the space may sometimes require the assessors to be l ess facilitative and more exigent, simply because the final conversation constitutes a formal judgement where the specific requirements of the Fellowship category must be met. Therefore, while the plan is to be value -based and affirming, it may be necessary to deliberately steer the conversation appropriately (R2:SC#3), a technique requiring 'interjecting' or 'chipping i n' at the right time (R3:SC#3) and which may be experienced as disruptive.
Keeping an open mind The design of our final assessed dialogues requires applicants to provide material in advance to scaffold the conversation. In some instance these materials may seem deficient, but as R3 notes the dialogue itself can 'confound expectations' (SC#1). From R2's perspective in this instance: … assessors need to make sure [it] doesn't cloud their judgment until the entire process is concluded (SC#1).
R3 concurs that some professional conversations may need to 'travel a distance', adding that assessors need to be 'capable of framing the conversation in such a way that it enables the person […] to surface the stuff that you haven't heard about or yo u need to hear about' (SC#1). Our data stress the skills needed to help navigation of the space , with R4 emphasising listening skills (SC#3) and R3 adding: You have to be agile enough to flex what you thought the shape of the conversation was and the direction you wanted to take it into something different as it evolves and without the person feeling manipulated (SC#3).
The indication here is that facilitating the professional dialogue is demanding, requiring skill, something we had understood , but perhaps not as starkly as became clear in our conversation.

Forward-wash
Quiet in our data, but still present was the journey's end. Captured by R2, forward -wash acknowledges that while we have been: […] steering it at the back, at some stage we need dropping off on that route because our job finishes at some point, but what we hope is that the other people in the canoe carry-on -wherever they're going we don't know, but what we hope is that it's got that persistence -they carry on down the river, in the canoe (SC#4) Quite where individuals go depends on them, but we know some mentees keep contact with mentors, described positively by R2 as 'an unintended consequence' (SC#2), and individuals become mentors and/ or assessors themselves because they want 'to give ba ck' (R4:SC#2). Another possibility relates to more reluctant participants.
I think we have to hope that some of the learning on their part, maybe more implicit and not as obvious to them, is also sticky […] hopefully some of that will persist (R3:SC#2).

R1 reiterates this hope in SC#4:
That's the persist, isn't it? That's the idea of, you know something stays with them once they're away on their own.
There is an element of trust in the hope expressed here, directing us to the theme which pervaded each of our four conversati ons, suggesting its primacy within our respective dialogic approaches.

Trust
Trust presents as a necessary condition, and from a risk perspective. Positively, for R2: it's about being able to provide a supportive developmental safe space with trusted coll eagues to be able to talk about your practice and try to share the things that you've been really good at (SC#1) For R3, trust has a number of elements. First, understanding the solitary nature of learning and teaching practices: we need to establish trust with them (the individuals) that it's ok for them to share their experiences good and bad […] so I think we have to build that relationship, build that trust very quickly (SC#3).
Secondly, the need to assure the individual that 'what happens is a trusted judgment […] that we're robust and appropriate, that we have the credentials' (SC#3). Here R3 references the tension identified earlier -when the verbal and non-verbal messages displayed in the dialogue do not align with the outcome: We can be so nurturing, so caring and kind, but that doesn't help us if we're actually having to say that didn't go very well .
R4 concurs with this contradiction, adding that while the whole process, including the mentoring, is designed 'to foster t hat sense of trust, it can be popped in a moment', either because of the negative mindset of the assessor(s), or because of the decisio n being made against Fellowship requirements.
Given the significance of trust in the professional conversation, but also the ease with which it might be compromised or lost, it seems essential to adopt the stance suggested by R4 -'you have to be in it (i.e. trust) to be able to live it' (SC#4) -knowing that despite every effort the process will not always have a positive outcome. The canoe may ground on the rocks.

Emergent Reflections
We entered this enquiry, adopting a collective autoethnographic approach because we wanted to learn from and with each other. We deliberately sought the opportunity to talk, share, listen an d question to better understand our own schemes where we know dialogic approaches thrive. We were open to what the data might offer, and were able to identify in them affirmations, surpri ses and ongoing challenges.
Starting with the affirmations, the data evidence demonstrated the power of the professional dialogue to transform thinking -that of the Fellowship applicant and the mentors/assessors, a similar finding to Matthews and Dobbins (2020). We lived our own experience of this through the collective autoethnographic methodology and can testify to the power of significant conversations to support and enable a culture of learning (Roxå and Mårtensson, 2009). As staunch advocates of dialogue, we challenge individuals operating Fellowship schemes which profess to use a dialogic approach to critically examine the extent to which their design is empowering, enabling of reciprocal learning and transformative change (Shaw, 2018).
We were surprised that our data spoke so much about t he conditions needed for the dialogue to embody characteristics of the intended design: enabling of voice, valuing of the person, respectful of practice. And while not a surprise as such, we are now wholly convinced that the dialogic approach is a process, not simply an event, within which scaffolded support is a key design feature.
Our data convinced us that the navigation of the dialogue space is highly skilled, and it is here that we reach out to the wider academic practice community. We argue that any type of assessment which adopts a dialogic approach in its design would benefit from ensuring that the person in the rear seat of the canoe can flex to the situation, while maintaining academic and/or professional standards. Therefore, we invite readers to contemplate not only the relational dimension of dialogic assessment, but the skillset needed and the means by which it might be enhanced.
Finally, back to where this paper started, as noted earlier, dialogues which result in a summative judgement are underused as an assessment approach. In many ways our data support academic practitioners who choose to stay away from them. They are complex, they require skill and careful thought in their design. And yet, we know they enable assessment for learning and moreover facilitate reciprocity in their process. As members of the academic practice community, wherein we know the debate continues in respect of authentic assessment, we argue for dialogic approaches, not just because they offer an alternative to written formats, but rather for the reason that they have the capacity to facilitate meaningful, relevant learning which privilege the real-world, not the performance. We would welcome further discussion and debate on our learning in the academic practice community. Equally we would encourage conversation about the use of collective autoethnography as a tool which can guide the process of reflective evaluation.