Supporting future scholars of engaged research

Researchers in the UK are taking on new roles and responsibilities to meet the requirements of an expanded agenda for generating and evidencing social and economic impacts from research. Within this wider context, culture change programmes have identified learning as an important driver of change. Here we outline a professional development programme designed to train postgraduate researchers studying environmental sciences in core engagement, influence and impact, governance and organization skills for research. We argue that training is an important step in further catalysing progressive culture change. However, our researchand experience-informed critical reflections in supporting researchers suggest that there is still significant work to be done: (1) to offer consistent messages to researchers at all grades about social impacts from research and (2) to ensure that engagement is seen as an aspirational activity, embedded within

• Systematic culture change requires interventions in terms of purpose, process and people; significant among these is training and support for people (researchers, support staff and non-academic stakeholders) at all stages in the research cycle, from conception to publication and beyond. (For the purposes of this paper, we take non-academic stakeholders to include end-users, members of the public and any other non-academic beneficiaries.) • Training and support should be combined with ongoing support mechanisms and measures to systematically recognize and reward excellence in engaged research.
Engaged research encompasses the different ways that researchers meaningfully interact with various stakeholders over any or all stages of a research process, from issue formulation, the production or co-creation of new knowledge, to knowledge evaluation and dissemination. (Holliman et al., 2015a: 3) In this context, 'stakeholders may include user communities, and members of the public or groups who come into existence or develop an identity in relationship to the research process' (Holliman et al., 2015a: 3). Not without its critics (for example, Watermeyer, 2014), this extended scholarship agenda includes a requirement to embed plans for generating and collecting evidence of research impact into research proposals.
Understandably, questions have been asked about the rationale for these changes (for example, Holmwood, 2014). Put simply, why should researchers work in this way when critics have argued that this could lead to research that has been conceptualized purely to meet the requirements of an external policy agenda, and not in the interests of relevant stakeholders, end-users and members of the public? Put more pragmatically, why should researchers work this way when they feel that it takes time away from doing research?
We share some of these concerns but argue that, if done well, engagement between researchers and research stakeholders can: (1) improve the quality of research and (2) improve the impacts arising from the research for those who participate in its production and those affected by the outcomes (Holliman et al., in press). Ideally, engaged research should address both points 1 and 2 through holistic, upstream planning, combined with participatory forms of downstream project management and governance (Holliman et al., forthcoming).
Major UK funders of researchers have worked to coordinate and support this expanded agenda for research, introducing principles for supporting culture change as universities and researchers look to engage with publics, stakeholders and enduser communities in ways that are relevant, meaningful and sustainable to the parties involved (for example, RCUK, 2013a;RCUK, 2010). To this end, one of the four principles codified in the Concordat for Engaging the Public with Research includes a requirement, 'Researchers are enabled to participate in public engagement activities through appropriate training, support and opportunities' (RCUK, 2010: 8).
Principles are important, of course, but consistent and effective implementation, which we argue should be informed by relevant research findings, is also key (Holliman et al., 2015a). Concerted efforts have been made by UK public funders to support embedding the principles of engagement within research, including the introduction of the UK's National Coordinating Centre for Public Engagement (NCCPE) and various levels of funding for more than 25 research-intensive universities. This funding includes, in chronological order: six Beacons for Public Engagement (NCCPE, 2012a); eight Public Engagement with Research Catalysts (RCUK, 2012); 12 School-University Partnership Initiatives (SUPIs) (RCUK, 2013b) and 10 Catalyst Seed Fund universities (RCUK, 2015b).
Through the Beacons for Public Engagement Initiative, the NCCPE (2012b) identified three broad areas that require strategic and operational attention if culture change is to be achieved: purpose, process and people. Each area includes three contributing elements: purpose (leadership, mission and communication), process (learning, support and recognition) and people (staff, publics and students). These nine key elements offer a framework in which systematic interventions can be introduced to Research for All 1 (1) 2017 support research culture change within universities (NCCPE, 2010). Among these nine, learning was seen as an important driver of change: Public engagement can bring great rewards -both for the university and for the public its staff and students engage with. But this mutual benefit can't be taken for granted: it relies on real skill and expertise. Many staff and students recognise that they need help to develop these skills and aptitudes, and an institution that wants to support them effectively needs to take account of this. (NCCPE, 2012b: 3) The need for learning identified by the NCCPE and the Beacons is further supported by research findings that show a mixed picture of how researchers define, practise and assess performance in engaged research (Jensen and Holliman, 2016;TNS BRMB, 2015). These findings, combined with our experiences of supporting more than 1,000 researchers through 60+ interventions across academic domains and with all academic grades, indicate a continuing need for learning, combined with ongoing support to embed new practises (Holliman et al., 2015a). It follows that we have concerns about the rapid nature of the changes required in the research practices of UK-based researchers. Researchers at all grades, including reviewers of grant proposals, are uncertain about the value of engagement in relation to the research impact agenda (Watermeyer, 2015) and, crucially, about what counts as excellence in this area, not least when compared to established forms of esteem (principally assessed by external funding and peer-reviewed publications in high-impact journals).
Furthermore, researchers are confused about the relative importance of communication within a research career. Put simply, should communication with nonacademic audiences, encompassing traditional and social media, be considered essential work for researchers ? If so, how much time should be spent on these activities, how is this activity supported by institutions, and how will excellence in this area be recognized and rewarded (Holliman, 2015)? Our previous research has shown that many researchers who have been developing skills and responsibilities end up 'muddling through'  as they learn through experience and anecdotal support from colleagues when planning pathways to generate social and economic impacts from their research. Many of these researchers are also supervisors of postgraduate research students, which is where the funded projects we describe in our acknowledgements come together. These projects aimed to support, and report evidence for, a cultural shift towards broadening and deepening engaged research practises across all academic domains (Holliman and Holti, 2014). In a response to research findings that highlighted a demand for professional development opportunities in engaged research  and to help researchers stop just 'muddling through', we developed a theoretically informed, hands-on training programme for postgraduate researchers. In supporting these future scholars of engaged research, we sought to change current and future practices. Here we offer some research-and experience-informed critical reflections that focus in particular on training and support offered to postgraduate researchers working in the environmental sciences. In so doing, we address the following questions: • What lessons can we learn from previous conceptualizations of open, digital and engaged scholarship to inform and support future scholars of engaged research?
• Which skills and competencies are required to be a reflective scholar of engaged research? • Which training and support resources might usefully be shared across the higher education sector?
In planning for the workshops (described in detail below), we drew on an earlier review of existing resources designed to support public engagement training through the Open University's Public Engagement with Research Catalyst (Holliman et al., 2015a). Several of these resources were developed through the Beacons for Public Engagement initiative (NCCPE, 2012a), others through teaching, training and research initiatives at the Open University. As such, they were designed to support a holistic conceptualization of scholarship, addressing: (1) sharing, principally teaching, (2) discovery and research and (3) application and integration, for example engagement to generate mutual benefits, changes and effects with relevant publics, stakeholders and end-users (adapted from Boyer, 1996). Our more limited remit was to address a scholarship of engaged research, in effect exploring the links between discovery, application and integration. We therefore needed to select resources to support this more focused remit. Furthermore, we struggled to locate resources to support forms of open, digitally mediated research (Wilks and Pearce, 2011;Weller, 2014;Weller, 2011).
In addressing this need, we drew on recently produced, research-informed resources developed to support the digital attributes of engaged research (Collins et al., 2015;.

Training programme overview
The training programme addressed two key themes: communication and engagement. We argue that an understanding of both is essential for an engaged researcher, yet research findings have identified confusion about the distinctions between them (Jensen and Holliman, 2016), and more recently about how they relate to the research impact agenda . In part, this comes back to well-rehearsed arguments about the purposes of communication and engagement and how, in turn, the purposes inform who is involved, how, when, where and so on (Holliman et al., forthcoming). In practice, we found Irwin's (2008) orders of engagement useful in this respect (see Table 1). Irwin (2008) describes three orders of engagement, which, in effect, map on to practices of communication and engagement. If planned for effectively, they can be deployed at different stages in the research cycle. As an example, first-order engagement is mainly about communication. This could be useful when funding is first announced, and towards the end of the research when findings need to be publicized. Second-order engagement lends itself to dialogue, which could be useful when consulting with a particular stakeholder on the direction of one or more aspects of the research. Finally, third-order engagement involves multiple stakeholders, which could be useful in the planning phase, and in connecting the findings of a complex engaged research project to the development of public policy and/or practice (Holliman et al., forthcoming). It follows that the training programme was designed to help the researchers consider whether and how a strategic approach to communication and engagement could improve the quality of their research, and how that research influenced relevant stakeholders. To complement existing training programmes we used the Researcher Development Framework, which was developed by Vitae (2010). More specifically, we used the Public Engagement Lens developed by Vitae in collaboration with the NCCPE and Research Councils UK (Vitae/NCCPE, 2013) to identify and develop skills and competencies in two main areas of open, digital and engaged scholarship with end-users (the latter was a requirement of the funding of the Innovation Award we secured). It is important to note at this point that, as researchers and supervisors, we are more than aware of the challenges that postgraduate researchers face in securing longterm employment in research following a successful viva voce (Leshner, 2015). Part of our rationale in preparing our plans for this training, therefore, was to offer delegates opportunities to develop and evidence transferable skills that could be useful within or beyond an academic context. As such, over the lifetime of the training, delegates explored conceptual issues related to: • Engaged research, for example: who are the end-users for your research; who could they be; how could you work with them in ways that are meaningful for your end-users; when should you engage with your end-users in the research cycle; through what methods; and how will you know the engagement was productive for the various participants? • Communication, for example: how do you best present your work in the public sphere to encourage end-users to work with you, and how can you create communities who connect with your research over time (Collins et al., 2015)? The researchers also developed and practised new skills in digital communication, such as creating an online profile, mediating scientific information for different audiences (Holliman, 2007), and working with recording equipment, producing pieces-to-camera and editing footage to produce web videos about the social and economic impacts of their research.

The training programme
We offered training to three cohorts of up to ten first-year postgraduate researchers in the environmental sciences. Delegates were required to do some preparation in advance of a full-time residential element running over five days. We also offered a number of follow-up activities.

Preparation for the workshop
We started our planning for the workshop from the premise that all researchers need some form of digital presence, one that allows end-users and other researchers to have easy and open access to an institutional profile (Collins et al., 2015;Terras, 2012). This had the secondary function of introducing the trainers to each of the research topics being studied by the postgraduate researchers. We therefore asked the delegates to submit a list of the potential or realized end-users of their research and how they might engage at different stages of the research cycle. In support of this, we circulated a set of slides that had been produced by the NCCPE outlining different types of end-user and the stages of research (see Figure 1 and Figure 2). We also asked delegates to submit a research summary of around 250 words in the form of an 'elevator pitch', aimed at one or more of the potential or realized endusers of their research. Together, these two activities provided a vehicle for starting to explore Domain D of the Researcher Development Framework, addressing core skills and attributes in D2: Communication and Dissemination and D3: Engagement and Impact, as well as Domain C1: Professional Conduct.

Developing skills in engaged research and communication
The research summaries were circulated to delegates in advance of the residential element. This allowed us to explore some of this information when we initially met. This was followed by a review of the objectives of the training, and a discussion about the research impact agenda and support for culture change in research (Holliman et al., 2015a). The discussion about the research impact agenda provided a vehicle for starting to explore Domain C of the Researcher Development Framework, addressing core skills and attributes in C2: Research Management.
Following this introductory session the focus switched to the development of practical skills in communication, for example, preparing recording equipment and techniques for interviews and presenting on camera. To this end, delegates were split into two equal groups, each working as production teams for the filming conducted throughout the rest of the week. As homework on Day 1, the delegates were asked to read a short article that offered a vision for what an engaging postgraduate thesis could look like in the year 2033 (Holliman, 2013), and to start sketching out ideas for their pieces-to-camera based on their introductory summaries.
The second day of the residential training began with a vote on whether the vision for what an engaging postgraduate thesis could look like in 2033 was one that they would buy into as potential supervisors (Holliman, 2013). In effect, the article offered a vision for the type of supervisor the current cohort of postgraduate researchers may (or may not) want to be. We were surprised, given the wide range of academic disciplines, that the number of votes for and against the vision were pretty even, and the number of 'it depends' responses was low. The voting process itself stimulated an interesting discussion, in particular addressing Domain D3: Engagement and Impact, but also aspects of Domain D1: Working with Others. For some, the vision was progressive and desirable. For others, it was incredibly problematic, mainly due to concerns about: (1) how to meaningfully engage stakeholders, end-users and members of the public with blue skies research, (2) a loss of control over the direction of research and (3) whether the time/effort spent on engaged research would be perceived as equivalent to more traditional research outputs.
The activity worked well to promote discussion among the delegates, in part because the moderator did not seek to introduce a right or wrong answer to the discussion. Rather, the activity provided a space to explore views on key aspects of the engaged research and communication agendas. Delegates also discussed how the increasing impact agenda would and could affect researchers' careers in the short-, medium-and long-term, for example by thinking about the potential to integrate engaged research into their current PhD projects and beyond.
Following this discussion, the delegates continued their work with the communication trainers, planning for their pieces-to-camera. The interviews were recorded so that delegates could review and reflect on their performances. If the delegates were happy with the results, they were posted online. As an example, see 'The social and economic significance of researching Masaya Volcano, Nicaragua', by Bethan Parkes: https://youtu.be/afZbiB3PTpk.
At the end of Day 2, the delegates were asked to begin planning their group video productions. Each group was asked to agree on a key theme or take-home message for their video and to begin storyboarding ideas for the content, and thinking about locations, expert commentary and props for filming. In successfully taking control of this aspect of the training, the postgraduate researchers addressed some of the advanced skills listed under Domain D1: Working with Others.
Day 3 began with the delegates returning to the engaged research agenda, exploring how this connects with planning for generating and collecting evidence of research impact and assessments of quality. The theme for this discussion was the assessment of quality, focusing in particular on Domain C3: Finance, Funding and Resources. The delegates were provided with two authentic Pathways to Impact plans. Both plans had been written by the same researcher: one prior to the introduction of institutional support for preparing these plans, the other produced in collaboration with an expert in engaged research. The delegates were asked to assess both plans using a set of criteria developed by the NCCPE, covering: (1) the non-academic stakeholders, (2) the aims and objectives, (3) the timing of the interventions in relation to the research, (4) the methods for engagement and (5) what measures were planned for capturing evidence of the social and economic impacts (see Figure 3). The delegates assessed the plans both quantitatively, scoring each of the five elements out of 20 (totalling 100), and qualitatively, through consideration of whether they considered the plans to have met the 'acceptable' threshold required by Research Councils UK (RCUK, 2015a). Finally, through discussion and reflection the delegates were asked to consider what constructive feedback they would provide for the researcher.
Following this activity, the delegates worked intensively on their plans for their videos, producing storyboards, sourcing props, writing scripts and interview questions, and producing a shooting schedule for Day 4, which was spent filming. Together, the planning and filming phases addressed Domain D1: Working with Others and D2: Communication and Dissemination (see Figure 4). Day 5, the final day, started with a reflective activity. Using the Public Engagement Lens developed by Vitae in collaboration with the NCCPE (Vitae/NCCPE, 2013) the delegates were asked to document their skills and competencies in engagement and communication. These skills were then mapped on to an authentic set of Further Particulars for an academic research position. In so doing, the delegates were asked to identify their strengths to add to their CVs, and weaknesses to address through future professional development.
The remainder of the final day was spent editing and reviewing the films, which you can view here: www.youtube.com/playlist?list=PL1-6ZFoq8C6fa5o_E9-T6FvG9yLSb7GIC.
As a final activity, the delegates were interviewed to explore their immediate perceptions of the training, and to explore their ideas and ambitions for engaged research in the future: What have I got out of this week? Quite a lot I would say. Not only just about the media and how to film and how to present, but also on the research engagement side. We did a lot a lot of good sessions about our CVs, and how to best approach research engagement. (Postgraduate researcher, 13 February 2015) Finally, we asked the delegates for suggestions to improve the training for future cohorts of postgraduate researchers. The findings from this informal evaluation informed the later training workshops; for example, we extended the amount of time available for editing the rough cuts following feedback from one of the earlier training weeks.

Post-residential training
Following the residential workshop, delegates were encouraged to revise their initial research summaries, using them to update their online profiles to include discussion of the engaged elements of their work with end-users. For an example, see Kerry Reid's profile: www.open.ac.uk/people/kr6473.
Similarly, delegates were encouraged to blog about their experiences, to reflect on the two key themes of communication and engagement: I think we were all apprehensive about being filmed. Top tips from Janet Sumner were to look animated, smile for the camera and not to use scientific jargon! We were all filmed in an interview situation about who we are, what we do, why we do what we do and the significance of our research and why is it important to the audience. (Clare Lawson, quoted in Holliman et al., 2015b) I like the fact that this course challenged me to present and explain my research to a range of audiences, as it gave me a moment to step back and think about the importance of my research and who is actually going to benefit from my work … Some really interesting topics were raised and we had a chance to look over the RDF [Researcher Development Framework] and talk about past experiences and opportunities we have taken part in and how they could enhance our profiles, emphasising the importance of a researcher who can address different audiences and communicate with a range of stakeholders. (Kerry Reid, quoted in Holliman et al., 2015b) The delegates were sent examples of authentic academic CVs where evidence of the products of knowledge exchange featured heavily, and encouraged to include evidence of engaged research on their CVs. Furthermore, delegates were given the opportunity to borrow recording equipment and to produce their own films. In reflecting on the training 12 months on, one of the delegates said: I have done some filming and spoken on camera a couple of times, but we have not managed to complete the projects yet. I definitely think that this course really contributed to that work and, as I said, built my confidence.

Reflections on the training
Planning for, delivering and reflecting on the training programme has raised a number of important issues worthy of reflection. The work of the Beacons and Catalysts highlighted that engaged research requires ongoing institutional support to flourish (NCCPE, 2012b;Holliman et al., 2015a). We note, therefore, that within our unit we have seen changes, partly enabled through, and partly supporting our work on, this Innovation Award. Gaining external funding for the training has helped to raise the profile of engaged research at a local level, where it is now routinely reported on at departmental meetings, but also across institutions through Doctoral Training Consortia.
More prosaically, funding has also been secured for equipment and presentations at international and national conferences. Furthermore, we drew on examples of authentic paperwork from our unit for recent recruitment and selection activities where engaged research was an essential requirement of the person specification. We used this paperwork to explore how the core skills developed by postgraduate researchers through the training could be used to apply for jobs.
In terms of our preparation and delivery of the workshops, we note that effective collaboration and cooperation require at least some level of shared language to aid understanding and, broadly speaking, common aims (Underwood and Underwood, 1999). We began this engagement with engaged research from very different perspectives. Although we are based in the same university department, our academic backgrounds could not be more different: social scientist meets Earth scientist. Prior to developing this training, we had different conceptualizations of '(the) public(s)', engagement and, to some degree, what we meant by research. We were united, however, in our common aim of supporting postgraduate researchers as they become encultured into the working practices of being a researcher in the twenty-first century, and such multidisciplinary challenges are representative of what institutions will probably have to overcome in the future.
The training provided an opportunity to make our thinking visible, as we questioned and clarified key concepts from our respective disciplinary backgrounds. In practice, this required us to put pen to paper on a range of important questions, such as, 'Where does "outreach" end and "engagement" start?' and 'Is public engagement a valid form of research impact?' We also spent time reflecting on how academics conducting 'blue skies' research can meaningfully connect their work with relevant publics. Developing our shared language between our social and Earth science backgrounds is still a work in progress, but co-authoring this paper has provided another opportunity to question and clarify our sense of key terms and concepts. We argue that a healthy respect for language and different forms of expertise, supported by ongoing dialogue, are features of any successful multidisciplinary endeavour (Schumner, 2009). They are also important aspects of any productive engagement involving researchers and non-academic stakeholders. It follows that a pluralistic appreciation of both language and expertise helped to shape and frame how we worked with postgraduate researchers through the training we offered.
Ultimately, though, the success or failure of the training should be measured by the delegates. We offered an introduction to some of the core skills and competencies in communicating and engaging with contemporary research. For the training to be successful in the long run requires that researchers gain useful conceptual knowledge and practical skills that can be applied during their PhD research. To explore this issue, we contacted the researchers more than 12 months after the workshops and asked them: Of the 25 researchers we contacted, 9 responded. The responses explored aspects of the two key themes for the workshops -communication and engagement: I have used the experience in several ways such as I did a camera interview which was then added to the University website as a virtual open day piece … The people undertaking the interview commented on how they could tell I was experienced in giving interviews! … I feel that I am much more conscious of what I am 'giving' to my collaborators, and how they will use the outputs of my research. With this in mind, this has enabled me to tailor my methodological design to make sure that my results are transferable to my end users. (Postgraduate researcher, 11 April 2016) I cannot think of a 'least useful' aspect of the course as I frequently now use much of what I have learnt on the course in activities ranging from CV writing, conference presentations (oral and poster) and wider engagement activities. As such, I can highly recommend the course and would go so far as to suggest that such courses should be made available to all earlycareer and post-doctoral researchers.
(Postgraduate researcher, 10 April 2016) While we are delighted to see evidence of the utility of the training, we also note two issues: (1) our sample of delegates was self-selecting and we should not make assumptions about the reasons why postgraduate researchers from the same Doctoral Training Partnerships chose not to attend the workshops when they were not full to capacity and (2) direct and immediate support from supervisors for engaged research is essential for researchers to see this approach as aspirational for their long-term careers (Holliman, 2015). We know of several instances when a lack of supervisory support for engagement has led to conflicts with postgraduate researchers.

Conclusion: Sustaining a scholarship of engaged research
Gradually, in part through sector-wide support within higher education, there is evidence emerging that cultures of research are adapting to the principles of engagement (Jensen and Holliman, 2016;Holliman et al., 2015a;Grand et al., 2015;TNS BRMB, 2015). We argue that training is an important step in further catalysing this culture change in progressive ways, demonstrating to researchers how, through effective engagement and communication, they can evidence a wide range of skills and competencies in job and promotion applications (Holliman, 2015;Vitae/ NCCPE, 2013).
Through our work with postgraduate research students studying environmental sciences, we have observed some of the ongoing tensions facing UK public funders, researchers and engagement trainers in supporting postgraduate researchers. For example, during the planning and delivery phases of the workshops, we encountered confusion about key terminology, such as 'engagement', 'communication', 'impact' and 'public'; a lack of shared criteria to assess what does and does not constitute high-quality engagement; confusion between the most appropriate methodologies for engagement and communication, and when to deploy them in the research cycle; conflicting priorities between research, communication and engagement; concerns about a lack of aspirational career paths for engaged research in some academic domains; and varying levels of enthusiasm for engagement. Through the training, we sought to address these serious issues and concerns, and in so doing address core skills in relation to engagement, influence and impact, and research, governance and organization (Vitae/NCCPE, 2013). It remains to be seen whether further training in advanced skills and attributes could further embed the engaged research agenda for different grades of researcher, but we note that leadership programmes have been successful in the past (see, for example, DeWitt, 2008).
However, training is only one step in this process. To embed cultural change requires attention to the other eight elements identified through the work of the NCCPE and the Beacons initiative (NCCPE, 2012a). Ultimately, for engaged research to become embedded requires that researchers value these activities as an aspirational route to a successful career (Holliman, 2015), and that consistent assessments of quality are made to ensure that excellence replaces acceptance as the threshold for planning for research impact (RCUK, 2015a). This requires ongoing support throughout annual cycles of workload management and mentoring, as well as consistent recognition and reward of excellence, and therefore a shared understanding and application of measures of quality.
programmes that allow students to acquire public engagement skills. Her publications are listed at: http://oro.open.ac.uk/view/person/cw6522.html.