A qualitative formative evaluation of a patient‐centred patient safety intervention delivered in collaboration with hospital volunteers

Abstract Background Evidence suggests that patients can meaningfully feed back to healthcare providers about the safety of their care. The PRASE (Patient Reporting and Action for a Safe Environment) intervention provides a way to systematically collect feedback from patients to support service improvement. The intervention is being implemented in acute care settings with patient feedback collected by hospital volunteers for the first time. Objective To undertake a formative evaluation which explores the feasibility and acceptability of the PRASE intervention delivered in collaboration with hospital volunteers from the perspectives of key stakeholders. Design A qualitative evaluation design was adopted across two acute NHS trusts in the UK between July 2014 and November 2015. We conducted five focus groups with hospital volunteers (n=15), voluntary services and patient experience staff (n=3) and semi‐structured interviews with ward staff (n=5). Data were interpreted using framework analysis. Results All stakeholders were positive about the PRASE intervention as a way to support service improvement, and the benefits of involving volunteers. Volunteers felt adequate training and support would be essential for retention. Staff concentrated on the infrastructure needed for implementation and raised concerns around sustainability. Findings were fed back to the implementation team to support revisions to the intervention moving into the subsequent summative evaluation phase. Conclusion Although there are concerns regarding sustainability in practice, the PRASE intervention delivered in collaboration with hospital volunteers is a promising approach to collect patient feedback for service improvement.

staff and patients to address this need. The intervention supplies health care providers with theory and evidence-based measurement tools for the routine collection of feedback around the safety of care from patients, alongside a framework for staff to interpret and act on that feedback. 1,[3][4][5][6][7] The efficacy of the PRASE intervention in achieving patient safety improvements over a 12-month period has previously been explored in a randomized controlled trial 5 with an embedded qualitative process evaluation. 8 The intervention comprises two measurement tools: first, the Patient Measure of Safety (PMOS), 3,4 which is a questionnaire based on the content of the Yorkshire Contributory Factors Framework; 9 second, the Patient Incident Reporting Tool (PIRT), 1,6 which enables patients to report detailed safety concerns and/or positive experiences. The anonymous feedback collected using these tools is presented to ward staff in the form of a feedback report. An iterative action planning cycle follows whereby ward staff consider the feedback in an action planning meeting and target improvements based on problematic areas. The three key stages of the PRASE intervention process are (i) measurement, (ii) feedback and (iii) action planning and change (see Figure 1). Early PRASE development work found a facilitated discussion at the patient's bedside to be the most successful in terms of volume, accuracy of patient reports, and "richness" of the data. 1,6 Although a facilitated discussion may introduce the potential of bias, offering participation in PRASE in this way aims to represent the best balance of richness of data, inclusivity in participation, and reduction of error in data collected.
A common challenge following the development of evidencebased interventions is how to spread innovation in practice. 10 The PRASE intervention does not prescribe who should collect the patient feedback. However, to provide a sustainable mechanism beyond the research studies in which the intervention was developed and tested, an improvement project is exploring the potential for hospital volunteers to collect the patient feedback. The project is implementing the PRASE intervention in acute care settings. Figure 2 describes the procedure for PRASE volunteers visiting wards to collect patient feedback.
With health care organizations globally functioning with strained resources, [11][12][13] and in the UK the National Health Service (NHS) under ever increasing pressure, 14 paid ward staff have little or no spare capacity to undertake this role. Furthermore, there is evidence to suggest that patients prefer to report safety concerns to an independent party rather than through established, staff-led reporting systems. 15 Therefore, the rationale for the use of hospital volunteers stems from the need for a method of capturing the patient perspective of safety which is viewed as "independent" from the patient perspective which is not overly resource intensive.
Studies have shown that volunteers can provide cost-effective support in resource constrained health care settings. 16,17 Research from the USA indicates that volunteers can offer significant cost savings to hospitals. 18 Further, evidence is building that demonstrates the positive impact health care volunteers can have on patient experience and quality of care. [18][19][20][21] Recent findings from the first national survey focussing on the scale and value of volunteering in NHS acute hospitals highlighted that the average number of volunteers in an acute trust is 471.
Administrative support (e.g general administration and administering patient surveys) was amongst the top five roles volunteers are engaged in. 22 Indeed, the nature of the role of volunteers in healthcare is evolving, and volunteers are increasingly being called upon to facilitate the collection of patient feedback. 23 However, there is currently little empirical evidence regarding the feasibility and acceptability of volunteers performing this role from an organizational perspective as well as from the perspective of the volunteers themselves. Therefore, we aim to address this gap within the context of PRASE.
This study explores the opportunities and challenges of implementing the PRASE intervention in collaboration with hospital volunteers within acute health care services. In this study, we present the results of a formative evaluation which spanned the pilot phase of the project. We explored the perceptions of key stakeholders (hospital volunteers, voluntary services, patient experience staff and ward staff) whose views we felt it particularly important to capture in the early phase of implementation. We intended to generate learning to contribute to the development of PRASE hospital volunteer recruitment, training, management and retention over the longer term. Involving hospital volunteers in the PRASE intervention is innovative; therefore, Therefore, to embed PRASE in hospital routines in a sustainable way, we needed to understand the context around the PRASE intervention with hospital volunteers from the perspective of these staff. Finally, we were also interested in the implementation of PRASE with hospital volunteers from the perspective of ward staff, particularly their views on hospital volunteers collecting PRASE feedback. Furthermore, as ward staff are integral to the "active ingredient" hypothesized to facilitate service improvement within the PRASE intervention (the action planning process), it was essential their perceptions and experiences were explored in the pilot phase.

| Research questions
Experiences of key stakeholders

| Evaluation objectives and approach
There are many uses of formative evaluation in the context of implementation research. 24 The main purpose of our formative evaluation was to address questions around feasibility and acceptability, and we aimed to provide the project implementation team with timely information to augment their learning prior to scale-up and spread.
A summative evaluation followed the formative work presented in this article. The summative evaluation phase focussed on whether the PRASE intervention with hospital volunteers resulted in improvements in quality and safety outcomes, as well as experiences of the key stakeholders over time.
Our evaluation approach was informed by recent publications which provide guidance on the evaluation of quality improvement work. [25][26][27][28] This literature posits a number of key principles that guided our evaluation approach: (i) flexibility and the need to be responsive to changes in implementation, (ii) iteratively revisiting the evaluation approach, and, (iii) the need to have an explicit programme theory, and a shared understanding of this amongst the wider project team. At the outset of the project, an earlier version of the PRASE intervention programme theory 6 was adapted as a result of on-going discussions between the wider project team. These adaptations aimed to reflect the addition of (i) hospital volunteer recruitment, training, management and retention, and, (ii) staff engagement to embed the PRASE intervention with hospital volunteers into routine ward activities. The programme theory delineated the hypothesized processes through which we expect the programme activities to result in both proximal and distal outcomes, with explicit reference to moderating factors known to F I G U R E 2 PRASE volunteer ward visit procedure affect these processes. We continually revised the programme theory throughout the pilot phase as a result of implementation learning and formative evaluation findings. We present a version of the programme theory which was agreed between the wider project team at the end of the pilot phase in the results section of this article. Being explicit about the programme theory allowed us to pinpoint the key stakeholders whose views it was important we capture within the formative evaluation.

| Patient involvement in the design and conduct of the study
Patient representatives were involved throughout the development of the PRASE intervention, and along with clinical staff, helped to codesign the measurement tools. 29 This project involved patient representatives in a number of ways. First, the research aim initially arose from discussions about the sustainability of the PRASE intervention with patients and health care professionals. Additionally, a patient representative was part of the initial application for funding, as well as invited to attend all steering group meetings. Their role was to provide input into both the implementation and evaluation components of the project, bring their experience as an active hospital volunteer, and to be involved in wider dissemination of the project progress.

| Design
A multiple methods qualitative approach was adopted including focus groups with hospital volunteers and interviews with voluntary services, patient experience and ward staff which spanned the pilot phase of the project. We held multiple focus groups with hospital volunteers to account for turnover. The focus groups and interviews followed a semi-structured format, and were mainly focussed around the participants' early experiences of their involvement in PRASE and their views on implementation (e.g barriers and levers) with specific questions varying for each participant group. Follow-up hospital volunteer focus groups revisited discussions from earlier focus groups where appropriate.

| Setting and sample
Participants were recruited from two acute NHS trusts in the UK between July 2014 and November 2015. We conducted five focus groups with hospital volunteers (n=15), interviews with voluntary services and patient experience staff (n=3), and ward staff (n=4).
Information pertaining to the characteristics of the participants is given in Table 1*.

| Data analysis
The focus groups and interviews were digitally recorded and transcribed verbatim. We analysed the data taking a framework approach, 30 and we used NVivo 10 Software for data management. The  T A B L E 1 Characteristics of the participants and some more so to the staff interviews. Therefore, we present key themes for these stakeholder groups separately.

| Procedure
Hospital volunteers were invited to take part in the focus groups by a member of the local project implementation team. Invitations and participant information sheets which explained the purpose of the evaluation and the focus groups were distributed in advance, and dates for the focus groups were organized on behalf of the evaluation team. At one of the sites, three volunteers were involved from the very early stages of the project and therefore participated in two focus groups, which allowed us to follow up on their experiences. Staff were approached directly by an evaluation researcher and invited to participate in the interviews, and participant information sheets were distributed in advance of the interviews. Written informed consent was obtained from all participants. One of the main aims of the formative evaluation was to feed back early evaluation findings (within two weeks of data collection) to the project implementation team, enabling learning to be applied to the on-going implementation of PRASE in a timely way.
Given the small numbers of staff involved, to protect confidentiality, these participants were shown the evaluation findings we intended to share with the project implementation team in advance. Staff were given the opportunity to delete information and make amendments † .

| RESULTS
We present key themes from the volunteer focus groups and staff interviews separately. Descriptions of each theme are provided, as well as illustrative excerpts. In addition, we present the participants' views on implementation. We also reflect on the impact of evaluation activities on implementation and provide examples of how the formative evaluation findings informed changes to project implementation, and finally, how these findings relate to programme theory development.

| The experience of hospital volunteers
Pertaining to the first research question "What are the expectations, concerns and on-going experiences of hospital volunteers involved in the PRASE intervention?" three themes were identified from the

| Skills, knowledge and experience
The volunteers acknowledged the diverse skill set and knowledge required for the PRASE role, some of which are also required for other hospital volunteer roles. Emphasis was placed on the ability to balance the emotional aspects of the role and communicate effectively with both staff and patients, and an understanding of patient safety.
In addition to skills around the use of the technology, being able to navigate a ward environment and maintain a neutral and diplomatic stance when collecting patient feedback were also recognized.  BOX 2 Hospital volunteer focus group excerpts reflective of the theme "Supporting hospital volunteers" "It would be nice for us to have feedback as to how we've been and to know this has made a difference or if it's not working…"

Hospital volunteer 1, Female
[Referring to group sessions]"…you could talk about your different experiences on the wards and different things that have cropped up… and you can say well how did you cope with that one, you know, how did you get round that one…"

Hospital volunteer 2, Male
"And I think my own feeling, this is well due, something like this because having been in hospital and by and large the care has been perfect, the care has always been good but the procedures might not be so good and it seems uncharitable to complain…"

Hospital volunteer 9, Female
BOX 3 Hospital volunteer focus group excerpts reflective of the theme "Skills, knowledge and experience" "…interpersonal skills, you know, to actually relate to someone at various levels on it. It would be nice to have different languages as well like, you know, being able to build a quick rapport isn't it, you know, and listening, active listening as well…"

Hospital volunteer 7, Male
"…being impartial because, yes I understand that we're not there to influence the patient in any way shape or form. I think that's a difficult thing but, you know, you've to deal with it."

| Involving volunteers in PRASE
Staff were overwhelmingly supportive of the implementation of PRASE with hospital volunteers. Given their independent standing, staff viewed volunteers as well placed to collect the feedback.
However, even as part of a pilot scheme the infrastructure needed for the implementation of PRASE with volunteers was viewed as substantial, the management and organization of which over the longer term would require dedicated capacity. In the early stages of the project, volunteers were recruited from existing hospital volunteer cohorts, and subsequently a person specification was developed for the PRASE volunteer role, which was advertised both internally and externally. Staff emphasized the need for this targeted recruitment to ensure volunteers with the right skills are identified for the role. Staff In addition to patient variation, there was an acknowledgement that there would be variation in how the volunteers collect the feedback and the supporting infrastructures in place which may have implications for data quality. This was mainly raised as an issue by ward staff who would be using the feedback to support improvements, and it was suggested that the provision of appropriate training and support may alleviate this potential variation. There was much discussion of the logistical and practical considerations surrounding PRASE with volunteers, not least the organization of volunteers and how this fits within existing workloads, but also the unpredictability of the ward environment and patient availability. Regarding staff views of pilot implementation, many staff found the project management structure to be somewhat unclear across the sites, but again there was recognition that this was to be expected in a pilot scheme. See Box 4 for illustrative excerpts.

| Moving from data to action
Although we were principally interested in the added dimension of involving hospital volunteers in PRASE, many of the discussions with ward staff focussed on their views of the intervention more generally.
PRASE from the perspective of ward staff has been explored comprehensively in the embedded qualitative process evaluation 8 within the randomized controlled trial of the intervention, 5 and many of our findings support some of the conclusions of this previous work. Staff viewed capturing patient feedback for service improvement as essential, and saw PRASE as a meaningful way of achieving this. However, some potential limitations were also discussed. For example, if over the longer term staff continue to receive feedback they are not able to put into action-for example because some issues are difficult to address at the ward level-their engagement in the process may decline. So although staff liked the fact the PRASE measurement tools collected more detail and context, there was a sense that the tools need to be fit for purpose if the aim of data collection is to lead to actionable change. Linked to this, the amount of information within a feedback report considered at a meeting needs to be manageable.
Staff also described barriers around the logistics of the action planning element of the intervention, in terms of getting the right staff together and finding time for these meetings. These challenges which relate to the intervention more generally, that is using the feedback to support  to a broader issue, that is the lack of dedicated time for staff to "think about safety." Staff also described the need for new initiatives to consider the current NHS context and existing initiatives. Relatedly, staff voiced that how PRASE feedback sits with other sources of patient feedback needs to be considered, and stressed the need for initiatives with similar aims to "tie together." Finally, although staff were supportive of PRASE and its potential benefits, concerns were raised around the evidence base in terms of improving patient safety outcomes and demonstrating cost-effectiveness. See Box 6 for illustrative excerpts.

| Impact of evaluation activities and early findings on implementation
In undertaking focus groups with volunteers and interviews with staff, it became clear that there was a sense of uncertainty, for example, about the specifics of the PRASE intervention and project timelines.
Therefore, participants valued the opportunity to share their experiences, get peer support (volunteers), and ask questions that they had about their wider understanding of the intervention. Whilst supporting the volunteers and staff is of course a useful activity, and we were pleased that they felt that participation was beneficial for them, it did highlight to us the issue of reflexivity in our evaluation activities. In

| Description of main findings in relation to existing literature
Hospital volunteers saw the value of their role in the PRASE intervention and the potential benefits for patients and for service improvement, but they also recognized that how the PRASE intervention sits BOX 5 Staff interview excerpts reflective of the theme "Moving from data to action" "…but there were some things that seemed really kind of obvious when you see it written down that yeah why aren't we doing that for example and also could readily be sorted with not a lot of resource and no financial kind of input and perhaps just changes in practice kind of thing…"

Ward staff
[Referring to action planning meetings] "…So I think a good facilitator will help to eat the elephant one bite at the time.
Which is needed and get to the real issues rather than be tied up with the periphery of it. Stuff that isn't well, it is important but only if it is indicating broader concern about you know communication for instance. It is not the individual exchanges that are crucial in that case. So yes, I think they need to be facilitated…"

Ward staff
in terms of the current NHS context (e.g staffing issues) needs consideration. The volunteers stressed the importance of ensuring PRASE was inclusive to as many patients as possible, and the need for the PRASE measurement tools to be fit for purpose. Referring to their involvement over the longer term, the volunteers felt the training and support mechanisms (e.g handover/debrief, group sessions, feedback on performance) would be influential, and this tied in with the different motivations volunteers had for their involvement, such as a desire to "give back" in a way that was also rewarding and a "need to be doing something." When we look to the wider literature around hospital volunteering, these findings are in line with research that highlights the importance of organizations understanding volunteers' motivations and meeting their expectations for retention. 31,32 As some of the elements of the PRASE volunteer role are different to traditional hospital volunteer roles, the volunteers described a diverse skill set which they believed to be important, for example communicating effectively with staff and patients, balancing the emotional aspects of the role and remaining neutral. Access to adequate support around the emotional aspects of the PRASE volunteer role seemed to be crucial. Indeed, the need for volunteer managers to be aware of how factors such as stress and sadness may affect volunteers in a particular role, and to provide the necessary support has been acknowledged in the wider literature. 33 Reflecting on the main findings from the voluntary services and ward staff perspective, overwhelmingly staff were enthusiastic about the potential for PRASE to support meaningful service improvement.
They were unanimous in their belief that volunteers were well placed to collect feedback from patients given their independent standing.
The importance of targeted recruitment was emphasized, and the need to fulfil the needs of individual volunteers to ensure retention.
Concerns were raised around the resource and infrastructure re-

| Implications
One of the main aims of the formative evaluation was to provide the implementation team with timely information which could be incorporated into project implementation. We achieved this aim, with our findings directly contributing to the volunteer retention plan (i.e the inclusion of regular volunteer group "catch-up" sessions), the shortening of the PMOS questionnaire, and the recommendation of hospital volunteer involvement in ward action planning meetings. The formative evaluation findings informed the refinement of the programme theory for PRASE delivered in collaboration with hospital volunteers which was taken forward to the roll-out phase of the project.
Regarding the use of hospital volunteers to facilitate the collection of patient feedback in health care more generally, which could be viewed as a departure from traditional hospital volunteering roles, our findings emphasize the need for targeted recruitment and adequate training and support, with particular support available for coping with the emotional consequences of collecting feedback directly from patients on a regular basis. However, we do of course recognize the more substantial nature of the PRASE volunteer role in comparison with other ways volunteers have been involved in patient feedback collection (i.e facilitating the collection of "Friends and Family Test" 39 in the UK).

| Limitations
This was a formative evaluation of an improvement initiative, meaning our research activities were limited to the organizations involved in the wider project. It is possible that this had implications in terms of generalizability, as the findings may be specific to the context of the two NHS trusts involved. However, given that our findings reflect issues from the wider literature on hospital volunteering, we are confident our findings are applicable more broadly. A further limitation of this formative evaluation is that the patient perspective was not within the scope of the study, in the sense that we did not explore patients' views of the implementation of PRASE with hospital volunteers. We know from previous research that patients prefer to report safety concerns to an independent party rather than through established, staff-led reporting systems, 15 and there is evidence demonstrating the success of collecting patient-reported safety concerns by facilitated discussion at the patient's bedside. 1 However, we do not know if the volunteers are viewed as "independent" from the patient's perspective. Although the need to maintain a neutral and diplomatic stance was acknowledged, given the many aforementioned motivations for participating, it is essential for the volunteer training and on-going support to reinforce the importance of impartiality and role boundaries to mitigate any concerns about perceived independent status of the volunteers. In the light of the above limitations, we recommended that future research explores patients' views of hospital volunteers facilitating the collection of patient feedback, to understand whether this approach is feasible and acceptable to patients.
We also advocate that research focusses on the impact of the PRASE intervention delivered in collaboration with hospital volunteers on quality and safety outcomes, in addition to cost-benefit analyses.

| Conclusion
This theory-led qualitative formative evaluation suggests the PRASE intervention in collaboration with hospital volunteers is feasible and acceptable to key stakeholders, whilst recognizing barriers from the perspectives of these stakeholders which need to be addressed. The formative evaluation findings were incorporated into project implementation, and offer valuable insights into the use of hospital volunteers as facilitators of patient feedback collection more generally.