Article Text
PDF
Abstract
Objectives Patient involvement in safety improvement is a developing area of research. The aim of this study was to investigate the feasibility of a patient feedback on safety intervention in primary care. Specifically, the intervention acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data were examined.
Design, setting and participants Mixed-methods feasibility trial with six purposively selected Australian primary care practices.
Intervention The intervention comprised an iterative process with a cycle of measurement, learning, feedback, action planning and implementation period of 6 months.
Primary and secondary outcomes Qualitative and quantitative data relating to feasibility measures (acceptability, fidelity, enablers, barriers, scalability and process of collecting safety data) were collected and analysed.
Results A total of n=1750 patients provided feedback on safety. There was a statistically significant increase in mean patient safety scores indicating improved safety (4.30–4.37, p=0.002). Staff deemed the intervention acceptable, with minor recommendations for improvement. Intervention fidelity was high and implementation enablers were attributed to the intervention structure and framework, use of intuitive problem-solving approaches, and multidisciplinary team involvement. Practice-based safety interventions resulted in sustainable and measurable changes to systems for safety, such as increased access to care and improved patient information accuracy.
Conclusions The findings indicate that this innovative patient feedback on safety intervention is feasible for scale-up to a larger effectiveness trial and further spread into policy and practice. This intervention complements existing safety improvement strategies and activities, and integrates into current patient feedback service requirements for Australian primary care. Further research is needed to examine the intervention effects on safety incident reduction.
- quality in health care
- primary care
- qualitative research
- health services administration & management
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
A feasibility study was conducted prior to the development and implementation of a large-scale effectiveness trial and wider spread and uptake into policy and practice.
Several feasibility domains were assessed including intervention acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data in a primary care.
A mixed-methods approach addressed each feasibility domain and included both qualitative and quantitative data collection and analysis.
A limitation is that the data collected will be mostly descriptive, and therefore, the generalisability of the findings may be limited to only one geographical area.
Introduction
Involving patients in error prevention and harm reduction activities has gained traction over the past decade.1–7 Patient engagement has been found to prevent or reduce adverse events, and increase awareness of potential safety risks.8 Much of this research has centred on hospital settings with the majority of interventions utilising patient feedback mechanisms for safety improvement.3 8–12 The evidence base regarding patient feedback on safety in primary care is considerably lacking by comparison.
In addition to reporting formal safety incidents,13–16 patient feedback about processes, systems and structures that lead to safety incidents is an essential piece of the safety intelligence ‘jigsaw’.17 Patients have demonstrated understanding and knowledge about the various conditions in the latent environment that influence safety, such as access to care; communication systems; information and care planning; and transitions between care settings.17–20 Capturing patient feedback about these contributory factors to safety incidents and using it for safety improvement work in primary care is a developing and novel field of research.21
Only one validated, real-time and theory-derived patient feedback tool for assessment of factors contributing to safety in primary care is currently available—the primary care patient measure of safety (PC PMOS).20 22 The PC PMOS aims to enhance or complement current data collection methods for patient safety in primary care.20 22 This self-administered tool is an acceptable, efficient and appropriate mechanism for engaging patients in safety improvement.11 13 17 23 The PC PMOS also facilitates primary care professionals and organisations learning, and drives implementation of real-time service improvements.20 21
The implementation and impact of interventions which use the PC PMOS tool for data-driven improvement and ongoing safety monitoring in primary care remains unexplored. Primary care, like most healthcare settings, is a complex system with multiple and multilevel factors likely to affect implementation of a patient feedback for safety improvement intervention.24 While common barriers and enablers to implementation of quality and safety improvement interventions have been published,25–27 the specific processes and outcomes of using the PC PMOS in a primary care safety improvement intervention is unknown. Advocates for complexity science and implementation science in healthcare-improvement-research recommend feasibility studies be conducted prior to the introduction of large-scale effectiveness trials or wider spread into policy and practice.24 25 28–30 Therefore, the aim of this study was to understand the acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data in a primary care patient feedback on safety intervention.
Methods
A detailed description of the study design and sampling frame, intervention, and primary and secondary outcome measures has been published in the study protocol.21 A brief overview is provided below.
Study design and sampling frame
This was a mixed-methods feasibility trial with six purposively sampled primary care practices from the southwest region of Victoria, Australia (online supplementary appendix 1).
Supplemental material
Intervention
Intervention tool: PC PMOS
The PC PMOS tool is an anonymous 28 item survey covering nine latent conditions in the primary care environment influencing safety incidents including: access to care, communication, the external policy environment, information flow, organisation and care planning, patient-related factors, the physical environment, referral systems and task performance (available on request).20 22 The PC PMOS consists of a five-point Likert scale with higher scores indicating safer primary care. The PC PMOS also captures patient-reported safety incident data.
Intervention phases
The intervention comprised an iterative process with a cycle of measurement, learning, feedback, action planning and implementation period of 6 months (figure 1).
Intervention phases.
Patient feedback about the safety of their care was measured using the PC PMOS tool at baseline (time 1-T1). Primary care teams then used patient feedback from the PC PMOS to develop and implement specific safety interventions over a 6-month period. Patient feedback about the safety of their care was measured again (PC PMOS) at the end of the intervention period (time 2-T2).
Primary care practices were asked to form safety improvement teams (SIT). These teams comprised a minimum of three members and included any combination of practice manager, practice nurse, receptionist or administration staff or general practitioner.
SIT members participated in two learning and development workshops on teamwork, communication, implementation planning, the model for improvement’s (MfI) Plan-Do-Study-Act (PDSA) methodology31 and trial information.
PC PMOS data from each practice were collated and presented to the SIT at an action planning meeting. SIT members considered which area(s) of safety improvement to target, and developed goals, measures, ideas and PDSA cycles. SIT members were responsible for implementing and monitoring their specific safety intervention/s through application of multiple PDSA cycles over the 6-month period.
Data collection
Primary outcome
Feasibility measures included acceptability, intervention fidelity, implementation enablers and barriers, and scalability. These data were collected using three qualitative methods:
Recordings and overt observations of SIT members at workshops and action planning meetings,
Semistructured interviews with SIT members at trial conclusion,
Reflexive researcher journaling.
Audio data were transcribed verbatim. Overt participant observation data were recorded using detailed field note diaries and regular researcher discussion and reflection. Approximately 31 hours of audio was recorded with participants at workshops (2×3 hours), action planning meetings (6×1.5 hours) and semistructured interviews (16 hours—13 discrete individual or group interviews).
Secondary outcomes
Patient feedback on contributing factors to safety
Every adult (≥18 years) presenting for their appointment was invited by the practice receptionist to complete the PC PMOS over a 3-week period. Patients returned their surveys via a secure survey return box in the practice waiting room. Surveys were anonymous and completion was voluntary.
Patient-reported safety incidents and concerns
The PC PMOS contains questions for patients to report any patient safety incident. Questions were adapted from the ‘Patient Incident Reporting Tool’ used in the Patient Reporting and Action for a Safe Environment intervention.32 The PC PMOS has an ‘other comments’ free-text question which also provides patients the opportunity to report safety incidents or concerns.
Staff safety culture
The validated Agency for Healthcare Research and Quality Medical Office Survey (MOS) on patient safety33 was used to obtain data about staff safety culture perceptions at baseline (T1) (prior to patient data collection) and after the intervention (T2). All staff were invited to complete the survey and return it to the researcher via a provided prepaid envelope. Surveys were anonymous and completion was voluntary.
Safety incident reports
Practice managers provided a deidentified copy of their practice’s clinical risk management/safety incident register from the previous 12 months at T1 and T2. Due to lack of detailed data provided on the register, specific analysis or categorisation of the safety incidents was unable to be performed. However, the type of incident and any patient demographic data (age, gender) were cross checked with the patient-reported safety incidents on the PC PMOS to assess for similarities or differences.
Data analysis
Primary outcome
Triangulation and thematic analysis techniques were employed to analyse the qualitative and content data. Both inductive and deductive approaches were used to undertake the analysis.34 Deductive approaches used the literature about healthcare culture and safety improvement, patient feedback and response theory, health service implementation science, and engagement and adaption theory.25–27 35 36 Inductive coding was also performed on qualitative and content data by three researchers (ALH, SJG and HB). The initial coding framework centred on the feasibility measures of intervention enablers, barriers, acceptability, fidelity and scalability. This framework was expanded through constant comparison with the data to create the final coding framework. Discrepancies between researchers were resolved through discussion. NVivo (QSR International) was used to support the analysis.
Intervention fidelity score
Intervention fidelity refers to the implementation of safety improvement interventions being delivered as intended.37 The number of safety interventions implemented at each practice was assessed by the research team using a three choice response option—yes, no or partially.
Secondary outcomes
Quantitative data were analysed using SPSS Statistics (IBM V.24). Continuous variables were compared preintervention and postintervention using t-tests, while comparisons for non-parametric data used the Mann-Whitney U test. Categorical variables were compared using X2-tests. Results were considered statistically significant where p≤0.05.
The MOS percent positive scores for each 10 patient safety culture composites, the average score across the 10 composites, and the overall patient safety rating were calculated at T1 and T2 for each practice, and overall using t-tests.
Patient and public involvement
Patients directly participated in the priority setting of safety interventions at a local level. Specifically, patients concerns or experiences with systems for safety in the primary care environment (eg, access to care, communication, information and referral processes, organisation and care planning) were acted on by primary care teams through development and implementation of interventions which prevent safety incidents from occurring.
Informed consent
Patient consent was implied by completion and return of the PC PMOS questionnaire. This was stated on the plain language statement accompanying the PC PMOS questionnaire. All staff who participated in a semistructured interview with the researchers provided written consent to participate. Each practice manager provided written practice consent for the research to be undertaken at their practice.
Results
Primary outcome
Representative participant quotes corresponding to feasibility measures are presented in table 1.
Key participant quotes corresponding to feasibility measures
Acceptability
Intervention acceptability
The majority of staff found the intervention acceptable. Staff reported that the intervention was predominantly positive and fitted within current organisational approaches to quality improvement.
Attitude towards patient feedback on safety
All staff valued patient feedback on safety. Positive feedback was welcomed and viewed as contributing to workplace morale, job satisfaction and reassurance that staff were meeting patient expectations. Feedback on safety was accepted when it aligned with staff awareness of issues. Furthermore, staff acceptance of the patient’s reality also influenced believability of the feedback.
Staff exhibited a range of responses to negative patient feedback, including: acceptance, feelings of empathy, surprise or uncertainty, or being dismissive of feedback.
Commonly mentioned reasons for dismissing patient feedback involved unrealistic patient expectations; deeming patient concerns too problematic to fix or out of the practice’s control; previous attempts to solve the problem have failed; or the patient was a known difficult patient (some staff speculated who a patient was even though the survey is anonymous).
Using patient feedback to make changes
Some staff were cautious about using the patient feedback for safety improvement activity. They contextualised the feedback in terms of where it may be coming from and how appropriate it would be to respond. Additionally, some mentioned difficulties in choosing priority areas to address due to largely positive patient scores limiting what they could respond to.
Four of the six practice teams saw this trial as a catalyst for undertaking improvements that aligned with previously identified staff priorities, and not responding directly to the patient feedback. Two practice teams attempted to link their chosen safety interventions back to domains of safety on the PC PMOS. For example, improving waiting time or availability of appointments was a focus area for staff yet the PC PMOS scores relating to access to care were largely positive. The other two practices did not attempt to link their previously identified target area to a PC PMOS domain of safety. The remaining two practices chose to address areas that were directly related to areas of concern highlighted from the patient feedback. This was either a patient-reported safety incident or a negatively scored PC PMOS domain.
Implementation of safety interventions
Intervention fidelity
The average intervention duration of 5.8 months was considered acceptable by most practice teams. Among the six practices, 25 safety improvement interventions were developed at the action planning meeting or during the implementation period. Of these, 17 (68%) were fully implemented, 2 (8%) partially implemented and 6 (24%) not implemented.
The safety priorities targeted at the six practices included improvement in the following areas: communication of patient recall and reminders, access to equipment and supplies, access to care, accuracy of patient information, management of staff time, patient experience of waiting time, and patient knowledge of registrar skills and abilities. There were no differences observed in success of interventions that addressed either relational (communication, behavioural change, etc) or transactional issues (data cleaning, equipment and supplies, etc). Other mediating and contextual factors in the practice environment were attributed to the success or failure of safety interventions by staff.
Barriers and enablers to intervention development and implementation
Developing interventions
Staff employed both intuition and problem-solving processes to develop safety interventions. This process appeared to be an enabler for practice teams. This often took the form of a rapid and informal root cause analysis where common sense and a pragmatic approach was apparent. This process did not require external facilitation and staff were easily able to identify latent conditions in the practice that contributed to the safety concern. Staff reported regular use of this approach for safety and quality improvement activities unrelated to this project, but had not recognised it as formal improvement work.
Some teams experienced challenges with translating their intuitive problem-solving approach onto the MfI framework. There was a perceived disconnect between the two problem-solving methods. This mainly related to adjusting to new habits or ways of working and adhering to a structured process. Practice teams with greater quality improvement experience were better able to integrate these approaches and adapt accordingly.
Implementing interventions
The high intervention fidelity shown in this trial was attributed to various factors. One key enabler was the multidisciplinary dynamic within the SIT. The teams largely consisted of a practice manager, administration staff member and a practice nurse. GPs adopted a more passive role in implementation. Nonetheless, GPs were engaged and supportive of the SIT and provided leadership and support when needed. Since most SITs comprised a practice manager, administration staff and a practice nurse, it was difficult to make comparisons about the effectiveness of teams that had different combination of staff roles.
Practice managers and administration staff often took primary responsibility and ownership for safety intervention implementation. As the interventions addressed the latent conditions within the primary care system that contribute to safety incidents, the corresponding activities and tasks often required input from administration staff rather than clinical staff. For example, ensuring patient demographic information was up to date or improving appointment scheduling were viewed as tasks to be undertaken by administration staff who are skilled and knowledgeable in this area.
Staff generally agreed that the MfI was a useful and familiar structure for implementing safety interventions. However, a few teams experienced some implementation challenges relating to the prescriptive nature and linear processes proposed in the model. Lack of model flexibility and adaptability were commonly cited as implementation barriers.
Staff also found measuring change difficult for various reasons. Identifying an appropriate measure directly relating to their safety intervention was challenging. For example, some staff indicated it was difficult to measure clinical outcomes or safety incidents averted. Often soft or proxy measures were used due to unavailability or inaccessibility of data.
Staff identified a number of other barriers to implementation. These were common across all practices and included lack of protected time, demanding priorities particularly for patient care, issues with staff recruitment and retention as well as staff leave, power and team dynamics, management support, and engagement from the wider practice.
Scalability
Staff recommended some improvements to the structure and components of the intervention that would enable future scale-up to a larger effectiveness trial or spread into policy and practice (box 1).
Recommendations for intervention improvement
Simplification of intervention framework.
Structured and defined intervention actions plans and corresponding safety measures for each of the primary care patient measure of safety domains of safety.
Electronic data collection platforms to enable real-time patient feedback.
Increased external intervention facilitation.
Modification to questionnaire collecting patient-reported safety incidents.
Existing practice infrastructure and resources were deemed adequate for participation.
The two learning workshops and facilitated action planning meeting with the research team were viewed as important. While the majority of staff felt that this level of facilitation was adequate, others suggested additional action planning meetings throughout the intervention phase would assist with accountability and implementation progress.
Patient data collection using the PC PMOS was considered relatively straightforward by practice staff. Only one practice (Practice A) failed to complete T2 data collection. Reasons for this included staff leave and patient survey fatigue. As the PC PMOS was a paper-based survey staff felt that improvements could centre on electronic data collection to increase the efficiency of real-time patient feedback, for example, via the use of waiting room iPads or emails to patients after their consultation.
Secondary outcomes
Patient feedback on contributing factors to safety: PC PMOS scores
A total of n=1750 patients completed the PC PMOS at T1 and T2 (n=839 T1, n=911 T2), representing a practice mean of 140 and 182 at T1 and T2, respectively. The crude response rate was 10.7%, however, the average response rate across the practices was 40.6%. Patient characteristics are presented in table 2. Patients completing the PC PMOS were significantly more likely to be older and female (online supplementary appendix 2). Mean age was 56 years (SD 18.2) and mean number of visits to the practice in the previous 12 months was 8 (SD 8.6).
Supplemental material
Patient demographic characteristics
The PC PMOS total mean scores and domain scores for each practice at both times points are presented in table 3. There was a significant increase in total mean PC PMOS score for all practices from T1 to T2 suggesting improved patient safety (4.30 (SD=0.49) to 4.37 (SD=0.47), p=0.002). There were also significant increases in mean scores for all practices from T1 to T2 for the following domains: access to care (4.09 to 4.23, p<0.001), communication (4.44 to 4.50, p=0.018), information flow (4.27 to 4.36, p=0.007) and patient-related factors (4.51 to 4.61, p<0.001). There was within and between practice variation for specific PC PMOS domain scores (table 3).
PC PMOS overall and domain specific scores by practice
Patient-reported safety incidents and concerns data
Patient-reported safety incident data are presented in table 4. There were n=11 patient-reported safety incidents at T1, and n=9 at T2. The mean severity rating at T1 and T2 was 7.4 (scale 1–10 with 10 being ‘extremely serious’). The median preventability rating of these safety incidents was ‘definitely preventable’ at T1, and ‘probably preventable’ at T2.
Frequency, preventability and severity of patient-reported incidents and concerns
An additional n=17 safety incidents at T1, and n=12 at T2 were identified from the ‘other comments’ section of the PC PMOS. Therefore, the total number of patient-reported safety incidents was n=28 at T1, and n=21 at T2. The number of patient-reported concerns (negative comments that were not a patient safety incident) decreased from n=45 at T1 to n=25 at T2 (table 4).
Practice measures of safety
Staff perceptions of safety culture
A total of n=57 staff completed the MOS survey at T1, and n=61 at T2. For the total sample there was an increase in the mean percent positive score for the overall patient safety rating between T1 and T2, although not significant (72% to 74%, p=0.851). For the majority of the patient safety culture composites and the average across the ten composites there was a reduction in mean percent positive scores, with only one significant reduction for the Teamwork composite between T1 and T2 (89%–80%, p=0.029) (online supplementary appendix 3).
Supplemental material
Safety incidents recorded on practice clinical risk management system
There was a reduction in the number of incidents recorded on practice’s clinical risk management system from T1 (n=32) to T2 (n=21) (online supplementary appendix 4). The incidents recorded on the practice clinical risk management system were different to the incidents reported by patients on the PC PMOS.
Supplemental material
Discussion
This is the first reported patient feedback on safety intervention in Australian primary care. The findings indicate that the intervention is feasible for scale-up to a larger effectiveness trial and further spread into policy and practice. Staff deemed the intervention acceptable, with minor recommendations for improvement. Intervention fidelity was high and implementation enablers were attributed to the intervention structure and framework, use of intuitive problem-solving approaches and multidisciplinary team involvement. Barriers to implementation reflected previously reported problems undertaking quality improvement in primary care, such as lack of time and staff, demanding priorities, power and team dynamics, and wider practice support and engagement.25–27 The process of systematically collecting patient safety data was achievable with n=1750 patient surveys completed. The utility of the PC PMOS tool as a measure for safety was demonstrated through the significant increase in mean scores for all practices from T1 to T2 (4.30 to 4.37, p=0.002).
It is widely acknowledged that patient feedback is rarely used for safety and quality improvement purposes.38–46 This study identified some enablers and barriers that impacted on the intervention development and implementation including the team dynamic, improvement framework and staff attitude.
A unique aspect of this patient feedback on safety intervention was the multidisciplinary dynamic of the primary care teams, particularly administration staff leadership. This was considered a key enabler to intervention adherence and acceptability. The safety interventions targeted the contributing factors to safety incidents; as such, administration staff were ideally placed for intervention delivery. Administration staff transcended professional boundaries to generate engagement and support, and implement changes at the latent end of the primary care system. In this respect administration staff acted as change agents and innovators35 47 48 and future safety improvement work should consider their current underused role.
Although the Mfi’s PDSA cycle31 is considered an effective, adaptable and flexible framework for quality improvement in some contexts, practice staff in this study identified it as a barrier to implementation. Formalising and documenting action plans in PDSA cycles was often in disconnect to their natural problem-solving approach and routine practice. In a time, resource and capacity scarce environment it is important that safety improvement frameworks are simple, and easily integrate or mimic everyday work flow. There are several well established quality improvement models49–51 that could be utilised for this patient feedback on safety intervention, however more research is needed to identify and investigate staff acceptability and appropriateness of the different frameworks in this context.52
Staff attitude towards patient feedback on safety was similar to previous research, which reveals staff difficulty to engage with or value patient feedback.36 38 41 42 45 53–57 While staff described the value and benefit of seeking patient feedback on safety, this was not entirely reflected in action plans or translated during intervention implementation. More than half of the practice teams undertook safety interventions that were a priority for staff rather than a priority for the patient. Recommendations to improve staff action on patient feedback could centre on providing staff with structured and specific intervention examples that correspond to particular domains of safety on the PC PMOS. Moreover, such intervention examples could have explicitly linked measures of safety to each of the PC PMOS domains which may address the challenges staff experienced with creating measures of change.58
The process of systematically collecting primary care safety data from the practice, staff and patients was acceptable and feasible, yet some consideration is needed when determining appropriate measures of intervention effectiveness in a larger trial. Data about patient safety in primary care in Australia is largely absent. Australia does not have a structured or connected reporting and learning system to understand the threats to patient safety, and there is no current systematic way to collect information about safety incidents or patient harm.59 60 Using available sources of patient safety data in this study revealed some limitations; as such, objective measures of intervention effectiveness like statistical control charts,61 62 PDSA cycle evaluation tools,63 64 and record review65 66 are recommended.
The findings from this study support results from other studies which have investigated patient feedback for safety improvement. While the majority of the research centres on hospital settings, the positive effect of patient feedback has been determined.3 8–12 One systematic review identified gaps in understanding regarding the enablers and barriers for implementation of patient feedback interventions.8 The findings from this study add to the discourse in this under-researched area.
A limitation of this study was the sample. The practices were from one regional area, which may limit the generalisability of the findings. However, the diversity within the practices was considered adequate for this feasibility study. All practices had participated in one or more of the Australian Primary Care Collaborative Programme67 waves previously. Their commitment, interest and understanding of safety and quality improvement processes was potentially already elevated prior to study commencement when compared with other practices. Results suggest the merit of conducting a larger scale effectiveness-implementation trial to determine the translatability of this intervention programme and safety outcomes to primary care practices more generally.
Conclusion
This study’s findings have demonstrated the feasibility of introducing an innovative patient feedback for safety improvement intervention in primary care, as well as contextual and intervention factors that promote safety improvement. The intervention complements existing safety improvement strategies and activities, and integrates into current patient feedback service requirements for primary care. Further research is needed to examine the intervention effects on safety incident reduction.
Acknowledgments
The authors acknowledge the practice staff and patients who participated in this study. The authors would also like to acknowledge the administrative support from Gillian Beard and Liz Jackway.
References
Footnotes
Twitter @andreahernan11
Contributors ALH conceived and designed the study, and was principal investigator for the study. SJG, KM, KK and VV contributed to the study design. ALH, SJG and HB were responsible for the qualitative data analysis and manuscript preparation. VV was responsible for the quantitative data analysis and manuscript preparation. ALH created the first draft of the manuscript and was responsible for its revisions. SJG, HB, KM, KK, MJB and VV contributed to specific sections of the manuscript. All authors read and approved the final version of the manuscript.
Funding This work was supported by the Western Alliance Academic Health Science Centre (grant number WA-733721).
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Ethics approval Ethics approval was obtained from Deakin University Human Ethics Advisory Group, Faculty of Health. Project number: HEAG-H 175_2017.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. Data are available on request to the authors.