Data-driven quality improvement program to prevent hospitalisation and improve care of people living with coronary heart disease: Protocol for a process evaluation

Background Practice-level quality improvement initiatives using rapidly advancing technology offers a multidimensional approach to reduce cardiovascular disease burden. For the “QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease” (QUEL) cluster randomised controlled trial, a 12-month quality improvement intervention was designed for primary care practices to use data and implement progressive changes using “Plan, Do, Study, Act” cycles within their practices with training in a series of interactive workshops. This protocol aims to describe the systematic methods to conduct a process evaluation of the data-driven intervention within the QUEL study. Methods A mixed-method approach will be used to conduct the evaluation. Quantitative data collected throughout the intervention period, via surveys and intervention materials, will be used to (1) identify the key elements of the intervention and how, for whom and in what context it was effective; (2) determine if the intervention is delivered as intended; and (3) describe practice engagement, commitment and capacity associated with various intervention components. Qualitative data, collected via semi-structured interviews and open-ended questions, will be used to gather in-depth understanding of the (1) satisfaction, utility, barriers and enablers; (2) acceptability, uptake and feasibility, and (3) effect of the COVID-19 pandemic on the implementation of the intervention. Conclusion Findings from the evaluation will provide new knowledge on the implementation of a complex, multi-component intervention at practice-level using their own electronic patient data to enhance secondary prevention of cardiovascular disease. Trial registration: Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

Background: Practice-level quality improvement initiatives using rapidly advancing technology offers a multidimensional approach to reduce cardiovascular disease burden.For the "QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease" (QUEL) cluster randomised controlled trial, a 12-month quality improvement intervention was designed for primary care practices to use data and implement progressive changes using "Plan, Do, Study, Act" cycles within their practices with training in a series of interactive workshops.This protocol aims to describe the systematic methods to conduct a process evaluation of the data-driven intervention within the QUEL study.Methods: A mixed-method approach will be used to conduct the evaluation.Quantitative data collected throughout the intervention period, via surveys and intervention materials, will be used to (1) identify the key elements of the intervention and how, for whom and in what context it was effective; (2) determine if the intervention is delivered as intended; and (3) describe practice engagement, commitment and capacity associated with various intervention components.Qualitative data, collected via semi-structured interviews and open-ended questions, will be used to gather in-depth understanding of the (1) satisfaction, utility, barriers and enablers; (2) acceptability, uptake and feasibility, and (3) effect of the COVID-19 pandemic on the implementation of the intervention.

Conclusion:
Findings from the evaluation will provide new knowledge on the implementation of a complex, multicomponent intervention at practice-level using their own electronic patient data to enhance secondary prevention of cardiovascular disease.Trial registration: Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

Background
Cardiovascular disease (CVD) including coronary heart disease (CHD) and stroke remains the leading cause of death and disease burden worldwide despite decades of significant advances in the prevention and management of CVD [1,2]..Globally an estimated 17.8 million people die every year from CVD constituting approximately one-third of global deaths [2,3].The burden of CVD continues to contribute heavily towards the global economic burden due to the associated direct and indirect effects including hospitalisations, medications, post-discharge primary care management, rehabilitation services, disability, and unemployment [4].As a result, the global cost of CVD is predicted to rise from US$863 billion in 2010 to US$1044 billion by 2030 [5,6].With the aging population and more people surviving initial cardiac events, the prevalence of CVD is increasing along with the economic cost [7].To reduce the risk of future cardiovascular events in those with established disease, secondary prevention strategies have become an international priority [8,9] and include the use of guideline-indicated medications, adopting a healthy lifestyle, implementation of chronic disease management plans and participation in a cardiac rehabilitation program following an acute event [8].Primary care plays an integral role in implementing successful secondary prevention strategies as the majority of people hospitalised for CVD regularly visit their primary care practitioners and use government-funded health services at least once a year following their acute CHD diagnosis [10][11][12][13].
Funded by the federal government, primary care is the first point of contact for all Australians to access care.Under the primary care system, individuals can receive services that includes treatment of acute conditions, chronic disease management, health promotion, prevention and early intervention [14].These services are provided via the general practices, community health centres and allied health practices with the help of Primary Health Networks (PHNs) [14,15].The Australian government has recently launched the Quality Improvement Practice Incentive Program (QI-PIP), which encourages primary care practices to collaborate with their PHNs and undertake quality improvement activities within their practices to provide high-quality patient care for better health outcomes [16].As a result, many primary care practices worldwide are rapidly adopting the use of quality improvement (QI) initiatives [17][18][19].QI initiatives offer an innovative, multidimensional approach to healthcare and have excellent potential to improve patient outcomes in primary care [20].Also, current technology has enabled the integration of automated data extraction leading primary care practices to consider data-driven QI programs to provide high-quality patient care [21,22].Primary care practices have been successfully implementing QI programs in several health conditions including asthma [23], diabetes, neonatal health [24,25].However, there is a paucity of research focused on evaluating the effectiveness of such QI interventions only at individual patient level rather than community or clinic level in CVD management [26,27]..The "QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease" (QUEL) study aimed to determine the effectiveness of a quality improvement program for improving CVD management [28].
For the QUEL cluster randomised controlled trial (cRCT), a structured QI program is delivered within Australian primary care practices to reduce CVD hospitalisations, improve CVD risk factors and medication adherence in patients with CHD over 24 months [28].The intervention practices are supported by the study team or their relevant PHNs to enhance efficiency in management and outcomes of CVD patients by better using their routinely collected data.The trial is ongoing involving 52 (27 intervention and 25 control) Australian primary care practices with approximately 15,000 CVD patients with 12-and 24-month followup with data collection scheduled for completion in mid-2022.The primary outcome is CVD hospitalisations, collected via linkage with state-based administrative data linkage centres that collect data on all hospitalisations in Australian hospitals and as such will not be adjudicated, and secondary outcomes are cardiovascular risk factors recorded electronically by the GPs in real time which will be collected routinely across all participating primary care practices using a standardised data extraction software; medication prescriptions and use collected by data linkage of the QUEL cohort with federal level Medicare Benefits Schedule (MBS) and Pharmaceutical Benefits Scheme (PBS) data; and deaths collected via linkage of the same cohort with the federal level National Death Index.Specific details on the cRCT including trial aims, design, sample size, and outcome measures are described elsewhere [28].
The QUEL intervention is based on the Collaborative Framework [29] and consists of (1) a virtual orientation session, (2) electronic data collection at baseline, and thereafter monthly, from the intervention practices via a practice-level software system that enables automated data extraction [30], (3) monthly data reporting, (4) completion of Plan, Do, Study, Act (PDSA) cycles that summarise practice-level progress towards pre-determined CVD indicators for the QUEL study, (5) a series of interactive learning and benchmarking workshops (1 in-person and 5 virtual sessions due to the COVID-19 pandemic) and ( 6) provision of support from PHNs and the study team.The intervention is delivered over a period of 12 months.
The study team is collaborating with five PHNs to ensure optimal delivery of the collaborative intervention.PHNs are independent organisations funded by the Australian government aimed to coordinate health services for the communities in a specific region [15].PHNs also work closely with the primary care practices and other health care professionals within the region to identify gaps and build capacity to ensure optimal service delivery [15].There are thirty-one PHNs operating in Australia including in remote and Aboriginal Torres Strait Islanders communities to encourage use of available health resources and access health care [15].All PHNs were invited to participate through a variety of communication channels, including a mailing list direct to PHN CEOs, University of Sydney's as well as research partners' networks.Five out of the thirty-one PHNs agreed to collaborate on the study based on their previous experience in QI collaborative and existing collaboration with the University of Sydney.For the QUEL study, each collaborating PHN nominates a primary contact to provide liaison, leadership and coordination to the participating practices within the PHN's jurisdiction.During the intervention period, PHN representatives play a key role in ensuring successful implementation of the intervention.The role of the PHN involves, but is not limited to, supporting practices to achieve pre-defined key performance measures to optimise outcomes, participating in program activity including training and learning workshops, encouraging practice level engagement in these activities and using PDSA cycles between activity periods, sharing practice achievement and providing additional support as required.
Process evaluation is particularly important in complex intervention trials as it provides in-depth information required to evaluate the intervention's effectiveness and investigate the implementation process.It provides valuable insights into describing the various intervention components [31] and identifying factors associated with successes and challenges of the programs in various healthcare settings [32].Use of process evaluation alongside complex interventions is increasing given because of the associated multisite, multicomponent features [33,34].However, little research has reported the mechanisms of impact, context and what constitutes effective QI interventions aimed at improving CVD management in primary care settings.
The QUEL QI program is a complex intervention with multiple interactive components, as such, process evaluation can accurately describe the intervention implementation, exposure of the intended intervention and real-time experiences of those involved [35].We hypothesise that evaluating the implementation of the multi-component QI intervention within the QUEL trial will help primary care practices to undertake further QI activities to improve care of CVD within their practices.The earlier protocol describes the cluster RCT itself [28], while this current protocol details the evaluation plan for the datadriven QI intervention program within the QUEL cRCT and its effects.The process evaluation aims to:

Study design
A mixed-methods approach will be undertaken using data from 27 intervention practices (out of 52 participating practices) from the QUEL cRCT [36,37].For this study, data will be collected only from the intervention practices as it aims to evaluate the effect of the QI intervention program.Qualitative and quantitative data will be collected both during and at the end of the trial intervention period.Semistructured interviews and open-ended questions will be used to collect qualitative data.Quantitative data will be collected from the intervention practices via multiple data sources throughout the intervention period.
A program logic model was developed to describe how, why and among whom the collaborative intervention works in practices within the QUEL cRCT (Fig. 1).This logic model is a visual representation of the intervention design and its intended implementation.The Cochrane Effective Practice and Organisation of Care Review Group (EPOC) checklist was used as a guidance to develop the logic model [38] to identify the key features of the intervention, check the fidelity of the implementation and assess participant's experience [31].The model includes 5 domains of the intervention: (1) input, (2) activities, (3) outputs, (4) outcome and ( 5) impact specific to the data-driven QI program that will be used to describe the study objectives [39].Inputs refer to various resources that are required to ensure program operation, activities refer to the planned actions, such as delivery of workshops, data collection that are an essential part of the implementation [39].Resources, inputs, and activities together form the program design.Outputs include the changes in the participant's behaviour, knowledge, skills, and awareness resulting from the activities and impact describes the fundamental changes occurring in the health services over a longer period as a result of the program activities [40].

Participants
Participants in the process evaluation will include practice team members (including general practitioners, nurses and practice managers) from primary care practices allocated to the intervention arm and PHN staff who are providing direct support to the intervention practices under their jurisdiction.At least two practice team members from all 27 intervention practices who were actively involved in QI activities in their practices; such as participated in QI workshops, submitted and carried out PDSA cycles and regularly communicated and shared reports with the study team on their activities during the intervention period will be approached to complete the surveys and participate in semistructured interviews.These participants will be able to understand sufficient English to provide written and informed consent.Practice team members from primary care practices allocated to the control group and any PHN staff not involved in the QUEL project will not be included in the process evaluation.All practice team members who are part of the intervention practices will be approached for recruitment to provide feedback and participate in interviews during and at the end of the intervention.

Data sources
Multiple data sources, collected throughout the cRCT, will be used in addition to surveys and interview data, to evaluate whether the complex intervention was delivered as planned.Combining these data sources will help to identify the key intervention elements, identify the dose, frequency and activities delivered to the intervention practices as well as describe barriers and enablers associated with the program implementation.To maintain balanced quality of information across the multiple data sources the research team will ensure a) close communications and interaction with practices and PHNs (e.g.workshops; practice visits, regular contact via email or phone calls) to promote quality of data collection; b) an experienced research officer is responsible for data collection throughout the study; c) all the practices are well informed on the study procedures before they are enrolled; d) the participating practices receive appropriate research support when required; e) routine extraction, monitor and check data for quality assurance and help practices solve issues if data is not returned.The data sources will include: 1) practice-level enrolment data, 2) attendance record, 3) SharePoint data, 4) practice correspondence record, 5) data collection record, 6) PDSA cycles, 7) learning workshop surveys, 8) end of program survey and 9) semi-structured interviews of practice team members and PHN representatives.These data sources will be used as credible evidence collected at different time point during the intervention period (Fig. 1).

Practice-level enrolment data
Practice-level enrolment data will be created at the time of recruitment and will be recorded in a Microsoft Excel (2016) spreadsheet.Information collected will include practice location (urban and rural), practice team members information, software compatibility, and  randomisation group.Urban and rural primary care practices were defined using the Australian department of Health's Health workforce classification guideline [41].This spreadsheet will be used throughout the main trial period and updated regularly with current dates and version numbers.We will use these data to identify intervention practices, software eligibility and installation requirement of the eligible software, practice support, describe practice type and gather details of the practice team members involved in the delivery of the intervention.

Attendance records
Participation of the practice team members in any events related to the intervention including orientation and learning workshops (both face-to-face and virtual) will be recorded in another Microsoft Excel (2016) spreadsheet and updated regularly throughout the intervention period.These data will provide information on the frequency of the workshops attended by the intervention practices, the number of staff from each practice attending the orientation and workshops.

SharePoint data
Microsoft SharePoint [42] is an online platform where a unique account is created for individual practices in both intervention and control arms.This platform is created for the practices to submit their PDSA cycle records and track improvements via monthly graphs which are uploaded in their respective accounts by the study team.From the SharePoint data, we will identify whether each practice had access to their account and all intervention materials including workshop recordings and lectures, monthly feedback reports, frequency and number of PDSA cycles submitted by each practice.

Practice correspondence record
The study team will be communicating with the intervention practices during the trial period and practices will also be encouraged to directly communicate with the study team as required.These communications will be undertaken via phone call, email or in-person site visits.Any communication will be saved and used to identify the reason, mode of contact (email, site visit or phone call), time spent on the contact, person contacted and solution provided in a Microsoft Excel (2016) spreadsheet.This document will be updated throughout the study period with current dates and version numbers.

Data collection record
The intervention practices will submit clinical data electronically in an aggregated and de-identified form monthly via the automated data extraction software [30].All aggregated data will be stored in the University's Research Data Storage (RDS).These data will be used to create practice level reports and will be uploaded to practices' SharePoint sites monthly as graphs for benchmarking their improvement for the predefined QUEL study performance measures.A Microsoft Excel (2016) spreadsheet will be used to record monthly data collection and reporting for each intervention practice.This spreadsheet will also be updated regularly throughout the intervention period with current dates and version numbers.

Plan, Do, Study, Act cycles
The PDSA cycle is a simple but powerful tool to measure improvements and increasingly used in many QI collaborative to boost quality of healthcare [43,44].It guides users to explicitly plan, implement, reflect on, and then repeat, incremental improvements as they make system changes to achieve the aim [45].Practices participating in the QUEL QI intervention are required to document and upload their PDSA cycles using a template.Training will be provided to the practices during the learning workshops on the process of completing PDSAs.Submitted PDSA cycles will be saved in their respective SharePoint accounts and the study team will be able to download a copy of the cycle when required.We will use all the PDSA cycles submitted by primary care practices during the intervention period to gather information on practice engagement, number of PDSAs submitted by each practice, identify key areas practices focused on improving and identify barriers and enablers to make improvement changes within the practices.

Learning workshop surveys
At least two practice team members (one clinical and one administrative) will be invited to participate in a series of six learning workshops that will be delivered during the intervention period.Six surveys corresponding to six workshops will be administered at the end of each workshop (paper-based for in-person workshops and online for the virtual workshops) to the workshop attendees.Each survey will contain questions that are specific to the workshop content and a set of common questions that will be asked at every workshop.The common questions include feedback on the workshop evaluation, learnings, satisfaction and suggestions for improvement collected as Likert scale and free-text response The surveys aim to evaluate practice engagement, workshop attendance, time commitment, staff skills and capacity involved in implementing QI changes in their practice.The survey will also be used to evaluate the appropriateness of content and the effectiveness in terms of practice-level implementation of QI.

End of program surveys
Practice team members including general practitioners, practice managers or nurses from QUEL study intervention practices who are actively involved in implementing QI changes within their practices will be invited to complete a comprehensive survey on the overall program at the end of the intervention.To ensure as many responses as possible, the survey will be sent by post, with a return address envelope, email, online or by direct contact.The survey aims to evaluate the whole intervention and examine acceptability, satisfaction, uptake, utility and feasibility among users.The survey will include fifty-five questions, of which forty questions require Likert scale responses focused on overall workshop content, design, facilitators, results and outcome; practice software usability, use of electronic data for the management of CVD patients, quality and satisfaction of care provided by the primary care team, impact of the intervention on the quality of care provided, leadership involvement and staff capacity.Six questions will require yes or no responses with possible further explanation which focuses on QI-PIP [16], SharePoint use and access.Nine questions will allow free text responses focused on sharing experience implementing changes within the practice, change in staff role, sharing feedback on different intervention components.These free text questions will also include questions and discussion points on the effect of COVID-19, which will provide detailed information to help us evaluate its effect on the intervention implementation.

Interviews with practice team members and PHN representatives
Practice team members participating in the QUEL study will be invited by email, telephone, or post to take part in a confidential one-onone interview at the end of the intervention.Practices will be selected based on their performance (high, low and medium); which will be defined by the practices' interaction during the intervention period such as participation on the learning workshops, submission of PDSA cycles.We will invite practice team members from at least three practices from each high, medium and low performing tier to participate in the interviews to ensure minimum bias.The purpose of these interviews is to evaluate workforce capability, describe perceived benefits, barriers or successes to implementation, uptake, and acceptability of the program.We will also be able to explore the differences in the dose, frequency and the activities implemented by the practices which were supported by the PHNs vs the study team.The interviews will also expand on themes within the surveys to triangulate these data; explore their experiences with QI strategies, gain detailed insight into the staff involvement, changes that occurred in the staff role due to the program.Interviews will be also used to explore the capacity of the practice software and the use of electronic data extracted from the software for QI program implementation.The semi-structured interviews will also explore how the COVID-19 affected the implementation of the intervention and will provide us with information on different approaches taken by the practices to overcome the challenges.
Interviewing the PHN representatives will enable us to describe the role of PHN in implementing QI intervention in primary care.With the interviews, we will obtain in detail PHN's perspective on the program, barriers and enablers to implementation, practice engagement, time commitment and efforts required by PHN representatives.
Semi-structured interviews are widely used in healthcare research to collect open-ended, qualitative data and to explore in-depth understanding of a specific topic [46,47].For the evaluation, we estimate a sample of approximately 10 interviews from different suburban locations, including a variety of practice team members such as GPs, nurses and practice managers and PHN representatives reflecting diverse participant demography.However, the final number of interviews will be dependent on the thematic saturation.A trained researcher will conduct and audio record the interviews of approximately 45 min duration at the practice or health service, or via telephone, as convenient for the participants [48].A topic-centred discussion guide will be used by the interviewer to conduct the semi-structured interviews to ensure the topics are systematically explored [48].The researcher may take notes during the interview to document relevant information.

Data analysis
Descriptive statistical analysis will be used to analyse quantitative data.Responses and measurements from all data sources will be presented as numbers and percentages for categorical variables and mean and standard deviation or median and interquartile intervals for continuous variables.This will help to understand the level of satisfaction and perceived utility associated with program implementation and explain the extent to which the intervention is delivered as intended.The quantitative data will be compared between the following subgroups: rural vs urban and small (≤2 GPs) and large primary (>2 GPs) care practices.Chi-squared test or Fisher's exact test will be used to compare categorical variables between the subgroups and independent t-test or Wilcoxon rank-sum test will be used to compare continuous variables.
Qualitative data, including semi-structured interviews and free-text responses from the surveys and PDSA cycles, will undergo thematic analysis [49].The thematic analysis will include preparing and transcribing the data, familiarising and coding, generating, reviewing and defining the themes and writing up the interpretation of the data [49,50].Two independent researchers will thematically analyse interview transcripts.All data collected will be converted into electronic format and stored in one location.Interviews will be recorded with consent and verbatim method will be used for transcription of the interviews.Interview transcript, any free text and notes from the interviews as well as the surveys and PDSAs will be coded and managed in NVivo Software.

Data storage, retention and disposal
All data collected for the process evaluation including personal information will be securely stored in the University's RDS database.Access to the RDS will require an employee unikey and password and only a limited number of people will have access to it.All data will be stored on The University's RDS for the duration necessary to comply with regulatory requirements; thereafter database will be destroyed in accordance with University's IT recommendations.Completed surveys and interview data will be stored securely for 5 years after publications, after which time, they will be destroyed securely.No personal information will be published.At the end of this data retention period, all files will be physically destroyed.

Discussion
This protocol outlines the systematic methods of a process evaluation of a complex QI intervention embedded within a cRCT to improve secondary prevention of CVD in primary care.The evaluation logic model is described along with methods for understanding the impact of the intervention and the context in which the impact occurs.It will assess successes and failures related to the program implementation in addition to determining factors associated with program scale-up and adaptation for other primary care settings [32].The EPOC framework does not provide information on describing the actual QI intervention, therefore this evaluation also includes interviewing of key participants at completion of the intervention.The study will contribute to stronger evidence around the use of QI in primary care to improve CVD outcomes as well as to literature through encouraging the development of process evaluation methodology in the design and promoting transparency in the reporting of the findings.
A strength of our study is the use of a mixed-methods approach.Mixed-methods research can strengthen data quality, improve interpretations of findings, and offer a more comprehensive understanding of the program implementation, and hence, it has become a very useful tool to evaluate complex interventions [51].Quantitative data will provide key information on what was effective and qualitative data will provide deeper understanding of why and in what context the intervention was effective.Therefore, combination of both will provide a more holistic understanding of the complex intervention than either method alone [52].Furthermore, this approach enables a richer perspective from a range of participants (GPs, nurses, practice managers, and PHN representatives) using various surveys and semi-structured interviews integrated within the main QUEL cRCT.Combining interview and survey data will enable in-depth knowledge on program utility, barriers, and likelihood of adoption.Findings from the process evaluation will also inform other primary care practices to implement data-driven QI programs and provide valuable insights to policymakers on wider adoption and scaling-up of such strategies.
While this process evaluation will enable evaluation of a complex QI intervention and barriers and enablers to its implementation, there are several limitations.One of the limitations is the data collected from the interviews may be subjected to recall bias as interviews will take place after the intervention.Another limitations is the cRCT is designed to be delivered in Australian primary care environment, hence it may only be relevant to health systems with similar contexts, funding, and infrastructures.

Conclusion
At the completion of the evaluation, we will gather rich data about collaborative implementation in terms of key features, impact, barriers and enablers and other factors including collaborative teams, staff resistance, use of experienced resources within the team to train staff specific to improving care of CVD management in primary care.Results from the evaluation will also contribute to further high-quality evidence regarding the implementation of quality improvement programs in primary care.This process evaluation will therefore help identify gaps in implementation and influence practice-level decision making in adopting data-driven quality improvement strategies and to improve CVD management.

Ethics and dissemination
The study is approved by the New South Wales Population & Health Services Research Ethics Committee (HREC/18/CIPHS/44).The ethics committee provides approval for all four participating states including New South Wales, Australian Capital Territory, Victoria, Queensland, and South Australia under the National Mutual Scheme.Participants who are participating in the process evaluation will be provided with a N. Hafiz et al.

N
.Hafiz et al.
Identify and describe barriers and enablers of the QI program. 5. Evaluate the effect of COVID -19 on the implementation of the QI program.