Facilitators and barriers for implementing patient-reported outcome measures in clinical care: An academic center’s initial experience (cid:2)

Objectives The aim of this study was to explore the perspectives of healthcare providers and researchers in a large academic hospital on facilitators and barriers for implementing patient-reported outcome measures (PROMs) in clinical care. Methods A customized web-based questionnaire was developed and disseminated to healthcare providers and researchers across multiple medical departments involved in a value-based health care initiative in the hospital. Questionnaire statements were rated using a 5-point Likert scale ranging from “strongly agree” to “strongly disagree”. In addition, 8 open-ended questions were included allowing respondents to mention additional facilitators and barriers for implementing PROMs. Descriptive statistics were used to summarize the results. Results In total, 61 participants from both surgical and non-surgical departments completed the survey. Most respondents (51%) were medical specialists and the median employment duration was 14 years. Frequently reported facilitators were the presence of a PROM coordinator in the (outpatient) clinic (85%), the integration of PROMs in the electronic health record (81%), and the intrinsic motivation of members involved in the implementation (N = 9 open responses). Commonly reported barriers were language barriers (76%), IT issues (N = 17 open responses), and time constraints (N = 14 open responses). Conclusions For the successful implementation of PROMs in clinical practice, it is imperative that health-care organizations consider supporting motivated healthcare professionals, involving PROMs coordinators, and investing in an adequate IT infrastructure, and removal of language barriers. © 2021 The Author(s). Published by Elsevier B


Introduction
Patient-Reported Outcome Measures (PROMs) have increasingly become a topic of interest in both research and clinical practice [1] , as part of an apparent shift in healthcare towards more valuebased/value-driven systems [2 , 3] .PROMs can be defined as feedback directly from the patient on his or her health status (e.g.wellbeing, symptoms) and/or treatment, without external interpretation by healthcare professionals [4 , 5] .PROMs can serve multiple purposes including monitoring the development of individual patient's symptom burden and quality of life over the course of the disease and/or treatment, facilitating patient-provider communication [6] , enhancing shared decision-making [7][8][9][10] , examining the quality of care [11] , comparative effectiveness research [12] , and their use in value-based payment [13] .
In 2013, a value-based health care (VBHC) strategy was initiated within the Erasmus University Medical Center, a large academic healthcare institution in Rotterdam, the Netherlands.This VBHC initiative included the definition of standardized outcome sets (including both provider-and patient-reported outcomes), and the development of a collection tool to routinely capture these outcomes.Most of these standardized outcome sets, which were developed by the International Consortium for Health Outcome Measurements (ICHOM), encompass both clinical (provider-reported) and patient-reported outcomes.The data collection tool was either linked to the electronic health record (EHR) or accessible through a web-based platform.Since the conclusion of the pilot phase of the initiative (2015-2019), the VBHC-concept has been successfully implemented for several conditions including breast cancer [14][15][16] , cleft lip and palate [17] , stroke [18] , familial hypercholesterolemia [19] , and Turner syndrome [20] .However, there have also been disease teams that did not (yet) succeed in implementing the collection of PROMs on a routine basis.Likewise, many hospitals and other healthcare organizations have had mixed success in implementing routine use of PROMs in a clinical setting with little insight in facilitating and impeding factors [21] .
Research aiming at obtaining a better understanding of healthcare professionals' experience or perspective on facilitators and barriers for implementing PROMs in clinical practice, is likely to be helpful to (other) hospitals or healthcare organizations that intend or are preparing to incorporate PROMs in clinical care.Therefore, the aim of this cross-sectional study is to identify the facilitators and barriers, as perceived by healthcare providers (e.g.physicians, nurses, paramedics) and researchers, to implementing PROMs in clinical care in a large academic medical center.

Study Setting
Since 2013 as part of the aforementioned VBHC initiative, 38 multidisciplinary "disease teams" within the Erasmus University Medical Center have attempted to define PROMs and implement their routine capture (before patient visits) with review and feedback by treating healthcare professionals in the consultation room.The goal of the initiative was ultimately to provide care tailored to "what matters to patients" and to improve patient experiences of care by enhancing patient-provider communication and shared decision-making [15] .Following a pilot phase, ten of these teams (i.e.breast cancer, familial hypercholesterolemia, bladder cancer, stroke, brain tumors, otolaryngology disorders, cleft lip and palate, Turner syndrome, pediatric sickle cell anemia, adult sickle cell anemia) managed to successfully implement the concept in daily practice.The remaining 28 disease teams were in the earlier phases of implementation (see Supplementary Table 1), with some teams failing to complete the entire PROMs implementation process.These teams were intentionally also invited to participate in the current study in order to gain insights into the obstacles they experienced.

Study Design, Web-Based Questionnaire, Participants & Data Collection
According to Foster et al .[21] , the implementation of PROMs can be divided into five stages: 1) Purpose and Intake (objectives and motivations for implementing PROMs) 2) Designing (making decisions on the process for implementing PROMs) 3) Preparing (investing time and resources to get an organization and its care providers ready to implement PROMs) 4) Commencing (the problems that ensue when organizations start to use PROMs) 5) Reflecting and Developing (evaluating the process of PROMs implementation and making adjustments) For each stage, the facilitators and barriers reported in the systematic review by Foster et al .[21] provided the basic structure to develop the current study's questionnaire.
Healthcare professionals involved in the disease teams were invited to complete a voluntary, anonymous, opt-in, web-based questionnaire.Depending on the implementation stage the healthcare professionals were in during the study period, either the full version of the questionnaire or a selection thereof was sent to the respondents (see below).Reminders were sent two weeks after initial distribution.The secure "LimeSurvey" online tool [22] was used to develop the questionnaire, which consisted of the following three sections: 1) Personal characteristics, including age, gender, professional role (e.g.doctor, nurse, researcher, physiotherapist), type of contract (e.g.part-time or full-time), department of employment, length of tenure at the Erasmus University Medical Center, and years of overall working experience within healthcare.
2) Eight open-ended questions on PROMs implementation within routine clinical practice, including perceived (dis)advantages, and perceived or experienced facilitators and barriers (Supplementary Table 2).This part was included in the questionnaire to give participants the opportunity to report their unique experiences that were not (fully) covered by the close-ended statements in the third section of the questionnaire.The openended questions were deliberately placed before the statements so respondents could not be influenced by them.Two questions were specifically related to facilitators and barriers, namely "What did you experience as the biggest factor promoting the implementation and/or use of PROMs in the (outpatient) clinic?" and "What did you experience as the biggest impeding factor when implementing and/or using PROMs in the (outpatient) clinic?".The responses to these two questions were analyzed and described in this study.The responses to the other six questions will not be discussed but are mentioned in Supplementary Table 2. 3) Statements about facilitators and barriers of PROMs implementation, with respondents being asked to rate their agreement on a five-point Likert scale (1 = strongly agree, 5 = strongly disagree).This part of the questionnaire is an adaptation of the previously validated "Barriers and Facilitators Assessment Instrument" (BFAI) developed by Peters et al . in the Netherlands [23][24][25] .The original 27-item BFAI instrument was developed to identify facilitators and barriers for implementation of preventive care (11 items) and innovations/guidelines (16 items), with facilitators and barriers being based on a literature study and an expert panel consensus procedure.The BFAI instrument allows for tailoring and addition of items [24] .
Additional statements in the current study's self-administered questionnaire were identified after literature review [8-10 , 21] by two authors (MA, AO) with added feedback from the other authors.The additional items were rephrased into statements and were then independently reviewed for relevance by two experts (one in VBHC and one in implementation research) within the Erasmus University Medical Center, and ultimately approved for inclusion by the research team.
According to Peters et al ., possible facilitators or barriers can be related to the characteristics of the interventions, care providers, patients, and implementation context [24] .Therefore, items were divided into four domains to enable a categorized presentation of results: 4) Context characteristics (e.g. group standards/socialization, presence of an information and administration system, setting).
A total of 37 statements (both positively and negatively phrased) were ultimately included in section three of the full version of the questionnaire.Since disease teams were in different phases of PROMs implementation and some facilitators or barriers were more (or even only) relevant for some phases, the number of statements included in the questionnaire was dependent on the implementation phases that respondents were currently in.Specifically, 28 statements were included in the questionnaire that was sent to participants that were in the "Designing" or "Preparing" phases, while the full questionnaire with all 37 statements was sent to participants that were already in the "Commencing" or "Reflecting and Developing" phases.Participants who were in the first phase ("Purpose and Intake") only responded to the questions in sections one and two of the questionnaire.
Although the accuracy and consistency of the questionnaire were not formally tested, it was pilot tested for comprehension among physicians and researchers (N = 10).The results of this pilot indicated that completion of the whole questionnaire took approximately 15-20 minutes and that it was well-understood and positively received by the respondents.Minor textual amendments were made to the questionnaire following the pilot.

Data Analysis
Frequencies and percentages were computed to describe respondents' characteristics and responses to questions and statements.In reporting the results, we focused on statements that at least half of the respondents (dis)agreed or fully (dis)agreed with.
Data derived from the free-text responses to open-ended questions in the second section of the questionnaire were summarized independently by two authors (MA, AO) and then compared.Exemplary quotes were selected to illustrate recurrent thematic elements and to provide more detailed context on experienced facilitators and barriers.
Analyses were performed with Statistical Package for Social Sciences (SPSS), Version 25.0 (IBM Corporation, Armonk, NY, USA).

Ethical Considerations
Ethical approval was not required for the anonymous webbased survey in this study as it was part of a quality-improvement project [26] .Informed consent of included participants was obtained, and their confidentiality was ensured.

Responses to Questionnaire Statements
At least half of the respondents (fully) agreed with eight of the positively phrased statements and (fully) disagreed with four of the negatively phrased statements ( Table 2 ).These twelve factors are briefly described below.Most respondents (89%) perceived the purpose of implementing PROMs to be clear.In addition, 63% found the administered PROMs to be relevant for their patient population with an acceptable number of questions (56%).The majority of respondents (69%) also felt that open communication was possible within their disease team when providing feedback on the PROM implementation process.In addition, most participants (85%) perceived the presence of a coordinator in the (outpatient) clinic to be essential for implementing PROMs.Many respondents (59%) also reported that reviewing PROMs prior to the consultation and discussing them during the patient encounter, made the consultation more efficient.Furthermore, most respondents indicated that they were open to changing their old work routines (65%) and many respondents (88%) were confident that if they would use PROMs, they would be able to provide better care for patients.Moreover, they reported a willingness to use them in the future (76%).The majority (67%) of respondents were unafraid that the quality of their care might be assessed based on PROM results.Lastly, respondents (81%) indicated that the use of PROMs is or would be facilitated if they are integrated into the EHR, and most also reported a digital dashboard to be necessary for the consultation room with patients (79%).
On the other hand, the majority of respondents (76%) experienced a language barrier for patients with a primary language other than Dutch to be an impediment to the implementation of PROMs in the (outpatient) clinic.There were no other negatively phrased statements that at least half of the respondents (fully) agreed with or positively phrased statements that at least half of the respondents (fully) disagreed with ( Table 2 ).

Responses to Open-Ended Questions
All 61 study participants responded to the open-ended question "What did you experience as the biggest facilitating factor in the implementation of PROMs in the (outpatient) clinic?" ( Table 3 ).Nine respondents (15%) reported that "motivation/enthusiasm/interest within the disease team" is needed for successfully implementing PROMs.This factor was the only reported facilitator by employees across most implementation stages, namely "Designing", "Preparing", "Commencing" and "Reflecting and Developing".Some participants (N = 4) noted "collaboration with multiple disciplines on every level of care" as an encouraging factor.A "more structured patient-provider communication about PROM results" was another reported facilitating factor (N = 6).Three respondents mentioned that "sufficient support from the central VBHC team" promotes the implementation.Other reported facilitators (N = 2 each) were "well-organized IT", "the provision of a guideline for interpreting PROMs", "the past experience of other colleagues", "the team-building around one clinical condition", "sufficient resources", and "the presence of a nurse practitioner".
All respondents also responded to the open-ended question "What did you experience as the biggest impeding factor in the implementation of PROMs in the (outpatient) clinic?" ( Table 3 ).Seventeen respondents indicated issues with the IT infrastructure to be impeding factors in the implementation process.These issues included "access to the web-based platform" (N = 8), the "visualization of the dashboard" (N = 1), the "response time of the IT-platform" (N = 3), and "integration of the PROMs into the EHR" (N = 5).In addition, "time constraints of care providers" in combination with the "labor-intensive nature of the incorporation of PROMs in care" were frequently reported (N = 14).These two barriers (i.e.IT-related challenges and time constraints/labor intensity) were most reported across most phases, including "Design-ing", "Preparing", "Commencing" and "Reflecting and Developing".Furthermore, "insufficient staff for support" and "insufficient staff for coordination" were also reported as barriers (N = 10).Other experienced barriers were "challenges in multidisciplinary collaboration due to conflicting interests, schedules, and task division" (N = 3), "lack of uniformity" (N = 3), and "excessive duration of the

ARTICLE IN PRESS
JID: HEAP [m5G; August 6, 2021;16:27 ] implementation process" (N = 2).Finally, barriers reported by single respondents were "insufficient information", "language barrier for patients with a primary language other than Dutch", "inappropriate organization of care", "learning new work routines with PROMs", and "different motivation levels within the disease team to use PROMs".

Discussion
Although the adoption of PROMs in routine clinical care has been widely advocated, few studies have analyzed facilitators and barriers to the implementation of PROMs in clinical practice as perceived or experienced by involved healthcare professionals [21] .This cross-sectional study sought to investigate the views of healthcare professionals on the implementation of PROMs in clinical care in the context of a VBHC-initiative in an academic medical center.Overall, our findings highlight that for a successful implementation of PROMs in clinical practice, it is essential for healthcare organizations to support motivated healthcare professionals, to arrange for a PROM coordinator in the (outpatient) clinic, to invest in an adequate IT infrastructure, and to remove language barriers for non-Dutch speaking patients by making multilingual surveys available on an interface.Many factors that influence healthcare professionals' ability and desire to implement the routine measurement of PROMs seem to be bi-directional: they can act as either facilitators or barriers.
Respondents had mainly positive attitudes towards implementing PROMs in clinical practice, with most reporting its purpose to be clear and the utilized questionnaires to be relevant to their patient population and not too burdensome to fill in.The majority of study respondents expressed confidence in PROMs contributing to better care, with most respondents reporting being open to changing old work routines and willing to use PROMs in the future.This is in line with a study by Boyce et al ., who found that "healthcare professionals value PROMs when they are useful for the clinical decision-making process" [9] .In the current climate of skepticism on the validity and interpretability of PROMs [27] , it is crucial that prior to implementation, a consensus is reached among all involved healthcare professionals on the goal of routine PRO measurement (e.g.detecting previously unrecognized health issues [28] , monitoring disease progression [29] , stimulating better communication [30] and promoting shared decision-making [31] ).
The responses to both questionnaire statements and openended questions demonstrated that respondents experienced open communication about the PROM implementation process within their disease team.This seems related to the factor "intrinsic motivation/enthusiasm/interest", which was the only factor that was reported as a facilitator across almost all implementation stages.Motivation has been associated with encouraging teamwork and creating a sense of solidarity between colleagues [32] .This essential component can enhance collaboration between healthcare professionals in a multidisciplinary team, which does not always turn out to be easy.In this study, "multidisciplinary collaboration" within a disease team was also mentioned as a facilitator, as was "teambuilding around one disease/condition".Correspondingly, "different motivation levels within the disease team to use PROMs" and "challenging multidisciplinary cooperation due to conflicting interests, schedules, and task division" were described as experienced barriers.Healthcare professionals need to learn to work together in a multidisciplinary way, by exploring the boundaries of each other's disciplines and being open to other insights, raise mutual trust, and create a sense of joint responsibility [33] .
Other interesting facilitators reported by healthcare professionals were (the prospect of) "better preparedness with more infor-mation on how patients are doing prior to consultation" and (the prospect of) a "more structured/ efficient patient-provider communication".In addition to these factors, the "involvement of patients in their own care" was also reported as a facilitator.These factors suggest that the (prospect of) engaging patients in their own care process seems to incentivize healthcare professionals to implement PROMs in clinical care, which subsequently may lead to a more individualized health service delivery and an improvement in the quality of care.
The presence of a PROM coordinator in the (outpatient) clinic, which some respondents felt should be the role of a nurse practitioner, was a frequently reported facilitator.Similarly, "the absence or lack of staff for coordination" was also a reported barrier in the current study.This factor may be related to other barriers that were frequently reported, namely "time constraints and/or the "labor-intensive nature of the implementation".Antunes et al .[8] and Dunckley et al .[33] also identified a coordinator as being essential for implementing PROMs in palliative care.In collaboration with the VBHC expertise team of the Erasmus University Medical Center, the coordinator assembles the members of the relevant disease team and explains what PROMs are and what is expected from them.In addition, the coordinator should be the linking pin between different levels in the organization (e.g. between management and in daily clinical practice) and support the multidisciplinary team.The coordinator also ensures that the multidisciplinary team comes together, prepares the office hours, and is the point of contact for the IT team that deals with the construction and implementation of the IT-system [8] .
Issues related to the IT infrastructure were also frequently reported as a barrier, with respondents reporting difficulty in accessing the web-based PROM platform, the slow working speed of the platform, the visually unappealing dashboard, and the lack of integration in the EHR.Correspondingly, van Egdom et al .[10] found the integration of PROMs in the EHR to be a facilitator for implementing PROMs in specifically breast cancer care.Electronic capture of PROMs in the clinic and between appointments allows real monitoring of symptoms, flexible scheduling of hospital appointments in response to PROM data, early detection of problems, and prompt clinical interventions [34] .However, the PRO data needs to be updated frequently and made accessible to patients and healthcare professionals.The use of PROMs in daily clinical practice also requires an adequate IT infrastructure for the design and visualization of these PROs in a clear and insightful way, in order to provide meaningful feedback to the patient and running smoothly in the consultation room.In addition, it should be able to show a trend in PROM scores after treatment to give patients insight into their own course of the disease [21 , 34] .
The presence of a "language barrier for patients with a primary language other than Dutch " was the only barrier identified through the questionnaire statements.In a study by Schamber et al ., non-native patients showed a trend towards a lower rate of survey completion although not statistically significant [34] .As one of the potential benefits of PROMs is the discussion of patient issues that otherwise would not have been identified, it is crucial in regards to healthcare equity that the needs of non-native speakers and patients with low health literacy skills are also taken into account.The availability of PROMs in multiple languages may be one effective and relatively inexpensive way of decreasing the reporting gap between native and non-native speakers.Some overlap was observed between the responses to questionnaire statements and the open-ended questions.Overlapping facilitators included the confidence that implementing PROMs would lead to improved/more efficient patient care, the presence of a PROM coordinator, and a well-functioning IT infrastructure.The only overlapping barrier was the "language barrier for patients with a primary language other than Dutch".These aforementioned

ARTICLE IN PRESS
JID: HEAP [m5G; August 6, 2021;16:27 ] factors were the most prominent in the current study, and need to be especially considered prior to and during future implementation plans.

Strengths and Limitations
A key strength of this study was the prospective nature of the data collection.In addition, while previous research on the implementation of PROMs as summarized by Foster et al .[21] has typically focused on a particular disease (e.g.cancer) [35] , the field of healthcare (e.g.palliative care) [8] or a specific phase of the implementation process [9] , the current study intentionally included all phases of the implementation process.Furthermore, the respondents were involved in the care of a variety of diseases, albeit selectively chronic conditions.The web-based questionnaire's combination of both closed and open-ended questions was another strength of this study, as it gave respondents the opportunity to contextualize their responses to the questionnaire statements and report additional facilitators and barriers not included in the statements.Despite the combination of both types of questions, this cross-sectional study was unable to delve deeper into underlying mechanisms between facilitators or barriers and the implementation process.Future research with focus groups of healthcare professionals and administrative personnel may provide further insights into these mechanisms.
However, several limitations should also be mentioned.Firstly, the questionnaire used in this study was an adapted version of the "Barriers and Facilitators Assessment Instrument", a questionnaire previously developed and validated by Peters et al .[24] .This adapted questionnaire, which consisted of revised or newly added statements/questions specific to the study's aim and setting, has not been validated or used in previous research.We acknowledge that, in retrospect, some statements (e.g."The purpose of implementing PROMs is clear") could have been interpreted by respondents as a general statement with which they could agree or disagree, and not as a facilitator/ barrier that they have actually experienced as such in practice.Nonetheless, we still believe that these statements embody important factors that should be taken into account before and throughout the different stages of PROM implementation.
Secondly, this study only involved healthcare professionals of a single academic medical center, which typically has more clinically complex patients, higher resource utilization, and a different workflow than e.g. a general hospital.This may limit the generalizability of the healthcare professionals' perspectives as identified in this study.
Thirdly, only 61 respondents out of 187 invitees responded, despite attempts to increase the response rate by sending out reminders twice.On the other hand, the response rate across different specialties was high considering the fact that 17 specialties/disease teams were represented by at least one respondent.
Lastly, as it is conceivable that the respondents are relatively highly motivated employees who favor the use of PROMs for the potential quality of care improvement for their patients, the results may somewhat be biased.Specifically, it is possible that respondents were more likely to have a positive attitude towards PROMs implementation and view certain factors as facilitators, which may also explain why only one barrier was experienced from the questionnaire statements.The responses to the open-ended questions, however, did mention a considerable number of barriers as well.

Conclusion
In this study, commonly reported facilitators for implementing PROMs in routine clinical care were the presence of a coordinator, intrinsic motivation of members within a multidisciplinary disease team, and the integration of PROMs in the EHR.On the other hand, frequently reported barriers were time constraints, IT issues, and language barriers for patients with a primary language other than Dutch.In general, these findings are consistent with previous research.It is imperative that healthcare organizations that plan to incorporate PROMs in routine clinical care, account for these facilitators and barriers prior to and during the implementation process.

Source of Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Fig. 1 .
Fig. 1.Flowchart of Included Study Participants from Different Phases of PROM Implementation.

Table 1
Demographic and Employment Characteristics of Study Participants (N = 61).

Table 2
Responses to Statements on Facilitators and Barriers of PROMs Implementation in Clinical Care.