Collective Wisdom in Faculty Development for Competency-Based Medical Education: A Needs Assessment and Survey

This article was migrated. The article was marked as recommended. Background: As programs transition to competency-based medical education (CBME), faculty development (FD) will be a key component of supporting successful implementation. Methods: Faculty at the University of Toronto (UofT) with leadership roles in residency education were invited to complete an online survey to explore their experiences with FD for CBME. Descriptive statistics were collected. Results were analyzed using thematic, frequency and comparative analyses between respondent subgroups to identify trends and theme categories relevant to the perceived most effective FD activities, most helpful FD topics as well as program/department needs for future FD initiatives. Results: The overall survey response rate was 44.6%. The most effective FD activity identified by survey respondents was a small group format. Perceived top FD topics included implementing CBME, assessment tools, feedback and coaching along with competence committees. The majority of programs felt that the ideal timing for CBME implementation was 6-12 months prior to implementation. The main perceived barrier to FD was lack of time amongst faculty. Conclusions: This data helped drive FD at UofT by supporting strategic planning for implementing competency based curricular reforms. The results have also informed the need for additional resources and enable focused FD on barriers and needs.


Introduction
Across Canada, undergraduate and postgraduate medical training programs are undergoing significant change as they transition to competency-based medical education (CBME) models.In postgraduate medical education (PGME), the College of Family Physicians of Canada (CFPC) has implemented the Triple C Curriculum which reflects the key elements of its competency-based curriculum: comprehensiveness, continuity in learning experiences as well as patient experiences and centered in the family medicine domain (Iglar, Whitehead and Glover-Takahashi, 2013;Gutkin and Triple, 2011).The Royal College of Physicians and Surgeons of Canada (RCPSC) is utilizing a stepwise approach to CBME implementation named Competence by Design (CBD) (Harris and Frank, 2014).Curricular modifications are well underway across universities including the University of Toronto (UofT); however, as programs transition to an outcomes-based approach to curriculum, assessment and evaluation, there is clear evidence that faculty feel unprepared for the tasks necessary to implement successful CBME (Harris and Frank, 2014).A recent scoping review on the landscape of FD in CBME has also shown that there is a paucity of literature in this area, despite the acknowledged importance of FD for CBME implementation and success (Sirianni, Takahashi and Myers, 2020).
It has been suggested that to ensure an effective CBME program there is a need for frequent, longitudinal assessments of learners, direct observation and narrative feedback from multiple sources (Harris and Frank, 2014;Holmboe et al., 2011;Dath and Iobst, 2010).Experts recommend a greater focus on faculty skill development for these tasks (Harris and Frank, 2014;Holmboe et al., 2011;Dath and Iobst, 2010).In particular, teachers working within this model need FD with regard to knowledge of competency-based education, facility in teaching within a CBME system, and new ways to assess and provide feedback to learners in a CBME context (Harris and Frank, 2014;Holmboe et al., 2010;Fraser et al., 2016).One potential challenge encountered when implementing CBME includes poor faculty attendance at events (Holmboe et al., 2010;Fraser et al., 2016).Suggestions for improving engagement include faculty incentives, making sessions easily accessible (e.g.online modules) and supporting the development of local CBME champions (Holmboe et al., 2011;Fraser et al., 2016).
Considering what is currently known about FD for CBME in the literature, the authors wanted to learn more about the local landscape of FD.The overall aim of this study was to harness the collective expertise and opinion of education leaders at UofT to determine their experience and needs with regards to FD for CBME.

Methods
An online survey was developed to explore the experiences of program directors and residency education leaders for programs currently in or transitioning to CBME.The survey design was informed by the Best Practices in Evaluation and Assessment (BPEA) Working Group.The working group suggested that the survey draw attention to the local context (i.e.university, program, department and site) and focus on change management principles, processes and practices are important factors that influence the successful implementation of CBME (Takahashi et al., 2017).The survey was developed and pilot tested through an iterative and collaborative process by the authors (SGT, GS) and a small group of colleagues with expertise in FD, medical education and survey design.A combination of multiple choice and open-ended questions were used.The survey was developed using Qualtrics and distributed via email to 168 faculty members.After the initial survey was sent in September 2018, four additional survey reminders were sent to potential respondents and the survey was closed in October 2018.Demographic information was also collected to inform subgroup analysis by program type (e.g.Family Medicine and Royal College).
Descriptive statistics were determined for each multiple-choice question and thematic analysis was applied to the qualitative data.The survey responses to multiple choice questions were summarized using frequency and cross tabulations performed using IBM® SPSS®.The analysis then focused on characterising the qualitative responses to the open-ended questions.One author reviewed the qualitative data (SGT) to identify response categories by grouping related responses into identified themes which was reviewed and refined by a second author (GS).Iterative review was done to reach consensus by all authors for the final categorization of responses.Ethics approval for this study was obtained from the University of Toronto Health Sciences Research Ethics Board.

Results/Analysis
See Table 1 for selected results.The overall survey response rate was 44.6%.Family Medicine faculty made-up 46.7% of respondents, followed by Medicine at 18.3%.56.7% of respondents were Program Directors and 26.7% were FD leads.Approximately 80% of respondents had programs that had already launched to CBME.When asked what type of FD activities respondents participated in, most attended workshops with small group activities (98%), accessed website/ online resources (94%) and reviewed newsletter updates (87%).The least commonly attended activities included webinars (29.6%), reflective exercises (5.6%) and online, self-instructional modules (3.7%).When asked what CBME-related topics the respondents planned on offering in their programs in the upcoming year, the top three subjects were Entrustable Professional Activities (69%), feedback and coaching (67%) and CBME/CBD update (60%).The least likely FD topics respondents were planning included learner handover (11%), change management (9%) and educational models and design (9%).
The top FD topics survey respondents would like to learn more about include implementing CBME, assessment tools, feedback and coaching and competence committees.The most effective FD activity identified by survey respondents was a small group format while the most helpful FD topic was coaching and feedback.The least effective FD activity was online modalities and resources.
The majority of programs felt that the ideal timing for CBME implementation was 6-12 months prior to implementation (45%); however, a large proportion (41%) also felt that FD should begin more than 1 year prior to CBME implementation.Survey respondents felt that the most common supports needed include administrative and technical support for program design/evaluation.
The main perceived barrier to FD was lack of time amongst faculty (87%), followed by lack of administrative support and lack of expertise in FD program design.81% of faculty respondents did not survey their own faculty at the program level regarding CBME FD needs.

Discussion
This survey is the first of its kind to evaluate the FD needs and experiences of postgraduate programs broadly at UofT.The data helped Postgraduate Medical Education (PGME) and the Centre for Faculty Development (CFD) at UofT in planning strategically for implementing competency based curricular reforms.The results have also informed the need for additional resources and enabled focused FD on barriers and needs.For example, this survey helped support the development of a feedback and coaching series along with a FD planning tool as a program resource.The survey data was further used to leverage resources for content development.
In terms of lessons learned, the authors found it interesting that, despite the convenience and flexibility of online modalities for FD, the respondents identified small group, interactive sessions as the most successful FD format, while also noting that online modalities were perceived as least effective.The narrative survey comments may help shed some light on this finding.One respondent stated that "faculty need the time and space to learn about CBME... the site-based approach... is best to transfer skills and knowledge on a practical level."The survey results revealed that one size does not fit all when it comes to the format of the FD activities, but also when it comes to topics of interest.Sessions on coaching and feedback were noted as most the helpful FD topics presented.In particular, respondents found this topic "most valuable for 'frontline' teachers." When it came to the ideal time for FD related to CBME implementation, most respondents felt that at least 6-12 months of lead time were needed.Respondents felt that faculty needed "adequate time for familiarity with CBD, but not so long that faculty feels it is irrelevant/forgotten."Those who felt that greater than 1 year of lead time is needed commented that "change management is a long term process" and that "faculty members need repetition".
A surprising finding was that 81% of respondents had not surveyed their local faculty about their FD needs with respect to CBME, especially in light of the importance that the local context plays in the design and implementation of FD activities. 9Although this survey has helped to inform the design of FD activities at the PGME and CFD level, the authors feel that this does not replace the need for program or department level feedback.

Limitations
Although the overall survey response rate was acceptable, the small number of respondents within each sub-group did not allow for statistical comparisons between sets.

Future Directions
This data will help inform future offerings.For example, the development of FD activities by PGME and the CFD that include peer mentoring and coaching, reflective exercises, change management or role-play with feedback may be of benefit to programs that are not prioritizing these topics currently.Furthermore, longitudinal follow-up of the programs surveyed may be helpful to monitor changes in FD gaps and needs over time.
The authors suggest that all FD should be data driven to spur content development as one size does not fit all.It is clear that FD should be centred on the needs of its learners.These needs will vary and may focus on content development, tool development, along with resources and supports to navigate the systems issues and barriers that can undermine FD for CBME.

Take Home Messages
Faculty Development (FD) will be a key component to supporting successful implementation of competencybased medical education (CBME).
Survey results indicate small group format as the most effective FD activity.
Top perceived FD topics include: implementing CBME, assessment tools, feedback and coaching along with competence comittees.
Majority of programs felt CBME implementation 6-12 months prior implemenation as a requirement was the ideal timing.
The main perceived barrier to FD was lack of time amongst faculty.This is an interesting article focusing on faculty development for competency-based medical education.In my country this will be a challenge, because in our time-based, master-apprentice-system we have not had any formal faculty and programmatic assessment is just a vision for the distant future.However, systematic and rigorous faculty development program will be the first step towards competency-based resident education.The authors made an online survey in order to explore experiences of the faculty working in programs in or transitioning to CBME.The test was developed and pilot tested by the authors.The response rate was acceptable (over 40 %).The results were interesting but not surprising, the most popular faculty development format was a small group activity, the least popular were an online modalities and the most frequent perceived barrier was lack of time.The problems are usually the same for everyone, but the solutions depend on the context.Thus, one size will not fit all.Like the authors point out, a local survey to find out the learning needs, preferred formats and timing of the educational intervention would be helpful.
Competing Interests: No conflicts of interest were disclosed.

Table 1 :
Summary of Key Results *Respondents were able to indicate more than one response Responses # (%)