Uncovering Hidden Scholar Feedback with Field Notes

Background The CanMEDS Scholar role remains one of the most neglected and diﬃcult to assess roles in medical education literature. Our study analyzes 'ﬁeld notes' (FNs) - frequent formative written feedback in postgraduate family medicine, and examines whether the Scholar role feedback can be found within the feedback for the Medical Expert role. Methods A deductive analysis was conducted on FNs generated by resident preceptors in the Department of Family and Community Medicine (DFCM) at the University of Toronto (UofT) between April 2017 and July 2017. We focused on Medical Expert role feedback. Scholarly roles derived from the CanMEDS Scholar role description were used to identify Scholar role feedback in these FNs. the Medical expert role were analyzed. Of these, 7.6% deﬁnitely described scholarly feedback. Two scholarly roles were highlighted - Evidence-based decision-maker and Lifelong learner. agreement test of a random subset 10% of comments. The number of deﬁnitely scholar identiﬁed FNs mapped onto each of the four scholarly roles was tabulated. The data were analyzed using Dedoose 8.0.35 web application (Los Angeles, CA). Approval for this study was obtained from the UofT Research Ethics Board (REB).


Introduction
CanMEDS is a competency-based medical education framework that was formally adopted in Canada in 1996 and implemented worldwide. CanMEDS is composed of seven distinct roles, each imperative to physician training. These roles include -Communicator, Leader, Professional, Collaborator, Health advocate, Scholar and the encompassing role Medical expert (Frank, Snell and Sherbino, 2015). The CanMEDS framework is used for evaluating and providing feedback for medical learners, through assessment of each individual role within the academic and clinical settings (CanMEDS, 2000). At the Family Medicine (FM) residency program at the University of Toronto (UofT), postgraduate learners are evaluated using an assessment tool called "Field notes". When used appropriately, comments in the form of field notes (FN) can provide necessary feedback for learner reflection and growth (Donoff, 2009).
One of the CanMEDS roles that has been recently described as "neglected" is the Scholar role (Ologunde, Di Salvo and Khajuria, 2014). CanMEDS states that as scholars, "physicians must demonstrate a lifelong commitment to excellence in practice through continuous learning and by teaching others, evaluating evidence, and contributing to scholarship' (The Royal College of Physicians and Surgeons of Canada, 2020b). CanMEDS further describes that as a 'scholar', a physician must uphold four key scholarly roles: Lifelong learner, Evidence-informed decision-maker, Teacher, and Researcher (The Royal College of Physicians and Surgeons of Canada, 2020b). While two scholarly roles, the teacher and the researcher, have been reported as easy to assess, assessment of the other roles has proven more challenging (Kassam et al., 2013). The inability to assess and provide feedback to learners pertaining to certain elements of the scholar role may be the explanation why many residents view this role as one of the least important ones (Ringsted et al., 2006).
As the 'Medical expert' role has been referred to as the "central and integrative role" (Whitehead, Austin and Hodges, 2011), we hypothesized that feedback relating to the other CanMEDS roles may be found in feedback evaluating the 'Medical expert' role. In this study we aim to discover whether or not learners, in receiving feedback on the 'Medical Expert' role in their field notes, are also receiving feedback on the intrinsic 'Scholar' role at the same time. Identifying and understanding the extent of the overlap of these two roles in field note assessments can inform educators on ways to enhance feedback on the Scholar role.

Methods
We extracted de-identified data from encounter-based FN written by multiple clinical preceptors from 14 different training sites affiliated with the UofT Department of Family and Community Medicine between April-July 2017. Four reviewers with different levels of medical education (GK, FL, BS, and JM) each independently coded 25% of the field notes. Following organized mixed-method approaches (Lockyer et al., 2018), each FN was coded into one of three categories -Definitely scholar, Somewhat scholar, or Not scholar. Code designation relating the relevance of the content of a free text comment to the scholar role was accomplished by referring to the CanMEDS Scholar role description as a guide. "Definitely scholar" notes were those easily related to one of four key scholarly roles, and the scholarly feedback was the focus of the comment. These notes were further mapped onto one of the four key scholarly roles -Lifelong Learner (LL), Evidence-based Decision-maker (EBDM), Teacher and Researcher. "Somewhat scholar" notes were identified as relating to 1 of the 4 key scholarly roles, but had a stronger relationship to other CanMEDS roles. "Not scholar" notes were those identified as not relating to any of the scholarly roles. Following completion of the coding process, all coders reviewed the code assignment and disagreement was resolved through consensus. Finally, internal reliability was tested with one of the initial reviewers (JM) through a code Kazevman G, Marshall J, Shachar B, Slater M, Leung F, Guiang C MedEdPublish https://doi.org/10.15694/mep.2021.000168.1 Page | 3 agreement test of a random subset 10% of comments. The number of definitely scholar identified FNs mapped onto each of the four scholarly roles was tabulated. The data were analyzed using Dedoose 8.0.35 web application (Los Angeles, CA). Approval for this study was obtained from the UofT Research Ethics Board (REB).

Results/Analysis
A total of 5224 field notes were written of which 3.0% (N=155) were identified by preceptors as 'Scholar' role feedback and 34.6% (N=1809) of the field notes written were identified by preceptors as 'Medical expert' feedback. During the analysis, we found that 7.6% (N=138) of all 'Medical expert' recorded feedback pertained to the 'Scholar' role, identified by the code "Definitely Scholar". While the data analysis generated 3 groups of codes (Definitely scholar, Somewhat scholar and Not scholar), we used only "Definitely Scholar" comments as they truly represent feedback related to the scholar role.
Evidence-based Decision-maker EBDM was the most identified key scholar role in the free text comments, seen in just over half of the "Definitely scholar" 'Medical expert' field notes. This role encompasses how well learners are able to identify and consolidate the best quality evidence into their practice. Frequent topics mentioned were proper utilization of common diagnostic resources and the ability to in prior knowledge to support decision making. One occasion this role was identified can be seen in the feedback of a preceptor to a resident that: "Although [the learner] had some knowledge limitations given the unique scenario, knowledge was appropriate for level of training and knew how to access resource to get information"

Lifelong Learner
This was the second most common scholar role identified in the examined field notes. This role entails feedback describing learners' reflection of their personal practice after a patient interaction and for times when learning plans were made to improve knowledge in the future. One example of reflection and learning plan creation can be seen in the following feedback: "wrong immunization given (adacel-polio rather than adacel); please review adult immunization schedules as per TPH resources/guidelines" Many of the comments pertaining to the LL role include specific resources that should be learnt by the resident. This can be seen in one preceptor's feedback: "Suggested atrovent treatment for asthma in a child. Review pediatric asthma guidelines re diagnosis/ treatment (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2831604/)"

Teacher
Only two comments were related to the 'Teacher' role, which pertains to times when a learner educated a medical student, or other allied healthcare professionals. One example is seen in a preceptor's feedback on a resident's procedure: "Excellent teaching-bringing a medical student to tongue tie procedure on a newborn with this condition…. Competent to do procedure on own next time" Kazevman G, Marshall J, Shachar B, Slater M, Leung F, Guiang C MedEdPublish https://doi.org/10.15694/mep.2021.000168.1 Researcher This role describes times when learners were involved in the creation or distribution of novel research. None of the free text comments in our analysis fulfilled this description.

Discussion
Only 3% of field notes written by FM preceptors were designated as feedback regarding the 'Scholar' role, showing that feedback related to this role is uncommon. However, an almost equivalent amount of feedback pertaining to the 'Scholar' role was found to be hidden within the 'Medical expert' labeled feedback. In particular, while feedback for the Researcher and Teacher key scholarly roles was rarely found in our analysis of the 'Medical expert' labeled FNs, the EBDM and LL key roles appear to be more commonly found in these evaluations.
The presence of EBDM comments in the 'Medical expert' feedback can be easily explained when examining the description for the 'Medical expert' role. CanMEDS describes that among other expectations, a medical expert must "Apply knowledge of the clinical and biomedical sciences relevant to their discipline" (The Royal College of Physicians and Surgeons of Canada, 2020a). This description is very similar to the 'Scholar' role's CanMEDS description stating that as scholars, physicians must "Integrate best available evidence into practice" (The Royal College of Physicians and Surgeons of Canada, 2020b). This overlap, together with the perceived importance of one role over the other, may be the reason that 'Scholar' role feedback is found within the 'Medical expert' feedback. On the other hand, the presence of LL feedback within the 'Medical expert' comments is harder to explain. Lifelong learning has been proven to be easily taught and evaluated within learning environments, such as with problem-based learning exercises often used in medical school education (Berkhout et al., 2017). Yet, it has been highlighted that teaching and evaluating this scholarly role within the clinical setting is much more difficult (Bammeke et al., 2015). In the academic context learners focus on learning, whereas in the clinical context, learners are focused on providing patient care which supports their learning process. Therefore, we believe that feedback for this key scholarly role will always be intertwined with medical expert feedback, as advice and guidance will arise after providing care for a patient.
In recent years, several publications have pointed out the importance of the existence of physician scholars (Bammeke et al., 2015;Puddester et al., 2015), and the growing concern among physicians for the fate of the scholar role (Ologunde, Di Salvo and Khajuria, 2014). Yet, in these publications, the term "Scholar" was often synonymous for "Researcher". This dissonance between the perceived and the actual description of the scholar role may be the reason why this role is less visible in learners' feedback. A study published in 2012 proffered that while the generation of new medical knowledge is an important competency, we must remember that it only represents one aspect of the scholar role (Koo et al., 2012). Our analysis suggests that in order to allow for more comprehensive scholar role feedback for learners, an assessment should be done of the tools used to evaluate performance in the scholar role. Specifically, we recommend that the way "evidence-based decision-making" and "lifelong learning" are evaluated should be addressed when modernizing the evaluation system. Lastly, we recommend adjusting evaluation tools to allow more nuance, such as those that allow preceptors to provide feedback for multiple roles witnessed in a single encounter.

Conclusion
This study highlights aspects of the Scholar role that can overlap with Medical Expert role feedback. In order to address this overlap, feedback tools should allow learners to receive comprehensive and rich feedback in their learning encounters. Moreover, more attention should be given to better prepare preceptors to evaluate and Kazevman G, Marshall J, Shachar B, Slater M, Leung F, Guiang C MedEdPublish https://doi.org/10.15694/mep.2021.000168.1 Page | 5 categorize these roles in the clinical setting.

Take Home Messages
The Scholar role remains one of the least evaluated competencies in post graduate medical education. 1.
Preceptors percieve significant overlap between the Medical Expert role and the Scholar role.

2.
Assessing postgraduate medical learners on their abilities to meet the scholar role criterias must describe 3.
more than just their abilties as researchers and teachers. More support must be provided to preceptors in order to improve the accuracy of assesment of the Scholar 4.
role in the postgraduate medical education setting. Better tools should be used in order to accuratly describe postgraduate medical learners' abilities in meeting 5.
expectations as scholars in a postgraduate medical program.