Four Years of Experience Using Problem-Based Learning as a Platform to Develop Student Teaching Skills: A Preliminary Report

This article was migrated. The article was marked as recommended. Introduction: A graduating medical student should be competent in both teaching and communication skills. This concept is supported by the Accreditation Council for Graduate Medical Education and the Association of American Medical Colleges. Our study describes how we utilized problem-based learning as a platform for developing student teaching skills and to examine preliminary outcomes. Methods: Since 2013, third-year medical students at Florida International University Herbert Wertheim College of Medicine have participated in a mandatory problem-based learning course in parallel to their clinical rotations. During the course orientation students have been led through interactive sessions on writing learning objectives and methods for effective micro-teaching sessions. During seven subsequent sessions, trained faculty members have assessed and provided narrative comments on students’ “Ability to Teach Peers” using an anchored developmental scale rubric. Data from four academic years were available for analysis. The Wilcoxon signed-rank test was used to test differences between the initial and final sessions. Results: At the initial session, 39.0% (n=147) received ratings of “mastering.” By the final session, 62.6% (n=236) received ratings of “mastering.” Conclusion: Our preliminary work demonstrates that a brief orientation to micro-teaching followed by repeated mandatory practice and feedback within our problem-based learning curriculum may serve to build students’ teaching skills.


Migrated Content
"Migrated Content" refers to articles submitted to and published in the publication before moving to the current platform.These articles are static and cannot be updated.Trevor Gibbs, AMEE 5.
Richard Hays, James Cook University 6.

Background
The importance of learning to teach as a key outcome in medical education now has widespread acceptance.The Accreditation Council for Graduate Medical Education (ACGME) competency of practice-based learning and improvement requires that trainees "participate in the education of patients, families, students, residents and other health professionals" (ACGME, 2009).Indeed, teaching and communication skills underlie three of the Association of American Medical Colleges (AAMC) Entrustable Professional Activites (EPAs), including EPA 6 (provide an oral presentation of a clinical encounter), EPA 8 (give or receive a patient handover to transition care responsibly), and EPA 11 (obtain informed consent for tests and/or procedures) (AAMC, 2014).The General Medical Council of the United Kingdom states that a graduating student should be able to "work effectively and appropriately as a mentor and teacher for other learners in the multi-professional team" (General Medical Council, 2018).
In a national survey conducted in 2008, 43 of 99 US medical schools reported offering a formal student-as-teachers programs with the majority occurring as fourth-year electives (Soriano et al., 2010).Our review of subsequent literature on this topic revealed several additional reports of elective experiences occurring in the fourth year (Shah et al., 2017;Yeung et al., 2018;Yoon et al., 2017).While these learning opportunities will support the development of skills among students who choose them, the challenge of promoting competency in all students calls for changes to the mandatory curriculum.With time constraints limiting curricular expansion, educators might identify opportunities in existing curricula to provide students both practice to teach and feedback on their teaching.One such opportunity includes student presentations incorporated within active learning pedagogies such as problem-based learning (PBL).
In a literature review using three databases (PubMed, ERIC, and EMBASE) with the terms "medical education, undergraduate", "students, medical", "teaching", "feedback", and then same with "problem-based learning," we found several references to students as near-peer and peer teachers (Nelson et al., 2013), to electives and modules specifically designed to develop educator skills (Dandavino et al., 2007), and to programs for developing students as PBL tutors (Blatt & Greenberg, 2007;Clark et al., 2008).However, we found no literature on the use or impact of feedback to students within problem-based learning curricula to improve their teaching skills.
The objectives of this study are to describe how we utilized PBL as a platform to develop student teaching skills and to examine preliminary outcomes.

Activity
Since 2013, a third-year mandatory PBL course running parallel to clinical rotations has served as a platform to utilize a rubric and provide feedback on student teaching skills.At our institution, PBL is conducted with groups of 5-7 students.In the morning, they work through an unfolding case in 4-6 segments, making and justifying clinical decisions at each juncture, and identifying learning gaps in the process.For some parts (the history and the laboratory values), they are given only the information they request with justification.The facilitator does not lecture or otherwise provide content.They do not use internet or other resources in the morning, to emphasize use of the group's collective intelligence.Students then have 2 hours to prepare short individual presentations for the afternoon on their assigned/chosen learning objective(s), during which time they do access the available resources of their choice.The afternoon session consists of the presentations and questions, followed by reflection on the overall session and individual contributions to group learning.Cases are diagnostically challenging, for example a woman in the third trimester of pregnancy with acute appendicitis presenting as pyelonephritis.
During the course student orientation, we conducted interactive sessions on writing learning objectives and structures and guidelines for effective micro-teaching sessions.The latter consisted of a simple structure emphasizing the importance of engaging the audience through initial interaction to activate prior knowledge and ensure connection to the case, use of visual aids (pictures, diagrams, algorithms, etc.) rather than text on PowerPoint to present information, and a closing summary to test understanding and highlight take-away points.
Unable to locate a rubric suitable for the assessment of short, quickly prepared learning sessions, we developed one ourselves.We borrowed the categorical descriptorsemerging, acquiring, masteringfrom the PBL rubric then used at the University of New Mexico.The behavioral anchors in the "mastering column" map to the guidelines provided in the orientation session.We used our own knowledge gained through observation of developing faculty to devise the anchors for emerging/developing (2 of us were the primary faculty developers at the time).
Faculty facilitators assessed and provided narrative comments, using this rubric with the domain labeled "ability to teach peers": Emerging (1-2): failed to relate learning issue back to case; rote recital of information from notes and/or PowerPoint; did not engage audience or check for understanding.
Acquiring (3-4): learning objectives stated or presented in beginning; organized; presented learning issue at appropriate level; related learning issues back to case.
Mastering (5): engaged audience through the use of frequent questions, visual diagrams, handouts or other activities; checked understanding of audience.
This rubric was discussed in detail at the faculty course orientation, with specific behavioral anchors of student performance at each level.
Data were available for a total of four classes (Classes of 2015(Classes of -2018)).Descriptive statistics included frequency distributions, means, and standard deviations on student performance at the initial and final sessions; the Wilcoxon signed-rank test was used to test differences between the initial and final sessions.The mean of student responses to a Likert-type item included in the post-course survey related to this activity ("The PBL sessions improved my ability to teach effectively") was also examined from two classes; ratings for this item ranged from 1 (Strongly Disagree) to 5 (Strongly Agree).All analyses were conducted using Stata 14 (College Station, Texas).This research was deemed exempt from review by the Florida International University Health Sciences Institution Review Board.

Results
Data were available for both the initial and final sessions for a total of 377 students.Results for these students are shown in Table 1.At the initial session, 39.0% (n=147) received a rating of "mastering", with a mean score of 4.17 (SD=0.81;range: 1-5).By the final session, 62.6% (n=236) were rated as "mastering", with a mean score of 4.58 (SD=0.58;range: 3-5).Results of the Wilcoxon signed-rank test indicated a significant difference between the initial and final sessions.
Post-course survey data were available for a total of 186 students from two classes.For the Class of 2016, the mean rating of the statement "The PBL sessions improved my ability to teach effectively" was 4.1 (SD: 0.75; n=83; response rate: 72.8%).The mean rating for the Class of 2018 was 4.0 (SD: 0.96; n=103; response rate: 85.8%).For logistical reasons, data were unavailable for the Classes of 2015 and 2017.
We did not systematically evaluate the quality of narrative feedback, nor investigate its relationship to changes in ratings of student performance.

Discussion
Our preliminary findings show that initial instruction in basic principles followed by narrative feedback within a problembased learning course may contribute to improving students' teaching skills.The lower standard deviation at the final session indicates that ratings of student performance became more standardized over the academic year.
The strengths of our work include the novelty of using PBL as a platform to assess student teaching skills as well as our large sample size.While the one-day version of PBL practiced at our institution is not typical, we can easily see easy transferability of our approach to more traditional versions, and to any version of case-based learning with required prepared peer teaching to address learning gaps.Another strength is the consistent finding of improvement over the academic year in each of the four classes.An important limitation of this study is our lack of comparison groups that did not engage in the practice or in the instruction on microteaching, and/or receive faculty feedback provided on their teaching.Neither can we isolate effects of the PBL course from the clerkships where students may receive feedback on didactic presentations.However, the PBL sessions serve as the most standardized and frequent didactic teaching practice our students receive in their third year.Additionally, student responses in the post-course survey support their perception that the PBL sessions played a key role in improving their teaching skills.Data for this question from all four of the classes would have strengthened this support.Finally, we have not yet been able to systematically explore possible validity evidence for our rating scale; we plan to do so in future studies.

Conclusion
Our preliminary work demonstrates that a brief orientation to micro-teaching followed by repeated mandatory practice and feedback within our problem-based learning curriculum may build students' teaching skills, as judged by faculty ratings as well as student perception.Future research should explore the content of narrative feedback from faculty with respect to effectiveness and developmental appropriateness.Our work has also uncovered the need to devise rating systems for use in PBL and other case-based settings with validity evidence to judge the quality and impact of student teaching and provide rigorous outcome assessment for future work in this area.

Take Home Messages
To our knowledge, this is the first report on the use of mandatory PBL or case-based curricula to develop medical student teaching skills.
Mandatory PBL and other case-based learning platforms requiring peer teaching may serve as an efficient platform for the development and improvement of medical student teaching skills.
Medical students perceive that a problem-based learning course offers the opportunity to develop teaching skills.
Orientations to micro-teaching can be easily incorporated into a PBL course orientation; quantitative and qualitative ratings of student micro-teaching performance can be easily incorporated into PBL rubrics.

Declarations
The author has declared that there are no conflicts of interest.certainly brings it into a clearer perspective.If an example of the case could have been provided it would have made it clearer if this was a case-based (CBL) study or a hybrid PBL.I still have worries about the methodology-sorry.I am not sure that a "true" PBL activity can be done in such a short period of time, particularly the discussive element and the possibility of addressing all three learning domains.However, and despite my reservations the authors have show an improvement in their students' teaching ability.
Whether this was a result of the PBL, a result of the repetitive action accompanied by feedback or just increased confidence, I do not think that this paper can confidently confirm; there would have to be a comparative cohort to show this.However I would still like to congratulate the authors again on their attempt to create a way in which students improve their teaching skills, whilst still developing their clinical skills; both I feel being so closely tied.
Competing Interests: No conflicts of interest were disclosed.

Version 1
Reviewer Report 13 August 2018 https://doi.org/10.21956/mep.18982.r27199 © 2018 Gibbs T. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Trevor Gibbs
This review has been migrated.The reviewer awarded 2 stars out of 5 An interesting although a rather confusing paper.I have not seen a "PBL" course that has been shortened to one day and like my colleagues I would not call what the authors describe here as a PBL session.It would have been helpful to see some of the various scenarios used to differentiate them from a casebased learning approachThat said, I also worry that the authors directly attribute the improvement in the students teaching to the PBL, given that they already had attended an orientation session on teaching and learning, had seven "goes" at teaching or presenting ( as the authors state", and had received some form of feedback through that seven opportunities time.I feel that there needed to be at least a comparative group who had not received those extra inputs to see if it was the PBL that brought about improvement.How we improve "teaching" within our graduates is very important since in everyday of our professional lives we teach, to patients, co-workers, peers, near-peers and such like.How we develop that is often found in our communication skills training, but we need to look at other methods.I congratulate the authors in attempting to combine teaching development with curricula learning, although in this case This is an interesting and well-written manuscript.It is important for medical students to be able to teach peers, colleagues, patients and the public as mentioned in the article.Third year medical students used PBL sessions to teach other members of the group and their performance were assessed by faculty members.All students were assessed and the results show a significant improvement in student performance as teachers.I assume that the faculty comments were based on the student presentations during the PBL.The authors have mentioned the limitations of their study and the take home points.I would be interested in greater information about the rubric used for assessing the student performance.The article will be of broad interest to medical educators.

Competing Interests:
No conflicts of interest were disclosed.there is a paucity of study on "the use or impact of feedback to students within problem-based learning curricula to improve their teaching skills".Below are my comments to improve the content.Methods section in the Abstract:-In the Methods section in the abstract, the number of the students who participated in the PBL should be stated given that abstract is crucial, and PBL is usually performed with a small group of learners but also a larger group.Methods section in the manuscript:-Some information is needed whether Developmental Scale Rubric reported in this study has been developed by the researchers themselves.If so, some references could be provided how the three criteria have been chosen: Emerging, Acquiring, Mastering.This rubric could be beneficial for medical teachers if they want to use.-Given that PBL is based on clinical problem, to contextualise this research and to show how this rubric can be used, a problem scenario from the research setting can be reported.-In the Discussion part, authors reported as follows: "Our preliminary findings show that initial instruction in basic principles followed by narrative feedback within a problem-based learning course may contribute to improving students' teaching skills."In the Abstract: "rained faculty members have assessed and provided narrative comments on students' "Ability to Teach Peers".Please provide some data regarding faculty members' assessment given that this research highlighted the significance of the use or impact of feedback to students within PBL.-There is a need to report about the resource materials used to use PBL (handouts or instructions during the teaching?)All in all, specific examples in the Methods section could make the PBL more concrete for the medical teachers.
Competing Interests: No conflicts of interest were disclosed.

Michelle McLean
1 Bond University 2 Bond University This review has been migrated.The reviewer awarded 2 stars out of 5 Thank you for asking me to review this short submission.As a someone who believes that we should all be on the same page when we talk or write about PBL or TBL (highly developed educational approaches underpinned by a series or steps or processes), there is not enough in this article to convince me that the method being described is PBL, i.e.Case Opening with five steps, a period (often a week) of self-study (Step 6), then returning for the Case Closing or Wrap-up (final one or two steps, with Step 7 being elaboration and application and Step 8 being reflection).PBL was designed as a whole curriculum approach so the fact that there were only seven one-day sessions in the academic year suggests that what is being described as PBL may not in fact be PBL in terms of the intended approach?The exercise of formally getting students to 'teach' is, however, well described and is something that could be introduced anywhere in the curriculum where the focus is on patient cases.The template and rubric could be shared with the medical education community.I could, however, see how students could contribute to peer learning during Step 6 by providing the 'expertise' in a hybrid PBL program, i.e. the supplementary sessions to assist learners to meet the learning issues they identified in Steps 1-5 as they worked through a patient presentation.How this currently works in one day (as described) is a difficult to imagine.
Notes On ContributorsDr.Carla S. Lupi is the Associate Dean for Faculty and Professor of Obstetrics and Gynecology at Florida International University Herbert Wertheim College of Medicine, where she also serves on the team working with the AAMC Pilot for the Core Entrustable Professional Activities for Entering Residency.Dr. Melissa Ward-Peterson is a Postdoctoral Associate in Department of Epidemiology, Robert Stempel College of Public Health and Social Work, Florida International University.She is currently supported by the National Institute of Minority Health and Health Disparities grant (1U54MD012393-01) for FIU-RCMI.
Dr. Ansley Splinter is an Assistant Professor of Pediatrics at Nationwide Children's Hospital and The Ohio State University.

Table 1 .
Four classes of performance data on student ability to teach peers at initial and final sessions in a third-year problem-based learning course Medical Teacher.35(2), pp.156-159.
This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.