The Experience of One Academic Medical Center Rapidly Transitioning Medical Student Clinical Teaching Using Video Telemedicine: Lessons Learned and Recommendations

Traditional clinical education for senior medical students has been disrupted by COVID-19. Telemedicine (TM) is a valuable tool that institutions can consider integrating into their curriculum to provide authentic patient experiences for medical students during this time of uncertainty. We describe our institutional transition to TM and the development of an educational framework to incorporate medical students into TM, evaluate its eﬀectiveness, and improve our process. Future work is required comparing video TM encounters to in-person visits exploring student, faculty and patient experience with multi-provider TM visits.


Introduction
Due to the emergence of the severe acute respiratory syndrome coronavirus-2 (COVID-19), traditional clinical education for medical students largely halted in accordance with guidelines issued by the Association of American Medical Colleges (AAMC, 2020). Educators responded by employing a variety of strategies to craft meaningful educational experiences for students including the utilization of virtual cases, shifting didactics earlier in the rotation with the hopes of later student reentry into the clinical environment, and creating non-direct patient care experiences (e.g. quality improvement projects) (Rose, 2020). Additionally, some explored involving students in the telemedicine (TM) environment. Kwan B, Charat S, Wooten D, Hill D, Restaino N, Friedman L MedEdPublish https://doi.org/10.15694/mep.2021.000014.1 Page | 2 While TM, defined here as the remote delivery of medical care using video visits, has been integrated into undergraduate medical education curricula at some institutions (Waseh and Dicker, 2019), a published roadmap for incorporating learners into TM is not readily available. In light of the uncertainty surrounding the duration or cadence of the COVID-19 pandemic and recognizing that future disruption to education may continue to occur in the form of quarantines and social distancing, we believe that TM will be a critical aspect of healthcare delivery and an essential component of medical student education for the foreseeable future. Patients have acclimated to the convenience of video visits, and trainees will need the experience and skill set to provide care using TM in a consumer-driven healthcare environment.
This article describes our institutional process in creating an educational framework to incorporate medical students into TM, evaluate its effectiveness, and improve our process. We outline the operational and training processes we undertook to successfully launch and sustain the use of TM video visits in the training of medical students.

Operational Background
In March 2020, the University of California, San Diego (UCSD) began planning for widespread community transmission of COVID-19. Anticipating a wider need for TM, training protocols were developed and distributed to providers. Primary care schedulers converted templates to accommodate TM visits, conducting patient outreach and education on visit logistics. TM was provided using the EMR patient web-portal (Epic, Verona, Wisconsin), allowing for consent, documentation and billing. Prior to March, less than 1% of all UCSD General Internal Medicine (GIM) faculty visits used video. Between March 16 th and 20 th , all GIM faculty received TM training, and by the week's end, nearly 80% of all patient visits were by video (Reeves et al., 2020).
The VA San Diego Healthcare System (VASDHC) had existing TM capability using the VA Video Connect (VVC) application, which was not extensively used prior to COVID-19. Scheduling staff converted most appointments to video or telephone, educating patients about VVC technology via phone calls and test-appointments. Between February 2020 to April 2020, the percentage of primary care providers conducting at least one video visit increased from 66.8% to 88.5%, while total number of video TM appointments among all specialties increased from 1,093 to 4,448 (VHA, 2020).

Educational Transition
We employed a two-pronged approach to TM education. The first phase of training targeted faculty in the form of written protocols and peer to peer education. The second phase of training involved educating students on the use of TM technology, best practices for conducting TM visits, and workflow protocols.
During the first two weeks of TM implementation in March 2020, four highly engaged third-year medical students were matched with GIM faculty to test educational workflows and provide feedback, resulting in trainee-facing and preceptor-oriented protocols. A construct consisting of four distinct phases (pre-implementation, preparatory, active, and debriefing) emerged from these efforts (Table 1). Orient medical students to video visit platform and basics of conducting physical exam via video (via didactic session, written tip-sheet, online training modules): Blood pressure and heart rate measurement or review of home readings Glucose measurement/review of glucometer readings General appearance Visual inspection of area of concern Assessment of tenderness by self-palpation Active range of motion Gait/mobility assessment Neurologic exam: limited cranial nerves, atrophy, symmetry, tremor, coordination, asterixis Mental status exam Educate students about best practices for Telemedicine: Quiet, private location Camera/device at eye level Adequate lighting and sound Professional attire and background Computer in reach to allow documentation while maximizing eye contact with patient Help from patient caregiver to hold device, flip camera to visualize areas of interest, etc. Pause before speaking to avoid speaking over patient Identify faculty conducting TM visits who are willing to include students/learners Student practices logging in to video platform prior to clinic session Gather and share personal contact information of student and preceptor pairs Visit Preparatory Phase Preceptor identifies 1-3 patients per session appropriate for student (ideally not scheduled back to back) and notifies them of student participation in appointment Student and preceptor brief over phone or video conference to address questions, visit workflow, and agenda setting for each patient prior to the clinic session. Preceptor orients student to methods of communication for questions during the visit (instant messenger programs, chat programs within the EMR, text messages) Preceptor reviews the contingency plan if the technology fails (e.g. convert the visit to a 3-way telephone encounter) If needed and no available clinical staff to do so, student calls patient 15-20 mins prior to appt to ensure no technical problems Visit Active Phase Patient Safety Pause: ensure patient in private location, confirm identity, confirm physical location and contact information for emergencies, obtain verbal consent for video visit, lock virtual room if applicable Technology Pause: ensure all parties can see and hear participants adequately, inform patient that provider and learner will be documenting during visit and that eye contact may be interrupted Visit Workflow: Student and preceptor start visit together, preceptor introduces student and patient and discusses flow of the visit Student facilitates setting agenda for the appointment with patient Student conducts history while preceptor listens without interrupting Preceptor asks additional or clarifying history questions Student conducts physical exam Student and preceptor leave visit (or mute/close camera) and discuss plan by phone or another video platform OR student presents assessment and plan in real-time with input from preceptor Student and preceptor rejoin visit and student explains care plan (if not already done) Student and preceptor close visit and log off Visit Debriefing Phase Preceptor moves to next patient while student works on note, writes patient instructions, and enters orders (if appropriate) Student sends preceptor note via secure email OR enters in EMR for review Kwan B, Charat S, Wooten D, Hill D, Restaino  Preceptor and student virtually discuss encounter, note, and any medical questions that arose during the visit (either in between patients or after the last patient of the session) In the pre-implementation phase, students gain access to and learn how to use the necessary technological platforms, orient to TM best practices such as appropriate setting, consideration of device positioning, and review of proper "webside manner" (McConnochie, 2019).
During the preparatory phase, students are paired with TM-practicing GIM faculty preceptor. The preceptor selects 1-3 patients for the student to evaluate per session. Students review patient data in the EMR or via notes provided over secure email in advance of the visit. The student and preceptor discuss pertinent medical and psychosocial information that should be addressed for each patient. The preceptor and student plan the agenda and workflow for the visits.
For the active phase, the student is encouraged to log onto the video platform prior to the first encounter. At the start of the visit, the student and preceptor join the encounter synchronously to perform patient safety, contingency planning for technological pauses, and student introduction. The student then executes the agreed-upon workflow of the visit.
In the debriefing phase, the preceptor and student discuss the encounter, documentation, and questions that arose during the visit. Faculty provides feedback to the student about history-taking technique and TM principles. Debriefing occurs between patients or at the end of the session.

Educational Process Insight
Early pilots of video visits allowed clinician-educators and learners to evaluate the feasibility of encounters in an under-developed TM environment, starting with the functionality of technological platforms and support of existing infrastructure, such as information technology bandwidth.
Two distinct TM workflow models arose from our pilot. In the first, preceptors chose to observe trainees throughout the entirety of the TM video visit. While the learner interviewed the patient, the preceptor provided a supporting role, conducting chart review, documentation, and order entry. Additionally, the preceptor could offer suggestions to the learner or pose questions directly to the patient. In the second, the learner conducted the history and physical in the virtual care room independently, presenting to the supervisor who would join the TM visit at a later time. Learner-facing and preceptor-oriented protocols including these workflow models were incrementally rolled-out to additional clinician-educators and trainees (Supplementary Files 1, 2).
We collected informal feedback from participating faculty via email and during routine medical education meetings. The former approach appeared to be favored by providers new to TM video encounters, those that had not yet formed a working relationship with their learners, and learners that were earlier in their training. This workflow model allowed the preceptor to directly observe the learner, develop trust, and provide real-time feedback on the history-taking and assessment. The supervisor could also document and enter orders during the encounter. Disadvantages included the total amount of time required by the preceptor, inability to see other private patients during the student's encounter, limited focus on learner clinical reasoning and presentations, and the potential for the trainee to perceive they have less autonomy in the care of the patient.
The latter model was preferred for more advanced medical students. Preceptors who have formed longitudinal relationships with learners or who oversee multiple learners simultaneously may also benefit in using this approach. Kwan B, Charat S, Wooten D, Hill D, Restaino N, Friedman L MedEdPublish https://doi.org/10.15694/mep.2021.000014.1 Page | 5 Within this model, the trainee can either present to the preceptor "live" in front of the patient (similar to a bedside presentation), or the preceptor and trainee can leave the virtual room to discuss the plan before returning to review with the patient. While there are several advantages to "live" presentations including opportunities for the patient to clarify the history and increased patient satisfaction related to involvement in the care plan (Anderson et al., 2002), inherent challenges include balancing the correction of trainee errors without undermining patient confidence in the learner and difficulty in broaching sensitive topics. Disadvantages include limited observation of learner interactions with the patient, additional complexity in coordinating learner and attending join and discussion times, and decreased opportunities to teach by modeling.
We sent emails to students in the pilot soliciting narrative feedback on the benefits and challenges of TM visits replacing in-person patient encounters. Students found participation in video visits to be an overall positive experience, appreciating the opportunity to engage with patients during COVID-19 and refine their clinical skills. Benefits included increased opportunities for direct observation and feedback by preceptors, while learning a new skill in TM. Drawbacks included limited ability to practice the physical exam, occasional technological difficulties, and adjusting to the virtual workflow (Table 2). "With the temporary suspension of our clinical rotations, medical students across the country have been trying to find creative ways to stay involved clinically... Joining Dr. X on a virtual visit was a perfect example of leveraging technology to do just that" "Remote visits have provided a source of coaching and feedback that otherwise are inaccessible during UCSD's clerkship pause" Direct Observation Increased opportunities for observed traineepatient interactions "Huge benefit was that Dr Y was able listen in to me doing the history/assessment/plan, and could provide more feedback than is usually possible at clinic" "I get more 1:1 feedback time with the attending, and the same amount of time with the patient" Skill Development Opportunity for growth and exposure to TM "Helped me feel a lot more comfortable doing virtual visits in general, since I haven't had exposure to virtual medicine previously in medical school" "More of a quality experience, than a quantity sort of situation, at least as folks are learning. I imagine, my classmates might feel similarly as most were hesitant to try the video visit, seems to be a bit anxiety provoking (understandably), so if we start small and attainable, more folks will surely realize it's not that bad, and perhaps a little fun even" Patient Care Delivery Advantages within video TM encounters "Easier to do medication reconciliation compared to in-person visits since patients have the medications with them at home" "With the virtual visit it was easier to type notes about what the patient was saying without worrying about not having enough eye contact (since it's hard to have complete eye contact using video chat)" Challenges within video TM encounters "Limited ability to do physical exam" "Sometimes I had difficulty hearing the patients because of the audio quality" Kwan B, Charat S, Wooten D, Hill D, Restaino N, Friedman L MedEdPublish https://doi.org/10.15694/mep.2021.000014.1 Page | 6 Comparison between inperson and video TM educational experience Perception of equivalence of TM educational experience "Overall I think it's an equivalent if not better experience than inperson clinic" "I was able to experience the same aspects of being a medical student (e.g. reading patient's chart, interviewing patient, post-visit feedback session with attending)" Perception of difference of TM educational experience "Besides not practicing in person physical exams, the biggest drawback to remote visits has been not being able to step out of the patient room to discuss the case/thought process with my attending." "I found it a bit tough to wait in between patients, you really just stare at your computer or phone, nervous and wondering when you will be called to participate (if your patient is late or cancelled.)"

Discussion
The COVID-19 pandemic has accelerated the adoption of TM in primary care practices, fundamentally altering the way medicine is delivered (Wosik et al., 2020). An important lesson is how widely accepted video visits became among patients and providers, and it is expected that perhaps as many of 30% of all primary care visits at UCSD may remain virtual post-pandemic (UCSD, 2020). Despite our institution's relative inexperience with TM prior to COVID-19, we were able to rapidly implement a primary care TM educational experience that was deemed effective by faculty and learners.
Virtual visits have the potential to increase access to care for some of society's most vulnerable patients including those who are homeless, cannot afford transportation or time off work to attend clinic for a visit, and/or have significant mobility limitations. Recognizing COVID-19's continued impact on societal norms coupled with the increasing acceptance of TM moving forward, educators must swiftly adapt to a "new normal" and rapidly integrate TM into internal medicine training at all levels of learners. As ambulatory clinics have widely embraced the use of video TM visits, it is important not to leave our learners behind.
Narrative comments from students in our educational pilot mirror those in other studies in recognizing TM as a valuable educational tool, not only for development of medical knowledge but also for its potential to improve their ability to deliver patient care (Waseh and Dicker, 2019;Pathipati, Azad, and Jethwani, 2016). Adding TM to already existing curricula may be a challenge, but the urgency has never been greater.
Conducting education via TM allows institutions to respond agilely in the face of uncertainty, providing a valuable clinical experience that circumvents in-person exposure. At a time when most students are disallowed from caring directly for patients with COVID-19, video TM encounters in an inpatient or intensive-care unit setting can give trainees valuable exposure and experience that will form a strong foundation for caring for these patients in the future. Finally, patients and payers will likely demand TM, and it will become an important tool in the creation of a value-based health care system.
Institutions and professional organizations have taken steps to mitigate some of the barriers that had previously limited the widespread adoption of TM, including improvement of technological infrastructure and dissemination of TM resources to providers. We hope that our framework targeting clinician-educators will reduce additional hindrances to incorporating learners in the provision of TM. The lessons learned with medical students can easily be applied to the training of medicine residents and fellows with progressive degrees of independence.