A comparison of medical and pharmacy student perspectives of a clinical interprofessional home-visit versus a virtual interprofessional workshop

Background: No Place Like Home is a clinical interprofessional education (IPE) activity whereby pharmacy and medical students conduct home visits under the guidance and supervision of a clinical preceptor to homebound patients. Purpose: We examined pharmacy and medical student perceptions of mastery of interprofessional competencies during an in-person clinical home visit pre-COVID-19 pandemic versus a virtual IPE learning activity consisting of didactic and case discussions in response to the global COVID-19 pandemic. Methods: We administered the same modified Interprofessional Collaborative Competency Attainment Survey (ICCAS) instrument, which uses a five-point Likert scale, to both the in-person and the virtual IPE students following their learning activity. Results: We received a total of 459 completed survey responses with an overall response rate of 84%. For both groups of students, the in-person format was preferred, however, to our surprise, the results indicated that students in the virtual group reported greater perceived gain in interprofessional skills than students in the in-person group. In addition, pharmacy students perceived greater gain from the interprofessional activity and offered more thoughtful reflections about their experience. Conclusions: Even though both groups of students preferred the in-person visit, the IPE objectives were equally (for medical students) or better (for pharmacy students) absorbed in the virtual environment than the in-person clinical home visit.


Introduction
Health care educators are tasked with creating and delivering discipline-specific curriculum that prepares learners for their field of practice and fulfilling the learning requirements of their institutions. Like most health care fields, pharmacy and medicine have interprofessional education requirements (Association of American Medical Colleges, 2022; American Association of Colleges of Pharmacy, 2022) that must meet the standards of their accreditors. As such, we created the No Place Like Home experience with the intention of meeting Interprofessional Education Collaborative (IPEC) (Collaborative IE, 2016) competencies. While the original in-person clinical home-visit experience was well-received, COVID-19 required a curricular pivot to a virtual workshop platform.

Background
The No Place Like Home (NPLH) activity is an IPE experience whereby medical students, pharmacy students, and either a physician or a nurse practitioner preceptor provide care as a team to patients in their homes (Hatfield et al., 2021). This educational activity was first introduced in 2012 at the Baylor College of Medicine (BCM) and the University of Houston College of Pharmacy (UHCOP) and has consistently been one of the most popular and highly rated IPE activities in our curriculum. This activity has been described in detail elsewhere in the literature (Misra et al., 2016). But briefly, NPLH pairs a medical student with a pharmacy student to spend one eight-hour day making house calls with either a physician or nurse practitioner facilitator (Misra et al., 2016). Early in this experience, we observed very high ratings of "overall satisfaction" associated with the activity. However, through an in-depth analysis of the evaluations, we discovered that although the students were awed by the clinical aspects of the session, they were mostly indifferent to the IPE objectives. Simply providing an opportunity for medical and pharmacy students to interact with each other was insufficient to meet IPE objectives. Thus, we redesigned the activity and reinforced training of the preceptors to focus explicitly on the IPE intentions. These corrections were successful, and the activity has continually met the IPE objectives ever since. The research described in this paper was performed entirely after these curricular modifications were made.
Like all educational activities, some changes have ensued over the years; however, the impact of the COVID-19 pandemic required the creation of a virtual, rather than in-person, curriculum, mirroring the efforts of many graduate medical institutions around the globe. (Sleiwah et al., 2020). The need to implement a virtual NPLH IPE activity was both unanticipated and urgent. BCM clinical educators and IPE experts met to brainstorm and develop a virtual activity that would meet the IPE and clinical knowledge objectives in a safe environment. Though research has shown high levels of student engagement in virtual learning platforms, (Kay & Pasarica, 2019) most notably for their flexibility, (Dost et al., 2020;Kay & Pasarica, 2019) it has also been reported that students worry that remote learning could negatively impact their ability to develop clinical competencies (Huddart et al., 2020). A meta-analysis of blended learning involving both face-to-face learning and e-learning demonstrated consistently better effects on knowledge outcomes when compared with traditional learning in health education (Vallee et al., 2020). Specific to team-based IPE virtual experiences, student attitudes were unchanged when comparing in-person learning activities to a virtual platform (DelNero & Vyas, 2021). One study by Robertson et al. (2021) suggests that IPE collaborative competencies can be met with a virtual platform.
The Interprofessional Education Collaborative (IPEC) is the recognized national authority on interprofessional curriculum. As such, they describe four core domains for IPE curricula: roles and responsibilities, interprofessional communication, teamwork, and ethics and values (Collaborative IE, 2016). We wanted and needed to know if the student learning objectives of the in-person, clinical IPE activity could be met in a virtual environment. Evaluation of delivery formats with the same learning objectives has important implications for medical educators across the county, especially where schools have multiple sites or campuses, as each site may be impacted by different resources. Specifically, the Liaison Committee on Medical Education (LCME) accreditation standard 8.7 states, "A medical school ensures that the medical curriculum includes comparable educational experiences and equivalent methods of assessment across all locations within a given course and clerkship to ensure that all medical students achieve the same medical education program objectives." (LCME, 2021). Therefore, the purpose of this paper is to compare evaluation data from both in-person and virtual IPE experiences to determine if the learning objectives specific to the IPE competencies above can be effectively met in the virtual format.

Methods
This was a non-experimental, cross-institute cohort study. The data was collected using course evaluation surveys that included Likert-scaled items for quantitative analysis and open-ended questions that allow for a qualitative analysis of text comments. Mixed methods (triangulation model) (Schifferdecker & Reed, 2009) were used for data analysis. We obtained approval for this study from the Baylor College of Medicine (protocol number: H-50687) and University of Houston (protocol number: STUDY00003318) Institutional Review Boards.
The in-person curriculum One pharmacy and one medical student are assigned to perform approximately four house calls with a trained preceptor for an entire day. The preceptor is either a physician specializing in geriatrics or a geriatric nurse practitioner. As a team, they visit patients in their homes, addressing medical, functional, and psychosocial aspects of care and offering patient/caregiver education. Opportunities for students to teach each other occur throughout the day and debriefing in the car between visits has proven a valuable part of the experience (Hatfield et al., 2021). Preceptors are directed to ask the medical and pharmacy students questions related to the IPE objectives to generate dialogue and reflection.

Participants
A total of 545 students participated in this project: 276 from Pharmacy (in-person: 125; virtual: 151); and 269 from virtual: 89). They were convenient samples of second-or third-year medicine or third-year pharmacy students who participated in the NPLH program from February 2019 to September 2021 (in-person: February 2019 to January 2020; virtual: October 2020 to September 2021). Other demographic information, such as gender and race is summarized in Table 1.

The virtual curriculum
The house call clinicians created a set of real-life cases, each of which illustrates an aspect of care such as deprescribing, recent hospital discharge, chronic care management, new hospice referral, to name a few. Two groups of two-to-five students each, half pharmacy and half medical, were assigned to a zoom call with one preceptor for a two-hour experience. Fewer resources were required to deliver this curriculum, in terms of time (eight versus two hours), resources (medical equipment and transportation), and number of preceptors (4 versus 2). Three cases were reviewed with the preceptor leading the discussion; every group discussed the transition of care case after hospital discharge, and the other cases were determined by the preceptor to offer some variety among the groups. Finally, to increase active engagement of the students, they were given a case to work on together, with the preceptor muted but observing. Debriefing of how they functioned as a team was deliberate in the final activity of the session.

Evaluation surveys
The evaluation surveys were designed to obtain data on students' perception of their growth in interprofessional skills and the overall learning experience of the NPLH program. For in-person and virtual curriculum, we used the same ten pairs (i.e., before and after participating in the learning activities) of five-point Likert scale items (1: poor; 5: excellent) that focused on IPE objectives and teamwork in a single survey collected after the experience. We included several open-ended questions focusing on strengths, weaknesses, and learning outcomes of the NPLH experience to elicit a deeper understanding of students' experiences. All medical and pharmacy students were asked to complete the survey after the experience, whether it was in-person or virtual.

Data analysis
Quantitative analysis. Descriptive statistics were tabulated by delivery format (in-person versus virtual) and discipline (medicine versus pharmacy). Non-parametric comparison (Wilcoxon signed-rank test) and parametric comparison (paired t-test) for related samples were conducted for each paired item to examine statistically significant differences in students' perceptions of interprofessional skills before and after this learning experience. Further statistical analyses were conducted using independent sample t-tests to examine if discipline (medicine versus pharmacy) or delivery format (in-person versus virtual) had an impact on interprofessional growth. Finally, we created two composite variables: Average-Before and Average-After. These two variables averaged the ten pairs of items for Before and After items. ANCOVA were used to examine all factors in an integrated model. We reported the p-values along with the effect sizes (Cohen's d) for all major statistical analyses. We used SPSS version 28 (IBM Corp, Armonk, New York) to conduct these quantitative analyses.
Qualitative analysis. Qualitative analyses focused on three domains: 1) types of learning that occurred in this experience; 2) superiority of in-person or virtual format; and 3) discipline differences. Two independent reviewers analyzed the text data and reached a consensus on themes for each of the three domains. The text comments were coded based on types of learning, separated by delivery format and discipline. The percentage of themes brought up by students can be compared between delivery formats or disciplines to examine superiority of teaching formats or discipline differences.

Results
Quality of data Pharmacy students, in general, provided higher quality (i.e., more thoughtful and descriptive) text responses to the series of learning experience questions (85%-100% provided text comments). Medicine students provided fewer responses (49%-99%) to text comment questions, but the available responses were sufficient for investigating our research question.

Quantitative findings
We received 459 completed survey responses: 201 from Pharmacy (in-person: 71; virtual: 130); 258 from Medicine (in-person: 176; virtual: 82), with the response rates of 57%, 86%, 98%, and 92%, respectively. The mean and SD for pre-and post-experience of each survey item are provided in Table 2, separated by disciplines and delivery formats. Note. ** indicates group comparison is statistically significant at the 0.01 level; * indicates group comparison is statistically significant at the 0.05 level, "ns" indicates group comparison is not statistically significant at the 0.05 level.

Effectiveness of the NPLH program
For each of the ten pairs of Likert scale questions, regardless of discipline and delivery format, the post perception mean scores of the post-experience were higher than the mean pre-experience scores. We found statistically significant differences in self-perception of interprofessional skills before and after this learning experience (Wilcoxon signed-rank tests: ps < 0.001 for all ten pairs of questions; Paired t-test: ps < 0.001 for all ten pairs of questions). The results provide evidence that the NPLH program enhances students' interprofessional skills, regardless of discipline and/or delivery format.

Difference in disciplines
In general, pre-experience mean scores for pharmacy students were lower than pre-experience mean scores for medical students and, the mean post-experience scores of for pharmacy students were higher than the mean scores for medical students for both in-person and virtual formats.
Pharmacy students' scores demonstrate more perceived gain than those of medical students (i.e., the difference between students' post-and pre-experience) in the ten pairs of Likert-scale questions. A series of independent sample t-tests were conducted to examine discipline differences on perceived gain. For the in-person format, one IPE survey item (i.e., address team conflict in a respectful manner) showed that pharmacy and medical students were statistically significantly different in gaining in IPE competencies (ps: 0.002-0.888, Cohen's ds: -0.060-0.436). For the virtual format, IPE improvement for pharmacy and medical students was statistically significantly different for all ten survey items (ps: <0.001-0.032; Cohen's ds: 0.304-0.576) (see Figure 1 for detailed results).

In-person format versus virtual format
In general, the pre-experience mean scores for the virtual group were lower than those for the in-person group across all ten questions. However, the mean post-experience scores of the virtual group were higher than those of the in-person group. The results might indicate that students in the virtual group reported more perceived gain in IPE skills than students in the in-person group.
More independent sample t-tests were conducted, examining the difference of perceived gain between delivery formats narrowed down by disciplines. The perceived gain in IPE competencies for the in-person and virtual format for pharmacy students was statistically significantly different in nine out of ten survey items (ps: < 0.001-0.127; Cohen's ds: 0.226-0.563) with the virtual method showing superiority. In contrast, the gain in IPE skills among medical students for the in-person and virtual formats was relatively equivalent with no statistically significant differences. The improvement in IPE proficiencies according to discipline and delivery format is shown in Figure 1.

Integrated model
Segregate analyses have shown a consistent pattern regarding the effect of discipline and delivery format. To estimate the impact of these two factors, we generated two composite scores: Average-Before and Average-After. We conducted an ANCOVA using the Average-Before as the covariate, discipline, and delivery format as two independent factors. The results indicate that the Average-Before was a statistically significant covariate (p < 0.001, partial Eta squared = 0.554), discipline and delivery format were both statistically significant factors (discipline: p = 0.001, partial Eta squared = 0.022; delivery format: p = .031, partial Eta squared = 0.010).

Qualitative findings
The findings revealed in the quantitative results are elaborated upon in the qualitative analyses by comparing the difference in coding themes across teaching formats or disciplines.

Student take aways
We coded the responses to the question, "What learning occurred for you in this experience?" Responses were matched to five themes: Communication, Interprofessional perspectives, Geriatric care, Clinical knowledge, and Others. When we noticed that some themes might need to be divided into even smaller categories, we created lower-level categories/themes to display the difference in learning between in-person and virtual formats. For example, for the theme "communication" we noticed that there might be differences in communicating with professionals or patients, so we created three sub-themes: communication with professionals, communication with patients, and communication in general, to capture the difference across teaching formats. Detailed coding results were showing in Table 3.
Which format was preferred, in-person or virtual? For the question "Please comment on the areas for improvement for this learning experience," no student who had an actual home visit proposed to switch this program to a virtual format. However, for those who received the training virtually, 24.1% of students at BCM and 13.7% of students at UHCOP indicated that they preferred to have an actual home visit. The results indicated that for both disciplines, if students were given a choice, students would prefer the in-person format to the virtual format.
"If possible, it would've been helpful to have the experience in person" (from a medical student)

"I would have loved to do this in-person instead of virtual."
(From a pharmacy student) We observed four common differences across disciplines pertaining to delivery format (in-person versus virtual), and the differences for the pharmacy students were more apparent than the medical students.

1) Communication with patients was not applicable in the virtual format because there was no real patient interaction 2) Students have enhanced interprofessional perspectives in the virtual format
3) Specific clinical knowledge pearls were delivered more effectively in the virtual, case-based format 4) Because actual visits provided students with the realworld experience of observing patients' challenges and conditions, students gained a more social-economic perspective and health care system knowledge in the in-person format We observed similar patterns to the questions, "What happened during your experience? What did you observe?" For the in-person format, most students highlighted the home visit. Some students mentioned their observed learning in the home setting related to the medication review and physical exams. For the virtual format, many students mentioned interprofessional collaboration between disciplines. Students also highlighted the case studies and explained what they learned from these cases.

Discussion
We created a virtual interprofessional educational activity for medical and pharmacy students related to home visits of geriatric patients. The goal of this activitiy was to meet the same IPE learning objectives previously approved by our Curriculum Committees during an international pandemic that protected the safety of the patients, learners and faculty. With the virtual curriculum, we continued to collect data which enabled us to compare the differences in perceived learning as reported by the students. To our surprise, the virtual format was rated equivalent if not more effective in meeting IPE objectives. In addtion, the pharmacy students had greater perceived gains in learning regardless of format.
Why were IPE learning objectives more effectively met in the virtual format? Across the entire curriculum, our experience at BCM is that almost all students prefer in-person learning compared to a virtual format. Therefore, we were somewhat surprised to learn that the IPE competencies were more consistently met in the virtual format. This phenomenon may be explained by the fact that the opportunities for non-IPE learning in the in-person home visit offer additional value to the experience but may detract from IPE goals. For example, immersion in the patient's home enviornment may be the first time the students experience the the socioeconomic realities of underservered populations, and this can overshadow the mechanics of the interprofessional team and team members' roles.
Likewise in the virtual experience, the faculty were able to create cases that purposely offered opportunities for interprofessional collaboration whereby students of both disciplines could suggest interventions, which, due to the exigencies of the in-person visit, cannot always be assured. Also, in the virtual format, students are given time to work as a group with minimal or no input from the preceptor. Preceptor involvement during the in-person format tends to be more consistent with fewer opportunities to independently work together. The role modeling of interprofessional teamwork is not always possible in the in-person format as we gather in the patients' homes though it is the culmination of the virtual session.
Why do pharmacy students seem to benefit more from the NPLH program? It appears the pharmacy students responded more thoughtfully to the survey in their text comments and provided more insightful thoughts/opinions. They seem to be more appreciative of the learning opportunities to observe how future doctors react to the same cases. This might be due to pharmacy students being third years and earlier in their training than the medical students who were third and fourth years. In addition, medical students have many more clinical opportunities and patient care experiences with different ages and diagnoses. Therefore, their enthusiasm for the NPLH activity may be blunted, and their perception of change may not be as pronounced as that of pharmacy students, as suggested in the comments below: "…having discussed with non-medical school students, the experience was novel for them. However, for us MS3s [thirdyear medical students], it was superfluous…after many similar activities during preclinical and clinical studies." (from a medical student)

"I think this overlapped strongly with our ARTS class [second-year medical student basic science course on aging], so it felt like more of a refresher than presenting new information." (from a medical student)
Should we replace home visits with a virtual experience?
As we have observed in previous research (Hatfield et al., 2021), in-person interprofessional home visits offer unexpected learning experiences that can overpower student emotions and undermine the IPE intentions. While these experiences have learning value in their own right, we must stay vigilant of the IPE objectives of the curriculum and continue to enforce those objectives through faculty-guided reflection and attention to student perceptions. We are confident that both aims can be accomplished as we have demonstrated in our own experience.
Limitations of this study include the generalizability of our evaluation and research. Our location in a large, urban enviroment and our student demographics may not be reprensentative of a majority of medical and/or pharmacy schools. In addition, not all programs have a county hospital system with a home visit option.
Our results suggest that a virtual session requiring fewer resources and time is just as effective, if not more so, than the in-person home visit in keeping the IPE goals in focus. Yet, as of February 2022, we resumed the in-person curriculum instead of substituting the in-person experience with the virtual session. We appreciate that the home setting offers more expansive opportunities for student growth, including but not limited to IPE alone. There is also the added benefit that students and preceptors alike prefer the in-person method. Going forward, we will incorporate elements of the virtual experience that seem particularly valuable into the in-person home visits (case study, more time to work together independently as a group with minimal preceptor involvement, and reflection on team performance). Our hope is that a blended experience will yield even better learning outcomes and represent the best of both worlds.

Conclusions
The ultimate aim of our research was to determine if a virtual IPE experience could serve as an alternative pedagogy for students to master IPE objectives required by our curiculum committees for graduation. We demonstrated that even though in-person clinical visits were preferred, the virtual experience was equivalent, if not even more effective, toward meeting IPE competencies.

Practice points
Medical and pharmacy students preferred in-person clinical learning for this IPE activity.
Virtual learning employing a workshop can provide an alternative teaching strategy to meet IPE learning objectives.
Meeting the IPE learning objectives in the virtual environment may exceed the learning obtained from the clinical environment.
Although students prefer clinical in-person learning, a virtual workshop may offer a viable alternative pedagogy during times of disruption to curriculum delivery.

Open Peer Review
This research article compares delivery formats by evaluating survey responses of two professions in order to distinguish if one delivery format is superior to the other. Delivery format superiority is determined by specific learning objectives derived from four core domains of the IPE curricula. This research is valuable and relevant to the advancement of student interprofessional learning within the medical field; however, there are a couple of constructive points that are highlighted below.
The research article's limitations have only partly been acknowledged. The article addresses challenges with generalizability due to location and a student demographic; however, it does not address limitations with the data collecting method used. Surveys are a subjective form of data that can be easily skewed. Survey participants may differ in their interpretation of the questions, may select answers without reading the questions at all, or may provide responses influenced by (other forms of) fatigue. Although surveys can provide useful data, it is important to be open about the limitations of the data collecting method before drawing conclusions based on the results. Because the data collection method leads to subjective data, it is important to consistently state that any improvements found were perceived. The article insufficiently addresses that the students' improvements were perceived in the section labelled "Which format was preferred, inperson or virtual". Explicitly stating it consistently throughout the article allows the conclusions to be less misleading.
Another recommendation is to change the verbiage of the umbrella term used to describe pharmacy students and medical students alike. In the article, "discipline" is the overarching term used to describe the two different types of students involved in the study. This is a common use of the term "discipline", but relatively imprecise. The term profession would be a more accurate representation of the two different students. An example of two different disciplines would be an internal medicine physician and a pediatric physician, but an example of two different professions would be a physician and a pharmacist. This article looks at student perceptions of an IPE curriculum in 2 different formats -the first was an 8 hour home visit curriculum and the second was a virtual 2 hour case-based small group session that was implemented in the context of the COVID 19 pandemic. Participants in the experience were clinical level pharmacy and medical students. I am assuming that the learning objectives for both curricular experiences were the same, though this wasn't explicitly stated in the paper.

○
For the evaluation instrument, the authors selected the ICCAS survey which asks students about their skills in 10 different interprofessional competency domains before and after participation in an experience. Additionally there were open ended questions.
○ While the authors are to be commended for using an instrument that was developed previously, it is important to acknowledge that this instrument through its measurement of self-assessed IP competencies is a limitation of this study. We know that self-assessment is not always an accurate measure of skill level. The authors at one point state, "The results provide evidence that the NPLH program enhances students' interprofessional skills, regardless of discipline and/or delivery format." However, I think this would be better phrased as "selfreported IP skills".

○
In the methods section the authors talk about a qualitative analysis, but it appears that they have actually done a comment count of the relative frequencies that specific themes came up. Their "qualitative analysis" mentions 3 domains with themes in each of the domains. It would be helpful to better understand why these 3 domains were chosen. Was there a specific theoretical framework the authors had in mind? ○ For Table 2 it would be helpful to include the Likert scale and to include the stem for the question for all the items.

○
In the section on "Which format was preferred, in-person or virtual" -I find this a bit problematic. It doesn't seem like students were directly asked this question and each group only participated in either the virtual or in person curriculum so hard for them to compare what they didn't experience. I am not sure that it is fair to say that because students who participated in the home visit didn't mention in the comments that they wished it had been virtual is enough to conclude that students preferred the in person version.
○ Also in that same paragraph the authors remark "We observed 4 common differences...." I am not sure where the points that follow come from -were these from the open ended questions or some other source of data that led to these observations. It would just be important to be clear on this point.

○
In the section on "Why do pharmacy students seem to benefit more from the NPLH program?" I found the first sentence a bit odd: "It appears the pharmacy students responded more thoughtfully to the survey in their text comments and provided more insightful thoughts/opinions." I am not sure that the more thoughtful comments means that they benefitted more from the experience. Maybe medical students have evaluation fatigue or ○ I appreciate that the authors sought to rethink a successful interprofessional experience into a virtual one. Along the way, they have gained insights and learnings elaborated in the manuscript that importantly highlight the dynamics of different learner populations, virtual learning, and curricula. I think that in particular the described learning activity brings learners together to learn from and with each other. I wonder if they expressed learners about each other and their respective professions.
I think that the learnings are more controlled in a virtual and case-based setting which yielded the desirable result of more interprofessional learnings compared to a live, in-person home visit. I do echo the authors' elaboration of the richness of such an in-person experience and the ad hoc interactions that could result going to, during, and after the encounters. I wonder what the role and/or expectations were of the pharmacy students in the activity as they likely had less clinical experience compared to their medical student peers. Nonetheless, I think that the virtual learning opportunity highlighted or more likely flattened that distinction by enabled them to focus more on their interactions and teamwork over the socio-environmental facets of an in-person encounter.
I also noted that pharmacy students were supervised by non-pharmacy faculty members (though this was not apparent in the manuscript) and wondered how that aligned with the pharmacy curriculum and expectations. Additionally, I wondered how students were prepared to engage in this experience (virtual and/or in-person). I appreciate that situating the experience in the senior year of medical school might not be ideal and would welcome the authors' continued innovation to right-fit the opportunity in the curriculum.
I look forward to learning from their continued efforts. Thank you.

Is the work clearly and accurately presented and does it cite the current literature? Yes
Is the study design appropriate and is the work technically sound? Yes

Have any limitations of the research been acknowledged? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes