Preparation for clinical clerkships: an evaluation of a continuity clinical experience for M.D./Ph.D. students

Introduction : We compared knowledge and self-perceptions among three groups: M.D/Ph.D. students who attended a continuity clinic; M.D./Ph.D. students who did not attend; and M.D. students. Methods : Group means from a test and a validated questionnaire were compared by ANOVA. Scheﬀe tests compared pairs of groups. Narrative comments were grouped according to key words and themes. Results : Eighty-two (20 M.D./Ph.D. clinic, 13 M.D./Ph.D. non-clinic and 49 M.D.) students participated. The average age was 26. The M.D. group had more females. ANOVA showed signiﬁcant diﬀerences in all questionnaire categories between the M.D./Ph.D. clinic group and other groups (p< .05). Analysis of paired groups showed a signiﬁcant diﬀerence between the M.D./Ph.D. clinic group and the others (p<.05). The test mean was 20.2 for the M.D/ Ph.D. clinic group, 18.4 for the M.D/Ph.D. non-clinic group, and 21.5 for the M.D. group (p<.05). Controlling for gender, questionnaire results were similar but the test score p value was not signiﬁcant. basic science. Conclustions : Although the M.D./Ph.D. clinic students had test scores similar to the other two groups, they perceived they maintained or improved their skills and conﬁdence in caring for patients.

Cumulatively for all pre-clinical courses, the 14,500 students from all U.S. medical schools responded only 2.97 (AAMC 2015). Moss and McManus (1992) surveyed 74 medical students beginning their clinical training and found that their top three anxiety-provoking situations were making the wrong diagnosis, presenting cases, and hurting patients. Of the remaining top 15 anxiety-provoking situations, those pertinent to the ambulatory clinic were dealing with psychiatric patients, making diagnoses and performing rectal and vaginal examinations.
In 1997, the director of the M.D./Ph.D. training program at Einstein, with the cooperation of the Department of Medicine at Jacobi Medical Center, designed an innovative clerkship for M.D./Ph.D. students. Interested students volunteer to attend a continuity general medical clinic one evening a week for at least a year. The goals of this experience for the M.D./Ph.D. student were, and still are, to serve as a bridge between the basic science and clinical years by maintaining clinical skills and confidence, encouraging the development of clinical questions as a stimulus for future research, and improving the students' knowledge of Ambulatory Medicine.
Currently, many M.D./Ph.D. training programs offer students an opportunity to gain clinical experience during the Ph.D. phase of the program. Some have clinics like our Continuity Clinic, while others create ward-based clinical experiences. The durations vary from program to program, some spanning the Ph.D. years with monthly half-day experiences, and others offering 3-6 month weekly or biweekly activities. Relatively few published studies have evaluated the effectiveness of these experiences (Bills, et al., 2013;Goldberg and Insel, 2013;Swartz and Lin, 2014).
Each week in our clinic, a student or faculty member presents a half-hour lecture on an ambulatory care topic, after which the students participate in the patient care session. In the clinic, the students take histories and perform physical examinations on new patients, make decisions about diagnoses and therapies, and provide follow-up care, functioning much like the fourth-year students who rotate through the clinic on their Ambulatory Care clerkships. The attending physicians, a mixture of primary care general internists and internists who are bench researchers, present a model of how doctors from the clinical and research realms can interact to deliver better patient care.

Methods
Rucker L, Steiner-Grossman P MedEdPublish https://doi.org/10.15694/mep.2017.000011 Page | 3 In order to assess students' perceptions of their skills in performing a complete history and physical examination, making diagnoses and clinical decisions, and their overall readiness to start their clinical clerkships, each student was asked to complete a questionnaire modeled on one previously used by the Uniformed Services University of Health Sciences. The questionnaire contained 38 questions asking students to self-assess their competency in taking a history and doing a physical examination, their ability to diagnose and treat common illnesses, and their general readiness to begin the third year.
The questionnaire was scored on a four-point Likert scale (1=poorly prepared, 4=very well prepared) and included seven demographic questions. Students were also given the opportunity to add their narrative comments about the clinical tasks they felt best prepared and least prepared to do, and to list their greatest concerns about beginning the third year as clerks.
The instrument was validated in three ways. The director of the Office of Educational Resources at Einstein helped in refining the questionnaire. It was then distributed to five third-year medical students who discussed the clarity of each question. This step was repeated with a revised questionnaire and another group of six students. The questionnaire was then distributed to 38 learners twice in a two-week period and subjected to Kappa testing to assess the consistency of responses. Kappa results were generally favorable, with two linked exceptions. The questions about diagnosing depression and anxiety had p values of .92 and .77, respectively. These questions were retained because of their importance and because of the difficulty in finding a better way to ask them.
To assess their knowledge of Ambulatory Medicine, the students completed a 40-item multiple-choice test developed and validated by the Clerkship Directors of Internal Medicine (CDIM) subgroup of the AAMC. The timing of the administration of the tests and questionnaires was vital: it had to be done after the end of the secondyear courses but before the students began their clinical rotations. The M.D. group comprised students beginning the orientation for their first clinical clerkship. The M.D./Ph.D. groups were tested the same week or the following week, with the exception of eight M.D./Ph.D. students who were tested the previous summer at the end of their year in clinic. All potential subjects participated voluntarily and anonymously. The test scores did not affect their course grades in any way.
Questionnaire responses were divided into four categories; history/general competency; readiness to begin clerkships and function as a doctor in the ambulatory setting; physical examination; and diagnosis and treatment. Predicting that aggregated responses would assume a normal distribution (univariate normal plots), it was decided to compare group means for the questionnaire categories and the test by ANOVA, using SAS data management software. Scheffe tests were done to evaluate differences between pairs of groups. Wilcoxon rank sum scores were also calculated, as would be appropriate for ordinal data. Kruskall-Wallis testing was preformed when indicated. Responses to open-ended questions were grouped according to key words or themes with the most frequent ones reported.  For the Diagnosis and Treatment category, Scheffe testing between paired groups confirmed significant differences. Wilcoxon rank sum scores and Kruskal-Wallis scores were in agreement. Table 2 summarizes the group means for the questionnaire categories and their comparisons. Because of the poor Kappa values for the question about anxiety and depression, the diagnosis and treatment section was re-analyzed without these questions. Results were similar to those reported in Table 2.

Results
The knowledge test mean was 20.2 for the M.D./Ph.D. with clinic group, 18.4 for the M.D./Ph.D. without clinic group and 21.5 for the M.D. group. The differences among the group means were nearly significant. Table 3 summarizes the mean test scores for the groups. Because of the predominance of females in the M.D. group, the groups were stratified by gender and the analysis rerun for males only. Results of the questionnaire were similar, but results on the knowledge test were quite different. The mean score for the M. were most concerned about their ability to do physical examinations, this concern was not mentioned often by either of the other two groups.
Asked about their greatest concerns in beginning the third year, the M.D. students overwhelmingly cited their anxieties about appearing incompetent or ignorant on the wards, making mistakes or hurting patients, and about personal issues like the long hours and lack of sleep. While the two M.D./Ph.D. groups also worried about making mistakes, the concern they mentioned most often was their fear of losing the basic science knowledge base they had accumulated in the first two years. In addition, the M.D./Ph.D. clinic group worried about staying current with their knowledge of medicine and maintaining the skills they had practiced in the continuity clinic.

Conclusions
The results of the questionnaire suggest that the M.D./Ph.D. clinic rotation succeeded in serving as a bridge between the basic science and clinical years and that participating students perceived that they maintained their skills and confidence in obtaining histories, performing physical examinations, and making diagnosis and management decisions. These results suggest that the students' skills and confidence are greater than those of their peers. In every Rucker L, Steiner-Grossman P category the M.D./Ph.D. students who attended clinic considered themselves significantly more prepared to function in the clinical setting than their counterparts.
Questionnaires can identify anxiety-provoking situations and assess whether students feel prepared, but their utility in assessing students' performance is equivocal. Brinkman et al. (2015) found poor correlation between students' self-confidence in their prescribing skills and their assessed competence. Liaw et al. (2012) found no significant correlation between the self-confidence and clinical performance of nursing students. In Sclabassi and Woelfel's (1984) study of 130 third-year students on an anesthesiology rotation, only 4.6% of their self-assessments were in general agreement with the evaluations prepared by their instructors. On the other hand, Herbert et al. (1990) found significant positive correlations between self-and departmental assessments for 142 medical students during their obstetrics and gynecology rotation. A high degree of correlation was noted between 24 surgery clerkship students' self-assessments and their evaluations by standardized patients in Kaiser and Bauer's (1995) study.
Regardless of the agreement of students' self-assessments with external forms of evaluation, confident students may have an advantage over those who are less confident. Orleans et al. (1980) showed that patients relate more favorably to learners who believe they have improved their clinical skills over the course of a rotation. Still, McCord et al. (1997) found that confident students assigned to teach a topic were no more likely to have accurate information than less confident students. Barr and Graffeo (2016) showed that students who had performed more clinical procedures were more confident than their peers. If perceptions of preparedness translate to confidence, the anxiety experienced by the M.D./Ph.D. students who attended clinic should be lower than that of their peers and the transition to the required third-year clerkships should be easier. Patients may relate even more favorably to these students. Whether scores on knowledge tests predict success in anything other than performance on future tests has long been a subject for debate. A reliable and validated knowledge test seems a useful way to assess knowledge, but not necessarily the skills, attitudes and fortitude required of a good physician.
What is surprising is that the test scores were so similar when stratified to include males only. Herbert et al.(1990) found no gender difference in test scores for their 142 medical students during the obstetrics and gynecology rotation. In our study females raised the mean score for the M.D. group and lowered the mean score for the M.D./Ph.D. groups. Bibbo et al (2015) showed that males scored higher on the MCAT test and the USMLE step 1 test although females scored higher on the OB/GYN NBME subject examination. Dixon (2015) found that among students at an osteopathy school, males had higher MCAT and COMLEX-USA scores, but not better grades or performance on clinical science subject examinations. In addition, since eight M.D./Ph.D. clinic students took the exam the year before completing the questionnaire, it is difficult to relate their group's score to the year they started school. The data do show that in this sample, the M.D./Ph.D. students who attended clinic performed slightly better on the test than the M.D./Ph.D. students who did not. At the very least, the M.D./Ph.D. students who attended clinic appear to have retained more knowledge of medicine.
Why did the M.D./Ph.D. students who attended clinic not do better on the test? One explanation for this finding is that the students no not learn medicine but think they do: they confuse their feelings of preparedness and comfort with the acquisition of knowledge. A more palatable explanation is that the test does not adequately evaluate what the students really learn in the clinic. For example, one test question referred to an 18-year old patient, but very few teens are seen in the clinic. Another question required the interpretation of pulmonary function tests, which are rarely ordered from the clinic. Perhaps a better assessment of performance, for our purposes, would have been to