Learning Technique Utility and Preferences Among Second-Year Medical Students: A Pilot Study of General and Pre-Exam Study Habits

Purpose: The purpose of this pilot study was to assess student preferences for different learning techniques and evaluate the efficacy of those learning techniques by correlating student use to academic performance. Methods: Second-year medical students reported their use of eight learning techniques for general study and preexam preparation through an optional survey during the final organ system course of an integrated curriculum. Optional quizzes were provided to assess academic performance. Results: Students preferred the lowest utility learning techniques, consuming 51% of their general study time. Two of the eight learning techniques, self-explanation and imagery for text, showed greater utility with our medical students than has been previously reported in general and undergraduate education. Significant changes were observed in student preference for three learning techniques as students approached an exam. Preference for highlighting (low utility) decreased, and preference for practice testing (high utility) and self-explanation (high utility) increased significantly during pre-exam study. Conclusions: Although student preference shifted toward higher utility techniques as their exam approached, they still spent more than half of general study time on learning techniques that had strong, negative correlations to


Introduction
Effective study skills are fundamental to academic competence and contribute to student success both inside and outside of the classroom. Study skills include a range of coordinated cognitive skills and processes that enhance the In contrast to the ASSIST instrument, that assesses student approaches and attitudes toward studying, Dunlosky and colleagues explored the efficacy of several learning techniques used by students to improve their academic success across a wide variety of content domains. The techniques assessed were classified by their utility based on their general benefits across several domains, including learning conditions, student characteristics, subject material, and criterion tasks such as outcome measures relevant to student achievement (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013). Techniques that were shown to boost students' performance and benefited learners of all ages and abilities, such as practice testing, were considered high utility learning techniques (HULTs). Low utility learning techniques (LULTs), including summarization, highlighting, mnemonics, imagery for text, and re-reading, did not consistently improve student performance and were difficult to implement. The moderate utility learning techniques (MULTs), including elaborative interrogation and self-explanation, showed promise but had limited evidence in regards to efficacy (Dunlosky et al., 2013). According to Dunlosky and colleagues, two of the techniques most favored by students, re-reading and highlighting text, were insufficient in terms of educational outcomes, including test scores and long-term retention. They also found that two of the most valuable techniques, practice testing and distributed testing, were unpopular among students (Dunlosky et al., 2013). The aforementioned studies targeted learning techniques in general education, and did not focus on medical education.
Effective learning techniques have helped students produce meaningful gains in academic performance (Dunlosky et al., 2013;Hattie, 2009). Although there are several studies that have evaluated the relationship between learning approach and academic performance, very few studies have looked at the potential relationship between specific learning techniques and academic performance of medical students. This study investigated the relationship between learning techniques and academic achievement for second-year medical students at Oakland University William Beaumont (OUWB) School of Medicine. The objectives of the study were to determine the different learning techniques used by second-year medical students at OUWB, as well as determine the relationship between the different learning techniques and academic performance. Ultimately, we investigated our hypothesis that medical students prefer LULTs, and that there will be a change in study behavior as an exam approaches. The study also allows us to compare our results from medical students with those known in general education to identify differences in learning technique efficacy and student preference. Lerchenfeldt S, Nyland R MedEdPublish https://doi.org/10.15694/mep.2016.000096 Page | 3

Participants
Second-year medical students enrolled in the Psychopathology course at OUWB were given the opportunity to participate in the study. For inclusion in the study, participants had to complete a consent form, 3-item learning techniques survey, and at least one practice quiz. All participants consented to the use of their data from the learning techniques survey and quiz scores.

Study Design
Students were introduced to the study at the beginning of the two-week Psychopathology course. A learning technique survey was made available to participating students at the beginning of the course. During the course, two optional quizzes were available through the school's learning management system, based on the material discussed in four pharmacology lectures given during the course. Student responses on the learning techniques survey were analyzed and correlated with their scores on the pharmacology practice quizzes (academic achievement). Students were given an additional two weeks after the end of the course to complete the informed consent, survey, and quizzes. The Institutional Review Board at Oakland University approved the study design.

Survey
The survey contained eight of the common learning techniques discussed in the paper by Dunlosky and colleagues (2013). The definition for each learning technique was provided on the survey (Table 1). Students were asked about their study habits during the majority of a hypothetical course in medical school (General Study), and during the time immediately preceding an exam (Pre-Exam Study). The pre-exam study period was self-defined by the respondent. For general study and pre-exam study, students were asked to estimate, in 10% increments, how much of their study time was devoted to each learning technique. An area for free text was provided for students who used other learning techniques not listed on the survey.

Learning Technique Definition
Practice testing Self-testing or taking practice tests over to-be-learned material Re-reading Restudying text material again after an initial reading

Self-explanation
Explaining how new information is related to known information, or explaining steps taken during problem solving Summarization Writing summaries (of various lengths) of to-be-learned texts Note. Definitions were taken from Dunlosky et al. (2013)

Quizzes
To provide a measure of academic achievement, optional quiz questions were provided for all students over the material discussed in each of the four pharmacology lectures. Topics covered in lecture included antidepressants and mood stabilizers, antipsychotics, anti-anxiety medications, and drugs of abuse. Each quiz contained 10 National Board of Medical Examiners (NBME) style questions written by pharmacology faculty at OUWB. To test long-term knowledge retention, some questions also required knowledge from past courses. The practice quizzes were not timed and did not count toward the students' final grade.

General and pre-exam study time
For each of the eight learning techniques, univariate linear regression models were run. Percent of time using the specific learning technique was the independent variable, and average quiz score was the dependent variable. All analysis was done in STATA 13.

Effectiveness of learning technique
For each of the learning techniques, online quiz scores were correlated to the amount of study time devoted to the individual technique. Correlations between quiz score and learning technique use were made for general study, preexam study, and total study time. Learning techniques with a negative relationship between use and quiz score were categorized as LULTs. Those with a positive relationship between use and quiz score were categorized as MULTs and HULTs.

Time spent by students using various techniques
Percent of time devoted to each learning technique was compared for general study and pre-exam study using a twotailed t test. Descriptive statistics were calculated, and statistical tests were performed. Of the 94 second-year students, 21 completed all necessary requirements to participate in the study, including the consent form, survey, and practice quizzes (Table 2).

Learning technique utility
In this study, four learning techniques showed negative correlations between overall time devoted to the technique and quiz performance, two learning techniques showed moderate, positive correlations, and two techniques showed strong, positive correlations ( Figure 1).

Change in utility with respect to time of use
Learning technique utility was compared for general study and pre-exam study ( Figure 2). Five learning techniques were found to have very similar utility during both general and pre-exam study time. These include elaborative interrogation, highlighting, keyword, self-explanation, and summarization. Three learning techniques showed a major shift in utility based on when the technique was used (Figure 3). Rereading has a small negative correlation to quiz performance when used in general study, r (19)

Figure 3
Change in learning technique utility relative to time of use. Three learning techniques showed a significant difference in utility when used during general study compared to pre-exam study. Techniques are categorized as low utility (light grey), moderate utility (striped), and high utility (dots).

Time spent by students using various techniques
When evaluating the utilization of low, moderate, and high utility learning techniques, imagery for text and self- explanation were reassigned based on efficacy in this population. Imagery for text was reassigned as a MULT, and self-explanation was reassigned as a HULT. Table 3 shows that more than one-half (51.1 %) of general study time was dedicated to highlighting, keywords, re-reading, and summarization. Less than one-third (32.5 %) of general study time was devoted to the highest utility techniques, practice testing and self-elaboration. The majority of preexam study time (46.7 %) was spent on practice testing, and 37.7 % of pre-exam study time was devoted to the lowest efficacy learning techniques.
Three of the eight techniques showed a significant (p = 0.05) shift in utilization between general study and pre-exam study (Table 4)

Discussion
Several studies in medical education have investigated learning approaches, including the relationship between study strategies and academic achievement. Many have investigated the learning approaches originally derived by Marton and Säljö (1976) and further developed by Entwistle and Ramsden (1983), comparing the deep approach to the surface approach. Students using the deep approach seek meaning in material, relate new ideas to previous knowledge, and use evidence critically, whereas students using the surface approach only memorize the material without a true understanding ( approaches used by medical students, our study evaluated student preference for each of the specific learning techniques, as well as the utility for each technique. Our study investigated the theory that study habits of medical students are similar to students in general, in which they spend the majority of their study time using low utility learning techniques. We surveyed second-year medical students on their use of eight different learning techniques previously evaluated by Dunlosky and colleagues; we also evaluated the learning technique utility (Dunlosky et al., 2013). Of the eight learning techniques evaluated in our study, the utility for six techniques were consistent with the literature. For example, highlighting/underlining, keyword/mnemonic, re-reading, and summarization had negative correlations, whereas elaborative interrogation had a moderately positive correlation and practice testing had a highly positive correlation with academic performance. There were two clear differences with learning technique utility of medical students versus the general student population.
The first difference occurred with self-explanation, a technique in which students describe their thought process and integrate new information with known information. Self-explanation was identified as a high utility technique in our study, although it was listed as moderate utility in the literature. A possible reason for this difference may be that self-explanation has a positive impact on development of clinical reasoning and problem solving skills, both of which are key competencies for medical students. Studies have shown that self-explanation is an effective technique for solving clinical cases at the clerkship level (Chamberland, St-Onge, Setrakian, Lanthier, Bergeron, Bourget, Mamede, Schmidt, & Rikers, 2011;Chamberland, Mamede, St-Onge, Setrakian, Bergeron, & Schmidt, 2015). According to Chamberland and colleagues, self-explanation for clinical reasoning works better with less familiar cases as it provides opportunities to link biomedical to clinical knowledge and leads to a more coherent representation of disease (Chamberland et al., 2011(Chamberland et al., , 2015. Although this was shown for third-and fourth-year medical students, similar findings may apply to first-and second-year students as well; especially since most case scenarios and patient problems are unfamiliar territory for new medical students. Even in the first two years of medical school, students are required to analyze clinical scenarios to reach correct diagnoses and treatment strategies. Most quiz and test questions assess the students' skills in interpreting patient data and making decisions, which may be why self-explanation, a technique that enhances clinical reasoning and problem solving skills, was shown as a high utility in our study. The second difference between the general student population and medical students occurred with the efficacy of imagery for text. According to Leutner and colleagues, constructing mental images has a positive impact on text comprehension and learning when it is performed directly while reading the text (Lautner, Leopold, & Sumfleth, 2009). Students who use imagery for text are able to form mental images of text materials while reading or listening to study material. Although it was considered a LULT among the general student population, the results from our study indicated that it was a MULT for medical students. Imagery for text may be more effective for medical students for several reasons. One potential reason is the vast variety of educational tools and proprietary resources available to medical students that turn textbook information into memorable images and stories. Another possible explanation for the efficacy of imagery for text in medical school is the material taught in medical school might be more easily described or represented by pictures. For example, developing a mental image of a nephron and its mechanism in body fluid regulation may be easier than developing an image for the conventions of Standard English grammar. One interesting outcome with imagery for text was that the technique was ineffective during general study time, but highly effective prior to the exam, showing that students may need to have a deeper understanding of the material prior to developing mental images of text materials.
Currently, there is limited research looking at the impact of re-reading on knowledge retention and educational outcomes; the studies that have been completed have shown that re-reading is generally an ineffective learning technique (Dunlosky et al., 2013;Callender & McDaniel, 2009). For example, a study by Callender and McDaniel found that re-reading and reading comprehension ability did not significantly increase performance on assessments (Callender & McDaniel, 2009). Our study showed that re-reading was ineffective during general study time and prior to the exam, although the ineffectiveness was much more evident prior to the exam. One potential reason for this finding could be due to differences in the processes required to learn and retain information. For instance, according to Millis and colleagues, initial reading of the text is used to establish surface levels of representation, in which the second reading is used to help build a deeper understanding with the use of mental representations (Millis, Simon, & ten Broek, 1998). Generally, students reread to gain a better understanding of the material, as well as to improve their memory of the information stated in the text. Medical students who used re-reading during general study time may have gained a better understanding initially, and therefore could use more effective study strategies, such as practice testing, prior to the exam. Medical students who chose to reread prior to the exam may not have had a clear enough understanding of the material, therefore they chose to reread instead of using more effective study strategies before the exam.
Dunlosky and colleagues ranked practice testing as a high utility learning technique, as it had broad applicability, proven effectiveness in educational contexts, and positive outcome measures and retention intervals (Dunlosky et al., 2013). One theory, by Carpenter and colleagues, stated that practice testing is effective because it can enhance retention by triggering an elaborative retrieval processes (Carpenter, Pashler, & Cepeda, 2009). Other evidence has suggested that practice testing can enhance how well students mentally organize information and process information, which can increase knowledge retention (Dunlosky et al., 2013;Hunt & Worthen, 2006). Our study also showed that practice testing was a high utility learning technique, although practice testing was most effective prior to the exam, compared to general study time. Practice testing may be effective for medical students for several reasons, including the fact that several banks that provide questions similar to those seen on their exams are readily available at all times. One potential reason why practice testing is more effective prior to the exam may be because students have a deeper understanding of the study material prior to the exam and are able to develop more thorough connections between different areas of the study material.
Our study also found that the learning techniques used by our students were similar to those used by students in general (Dunlosky et al., 2013). Consistent with studies of undergraduate students, medical students surveyed at OUWB reported spending the majority of their general study time using the least effective techniques, including highlighting and re-reading. However, there was a dramatic shift to the use of practice testing, a HULT, during their pre-exam study time, in which the majority of time was spent on practice testing (46.7 %). Even with this shift, a significant amount of pre-exam study time (37.7 %) utilized the lowest efficacy learning techniques. These results support the inclusion of study skills workshops to enhance their current study skills, as well as assist in the development of new, more effective study strategies.
A study skills workshop combined with continued reinforcement of effective study practices, may be a potentially useful strategy to boost confidence and increase student success, while educating them about effective and ineffective study techniques. In addition to providing insight about the distinctions in learning technique utility and preferences among medical students, the surveyed students will also provide a baseline reference for studying barriers to changing student study habits. We have recently implemented a study skills workshop during the first week of medical school; as it is a new addition to the curriculum, the surveyed students did not participate in the workshop. The study will be repeated with students who have taken the study skills workshop, in an effort to continually improve the workshop, as well as identify barriers that prevent students from implementing higher utility learning techniques.

Conclusions
Overall, our study showed that the learning techniques used by second-year medical students are similar to other reports, but with key differences. Like the general student population, second-year medical students continue to use the least effective study techniques during general study time, such as highlighting and re-reading. Although the utility for several of the learning techniques was similar to previous reports, our study showed that the utility for two of the eight techniques varied from the literature, which may be a distinction from studies in general education. Our data show self-explanation is a HULT, compared to the literature where it is labelled a MULT. We identified imagery for text as a MULT in our study, compared to its designation of LULT in the literature. This study provides a baseline reference on the distinctions in learning technique utility and preferences for medical students who were not given an opportunity to take the OUWB study skills workshop. Our goal is to repeat the study with students who received training about learning techniques during their first week of medical school, and further investigate our hypothesis that self-explanation is a high utility learning technique for medical students. We also hope to assess the impact of the study skills workshop provided at OUWB on student success, as well as identify the barriers that prevent students from using higher utility learning techniques.

Limitations
The authors are trained in writing NBME-style questions, and the quiz questions were representative of assessment items used throughout the first two years of medical school at OUWB. However, the quiz questions only covered pharmacology, and therefore may not have provided a comprehensive overview for the efficacy of each learning technique. Having a small sample size was also a limitation; this population may not represent the entire medical student population. Future studies will include a larger cohort of students and access to other indicators of academic performance including quizzes and exams from other courses, scores from standardized NBME exams, and undergraduate grades.

Take Home Messages
During the majority of study time, medical students use the least effective study techniques, including highlighting and re-reading. For medical students, there is a distinct shift in use of learning techniques during pre-exam study time, in which the majority of time is spent on practice testing. Despite this shift, students still spend a significant amount of pre-exam study time using low utility learning techniques. Imagery for text was identified as a moderate utility learning technique for medical students, indicating that constructing mental images during study time has a positive impact on text comprehension. Self-explanation was identified as a high utility learning technique for medical students, as it may aid in the development of clinical reasoning and problem solving skills. A study skills workshop that educates medical students about effective and ineffective study techniques, with continued reinforcement, may be a potentially useful strategy to increase student success.