An Analysis of Anki Usage and Strategy of First-Year Medical Students in a Structure and Function Course

It is well known that undergraduate medical education demands learners to master large amounts of material. To help cope with learning, medical students have resorted to e-learning resources that include video lectures, visual mnemonics, and flashcard systems. The purpose of the current study was to determine the usage of Anki (Damien Elmes, https://apps.ankiweb.net/) among first-year medical students in an integrated first-year module consisting of anatomy and physiology. Additionally, this study aimed to elucidate how students used Anki in conjunction with their lectures. Sixty first-year medical students were recruited in a twelve-week longitudinal study consisting of three surveys (Weeks 1, 7, and 13) about usage of Anki and their perceptions of how to use it with formal lecture. About 70% of participants utilized Anki in the course and the usage increased significantly from Week 1 to Weeks 7 and 13. There was variation to how many days a week students used Anki. Survey data shows that students value formal lectures and prefer to supplement their studies with Anki. A large proportion of first-year medical students use Anki to supplement their studies. Faculty should consider ways to incorporate Anki into their teaching to facilitate student learning through active recall and spaced repetition.


Introduction
Undergraduate medical education demands learners to master large amounts of material at an alarming pace. Additionally, students are learning other skills such as clinical reasoning and physical exam skills that can add to cognitive load. To help cope with learning pace, medical students have resorted to e-learning resources that include video lectures, visual mnemonics, and flashcard systems. Although Kim et al. found that most of the faculty in their study had positive perceptions of e-learning resources, far less incorporated any into their teaching. The barriers to utilizing e-learning resources were lack of resources relevant to lecture, lack of time to use it during lecture and lack of awareness of these resources [1]. Faculty need to understand how pervasive these e-learning resources are and how students are using them, to consider how to help students navigate them or incorporate features into the curriculum. It is known that large numbers of students use these tools to study for the United States Medical Licensing Exam (USMLE) Step 1. However, how students use these resources earlier in the medical school curriculum is less known [2][3][4].
Many e-learning resources are based on established learning principles. For example, Anki (Damien Elmes, https://apps.ankiweb.net/) is a software program that allows users to make digital flashcards, which can consist of pictures, videos, or audio [5]. Its success is based on the scientific principles of active recall and spaced repetition that involves showing new and more difficult flashcards more frequently than easier and less difficult flashcards to promote the "spacing effect" [6]. In 2015, Deng et al. showed that students had a one-point increase on their licensing exams for every 1,700 unique Anki flashcards used which has circulated on the online student forums [7]. There have been other studies reporting the positive effect of question banks on licensing examination performance as well [2,3,8]. The majority of these studies focused on resources students used outside of the formal curriculum. It has been acknowledged that spaced repetition should be included in future medical school education, yet there is limited literature to how [9,10].
Few studies ask students how they use e-learning resources. By knowing how and why students use elearning resources, faculty could integrate these educational tools into formal curricula to enhance learning. The purpose of the current study was to determine Anki usage among first-year medical students in an integrated first-year module consisting of anatomy and physiology. Usage consisted of number of cards, as well as the number of days a week a student utilized Anki. Additionally, this study aimed to elucidate how students used Anki in conjunction with their lectures.

Context and participants
The University of Central Florida's (UCF's) Institutional Review Board determined this study to be exempt. The current study took place during the Structure and Function Module of the UCF College of Medicine Medical Doctor (MD) Program. The module occurred in the first year of the program and ran from October to February. The module was 16 weeks long and included the disciplines of gross anatomy, microanatomy, embryology, medical imaging, and physiology. The pedagogies utilized in this module included lectures, gross anatomy dissections, small group case-based learning activities and high-fidelity patient simulations. Sixty of the 120 medical students (50%) at the UCF College of Medicine were recruited to participate in this study.

Anki usage survey and analysis
Participants were provided a Qualtrics survey at Weeks 1, 7, and 13 of the Structure and Function module. The survey asked students if they used Anki, the numbers of cards per week, and the number of days spent reviewing Anki flashcards. Participants were compensated for each survey of the study and were the same throughout the study weeks. A non-parametric Friedman test with a post-hoc Wilcoxon signed-rank test was used to determine whether there was a significant difference in Anki usage within the sample across the three time points.

Study habit survey and analysis
Participants were provided a Qualtrics survey at Week 13 that asked questions about study habits, how students perceived lecture and Anki, and how they utilized Anki. This 31-question survey used a seven-point scale with 1 = strongly disagree, 4 = neutral, and 7 = strongly agree. After converting students' responses to a numerical value, a T-statistic 99% confidence interval was generated for the sample mean for each response to determine if any of the responses were significant. A significant response is any generated confidence interval that does not include the theoretical mean of 4 (a value where students would neither agree nor disagree with the statement asked).

Results
A survey question asked the number of Anki cards that were used per week to determine how pervasive Anki usage was among the first-year medical students. Figure 1 shows that over the course of the study there was about the same percentage of students (28-29%) that did not utilize the Anki cards. The other 70% or so utilized various numbers of cards per week in Week 1 from less than 100 to over 800 cards. There was a significant increase in the numbers of cards used between weeks 1 and 7 (P = .001). There was no significant increase in the numbers of cards used between Weeks 7 and 13. These data show that most of the first-year medical students surveyed utilized Anki and that the number of cards used increased as the course progressed from Week 1 to Week 7.

FIGURE 1: Number of Anki cards reviewed per week
To further gauge the usage of Anki, a second survey question asked how many days per week the first-year medical students used Anki. Figure 2 shows that the same number of students did not use Anki at all similar to the first question. The other 70% students used Anki for different amounts of days per week with the largest percentage using Anki seven days a week. There were no significant differences in the days per week Anki was used across the course. These data suggest that how students employ Anki cards across the week is variable despite the increasing numbers of cards generated over time.

FIGURE 2: Number of days Anki was used
A seven-point Likert-like survey was deployed at Week 13 to elucidate student perceptions on the usage of Anki. The results are shown in Table 1. Twenty-one out of 31 questions were related to Anki or the use of external resources. Questions 3 to 6 provide data on the relationship of external resources and lecture. These data show that students still perceive value to lecture. However, students perceived that they need to supplement their learning with outside resources as the classroom lectures were not sufficient. This was supported by Question 9 in which students disagreed that Anki was a waste of time.

Mean SD
Lower Limit Upper Limit Significant?

TABLE 1: Survey questions and analyses
Likert scale values: 1 = strongly disagree, 4 = neutral, 7 = strongly agree. A significant response is any generated confidence interval that does not include the theoretical mean of 4 (a value where students would neither agree nor disagree with the statement asked).
Questions 18 to 31 provide perceptions of how students used Anki in relationship to their studies. For Question 19, students disagreed with the statement that Anki was used primarily to quickly cram material. Questions 27 and 28 indicate that students prefer to use Anki cards that were previously developed rather than create their own. Questions 29 and 30 show that students prefer to watch a lecture before doing Anki cards. These data support the thought that students like to supplement formal lectures with Anki cards as opposed to replace.

Discussion
The main finding from this study is that about 70% of the 50% of surveyed first-year medical students used Anki to help them study for the Structure and Function module at UCF College of Medicine. The second finding is that the number of Anki cards and daily use of Anki cards varied among users. Study habit and perception data about external resources suggested that students perceived the need to supplement their learning with external resources, that students preferred to use already made cards as opposed to creating their own, and students preferred to do Anki after watching a lecture.
In this study, about 70% of the student participants used Anki to supplement their studies in our first-year course. There were about 30% that did not use Anki and this remained steady throughout the course suggesting that those students most likely did not adopt this method of study. A recent study provided curriculum specific Anki cards related to anatomy and just over 50% of students used the cards and responded to the surveys [11]. It is important to note that the cards in that study were content related to the class and not focused on board preparation, which is similar in our study. A study by Banos et al. provided class access to a question bank for the initial 18 months of the curriculum without directives and found variable use in their systems-based modules ranging from 22% usage to 94% [2]. Other studies have shown the use of external resources such as First Aid and UWorld's Qbank to be over 99% although that was focused on Step 1 preparation and closer to the licensing exam as opposed to early in the curriculum [8]. Consistent with other studies, our study suggests that Anki flashcards are not used by a portion of students. Further studies are needed to understand why, or why not, a student chooses to use this tool in the early curriculum.
There have been less studies focusing on the motivation of students to use Anki. Surveys have shown that faculty have concerns regarding quality, support and incentives with the increasing prevalence of online learning [12]. Faculty concerns include the loss of the faculty role and the future of faculty [13]. Our data supports the use of lecture which is consistent with studies that show that the large majority of medical students watch or stream lectures by institutional faculty [14][15][16]. Our data suggests that students utilize Anki to supplement their studies as opposed to replace formal instruction. Importantly, students do not feel that formal lectures are adequate for studying (Questions 3 and 4). Our data suggests that students are using Anki to solidify their study efforts as they prefer to use it after lecture. (Questions 29-31) The use of Anki flashcards use active recall as students use them as a question bank, which is the most popular use of electronic online resources [17]. Overall, the survey data suggests that students are using Anki to supplement their classroom instruction because they find it relevant to their future and board exam preparation.
A limitation of the current study is that the data was collected in one medical school and involved a portion of the first-year medical student class. Since it only included one curriculum, we do not know if it is generalizable to the other schools that may have different curricula. There could also be sample bias as students interested in Anki could be more willing to participate. However, with 30% of the participants not adopting Anki usage and most likely never adopting through the study suggest that bias may be limited. This is also a self-reported survey, which may not actually reflect card usage.

Conclusions
The data suggest that large numbers of medical students use Anki for studying anatomy and physiology early in the curriculum. They also suggest that student use Anki to supplement the instruction provided by the faculty as opposed to replacing it. Since Anki is based on active recall and spaced repetition, faculty should consider how best to implement a similar strategy in their classroom to help a certain portions of student studying.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. University of Central Florida's Institutional Review Board issued approval determined this research to be exempt. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.