The Use of Smartphones and Mobile Clinical Decision Support Systems in Clinical Clerkships

—Smartphones and Mobile Clinical Decision Support Systems (MCDSS) could help to overcome information overload and support physicians in providing up-to-date medical care. However, would smartphones and MCDSS also be useful for medical students during their clinical clerkships? In this prospective pilot study eight students were asked to use a smartphone during their daily clerkship activities and specifically to evaluate a basic MCDSS. Students were asked to fill out a short survey and exit interviews were conducted. Most students found the smartphone, especially access to the internet, a useful tool during their clerkship. Some students thought that the internet was an easier accessible information source compared to many applications. Several students were not aware of any high-quality applications and most students were not willing to pay more than ten Euros for such an application. In the opinion of most students smartphones should not be used in the presence of a patient. Students expressed a desire for more basic content in the tested MCDSS. In conclusion, smartphones can be a useful tool for medical students during their clerkship. MCDSS for medical students should be designed to their needs. Further research is needed to guide MCDSS development specifically targeted at medical students.


I. INTRODUCTION
Medicine has changed over the past century. There has been an exponential growth of research and knowledge [1][2]. The volume of medical papers published almost doubles every decade [3]. Nowadays, PubMed cites more than 18 million papers [4]. The "gap of knowledge", the gap between what we can learn and what is known, is increasing constantly. Keeping up with the literature has already become an impossible task [5]. Literature alert services, good evidence-based resources and real-time decision support systems can help to overcome this information overload [6].
The use of computer systems that can aid clinical decision making is growing. Clinical Decision Support Systems (CDSS) could generally be described as computer applications designed to aid physicians in making diagnostic and therapeutic decisions [7]. However, CDSS could also be defined as: "providing clinicians or patients with computer-generated clinical knowledge and patientrelated information, intelligently filtered or presented at appropriated times, to enhance patient care [8]. According to Payne [7] there is substantial evidence from trials in a wide range of clinical settings that CDSS may help physicians to provide better clinical care to their patients and according to Garg et al. [9] many CDSS improve practitioners performances.
With the integration of cellular phone technology and additional hardware into Personal Digital Assistants (PDAs), the PDA has evolved into the smartphone [10][11]. The medical community could also benefit from this evolution [11]. Smartphones have the potential to become an important tool in providing up-to-date medical care: from having access to the latest medical research at the point of care [12] to decision support systems that can help to overcome the information overload [6]. In 2009, 64% of US physicians owned a smartphone, and this number is predicted to rise to 81% by the end of 2012 [13].
An increasing number of Mobile Clinical Decision Support Systems (MCDSS) is available to support physicians in providing up-to-date medical care, mostly as applications or so-called "apps" for modern smartphones. These apps are usually developed for physicians. Would they also be useful for medical students who are at the point of becoming a medical professional? Medical students also face the problem of information overload: they are expected to have basic up-to-date knowledge in all fields of medicine. However, they often lack the medical knowledge that is necessary to fully understand how to use a MCDSS made for physicians with far more background in a certain medical specialty. Therefore, medical students may have different preferences for a MCDSS than physicians. The study described in this paper was executed in the context of the clerkship neurology. This lead to the following questions: how do medical students in their clerkship neurology experience the use of a smartphone with a basic MCDSS made for physicians, and what other use has a smartphone for medical students in their clerkship neurology? To our knowledge this is the first prospective study that specifically addresses the needs of the medical student in MCDSS. Furthermore, the results of this pilot study will be used to improve the tested MCDSS.

II. MATERIALS AND METHODS
In this prospective pilot study eight students were equipped with an iPhone during their clerkship neurology. During this period the students were allowed to use the iPhone for work, study and private activities. All iPhones were equipped with unlimited internet use, a limited SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS amount of talk time and text messages, and a selection of pre-installed applications ( Table 1). One of the preinstalled applications about neurology and neurosurgery, was the application NeuroMind (version 1.3), developed by the last author. NeuroMind is a simple MCDSS that offers quick access to frequently used scores ( Figure 1) that can help in decision making. Additionally it contains anatomical images for explanation to patients and students ( Figure 2).
Upon distribution of the iPhones students were asked to use the smartphone during their daily clerkship activities and specifically to evaluate the application NeuroMind. Furthermore, students received a short manual in which some functions of the iPhone were explained. At the end of their clerkship students were asked to fill out a short survey. The survey consisted of 20 items (17 multiple choice questions and 3 open questions); (1) about the content, sort of use, and ease of use of NeuroMind, (2) open questions focusing on which elements students would like to keep or have changed in future versions of NeuroMind, (3) about acceptability of apps in the clinical setting, how much students would like to pay for applications, and which functions students used for their education. After filling in the survey the students were invited for an exit-interview of about 40 minutes, which was conducted with two students per interview and consisted of open questions and some in-depth questions about answers given in the survey. The exit-interviews were recorded on tape after informed consent from the participants, and used for re-listening the participants' answers if this was considered to be of added value. All students participated voluntarily and received a gift card of 30 Euros upon completion of the exit-interview. At the end of the pilot study a quantitative and qualitative data analysis was performed by analyzing the survey results and interview tapes.

III. RESULTS
Eight medical students, four women and four men, participated in this pilot study. All students were fifth year medical students during their clerkship neurology. Five students were six weeks in possession of an iPhone and three students four weeks. Seven participants possessed a smartphone, although none of them owned an iPhone at the start of the pilot study. One student was not in possession of a smartphone, but started thinking about purchasing one after the project.

A. Use of the smartphone and applications
Most students found the smartphone a convenient and useful tool during their clerkship, although most students stated that they used their own smartphones for most functions (except for the application NeuroMind) and not the smartphone they received at the start of the study. The majority of students reported that they used the internet on the smartphone frequently, mostly for quick access to medical information. Some students thought that internet (in particular using Google Mobile) was an easier accessible information source compared to many apps. Even though most students stated that they would like apps on their smartphone to support their education in becoming a doctor (Table 2), several students were not aware of any high-quality apps. Furthermore, none of the students were willing to pay more than 25 Euros for a high-quality app. Two students were not willing to pay any amount of money for a high-quality app, four students were willing to pay between 0 en 10 euro and two students were willing to pay between 10 en 25 euro for a high-quality app.

B. Acceptance of the smartphone in the clinic
In the opinion of most students smartphones should not be used in the presence of a patient for several reasons. Students believed that the use of a smartphone is not accepted by patients, and because the patient does not know what the physician is doing with his phone. One student explained: "I think that patients might not understand. They might think you are texting. That is not professional". Another student said: "If I want to look up something, then I will step out of the room". Most students agreed that if a physician or a medical student would like to use a smartphone in the presence of a patient, then the physician or student should explain what he or she is doing with the smartphone. Futhermore, the majority of the students felt that the use of applications by physicians and medical students is acceptable (Table 2).

C. NeuroMind
Four students have used the application NeuroMind fairly actively. The other four students almost never used NeuroMind and were therefore unable to give any feedback about the app. The opinion of multiple students was that NeuroMind is a useful app, but the information in the app is too limited. The majority of students that used NeuroMind would advise other medical students during their clerkship neurology to use the app. Furthermore, most students liked the navigational structure of NeuroMind. None of the students used NeuroMind in presence of a patient. The students also reported that they rarely changed their actions because of using NeuroMind in the clinic. One example of a student that did change his actions after consulting the MCDSS is a case in which the student had a patient with paraesthesias of the upper extremity. He did not know which dermatome or reflex was involved in this particular situation. After consulting NeuroMind he realized that he tested the wrong reflex, so he went back and elaborated his physical exam. This student explained that the use of NeuroMind had changed his actions and conclusions three or four times during his clerkship. A quantitative summary of the most important survey results is provided in Table 2.

D. NeuroMind strengths and weaknesses
Most students evaluated the anatomical images and the differential diagnosis as the most useful elements of NeuroMind. Almost all students agreed that the differential diagnosis should be extended. A number of students also reported that they would like more anatomical and radiographic images in NeuroMind. Moreover, the students would like diseases described in NeuroMind (no full texts, but preferably in a bullet-point style). Furthermore, all students agreed that providing a search module would be a great improvement.

IV. DISCUSSION
In this pilot study we investigated the use of a smartphone for medical students during their clerkship neurology. Most students in this study found the smartphone a useful tool and frequently used the internet on the smartphone for quick access to medical information. A systematic review by Kho et al showed that PDAs have become a valuable resource for both medical students and residents [14]. An overview by Lindquist et al also showed a positive attitude towards the PDA, which was regarded as a valuable tool for personnel and students in health care [15]. These findings about the use of PDAs among medical students correspond with the results of this study, in which students regarded the smartphone as a convenient and useful tool. Although the reviews by Lindquist et al en Kho et al included merely studies about PDAs and not smartphones as such, since smartphones were not yet developed then, it is reasonable to think that the results about the value of the PDA also applies to the contemporary smartphones.
The students stated that the internet was an easier and faster solution for finding information than the use of apps. However, several students were not aware of any high quality apps. Furthermore, most of them were not willing to pay more than ten Euros for a high-quality application, and none of them more than 25 Euros. This might be in contrast with physicians. In this context, there are two important differences between physicians and students. Physicians have a larger financial capacity compared to students. They are able to buy more expensive applications without exerting their financial means. However, no evidence was found in the literature that physicians are willing to pay more for a high-quality application. Another difference is that physicians are specialists in one field of medicine, and they only have to consider buying high-quality applications in their own field of medicine. This is in contrast to students, who are still exploring different fields of medicine and therefore may want to buy apps in several medical disciplines. Both selection of useful applications by the medical faculty, and possibly financial support, might help to facilitate the adoption of such high-quality electronic resources. Since this will allow students to try out more apps than they would be able or willing to pay for. This makes students more acquainted with these apps and may form a better foundation for their future use.
With regard to the acceptance of the smartphone in daily practice, the students in this study mainly stated that SHORT PAPER THE USE OF SMARTPHONES AND MOBILE CLINICAL DECISION SUPPORT SYSTEMS IN CLINICAL CLERKSHIPS in their opinion smartphones should not be used in the presence of a patient. One of the reasons the students gave was that they believed that the use of a smartphone is not accepted by patients. This is in contrast with existing literature. Rudkin et. al. concluded that patients' perceptions of physicians who use PDAs are neutral or favorable [16]. One might argue that modern smartphones equipped with phone and internet connection might be interpreted differently by patients than traditional PDAs. For example, a patient may think that the student is text messaging with friends instead of taking care of the patient. We did not find any articles that supported this hypothesis, and we did not measure patients' perceptions in this pilot study.
In this pilot study we also focused on the question whether smartphone applications (in particular MCDSS) for medical students should meet different requirements compared to applications made for physicians. NeuroMind was used as an example of such a MCDSS. As demonstrated in the introduction, the definition of (M)CDSS varies in the literature. For this study we applied the definition by Payne [7], which is less restrictive.
The results of this study showed that the students liked NeuroMind, a program originally designed for physicians, but that they preferred more basic information like anatomical images, extended differential diagnosis, and more disease descriptions in the application. This is in accordance with our hypothesis, that medical students have different preferences for a MCDSS than physicians, for example the need for more fundamental information in a MCDSS. The needs from the perspective of a medical student have hardly been reported in the available literature. Further research is needed to guide MCDSS development specifically targeted at medical students.
Another aim of the study was to improve the tested MCDSS. As a result of the outcomes of this study, the decision was made to focus NeuroMind primarily on the original target audience, which consists of physicians. From version 1.5 onwards the "Scores" category has more than doubled in number, and additional anatomical images have been added for explanation to patients and medical students. In contrast, the category "Differential diagnosis" has been discontinued, as it was known from personal communication that it was hardly ever used by physicians. To increase the reliability of the information presented in the MCDSS, references to peer-reviewed literature were added for all scores from version 1.5 onwards. The latest addition is an interactive form of decision support in version 2.0. A separate student version of NeuroMind is under consideration.

A. Study strengths and weaknesses
A particular strength of this study is the target group: medical students in their clinical clerkships. There is limited literature available on how to implement a mobile learning strategy for medical students. Another strength of the study is its prospective character and qualitative information collection and analysis. First the students were asked to fill out a survey, afterwards they were invited to comment on some of their answers and to provide general feedback about the test period with the smartphone.
A weakness of this pilot study is the small sample size, which is aggravated by the fact that the majority of students preferred to use their own smartphone instead of the iPhone with pre-installed applications that was handed over to them. Only half of the students used NeuroMind and were therefore able to evaluate this application. We learned from this pilot study that an effective mobile learning strategy needs to incorporate students' own devices. From a maintenance perspective this can be a nightmare, as many different software platforms (in particular operating platforms) have to be supported. This could be overcome by offering mobile websites instead of "apps", but in general apps offer a much better and more interactive user experience compared to mobile websites. Furthermore, apps can be developed in such a manner that no constant internet access is necessary. On the other hand, the current trend towards "cloud computing" (online information available through web access) may be preferred for rapidly changing information that does not require advanced visualization, and needs to be available on a wide range of devices. Of course, constant internet access is mandatory in that case.
Although we did not collect as much information on the effectiveness of implementing MCDSS for medical students in their clerkships as we intended, we did learn that using pre-installed devices for such a purpose is a recipe for failure, as almost all students are already in possession of such a device. This definitely needs to be considered by other departments who are training medical students and wish to enhance learning by adopting a mobile strategy.

V. CONCLUSION
In conclusion, smartphones can be a useful tool for medical students during their clerkship, but MCDSS made for physicians are not automatically suitable for medical students. Therefore, MCDSS for medical students should be designed to their needs. Although half of the students reported to use the MCDSS as a quick reference, they expressed a desire for more basic content. When developing a MCDSS for medical students one needs to take the willingness of students to pay for applications and their knowledge about high-quality applications into account. Both selection of useful applications by the medical faculty, and possibly financial support, might help to facilitate the adoption of such electronic resources. It seems mandatory to support students using their own smartphone devices instead of equipping them with preinstalled devices. Further research is needed to guide MCDSS development specifically targeted at medical students.