My personalised App: A Tool for Improvement of clinical Sessions. Usability and satisfaction of medical students

Background: Bedside-teaching provides a valuable opportunity for medical students to learn clinical-practical skills. Optimal preparation by students is of critical value. We have developed a speciﬁc mobile "learning App" that oﬀers students, course content and guidance on examination techniques in a preclinical physical examination course. In addition, learning objectives were stated which support preparation as well as provide relevant links to supplement further learning. The following is an evaluation report on the implementation of this course-speciﬁc App. Methods: A questionnaire, (comprising a 5-point Likert scale and two open questions) was used to evaluate the usefulness, versatility and satisfaction with the App. One hundred seventy-ﬁve students had the opportunity to use the App in parallel with a clinical examination course. This course concept, the development of the learning app and, the results and conclusions of the questionnaire are presented. Results: The App was favourably received and rated as helpful. Students requested further use with more detailed information. Students ranked the links to further resources as very favourable. Students should be informed about the use of smartphones and their applications for learning purposes in medical education. Conclusion: Students received this mobile support in learning clinical-practical skills positively, and results show there is increasing demand for this form of assistance. Our evaluation shows that the usability and the possibilities of mobile learning should be well established, well communicated.


Introduction Background
Bedside-Teaching is a valuable and useful method to impart clinical, practical and communicative skills to medical students. Teaching with real patients is, however, often unpredictable for lecturers and students and makes appropriate preparation difficult (Qureshi, 2014). Further, students often have different preconceived ideas regarding required knowledge and expected ability (Ramani and Leinster, 2008). Shorter patient hospital admission periods and increasing faculty workload provide obstacles to implementing optimal and structured courses (Garout et al., 2016). Despite clinical teaching being a valuable resource, this unpredictability is associated with complex challenges (Peters and ten Cate, 2014). Students should maintain a central role in their learning, and through optimal preparation, they can maximise their learning effect (Garout et al., 2016). Optimal preparation includes being informed on relevant learning objectives and course content, being able to retrieve and apply this information on demand and the ability to deal with the material during the course.
Accessing necessary information and learning aids during practical teaching sessions can be challenging due to changing geographical locations. Smartphones provide a convenient solution to this problem because of their immediate availability with constant access to information (Payne, Wharrad and Watts, 2012). Smartphones today are virtually indispensable in our everyday lives. Students also use tablets and laptops in lectures and seminars regularly (Patel et al., 2015). Mobile learning is intended to make knowledge acquisition independent of time and place, but it can also aid in acquiring accurate, instant information (El-Hussein and Cronje, 2010).
Students report the possibility of mobile learning as helpful and suitable for improving learning (Lee Ventola, 2014;Green et al., 2015). Usually, Apps or internet sources are used for literature research, drug reference information or medical calculators (Mosa, Yoo and Sheets, 2012;Short et al., 2014). The challenge appears to be not the availability of information but the breadth of knowledge available, i.e. the selection of a "good App" (Wallace, Clark and White, 2012). The search for concise but appropriate information, to cement clinical concepts, is often tedious and timeconsuming, especially when the aim of electronic information is precisely the opposite.
With the Learning Toolbox (LTB) © platform (Raycom BV, Utrecht, Netherlands), there is a customisable application that enables, supports and facilitates mobile and context-based learning. LTB allows the creation of course-specific learning content in the form of text and image-based illustrations. The learning App offers the possibility to retrieve or refresh concrete examination techniques and clinical information before, during and after the course. This convenience uniquely addresses the unpredictability of bedside patient contact in clinical sessions. In particular, unpredictable or spontaneous patient encounters can be better structured to student requirements, thus facilitating and maximising learning.

Course description
As part of a structured curricular "Clinical Examination" course offered in the first clinical semester at the medical faculty of the University of Duisburg-Essen, the learning App was implemented in the winter semester 2017/2018. One hundred seventy-five students participated in the course and sessions took place on two afternoons per week for twelve weeks. The course was offered at multiple addresses in various affiliated teaching hospitals in the Duisburg-Essen area. Students learned history taking and structured basic examination techniques for different specialities from neurology to the musculoskeletal system. All sessions occurred directly at the bedside using mostly in-patients or occasionally simulated patients.

Development of the LTB-App
A mobile learning Application (learning App) Learning Toolbox (LTB) © was designed, specifically for students taking this course. The App facilitates course preparation and provides on-demand assistance to students. Consistency and user-friendliness were maintained by creating a repeatable, fixed format and structure for every organ system. Links to publicly available online content were also included, such as examination videos or textbook content and references. These encouraged a more detailed engagement with various topics. The contents of the learning App included learning objectives, essential course content, short tutorials on practical skills and aids for the implementation in the course. In addition, the App allowed the review of student knowledge through a voluntary online quiz tailored to the course and App content.

LTB in the course
The learning App was developed based on the agreed course content. We decided on specific content and learning objectives following discussion and guidance with representatives from each speciality involved. The display content was consistent with course-scripts which were used throughout all relevant specialities. Essential examination techniques were divided into headings, which correlated directly to the structure in which the course was taught again, aiming for standardisation and consistency. (Figure 1).

Figure 1: Home of LTB -Clinical Examination Course with various organ systems.
We developed content themes such as history taking, cardiovascular, respiratory musculoskeletal and vascular examinations. In specific areas, presentation of the individual sub-topics pertaining to an examination was always listed in the same format ( Figure 2). With the help of short descriptions, listings and graphics, we designed the App to be as concise as possible. The intention of the App was not to replace textbooks but to offer the opportunity to refresh previously acquired knowledge or to ensure the correct conduct of an examination. We informed the students of the aims for App use during the usual course orientation session. A list of these intentions for use was also provided on the course Moodle platform. To offer an additional incentive to learn with the App, LTB was linked to an online quiz. Students were able to answer multiple-choice (MC) questions regarding the content of the course. Immediate feedback and answers were included for every test item. On conclusion of the test, the overall result was immediately provided. The use of LTB was voluntary and free for students.
As part of the orientation seminar at the beginning of the semester, we informed the students on the availability of the App and its recommended application. We presented the App as a new concept, which would be regularly developed and renewed over time. Students were provided with a direct access link with smartphone installation instructions (Android® or Apple iOS®) via a QRS code or website address. Additional instructions for troubleshooting were also provided should access be difficult. A direct web link was also provided by the learning management platform of the course via Moodle.
Following the course exam, a questionnaire survey in paper-pencil format was performed to evaluate the usefulness, flexibility and satisfaction with the learning app. Two open questions were asked to allow for individual feedback. Questionnaire participation was voluntary. The students were asked to assess different points, using a 5-point Likert scale (1 = strongly disagree 5 = completely agree). Technical aspects and depth of content desired were evaluated. Patterns and frequency of use of the App, such as whether LTB was used more at home or on the go were also assessed. In the two subsequent open questions, feedback on usability and miscellaneous issues were requested.

Results
Questionnaire (5-point-Likert-scale) 174 (99.4%) from a total of 175 questionnaires were completed. Results showed that LTB was used by 60% of the Herbstreit S, Mäker D, Szalai C MedEdPublish https://doi.org/10.15694/mep.2021.000153.1 Page | 5 students occasionally to frequently (Table 1). The majority of students who commented that they had not used the learning App, reported that they did not know about it or had forgotten about it. Some students said that they had had no time to use the App. LTB access was reported as usually error-free (4 ± 1[median ± standard deviation]). Most of the students recommend the App (4 ± 1). The students requested more explicit content (4 ± 1) from the App. The students stated that the learning app had helped in the practical implementation of examination techniques in the course and had also contributed to adequate exam preparation (3,5 ± 1,2).
LTB use was occasionally used in the home setting (3 ± 1) and somewhat less often on route to the course. In the course or usage on the hospital site, was reported as minimal (1 ± 1). The individual sections of the App, tailored to the themes and course content (such as history taking, abdominal examination, cardiovascular examination, respiratory and musculoskeletal systems) were used equally frequently. A slight focus was on the links to the videos with examination techniques was reported (3 ± 1,4).

Open questions (usability and miscellaneous comments)
Students especially highlighted Mobile usability in the free comment section. Comments such "mobile learning and viewing", "fast access on the go", "Always ready, mobile, modern, exciting, up-to-date", "always at hand on the mobile phone and usable on the train", prove that the app was used in the desired manner as a portable learning aid.
The technical implementation was reported positively with comments like "fast and convenient Operation", "easy installation and handling", "fast and convenient". The links were described as "Easy, quick access to knowledge ..." and "easy to navigate". General comments like "everything in a nutshell", "very well structured", "Instructions ... step by step", "... good explanations, the essentials summarised", "a lot of information clearly presented" and "good for in-between practising", reflected a positive evaluation of the App content. Linking to online videos to the most common examination techniques was highlighted as helpful. When asked about further comments on LTB, the students requested more content and visual aids such as videos. Students commented that not all topics of the examination course were available. Difficulties with access or other technical defects were rarely mentioned.

Discussion
The use of smartphones or tablets in medical education has rapidly increased (Payne, Wharrad and Watts, 2012;Shenouda, Davies and Haq, 2018). Evidence shows that around 80% of medical students own smartphones (Jebraeily, Fazlollahi and Rahimi, 2017). However, these apps focus predominantly on medical conditions, drug information, communication and documentation, e.g. through the management of timetables and lectures (Wallace, Clark and White, 2012;Robinson et al., 2013;Shenouda, Davies and Haq, 2018).
Studies show that students are receptive to the concept of smartphones as a mobile learning aid, especially with direct application to curricular events (Robinson et al., 2013). Although information about the learning app was given in the course orientation, and all corresponding information was additionally provided in the Moodle classroom, LTB was only used by 60% of students occasionally to frequently. This frequency may reflect unfamiliarity with a mobile platform as a learning adjunct. However, as this was the first presentation of a coursespecific app to the medical students at the University Duisburg-Essen, the initial reception may be interpreted as more than satisfactory. The students who used the learning App, strongly recommend LTB. Students felt supported by its focus on practical course implementation and in so doing, felt prepared for the course.
Although the students have recognised LTB as a mobile learning aid, they rarely used it on-site and only occasionally at home. This may be a result of direct discouragement of student use of mobile devices at the bedside as well as easier access to more comprehensive resources at home. Students may also perceive the usage of a mobile device at the bedside as a sign of being less competent despite believing that usage increases patient care (Quant et al., 2016). Investigations show that the use of smartphones in the clinical environment can also be considered irreverent or rude (Robinson et al., 2013;Wu et al., 2013). It is feared that the use of smartphones in a clinical context could be too distracting (Visvanathan, Gibb and Brady, 2011;Gill, Kamath and Gill, 2012;Wallace, Clark and White, 2012). However, recent studies show that patients tend to be more neutral or positive about the doctor's use of smartphones, especially if they are informed about it or directly involved (McCord et al., 2009;Asan, Tyszka and Fletcher, 2016).
Research also shows that there is increasing use of personal smartphones in the workplace as a tool in enhancing patient care (Patel et al., 2015). The use of smartphones as a supplement to learning activities in a clinical setting was also rated as positive. This positive reaction was due to the rapid access and deepening of knowledge through repetition. However, the interruption of communication with the patient is viewed in a somewhat negative light (Davies et al., 2012). Within the learning context, the smartphone is used by students as a reference book with the search function. Many commercial apps are available but are often costly and not tailored to specific faculty courses. Often there are also concerns about their validity and accuracy (Jebraeily, Fazlollahi and Rahimi, 2017) and therefore, their use is often only limited (O'Neill et al., 2013). Technical issues include the download of large volumes of data ((Mos, Yoo and Sheets, 2012). In the context of clinical-practical medical education, learning apps can be used to complement classroom teaching (Davies et al., 2012) and even improve learning practical skills (Low et al., 2011).
The App developed by us is directly tailored to a curricular course and can therefore only be used to a limited extent at other faculties. Students at our university used the App as preparation for their exams. This trend is also consistent with international research, showing increasing use of internet adjuncts for exam preparation (Payne, Wharrad and Watts, 2012). We were unable to find data for smaller-scale apps which were used within specific university curricular courses. In our study, a relevant proportion of students wished more in-depth or more content in the App and reported intermittently using other sources. However, it is essential to emphasise that the LTB is suitable for supporting the specific clinical examination course and providing expedient access to crucial information on course content. On the go access allows deepening of clinical knowledge, supports practical skills and where appropriate, helps to fill in waiting times. The App should serve as an adjunct to and not replace a comprehensive textbook. Thus, students must be informed on efficient App use in advance.

Conclusion
Students received this mobile support in learning clinical-practical skills positively. Results show there is an increasing demand for this form of assistance. Our research shows that the usability and the possibilities of mobile learning should be well established, well communicated. A specific course-based learning app can assist in the development of clinical-practical skills through mobile access to related course content and facilitates course preparation. Whether the App improves learning these clinical skills requires further research. Initial evidence suggests that positive effects on learning and competence can be seen (Cook et al., 2008).
An inference to long-term or deep learning is not possible at this time and has highlighted areas where research projects can be undertaken. We plan to perform a one-year longitudinal study assessing the repeated use of the App. Accessing learning apps on the smartphone, in the framework of clinical-practical teaching at the patient's bedside, should be communicated to tutors, patients should also be informed of their use. Direct patient communication should not be interrupted by smartphone use. Due to the favourable initial reception, similar Apps have been developed for use with various other courses (such as Anaesthesia and Emergency medicine) and are offered as part of the Faculty of Medicine eLearning program at the University of Duisburg-Essen.

Take Home Messages
My personalised App is a provides mobile support for clinical-practical learning. Digitalisation in medical studies is essential for the progressing technical development in medicine.
Mobile App provides course content and guidance on examination techniques for a clinical examination course. Our research shows that the usability and the possibilities of mobile learning are very useful in clinical skills. The App was favourably received and rated as helpful in preparation for university courses.