Design and Evaluation of Mobile Clinical Education via Interactive APP

Background: To design and evaluate the feasibility and ecacy of Mobile Clinical Education (MCE) based on the intelligent and social mobile app. Candidates: 181 students from Nursing Department of Nanjing Medical University. Methods: Those students were previously assigned into two classes based on their admission evaluation. MCE was randomly assigned for one class including 97 students, while control group had 84. The MCE app was developed and applied for Nutrition course during the whole semester, while the control group was taught by the traditional method. Results: MCE group had better performance for the theoretical test scores (t = 1.127, P = 0.26) and he nal evaluation (t = 7.419, P < 0.001). Based on teaching evaluation questionnaire, students under MCE presented more positive feedback: active communication, personal participation, update for the cutting-edge development, satisfaction of teaching quality, and overall course satisfaction. Conclusion: Mobile Clinical Education (MCE) improved the quality and ecacy of course teaching. With real-time and social advantage, MCE might arouse learning interest, bring various strategies for teaching theoretical knowledge and practical operation, and engage learning communication and collaboration. Mobile Clinical Education should be recommended for the future teaching and training.


Background
Medical education always includes complex course content, cutting-edge academic update and practical participation. Problem-Based Learning (PBL), Clinical-Based Learning (CBL) and other teaching methods were recently introduced to improve lecture modality and promote teacher-student interaction 1,2 . These methods cannot apply for the systematic and comprehensive mastery of knowledge in the theoretical discipline teaching, especially for large-scale medical talent education in China 2 . Several reports applied personal electronic devices for education. This approach could expand learning space, improve academic performance, and strengthen clinical thinking [3][4][5][6][7][8][9] . The mobile smart devices (such as Apple, Huawei, etc.) could facilitate active communication and arouse learning enthusiasm during education 3,10−14 . This study aims to explore a new teaching mode of "Mobile Clinical Education" (MCE), and evaluate its e cacy based on the intelligent and social mobile app . Methods

Subjects
In the rst semester of 2016-2017, research candidates were students from Nanjing Medical University for nursing major. The "Nutrition" course was selected for the experimental research. The same lecturers completed every teaching lecture for all students on the same day. Among them, 97 students were assigned for MCE, while 84 students were included in the control group. The participants` Ethic Approval and Consent was explained orally by one lecturer, and con rmed by one teaching assistant. There was no signi cant difference in gender, age, and course progress between the two groups (P > 0.05).

Teaching method
The experiment was designed and applied on the popular and commercial "WeChat" mobile app. The "Clinical Nutrition Online Teaching" (CNOT) program was developed for MCE under "WeChat" frame. All lecturers prepared all courses in advance, and uni ed the teaching methods for MCE group. All contents on CNOT program were selected and approved by the teaching committee. The research process was summarized on " Fig. 1. The MCE Research roadmap".
The control group was taught by the traditional method, and the same instructors were responsible for the overall teaching. The Nutrition course included 8 lectures and 2 clinical labs. All theoretical teaching was done in class, and no other exercises were assigned.
For MCE teaching, "Clinical Nutrition Online Teaching" (CNOT) program was applied, which included four modules: preview, after-school quiz, debate, and teaching evaluation. All learning could be achieved with time and dimensional limits. The CNOT system included the client and administrator applications. The program management structure was shown in " Fig. 2. Platform Management Structure". Via CNOT, we could upload the learning resources, collect client feedback and manage the online teaching. On Fig. 3.
The User Interface of CNOT" was displayed, including the QR code, login page and main program menu. The app details were explained as follows: 2.1 Preview: Before class, the lecturers were required to provide the academic knowledge and frontier about the teaching subject. Students could prep the lesson in advance, which would improve their engagement and learning e cacy.
2.2 Quiz after school: All quizzes were designed by instructors before class. Each quiz intended to cover the lecture content and evaluate the learning performance in time. The class quiz would be applied ten minutes before the end of class, and mandatory to be done in 15 min. The CNOT app could present the real-time evaluation about quiz results. The students could also overview all quizzes and the evaluation feedback, which was important to guide their understanding about the main knowledge and learning achievement. (refer to Fig. 2).
2.3 Debate center: As the core idea of MCE, this model contained two parts: "classroom debate" and "training class", which allowed to fully mobilize the students participation, initiative and creativity during teaching process. At the beginning, 2 lessons were taught by traditional class lectures about the basic nutrition knowledge. Based on the syllabus, MCE method was adopted for the remaining 8 chapters. As designed, the students for MCE were randomly assigned to participate 8 teaching activities (6 theoretical lectures and 2 internship courses). Each class, 10 students were selected for the pros and cons debate on one topic, which last about 40 minutes. The debate system referred to the "Singapore competition system" 15 , including thesis, essay, opening argument, debate, free debate, audience questioning, and conclusion. The competition system was adjusted appropriately (refer to Fig. 4). The Debate arrangement would be announced by the lecture instructor via CNOT program 1 week before the class. The debater could communicate with the teacher at any time through the WeChat. During the debate, the instructor should conduct the debate process, including time, order maintenance, and especially the debate content about the key theme. 60 participants joined the class debate, while the remaining 37 students were required for self-study. Each class, the on-site teaching was completed for random 6-7 students. Before the self-teaching, these students should prepare the lecture content in advance through the "preview module". The teachers, administrators and students of one topic class would form the communication group via social WeChat app, and the lecture content and teaching methods could be discussed freely through group chat. The quality of the clinical labs would be ensured in this way.
2.4 Teaching Evaluation: Students' feedback and advice were collected for MCE before and after class. The evaluation questionnaire was applied based on the Chinese version of the SEEQ "University Classroom Teaching Evaluation Questionnaire (Student Edition)", which was reported to have better stability and higher reliability 16 . The questionnaire contained 34 questions, which were divided into 8 parts: academic sense, teaching enthusiasm, content organization, teamwork, personal communication, teaching development, content di culty, and overall evaluation. All questionnaire was nished online via CNOT program. All data and evaluation were analyzed statistically.

Final Evaluation:
The nal evaluation for the control group were the theoretical test. For MCE students, the nal evaluation included 3 parts: the after-school exercises (10%), the class debate/self-teaching (20%), and theoretical examination (70%).

Evaluation of the teaching method:
Both subjective and objective Evaluation was carried out. The objective evaluation included two aspects: theoretical test and nal evaluation. The two groups received the same nal exam. All tests were scored according to the syllabus standard. All quiz and exam data was summarized, and the knowledge level of the two groups were statistically analyzed. The nal score of control group was the theoretical test score, while the nal evaluation of the MCE group was calculated as described above. The subjective evaluation was completed by the online questionnaire. All MCE students completed the SEEQ teaching evaluation before and after class. The control group only received one questionnaire survey after the semester.

Statistical method:
The test scores and questionnaire scores of the MCE and control group were analyzed by SPSS Statistics 22.0 (IBM Inc., Chicago, IL). P≤0.05 was considered to be statistically signi cant.

The theoretical examination:
The average score for theoretical test were (69.1 ± 7.5) in the MCE group, which was higher than the control group (67.9 ± 6.7; t = 1.127, P = 0.26).

Questionnaire:
The 2 SEEQ teaching evaluation questionnaires were statistically analyzed. 96 valid electronic questionnaires for MCE were collected before class, and 77 were collected after the semester. The SEEQ Evaluation of the MCE Group". For MCE, students claimed the better achievement for the following aspects: curriculum challenges, interest in learning, interpretation of teaching content, attraction of teaching methods, encouragement of discussion and communication, personal expression of ideas, and the latest development in the eld of teaching. The teaching satisfaction was also improved signi cantly for lecture content, teacher, and overall course. The positive feedback included teaching content extensibility (t = -4.44, P < 0.001), teaching method attractiveness (t = -4.75, P < 0.001) and personalized teaching interaction (t = -4.50, P < 0.001). The experimental group`s experience optimization is more prominent. The detailed results were provided on " Table 1. The SEEQ evaluation of the MCE group throughout the semester".
The SEEQ questionnaire for both MCE and control group were statistically analyzed after class. There were 77 valid questionnaires from the MCE group and 65 control group. The MCE students had better performance in encouraging discussion and communication and personal expression (t = 2.22, P = 0.03; t = 2.16, P = 0.03). The control group was more satis ed with teaching content and teacher preparation. MCE innovation was signi cantly effective (t = -2.54, P = 0.01; t = -2.89, P < 0.001) ( Table 2).

Discussion
The ultimate goal for medical education is to cultivate medical and healthcare talents. Different from other disciplines, medical education always includes complex course content, cutting-edge academic update and practical exercise. For undergraduate clinical teaching, traditional lecture methods are hard to keep students to the academic frontier and get practical skills for their carrer 17 . Besides the monolithic lecture, more intelligent approach should be considered to fully mobilize the students participation, initiative and creativity during the teaching process.
Tecent Wechat is one popular mobile app for social communication in China. Different from simple message, Wechat is the powerful and real-time platform for various social connectivity, such chatting, texting, le sharing, collaboration and more 18 . Besides individual connection, more smart program could be developed based on Wechat for varied application, such as lecture, advertisement, shopping, news and so on. "Mobile Clinical Education" (MCE) was developed to promote teaching innovation and meet increasing demand of medical talents in China. In our study, the "Clinical Nutrition Online Teaching" (CNOT) mini program was designed under WeChat frame. "Nutrition" course was selected to evaluate the feasibility and e cacy of MCE.
Our ndings con rm that MCE could improve the teaching quality, and promote students to deeply understand theoretical knowledge and develop clinical practical ability. With MCE activities, students presented better skills for social communication, medical practice and independent learning.
The analysis for teaching effect showed that the MCE was great for clarifying the status of students' teaching subjects, embodying individualized teaching, advocating active learning and formative evaluation. At this stage, students were more accustomed to the traditional "lecturer teaching, student listening" teaching style, while the new teaching mode has a better teaching experience. The acceptance of the teacher in the control group was better than that of the MCE group (Table 2, Question No. 10, t = -2.68, P = 0.01), which may be related to the compression of the teaching time of theoretical classroom teachers and the re nement of the teaching content.
Thus, MCE had the following advantages: 1) Topic teaching: Through "class debate" and "self-teaching lectures", MCE reinforced the teaching approaches for theoretical knowledge and medical practice. Students were encouraged for active learning and discussion about each lecture topic. Student-centered learning could improve the knowledge understanding, practical ability training, and independent learning. 2) Interactive teaching: Wechat learning group increased the lecture interactivity between teachers and students. Their communication could be established instantly to solve any question about the courses and learning subjects. Students reported better learning participation, creativity and satisfaction. 3) Open learning: Through MCE system, students could use online educational resources without time and space restriction. The active learning initiative was developed here for medical talents, which is important for their commitment about lifelong learning. 4) Academic frontier: MCE system contained the basic knowledge and academic frontier. All contents were provided and reviewed by the experienced teachers and scientists. All lectures were organized and updated according to recent discipline development and clinical practice. Students were free to learn the high-quali ed online resources related to the course topics. 5) Multidimensional evaluation system: MCE evaluation was designed for overall learning performance, including problem-solving quiz, active learning score and traditional exam scores. The multi-dimensional evaluation was always applied during the teaching process. Besides the core achievement of theoretical knowledge and practical operation, students were also questioned and tested for their ability for self-learning, individual analysis and social communication. All results would be comprised to evaluate the learning effect. 6) Evidence-based teaching: Through the application and management of the MCE platform, the teaching details and achievement could be recorded, traced, standardized and analyzed. These evidence and feedback could promote the continuous improvement of teaching design and MCE application.
This study was just the preliminary MCE application for clinical medicine. There were many challenges for its future popularization: 1) Comprehensive teaching ability: MCE reduced the lecture time, but the student-centered learning required more teaching tricks and social skills from teachers. For interactive classes, teachers should conduct active debat/discussion and keep it to the theoretical topic. The practice experience and clinical thinking should be also introduced during the teaching process. 2) Rigorous teaching management : WeChat is open for various learning resources and free communication. Based on MCE curriculum applet, the teaching team should review and provide high-quality resources related to learning topics. The MCE learning should have the timeline and dynamic evaluation. Appropriate teachning guidance was important to keep the learning quality and inspire students' creativity. More attention should be paid to encourage the slack students. 3) Teaching evaluation criteria: The class debates and self-teaching lectures in this study were judged by the responsible teachers. There was a lack of uni ed evaluation criteria. The standardized system should be explored to guide the process and result evaluation for MCE .

Conclusion
The 2015 Horizon Report (Higher Education Edition) highlighted that teaching role reversal, portable devices and adaptive learning would greatly impact on higher education 19 . The mobile intelligent platform should be introduced and developed accordingly. The CNOT mini program was designed as the great and effective trial for mobile clinical education in the era of "Internet +". Our results con rmed the MCE superiority to promote the active and lifelong learning. Besides theoretical lecture, student-centered MCE would arouse the learning engagement, improve social collaboration, and develop practical ability. The MCE innovation also bring up new requirements for teaching techniques, experience and course management. More studies should be developed to promote the MCE optimization and popularization. This research had no risks to the participants. All participants were not identi able from the data collected. The research did not involve any therapeutic intervention. This research presents no risks to the participants. All of our participants would not be identi able from the data collected. The Exception to Written Evidence of Consent was approved by the IRB of First A liated Hospital with Nanjing Medical University / Jiangsu Province Hospital. The participants` Ethic Approval and Consent was explained orally by one lecturer, and con rmed by one teaching assistant Funding: This research was supported by Nanjing Medical University in 2017 (QN2017151). The role demands of the founder throughout this research was to cover the expenses on design and maintenance of the CNOT mini program.

Acknowledgements:
Thanks to all the lecturers of Nutrition Department of Nanjing Medical University. Thanks to all the students participated in this research. Thanks to all of our research team and their families. With all their support led this submission and achievement.
Author`s contributions: XM was the leader of all the researchers included, who was responsible for decision making and research design. As the corresponding writer and the teaching secretary of Nutrition department, JW planned the research, submitted the study, responsible for organization of teaching activity and was a major contributor in writing the manuscript. JC and TZ were responsible for literature review beforehand. SG, FW and CP analyzed the results. All authors read and approved the nal manuscript.

Consent to Publication:
Not Applicable.  The MCE Research Structure  The SEEQ Evaluation of the MCE Group