Can online technology be reliably used to improve education? A prospective study using neurovascular examination of the upper and lower limbs

Objective This article assesses the efficacy of modern technology in improving education using neurovascular examination (NVE). To determine whether revision of NVE is facilitated with online video services. Methods This prospective study assessed medical students (n=260) and junior doctors (n=238) upper and lower limb NVE knowledge with an exam. A course utilising online videos was delivered. NVE knowledge was tested after the course. After one month, participants (n=100) were invited to watch the videos online and re sit the test. Results Mean score of participants before the course was 2.76/8 for UL and 2.67/10 for LL. After delivery of the course there was a significant improvement in scores to 7.83/8 for UL and 9.33/10 for LL (p=<0.005). Statistical analysis revealed no significant difference in scores from just after the course to one month after (p=0.765 UL, p=0.779 LL). Conclusions Integrating online video tutorials with traditional teaching methods demonstrated a significant rise in performance. Revision is facilitated due to its availability and ease of use. This helps with performing an adequate and complete NVE. We recommend academic institutions adapt such educational technologies to advance learning improve Aslam-Pervez N, Bradley D, Mushtaq V MedEdPublish https://doi.org/10.15694/mep.2017.000084 Page | 2 patient care. Educational theories should be utilised to optimise their delivery.


Introduction
The United Kingdom Department of Health noted that innovative education technologies offer unprecedented opportunities to provide trainees and practitioners to acquire, develop and maintain knowledge, skills, values and behaviours needed for safe and effective patient care (DH/Workforce, 2011). Sharing of educational resources through open online video sharing sites is an example of an innovative education technology. It comes under the broad heading of "eLearning" which relates to education using the Internet.
There are several open access websites allowing users to upload videos for social and educational purposes such as You Tube, Vimeo, iTunes etc. These video hosting services are described as a "collection of web based technologies which share a user-focused approach to design and functionality, where users actively participate in content creation and editing through open collaboration between members of communities of practice" (Hollinderbäumer, Hartz, & Uckert, 2013). Users can upload, watch, share and comment on published videos. The new generation of doctors who have been cultivated in an environment enriched in advancing information technology are dubbed "digital natives" (de Wet & Yelland, 2015). They hold aptitudes and learning styles in sync with their IT rich environment.
It has been shown that often, medical students use video hosting services as their first resource for learning and research (Kingsley et al., 2011).. Half of these videos are viewed on mobile devices, which highlight its accessibility to users (" YouTube," 2015). Many studies have shown its use as an effective educational tool due to its popularity, free content and ease of accessibility and use (Jaffar, 2012).
Azer et al screened 2240 videos relating to nervous system examination and concluded that YouTube is an adequate resource for medical students to learn nervous system examination (Azer, Aleshaiwi, Algrain, & Alkhelaif, 2012). A recent study of medical students who undertook the United States medical licensing examination demonstrated that the poorest physical examination performances were in musculoskeletal and neurology domains (Peitzman & Cuddy, 2015). This may have an impact on patients with a neurovascular (NV) deficit due to injury. The study calls on institutions to revise their curricula and traditional approaches to teaching to address this deficiency. With medical students turning to online services for their education, institutions may gain from using this emerging sector in order to reach them.
The failure of identification and inadequate documentation of a NV injury can lead to mismanagement and form a medico-legal pitfall (Mehta, Garbera, Kaye, & Ramakrishnan, 2015) , (Mayne, Perry, Stables, Dhotare, & Bruce, 2013). Trainees are often a first port of call for the assessment of a patient with a potential NV deficit. These include junior doctors and medical students. It is vital that their physical examination skills identify NV deficit so that prompt management and escalation to a senior can be made. total hip arthroplasty (Schmalzried, Noordin, & Amstutz, 1997).
This study aims to assess the efficacy of a course incorporating a video hosting service in improving NV examination knowledge in trainees of different grades and seniority. The study further assesses whether the revision of knowledge is facilitated with the availability of instructional online videos.

Methods
We invited trainees by distributing posters that advertised the course and contacting medical school coordinators in Leeds, Manchester, Hull and York and postgraduate education departments in 7 hospitals in the Yorkshire and Northwest Deanery of the UK. The number of trainees who responded to the adverts and were selected to be included in the study was 498. Flowchart 1 comprises a summary of the methods. A pre-course test to assess knowledge on peripheral NV examination was conducted and participants were asked to document how they would carry out a NV exam of the upper and lower limb using a clinical scenario as an example. The upper limb case was of an 11-year-old female with a supracondylar fracture. The lower limb case was of a 27-year-old with an open fracture of the tibia. Our test (Appendix 1) allowed participants to answer the two questions in free text. Our aim was to replicate the clinical situation where participants document their examination findings in patient notes. Data was gathered to determine their grade and whether they had passed the membership of the Royal College of Surgeon's (MRCS) exam to allow statistical comparison. The syllabus for the postgraduate membership of the Royal College of Surgeon's (MRCS) exam requires knowledge of NV examination ("MRCS syllabus," 2015) . Trainees should acquire detailed knowledge of the course of vessels, nerves and their supply and innervation to assess and manage injuries to them. Following the test, a three-hour course was delivered to 478 participants to teach NV examination of the upper and lower limbs. Of the 498 who responded 20 registrars were excluded as they did not participate in the course.
The course was delivered over the course of a year in multiple settings with a minimum of 10 participants in each sitting. A minimum of 10 participants ensured adequate numbers in divided groups so they may carry out the group work. An introductory lecture delivered the objectives and outline of the course. The course was split into two parts; upper and lower limb NV examination. The course incorporated demonstration of NV examination using a video resource. Participants were split into 5 groups to watch online videos on computers or their hand-held devices.
Each group was assigned to watch a video for the anatomy of one of the nerves or vessels and invited to present their summary to the other groups in a supervised setting with the facilitator. Volunteers from the groups were used so participants could physically draw the anatomy and course of nerves and vessels on their bodies for the demonstrations. Following this, videos on the clinical examination of peripheral nerves were shown. participants were divided into the same groups to practice their NV examination with supervision.
We used You Tube as our video hosting service for this study. This was based on the popularity of YouTube as a video hosting site(" YouTube," 2015). It also allowed the users to create a playlist pertaining to a topic of their interest. A playlist in YouTube was allocated to each part of the course; upper and lower limb. These were directed and published by the author (NAP). The videos show the author demonstrating surface anatomy of nerves and vessels and clinical examination on a volunteer. Video duration was restricted to a few minutes each to avoid any lapse in concentration. We used animations and annotations for key points. There were nine videos in each playlist for upper and lower limb NV examination. The final video was a summary of the examination for UL and LL which lasted a few minutes each.
Case studies were analysed in small groups to familiarise participants with cases where NV deficit occurs. These included cases with lacerations to forearm, mid shaft humerus fracture, total hip dislocation and open fracture to the lower limb.
Following this, three participants volunteered to simulate an injury with a NV deficit. They enacted a child with a supracondylar fracture with a median nerve injury, a patient with self-harm wounds to the forearm with injury to several NV structures and a patient with a hip dislocation and sciatic nerve injury. A volunteer was asked to perform a NV examination on the simulated patient and present their findings to the other participants. This helped reinforce the clinical NV examination in a setting with simulated pathology. At the end of the course participants were asked to re-sit the NV test (Post-course test). After the course they were provided with access to videos on YouTube to revise their knowledge. The widespread availability of the internet allows participants to access these, to revise their knowledge when seeing a patient to ensure their examination is complete and adequate.
After one month, the study group was invited to complete the NV test again. Because of the change in the geography of the participants many of them were not contactable for a follow up test. We therefore decided to invite 100 out of 478 participants from the original cohort to complete the NV test to assess their knowledge. There were 50 MS, 30 FY1, 10 FY2, 5 CT1 and 5 CT2. These participants were spread across grades similar to the original cohort that undertook the course so comparison can be made. We analysed the scores of the NV test done prior to the course. A comparison of upper and lower limb test scores between different grades was done using the Kruskal-Wallis test for non-parametric data as our data (scores) were not normally distributed. As our distribution was not similar in each grade we used mean ranks to compare grades.

Statistical analysis
We compared scores for participants with an MRCS to those without using the Mann Whitney U test for nonparametric data as our data distribution (scores) were of different shapes.
We used a paired t test to firstly compare the scores of participants prior to the course and immediately after to statistically determine the significance in change in their scores. Secondly, we aimed to determine whether there was a significant difference in scores of participants immediately after the course to 1 month after.

Comparison of different grades
Mean scores for UL and LL test scores was higher for more senior trainees (Figures 1 and 2 Pairwise comparisons were performed using Dunn's (1964) procedure with a Bonferroni correction for multiple comparisons across different grades. This particular procedure runs multiple Mann-Whitney U tests for each pairwise comparison with a correction for multiple comparisons (Bonferroni) but only uses data from the two groups being compared. Post hoc analysis and adjusted p-values are presented in Table 2. In UL test scores no significant difference was found in mean ranks between FY1 and FY2 (p=1), FY1 and ST2 (p=0.16), FY2 and ST2 (p=0.052), ST1 and ST2 (p=1), ST1 and registrar (p=1) and ST2 and registrar (p=1). In LL test scores no significant difference was found between FY1 and FY2 (p=1), ST1 and ST2 (p=1), ST1 and registrar (p=1) and ST2 and registrar (p=1). Medical students scored significantly less than any other grade (p=<0.001) for UL and LL.

Comparison of pre course test scores between participants with and without membership of the Royal College of Surgeon's exam (MRCS)
A Mann Whitney U test was conducted to determine if there were differences in UL and LL scores between participants with (n=65, 13%) and without (n=433, 87%) MRCS. Distributions for both UL and LL scores in participants with and without MRCS were not similar, as assessed by visual inspection. UL and LL scores for participants with MRCS (UL Mean rank = 456.43, LL mean rank = 464.19) were statistically significantly higher than for participants without (UL mean rank = 218.45, LL mean rank = 217.27) ( Table 3). Comparison of scores before and after the course A comparison NV UL and LL test scores among participants from before and after the course were statistically significantly different (UL p=<0.005, LL p=<0.005). 100 participants responded to a retest in 1 month. Their scores from just after the course to 1 month after (follow up score) were not significantly different.
In the results reported above we have presented the scores achieved against the mark scheme proposed in Table 1 to compare different grades and change in scores after the course. To ensure an adequate NV exam has been performed participants have to gain full marks. In Appendix 2 we report the percentage of participants who achieved an adequate NV exam according to grade and at pre course, post course and follow up intervals.  (Peitzman & Cuddy, 2015).
We found significantly higher scores for participants with a MRCS than those without. These findings were expected and concur with our belief that appropriate education can help improve NV test scores among trainees, especially those without the MRCS. Although higher mean scores were observed in those with the MRCS as compared to participants with no MRCS in UL (MRCS=7.3 vs No MRCS=2.3 p=<0.001) and LL (MRCS=8.5 vs No MRCS=2.1 p<0.001) NV exam, only 53.8% MRCS participants performed an adequate (i.e. full marks) upper limb NV exam and only 18.5% a lower limb NV exam (Appendix 2). The improvement in scores and percentage of participants with an adequate NV exam both with or without an MRCS immediately after the course suggests online educational materials through video hosting services can be successfully incorporated into courses.
At the one month follow up, 100 out of 478 participants were asked to watch the summary videos for UL and LL prior to the follow up test. There were no significant differences in scores in the Post-course and Follow up test. This demonstrates video hosting services are effective for knowledge revision after the course. Trainees can revise these prior to examining patients to ensure their examination is complete and adequate.
Due to geographical relocations and inability to contact all 478 participants, only 100 from the original group were followed up at one month. They were similarly distributed across the different grades as participants in the original group to ensure that grade did not affect one month follow up test scores. Nonetheless, this is a limitation of this study.
Despite the improvement in scores and percentage of trainees with an adequate NV exam, we found trainees were finding it difficult to retain knowledge for lower limb NV examination. The percentage of trainees with an adequate LL NV exam was lower than trainees with an adequate UL exam straight after the course (UL=82% vs LL=42%) and at follow up one month later (UL=86% vs LL=49%) (Appendix 2). This was mainly due to difficulty in retaining the five nerves that provide sensation to the foot. We have updated our videos to account for this and look to improve our course and videos to make this easier to retain.
The advantages of incorporating audiovisual elements to an educational instruction to enhance learning are well recognized with an increase in knowledge retention and technical proficiency (Collins et al., 2015) , (Mehrabi et al., 2000) , (Ridgway et al., 2007). Audiovisual instruction provides a mental standard to which trainees can compare their performance (Custers, Regehr, McCulloch, Peniston, & Reznick, 1999).
A recent survey in Ireland found that 78% of 73 undergraduate medical and radiation therapy students used YouTube as their primary source of learning anatomy (Barry et al., 2015). Many studies have explored the content of medical tutorial videos on You Tube. Whilst there were many high quality videos from recognized institutions, there were also videos published with compromised quality and wrong information (Azer, 2012) , (Akgun et al., 2014). In our study we have published our own videos to ensure the quality is in line with recommended standards published by academic institutions (Saxena, Natarajan, O'Sullivan, & Jain, 2008) , (Azer, Algrain, AlKhelaif, & AlEshaiwi, 2013). McAlister et al also published their own videos on YouTube to teach kinesiology to occupational health students. Their survey revealed that students perceived the videos improved the quality of the course, increased their level of engagement and learning, and boosted confidence in their manual skills (McAlister, 2014).
To understand this benefit and effectively deliver the utilisation education technologies such as video hosting services, Laurillard's conversational framework applies social constructivist theories to e-pedagogy in two main aspects. Firstly, on a discursive level theoretical ideas and concepts are exchanged, with critique, comments and group discussions involving reading, writing, communicating, articulating and presenting. Secondly, on the experiential level students often collaboratively work and learn by doing, practicing, analyzing and making  (Laurillard, 2002). We have included this model of learning for our trainees. In addition to using videos, our course used multiple modes of delivery to enhance the educational experience consisting of small group sessions with case studies, simulated patient scenarios and presentation to colleagues by drawing the courses of nerves and vessels on volunteers. After the course trainees had the chance to apply their knowledge in examining patients and watch our videos to keep their knowledge updated.
There are some limitations with this study. Firstly, the study lacks a control group. This would have allowed comparison of this educational technology to more traditional forms of teaching in our control group to determine whether this technology was more or less effective. Secondly, there was a relatively large number of medical students as compared to junior doctors (FY1, FY2, CT1, CT2). This reflects the ease of delivering teaching to medical students as they are available more frequently than doctors in training. Comparatively more busy schedules and on-call commitments meant fewer doctors were available for this study. Thirdly, there was a significant drop out in the repeat tests done after the one month interval.
The mark scheme was devised by the authors and covered the essential criteria expected in the examination routine. It was designed for the examination of injuries in the test scenarios and covers the examination routine for a vast majority of injuries to the limbs. There are other routines to the examination in more specific injuries that were not included in the mark scheme. An example is examination of the digital nerves in patients with injuries to their hands. Our teaching covered the main nerves and vessels and expressed the applicability of the examination routine taught.
The traditional learning approaches and the student/teacher relationships are now changing. The academic community will benefit in using these platforms to reach this new generation of "digital natives". With decreased contact time in anatomy labs and in job training to improve knowledge in NV examination, the rise of the online generation offers new possibilities for anatomic and clinical education.

Take Home Messages
There was a significant gain in knowledge of peripheral NVE by using video hosting services as part of our 1.
course. This study highlights the low levels of knowledge of NV examination among trainees which could be 2.
addressed by effective use of video hosting services to support traditional teaching methods. We recommend academic institutions combine online videos in their delivery of curriculum objectives to join 3.
in the emergence of new education technologies. Educational theories help describe and structure the delivery of education with new education technologies.

4.
Institutions wishing to use education technologies are advised to familiarise with these theories to ensure an enhanced educational experience for students.

Scenario 1
An 11 year old patient attends A+E with a deformed elbow. X rays show a displaced supracondylar fracture of the distal humerus.

Declaration of Interest
The author has declared that there are no conflicts of interest.