Randomized controlled trial on the effect of YouTube surgical videos on procedural knowledge and perceived self-efficacy of surgery residents

Background Multiple factors have challenged the surgical training which led to the development of the instructional videos to hasten the proficiency in performing surgical procedures. However, their educational effect has not been objectively studied yet. This study aims at objectively assessing the effect of instructional videos along with the subjective assessment of the self-efficacy of the training residents. The videos used were selected from YouTube, which is a valuable resource currently because of its ubiquitous availability and cost effectiveness. Methods A stratified randomized controlled trial was performed using an objective assessment tool for the procedural knowledge along with a questionnaire to assess the effect of videos on the perceived self-efficacy of the residents. Results There was a significant positive effect of watching instructional videos on the procedural knowledge. Even though residents report positive experience with using the videos, there was no significant effect of videos on the self-efficacy scores as reported by the residents. Conclusion instructional videos improved the procedural knowledge of the residents. This positive observation of the use of the YouTube videos offers great opportunities for the educationalists to select from thousands of videos instead of making the videos themselves.


Background
Proficiency in doing surgical procedures is one of the major competencies required to graduate an independent practicing surgeon.This operative competence requires more than the pure technical competence, the psychomotor skills.In fact, the cognitive domain, which entails the appreciation of the anatomy and the comprehension of the procedural steps, plays a critical role (1)(2)(3)(4).Indeed, the importance of the cognitive competence overcomes that of the pure technical counterpart (5).It was reported that only 25% of important events are related to the technical skills (6).Similarly, other researchers found that the non-technical aspects of performance accounts for many of operative adverse events encountered.This explains the great emphasis on focusing on the cognitive part of the procedural training.
The principle model used in procedural training is the observational theory, which states that residents learn by observation (7).In surgical training, this is very well encountered through the commonly used phrase "see one, do one, teach one" which is still used and said to depict the sequence of learning a surgical procedure (8).Interestingly, research has confirmed that observing a procedure prior to doing it results in better performance (9).
The observational theory stems from the famous social cognitive theory (10).This theory describes the concept of the interaction between the person, the surrounding environment and his/her behavior.More interestingly, Prof. Bandura describes a reciprocal effect of those factors as shown in figure 1.
This theory describes important phenomena.It shows the interaction between the environment and the behavior.First, people learn by watching and then imitating the way someone else performs a task.Second, people learn by observing others' success or failure during the performance of a task.
Those observations show how the environment affect behavior.On the other hand, the behavior of a person can affect the surrounding environment.
Another interesting and relevant phenomenon is the interaction between personal factors with behavior and environment (11).Personal factors refer to level of interest, self-efficacy and cognitive ability.Of interest, the perceived self-efficacy, which is defined as how someone believes in his ability to perform a task, affects his/her behavior and his/her learning from the environment.Self-confident people are more likely to have better learning from their environments and better performance of tasks.In the same time, the environment and the behavior affect the perceived self-efficacy of a person.For instance, having successful task performance and positive environment is more likely to result in improved feeling of self-efficacy.
Traditionally, for the procedural knowledge, residents prepare by using mainly atlases and textbooks (12).Interestingly, residents find those resources marginally useful to address their knowledge gaps (13,14).
Recently, the increasing number of residents which leads to fewer learning opportunities (15)(16)(17) along with increasing the cost on the institutions and the strain on the teaching staffs (18,19).
Additionally, there is an increased attention towards the patients' safety (20,21).Those factors promoted the call for changes in the surgical curricula.Multiple educational interventions, for instance, video demonstrations, simulations etc.
Instructional videos can be defined as videos which are made to teach a specific task or set of tasks.
Interestingly, instructional videos are shown to be the most time and cost effective, which drive the focus and the interest of this paper (22).Instructional videos are considered major vehicles for observational learning since the development of computers, internet and social media (23) and they are also are considered valuable sources for professional learning (7).Generally, instructional videos offer multiple advantages as shown in multiple domains; dentistry, computer science, etc.They are observed to be associated with better long-term retention as compared to the traditional methods (24,25).Students prefer videos over formal instructions because of reply function, being enjoyable, interesting and informative (26).Additionally, instructional videos are found to provide a uniform, efficient and safe learning environment (21).
In the context of surgical proficiency, the instructional videos are made to address mainly the procedural knowledge or specific technical skills.Surgical procedural knowledge has been defined as "the ability to enumerate the steps of a given surgical procedure in the order that they occur in the OR" (3).On the other hand, procedural skills are mainly about the technicality aspect used to perform a procedure.The primary intended rule of the instructional videos is adjunctive to the current surgical curriculum.As observed in the context of instructional videos made to model the procedural skills is which showed that they cannot substitute the observation of an actual procedure, rather, they can synergistically support the learning (7).
The procedural knowledge instructional videos were not investigated until recently when Hayden (2015) investigated the use of an instructional video for lobectomy, the removal of one lobe of a lung, made upon the needs assessment done in their institute.The video was made up of voice-over narration of schematic representation of the procedure or thoracoscopic video footage in addition to a thoracoscopic video explanation of the technique.The focus of the study was to test for the usefulness and the appropriateness of the video as assessed by residents and their training consultants assessed by a questionnaire.Residents who replied to the questionnaire reported that videos were helpful and give appropriate review of the procedure.The critique for this study is, first, that only 8/20 residents filled up the questionnaire.Second, the educational effect was not measured in this study objectively; rather, the assessment tool is subjective and asks whether the video was helpful or not.
In this paper, the aim is to study the effect of watching procedural knowledge instructional videos on trainers' self-efficacy and the procedural knowledge, which are the training outcomes that influence task performance (27).This topic is important to be studied because if those selected YouTube videos are shown to be effective, their ubiquitous availability makes them a valuable and costeffective resource for education in general surgery.

Research Question:
Do selected instructional videos from YouTube for general surgical procedures have an educational effect on the procedural steps knowledge of the residents?Do selected instructional videos from YouTube for general surgical procedures improve the perceived self-efficacy of the residents in general surgery?

Methods
This study adheres to CONSORT guidelines.
Setting: General Surgery department, Prince Sultan Military Medical City, Riyadh, Saudi Arabia.
Study participants: In the department of general surgery, there are 37 residents, all were offered to participate.The participation was voluntary and verbal consent for participation was obtained prior to enrolling in the study.An outline of the distribution is presented in figure 1.One fifth year resident elected not to participate as the procedures of interest in the study are basic procedures and she has already reached mastery in them.Hence, 36 residents were randomized.17 residents were randomized to the intervention group and 19 were randomized to the control group.During the process of data collection, 7 residents dropped out from the intervention group for multiple reasons outlined in figure 2.
Design: A stratified randomized control non blinded trial was used in which residents from each year were randomized to the intervention group or the control group.Computer generated randomization was used in which each participant was assigned a number by the department secretary and the https://www.randomizer.org/website was used by the principle author to randomize participants.In the intervention group watched the instructional videos on YouTube, while participants in the control group did not.The randomization was stratified.Each level of residency was randomized separately.
Procedural knowledge and self-efficacy were assessed at three times: before the intervention, one week after the intervention and one month after the intervention.The assessment was made using a questionnaire as shown in Appendix 1 and the Procedural steps knowledge assessment tool as shown in Appendix 2.

The intervention:
The procedural steps videos were selected from YouTube for open indirect inguinal hernia repair for men https://www.youtube.com/watch?v=H0wVNhvuljo and laparoscopic cholecystectomy https://www.youtube.com/watch?v=ej49YLgbqVs.Those videos were validated for their quality by the help of the training consultants.From practical point of view, the procedures chosen are seen in any rotation of the training and a month is usually enough to get enough exposure to the procedures.
Additionally, for the objectivity of our study, we used an existing validated tool for the assessment of residents on those specific procedures.

The measurement tools:
Two assessment tools were used.The first assessment tool is an objective tool that has been developed and validated for the assessment of procedural knowledge (3).It is composed of a checklist of the chronological order of the procedural steps from incising the skin up to closing the skin, see appendix 2a and 2b.Each procedure has number of steps and at the end a score is given to the participant, any incorrect ordered step or missed step was deducted from the score on that procedure.
The second assessment tool is a questionnaire aimed at assessing the perceived self-efficacy and confidence of the residents, see appendix 1.The scale was adapted from Hayden, Seagull, & Reddy, 2015 and (28) a 5-point Likert scale with 1 being very low and 5 being very high.
Analysis: Quantitative analysis for the differences in procedural steps knowledge and self-efficacy was done using SPSS 21.A descriptive analysis was first generated to present the data followed by a comparative analysis using independent T-tests to compare the intervention and the control group.
Paired sample T-tests were used to compare the effect of YouTube videos at one week and one month.The t-tests results displayed using the assumption of equality of variance as the Levene's test was higher than 5%.

Results
The effect of videos on the procedural knowledge, using the objective assessment: 36 residents were randomized.17 residents were randomized to the intervention group and 19 were randomized to the control group.During the process of data collection, 7 residents dropped out from the intervention group, 4 were too busy to have time for follow up and 3 were on leave on the time of final data collection.
The study started on March 1 st 2015 and the data collection was completed by the end of the month as the final assessment was completed.
The baseline characteristics of the participants in the two groups are similar as shown in table 1

Discussion
This study aimed to study the effect of watching procedural knowledge instructional videos on trainers' self-efficacy and the procedural knowledge.Our results show that instructional videos resulted in a statistically significant improvement in the procedural knowledge of the residents as compared to the control group.This difference persisted at one month.Of great interest, the videos showed the difference in the procedural knowledge which was observed at one week did not evolve at one month, which suggests a fast learning effect secondary to the videos.
This observation of educational effect of the instructional videos come in light with the observational theory which is described earlier, that people learn by observing others performing an action (23).
With regards to the medical field, instructional videos in general offer multiple advantages as shown in other medical domains e.g.dentistry.They provide long term retention as compared to the traditional methods (24).Students prefer videos over formal instructions because of reply function, being enjoyable, interesting and informative (26).Additionally, instructional videos are found to provide a uniform, efficient and safe learning environment (21).
Interestingly, the scores for the perceived self-efficacy were similar between residents who watched the videos and the control group.This could represent that confidence in performing a procedure is attained mainly from hands on practice rather than from knowing the procedural steps per se.

Limitations:
The number of the residents in our institute is limited to 37 residents which is a small number.The study is done in one institute, which is inferior to doing a multicentric study.There is a dropout of 7 residents in the intervention group which is due to pure chance and challenge to the results of the study.The videos used in the study were carefully selected from YouTube, similar caution should be exercised by the surgical educationalist to recommend any video from the YouTube.

Generalizability:
This study aims to open a great opportunity for surgical educationalists as YouTube contains thousands of instructional videos including procedures done with different techniques, rare procedures and difficult cases.This offers a great hope to open a great chance for the learners to use selected videos from the very rich resource being the YouTube.

Directions for future research:
Instructional videos are great tools for learning surgical procedures.As a follow-up to this first explorative study, we offer some directions for future research.First, we suggest that future research can employ a replication with a higher sample size and a multicentric approach.Second, we suggest that future studies could investigate other anatomical sites that extend those used in the previous study.Third, future studies can compare the educational effect of instructional videos for simple as compared to complex surgical routines because the level of surgical complexity could act as a moderator variable between self-efficacy and procedural knowledge gains.Finally, to assess validity, future research can address the fidelity of YouTube videos as compared to de novo made videos.

Conclusion
YouTube instructional procedural videos improved the procedural knowledge of the residents.This offers great opportunities for the educationalists to select from thousands of videos instead of making the videos themselves.However, the videos need to be carefully selected to ensure high quality education.

Figures
Figure 1 Social cognitive theory

table 2 .
and

Table 2
Baseline charactristics, level of participents in each group

Table 3
Baseline scores for the objective assessment of the operative knowledge

Table 4
Comparison between the two groups in objective assessment of the knowledge of procedural steps at baseline At one week, the objective assessment for the procedural knowledge is presented in table 5.The comparison between the two groups shows significantly higher scores for residents who watched the videos as presented in table 6.

Table 5
Scores of the objective assessment for the procedural knowledge at one week

Table 6
Comparison between the two groups using the objective assessment of the procedural knowledge at one week At one month, the objective scores are presented in table7.The comparison between the two groups showed significantly higher scores in the residents who watched the videos as represented in table 8.

Table 7
The scores of the objective assessment for the procedural knowledge at one month

Table 8
Comparison between the two groups at procedural knowledge using the objective assessment tool at one month The comparison between the effect of YouTube videos at one week and one month showed no significant difference as shown in table9.

Table 9
11mparison between the effect of videos at one week and at one month The effect of videos on the self-efficacy scores:The baseline scores in the self-efficacy questionnaire at the baseline are presented in table10.The comparison between the two groups in shown in table11which showed equivalence between the groups.

Table 10
Baseline scores for the self-efficacy questionnaire

Table 11
The comparison between the two groups using the self-efficacy scores At one week, the scores of the self-efficacy scores are presented in table 12.The comparison between the two groups showed no statistically significant difference as shown in table13.

Table 13
Comparison between the two groups using the Self-efficacy scores at the first week At one month, the self-efficacy scores are presented in table 14 whereas, the comparison between the two groups showed no statistically significant difference as shown in table15.

Table 15
16mparison between the two groups using the self-efficacy scores at one month The experience of residents who watched the videos are presented in table16which shows that residents find the videos very helpful, more fun to study than textbooks, the videos were scored to be more helpful than textbooks.Residents recommend strongly others to watch the videos and they strongly like to receive more videos in the future.

Table 16
Impression about the videosAll the residents who watched the videos reported that they watched the videos at work and individually.