YouTube as a Source of Patient and Trainee Education in Vascular Surgery: A Systematic Review

Objective Due to its video based approach, YouTube has become a widely accessed educational resource for patients and trainees. This systematic review characterised and evaluated the peer reviewed literature investigating YouTube as a source of patient or trainee education in vascular surgery. Data sources A comprehensive literature search was conducted using EMBASE, MEDLINE, and Ovid HealthStar from inception until 19 January 2023. All primary studies and conference abstracts evaluating YouTube as a source of vascular surgery education were included. Review methods Video educational quality was analysed across several factors, including pathology, video audience, and length. Results Overall, 24 studies were identified examining 3 221 videos with 123.1 hours of content and 37.1 million views. Studies primarily examined YouTube videos on diabetic foot care (7/24, 29%), peripheral arterial disease (3/24, 13%), carotid artery stenosis (3/24, 13%), varicose veins (3/24, 13%), and abdominal aortic aneurysm (2/24, 8%). Video educational quality was analysed using standardised assessment tools, author generated scoring systems, or global author reported assessment of quality. Six studies assessed videos for trainee education, while 18 studies evaluated videos for patient education. Among the 20 studies which reported on the overall quality of educational content, 10/20 studies deemed it poor, and 10/20 studies considered it fair, with 53% of studies noting poor educational quality for videos intended for patients and 40% of studies noting poor educational quality in videos intended for trainees. Poor quality videos had more views than fair quality videos (mean 27 348, 95% CI 15 154–39 543 views vs. 11 372, 95% CI 3 115–19 629 views, p = .030). Conclusion The overall educational quality of YouTube videos for vascular surgery patient and trainee education is suboptimal. There is significant heterogeneity in the quality assessment tools used in their evaluation. A standardised approach to online education with a consistent quality assessment tool is required to better support online patient and trainee education in vascular surgery.


INTRODUCTION
The internet has become a widely accessed tool for entertainment, communication, and knowledge translation. 1ith over two billion users and over 30,000 hours of uploaded video content per hour, YouTube (https://www.youtube.com/) is the second most visited website on the internet, 2 and is widely accessible on multiple platforms (e.g., smartphones, tablets, computers, etc.).Although originally designed as an entertainment platform, YouTube has also become a highly accessed source of information for patients to learn about various disease processes and their management. 3Similarly, YouTube has become a commonly used educational tool for surgical trainees. 4Despite its popularity, the quality of medical educational material found on YouTube has been called into question.For disease processes in several surgical specialties, including neurosurgery, 5 orthopaedic surgery, 6 general surgery, 7 and urology, 8 YouTube videos have been shown to be of low educational quality for understanding procedural steps and correct disease management.For patient education, You-Tube healthcare videos have often been found to contain misleading information across a breadth of pathologies. 3ouTube continues to grow in popularity owing to its accessibility, and sheer volume and breadth of publicly available videos. 9As such, patients and trainees alike will probably continue to access the platform for educational purposes.Patients may seek to educate themselves about their condition, prognosis, and expectations with surgical or non-surgical management. 10,11Similarly, trainees may use the platform to learn key procedural steps and how to expose procedure specific anatomy.While the educational quality of YouTube has been evaluated for several surgical and non-surgical specialties, 12e14 a comprehensive evaluation has not yet been conducted in the field of vascular surgery.YouTube may play an especially important role for patient and trainee education in vascular surgery given the breadth of pathologies in the specialty, the often high complexity and comorbidity of disease burden, and the generally high technical requirements for trainees.Given that the platform will inevitably be accessed in high volumes, evaluation of this educational content, identification of areas of improvement, and a review of the literature is critically important.As such, the purpose of this systematic review was to characterise and evaluate the current literature investigating YouTube as a means of education for both patients and trainees as it relates to the field of vascular surgery.

Study methodology and search strategy
This review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. 15The search strategy was developed in consultation with a health sciences librarian and peer reviewed according to the Peer Review of Electronic Search Strategies (PRESS) guidelines. 16Comprehensive searches of the literature were conducted using the databases EMBASE, MEDLINE, and Ovid Healthstar from inception (1974, 1946,  1966, respectively) until 19 January 2023.The keywords, YouTube, health, healthcare, information, education, and surgery were used.Search strategies in their entirety can be found in Supplementary Table S1.

Eligibility criteria
Studies were included if they were an original research article that evaluated or investigated YouTube as a source of patient or trainee education for any vascular surgery condition.Articles were excluded if they did not examine vascular surgery pathologies, did not use YouTube specifically as a video source, were narrative or systematic reviews (the citations of these papers were reviewed and extracted to the screening process), were commentaries, editorials, guidelines, news articles, or were written in a language other than English.

Study screening and selection
All citations were imported into EndNote X7 (2016) and underwent de-duplication, which was manually confirmed.Study screening was performed in two stages.First, two reviewers (A.P.J., M.V.) independently screened the titles and abstracts of each article in accordance with the inclusion criteria.This was preceded by pilots of 100 studies each to ensure that inter-rater reliability (kappa) was greater than 0.6. 17Two reviewers (A.P.J., M.V.) then independently conducted full text screening.During both stages, discrepancies were resolved through joint discussion and consensus between the two reviewers and a senior author (F.N.).

Data extraction and quality assessment
Data extraction occurred through standardised forms in Microsoft Excel in duplicate by two independent reviewers (A.P.J., M.V.).Discrepancies in data extraction were resolved through discussion with a senior author (F.N.).

Study and video characteristics
Data extraction parameters included study meta-data (year of publication, publishing journal), methodological data (study objective, search criteria and search methodology used, type of analysis conducted), vascular surgery pathology, quantitative data regarding videos examined (video length, views), educational video quality, intended audience, and uploading source.Video uploader source was divided into the following categories: academic institution/affiliation, advertisers/commercial/media, healthcare organisation, patient/public, physician, other healthcare practitioner, other.
For studies that reported only median and interquartile ranges (IQR) or ranges, mean AE standard deviation (SD) were calculated based on methods proposed by Luo et al., 18 Shi et al., 19 and Wan et al. 20 following normality tests.Studies with skewed quality scores were excluded from quantitative analysis.For studies with missing variance, analysis was completed using the largest reported SD. 21

Assessment of video educational quality
When quality of YouTube videos was assessed by a study, the overall educational quality of videos was categorised as either poor, fair, or good.Primarily this was determined by the authors' global assessment in the study conclusion or discussion.If authors did not explicitly comment on the overall educational quality of the videos, the overall assessment was determined according to a validated assessment scale if one was used.
If a validated assessment scale was not used and an author generated scale was used instead, the quality was deemed to be poor if the total score was 33% or lower, fair if between 33% and 66%, and good if above 66%.The authors of this review did not watch or evaluate the quality of any of the individual videos included in the study set.

Study quality appraisal
In contrast to educational quality of included videos in each study, quality appraisal of individual studies was conducted using the framework published by D'Souza et al. (M.V. and M.N.). 22This is a framework focusing on five overarching questions to assist authors in appraising studies using data from social media platforms.Each study was assessed according to the framework and a binary score (Y ¼ Yes or N ¼ No) was reported for each category.The overall study quality was graded as Good for a score !7 Fair, for a score six and !3, and Poor for a score 2.

Statistical analysis
Quantitative pooling of means and proportions were conducted using the meta 5.2e0 package in R 4.1.2(R Core Team, 2021, Vienna, Austria) with random effect, inverse variance weighting.Between study heterogeneity was determined using restricted maximum likelihood.Heterogeneity was quantified using I 2 where 30% < I 2 < 75% indicates moderate heterogeneity and I 2 !75% indicates serious heterogeneity.Pooled means were expressed as raw mean and 95% confidence interval (CI), and pooled proportions were expressed as percentage and 95% CI.For studies that reported 0% or 100% proportion, a continuity correction factor of 0.5 was used to complete the analysis.Descriptive statistics were used to evaluate trends and categorise data.Cohen's kappa was used to calculate interrater reliability.In all statistical tests, an alpha value of 0.05 was used to evaluate for statistical significance.

Study selection
The literature search across three databases yielded 8 273 initial citations (Fig. 1).Following the removal of 3 878 duplicate studies, a total of 4 395 citations remained for title and abstract screening.A total of 26 studies were evaluated in full text review, yielding a total of 24 23À46 studies for final analysis.

Study characteristics
Articles were published between 2015 and 2023, with the majority of papers published in the last four years (Fig. 2).Individual study characteristics are described in Table 1.Overall, 15/24 (63%) studies had multiple raters for video selection.Of those that did, 12/15 (80%) calculated an inter-rater agreement and reported it.All 24 included studies performed a content analysis and none of them conducted a thematic analysis (Table 1).

Video characteristics
Cumulatively, across all 24 papers, 3 221 videos provided a total of 123.1 hours of content and generated 37.1 million views.On average, studies analysed a mean of 134 videos.Overall, the pooled mean video length across all videos was 6.6 minutes (95% CI 5.8e8.0) with a pooled mean view count of 18867 (95% CI 11 233.1e26500) (Table 2).
Subgroup interactions between pooled mean video lengths of different vascular pathologies demonstrated statistically significant differences, driven largely by shorter videos for peripheral arterial disease (mean 5.9 minutes), diabetic foot care (mean 5.6 minutes), and deep vein thrombosis and or pulmonary embolism (mean 2.2 minutes).Subgroup interaction for pooled mean video views also showed statistically significant differences, driven by more views for videos related to peripheral arterial disease (mean 24,728), lymphoedema (mean 35 165), abdominal aortic aneurysms (mean 34,136), and multiple vascular pathologies (mean 76,658) (Table 2).
Overall, 41.3% of videos were uploaded by sources that would be considered educationally reputable (academic institutions, healthcare organisations, physicians, or other healthcare practitioner).Across all videos in the data set, 32.1% did not have a reported or recorded uploader source (Supplementary Table S4).Among videos uploaded specifically for trainee education, 53.5% did not report an uploader source (Supplementary Table S3).When sources were described, videos were most often uploaded by healthcare practitioners (15.1%), physicians (14%), advertisers, or commercial media (10%) (Supplementary Table S4).

Assessment of video educational quality
Among all included studies, 21 separate quality assessment tools were used to evaluate video educational quality (Supplementary Table S2).Among the 20 studies which reported on the overall quality of educational content, 10/ 20 studies concluded it was poor, 10/20 studies indicated that it was fair, and 0/20 indicated that it was good (Table 1).Videos characterised as poor were on average viewed more often than videos characterised as fair in quality (mean 27 348 views vs. 11 372 views, p ¼ .030,Table 3).In addition, videos intended for patients (mean 5.9 minutes) were generally far shorter than videos intended for trainees (mean 9.7 minutes) (Supplementary Table S3).Additionally, there was no difference between the proportion of videos that were poor in quality intended for patient education (8/15, 53%) compared with those intended for trainee education (2/5, 40%).

Study quality appraisal
When the quality of each included study was evaluated, 8/ 24 (33%) studies were found to be of good quality, with 16/ 24 studies (67%) being of fair quality (Supplementary Table S5).

Key findings
This systematic review characterised YouTube as a source of patient and trainee education in vascular surgery.The review found that (1) videos were poor to fair in educational quality, with the studies themselves being generally of fair to good methodological quality; (2) videos characterised as being poor in educational quality are viewed on average more often than higher quality videos; (3) there has been an exponential increase in the volume of literature evaluating YouTube in vascular surgery; (4) there is substantial heterogeneity in the quality assessment instruments used in the literature; (5) apart from videos without an identified uploader, most videos were uploaded by physicians or other healthcare practitioners.Taken together, these findings suggest that YouTube is seeing increasing use for both patient and trainee education in vascular surgery, but the overall educational and methodological quality of such information is lacking.
As evidenced by the 37.1 million views in this study, educational vascular surgery videos are in demand by both patients and trainees.Despite the popularity of YouTube, the overall educational quality of the videos is generally of poor to fair.It is also somewhat concerning that videos characterised as being poor in educational quality were viewed on average significantly more often than higher quality videos.Patients and trainees may mistakenly assume that a greater number of video views indicates a more reliable or more educationally reputable video.Both patients and trainees alike should exercise caution when interpreting the findings of these videos and should certainly not use them as a sole means of education.12e14 It is unsurprising that the educational quality of these videos on YouTube is lacking given the lack of a true peer review process and how the algorithm may not necessarily prioritise videos of high educational quality.Easily accessible and high quality peer reviewed information for patient  YouTube and open access online education may play a particularly important role for trainees as it relates to procedures that are technically complex or those that trainees have decreasing or limited exposure to, such as carotid endarterectomy or open abdominal aortic aneurysm repair.53e56 Trainees would probably benefit from simulation and pre-operative rehearsal in the form of watching these procedures before direct involvement in the operating room.57e59 In this study, the videos relating to trainee education for these pathologies were generally of poor to fair quality, and particular attention should be directed towards improving the quality of these educational videos as they may be of more utility to trainees.
The evaluation of YouTube as an educational platform has increased exponentially in the literature, with most of the studies in this paper being published in the last four years alone, a trend consistent with other specialties, with most studies being published in the last decade. 5,12The present study has identified several areas for improvement to augment the methodological rigor of this body of literature.Firstly, there is substantial heterogeneity in the scoring tools used to assess video quality, with 21 different scoring systems being used among the studies.Notably, 14 of these scoring systems were author generated and were often not externally validated.This presents challenges to the ability to compare educational quality across different studies.The use of standardised tools in future projects, such as the DISCERN score, 60 JAMA score, 61 and Global Quality Scale, 62 is encouraged.Utility in author generated scales exists when it provides additional granularity into a disease state and when they are externally validated or adequately supported by contemporary evidence, such as the Abdominal Aortic Aneurysm Specific Score. 26Additionally, the areas that were most lacking according to the methodological quality assessment of individual studies according to the framework proposed by D'Souza et al. pertained to the reproducibility of the study, exploration of the limitations of the study, and provision of future directions.Future studies should consult these guidelines when planning studies that evaluate open access educational materials. 22inally, it was found that many studies that evaluated YouTube as a source of patient or trainee education in vascular surgery either did not extract uploader source or were unable to identify an uploading source.This lack of source recognition is particularly worrisome in videos intended for trainees, where over 53% of videos had no identified uploader source.Lack of author identification may further contribute to confusion and difficulty with discerning video trustworthiness.The need for increased YouTube presence by academic and health institutions is encouraged, as has been suggested across the fields of urology, plastic surgery, and general medical education. 12,14,63

Strengths and limitations
This review has several limitations.Only videos shared to YouTube were included, excluding those from other video platforms or professional websites.This may create a bias towards content that is inherently unvetted and potentially of lower quality.YouTube specifically was chosen for this study due to its high traffic, popularity, and relatively global accessibility.Additionally, there was substantial heterogeneity in the educational quality assessment of individual studies, which precluded grouping together quantitative quality assessment ratings and instead having to rely on a global rating of poor, fair, or good.This reduces the precision of the estimate as it relates to the overall educational quality of videos.This is an intrinsic limitation of the study data set, and future reviews in this field may be able to perform a more precise analysis if there was more standardisation in the conduct and reporting of studies in this field, which may be made possible with the recommendations provided here.The highly sensitive search strategy is a strength of this review, increasing the number of studies for manual review in an effort to robustly review all available literature.Finally, while this paper suggests that the overall educational quality of vascular surgery YouTube videos is poor to fair, it is important to consider that there remain focal areas and channels with consistent high quality educational content which may be of particular benefit, such as the Houston Methodist DeBakey Cardiovascular Education videos. 64

Recommendations and next steps
This study demonstrates a need for higher quality YouTube videos in vascular surgery, such that they are created in correspondence with clinical guidelines and with reference to the peer reviewed literature.Academic incentives to publish on YouTube may be beneficial to improve overall video content, as suggested by Helming et al., 12 alongside institutions (academic journals, hospitals, university, etc.) working to produce high quality content.Additionally, a peer reviewed section on YouTube could perhaps be implemented to better support accurate information dissemination, as suggested by Farag et al. 65 Increased awareness of open access websites, such as the SVS OnDemand: https://www.vasculartraining.org, will probably also be beneficial for the future education of patients and trainees.Future research efforts should aim to create a unified video quality assessment tool to reduce heterogeneity and allow for ease of comparison, Study authors should consult the guidelines by D'Souza et al. when conducting future work evaluating YouTube and other open access online materials. 22Finally, vascular surgery societies may consider creating checklists for the standardised production, distribution, and assessment of open access educational resources, including YouTube videos.

Conclusion
YouTube is seeing increasing use by vascular surgery patients and trainees, and the literature in this field is seeing sustained growth.Despite this, the overall educational quality remains poor to fair, with most of the methodological rigor of individual studies being of fair quality.Moving forward, the standardised evaluation of educational videos on YouTube, the development of a peer review process and increased awareness of pre-existing high quality resources for patients and trainees are suggested.Additionally, the field would benefit from increased academic and healthcare institution presence on YouTube as it pertains to vascular surgery.

CONFLICTS OF INTEREST
None.

Figure 1 .
Figure 1.Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flow chart.

Table 1 .
Summary of key findings from 24 included studies.Fair and trainee education is available through several hospital organisations and societies.For example, a brief scan of open access resources reveals that both the Canadian Society for Vascular Surgery and Society for Vascular Surgery and several hospitals publish high quality information for patient education.47e51Similarly,theSocietyfor Vascular Surgery has compiled what it deems to be reliable, educationally reputable resources online on https:// svsondemand.vascular.org/.52Notably,when these resources do include videos, they often link back to YouTube itself.The findings herein point to the need for a peer review process for these videos before they are published in the public domain and a need for strategies to identify high quality educational videos for patients and trainees.

Table 2 .
Mean value calculated by total value divided by the number of videos, or total value calculated by multiplying mean value and number of videos.Borrowed from the study that reported the largest SD.YouTube in Vascular Surgery for Patient and Trainee Education Video characteristics based on topic discussed.Non-parametric data significantly skewed from normal were excluded.GQS ¼ Global Quality Scale; N/A ¼ not available.
* Imputed from non-parametric data and not significantly skewed.yImputedfrom non-parametric data and significantly skewed.zCombinedmean AE SD from more than one group.xk * y Subgroup interaction.z Fisher's exact test.

Table 3 .
Study characteristics based on video quality.Data are presented as n, n (%), 95% confidence interval.GQS ¼ Global Quality Scale; N/A ¼ not available.