Evaluation of the Quality of Dentistry Related Youtube Videos Made by Youtubers . Public Health Tool , or Large Scale Danger ?

Context and objectives: Considering the actual influence of YouTube, it seemed important to evaluate the quality of dentistry-related videos it offers. Indeed, youtubers are becoming more and more famous and their influence among communities is growing: their advice on oral health could represent an interesting public health tool, or a large-scale danger. Method: A quality-evaluating table has been built, including 18 items to obtain a score for each video. Inclusion and exclusion criteria were established: among other items, only videos that were made by amateur youtubers (“amateur” from a medical perspective, with no medical degree) were accepted. Then, youtuber videos were classified following their score. Results: Among a study of 100 videos, 58% were of poor quality versus 42% which were better. More specifically, videos about aesthetic dentistry (tooth whitening, etc.) were those having the worst results while surgery or orthodontics videos led to significantly better results. However, we observed a positive evolution of all the results from 2011 up to the end of the study in 2016. Conclusions: Aesthetic dentistry videos were evaluated as being potentially prejudicial. They can represent a danger for the patients-viewer’s oral health, whereas those relating to surgery or orthodontics were of better quality. These last ones, and finding new ways to improve the average quality of all dentistry-related amateur YouTube videos, could represent new ways to deal with oral prevention for a large audience, using youtuber’s influence or involving health professionals.


Introduction
YouTube, as a social media and network, has seen its popularity increase since 2011 and has become one of the most daily visited website on the Internet [1].To such an extent that young people think it is as important as television [2,3].By this mean, some new information or communication media, and new jobs like youtuber, were recently created [3].
The status of youtubers brings them success and attracts many big companies, offering them several ways of remunerations [4,5].Youtubers post videos on their own YouTube channels which are

Journal of Epidemiology and Public Health Reviews
Open Access Journal Inclusion and exclusion criteria of evaluated-videos were: Inclusion criteria -The video had to be uploaded on YouTube -The title of the video mentioned any dentistry topic -The content of the video was related to dentistry (whatever the topic: surgery, aesthetic, orthodontics, etc.) -The video was published in 2010 or after -The youtuber spoke in French or English -The video was presented and made by an «amateur» youtuber (with no medical training) Dentistry is a part of this phenomenon that might consequently be seen as a new way to improve public oral health.However, this phenomenon involves as well a large part of youtubers who are not health professionals.Several publications have already highlighted that the extent of their videos is important enough to be considered by health professionals [9,10]: on one hand the viewers are receptive to the content of the videos [11] and on the other hand their efficiency regarding the promotion of health has been proven when the information's are of good quality [12].Although the social networks contribution (such as YouTube) to improve the viewers and patients knowledge is real, the quality of the information's was only studied for health professional videos with a different presentation than the classic « facecam » videos or tutorials used by real youtubers [13][14][15].Knowing that some amateur-youtubers dentistry-related videos account for more than 1million of views, and thus can have a strong echo on patients-viewers, it seemed important to proceed to a thorough evaluation regarding different topics of dentistry, and to adapt this evaluation not only to professional videos or videos where the goal is to teach dentistry to practitioners or students.Our study's main goal was to evaluate the quality of the content of French and English amateur youtuber videos.Therefore, we observed the relevancy of the information's that patient-viewers could learn, measured a potential risk linked to those videos and eventually identified ways to improve their quality and the patient-viewers security as well.

Methods
In order to evaluate the quality of dentistry-related YouTube videos, we designed a quality-evaluating table with 18 items which are detailed in table 1.These items have been based on an existing table which was meant to be suitable for professional videos [9] and has been adapted for our study.The table that we used was then designed to concur with every oral health-related video and was validated using a test-retest reliability method (α = 0.97).Moreover, most regulatory and scientific notions we found were subject to reliability verifications (Haute Autorité de Santé (French High Health Authority) [16], Agence Nationale de Sécurité du Médicamentet des produits de santé (French National Agency for Drugs and Health Products) [17], American Dental Association [18], etc.).Indeed, to gain points, videos had to comply to their own country laws: this specific point largely concerned legislation related to the use of hydrogen peroxide.Actually, the concentration and usage of this product vary from one country to another.
One point could be added or subtracted for each item in the table, leading to final scores between -3 and 15.At the end of the study, the videos which were evaluated were ranked and compared regarding their score and quality: -From -3 to 3 points: Videos are considered of poor quality and present a potential danger for the oral health of the patient-viewers -From 4 to 7 points: Videos are considered of average quality and could have a negative impact on the oral health of the patient-viewers because of the lack of information -From 8 to 11 points: Videos are considered of acceptable quality and could have a beneficial impact on the oral health of the patientviewers but there is still a lack of information -From 12 to 15 points: Videos are considered of good quality and represent a potential new way of dealing with oral health prevention with patients-viewers.

Criteria Example
(The youtuber) provides basic hygien notions and/or motivates viewers regarding for their teeth or smile (+1) 1 Suggests to go see a dentist (or an oral health professionnal) (+1) Has contacted adentist/orthodontist (to inform himself)/ or is treated by a specialist (+1) 1 Quote one (or more) scientific references (+1) Produces a preventive video (+1) Masters the subjects, and has an adequate vocabulary (+1) 1 Explains the mechanism of action/ operative mode/ procedures/a treatment plan/a method/the causeof the problem(s) which is consistent andrecognized (+1) Explains the caution measures and/or recommendations recognized (+1) 1 Explains the different products or methods or recognized treatment options (and justify one of these) (+1) Produces a video with a thematic exclusively regarding dentistry, and doesn't digress (+1) 1 Explains clear and recognized indications concerning treatment/product and/or explains the aim of the given advice (+1) Explains the treatment's clear and recognized contraindications/product or explains the limit of the given advice (+1) Explains some of the side effects/complications/of the practice/of the method/of the advice (+1) Quotes a method/practice or a product/advice whose efficiency (or the input) is recognized (+1) Illustrates its explanation via different media sources (videos, photos, gestures) (+1) 1 Gives wrong or erroneous information (- Furthermore, to enrich the discussion, 11 videos made by health professionals were evaluated separately thanks to the same table (Table 1).

Results
In total, 100 random amateurs-videos consistent with the inclusion and exclusion criteria were assessed.51 were in French and 49 were in English (links are in the supplementary material section).As far as impact, these 100 videos represented more than 160 millions of views.

General results
Videos were classified following their score/quality: 19% of the videos were qualified as of « poor quality » and present a potential danger for the oral health of the patients-viewers; 39% are included in the « average » category; 38% are « acceptable »; and 4% of them in the « good quality » category which represents potential oral health prevention for the patients-viewers.
These general results, with an average score of 6, 6 over 15 points, have been detailed to highlight the difference between three types of videos: the aesthetic-related ones, the orthodontics-related ones and the oral surgery-related ones.The aesthetic category showed an average score of 5,3; the orthodontics-relating and oral surgery-relating categories presented average scores of respectively 7,8 and 8,3.The statistical analysis showed an extremely significant difference between aesthetically related videos and those dealing with orthodontics and oral surgery (Table 3).

Language
Results have been compared according to the language (French or English) of the videos and not according to thematics.The average score for the French videos was 6,3.For English videos, it was 6,9.The statistical analysis showed no significant difference (p=0,2877).

Year
In order to look over the results depending on the year they were put on YouTube, they were classified per year and analyzed with a Student t-test, revealing a positive increase in the quality of the videos particularly since 2015 (Table 4).It is to be noted here that the study took place over almost a year and ended August 6th, 2016.

Number of views
To improve the comprehension as far as the impact of videos, we have also looked at the average quality of the videos regarding the number of views.Thus, results have demonstrated that videos with the most views, all categories combined, had an average score of 5,8 over 15 (Table 5).

Exclusion criteria
-The video was presented by a health professional -The video was meant to be funny (like a sketch) -The video was meant to come as a backup to some pictures, like a diaporama -The video was published less than a month before the evaluation -The video had less than 1000 views At the same time, another table was made to reference some indications about videos (Table 2).This one is designed to take more information's into consideration like a number of views, youtuber's age and sex, year of publication, etc.Then, scores were compared according to different criteria (topic, year…).The statistical analysis was performed using Student t-tests, with biostaTGV (https://marne.u707.jussieu.fr/biostatgv/developed by INSERM, Paris, France).

Discussion
The results of this study have shown a large disparity regarding the oral health-related video's quality on YouTube.Incidentally, YouTube became an active research tool the same way its boss is: Google, the one that has proven that dentistry researches have increased over the last 10 years [19].The evaluated videos, on their own, gathered more than 160 million views while only representing a sample of the whole dentistry related subjects that you can find on YouTube.Regarding the variability of the information's provided by the videos, their evaluation following the quality-evaluating table was performed by two persons (test-retest method), obtaining sensibly the same results.On one hand, it seems to be a good indicator to overcome biased results between different videos; but on the other hand, there was no additional external validation and therefore the results may still be subjected to a general evaluation bias.
Three main categories were revealed: aesthetic dentistry; orthodontics and oral surgery (or maxillofacial surgery).None of the evaluated videos dealt with oral hygiene.Even if some videos were led to mention oral hygiene, none was specific to this topic and the subject was always reoriented to another thematic.
Results show that even if aesthetic videos appeared to be dangerous according to scores and notions which were given, videos dealing with surgery or orthodontics could add-up to a real benefit for the patients as much as for their psychological mindset before treatment as well as during the treatment.Indeed, it seems easier to use your imagination and « home-made » solutions (for example, Do-it-yourself aka DIY) for aesthetic-related videos, but their content can often be put into question, especially regarding whitening recipes.

Evolution
Our results show that the quality of the videos seems to improve over the years (older videos dating from 2011 to 2016 for the newest ones), giving an advantage to the informed youtubers who are more and more dedicated to their communities.This is probably the answer to the increasing demand in terms of quality from the patients-viewers.

Age
As far as the main youtuber's profile and results of regular research on « Google Trends » website (YouTube section -hot videos) go  or not, and patient-viewers to use it.This way, we're hoping to offer a new way to improve the quality of dentistry-related YouTube videos and promote an important vector in the oral health knowledge of patients.

Conclusions
Today, smile and appearance take up a huge part in everybody's life and keep feeding social networks, leading new generations to use and always produce more and more subjects related to oral hygiene, especially on the YouTube media.Aesthetic dentistry related videos seem more likely to be dangerous than other videos included in our studies.Indeed, surgery-related videos and those dealing with orthodontics subjects seem to bring a real benefit for the oral health knowledge of the patients-viewers.An improvement in the quality of the videos throughout years been noticed, reflecting the youtubers will to provide more accurate information and the trend to more professionalization on YouTube.
The general youtubers -and their subscribers-profile (notably their age), the average score of the videos (6.6/15), and the scores of the videos with the most views suggest that the dentistry related YouTube videos represent a large-scale danger.Nevertheless, this same general profile, the increasing quality of videos in time, and the average score of the videos made by health professionals (10.6/15) are in favor of dentistry related YouTube videos as a public health tool in the making, depending on how we act today to redress the balance.
Sharing our evaluating tool with youtubers could allow them to improve the quality of their oral health-related videos, and thus could be a real progress in terms of oral prevention and public health for the population before, during and after treatment.
[20], we can observe that videos are focusing more specifically on a young audience.It is striking how young some youtubers are.Their motivation to create and make videos are different and often legitimate but the lack of scientific experience is concerning.How can someone talk about medical or scientific subjects thoroughly if he/she is so young, and consequently, with scientific information's missing?There is obviously a risk to share false -and perhaps even dangerous -information's.While it seems impossible to impose to non-health professional youtubers to come up with highly accurate health-related videos, it could nevertheless be interesting to use YouTube robots.Even if they can't evaluate the quality or the potentially dangerous side of some videos, one could think that kids under 18 posting videos should not be receiving any financial reward.This would then stop many young youtubers from starting this actor business at a too early age.However, nothing has been found or proven to show a link between the quality of videos and the age of the youtubers.

Danger
The most famous videos (more than one million views) only have an average score of 5,8 and nearly 20% of all non-professional videos that were studied show a potential danger regarding oral health for the patients-viewers.
For instance, in some cases videos give a tip on how to whiten your teeth yourself, by using lemon and baking soda at the same time, twice a day: this is an acid and aggressive technique for the tooth structure.This should warn us on the urgent need to lead or provide tools to youtubers for improving the quality of their videos, because those could have an important impact on the population, especially the young one.The Madathil and et al. review in 2015 [11], pointed out that norms and controls are required regarding videos of youtubers which take an interest in health (oral or general health), although some controlling organizations already exist such as Health On the Net [21] which verify the information of some websites.The way to obtain these certifications is very strict and can't be applied by sharing websites like YouTube is.In fact, videos come from multiple sources, video formats vary a lot, etc.This is why no standard tools or control methods exist yet, therefore making our study unique.

Benefits and projects
Using YouTube as a new method for oral public health appeared quickly after observing the unrivaled quality of videos made by health professionals (more than 90% of the videos were of good or acceptable quality).Since YouTube is becoming more and more important, why not use it to raise awareness among the different populations?Indeed, new training and informing means could be an advantage for medical fields in exploring new ways of prevention.Using the short video format is indeed becoming more and more popular: some University Preventive Medicine Department is organizing international video contests, with health promotion as a theme [22].
It might be possible to request youtubers (via a partnership or not) to come up with entertaining and attractive videos dealing with oral health subjects, just like the French Ministry of Health mandated two famous youtubers to inform about the conduct to hold in case of a heat wave [23].This video was uploaded in July 2017 and counts more than 1,7 millions of views.Moreover, Google already offers different types of training to warn youtubers by means of YouTube Spaces.One might imagine Conference base ideas on medical subjects.
Obviously, the quality-evaluating table which has been designed for this study could be used for every potential project.Actually, we suggest setting it up on a free access basis allowing youtubers, trained Number (& percentage) of videos per year according to their quality and average score (*) p <0.1; (**) p < 0.05; (***) p < 0.01.

Table 1 :
Quality-evaluating table of dentistry related YouTube videos.

Table 2 :
Referencing videos related information table.

Table 3 :
Number of videos according to their category and their quality, and average scores (***) p < 0.01: extremely significant.

Table 5 :
Average scores of videos according to the number of views.