Introduction

The Internet has become a popular source of information for healthcare purposes in recent years [48]. In the USA, Internet-based healthcare information is the second commonest source of information for the patients, after doctors’ recommendations [16]. Likewise, in a survey conducted in Europe, 71% of Internet users reported that they used the Internet for healthcare purposes [3]. Regardless of location and time, individuals can search online for a wide range of health issues [12]. However, due to its uncontrollable nature, users should be cautious about the validity of the available data [13].

Like other areas of healthcare, information about orthodontics on the Internet is increasing, and more people use the Internet for various aspects of orthodontic treatment every day. However, the reliability and quality of the web-based information is crucial since it may affect patients’ compliance and adherence to treatment as well as their communication with and trust towards their doctors [29]. Concerns related to the nature of healthcare information have led to the development of various evaluation tools, which have been also used to evaluate the written data on the Internet [21, 28, 34]. The readability of Internet-based healthcare information is also evaluated because insufficient readability may limit the usability of websites [20, 42]. According to a National Assessment of Adult Literacy survey of adults in the USA, most adults (53%) have intermediate health literacy, and 36% have basic or below basic health literacy, indicating the skills for only simple and everyday literacy activities [26]. Universally, literacy skills and readability levels of texts are measured in terms of the average reading skill achieved at each year of schooling in the American public-school system [18]. With these in mind, writing healthcare information is recommended at the fifth- or sixth-grade level to make it more publicly understandable [18, 45].

Increasing expectations about physical appearance have also increased the demand for orthodontic treatment. However, the visibility of orthodontic appliances might sometimes cause esthetic concerns. Among other methods, clear aligner treatment (CAT) has been gaining more attention by orthodontic patients since it is esthetic, removable, and easy to carry and care for [46, 47]. In addition, CAT requires 67% fewer total appointments than conventional fixed appliances and significantly fewer emergency appointments [7]. Particularly during the global coronavirus disease 2019 pandemic, these features of CAT may be beneficial for potential orthodontic patients and increase their information-seeking tendency on the Internet. CAT is performed by general dentists in many countries without legal restrictions, and do-it-yourself or direct-to-consumer (DTC) orthodontic aligners, either with or without the consultation of a dental practitioner, have also been introduced [38]. With the growing attention to CAT from manufacturers, providers, and patients, this treatment modality has been heavily marketed in recent years. In a recent study, 73% of patients were informed about CAT by advertisements on social media and networks [17]. Despite the increased data on the Internet, limited information exists about the accuracy, reliability, readability, and overall quality of the information provided by websites on CAT [4, 31, 32]. The studies have varied in how they examined web-based data about CAT, such as being a part of different orthodontic treatment modalities [4] or DTC marketing [31]. The only study examining the quality of online information about CAT, by Meade and Dryer, [32] reported that websites contained mostly poor-quality data. However, that research lacked a detailed content analysis about features of CAT, except for the quality and readability assessment. Since CAT is a new and popular mode of orthodontic treatment, and professional websites are important ways to reach potential patients, additional data showing how these patients are informed from the Internet about CAT would add to the limited knowledge in this field.

Therefore, this study aimed to evaluate the quality, reliability, and readability of the information available on the Internet about CAT provided by different authorships, along with their contents reflecting how to reach prospective patients.

Materials and methods

Search strategy

The search terms for this study were determined using Google Trends [4]. Google Trends was used to define the most popular terms about CAT during the past 12 months (April 2019 to April 2020). The top three search terms related to CAT (“aligner”, “clear aligner” and “Invisalign” [Align Technology, Tempe, AZ, USA]) were further analyzed. The related terms were searched in three widely used search engines: Google (Google LLC, Mountain View, CA, USA), Yahoo (Yahoo, Sunnyvale, CA, USA), and Bing (Microsoft, Redmond, WA, USA) [2]. The search was performed in April 2020 by virtual private network (VPN) in the USA by a single researcher (MTA). A VPN extends a private network across a public network and enables users to send and receive data across shared or public networks as though their devices were directly connected to the private network. Since the authors of this paper were not based in the USA, a VPN was used to simulate the virtual environment of US citizens. The first 50 websites related to each keyword were recorded from each search engine. Before evaluation, duplicate websites, advertisements, links to scientific articles, videos, social media profiles, forums, blogs, discussion groups, and irrelevant websites were excluded. Only websites in English were analyzed. The websites were categorized according to authorship or ownership for further analysis.

Quality assessment

The websites were evaluated for quality and reliability using the DISCERN instrument, Journal of the American Medical Association (JAMA) benchmark criteria, and Health on the Net code (HONcode, Health On the Net Foundation, Geneva, Switzerland).

The DISCERN instrument is the first standard index to evaluate the quality of consumer health information [11]. The instrument consists of three sections and 16 questions scored from 1 to 5, where 1 indicates total rejection and 5 indicates absolute agreement. Section 1 is composed of eight questions to evaluate the reliability of the publication, and seven questions in section 2 analyze the quality of treatment options. Section 3 includes a separate question about the assessment of overall quality of the website (Table 1). A detailed handbook on the DISCERN website also includes factors to consider while deciding the score of each question (www.discern.org.uk). The websites were then classified into one of five categories by the total DISCERN score, except question 16 (range 15–75; 15–26: very poor, 27–38: poor, 39–50: average, 51–62: good, and 63–75: excellent) [44].

Table 1 Tab. 1 DISCERN instrument questions with the mean (standard deviation [SD]) score of each questionFragen des DISCERN-Instruments mit dem mittleren Score (Standardabweichung [SD]) jeder Frage
Full size table

The JAMA benchmark criteria are used to judge the reliability and reasonability of medical information on the Internet. They assess four basic criteria of the website: authorship (authors, contributors, affiliations, credentials), attribution (references and sources for the content and copyright information), disclosure (potential conflicts of interest), and currency (dates of posting and updates), and scores range from 0 to 4. The criteria have been proposed as a primary tool to assess the quality of health websites [40].

The websites were also assessed for the presence of the HONcode seal. HONcode is the oldest and most frequently used trust seal for quality healthcare information on the Internet. This corporation evaluates the volunteered applications of health-related websites based on standards for high-quality, transparent information, using eight criteria. If the website meets these standards, it is awarded HONcode certification for one year, with annual evaluations to keep it [15].

Content analysis

The form and function of websites were evaluated by a free online website assessment tool (https://www.bdc.ca/en/articles-tools/entrepreneur-toolkit/business-assessments/pages/free-website-evaluation.aspx by BDC, Montreal, Quebec, Canada). The website calculates an overall score between 0 and 100 after the URL of each website is entered. The scoring system is based on aspects such as web-optimized images, search terms and the amount of other content, links to social media, mobile speed, and mobile optimization. Higher scores mean better performance.

Additional information about the content of the data related to CAT features was also recorded and evaluated. These were categorized as information about cost, additional features, details about treatment protocol, case selection criteria, possible complications of CAT, comparisons with other orthodontic treatment modalities, and the presence of an image, video, or both about aligners.

Readability evaluation

The readability of the websites was evaluated by Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL) scores [6, 22]. The highest score that can be attained from the FRES is 100 and is directly related to the readability of the website. Scores between 90 and 100 reflect content very easily understood by people aged 10–11 years and are recommended. Scores 80–89 are categorized as easy, 70–79 fair, 60–69 standard, 50–59 fairly difficult, 30–49 difficult, and 0–29 very confusing [22]. The score was calculated automatically with an online FRES calculator (www.readabilityformulas.com) using an average of 300 words from each website [28]. The FKGL scores of the websites were determined by the same calculator. The FKGL gives a readability score corresponding to a USA education grade level, which represents the number of years of education required to understand the text context [25]. A score between 7 and 9 is considered to represent average readability.

Statistical analysis

All data were analyzed using a statistical analysis program (SPSS® Inc., version 20 for Windows; IBM, Armonk, NY, USA). Data were summarized as mean ± standard deviation (SD), median (minimum–maximum), and frequencies (percentiles). The data distribution was evaluated by the Shapiro–Wilk test. Intergroup comparisons analyzing the effect of authorship were performed by one-way analysis of variance for normally distributed data and by Kruskal–Wallis test for data not normally distributed. Fischer’s Exact test was used to compare JAMA benchmarks between groups.

All included websites were re-evaluated by the same researcher (MTA) 2 weeks after the initial evaluation for DISCERN and JAMA scores, and intraexaminer reliability was assessed by intraclass correlation coefficient (ICC). The ICC scores for DISCERN and JAMA were 0.983 and 0.872, respectively, demonstrating excellent intraclass correlation.

Results

A total of 111 websites were analyzed after the exclusion and inclusion criteria were applied (Fig. 1). When the websites were categorized according to authorship and ownership, almost half (n = 49; 44.2%) belonged to multidisciplinary dental clinics, and only three presented the name of an orthodontist. This group was followed by equal numbers of websites of orthodontists and aligner companies (each n = 26; 23.4%). In the category of aligner companies, the websites belonged to manufacturers that provided aligners either via a professional or DTC, except one that compared all companies and shared some informative content. Only 9% of the websites (n = 10) belonged to professional organizations including orthodontic societies and online general health information websites. Among all websites, only four had the HONcode seal (3.6%).

Fig. 1 Abb. 1
figure 1

Flowchart diagram of Internet search

Flussdiagram zur Internetrecherche

Full size image

When the scores for each DISCERN item were evaluated thoroughly, professional organization websites had the highest scores for all questions. The weakest element of all websites was the lack of information about what would happen if no treatment were used (question 12). Orthodontist and multidisciplinary dental clinic websites had similar scores for all questions. However, orthodontists’ websites presented slightly higher overall quality (question 16). Websites of aligner companies had the lowest scores, with some exceptions. For instance, these websites showed better performance than orthodontist and multidisciplinary dental clinic websites in presenting their aims (questions 1 and 2), descriptions of how CAT works (question 9), benefits of CAT (question 10), and its effect on quality of life (question 13; Table 1).

The overall scores of each instrument are shown in Table 2. Professional organization websites had significantly higher DISCERN scores (p < 0.001). The quality of professional organization websites was categorized as average, while the rest of the websites had poor information quality [46]. Regardless of authorship and ownership, 87.4% (n = 97) of the websites had very poor or poor quality, whereas only 1.8% (n = 2) were excellent according to DISCERN criteria (Fig. 2). No difference was found among groups in the average FRES and FKGL scores (Table 2). Irrespective of authorship, the readability of the written information on all websites was fairly difficult, with above-average FRES and FKGL scores (Fig. 3). Concerning BDC scores, one website was not suitable for the application of the tool and excluded from the statistical analysis. Professional organization websites had significantly higher scores (p = 0.027).

Table 2 Tab. 2 Demographic information for all websites together and grouped according to authorship with comparison of scores for quality scores among groupsDemographische Informationen für alle Websites zusammen und gruppiert nach Autorenschaft mit Vergleich der Punktzahlen für die qualitativen Scores zwischen den Gruppen
Full size table
Fig. 2 Abb. 2
figure 2

Distribution of DISCERN scores of analyzed websites

Verteilung der DISCERN-Scores der analysierten Websites

Full size image
Fig. 3 Abb. 3
figure 3

a Flesch Reading Ease Score (FRES) of analyzed websites, b Flesch–Kincaid Grade Level (FKGL) of analyzed websites

a FRES (Flesch Reading Ease Score) der analysierten Websites, b FKGL (Flesch-Kincaid Grade Level) der analysierten Websites

Full size image

When the content of the websites was evaluated in more detail, giving details about the treatment protocol was the strongest element of all websites, and the weakest was the presentation of the possible risks. Like orthodontists, most of the aligner companies provided information related to treatment protocols but paid more attention to case selection depending on the severity of malocclusion. Aligner companies and orthodontists both supported their content with images or videos more than did multidisciplinary dental clinics and professional organizations. Orthodontists were also superior in providing information about possibilities of aligner treatment for not only adults but also teenagers (Table 3).

Table 3 Tab. 3 Detailed evaluation of the content of websites about clear aligner treatment and variations related to different authorshipsDetaillierte Auswertung der Inhalte von Websites über die Clear-Aligner-Behandlung und Abweichungen in Bezug auf verschiedene Autorenschaften
Full size table

The JAMA benchmark scores of websites are presented in Table 4. When websites were categorized according to authorship, statistically significant differences existed between groups regarding authorship, attribution, and currency categories (p < 0.001). Professional organization and multidisciplinary dental clinic websites showed better compliance with JAMA benchmark criteria. The percentage distribution of all websites according to JAMA is shown in Fig. 4. Websites showed excellent performance (100%) according to JAMA’s disclosure criteria, but with respect to other principles, compliance was only 7.2–15.3%.

Table 4 Tab. 4 Comparison of JAMA benchmark scores between groupsVergleich der JAMA-Benchmark-Scores zwischen den Gruppen
Full size table
Fig. 4 Abb. 4
figure 4

JAMA benchmark scores of analyzed websites

JAMA-Benchmark-Scores der analysierten Websites

Full size image

Discussion

The Internet is an easy-to-access information platform. According to the National Health Interview survey in the USA, 74% of adults are Internet users and 61% use the Internet to search for medical information [14]. However, the quality of online sources varies. Therefore, orthodontists should be aware of the content, reliability, and quality of the information on the Internet due to the increasing interest of patients in online health-related information [8].

CAT is an esthetic and comfortable treatment option for patients and has continued to increase in popularity in recent years. However, the possible advantages of these systems attract potential patients to do online research before visiting healthcare providers [7, 46, 47]. Studies have investigated the content of YouTube™ videos (YouTube LLC, San Bruno, CA, USA) [27, 43] and social media tweets [35] related to CAT, but since these mostly originate from patients and reflect their perspectives, they may not show high quality and educational value [27]. To date, only a few articles have been related to this issue. Arun et al. [4] qualitatively assessed websites about orthodontic treatment modalities, including findings related to CAT. Recently, Meade and Dreyer [31, 32] conducted two very informative studies on this topic. One provided a general assessment of websites from different authorships in Australia, and the other focused on DTC aligner systems only. Adding to information from this literature, the present study aimed to analyze the content of websites with criteria specific to CAT and compare the findings with previous research.

In the worldwide desktop market share of leading search engines from January 2010 to January 2020, Google (87.35%) leads the industry, followed by Bing (5.53%) and Yahoo (2.83%). These three search engines account for 95.71% of the total market share, which is why they were selected for this study [41]. Three search terms were defined with the aid of Google Trends, which showed the most used keywords during online searches about CAT. The top 50 websites for each keyword were evaluated for each search engine. This was comparable to not only the number of evaluated websites in previous studies but also the real online search behavior of a potential patient [30, 32]. Together with the exclusion criteria, our search strategy was very similar to previous studies [4, 31, 32].

The display of the HONcode seal on a medical website can be regarded as meaning it presents understandable, accessible, and reliable information. However, it does not guarantee the accuracy of the information. For this reason, it may be more useful to evaluate the results with additional instruments. In our study, only four websites had the HONcode seal (4/111; 3.6%), which was more than in previous studies about adult orthodontics [30] and CAT [32], but much lower than in a recent study about periodontal diseases [24]. In addition, all websites with HONcode certification belonged to professional organizations. These varying numbers may be related to the renewal fee for the license of HONcode certification since 2014 [23], which may also explain why only professional organizations in our study had this.

DISCERN is a reliable quality assessment instrument available online [10, 11]. Although it was designed to assess written healthcare information and has not been updated for online content, it has been used frequently in previous studies dealing with online written healthcare information [1, 19, 21, 30,31,32, 37, 39].

Regardless of authorship, the mean total DISCERN score (except for question 16, in section 3) was 29.96. This is less than [4, 32, 33, 37] or similar to [31] the overall scores reported by various studies about the quality of online information related to different dental problems and procedures. These differences may be due to several factors such as the search terms, date, language or region of interest, and differences between examiners.

The professional organization websites showed good performance in DISCERN scores. The websites of these nonprofit groups had the highest scores almost for all questions in all sections (Table 1) and reached a mean total DISCERN score of 47.10, which was significantly higher than the other groups (Table 2). In addition, two websites that had excellent DISCERN scores belonged to professional organizations (Fig. 2). Likewise, Meade and Dreyer [32] reported the highest DISCERN scores for a national orthodontic society, as well as an online encyclopedia and a health information services website. This may be due to these websites being possibly less biased and receiving professional support. Despite this, the mean scores for all websites were average or substantially below. In addition, regardless of authorship and ownership, 87.4% (n = 97) had very poor or poor quality (Fig. 2), meaning that all websites need improvements.

Orthodontist websites presented similar overall quality to multidisciplinary dental clinics, of which only three officially presented an orthodontist on their websites. This may be because general dentists also provide CAT, and their numbers have increased dramatically in recent years, with almost the same number of treated cases as orthodontists [17, 38]. Since healthcare providers use the Internet to attract potential patients, providing better quality data is advantageous for all practitioners supplying CAT. Chambers and Zitterkopf [9] stated that with the aid of increased advertising, general dentists try to play more of an orthodontic care provider role rather than only making referrals to orthodontists. Together with the greater number of multidisciplinary dental clinic websites, our results may therefore show the effort of these practices to attract more patients for CAT.

With some exceptions, websites of aligner companies had the lowest scores for most of the DISCERN questions. Although the total mean DISCERN score for aligner companies was lower (28.5) than they reported, our results are in accordance with those of Meade and Dreyer [31], who also found lower DISCERN scores for manufacturers providing aligners via a dental professional (mean score 36.2) or DTC (mean score 33). Still, these websites showed a similar performance to orthodontists and multidisciplinary dental clinics in how they presented their aims, working principles, the benefits of CAT, and its possible effects on quality of life. This positive marketing results in increased recognition of the appliances before patients plan to pursue CAT. Olson et al. [38] stated that 45% of respondents had heard about DTC before, 30.5% of them from social media, internet advertisements, or search engines [39].

The mean BDC scores of websites of all authorships were higher than those reported by Oey and Livas for orthodontic practice websites in the Netherlands [36], reflecting better optimization of the websites evaluated in our study. Professional organization websites had the highest mean BDC score, followed by multidisciplinary dental clinics (Table 2). Website optimization requires professional assistance, which means increased financial loads. The financial backup and enhanced office functions of professional organizations may therefore explain this result. Similarly, consolidation of dental care by multidisciplinary dental clinics and dental practice chains may have enabled better integration of information technology services for multidisciplinary dental clinics [36].

The mean FRES score of all websites was 55.77 and showed no differences when they were grouped according to authorship (Table 2). Together with the mean FKGL scores, this result falls into the “very difficult to read” category (50–59 points), similar to the results of other studies [30, 32]. However, the number of adolescent patients seeking CAT is increasing, and they also go online to research alternatives for their possible treatment. Therefore, easily read online information may be more effective to inform and attract potential patients for CAT.

When the content of the websites was comprehensively evaluated with questions specific to CAT, giving details about the treatment protocol was the strongest element of all websites, and the weakest was the presentation of the possible risks (Table 3). Surprisingly, aligner companies were the ones competing with orthodontists’ websites in explaining CAT treatment protocols with images and videos. In addition, these websites provided more information than others about the cost and additional features of aligners, case selection criteria, and comparison with other treatment modalities. Notably, however, these results may be affected by the information provided by DTC aligner companies. Supporting this idea, Olson et al. [38] reported that DTC aligner companies attracted consumers by advertising the reduced costs, shorter treatment time, and greater convenience compared to traditional practitioner-based CAT. The increased content on aligner company websites about case selection criteria may also be related to DTC aligners. On the other hand, orthodontists shared more detailed information related to treatment protocols. Best et al. [5] reported that other than using similar interproximal reduction, orthodontists are more likely to use auxiliaries, supplemental techniques, elastics, and refinement stages during management of CAT when compared to general dentists. Therefore, our results may reflect the differences in management and perspectives of orthodontists and general dentists about CAT. In addition, orthodontists seemed to be the main source of information on CAT for adolescents compared to other websites, which reflects their ongoing potential as the primary care for this age group.

Only six websites met all four JAMA benchmarks. The highest-rated JAMA criterion was disclosure. This result was similar to a previous study evaluating the web-based information quality about lingual orthodontic treatment [37] but contrary to other studies about CAT and adult orthodontics that reported the authorship for this rank [30, 32]. This data is also interesting in the sense that all aligner companies presented disclosure, which may be part of positive marketing strategies for these companies.

This study has some limitations. The web search was limited to English websites in the USA, so the results are only valid for a limited population. The other limitation is that the websites were evaluated by a single researcher at a single time. To overcome the possible bias related to a single examiner, the websites were re-examined after 2 weeks by the same researcher, and the results showed excellent intraexaminer reliability. With changes in the ranking and contents of websites over time, the assessment of quality and readability instruments may also differ. However, online information sharing is a dynamic process, and such studies may help the development of current content.

Conclusions

  • The results of this study showed that websites in English about CAT (clear aligner treatment) presented low-quality information with poor readability.

  • Professional organizations showed the best performance in the quality assessment.

  • Orthodontists and multidisciplinary dental clinics presented very similar data quality.

  • Aligner companies used their content as a positive marketing tool to reach and convince more people.

  • If clinicians providing CAT are aware of the problems related to their online written content and improve it, they can help prospective patients to benefit more from their websites and ease their decision-making about CAT.