Knee osteotomy: Quality tools and readability data of information on the internet

Supplementary data for the article Osteotomy around the Knee: Assessment of Quality, Content and Readability of Online Information is provided. 45 unique websites were evaluated. The DISCERN score, JAMA (Journal of the American Medical Association) benchmark criteria and HONcode (Health On the Net) criteria are provided for reference. Readability of online information was analysed with Readability Studio Professional Edition, Version 2019 (Oleander Software Ltd.). The software assessed readability using eight different instruments: Flesch-Kincaid Reading Grade Level (FKGL), Flesch Reading Ease Score (FRES), Raygor Estimate, SMOG, Coleman- Liau, Fry, FORCAST and Gunning Fog. Data is also provided on the percentage of complex words, long words, Dale-Chall unfamiliar words, Fog words, as well as the number of ‘wordy’ items, overly long sentences and longest sentence length.


Specifications
Health and medical sciences Specific subject area Osteotomy around the knee Type of data Appendix (1) Table (3) Figure (1) Raw Data Document (1) How data were acquired Primary data was acquired by searching four Internet search engines (Google, Yahoo!, Bing and Ask) with the search terms "high tibial osteotomy " and "knee osteotomy ." Each website's text was converted into a Microsoft Word document (Microsoft, Redmond, WA). All hyperlinks, pictures, advertisements, copyright notices, and any other text that was not directly related to health information was subsequently removed. Secondary readability data was then acquired by analyzing the re-formated documents with Readability Studio Professional Edition, Version 2019 (Oleander Software Ltd.

Value of the Data
• Readability, a metric that determines the ease with which text can be read and understood, is a major factor affecting the ability of patients to utilize health information. Readability of information is a key component of health literacy. • Researchers involved with examining the quality and readability of health information, as well as those tasked with generating patient educational materials, will benefit from this supplemental material. Tools are provided to assess the quality and readability of healthcare information. The 'wordy' items list will act as a repository for commonly encountered complex words or phrases, with suggested alternatives, which content creators will be able to cross-reference against. • Future studies will be required to assess the progress made in providing accurate, high quality and readily comprehensible healthcare information. The supplemental material provided here may be used in designing such studies, and furthermore, will provide baseline readability data against which future studies may compare their findings.

Data Description
Appendix A . The DISCERN Instrument, JAMA Benchmark Criteria and HONcode Criteria A.1. The DISCERN Instrument The DISCERN Instrument was developed by an expert group in the United Kingdom to allow consumers and information providers assess the quality of written health information. The DISCERN Handbook ensures all users are able to understand and apply the instrument effectively [1] . 16 questions address clarity, balance, and content of information with each question representing a separate quality criterion. The original tool has 16 questions, each rated on a 5-point scale (1 = definite 'no'; 2-4 = 'partially meets criteria'; 5 = definite 'yes'). The first eight questions relate to the reliability of the publication and seven questions address specific details of the information about treatment choices. The final question is an overall quality rating. Similar to Weil et al. [2] ., we omitted the final question to obtain a minimum score of 15 and a maximum score of 75. Websites were then classified by their total score as 'excellent' (63-75), 'good' (51-62), 'fair' (39-50), 'poor' (27-38) or 'very poor' (15-26).
A.2. The Journal of the American Medical Association (JAMA) Benchmark Criteria The Journal of the American Medical Association (JAMA) benchmark criteria, originally published by Silberg et al. [3] , represent four core standards to determine whether a source of information is credible, reasonable or useful: authorship, attribution, currency and disclosure. Authorship requires that the authors and contributors provide their affiliations and credentials. Attribution entails listing of references and sources for all content and indicating any relevant copyright information. Currency refers to the provision of the dates that content was posted and updated. Disclosure ensures that websites have properly demonstrated 'ownership' with any sponsorship, advertising, underwriting, commercial funding arrangements or potential conflicts of interest noted. One point is allocated for each core criterion that is met, with a maximum score of 4.
A.3. The HONcode Criteria The Health on the Net Foundation is a non-profit, non-governmental organization, which seeks to establish ethical standards for publishing medical and health-related information on the internet. The Health On the Net code (HONcode) seal accredits websites that agree to comply with eight core standards (the HONcode Criteria [4] ) and publish transparent health related information.
An Internet search provided the primary data source for assessment. Four search engines (Google, Yahoo!, Bing and Ask) were searched for the terms "high tibial osteotomy " and "knee osteotomy ." Forty websites were analyzed from Google and ten each from Yahoo!, Bing and Ask. Following exclusion criteria and removal of duplicated websites, forty five unique websites were available for analysis.
Readability of a text is defined as "the determination by systematic formulae of the reading comprehension level a person must have to understand written materials [5] ." The scales used in our study include seven reading grade level (RGL) scores (the Flesch-Kincaid Reading Grade Level [FKGL], Raygor Estimate, SMOG, Coleman-Liau, Fry, FORCAST and Gunning Fog) and one index score (the Flesch Reading Ease Score [FRES]). RGL is reported as a United States grade level, which denotes the years of education (based on the U.S. educational system) required to easily read and understand a piece of text. For each website evaluated, the seven RGL tests generated seven RGL scores, as well as a mean RGL. The FRES formula presents readability as an index score, based on sentence length and number of syllables. The score ranges from 0 to 100, with a higher score indicating easier readability.
'Wordy' items are either complex words or phrases that contain too many words and therefore require more advanced reading skills. A complete list of all 'wordy' items encountered is provided, along with suggested alternatives to improve text readability.
A summary of important linguistic units that influence readability scores is presented ( Fig.  2 ). This provides an 'at a glance' overview of key elements analyzed by the readability formulae.
Readability Raw Data Document. Raw data from the readability analysis of the 45 unique websites is provided. This secondary data is provided as: • Readability Scores • Raw scores of each of the 8 readability formulae for each website.
• Summary scores of each readability formula for all 45 websites.
• Grade score summary for each website.
• Cloze score summary for each website. •

Experimental Design, Materials and Methods
Four search engines (Google, Yahoo!, Bing and Ask) were searched for the terms "high tibial osteotomy " and "knee osteotomy ." To reflect the disproportionate use of the search engines [6] , we analysed the first 40 websites from Google and 10 each from Yahoo!, Bing and Ask. Searches were performed on April 08, 2020. Websites were excluded from further analysis if duplicate findings were noted between search engines, if the site was inaccessible, if the site was solely for advertisement purposes or if it was in video format. A total of 45 unique websites were analysed ( Table 1 ).
All text from the articles was copied and pasted into separate Microsoft Word documents (Microsoft, Redmond, WA). All hyperlinks, pictures, advertisements, copyright notices, and any other text that was not directly related to health information were removed. Readability of the re-formated documents was then analyzed with Readability Studio Professional Edition, Version 2019 (Oleander Software Ltd.). All reading grade levels (RGL) were reported as a United States grade level, which denotes the years of education (based on the U.S. educational system) required to easily read and understand a piece of text. We selected eight different instruments to assess readability ( Table 2 ). These included seven RGL tests (Flesch-Kincaid Reading Grade Level [FKGL], Raygor Estimate, SMOG, Coleman-Liau, Fry, FORCAST and Gunning Fog) and one Index Score (the Flesch Reading Ease Score [FRES]). For each website, the seven RGL tests generated seven RGL scores, as well as a mean RGL. The FRES formula calculates the readability of a document, expressed as an index score, based on sentence length and number of syllables. The score ranges from 0 to 100, with higher scores indicating an easier readability. All websites were also analyzed for the percentage of complex words, long words, Dale-Chall unfamiliar words, Fog words, as well as the number of 'wordy' items, overly long sentences and longest sentence             'Wordy' items are complex words and phrases that contain too many words.

Fig. 2.
Linguistic unit summary of all articles Complex words are words with ≥ 3 syllables; Long words are words with ≥ 6 characters; Fog Words are words that contain ≥ 3 syllables that are not proper nouns, combinations of easy or hyphenated words, or two-syllable verbs made into three by adding -es and -ed endings; Dale-Chall unfamiliar words are words that do not appear on a list of 30 0 0 common words that are known to most 4th-grade students; Overly long sentences are defined as those with a word count greater than 22 words.
length. Complex words are defined as words with ≥ 3 syllables and long words as those with ≥ 6 characters. Dale-Chall unfamiliar words are defined as those that do not appear on a list of 30 0 0 common words that are known to most 4th-grade students. Fog words are words that contain ≥ 3 syllables that are not proper nouns, combinations of easy or hyphenated words, or two-syllable verbs made into three by adding -es and -ed endings. 'Wordy' items include complex words and phrases that contain too many words. Overly long sentences are defined as those with a word count greater than 22 words. Experimental design is outlined in Fig. 1 .

Ethics statement
This study analyzed the educational material on publicly accessible websites and as such was exempt from IRB review.