Accessibility and readability of online patient education on cutaneous lymphomas

Background Patients facing a cutaneous lymphoma diagnosis frequently turn to the internet for information but finding patient-accessible education may be a challenge. Objective To investigate accessibility and readability of patient-oriented online education on cutaneous lymphomas, including cutaneous T-cell and B-cell lymphoma subtypes. Methods This study queried a search engine for 11 cutaneous lymphoma terms, resulting in 1083 webpages. Webpages were screened using defined inclusion/exclusion criteria; literature directed to physicians and scientists was excluded. Webpages were stratified by academic/nonacademic and dermatology/nondermatology hosts and assessed by order of appearance. Readability, including text complexity, was analyzed for grade level understanding using 5 established calculators. Overall readability was assessed by Flesch–Kincaid Reading Ease. Results Academic webpages had earlier order of appearance. There was a dearth in dermatology-hosted webpages. Rarer cutaneous lymphomas yielded fewer patient-accessible results. Search term readability significantly exceeded the American Medical Association–recommended sixth grade level (P < .001∗), with higher grade levels for cutaneous T-cell lymphoma subtype webpages than cutaneous B-cell lymphoma subtypes. Limitations Webpage quality, accuracy, and language were not assessed. Conclusion Current online education for cutaneous lymphomas exceeds the American Medical Association’s maximum readability recommendation. There is a need for more patient-accessible education amidst predominance of scientific literature, greater dermatology host websites, and enhanced readability of existing online education.


INTRODUCTION
Most patients in the United States view online health resources upon diagnosis and for making healthcare decisions.Health Information National Trends Survey administered several times from 2008 to 2017, showed that nearly 69% of patients first use the internet to search for health information. 1This trend is particularly important for patients with rare diseases, such as cutaneous lymphomas.Cutaneous lymphomas are malignancies of T-and B-lymphocytes which primarily impact the skin.Primary cutaneous B-cell lymphomas (CBCLs) comprise approximately 25% of all primary cutaneous lymphomas in the United States, with cutaneous T-cell lymphomas (CTCLs) comprising nearly all the rest. 2 Although patients with early stage CTCL and CBCL generally have a good prognosis and an indolent disease course, 3 many patients experience significant health distress. 4This may, in part, be because of the uncertainty that accompanies a cancer diagnosis which may prompt patients to seek out additional information to make sense of their diagnosis and treatment options.
Online patient educational materials can be hosted or created by academic sources, such as hospitals, institutions, physicians, and academic organizations, or by nonacademic sources, primarily news sites and industry.Nonacademic pages may also provide insight into the patient's experience through personal stories on blogs and health sites.Availability of online materials hosted on dermatologic websites is of particular interest, given most diagnostic decisionmaking and multidisciplinary care coordination for cutaneous lymphomas are directed by dermatologists.
It is imperative that patient-oriented online education on cutaneous lymphomas is written in an accessible fashion, particularly given the complexity and rarity of many cutaneous lymphoma subtypes.The American Medical Association (AMA) recommends health information be presented at a maximum sixth grade reading level. 5The AMA recommendation for clinical practices with a high percentage of patients with limited literacy is third to fifth grade, noting the average reading skill of Medicaid enrollees is fifth grade level.
This study sought to assess both accessibility and readability of online educational materials that patients leverage in their pursuit of knowledge on cutaneous lymphomas, particularly CTCL and CBCL subtypes.To our knowledge, such a comprehensive and comparative investigation has not yet been conducted and published in the literature on cutaneous lymphomas.

METHODS
Internet searches for terms related to CTCL, CBCL, and their most common respective subtypes were performed through Google on incognito mode (limiting personalization/bias).Search terms were under 3 main categories: ''cutaneous lymphoma'' broadly, CTCL, and CBCL.For CTCL, cutaneous T-cell lymphoma, mycosis fungoides, S ezary syndrome, folliculotropic mycosis fungoides, lymphomatoid papulosis, and anaplastic large-cell lymphoma were queried. 6or CBCL, cutaneous B-cell lymphoma, follicle center lymphoma, marginal zone lymphoma, and diffuse large B-cell lymphoma were queried. 2 The first 10 pages of search results, as available, for each of the 11 total search terms were screened by inclusion and exclusion criteria.Criteria for exclusion included academic journal articles, articles with a paywall or subscription for access, videos (nonstill web media), books and encyclopedias, advertised search results, repeated links among search results, non-English articles, and clinic advertisements not featuring education.These criteria were set to exclude search results deemed inaccessible to patients, redundant, or not containing information on the term of interest.
Each included webpage was analyzed for academic/nonacademic hosting, order of appearance among search results, and dermatology/nondermatology hosting.Readability was assessed by established grade level calculators (FlescheKincaid Grade Level, Gunning Fog Score, SMOG Index, Coleman Liau Index, and Automated Readability Index) along with the FlescheKincaid Reading Ease score.FlescheKincaid Grade Level calculates readability as a grade-equivalent level, reporting American school grade required for reader comprehension. 5,7he Gunning Fog Index estimates years of formal education needed to comprehend a text passage upon initial reading, with a scale from 0 to 20. 7,8 It scores text with short sentences in plain English as better than longer sentences in complex language; the ideal Gunning Fog Index grading score is said to be 7 or 8, with anything higher than 12 deemed too complex for most people to read. 7The SMOG Index estimates years of education the average person needs to comprehend a piece of writing and was founded to improve upon the Gunning Fog Index. 7he Coleman Liau Index is designed to evaluate the American grade level necessary to understand a text, employing a formula based on characters (word length) instead of other syllable-based readability indicators. 7The Automated Readability Index is a grade level calculator derived from ratios representing sentence difficulty (number of words per sentence) and word difficulty (number of letters per word). 7Readability and text complexity were assessed via the WebFX Readability Test. 7Descriptive and comparative statistics were conducted in Microsoft Excel Version 16.66.1.

Search results, order of appearance, and website hosting
Following screening of 1083 webpages for the 11term web searches, 346 total webpages were included.There were 259 academic and 87 nonacademic webpages identified among the 11 cutaneous lymphoma search terms.Summary and comparative t test statistics for academic vs nonacademic webpages were computed for order of appearance (Table I).
Among all academic pages, 56 pertained to ''cutaneous lymphoma,'' 131 pertained to the CTCL terms, and 72 pertained to the CBCL terms (Supplementary Table I, available via Mendeley at https://data.mendeley.com/datasets/2f8rmmfxm5/1).Among all nonacademic pages, 8 pertained to ''cutaneous lymphoma,'' 46 pertained to the CTCL search terms, and 33 pertained to the CBCL search terms.Most criteria-excluded CTCL search terms were ''folliculotropic mycosis fungoides'' (4 academic and 7 nonacademic webpages) and ''lymphomatoid papulosis'' (9 academic and 5 nonacademic webpages); results mostly qualified for exclusion as academic scientific literature.''Lymphomatoid papulosis'' was the only study term to yield \100 results via the search engine, revealing relative dearth of accessible education.All other CTCL search terms, mostly reflecting more common subtypes, resulted in greater qualifying results (eg, at least 20 qualifying for inclusion among academic pages).These results highlight rarer subtypes having more excluded/ patient-inaccessible search results, potentiating worse patient comprehension.Comparing CTCL to CBCL, the number of qualifying academic webpages for the ''cutaneous T-cell lymphoma'' search alone is greater than twice the qualifying academic webpages for the ''cutaneous B-cell lymphoma'' search (48 vs 22).
Academic pages resulted more frequently than nonacademic pages, with an average order of appearance of 3.74 (range, 1.00-8.73;see Table I).Among nonacademic resources, the average order of appearance was 5.86 (range, 2.00-9.45).''Cutaneous lymphoma'' had an average academic search page order of appearance of 4.50 (vs 7.50 for nonacademic, P = .003*);CTCL and subtype search terms had an overall average order of appearance of 3.62 for academic vs 5.73 for nonacademic pages (P = .010*);and CBCL and subtype search terms had an average order of appearance of 3.74 for academic vs 6.07 for nonacademic pages (P \.001*).

Readability by established grade level and webpage text complexity
Academic vs nonacademic webpage stratification per search term was conducted for separate readability analysis, using established grade level readability calculators (FlescheKincaid Grade Level, Gunning Fog Score, SMOG Index, Coleman Liau Index, and Automated Readability Index).Differences among average grade level readability between academic and nonacademic webpages were recorded for ''cutaneous lymphoma,'' CTCL and subtypes, and CBCL and subtypes (Tables II and  III).Among all search terms, average readability grade level was higher for academic result webpages, as compared with nonacademic result webpages, for 3 out of 5 calculators (FlescheKincaid, Coleman Liau, and Automated Readability Index); academic webpage average grade levels ranged from 7.92 via Automated Readability Index to 15.79 via Coleman Liau (Supplementary Tables II and III, available via Mendeley at https://data.mendeley.com/datasets/2f8rmmfxm5/1).The other 2 grade level calculators showed nonacademic result webpages as scoring higher in average readability grade levels, 7.84 via SMOG Index and 10.03 via Gunning Fog (Supplementary Tables II and III), as compared with academic result webpages.Average readability Abbreviations used: AMA: Americal Medical Association CBCL: cutaneous B-cell lymphoma CTCL: cutaneous T-cell lymphoma of all search terms was found to significantly exceed the AMA's maximum recommendation of a sixth grade education (P \.001*).
Academic and nonacademic webpage results were also compared in grade level readability for CTCL search terms vs CBCL search terms.All grade level calculators, besides the Coleman Liau Index, scored CTCL search terms with higher average readability grade level than CBCL search terms; this trend showed higher average readability grade level for CTCL and CBCL search terms' resulting academic webpages, as compared with respective resulting nonacademic webpages.Among these academic results, average readability grade level for CTCL webpages ranged from 7.71 via SMOG Index to 9.83 via Gunning Fog Index, and for CBCL webpages ranging from 7.46 via SMOG Index to 9.44 via Gunning Fog Index (Supplementary Tables II and  III).The Coleman Liau Index for academic webpages for CTCL and subtypes vs CBCL and subtypes   II and III).The highest average Coleman Liau Index grade was 18.18 for ''folliculotropic mycosis fungoides,'' as calculated among academic webpages; ''Folliculotropic mycosis fungoides'' was also among the highest average readability grade levels for the FlesheKincaid calculator, with an academic webpage average of 10.20.Interestingly, the Coleman Liau Index calculator scored all search terms (among all webpages) with grade level averages requiring at least 13 years of education, necessitating patients have post-high school education for comprehension.
In addition, webpage text complexity was assessed through crude calculation for each search term, calculating percentage of complex words present on each webpage (Supplementary Tables II and  III).CTCL search terms had a higher percentage than CBCL search terms for academic webpages (22.58% vs 20.50%, respectively).However, for nonacademic webpages, CBCL search terms had a slightly higher percentage than CTCL search terms (22.84% vs 21.19%, respectively).''CBCL'' had an average text complexity of 26.11% among nonacademic pages and ''folliculotropic mycosis fungoides'' had an average text complexity of 26.07%among academic pages, representing 2 of the highest percentages of text complexity.

Overall readability by FlescheKincaid Reading Ease
FlescheKincaid Reading Ease is one of the most widely used measures of readability, even incorporated by United States military to evaluate their manuals. 7It is based on ranking between 0 and 100, with lower scores indicating less ease of text comprehension.Among all result pages, medians tended to be approximately 40 to 50, with an average median value of 45.42 for academic pages and 48.65 for nonacademic pages (Figs 1 and 2); this indicates all medians scored within college-level readability (30.0-50.0). 9 On average, CTCL search terms produced lower median FlescheKincaid Reading Ease scores among academic pages, as compared with CBCL search terms (CTCL average of medians 44.11 vs CBCL average of medians 47.41); this trend was also reflected among nonacademic result pages (CTCL average of medians 47.90 vs CBCL average of medians 49.60, see Supplementary Material, available via Mendeley at https://data.mendeley.com/datasets/2f8rmmfxm5/1).Overall, scores should be within 80 to 90 to meet the AMA's maximum recommendation of a sixth grade education, which all box plots failed to display. 9

DISCUSSION
In this study, the accessibility and readability of patient-oriented online education on cutaneous lymphomas were investigated for both CTCL and Results show earlier order of appearance for academic webpages than nonacademic host webpages.The result is within expectation: search engine prioritizing academic webpages as ''most relevant'' and/or ''most accessed.''Importantly, nonacademic webpages can highlight the patient experience/priorities, warranting study inclusion for accessibility.Our study highlights a dearth of search results from dermatology host websites, with most websites possessing oncology affiliations.One explanation may be that websites and/or organizations dedicated to cutaneous lymphomas are considered more relevant to oncology rather than dermatology (Supplementary Attachment).This could alternatively be attributed to limited representation, research, or sponsorship of cutaneous lymphomas in the field of dermatology; this presents an opportunity for the field to increase such representationdespecially since most cutaneous lymphoma care coordination is conducted by dermatologists.
Among readability calculators, cutaneous lymphoma search terms universally produced readability results that significantly exceeded the AMA's maximum sixth grade recommendation.1][12] These studies also found readability of respective online health resources exceeding AMA recommendations.Our study analyzed cutaneous lymphoma resource readability with greater granularity; comparing CTCL vs CBCL and between subtypes, we found CTCL subtype webpages mostly written for slightly higher education levels than for CBCL subtypes.The FlescheKincaid Reading Ease and Coleman Liau Index demonstrated readability scores even exceeding high school education; these scores could impress different formulaic approaches, although in line with other scores exceeding AMA Cutaneous lymphoma search term reading ease of academic pages.The boxplots showcase overall FlescheKincaid Reading Ease scores for academic page results of the cutaneous lymphoma search terms.The ease score is based on a ranking of 0 to 100, with lower scores indicating text is more difficult to comprehend and higher scores indicating greater ease of comprehension.The red line indicates a score of 85, between the recommended score range of 80 to 90.Cutaneous lymphoma search term reading ease of nonacademic pages.The boxplots showcase overall FlescheKincaid Reading Ease scores for nonacademic page results of the cutaneous lymphoma search terms.The ease score is based on a ranking of 0 to 100, with lower scores indicating text is more difficult to comprehend and higher scores indicating greater ease of comprehension.The red line indicates a score of 85, between the recommended score range of 80 to 90.Greater variability can be observed among the nonacademic webpages' box plots, as compared with those of the academic webpages in Figure 1.
recommendations.This trend was consistent across all grade level analytic tools, along with webpages presenting high text complexity.
To rectify this problem, organizations could implement readability evaluations.Websites could earn ''certification'' if content is presented at an appropriate reading level.Certified websites could be distributed to patients, which would incentivize organizations to improve readability to earn greater patient visibility.To improve readability, we suggest authors identify complex words and replace them with more readable words, possibly using artificial intelligence algorithms as guidance. 13For example, the algorithm may switch out ''lymphocyte'' for ''white blood cell'' or ''malignancy'' for ''cancer.'' One study limitation is that webpage quality was not evaluated.A recent study focusing on solely the most visible webpages for CTCL found approximately 25% of investigated webpages met Health On the Net code of conduct certification (HONcode), possessing higher quality information and adhering to specified ethical principles.The study found varying quality and popularity of websites, noting highly accessed websites tended to provide reliable patient information. 14Accuracy of patient-oriented education may be assessed in future investigations.6][17] Accessibility challenges are amplified for patients that do not speak their country's native language(s).Moreover, states and countries have different literacy levels, underscoring the equitable issue of readability.Future studies should investigate accessibility and readability from different geolocations or languages.
Given the rarity of cutaneous lymphomas and increasing dependence on online resources for patient education, we urge an increase in patientoriented education, greater webpage hosting by dermatology host websites, and enhanced readability of existing online patient-oriented education.Overall, this serves as an inflection point for the future of patient education in dermatology: toward pursuit of creating and populating more patientaccessible online education.

Fig 1 .
Fig 1.Cutaneous lymphoma search term reading ease of academic pages.The boxplots showcase overall FlescheKincaid Reading Ease scores for academic page results of the cutaneous lymphoma search terms.The ease score is based on a ranking of 0 to 100, with lower scores indicating text is more difficult to comprehend and higher scores indicating greater ease of comprehension.The red line indicates a score of 85, between the recommended score range of 80 to 90.

Fig 2 .
Fig 2.Cutaneous lymphoma search term reading ease of nonacademic pages.The boxplots showcase overall FlescheKincaid Reading Ease scores for nonacademic page results of the cutaneous lymphoma search terms.The ease score is based on a ranking of 0 to 100, with lower scores indicating text is more difficult to comprehend and higher scores indicating greater ease of comprehension.The red line indicates a score of 85, between the recommended score range of 80 to 90.Greater variability can be observed among the nonacademic webpages' box plots, as compared with those of the academic webpages in Figure1.

Table II .
Grade level calculator readability of academic pagesThe above table summarizes grade level readability of academic result pages for the chosen ''search term'' or group of cutaneous lymphoma terms.Analysis results from 1-sided 1-sample t tests against the AMA maximum sixth grade reading level recommendation are reported.

Table I .
Search page order of appearance among academic and nonacademic pages The above table summarizes search page order of appearance per cutaneous lymphoma ''search term'' or group, stratifying among academic and nonacademic results.Analysis results from 2-sided 2-sample t tests for equal variances are reported.CBCL, Cutaneous B-cell lymphomas; CTCL, cutaneous T-cell lymphomas.*Indicates significance with P \ .01. y Indicates significance with P \ .05.

Table III .
Grade level calculator readability of non-academic pagesThe above table summarizes grade level readability of nonacademic result pages for the chosen ''search term'' or group of cutaneous lymphoma terms.Analysis results from 1-sided 1-sample t tests against the AMA maximum sixth grade reading level recommendation are reported.CBCL, Cutaneous B-cell lymphomas; CTCL, cutaneous T-cell lymphomas.*Indicates significance with P \ .01.