Evaluation of Online Written Medication Educational Resources for People Living With Heart Failure

Background Patient educational resources on heart failure (HF) medications may improve patient understanding, which is critical for informed decision-making and patient self-efficacy. The purpose of our study was to evaluate the quality and readability of written medication educational resources available online. Methods Two investigators searched Google, Yahoo, and Bing for written patient educational resources that addressed at least one HF medication. We assessed educational quality using the Ensuring Quality Information for Patients (EQIP) tool (range 0 [worst] to 100 [best]), and we evaluated readability using the Flesch-Kincaid Grade Level. Results From 693 identified webpages, 39 HF medication educational resources met study eligibility. Among included resources, the median Ensuring Quality Information for Patients score was 61% (interquartile range 54%-68%), with 2 (5%) rated as high quality (score ≥ 75%). The median Flesch-Kincaid Grade Level was 8 (interquartile range 8-12), with 4 (10%) resources meeting the recommended 6th-grade reading level. Conclusions Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.

Heart failure (HF) affects approximately 750,000 Canadians. 1 Despite the high morbidity and mortality rates associated with HF, it can be successfully managed with a combination of healthy behaviours, medical devices, and medications. 2 The variety and complexity of HF treatment options may result in patients and their caregivers feeling overwhelmed, which can negatively impact patient outcomes. 3,4 One intervention to overcome this issue is patient education about HF, their disease condition, and available treatment options. 4,5 Provision of written patient educational resources may help improve patient understanding of their condition and treatment options. 6 Various types of educational resources exist, including those designed to enhance patient knowledge (eg, informational pamphlets, patient handouts) and specifically to support shared decision-making (ie, decision aids). Regardless of the type, ideally, written educational resources regarding HF should facilitate discussions between patients and their healthcare providers, should be based on high-quality evidence, and should be developed with the patient's informational needs and health literacy in mind. 4 Emerging evidence documents the suboptimal quality and readability of patient educational resources. [7][8][9][10][11][12][13][14] For instance, a 2014 review assessed the quality of available patient educational resources on left ventricular assist devices, for patients with advanced HF, using the Flesch-Kincaid Grade Level score, the Fry algorithm, and a modified version of the International Patient Decision Aid Standards. 8 This study found that most educational resources available online that address left ventricular assist devices are of suboptimal quality. 8 Despite the fact that pharmacotherapy is the cornerstone of HF management, educational resources on HF medication have not yet undergone the same level of rigorous evaluation. Therefore, the purpose of our study was to collect, categorize, and evaluate the quality and readability of online written patient educational resources regarding HF medication.

Search strategy
From May 2021 to June 2021, 2 investigators (S.L. and F.Y.) independently conducted simultaneous, comprehensive searches of the top 3 Internet search enginesdGoogle, Yahoo, and Bingdusing Google Chrome (the most commonly used browser) to collect Internet-based educational resources about HF medications. 7,9,11,15,16 Search queries included the following: (i) "Heart failure medication patient information"; (ii) "Heart failure medications patient information"; (iii) "Heart failure medication patient handout"; (iv) "Heart failure medications patient handout"; (v) "Heart failure drugs patient information"; and (vi) "Heart failure drugs patient handout". The search queries used were developed through discussion and collaboration between the researchers, with the goal of replicating search terms that patients looking for this information might use (ie, more general, broad terms, in lay language). The investigators used an empirical method to minimize the influence of prior search history, which involved clearing the cookies from their web browsers and entering "incognito mode" prior to each search. To emulate typical Internet searches of patients looking for health information, we considered the results from the first 2 pages of each search. 11,[17][18][19] The number of results per page was 10 for Google and Yahoo, and 8-10 for Bing, depending on ads. Additionally, we manually searched Web sites from key cardiovascular, HF, pharmacy, and patient advocacy organizations; HF medication manufacturers; and the Ottawa Decision Aid Inventory (Supplemental Table S1). Key Web sites were defined as being those that are well known Web sites of cardiovascular and HF societies, prominent Canadian cardiac centres, and health authorities. Evaluators assessed resources as a whole and multiple pages, if relevant, but hyperlinks to different URLs were not assessed. Further, S.L. and F.Y. individually removed any intra-assessor duplicate resources from their results prior to combining their lists. After the results were combined, between-assessor duplicates were removed. Lastly, although we were unable to determine the exact intention of the resources evaluated, the inferred intention was that they were developed to provide patient education. Therefore, we included resources irrespective of their stated (or unstated) intention if they met all the eligibility criteria below.

Eligibility criteria
We included educational resources that met the following criteria: (1) written resources directed at patients; (2) written in English; (3) included 10 sentences about HF medications (as per Iacovetto et al. 2014 8 ) to ensure inclusion of resources with substantive content on medications; (4) accessible without having to register and/or pay a fee; (5) specifically described 1 different HF medication treatment option (i.e. individual medication or drug class); and (6) provided medication information specific to their use in HF.

Primary outcome: EQUIP score
The primary outcome was educational resource quality, based on the Ensuring Quality Information for Patients (EQIP) score. EQIP is a validated 20-item questionnaire first described in 2004 that measures the quality of patient information on a  scale from 0% (worst) to 100% (best) by assessing the domains of completeness, appearance, understandability, and usefulness (Supplemental Appendix S1). 5 Each question has 3 possible answers with an accompanying score of 1 (yes), 0.5 (partly), or 0 (no). The total EQIP score was determined by averaging individual items' scores using the following formula: [((# of Yes*1) þ (# of Partly*0.5) þ (# of No*0))/20 e (# of does not apply))] *100 ¼ % score. 5 Domains that were not applicable to a given resource were excluded from the overall score calculation. Two reviewers (F.Y. and S.L.) evaluated each resource using the EQIP score, with a discrepancy between reviewers defined as a difference of 10% in EQIP score. Discrepant scores, which could occur due to the subjective nature of some of the questions, were reviewed by the 2 reviewers in order to reach a consensus, with disagreements resolved by a third author (R.T.). Following resolution of discrepant scores, the 2 EQIP scores calculated for each resource were averaged. This final averaged score was used in the analysis and to categorize each resource as one of the following: (i) well written, high-quality (score of 76%-100%); (ii) good quality with minor problems (score of 51%-75%); (iii) serious problems with quality (score of 26%-50%); and (iv) severe problems with quality (score of 0%-25%), consistent with categories recommended in the original EQIP development publication. 5

Secondary outcome: readability
Readability was assessed primarily using the Flesch-Kincaid Grade Level score, and secondarily, via the Flesch Reading   scores have previously been validated and used to measure the readability of patient educational resources. 8,12,13 In the US, an estimated 61% of adults read at a 6th-grade level, 9 and it is therefore recommended that patient information resources be written at a 6th-grade or lower reading level. 12,13,20 For this study, HF medication patient educational resources with a higher than 6thgrade reading level were considered to have low readability. We further used a Flesch Reading Ease score < 80 to indicate low readability. 13,21,22 Lastly, we collected data on each resource's total word count, for descriptive purposes.

Statistical analysis
We performed descriptive analyses, and we report data using medians and interquartile ranges (IQRs) for continuous variables, and frequencies and percentages for categorical variables. All analyses were performed using Microsoft Excel 2016 (Microsoft, Redmond, WA).

Results
From 693 search results, we identified 39 HF medication educational resources that met our eligibility criteria (Fig. 1). Despite the methods employed and the fact that the searches conducted were identical, F.Y.'s and S.L.'s search results differed slightly, likely due to differences in geographic location (Alberta vs British Columbia). Key characteristics of HF medication educational resources are described individually in Table 1, and summary statistics are provided in Table 2.

EQIP
The median EQIP score was 61% (IQR 54% to 68%; Table 2; Fig. 2A). Of the 39 included educational resources, 2 (5%) were of high quality, 30 (77%) were of good quality with minor problems, and 7 (18%) had serious problems with quality. Assessment of the specific components of the total EQIP score revealed that 36 resources (92%) used lay terms, 38 (97%) addressed the reader personally, and 39 (100%) had a respectful tone. However, only 2 resources (5%; including the highest-scoring resource) reported consulting with patients/family members during development. In terms of medication-specific information, 38 resources (97%) addressed the purpose of described HF medications, 28 (72%) qualitatively described benefits (eg, decrease mortality), 28 (72%) qualitatively described the medication risks and side-effects, and 21 (54%) addressed alternative medications (eg, angiotensin-receptor blocker if cough develops while taking an angiotensin-converting enzyme inhibitor). Further, 34 resources (87%) used generic names (instead of, or in addition to, brand names) and distinguished brand names as such, whereas the remaining 5 resources (13%) only discussed medication classes. Lastly, 24 resources (62%) did not address quality-oflife issues, with the remaining 15 (38%) only partly addressing this item. Only 5 resources (13%) included visual content to supplement the text.

Readability
The median Flesch-Kincaid Grade Level was an 8 th -grade reading level (IQR 8th to 10th grade) (Fig. 2B). Only 4 resources (10%) achieved the recommended 6th-grade reading level, and no resource had a 5th-grade or lower reading level. The median Flesch Reading Ease score was 59 (IQR 52 to 65), with all resources demonstrating low readability based on this score.

Discussion
HF is a chronic, complex condition that can be treated using a variety of healthy behaviours, medical devices, and medications. 2 Provision of written resource for patients to review on their own time and at their own pace can significantly improve information retention, compared to verbal instruction. 4,5 In this study, we found that most online HF medication patient educational resources were of good quality with minor problems, based on the EQIP tool. However, most HF medication educational resources were written at a grade level beyond the reading level of most people in the general population.
We identified several key areas for improvement among HF medication patient educational resources. In general, resources had a respectful tone, used generic names of medications, presented information in a logical order, and qualitatively described the purpose, side-effects, benefits (though not quantitatively,) and alternatives of medications. However, few followed a summative flow (ie, outlining the document content and covering what is stated), involved patients and caregivers in their development, or addressed quality-of-life issues.
Having a clear definition outlining the document content can help patients, caregivers, and healthcare providers understand whether the resource is suitable for the patient's specific informational needs. 5,23,24 Next, patients and healthcare providers may have different perceptions of what information is important to convey in educational materials. 5,23,25,26 Therefore, involving patients and caregivers in development of resources can help tailor educational materials to the needs of patients, making them more useful and relevant to their target audience. Further, describing the quality-of-life and practical impacts of medications ensures resource transparency, enables realistic patient expectations, 5,27 and allows patients to better understand and implement self-care and management strategies. 5,24 A recent study established a framework for patient-oriented practical issues to be included in evidence summaries and shared decision-making tools. 28 Such practical considerations align closely with quality-of-life issues incorporated into the EQIP questionnaire (eg, how a treatment impacts a patient's daily routine or social life) and should be included in HF medication educational resources. 5,28 These details are also more likely to align with patients' individual goals of therapy (eg, improve symptoms, daily functioning, capacity for social interaction, etc.) and are crucial to include in patient education to engage patients in their care and medication management. 3 Our search identified resources with a wide range of readability, with few resources achieving sufficient readability. The readability level of HF medication educational resources thereby poses a problem, as patients may not be able to fully comprehend the included information, regardless of its quality. An additional consideration for barriers to use of educational resources is the length of time required to read through them. The average adult reads at a pace of approximately 250 words/minute. 29 Of the top 5 highest-scoring resources based on the EQIP tool, the median word count was 4778, indicating approximately 19 minutes of reading time. For the average patient, who may be new to their diagnosis and receiving an overwhelming amount of information, resource length is likely a substantial barrier to absorbing and implementing the included information. 3

Limitations
Limitations to this study warrant discussion. First, we restricted resource inclusion to written materials available online. We did not capture information presented within social media communities (eg, Reddit, Facebook, etc.), video resources (eg, YouTube), or local/clinic-specific resources that are not available online. We anticipate greater variability of quality of these resources, as they may be subject to less curation. Second, the EQIP instrument was open to interpretation, which resulted in initial variability in scoring between the 2 reviewers. However, we mitigated this variability by incorporating duplicate scoring and an a priori plan to identify and resolve discrepancies. Third, the EQIP questionnaire used for our primary outcome defines quality from the perspective of healthcare professionals, rather than from the perspective of the end-user (ie, the patient). Therefore, it may not encompass all domains of quality or treatment considerations that are important to patients, or capture the relative value that patients place on each domain.

Conclusions
Most HF medication educational resources available on the Internet are of acceptable educational quality, but could readily be improved. Most resources were beyond the recommended reading grade level for educational resources, limiting their utility for patients with a low literacy level.
Funding Sources