Assessment of the quality and content of clinical practice guidelines for post-stroke rehabilitation of aphasia

Supplemental Digital Content is available in the text


Introduction
Stroke is one of the most serious global health problems, which accounts for approximately 5.9 million deaths annually. [1,2] Additionally, it is ranked the third as a cause of disabilityadjusted life-years (DALYs) according to the study of global burden of disease 2010. [3] Most of stroke survivors have persistent difficulty with daily tasks owing to serious sequelae. Aphasia, an acquired language disorder, is a common consequence of stroke, which significantly affects the individual's life such as relationships, social engagement, and independence. [4,5] It is estimated that about 25% to 40% of stroke survivors acquire aphasia. [6,7] The presence of aphasia is associated with increased length of stay, general decreased response to stroke rehabilitation interventions and an increased risk for mortality. [8] Several treatments have been used to improve functional communication but there is still a gap between different researches. [9][10][11] For example, it was stated that there was no clear evidence for efficacy of speech and language therapy in one guideline, [12] whereas it was recommended for aphasia in another guideline. [13] The guideline is systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances, [14] which plays an important role in health making decision. [15] The usefulness of guidelines primarily depends on the quality, rigorous methodology, and transparency of development. [16] High-quality guidelines based on the best available research evidence can provide optimal recommendations and optimize outcomes. [17] Therefore, it is important to determine whether the recommendations are indeed based on high-quality evidence. [18,19] The Appraisal of Guidelines for Research and Evaluation II (AGREE II) is a reliable tool used to assesses the quality of guidelines, which developed to address the issue of variability in guideline quality. [20][21][22] It has been widely used for different guidelines in recent years. [23][24][25] Therefore, the purpose of this study is to assess methodological quality of guidelines on the management of post-stroke rehabilitation of aphasia by using AGREE II instrument and identify gaps limiting evidence-based practice and highlight potential opportunities for improvement.

Materials and methods
The study was conducted in accordance with guidelines from the Preferred Reporting Items for Systematic Reviews and Metaanalysis group (PRISMA). Ethical approval was not necessary as no human subjects were involved. ]. Additional sources of information were found through Google Scholar and pearling the reference list of included guidelines. Terms searched included "stroke," "aphasia," "dysphasia," "rehabilitation," "practice guideline," "guideline," "guideline, * " "recommendations," and "consensus." Appendix 1, http://links.lww.com/MD/D144 presented search strategy used in the PubMed database.

Guideline selection
Two reviewers (QC and XW) independently reviewed titles and abstracts to identify eligible records. Differences opinions were resolved by consensus. The inclusion criteria were as follows: met the definition of a guideline as "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances'; contained rehabilitation recommendations regarding aphasia after stroke; available in English or Chinese; published between 2010 and 2018. The guidelines on stroke prevention or for patients were excluded. If the guideline had >1 version, only the most recent version was assessed. For each guideline ultimately included, we thoroughly searched for accompanying technical and supporting documents to better inform our assessments.

Data extraction
Two reviewers (QC and XW) independently extracted relevant information from each eligible guideline. Disagreements were resolved by consensus. The following characteristics of the guidelines were collected: year of publication, location where the guideline creation took place, the organization that created the guidelines et al. Besides, the information about the recommendation contents for assessment and recommendations for aphasia and the grade recommendations were also extracted.

Assessment of guideline quality
We employed the AGREE II instrument to evaluate each guideline meeting our inclusion criteria. [26,27] According to AGREE II handbook, each guideline was scored on 23 items within 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability and editorial independence. Information relevant to the rating of each of the 23 items with the AGREE II instrument were extracted from the included guidelines using the online tool My AGREE PLUS, which was freely available and accessible from the AGREE Enterprise website (http://www.agreetrust.org/). [20] Each guideline was scored by 2 independent reviewers (HL and SH). Before the assessment started, each topic of AGREE-II was intensively discussed to achieve homogeneity by twice preevaluation. Reviewers assessed each item and assigned a score from 1 (strongly disagree) to 7 (strongly agree). Domain scores were calculated by summing up all the scores of the individual items in a domain and by scaling the total as a percentage of the maximum possible score for that domain. Upon completing the 23 items, the overall assessment required the user to make a judgment as to the quality of the guideline, considering the criteria considered in the assessment process. The overall assessment included the rating of the overall quality of the guideline and whether the guideline would be recommended for use in practice.
Consistency of evaluations of the AGREE II domain was calculated using a -way analysis of variance with single-rater 2way intra-class correlation coefficients (ICCs) with 95% confidence interval (CI) for each domain across all guidelines. [28] The degree of agreement between 0.01 and 0.20 was deemed minor, 0.21 and 0.40 fair, 0.41 and v0.60 moderate, 0.61 and 0.80 substantial, and 0.81 and 1.00 very good. [29] Data analysis was performed descriptively and using the calculation of the total score by each reviewer and the score per domain.

Guideline characteristics
A total of 5008 records were retrieved, 61 records were considered potentially eligible for full-text screening, and 8 guidelines proved eligible [12,13,[30][31][32][33][34][35] (Fig. 1). Four guidelines were updated, and the others were original. Guidelines were published from 2010 to 2016. Of 8 eligible guidelines, 3 guidelines were from UK, 2 from Canada, 1 from United States, 1 from Australia, and 1 from Singapore. Six guidelines reported the quality evidence and recommendation grading system. The     Table 2).
The values of ICCs ranged from 0.60 to 0.99. The ICCs were highest in the "rigor of development" domain (0.99) and lowest in "clarity of presentation." All 6 domains scored >0.6, which indicated the intra-reviewer item score agreement was good ( Table 2).

Recommendation comparison
Regarding the assessment of aphasia, most guidelines recommended that patients with suspected communication deficits after stroke should be assessed using a simple and reliable tool by a speech and language therapist. The guideline of NSF 2010 [32] suggested that the instrument of the Frenchay Aphasia Screening Test had greater sensitivity and specificity and had been widely used in European countries. [36] The guideline of NCGC 2013 [35] stated that the assessment of aphasia should be within 72 hours of onset of stroke symptoms.
Various treatment recommendations for aphasia were described among included guidelines. Intensive speech and language therapy were recommended in 3 guidelines ( [31] Canada CSBPR Stroke rehabilitation Self-making system Yes 11 Fifth edition NSF 2010 [32] Australia NSF Stroke management The National Health and Medical Council (NHMRC)

Yes 11
Third edition SIGN 2010 [33] UK SIGN Stroke rehabilitation SIGN system Yes 3 Second edition ISWP 2016 [34] UK ISWP Stroke management -No 7 Fifth edition NCGC 2013 [35] UK  Table 3). The pharmacotherapy recommendations for aphasia were provided only in 2 guidelines. One guideline (NSF 2010) stated the routine use of piracetam was not recommended. One guideline (AHA/ASA 2016) indicated pharmacotherapy for aphasia may be considered on a case-by-case basis in conjunction with speech and language therapy, but no specific regimen was recommended for routine use.
The quality of guidelines was heterogeneous. The domains that generally score poorly were "applicability" and "rigor of development." Rigor of development, the most critical domain, markedly influenced the confidence for guideline implementation. [37] The most guidelines failed to discuss the strengths and limitations of the body of evidence and address the procedure for updating the guideline. Four guidelines (CSBPR 2015, NSF 2010, SMH 2011, and GPAC 2015) did not presented the criteria for selecting the evidence and 3 guidelines (AHA/ASA 2016, SMH 2011 and GPAC 2015) were ambiguous regarding systematic methods. Besides, the score of applicability domain was disturbingly low. The main limitation was that most guidelines did not clearly describe the facilitators and barriers to its application. The highest rated domain was the scope and purpose, of which the median score was 95.8%. Most guidelines described the overall objective, specifically health question and the target population in detail.
Previously a systematic review [38] that assessed the quality of clinical guidelines for aphasia in stroke management showed significant variability in methodological rigor, reporting of guideline development processes and scope of coverage of recommendations. Our research found the field of scope and purpose had significantly improved in recent years. Most guidelines specified the overall aim of the guideline, specific health questions, and target population. But the domain of rigor of development still existed some deficiencies and needed to further improve.
There existed some discrepancies between management recommendations for post-stroke aphasia. First, guidelines provided different therapy recommendations on post-stroke aphasia, and the total number of recommendations was various. Second, the recommendation grades of guidelines were generally low. It was indicated that related high-quality evidence was insufficient. Third, there were discrepancies in the grading of recommendation and the quality evaluation of evidence. Guidelines were based on different recommendation grade systems, including the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and the National Health and Medical Council (NHMRC) (Appendix 2, http://links.lww.com/ MD/D144). Therefore, it was recommended that guidelines development could be based on trustworthy consensus statements and a robust and transparent process. [16] There were 2 guidelines that stated the time and duration of therapy. SIGN 2010 stated that a minimum of 2 hours per week and a minimum period of 6 months for speech and language therapy should be provided where the patient is sufficiently well and motivated. NSF 2010 described that amount and intensity of therapy for communication difficulties should be provided as patient can tolerate and the timing of treatment should be offered as early as tolerated. The others failed to explicitly provide advice about the time, intensity, and duration of specific treatment Table 3 Treatment recommendations for aphasia management.

Guideline
Main recommendations interventions, grade and level of evidence SMH 2011 [12] Speech and language therapy D * 3 † AHA/ASA 2016 [13] Speech and language therapy I * A † ; communication partner training I B; intensive treatment IIa A; computerized treatment IIb A; group treatment IIb B; pharmacotherapy IIb B; Brain stimulation techniques III B GPAC 2015 [30] Intensive speech and language therapy CSBPR 2015 [31] Intensive language and communication therapy B * ; computerized language therapy C; conversational treatment and constraint induced language therapy B; group therapy and conversation groups B NSF 2010 [32] Constraint-induced language therapy B * ; group therapy and conversation groups C; delivery of therapy programs via computer C SIGN 2010 [33] Speech and language therapy B * ISWP 2016 [34] Practicing language and communication with a speech and language therapist or other communication partner; assistive technology and communication aids; training the careers and family of patients NCGC 2013 [35] Speech measures. But this information was usually of great significance for clinical treatment and nursing. Therefore, it is suggested that future study may consider the duration and intensity of therapy. Five guidelines classified the different types of aphasia (Appendix 3, http://links.lww.com/MD/D144). Six guidelines presented recommendations for dysarthria, 4 described apraxia of speech recommendations. However, none of included guidelines specified the management recommendations of aphasia according to the type of stroke. Besides, there were several factors that might affect the rehabilitation of patient with aphasia, including the period of admission in the hospital, inflammation, [39][40][41] and comorbidities. It was suggested that future guidelines should consider these factors to develop more detailed guidelines to facilitate readers to easily understand the recommendations and facilitate the implementation of recommendations by relevant clinical personnel. The 8 guidelines included in this study were developed by developed countries, and none came from developing countries. As recommendations of guideline were generally based on their local health resources, their applicability to developing countries may be dramatically reduced. Moreover, the prevalence of stroke is on the rise in low-and middle-income countries (LMICs). Research [42] found that upper-middle income countries accounted for the largest prevalence of stroke; low-income countries had experienced the steepest increase in stroke prevalence. In low-income and middle-income countries, the number of DALYs lost in people younger than 75 years exceeded those lost in high-income countries by almost 5 times. [1] Unfortunately, the present related studies of post-stroke aphasia were concentrated in the developed countries. In regions with low resources, relevant studies were insufficient. Therefore, it is recommended that future research should pay more attention to the treatment and rehabilitation of stroke survivors in LMICs. And it is also supposed to develop stroke recovery guidelines suitable for LMICs as soon as possible to improve the quality of life of stroke survivors and reduce the disease socioeconomic burden of stroke.
Strengths of this study include comprehensive search strategy with the use of multiple databases and the use of a structured, validated assessment tool. And we included the newest guidelines for last 8 years. Our review also has some limitations. First, study only included English language guidelines. This may result in the exclusion of guidelines designed for use in non-English-speaking countries which may have been relevant. Second, AGREE II instrument focused on methods of guideline development and the transparency of reporting, but did not involve the judgment of the rationality of recommendation opinions.

Conclusion
The quality of guidelines for post-stroke aphasia needs to be improved, especially in the fields of rigor of development and applicability. Besides, the treatment recommendations of aphasia also existed difference among the included guidelines.
The rehabilitation of posts-stroke aphasia is a complex and long-term process, which requires multiple support and participation. Guidelines based on high-quality evidence could provide clinical nursing staff with the optimal clinical advice and reference. Therefore, it is suggested that the formulation of guideline should pay more attention to the rigor of methodology and applicability. Future research should focus on the effectiveness, intensity, and duration of treatment measures.