Diversity of approaches in assessment of executive functions in stroke: Limited evidence?

Stroke is a leading cause of disability worldwide. Cognitive functions and, in particular, executive function, are commonly affected after stroke, leading to impairments in performance of daily activities, decrease in social participation and in quality of life. Appropriate assessment and understanding of executive dysfunction are important, firstly to develop better rehabilitation strategies for executive functions per se and secondly to consider executive function abilities on rehabilitation strategies in general. The purpose of this review was to identify the most widely used assessment tools of executive dysfunction for patients with stroke, and their psychometric properties. We systematically reviewed manuscripts published in English in databases from 1999 to 2015. We identified 35 publications. The most frequently used instruments were the Stroop, Digit Span and Trail making tests. Psychometric properties were described for the Executive Function Performance Test, Executive Clock Drawing Task, Chinese Frontal Assessment Battery and Virtual Action Planning — Supermarket, and two subtests of the Cambridge Cognitive Examination — Revised. There is a paucity of tools to reliably measure executive dysfunction after stroke, despite the fact that executive dysfunction is frequent. Identification of the best tools for executive dysfunction assessment is necessary to address important gaps in research and in clinical practice.


Introduction
Stroke is characterized by fast and focal development of neurologic symptoms that reflect loss of brain function due to ischemia or hemorrhage [1,2]. Strokes were responsible for 10% of all world deaths and 4% of loss of disability-adjusted life years in 2010 [3]. While agestandardized incidence of stroke significantly decreased by 12% in high-income countries, it increased by 12% in low-middle income countries between 1990 and 2010 [3]. Also, the number of stroke survivors increased by 84% in low-and middle-income countries between 1990 and 2010 [3].
More than two-thirds of patients with stroke have limitations to live independently. Stroke can cause a catastrophic impact in patients' lives, due to impairments in physical and psychological functions, as well as in cognitive, perceptual and communication skills [4]. Burden is not only caused by the direct deficits caused by acute stroke, but also by dementia. In a recent meta-analysis, rates of dementia after stroke varied from 9.1% (in population-based studies) to 14.4%, (in hospital-based studies) [5]. Cognitive dysfunction strongly contributes to disability and loss in quality of life. It can also often be a barrier for returning to work. Because cognitive impairments are 'invisible', patients have less awareness of them and it is more difficult to recognize the deficits in the workplace so that the necessary adjustments can be made. Studies in post-stroke patients often report scores in the Mini-Mental State Examination (MMSE) for cognitive evaluation, but this test is insensitive to detect executive dysfunction and does not capture more subtle deficits in cognition [5].
Executive function involves planning, problem solving, dealing with new situations, decision-making and performing complex tasks [10]. These functions are part of the cognitive process of acquiring, keeping and applying knowledge to behavior [11]. Theories about executive function have been proposed: (1) Single system: believes that injury of a single executive function process leads to impairment; (2) Constructled: working-memory and fluid-intelligence are the most important functions; (3) Multiple process: Executive functions are composed of different functions and processes working together during daily activities, but it is possible to evaluate each function separately; and (4) Singlesymptoms: there are two symptoms that are common in patients with EF deficits: confabulation (characterized by impairment in memory control) and multitasking (patients with impairments in higher-level functions, leading to a problems in organizing/planning daily routines) [11]. Executive function roles can also be divided into: (1) shifting: the capacity to initiate different tasks at the same time and return to each one; (2) updating: to monitor information and organize it according to a different objective, and retrieve it when necessary; and (3) inhibition: to inhibit one stimulus and focus on a task or problem [12]. These three functions are connected and can interfere with one another. Moreover, they contribute to performance of more complex executive functions and influence rehabilitation outcomes [12,13].
We conducted a review of the literature about tools for assessment of executive functions in stroke to identify tests performed in common practice worldwide and to determine the most appropriate evaluation instruments according to their psychometric properties, using an evidence-based approach.

Methods
This study was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses -PRISMA [14]. One investigator reviewed the manuscripts. Manuscripts published in English in Medline, Pubmed and Google Scholar from 1999 to 2015 were searched using the terms "stroke" and each of the following: executive function, executive dysfunction, working memory, rehabilitation or cognition. During the research we also considered the title, key word and abstract of the papers before including them in this review. Only original papers were considered.
We only included studies in which the primary goal was assessment of executive dysfunction in adults (18-90 years). Abstracts that did not specifically report the assessment of executive function or executive dysfunction, or in which executive function or dysfunction were not the primary objective, were excluded. Abstracts reporting on a mix of neurological disorders rather than stroke specifically were also not included. For those abstracts fulfilling the inclusion criteria, the full manuscripts were obtained to extract the methodological details listed below. In addition to the manuscripts identified by the key word search, we also included papers cited in other systematic reviews or meta-analysis.
We checked frequencies of use of the different tools of evaluation across studies. In addition, tools for evaluation of executive dysfunction were grouped in tables according to time from stroke described in the studies: less than one month, one-six months and six months or more. Tables 1 to 4 describe: (1) references, including authors and country of research; (2) instruments used for executive function assessment; (3) objective; (4) sample characteristics, taken from the methods sections of the original papers; including whether or not performance of patients was compared to performance of controls and if the lesion site was described; (5) psychometric data of the instruments: internal consistency (Cronbach's α), reliability (test-retest and inter-rater; intraclass correlation coefficient, ICC, or Pearson's r), concurrent validity (evaluated with Spearman's rho or Pearson's r) [15], and (6) main results: the relation between executive dysfunction and other conditions (such as depression or motor function). Table 5 describes the abbreviations of the tools listed in Tables 1 to 4.
The results of 35 papers were organized according to time from stroke: Table 1 (up to one month, 11 studies), Table 2 (one-six months; eight studies) and Table 3 (6 months and more, eleven studies). Five manuscripts did not specify the period of evaluation after stroke onset ( Table 4).
Modified versions of tests were reported in 6/35 manuscripts: Chinese Frontal Assessment Battery [24,25] Modified Wisconsin Card Sorting Test [16,17,25] and Swedish version of the Executive Function Performance Test [26].
One or two subtests of the following instruments of evaluation were applied in 7/35 studies: Initiation-perseveration subtest of the Mattis Dementia Rating Scale [24]; Zoo map, from the Behavioral Assessment of Dysexecutive Syndrome [21]; Digit Span Backwards [21,27]; Colorword Interference subtest [28,29] and Executive Functions subtests of the Cambridge Cognitive Examination -Revised [30]. These subtests were used in addition to other tests to evaluate executive dysfunction.
Detailed psychometric properties of the Executive Function Performance Test were described for the English (six months or more after stroke and up to one month after stroke) and Swedish (up to one month after stroke) versions [22,26,28]. This test includes the following tasks: simple meal preparation, making a telephone call, paying the bills and taking medication. Therefore, it evaluates initiation, organization, sequence, safety and judgment, and conclusion of the task. The test had excellent inter-rater reliability (ICC = 0.91) [22,28]. Internal consistency was also excellent (α = 0.94) [15].
Concurrent validity was poor when Executive Function Performance Test scores were correlated with scores in the Animal Fluency, Trail Making Test-B, Digit Span tests and the Functional Independence Measure [22], and moderate when correlated with the Assessment of Motor and Process Skills [26].
Psychometric properties of other executive dysfunction tests, listed below, were reported in four more studies (Tables 1-4): • Cambridge Cognitive Examination -Revised [30] (  Relations between executive dysfunction and other conditions were discussed in 15/35 manuscripts: depression [17][18][19][33][34][35], emotional incontinence [25], motor impairment and rehabilitation [20,21,27,36,37], coping [29], employment/productivity outcomes [38] and driving performance [39]. Altogether, patients with executive dysfunction presented poor performance on physical tasks often had depression and difficulties to return to a productive life. Long-term antidepressant treatment improved executive function in patients with stroke [34]. Only a few studies mentioned effects of antidepressants in their studies [18,35]. On the other hand, executive dysfunction was not significantly correlated with emotional incontinence or coping after stroke.

Discussion
The present study demonstrates that, in contrast to the extensive literature on motor function after stroke, the number of studies specifically evaluating executive function/dysfunction in stroke is relatively small (N = 35). These studies used a diverse range of assessment tools, and only a few provided adequate information on the psychometric properties of the tests employed. Literally all studies were performed in high-income, mostly western countries, despite the fact that most strokes occur in low-and middle-income countries. The study further shows that various assessment tools were used for different stages of recovery. And while most studies reported time since stroke at the point of assessment, the evidence-base available does not specifically address how stroke chronicity might have affected test performance. Several papers further highlighted the shortcomings of the evidence base concerning the association between executive dysfunction and other types of impairment, as well as rehabilitation interventions [18][19][20][21]25,29,34,35,37]. Moreover, more studies with larger sample sizes are necessary to improve the evidence base, and indeed, clinical practice in the area of executive function in stroke. The most frequently reported tools for assessment of executive function were the Trail Making Test, Stroop test, and Digit Span tests. They evaluate different aspects of executive functions. Digit Forward is a measure of attention and immediate memory, while Digit Backwards is related to more complex attention and working memory. Trail Making Test A is related to processing speed and flexibility, while Trail making B is a measure of mental flexibility, sustaining, shifting and dividing attention. The Stroop test is used to verify inhibitory control. Together these tests therefore provide an easy and quick to administer,   [49,50], Swiss-German [51], Portuguese [52,53], and Italian [54]. Psychometric properties of the Trail making test were further described for patients with brain damage in various languages [55,56]. Future research is required to define internal consistency, inter-rater and test-retest reliabilities, as well as discriminating, construct and concurrent validities of these instruments in patients with stroke, at different stages of recovery and with particular lesion locations. Moreover, it is important to consider the influence of motor or language impairment on test performance as well as the influence of educational level. The latter is particularly important in low-income countries where reading and writing skills are often quite poor.
Psychometric properties in patients with stroke were comprehensively described for the Executive Function Performance Test. This test had excellent inter-rater reliability and internal consistency. It was compared with measures of executive functions (Animal Fluency, Trail making Test-B, Digit Span tests and the Functional Independence Measure) [22], as well as performance-based tools for Activities of Daily Living and executive functions (Assessment of Motor and Process Skills) [26]. Concurrent validity was found to be moderate.
Besides the Executive Function Performance Test, psychometric properties were described for only four other instruments in patients with stroke. Test-retest reliability was reported to be moderate for the Executive Clock Drawing Task and was not described for other tests. Inter-rater reliability was excellent for the Chinese Frontal Assessment Battery, but not reported in other tests. Internal consistency was very good for the Chinese Frontal Assessment Battery. Moderate concurrent validities were described for executive functions subtests of the Cambridge Cognitive Examination -Revised [20], Chinese Frontal Assessment Battery [24] or Virtual Action Planning -Supermarket [32] and other tests.
As the discussion above highlights, psychometric data for executive function tests in stroke are insufficient, and this is a recognized demand. The NIH EXAMINER represents a new test battery for executive functions for neurological disease which has recently been validated in a multicentre study validated with 1248 participants (included 485 participants below the age of 18 years and 763 participants 18 years and older) [57][58][59]. However, patients with stroke were not included. Considering the impact of executive dysfunction in patients with stroke, we suggest that this instrument should be translated, validated and adapted to different cultures worldwide.
Executive function is a complex cognitive domain that influences and is influenced by other human functions such as behavior and emotional, motor and other cognitive domains. Executive dysfunction can compromise functional status, due to the interaction with other conditions such as depression [18,19,34,35], emotional incontinence [25], motor impairments [20,21,27,36] and driving performance [39]. In addition, impairment in these conditions can lead to poorer quality of life. Moreover, executive function can interfere in daily routine, including the capacity to deal with unfamiliar situations in new environments.
In particular, executive function is frequently impaired in elderly patients with depression [60], and is a predictor of depression [61,62]. Even after treatment of depression, impairments in executive functions can persist and are associated with worse outcomes [60]. Depression is common after stroke, with prevalence ranging from 25% to 70% in all survivors [63]. Therefore, patients may simultaneously present executive dysfunction and depression (co-morbid prevalence of 18.3%) [17][18][19]33,61]. Considering that depression and executive dysfunction interact with each other and are associated with poor outcomes after stroke, studies that report performance in executive function in patients with stroke should also investigate symptoms of depression.
Not only depression interferes on executive dysfunction after stroke, but also medication used for treatment may interfere with performance of the patient [64]. For this reason, medication should be considered during evaluations in clinical practice and in research studies in general; however not all studies described medications used.
In addition, not only the stroke lesion itself, but also vascular cognitive impairment can contribute to executive dysfunction [65]. Vascular cognitive impairment in non-demented patients is characterized by cognitive decline secondary to cerebrovascular disease and can present with executive dysfunction [66][67][68]. The rate of vascular cognitive impairment ranges from 10.5 to 37% of patients without stroke or dementia [69][70][71]. In Brazil, for instance, vascular cognitive impairment in patients with ischemic stroke was diagnosed in 16.8% of 172 subjects [72]. It is important that studies about executive function in patients with stroke define whether vascular cognitive impairment was an exclusion criterion.
Finally, details about lesion sites were scarce or absent in most studies that reported evaluation of executive dysfunction in patients with stroke. It is desirable that more information about lesion location is provided in future studies. Two studies used magnetic resonance imaging (MRI) for lesion characterization and compared the result with the measures for executive dysfunction [33,48]. Patients with executive dysfunction presented infarcts in the frontal-sub cortical circuit [48]. In addition, the aging brain suffers changes such as white matter lesions and microbleeds, which are associated with cognitive and functional decline [67].
Stroke is not the only cerebrovascular diseases that can cause cognitive impairment. White matter lesions [73] and cerebral microbleeds [74] may play an important role. White matter lesions are prevalent in people over 60 years, can be detected in up to 90% of neurologically symptom-free elderly and are associated with cognitive impairment [75,76]. Cerebral microbleeds are focal lesions that result from a deposit of hemosiderin that presumably leaks out from damaged small brain vessels [74]. The prevalence of cerebral microbleeds was estimated as 17.8% in persons aged 60-69 years, and 38.3% in people over 80 years [74]. The presence of numerous microbleeds correlates with worse cognitive performance [77]. Moreover, lobar microbleeds are independently associated with executive dysfunction in patients with stroke or transient ischemic attacks [77]. Executive dysfunction is more prevalent in patients with cerebral microbleeds (38%) than other cognitive impairments [78].
A limitation of this study was to only include articles published in English. Despite this, very few manuscripts published in other languages were identified: in Portuguese [79] and in Norwegian [80]. Manuscripts reporting scales in languages other than English may have been published in journals not indexed in Medline, Pubmed and Google Scholar. This limits evaluation of validity and cross-cultural comparisons.
It is further noteworthy that literally all studies included in this review excluded patients with language impairments and those with severe physical disabilities. The findings on executive function performance summarized here are therefore not entirely translatable to the stroke population as a whole.
In a recent systematic review about cognitive rehabilitation after stroke, the authors discussed the importance of cognitive rehabilitation and the lack of evidence in this area [81]. Moreover, the review revealed that some of the cognitive domains (attention, spatial neglect and motor apraxia) can improve with rehabilitation, but this improvement is not long lasting. The authors also described the major limitations of evidence and the need for more investigations in this area with appropriate methodological standards.

Conclusion
Appropriate assessment of the patient with stroke is essential to provide better treatments. Awareness and quantification of impairments can enhance the ability to plan rehabilitation and optimize long-term care provision, in order to tailor treatments according to specific individual needs. This is particularly important for patients who might be able to return to work if their deficits in cognition are recognized. To attain this goal, adequate tools of evaluation, adapted to local cultural specificities, are required. There is a paucity of tools to reliably measure executive dysfunction after stroke, despite the fact that executive dysfunction is frequent. Specifically, there is a great need to develop appropriate tools for developing countries. In addition, limited information is available about the relation between executive dysfunction measured with valid scales and stroke lesions, white matter disease, microbleeds, as well as with other conditions that can be associated with stroke such as vascular cognitive impairment and depression. There are deep gaps about executive dysfunction in stroke, to be filled in research and in clinical practice.

Funding
Annette Sterr, PhD is a Visitor Professor Neurostimulation Laboratory, Neurology Department at São Paulo University and received a "Science Without Boarders" scholarship (GRANT A 068/2013) from Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) and Coordenação de Aperfeiçoamento de Pessoal de Nivel Superior (CAPES). This investigation was supported by the Fogarty International Center of the National Institutes of Health under the NCD-LIFESPAN program (GRANT D71TW009132-01). Juliana Conti and Adriana Bastos Conforto received funding from the National Institutes of Neurological Disorders and Stroke (R01NS076348-01).