Comparison of the Diagnostic Accuracy of Five Cognitive Screening Tests for Diagnosing Mild Cognitive Impairment in Patients Consulting for Memory Loss

Objectives: We aimed to evaluate and compare the diagnostic capacity of five cognitive screening tests for the diagnosis of mild cognitive impairment (MCI) in patients consulting by memory loss. Methods: A cross-sectional study involving 140 participants with a mean age of 74.42 ± 7.60 years, 87 (62.14%) women. Patients were classified as MCI or cognitively unimpaired according to a comprehensive neuropsychological battery. The diagnostic properties of the following screening tests were compared: Mini-Mental State Examination (MMSE), Addenbrooke’s Cognitive Examination III (ACE-III) and Mini-Addenbrooke (M-ACE), Memory Impairment Screen (MIS), Montreal Cognitive Assessment (MoCA), and Rowland Universal Dementia Assessment Scale (RUDAS). Results: The area under the curve (AUC) was 0.861 for the ACE-III, 0.867 for M-ACE, 0.791 for MoCA, 0.795 for MMSE, 0.731 for RUDAS, and 0.672 for MIS. For the memory components, the AUC was 0.869 for ACE-III, 0.717 for MMSE, 0.755 for MoCA, and 0.720 for RUDAS. Cronbach’s alpha was 0.827 for ACE-III, 0.505 for MMSE, 0.896 for MoCA, and 0.721 for RUDAS. Correlations with Free and Cued Selective Reminding Test were moderate with M-ACE, ACE-III, and MoCA, and moderate for the other tests. The M-ACE showed the best balance between diagnostic capacity and time of administration. Conclusions: ACE-III and its brief version M-ACE showed better diagnostic properties for the diagnosis of MCI than the other screening tests. MoCA and MMSE showed adequate properties, while the diagnostic capacity of MIS and RUDAS was limited.


Introduction
Neurodegenerative diseases, and especially Alzheimer's disease (AD), are the most frequent cause of cognitive impairment [1].The early detection of cognitive impairment is increasingly relevant, as it facilitates an accurate diagnosis, treatment interventions, and patient and caregiver support.Furthermore, the new therapies recently approved or under development for AD are targeting the earliest stages of the disease [2].
Mild cognitive impairment (MCI) is a stage characterized by a decline in episodic memory or other cognitive functions, in which patients are at higher risk of developing dementia due to Alzheimer's disease or other causes [3].MCI patients show abnormal cognitive functions compared with the expected performance according to age and education levels, but without significant impairment in daily living activities.Neuropsychological assessment is an essential tool in the diagnostic framework of patients with MCI.Cognitive screening tests have been recommended as a first-line assessment of patients consulting by cognitive issues [4].Cognitive screening tests are carried out at different assistance levels, guiding the decisions to refer patients to more specialized centers (e.g., from primary care to neurology, or from general neurology to memory units) and for indicating neuroimaging or fluid biomarkers.The Mini-Mental State Examination (MMSE) is the most used cognitive screening test to assess patients [5].However, several studies have concluded a limited capacity to detect early stages of AD and especially to detect patients with MCI [6].In this regard, several tests have been developed and validated for detecting MCI [7,8].
Among the different cognitive screening tests, some of them have gained some popularity due to different reasons.The Montreal Cognitive Assessment (MoCA) includes some tasks to assess attention and executive function and could be more helpful in detecting the early stages [9].The Addenbrooke's Cognitive Examination (ACE-III) was developed as an expansion of the MMSE, but it added more visuospatial, executive, language, and memory tasks [10].Although the administration time is longer than the other tests, an abbreviated version (Mini-Addenbrooke's Cognitive Examination, M-ACE) seems to have greater sensitivity for screening [11].The Rowland Universal Dementia Assessment (RUDAS) has some advantages due to its favorable cross-cultural properties and low susceptibility to the educational level [12,13].Finally, the Memory Impairment Screen (MIS) is focused on the assessment of episodic memory and, given its brevity, has been suggested for screening uses in primary care [14][15][16].Nevertheless, even though these tests have been previously validated in different languages and clinical contexts, especially in comparison to the MMSE, as far as we know, few investigations have directly assessed the diagnostic capacity of multiple cognitive screening tests within the same study [17].
In this study, we aimed to evaluate and compare the diagnostic capacity of five cognitive screening tests (ACE-III and M-ACE, MIS, MMSE, MoCA, and RUDAS) for the diagnosis of MCI in patients consulting by memory loss.

Study Design and Participants
We conducted a cross-sectional investigation including patients consulting by memory loss to the Department of Neurology of a tertiary-care hospital in Madrid, Spain.The research protocol was approved by the Ethics Committee of our center.Written informed consent was obtained from all participants.
The patients met the following inclusion criteria: (a) cognitive symptoms mainly focused on memory complaints referred by the patients and/or confirmed by an informant; (b) absence of significant functional impairment according to the Functional Activities Questionnaire (FAQ) [18]; (c) absence of dementia; (d) Spanish-native speaker.The exclusion criteria were as follows: (a) neurological, systemic, or psychiatric disorder potentially impacting cognitive performance or associated with cognitive impairment (e.g., history of stroke, epilepsy, major depression, etc.); (b) visual or auditory impairment that may impair test performance.
The participants were consecutively enrolled among those patients referred for neuropsychological assessment in two periods: September 2021-March 2022 and September 2023-March 2024.Patients were referred to neuropsychological assessment after a consultation by a neurologist, who assessed clinical data reported by the patient and family.No cognitive instruments were administered in this consultation.
The total sample comprised 140 participants, 87 (62.14%) women, with a mean age of 74.42 ± 7.60 years and 9.70 ± 5.24 years of formal education.MCI was defined according to one of the following criteria: (a) at least one of the scores of the FCSRT measuring learning or delayed recall (total free recall, total recall, delayed recall) below seven age-and education-adjusted scaled scores (corresponding to >1 SD below the norms); (b) at least two impaired scores (>1 SD below the norms) within the same cognitive domain.These criteria are based on the Jack/Bondi criteria [33] but emphasize the importance of FCSRT in the early diagnosis due to the higher sensitivity of this test to capture the early stages of prodromal AD [34,35].Accordingly, the patients were classified as cognitively unimpaired (CU) (n = 84) or MCI (n = 56).
The main clinical and demographic characteristics of each group are presented in Table 1.

Cognitive Screening Assessments
Five cognitive screening assessments were administered during a single session, with the sequence of administration changed every thirty cases.Tasks that overlapped across different cognitive tests (such as drawing a clock or specific orientation questions) were presented only once.The following tests were conducted in their Spanish versions: ACE-III [36,37], MIS [39], MMSE [40], MoCA [41], and RUDAS [38].
Normative data were available for all these tests in our specific setting [39,40,[42][43][44].Raw data for all the tests were adjusted for age and education, following the procedures specified for each test, and scaled scores were calculated for ACE-III and MoCA, while corrected scores were computed for MMSE and MIS.For the RUDAS, z-scores were calculated.
These tests were selected based on the availability of a Spanish version, the use in our setting, and the previous literature suggesting their use in MCI.A summary of the main cognitive functions assessed in each test is depicted in Figure 1.
The time of administration for each test was as follows: 15 min for ACE-III, 5 min for M-ACE, 10 min for MMSE, 10 min for MoCA, 10 min for RUDAS, and 5 min for MIS.The five cognitive tests showed favorable psychometric properties according to previous studies, with high internal consistency.According to Cronbach's alpha, the internal reliability was 0.92 for ACE-III [10], 0.610 for MMSE [45], 0.83 for M-ACE [11], 0.88 for MoCA [9], and 0.876 for RUDAS [46].The inter-rater and test-retest reliability was high for all tests [9][10][11]45,46].These tests were selected based on the availability of a Spanish version, the use in our setting, and the previous literature suggesting their use in MCI.A summary of the main cognitive functions assessed in each test is depicted in Figure 1.The time of administration for each test was as follows: 15 min for ACE-III, 5 min for M-ACE, 10 min for MMSE, 10 min for MoCA, 10 min for RUDAS, and 5 min for MIS.The five cognitive tests showed favorable psychometric properties according to previous studies, with high internal consistency.According to Cronbach's alpha, the internal reliability was 0.92 for ACE-III [10], 0.610 for MMSE [45], 0.83 for M-ACE [11], 0.88 for MoCA [9], and 0.876 for RUDAS [46].The inter-rater and test-retest reliability was high for all tests [9][10][11]45,46].

Sample Size Estimation
To determine the sample size necessary for comparing two ROC curves using DeLong's method [47], we calculated first the variance of the difference between the AUC from the 95% intervals of the AUCs.According to our previous study comparing the different cognitive tests in AD dementia [48], it was less than 0.005 in all cases.Then, we calculated that a sample size of 140 individuals would detect a minimum significant difference of 0.0167 between the AUC, assuming a significance level of 0.05 and a statistical power of 0.80.

Statistical Analysis
Statistical analysis was conducted using IBM ® SPSS Statistics 26.0, Jamovi 2.3.21, and R3.6.3 (pROC package).The results are presented as frequencies (percentages) or mean ± standard deviation.Comparisons were made using Student's t-test (for independent samples) or chi-square tests as appropriate.Cronbach's alpha was used to estimate the internal consistency of each test.This parameter was interpreted as quite reliable (0.41-0.60), reliable (0.61-0.80), and very reliable (0.81-1.00).Receiver operating characteristic (ROC) curves and the areas under the curves (AUC) were employed to assess the discriminatory ability of each test between MCI and the control group (i.e., patients consulting due to cognitive symptoms but with the standardized neuropsychological assessment battery

Sample Size Estimation
To determine the sample size necessary for comparing two ROC curves using DeLong's method [47], we calculated first the variance of the difference between the AUC from the 95% intervals of the AUCs.According to our previous study comparing the different cognitive tests in AD dementia [48], it was less than 0.005 in all cases.Then, we calculated that a sample size of 140 individuals would detect a minimum significant difference of 0.0167 between the AUC, assuming a significance level of 0.05 and a statistical power of 0.80.

Statistical Analysis
Statistical analysis was conducted using IBM ® SPSS Statistics 26.0, Jamovi 2.3.21, and R3.6.3 (pROC package).The results are presented as frequencies (percentages) or mean ± standard deviation.Comparisons were made using Student's t-test (for independent samples) or chi-square tests as appropriate.Cronbach's alpha was used to estimate the internal consistency of each test.This parameter was interpreted as quite reliable (0.41-0.60), reliable (0.61-0.80), and very reliable (0.81-1.00).Receiver operating characteristic (ROC) curves and the areas under the curves (AUC) were employed to assess the discriminatory ability of each test between MCI and the control group (i.e., patients consulting due to cognitive symptoms but with the standardized neuropsychological assessment battery within normal limits).The method by DeLong et al. was used to compare the ROC curves [47].ROC curves were generated for both raw scores and adjusted scores using normative data.Youden's index J was applied to determine the optimal cut-off points.Cohen's d was calculated to determine effect size, while Pearson's coefficient was used to measure correlations between tests and between memory scores and the FCSRT.Effect sizes were classified as small (d = 0.2-0.49),moderate (d = 0.5-0.79),and large (d ≥ 0.8).Correlation coefficients <0.40 were interpreted as weak correlations, 0.40-0.69as moderate, and >0.69 as strong.A p-value < 0.05 was considered statistically significant.
Additionally, we estimated the diagnostic capacity of each cognitive test adjusted by the time of administration, as a measure of the efficiency of each tool.Accordingly, we divided the AUC of each total score by the mean time of administration of each test according to the literature [48].It quantifies how much the diagnostic capacity increases per minute of test administration.

Diagnostic Validity and Reliability
Patients with MCI showed lower scores in all the cognitive screening tests compared to the control group.Effect sizes were large in M-ACE followed by ACE-III, MMSE, MoCA, and RUDAS, and moderate in MIS (Table 1).
Additionally, we estimated the diagnostic capacity of each cognitive test adjusted the time of administration, as a measure of the efficiency of each tool.Accordingly, divided the AUC of each total score by the mean time of administration of each test cording to the literature [48].It quantifies how much the diagnostic capacity increases minute of test administration.

Diagnostic Validity and Reliability
Patients with MCI showed lower scores in all the cognitive screening tests compa to the control group.Effect sizes were large in M-ACE followed by ACE-III, MM MoCA, and RUDAS, and moderate in MIS (Table 1).

Correlations between Tests
Correlations between the total scores of the tests were all strong, except between the MIS and the other tests, which were moderate.All the correlations between the memory scores were moderate, except between MIS and MMSE (memory score), which was weak.All the correlations were statistically significant (Table 3).The correlation between ACE-III and FCSRT (free total recall, total recall, delayed free recall, and delayed total recall) was 0.796, 0.754, 0.743, and 0.703, respectively.The correlation between M-ACE and FCSRT was 0.805, 0.809, 0.773, and 0.743.The correlation between MMSE and FCSRT was 0.665, 0.683, 0.604, and 0.621, respectively.The correlation between MoCA and FCSRT was 0.77, 0.742, 0.716, and 0.685.The correlation between RUDAS and FCSRT was 0.718, 0.712, 0.705, and 0.673.Finally, the correlation between MIS and FCSRT was 0.629, 0.65, 0.555, and 0.593.All correlations were statistically significant (<0.001).

Discussion
Our study showed that the five cognitive tests distinguished between patients with MCI and the cognitively unimpaired, according to a diagnosis of MCI based on a comprehensive and co-normed neuropsychological battery.This finding is consistent with previous studies validating each individual test [7,8].However, the main aim of the study and the most interesting finding was the comparison between the different screening tests.In this regard, the ACE-III and its brief version (M-ACE) showed greater diagnostic accuracy than the other cognitive tests, including MoCA, MMSE, MIS, and RUDAS.As M-ACE may be administered in only 5 min [49], our findings suggest that the M-ACE should be the best option considering time of assessment and diagnostic accuracy to screen patients with MCI consulting due to memory loss.In this regard, we calculated an index to measure the increase in AUC per minute of administration time for each cognitive test, confirming that the M-ACE is the most appropriate test for screening, considering the balance between diagnostic capacity and time.This observation may be justified by the fact that the M-ACE only includes the most sensitive items in the early stages of MCI (i.e., temporal orientation, semantic fluency, verbal memory task, clock drawing), whereas the other tests include other items that, although informative about other cognitive domains, are impaired in later stages or in atypical cases (e.g., constructive praxis, naming) [50] (Figure 1).Additionally, the ACE-III provides a more complete assessment of cognitive functions than the other cognitive screening assessments.
As expected, the AUC of each test was reduced compared with a similar study in which we used the same tests for the discrimination between healthy controls and patients with mild dementia [48].However, the AUC values are still noteworthy and warrant the use of screening cognitive tests for the assessment of patients with memory complaints with no functional impairment.Another important finding is the assessment of reliability.In this regard, reliability was especially high for the ACE-III, good for MoCA and RUDAS, and low for MMSE.
The best cut-off points were estimated according to the Youden's index, which tries to maximize sensitivity and specificity.The optimal cut-off points estimated for the other tests are the same or similar to those proposed in other studies validating each test in our setting [16].Regarding the MMSE, the best cut-off was 26 or 29, which is relatively high and seems to confirm previous studies suggesting that MMSE is not sensitive enough to the early stages [51].However, the choice of cut-off point can also be adapted to the specific setting, according to the clinical needs of higher sensitivity, higher specificity, or predictive values.
Another interesting result is the correlation between the cognitive screening tests with the FCSRT.This test was selected because it is specifically recommended for the diagnosis of early stages of AD [52].Correlations were higher for M-ACE and ACE-III followed by MoCA and the other tests.These correlations may be interpreted as some sort of concurrent validity of cognitive screening tests, due to the high diagnostic capacity of FCSRT [53].
Our findings have important implications for clinical practice.Memory complaints are a frequent cause of medical consultation in the elderly population and are frequent symptoms in the general population.The early diagnosis and treatment of early stages of AD and other neurodegenerative disorders are increasingly necessary, as MCI patients are at risk of clinical progression.Our findings suggest the superiority of ACE-III and M-ACE over the other cognitive tests in detecting patients with MCI.
Importantly, when considering the time of administration, M-ACE is the most efficient test, whereas other tests like MoCA, MMSE, and RUDAS become more efficient than the ACE-III.This introduces the time of administration as a relevant parameter in selecting the most appropriate cognitive tool, which could be adapted according to different clinical scenarios and the available time to assess each patient.The search for brief tests, considering the time of administration and the associated economic costs, may be especially relevant when implementing modifications in clinical protocols [54].Additionally, the new landscape of pathology-modifying therapies necessitates optimizing available resources to limit the costs associated with screening, diagnosis, and follow-up [55].In this regard, our study evaluated the diagnostic capacity of several tests.However, the simultaneous or sequential administration of several brief cognitive tests [56], the incorporation of technological solutions [57][58][59], or the combination between a cognitive tool with a biological biomarker may be of great interest to increase diagnostic accuracy [60] and should be investigated in future investigations about cognitive screening tools.
Our study has some limitations.First, the MoCA optional items for memory were not administered and the MoCA-Memory Index Score was not calculated [61].Similarly, the MoCA has shown low performances in other Spanish cohorts, which could suggest limitations in the adaptation of the test to the Spanish population that could not be present in other versions [40,50].Second, the M-ACE was calculated after the administration of the ACE-III, but it was not administered specifically.This could imply a higher diagnostic performance because the time from the registration and recall of the memory task is longer in the ACE-III than in the M-ACE.Third, our study was performed in a tertiary center, although with direct access from primary care.Thus, the PPV and NPV are calculated for this sample, in which the prevalence of MCI was relatively high (40%).Fourth, our study was performed on Spanish-speaking patients to avoid cultural issues in the definition of MCI according to the neuropsychological battery.Future studies should evaluate the comparative diagnostic performance of cognitive screening instruments in multicultural settings [62].

Conclusions
In conclusion, our study found that the ACE-III and its brief version M-ACE showed better diagnostic properties for the diagnosis of MCI than the other screening tests.MoCA and MMSE showed adequate properties, while the diagnostic capacity of MIS and RUDAS was limited.The M-ACE showed the best balance between diagnostic capacity and time of administration.Future studies combining screening tests and other accessible biomarkers (e.g., plasma biomarkers) would be necessary to define the best strategy to identify patients at risk of developing dementia.

Figure 1 .
Figure 1.Summary of the main cognitive functions and tasks assessed by each cognitive screening test.

Figure 1 .
Figure 1.Summary of the main cognitive functions and tasks assessed by each cognitive screening test.

Figure 2 .
Figure 2. ROC curve analysis for the discrimination between MCI and CU.(A) Using total scores; (B) Using memory scores; (C) Using total adjusted scores.ACE-III is shown in blue (M-A in dark blue), MMSE in grey, MoCA in yellow, RUDAS in green, and MIS in red.

Figure 2 .
Figure 2. ROC curve analysis for the discrimination between MCI and CU.(A) Using total raw scores; (B) Using memory scores; (C) Using total adjusted scores.ACE-III is shown in blue (M-ACE in dark blue), MMSE in grey, MoCA in yellow, RUDAS in green, and MIS in red.

Table 1 .
Main characteristics of the sample.

Table 2 .
Optimal cut-off points, sensitivity, specificity, predictive values, and Youden index of each test.
* Adjusted scores are shown in scaled scores for ACE-III and MoCA and z-scores for RUDAS.