Skip to main content
SpringerLink
Log in

Prospective comparison of two cognitive screening tests: diagnostic accuracy and correlation with community integration and quality of life

  • Clinical Study – Patient Study
  • Published:
Journal of Neuro-Oncology Aims and scope Submit manuscript

Abstract

Cognitive screening tests are frequently used in brain tumor clinics. The Mini Mental State Examination (MMSE) is the most commonly used, and the Montreal Cognitive Assessment (MoCA) is an alternative. This study compares the diagnostic accuracy of both screening tests. Fifty-eight patients with brain tumors were prospectively accrued and administered the MMSE and MoCA, 67% of who completed a comprehensive neuropsychological evaluation as a gold standard comparison. Quality of life and community integration were measured with the Functional Assessment of Cancer Therapy-Brain (FACT-Br) and Community Integration Questionnaire (CIQ), respectively. At the pre-defined cut-off scores, the MoCA had superior sensitivity (61.9% vs. 19.0%, P < 0.005) and the MMSE had superior specificity (94.4% vs. 55.6%, P < 0.017). The areas under the ROC curve for the MMSE (0.615, standard error = 0.091) and MoCA (0.606, standard error = 0.092) were poor, indicating that at no single cut-off score is either test both sensitive and specific. Neither the MMSE (ρ = 0.12; P < 0.444) nor MoCA (ρ = 0.24; P < 0.108) were significantly correlated with the FACT-Br. The MoCA was modestly correlated with the CIQ (ρ = 0.35; P < 0.017), but the MMSE was not (ρ = 0.14; P < 0.359). The MMSE has extremely poor sensitivity. Using this test in clinical practice, research, and clinical trials will result in failing to detect cognitive impairment in a substantial percentage of patients. The MoCA has superior sensitivity, and is better correlated with self reported measures of community integration, and therefore should be preferentially chosen in practice and clinical trials.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Murray KJ, Scott C, Zachariah B, Michalski JM, Demas W, Vora NL, Whitton A, Movsas B (2000) Importance of the mini-mental status examination in the treatment of patients with brain metastases: a report from the radiation therapy oncology group protocol 91-04. Int J Radiat Oncol Biol Phys 48:59–64

    Article  PubMed  CAS  Google Scholar 

  2. Meyers CA, Hess KR, Yung WKA, Levin VA (2000) Cognitive function as a predictor of survival in patients with recurrent malignant glioma. J Clin Oncol 18:646–650

    PubMed  CAS  Google Scholar 

  3. Klein M, Heimans JJ, Aaronson NK, van der Ploeg HM, Grit J, Muller M, Postma TJ, Mooij JJ, Boerman RH, Beute GN, Ossenkoppele GJ, van Imhoff GW, Dekker AW, Jolles J, Slotman BJ, Struikmans H, Taphoorn MJB (2002) Effect of radiotherapy and other treatment-related factors on mid-term to long-term cognitive sequelae in low-grade gliomas: a comparative study. Lancet 360:1361–1368

    Article  PubMed  CAS  Google Scholar 

  4. Scheibel R, Meyers C, Levin V (1990) Cognitive dysfunction following surgery for intracerebral glioma; influence of histopathology, lesion location, and treatment. J Neurooncol 9:47–55

    Article  Google Scholar 

  5. Taylor BV, Buckner JC, Cascino TL, O’Fallon JR, Schaefer PL, Dinapoli RP, Schomberg P (1998) Effects of radiation and chemotherapy on cognitive function in patients with high-grade glioma. J Clin Oncol 16:2195–2201

    PubMed  CAS  Google Scholar 

  6. Taphoorn MJB, Klein M (2004) Cognitive deficits in adult patients with brain tumours. Lancet Neurol 3:159–168

    Article  PubMed  Google Scholar 

  7. Weitzner MA, Meyers CA (1997) Cognitive functioning and quality of life in malignant glioma patients: a review of the literature. Psychooncology 6:169–177

    Article  PubMed  CAS  Google Scholar 

  8. Meyers CA, Smith JA, Bezjak A, Mehta MP, Liebmann J, Illidge T, Kunkler I, Caudrelier J, Eisenberg PD, Meerwaldt J, Siemers R, Carrie C, Gaspar LE, Curran W, Phan S, Miller RA, Renschler MF (2004) Neurocognitive function and progression in patients with brain metastases treated with whole-brain radiation and motexafin gadolinium: results of a randomized phase III trial. J Clin Oncol 22:157–165

    Article  PubMed  CAS  Google Scholar 

  9. Sherman AM, Jaeckle K, Meyers CA (2002) Pretreatment cognitive performance predicts survival in patients with leptomeningeal disease. Cancer 95:1311–1316

    Article  PubMed  Google Scholar 

  10. Yaffe K, Blackwell T, Barnes DE, Ancoli-Israel S, Stone KL, For the Study of Osteoporotic Fractures Group (2007) Preclinical cognitive decline and subsequent sleep disturbance in older women. Neurology 69:237–242

    Article  PubMed  CAS  Google Scholar 

  11. Olson R, Tyldesley S, Carolan H, Parkinson M, Chhanabhai T, McKenzie M (2010) Prospective comparison of the prognostic utility of the Mini Mental State Examination and the Montreal Cognitive Assessment in patients with brain metastases. Support Care Cancer. doi:10.1007/s00520-010-1028-1

  12. Olson RA, Chhanabhai T, McKenzie M (2008) Feasibility study of the Montreal Cognitive Assessment (MoCA) in patients with brain metastases. Support Care Cancer 16:1273–1278. doi:10.1007/s00520-008-0431-3

    Article  PubMed  Google Scholar 

  13. Klein M, Taphoorn MJB, Heimans JJ, van der Ploeg HM, Vandertop WP, Smit EF, Leenstra S, Tulleken CAF, Boogerd W, Belderbos JSA, Cleijne W, Aaronson NK (2001) Neurobehavioral status and health-related quality of life in newly diagnosed high-grade glioma patients. J Clin Oncol 19:4037–4047

    PubMed  CAS  Google Scholar 

  14. Olson R, Parkinson M, McKenzie M (2010) Selection bias introduced by neuropsychological assessments. Can J Neurol Sci 37:264–268

    PubMed  Google Scholar 

  15. Hegi ME, Diserens A, Gorlia T, Hamou M, de Tribolet N, Weller M, Kros JM, Hainfellner JA, Mason W, Mariani L, Bromberg JEC, Hau P, Mirimanoff RO, Cairncross JG, Janzer RC, Stupp R (2005) MGMT gene silencing and benefit from temozolomide in glioblastoma. N Engl J Med 352:997–1003. doi:10.1056/NEJMoa043331

    Article  PubMed  CAS  Google Scholar 

  16. Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12:189–198

    Article  PubMed  CAS  Google Scholar 

  17. Brown PD, Buckner JC, O’Fallon JR, Iturria NL, Brown CA, O’Neill BP, Scheithauer BW, Dinapoli RP, Arusell RM, Curran WJ, Abrams R, Shaw EG (2003) Effects of radiotherapy on cognitive function in patients with low-grade glioma measured by the Folstein mini-mental state examination. J Clin Oncol 21:2519–2524

    Article  PubMed  Google Scholar 

  18. Meyers CA, Wefel JS (2003) The use of the mini-mental state examination to assess cognitive functioning in cancer trials: no ifs, ands, buts, or sensitivity. J Clin Oncol 21:3557–3558

    Article  PubMed  Google Scholar 

  19. Chinot OL, de La Motte Rouge T, Moore N, Zeaiter A, Das A, Phillips H, Modrusan Z, Cloughesy T (2011) AVAglio: phase 3 trial of bevacizumab plus temozolomide and radiotherapy in newly diagnosed glioblastoma multiforme. Adv Ther. doi:10.1007/s12325-011-0007-3

  20. Pereira J, Hanson J, Bruera E (1997) The frequency and clinical course of cognitive impairment in patients with terminal cancer. Cancer 79:835–842

    Article  PubMed  CAS  Google Scholar 

  21. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, Cummings JL, Chertkow H (2005) The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53:695–699

    Article  PubMed  Google Scholar 

  22. Smith T, Gildeh N, Holmes C (2007) The Montreal cognitive assessment: validity and utility in a memory clinic setting. Can J Psychiatry 52:329–332

    PubMed  Google Scholar 

  23. Gauthier S, Reisberg B, Zaudig M, Petersen RC, Ritchie K, Broich K, Belleville S, Brodaty H, Bennett D, Chertkow H, Cummings JL, de Leon M, Feldman H, Ganguli M, Hampel H, Scheltens P, Tierney MC, Whitehouse P, Winblad B (2006) Mild cognitive impairment. Lancet 367:1262–1270

    Article  PubMed  Google Scholar 

  24. Meyers CA, Brown PD (2006) Role and relevance of neurocognitive assessment in clinical trials of patients with CNS tumors. J Clin Oncol 24:1305–1309

    Article  PubMed  Google Scholar 

  25. Stern RA, White T (2003) Neuropsychological assessment battery: administration, scoring, and interpretation manual. Psychological Assessment Resources, Lutz

    Google Scholar 

  26. Reynolds CR, Kamphaus RW (2003) Reynolds Intellectual Assessment Scales (RIAS) and the Reynolds Intellectual Screening Test (RIST) professional manual. Psychological Assessment Resources, Lutz

    Google Scholar 

  27. Sander AM, Fuchs KL, High WM Jr, Hall KM, Kreutzer JS, Rosenthal M (1999) The Community Integration Questionnaire revisited: an assessment of factor structure and validity. Arch Phys Med Rehabil 80:1303–1308

    Article  PubMed  CAS  Google Scholar 

  28. Willer B, Ottenbacher KJ, Coad ML (1994) The community integration questionnaire. A comparative examination. Am J Phys Med Rehabil 73:103–111

    Article  PubMed  CAS  Google Scholar 

  29. Weitzner M, Meyers C, Gelke C, Byrne K, Levin V, Cella D (1995) The functional assessment of cancer therapy (FACT) scale. Development of a brain subscale and revalidation of the general version (FACT-G) in patients with primary brain tumors. Cancer 75:1151–1161

    Article  PubMed  CAS  Google Scholar 

  30. Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, Silberman M, Yellen SB, Winicour P, Brannon J (1993) The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol 11:570–579

    PubMed  CAS  Google Scholar 

  31. Beam CA (1992) Strategies for improving power in diagnostic radiology research. Am J Roentgenol 159:631–637

    CAS  Google Scholar 

  32. Dwyer AJ (1991) Matchmaking and McNemar in the comparison of diagnostic modalities. Radiology 178:328–330

    PubMed  CAS  Google Scholar 

  33. Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine [published erratum appears in Clin Chem 1993 Aug; 39(8):1589]. Clin Chem 39:561–577

    PubMed  CAS  Google Scholar 

  34. Hanley J, NcNeil B (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839–843

    PubMed  CAS  Google Scholar 

  35. Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52:591–611. doi:10.1093/biomet/52.3-4.591

    Google Scholar 

  36. Herman MA, Tremont-Lukats I, Meyers CA, Trask DD, Froseth C, Renschler MF, Mehta MP (2003) Neurocognitive and functional assessment of patients with brain metastases: a pilot study. Am J Clin Oncol 26:273–279

    PubMed  Google Scholar 

  37. Iverson GL, Brooks BL (2011) Improving accuracy for identifying cognitive impairment. In: Schoenberg MR, Scott JG (eds) The black book of neuropsychology: a syndrome-based approach. Springer, New York, pp 923–950

  38. Brooks BL, Iverson GL, White T (2007) Substantial risk of “Accidental MCI” in healthy older adults: base rates of low memory scores in neuropsychological assessment. J Int Neuropsychol Soc 13:490–500. doi:10.1017/S1355617707070531

    Article  PubMed  Google Scholar 

  39. Brooks BL, Iverson GL, Holdnack JA, Feldman HH (2008) Potential for misclassification of mild cognitive impairment: a study of memory scores on the Wechsler Memory Scale-III in healthy older adults. J Int Neuropsychol Soc 14:463–478. doi:10.1017/S1355617708080521

    Article  PubMed  Google Scholar 

  40. Brooks BL, Iverson GL, Feldman HH, Holdnack JA (2009) Minimizing misdiagnosis: psychometric criteria for possible or probable memory impairment. Dement Geriatr Cogn Disord 27:439–450. doi:10.1159/000215390

    Article  PubMed  Google Scholar 

  41. Andrews DW, Scott CB, Sperduto PW, Flanders AE, Gaspar LE, Schell MC, Werner-Wasik M, Demas W, Ryu J, Bahary J, Souhami L, Rotman M, Mehta MP, Curran J, Walter J (2004) Whole brain radiation therapy with or without stereotactic radiosurgery boost for patients with one to three brain metastases: phase III results of the RTOG 9508 randomised trial. Lancet 363:1665. doi:10.1016/S0140-6736(04)16250-8

    Article  PubMed  Google Scholar 

  42. Shibamoto Y, Baba F, Oda K, Hayashi S, Kokubo M, Ishihara S, Itoh Y, Ogino H, Koizumi M (2008) Incidence of brain atrophy and decline in mini-mental state examination score after whole-brain radiotherapy in patients with brain metastases: a prospective study. Int J Radiat Oncol Biol Phys 72:1168–1173. doi:10.1016/j.ijrobp.2008.02.054

    Article  PubMed  Google Scholar 

  43. Regine WF, Scott C, Murray K, Curran W (2001) Neurocognitive outcome in brain metastases patients treated with accelerated-fractionation vs. accelerated-hyperfractionated radiotherapy: an analysis from Radiation Therapy Oncology Group Study 91-04. Int J Radiat Oncol Biol Phys 51:711–717

    Article  PubMed  CAS  Google Scholar 

  44. Stupp R, Mason WP, van den Bent MJ, Weller M, Fisher B, Taphoorn MJB, Belanger K, Brandes AA, Marosi C, Bogdahn U, Curschmann J, Janzer RC, Ludwin SK, Gorlia T, Allgeier A, Lacombe D, Cairncross JG, Eisenhauer E, Mirimanoff RO, The European Organisation for Research, Treatment of Cancer Brain Tumor, Radiotherapy Groups, the National Cancer Institute of Canada Clinical Trials Group (2005) Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med 352:987–996. doi:10.1056/NEJMoa043330

    Article  PubMed  CAS  Google Scholar 

  45. Regine WF, Schmitt FA, Scott CB, Dearth C, Patchell RA, Nichols RC, Gore EM, Franklin RL, Suh JH, Mehta MP (2004) Feasibility of neurocognitive outcome evaluations in patients with brain metastases in a multi-institutional cooperative group setting: results of Radiation Therapy Oncology Group trial BR-0018. Int J Radiat Oncol Biol Phys 58:1346–1352

    Article  PubMed  Google Scholar 

  46. Shapiro AM, Benedict RH, Schretlen D, Brandt J (1999) Construct and concurrent validity of the Hopkins Verbal Learning Test—revised. Clin Neuropsychol 13:348–358

    PubMed  CAS  Google Scholar 

  47. Benton AL (1969) Development of a multilingual aphasia battery. Progress and problems. J Neurol Sci 9:39–48

    Article  PubMed  CAS  Google Scholar 

  48. Lezak MD (1995) Neuropsychological assessment. Academic Press, New York

    Google Scholar 

  49. Wechsler D (1981) Wechlser Adult Intelligence Scale (revised). The Psychological Corp, New York

    Google Scholar 

  50. Vardy J, Rourke S, Tannock IF (2007) Evaluation of cognitive function associated with chemotherapy: a review of published studies and recommendations for future research. J Clin Oncol 25:2455–2463. doi:10.1200/JCO.2006.08.1604

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Grant funding was provided by the Fraser Valley Interdisciplinary Research Grant and the Hershey and Yvette Porte Neuro-Oncology Fund. The authors thank Dr. Ann-Louise Ellwood for her input into the study design, Drs. Marie-Claude Roberge and Kyle Ferguson for help administering the neuropsychological evaluations, and Colleen McGahan for her statistical advice. Grant Iverson, Ph.D. has received research funding from Psychological Assessment Resources Inc., the company that publishes the Neuropsychological Assessment Battery (Stern and White [25]) that was used in this study. Brian Brooks Ph.D. receives funding from Psychological Assessment Resources Inc.

Conflict of Interest

No other potential conflicts of interest exist.

Author information

Authors and Affiliations

  1. BC Cancer Agency, Centre for the North, Prince George, BC, Canada

    Robert A. Olson

  2. University of British Columbia, Vancouver, BC, Canada

    Grant L. Iverson, Hannah Carolan & Michael McKenzie

  3. BC Cancer Agency, Vancouver Cancer Centre, 600 West 10th Avenue, Vancouver, BC, V5Z 4E6, Canada

    Robert A. Olson, Hannah Carolan, Maureen Parkinson & Michael McKenzie

  4. British Columbia Mental Health and Addiction Services, Vancouver, BC, Canada

    Grant L. Iverson

  5. BC Cancer Agency, Fraser Valley Cancer Centre, Surrey, BC, Canada

    Maureen Parkinson

  6. Alberta Children’s Hospital, Calgary, AB, Canada

    Brian L. Brooks

  7. University of Calgary, Calgary, AB, Canada

    Brian L. Brooks

Authors
  1. Robert A. Olson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grant L. Iverson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hannah Carolan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maureen Parkinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brian L. Brooks
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael McKenzie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert A. Olson.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Online Resource 1

MoCA test. Copywright Z. Nasreddine MD. Reproduced with permission. Available at www.mocatest.org (PDF 297 kb)

Online Resource 2

MoCA screening test scores, by category of assessment, with descriptive statistics (PDF 19 kb)

Online Resource 3

Results of self reported measures of community integration and quality of life, with descriptive statistics (PDF 23 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Olson, R.A., Iverson, G.L., Carolan, H. et al. Prospective comparison of two cognitive screening tests: diagnostic accuracy and correlation with community integration and quality of life. J Neurooncol 105, 337–344 (2011). https://doi.org/10.1007/s11060-011-0595-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11060-011-0595-4

Keywords

Search

Navigation