A comparison of multiple patient reported outcome measures in identifying major depressive disorder in people with multiple sclerosis

https://doi.org/10.1016/j.jpsychores.2015.08.007Get rights and content

Highlights

  • We compare the screening accuracy for major depressive disorder in people with MS.

  • 7 self-reported screeners were compared to the SCID interview.

  • No screener reached adequate diagnostic accuracy.

  • Self-reported screeners performed with similar accuracy.

  • Higher cutoff scores were suggested for most screeners.

Abstract

Background

Depression is one of the most prominent and debilitating symptoms in individuals with multiple sclerosis (MS), yet there is currently no consensus on the best instruments for depression screening in MS. More head to head comparisons of available screening instruments are needed to advise MS researchers and clinicians.

Methods

A cross-sectional comparison of the effectiveness of screening for MDD using multiple patient reported outcome (PRO) screeners against a modified SCID telephone interview was completed in 164 individuals with MS. Stratum goals were set for depression levels to ensure participation by people with borderline and higher levels of depression. Criterion standard was a modified SCID MDD module. PRO measures included the PHQ-9, BDI-FS, PROMIS depression, Neuro-QOL depression, M-PHQ-2, PHQ-2, and CESD.

Results

48 (29%) individuals met the modified SCID criteria for MDD. The sensitivity of the PRO measures ranged from 60% to 100% while specificity ranged from 46% to 86%. The ROC area for the PRO measures ranged from 0.79 to 0.83. Revised (higher) cutoff scores were suggested by the ROC analyses for most self-reported screeners.

Limitations

Enrollment was stopped early because of difficulties with recruitment. Several SCID recording could not be reviewed and diagnosis confirmed.

Conclusions

CESD-10 and PHQ9 had the best diagnostic performance using optimal cutoffs, but no one PRO measure stood out as significantly better than any other. Even when revised cutoff scores were used, none of the self-reported screeners identified people with MDD with adequate accuracy. More accurate self-reported screeners would facilitate diagnosing of MDD for both research and clinical purposes.

Introduction

Multiple sclerosis (MS) is a chronic neurologic disease characterized by inflammation, demyelination, and neurodegeneration in the central nervous system. People with MS often report physical, cognitive and psychological symptoms, of which depression is one of the most prominent and debilitating [49]. The life-time prevalence of major depressive disorder (MDD) in individuals with MS has been estimated at over 50% [38]. In addition, the 12-month prevalence of MDD is approximately twice that of the general population at 15.7% [38]. Depression is associated with a host of poor outcomes in people with MS, including poorer overall health, non-adherence to disease modifying medications [54], loss of employment or reduction in work hours [39], an increased risk of suicidal ideation and completed suicide [17], [18], [42], greater cognitive dysfunction [4], and an overall reduction in quality of life [15], [25]. Despite these poor outcomes in MS, MDD remains under-recognized and undertreated [16], [20], [25].

Numerous instruments are available to assess depressive symptoms in people with MS, and they can also be used to identify depression cases in need of treatment. In the MS literature these measures are commonly referred to as depression “screening” instruments or measures [19], [34]. However, only a few published studies have compared the agreement between depression measures and structured diagnostic interviews for MDD in people with MS. In a series of newly diagnosed individuals with MS, the original Beck Depression Inventory (BDI) [6] (cutoff 13) produced 71% sensitivity, 79% specificity when compared to the Diagnostic Interview Schedule for DSM-III disorders [53]. A more recent similar study of the BDI-II in a clinical MS population produced similar results at 85% sensitivity and 76% specificity compared to the Schedules for Clinical Assessment in Neuropsychiatry (SCAN) [56]. A two-item measure adapted from the Primary Care Evaluation of Mental Disorders (PRIME-MD) [51], [58] was compared to MDD diagnoses derived from the Structured Clinical Interview for DSM-IV Disorders (SCID) [23] and reported a 99% sensitivity and 87% specificity [35] in one study and 80% sensitivity and 93% specificity in a second [40]. The depression subscale of the Hospital Anxiety and Depression Scale (HADS) has also been validated against the SCID or the Schedules for Clinical Assessment in Neuropsychiatry. A cutoff of eight in two studies found 90% sensitivity and 87.3% specificity [28] and 85% sensitivity and 82% specificity [40] while a third study found 77% sensitivity and 81% specificity at a cutoff of 11 [56]. Recently a study of the Center for Epidemiologic Studies Depression scale (CESD) found that it provided 95% sensitivity and 73% specificity compared to the SCID [40]. These findings provide important information regarding the utility of each measure, however many instruments have only been examined once and in small clinical samples and still more depression instruments are available that have not been examined.

There is currently no consensus on best practices for what instruments to use to assess depression in people with MS. As noted by the American Academy of Neurology in their recent review of the evidence for depression screening measures in MS [34], a number of depression measures used in the MS field lack strong evidence for their utility in identifying cases of depression, particularly relative to other commonly used measures. More head to head comparisons of measures are needed to advise researchers and clinicians. Among the many common measures whose utility for case identification in MS have not been well studied are the Beck Depression Inventory-Fast Screen (BDI-FS) [5], Patient Health Questionnaire-9 (PHQ-9) [30], and the CESD [57]. The BDI-FS is made up of a subset of items from the Beck scale proposed for use in MS [7], while the CESD and the PHQ-9 are two of the most commonly used instruments in the literature. Additionally, new instruments were recently developed with modern psychometric methods, and their clinical utility as MDD screeners in MS has yet to be examined. These include the Patient Reported Outcomes Measurement Information Systems (PROMIS®) and Neurological Quality of Life (Neuro-QOL) depression item banks. Both instruments provide population norms and have an added benefit of the availability of administration through computerized adaptive testing (CAT) [24], [41]. The Neuro-QOL measure was specifically developed for use in neurological populations, including MS, and – the psychometric properties of the PROMIS Depression short form have been examined in people with MS and found to be acceptable [2], [46].

A recently published evidence-based guideline on the assessment and management of psychiatric disorders in individuals with MS emphasized the need for research comparing different self-report and diagnostic instruments for identifying psychiatric disorders, including MDD [34]. Therefore, the purpose of this study was to: (1) examine the correspondence between the standard diagnostic interview (SCID) and multiple self-report depression measures which are commonly used as tools for identifying MDD in MS; and (2) examine the published cutoffs for each measure and potentially identify optimal cutoffs for identifying people with MS for MDD.

Section snippets

Participants

Between September 2011 and March 2012 individuals with MS were recruited through invitation letters, print advertisements, and referrals from active research studies at the University of Washington (UW) in Seattle, WA. Individuals were sent invitation letters if they had participated in past research studies at the UW and indicated interest in future studies or were members of the UW disability registry. Individuals were required to be 18 years or older, self-report a definitive MS diagnosis by

Participants

A total of 372 individuals from past research studies were mailed invitation letters, 28 were referred from active research studies, and 11 responded to print advertisements. Of these, 50 declined to participate, 72 were determined ineligible at the time of enrollment (70 due to strata reaching recruitment capacity), three were deceased, one was unable to complete the interview due to severe comorbid conditions, and 119 did not respond or were lost to follow-up. A total of 166 individuals

Discussion

The purpose of this study was to examine the correspondence between the diagnosis of MDD based on the SCID and six self-report instruments used to identify people with MDD. Our results suggest that none of the self-report instruments identified people with MDD with adequate accuracy, suggesting a need for development of a more accurate screening instrument or scoring algorithm that results in more accurate diagnosis in persons with MS. Depending on their purpose, the researchers/clinicians may

Conclusions

In summary, our results as well as the Minden et al. [34] evidence based review suggest that there is much work to be done to improve accurate identification of MDD using screening measures in this population. It is possible that because of complexity of diagnosing MDD self-reported screening instruments are limited in their ability to accurately identify MDD in people with MS. It may be useful to examine whether a set of questions drawn from all the instruments would result in more accuracy,

Acknowledgments

The contents of this manuscript were developed under a grant from the Department of Education, NIDRR grant number H133B080025. However, those contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal Government. This review was also supported in part by a grant from the National Multiple Sclerosis Society, grant number MB 0008. Neither funding agency played any role in the study design or interpretation, or the

References (61)

  • D. Amtmann et al.

    Comparing CESD-10, PHQ-9, and PROMIS depression instruments in individuals with multiple sclerosis

    Rehabil. Psychol.

    (2014)
  • P.A. Arnett et al.

    Depressed mood in multiple sclerosis: relationship to capacity-demanding memory and attentional functioning

    Neuropsychology

    (1999)
  • A.T. Beck et al.

    BDI-Fast Screen for Medical Patients: Manual

    (2000)
  • A.T. Beck et al.

    An inventory for measuring depression

    Arch. Gen. Psychiatry

    (1961)
  • R.H. Benedict et al.

    Validity of the Beck depression inventory-fast screen in multiple sclerosis

    Mult. Scler.

    (2003)
  • R.H. Benedict et al.

    Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS)

    J. Int. Neuropsychol. Soc.

    (2006)
  • J. Bowen et al.

    Self-administered Expanded Disability Status Scale with functional system scores correlates well with a physician-administered test

    Mult. Scler.

    (2001)
  • D. Cella et al.

    Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology

    Neurology

    (2012)
  • S.W. Choi et al.

    PROsetta Stone Analysis Report: A Rosetta Stone for Patient Reported Outcomes

    (2013)
  • D.S. Conway et al.

    Long term benefit of multiple sclerosis treatment: an investigation using a novel data collection technique

    Mult. Scler.

    (2012)
  • P.C. Cozby

    Methods of Behavioral Research

    (2009)
  • S. D'Alisa et al.

    Depression is the main determinant of quality of life in multiple sclerosis: a classification-regression (CART) study

    Disabil. Rehabil.

    (2006)
  • A. Feinstein

    Multiple sclerosis, depression, and suicide

    BMJ

    (1997)
  • A. Feinstein

    An examination of suicidal intent in patients with multiple sclerosis

    Neurology

    (2002)
  • A. Feinstein

    Multiple sclerosis and depression

    Mult. Scler.

    (2011)
  • A. Feinstein et al.

    The link between multiple sclerosis and depression

    Nat. Rev. Neurol.

    (2014)
  • S.J. Ferrando et al.

    Patient Health Questionnaire-9 to screen for depression in outpatients with multiple sclerosis

    Int. J. MS Care

    (2007)
  • M. First et al.

    Structured Clinical Interview for DSM-IV-TR Axis I Disorders — Patient Edition (SCID-I/P, 4/2005 Revision)

    (2005)
  • M. First et al.

    Structured Clinical Interview for DSM-IV (SCID)

    (1995)
  • R.C. Gershon et al.

    Neuro-QOL: quality of life item banks for adults with neurological disorders: item development and calibrations based upon clinical and general population testing

    Qual. Life Res.

    (2012)
  • Cited by (0)

    View full text