Elsevier

Journal of Clinical Epidemiology

Volume 124, August 2020, Pages 106-117
Journal of Clinical Epidemiology

Original Article
Hamilton scale and MADRS are interchangeable in meta-analyses but can disagree at trial level

https://doi.org/10.1016/j.jclinepi.2020.04.022Get rights and content

Highlights

  • Trials in major depressive disorder reporting the results on both the Montgomery and the Hamilton rating scales have been studied;

  • None of the two scales is systematically more efficient than the other in detecting the treatment effect;

  • The same trial could have a very different interpretation depending on which one of the two scales is used as the primary outcome measurement tool.

Abstract

Background and Objective

Major depressive disorder is a multidimensional disease, in which demonstrating the efficacy of treatments is difficult. The Hamilton Rating Scale for Depression (HRSD) and the Montgomery–Asberg Depression Rating Scale (MADRS) cover different domains but are used interchangeably as primary measures of the outcome in trials and—with standardized measures—in meta-analyses. We aimed at understanding (i) whether the choice of the outcome measurement tool can influence the outcome of a trial, and if so, (ii) whether one systematically outperforms the other, and (iii) whether using standardized measures of the effect in meta-analysis is justified.

Methods

Short-term randomized trials in patients with major depressive disorder that used both the scales were systematically searched and the results were collected. To quantify the differences in the results—both in terms of the standardized mean difference (SMD) and odds ratio (OR) for response—and their range, data were analyzed and plotted with the Bland–Altman method.

Results

161 comparisons from 80 studies were included, involving a total of 18,189 patients. Neither of the two scales appears systematically more sensitive to the treatment effect than the other in terms of SMDs (P-value = 0.06, 95% CI −0.044 to 0.001) or ORs (P-value = 0.15, 95% CI −0.25 to 0.04). However, the variability of differences between the HRSD and MADRS largely depends on the number of patients included in the comparison.

Conclusion

No systematic differences between the two scales were found supporting the use of standardized measures in meta-analyses. However, the same trial may give very different results with either scale, especially in small trials. Further research is needed to understand the causes of this variability.

Introduction

Major depressive disorder (MDD) is an intrinsically multidimensional clinical entity [1] that creates a significant burden on individuals and societies [2]. The evaluation of patients with MDD is difficult [3], and a panoply of scales has been proposed. Among these, the Hamilton Rating Scale for Depression (HRSD) [4]—particularly its 17-items version (HRSD17)—and the Montgomery–Asberg Depression Rating Scale (MADRS) [5] are accepted [6] as primary outcome measurement tools in phase 3 trials. Drawing conclusions from studies using either tool implicitly assumes that they measure the same construct, i.e., that they measure in the same way the severity of depression.

This assumption is made explicit when the magnitude of the effect of different interventions is compared in meta-analyses that use standardized measures of the effect—standardized mean difference (SMD) and odds ratio (OR) of response—independently of which the outcome measurement tool originated the data. The use of these measures has come to be the widespread practice, despite early warnings that these should only be relied on when different scales are on the linear transformation of the other [7]. The less frequently used conversion tables ([8]) make a similar assumption at the patient level, although not the same distributional assumptions.

As shown in Table 1 and more formally demonstrated in the study by Fried and Santor et al [9,10], the HRSD17 and the MADRS differ in the domains covered. Hence, it is reasonable to presume that they might not measure the exact same phenomenon and that they might not be subjected to the same degree of change in response to the same underlying effect of a treatment. Indeed, building a scale more sensitive to the change was the intention in creating the MADRS [5]. If this is the case, the choice of measurement tool might determine whether a trial detects an effect, and, if so, how large. As a consequence, meta-analyses would be misled by using standardized measures of the effect including values deriving from both the scales.

This research aimed to answer the following questions:

  • (i)

    Would the results of trials differ depending on the choice of basing the primary inference on the HRSD or the MADRS?

  • (ii)

    Would trials be more efficient (i.e., systematically more able to detect differences) using one or the other scale?

  • (iii)

    Can the originating scale be ignored when comparing interventions through standardized measures of the effect in meta-analyses?

Section snippets

Study selection

Data from published randomized clinical trials in the MDD reporting results for both HRSD and MADRS were analyzed. Details of the literature search are reported in the supplementary material.

Inclusion criteria for trials were as follows:

  • -

    Enrollment of patients with MDD;

  • -

    Random treatment assignment;

  • -

    Date of publication between August 2002 and August 2017;

  • -

    Availability of short-term duration (up to 14 weeks) measures of the effect;

  • -

    The effect size for change from the baseline to the endpoint of both

Results

The search (see Supplementary Material) found 4,471 records. Of those, 549 correspond to studies that were retrieved and assessed for eligibility (see Figure 1).

After applying the prespecified eligibility criteria, 80 [[22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68]

Discussion

In our study, we did not find systematic differences favoring either scale when evaluating treatments for MDD, independently from the size of the effect and the drug class. These results support the use of summary measures for meta-analyses using data from both outcome measurement tools. The lack of a meaningful systematic difference in sensitivity contrasts with the intention—in developing the MADRS—to have an outcome measurement tool particularly fit to demonstrate treatment effects [5]. In

Conclusion

The interpretation of the same trial can be significantly different depending on the scale used to draw the primary conclusion. Further research is needed to clarify the nature of the observed differences. As already advised in the study by Fried and Nesse [115], a trial whose conclusions are supported by the results on two measurement scales would appear more robust. The results on only one scale in small trials are difficult to interpret. Research on new scales or methods of administration of

CRediT authorship contribution statement

Lorenzo Guizzaro: Conceptualization, Methodology, Investigation, Formal analysis, Writing - review & editing. David Dickinson Vaughan Morgan: Conceptualization, Formal analysis, Writing - review & editing. Andrea Falco: Methodology, Investigation, Formal analysis, Writing - review & editing. Ciro Gallo: Conceptualization, Formal analysis, Writing - review & editing.

Acknowledgments

The authors are grateful to Prof. Tony Fox, whose comments on the research proposal have been very helpful and to the reviewer Prof. Fried for his helpful contribution in the interpretation of the results of the study. L.G. wishes to express his gratitude to his colleagues Florence Butlen and Frank Pétavy and to Prof. Andrea Cipriani for the discussions that triggered the research question for this article. Evidence proved I was wrong.

References (123)

  • A.F. Leuchter et al.

    Efficacy and safety of low-field synchronized transcranial magnetic stimulation (sTMS) for treatment of major depression

    Brain Stimul

    (2015)
  • J.C. Nelson et al.

    Combining norepinephrine and serotonin reuptake inhibition mechanisms for treatment of depression: a double-blind, randomized study

    Biol Psychiatry

    (2004)
  • J.P. O’Reardon et al.

    Efficacy and safety of transcranial magnetic stimulation in the acute treatment of major depression: a multisite randomized controlled trial

    Biol Psychiatry

    (2007)
  • D.G.S. Perahia et al.

    Duloxetine in the treatment of major depressive disorder: a placebo-and paroxetine-controlled trial

    Eur Psychiatry

    (2006)
  • C. Plewnia et al.

    Treatment of major depression with bilateral theta burst stimulation: a randomized controlled pilot trial

    J Affect Disord

    (2014)
  • J. Sobiś et al.

    Therapeutic efficacy assessment of weak variable magnetic fields with low value of induction in patients with drug-resistant depression

    J Affect Disord

    (2010)
  • K.A. Tourian et al.

    Desvenlafaxine 50 and 100 mg/d in the treatment of major depressive disorder: an 8-week, phase III, multicenter, randomized, double-blind, placebo-controlled, parallel-group trial and a post hoc pooled analysis of three studies

    Clin Ther

    (2009)
  • P. Tran et al.

    Efficacy and tolerability of the novel triple reuptake inhibitor amitifadine in the treatment of patients with major depressive disorder: a randomized, double-blind, placebo-controlled trial

    J Psychiatr Res

    (2012)
  • A.P. Van Amerongen et al.

    A randomised, double-blind comparison of milnacipran and imipramine in the treatment of depression

    J Affect Disord

    (2002)
  • E. Vieta et al.

    Efficacy and tolerability of flexibly-dosed adjunct TC-5214 (dexmecamylamine) in patients with major depressive disorder and inadequate response to prior antidepressant

    Eur Neuropsychopharmacol

    (2014)
  • B. Yazicioglu et al.

    A comparison of the efficacy and tolerability of reboxetine and sertraline versus venlafaxine in major depressive disorder: a randomized, open-labeled clinical trial

    Prog Neuropsychopharmacol Biol Psychiatry

    (2006)
  • Diagnostic and statistical manual of mental disorders

    (2013)
  • The global burden of disease: 2004 update

  • M. Fava et al.

    The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach

    Psychother Psychosom

    (2003)
  • M. Hamilton

    A rating scale for depression

    J Neurol Neurosurg Psychiatry

    (1960)
  • S.A. Montgomery et al.

    A new depression scale designed to be sensitive to change

    Br J Psychiatry

    (1979)
  • Guideline on clinical investigation of medicinal products in the treatment of depression

    (2013)
  • L.V. Hedges

    Statistical Methodology in Meta-Analysis, Princeton, N.J.: ERIC Clearinghouse on Tests, Measurement, and Evaluation

    (1982)
  • T.A. Furukawa et al.

    Translating the BDI and BDI-II into the HAMD and vice versa with equipercentile linking

    (2020)
  • D.A. Santor et al.

    Eight decades of measurement in depression

    Meas Interdiscip Res Perspect

    (2006)
  • J.J. Deeks et al.

    Analysing data and undertaking meta-analyses’

  • L.V. Hedges

    Distribution theory for Glass's estimator of effect size and related estimators

    J Educ Stat

    (1981)
  • J.M. Bland et al.

    Statistical methods for assessing agreement between two methods of clinical measurement

    Lancet

    (1986)
  • J. Zang et al.

    Statistical modeling and verification for the synthesis of median survival time in multilevel meta-analysis of survival data

    J Epidemiol Res

    (2015)
  • D. Moher et al.

    Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

    PLoS Med

    (2009)
  • A.C. Del Re

    Compute.es: compute effect sizes

  • B. Lehnert

    BlandAltmanLeh: Plots (Slightly Extended) Bland-Altman Plots

  • H. Wickham

    ggplot2: Elegant Graphics for Data Analysis

    (2016)
  • M. Gordon et al.

    forestplot: Advanced Forest Plot Using 'grid' Graphics

  • D. Bakish et al.

    Levomilnacipran ER 40 mg and 80 mg in patients with major depressive disorder: a phase III, randomized, double-blind, fixed-dose, placebo-controlled study

    J Psychiatry Neurosci

    (2014)
  • J.G. Barbee et al.

    A double-blind placebo-controlled trial of lamotrigine as an antidepressant augmentation agent in treatment-refractory unipolar depression

    J Clin Psychiatry

    (2011)
  • R.J. Bielski et al.

    A double-blind comparison of escitalopram and venlafaxine extended release in the treatment of major depressive disorder

    J Clin Psychiatry

    (2004)
  • R.J. Bielski et al.

    Gepirone extended-release in the treatment of adult outpatients with major depressive disorder: a double-blind, randomized, placebo-controlled, parallel-group study

    J Clin Psychiatry

    (2008)
  • B. Binneman et al.

    A 6-week randomized, placebo-controlled trial of CP-316,311 (a selective CRH 1 antagonist) in the treatment of major depression

    Am J Psychiatry

    (2008)
  • J.A. Bodkin et al.

    Transdermal selegiline in major depression: a double-blind, placebo-controlled, parallel-group study in outpatients

    Am J Psychiatry

    (2002)
  • P. Boyer et al.

    Efficacy, safety, and tolerability of fixed-dose desvenlafaxine 50 and 100 mg/day for major depressive disorder in a placebo-controlled trial

    Int Clin Psychopharmacol

    (2008)
  • A.R. Brunoni et al.

    Trial of electrical direct-current therapy versus escitalopram for depression

    N Engl J Med

    (2017)
  • A.H. Clayton et al.

    Efficacy and safety of desvenlafaxine 50 mg/d in a randomized, placebo-controlled study of perimenopausal and postmenopausal women with major depressive disorder

    J Clin Psychiatry

    (2013)
  • D.L. Dunner et al.

    Efficacy and tolerability of adjunctive ziprasidone in treatment-resistant depression: a randomized, open-label, pilot study

    J Clin Psychiatry

    (2007)
  • E. Ehrich et al.

    Evaluation of opioid modulation in major depressive disorder

    Neuropsychopharmacology

    (2015)
  • Cited by (6)

    • Bifidobacterium breve CCFM1025 attenuates major depression disorder via regulating gut microbiome and tryptophan metabolism: A randomized clinical trial

      2022, Brain, Behavior, and Immunity
      Citation Excerpt :

      The packaging, color, and quality of the placebo sachet are the same as the probiotic sachet. The primary outcome for this study was the mean change of the depression symptoms assessed with psychometric questionnaires, including Hamilton Depression Rating scale-24 Items (HDRS-24) (Henigsberg et al., 2012), Montgomery-Asberg Depression Rating Scale (MADRS) (Guizzaro et al., 2020), and Brief Psychiatric Rating Scale (BPRS) (Lachar et al., 1999). Notably, the BPRS items were classified into five aspects, including “Hostility”, “Thinking disorder”, “Activation”, “Withdrawal-Retardation”, and “Anxiety-Depression” (Lachar et al., 1999).

    Conflicts of interest statement: The authors have no conflicts of interests to disclose.

    View full text