Discussion
Mega-trials vs. meta-analysis: Precision vs. heterogeneity?

https://doi.org/10.1016/j.cct.2006.11.007Get rights and content

Abstract

In recent years, several authors have suggested there is a need for more very large or “mega-trials” (defined in this manuscript as a trial powered to address subgroup differences/interactions/secondary analyses) to answer important clinical questions. Because mega-trials are expensive and funding for clinical research is limited, increasing the number of mega-trials limits funding for other research. The advantages of this approach compared with funding more focused RCTs needs to be debated. Because there is no method to determine gold standard for which method gives the correct answer, we provide theoretical arguments that demonstrate that the two approaches are similar with respect to sample size requirements and the mega-trial approach provides a small advantage with respect to minimizing confounding by chance. However, the inherent heterogeneity in a series of smaller trials may represent a significant advantage over a single mega-trial.

Section snippets

Manuscript

Empirical data suggest that the results of meta-analyses from multiple smaller trials are usually consistent with the results of a mega-trial (defined as powered to address subgroup differences/interactions/secondary analyses; usually considered more generalizable) [1]. Although the results may usually be consistent, several authors have suggested that the mega-trial approach is inherently superior to smaller randomized trials (even when the results of the latter are pooled together in a

Sample size calculations

The number of subjects required to answer research questions in the two approaches is similar. Let us assume that the 2-year mortality for a disease in untreated persons will be 15% and that treatment will reduce mortality by 50%. Setting alpha = 0.05 and power = 0.8, and using an intention-to-treat analysis, we would need 304 subjects per group for the main effect. In addition, we expect a loss to follow-up of 10%, 80% compliance and we would like to examine compliance-based effects in addition to

Confounding by chance

The mega-trial approach has some advantage over the multi-trial approach with respect to adequacy of randomization. As the number of subjects increases, the probability that an important confounder will be unequally distributed between the two comparison groups decreases. Although often taken for granted, it is possible to calculate the expected distributions. In brief, if the prevalence of the only important potential confounder for the study is 20% and we randomize 400 subjects into 2 groups,

Precision vs. heterogeneity

Because variance is related to the size of the sample under study, mega-trials provide a very precise estimate of the overall effect (e.g. across sex and age). The multi-trial approach provides the same precision if the studies used homogeneous methodology [3]. In both cases, one should include intra- and inter-centre variability in the analysis. In essence, a multi-centre mega-trial is a collection of many smaller studies performed within different institutions at the same time with the same

Time-dependent/population differences

The first smaller trials of magnesium in the immediate post-myocardial infarction (post-MI) period suggested it was beneficial with respect to both mortality and arrhythmias [9], [10], [11], [12], [13]. Later studies questioned the benefit and the mega ISIS-4 trial eventually concluded that there was no effect [14]. This mega-trial has been cited as evidence for the need of mega-trials [3]. However, the sample size of the mega-trial was only one difference between the trials. The original

Implications for systematic reviews and meta-analyses

On theoretical reasoning, a random-effects model is more rational than a fixed-effects model for most meta-analyses because individual studies with methodological differences do not come from one overall population of studies but from a series of populations; one should account for the methodological variability between studies [18]. In the random-effects model, the inter-study variance is not sample-size dependent, and therefore a new study of 10,000 subjects contributes much less weight to

Conclusion

Both the multi-trial and the mega-trial approaches allow for precise estimates of effect and subgroup analyses. The multi-trial approach provides additional information about heterogeneity between study methodologies and populations that might not be evident at the time of one mega-trial. If heterogeneity exists between smaller studies, a close examination of the methodologies may lead to new hypotheses — and these may not be possible with a mega-trial approach. Finally, if funding agencies

References (18)

  • C. Anello et al.

    Exploratory or analytic meta-analysis: should we distinguish between them?

    J Clin Epidemiol

    (1995)
  • J.C. Cappelleri et al.

    Large trials vs meta-analysis of smaller trials

    JAMA

    (1996)
  • J. LeLorier et al.

    Discrepancies between meta-analyses and subsequent large randomized, controlled trials

    N Engl J Med

    (1997)
  • M. Egger et al.

    Where now for meta-analysis?

    Int J Epidemiol

    (2002)
  • S. Yusuf et al.

    Why do we need some large, simple randomized trials?

    Stat Med

    (1984)
  • A. Agresti

    Categorical data analysis

    (2002)
  • G.D. Smith et al.

    Going beyond the grand mean: subgroup analysis in meta-analysis of randomised trials

  • S.G. Thompson

    Why and how sources of heterogeneity should be investigated

  • B.C. Morton et al.

    Magnesium therapy in acute myocardial infarction—a double-blind study

    Magnesium

    (1984)
There are more references available in the full text version of this article.

Cited by (33)

  • The Mapping of Rejection

    2021, JACC: Cardiovascular Imaging
  • Return to work after shoulder arthroplasty: a systematic review and meta-analysis

    2019, Journal of Shoulder and Elbow Surgery
    Citation Excerpt :

    To limit any bias introduced by heterogeneity, a random-effects model was used to perform the meta-analyses. In addition, although a potential drawback, the heterogeneous data are reflective of the reality of clinical practice and may therefore most accurately represent what surgeons actually encounter in the clinical setting.2,37 We report the rate of RTW after shoulder arthroplasty with a systematic review and meta-analysis combining the results of 317 patients.

  • Network Meta-Analysis in Pharmacy Practice Research

    2019, Encyclopedia of Pharmacy Practice and Clinical Pharmacy: Volumes 1-3
  • Grading the strength of a body of evidence when assessing health care interventions: An EPC update

    2015, Journal of Clinical Epidemiology
    Citation Excerpt :

    Scoring consistency requires independent investigations of the same treatment-outcome comparison in more than one study. EPCs cannot be certain that a single study, no matter how large or well designed, presents the definitive picture of any particular clinical benefit or harm for a given treatment [38–40]. Accordingly, we recommend that the consistency of a single-study evidence base be scored as unknown.

  • Electronic health records and the quest to achieve the "triple aim"

    2015, Journal of the American College of Cardiology
View all citing articles on Scopus
View full text