DiscussionMega-trials vs. meta-analysis: Precision vs. heterogeneity?
Section snippets
Manuscript
Empirical data suggest that the results of meta-analyses from multiple smaller trials are usually consistent with the results of a mega-trial (defined as powered to address subgroup differences/interactions/secondary analyses; usually considered more generalizable) [1]. Although the results may usually be consistent, several authors have suggested that the mega-trial approach is inherently superior to smaller randomized trials (even when the results of the latter are pooled together in a
Sample size calculations
The number of subjects required to answer research questions in the two approaches is similar. Let us assume that the 2-year mortality for a disease in untreated persons will be 15% and that treatment will reduce mortality by 50%. Setting alpha = 0.05 and power = 0.8, and using an intention-to-treat analysis, we would need 304 subjects per group for the main effect. In addition, we expect a loss to follow-up of 10%, 80% compliance and we would like to examine compliance-based effects in addition to
Confounding by chance
The mega-trial approach has some advantage over the multi-trial approach with respect to adequacy of randomization. As the number of subjects increases, the probability that an important confounder will be unequally distributed between the two comparison groups decreases. Although often taken for granted, it is possible to calculate the expected distributions. In brief, if the prevalence of the only important potential confounder for the study is 20% and we randomize 400 subjects into 2 groups,
Precision vs. heterogeneity
Because variance is related to the size of the sample under study, mega-trials provide a very precise estimate of the overall effect (e.g. across sex and age). The multi-trial approach provides the same precision if the studies used homogeneous methodology [3]. In both cases, one should include intra- and inter-centre variability in the analysis. In essence, a multi-centre mega-trial is a collection of many smaller studies performed within different institutions at the same time with the same
Time-dependent/population differences
The first smaller trials of magnesium in the immediate post-myocardial infarction (post-MI) period suggested it was beneficial with respect to both mortality and arrhythmias [9], [10], [11], [12], [13]. Later studies questioned the benefit and the mega ISIS-4 trial eventually concluded that there was no effect [14]. This mega-trial has been cited as evidence for the need of mega-trials [3]. However, the sample size of the mega-trial was only one difference between the trials. The original
Implications for systematic reviews and meta-analyses
On theoretical reasoning, a random-effects model is more rational than a fixed-effects model for most meta-analyses because individual studies with methodological differences do not come from one overall population of studies but from a series of populations; one should account for the methodological variability between studies [18]. In the random-effects model, the inter-study variance is not sample-size dependent, and therefore a new study of 10,000 subjects contributes much less weight to
Conclusion
Both the multi-trial and the mega-trial approaches allow for precise estimates of effect and subgroup analyses. The multi-trial approach provides additional information about heterogeneity between study methodologies and populations that might not be evident at the time of one mega-trial. If heterogeneity exists between smaller studies, a close examination of the methodologies may lead to new hypotheses — and these may not be possible with a mega-trial approach. Finally, if funding agencies
References (18)
- et al.
Exploratory or analytic meta-analysis: should we distinguish between them?
J Clin Epidemiol
(1995) - et al.
Large trials vs meta-analysis of smaller trials
JAMA
(1996) - et al.
Discrepancies between meta-analyses and subsequent large randomized, controlled trials
N Engl J Med
(1997) - et al.
Where now for meta-analysis?
Int J Epidemiol
(2002) - et al.
Why do we need some large, simple randomized trials?
Stat Med
(1984) Categorical data analysis
(2002)- et al.
Going beyond the grand mean: subgroup analysis in meta-analysis of randomised trials
Why and how sources of heterogeneity should be investigated
- et al.
Magnesium therapy in acute myocardial infarction—a double-blind study
Magnesium
(1984)
Cited by (33)
The Mapping of Rejection
2021, JACC: Cardiovascular ImagingReturn to work after shoulder arthroplasty: a systematic review and meta-analysis
2019, Journal of Shoulder and Elbow SurgeryCitation Excerpt :To limit any bias introduced by heterogeneity, a random-effects model was used to perform the meta-analyses. In addition, although a potential drawback, the heterogeneous data are reflective of the reality of clinical practice and may therefore most accurately represent what surgeons actually encounter in the clinical setting.2,37 We report the rate of RTW after shoulder arthroplasty with a systematic review and meta-analysis combining the results of 317 patients.
Network Meta-Analysis in Pharmacy Practice Research
2019, Encyclopedia of Pharmacy Practice and Clinical Pharmacy: Volumes 1-3Grading the strength of a body of evidence when assessing health care interventions: An EPC update
2015, Journal of Clinical EpidemiologyCitation Excerpt :Scoring consistency requires independent investigations of the same treatment-outcome comparison in more than one study. EPCs cannot be certain that a single study, no matter how large or well designed, presents the definitive picture of any particular clinical benefit or harm for a given treatment [38–40]. Accordingly, we recommend that the consistency of a single-study evidence base be scored as unknown.
Electronic health records and the quest to achieve the "triple aim"
2015, Journal of the American College of CardiologyRobust meta-analytic conclusions mandate the provision of prediction intervals in meta-analysis summaries
2012, Journal of Clinical Epidemiology