Extremely large outlier treatment effects may be a footprint of bias in trials from less developed countries: randomized trials of gabapentinoids

Highlights

• We found a large number of published studies with implausible, extremely favorable results for gabapentin and pregabalin.

• Most of these studies had been done in less developed countries and reported no information on funding or conflicts of interest.

• Most of the studies with unfavorable results came from more developed countries.

• Our analysis suggests strong bias in the global literature on a topic with an already well-documented manufacturer-orchestrated strategy that distorted evidence in randomized trials.

• Trials with extreme, implausible results should be scrutinized and raise suspicion for the integrity of the evidence included in a meta-analysis.

Abstract

Objectives

Court documents have proven that a manufacturer-orchestrated strategy tried to promote gabapentin by distorting evidence in randomized trials. Given this background, we aimed to assess whether implausibly large treatment effects for gabapentin and for a similar gabapentinoid, pregabalin may have been published.

Study Design and Setting

We identified meta-analyses on gabapentin or pregabalin on any outcome from Google Scholar, PubMed, and EMBASE. We explored excess of significance in meta-analyses and whether outlier studies with extreme results (differing >0.8 standard deviations from the summary effect of the meta-analysis) were scrutinized.

Results

All 10 evaluated meta-analyses showed statistically significant favorable findings. Heterogeneity I² estimates exceeding 90% were noted in four meta-analyses of postoperative pain. In these four meta-analyses, 77 studies had estimates differing >0.8 standard deviations from the summary estimate. Thirty-nine of 77 represented extremely favorable results, and 33 of them came from less developed countries with no tradition of clinical research, 22 reported no information on funding, and 20 reported no conflicts of interest. Conversely, 27 of 38 studies with unfavorable results came from more developed countries.

Conclusion

Extremely favorable outlier studies in the meta-analyzed literature of gabapentin and pregabalin may be a footprint of bias in studies done in less developed countries.

Introduction

In 1993, Neurontin was originally approved by the US Food and Drug Administration (FDA) for adults for partial seizures. However, the drug was increasingly used for off-label treatment of pain and psychiatric conditions. In 1996, a lawsuit was filed for illegal marketing of gabapentin for off-label use [1]. In 2004, Warner-Lambert, now a subsidiary of Pfizer, pleaded guilty to these charges and paid about $430 million dollars for criminal and civil liability [2].

Internal company documents released as part of the litigation [3] show that Pfizer and Parke-Davis executed a strategy to conduct trials and disseminate trial findings for off-label uses, influence publication content, select findings for presentation/publication, and have their staff author articles for others without attribution. Using these internal documents, an evaluation of 11 published gabapentin trials [4], [5] demonstrated extensive manipulation and selective reporting of outcomes. Other authors have dissected gabapentin seeding trials, that is, trials run for marketing rather than real clinical purposes [6].

Since gabapentin was released in 1993, many trials have assessed the utility of gabapentin for diverse off-label indications; perhaps, most commonly for the reduction of postoperative pain where many dozens of relatively small studies have been published. This evidence has been assessed by several meta-analyses that try to summarize these findings [7], [8], [9], [10], [11]. Even when the meta-analysts have no conflicts, they still synthesize primary data that may have been generated by a biased, skewed process that causes inflated summary results. One may be particularly worried about studies with very extreme effects because these would cause the greatest inflation of the summary result. Reported extremely large effects are implausible. Very few interventions in medicine have truly huge effects [12], [13]. Furthermore, the documented history of gabapentin marketing may add further suspicion when extremely large effects are reported in specific trials.

The present study aims to explore gabapentin meta-analyses for the presence of studies with implausibly favorable results. We first sought evidence of excess significance (having too many studies with significant results), which may reflect publication bias, selective analysis, and outcome reporting bias [14]. Then we focused specifically on studies with extremely large effect sizes and tried to examine their provenance, reported funding, and reported conflicts of interest. We considered both gabapentin and pregabalin, given their similarity in terms of drug class (gabapentinoids), mechanism of action, and manufacturer [15].