Elsevier

Journal of Clinical Epidemiology

Volume 90, October 2017, Pages 127-135
Journal of Clinical Epidemiology

Original Article
Searching ClinicalTrials.gov did not change the conclusions of a systematic review

https://doi.org/10.1016/j.jclinepi.2017.07.009Get rights and content

Abstract

Objectives

We assessed the effect of searching ClinicalTrials.gov on the conclusions of a systematic review.

Study Design and Setting

We conducted this case study concurrently with a systematic review. We searched ClinicalTrials.gov on March 9, 2016, to identify trial records eligible for inclusion in the review. Two independent reviewers screened ClinicalTrials.gov records. We compared conclusions and strength of evidence grade with and without ClinicalTrials.gov records for 31 comparisons and 2 outcomes.

Results

We identified 106 trials (53 in the peer-reviewed literature only, 23 in ClinicalTrials.gov only, and 30 in both sources). For one comparison, the addition of results identified through ClinicalTrials.gov reduced the pooled effect size. We found evidence of selective outcome reporting for two comparisons and suspected publication bias for another two comparisons. For all other comparisons, searching ClinicalTrials.gov did not change conclusions or the strength of evidence grading for the two outcomes.

Conclusion

Our search of ClinicalTrials.gov bolstered suspicions of reporting biases but did not change either the conclusions or the strength of evidence grading. Further research is needed to determine the effect of searching ClinicalTrials.gov on the conclusions of systematic reviews in different topic areas and as the new rules for registration of trial results take effect.

Introduction

The underlying principle of systematic reviews is a consideration of all relevant available evidence. As standards have developed on how to conduct and report systematic reviews, an Achilles heel has remained: are we really considering all available evidence? Missing relevant information in systematic reviews because of reporting bias, such as publication bias and outcome reporting bias, may lead to biased and incorrect conclusions. Systematic review guidelines, such as the Cochrane Handbook and the Agency for Healthcare Research and Quality's Methods Guide, recommend searching for trials through registries, such as ClinicalTrials.gov, to assess and possibly ameliorate the effects of reporting bias [1], [2].

We sought to assess the effect of searching ClinicalTrials.gov on the conclusions and strength of evidence grading of a systematic review.

Section snippets

Methods

We conducted this study concurrently with a systematic review on the effectiveness of treatment options for symptoms of diabetic peripheral neuropathy. Our protocol for this case study can be found at www.effectivehealthcare.ahrq.gov [3].

Search results

There were 81 randomized controlled trials from the published literature search of the concurrent systematic review [4]. Our search of ClinicalTrials.gov yielded 266 records. Of these, 53 records met the inclusion criteria (Fig. 1). From both the peer-reviewed literature and ClinicalTrials.gov, we included a total of 106 trials that evaluated pharmacologic interventions to treat the symptoms of diabetic peripheral neuropathy. Thirty trials were identified in both the peer-reviewed literature

Discussion

We identified 53 trials comparing the effectiveness of treatment options for symptoms of diabetic peripheral neuropathy by searching ClinicalTrials.gov. However, most of the identified trials had no results posted in the registry to inform our conclusions or evidence grades in the concurrent systematic review.

We noted discrepancies between the 30 ClinicalTrials.gov records and their matched publications in terms of the number enrolled, the reported primary outcome, outcomes results, and adverse

Conclusion

In this case study, our search of ClinicalTrials.gov was mostly useful in bolstering the suspicion of reporting biases but did not meaningfully change either the conclusions or the strength of evidence grading of a systematic review. Further research is needed to determine the usefulness of searching ClinicalTrials.gov on the conclusions and strength of evidence grading for systematic reviews across different topic areas and as the new rules for reporting results in ClinicalTrials.gov are

Acknowledgments

The authors would like to thank the other authors of the “Effectiveness and Treatment Options for the Prevention of Complications and Treatments of Symptoms of Diabetic Peripheral Neuropathy, including Wendy L. Bennett, MD, MPH; Allen Zhang, BS; Suzanne Amato Nesbit, PharmD, BCPS, CPE; Hsin-Chieh Yeh, PhD; Yohalakshmi Chelladurai, MD, MPH; and Dorianne Feldman, MD, MSPT, for sharing the results of their systematic review.

References (50)

  • D.M. Hartung et al.

    Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications

    Ann Intern Med

    (2014)
  • D.A. Zarin et al.

    The ClinicalTrials.gov results database–update and key issues

    N Engl J Med

    (2011)
  • J.E. Cornell et al.

    Random-effects meta-analysis of inconsistent effects: a time for change

    Ann Intern Med

    (2014)
  • J.P. Higgins et al.

    Measuring inconsistency in meta-analyses

    BMJ

    (2003)
  • D. Bansal et al.

    Amitriptyline vs. pregabalin in painful diabetic neuropathy: a randomized double blind clinical trial

    Diabetic Med

    (2009)
  • M. Ghasemi et al.

    The effects of intradermal botulinum toxin type a injections on pain symptoms of patients with diabetic neuropathy

    J Res Med Sci

    (2014)
  • S. Karmakar et al.

    Investigating the role of neuropathic pain relief in decreasing gait variability in diabetes mellitus patients with neuropathic pain: a randomized, double-blind crossover trial

    J Neuroeng Rehabil

    (2014)
  • R. Allen et al.

    Clinical experience with desvenlafaxine in treatment of pain associated with diabetic peripheral neuropathy

    J Pain Res

    (2014)
  • B. Parsons et al.

    The efficacy of pregabalin in patients with moderate and severe pain due to diabetic peripheral neuropathy

    Curr Med Res Opin

    (2016)
  • T.E. Bertorini et al.

    Electrophysiological effects of mexiletine in painful neuropathy

    Nerve

    (1998)
  • P. Raskin et al.

    Pregabalin in patients with inadequately treated painful diabetic peripheral neuropathy: a randomized withdrawal trial

    Clin J Pain

    (2014)
  • H. Yasuda et al.

    Superiority of duloxetine to placebo in improving diabetic neuropathic pain: results of a randomized controlled trial in Japan

    J Diabetes Investig

    (2011)
  • P. Raskin et al.

    Pregabalin in patients with painful diabetic peripheral neuropathy using an NSAID for other pain conditions: a double-blind crossover study

    Clin J Pain

    (2016)
  • Y. Gao et al.

    Treatment of patients with diabetic peripheral neuropathic pain in China: a double-blind randomised trial of duloxetine vs. placebo

    Int J Clin Pract

    (2015)
  • S. Schwartz et al.

    Safety and efficacy of tapentadol ER in patients with painful diabetic peripheral neuropathy: results of a randomized-withdrawal, placebo-controlled trial

    Curr Med Res Opin

    (2011)
  • Cited by (0)

    Funding: This project was funded under contract no. 290-2015-00006I-2 from the Agency for Healthcare Research and Quality and U.S. Department of Health and Human Services. The authors of this manuscript are responsible for its content.

    Statements in the manuscript should not be construed as endorsement by the Agency for Healthcare Research and Quality (AHRQ) or the U.S. Department of Health and Human Services. AHRQ retains a license to display, reproduce, and distribute the data and the report from which this manuscript was derived under the terms of the agency's contract with the author.

    View full text