Original Article
Concordance of randomized and nonrandomized studies was unrelated to translational patterns of two nutrient-disease associations

https://doi.org/10.1016/j.jclinepi.2011.07.006Get rights and content

Abstract

Objective

There are several examples in nutrition of discordance between the results of observational studies and randomized controlled trials (RCTs). We hypothesized that this discordance is attributable to differences in the translational paths of nutrient–disease associations. Translational paths can be assessed using citation analysis.

Study Design and Setting

We compared the characteristics of citation networks using examples, where RCTs and observational studies agreed (long-chain n-3 polyunsaturated fatty acids [n-3 PUFA]) or disagreed (vitamin E). We performed systematic reviews in each example, constructed citation networks, and compared them with respect to the number of articles and citation relationships between them, as well as the distribution of articles' hub and authority scores.

Results

For n-3 PUFA, meta-analyses of 14 RCTs and 10 observational studies both suggested that higher intake was associated with lower cardiovascular mortality. For vitamin E, the meta-analysis of 14 RCTs excluded a clinically significant effect, whereas 14 observational studies reported a significant inverse association. The respective citation networks consisted of 392 (n-3 PUFA) and 351 (vitamin E) articles. No differences between the characteristics of the two networks were identified. There was no evidence that the observational studies predated RCTs in the translational process in either example.

Conclusion

In the two examples, citation network characteristics do not predict concordance in the results of observational studies and RCTs.

Section snippets

Context

What is new?

  • •

    For many nutrient associations of disease, randomized controlled trials (RCTs) and observational studies reach conflicting results. Explanations for such discordances may be traceable in the translational paths leading to RCTs and observational studies—but this has not been studied till now.

  • •

    We analyzed the citation networks of publications on the associations of long-chain n-3 polyunsaturated fatty acids (n-3 PUFA) and vitamin E with cardiovascular mortality.

  • •

    We observed that the

Selection of the concordant and the discordant example

For proof of concept, we decided to study the evidence base for two nutrient associations of disease outcomes. After feedback from a technical expert panel composing of four nutritional epidemiology and methodology experts, we selected two nutritional associations with cardiovascular mortality: one where the results of RCTs and prospective cohorts (“observational studies”) were statistically significant and had the same direction (“concordant” example); and one where the summary of

Meta-analyses of RCTs and observational studies

We performed separate meta-analyses of RCTs and observational studies (prospective cohorts) for the n-3 PUFA and vitamin E examples. The online Appendix (on the journal’s Web site at www.jclinepi.com) describes the characteristics of the eligible studies for each systematic review, and the results of the quantitative analyses for cardiovascular mortality outcomes.

Briefly, in the n-3 PUFA example, we identified 14 RCTs (reported in 15 publications) [30], [31], [32], [33], [34], [35], [36], [37],

Discussion

We analyzed the citation networks of publications on the associations of n-3 PUFA and vitamin E with cardiovascular mortality. We observed that, in both examples, citation networks were grossly similar with respect to their quantitative characteristics (such as the number of articles and citation relationships among articles), and the connectivity of the articles. This was also true for citation networks limited to the subset of articles that described primary data in humans and were pertinent

Acknowledgments

Contributors: T.A.T. and J.Lau had the initial idea. T.A.T. designed the study and drafted the protocol with input from J.Lau. T.A.T., D.M., W.W.Y., J.Lee, and M.C. acquired the data. T.A.T. developed software for citation graph manipulation and analyses and wrote the first draft of the paper. All authors critically revised the paper. No other person including medical editors has assisted in any way in the writing or the preparation of the article. T.A.T. had full access to all of the data in

References (110)

  • G.C. Leng et al.

    Randomized controlled trial of gamma-linolenic acid and eicosapentaenoic acid in peripheral arterial disease

    Clin Nutr

    (1998)
  • M. Yokoyama et al.

    Effects of eicosapentaenoic acid on major coronary events in hypercholesterolaemic patients (JELIS): a randomised open-label, blinded endpoint analysis

    Lancet

    (2007)
  • M.H. Kamphuis et al.

    Depression and cardiovascular mortality: a role for n-3 fatty acids?

    Am J Clin Nutr

    (2006)
  • N.G. Stephens et al.

    Randomised controlled trial of vitamin E in patients with coronary disease: Cambridge Heart Antioxidant Study (CHAOS)

    Lancet

    (1996)
  • M. Boaz et al.

    Secondary prevention with antioxidants of cardiovascular disease in endstage renal disease (SPACE): randomised placebo-controlled trial

    Lancet

    (2000)
  • S. Devaraj et al.

    Effect of high-dose alpha-tocopherol supplementation on biomarkers of oxidative stress and inflammation and carotid atherosclerosis in patients with coronary artery disease

    Am J Clin Nutr

    (2007)
  • K.G. Losonczy et al.

    Vitamin E and vitamin C supplement use and risk of all-cause and coronary heart disease mortality in older persons: the Established Populations for Epidemiologic Studies of the Elderly

    Am J Clin Nutr

    (1996)
  • B. Buijsse et al.

    Both alpha- and beta-carotene, but not tocopherols and vitamin C, are inversely related to 15-year cardiovascular mortality in Dutch elderly men

    J Nutr

    (2008)
  • M. Dietrich et al.

    Vitamin E supplement use and the incidence of cardiovascular disease and all-cause mortality in the Framingham Heart Study: does the underlying health status play a role?

    Atherosclerosis

    (2009)
  • K. Klipstein-Grobusch et al.

    Dietary antioxidants and risk of myocardial infarction in the elderly: the Rotterdam Study

    Am J Clin Nutr

    (1999)
  • C.M. van Gent et al.

    Effect, on serum lipid levels of omega-3 fatty acids, of ingesting fish-oil concentrate

    Lancet

    (1979)
  • A.P. Simopoulos

    Omega-3 fatty acids in health and disease and in growth and development

    Am J Clin Nutr

    (1991)
  • E. Guallar et al.

    A prospective study of plasma fish oil levels and incidence of myocardial infarction in U.S. male physicians

    J Am Coll Cardiol

    (1995)
  • J.L. Breslow

    n-3 fatty acids and cardiovascular disease

    Am J Clin Nutr

    (2006)
  • K.F. Gey et al.

    Plasma levels of antioxidant vitamins in relation to ischemic heart disease and cancer 14

    Am J Clin Nutr

    (1987)
  • A. Koushik et al.

    Intake of the major carotenoids and the risk of epithelial ovarian cancer in a pooled analysis of 10 cohort studies

    Int J Cancer

    (2006)
  • J.E. Lee et al.

    Intakes of fruits, vegetables, vitamins A, C, and E, and carotenoids and risk of renal cell cancer

    Cancer Epidemiol Biomarkers Prev

    (2006)
  • J. Lin et al.

    Dietary intakes of fruit, vegetables, and fiber, and risk of colorectal cancer in a prospective cohort of women (United States)

    Cancer Causes Control

    (2005)
  • I. Shai et al.

    Ethnicity, obesity, and risk of type 2 diabetes in women: a 20-year follow-up study

    Diabetes Care

    (2006)
  • A. Schatzkin et al.

    Lack of effect of a low-fat, high-fiber diet on the recurrence of colorectal adenomas. Polyp Prevention Trial Study Group

    N Engl J Med

    (2000)
  • S. Yusuf et al.

    Vitamin E supplementation and cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators

    N Engl J Med

    (2000)

  • The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group

    N Engl J Med

    (1994)
  • H.D. Sesso et al.

    Vitamins E and C in the prevention of cardiovascular disease in men: the Physicians’ Health Study II randomized controlled trial

    JAMA

    (2008)
  • L.A. Bazzano et al.

    Effect of folic acid supplementation on risk of cardiovascular diseases: a meta-analysis of randomized controlled trials

    JAMA

    (2006)
  • C. Wang et al.

    Effects of omega-3 fatty acids on cardiovascular disease

    (2004)
  • H. Leon et al.

    Effect of fish oil on arrhythmias and mortality: systematic review

    BMJ

    (2008)
  • Z. Ye et al.

    Antioxidant vitamins intake and the risk of coronary heart disease: meta-analysis of cohort studies

    Eur J Cardiovasc Prev Rehabil

    (2008)
  • A. Mente et al.

    A systematic review of the evidence supporting a causal link between dietary factors and coronary heart disease

    Arch Intern Med

    (2009)
  • Batagelj, V. Efficient algorithms for citation network analysis. Technical report IMFM 897, arXiv.org:cs/0309023v1...
  • W. deNooy et al.

    Exploratory social network analysis with Pajek

  • J. Clark et al.

    A first look at graph theory

    (1991)
  • D. Koller et al.

    Probabilistic graphical models—principles and techniques

    (2009)
  • A.A. Alsheikh-Ali et al.

    The vulnerable atherosclerotic plaque: scope of the literature

    Ann Intern Med

    (2010)
  • I. Dahabreh et al.

    CYP2D6 testing to predict response to tamoxifen in women with breast cancer. Pharmacogenomic

    PLoS Curr

    (2010)
  • I.J. Dahabreh et al.

    Systematic review: anti-epidermal growth factor receptor treatment effect modification by KRAS mutations in advanced colorectal cancer

    Ann Intern Med

    (2011)
  • T. Terasawa et al.

    BCR-ABL mutation testing to predict response to tyrosine kinase inhibitors in patients with chronic myeloid leukemia

    PLoS Curr

    (2010)
  • E.B. Rimm et al.

    Vitamin E consumption and the risk of coronary heart disease in men

    N Engl J Med

    (1993)
  • P.N. Durrington et al.

    An omega-3 polyunsaturated fatty acid concentrate administered for one year decreased triglycerides in simvastatin treated patients with coronary heart disease and persisting hypertriglyceridaemia

    Heart

    (2001)
  • R. Marchioli et al.

    Antiarrhythmic mechanisms of n-3 PUFA and the results of the GISSI-Prevenzione trial

    J Membr Biol.

    (2005)
  • A. Macchia et al.

    Left ventricular systolic dysfunction, total mortality, and sudden death in patients with myocardial infarction treated with n-3 polyunsaturated fatty acids

    Eur J Heart Fail.

    (2005)

    • Cited by (14)

    • Tea in cardiovascular health and disease: a critical appraisal of the evidence

      2022, Food Science and Human Wellness
      Citation Excerpt :

      However, the wide range of catechin dosages administered in the available studies (208–1 344 mg/day [61]) do not allow any conclusions as to the dose-response relationship between tea intake and blood pressure-reducing effects. The findings of observational studies may help determine potential effective doses for testing in RCTs, which are frequently initiated before sufficient evidence of an effective dose is available [106]. In the absence of reports of toxicity associated with tea consumption, tea is considered to be a safe beverage.

    • Citations alone were enough to predict favorable conclusions in reviews of neuraminidase inhibitors

      2015, Journal of Clinical Epidemiology
      Citation Excerpt :

      These biases come in many forms, including the preferential inclusion of studies with positive outcomes and statistically significant results [11–13], from high-impact journals or authors with financial conflicts of interest [14–17], or disproportionate levels of self-citation [18,19]. Citation network analyses have been used to examine the incidence and potential implications of differences in evidence selection [1,20–24]. However, little is known about how the biases in evidence selection may affect the conclusions of reviews.

    • Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

      2024, Cochrane Database of Systematic Reviews

    • Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study

      2024, European Journal of Epidemiology

    • Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease

      2020, Cochrane Database of Systematic Reviews

    • Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease

      2018, Cochrane Database of Systematic Reviews
    View all citing articles on Scopus
    View full text
    Copyright © 2012 Elsevier Inc. All rights reserved.