Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Identifying competing interest disclosures in systematic reviews of surgical interventions and devices: a cross-sectional survey

BMC Medical Research Methodology volume 20, Article number: 260 (2020) Cite this article

Abstract

Background

A competing interest is an important source of bias in research and disclosure is frequently employed as a strategy to manage it. Considering the importance of systematic reviews (SRs) and the varying prevalence of competing interests in different research fields, we conducted a survey to identify the range of competing interests in SRs assessing surgical interventions or devices and explored the association between the competing interest disclosures and authors’ conclusions.

Methods

We retrieved SRs of surgical interventions and devices published in 2017 via PubMed. Information regarding general characteristics, funding sources, and competing interest disclosures were extracted. We conducted a descriptive analysis of the studies’ characteristics and compared them between Cochrane SRs (CSRs) and non-Cochrane SRs using the Chi-square test. Results were expressed as odds ratio and their 95% confidence interval.

Results

One hundred fifty-five SRs published in 2017 were included in the study. More than half of the SRs (58.7%) reported their funding sources and 94.2% reported authors’ competing interest disclosures. Among 146 SRs that stated competing interest disclosures, only 35 (22.6%) SRs declared at least one author had a competing interest. More than 40 terms were used to describe competing interests. Cochrane SRs (CSRs) were more likely to provide a detailed description of competing interests compared to those in non-CSRs (48.0% versus 25.4%, P = 0.023). No association between positive conclusions and competing interest disclosures was found (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no competing interest disclosure were more likely to report positive conclusions than those stating at least one type of competing interest, but the difference is not significantly different (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).

Conclusion

In surgical SRs, there is a high percentage of competing interest disclosures but without detailed information. The identification and statement of competing interests with a detailed description, particularly the non-financial ones, needs improvement. Some efficient and effective methods/tools for identifying, quantifying, and minimizing potential competing interests in systematic reviews remains valuable.

Peer Review reports

Background

A competing interest is defined as “a set of circumstances which create a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest” [1, 2] and considered to be “ubiquitous and inevitable in academic life” [3, 4]. A full disclosure is acknowledged as an important method for reporting and managing competing interests and serves to highlight the potential for bias [5,6,7,8]. Since 1988, the International Committee of Medical Journal Editors (ICMJE) has required authors to disclose financial or other relationships that might lead to a competing interest [9], and hundreds of other biomedical journals put similar policy into practice [10].

The number of systematic reviews (SR) increased rapidly in the recent year for providing end-users with a comprehensive, critical, and up to date method to synthesize available evidence. Meanwhile, studies regarding competing interests of systematic reviews increased for the concerns about suboptimal decisions made by end-users for potential competing interests [11,12,13,14,15,16]. Surgery, with its wide use of medical devices and strong personal preference for interventions, could be a field with high potential for competing interests [17]. In 2017, the JAMA published a collection to discuss issues relating to competing interests and stated that transparent disclosure is increasingly emphasized as an essential component in the reporting of both clinical trials and systematic reviews [18, 19]. Although numerous published studies addressed competing interests in medical research, there is a significant disproportion in the number of studies assessing competing interests, especially non-financial interests in surgical interventions and devices research [20,21,22].

Considering the importance of SRs and the varying prevalence of competing interests in different fields [23]. We, therefore, performed a survey to identify the range of competing interests in systematic reviews assessing surgical interventions and devices and proposed a potential checklist for competing interest disclosures in surgical research. We also explored the impact of competing interest disclosures on the study conclusions.

Methods

Eligibility criteria

A study was included if it was described as a systematic review or a meta-analysis and only included RCTs assessing at least one surgical intervention or device. Network meta-analyses, methodological systematic reviews or systematic reviews reported as conference abstracts and research letters were excluded. A systematic review was defined as described in the Cochrane handbook (a. a clearly stated set of objectives with an explicit, reproducible methodology; b. a systematic search that attempts to identify all studies that would meet the eligibility criteria; c. an assessment of the validity of the findings of the included studies, such as through the assessment of risk of bias; and d. systematic presentation and synthesis of the characteristics and findings of the included studies) [24] and the definitions of a surgical intervention and device have been described elsewhere [25]. A competing interest disclosure was defined as whether a competing interest disclosure was stated or not.

Data resource and study procedures

Systematic reviews were identified through PubMed, we limited our search to literature published in the English language, and between 1 January 2017 and 31 December 2017 due to a massive number of published surgical studies. The search strategy was based on MeSH terms and their variants and developed in collaboration with an experienced librarian (Additional file 1). Two reviewers (JY and GS), trained in trial and systematic review methods, used predefined, pilot-tested forms independently to screen abstracts and full texts for eligibility.

Data collection

Two teams of reviewers (JY and GS, YZ and ZY) independently collected data from eligible studies. Twenty systematic reviews met eligibility criteria were enrolled for a pilot study before the formal screening and data extraction. We iteratively adjusted the screening rules to improve the consistency between reviewers. Any conflicts not resolved by discussion between the reviewers were referred to the study team. Coher’s κ statistic was used to assess the concordance of reviewers, and a value of 0.75 or greater was chosen for satisfactory agreement.

The following general information was collected for each eligible study: (1) number of authors; (2) country of the corresponding author; (3) number of trials included; (4) total number of participants involved; (5) review type (Cochrane SR or non-Cochrane SR); (6) adherence to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses); (7) type of journal (general or surgical journal); (8) type of control (surgical intervention, non-surgical intervention or both); (9) involvement of methodologist (i.e. statistician, epidemiologist). We judged that a systematic review involved a methodologist if any of the authors were affiliated with a department of epidemiology, statistics, or evidence-based medicine, or listed in the acknowledgement section.

We extracted the funding sources and classified them as industry funding (e.g. funding from the device industry; company; insurance company), non-industry funding (e.g. funding from the academic institution; university; hospital; government bodies; foundation; charity et al.), no funding and not reported. We extracted the statement of competing interests verbatim and then categorized them based on their characteristics and nature.

The authors’ conclusion was labelled “positive” when the overall results favoured intervention over control or both groups were equal but the interventional group had minor advantages. A “negative” conclusion was defined if the results favoured control or both groups were equal [26].

Data analysis

We conducted a descriptive analysis of study characteristics of included systematic reviews. For all descriptive analyses, frequencies (and percentages) were used for dichotomous variables, and mean (and standard deviation) or median (and range) for continuous variables. We compared the funding sources and competing interest disclosures between Cochrane SR and non-CSR using Fisher’s exact test for dichotomous variables.

We categorized the studies into two groups based on whether the investigators disclosed competing interests or not. If they did, we then divided the competing interest disclosure group into two subgroups (at least one type of competing interest was disclosed and no competing interest disclosure group). The differences in the authors’ conclusions were compared between groups using the Chi-square test. Results were expressed as odds ratios and their 95% confidence interval. All analyses were performed by IBM SPSS Statistics 26.

Results

A total of 6256 systematic reviews were identified from PubMed. After the title and abstract screening, 479 were found to be potentially eligible, of which 155 were finally included based on the full-text article review (Fig. 1). The interobserver agreement on data collection was good (κ = 0.84). A full list of the included SRs is presented in Additional file 2.

Fig. 1
figure 1

Study selection

Full size image

General information from the included SRs

The general characteristics are listed in Table 1. The median number of included trials was 6 (range 2 to 21), 25 (16.1%) were Cochrane SRs and 89 (57.4%) mentioned PRISMA in their studies. 109 (70.3%) were published in general medical journals; 23 (16.1%) involved researchers with affiliations with either epidemiology or statistics departments, and 132 (85.2%) tested alternative surgical interventions.

Table 1 General characteristics of included systematic review
Full size table

Funding information from the included SRs

Of the 155 selected SRs, 42 (27.1%) reported review funding from governmental agencies and academic institutions, three (1.9%) reported review funding from pharmaceutical or devices company, 45 (29.0%) stated the reviews were not funded; and 64 (41.3%) did not provide the funding information (Table 2).

Table 2 Classification of funding source and competing interests in the included systematic reviews
Full size table

Competing interest disclosures concerning the included SRs

One hundred and forty-six studies (94.2%) disclosed the information of authors’ competing interests. However, of the 146 systematic reviews that provided competing interest disclosures, only 35 (22.6%) SRs declared at least one author had one competing interest. More than 40 terms were used to describe competing interests, and we categorized them into 12 terms that were the most common descriptions of competing interests, including grants/fellowship, honoraria, consulting, employment/salary, patent/copyright, equity/stocks/bonds, non-monetary support, service in other affiliations, founder or other leadership in a company, intellectual beliefs, experience, and personal relationship.

Of the 35 SRs that stated competing interests, Cochrane SRs were more likely to provide a detail description of the competing interests in comparison to those that are non-CSRs (48.0% versus 25.4%, P = 0.023). The most frequently reported types was “grant/fellowship” (n = 17, 11.0%), followed by “honoraria” (n = 15, 9.7%), “consulting” (n = 14, 9.0%), “non-monetary support” and “services in other affiliations” (n = 8, 5.2%) (Table 2).

We found no significant difference between SRs that had a competing interest disclosure and those that did not (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no competing interest disclosure were more likely to report a positive conclusion than those stating at least one type of competing interest. However, the difference is still not significant (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).

Discussion

Our survey shows that 94.2% of surgical systematic reviews published in 2017 reported authors’ competing interest disclosures. This result is in line with the strict journal requirement for more than 95% of medical journals have a policy, that refers to competing interest disclosure as an essential part of biomedical studies [16]. However, of the147 studies that declared their competing interests, only 35 (22.6%) provided a detailed disclosure.

There are several possible reasons for the low rate of detailed competing interest disclosures. One of the reasons is, most researchers are unaware of the fact that behaviours which include receiving support for food and beverages or academic competition are also competing interests and could affect their research [9, 27]. Another potential possibility is the policies regarding some competing interests are regularly obscure, particularly non-financial competing interests [16, 28,29,30]. We reviewed the “instruction to authors” on journals’ websites and found that the instructions of competing interests are limited, with most of them only involving a few types, such as a grant, consulting fee and employment. We have summarized a checklist of competing interests based on our results to assist researchers in identifying potential competing interests. Journals should also consider to include this checklist in their policies or instructions (Table 3).

Table 3 A potential checklist of competing interest disclosure
Full size table

Several methodological studies have been conducted to assess the reporting of competing interest disclosure, and there is substantial variability in the reporting rates among different study designs and specialties (17.0 to 71.2%) [16,17,18,19, 21, 22, 31,32,33,34]. Generally, authors of randomized controlled trials and cohort studies more likely to provide detailed information rather than those in systematic reviews. Our finding on the rate of competing interest disclosure is consistent with other studies assessing SRs [16,17,18] and no significant difference was found between trials in the drug and surgical field.

A competing interest in biomedical research has often been associated with a positive conclusion [19, 20, 35,36,37]; however, we did not have a similar finding in our study. Considering Cochrane’ policy regarding funding and competing interest is stricter than other journals [27], we also conducted sensitivity analyses that only included non-CSRs and the result was the same. Notably, our findings should be cautiously interpreted for the low percentage of SRs sponsored by industries in comparison to other published studies [19, 35].

We used a systematic method for data screening and extraction and provided a potential checklist of competing interests to readers and editors based on the results. However, our study also has some limitations. First, the restriction of our search to one database and year could compromise the generalizability of findings. Second, these disclosures have all been reported by authors, and we have limited methods to confirm or verify them. Thirdly, we only assessed SRs that included randomized controlled trials which could have underestimated the rate of competing interest disclosures.

Conclusions

In summary, 94% of SRs assessing surgical interventions and devices stated their competing interest disclosures but without detailed information. The identification and statement of competing interests, particularly non-financial interests, remains challenging for researchers, reviewers, and editors. The International Committee of Medical Journal Editors (ICMJE) and other communities need to continue to develop more efficient and effective methods/tools for identifying, quantifying, and minimizing potential competing interests in systematic reviews.

Availability of data and materials

The dataset supporting the conclusions of this article are included within the article and its additional files.

Abbreviations

SR:

Systematic review

ICMJE:

International Committee of Medical Journal Editors

CSRs:

Cochrane systematic review

References

  1. Schunemann HJ, Al-Ansary LA, Forland F, Kersten S, Komulainen J, Kopp IB, et al. Guidelines international network: principles for disclosure of interests and management of conflicts in guidelines. Ann Intern Med. 2015;163:548e53.

    Article  Google Scholar 

  2. Field MJ, Lo B. Conflict of interest in medical research, education, and practice. Washington, DC: National Academies Press; 2009.

    Google Scholar 

  3. DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA. 2001;286(1):89–91.

    Article  CAS  Google Scholar 

  4. Fontanarosa PB, Flanagin A, DeAngelis CD. Reporting conflicts of interest, financial aspects of research, and role of sponsors in funded studies. JAMA. 2005;294(1):110–1.

    Article  CAS  Google Scholar 

  5. Jureidini JN, McHenry LB. Conflicted medical journals and the failure of trust. Account Res. 2011;18(1):45–54.

    Article  Google Scholar 

  6. Eliades T, Turpin DL. Conflict of interest: always report it, and if in doubt, ask. Am J Orthod Dentofac Orthop. 2008;134(3):327–8.

    Article  Google Scholar 

  7. Cooper RJ, Gupta M, Wilkes MS, Hoffman JR. Conflict of interest disclosure policies and practices in peer-reviewed biomedical journals. J Gen Intern Med. 2006;21(12):1248–52.

    Article  Google Scholar 

  8. Lipton S, Boyd EA, Bero LA. Conflicts of interest in academic research: policies, processes, and attitudes. Account Res. 2004;11(2):119–39.

    Article  Google Scholar 

  9. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. BMJ. 1991;302(6772):338–41.

    Article  Google Scholar 

  10. Drazen JM, Van der Weyden MB, Sahni P, et al. Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med. 2009;122(1305):12–4.

    Google Scholar 

  11. Forsyth SR, Odierna DH, Krauth D, Bero LA. Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles. Syst Rev. 2014;3:122.

    Article  Google Scholar 

  12. Hakoum MB, Anouti S, Al-Gibbawi M, et al. Reporting of financial and non-financial conflicts of interest by authors of systematic reviews: a methodological survey. BMJ Open. 2016;6(8):e011997.

    Article  Google Scholar 

  13. Bou-Karroum L, Hakoum MB, Hammoud MZ, et al. Reporting of financial and non-financial conflicts of interest in systematic reviews on health policy and systems research: a cross sectional survey. Int J Health Policy Manag. 2018;7(8):711–7.

    Article  Google Scholar 

  14. Klanica K. Conflicts of interest in medical research: how much conflict should exceed legal boundaries? Ann Intern Med. 2006;144(3):225.

    Article  Google Scholar 

  15. Kaestner V, Prasad V. Financial conflicts of interest among editorialists in high-impact journals. Blood Cancer J. 2017;7(9):e611.

    Article  CAS  Google Scholar 

  16. Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD. Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ. 2012;345:e5155.

    Article  Google Scholar 

  17. Patel SV, Yu D, Elsolh B, Goldacre BM, Nash GM. Assessment of conflicts of interest in robotic surgical studies: validating Author's declarations with the open payments database. Ann Surg. 2018;268(1):86–92.

    Article  Google Scholar 

  18. Roseman M, Milette K, Bero LA, et al. Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments. JAMA. 2011;305(10):1008–17.

    Article  CAS  Google Scholar 

  19. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65.

    Article  Google Scholar 

  20. Okike K, Kocher MS, Mehlman CT, Bhandari M. Conflict of interest in orthopaedic research. An association between findings and funding in scientific presentations. J Bone Joint Surg Am. 2007;89(3):608–13.

    Article  Google Scholar 

  21. Agha RA, Barai I, Rajmohan S, et al. Support for reporting guidelines in surgical journals needs improvement: a systematic review. Int J Surg. 2017;45:14–7.

    Article  Google Scholar 

  22. Probst P, Huttner FJ, Klaiber U, Diener MK, Buchler MW, Knebel P. Thirty years of disclosure of conflict of interest in surgery journals. Surgery. 2015;157(4):627–33.

    Article  Google Scholar 

  23. Adie S, Ma D, Harris IA, Naylor JM, Craig JC. Quality of conduct and reporting of meta-analyses of surgical interventions. Ann Surg. 2015;261(4):685–94.

    Article  Google Scholar 

  24. Cochrane Handbook for Systematic Reviews of Interventions (version 6, 2019). https://training.cochrane.org/handbook/current. Accessed 15 Aug 2019.

  25. Yu J, Li X, Li Y, Sun X. Quality of reporting in surgical randomized clinical trials. Br J Surg. 2017;104(3):296–303.

    Article  CAS  Google Scholar 

  26. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325(7358):249.

    Article  Google Scholar 

  27. Cochrane Community. Disclosure of Potential Conflicts of Interest form: Instructions https://community.cochrane.org/help/tools-and-software/archie/resources-archie/disclosure-potential-conflicts-interest-form-instructions. Accessed 15 Jan 2020.

  28. Grundy Q, Mayes C, Holloway K, Mazzarello S, Thombs BD, Bero L. Conflict of interest as ethical shorthand: understanding the range and nature of “non-financial conflict of interest” in biomedicine. J Clin Epidemiol. 2019;120:1–7.

    Article  Google Scholar 

  29. Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Authors of clinical trials reported individual and financial conflicts of interest more frequently than institutional and nonfinancial ones: a methodological survey. J Clin Epidemiol. 2017;87:78–86.

    Article  Google Scholar 

  30. Shawwa K, Kallas R, Koujanian S, et al. Requirements of clinical journals for authors? Disclosure of financial and non-financial conflicts of interest: a cross sectional study. PLoS One. 2016;11(3):e0152301.

    Article  Google Scholar 

  31. Lerner TG, Miranda Mda C, Lera AT, et al. The prevalence and influence of self-reported conflicts of interest by editorial authors of phase III cancer trials. Ann Intern Med. 2012;156(11):809–16.

    Article  Google Scholar 

  32. Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance. BMJ Open. 2017;7(10):e015997.

    Article  Google Scholar 

  33. Cherla DV, Olavarria OA, Bernardi K, et al. Investigation of financial conflict of interest among published ventral hernia research. J Am Coll Surg. 2018;226(3):230–4.

    Article  Google Scholar 

  34. Bridoux V, Moutel G, Schwarz L, Michot F, Herve C, Tuech JJ. Disclosure of funding sources and conflicts of interest in phase III surgical trials: survey of ten general surgery journals. World J Surg. 2014;38(10):2487–93.

    Article  Google Scholar 

  35. Lundh A, Barbateskovic M, Hrobjartsson A, Gotzsche PC. Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study. PLoS Med. 2010;7(10):e1000354.

    Article  Google Scholar 

  36. Cherla DV, Viso CP, Olavarria OA, et al. The impact of financial conflict of interest on surgical research: an observational study of published manuscripts. World J Surg. 2018;42(9):2757–62.

    Article  Google Scholar 

  37. Fabbri A, Gregoraci G, Tedesco D, et al. Conflict of interest between professional medical societies and industry: a cross-sectional study of Italian medical societies’ websites. BMJ Open. 2016;6(6):e011124.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our deep appreciation for Dr. Mudathir (Maimonides Medical Center, Southeast University) for his linguistic assistance during the preparation of this manuscript.

Funding

The first author (JY) was supported by a grant from the “National Natural Science Foundation of China” (project No: 71704120), “National Science and Technology Major Projects” (project No: 2018ZX09201009–005-003), and International visiting program for Excellent Young Scholars of SCU. The funders had no involvement in study design, collection, analysis, or interpretation of data, writing the manuscript, and the decision to submit the manuscript for publication.

Author information

Authors and Affiliations

  1. Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, No. 37 Guo Xue Xiang, Chengdu, 610041, Sichuan, China

    Jiajie Yu & Youping Li

  2. Nuffield Department of Surgical Sciences, University of Oxford, Oxford, OX3 9DU, UK

    Jiajie Yu & Allison Hirst

  3. School of Preclinical and Forensic Medicine, Sichuan University, Chengdu, 610041, China

    Guanyue Su

  4. School of Medicine, PanZhiHua University, Panzhihua, 617000, China

    Zhengyue Yang & You Zhang

Authors
  1. Jiajie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanyue Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Allison Hirst
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengyue Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. You Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Youping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JY and YL conceived and designed this study, JY searched the literature, extracted data, synthesized data, and developed the first draft of the manuscript; GS, ZY, and YZ carefully checked studies and extracted data; HA provided critical methodological guidance. All authors critically revised the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Youping Li.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

All authors declare no competing interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Appendix Search strategy.

Additional file 2.

A full lists of included SRs.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Su, G., Hirst, A. et al. Identifying competing interest disclosures in systematic reviews of surgical interventions and devices: a cross-sectional survey. BMC Med Res Methodol 20, 260 (2020). https://doi.org/10.1186/s12874-020-01144-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-020-01144-2

Keywords

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us