Why Cochrane should prioritise sharing data

Packer1 says that the one who submits a research for public good should be ready to receive a request for data sharing for examination and re-analysis and that tax payers assume that a national agency is checking such data and analysis. Here we discuss Cochrane’s practice on data sharing.

Open science, as endorsed by the G7,2 includes sharing data, computer code, and materials. It is essential for reproducibility, collaboration, and innovation. We support the work of Cochrane, but are concerned that Cochrane is not sharing all its reviews’ data. These data should be fully accessible for reuse by third parties.

Cochrane, a non-profit private company3 and registered charity, produces and maintains systematic reviews in health and social care. Its work is undertaken by a global network of thousands of people,4 and its support largely comes from public funding.5 Most people producing Cochrane reviews are volunteers not specifically funded for this work,67 and Cochrane encourages “crowdsourcing” of work.8910

Cochrane editorial bases help volunteers obtain study reports and manually extract the wealth of data needed to generate systematic reviews.111213 Cochrane teams use RevMan software14 to produce files in standard format (XML), storing information on the studies, their methods, and results for publication in the Cochrane Library.

Benefits of sharing extracted data from trials and systematic reviews are well known, as are the costs of not sharing.13151617 Sharing maximises transparency, reliability of data extraction, and syntheses. It improves access to data—saving time and money—and opens new avenues of inquiry.18 Sharing is associated with increased citations,19 more publications,20 and reuse for new purposes.16

Structured data from Cochrane should be fully accessible for download, reuse, and review (box 1). Currently, they are not. Although Cochrane supports transparency initiatives such as AllTrials21 and is explicit about this in its policy,22 it has no similar clear principles on opening full access to the data in Cochrane reviews. Cochrane does provide access to results data from reviews but, crucially, these cannot be readily reused, and the available information is an incomplete set of the data generating these reviews, comes in a technically problematic format, and can only be viewed by those with access to the full content of the Cochrane Library.232425

Small amounts of Cochrane data have been released with bespoke arrangements for specific individuals. This sharing is welcome, but organisational culture, policy, and process regarding data release are lacking; there is no appeals process. For example, OpenTrials aggregates all accessible documents on all trials in an open database and makes it free for public reuse.2627 Thus far, OpenTrials has been unable to persuade Cochrane to share data for reuse. The Trip Database28 is a searchable library of evidence that asked if it could re-present structured data from Cochrane and also encountered barriers to access.29 Open sharing could foster collaborative ecosystems of digital innovation going beyond academic publications, with outputs that might include live, interactive presentations of summaries and results of trials produced by teams around the world, interactive decision support tools, and many more.

Cochrane’s non-release of data is unlikely to reflect the preferences of funders, publishers, the thousands of Cochrane volunteers, participants in trials, or patients. When asked, 83% of the members of the Cochrane Individual Participant Data Meta-analysis Methods Group supported sharing systematic review data through a central repository (recognising that these data might require some form of moderated access).30 Many funders now require that data arising from their grants are shared.31323334 Cochrane volunteer authors give tacit consent for use of their work in reviews but may not be aware of the restrictions placed on access to the data they worked so hard to prepare.25 This is morally and ethically questionable, potentially eroding public trust.1635

This issue of open science is now pressing, after recent moves by Cochrane to create more information and become a hub for systematic review data. This has the potential to improve evidence and patient care, but although the Cochrane Linked Data Project aims to share reusable data in some form,3637 there is not yet any information on how or when this will happen.3839 Furthermore, Cochrane is working towards “living” systematic reviews, with updates from data in real time.40 This is important work, but progress is slow. Opening up this work with shared data resources and in collaboration with the open source software community—where all can contribute—would accelerate progress and best reflect the culture of collaboration in science.

Open data offers a transformative, collaborative future for the systematic review community. Cochrane has enabled a vast workforce to painstakingly extract information for great benefit. It could act as a hub, harmonising data collected across groups and sharing these widely, reflecting the collective funding and volunteer workforce that produces them. This could include converting the morass of free text trial reports into machine readable curated data, in archived, citable, accessible, interoperable and reusable formats, as set out in the FAIR principles.4142 Cochrane could show leadership in supporting innovation and open science for clinical trials with full credit to all data extractors before43 and after review publication44 and, in this way, harness the greatest broadest impact. This reflects the exciting current move towards better use of data to produce digital tools of direct value to clinicians, rather than academic publications alone.

We have raised these issues with Cochrane and understand that the organisation is considering whether to start reviewing its approach to sharing data (D Tovey, personal communication, 2017). We hope that our setting out the benefits of open data is a helpful contribution to open that discussion.

We appreciate that Cochrane must focus on making itself sustainable and that open data sharing may be commercially sensitive.45 But making Cochrane a champion for openness, transparency, and sharing can only be beneficial for the organisation’s reputation—and finances. We encourage Cochrane leadership to create a policy that allows open data sharing and to make explicit any concerns they have on open data sharing so that these can be resolved.