Keywords
clinical trials, blockchain, data, bitcoin
At the request of the authors Greg Irving and John Holden, the article titled “How blockchain-timestamped protocols could improve the trustworthiness of medical science” has ... been retracted from F1000Research. The authors have taken this decision after considering the methodological concerns raised by a peer reviewer during the post-publication open peer review process. As the methodology has been deemed to be unreliable, the article is now retracted. This applies to all three versions of the article: Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 1; referees: 2 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.1) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 2; referees: 3 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.2) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 3; referees: 3 approved, 1 not approved]. F1000Research 2017, 5:222 (doi: 10.12688/f1000research.8114.3)
This article is included in the All trials matter collection.
At the request of the authors Greg Irving and John Holden, the article titled “How blockchain-timestamped protocols could improve the trustworthiness of medical science” has been retracted from F1000Research. The authors have taken this decision after considering the methodological concerns raised by a peer reviewer during the post-publication open peer review process. As the methodology has been deemed to be unreliable, the article is now retracted. This applies to all three versions of the article: Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 1; referees: 2 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.1) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 2; referees: 3 approved]. F1000Research 2016, 5:222 (doi: 10.12688/f1000research.8114.2) Irving G and Holden J. How blockchain-timestamped protocols could improve the trustworthiness of medical science [version 3; referees: 3 approved, 1 not approved]. F1000Research 2017, 5:222 (doi: 10.12688/f1000research.8114.3)
clinical trials, blockchain, data, bitcoin
The method we tested here was first proposed by Carlisle in the grey literature. Clear reference to the previously described method (Reference 6) has been added throughout the revised article.
To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.
Concerns have been raised about the overlap between Version 1 of this article and a previously published blog by Carlisle, who proposed the method 2 years earlier [Carlisle, Benjamin Gregory. “Proof of prespecified endpoints in medical research with the bitcoin blockchain”, 25 August 2014], and that the correction (Version 2) published soon after the original was not sufficient to rectify the overlap.
The case has since been discussed in a Committee of Publication Ethics (COPE) Forum, and COPE advised that the correction was sufficient to correct the scientific literature.
The case has been referred to the University of Cambridge for consideration.
Trust in scientific research is diminished by evidence that data are being manipulated1. Outcome switching, data dredging and selective publication are some of the problems that undermine the integrity of published research. The declaration of Helsinki states that every clinical trial must be registered in a publicly accessible database before recruitment of the first subject2. Yet despite the creation of numerous trial registries problems such as differences between pre-specified and reported outcomes persist3–5. If readers doubt the trustworthiness of scientific research then it is largely valueless to them and those they influence. Here we confirm the use of blockchain as a low cost, independently verifiable method that could be widely and readily used to audit and confirm the reliability of scientific studies.
A blockchain is a distributed, tamper proof public ledger of timestamped transactions. It provides a method for establishing the existence of a transaction at a particular time that can be independently verified by any interested party. When someone wishes to add to it, participants in the network – all of whom have copies of the existing blockchain – run algorithms to evaluate and verify the proposed action. Once the majority of ‘nodes’ confirm that a transaction is valid i.e. matches the blockchain history then the new transaction will be approved and added to the chain. Once a block of data is recorded on a blockchain ledger it is extremely difficult to change or remove it as doing so would require changing the record on many thousands computers worldwide. This prevents tampering or future revision of a submitted timestamped record. Such distributive version control has been increasingly used in fields such as software development, engineering and genetics. A method for using blockchain to provide proof of pre-specified endpoints in clinical trial protocols was first suggested by Carlisle in 20146. We wished to empirically test such an approach using a clinical trial protocol where outcome switching has previously been reported.
In this study we used publically available documentation from a recently reported randomized control trial7,8. A copy of the clinicaltrials.gov study protocol was prepared based on it’s pre-specified endpoints and planned analyses which was saved as an unformatted text file7. Following a method similar to that described by Carlisle the document’s SHA256 digest for the text was then calculated by entering text from the trial protocol into an SHA256 calculator (Xorbin©)6. This was then converted into a bitcoin private key and corresponding public key using a bitcoin wallet. To do this a new account was created in Strongcoin©9 and the SHA256 digest used as the account password (private key)6. From this Strongcoin© automatically generated a corresponding Advanced Encryption Standard 256 bit public key6. An arbitrary amount of bitcoin was then sent to a corresponding bitcoin address. To verify the existence of the document a second researcher was sent the originally prepared unformatted document. An SHA256 digest was created as previously described and a corresponding private key, public key and bitcoin address generated6. The exact replication of the bitcoin address (1AHjCz2oEUTH8js4S8vViC8NKph4zCACXH) was then used to prove the documents existence in the blockchain using blockchain.info©10. The protocol document was then edited to reflect any changes to pre-specified outcomes as reported by the COMPare group3. This was used to create a further SHA256 digest and corresponding public, private key and bitcoin address3.
Incorporating a transaction from the bitcoin wallet into the blockchain using a private key generated from the SHA256 digest of the trial protocol timestamped a record of the study protocol. The transaction took under five minutes to complete. The process cost was free as the nominal bitcoin transaction could be retrieved. Researchers were able to search for the transaction on the blockchain, confirm the date when the transaction occurred and verify the authenticity of the original protocol by generating identical public and private keys. Any changes made to the original document generated different public and private keys indicating that protocol had been altered. This included assessment of an edited protocol reflecting pre-specified outcomes not reported or non-pre-specified outcomes reported in the final paper.
Fraud in scientific methods erodes confidence in medicine as a whole which is essential to performing its function1. This study demonstrates that the method described by Carlisle provides an immutable record of the existence, integrity and ownership of a specific trial protocol6. It is a simple and cheap way of allowing a third party to audit and externally validate outcomes and analyses specified a-priori with the findings reported a-posteriori. It prevents researchers from changing study endpoints or analyses after seeing their study results without reporting such changes6. Transaction codes could be recorded in scientific papers, reference databases or trial registries to facilitate external verification. As discussed in the CONSORT guidelines, switching of outcomes in trials is sometimes necessary for perfectly legitimate reasons but this should be disclosed in the final report11. The use of blockchain timestamped protocols could facilitate trust in the reporting of this process by providing evidence of precisely when protocol changes took place. At the same time, fraudulent attempts to prepare multiple study protocols in advance would be technically possible but would also leave behind a publically available trail of evidence that could not be destroyed6.
The blockchain offers a number of advantages over the current approaches used trial registries or publishing protocols. Firstly, the blockchain would not be confined to the validation of clinical trials. The approach could be used for a whole range of observational and experimental studies where registries do not currently exist. Secondly, the blockchain provides a real-time timestamped record of a protocol. Such precision is important given persistent problems with protocol registration after trial initiation12. Thirdly, with over 30,000 trials currently published annually and rising, manual outcome verification is simply not possible13.
Blockchain-timestamped protocols can allow the exact wording and existence of a protocol at a given point in time to be verified. They have the potential to support automated, extremely robust verification of pre-specified with reported outcomes. This evidence should increase trust in medical science by diminishing suspicion in reported data and the conclusions that are drawn.
F1000Research: Dataset 1. Unformatted text file, 10.5256/f1000research.8114.d11459614
GI and JH carried out the research. GI prepared the first draft of the manuscript. All authors were involved in the revision of the draft manuscript and have agreed to the final content.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Competing Interests: CL is a cofounder and company advisor at Lumos Medica Srl, which provides software solutions for clinical trials.
Competing Interests: No competing interests were disclosed.
Competing Interests: No competing interests were disclosed.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||||
---|---|---|---|---|
1 | 2 | 3 | 4 | |
Version 3 (revision) 30 Mar 17 |
read | |||
Version 2 (revision) 25 May 16 |
read | |||
Version 1 26 Feb 16 |
read | read |
Click here to access the data.
Spreadsheet data files may not format correctly if your computer is using different default delimiters (symbols used to separate values into separate cells) - a spreadsheet created in one region is sometimes misinterpreted by computers in other regions. You can change the regional settings on your computer so that the spreadsheet can be interpreted correctly.
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
I encourage Irving & Holden to consult with the "second researcher" that replicated the address generation and publicly provide the cryptographic chain of operations that verifies their timestamp. Absent this validation, the study is meritless.
I encourage Irving & Holden to consult with the "second researcher" that replicated the address generation and publicly provide the cryptographic chain of operations that verifies their timestamp. Absent this validation, the study is meritless.