Reporting quality of randomized controlled trials in patients with HIV on antiretroviral therapy: a systematic review

Background To allow for correct evaluation of clinical trial results, readers require comprehensive, clear, and highly transparent information on the methodology used and the results obtained. This study aimed to evaluate the quality of reporting in articles on randomized controlled trials (RCTs) of antiretroviral therapy (ART) in the field of HIV/AIDS. Methods We searched for original articles on RCTs of ART developed in the field of HIV/AIDS in PubMed database by 5 April 2016. Searched articles were divided into three groups based on the revision year in which the Consolidated Standards of Reporting Trials (CONSORT) guidelines were published: Period 1 (1996–2001); Period 2 (2002–2010); and Period 3 (2011–2016). We evaluated the articles using the reporting rates of the 37 items in the CONSORT 2010 checklist, five items in the protocol deviation, and the three items in the ethics. Results Fifty-two articles were extracted and included in this study. Many of the reporting rates calculated using the CONSORT 2010 checklist showed a significantly increasing trend over the successive periods (65% in Period 1, 67% in Period 2, 79% in Period 3; p < 0.0001). The items with reporting rates < 50% were “the presence or absence of a protocol change and the reason for such a change,” “randomization and blinding,” and “where the full trial protocol can be accessed.” Reporting rates of deviations were as low as < 30%, while the reporting rates for patient compliance were the highest (>80% in Period 3) among the five items. The reporting rates for obtaining informed consent and approval by the ethics committee or institutional review board were high (>88%), regardless of the time period assessed. Conclusion In terms of representative RCT articles in the field of HIV/AIDS, the reporting rate of the items defined by CONSORT was approximately 70%, improving over the successive CONSORT statement revision periods. Electronic supplementary material The online version of this article (doi:10.1186/s13063-017-2360-2) contains supplementary material, which is available to authorized users.


Background
To allow for correct evaluation of clinical trial results, readers require comprehensive, clear, and highly transparent information on the methodology used and the results obtained. The Consolidated Standards of Reporting Trials (CONSORT) statement [1][2][3] aims to improve the scientific quality of reports of randomized controlled trials (RCT) and was developed in collaboration with clinical trial methodologists, guideline developers, knowledge translation specialists, and journal editors. This statement evaluates the quality of the reporting of clinical trial results, but not of the clinical trial itself. The first version of the CONSORT statement was developed in 1996 and was thereafter revised twice, in 2001 and 2010 [1][2][3]. The current CONSORT 2010 statement is composed of a checklist of 25 topics and includes 37 items and a flow diagram. It has been recommended that the statement be used in conjunction with the CONSORT 2010 Explanation and Elaboration [4], which was developed simultaneously with the statement [4].
Systematic reviews evaluating the reporting quality of clinical trials in the field of HIV/AIDS have included various articles. Some reports assessed the quality of reporting of adverse events using the CONSORT 2001 extension for adverse events reporting [5,6]. Another report targeted studies in which HIV interventions were performed by a pharmacist and their validity was assessed using the CONSORT 2010 statement and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement criteria [7]. A further report targeted non-inferiority RCTs [8] and yet another report evaluated the performance and reporting quality of RCTs aimed at prevention and treatment of HIV/AIDS in Africa [9]. However, to our knowledge, no study has comprehensively reviewed and evaluated the quality of reports on RCTs of antiretroviral therapy (ART), the standard therapy for HIV/AIDS using the CONSORT 2010 statement.
The CONSORT statement does not include a detailed protocol deviation item checklist. On the other hand, the report by Sweetman and Doig [10] classified protocol deviations into five items, i.e. enrollment, randomization, study intervention, patient compliance, and data collection. These items were defined as indices for the evaluation of clinical trial quality, with the purpose of examining relationships between protocol deviation reporting and the characteristics of clinical trials. Based on the results of their study, they proposed addition of the five protocol deviation items to the CONSORT statement in order to clarify the requirements for protocol deviation reporting. In the field of HIV/ AIDS, compliance with medication is an important index, because it is considered important for sustained HIV suppression and reduction of the risk of drug resistance [11]. Several compliance-related studies have been reported, including a study using a device that informs HIV patients with impaired memory when to take their medication [12] and another study on compliance rate and outcomes [13][14][15][16][17].
According to the 2016 fact sheet of the Joint United Nations Programme on HIV/AIDS (UNAIDS), there were 36.7 million HIV-infected individuals alive as of 2015 and 70% of those people resided in sub-Saharan Africa (19 million people in Eastern and Southern Africa and 6.5 million people in Western and Central Africa) [18]. The majority (33 of 48 countries) of sub-Saharan Africa countries are poorly developed countries [19]; this area includes many regions in which the literacy rate is low [20]. The United Nations defines "least developed country" based on the following three criteria: per capita income; human assets; and economic vulnerability [21]. It has been shown that informed consent processes require more effort in developing countries than in advanced countries [22] and it has been recommended that a local translator write the information pamphlet as well as the consent form and that informed consent should be acquired in the local language [23]. However, there has been no detailed review of the reporting on obtaining informed consent in RCTs in the field of HIV/AIDS. Thus, this study aimed to evaluate the reporting quality of articles on RCTs of ART developed in the field of HIV/ AIDS, by adding evaluation items related to protocol deviations and ethical aspects to the CONSORT 2010 checklist items.

Search strategy
Original articles on ART RCTs that had been reported between the start date of the PubMed database and 5 April 2016, were extracted by means of a PubMed search (Appendix 1 lists the search terms used). Additionally, when an extracted article was not the original article, the original article was manually identified from the reference list of the extracted article.

Article selection
Article selection criteria consisted of the following. Original articles on RCTs of ART developed in the field of HIV/ AIDS were included. Brief reports, rapid communications, short communications, conference abstracts, study protocols, and reports of non-RCTs, such as sub-studies and pilot studies, were excluded. One researcher reviewed all abstracts of the extracted articles to exclude ineligible abstracts. Thereafter, this researcher reviewed full texts to exclude ineligible articles. Another researcher independently reviewed the ineligibility of the excluded articles.

Data evaluation
One researcher evaluated the full texts and two other researchers independently assessed these evaluations. The four researchers resolved differences in evaluations by means of discussion.

Setting the evaluation period
The first CONSORT statement was published on August 1996, which was later revised in April 2001 and March 2010. Therefore, we divided the articles into three groups based on the year in which they were published: Period 1 (1996Period 1 ( -2001; Period 2 (2002-2010); and Period 3 (2011-2016). Evaluation items were compared the different versions of the checklist (Table 1). In the first revision, item numbers were changed, combined, or split. In the second revision, minor changes were made to item numbers, while major changes involved subdivision of existing topics, which increased the specificity and understanding of the evaluation items. Additionally, a new section, "Other information," was added, so that the number of sections increased from five to six.

Characteristics of subject articles/clinical trials
The characteristics of the extracted clinical trials, including the year of publications, number of study arms, blinding, region in which the trial was conducted, number of participant centers, number of enrolled patients, sponsor, and eligible age (individuals aged 18 years or older were classified as adults, whereas those 16 years or Evaluation of the quality of reporting in the selected articles The CONSORT 2010 checklist comprised six sections, 25 topics, and 37 items. If the description of the items was complete, we allocated a score of 1 point to the item, so the full mark was 37 points for an article. If there was no description regarding the item, we scored 0 points. When the reporting of an item was incomplete, it was allocated 0.5 points. Other items using specific scoring methods are defined in the footnotes (points 1-3). The items scores were totaled for each article. This sum was divided by the number of items which was determined by subtracting the number of irrelevant items excluded from the total number of items. To calculate the reporting rate, this value was expressed as a percentage. In the separate examinations conducted for the three periods, the reporting rates of the individual items were calculated by dividing the total score of the relevant item for articles in which the item was reported by the number of articles extracted in that period, and expressed as a percentage.
Point 1: Reporting rate of 37 items in the CONSORT 2010 checklist (Table 1) [3] Item 5 in the CONSORT 2010 checklist (#5). "The interventions for each group are given with sufficient detail to allow replication, including how Funding Sources of funding and other support (such as supply of drugs), role of funders --2 5 *Items were enhanced for specificity of appraisal † Items were simplified and clarified regarding the wording -indicates the absence of an item For the 2001 and 2010 CONSORT checklist, the item number in the article is shown. As for the 1996 checklist, there was no item number listed in the article, so item numbers were assigned according to the order shown in the original article and when they were actually administered. " We allocated a score of: 0.25 points for the descriptions of the daily dosage and interval; 0.25 points for the description of nutrition, such as fasting, preprandial, and postprandial administration; 0.5 points for a detailed description of scheduled visits. #11a and #11b. "Blinding. " Cases of non-blinded trials were excluded from the evaluation. #14b. "Why the trial ended or was stopped. " Cases in which trial progress could be judged as discontinued/completed/currently enrolled were allocated a score of 1 point. #20. "Limitations. " Cases in which the word "limitation" was clearly written were allocated a score of 1 point.
The reporting rate of each section (Title and Abstract, Introduction, Methods, Results, Discussion, and Other information) was calculated by dividing the sum of the percentage of reported items in each section by the number of reported items in each section.
Point 2: Reporting rate for protocol deviations (Fig. 3a) Sweetman and Doig recommended changes to the CONSORT statement that make reporting requirements for protocol deviations more explicit [10]. They analyzed the relationships between reporting of five individual items in protocol deviations (enrollment, randomization, study intervention, patient compliance, and data collection) and the trial characteristics of 80 articles published in four prominent medical journals [10]. The reporting rates for patient compliance and randomization were the highest and lowest among these items, respectively, although protocol deviations had not been reported frequently. They mentioned that protocol deviations might result in patient harm and introduce errors into clinical trial results. Therefore, we also assessed these items in this study. We allocated 1 point in cases where protocol deviations were clearly reported and 0.5 point in cases where the reporting was incomplete. We then compared our findings with those reported earlier [10].
Point 3: Reporting of ethical aspects The presence or absence of reporting on obtaining informed consent, acquisition of approval by the ethics committee or institutional review board (IRB) and statements of conflict of interest (COI) were extracted. In terms of COI, cases where "financial disclosures" were stated were allocated 1 point.

Statistical analysis
The reporting rate per period, expressed as the mean and standard deviation (SD), were compared using a one-way analysis of variance. Additionally, changes in reporting rates of individual evaluation items over the three time periods were examined using the Jonckheere-Terpstra trend test [24]. Since we had 45 items and six categories, we carried out 51 trend tests. A significant level of p = 0.05 was divided by the number of tests and adjusted to 0.001 on using the Bonferroni correction for multiple comparisons. R version 3.2.4-revised was used for statistical analyses.

Article identified
Seventy-eight articles were identified using the PubMed search. Nine original articles were additionally extracted by manual searches of the reference lists of non-original extracted articles. Abstract review excluded 22 articles from among the 87 non-redundant articles. Of the remaining 65 articles, 13 were excluded after full text review and the remaining 52 articles met the eligibility requirements ( Fig. 1) PRISMA checklist is provided in Additional file 1. No articles published before 1996 were found using our search criteria. Table 2 shows the journals in which the selected articles were published.

Characteristics of selected articles/clinical trials
Most articles identified in our search were published during Period 2 (52%, 27/52), followed by Period 3 (38%, 20/52), and Period 1 (10%, 5/52) Raw data is provided in Additional file 2. Table 3 shows the characteristics of the clinical trials stratified by period. Trials with two arms accounted for approximately 90% of trials, whereas double-blind trials accounted for 40%. Trials conducted in two or more regions accounted for 85% (44/52) of those included. The regions in which the trials were conducted were divided as follows: North America (39/44); Europe (25/44); Oceania (19/44); South America (16/44); Asia (11/44); and Africa (6/44). The median number of study sites was 79.5 (range = 19-234), whereas the median number of enrolled patients was 656.5 (range = 92-2463). The age of 18 years or older as an eligibility criterion was used in approximately 70% (36/52) of trials, whereas trials that allow enrolling younger patients accounted for 20% of the trials. In 10% of the articles, the age of participants was not reported. No trial involved children aged 11 years or younger as participants. Corporations acted as sponsors in 85% (44/52) of trials on average and this percentage increased over the successive time periods (Period 1, Period 2, Period 3 = 60%, 81%, and 95%, respectively). These results show that most of the ART RCTs reported in the selected articles were multicenter international collaborative studies; however, the RCTs conducted in Africa were infrequent.

Reporting of protocol deviations
Evaluation of individual items revealed that the rates of reporting on deviations in the study interventions, enrollment, and randomization were < 25%, regardless of the period, and the reporting rate on randomization was the lowest. Additionally, the reporting rates for data collection and patient compliance improved over the successive periods and the rate of patient compliance exceeded 80% in Period 3, revealing a substantial increase (p < 0.005, Fig. 3a). The mean 2014/2015 impact factors (IFs) of the articles assessed in this study and the study by Sweetman and Doig were 24.98 and 38.44, respectively. The overall reporting rate of the five items related to protocol deviation was 28.7% in our study and 29.8% in their study; these rates were similar and was < 30% in both studies (Fig. 3b).

Reporting rates of ethics
The rate of reporting on obtaining informed consent (Period 1, Period 2, Period 3 = 100%, 89%, and 100%, respectively) and that on approval by an ethics committee or IRB (Period 1, Period 2, Period 3 = 100%, 89%, and 90%, respectively) remained > 88% throughout the periods (Fig. 4). The rate of reporting on COIs increased over time; it was > 50% in Period 2 and was 84.2% in Period 3 (Fig. 4). In Period 1, financial disclosures rather than COIs were reported in 20% of the articles. Thus, reporting rates for obtaining informed consent and ethical approval were high throughout the study periods, even though the CONSORT checklist did not include either item.

Discussion
In this study, the quality of reports on RCTs of anti-HIV therapy was evaluated comprehensively and systematically, based on the CONSORT 2010 checklist, protocol deviations, and ethical considerations, which has not been previously reported.
The quality of reported articles evaluated using the CON-SORT 2010 checklist improved over time (65% in Period 1, 67% in Period 2, 79% in Period 3; p < 0.0001), although seven items had low reporting rates of < 50%, regardless of the study period ("Methods": 6/17 items; "Other information": 1/3 items). The six items in the "Methods" section were randomization or blinding (#9, #10, #11b), explanation of any interim analyses and stopping guidelines (#7b), and criteria changes after trial commencement (#3b, #6b). The one item in the "Other information" section was where the full trial protocol can be accessed (#24). Some journals require public posting of protocols as a condition of acceptance for publication [25]. Therefore, it is possible that articles published in the future may improve the reporting rate of item #24.
"Registration" (#23) was not reported at all in Period 1, but the reporting rates in Periods 2 and 3 were moderate (52%) and high (95%), respectively. The International Committee of Medical Journal Editors (ICMJE) requires the registration of clinical trials in a public trials registry. This abrupt increase after the second revision may involve the requirement for registration in Clinicaltrials.gov, etc., since 2000. "Registration" (#23) was added in the CONSORT 2010 revision.
Several studies have reported on compliance and outcomes of studies in the field of HIV/AIDS [13][14][15][16][17] and have demonstrated the close relationship between compliance rates and outcomes. In particular, in this study, the rate after 2010 was very high, at approximately 90%. This reflects the importance of compliance in this disease field. Although both the study by Sweetman and Doig and this study showed low reporting rates (approximately 30%) in overall deviations, the rate can likely be improved by means of a data manager, as management of deviation reporting is within the scope of data management.
In terms of ethical aspects, the rates of reporting on obtaining informed consent and approval by an ethics committee or IRB were high throughout the study period, as previously reported [26]. This is likely because the included trials complied with the Declaration of Helsinki [27], adopted in 1964, and the ICH-GCP guidelines [28], completed in 1996. The rates for reporting on COIs were 40%, 52%, and 85% in Periods 1, 2, and 3, respectively, and thus  [32], were likely to have facilitated reporting of COIs. This study has several limitations. Although PubMed is one of the largest medical publication databases, we did not evaluate reports that were not indexed in PubMed. Also, we searched for keywords in the Title/Abstracts only and did not search the literature by using relevant subject headings. The second limitation regards scoring. It was simple to score 0 or 1 for about half of the items (17/37 items), but some items could not be determined dichotomously (0 or 1), so we chose intermediate score levels (0.25, 0.5, and 0.75) for some items. This step was subjective in nature. Therefore, we provided a comprehensive description of item scoring in the "Methods" section. We defined the method to evaluate items such as "important changes to methods after trial commencement (such as eligibility criteria), with reasons (#3b)" and "any changes to trial outcomes after the trial commenced, with reasons (#6b)". But these items are difficult to assess unless the clinical protocol was reviewed. The third limitation is that there is no consensus if it is important to report ethical items for HIV/AIDS RCT articles. We found that earlier reports stating that informed consent required additional effort, due to low literacy rates and differences in language in developing countries [22,23]. However, we did not find trials involving participants exclusively from sub-Saharan Africa, so we could not assess the difference of reporting rates of ethical items between studies in developing and developed countries. Articles on trials targeting multiple regions accounted for 85% of the individuals included in this study. Although it is likely that, in such cases, informed consent forms were prepared in the language of the respective region, we could not confirm whether there were any problems related to language, as the articles did not report on language issues. The final limitation is that we were unable to find out when the author guidelines of each journal requested CONSORT compliance, so we evaluated all articles consistently under the CONSORT 2010 guideline. There was a possibility of including the journals that did not require authors to follow the CONSORT guidelines.

Implications for future studies
As mentioned in the background, it is very important to check if the patients take medicine as indicated in the HIV/ AIDS RCTs. To our knowledge, this is the first study to perform quantitative evaluation of patient compliance reporting, in addition to the CONSORT checklist items. The increasing trend of adherence to the CONSORT 2010 guideline and the improved reporting rate for patient compliance both imply that the quality of HIV/AIDS RCTs is improving. Our results may assist in further improving the quality of HIV/AIDS RCTs. As for randomization, many study reports referred to the mechanism used to implement the random allocation sequence, but did not refer to the steps taken to conceal the sequence until interventions were assigned, which was required by the CONSORT 2010 statement. By reporting such information on items of low reporting rates, it is expected that the reporting quality of HIV/AIDS RCTs will improve. Additionally, our methods could be applied to RCT reporting in other disease areas, in which patient compliance is critical for evaluation of efficacy and safety of a new drug or a therapy.

Conclusion
In conclusion, the representative RCT articles in the field of HIV/AIDS examined in this study, had reporting rates for items defined by CONSORT of approximately 70%; these rates additionally improved over time. However, the reporting rate was still low for some items related to randomization or blinding and for the description of changes after the trial commenced. Additionally, protocol deviations were not comprehensively reported.