Quality of randomized clinical trials published by plastic surgeons: a long-term follow-up study

to the 1996-2003 period ( p < 0.0004); however, there was no significant difference in the 2004-2008 period. Conclusion: There was no difference in the quality of the RCTs published by plastic surgeons in the 2009-2013 period compared to the previous five-year period (2004 to 2008). However, both periods indicated higher quality compared to the 1966-2003 period. ■


INTRODUCTION
A randomized clinical trial (RCT) is a prospective study design that compares the effect of interventions on humans in one or more groups against a control group.RCTs are the best source of evidence for health interventions.Evidence-based medicine (EBM) is defined as the use of the best existing scientific evidence, provided by appropriately designed and conducted RCTs results, combined with individual clinical expertise and patient preferences and values, for deciding on individual patient care [1][2][3][4] .EBM became popular in the 1980s and impacted all fields of medicine, including Plastic Surgery 5 .The application of EBM principles can not only determine the best treatment for the patient but can also reduce the costs of healthcare systems 2 .However, particularly in the surgical areas, there are challenges to overcome, including the belief that the application of EBM could reduce the autonomy of surgeons, and that the best scientific evidence does not always exceed the best Introdução: Em dois estudos prévios, avaliou-se a qualidade dos ensaios clínicos aleatórios (ECAs) com a participação de pelo menos um cirurgião plástico, em dois períodos: 1966 a 2003 e 2004 a 2008.O objetivo é avaliar a evolução da qualidade das publicações de ECAs por cirurgiões plásticos no período subsequente de cinco anos, de 2009 a 2013.Métodos: ECAs publicados de 2009 a 2013, em língua inglesa, com a participação de pelo menos um cirurgião plástico, foram identificados por busca eletrônica e classificados quanto ao sigilo de alocação, por dois avaliadores independentes.Os estudos com sigilo de alocação adequado tiveram a qualidade avaliada por dois avaliadores, utilizando-se a Lista de Delphi e a Escala de Qualidade de Jadad.Resultados: Dos 6.997 estudos identificados, 261 foram classificados quanto ao sigilo de alocação.Destes, 43 (16,47%) tinham sigilo de alocação adequado.Segundo a avaliação pela Lista de Delphi, houve melhora, em relação a 1966-2003, nos itens "características mais importantes do prognóstico" (p < 0,001), "uso de avaliador independente" (p = 0,0029) e "medidas de variabilidade e estimativa de pontos para a variável primária" (p = 0,0057); não houve diferença em relação a Several studies have indicated that plastic surgeons recognize the need to improve the level of evidence of research in Plastic Surgery, and this recognition reflects in the continuous increase in publications of clinical trials by this specialty 2,8,[12][13][14][15][16][17] .
A previous study identified RCTs with properly described allocation concealment, published by plastic surgeons between 1966 and 2003, and evaluated their quality 18 .Subsequently, in another study, the evolution was assessed over a subsequent five-year period (2004 to 2008), and a quantitative and qualitative increase in the RCTs published by plastic surgeons were observed 8 .This study aimed to test whether there was a quantitative and qualitative improvement in the RCTs in Plastic Surgery in another five-year period (2009 to 2013) compared to previously studied periods.

OBJECTIVE
To evaluate the evolution of randomized clinical trials in Plastic Surgery with adequately described allocation concealment, published between 2009 and 2013, compared to previously studied periods (1966-2003 and 2004-2008).

METHODS
The project was approved by the Research Ethics Committee of the Federal University of São Paulo -Paulista School of Medicine (UNIFESP -EPM), under the number 842.388, CAAE 37661814.8.0000.5505.The cases were selected by convenience, consisting of all the recovered RCTs that met the study eligibility criteria, published over a five-year period (January 2009 to December 2013) Electronic searches were performed to identify the largest number of RCTs that were published by plastic surgeons in English.Specific search strategies were developed for each studied database, CCTR (Cochrane Central Register of Controlled Trials), LILACS (Latin American and Caribbean Health Science Literature), EMBASE (Excerpta Medica Database), and MEDLINE (MEDLARS-Medical Literature Retrieval System -online).
The abstracts of all retrieved articles were read by an evaluator and those that met the eligibility criteria (possible RCTs in which at least one plastic surgeon participated and published in English between 2009 and 2013) were selected for the reading of their full texts.Studies that were not conducted by plastic surgeons or without the participation of at least one plastic surgeon, and those published in a language other than English, were excluded.At this stage, whenever there was any doubt, the study was selected for reading in its entirety.
The full texts of the articles which had their abstracts selected were read by an evaluator to confirm the eligibility criteria.Subsequently, two independent evaluators classified the selected articles according to their allocation concealment 19 , and the disagreements were resolved in a consensus meeting.The RCTs published by plastic surgeons, with adequately described allocation concealment, were selected and constituted the sample of this study.
The selected RCTs were then evaluated for their quality.The evaluation was conducted independently by two evaluators, followed by a consensus meeting.Two validated instruments were used for quality assessment: the Delphi List 20 and the Jadad Scale 21 .
The Delphi List is a list of generic criteria for quality evaluation of clinical trials that should be used in combination with other instruments.It does not use scores and all items have two answer choices: "yes" or "no" (Chart 1) 20 .
The Jadad Scale is based on scores: one point is given for each "yes" answer, and a zero point for each "no" answer.Points counted for the first two items (randomization and double blinding) depend not only on how they are described, but also on the use of appropriate methods for this purpose.If the methods are described and appropriate, an additional point is given for each item.If the methods used to generate the randomization sequence or create the blinding conditions are described but inappropriate, the item will receive a zero point.Therefore, the scale encompasses scores from 0 to 5. The study will be considered of poor quality if it receives two or fewer points (Chart 2) 21 .
The results were compared to those obtained in the two previous studies that used the same method to evaluate the quality of the RCTs published by plastic surgeons in the 1966-2003 and 2004-2008 periods 8,18 .

Kappa and
McNemar tests were used to analyze the concordances and disagreements between the evaluators.The Chi-square test was applied to compare the categorical variables evaluated in the three periods (1966-2003, 2004-2008, and 2009-2013).Kruskal-Wallis analysis of variance was used to compare the Jadad scores in the three periods.The Kolmogorov-Smirnov test was applied to compare, two by two, the Jadad scores in the studied periods 22 .
Statistical analysis was performed using the BioEstat 5.3 program (Instituto Mamirauá, Pará and Amazonas, Brazil).In all tests, the level of significance used was 0.05 or 5%.

RESULTS
The electronic search identified 6,997 articles in the analyzed databases.One evaluator selected 616 articles, excluding 6,381 that were clearly not RCTs, were repeated in different databases, or did not include the participation of at least one plastic surgeon.
After reading the full texts of the 616 publications, the evaluator excluded 336 articles for the following reasons: one article was published in a language other than English (Russian); 219 did not include the participation of at least one plastic surgeon; 116 were not RCTs.Of the 280 remaining articles, 19 were repeated within the same database.Thus, the final selection comprised 261 studies.
Two evaluators independently classified the 261 selected studies according to their allocation concealment 19 .The kappa coefficient of agreement (kw) between the two evaluators was 0.94 (p = 0.000).After a consensus meeting, 43 RCTs published in English, with the participation of at least one plastic surgeon, and with adequately described allocation concealment were selected.
The comparison of the Delphi list items between the 2009-2013 period and previous study periods (1966-2003 and 2004-2008) is shown in Table 1.A significant improvement in scores was observed when comparing the items "Groups were comparable in terms of the most important characteristics of the prognosis" (p < 0.001), "Inclusion and exclusion criteria were specified" (p = 0.0029), and "Measures of variability and estimation of points were presented for the primary variable" (p = 0.0057) in the 1966-2003 and 2004-2008 periods.
The kappa coefficient of agreement (kw) between the two evaluators in the evaluation of the 43 RCTs for Jadad scores 21 was 0.67 (p = 0.0000).Table 2 shows the distribution of the RCTs according to Jadad scores.

DISCUSSION
The present study evaluated the evolution of the quality of RCTs published by plastic surgeons since 1966.The long follow-up period of 47 years, using the same method, facilitated a clear analysis of the evolution of studies over this period.
Considering the growing demand for specialized treatments and the limitation of health resources, there has been a growing interest in the practice of so-called "evidence-based medicine," 12,14,23,24 and the qualitative and quantitative increase of publications of RCTs by plastic surgeons evaluated in the 47 years confirms this interest.
Conducting RCTs in surgical areas is accompanied by many difficulties and challenges, which includes ethical issues that make it impossible to use placebo procedures or even all procedures, compared to a surgical intervention; the impossibility of blinding the surgeon; learning curves; technical differences between surgeons, among others 25,26 .Thus, the production of RCTs in this field has been slower compared to other medical specialties 24,[27][28][29][30][31] .
Despite the difficulties in conducting a randomized clinical trial in plastic surgery, evidence-based medicine is the key to the progress of this specialty 15 .Thus, the clinical trials that are effectively conducted should follow strict quality standards and methodological 1.a.Were the patients randomly allocated to the treatment groups?1.b.If individuals were randomly allocated to the treatment groups, was concealment allocation maintained?

2.
Were the groups comparable in terms of the most important characteristics of the prognosis?

3.
Were the inclusion and exclusion criteria specified?

4.
Did an independent evaluator evaluate the results?

5.
Did the caregiver wear a mask?

6.
Did the patient wear a mask?

7.
Were the measures of variability and the estimation of the points presented for the primary variable? 8.
Did the study include an analysis by intention to treat (all allocated patients)?
Chart 1. Delphi List Items 20 1.a.Was the study described as randomized (use of words such as "random" and "randomized")?rigor, so that they may have more impact on clinical practice 11,12,32 .
In order to improve the quality of RCTs, a group of researchers and editors elaborated the CONSORT (Consolidated Standards of Reporting Trials) Statement, initially published in 1996 and updated in 2001 31,32 .From the review on the use of the CONSORT Statement, published in 2010, which comprises a checklist and a flow diagram, it became popular and was adopted by most medical journals as the standard for describing RCTs 33 .
The use of the CONSORT Statement has contributed to an increase in the quality level of the published RCTs.An important limitation of the present study was the lack of use of the CONSORT checklist to assess the quality of published RCTs.However, it should be noted that the checklist, in its current form, was not available when the two previous studies were designed and conducted, and this study aimed to strictly use the same method as the two previous studies, thus, allowing a long-term evolution assessment.
A progressive increase in the number of RCTs published by plastic surgeons was observed over time.No RCT appropriately describing allocation concealment was published by plastic surgeons from 1966 to 1983.The first RCT with these characteristics was published in 1984, comparing the occurrence of capsular contracture after the use of saline implants or silicone gel for breast reconstruction 34 .From then on, a progressive increase was observed, following the popularization of EBM from the 1980 decade 6 , but with a substantial increase only from the 2000s onwards.
In the present study, a higher concentration of publications was observed in Europe and North America, following a trend described in the other two periods for comparison (1966-2003 and 2004-2008).Momeni et al. 11 , when evaluating RCTs in three major plastic surgery journals from 1990 to 2005, also observed a higher number of publications in Europe and North America.
The Delphi List 20 , used in this study, evaluates three dimensions of the quality of an RCT: internal validity (degree of validity of the study for the assessed sample), external validity (degree of validity of the study in extrapolating its results to the population), and statistical analysis.The comparison of the Delphi List items between the present study (2009-2013) and the first studied period (1966-2003)  20 indicates a greater number of responses, with statistical significance, for the items: "The groups were comparable in terms of the most important characteristics of the prognosis," "The inclusion and exclusion criteria were specified," and "Measures of variability and estimation of points were presented for the primary variable."This shows an increase in the quality of RCTs published in the current period regarding these items.However, there was no change in the quality of the studies when the 2009-2013 period was compared to the previous fiveyear period (2004 to 2008).
The Jadad Scale is a short, simple, reliable, valid, and widely used instrument 21,35 .Originally designed to assess pain in RCTs, it can be applied in other fields of medicine, since the items are not specific 21 .Olivo et al. 35 , in a systematic review, analyzed the scales used to assess the methodological quality of RCTs in the health domain.They found that most of the scales did not have strict control over their development, nor were they tested for validity and applicability.They also observed that the Jadad Scale 21 has been one of the most cited and used in the academic community of the health domain, besides having the best evidence for validity and applicability.
The evaluation of the Jadad scores 21 indicated that there was a statistically significant increase in quality when comparing the 1966-2003 period to the 2004-2008 and 2009-2013 periods.However, there was no improvement in quality when comparing the 2009-2013 period to the previous five-year period (2004 to 2008), indicating a stabilization in the quality of RCTs.
Yu et al. 36 conducted a cross-sectional study aimed at assessing the quality of RCTs publications on surgery that were published in the 2003-2013 period.They used the conformity of the items to the CONSORT 2010 32,36 checklist as quality criteria.They observed that the studies published in 2013 obtained higher scores than those published in 2003, and this was statistically significant, suggesting an improvement in the quality of publications.They concluded that there has been an increase in the quality of RCTs publications on surgery in the last decade.However, this quality remains at suboptimal levels, especially regarding surgical interventions 36 .
In this study, it was observed that the studies indicated a stabilization in quality compared to the last analyzed period (2009 to 2013) and the period before that (2004 to 2008) regarding the methodological criteria assessed using the Delphi List 20 and the Jadad Scale 21 .This could indicate that plastic surgeons, after a significant improvement, may have reached a basic level of quality in terms of criteria used for the publication of RCTs.However, several other items would need to be incorporated in order to increase their quality.This need reflects in the constant improvement of the CONSORT 2010 37 checklist, and in the requirement, by an increasing number of journals, of compliance with the checklist items for an RCT to be accepted for publication.
Plastic Surgery is characterized by a long history of innovation, which continues to this day, and has many contributions to share with other medical, clinical, or surgical specialties 31 .Existing barriers should not be considered as obstacles to scientific growth of the specialty, but as challenges to be overcome.

CONCLUSION
There was no difference in the quality of randomized clinical trials (RCTs) with appropriately described concealment allocation, published by plastic surgeons from 2009 to 2013 compared to the previous five-year period (2004 to 2008).However, both periods had better quality RCTs than the 1966-2003 period.

COLLABORATIONS TBM
Analysis and/or data interpretation, conception and design study, data curation, final manuscript approval, formal analysis, methodology, project administration, resources, writing -original draft preparation.

DFV
Analysis and / or interpretation of data, statistical analysis, study design and design, project administration, methodology, writing -review and editing, supervision.

MSN
Final approval of the manuscript.

LMF
Final approval of the manuscript.

Table 1 .
Quality assessment using the Delphi List after consensus meeting and comparison with the 1966-2003 and 2004-2008 periods.

Table 2 .
Quality Scale scores after consensus meeting.