Methodological quality of clinical trials in pediatric dentistry research

Cite as: Jaime A, Rodríguez C & Aravena P.Methodological quality of clinical trials in pediatric dentistry research published in ISI journals, 2008-2012. J Oral Res 2015; 4(2): 109-115. Abstract: There are many studies about the methodological quality of articles in pediatric dentistry. However, there is no up-to-date information or the use of measurement guidelines designed and validated for this purpose. The aim of this article is to determine the methodological quantity and quality of clinical trials (CT) in pediatric dentistry published in four Web of Knowledge (ISI-indexed) journals between 2008 and 2012. Clinical trials published in four ISI pediatric dentistry journals were evaluated. Name of the journal, year of publication, country of the corresponding author, type of CT and main subject were registered. A scale of methodological quality (MINCIR) from 11 to 36 points was applied to evaluate each article, establishing 18 points as the cutoff for “good methodological quality”. The quality was calculated by country, subject area and journal. Of 1,151 published articles, 149 (12.5%) clinical trials were selected. Their methodological quality was 15.7±2.7 points. 17% of the CT achieved a score equal to or greater than a methodological quality considered “acceptable”. One in every 6 published clinical trials was of good methodological quality. This would actually make it difficult to adopt the results of these investigations into routine clinical practice or to include them in future systematic reviews and meta-analyses.


INTRODUCTION.
Evidence-based dentistry (EBD) represents a treatment approach that integrates the use of the best evidence available for clinical decision making 1 , helping professionals to stay informed about the areas of work prior to the treatment choice 2 . This approach requires measurement tools and reports of methodological quantity and quality, such as bibliometric analyses 3 , which can characterize the scientific production of people and institutions both in quantitative and qualitative terms, reducing the subjectivity inherent in the indexing of knowledge in a determined field of science to a minimum 4 .
In pediatric dentistry, bibliometric analyses have been performed on clinical trials (CT) and their results are varied. Nainar 5 reports that three out of four articles published between 1968 and 1998 were studies considered as having a low level of evidence (descriptive studies, case reports and expert opinions); while Poletto & Faraco 6 analyzed articles published in a Brazilian journal of pediatric dentistry, finding a frequency of only 4.3% in CT. For his part, Al-Namankany 7 analyzed the quality of CT publications, basing it on the CONSORT (Consolidated Standards of Reporting Trials) guideline, concluding that these articles were of poor quality and contained insufficient information to evaluate the validity of the study. All this indicates that studies in pediatric dentistry are in the main anecdotal in nature with a poor methodological quality.
Although the reports conclude that there is a need to evaluate and improve the quality of the evidence 5,7,8 , the instruments used to measure methodological quality do not present validity and reliability studies when determining this quality, which must be seen as a multidimensional concept 9,10 . One of the scales that measures the methodological quality of CT is the MINCIR (Methodology and Research in Surgery) scale, which has demonstrated its validity 9 and reliability 11 and proven its use in the bibliometric analysis of articles in dental therapy research 12 and oral-maxillofacial surgery 13 . Its use in pediatric dentistry would make it possible to quantify the methodological quantity and quality of CT, minimizing the chances of error in the determination of the quality of the scientific literature 14 and contributing to the most current quantification of the evidence obtained in this specialty.
The aim of this study is to determine the quantity and methodological quality of CT in pediatric dentistry published in four ISI-indexed journals between 2008 and 2012.

MATERIALS AND METHODS.
A bibliometric study was designed for CT published between 2008 and 2012 in the pediatric dentistry journals belonging to the SCI-Expanded database of the Thomson-Reuters (ex ISI) collection: Journal of Clinical Pediatric Dentistry; Pediatric dentistry; International Journal of Paediatric Dentistry and European Journal of Paediatric Dentistry.

Information search and compilation
One of the principal investigators (AJ) searched the SCI-Expanded database (www.isiwebofknowledge.com)  using a strategy and search limits recommended for the four pediatric dentistry journals (Table 1). Two investigators (JA, AP) selected the articles that had the CT design or something similar in their title or abstract 15 ("clinical trial/ study","randomized-single/double/triple-blind/masked -control/controlled trial", "randomized trial", "split mouth design/study", "crossover/parallel trial/study/design/ intervention", "cluster randomized trial", "randomized placebo controlled study) and whose participants were under 18  years of age. Articles were excluded where the design was descriptive or analytical, as well as literature reviews, letters to the editor and technical notes. The selected articles were analyzed by having two investigators (AJ, PA) read the complete text; these two ultimately selected those articles where the authors controlled one or more interventions on a population of subjects selected according to a properly designed protocol. Then, both investigators analyzed and independently tabulated the variables: i) journal name; II) publication year (2008-2012); III) country of corresponding author; IV) type of CT and v) main subject area of the article. (Table 2) For those articles where there was a discrepancy between the two reviewers in recording the type of CT and main subject area, a second investigator (PA) analyzed the document and classification was agreed by consensus.

Methodological quality
For the analysis of the methodological quality (MQ ) of each CT, the "MINCIR Therapy" instrument described by Manterola et al. 9 was used, which presents 3 domains (Study design; Size of study population and Methodolo-    gy used) and six items (Table 3). A score within a range of 11 (worst methodological quality) to 36 points (best methodological quality) was assigned to each article, establishing 18 points as the cutoff for "good methodological quality". Data analysis.
In order to verify interobserver reliability, two investigators (AJ and PA) analyzed, using the methodological quality instrument chosen for this study, 15 articles published in 2007 in the same four journals, determining an interobserver reliability in designs and MQ scores with a Kappa above 0.8. The variables journal name, publication year, country of corresponding author and main subject area of the article were analyzed using descriptive statistics. The average MQ score of each article was related to the country of origin, subject area and journal using tables. The data were tabulated in MS Excel 2009, from which the results tables were created.

RESULTS.
1,151 titles were identified in total from the search strategies: 234 articles were described as CT but ultimately 149 (12.5%) were selected and classified as CT (Figure 1).

Number of articles
Of the articles selected, the journal with the highest number of CT in the analyzed period was the Journal of Clinical Pediatric Dentistry (50 articles) and the year with the most publications was 2011 (40 articles) ( Table  4). The country with the most corresponding authors was India (16.8%), followed by Brazil and Italy (12%) ( Table  5). According to the subject area, most of the selected CT were categorized as Dental materials/Restorative Dentistry and Prevention/Dental education ( Table 6).

CT Quality
All the articles reviewed presented an average ±standard deviation (±SD) of 15.7±2.7. The journal with the highest average MQ score was the International Journal of Pediatric Dentistry with 17.1±3.7 (Table 4). Only 26 of all the articles reviewed obtained a score of 18 points or more. Among the countries, Finland shows the greatest average MQ with 21.6 points ( Table 5). The discipline with the highest MQ score was Sedation/Pain management, with an average of 16.4 points. No subject area reached 18 points ( Table 6).
The MQ instrument made it possible to classify 92% as simple CT, 2% presented clear objectives suitable to the study design, and 8.7% used calculation strategies for determining sample size.

DISCUSSION.
Our results show that in the last five years an average of 30 CT have been published every year, which is 12.9% of all the articles published from 2008 to 2012 by the four pediatric dentistry journals selected, a percentage similar to the 12.4% shown by Manterola et al. 12 in their study of MQ in surgery publications, but much higher than the 4.3% reported by Poletto & Faraco 6 for pediatric dentistry articles published by a Brazilian journal and the 3.6% for endodontics articles reported by Shafiei & Shahravan 16 .
About the quantity The frequency of published CT has been increasing over the past few years; but many of these, together with the contribution of new scientific writing guidelines, has not been enough to have any real impact on improving the quality of CT. The Journal of Clinical Pediatric Dentistry published the greatest number of CT; however, despite the quantity, the frequency per year of CT was observed as being homogenous in all the journals. If we compare the results with those of Nainar 5 , we observe that Pediatric Dentistry has published a steady number of CT over the years. 51.5% of the CT in the four journals analyzed are from India (16.8%), Brazil (12%), Italy (12%) and the United States (10.7%); these values are similar to the scientific productivity of these countries in other dental disciplines such as prosthodontics 17 and orofacial pain 18 , and are associated with public policies to incentivize publication, the presence of advanced research centers or the collaboration of researchers from other countries advanced in each area 17 .
The most published subject area in pediatric dentistry is "Dental material and Restorative dentistry", followed by "Prevention and Dental education", results very similar to those obtained by Poletto & Faraco 6 , but somewhat different from those reported by Nainar 5 , which reports that the main subject area studied is "Medicine, Pathology and Oral Surgery", which may be due to this author integrating every type of article design, of which the greatest proportion is descriptive studies, studies with no intervention risk.
About the quality In terms of the MQ, in general an average score of 15.7±2.7 was obtained. Only 26 CT (17%) obtained a score higher than the construct of good methodological quality proposed by the instrument (18 points). Independent of the instrument used to evaluate the MQ, the results are very similar to previous bibliometric reports in pediatric dentistry 7,8 and in other areas of dentistry 12,13,19,20 .
The countries with the greatest number of CT publications do not achieve a satisfactory methodological quality (Table 5). India has shown an increase in CT over the past few years, which has not correlated with its quality score as presented in this report (14.8). Finland reaches a mean that exceeds the cutoff of the scale, which is why as a country, its articles present a good methodological quality; nevertheless, the number of published articles is low, because in 2010 and 2011 only one article per year was chosen.
Although these results present data similar to the reports previously mentioned 7,8 , it is important to consider some limitations of the present study. First, the journals analyzed do not include all the CT generated worldwide in pediatric dentistry, and second, the low number of CT selected could be due to search and indexing errors as well as measurement bias on the part of the researchers. Small variations in the number of CT could have a great impact in the final figures. However, we consider it useful to present these results to the dentistry community, responding to the suggestions of previous reports 5,7,8 and using a completely valid and reliable MQ measuring instrument 11 .
The systems used to assess the quality of scientific publications have been varied; nevertheless, the MQ must be seen as a multidimensional concept. Although there are guidelines for the publication of results such as CONSORT (randomized clinical studies) and STARD (diagnostic accuracy studies), none of these was designed specifically for the evaluation of MQ and they have not been validated for this purpose 9 . In their report, Da Silva et al. 10 analyzed several psychometric scales and lists of methodological quality in clinical articles, observing that the CONSORT guideline reports only analysis of appearance, content and agreement between reviewers 10,21 which entails limitations, since they can be used in the evaluation of different types of studies, for different populations and different approaches in health care 10 . As a result, it becomes important to identify valid and reliable scales to use for a specific topic, thereby decreasing the possibility of errors in determining the quality of scientific literature 14 .
The MINCIR scale used in this study presents reports of reliability and validity in dental therapy research 12 , permitting analysis of the MQ based on the three basic concepts that the report must explicitly mention within a CT: type of study design, study population size and how it was calculated, and presentation of aims and clear and replicable selection criteria 9 ; for this reason, our study revealed that 92% of the CT were simple CT, and only three articles (2%) presented clear aims suitable to the study design. As a result, inadequate methodological quality was observed most frequently when the study design, the study population size and its justification were mentioned and when the aims were clearly and specifically outlined.
Despite the recommendation by editorial bodies concerning the use of scientific writing guidelines for CT such as CONSORT, their quality has not improved substantially since the publication of the guideline 7 . This implies repeating patterns of quality: instead of improving, they continue with a clear deficit in the generation of CT of good methodological quality, neglecting the editorial recommendations in the improvement of reporting quality.
In conclusion, the quantity and quality of clinical trials in pediatric dentistry research in the last five years has remained low. There is therefore a critical need to develop methodological strategies to improve the volume and quality of the scientific production in this area of dentistry. In light of these results, it is suggested that these assessment scales be disseminated more widely in order to verify the different items that can doubtless be corrected or improved.