Int. J. Morphol., 35(1):72-76, 2017.

 

Checklist for Reporting Results Using Observational Descriptive Studies as Research Designs. The MInCir Initiative

 

Lista de Verificación para Reporte de Resultados Usando Estudios Observacionales Descriptivos como Diseños de Investigación. La Iniciativa Mincir

 

Carlos Manterola^1,2,3 & Tamara Otzen^2,3,4

1 Department of Surgery, Universidad de La Frontera, Temuco, Chile.

2 Center for Morphological and Surgical Studies (CEMyQ), Universidad de La Frontera, Temuco, Chile.

3 Center for Biomedical Research, Universidad Autónoma de Chile, Chile.

4 Universidad de Tarapacá, Arica, Chile.

Correspondence to:

---

SUMMARY: Observational and descriptive studies (ODS) represent between 70 % and 80 % of the designs utilized in biomedical publications of the different scientific journals. Despite this, there are no tools to guide writers and to assist reviewers in reporting results with this type of research design. The aim of this study was to report the characteristics of a validated checklist for reporting the results using ODS as research designs in an English version. Two-stage study with qualitative methodology. In a first stage, a proposal was designed, by collecting items and domains from an extensive review of the literature. In the second, an instrument was developed by applying reduction items and domains through a panel of 45 experts comprised of clinical academics, reviewers and editors of biomedical journals, and experts in research methodology. These worked determining the validity of facade and content of the instrument. The items and domains incorporated into the final instrument were those in which over 80 % of agreement was achieved between the participants (36 of 45). In this way an instrument was created composed of 19 items, grouped into four domains. Characteristics of the design, construction and validation of a checklist that could help authors, reviewers and journal editors to write and review articles using ODS as research designs to report results was reported.

KEY WORDS: Observational studies; Longitudinal Studies; Epidemiologic Studies; Descriptive Studies; Cross-Sectional Studies; Case Reports; Case series.

---

RESUMEN: Los estudios observacionales descriptivos (EOD), representan entre el 70 % y 80 % de los diseños utilizados en las publicaciones biomédicas de las distintas revistas científicas; y, a pesar de ello, no existen instrumentos para guiar a los escritores en el reporte de resultados, como tampoco para colaborar con los revisores con este tipo de diseños de investigación. El objetivo de este estudio fue reportar las características de un sistema de verificación validado, para el reporte de resultados con EOD como diseños de investigación, en una versión en idioma inglés. Se llevó a cabo un estudio bietápico con metodología cualitativa. En una primera etapa, se diseñó una propuesta, mediante la recopilación de ítems y dominios a partir de una extensa revisión de la literatura relacionada. En la segunda, se construyó un instrumento, aplicando reducción de ítems y dominios a través de un panel de 45 expertos, compuesto por académicos clínicos, revisores y editores de revistas biomédicas; y expertos en metodología de investigación. Estos, trabajaron determinando la validez de fachada y de contenido del instrumento. Los ítems y dominios incorporados al instrumento final fueron aquellos en los que se logró más de un 80 % de acuerdo entre los participantes (36 de 45). Se generó de este modo un instrumento compuesto por 19 ítems, agrupados en 4 dominios. Se reportan las características del diseño, construcción y validación de una lista de verificación en versión en inglés, que puede ser utilizada por autores, revisores y editores de revistas, para la escritura y revisión de artículos en los que se utilicen EOD como diseños investigación.

PALABRAS CLAVE: Estudios observacionales; Estudios longitudinales; Estudios epidemiológicos; Estudios descriptivos; Estudios de corte transversal; Series de casos.

---

 

INTRODUCTION

One of the stages of the scientific research process is the communication of results, whose dissemination through scientific journals is fundamental. These remain the most significant channels of formal communication of new knowledge generated, representing therefore, the usual vehicle of disclosure of results, new methods and techniques, etc. (Manterola & Astudillo, 2013).

There is strong evidence that a large part of the questions that are generated in clinical research originate later observational studies (Glasziou et al., 2004; Torloni & Riera, 2010), and this, because observational studies play an important role in research on the benefits and harms of interventions (Black,1996; Grootendorst et al., 2010), detect rare or late adverse effects of treatments (Papanikolaou et al., 2006; Vandenbroucke, 2006), etc.

Thus, observational descriptive studies (ODS) constitute between 70 % and up to 80 % of articles published in scientific journals, and despite this, there are no instruments to guide authors in reporting results, as well as to help reviewers and publishers with publications that use this type of research designs (Funai et al., 2001; Scales et al., 2005; Manterola et al., 2006a; Manterola et al., 2006b; Manterola & Grande, 2010; Primo et al., 2014).

On the other hand, it is known that recommendations on how to report research results can help improve quality, which has been observed since theemergence of the CONSORT (Consolidated Standards of Reporting Trials) statement for clinical trials in 1996 (Moher et al., 2001), and other subsequent initiatives such as STARD (Standards for Reporting of Diagnostic Accuracy) (Bossuyt et al., 2003), MOOSE (Meta-analysis of observational studies in epidemiology) (Stroup et al., 2000), STROBE (Strengthening the Reporting of Observational Studies in Epidemiology), With special emphasis on cohort and case-control studies (von Elm et al., 2007); etc.

However, there is no guideline for ODS reporting, despite its high prevalence in biomedical journals, which is why in 2011 and 2013 we published the preliminary report of a study aimed at generating a proposal for this purpose (Manterola D. & Astudillo D., 2011; Manterola & Astudillo). These reports allowed the use of this proposal in publications of various disciplines, in Spanish and English. However, it was necessary to report the English version of this checklist, which will allow a better and more frequent use of it.

The types of design included in the concept of ODS are: case report, case series (retrospective and prospective), cross-sectional studies, diagnostic tests and concordance, population survey studies and correlational studies (Manterola, 2001; Burgos D. & Manterola D.; Manterola & Otzen, 2014).

The aim of this study was to report de characteristics of a validated checklist for reporting the results using ODS as research designs in an English version.

MATERIAL AND METHOD

Design: Bietapic study using qualitative methods for the generation of items and to built the instrument (Pope & Mays, 1995; Jones & Hunter, 1995). In the first step, items and domains were generated, and in the second one the instrument was designed and built.

Center: Department of Surgery and Center of Excellence in Morphological and Surgical Studies (CEMyQ), Universidad de La Frontera, Chile.

Methodology: In the first step, a proposal was designed by collecting items and domains from an extensive review of the related literature. In the second one, the instrument was constructed, applying a reduction of items and domains through a panel of experts, who were consulted through a semi-structured and self-administered questionnaire, composed of 25 items from the literature, to which the experts could add others according to their experience or particular vision of the situation. In this way, the validation of content, both facade and sampling, was generated, thus ensuring that the instrument contains representative items from all areas that define the concept or construct under study, which was secured by an exhaustive review of the literature, the experience of the research group and the expert panel.

Participants: The expert panel was comprised of 4 clinical epidemiologists, 5 biostatistics, 28 clinical academics that were members of reviewer panels of different biomedical journals, and 8 editors of biomedical journals (N=45). Median age of these was 52 years (35 to 70 years old). Thirty-nine of them are male (86.7 %) and 19 work in international centers (42.2 % [11 in South American, 4 in European and 4 in North American centers respectively]). They are medical specialists (n=33), biostatistics (n=5), nurses (n=5), dentist (n=1), physiotherapist (n=1. The median professional time experience of them was 28 years (5 to 47 years). Thirty-eight of them (84.4 %) work in universities.

Data collection: A qualitative strategy was developed applying the Delphi technique (Powell, 2003; Kennedy, 2004; Price, 2005). This one was applied in different phases: first of all, the experts were selected and the research question was asked, and then answers by way of a questionnaire to consult with respect to its agreement with the points on the elements that were considered fundamental were structured. In this case, items and domains that would configure the definitive checklist. With each feedback-response from the previous questionnaires the experts were able to establish a general consensus.

Analysis plan: After an extensive literature review, it was verified that there is no consensus embodied in a guideline for reporting ODS. The research question (first draft) was constructed. This, was raised to experts and they answered individually, private and anonymously. After which the responses were analyzed they were summarized and converted into items and domains. Thus, each member of the panel of experts evaluated each item according to their knowledge and experience (second draft). Subsequently, an ad-hoc database was generated for data storage, which was analyzed in a Stata 11.0 / SE® program. After performing an exploratory data analysis, descriptive statistics was applied calculating percentages, medians and extreme values.

Ethical principles: The identity of all members of the panel of experts was masked for which each of them was codified.

RESULTS

The items generated in the first stage (N = 25) were initially grouped into 5 domains: title, abstract, introduction, methodology, results, discussion and conclusions. When applying the questionnaire to the panel of experts, a reduction of items and domains was generated.

The items and domains incorporated into the final instrument were those in which more than 80 % agreement among the participants was achieved (in 36 of 45). Then, the items proposed by panel members were added, when at least two of them agreed on the relevance of incorporating them.

Thus, an instrument was constructed that was composed of 19 items: In study problem, objectives, design, center, characteristics of participants, inclusion and exclusion criteria, type of sampling used, variables studied, Follow-up, statistics used, observed ethical principles, general description of the in study sample, analysis of groups and subgroups, other analyses (if applicable), novelty of the proposal represented by the article, comments on the results obtained, limitations of the study and Conclusion (if applicable). These were grouped into 4 domains: Introduction, methodology, results and discussion.

Afterwards, the results were socialized, at which point the experts reviewed the items, domains and their respective explanatory comments, obtaining the final instrument that is outlined in Table I.

Table I. Generated Instrument. Checklist for Reporting
Results Using ODS as Research Designs. MInCir Initiative.

DISCUSSION

It is usually thought that only clinical trials (CT) are ideal primary studies to generate evidence, and there are those who even categorically reject the possibility of ODS serving this purpose. Thought associated with the fact that in the ODS the patient's allocation to a particular treatment or intervention is not controlled, transforming the researcher into a mere observer, descriptor and rapporteur of what happens. Nevertheless, there are groups who think that the restrictions imposed on patients included in a CT often cause them to differ from the characteristics of the habitual patient in clinical practice. In addition the participating centers tend to be specific, reference or university centers, also with special characteristics regarding the collective. That is why it is thought that the conclusions obtained from CTs are not always generalizable.

However, it seems more rational to adopt a more conservative position, admitting on the one hand that CTs are the best methodological tool to use when dealing with prevention and treatment, and on the other hand, that well-developed ODSs with adequately reported results are also appropriate methodological tools to generate evidence (Black; Grootendorst et al.).

Thus, it can be understood that what motivates ODS is to: describe unusual manifestations of an event of interest or the effect of an exposure that cannot be randomly assigned (for example undergo surgery, an invasive procedure, etc.), describe rare diseases or unusual events of interest, generate knowledge of the natural history or clinical course of a clinical entity, obtain frequencies of the different variables of an event of interest, allow the formulation of hypotheses of possible risk factors, conduct epidemiological surveillance, to study the external validity of the application of an intervention (verifying that the conditions of efficacy and tolerance in routine practice are met), to study the adherence to an intervention (degree of compliance of the treatment by the participants), etc. (Manterola; Manterola & Astudillo; Manterola & Otzen).

Despite this, it must be clearly understood that these types of designs have serious limitations, among which the following are noteworthy: they represent the limited experience of a research group, which results in the personal subjectivity of those who report, which can Generate classification and measurement biases. The problems generated by reporting biases related to selection and reference. Consider that the presence of a risk factor may be the result of chance. As the observation begins at different points in the course of the disease, or event of interest under study, it is difficult to be emphatic respect temporal associations. In summary, it must be remembered that ODS do not constitute solid evidence as a basis for altering clinical practice (Manterola; Manterola & Otzen).

Regarding the methodology it is worth mentioning some relevant facts: Firstly, experts were defined as subjects who could make valid contributions, that is to say that they possess knowledge based on the practice and experience updated with respect to the theme (Kennedy; Price; Jorm, 2015). We chose a heterogeneous group because the diversity of points of view is fundamental, which gives more interest and reflection on the participants (Powell; Birko et al., 2015).

And, secondly, we note that qualitative methodology (consensus techniques) allow obtaining quantitative estimates determining the degree of agreement among participants (Pope & Mays; Devers, 2011).

This strategy, called the Delphi technique, constitutes an effective and efficient way to generate consensus in a group without the members meeting physically. It is carried out at a distance by contacting the participants through a questionnaire by E-mail, through which the different prioritization wheels of the items and domains to be evaluated are carried out (Powell; Kennedy; Price; Birko et al.; Jorm). In summary, the advantages of this technique are:

1. The anonymity of contributions and ideas guarantees that each of them has the same value and equal importance in the further analysis.

2. The influence of the most experienced or reputed expert is eliminated, since the opinions of all the members have the same importance.

3. The interaction between participants is controlled and directed by a coordinator with a gradually feedback, so irrelevant information is eliminated (free exchange of information between experts is not allowed).

4. On the other hand, allows experts to express their ideas more frankly than in a formal meeting.

5. Although there is an instrument that could be considered similar to the current proposal known as the STROBE initiative, which has been widely disseminated in various journals, it is important to note that the STROBE Declaration is aimed at articles made with the three most important designs of observational analytical epidemiology: Cohort studies, case-control studies and cross-sectional studies. However, it does not consider the most common studies, case series and case reports, as well as population and correlational studies. In addition there are some limitations of the initiative, clearly mentioned in the original article (von Elm et al.).

By way of conclusion, it is possible to mention that characteristics of the design, construction and validation of a checklist that could help to authors, reviewers and journal editors to write and review articles using ODS as research designs to report results was reported. The generated checklist is intended to be a tool to value such studies quickly and efficiently.

REFERENCES

Birko, S.; Dove, E. S. & Özdemir, V. Evaluation of nine consensus indices in Delphi foresight research and their dependency on Delphi survey characteristics: A simulation study and debate on Delphi design and interpretation. PLoS One, 10(8):e0135162, 2015.

Black, N. Why we need observational studies to evaluate the effectiveness of health care. B. M. J., 312(7040):1215-8, 1996.

Bossuyt, P. M.; Reitsma, J. B.; Bruns, D. E.; Gatsonis, C. A.; Glasziou, P. P.; Irwig, L. M.; Moher, D.; Rennie, D.; de Vet, H. C.; Lijmer, J. G. & Standards for Reporting of Diagnostic Accuracy. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann. Intern. Med., 138(1):W1-12, 2003.

Burgos D., M. E. & Manterola D., C. Cómo interpretar un estudio sobre pruebas diagnósticas. Rev. Chil. Cir., 62(3):301-8, 2010.

Devers, K. J. Qualitative methods in health services and management research: Pockets of excellence and progress, but still a long way to go. Med. Care Res. Rev., 68(1):41-8, 2011.

Funai, E. F.; Rosenbush, E. J.; Lee, M. J. & Del Priore, G. Distribution of study designs in four major US journals of obstetrics and gynecology. Gynecol. Obstet. Invest., 51(1):8-11, 2001.

Glasziou, P.; Vandenbroucke, J. P. & Chalmers, I. Assessing the quality of research. B. M. J., 328(7430):39-41, 2004.

Grootendorst, D. C.; Jager, K. J.; Zoccali, C. & Dekker, F. W. Observational studies are complementary to randomized controlled trials. Nephron Clin. Pract., 114(3):c173-7, 2010.

Jones, J. & Hunter, D. Consensus methods for medical and health services research. B. M. J.,311(7001):376-80, 1995.

Jorm, A. F. Using the Delphi expert consensusmethod in mental health research. Aust. N. Z. J. Psychiatry, 49(10):887-97, 2015.

Kennedy, H. P. Enhancing Delphi research: methods and results. J. Adv. Nurs., 45(5):504-11, 2004.

Manterola, C. Estrategias de investigación. Diseños observacionales 1a parte. Estudios descriptivos. Rev. Chil. Cir., 53:229-33, 2001.

Manterola, C.; Busquets, J.; Pascual, M. & Grande, L. What is the methodological quality of articles on therapeutic procedures published in Cirugía Española? Cir. Esp., 79(2):95-100, 2006a.

Manterola, C.; Pineda, V.; Vial, M.; Losada, H. & the MINCIR Group. What Is the methodologic quality of human therapy studies in ISI surgical publications? Ann. Surg., 244(5):827-32, 2006b.

Manterola, C. & Grande, L. Methodological quality of articles on therapeutic procedures published in Cirugía Española. Evaluation of the period 2005-2008. Cir. Esp., 87(4):244-50, 2010.

Manterola D., C. & Astudillo D., P. Iniciativa Mincir para el reporte de estudios observacionales descriptivos. Reporte preliminar. Rev. Chil. Cir., 63(1):36-41, 2011.

Manterola, C. & Astudillo, P. Checklist for reporting of descriptive observational studies. MINCIR initiative. Int. J. Morphol., 31(1):115-20, 2013.

Manterola, C. & Otzen, T. Observational Studies. The Most Commonly Used Designs in Clinical Research. Int. J. Morphol., 32(2):634-45, 2014.

Moher, D.; Schulz, K. F. & Altman, D. G. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet, 357(9263):1191-4, 2001.

Papanikolaou, P. N.; Christidi, G. D. & Ioannidis, J. P. Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. C. M. A. J., 174(5):635-41, 2006.

Pope, C. & Mays, N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. B. M. J., 311(6996):42-5, 1995.

Powell, C. The Delphi technique: myths and realities. J. Adv. Nurs., 41(4):376-82, 2003.

Price, B. Delphi survey research and older people. Nurs. Older People, 17(3):25-31, 2005.

Primo, N. A.; Gazzola, V. B.; Primo, B. T.; Tovo, M. F. & Faraco, I. M. Jr. Bibliometric analysis of scientific articles published in Brazilian and international orthodontic journals over a 10-year period. Dental Press J. Orthod., 19(2):56-65, 2014.

Torloni, M. R. & Riera, R. Design and level of evidence of studies published in two Brazilian medical journals recently indexed in the ISI Web of Science database. São Paulo Med. J., 128(4):202-5, 2010.

Scales, C. D. Jr.; Norris, R. D.; Peterson, B. L.; Preminger, G. M. & Dahm, P. Clinical research and statistical methods in the urology literature. J. Urol., 174(4 Pt. 1):1374-9, 2005.

Stroup, D. F.; Berlin, J. A.; Morton, S. C.; Olkin, I.; Williamson, G. D.; Rennie, D.; Moher, D.; Becker, B. J.; Sipe, T. A. & Thacker, S. B. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA, 283(15):2008-12, 2000.

Vandenbroucke, J. P. What is the best evidence for determining harms of medical treatment? C. M. A. J., 174(5):645-6, 2006.

von Elm, E.; Altman, D. G.; Egger, M.; Pocock, S. J.; Gøtzsche, P. C.; Vandenbroucke, J. P & STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. B. M. J., 335(7624):806-8, 2007.

---

Corresponding author:

Dr. Carlos Manterola

Department of Surgery and CEMyQ
Universidad de La Frontera
Manuel Montt 112, Office 408
Temuco
CHILE

E-mail: carlos.manterola@ufrontera.cl

Received: 03-09-2016
Accepted: 22-11-2016