Sample size estimation practices in research protocols submitted to Danish scientific ethics committees

Background Sample size in research projects is estimated before initiation of the study to minimise type 1 and type 2 error, while keeping the study's financial cost and subject enrolment to a minimum. This study investigates project-specific factors potentially associated with correct estimation of sample size in study protocols. Methods Examination of 189 non-commercially sponsored study protocols (84 randomised controlled trials (RCTs) and 105 non-RCT studies) submitted to the Scientific Ethics Committees of The Capitol Region of Denmark from 2013 to 2015. Results 119 (63%) study protocols contained a sample size calculation, with a significantly higher rate of sample size calculations in RCT vs non-RCT study protocols (76% vs. 52%, p < 0.001). Significantly more intervention studies than non-intervention studies (69% vs 52%, p = 0.020), studies including blood samples compared to those without (69% vs. 55%, p = 0.045), studies funded by a foundation donation compared to those with no funding (68% vs. 49%, p = 0.040) performed sample size calculations. Further, increasing number of sick patients enrolled (p = 0.048) and newer studies (p = 0.032) were more likely to include a sample size calculation in the protocol. Conclusions Estimation of sample size is more often reported in RCT than non-RCT study protocols. Also, intervention studies, studies funded by a foundation donation, studies including blood samples, studies with a greater amount of sick participants and chronologically newer study protocols more often reported a sample size calculation.


Introduction
Estimation of projected sample size of a medical research study is based on the hypothesis proposed by the investigators. For various reasons investigators sometimes fail to determine sample size before initiation of the study and consequently base their projected number of participants on experience, logistics or something even more unscientific [1]. A sample size too small increases the risk of type 2 error, whereas a study with too many participants potentially exposes research subjects to unnecessary risk and uses up excess resources [2].
Previous studies have explored underpowered designs and how the concept of statistical power can be misused and is often misunderstood [2][3][4]. Recent studies have investigated sample size estimation empirically [1,5,6]. Notably, in regards to randomised controlled trials (RCTs) conducted in the UK one study from 2013 reported that only 42% of the protocols supplied all the information needed to repeat the sample size calculation. Non-commercially sponsored studies were less likely to report all the information and researchers tended to overestimate their sample size [1]. A French study from 2009 found that among selected MEDLINE RCT-articles, 43% did not present enough information to repeat the sample size calculation, while 5% did not report conducting a sample size calculation at all [5].An American study from 2015 showed that only 56% of clinical trials of analgesics reported a sample size calculation, but just 38% included all the parameters needed to recalculate the sample size. The authors also found that RCTs reported a sample size calculation more often than other experimental study types [6].
To extend previous knowledge, the aim of this study was to investigate sample size estimation practices and use of statistical sample size calculation in non-commercially sponsored original research protocols submitted to the Scientific Ethics Committees of The Capitol Region of Denmark. Specifically, we identified project-specific indicators possibly associated with correct or incorrect statistical sample size calculation and sample size estimation.

Identification of research protocols
In Denmark, all original research protocols are submitted to one of five central Scientific Ethics Committees corresponding to the five geographical regions in the country. One of these regions is the Region (greater Copenhagen), which receive approximately 700 original research protocols annually (an annual mean of 721 through 2013-2017). This number is roughly equal to the total number of research protocols evaluated by the other four Regional Scientific Ethics Committees combined.
We identified non-commercially sponsored study protocols submitted from January 2013 onwards and included, consecutively, all protocols until July 2015 when 89 RCT protocols were ascertained. Only non-commercial study protocols were included to ensure that sample size calculation was not performed or included in the study design by a commercial sponsor before the academic investigators were involved in the study design. Commercially sponsored studies usually include a power calculation as per company tradition, and therefore may not reflect the individual academic investigators' practices. During this period 100 random non-RCT protocols were included for comparison. Inclusion of the earliest versions of the protocols was prioritized to more reliably represent the original research intention of the investigators. As per Danish law research protocols must include some considerations about participant number but not necessarily a mathematical sample size calculation. This sampling strategy was based on the assumption that, in order to detect at least a 20% difference in the prevalence of reported sample size estimation in two groups (our primary study endpoint); RCT protocols = 75% and non-RCT protocols = 55%, 86 protocols were needed in each category with a power of 80% (β = 0.2) at the 95% significance level (α = 0.05) in the two groups.

Data extraction
For each study protocol data were extracted from the protocol report formula, the statistical subsection of the protocol, and the study participant information, which were the documents covered by the right of access. As per Danish law it is possible for all citizens (option pursued particularly by journalists and other researchers who perform meta-research) to apply for right of access to individual research project documents. The documents concerning this project were delivered to the authors by the administrative personnel of the Scientific Ethics Committees of The Capitol Region of Denmark via secure delivery of PDF (portable document files). The extracted data were organised in a database comprising the following exposure variables: study type (RCT/non-RCT), researcher's academic status (junior/senior), researcher's gender (female/male), institution (state hospital/regional hospital and non-hospital), intervention (treatment/other intervention/ none), biobank status (existing biobank/new biobank/none), study participant remuneration (yes/no), multistate study (yes/no), field of medicine (internal medicine/surgery/other), procedures involved (electromagnetic radiation exposure/blood sample/biopsy/none), economic funding (state funded/medicinal company funded/foundation donation funded/no funding), study length (months), participants (number of healthy/sick), study sessions (number), time to new year (weeks), time passed since 2013 (months).
Although study type was stated by the researcher in the protocol report formula, not always did it match the actual study type and subsequently the authors re-classified the study type by evaluating the participant information form and the study description found in the protocol report formula. Of the 89 protocols submitted as RCTs, only 84 were determined to be RCTs when the protocol report formulae and the study participant information were examined by the authors.

Evaluation of sample size estimation
Information on the following components of sample size estimation was extracted from the statistical subsection of the protocols: α (significance level), β (power), expected proportion of drop-outs, effect size (minimal relevant difference, MIREDIF) and standard deviation (or two proportions, if the study compared proportions). Further, an estimation of sample size was performed by the authors based on the reported components and compared to the reported sample size in the protocol.
Whether a sample size calculation was performed by the researcher in the study protocol was our primary outcome. As supporting secondary analysis, we constructed a sample size calculation component composite score (SSCCCS) ranging from 0 to 6 with one point given for enough information conveyed to reproduce correctly the calculation and achieving the same sample size estimate, and one point for each of the following five constants reported; α, β, expected dropout fraction, effect size and standard deviation (or two proportions).
The data analysis was conducted in SPSS version 25, using logistic and linear regression functions.
The two most common explanations for not including a sample size calculation in the protocols where sample size calculation was not performed were; missing data (38%), e.g. SD not accessible, and sample size calculation done prior or later (24%), reported in the protocol as some kind of sample size estimation calculation was to be done later or was done prior to the protocol but with no mention of the calculation itself, whereas the rationale for the proposed sample size was often not explained (35%) or based on the investigators research experience (32%) ( Table 2). The following extracted variables were statistically significantly associated with having performed a correct sample size calculation in the protocol (

Discussion
RCT protocols submitted to the Scientific Ethics Committees of The Capitol Region of Denmark were more inclined to include a sample size calculation compared to other study designs. This may be explained by the consensus that research trials including treatments or interventions should be conducted more carefully since the subjects potentially are at a greater risk compared to other study types. An increasing number of projected sick participants in the study also increased the likelihood of including a sample size calculation in the protocol, possibly also explained by the greater respect when the study involves treatment.
Studies funded by a foundation included a sample size calculation more often than studies funded by other means and studies not funded. More strict foundation regulations possibly explain this. Also, investigators pursuing funding with major funding bodies likely pay more attention to important details of their study's design, and consequently studies including realistic and scientifically sound considerations about projected sample size are more likely to obtain funding.
Interestingly, more sample size calculations were performed in newer studies (from 2013 through 2015). This could be explained by more awareness in general among researchers about power and sample size in scientific studies as time passes.
Our study has several strengths: the consecutive inclusion of study protocols during a specified time period led to a random pattern of study characteristics in our sample thus reducing bias. The wide variety of project-specific explanatory variables included increased the information extracted from this study. The centralised structure of the Danish scientific ethics committees ensured a complete coverage of protocols since no local institutional review boards exist. The single, homogenous protocol report formula proved to be an easy way to standardize data, which if multiple formulae types were involved could induce confusion and misunderstanding. Conversely, it is a limitation of the study that it only included a small sample of protocols in a limited geographical area.
Ideally, planning of all studies with quantitative analysis involving human subjects should include a sample size estimation [7]. However, some study designs make sample size calculations difficult, specifically  When sample size was estimated without a calculation, the reason for missing calculation (values: missing data/pilot study/incalculable design/calculated prior or later/none) and the sample size rationale (values: similar studies/experience/predetermined size/calculated prior or later/none) were found in the statistical subsection of the protocol. Explorative study: some constant or variable (e.g. SD) is unknown, often not specified which in the protocol.
Incalculable design: the sample size calculation deemed "incalculable" by the investigator due to the study design.
pilot studies within a field so unexplored that no constants can be used in the calculation [8]. The interpretation of studies with inadequate or missing sample size calculations could potentially differ greatly from studies with correct calculation of sample size. Studies with too few research subjects risk failing to reject the null hypothesis, but also risk accepting a false hypothesis that otherwise would have been rejected if additional research subjects had been enrolled. In contrast, studies with an abundancy of research subjects unethically waste funds and subject additional participants to unnecessary hazard without a clear justification. Our investigation confirmed that estimation of sample size is more often performed in RCTs than non-RCT studies. We extend previous findings by showing that several project-specific factors increase the probability of reporting a sample size calculation in scientific protocols. Particularly, intervention studies, studies funded by a foundation donation, studies conducted at the state hospital, studies with blood samples drawn, studies with a high number of sick participants and chronologically newer studies are more likely to report a sample size calculation. This study increases the knowledge within a sparsely studied area and highlight the importance of adequate sample size estimation in academic research projects. The knowledge generated by this study is important for researchers from academia as well as members of scientific ethics committees and institutional review boards when evaluating the impact and soundness of research. Further, it ultimately may prevent the initiation of unjustified and unsound research that exposes patients to unnecessary harm and possibly delays important knowledge from being adopted by the research community and in clinical practice.

Conflicts of interest
None.

Funding
None.