Meta-analysis of Drop out Rates in Cataract Surgery RCTs - An Update

the RCT meta data extraction major parts of the statistical meta data the meta data extraction RCT the statistical the FK the meta investigation the analysis the meta data the ABSTRACT Aims: A realistic sample size calculation is an essential step in planning a clinical trial. It includes the consideration of the expectable drop out profile during individual patient observation periods to ensure a sufficient sample size for statistical analysis. Aim of this meta-analysis was to estimate dropout rates for randomized controlled trials (RCT) on cataract surgery during a follow–up period of 6 and 12 months in order to optimize sample size calculation. Methodology: A full text hand search in five ophthalmological journals (publication period 01/2002 – 12/2012) for RCTs on cataract surgery was performed. The meta-estimation of the reported 6 and 12 months drop out rates was based on the random effects model and stratified for the trial design characteristic “comorbidities allowed by design”. Results: 35 RCTs reporting no comorbidities (total patient count n=3.055) and 9 RCTs reporting Original Research Article BJMMR, 1-17, comorbidities by design (n=8.631) met the inclusion criteria for the 6 months follow-up evaluation; 41 RCTs without comorbidities (n=3.384) and 7 RCTs allowing for comorbidities (n=1.082) were identified for the 12 months follow-up evaluation. Respective and 12 months meta drop out rates of 7.8% (95% CI 5.0 – 11.8%) and 16.3% (95% CI 13.2 – 20.0%) were estimated from RCTs without comorbidities. RCTs allowing for comorbidity by design demonstrated lower drop out estimates with 3.2% (95% CI 2.9 – 3.6%) after 6 months follow-up and 6.7% (95% CI 3.9 – 11.2%) after 12 months. Conclusion: Sample size calculation in cataract surgery should account for drop outs rates of at least 10% during a 6 months and of at least 20% during a 12 months follow-up period.


INTRODUCTION
Clinical trials are an important tool for improving medical knowledge. An essential determinant of their validity from the methological perspective is the appropriate design planning, which should be carried out with regard to the ICH guidelines E6 "Guideline For Good Clinical Practice" [1] and E9 "Statistical Principles for Clinical Trials" [2]. One important part of the design planning consists in the calculation of the minimum necessary (net) sample size for the trial alongside its primary clinical endpoint. If a trial suffers from a sample size too small to achieve a statistically significant trial result during statistical analysis, the overall patient trial may be considered unethical: It is not acceptable from an ethic point of view to randomize patients onto therapeutic alternatives, when the underlying trial design and its sample size cannot achieve of significant findings in the first place. Such trials also cause severe economical damage regarding millions of research investment necessary to implement, run and analyse a clinical trial. As a consequence, from either the medical, the ethical and the economical perspective the assurance of a sufficiently large number of trial patientsactually the minimum patient number necessary for statistical analysis with a sufficiently large power -is a crucial determinant of the intended clinical trial. In summary, the methodological impact of sample size calculation is directly motivated by its medical, ethical and economical consequences.
However, bearing unavoidable processes with the outcome of individual patient drop out in mind, such as withdrawal of informed consent, the necessary sample size must be increased by a context-specific drop out rate to avoid underpowering of the trial only because of drop outs [3]: "Underpowering" a clinical trial refers to the ethical perspective mentioned above by starting a trial with too few patients to achieve significant results in the first place. Note, however, that increasing the net sample size to account for drop outs is strictly connected to the ethical perspective in clinical trial planning as well: the net sample size has to be large enough to achieve statistically significant findings; On the other hand no more patients than necessary should be recruited for ethical reasons. An inappropriately large increase of the net sample size due to an unrealistic drop out rate presumption might therefore as well imply an unethical trial design and increases the cost of a study. Hence a valid estimate for the expectable drop out rate has to be included in the sample size calculation [4].
The ICH guideline E9 as well as the CONSORT Statement ("Consolidated Standards of Reporting Trials") postulate recommendation for documentation and reporting in clinical trials. For implementation the CONSORT Statement comprises a flow diagram which details the progress of all participants through the trial [4]. Usually trial planners refer to such diagrams presented in previous or similar trial publications to derive information on the order of the drop out rate, which has to be expected in their intended trial. These punctual informations, however, may be crucially biased (although unintended) as being drawn from only a single or very few trial reports and lack from representativeness. Whereas sample size calculation usually refers to a maximum of information on the expectable effect size in related clinical trial publications (mostly based on systematic reviews and meta effect estimates), the presumption for expectable drop out profiles are often tackled rather novercal. The aim of the present meta-analysis was therefore to illustrate a possible meta approach in evaluating expectable drop out rates. The approach will be illustrated in terms of the evaluation of randomized controlled clinical trials on cataract surgery with regard to a typical follow-up period of 6 to 12 months; The resulting drop out estimates shall then serve for optimized trial planning for this high volume research segment.

Search Strategy and Inclusion Criteria
Primary endpoint of the meta-analysis was the total drop out rate in prospective randomized clinical trials (RCTs) on cataract surgery after a 6 months individual follow-up. A key secondary endpoint was the corresponding total drop out rate after a 12 months follow-up to show a possible reduction in the number of compliant trial participants in the mid-term course. In order to identify a maximum number of reports on the primary endpoint "drop out rate" this review departed from standard systematic review strategies, usually based on electronic databases and pre-specified search keywords: A full text hand search in 5 pre-specified subjectspecific journals on cataract and refractive surgery with a major focus on RCTs was conducted instead.
The 5 subject-specific ophthalmological journals were consented by an ophthalmologist with an over 30 years experience in cataract surgery and a medical biometrician with an over 20 years experience in clinical trials in ophthalmology to ensure a sufficiently representative spectrum of RCT reports on refractive surgical procedures The search was restricted to publication dates between 01/2002 and 12/2012; Studies on cataract surgery, lens opacity and lens replacement were included without consideration of various operation techniques, lense types or study results in general. RCTs on the treatment of pediatric patients were excluded. Trials were excluded from the meta-analysis, as soon as they did not report a patient evaluation 6 months after randomisation by design; i.e. "time to event" trials only reporting patient information after a "median or mean" observation time of 6 months after randomisation were not considered for data extraction.

Data Extraction
The results for the primary and key secondary endpoint were extracted from full text of all included studies as well as the following design and report determinants of identified RCTs:  trial specification: title of the study, author(s), journal and year of publication  trial design and report determinants: Involvement of a methodological department (such as a medical statistics unit), involvement of industrial partners and sponsors, allowance for comorbidities in patient samples by design (such as acute primary angle closure, diabetes mellitus, diabetic retinopathy, penetrating keratoplasty or corneal astigmatism), masking level of the intervention (double blinded, patient blinded or open design), multicentric trial implementation.  endpoint raw data: net number of patients randomized into the study and number of patients not available for final analysis / missing after a follow-up period of 6 and 12 months, respectively.
Documentation of the raw data was carried out with the software Excel® (Office 2010 release for Windows®). For all identified studies satisfying the review's inclusion criteria the above information were independently extracted by two parallel reviewers. Subsequently any disagreement in report selection and raw data extraction were resolved by discussion with a third independent clinical trialist. In case of unclear documentation of drop out profiles, the study authors were contacted and requested for completion of the respective missing data information.

Risk of Bias in Individual Studies
To ascertain the validity of eligible RCT reports concerning putative trial-wise bias, the above parallel reviewers independently rated the adequacy of randomization as well as of concealment of allocation, blinding of patients and / or health care providers and / or data collectors and outcome assessors, information on early stopping of trials, the availability of a CONSORT flow chart in the trial report, and transparent information on the statistical sample size calculation. Note that in addition to the recommendation by Liberati et al.
[6] we included the CONSORT and sample size related criteria, as these were considered to be of relevance for the present investigation.

Statistical Analysis
During an exploratory pilot investigation [7] the review team identified only one of the above RCT design characteristics as potentially associated with the order of the RCT-wise 6 months drop out profiles: RCTs allowing for comorbid patient samples by design (and not only for patients exclusively suffering from cataract) showed somewhat smaller drop out profiles. As this finding was not derived by means of a formal meta regression analysis, but was rather presumed before any evaluation, the review team decided to stratify the overall metaanalysis for this possible determinant of drop out profiles. As a consequence, the following primary analysis concept refers to the evaluation of RCT reports not allowing for comorbidity by design; A parallel evaluation for RCTs considering patient samples with comorbidities by design was then performed in terms of an exploratory metaanalysis. Accordingly, the respective meta drop out estimates' 95% confidence intervals were not formally adjusted for multiplicity.
Based on the reported number of patients randomized and the net number of patients available at the 6 months follow-up evaluation of a trial, the trial-wise total drop out rate was estimated by means of an exact 95% confidence interval based on the Binomial distribution. For meta estimation of the total 6 months drop out rate a random effect model assumption was made and the DerSimonian-Laird estimator was used to estimate τ 2 ; The meta drop out rate was then again presented by means of its 95% confidence interval and the underlying total number of patients at recall. Statistical heterogeneity was explored by means of forest plots as well as I² statistics (iterative Paule-Mandel method to estimate the between-study variance); An I² value above 75% was considered to indicate high heterogeneity. Significance testing for heterogeneity was performed by means of Cochran's Q at the nominal 5% level.
Funnel plots were used for graphical presentation of trial-wise drop out rates (in logit scale) in relation to the respective trial size (in standard error scale) to account for asymmetric drop out profiles among trial reports. In case of significant heterogeneity among the identified RCTs' reported drop rates -in this review's setting implying a publication tendency towards trials with rather moderate drop outs -the Duval and Tweedie trim-and-fill method for publication bias was applied: the estimated drop out rate was adjusted for putative underreporting of trials with "larger" drop out profiles, again by stressing the random effects model assumption, to derive a conservative meta drop out rate estimate.
All analyses were conducted with the software Comprehensive Meta Analysis (CMA®) release 2.2.064, Biostat, Englewood 2011.

Study Selection
After completion of the hand search in the above five journals a total of 1.045 study reports were considered for further evaluation of the inclusion criteria; these were fulfilled by a total of 228 prospective RCTs. A total of 184 study reports had to be discarded from this 228 RCT reports' pool, because of missing information on the 6 months drop out rate as the primary endpoint of the meta evaluation. The remaining 44 RCT reports, 35 of them not allowing comorbidities by design, met all eligibility criteria and were included in the meta-analysis ( Fig. 1).
Similarly, a total of 182 study reports had to be discarded from the above 228 RCT reports' pool, because of missing information on the 12 months drop out rate as a key secondary endpoint of the meta evaluation. The remaining 46 RCT reports, 34 of them not allowing for comorbidity by design, met all eligibility criteria and were included in the exploratory meta-analysis ( Fig. 1). However, 5 of these 46 trials were identified as redundant publication reports and therefore discarded from the exploratory evaluation.
A total of 17 trial publications reported valid information on both the 6 and the 12 months drop out rate, and were therefore included in both respective meta-analyses.
The 41 RCT reports for the 12 months metaanalysis involved a total of 4.476 participants: 34 of these studies [11][12][13][14]16,17,20 Fig. 2). A funnel plot showed evidence of asymmetric reporting, which -in this setting -was an indication of publication bias towards studies with high overall compliance (actually due to a large number of studies with drop out rates ≤ 5%). The Duval and Tweedie trim-and-fill method imputed a total of 9 rates and estimated an adjusted 6 months drop out rate of 11.6% (95% CI 7.8 -16.9%).

Drop out Rate: 12 Months Follow-up, no Comorbidities Allowed by Trial Design
Only 1 of the 34 studies reported no drop outs after 12 months [11]; The largest drop out rate was quoted with 42.5% [65]. Random effect model estimation revealed a total drop out rate of 16.3% (95% CI 13.2 -20.0%), but was due to significant heterogeneity (I² = 80.1%; p<0.001, Fig. 4). A funnel plot showed evidence of asymmetric reporting; The Duval and Tweedie trim-and-fill method imputed a total of 3 rates and estimated an adjusted 12 months drop out rate of 17.5% (95% CI 14.1 -21.4%).

DISCUSSION
This meta-analysis sought to quantify the expectable short-and mid-term drop out rates in clinical trials on cataract surgery; by means of 44 RCT reports comprising a total of 11.686 participants a 6 months meta drop out rate of, in general, less than 10% was demonstrated. A total overall rate of 7.8% (95% CI: 5.0 -11.8%) was derived from 34 studies not allowing for comorbidities by design, whereas a total rate of 3.2% (95% CI: 2.9 -3.6%) arose from 9 trials in comorbid samples. The latter reduction is not too surprising: Patients with multiple disease burden can be expected to be more interested in midterm follow-up contacts with health care providers due to the non-cataract burden (in particular, if chronic diseases are involved). The "only cataract" patient, however, will benefit from the intervention quite soon and therefore become less interested in any further recall visits. RCT compliance is therefore expected to be reduced among "only cataract" patients even at the shortterm recall 6 months after surgery. Note, on the other hand, that the above stratification was only based on the overall trial reports information on inclusion / exclusion criteria. It must be presumed, that even a clinical trial excluding comorbidity by design will not exclusively recruit "just cataract" patients, as the "average" cataract patient aged 70-75 years will suffer from internal diseases even without knowing. Therefore it must be presumed, that the 7.8% drop out rate is actually not an "exclusively cataract patient" compliance characteristic, but rather refers to a "usual case mix" sample in RCTs on cataract surgery. Nevertheless, we decided to stratify the analysis for this comorbidity related trial design characteristic, as initial exploratory descriptions revealed it as potentially associated with the drop out profiles [7]. Since none of the other quality indicators considered in our investigation was found formally or plausibly associated with the 6 -12 months drop out profiles, we restricted the stratification to this design characteristic.
Depsite the above impact of "comorbidity" on a trial's drop out profile, we further sought to consider several putative drop out determinants such as demographic background patterns of trial samples or their respective insurance status. Unfortunately we could not stratify our meta analysis for such cofactors, as the underlying original publications did not provide such information on prospective drop out profiles in a stratified manner.

Drop out Profiles in Literature
Searching for similar investigations we found two meta-analyses estimating drop out rates for RCTs in osteoarthritis and allergen immunotherapy, respectively: Gehling et al.
(2011) published [77] meta drop out rates for RCTs on opioid analgesia for osteoarthritis pain and in placebo groups; they calculated drop out rates and odds ratios due to different risk factors such as adverse events and lack of analgesic efficacy. However, since cataract RCTs usually do not allow for placebo controls except for the rare case of "waiting designs" in health care research investigations, we could not derive corresponding characteristics from our investigation. The second study was published by Makatsori et al. [78], who examined drop out profiles in sublingual allergen immunotherapy and found an overall drop out rate of 14% (95% CI 12 -16%). In their publication several trial characteristics were considered such as its size and duration, the age of participants, the number of trial sites and the geographical allocation of the trial. In our investigation we previously examined several trial characteristics as potentially associated with the drop out profile [7], but only the design characteristic concerning sample comorbidities as mentioned above was found relevant for the short-and mid-term drop out profiles. Makatsori [78] found that the number of trial sites was not associated with the drop out rate; we could confirm this observation by stratifying the meta 6 and 12 months drop out rate estimates for the design characteristic "mono-versus multicentric".

Methodological Limitations
Based on the fact, that -even within these design strata -we accumulated data from diverse studies observing several interventions and lense types as well as trial objectives, it would have been unlikely that all studies were equivalent. Therefore we did not assume a common effect size and the random effects model was considered justified for this metaanalysis [76].
However, a possible limitation of the our results could be due to the fact that we did not implement a standard search procedure based on common online databases, but rather decided to perform a full text search based on five prespecified subject-sensitive journals. These journals were chosen by experts with regard to their relevance as high impact outlets for RCTs in cataract and refractive surgery. The advantage of maximum possible coverage for these five journals due to hand research (in contrast to the limited precision of an electronical research) must be contrasted to the possible selection bias caused by our journal selection process. Despite this restriction, however, the full-text hand search approach seemed appropriate for the target "drop out rate": Such characteristics cannot be searched for by means of standard tools such as the MeSH term approach in PubMed®.
Out of 1.045 initially identified RCT reports a total of 228 was found suitable for our purpose by means of the hand-search; an orientating PubMed search using the same search terms such as "cataract surgery" restricted to "clinical trial" publications between 2002 and 2012 revealed a total of 1.099 RCTs to cataract surgery, where only 610 of which were published in the five journals of our hand search. Since some of these 610 studies also refer to pediatric patients and to other follow-up schemes than the 6 month period, a notable loss in precision must be postulated for the electronic search as compared to our hand search. A future investigation will quantify the relative precision of the two search strategies, but will also account for the higher resource requirements of the hand search approach.
A definite restriction of our results arose from heterogeneity in the reported data, in particular for the 6 month follow-up (I² = 90.5% for trials on "only cataract" patients). This high order of heterogeneity can be explained by the large number of studies with very small drop out rates. It can be assumed that studies with (too) high drop out rate were rarely published or could not even be analyzed due to appropriate stopping rules in the Statistical Analysis Plan of the trial. As a consequence the actual drop out rates are a surrogate measure a publication bias -and the meta estimation of a drop out rate must be expected biased, accordingly, for under-reporting of trials. The trim-and-fill method uses an iterative procedure to remove the most extreme small studies from one side of the funnel plot, until the funnel plot is "symmetric". In theory, this will yield an unbiased estimate for the meta estimate under consideration [79]. Fig. 6 shows the funnel plot for the 6 months meta drop out rate correction with nine imputed studies. In that case the Duval and Tweedie trimand-fill method lead to an increased adjusted drop out rate of 11.6% (95% CI 7.8 -16.9%) instead of 7.8% (95% CI 5.0 -11.8%). The funnel plot plots illustrates the statistically significant heterogeneity between the trials (see above) underlying the initial meta drop out rate, but also demonstrates the trial series' tendency towards underreporting of RCTs with "larger" effective drop out rates.
However, as the trim-and-fill approach does not correspond to its usual interpretation of bias reduction in the recent setting, but rather provides a conservative drop out rate adjustment, one must reconsider these two rate estimates from a practical perspective: In the recent setting trim-and-fill accounts for unpublished trials with "large" drop out rates. As such trials would hardly be considered valid and representative, the adjusted rate estimate accounting for such trials might rather introduce a different kind of bias yielding unethical overestimation of sample sizes. In summary, the authors would recommend the initial estimate for net sample size increase during the planning phase for RCTs in cataract surgery.
However, although both above point estimates are of comparable order ("expect about 10% drop outs for a six months observational period"), one also must take the respective upper confidence limits into account: The latter propose an expectable drop out rate of 20% rather than 10%. In practice, trial planning therefore must sensitively consider, whether a "standard" or "worst case" scenario is more appropriate for the actual drop out correction of the intended net sample size.

CONCLUSION
This investigation sought to quantify the expectable short and mid-term drop out rates in RCTs on cataract surgery for optimized clinical trial planning from both a methodological and an ethical perspective. Sample size calculation in RCTs on cataract surgery should account for drop outs of at least 10% during a 6 months and of at least 20% during a 12 months follow-up period.