Difference in surgical outcomes of rectal cancer by study design: meta-analyses of randomized clinical trials, case-matched studies, and cohort studies

Abstract Background RCTs are considered the standard in surgical research, whereas case-matched studies and propensity score matching studies are conducted as an alternative option. Both study designs have been used to investigate the potential superiority of robotic surgery over laparoscopic surgery for rectal cancer. However, no conclusion has been reached regarding whether there are differences in findings according to study design. This study aimed to examine similarities and differences in findings relating to robotic surgery for rectal cancer by study design. Methods A comprehensive literature search was conducted using PubMed, Scopus, and Cochrane CENTRAL to identify RCTs, case-matched studies, and cohort studies that compared robotic versus laparoscopic surgery for rectal cancer. Primary outcomes were incidence of postoperative overall complications, incidence of anastomotic leakage, and postoperative mortality. Meta-analyses were performed for each study design using a random-effects model. Results Fifty-nine articles were identified and reviewed. No differences were observed in incidence of anastomotic leakage, mortality, rate of positive circumferential resection margins, conversion rate, and duration of operation by study design. With respect to the incidence of postoperative overall complications and duration of hospital stay, the superiority of robotic surgery was most evident in cohort studies (risk ratio (RR) 0.83, 95 per cent c.i. 0.74 to 0.92, P < 0.001; mean difference (MD) –1.11 (95 per cent c.i. –1.86 to –0.36) days, P = 0.004; respectively), and least evident in RCTs (RR 1.12, 0.91 to 1.38, P = 0.27; MD –0.28 (–1.44 to 0.88) days, P = 0.64; respectively). Conclusion Results of case-matched studies were often similar to those of RCTs in terms of outcomes of robotic surgery for rectal cancer. However, case-matched studies occasionally overestimated the effects of interventions compared with RCTs.


Introduction
RCTs are currently considered the standard for studying treatment effects in surgical research 1,2 . However, RCTs require considerable resources such as time, resources, costs, and collaboration among various specialists to ensure patient security, standardization of interventions, and data correctness. Although blinding is an important design feature of RCTs, blinding of outcome assessors, as well as for patients and surgeons, is difficult to achieve in surgical research, making it difficult to conduct high-quality RCTs 3,4 . Moreover, it is often impossible to conduct surgical RCTs for various reasons, such as feasibility and ethics 1 . Thus, findings from high-quality RCTs are not always available in surgical research 5 .
Recently, matching methods such as propensity score matching have been adopted as alternative methods to randomization. A number of studies using matching methods have been published, and such studies are generally referred to as casematched studies 1,[6][7][8] . However, only measurable confounding factors can be adjusted for in case-matched studies, and reports of such studies occasionally lack sufficient details of matching variables and patient characteristics [9][10][11][12] .
Both high-and low-quality RCTs and case-matched studies have been published. Apart from methodological differences between the two types of study, such as patient selection and adjustment for confounders, it remains unclear whether there are differences in results by study design 2,13 .
RCTs and case-matched studies have been conducted to examine the potential superiority of robotic surgery over laparoscopic surgery for rectal cancer, a topic of major interest among surgeons. However, no conclusion has been reached regarding whether differences exist by study design. On this basis, the present study aimed to examine similarities and differences in findings related to surgical outcomes for rectal cancer according to study design.

Methods
Eligible studies were those comparing robotic versus laparoscopic surgery for rectal cancer. Studies of transanal surgery were excluded. RCTs, case-matched studies, and cohort studies were subjected to analysis. Both prospective and retrospective studies were included in non-RCT studies. No restrictions were placed regarding methods of randomization or matching.
A comprehensive literature search was conducted on 12 June 2019 using PubMed, Scopus, and the Cochrane Central Register of Controlled Trials (CENTRAL). The following search terms were used: 'rectal cancer', 'surgery', 'robot', 'laparoscopy', and related terms (Appendix S1). Duplications were excluded by checking author names, year of publication, and study characteristics (such as study design, setting, and period). Two authors independently screened the extracted publications according to title and abstract, and then reviewed the full text of potentially eligible articles. Disagreement was resolved by discussion.
Data extracted included: study design and setting, number and characteristics of patients, type of surgery, and short-term surgical outcomes. The extracted data were checked for consistency, and discordance was resolved by discussion. For cohort studies, unadjusted data were extracted.

Outcome measures
Primary outcomes were: incidence of postoperative overall complications, incidence of anastomotic leakage, and mortality. Secondary outcomes were: duration of hospital stay, conversion rate, duration of operation, estimated blood loss, rate of positive circumferential resection margins, and quality of total mesorectal excision.

Statistical analysis
Data synthesis was performed using Review Manager 5.3 (The Nordic Cochrane Centre, Copenhagen, Denmark). A randomeffects model was used for all meta-analyses, as all types of rectal cancer surgery were included in the present review. An inverse-variance method was used for continuous variables, and the Mantel-Haenszel method for dichotomous variables. Mean difference (MD) with 95 per cent confidence interval was used for continuous variables when a single measure was included in the meta-analysis. Median (range) values were converted to mean(s.d.) 14 . Risk ratio (RR) with 95 per cent c.i. was used for dichotomous variables. When an outcome was rare, risk difference (RD) was used instead of RR. P < 0.050 (2-sided) was considered statistically significant.

Results
The literature search yielded 1091 articles in total. Among these, 426 duplicates were removed, and the remaining 665 articles were screened for eligibility based on title and abstract. After screening, 67 articles were subjected to full-text review, and 59 articles that met the inclusion criteria were included in the present review (Fig. 1)   . Reviewed studies included seven RCTs, 13 case-matched studies, and 39 cohort studies; two were conducted internationally, and 57 were reported from 17 countries. All casematched studies were retrospective. Among the 13 case-matched studies, propensity score matching was used in seven, and manual matching in one; no matching method was described in five. Variables used for matching included patient age, sex, comorbidity, tumour location and stage, and surgical procedure. Among cohort studies, one was prospective and 38 were retrospective ( Table 1).

Incidence of postoperative overall complications
Forty-five studies involving a total of 8390 patients (6 RCTs, 895 patients; 9 case-matched studies, 2582 patients; 30 cohort studies, 4913 patients) reported on the incidence of overall complications and were included in a meta-analysis stratified by study design. The incidence of overall complications did not differ significantly between robotic and laparoscopic surgery in RCTs (RR 1.12, 95 per cent c.i. 0.91 to 1.38; P ¼ 0.27) and case-matched studies (RR 1.01, 0.89 to 1.15; P ¼ 0.88). In cohort studies, however, robotic surgery was associated with a significantly lower incidence of overall postoperative complications compared with laparoscopic surgery (RR 0.83, 0.74 to 0.92; P < 0.001) ( Table 2 and Fig. 2).

Duration of hospital stay
Thirty-nine studies involving a total of 7651 patients (6 RCTs, 781 patients; 8 case-matched studies, 1904 patients; 25 cohort studies, 4966 patients) that reported on duration of hospital stay were included in a meta-analysis stratified by study design. Duration of hospital stay did not differ significantly between robotic and laparoscopic surgery in RCTs (MD -0.28 (95 per cent c.i. -1.44 to 0.88) days; P ¼ 0.64) and case-matched studies (MD -0.59 (-1.18 to 0.00) days; P ¼ 0.05). In cohort studies, however, robotic surgery was associated with a significantly shorter hospital stay than laparoscopic surgery (MD -1.11 (-1.86 to -0.36) days; P ¼ 0.004) ( Table 2 and Fig. 3).

Conversion rate
Fifty-three studies involving a total of 9813 patients (6 RCTs, 803 patients; 11 case-matched studies, 2976 patients; 36 cohort studies, 6034 patients) that reported on conversion rate were included in a meta-analysis stratified by study design. Conversion rate did not differ significantly between robotic and laparoscopic surgery in RCTs (RR 0.42, 95 per cent c.i. 0.17 to 1.03; P ¼ 0.06). On the other hand, robotic surgery was associated with a significantly lower conversion rate than laparoscopic surgery in case-matched studies (RR 0.40, 0.31 to 0.51; P < 0.001) and cohort studies (RR 0.34, 0.24 to 0.49; P < 0.001) ( Table 2 and Fig. S3).

Duration of operation
Forty-two studies involving a total of 7792 patients (six RCTs, 803 patients; seven case-matched studies, 1644 patients; 29 cohort studies, 5345 patients) that reported on duration of operation were included in a meta-analysis stratified by study design. Duration of operation did not differ significantly between robotic and laparoscopic surgery in RCTs (MD 33.53 (95 per cent c.i. -3.25 to 70.31) min; P ¼ 0.07). However, robotic surgery was associated with a significantly longer operating time than laparoscopic surgery in case-matched studies (MD 83.41 (54.37 to 112.45) min; P < 0.001) and cohort studies (MD 44.70 (32.40 to 57.00) min; P < 0.001) ( Table 2 and Fig. S4).

Estimated blood loss
Twenty-nine studies involving a total of 5783 patients (3 RCTs, 250 patients; 5 case-matched studies, 1095 patients; 21 cohort studies, 4438 patients) that reported on estimated blood loss were included in a meta-analysis stratified by study design.

Rate of positive circumferential resection margins
Forty-two studies involving a total of 8255 patients (3 RCTs, 664 patients; 10 case-matched studies, 2046 patients; 29 cohort studies, 5545 patients) that reported on the rate of positive circumferential resection margins were included in a meta-analysis stratified by study design. The rate of positive circumferential resection margins did not differ significantly between robotic and laparoscopic surgery in RCTs (RR 0.88, 95 per cent c.i. 0.46 to 1.69; P ¼ 0.70), case-matched studies (RR 1.05, 0.70 to 1.57; P ¼ 0.81) and cohort studies (RR 0.84, 0.63 to 1.12; P ¼ 0.23) ( Table 2 and Fig. S6).

Quality of total mesorectal excision
Fifteen studies involving a total of 1585 patients (4 RCTs, 686 patients; 2 case-matched studies, 133 patients; 9 cohort studies, 1585 patients) that reported on the quality of total mesorectal excision were included in a meta-analysis stratified by study design. The quality of total mesorectal excision did not differ significantly between robotic and laparoscopic surgery in RCTs (RR 1.08, 95 per cent c.i. 0.95 to 1.23; P ¼ 0.22) and casematched studies (RR 1.34, 0.74 to 2.42; P ¼ 0.33). In cohort studies, however, robotic surgery was associated with a significantly higher quality of total mesorectal excision than laparoscopic surgery (RR 1.14, 1.01 to 1.28; P ¼ 0.03) ( Table 2 and Fig. S7).

Discussion
The present systematic review and meta-analyses revealed that, among 59 studies that compared robotic versus laparoscopic surgery for rectal cancer, similarities and differences in findings  were observed by study design, particularly between RCTs and case-matched studies. Among the nine outcomes assessed, two (estimated blood loss and quality of total mesorectal excision) were difficult to compare by meta-analyses, as the number of included studies was small and the 95 per cent confidence intervals were wide. LAR, low anterior resection; AR, anterior resection; APR, abdominoperineal resection; HO, Hartmann's operation; PME, partial mesorectal excision; TME, total mesorectal excision; CAA, coloanal anastomosis; ISR, intersphincteric resection; TPE, total pelvic excision; TPC, total proctocolectomy; SSP, sphincter-saving procedure.
With respect to the incidence of anastomotic leakage, mortality, and rate of positive circumferential resection margins, metaanalyses for each study design revealed no significant differences between robotic and laparoscopic surgery, suggesting that findings related to these outcomes did not differ by study design. On the other hand, meta-analyses of case-matched studies and cohort studies, but not RCTs, revealed significant differences between robotic and laparoscopic surgery with respect to conversion rate and duration of operation. However, the number of included patients was lower for RCTs than for casematched studies and cohort studies, and 95 per cent confidence intervals were also wider, suggesting that the statistical power might have been lower. Given the wide range of 95 per cent confidence intervals and lower statistical power, the difference between the three study designs in terms of conversion rate and operating time in the meta-analysis could be considered minimal.
The incidence of postoperative overall complications (primary outcome) and duration of hospital stay (secondary outcome) did not differ significantly between robotic surgery and laparoscopic surgery in RCTs and case-matched studies, whereas significant differences were observed in cohort studies. In-depth analyses of the distribution of 95 per cent confidence across study designs showed that outcomes from case-matched studies fell between those of RCTs and cohort studies in meta-analyses. Specifically, superiority of robotic surgery was most evident in cohort studies, least evident in RCTs, and intermediate (between cohort studies and RCTs) in case-matched studies. These differences by study design might reflect the degree of adjustment for confounding factors between study designs. All confounding factors including measurable and unmeasurable factors could be adjusted for in RCTs, whereas confounding factors in cohort studies were not controlled for in the present meta-analyses because the data were unadjusted.
In this review, the results of meta-analyses did not show differences in most of the outcomes assessed. This is consistent with a previous report 2 that results of RCTs were similar to those of case-matched studies in cardiac surgery. On the other hand, other authors 13 reported that case-matched studies tended to overestimate the efficacy of interventions compared with RCTs in patients with acute coronary syndrome. In the present review, the incidence of postoperative overall complications differed by study design, whereas that of anastomotic leakage did not. Postoperative overall complications include anastomotic leakage and so the rates are higher for postoperative overall complications than for anastomotic leakage. Because the statistical power was greater for postoperative overall complications than for anastomotic leakage, the difference in power might have had some influence. Moreover, although anastomotic leakage can be assessed objectively, other complications such as surgical-site infection and ileus are often influenced by subjective judgements. Duration of hospital stay can also be influenced by subjective judgements because the timing of discharge may depend on surgeon preference. In addition, experimental and comparator interventions are usually performed during the same interval in RCTs, whereas historical comparators are sometimes used in cohort studies. Duration of hospital stay tends to shorten as time progresses owing to the introduction of newer and more effective treatment modalities. In this regard, robotic surgery is a newer technique than laparoscopic surgery. Thus, hospital stay after robotic surgery might be shorter in RCTs than in cohort studies. Clinicians should  57 et al. 58 et al. 61 Panteleimonitis et al. 62 et al. 63 et al. 65 et al. 66 et al. 67 et al. 68 et al. 69 et al. 70 et al. 71 ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) 603 of 2823 53. interpret findings related to these outcomes with caution, and consider the study design when doing so. The strength of the present review is the large number of studies examined. In total, 59 studies were reviewed, compared with 5-23 in previous systematic reviews 74 . Moreover, previous studies that focused on differences by study design often investigated a single outcome for each comparison 5,75 , whereas nine outcomes for a single comparison (robotic versus laparoscopic surgery) were investigated here to highlight differences in surgical outcomes.
However, this study also has some limitations. The numbers of studies and patients differed among the three types of study, and tended to be lower in RCTs. The present review included only published data and did not consider the quality of each study.

Funding
The study received no funding.