Types of studies

This review will include randomized controlled trials (RCTs), controlled clinical trials (CCTs), controlled before‐after (CBA) and interrupted time series (ITS) designs. As surgical consolidation is a major, hospital‐ or system‐wide intervention, we anticipate that few if any RCTs will exist, with most of the evidence coming from ITS or CBA studies. In keeping with EPOC criteria, ITS studies must include at least three points before and three after the intervention; we will not include simple pre‐post designs. CBA studies must include at least two intervention groups and at least two comparable control groups, with contemporaneous data collection. In addition, the review will exclude studies (or outcomes) that feature only a “post” measure and no comparator, studies that do not directly test an ACS model (e.g. those investigating which tasks can be performed by various types of surgeons), or process/implementation evaluations without a quantitative outcome evaluation. We will not include qualitative studies of staff or patient perspectives in this review, but we will flag these for potential use in a future review.

Both published and unpublished sources, written in any year and in any language, are eligible for inclusion.

Types of participants

Patients. The focus will be on patients (of any age) who require non‐elective surgery. Studies may include all such patients or only a subset thereof (e.g. patients with appendicitis). We will also include studies that consider the ACS model's effects on other segments of the surgical patient population (e.g. elective).  Only studies of actual patients are eligible, not simulation studies.

Providers. We will consider program impacts on any healthcare professional, including but not limited to surgeons, other physicians, residents, nurses, and allied health professionals.

Settings. We will include studies that take place in hospitals or multi‐hospital systems. While we suspect that this topic is most likely to have been studied in developed countries, studies conducted in any country are eligible for inclusion.

Types of interventions

In order to include all potentially relevant studies, this review will consider any model in which all emergency or non‐elective surgery is assigned to designated person(s) or place(s). This includes the new role, trauma‐service expansion, dedicated emergency OR, and regionalization interventions described above, or any combination thereof. Should other relevant models be identified in the course of the literature review, we will include these also. On the other hand, we will not include the creation of a trauma system or centre and the ring‐fencing of elective surgery. Whereas a dedicated emergency OR is an eligible intervention, this review is not designed to assess models of scheduling emergency surgery in general (e.g. blocking).  Although the new role for acute‐care surgery is sometimes called the “surgical hospitalist,” we note that this review does not concern hospitalist physicians who are not surgeons (nor their interactions with surgeons who are not hospitalists). 

Depending on the variant of the ACS model, eligible interventions may consolidate both non‐trauma and trauma surgery, or may be limited to non‐trauma surgery. We will not include interventions that separate out only one type of surgery (e.g., through a centre of excellence) ‐ although, as noted above, we will include studies that measure outcomes for only one type of patient. 

Types of outcome measures

Electronic searches

Electronic Sources

We will search MEDLINE (OVID), EMBASE (OVID), CINAHL (EBSCO), and Dissertation Abstracts for potentially relevant sources. We have provided the proposed search strategy, which includes both MeSH keywords and free‐text operators, below. We will also undertake a narrower Google search for evaluation reports, and browse websites that may be sources of relevant grey literature (including the websites of national and international surgical organizations and other sites identified through CADTH’s “Grey Matters” checklist). Additionally, we will search the Cochrane EPOC Group Specialised Register (See SPECIALISED REGISTER under GROUP DETAILS). (See Appendices 1 and 2 for MEDLINE and Google search strategies.)

Searching other resources

We will attempt to identify additional as available by:

• canvassing review team members’ personal collections and contacts;

• e‐mailing all corresponding authors who have published an eligible article in the past five years, to ask whether they also know of unpublished studies;

• examining the reference lists of included studies and any relevant reviews identified during the search.

Selection of studies

We will use the inclusion/exclusion criteria described above to develop a screening guide, which we will pilot to ensure that the criteria are clear to, and can be consistently applied by all screeners. Two team members (SK and ES or PB) will independently screen each record; we will retrieve all reports that we deem potentially relevant.  At least two authors will screen each article; we will resolve disagreements by consensus, and where necessary, in consultation with other members of the team. In the event that a member of the review team is an author of a potentially eligible report, we will not involve this person in its screening, assessment, or data extraction.

The author who reads all of the reports (SK) will seek to identify any duplicate publications or sister studies. We will combine reports with the same intervention and participants and analyze them as a single study. We will analyze reports that concern different participants separately, but we will compare information about the intervention, timeframe, and context across such reports, and note any discrepancies.

Data extraction and management

We will develop a data extraction template, following a modified version of the EPOC data collection checklist, and the full team will review before use. Two team members will independently extract data from each report (SK and one other; studies will be randomly divided among team members). We will extract the following types of data.

• General information about the report (year, country, author, any duplicate or sister publications)

• Description of the context (e.g. type of hospital or system, stated reason(s) for introducing the intervention, including any theory behind model selection)

• Description of the intervention (components; which model(s) it corresponds to, if any)

• Classification of the intervention according to the EPOC taxonomy

• Description of any co‐interventions or concomitant organizational changes

• Information about study design, methods, and quality (as per EPOC Data Collection Checklist)

• Information about the patient and staff populations involved (as per EPOC Data Collection Checklist)

• Information about the comparator(s)

• Information about outcomes (what specifically was measured, when it was measured, and appropriate measures of effect size and variability)

For both data extraction and risk of bias assessment (see below), we will resolve factual differences between the two authors’ accounts by re‐checking the source. We will resolve any differences of interpretation by consensus, and where needed, in consultation with other team members, the contact editor, or both.

Assessment of risk of bias in included studies

Using the EPOC Risk of Bias Guidelines for experimental and quasi‐experimental studies, two reviewers will independently assess risk of bias while they are doing data extraction.  

Apart from ensuring that study designs meet the EPOC criteria for inclusion, we will not exclude any studies on the basis of methodological quality. We will provide a Summary of Findings table specific to the risk‐of‐bias assessment, including at least five columns: design issues (e.g. randomization lacking/improper); participant issues (e.g. uncontrolled or unknown differences in patient populations between groups or time periods); data quality issues (e.g. missing data, un‐blinded assessment); analysis issues (inappropriate analysis that could not be redone); and confounding events (e.g. other changes occurring during an ITS study). We will consider risk of bias when exploring heterogeneity (see below), and will undertake sensitivity analysis, if suitable, excluding studies with high risk of bias.

Measures of treatment effect

We will report outcomes in natural units. For between‐groups analyses, we will present the mean difference for continuous measures (using standardized mean difference if the outcome is not measured in comparable units, which might occur for some of the secondary outcomes), and odds ratio for dichotomous measures. 

For ITS analyses, we will present effect sizes of both the change in level after the intervention, and change in slope of the regression line.  Level effects will be assessed at 12 months, 24 months, and 36 months. If the report features an inappropriate analysis, or no analysis, we will reanalyze the data where possible. In a short ITS design (fewer than 20 points before the intervention), the analysis would control for the linear effect of time and, where possible, for seasonality; in a long ITS design, it would also control for autocorrelation. We will annotate any studies that we have reanalyzed.

For all non‐randomized studies, we will also record any adjusted measures of intervention effect, and what variables were controlled to obtain them. Some potential confounders that might be controlled include patient age, sex, type of condition, and disease or injury severity; and other organizational changes or interventions implemented during the study period. If there is more than one adjusted measure, we will record the one that incorporates the greatest number of variables listed in the preceding sentence.

Unit of analysis issues

Some cluster‐randomized trials or CBA studies may mistakenly treat the individual as the unit of analysis, without taking into account that assignment to treatment conditions was done at the group level. Under such conditions, we will use the intraclass correlation coefficient (if reported or available from the authors) to re‐estimate P values and confidence intervals. If this information is not available, we will note that the study contained a unit of analysis error, and suppress the P value and confidence interval. (Note that we are not planning a meta‐analysis, so it will not be necessary to calculate the effective sample size.)

Dealing with missing data

We will contact authors by e‐mail where information in their report(s) is incomplete or unclear.

Where particular statistics are not reported, we will calculate standard deviations, confidence intervals, and P values from other available statistics, and odds ratios from the numerator and denominator provided. If a study does not report the numerator and denominator for binary outcomes, but only percentages, we will impute the most plausible numerator and denominator from the exact percentage values reported. (For instance, a value that rounds to 12.3% would be possible with some denominators but not others.) We will not use any other forms of imputation. We will convert graphical results to numbers by measuring the distances on the graph (see Rotter 2010). We will identify any results that we have reanalyzed.

Where outcome data are missing for some participants, we will use an available case analysis, and conduct sensitivity analysis as needed (e.g. for a dichotomous variable, by assuming that missing participants had the same rate of events as control group participants). In the course of risk of bias analysis, we will flag studies with non‐trivial (more than 5%) or unbalanced rates of missing data.

Assessment of heterogeneity

If there are at least three studies that measure a primary outcome in a comparable way, we will inspect a forest plot to gain a sense of the heterogeneity of estimates. Some predicted sources of heterogeneity include type of model, type of study design, and base rate of the outcome (i.e. how much room there is for improvement), hospital or system characteristics (e.g. teaching versus community hospital), and risk of bias.

Assessment of reporting biases

We will attempt to mitigate the effects of publication bias through efforts to locate unpublished studies and evaluations. If there are at least 10 studies that report on a primary outcome, we will inspect a funnel plot for evidence of potential publication bias. Since non‐randomized studies typically do not have a registered protocol (and may not have a protocol at all), it will be difficult to assess selective reporting. However, we will note (a) whether any outcomes are mentioned in the methods section but not the results section, or vice versa; and (b) whether any accompanying information (e.g. the authors reply in a discussion) reveals that certain analyses were not reported.

Data synthesis

We will analyze each of the four (or more) ACS models separately in the first instance, as the outcomes of one model may not be generalizable to another. We will categorize studies in terms of the model they most closely reflect; this decision may depend both on the intervention itself and on the focus of the study (e.g. participants, comparator). For example, an intervention to create a new acute‐care surgery program that incorporates an existing trauma service might be categorized as “new role” if the study focuses on how the presence of a designated surgeon affects non‐trauma patients, but as “expanding a trauma service” if the study focuses on how trauma patients fare when their surgeons take on additional responsibilities.  We will analyze randomized studies, if any, separately from non‐randomized studies.

We will present characteristics of included studies and a summary of findings table format (along with a list of excluded studies). As this topic area does not lend itself to RCTs, we do not anticipate carrying out a meta‐analysis. Where more than one study reports on the same outcome, measured in the same way, for the same model, we will present the median and range of effect sizes as an alternative to “vote‐counting.” We will synthesize findings narratively, noting the overall pattern of findings (within and across the model types) and quality of evidence (taking into account the risk‐of‐bias assessment). We will identify the intervention components and other factors (e.g. hospital characteristics) that appear to be consistently associated with positive or negative outcomes, while noting that such observations must be treated as hypothesis‐generating, not hypothesis‐confirming. In the event that we find few or no studies with eligible designs, the review will provide some discussion of the characteristics and findings of studies with ineligible designs. The conclusion will assess the extent to which there is adequate evidence for or against the acute care model(s), and identify areas requiring future research.