The Impact of On-duty Emergency Medicine Trainees on Left-Without-Being-Seen Rates in an Academic Emergency Department

Objectives: One of the endpoints for assessing the emergency department (ED) performance is the left-without-being-seen (LWBS) proportion. This study aimed to evaluate the impact of increasing proportions of on-duty emergency medicine (EM) trainees on LWBS rates in clinical shifts. Methods: The study was conducted at an urban-academic-ED (annual census: 452,757) over a period of one year. We employed multivariate linear regression (p < 0.05) defining significance to identify and adjust for multiple LWBS influencers related to patient care. Results: After analyzing over 1098 shifts, the median LWBS rate was 8.9% (interquartile range 5.3% to 13.5%). The increasing number of EM trainees in the ED did not adversely impact the LWBS; the opposite was noted. In univariate analysis, the increasing proportion of on-duty EM trainee physicians was significantly (p < 0.001) associated with a decrease in the LWBS rates. The multivariate model adjusted for the statistically significant and confounding LWBS influencers, with an absolute increase of 1% in trainees’ proportion of overall on-duty physician coverage, was associated with an absolute decrease of 2.1% in LWBS rates (95% confidence interval 0.43% to 3.8%, p = 0.014). Conclusions: At the study site, there was a statistically and operationally significant improvement in LWBS associated with partial replacement of board-certified specialist-grade EM physicians with EM residents and fellow trainees.


INTRODUCTION
Over recent years, the government-operated healthcare system in the State of Qatar has evolved from a solely service-based organization to a recognized academic medical center status. Accreditation by the Joint Commission International group as an academic center has been accompanied by substantial evolution of the government's major health provider, Hamad Medical Corporation (HMC). Changes in service model have been most pronounced at HMC's (and the country's) sole tertiary care center, Hamad General Hospital (HGH). Historically, at the emergency department (ED) of HGH, board-certified specialist-grade (SG) doctors provided most hands-on care under consultant supervision. The role of trainees in emergency medicine (EM) training programs, at the HGH ED, has grown from supernumerary ED positions to significant clinical care responsibilities. As of this year, trainees in the HGH Department of EM outnumber the SG physicians. The most significant operation endpoints for an ED operational performance are the time interval from patient presentation to initial physician evaluation (tMD), length of stay (LOS), and the patient proportions of those who leave the ED without being seen (LWBS). 1 -3 There are fewer studies that examined the impact of trainees on LOS for patients presenting at academic EDs and found patients seen by trainees had longer LOS. 4 -6 An initial report from the study center found that tMD was not adversely affected by the increasing trainee proportion of ED physicians. 7 However, literature studying the impact of on-duty trainee proportion on LWBS is limited. 8 LWBS is both intrinsically important and also useful as an end point for the ED operations studies of related endpoints such as tMD. 9 -12 The move from an SG-centered service model to a trainee-centered academic model was recognized from its inception as a natural experiment to study the impact of increasing trainees on the overall ED operations. However, a priori plan for following the transition included an assessment for adverse impact on ED operation endpoints. This study aimed to determine the impact of LWBS associated with increasing trainee numbers as a proportion of overall on-duty physician in an academic ED.

METHODS
The study is a retrospective analysis of administrative data assembled as part of daily routine ED operations. The study anonymized all data. There was no protected health information reviewed or reported, and individual patients were not able to be identified based on the data reviewed. The institutional ethics board of HGH approved the study. The HGH ED is a tertiary hospital for a country with a population approaching 2.7 million. The annual overall ED census for the study year was 452,757. The country's network of pediatric emergency centers constitutes the designated receiving facilities for pediatric (, 15 years) cases that do not involve major or minor trauma. HGH sees all injured children and occasional nontrauma cases that are transported by private vehicle; pediatric cases constitute approximately 10% of the overall volume. Physicians work in three shifts: The day shift runs 06:00-14:00, evening 14:00-22:00, and night 22:00-06:00. There are also a few "mid-shifts" (for example, 09:00-17:00), and the number and timing of which changed throughout the study period. The study regression modeling included adjustment for time shifts. On each shift, there is at least one EM consultant (EMC) in the ED. As of the end of the study period (end of 2016), there were 47 EMCs, which is 11 more than there were at the study period's commencement (early of 2016). The study regression modeling included adjustment for the total number of EMCs on duty for a given shift. Working under the consultants are SG physicians, who are nearly all EM board certified (some have been at HGH for years with primary training in medicine or surgery). Over the study year, the number of SG ED physician numbers dropped from 110 to 70. As the SG physician numbers dropped, the clinical responsibilities of the EM resident (EMR) and EM fellow (EMF) training program participants increased, but the trainee numbers remained relatively stable (that is, program sizes did not increase). The EMR now comprises 40 physicians (10 per year in a PGY1-4 format), and the three-year EMF program had an approximately the same number of participants (the EMF program has a less-rigid entry and graduation calendar, and participant numbers varied slightly during the study year). The EMR is accredited by the Accreditation Council for Graduate Medical Education-International and follows that group's graded responsibility guidelines for trainees. EMRs do not function autonomously; they present all cases to a supervising EMF or EMC. The EMFs essentially function as would western-model PGY5-7 residents. In the initial year or two, they discuss cases with a senior EMF or EMC, and in their final year, they have increased autonomy (although there are always senior physicians available on-hand for supervision). For analysis, EMRs and EMFs were both considered "trainees" as part of a longitudinal seven-year program. Nurses staffing in the study ED were all registered nurses (RNs), and the number of daily on-duty RNs (nearly 100 each day) did not change during the study period. The number of other ancillary staff (for example, X-ray technicians) also remained constant. For the entire study period, physician-staffing numbers were taken directly from the on-duty physician roster on an ongoing daily basis. Because of the ED's transitioning to an electronic health record (EHR) midway through the study period, the operation data source also changed six months into the study period. For the initial six months of operation data, the information source was the ED's Excel-based administrative database (EDAD). The study-site EDAD's hand-executed process of data entry by a team of nurses reading every paper ED chart, as well as the validation of the completeness and accuracy of this process, has been previously reported in detail. 13 Data for the latter six months of the study came from the EHR (Cerner Millennium, Kansas City, MO, USA). The EHR data storage systems allowed for direct data export into an Excel format that was then added to the Excel sheet with EDAD for the initial half-year of study data. The transition from paper charts to the EHR was judged a priori to be sufficiently important to include this dichotomous (before and after EHR) calendar variable in multivariate analysis. The study also recorded other calendar variables that were potentially related to LWBS: study month (ED Facilities section), day of the week, and holiday status. Study ED operations data were collected for each shift (unit of analysis section). Demographic information included age, sex, and nationality reported dichotomously as Qatari national versus expatriate. ED workload was assessed using patient census and the number of ED boarders (that is, inpatient-admitted cases still in the ED because of the lack of other bed availabilities). Shift acuity was assessed by ascertaining the percent of patients who arrived by ambulance (EMS), the percent with the highest two levels of the five-point Canadian Triage Acuity Score, 14 and the percent of inpatient admission. Each shift's performances for time intervals from presentation to triage and tMD were also assessed. From the combined Excel spreadsheet, data were exported to the statistical software Stata (Version 14MP, Stata Corp, College Station, TX, USA). This study used Stata for all analyses, with significance set at the p , 0.05 level and confidence intervals (CIs) reported at 95% levels.

Unit of analysis
The study unit of analysis was the shifts. Each day consisted of three shifts (day, evening, and night). Since the study (leap) year comprised 366 days, the study number of shifts was 1098 (that is, three shifts per day). Because of an HMC-wide EHR crash, there were limitations of data availability for some variables for one study day (December 25, 2015); therefore, the total shift number was 1095 instead of 1098. The outcome variable (percentage LWBS) and the independent variables (for example, age, sex, and physician-staffing number) were documented on a per-shift basis. Data such as patient census were analyzed as "count" data (that is, the number of patients per shift). Continuous parameters such as age were analyzed as the median for the shift and categorical variables such as sex as the percentage for the shift.

Analytic methods
Stata's Shapiro -Francia testing procedure was used for initial tests of normality, with follow-up skewness-kurtosis testing to define reasons for nonnormality. 15 For data identified as normal, mean^standard deviation was used for central tendencies. Median with interquartile range (IQR) was used to report nonnormal variables' central tendency. Univariate linear regression was used to assess categorical data and nonparametric (Kruskal-Wallis) for continuous variables. A univariate cutoff of p , 0.20 was used to define covariates for multivariate modeling assessment. 16 Multivariate analysis entailed execution of linear regression with the dependent variable LWBS as the outcome of interest. Stepwise model-building was used, and as model-building proceeded, effect modification and confounding were assessed with interaction terms and covariate reintroduction. 16 Where robust variance estimators were necessitated by model heteroscedasticity (as assessed with the Breusch-Pagan/Cook-Weisberg test), the Wald test was used to determine the significance of covariates in multivariate regression; the likelihood ratio test assessed covariate significance. This study used a variety of approaches to assess the overall linear regression model performance. Adjusted r 2 was calculated to determine the proportion of overall dependent variable variance accounted for by the model. When heteroscedasticity was identified, robust standard errors were used to optimize the accuracy of b estimates. 17 The link test was used to assess the linear regression model specification. The contribution of individual covariates was confirmed using the nested-model likelihood ratio testing. After model generation, marginal predictive probability analysis was executed with Stata algorithms as outlined by Mitchell. 18 Since marginal analysis cannot be executed on noninteger factor variables, the study's marginal analysis plan employed translation of continuous (percentage) data of trainee proportion of on-duty physicians into quantiles. As recommended for Stata's margin calculations, the contrast command was used to ascertain the significance of the linear trend from one quintile to the next. 18

Descriptive and univariate statistics
The annual overall ED census for the study year was 452,757. The overall LWBS (expressed as a percentage) was a median of 8.9 with IQR (5.3-13.5). Table 1 shows summary statistics for the ED operations data per study shift (n ¼ 1098). Table 1 also shows the univariate p-value for the association between the parameter of interest and the outcome LWBS. Physician-staffing numbers at each level (EMC, SG, and trainee) were all significantly (p , 0.001) associated with LWBS. Table 2 shows these numbers. The number of trainee physician's levels varied substantially. In addition to the physician numbers' IQRs as outlined in Table 2, the number of on-duty trainees for a shift was as low as 0 (on lecture days) and as high as 19. Since the number of SG physicians varied inversely with the number of on-duty trainees (to maintain a stable overall number of on-duty physicians), the proportion of trainees on-duty during a shift ranged broadly, from 0% to 75% (median 23% and IQR 15% -32%).
Calendar-related parameters were also significantly associated with LWBS in univariate testing. Time of day (that is, day, evening, or night shifts), day of the week, study month, and Friday (holy day) status were all significantly (p , 0.001) associated with LWBS. Two other calendar-related markers were also assessed for association with LWBS. The first was "holiday" status. In univariate testing, holiday status was not associated with LWBS (p ¼ 0.615). The second was an indicator of whether a given shift occurred during the EDAD (paper-chart) or the EHR era. The EHR covariate was strongly (p ¼ 0.0001) associated with LWBS in univariate analysis.

Multivariate linear regression: factors associated with LWBS
Inclusion of all covariates, with the removal of those with the least statistical significance (that is, highest p-value), yielded the final models depicted in Table 3.
In Table 3, each covariate's b indicates the direction and magnitude of that parameter's influence on the continuous response variable percentage LWBS. Positive b indicates that increasing the covariate's value increased the percentage LWBS, and negative b indicates an inverse (that is, favorable) association between the variable and LWBS. The null value (that is, no effect) for b is 0. The further the b deviates from 0, the more profound the effect of the parameter's one-unit change on LWBS. For example, if a baseline LWBS is theoretically presumed to be at 5.1%, prolongation of tMD by one minute would be expected to be associated with a 0.1 decrease in LWBS down to 5.0%; an increase of 1% (absolute) in the proportion of trainees working in the ED would be associated with an LWBS drop to 3.0%. Diagnostic testing demonstrated model acceptability. The adjusted r 2 ¼ 0.64 indicates that the model explains 64% of all LWBS variation. Post-estimation evaluation of the model suggested heteroscedasticity (p , 0.001), so Table 3 presents CIs obtained using robust standard errors. The link test was consistent with good model specification (p ¼ 0.876).

Marginal predictive probability analysis
The first step was to divide the percentage trainee numbers into quintiles, and the quintiles were assigned numbers 1 (for lowest quintile) through 5 (highest quintile). After the replacement of the trainee percentage continuous variable with the ordinal (quintile) score, data in Table 3 confirm no major drop-off in model performance. The new model's adjusted r 2 was 0.63 compared with 0.64 in the initial model of Table 3, and the percentage trainee quintile term was significant (p ¼ 0.033), so the new model was judged appropriate for demonstrative marginal analysis. The linear trend depicted in Figure 1 was statistically significant (p ¼ 0.0391). The "hockey-stick" curve reversal between the fourth and fifth quintiles was not statistically significant (p ¼ 0.461). Given the study's null hypothesis (and hoped-for best case finding) of no association between trainee percentage of on-duty physicians and LWBS, there was no a priori plan for the statistical exploration of reasons for any improved LWBS associated with increasing trainee numbers. As a post hoc univariate exploratory assessment, the nonparametric trend test (an extension of the Kruskal -Wallis test) was performed to assess whether there was an association between increasing quintile of percentage trainees and faster tMD. The trend test was significant, (p , 001), but the finding is reported only as a basis for further hypothesis generation and evaluation.

DISCUSSION
LWBS is important ED operation endpoints, because of its objective nature and claimed relationship to care timeliness and quality. 19,20 LWBS is both intrinsically important and also useful as an endpoint for other ED operation studies related to endpoints such as tMD. 21 -23 While not overly clinical in their outcome, operation endpoints such as LWBS are broadly accepted as partial contributors to the assessment of care quality and efficiency in EM. 24 -27 This study set out to reassure the medical center administration and our patients that HGH's transition toward an academic model was not associated with a decrease in ED operation performance as assessed by LWBS. The partial replacement of board-certified SG physicians in the ED with trainees is an important change in the ED operation. Since trainees inherently require more contact and supervision with supervising physicians, there is a logical concern that LWBS could be increased because of movement from an SG-only ED physician model to an academic model in which trainees have significant-but properly supervised-clinical responsibilities. The study failed to find evidence of any increase in LWBS associated with increasing trainee roles in the ED. In fact, there was a surprising finding that LWBS decreased (that is, improved) as the trainee proportion of on-duty physicians increased. The study has shown a significant relationship of LWBS with time to physician that is one factor for patient satisfaction. 28 It also revealed less number of patients, and patients admitted as inpatient also influenced the LWBS. Patients arriving in an ambulance and high-level acuity patients also showed a statistical significance to LWBS. Although these unexpectedly positive results deserve exploration, some limitations to the current study should first be noted. First, the study shares with other administrative and database studies that the characteristics of the results are only as good as the data quality. 29 The initial half of the study used hand-entered operation data in a department-specific database (EDAD). The EDAD has been in place for years and has been previously validated with cross-checking of entries and confirmation of lack of identifiable bias. However, the   30 If the conclusions of the current study are to be considered valid, they are only valid as applied to the study ED. The ED is far busier and has far more physician staffing than most other EDs (academic or nonacademic). Furthermore, the ED's operation parameters such as tMD and LWBS were grossly in line with the findings of other EDs. 21,26 Many EDs boast an LWBS that is far less than that seen in this study. The fact that the study ED population is younger and more likely men (Table 1) than the ED patients seen in many other EDs may also limit external generalization.

Implications of this study
Encouraging and supporting the academic health system model are the best proposed results in other healthcare practices. General apprehension about the training load adversely impacting the healthcare service delivered was not supported by the results. Individual teaching hospitals can use a similar methodology to evaluate the impact of trainees' presence in the ED on the operational parameters. Such data can be useful to gain support from the hospital managers and assure the public about the safety of running an academic healthcare model.

CONCLUSION
With considerations of the limitations enumerated above, the conclusion regarding the current data is that the model and its association marginal analysis provide consistent and robust evidence supporting the conclusion that trainees' proportions do not prolong LWBS. Not only was there a failure to see any adverse impact on the operational parameter of focus, but these data strongly suggested the opposite finding of improved ED operations. Further study should confirm the results, but in the intervening time, the available results support continuing the study center's move toward an academic physician-staffing model.

Declaration of Conflict of interest
There is no conflict of interest among the authors.