Long-term Effects of an Evidence-based Guideline for Emergency Management of Pediatric Syncope

Supplemental Digital Content is available in the text.


INTRODUCTION
Syncope occurs in up to 40% of children. 1 Extensive evidence supports that the majority of syncope is a typical neurally-mediated syncope. Despite the mostly benign nature of pediatric syncope, 2,3 clinicians often obtain low yield, high cost, and resource-intensive diagnostic evaluations in these children. [4][5][6][7] Rare, life-threatening etiologies of syncope, including cardiac disease, may drive clinicians to pursue unnecessary evaluation in the absence of concerning history or physical findings. 8 Wide variation in clinical practice and resource utilization for pediatric syncope exists in the emergency department (ED). 9 Therefore, the pediatric ED staff at Boston Children's Hospital implemented an evidence-based guideline (EBG) for the evaluation and management of pediatric syncope in July 2011. 10 The EBG for syncope is one of several EBGs associated with short-term changes in clinical practices. [11][12][13] In the year following implementation of the syncope EBG, clinicians obtained significantly fewer evaluations not routinely recommended in the EBG and obtained significantly more of the recommended tests. 10 These improvements in guideline adherence and clinical care occurred in the setting of resources dedicated to EBG implementation.
Abstract Introduction: Variability exists in the management of childhood syncope as clinicians balance resource utilization with the need to identify serious diseases. Limited evidence exists regarding the long-term impact of evidence-based guidelines (EBGs) on clinical practices. This study's objective was to measure long-term changes in the management of syncope after implementing a syncope EBG in a single pediatric emergency department following the redistribution of resources to facilitate compliance over time. Methods: We included healthy patients aged [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22] years, presenting to the pediatric emergency department with syncope between 2009 and 2017. Interrupted time series analysis compared testing rates and length of stay among the pre-EBG, short-term follow-up, and longterm follow-up periods. Results: The study included 1,294 subjects. From the pre-EBG period to the long-term follow-up period, recommended electrocardiogram and urine pregnancy test rose significantly [level change odds ratio (95% confidence interval) 5 Limited data exist on the long-term influences of EBGs on clinical practices. 14 The reduction of intensive EBGdedicated resources may affect adherence to the guideline over time. Lack of evidence regarding the long-term effects of EBGs represents a critical gap in knowledge. This study's objective was to characterize the long-term changes in the diagnostic evaluation and management of syncope in children in the pediatric ED following the implementation of an EBG. We hypothesized that in the absence of resources dedicated to the EBG, clinicians would exhibit a decrease in adherence to the EBG recommendations in the long-term follow-up period.

Evidence-based Guideline for Management of Syncope in the Pediatric Emergency Department
In July 2011, clinicians introduced an EBG for the evaluation and management of pediatric syncope in the pediatric ED at Boston Children's Hospital, a large, urban tertiary care children's hospital with an annual volume of approximately 60,000 visits per year. The guideline has been previously published (Fig. 1). 10 Approximately 62 board-certified pediatric emergency medicine physicians, 14 board-certified general pediatricians, 18 pediatric emergency medicine fellows, and over 200 rotating residents from 4 pediatric and emergency medicine residency programs staff this pediatric ED. Other staff include rotating medical students, over 100 nurses, and 40 clinical assistants. The EBG routinely recommends an electrocardiogram (ECG) for all patients and urine pregnancy testing for post-menarchal females. The EBG does not routinely recommend serum laboratory testing, radiographic imaging, intravenous (IV) fluids, medications, or hospital admission. It provides focused recommendations for urgent or elective neurology or cardiology consultation.
A multi-disciplinary collaborative group that included clinicians from emergency medicine, cardiology, neurology, and nursing created the guideline by incorporating the best available evidence from the literature. A pediatric emergency physician and nurse led the iterative process, including 12 revisions over 1 year. Local experts from cardiology and neurology vetted and disseminated the guideline to inform their colleagues and consulting fellows of the collaborative effort optimally.
The introduction of the EBG included paper and electronic copies of the algorithm, and an associated order set in the electronic medical record. The authors of the guideline promoted the EBG at a staff meeting, sent e-mail reminders, and distributed a pocket-card to faculty and trainees at the time of implementation. At this time, a temporary multi-disciplinary team dedicated time and resources to EBG implementation. 10 There were minimal long-term resources dedicated to promoting EBG adherence by clinicians working in the ED. Currently, the Division of Emergency Medicine has 45 guidelines, which have been recently termed "clinical pathways." An attending physician serves as the champion for each clinical pathway. Champions complete a review of quarterly logs to assess for regressions to prior practices and review new relevant literature to inform an annual update if indicated. The champion circulates the annual updates to emergency medicine staff by email or presentation at a weekly departmental educational conference. New providers and trainees complete orientation videos, which includes a highlight of the departmental website containing all of the guidelines, and the associated order sets. The video does not specifically highlight the guideline and order set for syncope. During annual performance evaluations, the division chief provides attending physicians in emergency medicine with metrics on their adherence to selected clinical pathways. The division chief selects different pathways annually and does not inform the physicians of the selections ahead of time.

Study Design
The Committee for Clinical Investigation at Boston Children's Hospital approved this retrospective cross-sectional time-series study. The study included children aged 8-22 years who presented to the ED at Boston Children's Hospital with syncope or pre-syncope from July 2009 to December 2017. The definition of syncope is a transient, brief loss of consciousness associated with loss of postural tone with spontaneous recovery. [15][16][17] Pre-syncope is a near-fainting episode, which may include lightheadedness, dizziness, severe weakness, and blurry vision without loss of consciousness. The study included children with syncope and pre-syncope, given the significant overlap and difficulty in differentiating the 2 at the time of evaluation in the ED.
We identified potentially eligible study subjects from chief complaint codes of syncope or dizziness assigned by the ED triage nurse from a pre-populated list. The study excluded patients who were not well-appearing or had a history of cardiac disease, neurologic disease, or other significant co-morbidity, preceding major trauma, or toxic ingestion. These patients mirror the inclusion and exclusion criteria of the EBG. We also excluded children who underwent diagnostic evaluation at outside institutions before transfer. We applied consistent exclusion criteria across the study periods.

Data Collection and Measures
We performed both chart review and automated abstraction from electronic medical records to collect the following data: demographics, historical information, diagnostic test results [ECG, urine pregnancy test, complete blood count (CBC), serum electrolytes, point of care (POC) glucose, chest radiograph, head computed tomography (CT) scan], cardiology or neurology consultation, IV fluid administration, patient disposition, and length of stay (LOS) associated with each visit. Follow-up data included subsequent visits to the Boston Children's Hospital ED in the 6 months after the initial ED visit or any outpatient visit to the cardiology clinic or neurology clinic, as well as associated subsequent testing, delayed diagnoses, or death. We collected and managed the data using the Research Electronic Data Capture tools hosted at Boston Children's Hospital. 18,19

Statistical Analysis
We divided the study period into the following 3 segments: (1) the pre-EBG period (the 2 years before the introduction of the EBG, from July 1, 2009 to June 30, 2011); (2) the short-term follow-up period (the year immediately following the introduction of the EBG from July 1, 2011 to June 30, 2012); and (3) the long-term follow-up period (the years following the short-term follow-up period, from July 1, 2012 to December 31, 2017).
With descriptive statistics, we characterized the patients' demographic features, using frequencies with proportions and medians with interquartile ranges for categorical and continuous variables, respectively. We used chi-square and Kruskal-Wallis rank tests for categorical and continuous variables to assess differences in patient demographics and diagnostic testing rates and other management decisions across periods, respectively.
We evaluated the impact of the EBG on clinical practices in syncope management using a segmented interrupted time series analysis. We chose this method instead of run charts and statistical process control to measure preexisting trends in clinical practices in the pre-EBG period and assess changes in the levels and trends of those practice outcomes across the different periods. The interrupted time series analysis produced 2 tests of interest: the level change and the slope comparison. A significant level change indicates the presence of a one-time shift in the level of the outcome at the beginning of the post-intervention study period, suggesting a significant change in the occurrence of the outcome at the time of its introduction. A significant slope comparison indicates the presence of a change in the trend over time of the outcome between the 2 study periods, suggesting that the intervention accelerated or decelerated change in the rate of the outcome.
First, we estimated a set of regression models with each clinical practice outcome as the dependent variable and time (measured bi-monthly), study period (pre-EBG period versus the short-term and long-term follow-up periods), and the time-by-study period interaction term as the independent variables. We used logistic models for binary outcomes and negative binomial models for the count outcome (LOS). As described above, these models compared the pre-EBG period to the follow-up period rates over time (ie, comparison of slopes) and the pre-EBG period versus follow-up period intercepts (ie, the level-change), while accounting for pre-EBG trends in the outcome. Then, we repeated these analyses while comparing the pre-EBG period to the long-term follow-up period.
Logistic and negative binomial models produce odds ratios (ORs) and incidence rate ratios (IRRs), respectively, as measures of association. An OR or IRR with a 95% confidence interval (CI) that does not include 1 is statistically significant. A CI greater than 1 for a level change assessment indicates an increase in the outcome rate after the intervention. In contrast, a CI less than 1 indicates a decrease in the outcome after the intervention. A CI greater than 1 for a slope comparison indicates an accelerated trend over time in the rate of the outcome after the intervention. A CI of less than 1 indicates a decelerated trend over time in the rate of the outcome after the intervention.
The clinical practice outcomes comprised of testing recommended by the EBG (ECG and urine pregnancy test for post-menarchal female patients) and testing and management not recommended by the EBG (CBC, serum electrolytes, POC glucose, chest radiograph, head CT, cardiology or neurology consultation, IV fluid administration, and admission). We evaluated the LOS of the pediatric ED visit as an outcome, excluding patients with a chief complaint of mental health diagnosis or homelessness across all study periods.
Finally, we calculated 6-month follow-up rates of testing, delayed diagnosis, and death following the introduction of the EBG. Statistical analyses were conducted using Stata SE, version 14 (Stata Statistical Software; StataCorp, College Station, TX). We calculated all tests as 2-tailed and set α level at 0.05.

Patient Demographics
In total, 3,426 patients aged 8-22 years during the study period presented with a chief complaint of syncope or dizziness, 2,468 of which had syncope or pre-syncope. After applying the inclusion criteria, the study included 1,294 patients (see Figure 1, Supplemental Digital Content 1, http://links.lww.com/PQ9/A218). No significant differences existed between the study periods in patient demographics, including sex, age, race, ethnicity, and insurance type ( Table 1). Table 2 shows the proportion of patients receiving diagnostic evaluations or interventions for syncope stratified by the study period. Rates of urine pregnancy testing and ECGs significantly increased from the pre-EBG period to the long-term follow-up period (P < 0.001 and P = 0.001, respectively). Rates of CBC, serum electrolytes, POC glucose, chest radiograph, IV fluid administration, and head CT significantly decreased (all P values <0.05). Table 3 and Figure 2 show the interrupted time series analysis of diagnostic evaluations recommended by the EBG in the pre-EBG period versus the short-term and long-term follow-up periods. Before the EBG, there was a significant trend in rising rates of ECGs in children with syncope (referent slope OR 1.21, 95% CI 1.05-1.39). Still, there were no significant changes in urine pregnancy testing. In the pre-EBG versus the short-term follow-up periods, there were no significant ECGs or urine pregnancy testing changes.

Diagnostic Evaluations Recommended by Evidence-based Guideline
Rates of ECGs and urine pregnancy testing increased significantly in the long-term follow-up period (level change OR 5.56, 95% CI 1.73-17.91 and 3.15, 95% CI 1.07-9.32, respectively). The trend of obtaining ECGs remained favorable in the long-term follow-up period compared to the pre-EBG period (slope comparison OR 0.85, 95% CI 0.74-0.99).    Table 4 and Figure 2 show the interrupted time series analysis of diagnostic evaluations and management decisions in the pre-EBG period versus the short-term and long-term follow-up periods. Before the EBG, there were no significant trends over time in any outcome. No significant changes occurred in any outcome between the pre-EBG and short-term follow-up periods.

Diagnostic Evaluations and Management not Routinely Recommended by Evidence-based Guideline
In comparing the pre-EBG and the long-term follow-up periods, there were significant level reductions in rates of CBC [level change OR 0. 19

Length of Stay
The median (interquartile range) LOS was 205 minutes (140-288) in the pre-EBG period, 176 minutes (126-251) in the short-term follow-up period, and 170 minutes (118-234) in the long-term follow-up period. Before the EBG, there was a significant trend in decreasing the LOS for patients with syncope (referent slope IRR 0.98, 95% CI 0.96-0.99). A reduction in LOS in the short-term follow-up period versus the pre-EBG period persisted into the long-term follow-up period [level change IRR 0.68 (95% CI 0.52-0.88) and 0.70 (95% CI 0.59-0.83), respectively]. No significant slope differences in LOS existed among the study periods. In comparison, there were no significant differences in LOS for all patients seen in the ED among the study periods.

Balancing Measures
Following the introduction of the EBG, 17 patients (1.7%) returned to the Boston Children's Hospital ED for a similar event within 6 months following their initial visit. Eighty-two patients (8.2%) visited the cardiology clinic, and 31 patients (3.1%) visited the neurology clinic at any time after their initial visit. Twenty-seven patients (2.7%) had echocardiograms, 10 patients (1%) had EEGs, 4 patients (0.4%) had brain MRIs, and 7 patients (0.7%) had laboratory tests. As a result of these encounters following the initial ED visit for syncope, no patients received an alternative serious cardiac or neurologic diagnosis at Boston Children's Hospital. There were no deaths documented in the Boston Children's Hospital electronic medical records.

DISCUSSION
The pediatric ED staff implemented an EBG for syncope in 2011 as a quality improvement intervention. In the following year, relevant testing increased, and low-yield diagnostic studies decreased. 10 The current research suggests that the EBG continued to influence clinical practices for syncope management for 6 years after its implementation, despite the decrease in resources dedicated to adherence. Interrupted times series analysis identified rising rates of ECGs in children with syncope before introducing the EBG, but no other significant trends in testing existed at that time. Attention to high-value testing amidst rising healthcare costs may have influenced local trends in care involving increasing rates of ECGs before the introduction of the EBG. Follow-up data show that despite the reductions in low-yield diagnostic testing, there were no delayed critical diagnoses.
A robust culture of quality improvement in the Division of Emergency Medicine at Boston Children's Hospital and the introduction of EBGs for other conditions in the ED during the study period may have influenced the sustained success of the EBG. The proliferation of guidelines and clinical pathways for other conditions may have contributed to the sustained momentum of clinician adherence. Also, the syncope EBG development team included staff from cardiology, neurology, and emergency medicine, representing the best local practices and potentially promoting acceptance by clinicians. Coincident with this EBG, the Department of Cardiology at Boston Children's Hospital also examined their ambulatory practices in the evaluation of syncope. 20 The division chief's annual evaluation of physician performance, including rotating metrics of adherence to a single guideline as a marker of evidence-based practices, likely promotes adherence to the guidelines. Lastly, escalating national discussion on cost containment in healthcare may have impacted clinical practices.
We did not implement intensive efforts to promote a further reduction in rates of tests or interventions that are not routinely recommended by the guideline, including CBC, serum electrolytes, point-of-care-glucose, and IV fluid administration. We do not target complete elimination of this testing but encouraged low rates of these practices in cases lacking concerning features. Furthermore, the resources required to achieve relatively small additional reductions in already low rates of testing may be dedicated to other quality improvement initiatives.
A key lesson from this study is that pediatric EDs can successfully maintain local clinical guidelines to change practices with minimal, focused efforts in the setting of a robust culture of quality improvement. In the context of prior literature supporting the short-term effectiveness of EBGs, 10-13 our analyses suggest that EBGs are useful in creating sustained changes in clinical practices. Clinicians need simple tools like EBGs to optimize decision-making and support efforts regarding healthcare costs, 21,22 quality improvement, and implementation of research findings into practice.
The strengths of this study include a large sample size and extended length of the follow-up periods. Limitations of this study include the single-center design at a large, urban, free-standing tertiary care pediatric hospital, which may impair generalizability. Additionally, the follow-up assessment did not include return visits outside of this institution. Lastly, we were unable to control the effects of the implementation of other EBGs on the adherence to this guideline.
Further investigation may shed light on the role of EBGs or clinical pathways for other clinical conditions and different practice settings. Ongoing research should ensure that guidelines do not have unintended consequences regarding outcomes, missed or delayed diagnoses, return visits, or other cost implications.
In conclusion, this study demonstrates that an EBG for syncope management in the pediatric ED influenced longterm changes in clinical practice to promote evidence-based decisions and reduce unnecessary testing without unintended adverse effects. The findings provide strong evidence that local EBGs are a valuable tool to change physician practices in a sustained manner. ED clinicians may utilize EBGs as inexpensive tools to promote safe, cost-efficient, and high-quality management of common diagnoses amid rising health costs in the United States.

DISCLOSURE
The authors have no financial interest to declare in relation to the content of this article.