Comparing Adherence with Best Practices in End-of-Life Care After Implementing the End-of-Life Order Set: A Quality Improvement Project in an Ottawa Academic Hospital

Background: Physicians in acute care require tools to assist them in transitioning patients from a “life prolonging” approach to “end-of-life care,” and standardized order sets can be a useful strategy. The end-of-life order set (EOLOS) was developed and implemented in the medical wards of a community academic hospital. Objective: To compare adherence with best practices in end-of-life care after implementing the EOLOS. Methods: We conducted a retrospective chart review of admitted patients with expected deaths in the year preceding EOLOS implementation (“before EOLOS” group), and in the 12 to 24 months following EOLOS implementation (“after EOLOS” group). Results: A total of 295 charts were included: 139 (47%) in the “before EOLOS” group and 156 (53%) in the “after EOLOS” group, of which 117/156 charts (75%) had a completed EOLOS. The “after EOLOS” group demonstrated more “do not resuscitate” orders and more written communication to team members about comfort goals of care. There was a decrease in nonbeneficial interventions in the last 24 hours of life in the “after EOLOS” group: high-flow oxygen, intravenous antibiotics, and deep vein thrombosis/venous thromboembolism prophylaxis. The “after EOLOS” group demonstrated increased prescription of all common end-of-life medications, except for opioids, which had a high preexisting rate of prescription. Patients in the “after EOLOS” group showed a higher rate of spiritual care and palliative care consult team consultation. Conclusion: Findings support standardized order sets as a good framework allowing generalist hospital staff to improve adherence to established palliative care principles and improve end-of-life care of hospital inpatients.


Introduction
How and where we die matters to patients. [1][2][3][4][5] We know that a high percentage of people die in hospitals. An international study looking at place of death in 45 populations, confirmed that 54% of deaths occurred in hospitals. 6 In Canada, this number is closer to 59%. 7 Generalist physicians, such as family doctors and internal medicine specialists, 8,9 provide the majority of inpatient end-of-life care. Within this group, there are varying levels of expertise 10,11 and managing endof-life in an acute care setting brings additional challenges when compared with dying at home or in a hospice setting. The well-known SUPPORT trial 12 published 30 years ago highlighted shortcomings in communication, high frequency of aggressive treatments, and suboptimal symptom management in hospitalized patients at the end of their life.
In recent years, numerous studies [13][14][15][16][17] reconfirm that in an acute care setting, intensive treatments are often ongoing for patients in their last hours of life. These interventions include parental hydration, intravenous (IV) antibiotics, transfusions, surgery, and life support treatments (e.g., mechanical ventilation), which are nonbeneficial and contribute to discomfort. 17 This is contrary to what patients have defined as a quality death, which involves adequate symptom management and avoidance of prolonging dying phase. 18,19 In a literature review, 19 defining a ''good death'' encompasses respect of preferences for the dying process such as control over the circumstances of the dying and preparation for death (reported in 94% of articles); relieving suffering (81%); and supporting emotional well-being (64%).
Furthermore, studies have demonstrated poor documentation of patients' end-of-life wishes. 12,20,21 Incomplete documentation of goals is linked to inconsistent messaging among team members and to suboptimal care delivery. Assuring that teams focus on patientspecific goals and care plans promotes continuity and satisfaction. 18 To address these gaps in providing quality care for patients dying in acute care hospitals, strategies to support the transition from ''disease modifying treatment'' to ''comfort care'' are needed. Standardized order sets are clinical decision support tools that aim to help physicians prescribe appropriate treatments using a predefined set of recommendations linked to evidence-based practice. 22 They have become common in clinical practice including in palliative care and have proven to increase physician comfort while prescribing less familiar drugs, to improve symptom management by reducing errors [23][24][25] and variability of orders among clinicians. 26 In our hospital in Ottawa, Ontario, the palliative care consult team (PCCT) noted anecdotally that practices around managing end-of-life varied greatly, often dependent on where physicians had trained. 27 The PCCT was approached by physicians, nursing leaders, and the medical chief of staff requesting the development of a standardized order set as a resource for providing end-of-life care. In response, the PCCT reviewed the evidence 21,24-26,28 for using standardized order sets, aware of concerns that order sets have been compared with ''tick box'' exercises. 26 Satisfied, we aimed to develop an order set that integrated best practices for end-of-life care.

Development of the End-of-Life Order Set
Standardized order sets were already common within our hospital culture for conditions such as pneumonia, stroke, and heart failure. In September 2015, the PCCT and a pharmacist developed the end-of-life order set (EOLOS) (Supplementary Appendix SA1). The team was guided by examples of ''comfort measures order sets (CMOS)'' obtained via a literature review, looking specifically for articles that included their copy of the order set. 24,26,[28][29][30] The revision process included regular interdisciplinary group meetings, feedback from our key institutional committees, and consultation with local experts. The EOLOS was also beta-tested by the PCCT before its formal implementation.
The EOLOS is a bilingual (French and English), twosided, paper-based document designed to be completed by a physician. The first page of the EOLOS prompts the initiation of nonpharmacological interventions (e.g., the insertion of a urinary catheter as needed) and cessation of nonbeneficial medical interventions (e.g., IV fluids). There is an option to request a palliative or spiritual care consultation. Physicians then prescribe standard palliative care medications. Certain medications, such as opioids and antipsychotics (haloperidol, methotrimeprazine), with a need for more flexibility around dose and frequency, are completed by the physician. We opted to prefill the dose and frequency for other medications with less variability in their start doses (anticholinergics, midazolam, and lorazepam for seizures).
The second page features ''clinical pearls'' as an educational resource, including suggested start doses for opioids, haloperidol, and methotrimeprazine; indications or rationale for medications (e.g., haloperidol is a potent antiemetic); and tips on various end-of-life interventions such as hydration, oxygen use, and managing secretions.
Three criteria are required for a physician to initiate the EOLOS: (1) a ''do not resuscitate'' (DNR) order; (2) the patient is dying of a life-limiting illness, prognosis hours/days (Palliative Performance Scale £30% 31-33 ); and (3) the care goals have been discussed with the patient or the substitute decision maker.

Aim
To compare adherence with best practices in end-oflife care after implementing the EOLOS in our hospital.

Setting and context
The setting is a 289-bed community academic hospital in Ottawa, Canada, with *140 medicine beds. Hospitalists are responsible for most inpatients at end of life, and the PCCT, comprising a palliative care nurse and physician, is available on weekdays. The hospital transitioned from a paper-based medical record system to an electronic medical system in July 2015, just before the implementation of the EOLOS.

Implementation of the EOLOS
The EOLOS was implemented in September 2015, as part of a multiprong approach to improve end-of-life care at our institution. Along with the EOLOS, we designed education sessions for physicians and nursing staff, as well as an end-of-life booklet for families. The staff physicians, medical learners (medical students and residents), and nurses received separate training sessions from the PCCT, detailed in Table 1. A goal of the educational plan was to integrate ongoing clinical and academic activities, rather than add to the busy workload of clinicians and learners. In addition to formal teaching sessions, informal mentoring (coaching at the bedside; reviewing EOLOS prescription with clinicians 1:1) from the PCCT was delivered to nurses and physicians to assure proper understanding and use of the EOLOS.
Evaluation method This quality improvement project used a retrospective chart review before and after the implementation of the EOLOS. Retrospective chart review We conducted a retrospective chart review of admitted medicine patients with expected deaths that occurred between September 1, 2014, and September 1, 2015 (i.e., 12 months before EOLOS implementation-the ''before EOLOS'' group), and between September 1, 2016, and September 1, 2017 (i.e., 12-24 months following the EOLOS's implementation-the ''after EOLOS'' group). The charts of all deceased patients from the medical wards during the ''before EOLOS'' and ''after EOLOS'' periods were reviewed. An experienced palliative care nurse reviewed the charts and determined eligibility based on the inclusion and exclusion criteria. A death was determined to be ''expected'' if indicated in the progress notes or orders (e.g., ''comfort care,'' ''palliative patient,'' or ''expected death''); a completed DNR order on its own was insufficient. Patients were excluded if they died outside the medical wards. Unexpected deaths on medical wards, deaths occurring when the patient's goals were for active medical management, or charts with privacy restrictions were also excluded. Patients who had received a PCCT consult were not excluded, as this was part of ''usual care.'' The EOLOS was considered completed if it was present and signed by a physician in the patient's chart. Not all sections were required to be ''checked off'' for them to be deemed completed, leaving room for individual patient care needs and physician practice.
Data collection and evaluation. A data extraction spreadsheet template was created by a palliative care nurse and physician. After piloting the template on the first 20 charts, iterative changes were made. Data were extracted from both paper charts (''before EOLOS'' group) and patients' electronic medical records (''after EOLOS'' group).
This palliative care nurse and a medical student (trained by the palliative care nurse) extracted the following data: (1) sociodemographic data, including age, sex, date of death, and location before hospital admission; (2)

Analysis
Descriptive statistics were calculated for all variables. Continuous variables were presented as means and standard deviations. Categorical variables were presented as total numbers and percentages. Sociodemographic and characteristics were compared for differences between the before and after EOLOS implementation groups. Statistical analysis included unpaired t-test for continuous variables and chi-square for categorical variables to determine the significance of means and the significance of group characteristics. A post hoc pairwise z-test with Bonferroni correction was performed for significant chi-square results, with more than two categories, to determine which percentages differed significantly from each other. Data were analyzed using IBM SPSS Statistics for Windows, Version 22. 34 A p-value of 0.05 indicated statistical significance.

Results
A total of 295 charts met the inclusion criteria, with 139 (47%) in the ''before EOLOS'' group and 156 (53%) in the ''after EOLOS'' group. Numbers of charts were slightly lower when looking at nonbeneficial medical treatments 24 to 48 hours before death (132 ''before EOLOS'' and 130 ''after EOLOS''), reflecting the fact that some patients arrived at hospital with death imminent, and died within 24 hours of admission. There was no significant difference between patients' age, sex, or location before hospital admission. Primary diagnosis (e.g., cancer; respiratory [e.g., COPD]; cardiac [e.g., congestive heart failure]) between the groups was similar, apart from an increased number of ''other'' conditions and a decreased hematological diagnosis in the ''after EOLOS'' group (Table 2). In the ''after EOLOS'' group, 117/156 charts (75%) had a completed EOLOS. Our team opted to analyze the data of the ''after EOLOS'' group jointly (patients with and without EOLOS), as this represents the systemic impact of EOLOS as a whole.
In comparing the two groups, we noted that documentation of DNR increased (from 90% to 100%, p ‡ 0.001). The evidence of clear documentation communicating comfort care and expected death increased from 72% to 92%, p ‡ 0.001.
There was a statistically significant decrease in the following nonbeneficial interventions in the last 24 hours of life in the ''after EOLOS'' group: high-flow oxygen (decreased from 18% to 10% of patients, p = 0.036), IV antibiotics (decreased from 23% to 11%, p = 0.005), and deep vein thrombosis prophylaxis (decreased from 19% to 10%, p = 0.038). Considering the last 48 hours of life, the only significant reduction between the ''before EOLOS'' and ''after EOLOS'' groups was seen in the use of IV fluids (from 24% to 13%, p = 0.018).
Patients in the ''after EOLOS'' group showed a higher rate of consultation requests for spiritual care (from 41% to 62% p < 0.001) and to the PCCT (from 57% to 74%, p = 0.002; Table 3).

Discussion
Our project aimed to compare adherence with best practices in end-of-life care, 28 after implementing the EOLOS in an academic hospital; in particular, communication of a ''comfort'' approach to team members, discontinuation of nonbeneficial medical interventions, and prescription of standard symptom management medications. These principles were chosen to reflect priorities outlined in the literature regarding patients' preferences at their end of life. 18,19,35 Consistent with previous authors, 21,23,28 we found an overall improvement in the documentation of care goals, discontinuation of nonbeneficial medical interventions, and the prescribing of common symptom management medications in the last days of life. First, the outcome of clearer communication regarding care goals should result in the delivery of more consistent messaging to family, care that is more aligned with the patient's wishes, improved continuity, and overall satisfaction. 36 Acknowledgment that the patient is at end of life might allow timely exploration of the patient and family's preferences surrounding the dying phase, preparation for the death, and improved emotional support-aspects highlighted as important for a ''good death.'' 19 Second, we found that the rate of nonbeneficial medical interventions decreased after initiating the EOLOS, mainly in the last 24 hours of a patient's life. This may indicate that earlier signs of imminent death (e.g., in the last 48-72 hours of life) remain poorly recognized within acute care hospitals. A recent study 37 concluded that the majority of providers caring for hospitalized patients at the end of life felt that comfort care was initiated too late, resulting in moral Each superscripted letter indicates a subset of group categories whose column proportions do not differ significantly from each other at the 0.05 level. Within each row, percentages that do not share a superscript are significantly different (indicated in bold). Other: Mostly refers to patients with extensive comorbidities where no primary diagnosis could be clearly established. distress for health care providers. It is important to acknowledge that a limitation of the EOLOS is that imminent death must be recognized for it to be useful.
Third, we demonstrated an improved prescription of standard end-of-life medications after implementing the EOLOS; this may have led to improved symptom management. However, actual medication administration or its effects were not part of the scope of this study. The increased rate of palliative medication prescriptions was significant in every category, except for opioids, which was not significant due to a high preexisting rate of opioid prescription at the end of life. This differs from prior studies that demonstrated improvements in opioid prescription with the use of CMOS. 28 Regarding antipsychotics, our findings are consistent with a recent study evaluating the pharmacological management of terminal delirium in dying patients, demonstrating that patients with a CMOS in place were more comfortable than those without. 38 Another study 26 demonstrated a decreased need for medication adjustment for end-of-life symptoms when a CMOS was used. Therefore, CMOS, along with other institutional strategies, likely contribute to more appropriate pharmacological management at the end of life. [23][24][25][26]28,29,38 Notably, one year after the EOLOS's introduction, 75% of patients with an expected mortality rate were prescribed the EOLOS, which indicates excellent uptake at our institution. We opted to analyze the data of the ''after EOLOS'' group jointly, without separating the patients without completed EOLOS, as this represents the systemic impact of the EOLOS on our institution. For example, even if a patient was not directly prescribed the EOLOS, they might have benefited from the teaching the provider had received.
As other studies have found, 26,28,39,40 standardized order sets can be useful tools in promoting systemic change within hospitals. The 75% uptake in our hospital contrasts with previous studies where 20% to 67% of those referred to palliative or end-of-life care had standardized orders initiated. 26,29,30 This success might be explained partly by the culture of our hospital, where order sets are common practice, and by the fact that the providers themselves identified the need for the EOLOS. The initial implementation strategy and education program, as well as ongoing education and mentoring from the PCCT, certainly contributed. In fact, we noted that the EOLOS was quickly embraced by the nursing staff, who strongly advocated its use.
From this, we extrapolate that the EOLOS gave the nurses clear messaging regarding care goals, allowing them to take charge of the end-of-life care plan while giving them the tools to manage the expected symptoms. This points to the EOLOS's significant impact on end-of-life care in our hospital. Sustainability Education sessions are ongoing to ensure that all cohorts of trainees receive proper EOLOS training, and new medical staff are provided with informal, individual teaching regarding the EOLOS by the PCCT. All family medicine residents assigned to our institution receive a 1.5-hour lecture by a palliative care physician on the EOLOS. Newly hired nurses are taught separately by the palliative care nurse during their orientation training (60-minute session; approximately eight times per year).
With our institution's transition to electronic prescriptions, the EOLOS was easily adapted to fit our electronic medical records.

Project strengths
The strengths of this project include its considerable sample size, multiple endpoints, and longitudinal sampling (performed over one year for each group). Another strength was the multimodal intervention, which included the EOLOS and ongoing education, aimed at both physicians and nurses. Finally, the chart eligibility was determined by one person with expertise in the matter (palliative care nurse), and the data extraction was performed by only two people, assuring consistency.

Project limitations
This project was a retrospective study in a single center. We did not evaluate whether the prescription of medications for symptom management translated into improved symptom control. Furthermore, we evaluated communication among the care team but not among patients or their families. Our hospital's transition to electronic medical records in July 2015 led to a slightly higher number of digitized medical charts meeting the inclusion criteria in the ''after EOLOS'' group.

Conclusion
In conclusion, this quality improvement project supports using standardized order sets in palliative care. Although they are not a substitution for quality, individualized palliative care, standardized order sets represent a framework allowing generalist hospital staff to improve adherence to established palliative care principles. This project was not designed to evaluate patient outcomes but does point to improved patient care at the end of life. Future work should further evaluate the impact of the EOLOS on symptom management and whether it does indeed contribute to a more peaceful death and greater family satisfaction.