Adherence to the cardiac surgery checklist decreased mortality at a teaching hospital: A retrospective cohort study

Highlights • Checklists avoid human errors and are commonly used in high-reliability industries. • The “InCor Checklist” was associated with decreased mortality over time. • Adherence, completeness, and sustainability within public policies are necessary.


Introduction
In the health system, adverse events cause more deaths annually than road traffic accidents, breast cancer, or acquired immunodeficiency syndrome. 1 Global estimates suggest that every year 1 million people die after surgery. A scenario in which 50% of deaths would be preventable. 2 Checklists began to be used in aviation in the 1930s to avoid human errors and are currently common in high-risk industries. 3 In this way, safety checklists have been incorporated as an essential part of a safety culture. 4 However, while surgery has continued to increase, little progress has been made in patient safety.
In 2009, World Health Organization (WHO) led the development of a checklist for surgery. 5 The results were striking showing that the instrument was able to reduce mortality by up to 47%. 6 The benefits seem to be directly related to an improvement in the team's communication and situation awareness, just before starting the procedure. It did not take long for the American and European Society for Cardio-Thoracic Surgery (EACTS) to consider the checklist as a class I recommendation to be applied in all cardiac surgeries. 7,8 The morbidity and mortality after cardiac surgery decreased over time; however, avoidable errors persisted causing undesirable results. It is a complex scenario that involves sophisticated techniques and equipment, cardiopulmonary bypass, several professionals, and especially, high-risk patients. Thus, adverse events can occur due to both communication failures and lack of teamwork. 9 Seeking to build a surgical patient safety culture, the hospital team adapted and structured its safety verification system, the InCor-Checklist "Five steps to safe cardiac surgery". The aim of this publication is to describe the implementation and adherence of the InCor-Checklist over time, as well as its association with the mortality results after cardiac surgery.

Study population, inclusion, and exclusion criteria
In this analysis, 8139 patients who underwent coronary artery bypass grafting and/or heart valve surgery from 2013 to 2019 were analyzed. In total, 5 types of procedures were analyzed: coronary artery bypass graft surgery, aortic valve surgery, mitral valve surgery, aortic valve + mitral valve surgery, and coronary + heart valve surgery. Clinical Characteristics and preoperative parameters of the population analyzed are detailed in Table 1. Therefore, most surgeries were analyzed except for congenital diseases, aortic diseases, cardiac tumors, pericardial diseases, heart transplants, catheter therapies, pacemaker implantation, and patients who had salvage cardiac surgery. Details of outcomes and length of hospital stay of the population analyzed between 2013 and 2019 are detailed in Table 2.

Study design
This is a single-center retrospective analysis performed by the Quality and Safety Surgical Unit (UCQSP). The authors complied with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement. 10 In total, 8139 patients were analyzed. It is worth noting that, in 2015, the Continuous Quality Improvement Program in Cardiovascular Surgery (PMCQ) was established. The public and monthly presentation of the results, the monitoring of the mandatory use of the surgical InCor-Checklist, the establishment of the clinical and surgical outpatient clinic, the root-cause analysis of surgical mortality, the development of Quality and Safety Research and the multidisciplinary discussion of the ideal moment to approach emergency patients are among the actions of this program. Therefore, even though the implementation and monitoring of the mandatory use of the InCor-Checklist was the intervention with the greatest impact within the PMCQ, the authors believe that the results may have also been influenced by the other actions oriented during this period. The main outcome was operative mortality defined as when it occurred during the hospitalization in which the operation was performed, even after 30 days; as well as all deaths that occurred after hospital discharge, but before the end of the thirtieth postoperative day. The EuroSCORE II was used as a reference to assess the risk of expected mortality. The flowchart of the study design is shown in Fig. 1. All patients operated on during the study period were included in the analysis and there were no missing regarding the operative mortality.

Implementation of the InCor-Checklist
In 2014, a questionnaire was applied to assess the need for the use of an InCor-Checklist by the teams (surgeons, anesthetists, perfusionists and nurses) in surgical patients (Table 3), and from this, the InCor-Checklist was built.
This model was established within an educational program composed of standardized classes, teaching materials, videos and simulations in scenarios set up in the operating room. In 2015, the checklist was implemented in the surgical routine to be used in all cardiac surgeries. The idealized model includes 5 sequential phases: Briefing (team planning in relation to the patient and specific surgery), Sign In (before the patient enters the operating room), Time out (before skin incision), Sign out (before the patient leaves the operating room) and Debriefing (report of what happened and how to improve) (Fig. 2). The detailed flow of the InCor Checklist application is shown as supplementary material.
Training with the teams were carried out every six months. Theoretical and realistic simulation classes were included with the purpose of evaluating its applicability, identifying possible adaptations in the process and/or in the instrument, and avoiding the deterioration of the processes.

Statistical analysis
Regarding the observed mortality, the periods were compared using the two-tailed test to compare proportions. Only the annual average of surgeries performed was analyzed. For the variable number of surgeries, the two-tailed Mann-Whitney test was used. Correlation analyzes of the annual mortality with the level of adherence and the degree of completeness of the instrument were performed. To perform the correlation,  Pearson's coefficient was used. The level of significance was set at 0.05. The R software (version 3.5.3) was used for analysis and charts. The Excel program was used to consolidate the original database. As treatment of missing data, the variables "left ventricular ejection fraction", "CABG time" and "anoxia time" had blank observations filled in with the database mean, similarly, in the analysis of the multiple logistic models as continuous variables were filled in with the observed mean.

Ethics and consent form
This project was approved by the Ethics Committee for Analysis of Research Projects (CAAE: 4141821.9.0000.0068). The analysis was made from the institution's database and validated by UCQSP without identifying the patients. For this reason, informed consent was waived.

Results
The mean age of the study population was 60.31±2.09. A total of 5009 (61.54%) patients were male, and 17.16% of the patients underwent urgent surgery. In adults, the most common type of procedure was isolated coronary artery bypass grafting surgery (4146 procedures = 51%).
The average annual mortality was 5.98% (8.22% in 2013, 6.82% in 2014, 7.49% in 2015, 6.5% in 2016, 5.69% in 2017, 4.66% in 2018, and 3.13% in 2019, as shown in Fig. 3 Regarding the completeness of the 5 phases of the InCor-Checklist, it evolved from 3.3 in 2015 to 3.9 in 2019 according to sample evaluations year by year (questionnaires). Whenever the InCor-Checklist was used in the operating room, the phases always performed were: Sign-in, Time out, and Sign out, followed by Briefing and finally Debriefing ( Table 4). The implementation of the checklist was done gradually among the surgical teams, in a strategic way, aiming to stimulate adherence to the system, avoiding fatigue of those involved in the checking system, until this process became a hospital routine.
Over time, the use of InCor-Checklist for these procedures has been progressive, reaching 75% of all cardiac surgeries in 2016 and 100%   ∼ Approximately.
in 2019 in an institution that performed an average of more than 100 of these cardiac surgeries per month during this period.

Discussion
Studies show that adverse events happen more commonly in the operating room (43%) for patients admitted for surgery procedures, and most of them are preventable. 1,11 The present study's results show that the use of InCor-Checklist was associated with a 62% reduction in mortality. The InCor-Checklist was adapted from the WHO model 5 and follows the guidelines of the cardiac surgery societies, 6 and the American Heart Association, 12 with the aim of reducing the risk of human errors.
The InCor-Checklist was 37% more effective than the checklist used by the National Patient Safety Program in Scotland, 13 and less than that achieved by a Boston group in a randomized, multicenter study in which a reduction of 47%, 6 was seen, both in cardiac and non-cardiac surgeries. On the other hand, another study, also performed in non-cardiac surgeries, in hospitals of Ontario, Canada, did not show significance, with a mortality reduction of only 8.5% before and after implementation. 14 Perhaps the study most similar to ours in relation to the 5-year period analyzed was conducted in Australia and had a mortality reduction of 23.3% after implementation of the checklist, regardless of the duration of surgery. 15 An Italian study showed that after implementation there was a 27% decrease in mortality within 90 days after surgery but no difference in relation to the 30-day mortality. 16 However, most of these studies were done in non-cardiac surgeries. In cardiac surgery, little has been published about the impact of the surgical checklist. Perhaps the most influential study is the one recently published, performed in 7 Dutch hospitals, in which the implementation of a checklist was associated with a 43% reduction in mortality up to 120 days after cardiac surgery. 17 The decrease in mortality in the present study was associated with the implementation, adherence, and advancement of the 5-year checklist program. The evolution from high mortality values to values similar to those of international reference centers may explain the 58% reduction in mortality.
Regarding the importance of performing a training program, an analysis showed that hospitals in South Carolina (USA) that completed the checklist program (14 hospitals) had a greater reduction in mortality than other hospitals (44 hospitals). There was a 16% mortality reduction before and after implementation of the checklist while there was no difference in the group that did not participate in the program. 18 As in the present study, adherence to the checklist was also related to leadership involvement, the commitment of the surgical team, and the improvement of communication. 19 Regarding the impact of completeness of the checklist, a study also showed that there is a lower risk of mortality when the 3 main phases (Sign in, Time out and Sign out) were applied. 20 Thus, the influence of the use of the checklist was related to risk management before surgery, teamwork, and communication. 6 Communication mistakes are the most common cause of adverse health events. These mistakes occur because the information does not reach the right person, is inaccurate, or because the problems remain unsolved until they become critical. 21 Therefore, it would not be simply the implantation of one more questionnaire, but the full conviction that the authors are within a system in which the result will depend on the effective communication of the team and between the teams in the different modalities of assistance to the surgical patient. 22 The InCor-Checklist includes Briefing and Debriefing within its flow. This was based on a study in which the use of a structured briefing managed to reduce the average number of communication failures by surgery from 3.95 to 1.31 (p < 0.001). A structured briefing is associated with an improvement in situational awareness, decision making, teamwork, and the reliability of clinical interventions. 23,24 The US Veterans Health Administration training program, 25 that included briefing and debriefing, in addition to the use of the surgical checklist (Sign in, Time out and Sign out), achieved a 50% greater mortality reduction in the trained group than in the untrained group (p = 0.01). Another study also showed that, after adopting briefing and debriefing as a complement, communication improved. This could be the initial step towards a substantial and sustainable organizational transformation. 26 We believe that the construction of a safety culture for surgical patients at the present institution was positive and progressed in a sustainable manner over 5 years, reaching 100% adherence. The impact of the checklist on the formation of the Safety Culture has been positive in previous studies. [27][28][29] Considering that approximately 30%-47% of the complications of the patients admitted to surgery are related to the operating room, a more comprehensive checklist strategy may be ideal. This was assessed by the SURPASS study in the Netherlands and published in NEJM in 2010. 30 This study not only implemented a perioperative checklist, but also considered all phases of the surgical process and combined it in a uniform way from admission to discharge.

Limitations
One of the limitations of this study is that it is an observational and retrospective study based on the institutional database. However, all data were validated by the Informatics Service, the Hospital Medical Information Unit, and the Quality and Safety Surgical Unit. The second limitation is that other actions were also included in the same period within the quality improvement program at the institution, the same ones that may have influenced the results of surgical mortality. 31 For this reason, the authors cannot highlight that there was causality, only association. However, it is undeniable to highlight the correlation between a progressive and sustainable adherence to the checklist and the reduction of mortality. Another limitation of this study is related to the fact that the idealized checklist had no adherence to savage surgery, nor includes crisis situations that may arise during surgeries. A checklist for crisis situations was developed by the Harvard group and resulted in a 6-fold reduction in adherence failure in critical stages in the evaluated scenarios. 32 The impact of this model on our practice will be assessed in further studies.
It seems that it is not only the technical skill but also the surgeon's behavioral patterns and non-technical skills (leadership, teamwork, problem-solving, decision making, and situational awareness) that affect the surgical results. 33 A multicenter study on congenital heart surgery in the United Kingdom indicated that the results were not solely related to the technical difficulties of the surgery, as there was a strong relationship between the non-technical skills of the surgeon and adverse events, including death. 34 Although there are still limitations to the completeness of the 5 phases, the authors can say that the InCor-Checklist achieved your goal: the formation of the safety culture of the surgical patient. Thus, the authors can see that the implantation and sustainability of a surgical checklist are not as simple as it seems; it requires a lot of leadership, humility, and teamwork. 35

Conclusion
In the formation of the surgical patient safety culture, the implementation and adherence to the InCor-Checklist "Five steps to safe cardiac surgery" was associated with decreased mortality after cardiac surgery. The authors recommend that hospitals, in addition to implementing a surgical checklist, should develop strategies to encourage adherence and completeness, as well as sustainability within public policies.
Author's contribution OAVM and FBJ conceived, designed, and directed the project. OAVM, FCCM, and MP were involved in planning and supervising the work. FCCM, LABNS, FRBGG, LFC, and JFMF Collected the data. LRPD and LAFL performed the analysis, drafted the manuscript, and designed the figures. OAVM aided in interpreting the results and wrote the manuscript. All authors discussed the results and revised the final content of the manuscript.

Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.