Surgical Safety Checklists in Children's Surgery: Surgeons' Attitudes and Review of the Literature.

Introduction
Surgical safety checklists (SSCs) aim to create a safe operating room environment for surgical patients. Provider attitudes toward checklists affect their ability to prevent harm. Pediatric surgeons' perceptions surrounding SSCs, and their role in improving patient safety, are unknown.


Methods
American Pediatric Surgical Association members conducted an online survey to evaluate the use of and attitudes toward SSCs. The survey measured surgeons' perceptions of checklists, including the components that make them effective and barriers to participation. To better evaluate the available data on SSCs, the authors performed a systematic literature review on the use of SSCs with a focus on pediatric studies.


Results
Of the 353 survey respondents, 93.6% use SSCs and 62.6% would want one used in their own child's operation, but only 54.7% felt that checklists improve patient safety. Reasons for checklist skepticism included the length of the checklist process, a distraction from thoughtful patient care, and lack of data supporting use. Literature review shows that checklists improve communication, promote teamwork, and identify errors, but do not necessarily decrease morbidity. Staff perception is a major barrier to implementation.


Conclusions
Almost all pediatric surgeons participate in SSCs at their institutions, but many question their benefit. Better pediatric surgeon engagement in checklist use is needed to change the safety culture, improve operating room communication, and prevent harm.


INTRODUCTION
Since the 1930s, checklists have been used in aviation to prevent accidents due to human error. 1 In 2008, the World Health Organization (WHO) launched the "Safe Surgery Saves Lives" campaign and developed a Surgical Safety Checklist (SSC); 2,3 various forms have been adopted throughout the world. After implementation of the WHO's 19-item surgical checklist in 8 different hospitals, the death rate decreased from 1.5% to 0.8%, and the complication rate decreased from 11.0% to 7.0%. 4 On the basis of an annual number of 234 million operations performed each year globally, the WHO estimates that effective implementation of the WHO SSC could prevent at least half a million deaths per year worldwide. 2,4 In the United States, since calendar year 2015, the Centers for Medicare and Medicaid Services has made "safe surgery checklist use" one of the measures in the Ambulatory Surgical Center Quality Reporting payment program. 5 With the WHO SSC as a guideline, institutions throughout the world are adopting their versions of the surgical checklist with opportunities for a pause before induction of anesthesia, before incision, and at the end of the case. 6 Prevention of death and complications is a clear goal of SSCs, but it is not the only goal. Checklists have additional benefits that indirectly affect patient care. SSCs are shown to improve communication [7][8][9] and teamwork 8,10,11 and reduce observable errors relating to poor team skills. 10 However, not all studies have found checklists to be effective. The checklist-based quality improvement program in Michigan, known as Keystone Surgery, did not improve surgical complications or 30-day mortality rates. 12 Successful implementation of an SSC depends on individual "buy-in" and surgeon engagement in the process. 11,13 Most institutions use a checklist because it is a safety measure; however, meaningful use of the checklist is highly variable. Furthermore, checklist adoption, acceptance, and adherence may differ between surgical subspecialties and between surgeons caring for adults versus children. Checklists are not always designed for pediatric patients and may have irrelevant components if used for both adult and pediatric patients. Workflow can be negatively affected by an irrelevant checklist. Therefore, a positive perception of the SSC is paramount to its success.
Since the hallmark WHO report, 4 the majority of studies show that checklists improve patient safety and outcomes in adults. However, researchers have not adequately investigated the effect of SSCs in the pediatric population. The current investigators designed a survey to assess safety knowledge, attitudes, and perceptions of North American pediatric surgeons and to specifically gauge the "buy-in" of the American Pediatric Surgical Association (APSA) membership on checklists. To support understanding of surgical checklists, the investigators conducted a literature review aiming to summarize the current data on the utility of SSCs in both adult and pediatric surgery.

Study Design
All active APSA members were e-mailed an invitation asking them to participate in a survey on SurveyMonkey. The survey was developed by the APSA Quality and Safety Committee to measure patient safety attitudes and perceptions of the membership. It collected demographic information and practice setting and asked surgeons whether they functioned in any formal leadership, education, or safety roles within their hospital. This report focuses on checklist-specific survey content. Questions were designed to quantify surgeon participation in the preinduction, preincision, and postoperative debriefing checklists and measure surgeons' attitudes about the effectiveness of checklists using Likert scales. Respondents had the opportunity to elaborate on certain survey questions with open-ended responses. APSA members were encouraged to participate in the survey with 2 follow-up reminder e-mails over a 3-week interval.

Variables and Data Analysis
The current investigators performed standard frequency analyses for surgeons' responses regarding whether checklists exist in their hospital and to summarize surgeons' beliefs about whether checklists improve patient safety. They tested associations between surgeon characteristics (years since fellowship, practice setting, leadership/safety/educational roles) and attitudes toward the SSC using a chi-square test for categorical variables and Kruskal-Wallis test for continuous variables. Likert scale responses were dichotomized into agree/strongly agree versus neutral/disagree/strongly disagree. A P value < 0.05 was statistically significant.
The authors analyzed open-ended response questions using content analysis to enhance understanding of quantitative findings. The content analysis method is a systematic data coding and analysis procedure 14-16 that allocates specific quotes from respondents into a code structure that is developed through an iterative process. The codes are then used to group the data into representative themes. The investigators selected specific quotes to illustrate the broader themes, and these are noted in italics. Four separate reviewers (including an expert in qualitative research) evaluated the open-ended responses and formed consensus on the themes that emerged.

Literature Review
In addition to the survey, we conducted a literature search to identify systematic reviews of safety checklists in surgery and studies focusing on checklists in the pediatric surgical population. The online databases Medline, Embase, and PubMed were searched using the terms "surgical checklists" AND "systematic review," "safety checklists" AND "systematic review," "surgical safety checklists" AND "systematic review," "pediatric surgery," "checklist," AND "systematic review," and "children's surgery," "checklist," AND "systematic review." Studies were limited to those that focused on surgery/perioperative checklists, were written in the English language, and were peer-reviewed and included an abstract.

Demographics and Other Respondent Characteristics
A total of 928 APSA members received the survey, and 353 responded (38% response rate). The majority of respondents operate primarily within a children's hospital, 49.7% freestanding and 38.1% within an adult medical center. Most respondents were in an academic (65.3%) or mixed (25.3%) practice, with only 9.4% in private practice. The majority of respondents (56.7%) self-reported holding leadership positions at their institutions (eg, chair or division chief), with 43.4% in education and 21.0% in safety positions. The median number of years since completion of pediatric surgery fellowship was 13 (range, 0-32 years).

Practice and Attitudes Surrounding SSC
Most respondents (93.6%) reported compliance with SSCs at their institution, but only 54.7% felt that checklists improve patient safety. Respondents most consistently reported doing the preincision checklist (83.8% of respondents stated that they do it all of the time), followed by the preinduction checklist (79%), and then the postoperative debriefing checklist (29%). The majority of respondents (54.5%) agreed strongly that the checklists improve operating room communication, but only 35.6% agreed that they prevent harm and 24.1% that they benefit every case. Only 62.6% agreed strongly that they would want it used in their own child's operation. Being in a safety position was the only respondent characteristic correlated with believing that checklists improve patient safety or wanting the checklist used in one's own child's operation (Table 1).
In the open-ended questions, several themes emerged to explain a lack of enthusiasm for SSCs/handoffs: 1) Checklists are too long and contain information that is not relevant to the current case. Three respondents had these comments: "Checklists only work if they are short. So many checklists are filled with useless info. As a pilot, I use a checklist when I fly. It encourages safety by being short, simple, and to the point." "There are generic elements to all surgical procedures that have some applications in the generic checklists currently available. What would be far more effective would be computer generated checklists SPECIFIC for the procedure scheduled." "Safety checklist has become too encompassing beyond safety issues so that safety factors are getting lost in the long checklist, which has become another form to complete." 2) Checklists result in mindlessness as opposed to mindfulness and can distract from thoughtful patient care as described by these 4 respondents: "All this devotion to checklists and process has become a form of fetishism, and often is a distraction from the task at hand." "The surgical checklist is highly ineffective in modern Western hospitals. This was shown, but ignored, in the original paper. It is a meme, a trend, and a fad. It does not actually improve real mindfulness, and often distracts from it." "People sometimes go through the motions but don't really pay attention." "Like any checklist, it is used with varying degrees of a mechanical/begrudging fashion." 3) Surgeons want to see data proving that checklists prevent adverse events. As 2 surgeons stated: "There may be one case in 10,000 in which it makes a difference. I personally haven't seen it." "Of the tools that are available, none have led to new systems that demonstrably decrease error or improve safety." In contrast to the negative comments summarized above, some surgeons saw some utility in the checklist depending on the situation.
"It was my practice to discuss the more complicated cases at the beginning of the day to identify the need for additional anesthesia/nursing/ surgical care or unusual equipment. Current safety lists may have influenced some groups to use a similar practice. However, for most routine cases, the safety lists (required for all cases) add little or nothing to the case or safety of the patient." "It has improved patient safety in SOME cases." Many other respondents voiced strong support for SSCs as illustrated by these 3 respondents: "SSC has definitely improved OR [operating room] safety. NO wrong side procedures, NO wrong procedures since it was introduced and adopted. Also caught several 'near misses,' also improved timeliness of preop antibiotics." "I have been trying for years to implement a safety checklist." "It has become hard-wired and is an improvement."

Literature Review
The investigators identified 34 manuscripts, of which 11 met inclusion criteria and were reviewed in detail. Only 2 studies address pediatric surgery patients, 17,18 and the remainder focus on the adult literature. The first pediatric surgery study, by MacDonald and Sevdalis, 17 systematically reviews 20 studies of patient safety interventions including the SSC. The authors note that pediatric cardiothoracic surgery account for the majority of the children's safety literature. Fifteen studies (75%) use a checklist tool as part of their safety intervention; however, only 5 studies use checklists as their primary safety intervention. The 5 checklist-oriented studies evaluate checklist adherence and perioperative patient safety, but none assess the rate of complications, and only 1 addresses the near misses as a result of checklist adherence. The second pediatric surgery study, a systematic review, by Lagoo and colleagues, 18 evaluates the effectiveness and meaningful use of pediatric SSCs and their implementation strategies. The authors include 26 studies, all of which are cross-sectional or cohort studies. They rate 6 of them as poor overall. Of the 20 studies rated fair or good, only 16 evaluate SSCs in the operating room or perioperative setting. Three studies measure the effectiveness of checklists on morbidity and mortality with mixed results; one of these is primarily in adults with a small pediatric component. Four studies assess parental involvement in SSCs. Nine studies evaluate compliance with checklists, 11 studies investigate the implementation of checklists, and 8 studies assess attitudes related to checklist use and the effect of SSCs on safety culture. Table 2 summarizes the results of the pediatric surgery-specific checklist studies written in the English language.

Review of the Adult Surgical Literature on Checklists
The adult literature yielded 9 systematic reviews that met the inclusion criteria, and these are reviewed here in more detail starting with oldest to newest. 11,[19][20][21][22][23][24][25][26] Unlike the majority of the pediatric surgery literature, several of these studies did evaluate the effect of checklists on morbidity and mortality. It is important to note that many of the individual studies included in each systematic review overlap those included in the other reviews, as would be expected. Therefore, there is some redundancy between systematic reviews.

DISCUSSION
For nearly a decade, checklist implementation has been a surgical safety initiative around the world. Since the landmark 2009 NEJM report showing a decrease in deaths and surgical complications after the implementation of the WHO's SSC in 8 different hospitals, 4 numerous studies evaluate the role of checklists in surgery. The NEJM report by Haynes et al. 4 has been challenged, and some studies show that SSCs cause no reduction in morbidity and/or mortality. 21,27 However, the majority of studies show that checklists improve patient outcomes, teamwork, and communication. This national survey of APSA members shows that pediatric surgeons are skeptical about the value of SSCs. The current investigators sought to understand why such skepticism exists and how to make the SSC a more meaningful safety tool for their pediatric patients.
In the current survey, the authors found that the majority of pediatric surgeons believe that checklists improve operating room communication. Both the pediatric and adult literature support this notion; checklists are one tool used to facilitate better communication in surgical settings and are shown to improve operating room teamwork. 11,28 In 1 study of all root cause analyses submitted to the Joint Commission on Accreditation of Healthcare Organizations, communication is the most common root cause of sentinel event wrong-site surgeries. 29 Ideally, good communication is efficient. In the current survey, several pediatric surgeons express opinions that the checklist is too long or contained elements that are irrelevant to a particular case. A checklist that contains too many elements may be thought to inhibit effective communication. As 1 pediatric surgeon said, "Like any checklist, it is used with varying degrees of a mechanical/begrudging fashion." Acceptance of the checklist as an important safety tool means realizing that not all checklist items will benefit every patient, but all patients will benefit from some portion of the checklist. Customizing the checklist to pediatric general surgical patients should be a goal, but maintaining a routine set of items ensures that communication is standardized.
Across several studies in both the adult and pediatric literature, staff perception is noted to play a large role in the acceptance and successful adoption of checklists. To maximize its safety benefit, those people designing and using the checklist need to be engaged in the process. Some pediatric surgeons surveyed express support for the checklist and it is becoming "hard-wired" as an expected part of surgery. Perceived ownership of the checklist varies. Having an organizational culture of safety that begins with executive leadership can help foster an environment where the introduction of a checklist is well received. The onus for completing a checklist may be driven by hospital administration to satisfy a Joint Commission requirement but, ideally, is most closely overseen by leadership within the operating room who may be more "in touch" with the day-to-day use of the checklist. The investigators did note that being in a safety position is the only surgeon characteristic that correlates with believing that checklists improve patient safety or wanting the checklist used in     25 states, "The motivation for implementing an SSC differs between health care providers and hospital management." In the Boston Children's study, feedback from surgeons, anesthesiologists, and nursing staff is "overwhelmingly positive," 30 and this may have been due to participant engagement in the initiation of the checklist. After a first pilot test, feedback resulted in changes to improve the second version. Participant involvement in the creation and modification of the checklist can be beneficial but is not enough. Implementation of the checklist needs to be done thoughtfully. Repeatedly, it is seen that effective implementation strategies incorporate education and in-service training that addresses the concerns expressed by staff. 25,30 As many studies have shown, when implemented well, checklists improve the safety culture. 23,25,31,32 Clearly, there is a difference between checklist compliance (participants did the checklist), adherence (participants did all the parts of the checklist as it was designed), and fidelity (participants did the checklist with intent and meaning). Multiple studies show that checklist adherence is lower than compliance. 23,33 In the pediatric literature, SSC compliance has been good in most studies. Not all elements of the checklist are being done routinely, however, perhaps because the checklist is too lengthy. The current study found that pediatric surgeons surveyed do use SSCs regularly, but that compliance is best with the preinduction and preincision, and that the postoperative debriefing checklist is much less common. It would be interesting to know why this is the case among the surgeons surveyed. It may be easier to overlook a less formal postoperative debriefing than a more formal time-out. It may be because the lead surgeon or the lead anesthesiologist or both are not always both present at the end of the case to lead the debriefing. It may be due to a lack of institutional awareness of the potential benefits of the debriefing. 34 The relevance of the postoperative debriefing may also be a factor. If the debriefing contains elements that pertain to the transition of care of the patient leaving the operating room, it represents a key moment of team communication. However, if it contains only a review of wound classification and name of the case, it may be less valued as a component of patient safety. Better education on the benefits of doing all 3 parts of the SSC as it is designed is needed to make the debriefing an essential part for pediatric surgeons.
In the current survey, the authors found that pediatric surgeons express concern about the checklist being done in a mindlessly. When the operating room team does not embrace checklists or uses them improperly, they can have a negative effect on team dynamics. 11 When used properly, checklists strengthen the "team feeling" in the operating room, increase discussion of critical events, improve decision making, and lead to better inter-professional coordination and assignment of tasks. 11 One theme noted in the comments is the concern that the mindless use of the checklist can distract from thoughtful patient care. "Going through the motions" and not paying attention to the components of the checklist can detract from its utility and can contribute to surgeons' skepticism about its benefit. By contrast, thoughtful use of the checklist can result in "good catches" or "near misses," which will ultimately improve patient safety.
This survey shows that pediatric surgeons question the checklist's ability to prevent adverse events. Although the majority of the pediatric literature on checklists does not address effects on complications and mortality, the adult literature shows that checklists reduce both. [19][20][21][22][23][24][25][26] A recent retrospective study from Ontario showed no reduction in perioperative complications when SSCs were used in pediatric patients. 35 In one of the other few studies that address outcomes in pediatric surgical patients, there is no significant decrease in morbidity or mortality, except in developing countries when a broad safety strategy that includes practice bundles and team-based practice is employed. 36 In the pediatric population, perhaps a better measure of outcome success would be to look at the ability of the checklist to prevent errors and identify "near misses" or "good catches." As suggested by Putnam et al., 37 meaningful use or good catches may be a more appropriate metric for checklist effectiveness. Wrong-site surgery is unacceptable but exceedingly rare. Still, current site-verification protocols could have prevented only twothirds of the wrong-site surgery examined cases. Many protocols involve considerable redundancy without clear added benefit. 38 A preincision checklist that includes site verification and pertinent radiological imaging, and has engaged participants, is a more efficient way of confirming the correct surgical site. Other good catches including identification of medication issues and allergies and confirmation of the availability of essential implants or blood products can have safety implications for a pediatric patient. As 1 pediatric surgeon responded to the survey, "SSC has definitely improved OR safety. NO wrong side procedures, NO wrong procedures since it was introduced and adopted. Also caught several 'near misses,' also improved timeliness of preoperative antibiotics." The effect of the SSC can be measured by its ability to prevent harm before it happens.
This study has several limitations. The survey questions were selected based on what the APSA Quality and Safety Committee subcommittee deemed important to evaluate checklists in pediatric surgery. The response rate was only 38%, which may reflect a lack of interest in or support for checklist use among the APSA membership. The majority of respondents reported holding leadership positions at their institutions, and those in leadership positions may be more likely to support a checklist because of the belief that it improves the safety culture. In general, those who responded to the survey are more likely to be involved in patient safety initiatives, so the survey likely overestimates pediatric surgeons' enthusiasm for checklists.
Despite being driven by Centers for Medicare and Medicaid Services and the WHO to use them, belief in SSCs among pediatric surgeons is lacking. Checklists are often implemented as part of a multi-step process to improve care. Studies show that safety attitudes and staff engagement affect successful implementation, adherence, and fidelity of the SSC. The current survey results support this. Checklists work, but only if surgeons believe in them and are engaged. Those surgeons who believe in the safety culture understand why SSCs are important. Clearly, there needs to be more education to get pediatric surgeons on board with checklists and to get them involved in the development and implementation of checklists for the pediatric surgical patients. Their "buy-in" is essential to their effective use, and the current survey shows that an alarming number of pediatric surgeons have not "bought in." In the pediatric setting, investigators should judge the effectiveness of a checklist on its ability to foster teamwork, good communication, and improve the safety culture. With pediatric patients, the SSC can certainly prevent errors, but its benefit may not be in its direct effect on patient outcomes. When compared with adults, the pediatric adverse and "never" event rates are very low. Therefore, using complications and mortality rates as a checklist metric is unrealistic. At a time when quality improvement is at the forefront of health care, and when the Children's Surgery Verification program is beginning to set expectations for pediatric surgical care in the United States, one must look to safety initiatives that have shown promising results to be the foundation of surgical care for children. Checklists are such an initiative. One must continually engage in such safety efforts when seeking to provide better, safer care for the patient population.