1 Introduction

Due to increasing complexity of care, modern pediatric intensive care unit (PICU) is prone to human errors. Data overload, complicated medical conditions, need for urgent decision making, commotion, involvement of multiple physicians and inefficient handoffs are some of the factors that can contribute to errors. These obstacles need to be surmounted to prevent patient harm in the process of care giving [1]. Out-of-context presentation of large amounts of data impedes health care providers from making correct decisions, leading to omissions and delays. Error prevention with the use of checklists and electronic decision support algorithms has been long used in other complex industry environments, but has only recently been applied in acute care hospitals [2].

Daily morning rounds are an essential component of the Pediatric ICU practice. Increasing complexity of monitoring data and multitude of lab results that need to be evaluated can lead to information overload and potential errors, due to overlooking of pertinent data and shortcomings in information transfer. Focusing on acute, potentially life threatening conditions, at times, leads PICU physicians to overlook essential factors that may play an important role in patient’s recovery. Adverse events, both preventable and non-preventable, are a common problem in PICUs. The international incidence is 2.7–33.4 adverse events per 100 PICU-days [2]. The innovations in medical technology and bio-medical engineering have tried to enhance decision making in clinical workflow by providing us with tools (checklists) that can be applied in complicated and busy hospital settings. This is especially true in Pediatric ICU where multiple distractions can lead to lack of focus [2, 3]. Previous checklists used in the pediatric wards and PICU included DEFAULT (DNR status, ET tube safety, Fluid strategy/feeding plan, Analgesia, Ulcer skin and gut, Lines out and Tidal volumes <8 ml/kg) [4]; KIDS SAFE (Kid’s Development needs, Infection, Deep vein thrombosis prophylaxis, Skin integrity, Sedation, Analgesia, Family and Enteral needs) [5], etc. By incorporating checklists into the daily rounding process, the studies demonstrated improved teamwork and collaboration, while reducing inertia of previous practice and lack of agreement on goals and measures [6, 7]. However, there is limited adaptation of the checklists in routine practice [4, 5]. We believe that lack of routine use may be related to their utilization as an additional rounding step rather than an inbuilt “rounding tool” which can serve as a framework for complete rounds with an ability to transform to clinical notes, along with decision support capability.

Checklist for early recognition and treatment of acute illness and injury in pediatrics (CERTAINp) was designed as critical care decision support tool to facilitate timely and improved best-practice delivery and a reduction in preventable death and complications in critically ill children compared to current practice. The CERTAIN information display and functionality were designed using cognitive ergonomic principles and integrated into the workflow to facilitate high quality; high value health care behaviors. The CERTAIN information display and functionality were designed using cognitive ergonomic principles in its mainframe called—ambient warning and response evaluation (AWARE) and integrated into the workflow to facilitate high quality; high value health care behaviors [8]. To facilitate better PICU rounding we developed an electronic rounding tool as part of the broader CERTAINp (Fig. 1). Adult version of this rounding tool was previously validated in a simulation environment [9]. This study was conducted with an objective to alpha test this tool in a clinical setting. The study was reviewed and deemed exempt by Mayo Clinic IRB.

Fig. 1
figure 1

Snapshot of the electronic rounding tool of CERTAINp

2 Methods

A prospective pre and post-interventional study was carried in the pediatric intensive care unit (PICU) of the Mayo Clinic at Rochester, Minnesota, in the months of July and August, 2015. This is a 16 bed mixed medical/surgical unit (non-cardiac). Patients in the PICU include both critically sick children and also intermediate status patients for monitoring or chronic ventilator management.

Assessment of team performance was carried out by direct in-person observation by two investigators who were not a part of the clinical PICU team (HK and MH). Pre-implementation/Phase-1 of the study was carried out in the month of June 2015. Following the baseline assessment, we implemented CERTAINp rounding tool. Assessment was continued for further 2 weeks to assess impact of CERTAINp rounding tool on the process. Various characteristics such as discussion of important patient safety characteristics and adherence to standard guidelines were noted in a designated form. (Appendix 1). Standard operating procedures for accessing the compliance were pre-defined to prevent unconscious bias during documentation of results. The total checklist compliance score was generated at the end of each patient encounter by dividing the number of standard of care adherence guideline data points discussed during that encounter by the total number of eligible data points for that encounter. A total of 16 data points were identified as described above. The need for devices was further subcategorized to need for central line, need for ET Tube and need for arterial line. Thus a total of 18 unique data points were described and discussing all 18 of them amounted to a perfect compliance score of 100 %.Time duration for each patient encounter was recorded via direct observation by means of a stopwatch.

At the end of morning rounds, perception surveys (Appendix 2) were handed over to fellows, nurses and residents. Survey was based on a 10-point Likert Scale, 1 being the lowest and ten being the highest for questions 1–4. Question five asked for their perception of rounds in the scale of 1–10 where lower number meant longer rounds time and higher number meant adequate time. Question 6 assessed potential for missing information, where higher number meant more potential for missing information. Surveys were collected back immediately. In person survey at the end of rounds each day ensured that the staff gave a quick feedback of their perception of various aspects of the rounds, while the experience was fresh in their minds.

Attending physicians and staff were blinded to the components of the rounds that were scored. The rounds and surveys were carried out during rounding by different attending physicians and residents in order to introduce variability. A total of five unique attending physicians rounded over the 2 months during which the study was carried out. The group of residents, fellows and nurses underwent changes as their group moved over to different rotations, and a new group of residents started their rotation in the PICU. Thus we were able to obtain a diverse range of perspectives both in pre and post implementation phases.

Study data were collected and managed using REDCap electronic data capture tools hosted at Mayo Clinic. Research Electronic Data Capture (REDCap) is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources [10]. Raw data from Internal Redcap data collection software was exported to an electronic file for analysis. Analysis was done using JMP© software (SAS institute, NC). A checklist compliance score was generated after analysis. Shapiro–Wilk test on the continuous variables showed non-normal distribution. Non –parametric (Wilcoxan/Kruskal–Wallis test) was used to analyze continuous data. For binary variables, Chi-square test was used when applicable. For variables with expected count <5 in 20 % of the cells, Fischer exact test was used. Two-sided p value of less than 0.05 was considered statistically significant.

3 Results

A total of 113 patient encounters were recorded by direct personnel observation and 108 staff members filled out the staff perception surveys in the baseline phase of the study. 114 patient encounters and 80 staff perception surveys were recorded in Phase 2 (Post implementation phase). Discussion of standard of care guidelines improved to >97 % in all the compliance to standard of care guideline data points. Maximum increase was seen in discussion of head of bed elevation [part of the Ventilator Associated Pneumonia (VAP) bundle], DVT prophylaxis, glucose control and skin and wound care (from 3.9, 5.3, 15.5, and 19 % respectively in Phase-1 to 100 % for all in Phase 2; p < 0.01). The discussion of ulcer prophylaxis and bowel protocol (to reduce opioid induced constipation in applicable patients) increased from 28.1 and 27.7 % respectively in Phase 1 to 100 % in Phase 2 (p < 0.01). Significant increase was also observed in the discussion of spontaneous breathing trials, sedation breaks, need for devices, fluid balance and nutrition (56, 55.2, 57.2, 80.5 and 84 % respectively in Phase-1 to 100 % for all in Phase 2; p < 0.01). Post intervention compliance was significantly higher but not 100 % for pain treatment (45.3–99.0 %), ‘need for central line’ (70.8–97.8 %) and ‘need for Endotracheal (ET) tube’ (58.6–97.9 %). No significant difference was observed in the ‘need for arterial line’ and ‘family concerns addressed’. Total compliance to the checklist and thus compliance to discussion of standard of care guidelines significantly improved from a Mean value of 49.17 ± 16.5 % in Phase 1 to 100 % in Phase 2 (Table 1).

Table 1 Adherence to standard of care guidelines before and after implementation of CERTAINp rounding tool

Rounding time was recorded for each patient encounter for 113 and 114 encounters in Phase 1 and Phase 2 respectively. Both mean rounding time with standard deviation and median with Interquartile range (IQR) was calculated for each phase. Rounding time duration per patient increased from a mean of 8.03 ± 5.8 min in Phase 1 to 9.9 ± 5.7 min in Phase 2, p = 0.0002 (Table 2).

Table 2 Average rounding time duration per patient (minutes)

Residents, fellows and nurses reported improved perception of rounding aspects such discussion of pertinent information and clarity with goals at the end of rounds. The staff felt that more pertinent clinical information was discussed after the implementation of CERTAINp rounding tool (mean of 8.6/10 points on a Likert scale in Phase 1 to 9.4/10 points in Phase 2, p < 0.01). They also felt that rounds were more systematic and there were more teaching and learning opportunities during rounds in Phase 2 (mean 8.0/10 and 6.7/10 respectively in Phase 1 to mean 8.7/10 and 7.7/10 in Phase 2 respectively; p 0.004 and 0.02). Staff perceived an increase in rounding time, however the difference was insignificant. They also subjectively reported a reduced potential for missing information (3.1/10 in Phase 1 to 2.0/10 in Phase 2), p < 0.01 (Table 3).

Table 3 Staff perception survey results

4 Discussion

In this study we have shown that implementation of the checklist based rounding tool (CERTAINp) improved the discussion of standard of care guidelines to >97 % in all pertinent data points. Based on the expert’s consensus, evidenced based daily key clinical care process were selected as checklist for the Rounding tool. All staff reported improved aspects of rounding processes including clarity of goals, learning opportunities and more systematic execution of processes of care. Staff also felt the rounding tool reduced the potential for errors due to shortcomings in information transfer and improved oversight of issues that could affect patient care. The rounding time duration was increased by only a minute, making the process overall very efficient.

Although this increase was statistically significant, from a clinical perspective it is a negligible increase. The benefit gained from covering all data points with an additional expenditure of only a minute might be desirable by the majority. Other factors that might have caused this include team’s adoption of a new tool and the learning that was required to incorporate this into daily rounds.

Communication failure leading to errors and harm in medicine is very common. ICUs are one of several clinical environments that characteristically have high numbers of staff, time pressures and patients with complex and often rapidly changing needs [3]. Children in PICU are particularly victims of errors because of intrinsic complexity involving various organ systems and the urgent need for decision-making required for unstable patients [4]. Utilization of checklist during rounding is not a novel concept. Checklists have become ubiquitous in many industries and specialties in medicine, in order to ensure adherence to guidelines and avoid omission of pertinent data points in care processes [1117]. Use of checklists has shown to improve patient outcomes [18, 19], while also improving intangible elements such as improved team work [20], communication and co-ordination among the caregiving team [16]. The DEFAULT study demonstrated that simple structured communication in wards through the use of a mnemonic(s) improved patient safety measures, including reduction of accidental extubation rates and an increase in the proportion of pediatric patients ventilated with target range tidal volumes. It further empowered nurses and junior staff to actively participate in the rounding process. However, the authors observed that the checklist was sometimes recited unconsciously without putting a lot of thought into it [4]. Implementation of rounding stickers at Arkansas Children’s hospital reduced the rates of UTI, while improving device usage and administration of DVT and GI prophylaxes [21]. Children’s hospitals and clinics of Minnesota implemented a PICU safety checklist and identified improvements in all quality and safety metrics, such as use of invasive devices, antibiotic and laboratory test use, and compliance with standards of care, while improving staff satisfaction rates to >80 % for safety and collaboration [3].

A common theme that emerges from the prior use of checklists in the PICUs is that they have traditionally been utilized as an adjunct to the conventional rounding process. Furthermore, the mnemonic, as simplified as it may be, is still dependent on recall by memory. This has the potential to lead to “checklist fatigue” in the long run. Repeating the checklists at the end of rounds is sometimes done at a subconscious level, thus not adequately addressing their significance. Our study utilized a novel approach by means of utilizing information technology, to incorporate the rounding checklist into a real time decision support tool, which also had the functionality to provide a dashboard view of the case scenario and print out daily progress notes, which are integral aspects of the rounding process. User friendly interface facilitated ease of navigation with minimal training. Electronic checklist also enables streamlined data capture and reporting without extra staff for data collection. Success has been shown in this arena through the EMR enhanced and unit wide dashboard depicting adherence to CLABSI prevention checklist, in a previous cohort study [22]. More widespread use of technological interventions such as these is warranted.

Our study did have some limitations. First, it was confined to a single location with highly sophisticated clinical processes, ease of using new software and real time tools. Its extrapolation to resource poor settings, with limited technology capabilities and poor connectivity issues has to be determined with further studies in such settings. Second, a wash-out period was not incorporated in study design. This might have enhanced the effect of CERTAINp, but discussion reaching near perfect levels in post-phase has nullified this oversight. Third, our study does not yet determine ultimate patient outcomes after the implementation of tool. It would be beneficial to have further exploration of outcomes and thus the cost-effectiveness of using the tool. Its impact on clinical outcomes is currently being evaluated by a multinational clinical trial (www.clinicaltrials.gov: NCT0239898) to test the wider adoption of the CERTAINp tool. We further recognize that certain behaviors of care providers may have been altered due to the Hawthorne effect (observer effect) [23]. Study personnel directly observed rounds and this may have impacted the behavior and perceptions of the rounding team.