Objective Evaluation of a Simulation Course for Residents in the Pediatric Emergency Medicine Department: Breaking Bad News

Introduction: Breaking bad news (BBN), especially in the pediatric emergency medicine department, requires significant skill and delicacy due to the acute context of a busy emergency department (ED) and the lack of prior rapport with the patients and families. Pediatric literature on breaking bad news has mostly focused on pediatric oncology and pediatric critical care, with limited literature focused on pediatric emergency medicine. Review of the literature also reveals that most existing studies solely assess the learners’ self-ratings of efficacy and comfort, and far fewer studies objectively evaluate learners’ actual performance using simulation. Our objectives for this study was to use an objective assessment tool to assess residents’ breaking bad news skills, pre- and post-simulation training, specifically in the setting of a pediatric emergency medicine department. Methods: 34 residents were evaluated on their performance in breaking bad news via videotaped simulation encounters before and after teaching intervention. The "Modified Breaking Bad News Assessment Scale” (mBAS) was used as the assessment tool. A paired t-test analysis was conducted to examine the mean difference in pre- and post-simulation scores in each of the five mBAS domains. Results: Breaking bad news performance score improves one to two weeks post-intervention, and was statistically significant in three of five domains. Conclusion: Our study shows that breaking bad news is a teachable skill that can be improved by simulated education in the pediatric emergency medicine department. This study demonstrates the utility of simulation course in improving breaking bad news skills in the pediatric emergency medicine department. Future work in developing focused simulation curriculums is important to improve provider communication skills and patient-physician relationships.


Introduction
Breaking bad news (BBN), especially in the pediatric emergency medicine department, requires 1 2 2 3 4 Simulation has long been used to teach trainees procedures and communication skills, and BBN training has become a recent area of interest in simulation [19]. This idea originated from the field of oncology, where research showed poor skills of providers in BBN, leading to patient dissatisfaction and provider stress [19]. Simulation enhances residents' comfort in BBN, which translate into more effective communication skills throughout their training [15]. This skill is particularly important in the pediatric emergency medicine department due to the acuity and unexpected adverse events that increase physician stress and patient dissatisfaction.
Pediatric literature on BBN has mostly focused on pediatric oncology and pediatric critical care, with limited literature focused on pediatric emergency medicine [20]. Review of literature also reveals that most existing studies solely assess the learners' self-ratings of efficacy and comfort, and far fewer studies objectively evaluate learners' actual performance using simulation [21]. Our aim for this study was to use an objective assessment tool to assess residents' BBN skills, pre-and post-simulation training, specifically in the setting of a pediatric emergency medicine department.

Participants
Pediatric, emergency medicine and family medicine residents rotating through the ED in our institution, a free-standing tertiary children's hospital and an academic center, were invited to participate. The study period was from July 2017 to November 2017. There were no specific selection criteria, and residents participated based on their availability and willingness. The only exclusion criteria included residents who had greater than five hours of prior formal didactic training in BBN. Institutional Review Board (IRB) and site approval from our hospital were obtained prior to the recruitment of trainees.

Rationale of the assessment tool selected
The objective assessment tool selected for this study was the "Modified Breaking Bad News Assessment Scale" (mBAS) (Appendix 1), originally designed and published by Miller et al. [22], in the British Journal of Cancer, as a collaboration between the oncology and psychiatry departments, of the University of Oxford, UK. The scale was later validated in a BBN study on medical students in 2012 [23]. It is a detailed and structured checklist that contained 22 items distributed over five different domains of communication behaviors relevant to BBN. For ease of scoring, the questions are grouped by sections into the chronological order one would expect in an actual patient encounter [22][23]. The five domains include A: setting the scene, B: breaking bad news, C: eliciting concerns, D: providing information, and E: general considerations. These items are rated on a Likert scale of one (very good) to five (very poor). The mBAS was selected for several reasons. First, it was shown to generate scores for each category listed above, and therefore helped identify strengths and weaknesses in individual performance. Second, it standardized scores for overall performance and demonstrated good utility in that raters required minimal expertise in the field and minimal training to complete the mBAS. Third, there is limited availability of validated tools to objectively evaluate performance on BBN.

The simulation cases
The Principal Investigator (PI) created two simulation cases to be randomly assigned in the study. Case one involves a one-year-old who presents with viral symptoms found to have newonset leukemia. The second case is a four-year-old drowning patient with hypoxic-ischemic brain injury. Residents were initially randomly assigned a scenario and asked to break the bad news to the mother, played by a trained standardized patient (SP). Trained SPs were utilized from the standardized patient Program at the University of Texas Southwestern (UTSW) Medical Center. The UTSW Pediatric Emergency Medicine Division provided monetary compensation for the SPs. All simulated encounters were recorded in the ED family consultation room.

Simulation and educational intervention
Residents participated in two randomly assigned simulations: before teaching intervention (pre-test), and one to two weeks post-intervention (post-test). Residents were randomized to either the leukemia or the drowning case. Immediately after the first simulation, the resident was debriefed and given the teaching intervention. The intervention consisted of a 15-20 minutes lecture on BBN, created by the PI in collaboration with the UTSW Palliative Care Department. Content of the lecture included background on BBN, obstacles to delivering bad news, how patients want bad news to be delivered and how to break bad news. One to two weeks later, there is a crossover in the cases. The resident participated in a second case followed by another debriefing session. Three investigators independently scored all pre-and post-test videotaped encounters using the mBAS tool. The investigators were blinded to the order of the videos and to each other's scores. Investigators were all Pediatric Emergency Medicine attendings and were trained on how to use the mBAS tool.

Data analysis
Data analysis was done using SPSS statistical package software (IBM SPSS Statistics for Windows, Version 22.0, Armonk, NY: IBM Corp; 2013). Descriptive analysis was done to characterize the participants by gender, number of years in training, prior training hours, and residency specialty. All analysis was done separately for each of the five mBAS domains and assumed a p-value <0.05. For each participant, the domain score was calculated by obtaining the mean of the graded scores. The inter-rater reliability was obtained using intraclass correlation (ICC) estimates and their 95% confident intervals based on a mean-rating (k = 3), absolute-agreement and two-way mixed-effects model. An average of the graded scores for the three raters was calculated to obtain the final score. ANOVA F-test was used to determine if the pre-post mean difference significantly differed by participant characteristics. A paired t-test analysis was done to determine if a significant mean difference existed between pre-and posttest scores in each of the five mBAS domains.

Participation
A total of 35 residents (20 pediatrics, eight emergency medicine, six family medicine, one medicine-pediatric) participated in the course, with one emergency medicine resident unable to finish post-test simulation due to clinical obligations, and therefore was excluded from the study.

Inter-rater reliability
Three raters independently scored both pre-and post-simulation encounters.

Pre-test and post-test ratings
Pre-test and post-test scores were calculated for each category of the mBAS tool (

Discussion
This study supports that residents' BBN skills in the ED improve one to two weeks after a simulation course with educational intervention, suggesting that BBN skills are teachable. In addition, our results demonstrate that there is a lack of communication training for residents and that it is important to improve education on this topic. An increase in communication training using simulation in residency programs can produce significant and meaningful improvements in these skills.
This is the first study to use an objective tool in assessing BBN performance in the pediatric ED. The improvement in the performance before and after intervention is encouraging. However, this study also demonstrates the need to develop an objective BBN assessment tool for use in the often-chaotic pediatric ED.
Delivering bad news in an emergency setting has its unique challenges, relative to other specialties. The ED is a busy and acute environment, often frightening for patients and parents. Often times, visits represent the first encounter between the patient and the physician, leading to lack of rapport. In addition, ED physicians are often feeling pressured, distracted or preoccupied and juggling a number of patients with life-threatening problems simultaneously. These environmental and emotional factors make BBN even more difficult and delicate in the ED.
While mBAS is an excellent tool for BBN in the Oncology department, its application in the ED may be limited due to the different pace and doctor-patient relationship in the two fields. The raters in our study found the mBAS tool difficult to use since certain questions were not applicable in an emergency setting. For example, question 14 (Appendix 1) asks if the doctor manages to focus on any positive aspect. This may explain the poor inter-rater correlation.
Future studies with larger sample sizes are necessary to create an assessment tool specific for BBN in the Pediatric ED.
There were several limitations in our study, including small sample size, single institution, and lack of a control group. There was potential selection bias, given that participants were enrolled on a voluntary basis. Finally, the mBAS tool was created specifically in the oncologic setting, therefore its generalizability in the ED may be limited.

Conclusions
Breaking bad news well is an important communication skill every physician should have especially when working in the pediatric ED. This study successfully demonstrates the improvement of BBN performance in residents rotating through the pediatric ED after an educational intervention with simulation. The data support that communication curriculum in residency training programs can improve resident communication skills. This study supports the need for continued focused resident education in communication and BBN.

Appendices Appendix 1. Modified Breaking Bad News Assessment Scale (mBAS)
When marking please place a circle around the number which best reflects the score you wish to give. The scale below each question is for guidance only.

A. Setting the scene: This section looks at whether the doctor established an initial rapport before breaking the bad news. This can be done by creating an environment which allows both private and comfortable communication, by the doctor introducing him/herself, and by the doctor showing an interest in the patient as an individual.
1. How did the doctor arrange the environment?
The doctor may have: · Placed the chairs at an angle which allowed unforced eye contact.
· Ensured that the desk was not in between him/her and the patient.
· Ensured that the wastebasket was out of the way.
· Prepared for the patient becoming upset-for example, by placing the tissues so the patient could reach them.

·
Taken measures to prevent interruptions-for example, by disconnecting the phone. B. Breaking bad news: This section specifically focuses on whether the doctor was sensitive to the patient's perspective when he/she delivered the news (the establishment of rapport is scored in the above section). The amount of information given to the individual patient may vary depending on what the patient already knows. Individual patients may differ in the amount of information they wish to receive during this interview, and in the rate at which they assimilate the news. Did the doctor: · Gently inform the patient that what followed was going to be important, before using any specific terms? · Take cues from the patient on whether to speak or listen after breaking the news?
6. When delivering the news did the doctor utilize an appropriate level of detail and language for the patient's understanding?