Knowledge, skills and competency retention among health workers one year after completing helping babies breathe training in South Sudan

Introduction This study aimed to evaluate the long-term retention of knowledge, skills, and competency of health workers who completed a Helping Babies Breathe (HBB) training program and its effect on newborn mortality. Methods A quasi-experimental pre- and post-intervention study was conducted. Participants were health workers selected based on their previous training on HBB protocols. Participants were assessed for knowledge, skills, and competency in March 2017 (immediately before and after training), June 2017 (three months after training), and September 2018 (one year after training). Assessments were conducted using HBB questionnaires, checklists and practical skill drills. Mean scores were obtained and ANOVA, chi-squared test, and Pearson's test were used for pre intervention, post intervention, and one-year-after comparisons. The effect of training on the management of newborn asphyxia was assessed based on a review of the delivery registry at a maternity and children's ward. The scores were group into percentages and averages means and were computed using chi-squared tests. Results Despite improvements in knowledge, skills, and competency three months after training, participants showed a marked decline one year after training. Knowledge increased from 42.5% pretest to 97% posttest but decreased to 84.5% three months' post training and further decreased to 69.4% one year post training. Skills increased from 26.1% pretest to 94.4% posttest, remained at 94.4% at three months, and decreased to 77.0% at one year. Simple resuscitation scores increased from 26.9% to 88.8% pre- and posttest, remained roughly at three months and decreased to 76.4% at one year. For complex resuscitation, scores decreased from 90.9% posttest to 76.9% at one year. The assessments at one year indicated a need for support and practice, especially with bag-mask ventilation. Conclusion The immediate evaluation of health workers after HBB training showed significant increases in knowledge, skills and competency in neonatal resuscitation. However, this declined after one year. The training also resulted in decreased neonatal mortality.


Introduction
Of the 200,000 children born annually in South Sudan, an estimated 40% die in the first month of life [1]. Newborn mortality accounts for 39% of all under-five deaths in South Sudan. Over the last decade, the decline in the newborn mortality rate in South Sudan has been slower than the global average [2]. An estimated 136 million infants are born each year and this figure is expected to rise in the coming years. The highest risk of newborn mortality is present during the first day of life. Death during this period accounts for almost 5% of newborn deaths worldwide [3,4]. Globally and in South Sudan, the main direct causes of newborn deaths are usually infection-related complications (26%); intrapartum complications (24%), including birth asphyxia; preterm delivery (34%) and congenital abnormalities (9%). In South Sudan, 20% of newborn deaths are associated with birth asphyxia [5,6]. The transition from intrauterine to extra uterine life requires the initiation of breathing, which is a critical physiological change required for newborn survival. Research has indicated that most newborns initiate breathing within 30 minutes, and an estimated 10% breathe when they receive drying and stimulation from health workers. Three percent of newborns require positive pressure (PPV) while another 2% need ventilation and intubation [7,8]. Training health workers on effective and timely newborn resuscitation could reduce newborn asphyxia and improve survival rates. To address inadequate resuscitation training in health facilities in low-resource areas and improve neonatal outcomes, in 2010 the American Academy of Pediatrics developed a newborn-care training program for healthcare professionals called Helping Babies Breathe (HBB) [9].
The HBB curriculum is designed to train birth attendants in lowresource countries in neonatal resuscitation. It is evidence based and specifically geared toward reducing global neonatal mortality. HBB focuses on the essential steps of resuscitation, including the birth evaluation of the infant, breathing stimulation, and ventilation during the critical "golden minute" after birth [10]. The HBB curriculum promotes active learning and hands-on practice using newborn simulators, self-reflection, and group discussion and feedback after completing a task. It also focuses on paired learning, which is tested based on four formative assessments. Competence and performance can also be determined based on multiple-choice questionnaires, bagmask ventilation (BMV) tests, and objective structural clinical examination consisting of sections A and B (OSCEA&B). Before its global launch, HBB curriculum assessment was modified based on the study of Singhal et al. on the educational evaluation of HBB assessment in Kenya and Pakistan. Further studies of HBB curriculum assessment in Africa and Asia using pre-and posttest scores showed significant gains in resuscitation knowledge and skills immediately after training [11,12]. However, those studies did not conduct detailed examinations of learners' qualifications and the characteristics of their performance. Understanding learners' performance and competence could help refine the HBB curriculum to promote the retention of neonatal resuscitation knowledge and skills tailored to specific learners. Studies on the effectiveness of newborn resuscitation training programs have shown increased knowledge, skills, and competency after training, which was sometimes retained for one year. The evaluation of newborn resuscitation training has shown immediate increases in knowledge, practical skills, and competency, with improved newborn outcomes [13]. Nevertheless, there has been limited rigorous evaluation of the retention of knowledge, skills, and competency, and of subsequent newborn outcomes, in low-resource and post conflict settings, such as South Sudan. Training outcomes depend not only on the extent to which such knowledge and skills are retained but also on the ability to apply them at appropriate times. This depends on several factors, including conditions in the clinical environment, regular supervision, settings, practice opportunities, and the availability of supplies and appropriate equipment [14]. Providing appropriate training, offering refresher training and support, and ensuring good training environments facilitate long-term knowledge, skills, and competency retention among health workers in lowresource settings [14]. In South Sudan, nurses and midwives typically manage normal deliveries, and birth asphyxia is usually not recognized early enough. Due to critical shortages, medical doctors are usually not involved until the late stages of managing birth asphyxia, even in major hospital settings. The present study aimed to evaluate the retention of knowledge, skills, and competency among health workers and the effects on newborn mortality one year after implementation.

Methods
Trial background: the trial in this study was registered with the Pan African Clinical Registry [15]. The study trend statement is available as a supporting document. The study originally aimed to measure only improvements in health workers' knowledge, skills and competency. The measurement of newborn mortality was conceived later, leading to late registration.
Sample size: sample-size calculation was based on the ability to detect a 20% increase in knowledge, practical skill, and competency, as well as a 20% reduction in newborn mortality, with an error of 0.05, 20% and a dropout rate of 50%. Using G*Power v. 3.1, we determined a sample size of 74 participants in each group to account for losses. However, due to the ongoing conflict in South Sudan, the actual participants for both groups were less than the estimated sample size.
Selection criteria: the selection criteria were as follows: medical officers/doctors, nurses, midwives, maternal child health officers, community health workers, and clinical officers working and practicing in maternities, operating theaters, or children's wards; health providers self-reporting that they provide routine care services in delivery and neonatal units or departments and health workers willing to be available for data collection during the period of study.
Recruitment setting: health workers were identified and recruited from maternity wards, newborn operating theaters, and children's wards at Juba Training Hospital (intervention site) and Wau Teaching Hospital (control site). After completing the recruitment process, invitations were sent to those who met the criteria to participate in the study. Written informed consent was obtained from the participants. All newborns delivered in maternities, newborn units and operating theatres who met the inclusion criteria were included in the study. criteria. The objective of this study was to assess changes in health workers' knowledge, psychomotor skills, and competency regarding managing neonates with birth asphyxia after receiving training, as well as the changes in perinatal mortality rates due to asphyxia within 24 hours of birth. We hypothesized that HBB training would result in a 20% increase in knowledge and skills, and a corresponding reduction in mortality among newborns with asphyxia. The primary expected outcome was an improvement in health workers' knowledge, skills, and competency after HBB training. The secondary expected outcome was a reduction in mortality among newborns with asphyxia in the first 24 hours after birth. The study design was approved by the South Sudan Ethical Committee, Ministry of Health. Raters assigned a 1 when the task was correctly completed and a 0 otherwise. Health workers needed to obtain 80% or above in all three aspects of knowledge, skills and competency to be considered competent to help newborns breathe. Post training and intervention assessment were done immediately and at three months using the HBB knowledge, skills and competency checklist. Assessment were categorized into knowledge, practical skills and competency. The sum of the scores was calculated by determining the degree of completion of each task in knowledge, skills and competency. The total score for each task was 80% for knowledge, skills and competency. All participants who completed the initial HBB training and were present one year later were asked to participate in the HBB assessments to determine their long-term knowledge, skills and competency retention. The one-year assessment used the same checklist that was used in the initial training. In the one-year evaluation, each participant was assigned a knowledge, skills and competency score presented as the percentage of correctly performed items.
Participants were asked to complete the same five-point Likert scale survey and express their confidence in HBB practice. The mean scores and significance levels within and between the groups were tested using repeated ANOVA and chi-squared tests in the three performance areas of knowledge, skills and competency. Newborn mortality was determined and tested with Pearson's chi-squared test.
Statistical analyses were performed using SPSS v. 20 and p<0.05 was considered statistically significant.    (Table 3).

Discussion
This study aimed to evaluate knowledge, skills and competency retention as well as the effect of HBB training on newborn mortality at a public tertiary hospital in South Sudan. To our knowledge, this is the first rigorous study of HBB conducted in South Sudan The knowledge, skills and competency of the intervention and control participants were evaluated one-year post training and compared to the status at three months.

Participants' HBB knowledge
The mean knowledge passed rates using the HBB multiple-choice questionnaire were 42.5% and 48% for the intervention and control groups, respectively. There were no significant differences between the intervention and control groups during the baseline assessment, Importantly, the health workers who received training were able to achieve high scores in simulated environments despite a lack of previous HBB training. Furthermore, the results showed that helping babies is an effective way to address a lack of knowledge among the health workers who are often the first to act in resuscitating newborns with asphyxia. A study of HBB implementation in Honduras obtained a mean baseline knowledge score of 46%, which is close to our results for both the intervention (42.5%) and control groups (48%) [16]. The same study found that health workers with prior training scored higher at 69%. This is different from our results for the control group, where most had been exposed to resuscitation training but had lower scores. In India, health workers receiving HBB training for the first time did not score higher than those who had received it previously.
Similarly, our study obtained low baseline mean knowledge scores for first-timers, raising concerns about the retention of HBB resuscitation knowledge [16]. The level of knowledge attained after training declined at three months and further declined after one year. In an evaluation of HBB education in Kenya, the pass rate for knowledge increased among trainers (nurses and doctors) from 75% to 95% after a similar training intervention. In the same training, the pass rate for the simulated practical skills and competency was 20% for the health workers (learners) [11,14]. Using a simulated learning environment greatly improved the participants' knowledge of neonatal resuscitation. However, that knowledge was not sufficiently retained after one year. The creators of HBB training noted that longterm knowledge retention posed a significant challenge. Most prior studies have found a similar rapid deterioration in acquired skills and, to a lesser extent, knowledge in the months following the training [17].

HBB psychomotor skills and competency
In most prior studies of HBB training and evaluation, baseline BMV scores were not obtained [18]. Despite this decrease, there was a significant reduction in neonatal mortality. It was surprising to find that health workers retained skills and competency after three months. Similar studies of skills and competency retention in Rwanda and Kenya found that 3-6 months after intervention was the most difficult period for retaining such skills and competency [17].
Another study evaluating a performance checklist to assess neonatal resuscitation skills found that, despite significant improvements in overall test scores and pass rates after training, scores and improvements were disproportionate for those assessments that

Early neonatal mortality (ENM) within 24 hours
The training intervention in this study had an impact on newborn outcomes. The data indicated that, prior to the training, most newborns had not received resuscitation following HBB protocols.
Health workers were not following the correct steps for resuscitating newborns with asphyxia, including those who required stimulation or BMV for neonates with breathing problems. Additionally, the health workers had previously had difficulty properly identifying newborns with breathing problems and initiating resuscitation within the "golden minute". However, after HBB training and implementation, the participants' resuscitation skills increased twofold and such skills had been retained at the three-month follow-up.
This study also found a trend toward an overall reduction in newborn mortality within 24 hours three months after implementation at both sites. At the intervention site, newborn mortality within 24 hours decreased by half after the administration of resuscitation by the trained participants. Meanwhile, there was no significant decrease in ENM at the control site. Despite the notable decrease in early newborn deaths at the intervention site, the short period between the baseline and implementation was not sufficient for the results to be conclusive and generalizable. Thus, there is a need for a longer period between baseline and implementation given the variations in newborn registry data, especially in political conflict areas. Nevertheless, this study has shown that training health workers has an effect on their knowledge and practical skills, in addition to reducing newborn mortality.
Few existing studies have investigated the long-term effects of HBB training on early neonatal outcomes. One large before-and-after study of eight hospitals in Tanzania found that HBB training and implementation was associated with a significant reduction in ENM, fresh stillbirth rates and early perinatal mortality [19]. In that study, ENM was reduced from 13.4 to 7.1 per 1,000 live births, and the reduction in ENM was significant for both normal and low birth weights, as well as term and preterm infants. The only difference from our study is that we excluded stillbirth and considered neonates identified as having breathing problems at birth. Finally, we thank our research team for their tireless efforts through training, simulation rating, data analysis, and the production of the manuscript.       <0.001 Scores expressed as mean difference. *Significance level at 0.005 post intervention, 3-month follow-up, and one year. P-value within intervention group tested by repeated ANOVA, 0.001, and 0.001 between immediate post intervention and 3-month followup. **No baseline conducted for control group for bag and mask, OSCEA&B.