https://orcid.org/0000-0002-5813-4779
Figures
Abstract
Background
Helping Babies Breathe (HBB) is a newborn resuscitation training program designed to reduce neonatal mortality in low- and middle-income countries. However, skills decay after initial training is a significant barrier to sustained impact.
Objective
To test whether a mobile app, HBB Prompt, developed with user-centred design, helps improve skills and knowledge retention after HBB training.
Methods
HBB Prompt was created during Phase 1 of this study with input from HBB facilitators and providers from Southwestern Uganda recruited from a national HBB provider registry. During Phase 2, healthcare workers (HCWs) in two community hospitals received HBB training. One hospital was randomly assigned as the intervention hospital, where trained HCWs had access to HBB Prompt, and the other served as control without HBB Prompt (NCT03577054). Participants were evaluated using the HBB 2.0 knowledge check and Objective Structured Clinical Exam, version B (OSCE B) immediately before and after training, and 6 months post-training. The primary outcome was difference in OSCE B scores immediately after training and 6 months post-training.
Results
Twenty-nine HCWs were trained in HBB (17 in intervention, 12 in control). At 6 months, 10 HCW were evaluated in intervention and 7 in control. In intervention and control respectively, the median OSCE B scores were: 7 vs. 9 immediately before training, 17 vs. 21 immediately after training, and 12 vs. 13 at 6 months after training. Six months after training, the median difference in OSCE B scores was -3 (IQR -5 to -1) in intervention and -8 (IQR -11 to -6) in control (p = 0.02).
Citation: Chan NH-M, Merali HS, Mistry N, Kealey R, Campbell DM, Morris SK, et al. (2023) Utilization of a novel mobile application, “HBB Prompt”, to reduce Helping Babies Breathe skills decay. PLOS Glob Public Health 3(5): e0000705. https://doi.org/10.1371/journal.pgph.0000705
Editor: Julia Robinson, PLOS: Public Library of Science, UNITED STATES
Received: June 6, 2022; Accepted: March 20, 2023; Published: May 8, 2023
Copyright: © 2023 Chan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data sets available upon request due to data containing potentially identifying and sensitive information. Please contact Francis Bajunirwe (fbaj@must.ac.ug) from the Mbarara University of Science and Technology Research Ethics Committee to request access to data.
Funding: This study is funded by Grand Challenges Canada Stars in Global Health, project number: R-ST-POC-1707-07454. All authors on the manuscript were the named as recipients of this grant (NHMC, HSM, NM, RK, DMC, SKM, SD). The funder had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. https://www.grandchallenges.ca/programs/stars-in-global-health/.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: Santorino Data is a co-creator of the Augmented Infant Resuscitator, which is used in phase 2 of this study. All other authors declare that they have no competing interests.
Introduction
Neonatal mortality continues to be a critical problem worldwide. The first 28 days of a child’s life is the most vulnerable period for survival and, in 2020, 2.4 million newborns died in this period [1]. Sub-Saharan Africa continues to have a very high neonatal mortality rate (NMR), with 27 deaths per 1000 live births, including Uganda which is estimated to have an NMR of 19 [1]. If we are to achieve the Sustainable Development Goal (SDG) of reducing the NMR to 12 by 2030, then significant advancement in providing quality healthcare by skilled health workers is crucial [2].
Helping Babies Breathe (HBB) is an evidence-based newborn resuscitation training program designed to be used in low-resource settings [3]. Utilising the principles of adult learning, HBB training sessions incorporate small group discussion, peer-to-peer facilitation, simulation, and debriefing. Sessions are run with one instructor and six participants split up into groups of two. Each pair uses a NeoNatalie (Laerdal Global Health) simulator, a low-cost mannequin that can demonstrate chest wall excursion and an umbilical pulse [4]. Since 2010, HBB has reached an estimated 850,000 providers in 158 countries [5]. In large-scale studies, HBB has been shown to decrease early neonatal mortality and fresh stillbirths in low- and middle-income countries (LMICs) [6–8]. One of the primary challenges for HBB, as with any educational program however, is the knowledge and skills drop-off that occurs over time [9–13]. Frequent practice both with self and peer-to-peer learning [11, 14, 15], as well as expert feedback have been shown to be helpful methods to sustain competencies [16]. In a recent qualitative study examining 10 years of HBB facilitator data, one of the themes that emerged as critical to the program was “frequent and sustained hands-on practice during and after the course [5].”
One method to facilitate low-dose high frequency (LDHF) practice is the use of mobile applications [17–20]. Mobile applications lend themselves well in assisting HCWs, given the: ubiquity of smartphone availability in LMICs [21, 22], the cost savings of using mobile health technologies (mHealth) [23], and, as the global COVID-19 pandemic has shown, the importance of being able to access this information outside of a traditional classroom-based approach [24]. In the decade since HBB was launched, several digital support tools have been created to assist facilitators and providers with data collection, education and training, clinical decision support, and quality improvement [25]. The applications that assist with education and training provide digital information [26–29], questions [27–29], case-scenarios for practice [29], and, more recently, virtual reality (VR) scenarios that can be used with low-cost VR headsets [25, 30]. It is within this context that a novel mobile application, HBB Prompt, was created in 2019 [31]. This mobile application was created using user-centered design (UCD) [32] and has four parts, including: Training mode, to demonstrate with videos the steps of the HBB algorithm; Simulation mode, to facilitate individual and peer-to-peer practice of resuscitation scenarios; Quizzes/Knowledge check, to refresh knowledge about HBB; and a Scoreboard/Dashboard that summarises user performance and app usage, to increase user motivation and accountability for maintaining skills.
The purpose of this study is to report on the results of our comparative analysis, which took place at two hospitals in Southwestern Uganda.
Methods
Study overview
This was a comparative study at two hospitals in Southwestern Uganda, in collaboration with the Mbarara University of Science and Technology (MUST). Details of the protocol can be found in the study’s published protocol paper [31]. The study (Clinicaltrials.gov: NCT03577054) consisted of two phases. Phase 1 was the development phase where HBB Prompt was created via an iterative approach utilizing UCD and focus group discussions (FGD) [32]. Phase 2 is the assessment phase where we collected data at the two hospitals—one that was exposed to the app, and the other that was not. This paper focuses on Phase 2 of the study. One significant change from our protocol is that we were only able to follow the defined outcomes to six months, rather than one year. This was due to the COVID-19 pandemic that occurred in the second half of our data collection period, which restricted in-person assessments, gatherings for group practice, and required redeployment for some of the HCWs in the study.
Objectives
At the outset of the study, we had three main objectives. The first was completed in Phase 1 of the study and that was to develop HBB Prompt through UCD and input from frontline birth attendants in Uganda. The remaining two objectives for this study, modified for the follow-up timeframe, were:
- Train a minimum of 20 health workers from two district hospital in Southwestern Uganda in HBB.
- Determine the impact of HBB Prompt on HBB skills retention of providers in the intervention site compared with providers in the control site as measured by the HBB version 2.0 OSCE B test scores.
Participants and hospital intervention
Participants for Phase 2 of this study included frontline birth attendants from two district level hospitals in Southwestern Uganda that have HBB providers but not trainers. Written consent to participate was obtained both from the medical superintendent at each hospital and the individual birth attendants. Using a random sequence generator Kitagata Hospital was randomly assigned to be the intervention hospital and Itojo Hospital was assigned to be the control hospital.
HBB Prompt
User-centred design of HBB Prompt.
HBB Prompt was designed in Phase 1 of this study with a participatory UCD approach. HBB trainers and providers were invited to participate in an iterative process to help design a tool to improve HBB learning and skills maintenance. Participants provided input regarding the process of learning and maintenance of HBB skills and knowledge, which informed the development of the key components of HBB Prompt. Subsequently, participants interacted with a prototype version of HBB Prompt (v1) and gave feedback through FGDs to further optimize the functionality, content, flow, and usability of the app. For example, peer-to-peer learning and timely feedback were identified as crucial aspects of HBB skills maintenance; therefore, HBB Prompt was designed to have Simulation mode to help facilitate real-time simulation of applying HBB skills to newborn care. Within Simulation mode, rater mode shows users an HBB checklist that can be used to provide feedback serving as a peer-to-peer learning tool. Further details can be found in our publication outlining the details of HBB Prompt [32].
HBB Prompt app features.
HBB Prompt v2 was available to participants in the intervention hospital for the duration of the study. The key components of the app included Training mode, Simulation mode, Quizzes/Knowledge check, and a Scoreboard/Dashboard (Fig 1). Training mode consisted of a series of videos to demonstrate various steps in the HBB Action Plan and how to reprocess equipment. Simulation mode assisted providers in going through HBB practice scenarios in real time either alone or with a partner. Simulation mode was structured to transition in three phases: preparation, scenario execution and review with the checklist. In the preparation phase, users are provided the scenario stem and time to assemble all equipment and supplies for the scenario. During the scenario, the app displays the state of the baby using audio cues and associated graphics from the HBB Action Plan. Users are expected to follow the HBB Action Plan to appropriately manage the baby as the scenario progresses. At the end of the scenario phase, a performance checklist for that scenario appears on the screen to aid personal reflection on performance or peer feedback. The Quizzes/Knowledge check section provided practice questions to users with feedback to assist with retention of key pieces of information. Finally, the Scoreboard/Dashboard feature allowed users to see personal performance as well as comparison with other users.
Main menu screen for HBB Prompt showing the four main app features: Training mode, Simulation Mode, Quizzes/Knowledge check and Scoreboard/Dashboard.
Phase 2
Following recruitment, each HCW underwent standard HBB training in-person with a Master Trainer. Separate sessions were held at each of the two hospitals. A skills corner was set-up in each hospital that had a NeoNatalie simulator and resuscitation equipment so that participants could practise their skills. Participants in both hospitals were asked to practise their skills at least once per shift and record practice sessions in a logbook. In addition to training, the intervention hospital was provided with HBB Prompt on one designated Android tablet computer for the duration of the study. Intervention group participants were asked to use HBB Prompt as part of their regular skills practice. The control group did not have access to HBB Prompt and it was not made publicly available for download. Data analytics were stored on the tablet and subsequently downloaded to the HBB Prompt server at the end of the study. Phase 2 took place from April to October 2019.
Measurement of outcomes
During initial training, and at the 6-month follow-up, each participant completed a knowledge check, and OSCE B from the HBB program. At the end of the study, participants completed a questionnaire regarding their perception of HBB Prompt and had exit FGDs. See S1 and S2 Texts. The following outcomes were measured:
Primary Outcome
- Comparison of OSCE B scores in intervention versus control group at 6 months after training
Secondary Outcomes
- Comparison of knowledge assessments in intervention and control group before initial training and after 6 months.
- App analytics in the intervention group to assess pattern of usage, frequency of app use, and scores in quizzes.
- Exit FGDs at both sites to understand the impact of HBB Prompt and HBB training.
Data processing and analysis
Stata Corporation, Version 13.1 was used to perform statistical analyses. Descriptive statistics were used to compare baseline characteristics of participants in the intervention and control groups. OSCE B and knowledge check scores were compared using Wilcoxon ranked-sum analysis. In the intervention arm, app analytic data on practice frequency were compared with logbook data using the Kappa statistic. For the FGDs, audio recordings were transcribed and then qualitatively analyzed to identify major themes. For the intervention group, multivariate linear regression models were used to assess variables that may confound the primary outcome, including: frequency of practice and frequency of app usage. For the purposes of this study, only simulation practice was recorded and analyzed, and not any clinical practice outside of the study protocol.
Results
In total, 29 HCWs were recruited for the study: 17 in the intervention cohort and 12 in the control cohort (Fig 2). Participant demographic and clinical characteristics were similar across the two sites (Table 1).
Six months after HBB training, 10 HCWs were evaluated in the intervention group and 7 in the control arm. The OSCE B has a total score of 23, and the knowledge check has a maximum score of 18. When examining only participants who were evaluated six months after training, there was a statistically significant difference in the OSCE B scores with a median difference of -3 (IQR -5 to -1) in the intervention group and -8 (IQR -11 to -6) in the control group (p = 0.02). The median difference in knowledge check scores was 0 (IQR 0 to 0) in the intervention group and 0 (IQR -1 to 1) in the control group (p = 0.68). Fig 3A and Table 2 illustrate the distribution of OSCE B scores at different time points for every participant evaluated at all time points including those who were lost-to follow-up at six months. Fig 3B shows only the participants who were evaluated at six months. Fig 4 and Table 3 show the distribution of knowledge check scores. 17 out of 23 is considered a passing score and only 2 participants (both in the intervention arm) had passing scores.
(A) Both groups had significantly improved scores immediately after training (p = 0.0003 in intervention, p = 0.002 in control). (B) When those lost-to follow-up were excluded, both groups still had significantly improved scores immediately after training. Within each group, there was also a significant decrease in scores six months post-training when compared with immediately after training (p = 0.006 in intervention, p = 0.02 in control). The intervention group showed less skills decay after six months, as indicated by smaller change in median difference of OSCE B scores (p = 0.02).
Both groups showed improvement in knowledge after training (p = 0.004 in intervention, p = 0.003 in control). In contrast to skills retention, knowledge retention remained high in both groups with no significant decay (p = 1 in intervention, p = 0.66 in control).
Lost-to follow-up was similarly high in both groups, 7 of 17 (41%) participants in intervention and 5 of 12 (42%). Reasons for lost-to follow-up were also similar, including leaves of absences and moving away from the district (Fig 2). For participants evaluated at six months, median deliveries per month at the start of training and years of experience did not significantly differ between the intervention and control groups (Table 4). Within each group, participants who were evaluated compared with those who were lost to follow-up did not differ in their years of experience as a healthcare provider. In the intervention group, participants who were evaluated attended less deliveries per month compared to those who were lost-to follow-up, 15 (IQR 0 to 20) vs. 30 (IQR 20 to 32), p = 0.03.
Average delivery volume per month was 350 in the intervention site and 200 in the control site. Three-month running average rates of infants with asphyxia and fresh stillbirths are shown in Table 5. Birth asphyxia was defined as a failure to initiate or sustain spontaneous breathing at birth [33]. Compared to the baseline rate in the three months prior to HBB training, the rate of infants with asphyxia decreased significantly in the intervention site six to nine months after HBB training (p = 0.0004). When compared to baseline rates at each site, there was a downward trend in fresh stillbirth rates in the intervention site that was not statistically significant (p = 0.37) while there was an upward trend in the control site (p = 0.43).
App usage in intervention group
Of 17 trained users in the intervention group, six individuals (four of whom completed the six-month evaluation) used HBB Prompt with their own unique login. App usage varied across participants who used the app, and frequency of use increased three months after initial HBB training (Fig 5A and 5B). One user represented nearly half of the total logins, 17 out of 31. Of note, users continued to use the HBB Prompt after the intervention period, including users who had not previously logged in before. The most frequently used feature of the app was Quizzes/Knowledge check, followed by Training mode and Simulation mode (Fig 6). No users accessed the Scoreboard/Dashboard feature.
(A) Usage by login frequency per user and (B) usage over time, by months since HBB training.
Using simple linear regression to examine the relationship between app usage and HBB skills and knowledge revealed no significant findings. No relationship was found between increased app usage and the OSCE score at six months, or the change in OSCE score immediately after training compared to six months after training. Similarly, there was no relationship between increased app usage or any app usage and knowledge check score at six months, or the change in knowledge check score immediately after training compared to six months after training. Simple linear regression to evaluate the influence of years of experience or deliveries per month on the OSCE score at six months, or change in OSCE score did not reveal any significant findings in both intervention and control groups.
Both intervention and control sites were asked to track their practice patterns using a logbook. In the intervention site, the logbook entries did not match the time stamps of app usage for nearly all the entries. Logbook data was requested but the logbook was not used by participants at the control site, although these healthcare workers did say they practised. However, because it was not possible to ascertain which users practised and the frequency of practice, it was not possible to conduct analysis on the influence of practice on OSCE performance in the control group.
Focus group discussion findings
In the FGD at the control site, themes emerged regarding the importance of HBB training, the lack of continuing practice, disagreement between healthcare providers regarding neonatal resuscitation steps, and potential solutions to improve practice (Table 6). The discussion primarily focused on the lack of continuing practice and reasons that were given included not being required to be at a resuscitation, workload demands, and concerns of what colleagues would think of their skills if they were inadequate. Potential solutions included having a dedicated partner and reserved time for practice.
At the intervention site, themes emerged regarding the utility of the app including the value of keeping consistency in practice to providers, the usefulness of the app as a clinical reminder, and the value of using the app to get feedback (Table 6). The participants stated that they had not used Simulation mode for a wide variety of reasons including the difficulty of finding partners who had time, workload demands, and only having the app available on one device in the hospital. They did say, however, that the videos and quizzes were used frequently as a clinical reminder. For improvement, several participants discussed the importance of receiving reminders. Feedback collected from anonymous surveys regarding HBB Prompt was consistent with findings from the FGD (Table 7).
Discussion
Over a six-month period, there was less skills decay in HCWs who had access to HBB Prompt, which had features that were developed to facilitate simulation-based practice of HBB skills. Our study results are in keeping with Umoren et al.’s [34] study on eHBB, which demonstrated the superior effect of supporting HBB training with VR technology compared to videos alone (passive reinforcement) or standard of care (access to HBB algorithm and teaching materials). In our study, the Simulation—Alone mode was used for the greatest amount of time, compared to other app features. This is similar to how eHBB is used as that program only allows for single user simulation. More broadly, these findings suggest that improving skills retention can be achieved with technology that supports simulation, an educational method that has gained increasing importance in a wide variety of pediatric and adult resuscitation education programs, including HBB [4, 35].
Overall, there was low frequency of practice captured by the app, with the highest app use during the latter part of the intervention period, potentially because participants knew there was to be an upcoming evaluation, or they recognized that knowledge and/or skills decay occurred and responded by refreshing skills and knowledge. Unfortunately, users in both sites did not adequately record their practice sessions using the logbook that was provided, and so practice trends could not be compared reliably between sites. Practice frequency may not have been well captured if users did not use the app as intended. Use of new technologies such as manikins that can capture practice frequency such as “NeoNatalie Live” (Laerdal Global Health) may mitigate this lack of objective data in the future [34]. Even though usage of HBB Prompt was low, it may be possible that horizontal transfer of knowledge occurred, or group practice occurred that was not captured by the way users accessed the app.
While both sites had decreasing rates of infants with asphyxia after HBB training, the decrease was only statistically significant in the intervention site. Additionally, while there was an upward trend of fresh stillbirth rates in the control site, there was a downward trend in the intervention site that appeared to be sustained six months after HBB training. Although these findings are encouraging, our study was not powered to discern whether these clinical trends were an effect of having access to the app or if they were due to other factors that were not captured in our dataset.
Lost to follow-up was significant in both groups, which further undermined the small number of participants in this pilot project. Although it is common for HCWs to be moved around from one health facility to another within regions in Uganda, the leaves of absence were not expected. It would have been difficult and inappropriate to exclude participants at the outset due to upcoming leaves. Another limitation regarding the healthcare workers in this study is that we could not capture informal methods of training prior to the study. Our demographic data specifically asked about formal course completion, and given that HBB has been in Uganda since 2010, some healthcare workers were exposed to HBB methodology prior to taking the formal course. The Ugandan Ministry of Health has also led initiatives to build capacity with HBB and provide facility-based mentorship to newborn providers. Of note, older HBB 1st Edition posters were accessible in delivery rooms at both hospitals prior the start of this study. Data collection was affected by the COVID-19 pandemic and although the study protocol was set for 12 months after HBB training, issues with funding and the pandemic resulted in early termination of the study.
Similar to findings of other resuscitation training programs, knowledge retention does not translate to skills retention [10]. Therefore, ongoing efforts to improve sustainability of training need to focus on how to maintain resuscitation skills. Although Phase 1 [32] of this study attempted to address the issue of sustainability by focusing on UCD of HBB Prompt to suit end-user needs for acquisition and maintenance of skills of knowledge, the low frequency of usage of the app appeared to be mostly influenced by daily realities and limitations due to structural factors and daily workflow. While some of these issues were raised during the design phase, they were deemed out of scope of the development of HBB Prompt. The results of Phase 2 of this study suggest that more attention needs to be paid to address barriers in daily realities of HBB providers, such as workload, with the goal of integrating strategies for skills maintenance into daily workflow. Adding an extra tool like HBB Prompt that is not seamlessly integrated into the regular routine of HCWs may have been a reason for low app usage. Better mapping of the local HCW division of labour and workflow may have helped improve the ability to facilitate practice of HBB skills. For example, understanding the rotational nature of health workers going from labour and delivery to the postpartum and newborn wards and consideration of using natural transition points may improve uptake of HBB practice.
While health system challenges may be complex and difficult to address, employing UCD to understand the impacts of these challenges at the end-user level can still be advantageous in improving HBB Prompt or other solutions that increase sustainability of HBB. The scope of UCD can be expanded beyond just understanding how users will use a tool itself to more broadly understand how the tool will be integrated within the existing environment. Through an iterative process of UCD, systems-based gaps can be identified, and solutions tested. Use of UCD is still a worthwhile undertaking as it tailors interventions to the needs of end-users, and can improve efficiency and effectiveness of solutions.
Examining various aspects of usage can help better understand trends. For example, frequency of access versus duration of access may differ depending on the nature of how users interacted with the app. In this study, the total and average time spent in Simulation mode was higher (total 104 minutes, average 5.8 minutes) than in Training mode (total 50 minutes, average 1.8 minutes). Within Simulation mode, only Simulation—Alone was utilized and not the Simulation—Group function. This may have been because users preferred to do their own simulations, or they may have had difficulty in finding peers for group practice, as they indicated in the FGDs. Although simulation is a methodology used in HBB training, using the app to practise is different than going through a peer or trainer-led simulation. Although one intention of creating Simulation—Alone mode was to facilitate individual simulation-based practice, users may not have spent adequate time training to be able to use this feature confidently on their own. More time spent with the target end users in the user-centred design cycle may have been able to uncover any additional app functions and features to improve usage. In terms of frequency, quizzes were the most accessed part of the app, followed by the training videos and Simulation mode. The self-study nature of quizzes, ease of use and instant gratification of getting answers marked as correct may have influenced the higher frequency of access to quizzes. Tracking app usage through built-in analytics enabled better understanding of user behaviour. However, one significant limitation to understanding app usage and its impact on skills decay was that one user represented nearly half of all logins. This likely affected the ability to determine any meaningful relationships at the group level when evaluating the link between app usage and intervention outcomes such as skills retention. Despite this limitation, there was a distinct difference in less skills decay in the intervention group, which may be secondary to potential unmeasured effects of having access to the app.
Having the app deployed on one tablet for the ward rather than on personal devices made it more difficult to coordinate access. On the other hand, it would have been challenging to demand specific technical requirements for personal devices or to offer internet credit in order to have their app analytic data uploaded. During Phase 1, participants had more time exposed to the app with the guidance of the study team. More streamlined tech support may have been helpful as access and login were cited as important initial barriers to using the app. During the FGD participants did not identify significant difficulties in using the app; however, they may not have been forthcoming with their true thoughts since the facilitators of the FGD were the study investigators. Similarly, the positive review of the app and its features as captured by the post-intervention survey was not congruent with the lack of app usage. This highlights the importance of ongoing and dedicated user-centred feedback to truly capture the nuances of how technology can be integrated into user’s daily activities, workflow and ultimately how to continue to adapt technology in a way that truly benefits users and targeted outcomes such as skills retention.
Reminders and strategies to prompt LDHF practice have been shown to be effective in other settings [17–20, 36]. The Scoreboard/Dashboard feature was created to help hold users accountable for their own practice habits and those around them; however, this feature was not utilized by the users at all. One potential explanation is that this feature is a passive way to motivate users to practise their skills. Employing strategies that rely less on users to login to the app to practise in the first place may be useful. For example, perhaps the dashboard could have been displayed at a central location, and linked to the tablet so that it would have been prominently displayed to trigger participants to turn on the app and practise. Specific targeted reminders that change with participant behaviour may be helpful as an improvement. For instance, if participant A has not practised in a month versus participant B has practised once a day, A’s reminder would highlight the lack of practise, and encourage them to start practising and B’s reminder would be more positive reinforcement.
Skills retention remains a significant challenge for sustainability of impact of initial HBB training. Refresher trainings and increased supervision are strategies that have been shown to improve retention [16, 20, 37]; however, these methods are resource intensive. Technology based solutions that are integrated into daily routines of users, such as mobile apps like HBB Prompt, enhanced manikins like “NeoNatalie Live” [34, 36] or VR technology like eHBB [34] may provide feasible solutions that are more easily scalable. Without specific integration into daily workflow, there was a trend towards improved retention with use of HBB Prompt, similar to eHBB VR. With ongoing careful design and integration into the routines of health workers at the frontlines, there is potential for technologies to exert an even larger impact [38].
Conclusions
There was less neonatal resuscitation skills decay in HCWs with access to HBB Prompt, despite low usage. Further studies on use of mobile apps to improve skills retention should focus on integration of the app into the daily routines of end-users to maximize the app’s utility.
Supporting information
S1 Text. Focus group discussion guide for control site at 6 months.
https://doi.org/10.1371/journal.pgph.0000705.s001
(DOCX)
S2 Text. Focus group discussion guide for intervention site at 6 months.
https://doi.org/10.1371/journal.pgph.0000705.s002
(DOCX)
S3 Text. HBB Prompt study protocol version 2018–02.
https://doi.org/10.1371/journal.pgph.0000705.s003
(PDF)
Acknowledgments
The authors acknowledge the work of Mukama Martin and Openja Moses at the Consortium for Affordable Medical Technologies in Uganda (CAMTech Uganda) for developing the software for HBB Prompt, Tumwesigye Paul for data collection, Sabrina Kernerman for editing the manuscript, and Dr. Diego Bassani for assistance in study planning. The authors also acknowledge the innovation design process of the CAMTech Uganda that enabled the development of the HBB Prompt app using UCD approaches.
References
- 1.
Sharrow D, Hug L, Lee S, Liu Y, You D. Levels and trends in child mortality—Report 2021—Prepared by UN-IGME. UNICEF; https://data.unicef.org/wp-content/uploads/2022/01/UNICEF-IGME-2021-Child-Mortality-Report.pdf
- 2. Patterson J, Niermeyer S, Lowman C, Singhal N, Kak LP. Neonatal Resuscitation Training and Systems Strengthening to Reach the Sustainable Development Goals. Pediatrics. 2020;146: S226–S229. pmid:33004645
- 3. Niermeyer S, Little GA, Singhal N, Keenan WJ. A Short History of Helping Babies Breathe: Why and How, Then and Now. Pediatrics. 2020;146: S101–S111. pmid:33004633
- 4. Singhal N, McMillan DD, Savich R, Matovelo D, Santorino D, Kamath-Rayne BD. Development and Impact of Helping Babies Breathe Educational Methodology. Pediatrics. 2020;146: S123–S133. pmid:33004635
- 5. Merali HS, Visick MK, Amick E, Savich RD. Helping Babies Survive: Lessons Learned From Global Trainers. Pediatrics. 2020;146: S134–S144. pmid:33004636
- 6. Msemo G, Massawe A, Mmbando D, Rusibamayila N, Manji K, Kidanto HL, et al. Newborn Mortality and Fresh Stillbirth Rates in Tanzania After Helping Babies Breathe Training. Pediatrics. 2013;131: e353–e360. pmid:23339223
- 7. Goudar SS, Somannavar MS, Clark R, Lockyer JM, Revankar AP, Fidler HM, et al. Stillbirth and Newborn Mortality in India After Helping Babies Breathe Training. Pediatrics. 2013;131: e344–e352. pmid:23339215
- 8. Kc A, Wrammert J, Clark RB, Ewald U, Vitrakoti R, Chaudhary P, et al. Reducing Perinatal Mortality in Nepal Using Helping Babies Breathe. Pediatrics. 2016;137. pmid:27225317
- 9. Reisman J, Arlington L, Jensen L, Louis H, Suarez-Rebling D, Nelson BD. Newborn Resuscitation Training in Resource-Limited Settings: A Systematic Literature Review. Pediatrics. 2016;138: e20154490. pmid:27388500
- 10. Bang A, Patel A, Bellad R, Gisore P, Goudar SS, Esamai F, et al. Helping Babies Breathe (HBB) training: What happens to knowledge and skills over time? BMC Pregnancy Childbirth. 2016;16: 364. pmid:27875999
- 11. Morris SM, Fratt EM, Rodriguez J, Ruman A, Wibecan L, Nelson BD. Implementation of the Helping Babies Breathe Training Program: A Systematic Review. Pediatrics. 2020;146: e20193938. pmid:32778541
- 12. Jani P, Blood AD, Park YS, Xing K, Mitchell D. Simulation-Based Curricula for Enhanced Retention of Pediatric Resuscitation Skills. Pediatr Emerg Care. 2021;37: e645–e652. pmid:31305500
- 13. Yang C-W, Yen Z-S, McGowan JE, Chen HC, Chiang W-C, Mancini ME, et al. A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation. 2012;83: 1055–1060. pmid:22391016
- 14. Drake M, Bishanga DR, Temu A, Njozi M, Thomas E, Mponzi V, et al. Structured on-the-job training to improve retention of newborn resuscitation skills: a national cohort Helping Babies Breathe study in Tanzania. BMC Pediatrics. 2019;19: e396. pmid:30732580
- 15. Gurung R, Gurung A, Sunny AK, Basnet O, Shrestha SK, Gomo ØH, et al. Effect of skill drills on neonatal ventilation performance in a simulated setting—observation study in Nepal. Stem Cell Res Ther. 2019;10: 189.
- 16. Mduma E, Kvaløy JT, Soreide E, Svensen E, Mdoe P, Perlman J, et al. Frequent refresher training on newborn resuscitation and potential impact on perinatal outcome over time in a rural Tanzanian hospital: an observational study. BMJ Open. 2019;9: e030572. pmid:31562152
- 17. Mduma E, Ersdal H, Svensen E, Kidanto H, Auestad B, Perlman J. Frequent brief on-site simulation training and reduction in 24-h neonatal mortality—an educational intervention study. Resuscitation. 2015;93: 1–7. pmid:25957942
- 18. Arabi AME, Ibrahim SA, Manar A-R, Abdalla MS, Ahmed SE, Dempsey EP, et al. Perinatal outcomes following Helping Babies Breathe training and regular peer–peer skills practice among village midwives in Sudan. Archives of Disease in Childhood. 2017;103: 24–27. pmid:28821501
- 19. Gomez PP, Nelson AR, Asiedu A, Addo E, Agbodza D, Allen C, et al. Accelerating newborn survival in Ghana through a low-dose, high-frequency health worker training approach: a cluster randomized trial. BMC Pregnancy Childbirth. 2018;18: 72–83. pmid:29566659
- 20. Ugwa E, Kabue M, Otolorin E, Yenokyan G, Oniyire A, Orji B, et al. Simulation-based low-dose, high-frequency plus mobile mentoring versus traditional group-based trainings among health workers on day of birth care in Nigeria; a cluster randomized controlled trial. BMC Health Serv Res. 2020;20: 586. pmid:32590979
- 21. Lee S, Cho Y, Kim S-Y. Mapping mHealth (mobile health) and mobile penetrations in sub-Saharan Africa for strategic regional collaboration in mHealth scale-up: an application of exploratory spatial data analysis. Globalization and Health. 2017;13: 932. pmid:28830540
- 22. Madanian S, Parry DT, Airehrour D, Cherrington M. mHealth and big-data integration: promises for healthcare system in India. BMJ Health Care Inform. 2019;26: e100071. pmid:31488497
- 23. Iribarren SJ, Cato K, Falzon L, Stone PW. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. Mihalopoulos C, editor. PLoS ONE. 2017;12: e0170581. pmid:28152012
- 24. Merali HS, Wood A, Fernando A, Abdalla K, Amick E, Hemed M, et al. Telementoring Education and Mentorship for Newborn Providers in Kenya, Pakistan, and Tanzania. American Academy of Pediatrics National Conference & Exhibition—Virtual 2020.
- 25. Bucher SL, Cardellichio P, Muinga N, Patterson JK, Thukral A, Deorari AK, et al. Digital Health Innovations, Tools, and Resources to Support Helping Babies Survive Programs. Pediatrics. 2020;146: S165–S182. pmid:33004639
- 26. Crehan C, Kesler E, Nambiar B, Dube Q, Lufesi N, Giaccone M, et al. The NeoTree application: developing an integrated mHealth solution to improve quality of newborn care and survival in a district hospital in Malawi. BMJ Glob Health. 2019;4: e000860–12. pmid:30713745
- 27. Thomsen CF, Barrie AMF, Boas IM, Lund S, Sørensen BL, Oljira FG, et al. Health workers’ experiences with the Safe Delivery App in West Wollega Zone, Ethiopia: a qualitative study. Reproductive Health. 2019;16: 1319. pmid:31072399
- 28. Bolan NE, Sthreshley L, Ngoy B, Ledy F, Ntayingi M, Makasy D, et al. mLearning in the Democratic Republic of the Congo: A Mixed-Methods Feasibility and Pilot Cluster Randomized Trial Using the Safe Delivery App. Glob Health Sci Pract. 2018;6: 693–710. pmid:30591577
- 29.
Bucher S, Sammy B, Md C, Jecinta M, Bonnie M, Geoffrey M, et al. Using mobile phones to support Helping Babies Breathe training in Kenya. Pediatric Academic Societies Meeting 2015.
- 30. Ghoman SK, Patel SD, Cutumisu M, von Hauff P, Jeffery T, Brown MRG, et al. Serious games, a game changer in teaching neonatal resuscitation? A review. Archives of Disease in Childhood—Fetal and Neonatal Edition. 2019;105: 98–107. pmid:31256010
- 31. Merali HS, Chan NH-M, Mistry N, Kealey R, Campbell D, Morris SK, et al. Designing and evaluating a novel mobile application for Helping Babies Breathe skills retention in Uganda: comparative study protocol. BMJ Paed Open. 2019;3: e000561. pmid:31549001
- 32. Chan NH-M, Merali HS, Mistry N, Kealey R, Campbell DM, Morris SK, et al. Development of a novel mobile application, HBB Prompt, with human factors and user-centred design for Helping Babies Breathe skills retention in Uganda. BMC Med Inform Decis. 2021;21: 39. pmid:33541340
- 33. Moshiro R, Mdoe P, Perlman JM. A Global View of Neonatal Asphyxia and Resuscitation. Front Pediatr. 2019;7: 957–6. pmid:31850287
- 34. Umoren R, Bucher S, Hippe DS, Ezenwa BN, Fajolu IB, Okwako FM, et al. eHBB: a randomised controlled trial of virtual reality or video for neonatal resuscitation refresher training in healthcare workers in resource-scarce settings. BMJ Open. 2021;11: e048506. pmid:34433598
- 35. Mundell WC, Kennedy CC, Szostek JH, Cook DA. Simulation technology for resuscitation training: A systematic review and meta-analysis. Resuscitation. 2013;84: 1174–1183. pmid:23624247
- 36. Vadla MS, Mdoe P, Moshiro R, Haug IA, Gomo Ø, Kvaløy JT, et al. Neonatal Resuscitation Skill-Training Using a New Neonatal Simulator, Facilitated by Local Motivators: Two-Year Prospective Observational Study of 9000 Trainings. Children. 2022;9: 134. pmid:35204855
- 37. Mubeen K, Baig M, Abbas S, Adnan F, Lakhani A, Bhamani SS, et al. Helping babies breathe: assessing the effectiveness of simulation-based high-frequency recurring training in a community-based setting of Pakistan. BMC Pediatr. 2021;21: 555. pmid:34876070
- 38. Perlman JM, Velaphi S, Massawe A, Clarke R, Merali HS, Ersdal H. Achieving Country-Wide Scale for Helping Babies Breathe and Helping Babies Survive. Pediatrics. 2020;146: S194–S207. pmid:33004641