A comic based interactive digital intervention to enhance facilitation skills of nurse mentors in public facilities – results of a pilot intervention in Bihar, India

ABSTRACT Background Various trainings are designed to educate nurses to become simulation educators. However, there are no good strategies to sustain their learnings and keep them engaged. We developed a series of 10 interactive digital storytelling comic episodes ‘The Adventures of Super Divya (SD)’ to strengthen simulation educator’s facilitation knowledge, skills, confidence, and engagement. This endline evaluation presents results on the change in knowledge after watching the episodes and retention of that knowledge over 10 months. Objectives The objectives of this pilot study are to: 1) assess the change in knowledge between the baseline and post-episode surveys; and 2) understand the retention of knowledge between the post-episode and the endline survey. Methods A human-centred design was used to create the episodes grounded in the lived experience of nurse simulation educators. The heroine of the comic is Divya, a ‘Super Facilitator’ and her nemesis is Professor Agni who wants to derail simulation as an educational strategy inside obstetric facilities. Professor Agni’s schemes represent real-life challenges; and SD uses effective facilitation and communication to overcome them. The episodes were shared with a group of nurse mentors (NM) and nurse mentor supervisors (NMS) who were trained to be champion simulation educators in their own facilities. To assess change in knowledge, we conducted a baseline, nine post-episode surveys and an endline survey between May 2021 and February 2022. Results A total 110 NM and 50 NMS watched all 10 episodes and completed all of the surveys. On average, knowledge scores increased by 7–9 percentage points after watching the episodes. Comparison of survey responses obtained between 1 and 10 months suggest that the gain in knowledge was largely retained over time. Conclusions Findings suggest that this interactive comic series was successful in a resource limited setting at engaging simulation educators and helped to maintain their facilitation knowledge over time.


Introduction
Digital training in medicine is gaining foothold, albeit slowly, especially in low-and middle-income countries (LMICs). In this manuscript, we present a digital training modality with unique concepts for front-line intrapartum care mentors and providers in one of the poorest regions in South Asia, Bihar, India. Bihar is an eastern Indian state with neonatal mortality rate and maternal mortality ratio of 27 per 1,000 and 165 per 100,000 live births, respectively [1]. Despite having less than 10% of the country's population, Bihar accounts for approximately 15% of neonatal deaths [2][3][4]. Primary health centres (PHCs) are the most common location of birth for the over 44 million women in the state. However, the PHCs are plagued with limited supplies and staffing with no obstetricians or paediatricians [1, 3,5]. In 2015, the state Government of Bihar with a non-governmental organisation, CARE India, as the technical support unit, started implementing a statewide nursementoring programme aimed at reducing maternal and neonatal deaths. Shortly thereafter, PRONTO International was invited to join the initiative and integrate its highly realistic in-situ simulation and team-training into the nurse-mentoring programme [6,7]. The resulting programme created a cadre of trained facility-level nurses who were encouraged to become champion simulation educators and independently conduct simulations as well as teamwork and communication (T&C) activities at their home facilities. As the cadre of champion simulation educators grew, a strategy was needed to sustain their learning and keep them engaged.
The LINQED (Leading Innovation in Quality-of-Care Education Development) team developed an innovative series of interactive digital comic episodes 'The Adventures of Super Divya (SD)'-a simulation educator superhero [8]. The goal of these storytelling comic episodes was to strengthen simulation educator's facilitation knowledge, skills, confidence, and engagement. Comics and creative graphics have been used previously to communicate complex ideas about health [9]. The medium conveys nuanced concepts, explores emotional and social issues, and allows readers to develop a level of empathy beyond what can be achieved through textbooks [10]. Graphic and visualised contents have been shown to help readers retain information longer, improve comprehension, and understand complex concepts [9,[11][12][13]. In formal health education settings, comics have been used among medical, nursing, and psychiatry students, to teach diverse topics from empathy to anatomy [11,14,15]. Although there has been some research into the use of digital comics in teaching, it is not known whether this method is effective with frontline nurses, specifically those individuals from rural areas where they may not have good access to the internet or have trouble navigating a smart phone. We chose to use the interactive graphics format because the topics we needed to cover were abstract and subtle, focused on the development of learners' empathy skills, and facilitation of learning in resource-limited settings.
This endline evaluation presents results from the complete set of 10 SD comic episodes that were implemented with two cadres of simulation educators -nurse mentors (NM, frontline government staff nurses with a diploma in General or Auxiliary Nursing and Midwifery) and nurse mentor supervisors (NMS, nursing professionals in a mentoring role with a Bachelor's or Master's degree in Nursing, hired by CARE India). The objectives of this pilot study were to: 1) assess the change in knowledge between the baseline and post-episode surveys; and 2) understand the retention of knowledge between the postepisode and the endline survey.

Pilot study setting and participants
The idea of the SD intervention was conceived following the statewide AMANAT and AMANAT Jyoti programmes. AMANAT was a quality improvement initiative implemented by CARE India in collaboration with the Government of Bihar, in which among other components, visiting NMS conducted facilitybased capacity building activities including in-situ highly realistic simulations. NMS mentored and supervised government facility-based nurses following a repeated cyclical pattern over approximately 8-months [16,17]. To promote sustainability, AMANAT was followed by the AMANAT Jyoti programme, in which NMS trained a group of about 800 government staff nurses, the NM from about 400 facilities in Bihar. These peer mentors then provided simulation trainings in their respective facilities [18]. A success of the programme was the creation of the educator infrastructure needed to achieve change at scale and sustain it over time in the state. In order to support the sustainability efforts, we began development of the SD digital storytelling video episodes. These digital, interactive episodes were targeted at the NMS and NM who had two prior simulation educator trainings (a 5-day introductory simulation and T&C facilitation training, and a 4-day advanced training). They were also provided with reviews of topics previously covered as part of the AMANAT and AMANAT Jyoti curriculum prior to the introduction of SD comics. The digital SD episodes were asynchronous, allowing individuals to progress at their own speed.

Super Divya intervention
A human-centred design approach was used to create the storytelling episodes grounded in the lived experience of conducting simulation in the facilities in Bihar as well as to ensure the episode content and style were relatable [19]. The development of the episodes followed an iterative design, which included beta-testing, focus group discussions and a pilot survey before finalising and disseminating to the NM and NMS. After integrating feedback from the design process and from the initial qualitative discussions, 10 short (15-20 min) episodes in English and Hindi were designed by the LIFT Simulation Design Lab at the University of Utah in close collaboration with team members of PRONTO India Foundation and NuggetHead Studioz, LLC. Each episode focuses on a specific simulation facilitation topic with predefined objectives ( Table 1). The Hindi version of the episodes was sent via WhatsApp to NMS and NM working in PHCs, District Hospitals, Subdivisional Hospitals and Referral Hospitals in Bihar. Table 1 presents the objectives and timeline of each episode.
The Adventures of Super Divya stories revolve around two characters: Divya, a 'Super Facilitator' and her nemesis, Professor Agni. Divya's mission is to spread the magic of simulation and guide learners through practicing and understanding the nuanced elements of facilitation. Professor Agni's goal is to derail Divya's efforts by distracting simulation educators and leaners from meaningful engagement in simulation experiences and by taking away the safe learning space. Professor Agni's schemes are designed to represent real-life challenges that nurse educators face in their daily work. SD displays leadership and uses effective facilitation and communication to overcome hurdles encountered while conducting simulations. Each episode contains interactive components and quizzes to keep learners actively engaged.
The SD comic intervention uses metaphors to convey abstract concepts. SD has a set of magical tools in her PRONTO Pouch, which represent advanced skills that educators can use to focus and manage their biases and distractions ( Figure 1). For example: the Facilitator spray helps SD to recognise and name her own feelings, a critical step in selfreflection that is needed to prevent the educator from allowing their own issues into the learning space. The Empathy goggles help facilitators recognise clues from participant's body language and facial expressions that help them to empathise with how the participants might be feeling at that moment. The Curiosity antenna helps facilitators to stay curious and reminds them to ask open-ended questions during the simulation debrief. The Learning lens helps take 'snap shots' during the simulation scenario. These snapshots enable the facilitators to capture key moments during the simulation, link them to the overall programme learning objectives and then use those as examples in the debrief. In addition, the episodes introduce a set of Genuine self-practices, which help in establishing a practice of mindfulness and help the educator stay focused and grounded in the face of challenges. Genuine self-practices include: Ready-set-Breathe in which SD recommends taking a moment to pause before facilitating an activity, and take three deep breaths; the PRONTO pose recommends assuming a 'power' pose of one's choice and holding it for a minute while breathing deeply in and out, thus increasing confidence; Be Prepared minds the educator to prepare for the session by studying the curricular guide and to brainstorm solutions ahead of time for common problems encountered when conducting simulation; and Get Support helps to reach out to someone from work for support if you are struggling with any part of facilitation. The Energy Scanner is a 3 min guided meditation and body scan that the educators can use anytime they feel the need to recharge. All of these tools are used by SD to fight off a series of 'bugs' sent by Prof. Agni to derail the simulation. The bugs represent common educator and participant attitudes and distractions that pose a challenge to even the most experienced facilitator. Some of these bugs include fluster, fear, ego, disinterest, self-doubt, defensiveness, and frustration. The use of the bug imagery conveys the message that participants are not inherently difficult rather they have been bitten and are temporarily inflicted by the negative attitude that prevents their full engagement in the learning.

Survey description and sample selection
Web surveys in English and Hindi were designed using first Qualtrics and later Google (the shift was to overcome accessibility issues identified early on in the study). Prior to distributing the first episode, participants completed a baseline survey containing 20 true/false questions that covered topics from all 10 episodes (Figure 1, Supplementary Table S1). Participants then received a short post-episode survey after they watched each episode (Figure 1). Postepisode surveys repeated two questions (shown as B# in Figure 1) from the baseline survey and two to four new in-depth questions on the topic of the episode (shown as PE# in Figure 1). After the last episode, an endline survey was conducted that repeated all the new in-depth questions (i.e. PE 1-37 in Figure 1) from the post-episode surveys. The endline survey did not repeat the baseline survey questions because they were already administered twice and provided necessary data to measure change. Thus, we had the baseline survey, nine brief post-episode surveys and the endline survey. Across all surveys, each question was asked twice. One set of pre-post questions (B# in Figure 1) allowed us to assess change in knowledge, after watching the episodes. The second set of pre-post questions (PE# in Figure 1) allowed us to assess retention in knowledge, over 1-10 months. Nurses completed the surveys on their smart mobile devices. All of the questions were mandatory. The survey offered true/false, multiple choice, or select all that apply responses, as appropriate. The correct answers to each question appeared after the responses were submitted. A convenient sample of 148 NM out of 685 were selected based on their availability and willingness to participate. The selected NM were oriented by their respective NMS in their facility on how to access the SD digital episodes. A sample survey link was also shared. They were asked to watch a SD trailer on their smart phone and respond to the sample survey, to demonstrate that they grasped the necessary steps. All 77 NMS participated in the pilot intervention.

Statistical analysis
We matched the responses of individual NM and NMS across the baseline, post-episode and endline surveys. Score for an individual was the number of correct responses divided by the total number of questions multiplied by 100. For each participant, we generated their baseline score, two sets of postepisode survey scores and the endline score. The baseline score was generated using the responses to the 20 questions asked in the baseline survey ( Figure 1). The first set of post-episode survey scores (Post-episode score 1) was generated using the post responses to the 20 baseline questions (shown as B# in Figure 1) episode. The second set of post-episode survey scores (Post-episode score 2) was generated using the responses to the 30 new in-depth questions (not part of baseline) asked in the post-episode survey (shown as PE# in Figure 1). Finally, the endline score was generated using the repeat responses to 30 new in-depth questions repeated in the endline survey. We compared the baseline score with the post-episode score 1 to evaluate knowledge gained after they watched the SD episodes. Next, we compared the Post-episode score 2 with the endline score to evaluate knowledge retained by the end of the last episode. In other words, participants' responses were compared for the 20 baseline questions that were repeated in the post-episode surveys to understand immediate improvement, while the comparison of the 30 post-episode survey questions that were repeated in the endline helped to understand retention of knowledge. The individual scores are reported in four categories: 100%, 99-80%, 79-60% and below 60%. In addition, we present the mean scores with the standard deviation. Analyses were conducted and reported separately for NM and NMS. Excel and Stata 17.0 were used for the analysis, and SQUIRE guidelines were followed for reporting in this manuscript [20].

Results
A total 110 NM (out of 148, 74%) and 50 NMS (out of 77, 65%) participants completed all of the surveys and were included in this analysis. The vast majority (83%) of the NM had completed a diploma in general nursing and midwifery, while a majority (78%) of the NMS completed bachelor's degrees ( Table 2). About 94% of NM and 66% of NMS had participated either in the AMANAT or AMANAT Jyoti nursementoring programmes. 80% of the NM and all NMS were trained by PRONTO International on facilitating simulation and T&C activities either through in-person or virtual training prior to inclusion in this pilot study. About half (48%) of the NM and two-thirds (66%) of the NMS facilitated at least one simulation per month prior to COVID-19 related interruptions (Table 2).
Overall, the individual scores showed improvement between baseline and post-episode survey score 1 (Table 3). A higher proportion of NM scored in the 80-99% category in the post-episode survey score 1 (64%) than at baseline (30%). Simultaneously, a smaller proportion scored in the 60-79% category in the post-episode survey score 1 (32%) than at baseline (64%). Similarly, the NMS scores also showed an improvement (Table 3). For example, none of the NMS achieved 100% at baseline, whereas in the post-episode survey score 1, 8% of the participants scored 100%. Moreover, the proportion of NMS who scored in the 80-99% category increased from 40% at baseline to 72% in the post-episode survey score 1, whereas those in the 60-79% category decreased from 60% at baseline to 20% at postepisode survey score 1. The average scores between baseline and post-episode surveys score 1 increased by about 9 and 7 percentage points for the NM and NMS, respectively ( Table 3). Scores of individual questions are presented in Supplementary table S2. A comparison of the post-episode survey score 2 with the endline survey score indicates that both cadres largely retained knowledge over a period of maximum 10 months (Table 3). The proportion of NM who were in the 100% score category decreased from 9% in the post-episode survey score 2 to 0% at endline. However, in the 80-99% score category an upward trend was observed from 27% in the post-episode survey score 2 to 45% at endline. For the NMS, an upward trend was observed in the 100% score category as none of NMS achieved that score in the post-episode survey score 2, whereas in the endline survey 10% achieved that score. The average scores for the two timepoints were largely similar for both cadres (Table 3). When the questions that were asked both at the post-episode survey and in the endline survey were examined individually, we see less than 10% decrease in score between the two timepoints for all questions (Supplementary Table S3).

Discussion
In this pilot study, we explored whether The Adventures of Super Divya, an interactive digital comic series, could sustain and improve simulation educator knowledge among front-line intrapartum care nurses in Bihar, India. A comparison of results from surveys conducted at several timepoints suggests an improvement in knowledge. More importantly, the knowledge gained from the interactive comics was largely retained over a maximum of 10 months. Note, the maximum gap in time was 10 months between the 1 st and the 10 th episode, while the minimum gap was 1 month between the 9 th and the 10 th episode. These findings indicate that the comic episodes appear to be successful in improving and retaining knowledge pertaining to best-practices for successfully conducting simulation training (e.g. creating a safe learning environment, conducting a pre-brief, facilitating a debrief and understanding T&C techniques [21]).
A few gaps emerged that point towards the need for additional reinforcement for some of the novel and relatively complex concepts of a simulation, specifically related to conducting the pre-brief and phases of a debrief. While some of the new concepts (e.g. Genuine self-practice, PRONTO pose, Facilitator spray, Fluster bugs) were well-understood by the nurse educators, others (e.g. Empathy goggles, Curiosity antenna, Learning lens) require further investigation to understand whether the concepts behind the 'tools' were not grasped or the lack of change in scores was due to the manner of assessment. Like the specific concepts, individual episodes may have been at varying levels of difficulty, because performance across episodes varied, suggesting improvement and/or retention of knowledge may not have been uniform. For example, in the  *Post-episode score 1 was generated using the 20 baseline questions that were repeated in the post-episode surveys (shown as B # in Figure 1). # Post-episode score 2 was generated using the 30 new in-depth questions that were not part of the baseline survey (shown as PE # in Figure 1). ! Endline score was generated using the 30 new in-depth post-episode questions that were repeated in the endline survey.
comparison of the post-episode score 2 with endline, we found both cadres registered best performance in the Origin Story and Facilitation Secrets Part 2 episodes and worst performances in the Professor Agni Attacks the Pre-brief episode. Nevertheless, this study demonstrates that using digital comics, basic simulation and T&C facilitation concepts can be maintained and nuanced concepts can be introduced, even in this low-resource setting. To the best of our knowledge, this interactive comic series is the first report of using this modality to strengthen knowledge about complex facilitation concepts for frontline nurse educators in a resource-limited setting.
It has been suggested that the use of comics in healthcare education can provide an alternative mode for reflection, (compared to reflection through writing or verbal communication), foster thinking, help linking theory with practice, and simplify communication by adding humour [22]. Comics can also offer a safe space for expression [23], including sharing life experiences, though all of these instances are in the Western settings [24]. Recently, a report presented the perception of medical and nursing students about the use of comics in health education, in New Delhi, India. While only one-fifth reported having heard about graphic education, three-quarters wanted comics to be used in health training [25]. With the rapid spread of internet and mobile digital platforms, access to the digital environment is getting easier, even in the resource-limited settings. This study provides a proof of concept that digital comics can be effectively used for training frontline health providers.
The intervention was strengthened by the creativity and innovation of the designers through the use of a super hero nurse. Each episode had specific learning objectives, laid out at the outset of each episode. The graphics were attractive and aimed to be relatable because they were designed using images of local nurses and the settings reflected a health centre in a LMIC. The story line was appealing because of the depiction of clashes between a protagonist and an antagonist, in a classic good-vs-evil struggle. The nurse educators felt connected because the challenges depicted in the episodes were among those the nurses routinely encounter. Use of local language (Hindi) made the episodes culturally appropriate and easy to understand, though an English version is also available. Lastly, the design of the knowledge assessment surveys was similar to a case-crossover study where the participants served as their own controls.
There were several challenges, including difficulties accessing the episodes and registering responses to the online surveys, due to intermittent connectivity issues encountered by a small number of participants.
To alleviate these problems, PRONTO provided free data to watch the episodes and to complete the surveys. There was no way to ensure that nurses responded to post-episode surveys after viewing the episodes; some nurses may have responded to surveys without watching the SD episodes. It is also possible that varied participation in previous PRONTO and/ or AMANAT trainings as well as years of experience as a simulation educator may have influenced the results. We also faced challenges with the roll-out of the episodes and retention of participants over time. Initially, we distributed the episodes to 148 NM and 77 NMS, of which 110 NM and 50 NMS were retained. The loss of NM was due to reassignment to COVID-19 services or due to transfer to other departments within the same or another facility. The loss to follow-up may have introduced bias in our results. Due to constraints beyond the control of the study team, such as staff turnover or transfer to a different district during the AMANAT programme, we lost some NMS from this pilot project. Another limitation was not having a control group and the intervention was not assigned in a randomised manner, limiting the interpretational ability of the results. Finally, the initial selection of the NM and NMS was based on their availability, workload and willingness to participate in the SD intervention. Hence, results may or may not be generalisable to all NM and NMS.

Conclusion
Overall, the Adventures of Super Divya episodes appear to have helped improve and sustain knowledge for two cadres of simulation educators in Bihar. The results from this pilot study are encouraging, especially during a global pandemic when digital learning and training became imperative, not to mention the cost savings in using this digital environment. These findings suggest The Adventures of Super Divya interactive comic series is an innovative training tool for maintaining simulation educator knowledge and skills in resource-limited settings. Effective training and engagement of educators has the potential to improve the quality of obstetric and neonatal care and ultimately patient outcomes. and team training as part of AMANAT and AMANAT Jyoti programmes. We are inspired to have had the opportunity to work with the countless nurses and other health providers from public health facilities in Bihar. Last but not the least, our deepest gratitude to pregnant and birthing individuals for whom we are striving to make a small difference.