A mixed methods evaluation of Advanced Life Support in Obstetrics (ALSO) and Basic Life Support in Obstetrics (BLSO) in a resource-limited setting on the Thailand-Myanmar border [version 1; peer review: 2 approved]

Background: Short emergency obstetric care (EmOC) courses have demonstrated improved provider confidence, knowledge and skills but impact on indicators such as maternal mortality and stillbirth is less substantial. This manuscript evaluates Advanced Life Support in Obstetrics (ALSO) and Basic Life Support (BLSO) as an adult education tool, in a protracted, post-conflict and resource-limited setting. Methods: A mixed methods evaluation was used. Basic characteristics of ALSO and BLSO participants and their course results were summarized. Kirkpatrick’s framework for assessment of education effectiveness included: qualitative data on participants’ reactions to training (level 1); and quantitative health indicator data on change in the availability and quality of EmOC and in maternal and/or neonatal health outcomes (level 4), by evaluation of the post-partum haemorrhage (PPH) related maternal mortality ratio (MMR) and stillbirth rate in the eight years prior and following implementation of ALSO and BLSO. Results: 561 Thailand-Myanmar border health workers participated in ALSO (n=355) and BLSO (n=206) courses 2008-2020. Pass rates on Open Peer Review


Introduction
Skilled attendance at birth providing quality emergency obstetric and newborn care is an essential element for reducing the high burden of maternal and neonatal morbidity and mortality in resource-limited settings (RLS) 1,2 .
A number of practical skills or "hands-on" short course courses are aimed at improving emergency obstetric care (EmOC) 3 . The Advanced Life Support in Obstetrics (ALSO) course developed by the American Academy of Family Physicians is one of the oldest commencing in 1991, is standardized and evidenced-based, and available in multiple languages 4,5 . ALSO promotes inter-professional and multidisciplinary training to equip the maternity team (nurse midwives, registered nurses, physicians, residents and other members) to improve patient safety and positively impact maternal and newborn outcomes in obstetric emergencies. Positive ALSO evaluations from high-income countries 6,7 and low-income and RLS (Colombia, Guatemala, Honduras, and Tanzania) report decreased in-hospital maternal mortality, episiotomy use, and postpartum haemorrhage (PPH); and active management of third stage of labour and vacuum-assisted vaginal delivery increased in frequency after ALSO training 8 . In 2010 a systematic review in low-resource environments suggested there was a lack of strong evidence of the effectiveness of emergency obstetric trainings 9 . In 2020 a global review reported improved provider knowledge/skills and change in clinical practice but evidence for a reduction in the number of cases of post-partum haemorrhage (PPH), case fatality rates, stillbirths and institutional maternal mortality were less strong 3 .
The ALSO approach to learning is adult based. Learning suits professionals working in government health systems where tertiary education and certification of health care training is the norm. Tertiary qualified staff may not be available in postconflict and politically unstable RLS with broken education and health infrastructure 10 . Evidence for the type of learning and teaching suitable for healthcare workers in politically unstable and post-conflict settings is limited 11 . ALSO requires a degree of health literacy that may challenge healthcare workers based in countries rebuilding education and health systems 12, 13 .
Basic Life Support in Obstetrics (BLSO) is complementary to ALSO but structured for pre-hospital health care providers, first responders, and medical, nursing and physician assistant students 14 . The objective of BLSO is to improve the management of normal deliveries, as well as obstetric emergencies by standardizing the skills of frontline responders. BLSO is more difficult to compare internationally as there is more flexibility on topics included in the course, realizing the need for context appropriate training.
Reported success of training in emergency obstetric care in low-income countries within government health systems is commendable but are these replicable in post-conflict or politically unstable health care systems? 10,[15][16][17][18] Shoklo Malaria Research Unit (SMRU), a humanitarian and research organization on the Thailand Myanmar border, has provided antenatal care and childbirth facilities for refugees since 1986 and migrants since 1998. Facility based ALSO and BLSO training for Karen and Burmese healthcare workers was introduced in 2008. This was the first time the ALSO course has been held in a refugee camp. The objective of this analysis is to evaluate ALSO and BLSO as an adult education tool in emergency obstetric care, from the first course in 2008 to the most recently completed course in May 2020, in a protracted, post-conflict, RLS.

Study design
This mixed methods study collates several data sources to evaluate ALSO and BLSO as an adult education tool over the time period of 2008 to 2020. Before each course participants provided information about themselves using a standard form. Change in perceived confidence was measured before and after the course, and a sub-cohort (of the 2008 course) had a 12-month evaluation. The course theory and skill results were summarized. Course strengths were evaluated through an openended question. Finally, obstetric indicators including PPH related maternal mortality and stillbirth in the eight year block preceding the implementation of ALSO and BLSO and in eight years after provided were sourced from routine surveillance data.  Figure 1). Initially, P. falciparum infection accounted for the largest proportion of maternal mortality (estimated at 1,000 per 100,000 live births in 1985-1986 19 ) and was the focus of activity for the first few decades. In 1994 basic tools such as the World Health Organization (WHO) partogram in labour and essential medications such as magnesium sulphate for eclampsia and severe pre-eclampsia were introduced. To standardize interventions for common obstetric emergencies and medical complications SMRU produced a three language (Karen, Burmese and English) 'Manual of Obstetrics' in 1998, which has subsequently been revised three times (English only). As post-partum haemorrhage (PPH) 20 replaced malaria as the largest cause of maternal mortality, the need further training in obstetric emergencies was recognized.

Participants
Human resources. There are significant barriers to displaced persons in Thailand (refugees or migrants) obtaining formal training and certification as health workers in either Thailand or Myanmar. For more than three decades healthcare work by NGOs and humanitarian organizations on the border including SMRU and MTC have been provided by local Karen and Burmese from the community trained to work as medics, nurses, midwives, sonographers or counsellors as defined here: Medics: A 'medic' takes the role of doctor by doing clinical examinations, ordering laboratory tests, making the diagnoses and prescribing treatment. They have normally received a minimum of 6-12 months of theory training and a further 6-12 months of on the job practical training 12,13 .
Nurse: A 'nurse' works in the same capacity as nurses in developed countries and have normally received 3-6 months of theory and 3-6 months of practical training.
Midwife: A 'midwife' (or skilled birth attendant) provides antenatal and delivery care services. SMRU developed its first curriculum in 1994, with the current course taking a minimum of 15 months (30 deliveries) 21 .
Sonographer: SMRU trains local staff in ultrasound gestational age assessment. They have the hardest entrance test compared to the other health staff and the quality of their work is reported elsewhere 22 .
Counsellor: A 'counsellor' provides antenatal care counselling particularly in explaining screening for pregnant women (blood tests, gender-based violence, mental health, nutrition).
Myanmar doctors: SMRU employs doctors to support the health workers and some come from Myanmar.
All these health workers were invited to participate on courses as they were offered.

ALSO and BLSO
With support from ALSO Australasia, it was decided to launch the program at SMRU and MTC. The initial course was supported by four instructors from Australia. The ALSO course includes a syllabus to be studied in advance of the course, an intensive two-day course reviewing obstetric emergencies, and an examination of theory and skills ( Table 1). The BLSO course is conducted in one to two days, context adapted for front line responders (Table 1), with the emphasis on skills.
Instructors for ALSO and BLSO. ALSO and BLSO instructors must have successfully passed both ALSO and the ALSO Instructors course, detailed elsewhere but briefly a short intensive training of the ALSO teaching methods 4,5 . A range of languages are used in the clinics with most local staff speaking one or two dialects of Karen (Sgaw Karen or Pho Karen), Burmese and English as a 2 nd , 3 rd or 4 th language. Following the first course, local staff were selected as Instructors based on qualities such as being a good team player, performing well on the actual ALSO course and languages spoken 23 . Perinatal loss Safety in Maternity care/Universal precautionŝ As maternal venous thrombosis mortality in the area is much lower than mortality from severe malaria and sepsis; it was de-emphasized while severe malaria and sepsis were included from 2008. ALSO for the Thailand-Myanmar border. SMRU course organizers reduced the volume and complexity of the English course material to reflect the baseline education of the group and to ensure the content was relevant to the local setting e.g. including malaria and sepsis as a cause of maternal mortality (Table 1). ALSO course teaching was structured into two parts: the preparation phase and the standard two day intensive course. The preparation phase reviewed one to two chapters per week, mostly in English (60-120 minutes, led by a doctor). A study group for 2-3 hours later in the week, mostly in Karen and Burmese language led by a senior midwife or medic included practice of skills using donated mannequins.
The ALSO written exam was also modified for relevance and clarity, removing difficult English e.g. double negatives. The exams were available in Burmese starting in 2008 and Karen in 2013. Translation verification included back translation. Courses and exams were updated regularly, and numerical scores were given to participants.

BLSO for the Thailand-Myanmar border.
BLSO was directed at frontline staff in border clinics who might encounter pregnant women or neonates in their work. English literacy was a recognized barrier to learning for these staff. Student midwives with little or no clinical experience were also encouraged to participate in BLSO before the end of the first year of training to promote confidence in the workplace and familiarity with some emergencies.
BLSO does not have a preparation phase for participants, the content and the assessment all took place in a two-day course. Material was prepared in English and taught in Burmese and Karen, with a 12-page handout (>50% pictures) provided.
The BLSO was assessed by 10 short answer questions and was available in four languages (Burmese, Karen, Thai and English). The practical component assessed emergencies that were likely to be seen in the community and scored as pass or fail.
Course fees and size. Sustainability in high-income countries comes at a cost to the participant. In this RLS both ALSO and BLSO were conducted by SMRU without cost to the participant. Doctors were asked to pay for the cost of copying the materials. Course size was limited by mannequins to a maximum of eight per group and three groups i.e. 24 participants at a single course.

Evaluation of adult education effectiveness
Kirkpatrick's framework for assessment of effectiveness of adult education includes four levels of data: (level 1) participants' reactions to training, (level 2) change in knowledge and/or skills, (level 3) change in behaviour or clinical practice and (level 4) change in the availability and quality of EmOC and in maternal and/or neonatal health outcomes 24 . In this analysis data on level 1 and level 4 were measured.
All level 1 data was voluntary following a verbal explanation to participants at course orientation. Two measures were included: perceived confidence and self-reported strengths of the course. Prospective repeated-measures surveys at three time periods on perceived confidence: pre course, immediate post course and in a sub-cohort from the original ALSO course in 2008, at 12 months post baseline. The survey tool was adapted with permission from ALSO courses in the USA 6,25 .
Self-reported strengths of the course were measured for ALSO participants by response to an open-ended survey question: "What are the strengths of the course?". Answers in native language were encouraged with responses in Karen and Burmese translated to English.
Level 4 data were extracted from the routine birth facility data curated by SMRU, and measured pre and post ALSO and BLSO in two equal year blocks: before (2000-2007) and after (2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016) implementation. The eight year time frame was chosen as all archived data were cleaned and data checked, while more recent data was not.

Variables
Participant characteristics included age, gender, profession, years of practice, workplace, high school and university attendance, and theory and skills results (as a percentage).
For qualitative analysis perceived confidence to manage the obstetric emergency situations covered in the ALSO or BLSO curriculum (detailed in Table 1) was measured using a fourpoint [1 terrified, 2 scared, 3 coping, 4 comfortable] rank-ordered response Likert scale.
For quantitative analysis extracted variables related to maternal mortality and stillbirth included: status (migrant/refugee), parity, smoking, underweight (<40 kg), four or less antenatal consultations, anaemia the first antenatal visit (haematocrit < 30%) and falciparum malaria confirmed by microscopy by active and frequent screening in pregnancy.

Statistical analysis
The analysis was conducted using SPSS 23.0 for Windows. Participant characteristics such as gender were summarized mostly by proportions (n, %) while age and scores on the course were summarized using the median (interquartile range). The Likert scale responses were summarized by median, interquartile range (IQR) or range (minimum-maximum (min-max)), or proportions, as appropriate. The 95% confidence intervals of proportions were calculated using Wilson's method 26 . The comparison of confidence scores was based on individual paired testing using the Wilcoxon signed rank test.
Qualitative data were analyzed using conventional content analysis to identify major themes 27 . Two authors independently and manually reviewed printed transcripts of the data and suggested possible codes. After discussion, and analysis of the relationship between proposed codes, three common themes were agreed on for the final analysis. Data review was repeated and content was summarized into these three themes. Each reviewer independently chose exemplary comments for each of the three themes and final inclusion was based on discussion and by mutual agreement.
A word cloud was developed using free online software Voyant Tools (https://voyant-tools.org/), to examine the most commonly used words from all the responses to the openended question.
The odds ratio (OR) and confidence interval (CI) for factors associated with the binary outcomes, PPH related maternal mortality (yes/no) and stillbirth (yes/no), were calculated by univariate analysis. The outcome of interest, birth before ALSO and BLSO implementation (yes/no) was retained in the models, as were factors with a p < 0.10 in univariate analysis. These factors were entered in their respective logistic regression model, and were included in the relevant tables, along with adjusted odds ratios (aOR) and 95% CI.

Bias
The participants include all those who participated to the ALSO and BLSO courses reducing selection bias. The qualitative data was analyzed by two of the local instructors who support the course and therefore are at risk of 'positive' bias however they are not responsible for the numerous volunteered responses from the participants. The quantitative data is limited to what data was available and at risk of not being unable to account for factors that can affect maternal mortality and stillbirth. For example 'socio-economic status' was not available and may have contributed to improvements although refugees and migrants have remained marginalized populations over the entire period.

Ethical approval
As part of standard procedure in this setting the details of this project were presented, discussed and approved by the local Community Advisory Board (TCAB-311-12,016), Mae Sot Thailand. No ethical approval was sought for the education evaluation data which was fully anonymized and previously collected. Oxford University Ethics Committee, ethical approval reference code OxTREC 28-09 was obtained for extraction of PPH related mortality and stillbirth data from anonymized hospital records.

Results
First ALSO in a refugee camp and migrant clinics In 2008, ALSO training took place for the first time in a refugee camp with 18 participants from Maela Camp; and in Wang Pha clinic where 19 participants came from three border clinics (Wang Pha, Maw Ker Thai and Mae Tao Clinic in Mae Sot). The majority of participants were midwives 78.4% (n=29/37), 73.0% (n=27/38) had five or less years of work experience 25 . Only 16.2% (n=6/37) had been to university in Myanmar and none had the opportunity to study courses related to medical health care. In total, 94.7% (n=35/37) passed the practical exam while only 29.7% (n=11/37) successfully passed the theory component to ALSO standards. Eleven participants were invited to become instructors to facilitate future courses in local languages. Local participant characteristics included ( Table 2) young age (median <30 years), female predominance 84.8% (476/561) with a majority with five or less years of practice. There was a high uptake to the courses by midwives (none were male), and a variety of workplaces (all non-tertiary) were involved. Less than four years high school was reported by one in four participants and university attendance (not completion) in less than one in five participants

Subsequent ALSO and BLSO courses
The main written and spoken language of participants confirmed that no one language was available to all participants ( Table 3).

Results of the ALSO and BLSO courses
Results on the skills component of ALSO have been consistently high, with a median score of 93 [35-100], and 98.6% (n=350/355) passing on first attempt. This contrasts to the theory component with a median score 66 , with 26.8% (95/355) passing on the first attempt, and one in two participants passing on a retake (50.6%; n=43/85). Retakes were not offered in 2008.
Overall, 88.9% (8/9) doctors passed both practical and theory but all attended tertiary education.

Changes in perceived confidence
Complete data was available for 97% (n=345/355) of ALSO participants for pre-and immediate post-course paired testing of perceived confidence. For all items there was a strong and significant signal for an increase of median perceived confidence, with p<0.001 for all comparisons (median scores not shown). The proportion of participants who were terrified, scared, coping or comfortable were summarized and breech, vacuum, shoulder dystocia and neonatal resuscitation showed the greatest improvement in the proportion 'comfortable' from baseline ( Figure 3, Panel A).
Of 36 participants from the first ALSO cohort in 2008, 32 had a 12-month post-course evaluation: perceived confidence remained significantly higher than baseline for 8 of 13 situations; and was significantly lower for 5 of 13 situations at 12 months compared to the immediate post ALSO course evaluation (Table 3).
There were 89.3% (184/206) of participants on BLSO who completed pre-and immediate post-course evaluation. BLSO perceived confidence at baseline was low except on universal   Missing data: n=7 ALSO, n=3 BLSO precautions; and the median post evaluation scores were significantly higher for all items (data not shown) p<0.001. The proportion of participants who were terrified, scared, coping or comfortable were summarized (Figure 3, Panel B). Despite improvements most were still 'scared' of normal delivery at the end of the course.

Thematic analysis course strengths
Almost all (97.7%; n=347/355) ALSO participants responded in their language of choice to the survey "What are the strengths of the course?": Karen (21%), Burmese (16%) and English (63%). The most common themes to emerge from the responses were: "Improved knowledge", "I can do it" and "Teamwork". A summary of the most common terms from all the comments on the strength of the course from all participants were collated in a word cloud (Figure 4).  The proportion of PPH related mortality and stillbirths with associated factors were summarized (Table 4). After adjustment, birth before ALSO and BLSO was implemented (aOR 3.825, CI 1.1233-11.870), having migrant (not refugee) status (aOR 3.814, CI 1.241-11.718) and attending four (or less) consultations (aOR 3.648, CI 1.189-11.191) were all associated with PPH related mortality. Seven of the eight factors including birth before ALSO and BLSO was implemented (aOR 1.235, CI 1.018-1.500) were associated with increased odds of stillbirth (Table 4).

Discussion
This evaluation of ALSO and BLSO as education tools for health care workers mostly without tertiary qualifications in a post-conflict RLS demonstrates that they can achieve excellent standards for practical components of emergency care in obstetrics. The increased perceived confidence on most items, including at 12 months post baseline, and self-declared benefits from the training in knowledge, skills and teamwork, is consistent with former evaluations in other settings [6][7][8] . The risk of PPH-related maternal mortality and stillbirth declined following the implementation of ALSO and BLSO in this setting, suggesting a meaningful impact on the quality of care.
On the Thailand-Myanmar border, an area of protracted conflict resulting in disrupted education systems, the written theory test has proved to be challenging for health staff 28 . Approximately one in six participants attended (not necessarily completed) tertiary education in Myanmar though none studied health related subjects. Despite low overall pass marks on theory results 'increased knowledge' emerged as one the strongest themes from the qualitative analysis. However, unlike the ALSO course administration in high income countries, this increased knowledge required months of guided study. Provision of the syllabus without facilitation was impossible given the baseline education and diverse language abilities among the staff.
BLSO on the Thailand-Myanmar border has been used as a confidence builder and tool for raising awareness in maternal and child health for staff working mostly outside the delivery room. While BLSO significantly increased perceived confidence, the short nature of this course is no replacement for comprehensive midwifery and nursing training. This is perhaps best reflected by the participants' self-assessment, where 50% reported being 'scared' of normal vaginal delivery after the BLSO course.
The use of historic comparison, before and after ALSO and BLSO were implemented, is a weakness of this evaluation, as changes in the time frame may have occurred that impact on PPH related maternal mortality and stillbirth, and have not been accounted. Nevertheless, the factors associated with PPH maternal mortality highlight known problems in the area related to access to antenatal care 29-31 , and delivery before or after implementation of the courses remained a significant factor after adjustment for known confounders. Reduction of MMR over time is not smooth or inevitable in this setting. The PPH-related MMR (25.4 per 100,000 live births) on the Thailand Myanmar border after ALSO and BLSO were implemented was higher than the all cause official maternal mortality ratio in Thailand 32 .
The factors significantly associated with stillbirth including grand multiparity, smoking, underweight, four or less antenatal consultations, anaemia and falciparum malaria, all typical of RLS also stress the need for improved antenatal care and may in part be addressed by improved access to family planning 33 .
SMRU implemented the WHO partogram in 1994 and while the stillbirth rate of 12.1 (95%CI 11.1-31.2) per 1000 total births is at the global target, like maternal mortality it may well be under-estimated 34 .
ALSO sets a standard. In protracted conflict and post-conflict settings where education as well as health are neglected reaching standard levels of practice are critical 28 . Standards are the main reasons for the minimal modifications of ALSO on the Thailand-Myanmar border, limited to only those situations that are extremely rare or skills requiring equipment that is unavailable. This long and positive evaluation with improved outcomes while ALSO and BLSO have been the only emergency obstetric courses on offer in SMRU clinics and where the level 4 data was reviewed, supports their benefit 35 and sustainability in RLS 36 .

Limitations
This is an evaluation of a single emergency obstetric course, although many exist, and the comparison for PPH related maternal mortality and stillbirth is historical which is acknowledged as imperfect 37 . Deeper analysis into the factors associated with each case of PPH related mortality e.g. timeliness of uterotonics; and stillbirth e.g. pre or intrapartum; would also be helpful to understand if ALSO skills could have prevented the adverse outcome 38 . ALSO relies on mannequins for the practical component and is a burden for most RLS to purchase or maintain but invaluable in visualization of emergency obstetric skills. A complete cost analysis has not been conducted but the cost of replacing mannequins far exceeds other facility-based course costs as instructors work on a voluntary basis 39 .

Conclusion
The best obstetric emergency course for RLS is unknown, however ALSO and BLSO are clearly feasible, effective and sustainable for adult education. Salem, NC, USA 2 American Academy of Family Medicine, Society of Teachers of Family Medicine, Leawood, KS, USA I believe it is a very good study and report to be published. The article addresses improved confidence in clinical performance hands on skills and knowledge after receiving training in ALSO and BLSO Obstetrics Emergency Courses. In a post-conflict limited resource setting Thailand-Myanmar border. Other outcomes studied were the rates of MMR and stillbirth comparing the intervals pre-course from 2000-2007 and post-course 2008-2020. Methods including participants, who were eligible for the ALSO and BLSO courses, the structure of the courses were reported and the assessment of the educational effectiveness qualitative, quantitative then chart review. It is unclear in the methods page 6/16, the first paragraph under Table 1 the ALSO course was standard 2 day course but the mentioned the preparation phase reviewed 1-2 chapters per week (60-120) minutes. (Question: How many days/weeks took for the preparation of the chapters) the ALSO course is designed as a 2 day course. How many questions were used for the ALSO written examination? you mentioned BLSO using 10 questions.

Data availability
Results: Results were clear in regard to passing rates for both ALSO and BLSO. the clinical assessment hands on were high rate passing >90%, but the written exam was 50% passing for ALSO evaluation. the reported self assessment of the participants was showing clear confidence in 8/13 situations which seems to persist after 12 months. The comparison of the outcome related to PPH caused mortally pre-course and post-course were positive results. although I agree with the limitation mentioned here that they did not look at the reasons/ etiology/risk factors for PPH if preventable or not, the availability of uterotonic in a resource limited area. And I think another limitation included: researchers did not have the chance to study if the participants in these PPH scenarios had received the ALSO course training? Did they implement what they learned and used a team based approach?
The stillbirth result was reported well with a good conclusion. The tables numbering are incorrect:

Is the study design appropriate and is the work technically sound? Yes
Are sufficient details of methods and analysis provided to allow replication by others?