Model fidelity of group antenatal and postnatal care: a process analysis of the first implementation of this innovative service model by the Preterm Birth Initiative-Rwanda

Introduction

This article reports process results associated with a study of group antenatal and postnatal care (ANC and PNC) conducted by the Preterm Birth Initiative-(PTBi) Rwanda in 2017–2019 (Musange et al., 2019). In the parent study, 18 Rwandan health centers were randomized to provide group ANC and PNC instead of the standard individual ANC and PNC, while 18 pair-matched health centers continued to provide individual ANC and PNC. The primary outcome of the parent study is gestational length; that analysis will be reported separately. This nested study examines the model fidelity of group ANC and PNC implemented for the purpose of this trial and the implications of our results on scaling implementation of group ANC and PNC in Rwanda and in other contexts.

Methods

A total of 5/30 Rwandan districts were selected for the parent trial in collaboration with the Rwanda Ministry of Health. Within those 5 districts, 55 health centers were assessed with a standardized tool for number of providers, ANC volume, suitable space for group care, services, and equipment. Health centers that reported that they allocate at least two providers to ANC services on any day that ANC is offered were selected for this trial, for a total of 36 health centers. These 36 health centers were pair-matched and then randomized to either continue individual ANC and PNC or switch to group ANC and PNC. A data collector was embedded at each of the 36 study sites.

To both monitor and study model fidelity during the trial, the study partners developed a strategy to: 1) train Rwandan group ANC/PNC Master Trainers who in turn trained ANC/PNC providers to deliver the group care model; 2) provide regular support and feedback to group care providers by Master Trainers who visited and observed group visits at all intervention clinics; 3) collect model fidelity data by Master Trainers during observed group visits; and 4) instruct group care providers to complete a self-assessment instrument after each group visit, observed or unobserved by the Master Trainer.

This analysis includes data collected by two groups between late June 2017 and early January 2019. The first group includes nurses, midwives, and CHWs who were recruited by facility directors to participate in the PTBi-Rwanda trial as group ANC and PNC facilitators. The second group includes one nurse, five midwives, and one physician who served as group ANC/PNC Master Trainers; the preparation of these Master Trainers is described elsewhere (Sayinzoga et al., 2018). We used two discreet questionnaires to collect data from these two groups about the fidelity with which the group ANC/PNC model was implemented during this trial.

Results

Of about 3000 expected group visits, 2763 total GVDQs were available (approximately 90%); 17 records were excluded for missing or nonsensical data. Of 162 expected Master Trainer observations, 149 MFAs were available, but 9 of these were missing data for one or more items and 140 were analyzed. Both GVDQs and MFAs were available for 84 visits. Table 1 reports documentation of group ANC and PNC visits recorded by facilitators in the GVDQ, both during the 84 visits for which MFAs were also available and for all documented group visits (2763). Of note, women required some amount of additional, one-on-one care during or after about one-third of group visits, and the average number of women who required this individual care was one woman per group. Co-facilitators wrote in descriptions of this additional, one-on-one care for only about 20% of these (n=175). In total, 87% of these free-text responses stated that the additional care was for preventative services including provision of a family planning method (after PNC), provision of an insecticide-treated bed net, and tetanus toxoid vaccination. A total of 13% these responses documented the management of abnormal findings, including symptoms of malaria, abnormal health assessment findings, and transfer to hospital.

Table 2 reports self-assessment items in the GVDQ, among both all documented group visits and the sub-set of visits observed by a Master Trainer. In general, group visit facilitators self-reported positive characteristics of the visit more often among all visits than among the sub-set of visits observed by Master Trainers. The mean score for all 140 complete MFAs was 3.17 (5-point scale from 0-4). The mean MFA score among the sub-set of records with an available GVDQ (n=84), was 3.16. (1.09 to 4.0). The average visit score among visits at which a midwife was present (19% of visits) was slightly higher than visits at which a midwife was not present (81% of visits), but this difference was not statistically significant. There was a range of mean score calculated by health center, from 2.95 to 3.50. The trend line for the average score increased approximately 13% from the beginning to end of this 18-month period. We compared MFA score with 6 characteristics of each group visit: 1) the health center where the group visit was observed, 2) whether or not water was prepared for the women’s refreshment, 3) the title of the facilitator present with the highest level of education (nurse or midwife), 4) the number of facilitators present, 5) the number of women present for the group visit, and 6) the length of time in days since the study began. There was a correlation between length of time since the study began and MFA score (F-score=0.0003, r2=0.15). However, there were no significant correlations between the MFA score and any of the other characteristics listed above.

Table 3 shows the average MFA score for each item, across all 140 complete MFA records. The highest average score (3.46) for a single item was “The co-facilitators provided ANC/PNC screening, medications, and referrals as indicated, consistent with the Rwanda FANC and PNC packages,” indicating that the overall quality of service package delivery was high. The item with the lowest average score was “Kept time,” (2.70) which is consistent with the result reported in Table 1 that at least 25% of group care visits lasted more than 2 hours. Table 4 shows the correlation scores resulting from bivariate regression analysis comparing each of the 11 MFA items to the overall MFA score. Four MFA items had correlation scores between 0.71 and 0.76 (highest correlation score), while 7 MFA items had correlation scores between 0.45 and 0.69. Several self-reported items in the GVDQ are conceptually closely related to items in the MFA; these conceptually-related items appear in Table 5. We performed a bivariate regression analysis with the sub-set of records with data from both tools (n=84) to discover whether a “yes” answer to each of this sub-set of items in the GVDQ could predict whether the related item in the MFA would be scored 3 or 4 (0-4 scale with 4 being the highest possible item score) by the Master Trainer. There was not a strong relationship between self-reported and Master Trainer scores for any of these conceptually-related items; self-assessments both over- and under-reported soft skills compared to Master Trainer assessments. However, two GVDQ items agreed with MFA items ≥68% of the time: we were well-organized and we followed the lead of the women.

Discussion

Our results are consistent with previously reported results of group ANC model fidelity by Novick et al. in the United States (Novick et al., 2013). Those authors reported that fidelity to intended group ANC process was 77% (range, 54–97%) and intended group ANC content was 70% (range, 44–100%). We did not specifically measure content fidelity, but overall model fidelity—focused on process fidelity—during this study period was estimated to be 80% (average MFA score was 3.18 on a 0-4 scale). The Master Trainers observed that, in general, this cohort of group ANC and PNC adhered to the Rwanda package of ANC/PNC services while implementing this alternative model of service delivery. By objective descriptive data, the intervention was implemented as intended. The “soft” skills fundamental to the success of the group care intervention were challenging to learn and implement (as expected), but Master Trainers observed that, across this study period, providers encouraged participant engagement, asked open-ended questions, and spoke less than participants during discussions to a degree estimated to be, on average, 80% of the ideal. However, when co-facilitators rated themselves on these soft skills, their answers did not consistently agree with Master Trainer assessments. Self-assessment of these soft skills may be less discriminating than Master Trainer assessment of these important facilitative leadership skills.

Model fidelity scores were not significantly different depending on whether a nurse or a midwife was the “highest-level” provider present at the group visit. We interpret this result to mean that in this context nurses and midwives are equally able to successfully provide group care. The presence of 1, 2, or 3 facilitators also had no relationship to the MFA score. We interpret this to mean that while 2 co-facilitators may make it easier to share the labor of providing the group visit, 1 provider is equally as likely as 2 or more co-facilitators to achieve a high MFA score. Because CHWs in Rwanda do not independently complete blood pressure and abdominal examination at this time, we assume that they cannot independently convene a group visit that is meant to include these assessments and clinical decision-making for abnormal findings.

In future implementation of group ANC and/or PNC care in Rwanda, we recommend two abbreviated group ANC/PNC model fidelity assessment tools, one for facilitators and one for expert observers (Table 6 and Table 7). These abbreviated tools were created by the Rwanda Group ANC/PNC Technical Working Group after reviewing our results. The Rwanda Group ANC/PNC Technical Working Group’s activities and composition are described in a separate publication (Sayinzoga et al., 2018). These simplified assessment questionnaires could be integrated into a streamlined monitoring strategy. We conclude that future implementation of group ANC/PNC in Rwanda will benefit from continued collection of self-assessment data by group visit facilitators, expert coaching and mentoring, and assessment by expert observers. Future research is needed to understand the optimal schedule of observation visits by quality assurance supervisors.

While about 25% of group visits lasted longer than 2 hours, we found that it is feasible in this context to plan for three group ANC and one group PNC visits that last an average of 2 hours with an average of nine women in attendance. These results give Rwandan policy makers information when they consider whether patient volumes and human resources necessary for future implementation of group ANC and PNC are aligned. For example, a health center that provides comprehensive ANC for an average of 50 women who expect to give birth every month, and that invites all pregnant women to three follow-up group ANC visits over the course of pregnancy, should plan for about 40 hours per week of work dedicated exclusively to group ANC provision.

Conclusion

Our findings from the first implementation of group ANC and PNC in Rwanda suggest that the model was delivered as intended during the majority of group visits, with evidence from both self-assessments and expert observations. However, we did not find significant correlations between MFA scores provided by Master Trainers and self-assessment data reported by group visit facilitators. At this time and in this context, we cannot recommend relying on facilitator self- assessment alone to monitor group ANC/PNC process fidelity. We recommend both the continued participation of expert observers at new and existing group care sites and the ongoing development of self-assessment techniques that may lead to more reliable self-monitoring methods for even larger-scale group ANC/PNC programs.

Ethical statement

Ethical approval for all study activities, including the administration of these two questionnaires, was granted by the Rwanda National Ethics Committee (0034/RNEC/2017) and University of California, San Francisco Institutional Review Board (16-21177). Two discreet written informed consent forms were obtained from each provider and CHW prior to the first group ANC or PNC visit in which she/he participated as a facilitator: one consent form for completing GVDQ and the other consent form for being observed by Master Trainers while facilitating a group ANC or PNC visit. No personal identifiers of providers or CHWs were recorded. Study staff protected all data as confidential.

Data availability