Implementation research to improve quality of maternal and newborn health care, Malawi

Abstract Objective To evaluate the impact of a performance-based financing scheme on maternal and neonatal health service quality in Malawi. Methods We conducted a non-randomized controlled before and after study to evaluate the effects of district- and facility-level performance incentives for health workers and management teams. We assessed changes in the facilities’ essential drug stocks, equipment maintenance and clinical obstetric care processes. Difference-in-difference regression models were used to analyse effects of the scheme on adherence to obstetric care treatment protocols and provision of essential drugs, supplies and equipment. Findings We observed 33 health facilities, 23 intervention facilities and 10 control facilities and 401 pregnant women across four districts. The scheme improved the availability of both functional equipment and essential drug stocks in the intervention facilities. We observed positive effects in respect to drug procurement and clinical care activities at non-intervention facilities, likely in response to improved district management performance. Birth assistants’ adherence to clinical protocols improved across all studied facilities as district health managers supervised and coached clinical staff more actively. Conclusion Despite nation-wide stock-outs and extreme health worker shortages, facilities in the study districts managed to improve maternal and neonatal health service quality by overcoming bottlenecks related to supply procurement, equipment maintenance and clinical performance. To strengthen and reform health management structures, performance-based financing may be a promising approach to sustainable improvements in quality of health care.


Introduction
Improving health-care quality is essential to health systems strengthening. 1 Financial, material and human resource limitations challenge low-and middle-income countries to sustain satisfactory health-care quality performance. 2 Health-care quality improvement strategies differ in focus. Some strategies focus more on health workers' performance and the immediate micro-context in which health workers provide care. Other strategies focus more on the health system's macro-context to address financial and fiscal constraints, inefficient resource allocations, limited donor coordination or ineffective health sector regulation. [3][4][5] Performance-based financing schemes are health financing strategies that link economic aspects of health care provision (macro-context) directly to bottlenecks at the frontline of actual service delivery (micro-context) aiming to improve health-care quality. 6,7 The schemes commonly tie payments to front-line service providers (health worker or facility) to predefined clinical and/or structural performance measures. 8 Performance-based payments are usually managed and directly invested in further service improvements by facilities. Moving some fiscal decision-making towards front-line providers allows for additional mechanisms to overcome day-to-day limitations in service provision (such as inadequate supplies or low staff motivation) faced by health facilities. 9,10 Many low-and middle-income countries have adopted performance-based financing schemes because of their potential to optimize both financial management and service quality (micro-context). [11][12][13] The schemes may also offer more sustainability to health-care quality improvements by strengthening or redefining responsiveness and accountability across various service providers and health-system actors (macro-context). 10,14 Failure to maintain essential service inputs, such as adequate staffing, drug procurement, equipment repair, etc. risks the overall success of health-care quality improvement programmes. 15,16 Health-care quality depends on adequate human resource and supply chain mechanisms for procurement of drugs and supplies, particularly in systems exposed to extreme resource constraints. 17 In many low-and middleincome countries, central-level health officials are responsible for maintaining drug stocks, essential equipment and adequate staffing across health facilities, and are thus ultimately liable for service quality at the periphery. 18 Therefore, centrally organized health systems may benefit from strategies that introduce more decentralized structures of decision-making, responsiveness and accountability.
While performance-based financing schemes in principle are designed to help health workers and service managers to achieve better health-care quality outcomes, their actual effect in doing so remains unclear. 7,9 Recent performance-based financing evaluations in Afghanistan 19 and Burundi 20 failed to detect strong effects of their schemes on health facilities' ability to secure essential drugs and supplies. A Tanzanian scheme directly providing incentives for facilities to manage essential drugs and supplies failed to prevent stock-outs of essential equipment, medicines and commodities. 21 The study concluded that incentives provided to facilities to improve quality-of-care performance that are tied to drug and supply management cannot overcome systemic shortcomings, such as inadequate supply chain structures. 21 Conversely, a Objective To evaluate the impact of a performance-based financing scheme on maternal and neonatal health service quality in Malawi. Methods We conducted a non-randomized controlled before and after study to evaluate the effects of district-and facility-level performance incentives for health workers and management teams. We assessed changes in the facilities' essential drug stocks, equipment maintenance and clinical obstetric care processes. Difference-in-difference regression models were used to analyse effects of the scheme on adherence to obstetric care treatment protocols and provision of essential drugs, supplies and equipment. Findings We observed 33 health facilities, 23 intervention facilities and 10 control facilities and 401 pregnant women across four districts. The scheme improved the availability of both functional equipment and essential drug stocks in the intervention facilities. We observed positive effects in respect to drug procurement and clinical care activities at non-intervention facilities, likely in response to improved district management performance. Birth assistants' adherence to clinical protocols improved across all studied facilities as district health managers supervised and coached clinical staff more actively. Conclusion Despite nation-wide stock-outs and extreme health worker shortages, facilities in the study districts managed to improve maternal and neonatal health service quality by overcoming bottlenecks related to supply procurement, equipment maintenance and clinical performance. To strengthen and reform health management structures, performance-based financing may be a promising approach to sustainable improvements in quality of health care. scheme in Zambia provided incentives to health facilities, which were tied to a programme on essential drug and supply management. The scheme enabled participating facilities to use financial rewards gained for positive performance to purchase additional drugs, which successfully reduced subsequent stock-outs. 22 Yet generalized evidence on performance-based financing schemes' effects on improving health-care service management processes remains limited. This is partially because many schemes narrowly focus on service utilization or the clinical aspect of health-care quality, often overlooking elements directly tied to the structural aspect. 23 We evaluated a performance-based financing scheme in Malawi to further understand the potential of such a scheme to improve both clinical (i.e. adherence to obstetric care treatment protocols) and structural (i.e. provision of essential drugs, supplies and equipment) aspects of health-care that have an effect on providing quality care.

Study Setting
Malawi has a centralized tax-based health system that receives significant external donor support. 24,25 In 2010, basic emergency obstetric care was provided free of charge by 98 (90%) public and contracted not-for-profit health facilities nationwide. 25 Public health-care facilities receive general budget allocations from the Ministry of Health for infrastructure, equipment and salaries. 26 District health management teams receive additional budget allocations from the Ministry of Local Government and Rural Development to procure drugs and supplies from central medical stores for all public facilities within their districts. 25,27 Malawi's health system faces two major challenges. First, the country has inadequate human resources and suffers from shortages of qualified health workers, poor remuneration, limited career opportunities and insufficient supervision and training opportunities. 28,29 Second, it has a wide-spread lack of essential equipment at health centres with recurrent stock-outs of drugs and consumables. 30,31 Consequently, public facilities struggle to provide quality health-care services. 32 To address some of these limitations and to meet the mil-lennium development goals 4 and 5, to reduce child and maternal mortality, the Malawian Ministry of Health introduced the Results Based Financing for Maternal and Neonatal Health (RBF4MNH) Initiative in 2013 at several emergency obstetric care facilities. Similar to other performance-based incentive schemes, the initiative provides financial rewards for maternal care providers and district health management teams upon meeting defined performance goals (performance-based incentives). It also provides conditional cash transfers (financial rewards) for pregnant women upon meeting defined compliance targets. 1 Details about the initiative and its implementation design are presented in Box 1.

Study design
This paper focuses on the performancebased financing component of the initiative and assesses its effects on maternal and newborn health service quality. The study is part of a larger impact evaluation that uses a non-randomized controlled before and after study design with independent controls. 34 The intervention sample for this study initially consisted of all 18 facilities contracted by the initiative at the beginning of phase I: four public district hospitals, and 14 public health centres providing at least basic emergency obstetric care. The control sample included all health facilities within the same study districts that were not, or not yet, enrolled in the initiative at baseline: one mission hospital, two private and 12 public health centres providing basic emergency obstetric care. Several months prior to the last data collection round, five previous control facilities (a mission hospital, one private and three public health centres) were contracted by the initiative (phase II) and thus moved into the study's intervention arm. Data were collected at three time-points: before the initiative was implemented (March/April 2013) used as baseline, at mid-point (June/July 2014) and at the end (June/July 2015). In addition, at each time point we identified a small number of pregnant women giving birth per each sampled facility using a convenience-sampling approach.

Outcome indicators
We measured the scheme's effect on quality using two sets of performance outcomes: structural performance and clinical performance (Box 2). For structural performance, we assessed the availability and functionality of service input elements, such as equipment, drug and supply items included in the initiative's tracer lists at each sampled facility. To measure effects on clinical performance, we observed the following processes: (i) routine care activities required to identify respective emergency conditions; (ii) the overall number of pregnant women monitored by partograph; (iii) the completeness of partograph documentation; (iv) a set of common routine infection prevention measures; (v) oxytocin ad-

Box 1. Malawi's Results-Based Financing for Maternal and Neonatal Health Initiative: implementation design
The Results-Based Financing for Maternal and Neonatal Health Initiative was introduced in Malawi in April 2013 to improve the quality of facility-based care provided to women and newborns during and within 48 hours after delivery. The initiative was implemented in selected emergency obstetric care facilities across four districts (Balaka, Dedza, Mchinji and Ntcheu). Similar to other performance-based incentive programmes, the initiative combines financial rewards for maternal care providers and district health management teams upon meeting defined performance goals (performance-based incentives) with conditional cash transfers (financial rewards) for pregnant women upon meeting defined compliance targets. 33 Across the four districts, the initiative contracted 18 health facilities (four hospitals and 14 health centres) during phase I (April 2013-October 2014) and then expanded to include a total of 28 health facilities (5 hospitals and 23 health centres) during phase II (ongoing until 2017). Enrolment of facilities into the initiative was non-random and was based on performance of emergency obstetric care signal functions, functionality of referral system, geographical emergency obstetric care coverage and catchment population size. Each enrolled facility received initial start-up financial support conditional on need related to immediate infrastructural requirements (i.e. potable water, electricity, waste disposal, building structures, etc.). Incentives for facilities are largely tied to clinical and general service management performance; district health management team incentives are largely tied to equipment maintenance and drug supply management across all facilities within each district. At least 30% of received payments are re-invested to further improve health care quality at district and facility levels, while the remaining portion is directly shared among staff as personal bonuses.

Research
Maternal and newborn health care, Malawi Stephan Brenner et al.
ministration; and (vi) other essential elements of active management of third stage labour (AMTSL), such as controlled cord traction, uterine massage, completeness check of placenta and estimation of overall blood loss.

Data collection
We measured changes in the outcome variables (Box 2) over the study period using two different survey instruments: (i) facility inventory checklists to collect information on structural performance; and (ii) an observation checklist to document non-participant observations of routine labour cases to assess birth attendants' clinical performance. 35 We trained research assistants with a back-ground in midwifery to observe and record routine clinical care processes in uncomplicated labour presenting to a facility during a 5-day visit, long enough to observe an adequate number of deliveries. We observed only for 3 days during baseline data collection. Clinical performance was measured against national midwifery standards. 36 All uncomplicated labour case observations began when a woman in labour was taken to a maternity unit and ended two hours after childbirth. To ensure data completeness and accuracy, checklists were designed to prompt research assistants to complete or indicate any missing information during and immediately after data collection. Written informed consent was obtained from both patients and birth attendants before observation. Both survey instruments were used at all study facilities during each of the three data collection time points. We observed 33 health facilities (23 intervention facilities and 10 control facilities) and 401 pregnant women across four districts. Table 1 presents a summary of the number of sampled facilities and pregnant women observed at each data collection time.

Analytical approach
To estimate the scheme's effect on each outcome variable, we used a differencein-differences (DiD) linear regression model. 37

Box 2. Performance indicators and related measures of the performance-based financing scheme in Malawi Clinical performance indicators and process measures a
Maternal and newborn health care, Malawi Stephan Brenner et al.
The model is as follows: where, Y i represents our outcome variables, (structural or clinical performance); β 0 is the average frequency of observations among controls at baseline; t is a dummy variable indicating the observation time point, with t1 being observation at mid-term and t2 being observation at end-term; T is another dummy variable that indicates the treatment time point, with T1 being treatment distribution during phase I (April 2013-October 2014) and T2 being treatment distribution at phase II (from November 2014 onwards). T*t denotes the interaction between treatment and time point, where T1*t1 denotes interaction at mid-term and T2*t2 at end-term; X k , i denotes potential confounders; ρ is within-cluster correlation; m is the number of clusters and c denotes the number of individual observations. The overall effect size estimate at end-term is represented by β 6 , which served as overall effect estimate for the entire initiative.
We adjusted all models for potential confounders, i.e. facility type (hospital versus health centre) and ownership (public versus faith-based). Models used in analysing clinical performance were further adjusted for duration of stage 1 labour (accounting for late presenting women unlikely to be monitored by partograph), number of birth attendants available during a case (accounting for workload constraints) and birth attendants' participation in in-service trainings (accounting for recent skill changes independent of the scheme). We further controlled clinical performance for clustering effects at the level of the facility. Given the relatively small sample sizes, we used bootstrapping to derive more robust estimates of the underlying population parameters. 38 Table 2 summarizes changes and effect sizes for equipment maintenance. Positive DiD coefficients indicate a comparatively larger increase in availability of operational equipment at intervention facilities. We found statistically significant positive effects for nearly all essential equipment items related to routine obstetric care and vital sign recording. For items related to emergency and ancillary equipment, only a few items showed a significant positive effect (vacuum extractors, sterilizers and examination beds). We observed no significant effects for delivery beds. For blood pressure machines and newborn resuscitation equipment, improvements in both intervention and control facilities over time led to no significant change.

Results
The scheme's effects on drug and consumable stocks varied greatly (Table 3). We observed statistically significant positive effects for ampicillin, oxytocin, dextrose 5% saline solution, pethidine, and human immunodeficiency (HIV) drugs used for prevention of mother-to-child transmission. We observed significant negative effects for benzyl penicillin, normal saline and Ringer's solution. In both intervention and control facilities, diazepam and urine testing strip stocks improved but stocks for magnesium sulfate worsened.
The effects on clinical performance were diverse (Table 4). Positive, but non-significant effects were observed for review of pre-eclampsia symptoms and timely set-up of sterile birth equipment. Negative effects were found for partograph monitoring and AMTSL. Initially, parallel improvements across intervention and control facilities occurred for examination of pre-eclampsia signs, partograph documentation, infection prevention techniques during vaginal examinations and births, as well as medical and practical elements of AMTSL.

Discussion
Our findings demonstrate that performance incentives directed to a range of health system actors beyond the actual frontline providers can affect health service quality positively. In our study, direct district level involvement not only improved supply management at intervention facilities, but likely produced additional benefits to control facilities.
Our findings indicate an overall positive impact on both equipment maintenance and drug stocks as a result of the incentives provided through the scheme. Though limited, this evidence suggests that performance-based financing may effectively sustain service inputs and processes once incentives are set across relevant health system levels. Qualitative evidence from interviews with the initiative's district health and facility managers illustrate that local adoption has led to further decision-making by managers about resource allocation to extend beyond the initiative. 39 The decision allowed facilities not included in the performance-based financing initiative to receive procurement support. 39 This local adoption effect on equipment availability was also demonstrated by the Salud Mesoamérica initiative -a results-    e Defined as documentation of fetal heart rate every 30 minutes, uterine contractions/maternal pulse/maternal blood pressure/fetal descent every 60 minutes. f Defined as washing of hands with water and soap. g Includes setting up sterile delivery kit contents, sterile cord clamps, sterile gloves. h Parenteral administration of oxytocin once stage 3 is entered. i Controlled cord traction and uterine massage to actively support delivery of placenta. j Placenta examined for completeness and estimation of overall blood loss during delivery to determine woman's risk of bleeding. Note: DiD estimates are calculated across years 1 and 2 while controlling for duration of stage 1 labour, number of birth attendants available during a case, number of birth attendants' participation in service trainings, as well as clustering effects at the level of the facility. based funding programme that provides incentives to central health system levels to indirectly support front-line service providers. 40,41 However, in our study, for some items availability did not improve. The number of delivery beds, for instance, remained unaffected, probably because most facilities counted at least two delivery beds already at baseline. Some drugs also remained unchanged (co-trimoxazole, paracetamol) or declined (benzyl penicillin, magnesium sulfate, isotonic fluids), either indicating increased use (e.g. in case of the incentivized administration of magnesium sulfate) or worsening central drug shortages. A randomized-controlled trial from the Democratic Republic of the Congo comparing performance-based versus fixed payments observed significant decline in essential equipment availability at facilities supported by a performancebased financing scheme. 42 However, in this scheme, incentives were provided directly to the facility level without including district or central level entities.
In our study, Malawi's nationwide economic situation deteriorated before our mid-term data collection which may have had an effect on drug and consumable stocks. Mixed or inconclusive effects were also reported from performance-based financing experimental work in the Democratic Republic of the Congo 42 where after the implementation of the scheme, drug stock-outs decreased, though stock-outs of vaccines increased. Findings like these point towards overall political or economic challenges affecting health system and performance-based financing scheme operations. In contexts where availability of medical supplies is negatively affected by performance-based financing schemes, a detailed investigation may provide further understanding of system-wide determinants of performance-based financing implementation.
Among those essential drugs whose observed availability was not strongly affected by the initiative, many (except penicillin and Ringer's lactate) nevertheless remained in stock in more than 70% of intervention facilities at end-term, which may reflect facilities' new financial autonomy, such as using reward earnings to purchase additional drugs from private pharmacies. This aligns with findings from another performance-based financing evaluation in the Democratic Republic of the Congo, where facility managers with financial autonomy to use performance rewards to purchase drugs and supplies made decisions resulting in significant improvements in drug availability. 43 In our study, as most of these less available drugs are essential to the medical management of obstetric complications (i.e. broad-spectrum antibiotics, crystalloid fluids, anticonvulsants), some shifts in facility stocks towards second-line tracer drugs (i.e. ampicillin, dextrose 5% saline, diazepam) occurred. These shifts likely indicate better strategic procurement decisions by district and facility managers in response to variations in first-line drug availability at central stores.
Improved availability of some items (e.g. blood pressure machines, newborn resuscitation equipment, obstetric examination beds, diazepam, Ringer's solution) was also observed in control facilities. District-level performance indicators were purposefully designed to introduce spillover benefits to those facilities not yet enrolled. Improved equipment and supply availability in control facilities may represent such programme-induced spillover. In addition, qualitative evidence from interviews with the initiative's health managers in the study districts demonstrate that the direct involvement of the district health management team led to more frequent performance review meetings at district levels and closer performance supervision across facilities. 39 Our findings show few positive effects on clinical processes due to the scheme. For pre-eclampsia assessment of risk factors, we only observed improvements in symptom review, while physical examination of clinical signs increased only during mid-term data collection. There was also no effect observed on the routine review of patients' HIV status. Mixed effect patterns regarding clinical assessment practices in maternal and child services, both history-and exam-based, were also observed in performance-based financing schemes in Afghanistan and Egypt. 19,44 Although less obvious, these findings may nevertheless point towards the potential the schemes bring to improve providers' adherence to routine care standards and clinical guidelines.
In respect to infection prevention processes, we observed non-significant positive effects on sterile set-up of birth equipment, but no effects on hand washing, use of gloves and perineal cleansing. This may reflect a complementary effect regarding infection prevention processes between clinical performance and traced supply incentives, which included sterile delivery packs and functional sterilizers, but not gloves, topical antiseptic solutions or other consumables relevant to infection prevention. The role of complementary use of incentives may also explain findings in a study from Afghanistan, 19 which failed to detect significant effects on universal precautions in the absence of relevant equipment and supply improvements.
We further observed significant negative effects on processes related to partograph monitoring and AMTSL. As oxytocin (tracer item) and partograph (not listed as tracer item, but followed by the study's facility inventory) availability significantly improved in intervention facilities, stock-outs don't seem to be a likely cause of the negative effects. The negative trends for AMTSL performance might rather relate to district level spillover benefits to control facility performance as indicated by qualitative evidence outlining how district managers reinforced clinical standards, supervision and staff coaching across all facilities. 39 In the case of AMTSL, control facilities -starting at a much lower performance level, but rapidly catching up over time -probably benefitted more than facilities in the intervention arm. A recent study from Burundi found that introduction of a performancebased financing scheme encourages guideline adherence through increased supervisory, coaching and technical support by senior staff. 45 Regarding labour monitoring, birth attendants at both intervention and control facilities improved partograph documentation, a relatively time-consuming process. Yet, the proportion of overall partographmonitored cases stagnated in the intervention facilities, which may simply reflect capacity limitations due to staffing constraints once service utilization at intervention facilities increased.
Our study has some methodological limitations. As with other studies on quality of clinical care, our sample sizes for both facilities and observed cases

Research
Maternal and newborn health care, Malawi Stephan Brenner et al.
were relatively small therefore our study lacked the statistical power to detect additional effects that may have been produced by the scheme. Also, due to logistical and cost considerations, our evaluation design had to rely on control facilities within intervention districts, falling short in conceptually isolating district-level from facility-level effects.
Our study therefore was not able to discern whether improvements in control facilities represent district-level induced spillover or rather resulted from other secular trends. ■

Resumen
Investigaciones sobre la ejecución para mejorar la calidad de la atención sanitaria materna y obstétrica, Malawi Objetivo Evaluar el impacto de un plan de financiación basado en el rendimiento en la calidad del servicio sanitario materno y neonatal en Malawi. Métodos Se realizó un estudio controlado no aleatorizado del antes y el después para evaluar los efectos de los incentivos de rendimiento a nivel de distritos y de centros para trabajadores sanitarios y equipos de administración. Se evaluaron los cambios en las existencias de medicamentos esenciales, el mantenimiento del equipo y el proceso de atención obstétrica clínica en los centros. Se utilizaron modelos de regresión de diferencia en diferencia para analizar los efectos del plan sobre la adherencia a los protocolos de tratamiento de atención obstétrica y el suministro de medicamentos esenciales, suministros y equipos.