Cost–effectiveness of results-based financing, Zambia: a cluster randomized trial

Abstract Objective To evaluate the cost–effectiveness of results-based financing and input-based financing to increase use and quality of maternal and child health services in rural areas of Zambia. Methods In a cluster-randomized trial from April 2012 to June 2014, 30 districts were allocated to three groups: results-based financing (increased funding tied to performance on pre-agreed indicators), input-based financing (increased funding not tied to performance) or control (no additional funding), serving populations of 1.33, 1.26 and 1.40 million people, respectively. We assessed incremental financial costs for programme implementation and verification, consumables and supervision. We evaluated coverage and quality effectiveness of maternal and child health services before and after the trial, using data from household and facility surveys, and converted these to quality-adjusted life years (QALYs) gained. Findings Coverage and quality of care increased significantly more in results-based financing than control districts: difference in differences for coverage were 12.8% for institutional deliveries, 8.2% postnatal care, 19.5% injectable contraceptives, 3.0% intermittent preventive treatment in pregnancy and 6.1% to 29.4% vaccinations. In input-based financing districts, coverage increased significantly more versus the control for institutional deliveries (17.5%) and postnatal care (13.2%). Compared with control districts, 641 more lives were saved (lower–upper bounds: 580–700) in results-based financing districts and 362 lives (lower–upper bounds: 293–430) in input-based financing districts. The corresponding incremental cost–effectiveness ratios were 809 United States dollars (US$) and US$ 413 per QALY gained, respectively. Conclusion Compared with the control, both results-based financing and input-based financing were cost–effective in Zambia.


Introduction
Over the last two decades, Zambia's health indicators have improved substantially. Between 1990 and 2013, the underfive mortality rate dropped from 193 to 87 deaths per 1000 live births, and the maternal mortality rate fell from 580 to 280 deaths per 100 000 births. 1 There has been a substantial reduction in mortality from human immunodeficiency virus and acquired immune deficiency syndrome (HIV/AIDS) and malaria. 2 Despite these improvements, the under-five mortality rate and maternal mortality rate in Zambia (population 15.15 million in 2013) remain high by regional and international standards. Furthermore, the use of key maternal and child health services remains low and inequitable. 2 The country faces great challenges in financing the health system, and delivering quality and equitable health services. 3,4 Results-based financing, an approach to incentivize providers to improve the quality and quantity of health services, has been implemented in many low-and middle-income countries. 5 Despite mixed results, 6,7 results-based financing has generally shown promise as a way to address maternal and child health concerns and catalyse health-care reforms. [8][9][10][11][12][13] These findings prompted Zambia to initiate a results-based financing programme. With partial funding from the World Bank, Zambia implemented a project in 2008 in Katete district as a pre-pilot site, trying to realign payment to outputs rather than inputs.
Given the promising results from the pre-pilot project, results-based financing in Zambia was expanded to 10 additional districts under the pilot phase in April 2012, covering a population of 1.33 million. The programme design was a contract-in model, whereby health centres were contracted to deliver a specified package of essential maternal and child health services. Once they fulfilled their obligation, payments were made using predefined maternal and child health and quality indicators. 14 The pilot programme ended in October 2014.
Although many results-based financing programmes have incorporated impact evaluations, the majority of these have focused on use of services. To our knowledge, few publications have systematically combined the cost and health impact to estimate the value for money of results-based financing. 6,15 Given increasing demand for health services, governments face financial constraints in funding programmes. This study aimed to evaluate both the cost and health outcomes of Zambia's results-based financing programme against two counterfactual policies: enhanced financing not explicitly tied to performance (input-based financing) and the existing system of funding without additional financing (control).
Results-based financing, Zambia Wu Zeng et al.

Research design
The study used a triplet-matched cluster randomized design to evaluate the impact on health service delivery in a total of 30 districts from eight provinces. District-level data were collected and scores were assigned to districts for population health, socioeconomic conditions and health governance capacity. In most provinces, three districts at or near the provincial median index score in that province were selected and then randomly allocated among the three different groups: results-based financing, input-based financing or control. Health facilities under results-based financing received incentives tied to performance on pre-agreed indicators and were required to use at least 40% of the incentive payment for operational activities. A maximum of 60% of the incentive payment could be used for staff bonuses. Facilities under input-based financing were intended to receive approximately the same amount of funding as those in results-based financing districts, but their funding was not tied to performance. The payment received was used only for operational activities. Given this additional funding and potential expectation of joining the results-based financing programme in the future, health facilities in the inputbased financing group could organize themselves for better service provision. Control facilities represented the existing financing method in Zambia, receiving no additional funds. There were 175, 173 and 175 primary health-care facilities in the results-based financing, input-based financing and control groups, serving populations of 1.33, 1.26 and 1.40 million people, respectively.
Health facilities under results-based financing received incentive payments based on the quantity of nine health services (Box 1) and the quality of 10 aspects of care. Fig. 1 shows the distribution of the incentive payments by service.

Incremental cost assessment
To provide practical recommendations for decision-making, we conducted the cost-effectiveness analysis from a health-system perspective and examined financial costs rather than economic costs that capture additional opportunity costs. In this analysis, we focused on the incremental cost (additional cost) incurred in the results-based financing and input-based financing groups, compared with the control. Thus, for the cost analysis, the additional costs that we included in the analysis were: the costs of consumables (e.g. drugs and supplies) due to increased services; the costs incurred at World Bank headquarters for designing, implementing and monitoring the programme; and the field costs of implementing the programme in Zambia. All costs were measured in United States dollars (US$) over the period of the programme's implementation from April 2012 through June 2014.
We obtained the costs of consumables from a data set compiled by Medical Stores Limited, the national pharmaceutical distribution hub, from January 2011 through June 2014. The cost of consumables for each health facility was calculated as the cost per capita per quarter and then a difference-in-differences approach, which adjusted the baseline differences, was used to determine the incremental cost of consumables. We allocated the World Bank headquarters' costs to the results-based financing and input-based financing groups in proportion to the implementation costs of each group. We obtained the programme costs from the World Bank Zambia office. Local costs were primarily for administration of the programme (e.g. costs of op-

Use of health services
To assess the impact of results-based financing on service delivery, the World Bank conducted household surveys before and after the implementation of results-based financing in November to December 2011 (baseline) and November 2014 to January 2015 (endline), respectively. These surveys included 3064 and 3500 households, respectively, with women who had had at least one birth within 2 years before the survey. The household survey results provided estimates of coverage of services for antenatal care, postnatal care, institu-tional delivery, immunization and use of intermittent preventive treatment for pregnant women. 16 An impact evaluation was conducted by the World Bank, using the household surveys and a difference-indifferences approach, to examine the effect of results-based financing on key maternal and child health services. 16 Key advantages of using difference-indifferences analysis are its adjustment for the baseline differences among the three groups, and robustness in estimating the impact. 17 From the household survey, the following services were included: antenatal care, postnatal care, institutional delivery, intermittent preventive treatment for pregnant women and immunizations.
At the same time as the household surveys, the World Bank also made health-facility surveys, each of which covered 176 and 210 representative health facilities at the baseline and endline and were verified by field supervisors. For this study, the health-facility survey provided information on use of injectable contraceptives, the general quality of care and service-specific quality measures for constructing quality indexes. We used the use of injectable contraceptives for impact evaluation, after converting the measure to population coverage, as this was the only family planning service showing any statistically significant differences. The calculated national coverage of injectable contraceptives was 21.9% (62 439/285 113) in 2014, similar to the national estimate of 19.3% (1903/9859) in 2013. 2 The quality measures used by the programme for incentive payments were not used for impact analysis, as they were assessed only for facilities receiving results-based financing.
Curative services and HIV/AIDS services, not available in the household survey, were not statistically significant based on the data from the health-facility survey, and the Lives Saved Tool could not handle the analysis of the curative care. These two services were therefore excluded from the analysis, but all other incentivized services were included.

Quality of health services
The health-facility survey measured (i) general quality, (ii) clinical process, (iii) availability of drugs and supplies, (iv) equipment and (v) qualified human resources. We selected questions on these five categories from the healthfacility survey. We convened a Delphi panel with expertise in epidemiology and clinical medicine in Zambia in November 2014 to determine the relative importance of each quality component, and generated a quality index (ranging from 0 to 1) for each service. Fig. 2 shows the relative importance of each quality component by service. The score of the constructed quality index was similar to that developed from items measuring process quality for the incentive payments in results-based financing facilities.
Quality of care may not have a linear relationship with health benefit gained from the care (e.g. 80% quality of care does not necessarily mean the care will gain 80% of its full potential of health benefits). To ascertain the impact of quality of care on potential health benefits from the care, we used the same expert panel to generate a health-effect index of quality of care using a quadratic function. The health-effect index estimated how much compromised quality affected the health benefits of a service, expressed as a percentage of the full potential of health benefits when the service was provided optimally. The Results-based financing, Zambia Wu Zeng et al.
quadratic function was used, because of its flexibility to accommodate different relationships between variables.

Modelling outcomes
We generated a measure of effective coverage by multiplying the health-effect index by the coverage of corresponding services. The result was treated as quality-adjusted coverage to feed into the Lives Saved Tool. [18][19][20][21] The tool is widely used to estimate maternal and child health outcomes (mortality), with good validity. 22 However, it only handles a set number of interventions. The tool cannot deal with morbidity, nor can it implement probabilistic sensitivity analyses. We used key parameters from the Zambia data preloaded in the tool and adjusted the population size to that covered by results-based financing. The tool converts the coverage of health services to the number of lives saved from improved services. We converted lives saved into quality-adjusted life years (QALYs), applying the formula for fatal cases 23 in Equation 1: Where Q is average quality of life if one survives, r is the annual discount rate (we applied 3% in the analysis) and L a is life expectancy at the age of death of a, estimated from Zambia's life table. 24 Q was estimated from the disease burden in Zambia 25 by Equation 2 as: Where d is disability, a is adjusted life years due to morbidity, h is health life expectancy and p is population size.
Years gained in early life play a more important role in determining QALYs. We estimated total QALYs gained by multiplying the number of cases saved by QALYs gained per case. The QA-LYs gained were then rescaled by the population covered by health facilities to estimate QALYs gained per capita.

Cost-effectiveness analysis
Based on incremental costs and effectiveness per capita, we estimated the incremental cost-effectiveness ratio under two scenarios: one without quality improvement and the other with it.
To examine the uncertainty of incremental cost-effectiveness ratios, we focused on the uncertainty of the impact of results-based financing on use of health services. To be conservative (i.e. having wider uncertainty intervals) and for the ease of conducting sensitivity analyses, we assumed the independence of the eight indicators included in the analysis. Thus, we used 31.2% confidence intervals (CI) of each indicator to estimate the 95% CI of incremental cost-effectiveness ratios.
While thresholds of cost-effectiveness analysis have been debated, 26-28 a study evaluating returns on investment specific to maternal and child health valued a healthy life year as 1.5 times gross domestic product (GDP) per capita, 19 which we used as the threshold to interpret the results. In 2013, GDP per capita was US$ 1759 in Zambia, 29 and thus the threshold was estimated at US$ 2639. Interventions with incremental cost-effectiveness ratios smaller than the threshold were regarded as cost-effective.

Results
Over the 2.25 years of the programme, costs after annualization were US$ 13.26 million in total, of which US$ 10.54 million (US$ 7.91 per capita) was used in the results-based financing districts, while US$ 2.72 million (US$ 2.16 per capita) was used in the input-based financing districts. Fig. 3 presents the distribution of non-annualized resultsbased financing and input-based financing programme costs by function. About half of the cost (US$ 8.49 million of the US$16 505 963) was used to pay incentives for results-based financing facilities and to provide comparable funding to input-based financing facilities. The World Bank headquarters' costs were US$ 566 711, with US$ 450 427 (i.e. US$ 0.34 per capita) and US$ 116 284 (i.e. US$ 0.09 per capita) for the resultsbased financing and input-based financing groups, respectively.
The drug and supply costs per capita per quarter in the period before implementation of the programme were estimated at US$ 0.26 for the resultsbased financing group, US$ 0.50 for the input-based financing and US$ 0.42 for the control (Table 1). These numbers increased to US$ 0.59, US$ 0.77 and US$ 0.64 in the period after programme implementation. Difference in differences estimated that costs of the resultsbased financing group were US$ 0.57 and US$ 0.97 more than the input-based financing and control groups, respectively, over the implementation period of the programme (9 quarters; Table 2).   4 summarizes the overall incremental costs per capita during the programme implementation period. Costs in the results-based financing group were US$ 6.56 and US$ 9.21 per capita more than in the input-based financing and the control groups, respectively. In input-based financing group, costs were US$ 2.66 per capita more than in the control group. Table 3 shows coverage of the key maternal health services and quality of care of services at baseline and endline from the household or health-facility surveys. 16 A statistically significant difference was found in both Haemophilus influenza type B vaccination and use of injectable contraceptives, where the results-based financing group had higher use than the input-based financing group. The uptake of injectable contraceptives, for example, increased from 6.5% to 34.0% and from 9.9% to 15.6% in the results-based financing and inputbased financing groups, respectively. Compared with the control group, the provision of services increased in the results-based financing group by 12.8% for institutional deliveries, 8.2% for postnatal care, 19.5% for injectable contraceptives and from 3.0% to 20.4% for vaccinations. The input-based financing group had significantly greater improvements in institutional deliveries and postnatal care versus the control group.
Quality-of-care indices in the results-based financing group increased more than in the input-based financing group for institutional deliveries (0.7%), vaccinations (3.2%) and injectable contraceptives (4.9%; Table 3). Compared with the control group, the results-based financing group had greater improve-ments in quality of care: by 3.1% for institutional deliveries, 2.9% for antenatal care, 2.3% for postnatal care, 3.8% for vaccinations and 9.7% for injectable contraceptives. The input-based financing group also had improved quality of care in comparison with the control group.
Among the total 1.33 million population, 11 more lives were saved among pregnant women and 214 more lives among children younger than 5 years in the results-based financing group compared with the input-based financing group over the programme implementation period (without including quality of care in the analysis; Table 4). In the results-based financing group, 22 lives were saved among pregnant women and 497 lives among children younger than 5 years compared with the control group. After adjustment for improved quality of care, the estimated number of lives saved in the results-based financing group doubled. Specifically, 279 more lives were saved among mothers and children (lower-upper bounds: 214-324) in the results-based financing group versus the input-based financing group, and 641 more lives were saved (lower-upper bounds: 580-700) versus the control group.
When converting health benefits to QALYs gained, 5325 QALYs were saved (lower-upper bounds: 3948-6317) in the results-based financing group versus the input-based financing group, and 12 291 QALYs (lower-upper bounds: 10 905-13 639) versus the control group, when quality was not adjusted (Table 5). These were equivalent to gaining 0.0041 QALYs and 0.01 QALYs per capita. The number of QALYs gained was doubled when the improvement of quality of care was considered.
The incremental cost-effectiveness ratios of results-based financing were US$ 1642 and US$ 999 per QALY gained when compared with the input-based financing and control groups, respec-  Results-based financing, Zambia Wu Zeng et al.
tively, without adjustment for quality of care (Table 6). These ratios fell to US$ 1324 and US$ 809 per QALY gained if quality of care was included. The incremental cost-effectiveness ratios for the input-based financing versus the control group were US$ 508 and US$ 413 per QALY gained, without and with quality adjustment, respectively.

Discussion
This study was an attempt to estimate the value of results-based financing on both costs and effectiveness of maternal and child care. The results showed that the results-based financing programme in Zambia, in comparison with the control group, is a cost-effective approach to improving maternal and child health. The mid-point incremental cost-effectiveness ratios were US$ 999 and US$ 809 per QALY gained, without and with quality adjustment, respectively. Both values were less than 1.5 times the GDP per capita in 2013 in Zambia. 29 In comparison with input-based financing, there were greater health benefits in results-based financing districts, but with higher costs. However, the resultsbased financing programme remained cost-effective using the same threshold, and the mid-point incremental cost-effectiveness ratio was US$ 1324 per QALY gained.
This study is consistent with many studies showing a favourable impact of results-based financing in improving the uptake of maternal and child health services. 8,10 We found that the major increases in quantities of health services were institutional deliveries, postnatal care visits, Haemophilus influenza type B vaccination and family planning using injectable contraceptives. The potential improved motivation among staff, greater flexibility in managing financial resources and strengthened supervision through results-based financing may contribute to such improvements. Concerns have been raised whether results-based financing neglects non-incentivized services. 11 The chance of such an effect in our study seems slim given that the incentivized indicators spanned the majority of services provided in a health facility. In Haiti, a study showed no negative effect of results-based financing on non-incentivized services. 11 Compared with the control districts, facilities in results-based financing districts showed substantial improvements in using injectable contraceptives for family planning. The improvement of injectable contraceptives is an important factor in determining the cost-effectiveness ratio. The resultsbased financing programme spent about 29% of incentive payments on family planning. Investment in family planning is regarded as one of most cost-effective approaches to reducing both maternal and child mortality rates, 19,30,31 not only avoiding unintended pregnancies, but also reducing health risks for both pregnant women and infants themselves.
Few studies have examined increased financing alone against the usual financing method without additional funding. Compared with control, the input-based financing method was costeffective, with a lower incremental costeffectiveness ratio than results-based financing. In health facilities where resources are constrained, this suggests that simply providing more financial support would improve maternal and child health substantially. However, input-based financing also contained a directive that these funds be spent on maternal and child health services. This signalling may have played an important role in coordinating efforts around the additional resources for maternal and child health at the district and facility level. As none of the input-based financing resources were allowed to be spent as staff bonuses, the amount of financing available for facility investments was similar to that with results-based financing after subtracting staff bonuses (47% of results-based financing transfers to facilities were allocated to staff bonuses).
We incorporated quality of care in the results-based financing cost-effectiveness modelling. Given that quality of care is one of the major components of the programme and a component of the payment formula to health facilities, an analytical model not incorporating quality of care would underestimate the cost-effectiveness of results-based financing. Studies show that improving quality of care is important for achieving better health outcomes and could greatly reduce mortality rates. [32][33][34] Services with high baseline coverage, such as antenatal care and vaccina-   The impact was estimated with linear probability models with difference-in-difference specification, including controls for district stratification. Errors are clustered at the community level. b Data were from the health-facility survey.
c Quality index was constructed at the group level, through averages of key quality dimensions across-facilities, and thus no statistical test was conducted. Notes: Results-based financing facilities received increased funding tied to performance on pre-agreed indicators; input-based financing facilities received increased funding not tied to performance; control facilities were under the usual funding system without additional financing in Zambia.
Results-based financing, Zambia Wu Zeng et al.
tion, showed the least improvement under results-based and input-based financing. Results-based financing continued incentivizing those services. As a means to enhance the cost-effectiveness of the programme, payments to services with high coverage may need to be reduced. Reducing excessive verification costs could be another approach to improve the efficiency of the programme. In Zimbabwe, for better efficiency, a risk-based verification approach was implemented. 35 Cost-effectiveness is not the only criteria for decision-making. In scaling up the results-based financing programme, other aspects need to be considered, such as affordability, spillover effects, sustainability, equity and political factors. 27 This study has several limitations. First, the focus of the study was financial costs instead of economic costs. Economic costs in the results-based financing districts may be slightly higher than those in other districts, as the staff members tend to be more satisfied with their jobs and work additional time. Second, converting the coverage of health services to health benefits relied on both the international literature about the effectiveness of the interventions and the overall national statistics in Zambia. These sources do not necessarily represent the parameters in the resultsbased financing intervention areas in this study. Third, the CIs of incremental cost-effectiveness ratios assumed that the indicators analysed are statistically independent; as costs and services are often positively correlated, these CIs should be regarded as the outer limits of the true intervals. Fourth, payments to input-based financing facilities deviated from the original plan. Only 56% of the results-based financing was transferred to input-based financing districts, adding complexity to the comparisons. Fifth, the cost estimates may have included certain cost categories such as introductory training that are generally present in pilot programmes, but not mature programmes. This element would overstate the costs of resultsbased financing against the alternatives. Sixth, the baseline for some indicators is not well-balanced. The control districts had a relatively high coverage of some maternal and child health services when compared with input-based financing. Lastly, the sample population for the household survey was from households with recent births, and some measures,     such as family planning, were estimated from health-facility data to mitigate the potential bias. This study provides empirical evidence on cost-effectiveness of results-based financing using a rigorous randomized trial. Both quantity and quality improvements contributed to the cost-effectiveness of incentivized financing. The analysis compared resultsbased financing with an input-based financing method, demonstrating the pure impact of incentives alongside associated activities while controlling for total financial flows. Additionally, this study presented an approach to model the impact of quality improvement on health outcomes through a Delphi panel process. ■ ment partners during the Delphi consultation workshop in Lusaka, Zambia in November 2014. The study is registered at ISRCTN, and the trial registration number is ISRCTN14332616.