Postpartum family planning in Rwanda : a cost effectiveness analysis

Globally, there is a large unmet need for family planning in Background: the postpartum period: 90% of women in this group want family planning for birth spacing or to avoid unintended pregnancies and stop child bearing once desired family size has been reached. In total 76% of Rwandan women want family planning postpartum, yet a 26% unmet need remains. Currently, the four most commonly used postpartum family planning methods in Rwanda are injections, subdermal implants, pills, and condoms. The economic and health benefit impact of the current method selection has not yet been evaluated. To evaluate the impact of current usage rates and method types, Methods: this cost effectiveness analysis (CEA) compared the most frequently used family planning methods in Rwanda broken into two categories, longer-acting reversible contraception (LARC) (injections and subdermal implants) and shorter-acting reversible contraceptives (non-LARC) (pills and condoms). A time horizon of 24 months was used to reflect the World Health Organization suggested two-year spacing from birth until the next pregnancy, and was conducted from a health systems perspective. This CEA compared two service package options to provide a comparator for the two method types, thus enabling insights to differences between the two. For women of reproductive age (15-49 years) in Rwanda, Results: including LARC postpartum family planning methods in the options, saves $18.73 per pregnancy averted, compared to family planning options that offer non-LARC methods exclusively. There is an opportunity to avert unplanned pregnancies Conclusion: associated with increased utilization of LARC methods. The full benefits of LARC are not yet realized in Rwanda. Under the conditions presented in this study, a service package that includes LARC has the potential to be cost-saving compared with one non-LARC methods. Effective health messaging of LARC use for the postpartum population could both enhance health and reduce costs. 1 1 1 1

report report report report

Introduction
A multitude of variables influence maternal and child health outcomes, such as delivery of and access to antenatal care, giving birth in facilities, and many more. However, the postpartum period, is equally important and often overlooked. The evidence of reduced maternal mortality upon engagement in postpartum family planning (PPFP), and the absence of robust PPFP programs globally, leads the World Health Organization (WHO) to distinguish the postpartum period as a key opportunity for promoting the health of mothers and babies 1 .
Women spend on average about 30 years, or three-quarters of their reproductive lives, attempting to avoid pregnancy 2 . Globally, there is a large unmet need for family planning in the postpartum period: 90% of women in this group want family planning, for birth spacing or to avoid unintended pregnancies and stop child bearing once desired family size has been reached. In Rwanda, where there has been a coordinated response to maternal and child health; family planning provides a platform to continue the nation's trend of improved maternal and child health. The WHO recommends at least 24 months between a birth and the next pregnancy for improved maternal and child health outcomes 3 . Rwanda's commitment to scale up PPFP is well established 4 . In total 76% of Rwandan women want PPFP and a 26% unmet need remains 5 . Despite this, about one half of births are conceived before the recommended interval of 24 months 6 .
Faith-based health facilities make up 30% of Rwanda's healthcare system and fill critical gaps in care 7,8 . Some denominations do not include comprehensive contraceptive options, leading to possible barriers to access 9,10 . "More effective" family planning methods, such as those included in this analysis, remain absent at these facilities but are made available at nearby health posts 9 . Thus, attention to potential cost savings with the inclusion of these methods at all of Rwanda's health facilities warrants further attention.
This cost effectiveness analysis (CEA) compared two categories of family planning methods of postpartum women of reproductive age (15-49 years) in Rwanda. Each consisted of one of the two most utilized methods in Rwanda: longer-acting reversible contraceptives (LARC) identified as injections and subdermal implants; and shorter-acting reversible contraceptives (non-LARC), pills and condoms 11 . A total of 45% of women do not use contraception postpartum, and this comparator is also represented in the model 5 .

Methods
This CEA compared two interventions, addressing whether the additional cost of LARC is justified by the additional health benefits from a health systems perspective ( Figure 1). TreeAge Pro 2018 R1 was used to develop the model and run the sensitivity analyses a . A time horizon of 24 months was used to reflect a Alternative software to run the analysis: Microsoft Excel.

Amendments from Version 1
This revision incorporates a re-framing of the conclusion, corrections to some references, and other clarifying points highlighted by the thoughtful comments of the first reviewer. Figure 1. The two primary arms comparing family planning menu types. One includes the most common longer-acting reversible contraceptives (LARC), injections and subdermal implants; the second only provides the most common shorter-acting reversible contraceptives (non-LARC), pills and condoms 11 . Decision nodes are indicated with a circle; probability values are listed below each type and method. One-way sensitivity analyses were executed for variables with the greatest impact on averted pregnancies, as determined by a tornado diagram. Input ranges were set to 50-150% of original value.

Results
The use of LARC methods saved $18.73 per pregnancy averted compared to contraceptive selections with non-LARC methods only (Table 2). When LARC is included in the menu, LARC is the dominating option among the contraceptive methods -the use of this contraceptive type both saves money and averts unwanted pregnancies with higher probability when compared with the non-LARC or no contraception use options.
Sensitivity analysis outputs are illustrated in (Figures 2a-c).

Limitations
There is uncertainty surrounding the following elements: 1. Study population -This analysis is limited to the evaluation of women in Rwanda who access care at a public facility. It fails to capture those who do not seek care at a government institution. However, since about 92% of Rwandan women deliver at a public healthcare facility, our analysis reflects the majority of the population 5 .
2. Contraception type -This analysis included two LARC methods and two non-LARC methods and therefore excludes some other contraceptive options. These four methods make up 77% of the contraceptive uptake in Rwanda and therefore is a reasonable representation of the population 11 . Yet, possible effects of the introduction of LARC methods on the methods not portrayed here could change the results. However, since these other methods are less effective than LARC, their inclusion is unlikely to diminish the estimated cost-effectiveness of LARC.
3. Tapering effectiveness upon discontinuation of injection method -It is reported that the injection method continues to provide protection for nine months following the last injection 19 . This phenomenon is modeled through an altered discontinuation rate if injection follow-up stopped at nine months postpartum.
4. This analysis does not include the societal perspective; LARC savings are thus likely undervalued.

Conclusion
This evaluation shows that the inclusion of LARC methods in contraceptive options results in savings of $18.73 per pregnancy averted compared to family planning menus that offer only non-LARC methods for women in Rwanda for two years following birth. With Rwanda's current population of 12.8 million, a birth rate of 32.23/1,000, a 37% unplanned birth rate of which 50% are conceived before the recommended interval, if an additional 25% of this group uses a LARC method as their PPFP method choice, $951,000 US$ per year can be saved 6,20,21 . These are funds that the Rwanda Ministry of Health can apply to other high value interventions. With the incorporation of a greater than two-year postpartum window, additional cost savings is projected. This model does not capture the health benefits to the mother and second baby incurred if the two-year minimum window is observed. To increase rates of LARC PPFP use, the Rwanda Ministry of Health may see increased rates of uptake with public health messaging specifically targeting the benefits of LARC use and the identified barriers to LARC use such as male involvement and postpartum risk of pregnancy, among others 22,23 . Future areas of research include: analysis of barriers and facilitators to PPFP uptake with particular focus to currently used LARC methods, exploration of barriers to access for women utilizing neighboring health posts of faith-based facilities in lieu of government health facilities, and investigation of reasons for low intrauterine device utilization (<0.07%) 5 .

Data availability
All data underlying the results are available as part of the article and no additional source data are required.

Grant information
This work was supported by the Bill and Melinda Gates Foundation [OPP1181398].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgments
The preliminary stage of this work was supported by the University of California "… including LARC postpartum family planning methods in the options saves $18.73 per pregnancy averted, compared to family planning options that offer non-LARC methods exclusively." I still do not understand the rationale behind the research question. LARCs (defined here as injectables and implants) are already included in the FP method options in Rwanda. They are already offered and providers are already trained to provide implants and injectables. The counterfactual of exclusive offer of non-LARC methods is not the current standard of care, nor will it ever be a real counterfactual scenario in Rwanda. Is this another instance where the authors need to reframe their rationale to be talking about uptake instead of accessibility? If so, please see and revise mentions of 'LARC inclusion' or 'when LARC is included'. Effective public health messaging campaigns and other promotion targeting current resistance to LARC use for the postpartum population could both enhance health and save public health funds." What current resistance to injectables and implants? This needs elaboration with citations in the discussion section if this is actually occurring.

Introduction:
Thus, attention to potential cost savings with the inclusion of these methods at all of Rwanda's health facilities warrants further attention." I still do not understand this point. Please see my previous comment. Are the authors saying that methods should be added to the faith-based facilities (since they are already offered at all other government facilities)? If this is not a focus of the analysis, suggest to remove.
I suggest you provide both the percentage of unmet need for spacing and the Comment/response 7: percentage for limiting, as this gives the reader a better idea of the large unmet need for FP in postpartum periods in the population.
Of course the focus on PPFP is critical. The comment was asking the Response to comment 11: Of course the focus on PPFP is critical. The comment was asking the Response to comment 11: authors to explain why they were considering scenarios with limited accessibility for PPFP, which in reality do not exist. I understand that they are now trying to re-frame their argument to focus on different uptake scenarios (not service provision scenarios which will never include no LARC options); more work needs to be done to clarify this.
: "…family planning menu types" Again, these are not realistic 'menu types.' No facilities that Figure 1 title provide FP are providing non-LARC only (again, the accessibility versus uptake issue). The tree would need to be re-framed as differences in uptake scenarios.
To re-frame the research question to one of different uptake scenarios Response to comment 12: (versus accessibility), the authors could easily use a realistic control comparator of the current distribution of PPFP methods. And then explore scenarios where uptake of LARC methods is increased. The current 'non-LARC only' arm is not standard of care and doesn't provide a realistic comparison scenario.
Acceptability of the IUD has been shown to be high when the method is Response to comment 13: accompanied with educational counseling and availability from well-trained providers. Please see Ingabire (2019 ) and other salient articles. This is a method that is very attractive to governments et al. due to low commodity cost and high user acceptability, and at the very least should be mentioned in the discussion.
This response confuses me. The authors have kindly clarified that they are Response to comment 14: not looking at different accessibility scenarios as presented in the first version (because as I think we agree, the non-LARC scenario is not a realistic comparator), but are in fact considering different uptake scenarios. If that truly is the intent, then the authors must recognize that increasing uptake requires demand creation which comes at a cost.
Were the effects also discounted at 3%/year? Please clarify in the text.

Response to comment 15:
Are the commodity costs for POP versus COC different?

Response to comment 19:
The methods for the weighting should be clarified in the methods. How can Response to comment 20: the usage prevalence estimates be weighted based on usage prevalence? Perhaps this was a typo.
The response to 20 did not clarify this question for me. Perhaps I missed Response to comment 21/26: it in the paper, but where do the assumptions of 48% and the other distribution estimates come from?
A rate is not a probability. Are these annual rates? If so, please state Response to comment 23: '3%/year'.

Methods:
It was unclear in the first version of the paper that the authors were only considering of spacing pregnancies for 2 years, and I do not see the rationale for excluding women who want to limit pregnancies as well. "…addressing whether the additional cost of LARC is justified by the additional health benefits from a health systems perspective." I would re-frame this as addressing whether efforts to increase LARC uptake are cost-effective. Dr. Wall, Thank you for the thorough review and attention dated June 5, 2019. We have carefully checked the comments and revised the manuscript accordingly. Our responses are given in a point by point manner below. Sincerely, Author Team Abstract Reviewer comment (RC) 1: "… including LARC postpartum family planning methods in the options saves $18.73 per pregnancy averted, compared to family planning options that offer non-LARC methods exclusively." I still do not understand the rationale behind the research question. LARCs (defined here as injectables and implants) are already included in the FP method options in Rwanda. They are already offered and providers are already trained to provide implants and injectables. The counterfactual of exclusive offer of non-LARC methods is not the current standard of care, nor will it ever be a real counterfactual scenario in Rwanda. Is this another instance where the authors need to reframe their rationale to be talking about uptake instead of accessibility? If so, please see and revise mentions of 'LARC inclusion' or 'when LARC is included'.

Response (R) 1:
The rationale behind this analysis is to provide the Rwanda Ministry of Health (MOH) a platform to make informed choices on next best steps to encourage postpartum family planning (PPFP) to more fully realize the maternal and child health outcomes of when birth spacing of 2 years is observed. This analysis is not a representation of family planning methods access in Rwanda. The "Non-LARC only" arm of the model is needed to provide a comparator by which to determine the cost effectiveness of the longer acting options. It does not represent one type of facility versus another. By using the four most commonly utilized methods of PPFP in this model, which make up 77% of PPFP methods in Rwanda, we are able to highlight opportunities for informed next steps pertaining to PPFP public messaging leveraging already socially accepted methods. 77% of PPFP methods in Rwanda, we are able to highlight opportunities for informed next steps pertaining to PPFP public messaging leveraging already socially accepted methods.

RC2:
Effective public health messaging campaigns and other promotion targeting current resistance to LARC use for the postpartum population could both enhance health and save public health funds." What current resistance to injectables and implants? This needs elaboration with citations in the discussion section if this is actually occurring.

R2:
Thank you for pointing out this sentence. Upon reconsideration, we believe it is more accurate to state as: "Effective public health messaging LARC use for the postpartum population highlighting could both enhance health and save public health funds."

RC3:
Thus, attention to potential cost savings with the inclusion of these methods at all of Rwanda's health facilities warrants further attention." I still do not understand this point. Please see my previous comment. Are the authors saying that methods should be added to the faith-based facilities (since they are already offered at all other government facilities)? If this is not a focus of the analysis, suggest to remove.

R3:
Thank you for providing the opportunity to clarify. Faith-based health facilities make up 30% of Rwanda's healthcare system and fill critical gaps in care.[1,2] Some denominations of faith-based facilities offer natural methods only (rhythm beads), leading to possible gaps in care.[3,4] "More effective" family planning methods, such as the four most utilized by postpartum women in Rwanda modeled in this analysis, remain absent at these facilities.
[3] Rwanda provides a suite of contraceptive options at its health facilities, and nearby health posts for facilities managed by some denominations. Generally, a gap remains in data and knowledge on the subject of faith-based organizations' contribution to reproductive, maternal, newborn, and child health (RMNCH) healthcare delivery, particularly in low-and middle-income (LMIC) countries. [5,6] The MOH in Rwanda has attempted to address the gap in family planning services accessibility through the establishment of health posts in these areas to provide access to these family planning methods, however, whether this has addressed the gap in access remains unknown. Because no research on usage rates comparing PPFP usage and access between faith-based facilities and their corresponding health posts versus government health clinics, the authors agree it is pertinent to consider the differences in PPFP access between these two facility types. The authors are not insinuating any action beyond what is stated in the sentence, which is the open acknowledgement that not all facilities carry all family planning methods, usage rates between the two types are unknown, and thus, with consideration for potential cost savings, further attention to this sector of the health system is warranted.

RC3a:
Comment/response 7: I suggest you provide both the percentage of unmet need for spacing and the percentage for limiting, as this gives the reader a better idea of the large unmet need for FP in postpartum periods in the population.

R3a:
Thank you for these insights. The next version of this article will include the following in the second paragraph of the introduction: "In total 76% of Rwandan women want PPFP; a 51% unmet need for PPFP exists of which 26% is an unmet need for spacing and 25% an unmet need for limiting."

RC4:
Response to comment 11: Of course the focus on PPFP is critical. The comment was asking the authors to explain why they were considering scenarios with limited accessibility for PPFP, which in reality do not exist. I understand that they are now trying to re-frame their argument to focus on different uptake scenarios (not service provision scenarios which will never include no LARC options); more work needs to be done to clarify this.

R4:
Thank you for the opportunity to clarify. Your previous comment sought additional explanation for the selection of the postpartum population focus in this model as compared to nulliparous women. Unlike nulliparous women, parous women prioritize the ability to manage inter-pregnancy intervals, avoid unintended pregnancies, and halt childbearing when desired.
[1] Thus, distinctive interventions for this population, compared to women who have not been pregnant, are necessary. Additionally, because the postpartum period has been identified as an opportunity to continue the Rwanda's trend towards improved RMNCH, and an area of specific interest by the Maternal, Child, and Community Health Division, this population was the focus for this analysis. With greater understanding to the cost-benefits tradeoffs, this analysis has the potential to influence RMNCH resource allocation and decision-making in Rwanda and thus the promotion and uptake of LARCs. Additionally, in alignment with the priorities of the Gates Foundation, data generation, particularly in the context of underserved and under-researched populations such as postpartum women in a low income country setting, is encouraged. Thus, there is value to greater understanding of the current environment. 1.
Ankomah A. Myths, misinformation, and communication about family planning and contraceptive use in Nigeria. Open Access J Contracept. 2011;Volume 2: 95. doi:10.2147/OAJC.S20921 Figure 1 title: "…family planning menu types" Again, these are not realistic 'menu types.' No facilities that provide FP are providing non-LARC only (again, the accessibility versus uptake issue). The tree would need to be re-framed as differences in uptake scenarios.

R5:
Thank you for highlighting this point. We have changed Figure 1 to the suggested phrasing of "uptake scenarios."

RC6:
Response to comment 12: To re-frame the research question to one of different uptake scenarios (versus accessibility), the authors could easily use a realistic control comparator of the current distribution of PPFP methods. And then explore scenarios where uptake of LARC methods is increased. The current 'non-LARC only' arm is not standard of care and doesn't provide a realistic comparison scenario.
We're looking at understanding the cost impact of two family planning method R6: current categories. Two service package options are modeled as is required to isolate differences between the method types. The two uptake scenarios as modeled in Figure 1 do represent existing not two service package options. The Non-LARC Only arm serves as a comparator to illuminate the benefits that LARC provides. This model does not project costs for future training and higher uptake levels of methods currently less utilized. We agree with the reviewer that the IUD has much untapped potential in Rwanda. This model provides a baseline for which to compare once the IUD has greater acceptance.

RC7:
Response to comment 13: Acceptability of the IUD has been shown to be high when the method is accompanied with educational counseling and availability from well-trained providers. Please see Ingabire et al. (20191) and other salient articles. This is a method that is very attractive to governments due to low commodity cost and high user acceptability, and at the very least should be mentioned in the discussion.

R7:
Thank you for this point. The purpose of this research document is to provide cost information for current highly utilized PPFP methods. The inclusion of the IUD in this model, along with other methods utilized by a minority of postpartum women in Rwanda, have low cost impact as a result of their low utilization (e.g. IUD usage rate of <0.07%). This model does not project costs for future training and higher uptake levels of methods currently less utilized. We agree with the reviewer that the IUD has much untapped potential in Rwanda. This model provides a baseline for which to compare once the IUD has greater acceptance. The following sentence will be added to the next article version: "To increase rates of LARC PPFP use, the Rwanda Ministry of Health may see increased rates of uptake with public health messaging specifically targeting the benefits and identified barriers to LARC use such as male involvement, postpartum risk of pregnancy, and intrauterine device (IUD) uptake, among others."

RC8:
Response to comment 14: This response confuses me. The authors have kindly clarified that they are not looking at different accessibility scenarios as presented in the first version (because as I think we agree, the non-LARC scenario is not a realistic comparator), but are in fact considering different uptake scenarios. If that truly is the intent, then the authors must recognize that increasing uptake requires demand creation which comes at a cost.

R8:
The reviewer and authors agree that a non-LARC only scenario is representative of a current not service package. Modelling different uptake scenarios is an area for future research. The purpose of the current report is not to model increased uptake, thus including costs associated with uptake is also not the purpose, but another ripe area for future research. The model displays current usage rates and costs of the four most utilized methods for this population. The selection of a LARC 1.
rates and costs of the four most utilized methods for this population. The selection of a LARC method by a Rwandan woman within the two year postpartum window saves $18.73 per pregnancy averted. Thus, because we are projecting increased uptake, but rather illustrating not the cost benefit at existing usage rates, the costs that would be required to accurately represent such a scenario are not appropriate for this model.

RC9:
Response to comment 15: Were the effects also discounted at 3%/year? Please clarify in the text.

R9:
Thank you for this point. Cost data only was discounted and is clarified in the following sentence in the Methods: "All cost data was: converted to dollars cost in Rwanda based on mean market exchange rate, adjusted to 2018 US$ for the first year, and applied a 3% inflation rate for the second year of the 24-month time frame for the analysis. The health effects (number of pregnancies averted), were discounted at the same rate."

RC10:
Response to comment 19: Are the commodity costs for POP versus COC different?

R10:
The difficulty in accurately distinguishing the combined oral contraceptive (COC) and progestogen-only pill (POP) is described in the literature.
[1] Oral commodity costs are considered as one value because the usage rates of oral contraceptives in Rwanda do not distinguish between the two types. In the literature, these costs are not disaggregated and thus are reflected as one cost in this analysis.

RC11:
Response to comment 20: The methods for the weighting should be clarified in the methods. How can the usage prevalence estimates be weighted based on usage prevalence? Perhaps this was a typo.

R11:
Thank you for highlighting this inconsistency. In Table 1, the four family planning type rows (injectable, implant, male condom, and pills) have two values in the probability column. The second value, indicated with an *, is the calculated weighted value The following will be added to the Table 1 footer: "The three nodes stemming from the "LARC+Non-LARC" arm are weighted. LARC (0.48) +Non-LARC (0.29) + No Contraception (0.23) = 1, or 100%; the same method applied to the "Non-LARC Only" arm. Decision nodes must equal 1; thus, as defined by our parameters of the family methods included in this analysis, values were weighted based on percent uptake (proportion of women selecting the method)."

RC12:
Response to comment 21/26: The response to 20 did not clarify this question for me. Perhaps I missed it in the paper, but where do the assumptions of 48% and the other distribution estimates come from?

R12:
A table with the weighted calculation details has been included with the next manuscript revision.

RC13:
Response to comment 23: A rate is not a probability. Are these annual rates? If so, please state '3%/year'. '3%/year'.

R13:
Thank you for this point. We misinterpreted your question on the previous comments. Yes this is 3%/year and will be changed in the next version.

RC14:
It was unclear in the first version of the paper that the authors were only considering spacing of pregnancies for 2 years, and I do not see the rationale for excluding women who want to limit pregnancies as well.

R14:
This analysis is specific to the 24 month birth spacing recommended by the WHO, supported by citation 3 in the manuscript in the Introduction. The Methods section includes rationale for this point: "A time horizon of 24 months was used to reflect the WHO suggested two-year spacing from birth until the next pregnancy, defined for this analysis as the postpartum period, with time zero designated as time of birth. Pregnancy averted within this time horizon is the outcome of interest." This population subset was the focus in order to highlight the health benefit opportunities specific to birth spacing.

RC15:
"…addressing whether the additional cost of LARC is justified by the additional health benefits from a health systems perspective." I would re-frame this as addressing whether efforts to increase LARC uptake are cost-effective.

R15:
Thank you for this point. The following modification will be in the next version: "…addressing whether efforts to increase LARC uptake are cost-effective." No competing interests were disclosed.

Competing Interests:
Please define the 'postpartum' time period (for example, different sources may define this period as 1 year, 2 years, etc.).

"$2.8 million US$ per year can be saved if LARC is included as a contraceptive choice across all health centers in Rwanda…" Does this mean if LARCs (defined here as injectables and implants)
were included at the few (primarily Catholic) clinics where these methods are not offered? Injectables and implants are already offered at the majority of government clinics in Rwanda.

Introduction:
In It needs to be mentioned that while faith-based health facilities may not provide family planning in Rwanda, such health facilities must have a family planning post or point of access nearby and refer patients to those posts (which is mentioned in reference 9 and here: ). https://pdf.usaid.gov/pdf_docs/PA00HQSV.pdf "Some denominations…" Is this analysis comparing scaling up injectables and implants in the relatively few clinics which do not offer family planning due to religious reasons, versus the current situation in which those clinics do not offer any family planning (including OCPs and condoms), but instead refer women to nearby health posts?
From reading the introduction, I think the concept of the paper is potentially problematic. The authors make the case that faith-based organizations do not provide family planning methods and then say "Thus, attention to potential cost savings with the inclusion of these methods at all of " The vast majority of government facilities in Rwanda's health facilities warrants further attention. Rwanda are already providing the full range of contraceptive options (acknowledging occasional method stockouts and limited supplies of IUDs which have lower demand and are not modeled here). The faith-based organizations all now have nearby health posts that provide family planning and to which the faith-based organizations refer to (as well as community health workers who now provide some methods like the injectable in the community). Are the authors arguing that the faith-based (namely Catholic) organizations should themselves have to provide all family planning methods even though the nearby posts provide such methods? To what facilities are the authors suggesting that LARC (defined here as implants and injectables) be added? Additionally, the comparator in this analysis is offer of OCPs and condoms, which the Catholic facilities are also not providing.
The rationale for looking at postpartum periods specifically as it is modelled is not totally clear -are The rationale for looking at postpartum periods specifically as it is modelled is not totally clear -are the authors saying that postpartum of family planning services is lower than for provision non-postpartum family planning (I certainly recognize that is lower and thus unit need demand during postpartum periods is higher than for non-postpartum periods, but this article seems to be comparing different service provision scenarios)?
This article is comparing provision of LARC (injectables+implant) along with OCPs+condoms versus OCPs and condoms alone. Again, I don't think that any facilities in Rwanda are only providing OCPs+condoms.
One of the most cost-effective methods of contraception (with very low commodity costs), the copper IUD, is not included in this analysis. This LARC method has relatively low uptake and relatively few providers are trained to insert this method, particularly postpartum. I would strongly encourage the authors to include increasing the provision of the copper IUD as a scenario in their modelling.
The authors may be better served by reconceiving the research question to compare scenarios where of postpartum family planning (including the IUD) is increased (via demand uptake generation and IUD skills training), recognizing that implant, injectable, condom, and OCP provision are already being provided in most clinics.

Methods:
Table 1 specifies a 3% discount rate applied to costs, but this is not mentioned in the methods.
Were the effects also discounted? The table should clarify that this is 3%/year, if in fact the discount rate was applied annually. The column heading says 'estimate 2 years (USD)' so this is unclear.
I disagree with combining injectables and implants and naming them 'LARC'. Typically, LARCs are defined as the IUD and implant.
Age of consideration in the analysis is 12-49, but the model parameters come from DHS data which uses a 15-49 age range.
Breastfeeding postpartum women (the majority of postpartum women breastfeed in Rwanda) are not eligible for injectables per WHO guidelines until they are 6 weeks postpartum. Is this considered in the model?
POPs are available to postpartum women who are breastfeeding but COCs are recommended for non-breastfeeding women beginning at 3 weeks. Have these timing issues and differences in POP versus COC commodity costs been considered?
If '45 ' as stated, how is this represented in the % of women do not use contraception postpartum decision tree? In the no-LARC arm, 47% of women do not contracept. But in the LARC+non-LARC arm, only 23% of women do not contracept.
How do the authors arrive at the estimate that 48% of women will select LARC (per the decision tree)?
The methods are very underdeveloped. It would be helpful if the authors describe: the process for The methods are very underdeveloped. It would be helpful if the authors describe: the process for analyzing for the most influential variables in the model, the sensitivity analyses, how the cost ranges were selected in the sensitivity analysis, and how method discontinuations were handled in the model. What methods were women who discontinued condoms, pills, or no contraception assumed to take up?
What are the units of the discontinuation rates in Table 1?
The second paragraph of the results should be in the methods section, and expanded upon.
Are the 'chance of 'pregnancy' estimates given per year? The denominator should be stated. Regardless of the units, I do not think that the chance or pregnancy for injectables is 0.03% while for implant it is 1%. The implant is more effective than the injectable at pregnancy prevention per CDC typical-use failure rate estimates.
It is unclear how the distribution of method use was derived in Table 1 and Figure 1. Does this assume the current distribution of uptake in Rwanda? Table 1 footnote mentions that there was some weighting of estimates, but this is not described in the methods.
The outcome of interest (pregnancies averted) is never mentioned in the methods and should be explicitly stated.
Similar to the issues mentioned previously, the tile for Figure 1 states "The two primary arms comparing family planning menu types," but the non-LARC only option is not an option at government facilities.  Table 1, how is the cost estimate for pregnancy hospitalization taken from a study on post-abortion care (reference 18)?

Limitations:
It is worth mentioning that the societal perspective is not included, as is recommended.

Conclusions:
"This evaluation illustrates the inclusion of LARC methods in contraceptive options results in savings of $18.73 per pregnancy averted compared to family planning menus that exclusively include non-LARC methods for women in Rwanda for two years following birth." "…$2.8 million " US$ per year can be saved if LARC is included as a contraceptive choice across all health center. Again, it is unclear to me whether the authors modelled adding injectable/implant options to the very few clinics which do not offer them (which would primarily be clinics which offer no family planning methods at all, and not their counterfactual scenario of OCPs and condoms). If so, did the authors also model adding OCPs and condoms to those clinics?
With consideration for the country's low postnatal care attendance rate…" First, this sentence With consideration for the country's low postnatal care attendance rate…" First, this sentence needs a reference. Second, what are the authors referring to? A very high percentage of Rwandan women attend postnatal infant vaccination visits.

References:
Some references are incorrect, please double-check them (e.g., reference 7 is incomplete, reference 4 is not provided correctly, the link for reference 13 and 14 are the same/duplicated, referencing blogposts (reference 22) is not appropriate and the peer reviewed literature is preferred).

Is the work clearly and accurately presented and does it cite the current literature? No
Is the study design appropriate and is the work technically sound? Partly

If applicable, is the statistical analysis and its interpretation appropriate? Partly
Are all the source data underlying the results available to ensure full reproducibility? Yes

Are the conclusions drawn adequately supported by the results? No
No competing interests were disclosed.

Competing Interests:
Reviewer Expertise: Infectious disease epidemiology, HIV prevention and treatment, family planning, cost-effectiveness I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above. Sincerely,

RC12:
This article is comparing provision of LARC (injectables+implant) along with OCPs+condoms versus OCPs and condoms alone. Again, I don't think that any facilities in Rwanda are only providing OCPs+condoms.

R12:
The purpose of this model was to compare the two most commonly utilized longer-acting methods with the two most commonly utilized shorter acting methods in Rwanda. This is stated in the Abstract and last paragraph of the Introduction. The "Non-LARC only" arm of the model is needed to provide a comparator by which to determine the cost effectiveness of the longer acting options. It does not represent one type of facility versus another.

RC13:
One of the most cost-effective methods of contraception (with very low commodity costs), the copper IUD, is not included in this analysis. This LARC method has relatively low uptake and relatively few providers are trained to insert this method, particularly postpartum. I would strongly encourage the authors to include increasing the provision of the copper IUD as a scenario in their modelling.

R13:
Thank you for this point. As stated in the Abstract and Introduction, only the most common methods currently used by postpartum women in Rwanda were included in this analysis in order to more accurately model contraceptive usage and thus realistic cost. Very low acceptance of IUDs currently exists (<0.07%) (Rwanda DHS 2016). Despite the copper IUD's cost effectiveness, cultural acceptance has not yet occurred and is thus not relevant for this analysis. An investment in current attitudes and behavior change initiatives would be required before this is applicable in the Rwandan context and could relevant information to shift method use.

RC14:
The authors may be better served by reconceiving the research question to compare scenarios where uptake of postpartum family planning (including the IUD) is increased (via demand generation and IUD skills training), recognizing that implant, injectable, condom, and OCP provision are already being provided in most clinics.

R14:
The purpose of this research document is to provide cost information for currently utilized PPFP methods. It does not aim to project costs for future training and uptake of methods not currently accepted.

R16:
This analysis defined longer-acting methods as those not requiring intervention by the user on a daily or per sexual encounter basis.

RC17:
Age of consideration in the analysis is 12-49, but the model parameters come from DHS data which uses a 15-49 age range.

R17:
Thank you for pointing out this discrepancy. An age adjustment has been made to match the age provided in the DHS data.

RC18:
Breastfeeding postpartum women (the majority of postpartum women breastfeed in Rwanda) are not eligible for injectables per WHO guidelines until they are 6 weeks postpartum. Is this considered in the model?

R18:
Thank you for highlighting this point. For this model we used the higher dollar value to illustrate the more conservative perspective, representing the full 104 weeks ($22.15), as opposed to 98 weeks ($20.87) for this method. This representation of cost varies in error in favor of the non-LARC methods in the model, and thus if the injection supplier, type, and cost stays the same, it makes LARC methods more cost saving then shown. Reports have indicated that supply chains for the current injection method may not be reliable, thus, we selected the more conservative cost to be more inclusive of supplier and injection type changes.

RC19:
POPs are available to postpartum women who are breastfeeding but COCs are recommended for non-breastfeeding women beginning at 3 weeks. Have these timing issues and differences in POP versus COC commodity costs been considered?
Thank you for this point. Per WHO, "The number of users of combined orals in any situation R19: almost always exceeds the number of users of progestin-only orals. In addition, many service providers do not have clear guidelines about how to counsel women who might be eligible to use progestin-only orals-this makes it more difficult to accurately estimate the demand for POPs." http://apps.who.int/medicinedocs/documents/s21863en/s21863en.pdf For this reason, we did not differentiate between types of oral contraceptives. Effectiveness rates between the two are very similar.

RC20:
If '45% of women do not use contraception postpartum' as stated, how is this represented in the decision tree? In the no-LARC arm, 47% of women do not contracept. But in the LARC+non-LARC arm, only 23% of women do not contracept.

R20:
The three nodes stemming from the "LARC+Non-LARC" arm are weighted. LARC (0.48) The three nodes stemming from the "LARC+Non-LARC" arm are weighted. LARC (0.48) +Non-LARC (0.29) + No Contraception (0.23) = 1, or 100%; the same method applied to the "Non-LARC Only" arm. Decision nodes must equal 1; thus, as defined by our parameters of the family methods included in this analysis, values were weighted based on usage prevalence.

RC21:
How do the authors arrive at the estimate that 48% of women will select LARC (per the decision tree)? R21: See Response 20.

RC22:
The methods are very underdeveloped. It would be helpful if the authors describe: the process for analyzing for the most influential variables in the model, the sensitivity analyses, how the cost ranges were selected in the sensitivity analysis, and how method discontinuations were handled in the model. What methods were women who discontinued condoms, pills, or no contraception assumed to take up? R22: Thank you for highlighting an area that needed further explanation. The process for analyzing the most influential variables in the model was determined by a tornado diagram, which compares the relative importance of variables. Sensitivity analyses were processed through the use of "chance" nodes in TreeAge Pro, which provides the functionality of sensitivity analyses within the software. The cost ranges were selected in the sensitivity analysis and are as stated in the second paragraph of the methods which states: "Input ranges were set to 50-150% of original value." In regard to method discontinuations in the model, discontinuation rates and subsequent uptake probabilities are provided in Table 1. Discontinuation was modeled after 1 year of use (1 year postpartum) with consideration for effectiveness tapering (effectiveness is maintained for 6 months following missed injection date). Additionally, limitation 3 provides further detail of the consideration given to discontinuation: "Tapering effectiveness upon discontinuation of injection method -It is reported that the injection method continues to provide protection for nine months following the last injection. This phenomenon is modeled through an altered discontinuation rate if injection follow-up stopped at nine months postpartum."

RC23:
What are the units of the discontinuation rates in Table 1? R23: Percentage (%) -the unit indicated with the column heading of probability (the likelihood or chance of an event occurring).

RC24:
The second paragraph of the results should be in the methods section, and expanded upon.

R24:
This sentence was moved and expanded upon.

RC25:
Are the 'chance of 'pregnancy' estimates given per year? The denominator should be stated. Regardless of the units, I do not think that the chance or pregnancy for injectables is 0.03% while for implant it is 1%. The implant is more effective than the injectable at pregnancy prevention per CDC typical-use failure rate estimates. CDC typical-use failure rate estimates.

R25:
Thank you for pointing out this discrepancy. The chance of pregnancy value reflects the chance of pregnancy within the two year time horizon of this analysis. Both methods provide a <1% chance of pregnancy. To conduct the analysis from a more conservative perspective, both values could reflect the 1% chance of pregnancy in future iterations.

RC26:
It is unclear how the distribution of method use was derived in Table 1 and Figure 1. Does this assume the current distribution of uptake in Rwanda? R26: See Response 20. Table 1 footnote mentions that there was some weighting of estimates, but this is not described in the methods.

RC28:
The outcome of interest (pregnancies averted) is never mentioned in the methods and should be explicitly stated.

R28:
The pregnancies averted outcome has been explicitly stated in the methods.

RC29:
Similar to the issues mentioned previously, the tile for Figure 1 states "The two primary arms comparing family planning menu types," but the non-LARC only option is not an option at government facilities. R29: See Response 12. Table 1 footnote states "Value established from range". What range? What does this refer to? R30: The ** asterisks in the table legend is reflected in the table for the "normal vaginal delivery" row. A range of costs for a normal vaginal delivery are cited in the literature. This is discussed in the citation provided (page 26).

RC31:
In Table 1, how is the cost estimate for pregnancy hospitalization taken from a study on post-abortion care (reference 18)?

R31:
The referenced study provided general Rwanda health systems costs and can thus costs for other facility based care.

RC32:
It is worth mentioning that the societal perspective is not included, as is recommended.