INTRODUCTION

Low patient accrual, variable disease progression, lack of objective outcomes, relatively high overall trial costs, and difficulties in regulatory approval are some of the challenges faced in conducting clinical studies of rare diseases.1 Over recent decades, regulatory bodies in several countries have implemented special policies and regulations to support research and development of drugs for rare and ultra-rare diseases.1 Although existing policies facilitate the market approval of such drugs, decisions for timely access and reimbursement are not universal and are usually made at the national, regional, state, or even provider level.2 This is because most drugs for rare diseases are expensive and do not meet currently acceptable cost-effectiveness thresholds used in resource allocation decisions. Furthermore, in some cases, even when cost-effectiveness is near acceptable threshold values, there is a great deal of parameter uncertainty, and a drug or technology may receive conditional funding with the option to revise as new evidence becomes available.3 , 4

Considering the difficulties in conducting research in rare diseases, future trials should be designed to maximize the difference between the expected value of the information provided by the results and the expected costs of the trial. The traditional tests of hypothesis and power considerations do not take this into consideration and ignore the potential costs of Type I or Type II errors. Current evidence may indicate that intervention A is superior to intervention B. However, due to uncertainty in the evidence to date, the decision to adopt A is associated with an expected opportunity loss, and collecting additional evidence (e.g., conducting another clinical trial) is expected to reduce uncertainty.

As an alternative to the traditional approach, many authors have proposed value-of-information (VOI) methods. Grounded in decision theory, VOI methods can be used to assess the uncertainty associated with the existing evidence in the comparison of two competing interventions and identify optimal sample sizes if future research is still needed.5 – 16 The methods allow determination of the expected value of conducting additional research (e.g., a clinical trial) in monetary terms. The value of additional research, referred to as the expected value of sample information (EVSI), is the amount by which it reduces the expected opportunity loss associated with making decisions in the face of uncertainty. The expected total cost (ETC) of additional research is the sum of its financial and opportunity costs. The EVSI is a function of sample size, as is the cost of the trial. If for all sample sizes the EVSI is less than the cost, current evidence is sufficient for adopting A, since it is irrational to conduct a trial where the trial cost exceeds the value. On the other hand, if for any sample size the EVSI exceeds the trial cost, current evidence is insufficient, and performing another trial is the optimal decision. Expected net gain (ENG) is the difference between EVSI(n) and ETC(n), and n* is the optimum sample size that maximizes this difference. More specific details regarding VOI methods are presented in online Appendix B.

The purpose of this article is to illustrate, in a decision-analytic context, the application of VOI methods for assessing current evidence for the use of expensive health care interventions in rare diseases, in particular treatment of children with severe hemophilia A using recombinant Factor VIII (FVIII).

Hemophilia and Factor VIII Treatment Strategies

Hemophilia is a sex-linked rare genetic disorder with an annual incidence of approximately one in 5000 live male births.17 The worldwide prevalence of hemophilia is about 400,000, and only 25 % of affected patients receive adequate treatment.18 The disease is clinically manifested by repeated bleeding due to deficiency of clotting factor VIII (hemophilia A, 80–85 % of cases) or factor IX (FIX) (hemophilia B). About 50 % of all patients with hemophilia A have severe hemophilia, defined by FVIII levels lower than 1 or 2 % and recurrent bleeds, mostly into joints—either spontaneously or after minor injuries—that over time lead to arthropathy and lifelong disability.19 The preferred treatment in many developed countries is the regular intravenous administration of FVIII to prevent bleeding; however, prophylaxis doses and regimens vary across and within countries.20 High-dose primary prophylaxis (the Malmö regimen) involves the intravenous administration of 20–40 International Units of FVIII per kilogram of body weight given on alternate days (minimum three times per week), started before the onset of any overt musculoskeletal disease. This is considered the gold standard of preventive care and is implemented in many Scandinavian and Northern European countries.21 , 22 Although the World Health Organization, the National Hemophilia Foundation, and the World Federation of Hemophilia recommend FVIII prophylaxis as the main approach for treating children,20 in several countries patients are treated with an on-demand regimen, that is, FVIII infusions given only when bleeding occurs.

Hemophilia A is a very expensive disease to treat, primarily due to the high cost of FVIII, which accounts for 90 % of total treatment costs in hemophilia.23 , 24 In a study in France, the mean annual per-patient treatment cost with on-demand therapy was estimated to be USD $73,029.25 In Germany, where guidelines advocate alternate day prophylaxis for patients with severe hemophilia A, the average annual costs vary from €40,000 to €120,000 per patient.26

There have been only two randomized clinical trials (RCT) of primary full-dose prophylaxis versus on-demand therapy in young boys with severe hemophilia A.27 , 28 The first multi-centered trial was conducted in the United States and compared alternate day prophylaxis (AP, 25 IU/kg) with on-demand (OD) or enhanced episodic treatment among boys less than 30 months old with severe hemophilia A.27 By the age of 6 years, 93 % (25/27) of patients in the AP arm and 55 % (16/29) in the OD arm did not have magnetic resonance imaging (MRI)-detected joint damage. Using an estimate of FVIII cost of USD $1 per unit, the average (standard error) per-patient treatment cost was $352,793 ($26,597) for patients in the AP arm and $113,237 ($11,404) for patients in the OD arm. The second was an RCT in Italy that found that after 10 years, only 29 % of children on alternate day prophylaxis with FVIII developed radiographic arthropathy compared with 74 % of children randomized to episodic treatment (p < 0.05).28 Arthropathy was not assessed by MRI in this study.

Concurrent with the U.S. study, a single-arm, multi-center trial in Canada enrolled patients who met the same eligibility criteria, but were treated with tailored prophylaxis (TP). All patients in this cohort (n = 24) started with once-weekly prophylaxis and escalated to twice-weekly or alternate day prophylaxis if the bleeding pattern met specified escalation criteria.29 At 5 years of follow-up, 40 % of subjects were on once-weekly prophylaxis, 32 % were on twice-weekly prophylaxis, and 28 % on alternate-day prophylaxis. At the time of MRI joint assessment, children in the Canadian cohort were 8.8 years old (range 6.2–11.5 years) and 12 out of 24 patients (50 %) had MRI-detected osteochondral joint damage.30

A recent systematic review of hemophilia treatment strategies by the Swedish Council on Health Technology Assessment concluded that there is insufficient evidence regarding the optimum time to start FVIII treatment, the optimum dose and dose interval, and its cost-effectiveness.31 The objective of this study was to use the information from the U.S. and Canadian studies to evaluate if it is worth to conduct another trial of FVIII prophylaxis in children with severe hemophilia A.

METHODS

The net benefit of a health care intervention is defined as the mean measure of health outcome times the threshold value of the willingness to pay for a unit of health outcome minus the mean cost. In this formulation, the measure of health outcome is scaled so that larger values are more desirable. In the hemophilia example, the measure of effectiveness is scored as 1 if a patient does not have MRI-detected joint damage/arthropathy and 0 if he does. Thus, the estimated mean measure of health outcome is simply the proportion of patients free of joint damage. The threshold value (λ) is determined by how much a decision-maker would be willing to pay (in the current example) to avoid one case of hemophilic arthropathy.

As described in online Appendix B, EVSI is the reduction in expected opportunity loss per patient (if a new trial is conducted) multiplied by the number of patients (N) who would benefit from the decision. Since FVIII prophylaxis is recommended for children who are less than 2 years old and have had no more than one joint bleed, the decision will affect newly diagnosed or incident cases of hemophilia. N directly depends on the incidence of the disease, trial duration, and time horizon for the decision. Since EVSI increases with incidence, the application of VOI methods in a rare disease may often result in a situation where EVSI is less than ETC, leading to the conclusion that current evidence is sufficient for decision making. By contrast, standard frequentist methods are independent of incidence and may lead to unnecessary research.

Applying VOI Methods in Hemophilia

Consider the possibility of conducting a new, multi-center randomized controlled trial (RCT) in North America and enrolling patients with the same eligibility criteria as the previous trials, namely children with severe hemophilia A, less than 30 months old, with no joint damage and no clotting factor inhibitors. The combined annual incidence (k) in Canada and the U.S. is assumed to be 200, based on the total number of live male births in Canada32 and the U.S.33 in 2010. The summary statistics for all three treatment strategies are given in Table 1. To estimate the MRI score at the age of six6 yearsr the Canadian cohort, we examined clinical variables that predict MRI joint damage at the actual time of MRI assessment. Using the MRI joint score linear prediction rule at the time of MRI assessment (including target joint development, defined as at least three bleeds into the same joint within three consecutive months34 and the number of bleeds per year), the predicted number of children with MRI-detected joint damage at the age of 6 years would have been five out of 24, corresponding to 79 % with no damage (95%CI: 60 to 91 %). Considering the fact that FVIII costs vary widely in the current market depending on the country, product type, and contract negotiations, we conducted base case analysis using a cost estimate of 0.50 U.S. cents/FVIII unit.

Table 1 Summary Statistics by Treatment Strategy
Full size table

The values of the other parameters required for VOI determinations are given in Table 2 for base-case and sensitivity analyses. It was assumed that the decision would affect only new incident cases. Following the example of the U.S. trial, the proposed study would have 5 years of follow-up, with an additional year for data analysis and dissemination of the results (i.e., τ = 6). It was assumed for each comparison that if the trial were not conducted, future patients would receive the treatment strategy with the highest expected net benefit. It was also assumed that if a trial was conducted, patients not in the trial would receive the treatment strategy with the lowest expected net benefit, i.e., the treatment strategy with the lowest expected net benefit was assumed to be the standard. It was also assumed that while the trial is underway, no new treatment was developed and implemented (e.g., gene therapy) during the time horizon, and that no planned interim analysis or early stopping rule would be applied. For simplicity and due to low incidence,27 , 29 we ignored the risk of FVIII inhibitor development and its effect on ETC.

Table 2 Parameter Estimates for Sensitivity Analysis
Full size table

We performed three comparisons: Comparison 1 was AP vs. TP; Comparison 2, TP vs. OD; and, Comparison 3, AP vs. OD. We also conducted two base-case analyses using threshold values of $200,000 and $400,000. VOI methods were used to determine whether current evidence is sufficient for decision making and, if not, the optimal sizes of future clinical trials. A sensitivity analysis was conducted to examine the robustness of some of the assumptions. An MS-Excel spreadsheet provides optimal sample size calculation for base-case scenario for Comparison 1, along with the estimations of EVSI, ETC, and ENG (available at www.andywillan.com/downloads).

RESULTS

The expected net monetary benefit of each strategy was estimated for threshold values for the health outcome (MRI-detected joint damage) ranging from $0 to $800,000 (Fig. 1). For threshold values lower than $200,000, OD has a higher expected net benefit than either AP or TP. The differences between strategies are less pronounced for threshold values between $200,000 and $400,000, but for threshold values above $400,000, AP and TP have a higher expected net benefit than OD.

Figure 1
figure 1

Net monetary benefit of all treatment methods. Net monetary benefit was calculated as λ*E – C where λ = threshold value for a unit of effectiveness, E = effectiveness (MRI-detected joint damage) and C = costs.

Full size image

The optimal sample size (i.e., the sample size that maximizes ENG), as a function of the threshold value, is plotted in Figure 2 for all three comparisons. In Comparison 1 (TP vs. AP), for threshold values below $360,000 (area A), there is sufficient evidence to adopt TP (and the optimal sample size is 0). For threshold values greater than $550,000 (area D), there is sufficient evidence to adopt AP. Between the values of $360,000 and $550,000, the EVSI exceeds ETC and the optimal sample size is positive, making a new trial worthwhile. The evidence is insufficient for decision making, and a new trial is worthwhile for values between $220,000 and $300,000 in Comparison 2 (OD vs. TP), and between $290,000 and $350,000 in Comparison 3 (OD vs. AP).

Figure 2
figure 2

Optimal sample size as a function of threshold value (λ). For all sample size and threshold value combinations, in area A, the optimal decision is to retain the standard treatment; in area B, the current evidence supports the standard treatment (lower cost, lower effectiveness), but a new trial offers a reduction of uncertainty and an expected net gain; in area C, the current evidence favors the experimental treatment (higher cost, higher effectiveness), but a new trial is warranted to reduce the uncertainty; and in area D there is enough evidence to adopt the superior experimental treatment.

Full size image

In base-case analyses with a threshold value of $200,000, OD was the optimal treatment, since there was enough evidence to choose TP over AP, OD over TP, and OD over AP. For a threshold value of $400,000, there was sufficient evidence to choose either prophylaxis method over OD (Comparisons 2 and 3). However, there was insufficient evidence for Comparison 1, and a new trial comparing the two prophylaxis methods was the optimal decision. Using the base-case parameters, the optimal sample size was 38 patients per arm, with an ETC of $9,341,061 (financial cost of $1,382,000 and opportunity cost of $7,959,061) and an ENG of $17,113,972 (Fig. 3). For comparison, if we use a traditional sample size calculation to plan a study to test the difference in mean effectiveness of TP and AP, assuming a type I error rate of 0.05 and a power of 0.80, we would need a total sample size of 210, or 105 patients per arm.

Figure 3
figure 3

Expected value of sample information (EVSI) and expected total cost (ETC) as a function of sample size for Comparison 1. Comparison 1: Tailored prophylaxis vs. Alternate day prophylaxis using a threshold value of $400,000. EVSI is the expected value of sample information, ETC is the expected total cost, n* is the optimal sample size, and ENG is the expected net gain.

Full size image

Table 3 contains the results of the sensitivity analyses for Comparison 1 with a threshold value of $400,000. The change from base-case parameters to both high and low trial costs made little difference in the ENG and the optimal sample size. Applying discounting rates of 3 % and 5 % decreased the EVSI, ETC, and ENG. Optimum ENG remained positive for all sensitivity analyses except the unit cost of FVIII, indicating the need to conduct a new trial. The change of FVIII cost from 50 cent/unit in base-case analysis to 75 cent/unit or USD $1.00 resulted in negative ENG, indicating sufficient evidence to adopt TP, and the cost of a trial of any size would exceed the value of the information it produced.

Table 3 Sensitivity Analyses for Comparison 1: Tailored Prophylaxis Versus Alternate Day Prophylaxis (Threshold Value = $400,000)
Full size table

DISCUSSION

In decision analysis, the recommendation for the adoption of new health interventions is made on the basis of a positive expected incremental net benefit. However, in rare diseases, the evidence is often scarce and insufficient for decision making. In addition, although the number of people affected by a decision can be small, the overall opportunity loss can be significant. In considering the funding of new research and patient reimbursement in rare diseases, VOI methods may provide more relevant determinations of the value and costs of additional research than traditional sample size determinations. VOI methods take into account the current evidence, the value placed on health outcomes, the cost of research, the discount rate, disease incidence, and the appropriate time horizon.

Using VOI methods and the cost and effectiveness estimates of hemophilia treatment from the U.S. and Canadian trials, we compared current treatment strategies for FVIII use in hemophilia A and sought to determine whether current evidence is sufficient for making an evidence-based decision. We found that the direction of the incremental net benefit for the paired comparisons depended on the threshold value. As would be expected given the high cost of prophylactic treatment, low threshold values favored on-demand therapy, while higher threshold values favored prophylactic treatment strategies.

A recent review of FVIII treatment strategies in severe hemophilia A called for more studies to establish the optimal age to begin treatment, optimal dosage and schedule, and objective outcomes that consider also the costs of inhibitor development and cost of therapy.20 Using a specific threshold level and applying VOI methods, a decision maker can identify treatments for which there is enough evidence, treatments for which there is not enough evidence and new trials are warranted, and the optimal sample size that maximizes the ENG. For example, in our study, for a threshold level of $400,000 to avoid arthropathy, there is sufficient evidence to choose either prophylactic strategy over on-demand therapy. However, a new trial would be needed to make an optimal choice between the two prophylaxis strategies.

The robustness of these findings can be evaluated by determining the impact of various assumptions and parameters using sensitivity analyses. We illustrate this using the example of Comparison 1 (AP vs. TP) with a threshold value of $400,000; the results were robust for parameters of discount rate, trial fixed costs and variable costs, enrollment fraction, and the time horizon. The time horizon had a large impact on the optimal sample size, because it directly affected the number of people who would benefit from the treatment after an optimal decision was made. As expected, the unit cost of FVIII had the largest impact on the decision about treatment adoption, and an increase of the FVIII cost favors treatment modalities with the lowest FVIII utilization (i.e., TP over AP). However, the cost of FVIII varies across countries and depends—for example—on whether the product available is recombinant, plasma-derived, or virus inactivated, and whether the purchaser is able to secure a discount from the manufacturer based on the terms and duration of the contract.

Evidence shows that FVIII prophylaxis strategies are superior to on-demand therapy in preventing bleeding, further joint damage, and long-term disability in patients with severe hemophilia.27 , 35 , 36 Although even a single bleed can cause arthropathy and long-term disability, to our knowledge, no study has attempted to estimate the different decision makers’ (i.e., patients, caregivers, healthcare providers, third-party payers) willingness to pay for avoiding joint damage in hemophilia. The results from past cost-utility (modeling) studies conducted in the UK, Germany, and Canada varied significantly, with incremental cost-effectiveness ratios ranging from low £37,000 per quality-adjusted life years (QALYs) to above €1million per QALY when comparing FVIII prophylaxis with on-demand therapy.37

Our study results must be interpreted in the light of potential limitations. We made assumptions about several parameters, including patient accrual fraction, time horizon of the decision, and trial costs, that may have been incorrect. However, these assumptions were explored in sensitivity analyses, and our results were found to be robust. Recent articles about VOI methodology have tested the impact of other parameters and assumptions not considered in this study.5 – 7 , 15

Our results can be used to help plan RCTs in other rare diseases with high treatment costs. As an example, the per-patient annual cost of enzyme replacement in Gaucher disease with imiglucerase is ₤70,100, with agalsidase in Fabry disease it is ₤109,600, and with laronidase in mucopolysaccharidosis 1, it is approximately ₤311,000.4 A careful analysis of future expected costs and benefits in planning RCTs in these diseases using VOI methods and realistic estimates of disease incidence, threshold value of health outcome, costs and time horizon might maximize both the information provided by future trials and the resulting benefits.

In conclusion, we used VOI methods to determine the expected value of sample information, the expected trial costs, and the expected net gain for therapeutic trials in boys with severe hemophilia—a rare disease with high treatment costs. We demonstrate that there remains considerable uncertainty about the optimal cost-benefit of therapy for hemophilia, and several scenarios in which future clinical trials would be the best decision. We hope that the results of the current study will help in decision making for treatment strategies in hemophilia and in planning of future studies of this condition and other rare diseases.